News and Comment

Ganzfeld Studies: First Detailed Appraisal Finds Serious Flaws, No Evidence of Psi

N THE 11 YEARS since publication have been presented as evidence for psi. Iof the first "ganzfeld" ESP experi­ The first detailed scholarly evalua­ ment in 1974, reporting pro-psi results, tion of the ganzfeld studies has now a series of similar experiments have been been published. The critique, prepared published in the parapsychological liter­ over a period of several years by psy­ ature. Parapsychologists and others chologist (and CSICOP Executive have considered these to be among the Council member) Ray Hyman, of the strongest scientific evidence for the University of Oregon, gives little com­ existence of extrasensory perception. fort to proponents of the ganzfeld Ganzfeld experiments are based on experiments as the best hope for proving the idea that sensory deprivation is con­ psi abilities exist. ducive to the manifestation of psi abili­ "... 1 believe that the ganzfeld psi ties. The research subject is generally data base, despite initial impressions, is isolated from visual and other sensory inadequate either to support the conten­ contact. Then various experiments are tion of a repeatable study or to demon­ carried out to test the subject's ability strate the reality of psi," Hyman con­ to perceive outside information. cludes in his 47-page critical appraisal, For instance, a person undergoing published in the March 1985 Journal perceptual deprivation might be asked of Parapsychology. "Whatever other to "receive" an image from a photo­ value these studies may have for the graph randomly selected from four parapsychological community, they photos and being concentrated on by a have too many weaknesses to serve as "sender" in another location. While the basis for confronting the rest of the doing so he might be requested to ver­ scientific community. Indeed, parapsy­ balize his thoughts, feelings, and images. chologists and others may be doing Afterward, the subject and, in some themselves and their cause a disservice cases, independent judges would be by attempting to use these studies as asked to assess the degree of cor­ examples of the current state of their respondence between the picture and field." the subject's imagery. Positive results The journal follows Hyman's from a number of such experiments analysis with an equally detailed

2 THE SKEPTICAL INQUIRER, Vol. 10 response by parapsychologist Charles ganzfeld study he knew of. Honorton disputing many of his con­ As a result, Hyman evaluated all clusions. Honorton carried out the first 42 studies reported from 1974 through reported ganzfeld ESP experiment in 1981. Honorton classified 23 of them 1974, and his subsequent ganzfeld as having achieved significance as studies include some of the most posi­ evidence of psi. This amounts to a tive (pro-ESP) results. claimed replication rate of 55 percent. Hyman had been asked to prepare Hyman prepared a preliminary a critical appraisal of parapsychology. critique, which he presented at the com­ Rather than attempting to take on the bined meetings of the Society for Psy­ whole field, he looked for a systematic chical Research and the Parapsycho- research program that parapsychologists logical Association in Cambridge, considered especially promising. It England, in August 1982 (SI, Winter needed to consist of a series of studies 1982-83). As a result of comments on carried out by a variety of researchers. that paper by Honorton and others, he He chose the ganzfeld psi studies. prepared a new and more systematic Respected investigators had conducted analysis of the data. them. Hyman was intrigued by their Hyman focused on two questions: claims that significant psi scores had (1) Does the data base, taken as a been achieved in more than half of the whole, supply evidence for psi? (2) Does experiments and that the studies had the ganzfeld psi study yield evidence been conducted with a high level of for psi that is replicable? sophistication and rigor. The basic index for these questions Hyman sought Honorton's cooper­ is some measure of hitting or target- ation. Honorton felt it important to matching compared with a chance base­ have an outside critic like Hyman assess line. This, Hyman noted, creates special the ganzfeld literature. He supplied problems; assumptions about chance Hyman with a copy of every reported levels and probability distributions take

Fall 1985 3 on a great burden. gests a tendency to report studies with He divided his critique into four a small sample only if they have sig­ phases: nificant results." —Rechecking the "vote count." Another, related bias Hyman calls —Assessing the actual opposed to a "retrospective bias." "This is the the assumed level of significance. tendency to decide to treat a pilot or —Assigning procedural flaws to the exploratory series of trials as a study if studies. •it turns out that the outcome happens —Analyzing correlations among to be significant or noteworthy." He flaws, positive effects, and significance. found two studies in the data base that The "vote count" check assessed were clearly retrospective and strong whether the studies claimed to be suc­ circumstantial evidence of four others. cessful really amount to 55 percent of Next Hyman considered whether the total. Hyman found a lot depends the chances of getting successful results on how studies containing multiple in ganzfeld ESP studies without invok­ conditions are divided up. He found, ing psi are really as low as psi pro­ for instance, that a study counted as ponents suggest. The studies varied one "successful" replication could be widely in variables and in the questions viewed "with equal justification" as being asked. Notes Hyman, with some adding one successful and 11 unsuccess­ understatement, "Many confusing ques­ ful replications to the total. tions arise about what probability levels Then he considered the "file- to assign to the various tests of signifi­ drawer" problem. How many ganzfeld cance." studies have been conducted but not Generally, a ganzfeld experiment reported? Surveys have identified other is taken to show evidence of psi if the studies, and their inclusion tends to statistics indicate there is no more than lower the success rate. a .05 probability that the results are The important question here was due to chance. Hyman's various analy­ whether there was evidence for biased ses found that the probability of obtain­ reporting—specifically, is there a possi­ ing at least one significant outcome per bility that only those experiments that experiment was instead .24—"over four begin with a string of successes end up times the assumed level of .05." being reported? "It is [easy] to imagine The discrepancy results from the that a large number of experimenters use of multiple indices—the availability . . . might have begun conducting some of several different ways of getting "hits" trials and then abandoned the study without their being included in the when the first few trials turned out to probability estimates. Hyman found be unpromising. On the other hand, a that more than half the studies he eval­ few of these exploratory ventures might uated "clearly used multiple indices have started with initially successful without taking this into account in trials, encouraging the experimenter computing their statistical significance." either to continue or to stop and write Multiple indices was but one of six up the result as a successful replication." categories of multiple testing he checked Is there any evidence for such a the studies against. Forty percent of the suggestion? Yes, Hyman says. He found studies, for instance, used multiple a tendency for the studies with the fewer baselines; 64 percent used multiple trials to have a higher proportion of groupings. significant outcomes. "The most obvious Using this kind of analysis, Hyman conclusion is that such a strange rela­ found one study that had increased the tionship is due to selective bias. It sug- probability of getting successful results

4 THE SKEPTICAL INQUIRER, Vol. 10 ''"&. V, "^^7: ..I'1"'

J ,.*•> >" ,vl" "

— \ "s"»'.,x lie* \ .

i":'""""wv. *»••». i . "'•"'l.^,*r""'"'inJL**'«.l.l, /

"'"fw *"""»'•. -n !/"•*'••• I 1 *" *nnHn I Ganzfeld debate: Hard-hitting critique, spirited rebuttal.

"almost surely beyond .50." In other had inadequate security, and 29 percent words there was a better than 50-50 appeared to use erroneous statistical chance of getting a positive, "pro-psi" procedures. result just by chance. "Indeed, if we Hyman says he was very conserva­ consider the eight intervening practice tive in assigning these flaws. Those that conditions, the chances of coming up were not too common or depended on with a significant outcome are well over suspicions or hard-to-objectify criteria .80! And this is just one of the many he did not count. "In any case the studies in this data base that exhibit existence of so many elementary defects such complex options either explicitly in this data base is both disturbing and or implicitly." surprising. Only two studies were Hyman found that the actual rate entirely free of the six procedural flaws. of successful replication is less than 30 And if we include multiple-testing percent. "And the arguments in this sec­ errors, not a single study in this data tion strongly suggest that this rate of base was flawless. 'successful' replication is probably very "It is important to realize that the close to what should be expected by defects being discussed are not obscure chance given the various options for or subtle. Rather, I suspect that a multiple testing exhibited in this data typical parapsychologist would spon­ base." taneously list them as being unaccepta­ Hyman then turned to procedural ble in a psi experiment." flaws in the studies. He found that 36 Are these defects important? Yes, percent of them used improper random­ Hyman believes, in two ways. First, they ization procedures, 55 percent used only are a symptom of something seriously a single target (which allows various wrong. When studies put forth as chances for sensory leakage), 24 percent among the field's strongest evidence of allowed contaminating feedback, 38 psi have so many elementary deficien­ percent of the published studies (and cies, it is a sign that quality control is 81 percent of the unpublished ones) gave lacking. There are so many problems inadequate documentation, 24 percent with these studies in fact that Hyman

Fall 1985 5 told the SKEPTICAL INQUIRER he just such guidelines. Hyman has believes most were informal exploratory accepted an offer to serve on the com­ studies that were reported only because mittee.) Only if a large body of studies they gave positive results. No one knows meeting such rigorous guidelines can be how many exploratory studies giving accumulated should the scientific com­ null results have been carried out for munity have any obligation to take each one giving a positive result. This notice, Hyman concludes. is an old problem in parapsychology. In Honorton's 41-page response to Until it is solved, it makes the statistical Hyman's analysis, he presents his own case for psi almost meaningless. meta-analysis of ganzfeld research, Second, the tendency to get which he says eliminates the multiple- stronger results correlates with greater analysis problems Hyman criticized. He presence of experimental deficiencies. disputes the view that selective-report­ Hyman examined the pattern of rela­ ing bias has anything to do with the tionships among indices of success and positive results reported in the literature. various flaws. He compared the pres­ He contends that, contrary to Hyman's ence of flaws in each study with its out­ assessment, no significant relationship come. The flaws concerning randomiza­ is found between study outcomes and tion, feedback, documentation, and measures of study quality. He also dis­ statistics seemed to correlate with three agrees with some of Hyman's assign­ different measures of significance. "The ments of flaws. "Is there a significant more likely a study was to be assigned psi ganzfeld effect?" Honorton asks. "I any of these flaws, the more likely it believe my evaluation of direct-hits was to be classified as significant." A studies justifies an affirmative answer similar but weaker pattern was found to the question." in the size of the effect reported by the Hyman will prepare a rebuttal to study. Honorton's response for a future issue Hyman told 57 he doesn't contend of the Journal of Parapsychology. He that there is a one-to-one correspond­ points out that Honorton, in his ence between a flaw and a positive out­ response, concentrates only on Hyman's come. "I argue that it's not any one correlation analysis while ignoring the defect alone. Probably they work in larger issue. "He ignores the fact that combination." all the experiments are flawed. In ten "Whatever the reasons, the 42 years, why hasn't anyone done them studies in the present data base cannot right? He hasn't faced up to that." by any stretch of the imagination be Hyman's published evaluation included characterized as flawless, and I suspect all ganzfeld studies published through that most of them were not well 1981, but he says he hasn't seen one planned," Hyman concludes. "The cur­ reported since then that avoids the rent data base has too many problems problems he cites. to be seriously put before outsiders as Hyman's is the most detailed cri­ evidence of psi." tical analysis of the ganzfeld ESP In the concluding section, Hyman studies ever done. Constructive in offers a number of suggestions by which intent, it is nevertheless sobering, even the Parapsychological Association and damning, in result. It is clear from others could establish guidelines for Honorton's spirited rebuttal that the what should constitute an adequate controversy will continue. But parapsy- confirmatory study. (The Council of the chologists who hoped that ganzfeld ESP Parapsychological Association has now studies would at last achieve the scien­ commissioned a study group to develop tific demonstration of psi have been

6 THE SKEPTICAL INQUIRER, Vol. 10 given a good dose of the kind of critical scrutiny their claims will have to face and overcome. If future studies can pass that kind of test, then they would deserve the attention of outside scien­ tists; if not. it appears that oblivion is their destiny.

Kendrick Frazier

Koestler Appointment Goes to Robert Morris

HE ENDOWMENT of a chair in Tparapsychology by writer Arthur Koestler and his wife Cynthia in the will left alter their joint suicide in 19X2 set off a small storm of academic con- [roversy in Britain. Was the study of parapsychology a proper activity al a university'.' Oxford and Cambridge expressed no interest in the £500.000 1978. In it. he revealed crucial discrep­ bequest and eventually the Arthur ancies between the book and the facts Koestler Chair of Parapsychology was and outlined 12 basic problems facing established at Edinburgh University spontaneous investigations in parapsy­ (See Martin Gardner's "Koestler Money chology. He was also an invited speaker Down Psi Drain?" SI. Fall 1984.) at CSICOP's international conference Now. after a long search for the in Buffalo. NY., in October 1983. He best candidate. Edinburgh has selected spoke of a possible "residue of useful an American parapsychologist. Robert information" in experiments on ESP. Morris, to occupy the post. Morris, referred to his "high tolerance for senior research scientist at the School ambiguity." complained of some of the of Computer and Information Science more emotional criticisms of parapsy­ at Syracuse University, is expected to chology, and called for intelligent cri­ begin on January I. 19X6. ticism and active cooperation between critic and researcher ) Morris. 42. is a well-known para- psychological researcher in the United The Koestler chair is the only pro­ States. He is generally respected by both fessorial post for parapsychology in proponents and critics of the field. Britain. It will be in the Psychology President of the Parapsychological Department "Given the unique nature Association, he received his Ph.D. in of this appointment. Dr. Morris faces a animal social behavior at Duke Uni­ considerable challenge." said John versity and has published papers in that Burnett, principal of Edinburgh Uni­ field as well as detailed scientific asse^- versity "He is well equipped to meet ments and popular critical accounts of that challenge and cope with the wide paranormal claims. public interest his research program will attract " (longtime SI reader may recall Morris's critical investigative review of I he university had drawn up a list The Amityville Horror. Spring Summer of .'(I applicants who it indicated had

Fall 19X5 " appropriate qualifications in the field Randi, Harvard professor Stephen Jay of psi research. It narrowed that list to Gould, Bermuda Triangle expert Law­ 8 before selecting Morris. He will rence Kusche, CSICOP chairman Paul receive an annual stipend of £18,000. Kurtz, UFO expert Robert Sheaffer, "While occupying the Koestler and Drs. Bernard Leikind and William Chair," Morris told the SKEPTICAL McCarthy, experts on firewalking. INQUIRER, "I intend to promote para­ For its April lecture, SCS held a psychology as an interdisciplinary prob­ demonstration of the physics of fire- lem area, seeking to understand as best walking on the Caltech sports field. It it can the causative bases of the human included a demonstration and voluntary experiences commonly labelled as 'psy­ audience participation. Approximately chic' Such understanding may lead us 1,000 persons attended and 125 of them in major new directions, and we must barefooted over the 1,400° F coals. be prepared to accept this. It most Newspapers in Southern California car­ certainly will lead us, at least in part, ried front-page stories and pictures to a better understanding of the means about the demonstration, and reports by which we infer causal connections were aired on CBS and CNN national between ourselves and our environ­ news the following week. The Los Ange­ ments, including means by which we les Times followed this with an enthusi­ can be deceived. astic editorial about SCS in their Sun­ "Parapyschology is not a belief day edition, headlined "Go Skeptics!" system, and should not be practiced as Aside from the letters we have such by those who would make strong received from doctors, college pro­ claims about the nature of psychic fessors, psychologists, and high school experiences in the guise of science but teachers, thanking us for our efforts, without adequate scientific support." we have had many letters and phone —K.F. calls from believers who changed their minds after hearing common sense from us. One man, for example, who had Southern California Skeptics been attending a local firewalking semi­ Fast-Rising Star in LA nar read our explanation in his local newspaper. He called to thank us for N NINE SHORT months, the setting him straight. Probably the most ISouthern California Skeptics (SCS) grateful were the two callers who were has become the fastest-growing saved from investing $3 million in a local skeptics group. SCS has already perpetual-motion machine. achieved a membership of 700 and has One of our main goals is to reach received favorable reports in almost children at the junior-high and high every major newspaper in the country. school levels. SCS is proud of the suc­ SCS spokesmen have been interviewed cessful visit by one of its speakers to a on more than 50 radio talk-shows class in a San Fernando Valley high around the country. The requests for school where the teacher, Joe Feinstein, SCS representatives to speak to news­ is an unabashed supporter of paranor­ papers, radio and television shows, club mal claims. He had been making his banquets, and schools continue to come views known in his high school "Intro­ in at an impressive rate. duction to Sociology" classes, where he SCS monthly meetings and lec­ believes "a new type of thinking is tures, held at the California Institute of required in today's world." SCS asked Technology, have featured such distin­ to present a different point of view to guished speakers as magician James his class. Mr. Feinstein agreed.

8 THE SKEPTICAL INQUIRER, Vol. 10 Feinstein wrote to SCS giving the results. "After spending 15 weeks pre­ senting a wide array of psychics, graph­ ologists, and parapsychologists of all descriptions, you totally annihilated all their efforts in 40 rapid minutes. I'm not sure if I'm happy or sad about that. However, you were very effective in giving my students somethings) to think about. And, after all, isn't that what school and education are supposed to be about? Since you've been such a marvelous irritant and catalyst, please accept my invitation to join us once again ... to do your number." SCS returned to his class in June and received a standing ovation from the students after the presentation. SCS is not, however, entirely suc­ cessful. We never thought we would be. In a sense we are like Sisyphus pushing that rock up the proverbial mountain of superstition only to watch it roll back Australian organizers were appointed, down on occasion. A week after a leng­ and an office with a toll-free number thy article about us appeared in the Los was opened. All was set for a profitable Angeles Herald Examiner, it published run. Robbins was to tour from May 8 an equally lengthy article about a to 20, 1985. woman who edits a magazine contain­ Robbins clearly hadn't counted on ing only articles contributed by ghosts. the Australian Skeptics, the Australian I wonder which is the better buy—the branch of the Committee for the Scien­ magazine that tells you, for $25, that tific Investigation of Claims of the there are ghosts, or the SKEPTICAL Paranormal. INQUIRER, for $18, which tells you that The news of Robbins's intended the former is probably using a different tour reached the Australian Skeptics in type of ghostwriter? Nevertheless, we the same week as a copy of a paper on have our successes, so we will continue firewalking from Southern Californian to keep pushing that rock up the hill. Skeptics Bernard Leikind and William McCarthy (based on their article in this —Al Seckel, SCS Chairman issue), both members of the Southern California Skeptics. The Australian Skeptics rushed the Firewalker Challenged, paper over to one of their patrons, Gets Cold Feet national columnist Phillip Adams. Adams made it the topic of his weekly OP CALIFORNIA firewalker column in Australia's national news­ TTony Robbins prepared well for paper The Australian on March 30. his first Australian tour. Advance press Millions of Australians read the scien­ releases were circulated to the Austral­ tific explanation of firewalking. ian media, full-page advertisements were The following weekend the placed in the "alternative press," Australian Skeptics held their first

Fall 1985 9 national convention in Sydney. In his volcanoes, and other surface features opening address the Skeptics' national on Mars, one photo of a kilometer-wide president, Mark Plummer, challenged formation that superficially looked like Robbins to allow his claims to be tested a giant stone face caused some momen­ on arrival in Sydney. The challenge was tary amusement. No one took the "face" widely reported by the media. seriously, but we knew that the fringe- At the same time another Ameri­ science fans would probably try to make can, Robert Young, from Hawaii, was something mysterious out of it. The touring Australia trying to stir interest only thing surprising about the recent in his firewalking courses. Not only was flap in the media about the stone face his pre-publicity not as good as Rob- on Mars (see Martin Gardner's column, bins's but everywhere he turned up there p. 14) is that it occurred nearly nine were Skeptics handing out flyers giving years after first publication of the photo. the scientific explanation of firewalking. At Science News, where I was editor Young debated Mark Plummer on during the Viking mission, we published one radio station and challenged him the photo in our August 7, 1976, issue to walk over the fire. Plummer said he (whose cover was the first color photo would consider it if the Skeptics' scien­ from the surface of Mars, taken on July tists were allowed to take heat measure­ 21 by the Viking 1 lander) and gave it ments of the fire first. a justifiably brief two-sentence reference When the night came, Young's at the very end of a 3-page article on group refused to allow the measure­ the Viking findings by our space- ments to be taken. The following day sciences editor, Jonathan Eberhart. The Plummer was given time on the same reference quoted an apt quip by Viking radio station to blast Young for not site-selection chief Harold Masursky honoring the agreement. Although about the stone face: "This is the guy Young was invited to give his side of that built all of Lowell's canals." the story, he declined to appear. The Skeptics kept up the barrage ***** in the media, telling more and more people about the realities of firewalking A 1985 George Norlin Award from the and repeating the challenge to Robbins. University of Colorado alumni associa­ Young found interest in the courses tion to your editor makes special note waning as the Skeptics' publicity of the SKEPTICAL INQUIRER. The cita­ increased. tion refers to a journalism of public Ten days before Robbins was due service. "Editor since 1977 of the to arrive, Mark Plummer telephoned SKEPTICAL INQUIRER, a quarterly that his office to get details of his arrival so is recognized worldwide, he speaks that the Skeptics could plan their own calmly, authoritatively and fairly in a press conference. He was told that the mine field of controversy against pseu- Robbins tour was "postponed." The doscience, superstition, and the irra­ Californian firewalker had got cold feet! tional fringe."

***** Faces, Flyers, and Frauds . . . and an Award Many readers (presumably relatively new ones) have written asking us to do N THE SUMMER of 1976, when something about the pro-paranormal Ithe Viking 1 orbiter was daily taking' flyer the Reader's Digest has been extraordinary photos of huge canyons, distributing this year in massive"quanti-

10 THE SKEPTICAL INQUIRER, Vol. 10 ties. They all have complained that the birth, received an "Emergency Notifica­ four-color foldout is paranormal pan­ tion Memo." It was accompanied by a dering of the worst sort. ("There Are Lucky Number Horoscope with half of Strange Things Going On Around Us," the page torn out and missing. "I had it begins.) In fact, this is another mass prepared your horoscope for internal mailing of exactly the same promotional research use only," says the letter. "But flyer we have already condemned in after reading your horoscope, I was these pages ("Promoting the Paranor­ forced to halt my research project and mal: The Digest's Glossy Sales Pitch," send you this emergency notification." Fall 1983, pp. 5-8). We criticized both It goes on to exclaim that "your horo­ the tone and the facts of the promo­ scope" shows that opportunities for tional piece, in some detail, and said "extraordinary luck, love, wealth, and Reader's Digest Books ought to be happiness" are about to occur in the ashamed of it. We even brought our weeks ahead. But to take advantage of critique to the attention of the editor of these opportunities the recipient must, Reader's Digest Books, in a personal according to the writer's "research," letter. There was no reply. know the exact minute when his or her significant time periods begin. That is why, says the letter, "1 felt obligated to make this emergency notification." How does one get the missing part of the The American Astrological Association, horoscope? By sending $10.95 to the Canton, Ohio, has descended to a new Association. Asks one recipient, "Is this low in direct-mail solicitation. Recipi­ mail fraud?" ents, allegedly those who had already sent in their exact time and place of -K.F.

Australian Skeptics' Quarterly

The first 18 issues of the Australian Skeptics' quarterly magazine, The Skeptic (through the July 1985 issue), are available for $18 U.S. Order from Dr. J. Lattanzio, Canadian Institute of Astrophysics, University of Toronto, Toronto, Ontario M5S IAI, Canada.

Fall 1985 11