sign in sign up
<<
Home , Charles Honorton, James Alcock, James Randi, Marcello Truzzi, Ray Hyman, Skeptical movement

Skeptical Responses to Psi

By Ted Goertzel and Ben Goertzel

From Evidence for Psi: Thirteen Reports, edited by Damien Broderick and Ben Goertzel, McFarland and Company, 2015.

Editors’ Note

One of the peculiarities of psi research is that if any scientific work or publication on the topic gets attention at all outside the community of serious psi researchers, it is met by a barrage of derision from the community of self-styled psi “skeptics” (some of whom, it has been argued, might be better termed “denialists”). We consider it fairly likely that this edited volume attract a familiar sort of negative attention from the psi skeptic community, and are certainly prepared to debate any of their critical responses to the evidence we have gathered here in a careful and public way. However, given the prominence of the “psi skeptic” in the modern scientific and popular-media discussions of psi, we also deemed it worthwhile to include here an explicit analysis of the psi skeptic community itself and its weaknesses and motivations.

Toward that end, we invited a brief chapter from a sociologist with expertise in both social movements and associated systems, and statistical data analysis—Ted Goertzel, who also happens to be the father of one of the editors. Ted has no historical connection with psi research, and brought to the project an attitude of healthy toward claims of the of psi phenomena. One of the thrusts of Ted’s research career has been the careful skeptical investigation of claims made by various parties based on statistical and evidence, mostly related to social policy (the effect of capital punishment on homicide rates;1 the impact of welfare reform,2 etc.), but also including other topics such as UFO abductions.3 Ted’s initial chapter draft ultimately resulted in the jointly authored chapter you will find below.

1“Capital Punishment and Homicide: Sociological and Econometric Illusions,” , July/August 2004, pp. 23-27. Reprinted in , ed., and . , 2007, pp. 184-192. Reprinted in Diane Henningfeld, ed., The Death Penalty: Opposing Viewpoints, New York: Greenhaven Press, 2006, pp. 99-104.. 2 “New Jersey’s in Welfare Reform,” The Public Interest, Fall 1996, pp. 72-80. With Gary Young. 3 “Measuring the Prevalence of False Memories,” Skeptical Inquirer, 18 (3): 266-272, 1994. Skeptical Responses to Psi Research

By Ted Goertzel and Ben Goertzel

Psi research is unusual, but not unique, in that the community of researchers is complemented by a well-organized community of professional “skeptics,” who make it their business to attempt to debunk published psi research results via exposing them as erroneous or fraudulent. This chapter presents a brief investigation and discussion of the psi skeptic movement. Who are these skeptics? What are their motivations? How valid are their criticisms? What might psi researchers do to convince the skeptics, or counter their arguments?

Historical Roots of the

The organized skeptical movement has had a long and contentious relationship with the psi research community (Frazier, 1998; Kurtz, 2001; Carter, 2012). The conflict goes back to 1976 when philosopher Paul Kurtz and sociologist Marcelo Truzzi joined forces with a group of , writers and activists who felt a need for an organization to promote rational and scientific thinking. Prominent figures in the movement included psychologist , mathematician and science writer , magician , astronomer and science fiction writer . Paul Kurtz, the central figure, had been a leader in the secular humanist movement and was highly skeptical of religious thought. But religious thinkers generally acknowledge that their beliefs are based on , which makes them poor targets for rational or scientific criticism.

The organizers decided to focus their efforts on the “” because the paranormal movement made empirical claims that they believed to be irrational and unscientific. They called their new organization the Committee for the Scientific Investigation of Claims of the Paranormal, or CSICOP (Frazier, 1996). Later on, they expanded their focus and renamed it the Committee for Skeptical , or CSI.

CSICOP started by conducting a study under its own auspices, designed to refute the claim of a “Mars Effect” by French psychologist Michel Gauquelin. They were quite shocked when their independent analysis actually confirmed some of Gauquelin’s findings, and there was a minor scandal when they were accused of trying to suppress their own results (Carter, 2012; Rawlins, 1981; Frazier, 1996). Astronomer Dennis Rawlins, who protested the suppression, was dropped from the organization’s board and CSICOP decided not to do any more of its own original research. Instead, it found its niche in science journalism, and achieved some success with its magazine, The Skeptical Inquirer.

The magazine was originally founded by Marcelo Truzzi , who called it The Zetetic. Truzzi wanted it to be a balanced popular scientific journal with contributions from both sides of controversial issues. He favored inviting psi researchers into the organization, a step that was strongly opposed by other key leaders. As a social , Truzzi sympathized with the psi researchers in their earnest efforts to apply rigorous statistical methodology. He observed that:

Parapsychologists really want to play the game by the proper statistical rules. They're very staid. They thought they could convince these sceptics but the sceptics keep raising the goalposts. It's ironic, because real researchers are very committed to doing real science, more than a lot of people in science are. Yet they get rejected, while we can be slipshod in psychology and and and get away with it. We're not painted as the witchdoctors, but they are (Margolis, 1999).

Extraordinary Claims and Extraordinary Evidence

One phrase often heard in the discussion of psi research is “extraordinary claims require extraordinary evidence.” While this sounds sensible at first, it’s actually not so obvious, and merits a bit of reflection and analysis. What is, actually, the evidence for this claim itself?

Truzzi (1978) first coined the dictum “an extraordinary claim requires extraordinary proof” in trying to explain why the parapsychologists were so unsuccessful in persuading the skeptics with their conventionally rigorous research. This slogan became an axiom of the skeptical movement, thanks largely to its popularization by Carl Sagan who rephrased it slightly as “exceptional claims require exceptional evidence.” The dictum was useful as a rhetorical meme, so the skeptics used it that way without subjecting it to a critical analysis. Truzzi tried, defining extraordinary claims as those that contradicted well established scientific principles, but he failed in his effort to clarify what kind of extraordinary evidence might be sufficient to accept such claims.

When philosopher Theodore Schick (2002: 332) finally examined the principle seriously, for a skeptical encyclopedia, he concluded quite sensibly that “extraordinary claims do not require extraordinary evidence. It can be reasonable to accept an extraordinary claim in the absence of extraordinary evidence as long as it provides the best explanation of the evidence available; that is as long as it meets the criteria of adequacy better than any other explanation.” Some skeptics have suggested abandoning the “exceptional evidence” argument in favor of simply requiring rigorous, high quality . But they have been reluctant to do so, because they realize it might force them to actually accept some paranormal claims. Philadelphia Skeptics leader Ed Gracely (1998) defended the “extraordinary proof” criterion because “skeptics are not willing to accept the plausibility of most paranormal claims unless the evidence is extremely strong. We risk perceived (correctly) as disingenuous if we call for solid quality research, then revert to the extraordinary claims argument should it in appear.”

It seems plain that requiring extraordinarily strong evidence for any claim that is “extraordinary,” in the sense of going against received thinking, will sometimes have the impact of preserving an existing system of belief beyond what is commonsensically warranted by the evidence. If a community holds evidence in favor of its current theoretical understanding to lower standards than evidence challenging this understanding, then obviously this will bias that community against changing its understanding. Examples of this phenomenon abound in the of science, politics and other areas of human endeavor.

Time and time again, the scientific establishment has taken what, in hindsight, appears an irrationally long time to incorporate new evidence contravening its commonly held assumptions. Magnetic field lines, evolution by natural selection, quantum indeterminacy and numerous other scientific concepts, now considered well demonstrated, were initially rejected by the majority of scientists even after the presentation of experimental evidence, now considered reasonably compelling, in their favor. Compared with most other fields of human endeavor, science has proved remarkably good at revising its deepest assumptions based on new evidence. Nevertheless, there have been many, many cases where scientific progress has been slowed by inadequate processing of evidence related to claims found counterintuitive at the time.

On the other hand, while the dangers of excessive skepticism toward evidence for new ideas are well-demonstrated, the conceptual foundation of the “extraordinary evidence” idea is also intuitively clear. Suppose one has two hypotheses, A and B, where: A is extraordinary relative to a well- established theoretical system S, whereas B is ordinary relative to this system. One could argue that A requires more evidence than B, because B picks up some extra evidential weight indirectly from the other evidence already gathered in favor of the theoretical system S.

But while this makes perfect sense intuitively, rigorous elaboration of this sort of conceptual argument becomes quite subtle. The problem is that there is no generally accepted, scientific -independent approach to formalizing , weighing and comparing evidence, and so forth (Balashov and Rosenberg, 2001). Different tend to work quite differently in these respects; and things can shift dramatically within an individual science when a revolutionary advance happens, as in the emergence of quantum physics, molecular biology or cognitive or behavioral psychology. There is no clearly defined baseline “theory of science” that can be used to provide a generally acceptable, rigorous version of the notion that “extraordinary claims require extraordinary evidence.” Rather, our understanding of science and how it works is somewhat nebulous and seems to be continually evolving along with science itself. The and the study of scientific methodology seem mainly to have adapted to describe what scientists do, rather than providing a clear foundation to guide the work of science. And what scientists do is pretty diverse, and only partially systematic. Some have sought to formalize science on Bayesian grounds (Jaynes, 2003). Without accepting this as the end-all philosophy of science, we do consider this an interesting approach to understanding scientific claims and evidence. It has become common for psi researchers to publish Bayesian analyses of their results, often alongside traditional or frequentist statistical analyses. And when one views the matter in Bayesian terms, the need for extraordinary evidence to bolster extraordinary claims is not particularly clear.

If one takes the corpus of currently accepted scientific data to form one’s prior distribution, then one can assess new hypotheses conditionally on this corpus. In this framework, it is easily demonstrable that to bolster a that goes against currently accepted data in some blatant way, requires a lot more evidence than to bolster an hypothesis that agrees with currently accepted data. But this isn’t really relevant to the situation with most psi research. Psi results don’t tend to contravene specific data gathered in previous physics ; rather, they contravene hypotheses people have abstracted from the data gathered in previous physics experiments.

So, how can it be claimed that psi-based hypotheses should have a low probability according to a Bayesian approach, in spite of the apparently statistically significant results of many psi experiments in terms of Bayes factors? To make such a claim, one needs to argue that psi-based hypotheses should have a low prior probability for some reason. But why? The Bayesian perspective does not especially justify a psychological bias against psi, or against new hypotheses that appear “extraordinary” relative to currently accepted scientific hypotheses.

One could argue that psi is undesirable from an Occam’s Razor perspective; i.e. psi-based hypotheses are not simple explanations, because there is as yet no coherent, reasonably fleshed-out theory tying psi to physical mechanisms. But this also becomes somewhat subjective. Occam’s Razor is more an intuitive heuristic than a rigorous guide to formulating priors; and one person’s simplicity may be another’s complexity.4 For instance, one may choose to explain the corpus of ganzfeld results as resulting from either: A) a complex, ill-specified combination of experimental error, experimenter fraud, publication bias and questionable laboratory management methodology; or B) a weak but persistent psi phenomenon, with incomplete dependencies on a variety of factors. Which of these hypotheses is simpler, A or B? The choice of prior probability to assign to A versus B is driven by subjective preference, not evidence. One person might consider A more extraordinary than B; the other vice versa. The Bayesian approach to formalizing science provides no escape from the theory-dependence of evidence assessment.

Setting aside all these complexities and taking a more simplistic empiricist view, we note there is no clear evidence for the utility of the dictum that “extraordinary claims regard extraordinary evidence.” It is by no means clear, from the historical record, that science has been more successful in those cases where it has followed this dictum more closely. The various conceptual arguments for and against the dictum are difficult to weigh; and the reality of science and its historical and current handling of ordinary and extraordinary claims and evidence is a subtle beast.

4 Even if one formalizes Occam’s Razor mathematically, some arbitrariness remains. E.g., one can define the simplicity of an hypothesis as its length when expressed in some formal language. But then one must choose which formal language to use. All sufficiently powerful formal languages are equivalent in the limit, but this doesn’t necessarily help when processing finite datasets and real hypotheses about them. There is no known way to turn Occam’s Razor or any other heuristic into a truly objective, theory-independent means of choosing prior distributions. General Issues with Scientific Evidence and Methodology

The incomplete of our understanding of how science works, even in ordinary cases, is not a purely theoretical phenomenon, of concern to philosophers only. There are vexing practical difficulties with ordinary everyday science and its handling of theory and experiment, which are not well understood by most practicing scientists, and which make the issues dividing psi researchers and skeptics subtler than might be imagined.

Over the years, substantial evidence has emerged that—psi and other anomalous phenomena aside—science has a troublesome issue in not being demanding enough of ordinary claims. Frequently, scientific findings that make intuitive sense are accepted on the basis of a single study without rigorous replication. At times, this practice has led to serious errors. Social psychologist Jonathan Schooler built a successful career around a study of “verbal overshadowing” based on an initial experiment (Lehrer, 2010). The theory seemed reasonable and became part of the canon of mainstream social psychology. But years later, when Schooler decided to replicate the experiment, he found that it didn’t work nearly as well as it had in his own original study. The next year it was even worse.

This problem is not restricted to psychological and social sciences. The pharmaceutical firm Amgen recently repeated 53 “landmark” cancer research studies and found it could replicate only six of them (Begley and Ellis, 2012). This was a real problem, because drug companies often spend millions of dollars following up on these studies. Many drugs that apparently work well in initial studies seem to lose their effectiveness when follow-up studies are done. In a paper provocatively titled “Why Most Published Research Findings are False,” epidemiologist John Ionnidis (2005) argued that “for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.” The extent of this problem is currently under investigation; a recent set of replications of 13 important psychological studies by a large international group found that 10 could be replicated (Yong, 2013).

These problems seem to be due mainly to aspects of the social organization of science. Scientific careers depend on publication in selective refereed journals, and these journals select studies that appear to offer important new findings. Studies with negative results, or studies that replicate what has already been done, end up in the researcher’s “file-drawer.” Tests of statistical significance allow for a certain percentage of false-positive results, often set at 5%. So chance findings are likely to be published, and if they seem reasonable and “ordinary” no one has reason to question them.

Mainstream social and medical scientists have become acutely aware of these problems only in the last few years, but psi researchers have been sensitive to them for decades because of the way their research is relentlessly criticized by skeptics. This criticism continues even though the work of the psi researchers is often more rigorous methodologically than that of scientists in less controversial fields. Skeptic (2011) severely criticized social psychologist ’s (2011) article on retroactive published in a leading social psychology journal. But Bem’s article had been reviewed by four leading researchers in the field who examined it with exceptional care because of its counterintuitive findings. No such criticism would have been forthcoming if his findings had been “ordinary” rather than “exceptional.”

The Committee for Skeptical Inquiry’s web site defines as its mission “to promote scientific inquiry, critical investigation, and the use of reason in examining controversial and extraordinary claims.” They seldom if ever turn their skeptical eye on mainstream, “ordinary” science, even though this science might be in dire need of skeptical review. Others, however, have taken over the skeptical meme and used it to attack mainstream science in areas such as global warming and the theory of evolution. The CSI objects to the use of the word “skeptics” by those they call “global warming denialists,” some of whom have left the organization in protest. But they have not articulated a convincing rationale for being skeptical of some fields and not others, especially given recent findings about the non- of much mainstream research.

While skepticism has deep roots in Greek philosophy, today it is used whenever any group wants to deny any scientific finding. When I (Ted Goertzel) spoke at a CSI conference in New Orleans a few years ago, I noticed that they were selling a button with the simple slogan “I Doubt It!” I pointed out that they could sell the same button at a global warming skeptics conference, a holocaust revisionist conference, an conference, or a conference. They conceded the point, but a sticker with this slogan is still offered as a free promotion on their web site.

Skepticism, of course, is a valid and often valuable attitude and motivation. But the choice of where one’s skeptical attitude is directed, in practice, often reflects particular assumptions and biases. It is appropriate to be skeptical about the choices self-styled skeptics make, about where to direct their skeptical attention.

Some of the topics addressed in The Skeptical Inquirer are important to society, such as “alternative” medical claims, promotion of “recovered” memories that tear families apart, conspiracy theories that distort public policies, and attacks on vaccination, genetic and treatment for AIDS. The magazine plays a useful role in examining these areas. Other topics addressed in the magazine are amusing but not really harmful, such as stories, Big Foot and sightings, and tales of abduction by space aliens. But these topics are not really amenable to scientific testing. Psi has the advantage of getting easy laughs and also being amenable to scientific testing, so it is likely to be of continuing interest to the skeptics.

Skeptical Analyses of Psi Phenomena

Things really haven’t changed much since observed (Caldwell, no date) that skeptics “are not agnostic towards claims of the paranormal, they are out to knock them. When an experiment of the paranormal meets their requirements, then they move the goal posts. Then, if the experiment is reputable, they say it's a mere anomaly.” Adopting Bayesian statistics doesn’t seem to help either, since the priors can be set in different ways, and with reasonable priors the results of psi experiments are not very different with Bayesian or traditional statistical methods.

Meta-analytic studies have been seen as a promising method for increasing sample sizes and providing more rigorous tests of hypotheses with weak effect sizes. But the results vary depending on the criteria used to select studies, the weighting of studies, and the point in time at which they are selected, so there are often meta-analytic studies on both sides of a question. In an effort to encourage widespread replication of his controversial “Feeling the Future” studies, Bem (2011) provided software for exact replication. This led to a great many replications, many of which, however, changed the procedures in various ways. Skeptics were quick to publish replications and even to do meta-analytic studies even though the number of studies was limited (Ritchie et al, 2012; Galak, et al, 2012). Journals were quick to publish the earliest studies, with negative results, rather than waiting for the full body of studies to be published. A more complete meta-analysis showed a success rate similar to Bem’s original research for those studies that replicated his procedures strictly (Tressoldi, et al, forthcoming).

Skeptics sometimes resort to dramatic public relations gestures instead of conducting rigorous research. Stage magician and professional skeptic James Randi has gained considerable press via his “million dollar” psi challenge (Wikipedia, 2003). However, the conditions of this challenge are not at all sensibly laid out, given the empirical data regarding psi phenomena. Randi’s preliminary test, which must be passed before an applicant can try for the million, demands odds against chance of 1,000 to 1. The second test, to win the million, requires the applicant to show results at better than a million to one against chance. Of course, all psi researchers would love to be able to produce, within a single laboratory experiment, results providing such a compelling demonstration of psi phenomena. But the available corpus of psi results suggests strongly that, if psi is a real phenomenon, this just isn’t how it works.

It seems clear from the available data that, if psi is a real phenomenon, it is generally weak and finicky, and varies based on a host of hard-to-pin-down variables, emerging strongly only under specific conditions that are hard to engineer in the lab. These aspects of the available psi data are frustrating to deal with; but scientific research is often frustrating in many different ways, and must be grappled with on its own terms. Positing and publicizing challenges that don’t match the available data regarding the phenomenon being studied is not the way to advance understanding. Randi might have done some good by exposing some quacks and , but he has also done damage via his glib dismissal of the whole corpus of serious psi research. He has encouraged the public to view psi in a way that doesn’t distinguish between theatrical, money-seeking alleged-psi practitioners like television faith healers, and scientists doing serious research on potential psi phenomena in the laboratory. But any careful, reasonably dispassionate observer, regardless of their views on the reality of psi, would have to admit these two communities have almost nothing in common.

Some skeptics have engaged with the psi research literature and community much more seriously than Randi. A classic example of this was the debate in the 1980s between skeptic Ray Hyman and psi researcher regarding meta-analysis of ganzfeld ESP experiments. In this exchange, Hyman did look in detail at the ganzfeld data available at the time, and proposed his own meta-analysis of the overall corpus of data, getting deep into specific criticisms of various experiments and analytic methodologies. But his conclusions were confusing and ambiguous.

In a published “Joint Communiqué” co-authored with Honorton, Hyman noted that “there is an overall significant effect in this data base that cannot reasonably be explained by selective reporting or multiple analysis” and that “significant outcomes have been produced by a number of different investigators” (Hyman and Honorton, 1986). Yet, in his contribution to a National Research Council report covering , he included no such comments and presented a more negative perspective (Druckman & Swets, 1988; for a response to the NRC report, see Palmer, Honorton, & Utts, 1989). In the years since the Joint Communiqué, Hyman has continued his role as an unusually research-literature-savvy and detail-oriented psi skeptic. Among his complaints has been that there is no clear definition of what “psi” entails, aside from anomalous results not clearly explicable in terms of known science. In 2007 he opined that:

Until parapsychologists can provide a positive way to indicate the presence of psi, the different effect sizes that occur in experiments are just as likely to result from many different things rather than one thing called psi. Indeed given the obvious instability and elusiveness of the findings, the best guess might very well be that we are dealing with a variety of Murphy's Law rather than a revolutionary anomaly called psi. (Hyman, 2007).

Here Hyman is using humor (“Murphy’s Law”) to sidestep the fact that he hasn’t yet managed to deliver a knockout blow to the ganzfeld data. Yes, the findings regarding potential psi phenomena, in the ganzfeld paradigm and otherwise, are obviously unstable and elusive. And yet, with all his skeptical statistical prowess, Hyman hasn’t quite managed to eliminate the presence of something peculiar and anomalous in the ganzfeld data.

And ganzfeld ESP research goes on, alongside research on many other experimental psi . In 2010, Storm et al analyzed 29 ganzfeld studies from 1997 to 2008. Of the 1,498 trials, 483 produced hits, corresponding to a hit rate of 32.2%, and providing statistical significance with p < .001. Participants selected for personality traits and personal characteristics thought to be psi-conducive were found to perform significantly better than unselected participants. Shortly afterwards, Hyman (2010) published a rebuttal to Storm et al; and Storm (2010a) published a rebuttal to Hyman’s rebuttal. There was a later meta-analysis by Williams (2011), discussing the relation between all the previous ones. The game continues, here much as in some other contentious areas of science.

Over the years, Hyman’s published views have maintained that there is not yet extraordinary enough evidence to consider psi a reality, but yet that the existing corpus of psi data contains anomalous results that he hasn’t yet figured out how to explain away. This sort of measured, ambiguous response is what generally ensues when skeptics take the psi data seriously. However, a Hyman-Honorton style academic debate, or a long trail of papers containing dueling statistical meta-analyses, are not nearly as exciting as a million dollar prize and the promise of debunking fraudsters, and so Randi and his ilk tend to get far more press and popular attention.

In encounters like Hyman versus Honorton, or Hyman versus Storm, one sees clear examples of the role prior assumptions play in guiding scientists’ consideration of theories and evidence. From Hyman’s view, the prior probability of psi being real is very low, so that nebulous, as-yet undetermined potential alternative explanations of the admittedly anomalous aspects of the ganzfeld data still seem more plausible. From Honorton’s or Storm’s point of view, the prior probability of psi being real is reasonably high (based, e.g., on the numerous anecdotal reports of psi phenomena), so that a direct psi-based explanation of the ganzfeld data seems far simpler than an unspecified non-psi-based alternative.

Bridging the Gap

Given the attitude of the skeptical community and the nature of current psi research, the conflict between psi researchers and skeptics/denialists seems unlikely to be resolved anytime soon. The psi community seems justified in focusing on moving their research forward in ways that make sense from their own point of view, rather than on convincing the skeptics. However, it is partly due to certain systematic weaknesses in the corpus of evidence for psi that the skeptical community finds such fertile ground in psi for ongoing criticism. Resolving these weaknesses seems important if psi research is to move forward dramatically, for reasons beyond the value of convincing skeptics.

Arguably, the most important challenge for psi researchers is to increase the effect size in their experiments. There have been calls for increasing the number of replications that reproduce a study as exactly as possible (LeBel and Peters, 2011). But replication of studies will not be very convincing to skeptics as long as the binary success rates are only 53% when compared to 50% expected by random chance. The only way to improve the effect rate is to keep running experiments with different parameters and under different conditions, rather than repeating the same experiments over and over without change. To the extent that resources are limited, it would seem more worthwhile to put them into efforts to untangle the causal mechanisms and increase the effect size.

Statistically, this is in keeping with recent advocacy of replacing null hypothesis significance testing with confidence intervals, measures of effect size and other estimation techniques (Cumming, 2013). This is good advice for researchers in all areas, including psi research, and psi researchers have been at the forefront in incorporating these measures in their publications.

Skeptics and the general public would also be impressed if practical benefits could be shown for using psi techniques. The U.S. military gave up its research project on psi, not because they were certain it did not exist, but because they couldn’t find a practical military use for it. Similarly, Sony’s successful ESP research program was shuttered because the firm couldn’t figure out how to make their statistically significant but highly erratic ESP results contribute to the company’s bottom line. As Sony spokesman Masanobu Sakaguchi told the South China Morning Post when questioned about the Sony ESP project and its termination after nearly ten years of research: “We found out experimentally that yes, ESP exists, but that any practical application of this is not likely in the foreseeable future” (Semel, 2013).

It’s natural to wonder why psi has not been more extensively and explicitly applied in the gambling domain. Millions of people are already risking their hard earned money trying to anticipate the behavior of roulette wheels. If psi could get their success rate up to 53%, the consequences would be quite convincing. Indeed, some psi researchers have reported limited success with the use of psi for gambling (noted in Schwartz’s chapter in this volume); but there are have been no well-documented dramatic long-term successes. There have also been some theoretical arguments regarding why this sort of application might be unlikely to succeed (Moddel, 2004). Still, this could be an interesting avenue for future exploration.

Alternately, a detailed understanding of causal mechanisms underlying psi would shift the landscape of psi research dramatically. If we had an underlying scientific explanation that told us more exactly why psi experiments have such small effect sizes, and why psi phenomena depend on so many factors in such complex ways, then these complicating aspects of psi would be a lot less troubling. Experiments aimed at understanding underlying causal mechanisms, rather than merely further gathering statistical evidence in favor of the of psi phenomena, would seem a high priority.

All in all, until the effect size goes up substantially, or some practical benefit is demonstrated, or underlying causal mechanisms are much better understood, psi is likely to remain a marginal research area. The demand for “exceptional evidence,” never precisely defined, will be used to deny findings even when they meet usual scientific standards; and the debate between psi researchers and skeptics will likely persist in familiar fashion, as the meta-analyses continue to accumulate. References

Alcock, James. (2011, January 6). “Back from the Future: Parapsychology and the Bem Affair,” Skeptical Inquirer. Accessed at: http://www.csicop.org/specialarticles/show/back_from_the_future

Balashov, Yuri and Alex Rosenberg (2001). Philosophy of Science: Contemporary Readings. Routledge.

Begley, Glenn and Ellis, Lee. (2012). “Raise standards for preclinical cancer research,” Nature 483: 531-533. Retrieved from: http://www.panoptic.be/plicploc/blog/wp-content/uploads/2012/06/raise- standards-483531a.pdf.

Bem, D. J. (2011, January 31). Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect. Journal of Personality and Social Psychology. doi: 10.1037/a0021524. Retrieved from: http://caps.ucsf.edu/wordpress/wp-content/uploads/2011/02/bem2011.pdf.

Bem, Daryl J. (2011, January 6). “Response to Alcock’s “Back from the Future: Comments on Bem,” Skeptical Inquirer Retrieved from: http://www.csicop.org/specialarticles/show/response_to_alcocks_back_from _the_future_comments_on_bem.

Caldwell, Daniel. (no date). “Parapsychology, Anomalies, Science, Skepticism, and CSICOP,” retrieved at: http://blavatskyarchives.com/zeteticism.htm.

Carter, Chris. (2012). Science and Psychic Phenomena: The Fall of the House of Skeptics. Inner Traditions Press. Cumming, Geoff. (2013, November 12). “The New Statistics: Why and How,” Psychological Science Online First doi:10.1177/0956797613504966.

Delgado-Romero, Edward and Howard, George. (2005). “Finding and Correcting Flawed Research Literatures,” The Humanistic Psychologist 33(4): 293-303.

Druckman, D. and Swets, J.A. (Eds.). (1988). Enhancing Human Performance: Issues, Theories, Techniques. Washington, DC: National Academy Press.

Frazier, Kendrick (1996). “Committee for the Scientific Investigation of Claims of the Paranormal (CSICOP),” pp. 168-180 in The Encyclopedia of the Paranormal, Gordon Stein, editor, Prometheus Books. Retrieved from: http://www.csicop.org/about/csicop/.

Frazier, Kendrick (1998). Encounters with the Paranormal: Science, Knowledge and Belief. Prometheus Books.

Frazier, Kendrick. (2013, March). “Failure to Replicate Results of Bem Parapsychology Experiments Published by Same Journal,” Skeptical Inquirer 37.2. Accessed at: http://www.csicop.org/si/show/failure_to_replicate_results_of_bem_parapsy chology_experiments_published_by/

Galak, Jeff; LeBoeuf, Robyn; Nelson, Leif; Simmons, Joseph. (2012, December). “Correcting the Past: Failures to Replicate Psi,” Journal of Personality and Social Psychology 103(6): 933-948.

Hyman, R., and Honorton, C. (1986). A joint communiqué: the psi ganzfeld controversy. Journal of Parapsychology, 50. pp. 351-364. Hyman, Ray (2007).. Evaluating Parapsychological Claims in J. Sternberg, Henry L. Roediger, Diane F. Halpern. in Psychology. Cambridge University Press. pp. 216-231

Hyman, R. (2010). Meta-analysis that conceals more than it reveals: Comment on Storm et al. (2010). Psychological Bulletin, 136. pp. 486-490.

Ioannidis, John. (2005, August 30). “Why Most Published Research Findings Are False,” PLOS Medicine. DOI: 10.1371/journal.pmed.0020124. Accessed at: http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.p med.0020124.

Jaynes, E.T. (2003). Probability Theory: The of Science. Cambridge University Press.

Kurtz, Paul. (2001). Skeptical Odysseys: Personal Accounts by the World's Leading Paranormal Inquirers. Prometheus Books.

LeBel, Etienne and Peters, Kurt. 2011. “Fearing the Future of Empirical Psychology: Bem’s (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice,” Review of General Psychology, pp 1-9. DOI: 10.1037/a0025172.

Lehrer, Jonah. (2010, December 13). “The wears off: is there something wrong with the ?” The New Yorker. Retrieved From: http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer.

Margolis, Jonathan. (1999). : Magician or Mystic. Welcome Rain Publishers. McGreal, Scott. (2012, October 8). “: Science or Fanta-Psi?” Psychology Today. Retrieved from: http://www.psychologytoday.com/blog/unique-everybody- else/201210/precognition-science-or-fanta-psi.

Meehl, Paul. (1978). “Theoretical Risks and Tabular Asterisks: Sir Karl, Sir Ronald, and the Slow Progress of Soft Psychology,” Journal of Consulting and Clinical Psychology, 46(4): 806-834.

Moddel, Garret. (2004). Entropy and Subtle Interaction. Journal of Scientific Exploration, Vol. 18, No. 2, pp. 293–306, 2004

Palmer, J.A., Honorton, C. & Utts, J. (1989). Reply to the National Research Council study on parapsychology. Journal of the American Society for Psychical Research, 83, 31-49.

Rawlins, Dennis. (1981, October). “sTARBABY,” Fate Magazine 34: 67-98. Retrieved from: http://cura.free.fr/xv/14starbb.html

Ritchie SJ, Wiseman R, & French CC (2012). Failing the future: three unsuccessful attempts to replicate Bem's 'retroactive facilitation of recall' effect. PloS one, 7 (3) PMID: 22432019.

Schick Jr., Theodore. (2002). “Do Extraordinary Claims Require Extraordinary Evidence? A Reappraisal of a Classic Skeptics’ Axiom.” Pp. 327-333 in The Skeptic: Encyclopedia of . ABC-CLIO.

Semel, Gene (2013). E.S.P. Exists: Inside Sony’s Corporate Research. Disinfo.com, http://disinfo.com/2013/03/e-s-p-exists-inside-sonys-corporate- research/. Downloaded Nov 24, 2013. Storm, Lance, Patrizio Tressoldi and Di Risio (2010). “Meta-Analysis of Free-Response Studies, 1992–2008: Assessing the Noise Reduction Model in Parapsychology.” Psychological Bulletin, 138 (4): 471–85.

Storm, L., Tressoldi, P. E., & Di Risio, L. (2010a). A meta-analysis with nothing to hide: Reply to Hyman (2010). Psychological Bulletin, 136, 491– 494.

Tressoldi, P. E. (2011). “Extraordinary claims require extraordinary evidence: The case of non-local perception, a classical and Bayesian review of evidences.” Frontiers in Psychology, 2, 1–5. doi:10.3389/fpsyg.2011.00117.

Tressoldi, Patrizio; Rabeyron, Thomas; Duggan, Michael; and Bem, Daryl. (forthcoming). “Feeling the Future: a Meta-analysis of Experiments on the Anomalous Anticipation of Random Future Events.”

Truzzi, Marcello. (1978). “On the Extraordinary: At Attempt at Clarification. Zetetic Scholar 1: 11-22. Retrieved at: http://rr0.org/time/1/9/7/8/Truzzi_OnTheExtraordinaryAnAttemptAtClarific ation/index.html.

Truzzi, Marcello. (1989, November 29). “Reflections on the Reception of Unconventional Claims in Science,” colloquium presented at Eastern Michigan University. Retrieved from: http://www.ufoevidence.org/documents/doc1226.htm.

Yong, ed. (2013, November 26). “Psychologists strike a blow for reproducibility,” Nature:International Weekly Journal of Science. doi:10.1038/nature.2013.14232. Wagenmakers, E. J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100, 426–432. doi:10.1037/a0022790

Wikipedia (2003). One Million Dollar Paranormal Challenge, http://en.wikipedia.org/wiki/One_Million_Dollar_Paranormal_Challenge. Downloaded Nov 24, 2013.

Williams, Bryan (2011). Revisiting the Ganzfeld ESP Debate: A Basic Review and Assessment.