6/20/2014 Explicit Anomalous Cognition: A Review of the Best Evidence in Ganzfeld, Forced-choice, Remote Viewing and Dream Studies Johann Baptista, Max Derakhshani and Patrizio Tressoldi To appear in The Parapsychology Handbook. Etzel Cardeña, John Palmer, and David Marcusson-Clavertz Eds. In its quest to demonstrate the reality of psi phenomena to the mainstream scientific community, parapsychology has rarely been afforded the luxury of dealing in averages. On the contrary, the field has had to develop a progressive ethos, a long and distinguished tradition of adopting the more rigorous and innovative techniques in science—sometimes even to the point of creating them. It is no coincidence, for example, that the first comprehensive meta-analysis in scientific history was performed by Rhine et al. on ESP card studies (Bösch, 2004), or that the first official policy of publishing null results was set out by the Parapsychological Association (Carter, 2010). Neither is it trivial that psi research has kept pace with associated mainstream and behavioral fields in terms of reproducibility—on a minimal budget (Baptista & Derakhshani, 2014); exceeded their criteria with respect to masking practices (Watt & Nagtegaal, 2004); and exceeded them again in terms of reporting negative results (Mosseau, 2003)1 . It should indeed be this way, for not only has parapsychology been subject to (and strengthened by) an intensified back-and-forth between proponents and skeptics along its history, but the claims it propounds have always demanded high standards of evidence. This is a fact recognized by sides of the psi debate. Our contribution to the Handbook is an attempt to further this point of agreement. We propose changes that will add constructively to the face validity of parapsychological claims by combating potential flaws before they occur—changes informed by our review of four key, conceptually simple domains of experimentation: ganzfeld, forced-choice ESP, Remote Viewing, and dream ESP. Subsequent to our review of these studies, we provide further suggestions on the statistical treatment of meta-analysis, applicable to ESP research generally. Although the evidence for explicit anomalous cognition (EAC) so far produced in parapsychology has been resistant to criticism and demonstrated compelling trends (Storm, Tressoldi, & DiRiso, 2010; Tressoldi, 2011; Baptista & Derakhshani, 2014), in the face of our analysis we find the most prudent course of action is to agree with our critics that the evidence can be—and ought to be— improved, if for no other reason than that improvement is inherent to the parapsychological enterprise. Improvement consists in obviating potential flaws before they have the chance to occur (whether or not they do) and/or in making the EAC effect more robust, generally (so as to a priori rule out explanations based on flaws). Ganzfeld We review the ganzfeld studies first by summarizing their history from the early 1970s until the early 2000s. Then we focus on the most contemporary evidence available, from the Storm, Tressoldi, and DiRisio (STDR; 2010) database. Afterwards, we present several new meta-analytic findings from Baptista and Derakhshani (2014) relevant to the current state of the research—as well as to other altered states EAC studies. Early History From 1974 to 1981, 42 ganzfeld studies were conducted by 47 different investigators, analyzed by both Hyman (1985) and Honorton (1985). Hyman provided a skeptical appraisal, identified methodological weaknesses, and performed a factor analysis that found evidence of a significant 1 For a comprehensive overview of various benchmarks of good science, as well as how parapsychology fares on them, we refer the interested reader to the cited papers, with emphasis on Mosseau (2003). correlation between specified flaw indices and ganzfeld rates of success. In response, Honorton criticized Hyman's flaw categorizations and asked psychometrician David Saunders to comment on his method, to which Saunders (1985) stated that factor analysis was inappropriate given the small sample size of Hyman's database. Honorton also attempted to counter Hyman's multiple-analysis criticism by meta-analyzing only the 28 ganzfeld studies in the original database of 42 that used direct hits as their measure of success. He found an unweighted Stouffer’s Z = 6.6, p = 10-9, where 43% of the studies were significant at the p < .05 level, with a hit rate (HR) of 36.87% for the 26 studies of four-choice design. The debate ended with the publication of the Joint Communiqué (Hyman & Honorton, 1986), in which the authors agreed that there was an overall effect in the database but differed on the extent to which it constituted evidence for psi. Stricter methodological guidelines were proposed by both authors, and a consensus was reached that “the final verdict will await the outcome of future experiments conducted by a broader range of investigators and according to more stringent standards.” p. 351. Eight years later, Bem and Honorton (1994) published a meta-analysis of 10 automated ganzfeld studies conducted in Honorton’s Psychophysical Research Laboratories (PRL) designed to meet these more stringent standards, comprising 329 sessions in total, with a Z = 2.89, p = .002, and an hit rate of 32.2%. This successful effort made inroads towards the aims of the Joint Communiqué and was promoted by the authors as evidence that the ganzfeld psi effect was both robust and reproducible. However, a later meta-analysis by Milton and Wiseman (1999) conducted on all the studies that came after the PRL (from 1987 to the later cutoff of 1997) did not yield similar findings. Comprising 30 studies, it reported a null result: Z = 0.70, ES = 0.013, p = .24, hit rate = 27.55%. Milton and Wiseman surmised from this that the PRL had not been replicated, and that the ganzfeld paradigm did not provide evidence for psychic functioning—a conclusion that sparked an involved debate in the parapsychology community (Schmeidler & Edge, 1999), and was frequently mentioned in skeptical circles as evidence of the irreproducibility of psi. Although the appropriateness of the statistical test in Milton and Wiseman (1999) has been questioned (Carter, 2010; Baptista & Derakhshani, 2014), its identification of a significantly reduced overall effect size for the ganzfeld database has not. The hit rate drop from 32.2% (PRL) to 27.6% (Milton & Wiseman, 1999) required an explanation. Two years afterwards, a potential one was supplied by Bem, Palmer, and Broughton (2001). Bem et al. presented a meta-analysis of their own, arguing that the reason for Milton and Wiseman's (MW's) effect size plunge was that many studies subsequent to the PRL studies had employed novel, nonstandard procedures that were at greater risk for failure (such as Willin, 1996, which used musical targets). Their meta-analysis added 10 new studies published after the MW cut-off and included a 7-point standardness scale for each study, where ranks were awarded by masked raters. For this database of 40 studies, Bem et al. (2001) found a hit rate of 30.1%, an ES = 0.051, and a Stouffer’s Z = 2.59, p = .0048. Additionally, their hypothesis was supported by the fact that those studies that ranked above the midpoint (4.0) for standardness yielded significant results at a hit rate of 31.2% (1278 trials, 29 studies, exact binomial p = .0002) and those that fell below the midpoint supplied a nonsignificant hit rate of 24%—and the difference was significant (U = 190.5; p = .020). But their analysis possessed one weakness: with the addition of the 10 new studies, one of which was the very successful Dalton (1997) study with artistic subjects—hit rate of 47% and 128 trials— their new hit rate of 30.1% was already independently highly significant. Additionally, as Baptista and Derakhshani (2014) argued, since almost 100% of PRL subjects had at least one of the psi conducive traits used by Storm et al. (2010) in their criteria for selected subjects2 (88% personal psi experience; 99.2% above midpoint on psi belief; 80% training in meditation; Honorton, 1990), it is appropriate to ask what the hit rate was for subjects who possessed these traits in the Milton and Wiseman database. It turns out to be between 30.88% and 34.2% (see next section). So Bem, Palmer, and Broughton's result should be taken with a grain of salt; nevertheless, that the hit rate for their studies rose to 33% (974 trials, 21 studies) for experiments that ranked 6 or above on the similarity scale—almost exactly PRL's own hit rate— shows that study standardness at least captured something that correlated positively and reliably with experimental results. Aside from Bem, Palmer, and Broughton's (2001) comparative meta-analysis, one other meta-analysis was published in the same year by Storm and Ertel (2001) which included all 79 reported studies from 1982 to 1997, combining the 10 PRL studies in Bem and Honorton (1994) with Honorton's (1985) and Milton and Wiseman's (1999) databases: ES = 0.138, Stouffer’s Z = 5.66, p = 7.78 × 10-9. They criticized Milton and Wiseman’s method of analyzing only the post- PRL studies and concluded that, despite their dip in effect sizes, the ganzfeld experiment remained a viable method for eliciting psi and would benefit from further replication. A Reanalysis of the Milton and Wiseman Database As was commented above, it is appropriate to ask what percentage of participants in the Milton and Wiseman database had "selected" characteristics, and what their success rate was. The answer to this question is not so straightforward. In procuring an approximate one, it is well to make a distinction between "selected subjects"—participants explicitly selected on the basis of Storm et al.'s criteria—and "happenstance" subjects who fortuitously possessed at least one psi- conducive trait.