Over at Boing Boing, Cory Doctorow has posted a story (contributed by Clay Shirky) on the attempted replications of Daryl Bem’s controversial ‘feeling the future‘ precognition experiments, which he has titled “ESP proponents claim that ESP skeptics are psychic, and use their powers to suppress ESP” (linking to the original story “Wait, Maybe You Can’t Feel the Future”):
Clay sez, “Stuart Ritchie, a psychology doctoral student in Edinburgh, worked with two colleagues to try to replicate the results of a famous recent experiment, claiming people could predict in advance whether they were about to be shown erotic images. When the three failed to find any such evidence for ESP they sent their results out for publication, and the British Psychology Journal, one of the journals to which it was sent, in turn sent the trio’s article out for review. When Ritchie et al got the responses back ‘…there were two reviews, one very positive, urging publication, and one quite negative. This latter review didn’t find any problems in our methodology or writeup itself, but suggested that, since the three of us (Richard Wiseman, Chris French and I) are all skeptical of ESP, we might have unconsciously influenced the results using our own psychic powers.’ They are still looking for a place to publish their findings.
Now personally, I don’t agree with the reviewer’s grounds for rejecting the replication study. Though I’ve read about the alleged ‘experimenter effect’ before, and consider it an interesting sideline topic, I think the paper by Ritchie, Wiseman and French deserves to be published regardless. Additionally, they already *do* address the possibility of the experimenter effect in their paper explicitly, in referring to the original experiment set-up by Bem:
When discussing the issue of replication, Bem drew special attention to the role of experimenter effects, arguing that a skeptical experimenter might be more likely to obtain a null effect than one more open to the possibility of psychic ability. To help overcome this potential issue, Bem describes how he specifically designed the study to be run by a computer (thus minimizing the experimenter’s role) and using undergraduate experimenters that were given only informal training. In line with these guidelines, only Replication 1 was carried out by the Principal Investigator – Replication 2 was conducted by the Principal Investigator’s research assistants, and Replication 3 was carried out by an undergraduate student as part of a project being supervised by the Principal Investigator.
(Though there could be an argument that Replication 1 did not address the issue raised by Bem, and the further possibility that the research assistants and undergraduates under supervision may have shared the skeptical view of the Principal Investigator due to their working relationship).
What I wanted to address more though is the handling of it at Boing Boing. Perhaps I’m reading too much into it, but the title seems to me to be designed to say “those whacko ESP proponents, justifying their belief in any way they can”. The addition of the ‘Psychic Reader’ image helps reinforce the woo-woo component. The problem is this: though it has been noted for decades, the ‘experimenter effect’ has come to the fore in recent years chiefly due to research performed by Richard Wiseman, one of most high-profile *skeptics* in the world. In a joint study with ‘psi proponent’ Marilyn Schlitz, of the Institute of Noetic Studies, Wiseman found (very tentative) evidence that it might be possible that results may differ depending on the attitude of the researcher. In “Experimenter effects and the remote detection of staring” (full-text PDF), Wiseman and Schlitz discussed the possible explanations for the discrepancy in Wiseman’s negative results and Schlitz’s positive results:
Finally, it is also possible that both RW and MS used their own psi abilities to create the results he/she desired. This interpretation, if genuine, supports past research which suggests that ‘successful experimenters’ (i.e., those that consistently obtain significant effects in psi studies) outperform ‘unsuccessful’ ones on a variety of psi tasks (see Palmer, 1986 for a review of the literature supporting this notion).
Wiseman and Schlitz have collaborated three times on investigating experimenter effects – the first two resulted in positive evidence, but the most recent experiment failed to replicate their previous findings. Their conclusion at this time regarding the experimenter effect was that “the inconsistent nature of our findings does not allow for a firm acceptance or rejection of either interpretation and the issue will only be resolved by further research”.
Now, given that Bem explicitly notes his concern over ‘experimenter effect’, and further that Richard Wiseman was also one of the co-authors of the failed Bem replication, some might say it’s fair enough to raise the experimenter effect as a possible variable, given it’s part of Wiseman’s own research corpus and that he has actively stated it as a possible cause of failed results. As I said at the beginning, I do think it’s worthy of publication all the same. But the Boing Boing title is misleading, and leads to a vast comment thread with a number of boorish and uneducated ‘skeptical’ comments. Which is a shame, because it’s a fascinating field of research no matter what the final outcome is.
You might also like…
