For ten years, the modern skeptical movement has wielded a cudgel against claims of the paranormal: the James Randi Million Dollar Challenge [2]. In many debates over the possibility of psi abilities, the Challenge provides a final word for one side..."has so-and-so applied for the Challenge?" The financial reward offered by the James Randi Educational Foundation is seen by many skeptics as providing an irresistible motivation for anybody with paranormal ability - after all, if someone could genuinely exhibit such powers, surely they would step forward to take the million?

However, after ten years, the James Randi Educational Foundation (JREF) says nobody has even got past their preliminary testing. Furthermore, none of the 'big fish' - medium John Edward, spoon-bender Uri Geller, psychic Sylvia Browne - have applied (although Sylvia Browne did accept [3] James Randi's direct challenge on Larry King Live, without going any further). And now, perhaps as a result of that fact, James Randi has announced that the Challenge will come to an end in two years, on March 6th, 2010.

But does the challenge really make a statement about the existence of the paranormal and/or psi abilities? According to paranormal investigator Loyd Auerbach (who, like Randi, is a member of the magic fraternity):

The suggestion that ending the Challenge after 10 years supports any statement that psi does not exist or someone would have won the challenge, is absurd on many levels.

The procedures for the Challenge included several hurdles in favor of, and multiple "outs" for Randi and the JREF that any discerning individual capable of any kind of extraordinary human performance would think twice about (and here I'm not just referring to psychics and the like).

What are these hurdles that Auerbach refers to?

Chances, of Anything...

First, and perhaps the most important, is the effect size required to win the challenge. While the JREF says that "all tests are designed with the participation and approval of the applicant", this does not mean that the tests are fair scientific tests. The JREF need to protect a very large amount of money from possible "long-range shots", and as such they ask for extremely significant results before paying out - much higher than are generally accepted in scientific research [4] (and if you don’t agree to terms, your application is rejected). In the case of parapsychological research, however, where effect size is often small (though apparently robust), this means most researchers would have to go to extraordinary lengths to win the million dollars. As one psi researcher pointed out to me:

In the ganzfeld telepathy test [5] the meta-analytic hit rate with unselected subjects is 32% where chance expectation is 25%. If that 32% hit rate is the "real" telepathy effect, then for us to have a 99% chance of getting a significant effect at p < 0.005, we would need to run 989 trials. One ganzfeld session lasts about 1.5 hours, or about 1,483 total hours. Previous experiments show that it is not advisable to run more than one session per day. So we have to potentially recruit 989 x 2 people to participate, an experimenter who will spend 4+ years running these people day in and day out, and at the end we'll end up with p < 0.005. Randi will say those results aren't good enough, because you could get such a result by chance 5 in 1,000 times. Thus, he will require odds against chance of at least a million to 1 to pay out $1 million, and then the amount of time and money it would take to get a significant result would be far in excess of $1 million.

Furthermore, applicants must first pass a 'preliminary test', before they are allowed to progress to the actual 'formal' test which pays the million dollars. So an applicant must first show positive results in a preliminary test (yielding results against chance of at least 1000 to 1, apparently [6]), then once through to the next stage they would then have to show positive results against much higher odds to claim the prize (by all reports, at odds of around 1 million to 1). Failure in either test means no cash prize, and a fail beside their name. It many respects it would be like telling a professional golfer to shoot 63 around Augusta National, then come back and shoot 59, to prove that he can play golf. In the words of Chris Carter, author of Parapsychology and the Skeptics:

If Randi were genuinely interested in testing unusual claims, then he would also not insist upon odds of at least one million to one against chance for the results. Anyone familiar with scientific studies will be aware that experimental results against chance of say, 800,000 to one would be considered extraordinary; but results this high would be, according to Randi, a “failure.”

Dr Michael Sudduth of San Francisco State University also pointed out to me a wonderful irony in one of the rules. Challenge rule #3 states: "We have no interest in theories nor explanations of how the claimed powers might work." As Sudduth puts it: “Curiously, Randi's challenge itself is saddled with assumptions of this very kind. The challenge makes little sense unless we assume that psi is the sort of thing that, if genuine, can be produced on demand, or at least is likely to manifest itself in some perspicuous manner under the conditions specified by the challenge.”

Researchers Step Up to the Plate

As a consequence, you might well say "no wonder no serious researcher has applied for the Challenge." Interestingly, this is not the case. Dr Dick Bierman [7], who has a PhD in physics, informed me that he did in fact approach James Randi about the Million Dollar Challenge in late 1998. Bierman reported a success in replicating the presentiment experiments of Dr Dean Radin [8] (where human reactions seem to occur marginally before an event occurs), and was subsequently asked by Stanley Klein of the University of California why, if his results for psi effects were positive and replicable, he didn't respond to Randi's challenge. Bierman replied that he would rather invest his time in good scientific research, rather than convincing skeptics in a one-off test. However, after further discussion, he decided that he may be able to combine the two:

After some exchange of ideas I was brought into contact with Randi. Randi sounded sincerely interested and I worked out a proposal for an interesting experiment that would last about a year. Experimental effects in this type of research are small and require a lot of measurements to reach the required statistical significance (I think Randi wanted a p-value of 0.000001).

Note that he didn't insist on showing the effect on stage. Rather I proposed to do a kind of precognition (actually presentiment) experiment on-line over Internet where he or some other independent skeptic could generate the targets once the responses were communicated over the Internet (all this would be done automatically on a computer under his control within a second). This would prevent cheating from the experimenter's side but we still had to work out how to prevent cheating from the Randi-side.

At that point Randi mentioned that before proceeding he had to submit this preliminary proposal to his scientific board or committee. And basically that was the end of it. I have no idea where the process was obstructed but I must confess that I was glad that I could devote myself purely to science rather than having to deal with the skeptics and the associated media hypes.

Bierman said I should also contact Suitbert Ertel, Professor Emeritus of Georg-August-University of Göttingent, who has developed a new type of parapsychology experiment [9] which seems to facilitate large-scale psi effects - which would be much more suited to the Randi challenge. Ertel, I was told, had apparently also discussed the challenge with James Randi, after his results had been replicated by other skeptical researchers. Ertel replied to my query by explaining his involvement with not just Randi's challenge, but also a separate 'Prize Challenge' offered by a German skeptical group:

My first approach [to Randi] was made because I thought the prize might be achieved by the Gauquelin planetary effect, a statistical "paranormal" or "neo-astrological" effect, with which I was very familiar as researcher. The problem was that decisions regarding the sample which would amount to 1000 natal charts was dependent on much informed thought, and Randi didn't know how to deal with the conditions. So the correspondence came to an end.

The second approach was made because I had applied to win the prize of 10,000 EURO which the German GWUP promised to give to someone who would be able to demonstrate large psi effects. Winning this prize would have been considered by Randi as passing his preliminary test, his first test which must be passed before someone is allowed to apply for Randi's main $1 000 000 test.

The psi effect demanded, even for the GWUP test = Randi's preliminary test, was so large that I was not hopeful that I would be able to show so much of psi, with the help of my psi-gifted students which I selected by my "pingpong ball test". My only goal was to achieve a statistically significant effect so as to make the skeptics admit that they observed a significant psi effect. This goal was achieved by my first test trial (one psi-gifted participant) in 2005. In 2006 another test was conducted with the presence of GWUP people: two of my students, psi-gifted in earlier tests, participated. In this test the effect was not significant.

One of the apparent reasons for this failure was that the skeptics had changed the conditions of this test arbitrarily in many ways so that the participants felt uneasy under strong control - such feelings have psi-reducing effects.

Ertel's first test with the GWUP had a p value of .018. He mentioned however, that two additional students among a number of observers also participated, secretly, during the test. Their results were also significant, giving a total significance p-value of .002. Ertel told me that the GWUP skeptics, to their credit, did note the results of the two students who had participated secretly.

Ertel thinks that the Million Dollar Prize is winnable, though obviously the odds required are not 'fair' scientifically. However, as one of the rules is that applicants must pay all their own expenses, he estimates that he would need at least $US10,000 to make a ‘long shot’ bid for the formal challenge. He would also like to have a personal attorney present and another independent scientist as observer, and would need to select 3-4 psi-gifted participants near the JREF institution where the tests would be performed:

But winning the prize would not be my main concern. My main concern is to achieve high levels of significance under control by the skeptics. Psi effects would have to be acknowledged as existent by the science community if they were achieved, i.e. replicated (because they would have been observed before the Randi test was made) with, say, p = .0001 - it need not be .000001. Winning the Randi prize is no scientific standard for acknowledging the existence of causal effects. P = .0001 or so obtained under control of people who are experts in deception (so that this factor is ruled out) and whose intention and bias is to prove that psi does NOT exist (so bias is also ruled out) would let psi appear existent beyond reasonable doubt.

Would You Trust This Man?

Ertel's mention of the expenses required to engage in Randi's challenge, returns us to to the "hurdles" mentioned by Loyd Auerbach. Perusing the rules [10] of the Million Dollar Challenge would certainly give most people cause for concern. Two of the most important, especially when combined, are rules #4 and #8:

4. Applicant agrees that all data (photographic, recorded, written, etc.) gathered as a result of the setup, the protocol, and the actual testing, may be used freely by the JREF.

8. When entering into this challenge, as far as this may be done by established legal statutes, the applicant surrenders any and all rights to legal action against Mr. Randi, and/or against any persons peripherally involved, and/or against the James Randi Educational Foundation. This applies to injury, and/or accident, and/or any other damage of a physical and/or emotional nature, and/or financial and/or professional loss, and/or damage of any kind. However, this rule in no way affects the awarding of the prize, once it is properly won in accord with the protocol.

In other words, applicants give the JREF/Randi virtually absolute license to use the data as best suits their publicity needs, without any legal recourse for the participant. Not exactly enticing for an applicant, although if James Randi was held in higher esteem by the parapsychology research community then it might not matter so much. However, a number of scientists iterated to me their distrust of Randi...and a number of them appear to have good reason for that judgement. When I asked Rupert Sheldrake about the Million Dollar Challenge – a scientist who has investigated ‘telephone telepathy’, the sense of being stared at, and possible psychic talents in animals, Sheldrake told me quite simply: "I don’t take the prize seriously, and above all I don’t trust Randi since I’ve found him to be dishonest...He is not a scientist, has no scientific credentials, and is essentially a showman and an expert in deception". Sheldrake pointed out a previous confrontation as evidence for his distrust of James Randi:

The January 2000 issue of Dog World magazine included an article on a possible sixth sense in dogs, which discussed some of my research. In this article Randi was quoted as saying that in relation to canine ESP, "We at the JREF [James Randi Educational Foundation] have tested these claims. They fail." No details were given of these tests.

I emailed James Randi to ask for details of this JREF research. He did not reply. He ignored a second request for information too.

I then asked members of the JREF Scientific Advisory Board to help me find out more about this claim. They did indeed help by advising Randi to reply. In an email sent on Februaury 6, 2000 he told me that the tests he referred to were not done at the JREF, but took place "years ago" and were "informal". They involved two dogs belonging to a friend of his that he observed over a two-week period. All records had been lost. He wrote: "I overstated my case for doubting the reality of dog ESP based on the small amount of data I obtained. It was rash and improper of me to do so."

Randi also claimed to have debunked one of my experiments with the dog Jaytee, a part of which was shown on television. Jaytee went to the window to wait for his owner when she set off to come home, but did not do so before she set off. In Dog World, Randi stated: "Viewing the entire tape, we see that the dog responded to every car that drove by, and to every person who walked by." This is simply not true, and Randi now admits that he has never seen the tape.

Dr Gary Schwartz has often come under attack from James Randi for his research into mediumship. Labelled "Gullible Gary" by Randi, and accused of believing in the tooth fairy, Dr Schwartz refused an invitation from Randi to allow an "independently qualified panel" to hold forth on the data he has collected. According to Dr Schwartz: "He calls it an 'independently qualified panel', but it is composed mostly of people hand-picked to guarantee the decision would likely be a foregone conclusion, merely rubber-stamping his prejudices". In this case, Randi suggested a panel comprising of Ray Hyman (CSICOP Fellow), Marvin Minsky (CSICOP Fellow), Michael Shermer (CSICOP Fellow) and Stanley Krippner (a parapsychologist whom Randi is familiar with). Not exactly “independent”, one would surmise. Unfortunately, according to Dr Schwartz:

James Randi has a history of engaging in the twisting of the truth...Randi's recommendation of Dr. Krippner was certainly acceptable to me. However, when I contacted Dr. Krippner directly to see if Mr. Randi’s statement about him serving on the panel was correct, Dr. Krippner was concerned. Dr. Krippner explained that he had previously emailed Mr. Randi stating that he would not agree to serve on such a committee. The truth is, Dr. Krippner was not willing to serve on the panel, and he made this clear to Mr. Randi.

Lastly, despite James Randi's assurances that applying for the prize is a simple matter, this seems not to be the case. A number of the more 'general' applicants have waited multiple years to have their claim tested; one of the more recent, Carina Landin [11], went through a 3 year process just to reach the preliminary test, and after failing her test (achieving above chance results, but not to a significant level) found that her protocol had not been adhered to...and so is now waiting to be retested. According to 'Kramer', a former JREF employee who helped with applications:

We experience this a lot, and this most certainly leads many applicants to the conclusion that JREF is "jerking them around", forgetting that no JREF representative is involved in testing, and that tests are determined with Randi's approval, but without his direct involvement, in order to insure absolute impartiality in the testing procedure. JREF cannot guarantee the continued involvement of any third parties who volunteer their time (without any form of compensation) on behalf of The JREF Challenge."

All in all, it's rather easy to see why 'psychic personalities' would ignore the Million Dollar Challenge, irrespective of anyone's opinion as to whether their talents are real or fraudulent. It asks them to risk their careers on a million to one shot (assuming they are not fraudulent), putting all the power into the hands of a man they distrust - and who has been antagonistic towards them over a number of years - with no legal recourse available to them.

On the other hand, although parapsychologists face similar worries, it is now apparent that some are so determined to show the evidence for psi effects that they are willing to risk a failure in order to make an impression. Both Dick Bierman and Suitbert Ertel feel that there is a robust enough effect for them to at least scientifically prove to the skeptics that something interesting is going on. And perhaps others are aware of this fact...

You Say Paranormal, I Say Perinormal

At The Amaz!ng Meeting #3 (TAM3), the JREF-sponsored conference held in January 2005, Richard Dawkins made an intriguing comment [6] during an on-stage chat with James Randi:

About the million dollar prize, I would be worried if I were you because of the fact that we have perinormal possibilities. I mean, what if somebody - what if there really is a perinormal phenomenon which is then embraced within science and will become normal, but at present is classified conventionally as paranormal?

Certainly, suspicious (some might say 'skeptical') minds might wonder whether the influx of positive “perinormal” results - such as from the decades of Ganzfeld telepathy research, replicated presentiment experiments, and Ertel’s new ball-drawing test - may have influenced the JREF’s decision to withdraw the Challenge. It’s interesting to note that Rule #14 of the challenge states:

This prize will continue to be offered until it is awarded. Upon the death of James Randi, the administration of the prize will pass into other hands, and it is intended that it continue in force.

Similarly, in a previous discussion regarding the Challenge, Randi had stated: “...the million dollars is not my million dollars, sir, it belongs to the foundation I represent, and it cannot be used for any purpose other than as prize money in the challenge." It would seem this is no longer the case…

Whatever the reasoning behind the withdrawal of the Million Dollar Challenge, it has little impact on scientific acceptance of psi effects. Even if a challenger took on the risks and won the million dollars – despite Suitbert Ertel’s best intentions - it is doubtful that skeptics would be convinced. According to CSICOP Fellow Dr Ray Hyman:

Scientists don't settle issues with a single test, so even if someone does win a big cash prize in a demonstration, this isn't going to convince anyone. Proof in science happens through replication, not through single experiments.

Ultimately, Irrelevant

It would seem the modern skeptical movement has all bases covered. If you don’t apply, it shows you have no evidence of the paranormal. If you do apply and fail, ditto. If you put your career on the line and apply, beat initial odds of 1000 to 1, and then 1,000,000 to 1, to win the Challenge, then it still offers no proof of the paranormal.

Ironically, paranormal investigator Dr Stephen Braude agrees with Ray Hyman about the merits of the Challenge: “The very idea that there could be a conclusive demonstration to the scientific community of psychic functioning is fundamentally flawed, and the suggestion that a scientifically ignorant showman should decide the matter is simply hilarious.“

Skepticism is certainly demanded in examinations of paranormal claims (not to mention, in all facets of life). However, the JREF Challenge seems to be primarily aimed at providing the modern skeptical movement with a purely rhetorical tool for attacking the topic of the paranormal. In a recent newsletter, James Randi says as much: “The purpose of the challenge has always been to provide an arguing basis for skeptics to point that the claimants just won’t accept the confrontation.” It appears though that some parapsychology researchers are actually more willing than Randi thought...

It seems quite obvious that the Million Dollar challenge does not offer - and has not offered in the past - a fair scientific evaluation of paranormal claims - rather, the statistics employed are primarily based on ensuring the million dollars remains safe. Other rules further stack the deck against participants, by handing control of publicity to the JREF. Suitbert Ertel commented to me:

Randi and those who offer a large monetary prize for psi effect demonstrations are entitled to demand unachievable psi effects. It's their money and they must be careful not to lose it. Everybody must admit that this is reasonable economically. But careful reasoning about money and property is quite a different thing than careful scientific reasoning.

Dr Dean Radin was more blunt in his assessment:

This 'challenge' was like Evel Knievel's steam-powered motorcycle jump over the Snake River Canyon: A great stunt, accompanied by pomp and bluster, but ultimately irrelevant.

-------------------

Update: James Randi has responded to this post in his JREF newsletter dated 29/02/2008: "The Grubbies Attack [12]". While I don't consider this article an "attack" (nor consider myself "grubby"), I do thank Randi for responding. To be clear: I contacted the JREF three times while writing this article, and extended the deadline by a week, to allow for responses and clarifications from Randi (or JREF officials). I would have preferred that, rather than a rhetorical and selective newsletter 'debunking', but Randi is entitled to do what he likes.

Although I would like to leave the article to stand alone, rather than debating points, Randi makes some unfortunate errors in his newsletter, which I feel bound to point out here. Most importantly, in multiple passages, Randi refers to the words of "Loyd Auerbach" - these are not Auerbach's words, they are mine (apart from one short quote from Auerbach). This is unfortunate, as Randi directly addresses Loyd Auerbach in a rhetorical fashion on multiple occasions, when Auerbach did not say the words Randi attributes to him.

Other than that: I am not "chortling" over the end of the challenge, nor is this a "19,000 word tirade" (it doesn't even measure 4000 words, and it is simply an examination of the challenge). Surely Randi is not so sensitive about people offering skeptical analyses (this is his raison d'être, after all) of his own work, as to label them "tirades" (three times no less), when it most obviously isn't?

Randi defines "applied" for the challenge as it suits him. Sylvia Browne "applied", according to Randi, by responding on national TV after being "forced into it" (labeling my statement "wrong" as a consequence). Later, Professor Dick Bierman did not "apply", despite approaching Randi without being forced into it, because "his name appears in none of the application files". For the record, when I queried Randi about his in a private email, he confessed that "Browne never applied."

The passage about "none of the “big fish”" having applied is not a "canard", as Randi labels it - it is in fact a point in favour of the Challenge. For Randi's own edification, I am in agreement with him regarding Sylvia Browne.

In the only correct attribution to Loyd Auerbach, Randi says "we have never said nor even suggested [that the challenge disproves psi]. Loyd invented that, all by himself." Loyd did not claim that Randi made that statement. However, numerous self-described skeptics *have* suggested it. Auerbach had no need to "invent" it (a wonderfully descriptive phrase by Randi though, credit where due for his rhetorical skills).

Randi says "the applicant invests nothing, has nothing to lose, and should be able to beat the odds in the same way that any person could ." This is patently untrue, as the article shows.

Randi: "Again, nonsense. We have NEVER had an applicant fail to come to agreement with us when terms were negotiated, and every one of those applicants simply failed and did not re-apply." I stand corrected.

Randi: "What Auerbach purposely fails to understand – in order to have an argument – is that a pole-vaulter should be able to pole-vault, a cook should be able to cook, and a psychic should be able to do what he/she claims, to better than 1/100 odds."

Nonsense, Randi has no such knowledge that a psychic should do better than his arbitrary 1/100 odds - it is his personal opinion. Would it be snarky of me to point out that in earlier paragraphs Randi claimed to have an "abysmal ignorance of statistics"?

Randi says: "And, I have to wonder why Dr. Bierman did not press me to pursue the matter, since he reports that it seems to have simply vanished. We’ve had many of such disappearances, in which apparently interested persons, scientists among them such as Dr. Wayne E. Carr – also a PhD, so we know he’s a real scientist – who negotiated with us literally for years before backing out. "

Randi turns this around rather deftly with some rhetorical sleight-of-hand. According to Dr Bierman, the ball was in Randi's court when the application "disappeared"; Bierman did not "back out". Randi need not have "wondered" why Bierman did not follow up - Bierman says himself in the article. Further, Randi says his correspondence with Bierman terminated in 1983...I'm not sure of this date, as Bierman's email correspondence about presentiment was in 1998.

[The mention of Victor Zammit's own attack mid-response is nothing to do with my article.]

However, I am glad to see that my article has prompted Randi to lower the odds (to 1 in 100 for the preliminary, and 1 in 100,000 for the main challenge). This may make the Challenge a more attractive proposition for parapsychology researchers. It certainly remedies (to a degree) one of the main problems with the challenge - that the odds are so long. One in one hundred thousand is still no easy task however.

--------------