A story of “Cooked Lists” and “Permutation Races.”

The Torah Codes: Puzzle and Solution

Maya Bar-Hillel, Dror Bar-Natan, and Brendan McKay In 1994, Statistical Science published remained hidden. Now it has been What is this Torah Code, astonishing statistical evidence proving unlocked by computer—and it may the existence of a hidden code in the reveal our future.” Using this code, the and What are We to Book of Genesis relating to future author, a self-described “skeptical secu- Make of It? events. New research deprives this evi- lar reporter” named Michael Drosnin, dence of its import by proving that the discovered what he took to be a predic- The Torah Code same code can be found in the Hebrew tion of the assassination of the late translation of War and Peace. , then prime minister of . Lest skeptics dismiss this claim Take the Book of Genesis in Hebrew. If out of hand, readers were told, “The you start from the very first (Hebrew code was broken by an Israeli mathe- equivalent of the letter) T and skip 49 The matician, who presented the proof in a letters, the 50th letter is an O. Skipping major science journal, and it has been over another 49 letters brings you to an On Thursday, May 28, 1997, Simon & confirmed by famous mathematicians R, and a final 49-letter skip brings you Schuster took out a full page ad in The around the world” and “In a few dra- to H, thus spelling the Hebrew word New York Times, announcing a new matic cases [Rabin’s assassination for Bible, or Torah, in the form of an book with the following caption: “In all being one of them] detailed predictions equal-letter-skip, or ELS for short. This of history, few books have completely were found in advance—and the ELS, and some others like it, were dis- changed the way we view the world. events then happened exactly as pre- covered several decades ago by the late The Bible was one. The Bible Code is dicted” (all quotes are from the book Rabbi Weissmandel. another.” Within a week, the book had jacket). Astonishingly, these assertions How remarkable is this discovery? been featured on CNN and in The are actually more or less truthful: An easy combinatorial argument shows New York Times, TIME magazine, and Evidence for the so-called Torah code that if a word is not too long (say, up to Newsweek (in the Religion section). In was published in 1994 in the six letters), or its letters not too rare, quick succession, it was featured in respectable peer-reviewed journal then the chances of finding it as an ELS the leading U.S. newspapers and on Statistical Science (Witztum, Rips and in a text as long as Genesis—78,064 the Today Show and the Oprah show. It Rosenberg 1994, hereafter WRR), it Hebrew letters long—is excellent. was translated into many languages received a degree of endorsement from Indeed, based on the letter frequencies and topped bestseller lists all over the several famous Jewish Orthodox math- in Genesis, the expected number of world for weeks. Warner Brothers ematicians from Harvard, Yale, and The appearances of the word TORH in bought the movie rights. Hebrew University, and reliable wit- Genesis as an ELS with any given skip The claims made for the book were nesses confirm that Drosnin took pains size is 3. In fact, TORH appears 56,769 of National Inquirer quality: “For three to warn Rabin of the danger to his life a times in Genesis as an ELS. In that thousand years a code in the Bible has full year before the assassination. respect, the discovery, appearance

CHANCE 13 notwithstanding, is unamazing. How written as a page of 78 rows, each 1,000 ders.” These words appear as plain text remarkable is it, though, that one of letters wide, or as a page of 780 rows, in Deuteronomy, chapter 4, verse 42; these should start, of all places, at the each 100 letters wide, and so on. In this The Bible translates them to very first T in Genesis, and have a skip rectangular letter array, ELS words are “a man ... who had killed his fellow;” size of 49, a significant number in allowed to be read horizontally, vertical- Drosnin translates them as: “an assassin Judaism? The problem with putting a ly, or diagonally, and in any direction— who will assassinate;” the skip in Rabin’s number on the remarkableness of this left to right or right to left, top down or name is of almost 5,000 letters. Striking detail is, of course, that it is ad hoc. bottom up. This enables the presenta- as these arrays may seem to the naive What Weissmandel saw with the tion of ELS words in a visually ready reader, the unsystematic nature of the unaided eye can nowadays be sought manner even though in the original text search methodology makes the calcula- with the aid of computers, spanning the letters constituting an ELS might be tion of a responsible number for the much longer text segments, and much separated by hundreds, if not by thou- probability of any particular discovery larger letter skips. In the mid 80s Ilya sands, of other letters. impossible to do analytically (although Rips, a professor of mathematics at The Witztum and Rips were struck by irresponsible calculations abound). Hebrew University, and Doron the occasional discovery of thematically Witztum, who, like Rips, is an Orthodox related ELS’s in proximity—namely, in a Jew, used the computer to search for all relatively compact rectangle of letters. The Rabbis Test kinds of words and word combinations Such a rectangle can be thought of as a in Genesis. Witztum had an idea—to small “window” cut out of some gigantic Concerned over the possibility that in write Genesis as a single, extended page Genesis page. An example is the famous any text of sufficient length it is possi- consisting only of letters (no spaces and array on the cover of Michael Drosnin’s ble to find ELS clusters like those no punctuation marks). This huge page book, The Bible Code (Drosnin 1997), found in Genesis, Witztum and Rips could have various dimensions. For containing Rabin’s full name in Hebrew decided to run a kind of statistical test example, the 78,064 letters could be and the words: “A murderer who mur- and calculate the significance—statis- tical and otherwise—of their discover- ies. In selecting a sample, they set Table 1—Alternative Date Choices and Proximity themselves a goal of “uniformity and Measures in Genesis objectivity with regard to the choice of the pairs” (WRR, p. 431). The First List Second List Encyclopedia of Great Men of Israel †Only death dates 144 4 (1961) contains short biographies of †Only conventional Jewish Rabbis who lived between the form for 15, 16 of the month 107 20 8th and the 19th centuries. A minority of the biographies contain a death date b’alef b’Tishrey date form 301,391 521,298 or a birth date (Hebrew dates are writ- alef l’Tishrey date form 930,257 274,167 ten using letters only. Numbers are b’alef l’Tishrey date form 637,082 749,344 represented by letters in a systematic alef shel Tishrey date form 869,341 105,694 way, with the first 10 letters of the b’alef shel Tishrey date form 617,047 225,328 alphabet representing the numbers 1 to 10, etc.) Witztum and Rips drew up Dates and forms of Margalioth 288,779 360,472 a list of those rabbis to which the Encyclopedia devotes three columns Distance measured by minimal and more of text and mentions a date area of enclosing rectangle 5,547 63 of death or birth. They found 34 such Distance measured by minimal rabbis (actually, they erred slightly in diagonal of enclosing rectangle 24,554 425 composing the list, including in it one rabbi who didn’t have a long enough Distance measured by minimal entry and missing one rabbi who did), perimeter of enclosing rectangle 22,160 434 searched for their names and dates as Distance between the closest two letters 47,965 617 ELS’s in Genesis, and computed a Distance between the farthest two letters 21,398 510 kind of distance function between the name of a rabbi and his date. The exact † Note: Only a small number of dates were removed in this check. manner in which Witztum and Rips computed the distance is technically The number in the table gives the rank of the correctly paired rabbis’ list in a race complicated and will not be described against 999,999 competing permutations. The race was run with respect to the here. Suffice it to say that proximity P2 statistic of WRR, in the Book of Genesis. The numbers are, of course, subject between a pair of ELS’s was not mea- to minor sampling errors. sured by the size of the letter array enclosing them.

14 VOL. 11, NO. 2, 1998 talk of Witztum and Rips, in the order in which they chose to author their papers. The third author, Rosenberg, only did the programming for them and will not be mentioned further.

Reactions to the Rabbis Test

Although a proper significance test obviates the need to test the rabbis list on other texts (since it answers the question, “What is the chance of find- ing this kind of result by chance alone?”), skeptical referees asked that Figure 1. A sample of the Torah Code being decoded. “control texts” (i.e., texts that ought to yield a significant result) be subjected to the same procedure. Of several control A list of numbers was obtained that was paired with the date against which texts used, one, at the explicit request of describes the “distance” between the the permutation pitted him. For each of one of the referees, was Tolstoy’s War name of a rabbi and his date (in some the 999,999 permutations, the dis- and Peace (first 78,064 letters of the cases, no ELS was found for a rabbi or tances between the rabbis’ names and Hebrew translation). Another control his date, and then, of course, no dis- the date they happened to be paired text was the book of Isaiah, although tance was computed). Here and there with in that permutation were comput- Drosnin (p. 36) reported on codes he some rabbi was particularly close to his ed. The original, correct, list of dis- found in Isaiah and claimed (p. 185) own date. At the same time, most of the tances was compared to 999,999 lists that Rips “agrees that other parts of the rabbis were actually closer to some other of distances in which each rabbi was Old Testament, like Daniel and Isaiah, rabbi’s date. To test if there was an extra- paired with a random date, and its prox- might also be encoded.” The rabbis list ordinary closeness in Genesis between imity rank was taken to be its statistical did not excel in the permutation race in the rabbis’ names and dates, some statis- significance. Lo, the correct pairing any of the control texts. Genesis was tical significance test was required. The achieved one of the first places in this clearly special. In addition to the “con- original method whereby Witztum and “permutation race!” Of course, if there trol texts,” Diaconis also asked for a Rips attempted to compute the statisti- were nothing special in this list or in cross-validation, or replication (i.e., a cal significance of their results made no the Book of Genesis, there would be no new sample, which is expected to yield mathematical sense and will not be a priori reason for it to excel like that. a significant result). If a control text is a described here. Professor Persi The preceding account is an over- prudent check for artifacts in the statis- Diaconis, then of Harvard University, simplification in several respects (for tical method, a replication is prudent proposed an alternative method, whose example, Witztum and Rips used only “[i]n order to avoid any conceivable central idea was to compare the dis- ELS’s with a near-minimal skip). The appearance of having fitted the tests to tances in the rabbis list to a large num- actual details are somewhat more com- the data” (WRR, p. 431). ber of alternative control lists. plex than we described them, but the WRR’s new sample consisted of Imagine that each rabbi is paired simplification smooths the story line. those rabbis to whom Margalioth’s not with his own date of death but Our account also pretends, contrary to Encyclopedia devoted between 1.5 and rather with some other date, sampled at historical fact but for narrative clarity, 3 columns of text, and contained a date random from the list of dates. It is pos- that the published method was the one of death or birth, “without changing sible, of course, to compute the dis- used all along. We did not describe how anything else” (WRR, p. 431). The sec- tance between the rabbis’ names and a list of distances was reduced to a sin- ond list, consisting of 32 rabbis (there such random dates. If there is anything gle number expressing the proximity of were some errors of inclusion again), special in the Book of Genesis and the the entire list. It is noteworthy, howev- was subjected to the exact same proce- rabbis’ names really appear exceptional- er, that the method reported in dure as the first. Again, it arrived almost ly close to their dates of death, then the Statistical Science is an inferior varia- first in a race among one million con- distances between correct name-date tion of Diaconis’s proposal, and yields testants. Following these additional pairs should be, on average, closer than results that are hundreds of times more tests, the article was finally accepted between random name–date pairs. “significant” than Diaconis’s original for publication in Statistical Science. Diaconis suggested running a kind of suggestion would have yielded. In addi- WRR’s only conclusion from their “race” in which 999,999 permutations tion, although the popular press regards extraordinary results is that “the prox- of the 34 rabbis were chosen at ran- Rips, a reputable mathematician, as the imity of ELS’s with related meanings in dom. In every permutation, each rabbi central figure in ELS research, we shall the Book of Genesis is not due to

CHANCE 15 chance” (WRR, p. 434). Their conclu- impressed by the fact that Drosnin exhaustive list, either. What was the sion is more noteworthy for what it apparently predicted Rabin’s assassina- protocol whereby Witztum and Rips does not say (but everyone under- tion before it actually took place should chose the particular names and date stands) than for what it does: Since the note that his actual warning to Rabin forms in their lists? Their article is mum rabbis were born and died decades only said: “ I believe you are in real dan- on this question. It states only that “we after the Torah was written, only a clair- ger, but that danger can be averted” used three fixed variations of the format voyant could have coded them into it. If (Drosnin, p. 187). That much, however, of the Hebrew date”, and that “The list you will, Witztum and Rips had discov- was a common perception at the time. of appellations ... was provided by Prof. ered a statistical proof of the divine, or And of course, this prediction would S. Z. Havlin” (WRR, p. 436). Written at least supernatural, origin of the Book have been confirmed whether Rabin and oral exchanges of the present of Genesis. Professor Robert Kass, then had been assassinated or not. authors with Witztum and Rips give a editor of Statistical Science, prefaced Whereas Drosnin’s findings, as well more comprehensive—and startling— the article with the words, “Our refer- as Witztum and Rips’s earlier word answer, however. There was NO proto- ees were baffled: Their prior beliefs clusters, are mere anecdotes, the rab- col! The official line is that the list of made them think the Book of Genesis bis list is much harder to shrug off. The names and appellations merely reflects could not possibly contain meaningful work described previously has an Havlin’s judgment, and the choice of references to modern-day individuals ... appearance of scientific rigor. Its faults date forms is credited to the late Dr. Y. The paper is thus offered ... as a chal- escaped detection by a bevy of sophis- Orbach. lenging puzzle” (p. 306). Kass later ticated referees. It is far and away the It is also not quite clear who is reacted strongly against claims that the strongest evidence for the existence of responsible for the fact that many of the paper had convinced him, telling a codes in the Torah. It was time for dates mentioned in Margalioth’s reporter “I’d be very surprised to see some statistical sleuthing. Encyclopedia were altered, corrected, any statistician believe any of this.” discarded, exchanged, or added to, a fact that is not explicitly mentioned by A Closer Look at the WRR (though it is mentioned in earlier Reactions to Drosnin’s preprints). The odd fact is that although Rabbis List it was necessary for a rabbi’s encyclope- Book dic entry to contain a date for him to be It is difficult for a non-Hebrew speaker included, the date mentioned was not The puzzle did not attract much atten- to appreciate the details of the rabbis binding on the authors. They used tion, until Drosnin’s book appeared to list as they appear, in Hebrew, in WRR. whichever date they considered most instant best-sellerdom and directed If hitherto our description imparted the authoritative. Be that as it may, whoever some publicity to WRR’s work. impression that each rabbi was repre- was responsible for the choices made, Drosnin’s book was panned not only by sented by a single name and one or two the end result is that in spite of Witztum skeptics but even by Witztum and Rips dates, this turns out not to be so. The and Rips’s declared intention of “unifor- themselves. One stated reason was rabbis were known by many names and mity and objectivity with regard to the Drosnin’s attempts to use codes to pre- appellations, and they entered the list choice of pairs” (WRR, p. 431), the list dict future events, which Witztum and with many names and appellations— they actually compiled was neither uni- Rips claimed to be impossible (though as many as 11 in one case! By way of form nor objective: It necessitated the it is not clear why). Another was its lack illustration, the current president of the discretionary judgment of experts— of rigor. Obviously, “messages” such as can be called, Mr. judgment which, we shall show, is those Drosnin found supposedly President, Mr. Clinton, Bill, Bill arguable. Moreover, we shall also show “encoded” in the Bible could be found Clinton, Clinton, and so on. Dates also how the fact that the list of names and in any text. Indeed, the triviality of appear in more than one form, up to six date forms was a subset of all names Drosnin’s discoveries is highlighted in in one case. By way of analogy, consid- and date forms left room for the very the following anecdote. In a Newsweek er various date writing forms in English: “fitting of tests to the data” that the sec- interview, Drosnin had stated: “When January first, the first of January, on ond list was designed to prevent. It is in my critics find a message about the January first, on the first of January, the multitude of possibilities for writing assassination of a prime minister January one, on January one, New names and dates, and the even greater encrypted in Moby Dick, I’ll believe Year’s day, on New Year’s day, and so on. multitude of possibilities for choosing them” (Begley 1997, p. 51). McKay (In Hebrew, names and dates form among these possibilities, that the key promptly found a slew of letter arrays in much shorter letter sequences than in to the Torah codes puzzle lies. Moby Dick, encoding a series of famous English, and prepositions such as “on” assassinations, including, in addition to and “of” are single-letter prefixes.) Rabin’s, those of Kennedy, Martin Casual observation reveals that the Luther King, Trotsky, and even a date forms do not exhaust all the differ- Degrees of Freedom tongue-in-cheek “prediction” of the ent ways in which the dates could have in the Rabbis List murder of Drosnin himself, in a format been written, and careful research of closely mimicking Drosnin’s own Rabin the rabbinical literature shows that the On the face of it, the second list assassination array (see Fig. 1). Those names and appellations are not an answered the concerns raised by both

16 VOL. 11, NO. 2, 1998 the lack of objectivity and the suspi- ond list is in fact? Many of those who tions in Witztum and Rips’s list, 20 cion of a lack of a priority of the first readily see that the appearance of words were dropped, and 30 were added. list. Whatever degrees of freedom were and of word combinations as ELS’s, For example, the name exploited in the choice of appellations even in proximity, is possible in any text “Oppenheim” spelled with a single and date forms in the first list had been find it hard to believe that, under the Hebrew letter yod (corresponding exhausted, and the second list was heavy constraints set by the first list of roughly to I) was exchanged for bound to the choices made in the first rabbis, Witztum and Rips’s results that name with a double yod list. In requesting a new sample, (namely, outstanding performance in a spelling. The deletions and addi- Diaconis had hoped that the con- permutation race) can also be replicated tions were all based on research, straints set by the first sample would in any text. But recently, this too has and were governed by principles guarantee the objectivity of the new been unambiguously demonstrated. and consistency considerations to one, and this objectivity was supposed the same extent as Witztum and to make it more credible that it was Rips’s list was. Menahem Cohen, a also a priori. But in fact, the rules and The Rabbis List in War colleague of Havlin’s from the constraints laid down by the first list Faculty of Jewish Studies at Bar- left sufficient room for maneuvering in and Peace Ilan University, has stated in writ- the second list, allowing for the “cook- ing that “The list prepared by Prof. ing” of a second list that would be no Bar-Natan and McKay undertook the Havlin, ... has, in my humble opin- less successful than the first list. following challenge: To take some text ion, no scientific basis. ... I see no Where did any degrees of freedom of the length of Genesis and cook the essential difference between the remain in the transition between the second rabbis list so that it would excel two lists for the purpose of using first list and the second? In regard to the in a race of permuted lists run on this them for ELS experiments.” We date forms, the second list did indeed text, yet follow the guidelines govern- note that the cooked list is not follow the first list. But in writing names ing Witztum and Rips’s first list just as claimed to be perfect but only as and appellations, there aren’t rules as closely as their own second list did. error-free and self-consistent as strict and binding as there are for date The text chosen for this purpose was Witztum and Rips’s list, or better. writing. Different people, even if given War and Peace (the first 78,064 letters) Additional detail can be found on the same name at birth, may come to be because of its earlier role in WRR. Bar- McKay’s Web site http://cs.anu called differently in the course of their Natan and McKay’s list was construct- .edu.au/~bdm/dilugim/torah.html. ed as follows: life time (a Robert can be Bob, Bobby, In a race of ten million permuta- Rob, Robby, Bert, Bertie, etc.). If one 1. The list of rabbis was chosen by tions, the modified list finished in 12th Robert is also called Bob and another is Witztum and Rips’s criterion— place! It is possible to summarize mat- not, this is not an inconsistency in namely, rabbis to whom the ters thus: Within the boundaries of choosing appellations—rather, it Encyclopedia allotted between 1.5 mostly trivial changes, all of which are reflects reality. And the facts of the mat- and 3 columns of text and whose justifiable, consistent, and legitimate ter regarding names and appellations entry included a date of death or no less than Witztum and Rips’s, the cannot be derived from rules; they must birth. One rabbi, who was includ- astonishing result from Genesis can be be historically known. Hence, allowing ed in Witztum and Rips’s list by replicated in War and Peace. nicknames and appellations makes it mistake, was removed. A missing Of course, from the fact that a list difficult to specify an “objective” list of one was added. could be, and was, cooked for War and names governed by strict rules. The 2. The dates, including their form, Peace, it does not follow that Witztum appellations are included because the were identical letter by letter to and Rips cooked their list for Genesis. person who draws the list thinks they those used by Witztum and Rips. They persist in claiming that their list, should be included. It is a matter of dis- unlike Bar-Natan and McKay’s, was a cretionary judgment. Indeed, Havlin, in 3. The computational details were the same as those used by priori, that it was chosen in good faith, a written description of his considera- and that the choices were blind with Witztum and Rips, with the fol- tions when constructing the lists of respect to their success probability. lowing exception: Witztum and names and appellations, admitted Because none of the critics and skep- explicitly that he used a great deal of Rips included two rabbis even tics were present when the list was judgment when constructing the list. It though they had discarded the drawn up, the decision whether or not follows, therefore, that the second list dates given them by the encyclo- to believe the claim that the second list did not solve the problem it was intend- pedia, thus leaving them unpaired was a priori remains as much an act of ed to solve, namely—guaranteeing that in the correct list. Bar-Natan and faith as it was with regard to the first the second list could not be fiddled with McKay’s list discarded these rab- one. Nonetheless, we performed other and “cooked” to success. bis, who necessarily contributed analyses, also statistical in nature, The reader might wonder at this only noise to the computation, which indicate that if Witztum and point: Is it really possible to “cook” a altogether. Rips’s list was not cooked then its cre- second list, following exactly the rules 4. The “cooking” was confined to the ators enjoyed a fantastic streak of luck, set by the first list so that it would be as list of names and appellations. Out to say the least. These findings, to be successful as Witztum and Rips’s sec- of close to 90 names and appella- described in the next section, seriously

CHANCE 17 question the claims of a prioricity. But The distance metric also provided a tion of fortuitous choices is expected to we shall leave it to the readers to draw fertile ground for choices. Proximity be no higher than 50%. Wonder of won- their own conclusions. between word pairs could have been ders, however, it turns out that almost measured any number of ways—area of always (though not quite always) the the minimal rectangle enclosing the allegedly blind choices paid off: Just Choices, Choices pair; diagonal of the minimal rectangle about anything that could have been enclosing the pair; circumference of the done differently from how it was actual- minimal rectangle enclosing the pair; ly done would have been detrimental to Every researcher in every research Euclidean distance between the two the list’s ranking in the race. must make many methodological and farthest letters in the pair; Euclidean In particular, all the choices listed in substantial choices. So too, Witztum distance between the two nearest let- the present section were fortunate for and Rips, or their consultants, faced ters in the pair, and so on and on. How Witztum and Rips. Had any of them many choices and decisions in imple- to measure distance was a choice. been different, the ranking of the lists menting their selection of a sample of Finally, we describe one of the in the permutation race would have word pairs and in their determination purely technical choices made. If a gone down. Sometimes by a small of “distance.” The choices that existed word is expected (on the basis of the amount, sometimes by an order of mag- with regard to the matter of dates pro- letters in it and their frequency in nitude, but always down. Some might vide some convenient examples. Genesis) to have more than 10 ELS’s claim that it is not “fair” that the choic- 1. One date, or two? Only four of the in Genesis, Witztum and Rips put a es were tested with respect to their 34 rabbis in the first list and one limit on the size of the skip so as to effect on the permutation race rank, in the second had valid birth dates reduce the expected number of ELS’s because this statistic had not yet been in the Encyclopedia. All but one of to 10. This is an arbitrary number, the developed when the choices were them also had death dates. stated reason for which was just to save made. We are not claiming, however, Hence, Witztum and Rips could computation time. It is thus not inher- that choices were made in a deliberate have settled for dates of death ently better than aiming for any other process vis-à-vis any particular statistic. only (they would have lost but one expected value. We are merely claiming that blind rabbi out of 66), or they could These are just a sample of the many choice is expected to have yielded dif- have taken dates of birth as well as choices Witztum and Rips faced. ferent results than those our analyses dates of death (just birthdates Suppose, now, that all these choices actually found. Table 1 lists some of the would have been impractical, as were made, as claimed, in an a priori tests we did and their results. there are not enough of them). fashion—namely, without knowing how By the way, the rationale that Thus the choice was between the choice would affect the outcome. It Witztum and Rips gave for not having death date or both dates. stands to reason that where a choice used a single date form is that they could have been realized in one way or 2. Conventional forms, or uncon- couldn’t know in advance which form another, it would turn out a posteriori ventional ones, too? “The 15th was the one used in the code. We now that the blind choice was fortunate (i.e., know, however, that no single date form and 16th of any Hebrew month improved the ranking in the race) about can be denoted [two ways]. We is “best.” In other words, use of a com- as often as it was unfortunate (i.e., hurt bination of date forms (and also using used both,” said Witztum and the ranking). So it is possible to set up a Rips (WRR, p. 436). What they both forms of the 15th and 16th of the null hypothesis of blind choice (blind month) is superior to any single date did not say is that one of these is with respect to its implications for the conventional and the other, form. Moreover, the triplet of date success of the list in the permutation forms used by WRR is superior to any of though it used to exist, no longer race), according to which the propor- does. Use of the conventional the other 14 choices that could have form only is unlikely to have been questioned, because few people Table 2—Replications are even aware that there is an alternative. In any case, they were WRR Original Lists not compelled to use it—they The text List 1 List 2 chose to. Genesis 38 2 3. “In designating dates, we used Exodus 23,888 189,486 three fixed variations of the format Leviticus 698,915 947,860 of the Hebrew date” (WRR p. 436). In fact, there are as many as Numbers 4,573 916,271 eight standard ways of writing a Deuteronomy 135,101 760,645 Hebrew date, and for some dates (particularly holidays), even more. Note: The number in the table gives the rank of the correctly paired rabbis list in a The choice of three particular race against 999,999 competing permutations. The race was run with respect to the forms is arbitrary: It is possible to P2 statistic of WRR, in the Book of Genesis. The numbers are, of course, subject to choose any one form, pair of forms, triplet of forms, etc. minor sampling errors.

18 VOL. 11, NO. 2, 1998 been made from the power set of the legitimate category of texts. Running sively in Genesis. That list performed possibilities: alef Tishrey, b’alef Tishrey, Witztum and Rips’s two lists letter for quite poorly in War and Peace, and in alef b’ Tishrey, b’alef b’ Tishrey. As to letter on the other four books of the several other control texts (though we how the list would have ranked in the Pentatuech provides eight such replica- now know that it also performs poorly in permutation race had some other values tion opportunities. They all failed (see Exodus, Numbers, Leviticus, and been set for the expected number of Table 2). Deuteronomy). Another list, however, ELS’s rather than 10, the ranks for the Witztum and Rips are often asked consistent and correct to roughly the choice of 2, 5, 15, 20, 25, or 50 are, why they didn’t check the other books same degree as the original list, did respectively, 210, 55, 3, 10, 20, 159. themselves. The standard answer is, achieve in War and Peace a measure of The rank for the original choice of 10 is “We are only making claims for the success as impressive as that of 2, smaller than for any of these alterna- Book of Genesis.” By the same token, Witztum and Rips’s list. So it is not the tive choices. of course, they could have refused to Book of Genesis that is remarkable; it is Of course, the probability of blindly run any new sample, with the excuse Havlin’s lists, which seem made-to- choosing so felicitously again and again that they are “only making claims for order for Genesis. and again diminishes rapidly with the the first list.” What then remains of the “scien- number of such choices. By the same Another replication attempt is to use tific” experiment and the statistical statistical logic that gave power to the the list of names and appellations letter significance testing? We are back to WRR results, the results of the present for letter but use a different date square 1. We started by showing that analysis suggest that the chance of form—any single, uniform date form, single words in ELS form can be drawing up such a successful list that would disallow any data fitting. The found not only in Genesis but also in blindly is very remote. forms that were not chosen by WRR any other sufficiently long text. Now Lest there be a misunderstanding, provide several such replication oppor- we see that even a list of rabbis and we hasten to repeat that the fact that a tunities. These replications also failed their dates can be found in remark- particular choice made by Witztum and (Table 1). (We do not wish to argue the able proximity not only in Genesis Rips turned out to be better than its distinction between a replication, and a but also in any other sufficiently long alternative by no means implies that mere “what if” check. Roughly speak- text. To be sure, “the proximity of both were checked and the superior one ing, if the check affects all or most of ELS’s with related meanings in the the rabbi pairs, it is probably fair to call was chosen. The method whereby the Book of Genesis is not due to it a replication, if it affects just a few— War and Peace list is cooked did not chance” (p. 434). It must therefore not. A replication should, in principle, involve any of these choices, because be due to design. The design, howev- allow the computed rank to be any- er, may well be human, not divine. they were imposed already. All choices where, whereas the checks are often were limited to which names and appel- [The Chance article is loosely based predestined to be not very far from the on a Herbrew article (Bar-Hillel, Bar- lations to include and how to spell rank of the original list.) them. Nonetheless, our list would have Natan, and McKay 1997), insofar as Over the years, Witztum, Rips, and both tell essentially the same story, but fared similarly to theirs under the same Rosenberg carried out their own repli- checks. If a list of names is cooked to this article is expanded and more statis- cations, reporting additional positive tically sophisticated.] optimize some statistic given some results with a few similar experiments choices, the choices look as if they were on different topics. All of them, howev- References and Further Reading cooked to optimize the statistic given er, suffer from the same problem as the the list of names. one that plagues the Statistical Science Bar-Hillel, M., Bar-Natan, D., and samples: they are not tight enough, McKay, B. D. (1997), “There Are leaving too much room for data fitting. Codes in War and Peace Too,” Replication They all involve pair lists that could eas- Galileo, Vol. 25, 52–57 (in ily be different from what they are and Hebrew). Recall that the Statistical Science article still remain within the stated criteria. Additional information can be found at Begley, Sharon, “Seek and Ye Shall reported several control tests and only McKay’s Web site. A partial rely by Find,” Newsweek, June 9, 1997, one replication test—the second list. Witztum, with which we very strongly 66-67. The fact that the second list had too disagree, can be found at his Web site much slack raises the question of Drosnin, M. (1997), The Bible Code, www.torahcodes.co.il. whether a replication could be per- New York: Simon & Schuster. formed that would be completely tight. Margalioth, M. (1961), The We believe that the answer is yes—and Conclusion Encyclopedia of Great Men of that we have done some such replica- Isreal, Tel Aviv: Joshua Chachik. tions. The (failed) goal of the replication reported by WRR, recall, was “to avoid This analysis completes the critique of Witztum, Rips, and Rosenberg, any conceivable appearance of having the seemingly astounding statistical (1994), “Equidistant Letter fitted the tests to the data” (p. 431). results that were published by WRR. A Sequences in the Book of Rather than change the list, one could, list was presented there which per- Genesis,” Statistical Science, of course, change the text within the formed most surprisingly and impres- 429–438.

CHANCE 19