Exploring Randomness and the Unknowable Reviewed by Panu Raatikainen
Total Page：16
File Type：pdf, Size：1020Kb
rev-panu.qxp 8/17/01 9:46 AM Page 992 Book Review Exploring Randomness and The Unknowable Reviewed by Panu Raatikainen Exploring Randomness What Gödel proved in 1931 is that in any finitely Gregory Chaitin presented system of mathematical axioms there are Springer-Verlag, 2000 sentences that are true but that cannot be proved ISBN 1-85233-417-7 to be true in the system. Church showed in 1936 176 pages, $34.95 that there is no general mechanical method for deciding whether a given sentence is logically valid The Unknowable or not and, similarly, that there is no method for Gregory Chaitin deciding whether a given sentence is a theorem of Springer-Verlag, 1999 a given axiomatized mathematical theory. Such an ISBN 9-814-02172-5 impossibility proof required an exact mathemati- 122 pages, $29.00 cal substitute for the informal, intuitive notion of a mechanical procedure; Church used his own λ- In the early twentieth century two extremely in- definable functions. Turing arrived independently fluential research programs aimed to establish at the same results at the same time. Moreover, he solid foundations for mathematics with the help gave a superior philosophical explication of the con- of new formal logic. The logicism of Gottlob Frege cept of mechanical procedure in terms of abstract and Bertrand Russell claimed that all mathemat- imaginary machines, known today as Turing ma- ics can be shown to be reducible to logic. David chines; this advance made it possible to prove ab- Hilbert and his school in turn intended to demon- solute unsolvability results and to develop Gödel’s strate, using logical formalization, that the use of incompleteness theorem in its full generality. This infinistic, set-theoretical methods in mathemat- identification of the intuitive notion of mechanical ics—viewed with suspicion by many—can never method and an exact mathematical notion is usu- lead to finitistically meaningful but false state- ally called Church’s thesis or, more properly, the ments and is thus safe. This came to be known as Church-Turing thesis. It is the fundamental basis Hilbert’s program. of all proofs of absolute unsolvability. These grand aims were shown to be impossible One of the greatest achievements of modern by applying the exact methods of logic to itself: mathematical logic was certainly the proof by Yuri the limitative results of Kurt Gödel, Alonzo Matiyasevich in 1970, based on earlier work by Church, and Alan Turing in the 1930s revolution- Julia Robinson, Martin Davis, and Hilary Putnam, ized the whole understanding of logic and that the tenth problem of Hilbert’s famous list of mathematics (the key papers are reprinted in [5]). open mathematical problems from 1900 is in fact Panu Raatikainen is a fellow in the Helsinki Collegium for unsolvable; i.e., there is no general method for de- Advanced Study and a docent of theoretical philosophy ciding whether a given Diophantine equation has at the University of Helsinki. His e-mail address is a solution or not [11]. This result implies that in [email protected]. any axiomatized theory there exist Diophantine 992 NOTICES OF THE AMS VOLUME 48, NUMBER 9 rev-panu.qxp 8/17/01 9:46 AM Page 993 equations that have no solution but cannot be a comprehensive survey of the theory and its proved in the theory to have no solution. applications). However, it is now a widespread view, espe- Chaitin was active in developing this approach cially in computer science circles, that certain into a systematic theory (although one should variants of incompleteness and unsolvability not ignore the important contributions by many results by the American computer scientist Gregory others). From the 1970s onwards Chaitin’s inter- Chaitin are the last word in this field. These vari- est has focused more and more on incompleteness ants are claimed to both explain the true reason and unsolvability phenomena related to the notion for Gödel’s incompleteness theorem and to be the of program-size complexity. Indeed, he now ultimate, or the strongest possible, incomplete- says that “the most fundamental application” of ness results. Chaitin’s results emerge from the the theory is in “the new light that it sheds on theory of algorithmic complexity or program-size the incompleteness phenomenon” (The Unknow- complexity (also known as “Kolmogorov com- able, pp. 86–7). plexity”); Chaitin himself was, in fact, one of the It was known from the beginning that program- founders of the theory. size complexity is unsolvable. Chaitin, however, The classical work on unsolvability dealt solely made in the early 1970s an interesting observation: with solvability in principle: one abstracted from Although there are strings with arbitrarily large the practical limits of space and time and required program-size complexity, for any mathematical only finiteness. From the late 1950s onward, how- axiom system there is a finite limit c such that in ever, more and more attention has been paid to dif- that system one cannot prove that any particular ferent kinds of complexity questions—at least in string has a program-size complexity larger than part because of the emergence of computing ma- c [1]. Later Chaitin attempted to extend his “in- chines and the practical resource problems that ac- formation-theoretic” approach to incompleteness companied them. In logic and computer science theorems in order to obtain “the strongest possi- various different notions of complexity have been ble version of Gödel’s incompleteness theorem” ([3], studied intensively. First, computational complex- p. v). For this purpose he has defined a specific ity measures the complexity of a problem in terms infinite “random” sequence Ω. of resources, such as space and time, required to As was noted, one of the major sources that solve the problem relative to a given machine originally motivated the development of the model of computation. Second, descriptive com- theory of program-size complexity, especially in plexity analyzes the complexity of a problem in Kolmogorov’s case, was a problem in the theory terms of logical resources, such as the number of of probability, viz. that of giving a precise and variables, the kinds of quantifiers, or the length of plausible definition for the notion of a random a formula required to define the problem. And fi- string. The problem is related to the paradox of nally, by the algorithmic complexity, or the pro- randomness, which may be explained as follows: gram-size complexity (or Kolmogorov complexity), Assume we are given two binary strings of 20 of a number or a string, one means the size of the digits each, and we are informed that they were shortest program that computes as output that both obtained by flipping a coin. Let these two number or string. strings be: x = 00000000000000000000 Theory of Algorithmic Complexity The basic idea of the theory of algorithmic com- and plexity was suggested in the 1960s independently y = 01001110100111101000. by Ray J. Solomonoff, Andrei N. Kolmogorov, and Gregory Chaitin. Solomonoff used it in his com- Now according to the standard theory of putational approach to scientific inference, probability, these strings are equally probable. Kolmogorov aimed initially to give a satisfactory And yet intuitively one tends to think that x definition for the problematic notion of a random cannot possibly be a randomly generated string— sequence in probability theory, and Chaitin there is too much regularity in it—whereas y first studied the program-size complexity of appears to be genuinely irregular and random Turing machines for its own sake. Kolmogorov and may well be the result of a toss of a coin. The went on to suggest that this notion also provides algorithmic theory of randomness explicates this a good explication of the concept of the informa- idea of regularity with the help of Turing machine tion content of a string of symbols. Later Chaitin programs. One considers a finite string as regular, followed him in this interpretation. Consequently, or nonrandom, if it can be generated by a simple the name “algorithmic information content” has program, i.e., if its program-size complexity is frequently been used for program-size complex- considerably smaller than its length. Accordingly, ity, and the whole field of study is very often a finite string is defined to be random if its called “algorithmic information theory” ([10] is program-size complexity is roughly equal to its OCTOBER 2001 NOTICES OF THE AMS 993 rev-panu.qxp 8/17/01 9:46 AM Page 994 length, i.e., if it cannot be compressed to a shorter language provides exactly what is needed, so he program. (Note that this notion is relative to a invented a new version of LISP. Chaitin begins The chosen programming language or coding system; Unknowable by saying that what is new in the book a finite string may be random in one but nonran- is the following: “I compare and contrast Gödel’s, dom in another.) Turing’s and my work in a very simple and straight- Extending this approach to infinite strings forward manner using LISP” (p. v). According to turned out to be, however, more difficult than was Chaitin this book is a “prequel” to his previous thought. Kolmogorov’s first book, The Limits of Mathematics (Springer-Verlag, idea was that an infinite string 1998), and is an easier introduction to his work be considered random if all of on incompleteness. In Exploring Randomness its finite initial segments are Chaitin in turn writes that “[t]he purpose of this random. But Per Martin-Löf book is to show how to program the proofs of the showed that this definition does main theorems about program-size complexity, not work and then gave a more so that we can see the algorithms in these proofs satisfactory definition in mea- running on the computer” (p.