INFORMATION, RANDOMNESS & INCOMPLETENESS Papers On
Total Page:16
File Type:pdf, Size:1020Kb
INFORMATION RANDOMNESS INCOMPLETENESS Pap ers on Algorithmic Information Theory Second Edition G J Chaitin IBM P O Box Yorktown Heights NY chaitinwatsonibmcom Septemb er This collection of reprints was published byWorld Scientic in Singap ore The rst edition app eared in and the second edition app eared in This is the second edition with an up dated bibliog raphy Acknowledgments The author and the publisher are grateful to the following for p ermis sion to reprint the pap ers included in this volume Academic Press Inc Adv Appl Math American Mathematical So cietyAMS Notices Asso ciation for Computing Machinery J ACM SICACT News SIGACT News Cambridge University Press Algorithmic Information Theory Elsevier Science Publishers Theor Comput Sci IBM IBM J Res Dev IEEE IEEE Trans Info Theory IEEE Symposia Abstracts N Ikeda OsakaUniversityOsaka J Math John Wiley Sons Inc Encyclopedia of Statistical Sci Com mun Pure Appl Math I Kalantari Western Illinois UniversityRecursive Function The ory Newsletter MIT Press The Maximum Entropy Formalism Pergamon Journals Ltd Comp Math Applic Plenum Publishing Corp Int J Theor Phys I Prigogine Universite Libre de Bruxelles Mondes en Developpe ment SpringerVerlag Open Problems in Communication and Compu tation Verlag Kammerer Unverzagt The Universal Turing Machine A HalfCentury Survey W H Freeman and CompanySci Amer Preface Go d not only plays dice in quantum mechanics but even with the whole numb ers The discovery of randomness in arithmetic is presented in my book Algorithmic Information Theory published by Cambridge Univer sity Press There I show that to decide if an algebraic equation in integers has nitely or innitely many solutions is in some cases ab solutely intractable I exhibit an innite series of such arithmetical assertions that are random arithmetical facts and for which it is es sentially the case that the only waytoprovethemistoassumethem as axioms This extreme form of Godel incompleteness theorem shows that some arithmetical truths are totally imp ervious to reasoning The pap ers leading to this result were published over a p erio d of more than twentyyears in widely scattered journals but b ecause of their unity of purp ose they fall together naturally into the present book intended as a companion volume to my Cambridge University Press monograph I hop e that it will serveasa stimulus for work on complexity randomness and unpredictabilityinphysics and biology as well as in metamathematics For the second edition I have added the article Randomness in arithmetic Part I a collection of abstracts Part VI I a bibliogra phyPart VI I I and as an Epilogue two essays whichhave not b een published elsewhere that assess the impact of algorithmic information theory on mathematics and biology resp ectively I should also liketo p oint out that it is straightforward to apply to LISP the techniques used in Part VI to study b oundedtransfer Turing machines Afew fo otnotes have b een added to Part VI but the sub ject richly deserves b o ok length treatmentandIintend to write a b o ok ab out LISP in the near future Gregory Chaitin LISP programsize complexity is discussed at length in my book Information Theoretic Incompleteness published byWorld Scientic in Contents I Intro ductoryTutorial Sur vey Pap ers Randomness and mathematical pro of Randomness in arithmetic On the diculty of computations Informationtheoretic computational complexity Algorithmic information theory Algorithmic information theory II Applications to Metamathematics Godels theorem and information Randomness and Godels theorem An algebraic equation for the halting probability Computing the busy b eaver function III Applications to Biology To a mathematical denition of life Toward a mathematical denition of life IV Technical Pap ers on SelfDelimiting Pro grams A theory of program size formally identical to information theory Incompleteness theorems for random reals Algorithmic entropyofsets V Technical Pap ers on BlankEndmarker Pro grams Informationtheoretic limitations of formal systems A note on Monte Carlo primality tests and algorithmic information theory Informationtheoretic characterizations of recursive in nite strings Program size oracles and the jump op eration VI Technical Pap ers on Turing Machines LISP On the length of programs for computing nite binary se quences On the length of programs for computing nite binary se quences statistical considerations On the simplicity and sp eed of programs for computing innite sets of natural numbers VI I Abstracts On the length of programs for computing nite binary se quences by b oundedtransfer Turing machines On the length of programs for computing nite binary se quences by b oundedtransfer Turing machines I I Computational complexity and Godels incompleteness theorem Computational complexity and Godels incompleteness theorem Informationtheoretic asp ects of the Turing degrees Informationtheoretic asp ects of Posts construction of a simple set On the diculty of generating all binary strings of com plexity less than n On the greatest natural numb er of denitional or informa tion complexity n A necessary and sucient condition for an innite binary string to b e recursive There are few minimal descriptions Informationtheoretic computational complexity A theory of program size formally identical to information theory Recentwork on algorithmic information theory VI I I Bibliography Publications of G J Chaitin Discussions of Chaitins work Epilogue Undecidability randomness in pure mathematics Algorithmic information evolution Ab out the author Part I Intro ductoryTutorialSurvey Pap ers RANDOMNESS AND MATHEMATICAL PROOF Scientic American No May pp by Gregory J Chaitin Abstract Although randomness can beprecisely denedandcan even bemeasured a given number cannot beprovedtoberandom This enigma establishes a limit to what is possible in mathematics Almost everyone has an intuitive notion of what a random numb er is For example consider these two series of binary digits Part IIntro ductoryTutorialSurvey Pap ers The rst is obviously constructed according to a simple rule it consists of the numb er rep eated ten times If one were asked to sp eculate on how the series mightcontinue one could predict with considerable condence that the next two digits would b e and Insp ection of the second series of digits yields no such comprehensive pattern There is no obvious rule governing the formation of the numb er and there is no rational way to guess the succeeding digits The arrangement seems haphazard in other words the sequence app ears to b e a random assortment of s and s The second series of binary digits was generated by ipping a coin times and writing a if the outcome was heads and a if it was tails Tossing a coin is a classical pro cedure for pro ducing a random numb er and one might think at rst that the provenance of the series alone would certify that it is random This is not so Tossing a coin times can pro duce anyoneof or a little more than a million binary series and each of them has exactly the same probabilityThus it should b e no more surprising to obtain the series with an obvious pattern than to obtain the one that seems to b e random each represents an event with a probabilityof If origin in a probabilistic eventwere made the sole criterion of randomness then b oth series would have to b e considered random and indeed so would all others since the same mechanism can generate all the p ossible series The conclusion is singularly unhelpful in distinguishing the random from the orderly Clearly a more sensible denition of randomness is required one that do es not contradict the intuitive concept of a patternless num ber Such a denition has b een devised only in the past years It do es not consider the origin of a numb er but dep ends entirely on the characteristics of the sequence of digits The new denition enables us to describ e the prop erties of a random numb er more precisely than was formerly p ossible and it establishes a hierarchy of degrees of ran domness Of p erhaps even greater interest than the capabilities of the denition however are its limitations In particular the denition can not help to determine except in very sp ecial cases whether or not a given series of digits such as the second one ab ove is in fact random or only seems to b e random This limitation is not a aw in the de nition it is a consequence of a subtle but fundamental anomaly in the foundation of mathematics It is closely related to a famous theo Randomness and Mathematical Pro of rem devised and proved in byKurtGodel which has come to b e known as Godels incompleteness theorem Both the theorem and the recent discoveries concerning the nature of randomness help to dene the b oundaries that constrain certain mathematical metho ds Algorithmic Denition The new denition of randomness has its heritage in information theory the science develop ed mainly since World War I I that studies the transmission of messages Supp ose you have a friend who is visiting a planet in another galaxy and that sending him telegrams is very exp ensive He forgot to take along his tables of trigonometric functions and he has asked you to supply them You could simply translate the numb ers into an appropriate co de such as the binary numb ers and transmit them directly but even the most mo dest tables of the six functions have a few thousand digits so that the cost would b e high A muchcheap er way to convey the