John von Neumann: The Founding Pierre Marchal Centre Suisse d’Electronique et Father of Artiﬁcial Life de Microtechnique SA CH-2007 Neuchatelˆ Switzerland [email protected] http://www.csem.ch

Abstract Aside from being known for his contributions to mathematics and physics, John von Neumann is considered one of the founding fathers of computer science and engineering. Not only did he do pioneering work on Keywords sequential computing systems, but he also carried out a major computer architecture, universal ma- investigation of parallel architectures, leading to his work on chine, cellular automata, self-repro- ducing machine cellular automata. His exceptional vision and daring, borrowing from biology the concept of genomic information even before the discovery of DNA’s double helix, led him to propose the concept of self-reproducing automata.

1 The Life of John von Neumann

John von Neumann, or rather Margittai1 Neumann Janos Lajos was born in Budapest (Hungary) on December 28th, 1903. His father, Miksa, was a successful lawyer who became the chief counsel and director of the Magyar Jelzalog Hitelbank. His mother, Margit, ran the home. Miksa and Margit fostered a rich intellectual environment in their household, where the entire family participated in regular discussions of current events, science, music, literature, and theater, as well as school subjects. Only governesses able to teach French and German were selected. Growing up in an extended family, with his parents, cousins, and two younger brothers, Michael and Nicholas, Jancsi, as he was called by the family, was a precocious child with a remarkably quick mind. He read avidly with almost photographic memory. As a child, he read a popular 44-volume set on general history. Von Neumann received private tutoring at home until the age of 10, when he was enrolled in the Lutheran Gymnasium. In addition to the usual courses, he received—as did all Jewish students—the instruction of a visiting rabbi teaching Hebrew writing and literature. His talent in mathematics was already recognized as early on as secondary school. The senior mathematics teacher, L. Racz, encouraged his parents to provide him with training beyond that offered by the school. A young mathematician, M. Fekete, tutored von Neumann at home throughout his eight years of secondary school. He read mathematics at the University of Budapest in 1921; however, due to a rather turbulent period in Hungary’s history, he adopted the unorthodox practice of attending the university only at the end of each term when examinations were given. To obtain the practical education desired by his father, von Neumann enrolled in the Eidgenossische¨ Technische Hochschule in Zurich in 1923 to study chemical engineering. He received his diploma in 1925. In Zurich, much of his spare time was devoted to mathematics, writing publications, and corresponding with mathematicians. Thus, one year later

1 His father was awarded the von Margitta title of nobility by Emperor Franz Joseph in 1913 for his contribution to the economic development of Hungary, which he passed on to his sons. When von Neumann moved to Zurich, he used the name Johann Neumann von Margitta, which the German-language publishers shortened to von Neumann.

c 1998 Massachusetts Institute of Technology Artiﬁcial Life 4: 229–235 (1998)

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/106454698568567 by guest on 30 September 2021 P. Marchal The Founding Father of Artiﬁcial Life

(1926), he received his doctorate in mathematics from the University of Budapest, having written a thesis on set theory. Von Neumann arrived in Princeton in February 1930, returning to Europe every summer from 1930 to 1938. In the summer of 1932 he came back with his bride, the former Mariette Kovesi, their marriage producing his only child, Marina, in 1935. The marriage ended in divorce in 1937. That same year he was naturalized as a U.S. citizen. In 1939, von Neumann returned from a summer vacation in Hungary with a new wife, the former Klara Dan, who became one of the ﬁrst computer programmers. Throughout his life, von Neumann received numerous honors and awards: hon- orary degrees from Princeton University (1947), University of Pennsylvania, Harvard University (1949), Istanbul University, Case Institute of Technology, Maryland Univer- sity (1952), Munich Polytechnic Institute (1953), and Columbia University (1954). He also received the Medal of Merit—Presidential Award (1947), the Science Award (1955), the Medal of Freedom—Presidential Award (1956), the Enrico Fermi Science Award (1956), and the American Meteorological Society Award for extraordinary scientiﬁc ac- complishment (1957). His professional life began with a Rockefeller Fellowship at Gottingen¨ University for 1926–1927. In 1927 he became Privatdozent at Berlin University, the youngest ever nominated. At the beginning of the academic year 1929–1930 he was also Privat- dozent at Hamburg University. In February 1930, he was nominated visiting lecturer in mathematics at Princeton University (invited by Oswald Veblen). In January 1933, he became one of four professors appointed by the world-famous Princeton Institute for Advanced Studies (IAS), along with Oswald Veblen, James Alexander, and Albert Ein- stein. He remained there for the rest of his life. During the war he was involved in sev- eral government consultancies: Ballistics Research Labs, Navy Bureau of Ordnance, Los Alamos Scientiﬁc Laboratory, Naval Ordnance Laboratory, Washington DC Research and Development Board, Oak Ridge National Laboratory, Armed Forces Special Weapons Project, Weapon Systems Evaluation Group, U.S. Air Force Scientiﬁc Advisory Board, and Member General Advisory Committee of the U.S. Atomic Energy Commission, Cen- tral Intelligence Agency, Livermore Radiation Laboratory, Strategic Missiles Evaluation Committee of the U.S. Air Force, National Security Agency Advisory Board at Washing- ton DC, Nuclear Weapons Panel and Scientiﬁc Advisory Board of the U.S. Air Force, Technical Advisory Panel on Atomic Energy for the Department of Defense, and Ad Hoc Panel on University Computing Facilities for the National Science Foundation.

2 Scientiﬁc Contribution

John von Neumann’s scientiﬁc contribution may be divided into three periods of his life. The ﬁrst period began with his university days and ended with his work with government consultancies during the war. The second period corresponded mainly to his wartime occupation. The last period, devoted to computer science and engineering, was driven by the huge number of calculations necessary to resolve many war-related problems.

2.1 The University Days During the ﬁrst period, von Neumann addressed the topics of measure theory (Haar measure), the theory of integration, and game theory and its application to economics (thus founding the ﬁeld of mathematical economics). Aside from this work, he concen- trated on set theory, logical foundation of mathematics (Hilbert’s formalist program2),

2 As early as 1925, von Neumann had some inkling of problems with the formalist program and it was a great disappointment to him not to have discovered the incompleteness of the results himself.

230 Artiﬁcial Life Volume 4, Number 3

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/106454698568567 by guest on 30 September 2021 P. Marchal The Founding Father of Artiﬁcial Life

Table 1. Some “offspring” of the IAS machine.

Name Institution Avidac Argone National Lab Besk Swedish Board of Computing Machinery Besm Academy of Sciences, Moscow Dask Danish Academy, Institute of Computing Machinery George Argone National Lab IBM 701 IBM Corporation Illiac University of Illinois Johnniac RAND Corporation Maniac Los Alamos Scientiﬁc Lab Msudc Michigan State University Oracle Oak Ridge National Lab. Perm Technische Hochschule, Munich Silliac University of Sydney SMIL Lund University TC-1 International Telemeter Corporation Weizac Weizmann Institute, Rehovoth

spectral theory of multidimensional Hilbert spaces, ergodic theory, rings of operator theory (now known as von Neumann algebras), as well as the mathematical founda- tions of quantum mechanics.

2.2 The War Epoch Starting in 1941, von Neumann began to contribute increasingly to the ongoing war effort. His ﬁelds of contribution covered the theory of detonation, arrangement of high-power explosives in shaped charges, physical effects of detonation and mine war- fare, mine countermeasures, aerodynamics (with Theodore von Karman), atomic bombs (Los Alamos), interaction and reﬂection of blast waves, effects of explosive blasts, and effects of projectile impact on various structures. This work tended to generate a huge number of calculations, leading von Neumann to develop appropriate numerical meth- ods. These were followed by computing methods for high-speed calculating devices as soon as the ﬁrst computing devices appeared.

2.3 The Birth of Computer Science Driven by the necessity of using computational devices, John von Neumann partici- pated in discussions on the design of the EDVAC machine (electronic discrete variable arithmetic computer) with J. Prespert Eckert and John W. Mauchly, addressing logical design as well as programming aspects. Then he turned toward engineering the IAS computer, where he participated in the early design, solving the memory problem, and was also involved in the computer’s maintainability. The IAS machine gave birth to several offspring (see Table 1). He also concentrated on numerical analysis: evaluation of algorithm complexity, stability analysis (round-error, cumulative error), solving matrices and systems of linear equations, addressing partial differential equations, and random numbers (together with S. Ulam he developed the Monte Carlo method). His work on numerical linear algebra was partly driven by his desire to adapt hand-numerical methods to computers. Finally, he provided fundamental insight into the mathematical theory of linear programming.

Artiﬁcial Life Volume 4, Number 3 231

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/106454698568567 by guest on 30 September 2021 P. Marchal The Founding Father of Artiﬁcial Life

3 From Calculating Devices to Computing Machines

3.1 Electronic Storage: Memory Without a doubt, von Neumann’s stroke of genius was the concept of the stored pro- gram [3]. Its impact on machine architecture was to transform calculating devices into computing machines. Early calculating devices were composed of sets of operators and a few registers. The inputs were wired through huge boards, called function tables, covered with hundreds of decimal switches. Specially trained people, called “program- mers,” set the switches and plugged in cables to activate the computations in a lockstep fashion. Running a single program on the ENIAC generally required several days of wire connecting by a team of programmers. These programmers played the role of the control unit of present-day computers: reading instruction codes—on a sheet of paper—and translating them into sequences of control signals (a single cable with a large fanout for each step) that drove operators. Von Neumann wanted to automate this process. His key idea was to code the instructions and to store them in memory, as was done with the data on which the computations were performed. A new unit hence needed to be added: the control unit. The role of this unit was to fetch an instruction from the memory, interpret it, and control the arithmetic and logic units as well as the memory accesses, according to the instruction code, so as to perform the correct operations. This is the basis of the so-called von Neumann architecture.

3.2 Numbers: Binary Representation A second improvement made by von Neumann was the introduction of the binary digit numbering scheme. At that time the tuning, as well as the storage of the function tables, was realized using decimal numbers. This process results in an overuse of ﬂip-ﬂops. In addition to the fact that “the ﬂip-ﬂops and memory elements were truly binary devices” [1] (p. 104) and were hence better adaped to a binary numbering scheme, a second justiﬁcation was that “an important part of the machine was not arithmetical but logical in nature, and logic, being a yes-no system, was fundamentally binary” [1] (p. 105).

3.3 Foundations of Computer Science John von Neumann managed to render extremely complicated things crystal clear. To- ward this end, after addressing the logical foundations of mathematics, and the math- ematical foundations of quantum mechanics, he ﬁnally focused on the mathematical- logical-biological foundations of computer science. He fathered the ﬁrst computa- tional approach to the generation of lifelike behavior. In his pioneering work on self-reproducing automata, von Neumann aimed at abstracting the logical underpin- nings of natural self-reproduction. His ﬁrst attempt was oriented at a kinematic model: an automaton that ﬂoats in a reservoir of liquid ﬁlled with a virtually unlimited supply of elementary parts from which it is built. Given a set of instructions, the kinematic automaton will sort through the parts, locate those it needs, and assemble its progeny according to the instructions. This three-dimensional model was much too complex to work with. Stanislaw Ulam3 suggested that the notion of self-reproducing machines would be amenable to rigorous treatment if it could be described in a “cell-space” format. The cellular approach had the added advantage of reducing the spatial com- plexity of the self-reproducing automaton from three dimensions to two. Von Neumann adopted this formalism, now known as cellular automata, for its crystalline regularities. He developed the theory of automata [2]: “an interdisciplinary subject lying in the in- termediate area between logic, communication theory and physiology” (p. 204), as he

3 A mathematician friend of von Neumann who worked on crystal growth and investigated the dynamics of pattern production and competition.

232 Artiﬁcial Life Volume 4, Number 3

liked to deﬁne it. He tackled the deﬁnitions of logical universality, construction univer- sality, and evolvability in computing machines and automata. Von Neumann addressed ﬁve basic questions.

3.3.1 The Universal Machine His ﬁrst question was, what is the class of automata able to perform all logical operations using ﬁnite means? Taking advantage of Turing’s contribution to the decision problem,4 he answered the question by formulating Turing’s machine model in terms of cellular automata. We note in passing that the cutting edge of today’s VLSI technology is represented by ﬁeld programmable gate arrays, or FPGAs. They are the most complicated circuits built nowadays (65 million transistors at the end of 1997). It is amazing to see that these FPGAs are nothing but cellular arrays of programmable logic cells. The programming bits deﬁne the functionality of the cell (automaton). With respect to von Neumann’s universal machine, slight modiﬁcations have been made: The interconnection scheme has been improved, and the constraints of a ﬁnite-state machine have been relaxed [5].

3.3.2 The Constructing Automaton His second question was, what is the class of automata that can be constructed from raw materials by an appropriately deﬁned automaton? He showed how to adapt the Turing machine, transforming it into a constructor, by using a constructing arm that could move about and deﬁne the functionality of any cell of the cellular automaton. The key idea introduced by von Neumann was the elemental cell’s ability to change functionality from one moment to the next. This resolved the major problem of the kinematic model, allowing the programming of a constructor that could perform three complex tasks: sorting through the parts, locating the adequate ones, and assembling them.

3.3.3 The Universal Constructing Machine Von Neumann’s third question was, is there an automaton able to construct any other automaton? The solution to this problem was the introduction of the universal con- structor. A major problem was the complexity paradox: An automaton able to produce another automaton should be at least as complex as the one being built. Because the description of the automaton uses a ﬁnite set of symbols (states), the higher the com- plexity of the automaton to be built, the longer the description “tape” (which contains the construction instructions).

3.3.4 The Self-Reproducing Automaton His fourth question was, is there an automaton able to construct a copy of itself? This question deﬁned the concept of self-reproduction, a special case of universal construction. Von Neumann’s solution was to build an automaton composed of three subparts (Figure 1): Part A is a universal constructor, part B is an automaton able— given any machine description—to build a copy of it, and part C is an automaton controlling the execution of subparts A and B. Self-reproduction takes place when the description of a universal constructor (A B C) is given to the universal constructor itself; the description is placed on the tape,+ enters+ the automaton B, which then provides a copy to automaton C. This copy is transferred to automaton A to physically build the progeny. Afterward, a second copy of the description tape is transmitted to the progeny

4 Turing’s solution of the decision problem (Entscheidungs-problem), as formulated by Hilbert, was framed in terms of theoretical machines able to carry out the decision procedure. In effect, the universal Turing machine is nothing but a mathematical model of a stored program computer.

Artiﬁcial Life Volume 4, Number 3 233

Figure 1. Self-reproducing universal constructor.

so that both machines (mother and daughter) possess a copy of their own description. This is genuine self-reproduction. The contents of the description tape are read once as instructions to be executed to build the offspring and once as data to be copied to provide the offspring with its own description. Such a machine has never been realized either in hardware or in software. In fact, more than two hundred thousands cells are necessary for its realization. But, as the system is modular, each part has been tested separately, enabling one to prove the correct behavior of the complete system [7]. The ﬁrst physical implementation of von Neumann’s automaton was realized in 1996, while a genuine self-reproducing machine was built in 1994, and improved versions have been realized since then [6]. A remaining problem of self-reproduction is, at what time does the offspring begin to function? Examination of the possible states of the automaton shows how von Neu- mann resolved this problem. The existence of excited (respectively, unexcited) states corresponds to functional (respectively, not-yet-functional) wires. Present implemen- tations make use of two clocks: a developmental clock and a functional clock.

3.3.5 Increasing Complexity The ﬁfth and ﬁnal question was, can the construction capacity lead to the evolution of ever-increasingly complex machines? He brieﬂy discussed this ﬁnal question but gave no deﬁnitive answer. This question is related to current-day research on the origin of life and the evolution of species.

4 Only Human in Spite of Himself

In addition to the quantity of work he had done (nearly two hundred publications), John von Neumann enjoyed night life and cabarets more than most of his colleagues, who saw him as a bon vivant. Von Neumann is also remembered as a man who exuded bravado and a zest for life, and whose involvement in mathematics was no bar to the enjoyment of worldly pleasures. John von Neumann was a cheerful man, “very friendly and accessible,” “warm-hearted and friendly with people,” “easy to get along with ... didn’t give himself airs” [4]. This demigod, as described by some of his colleagues, was “only human in spite of himself” [4]. Unfortunately, on February 8, 1957, John von Neumann died from cancer. He tried to make advances in the “technology of life” right up to the end, possibly in an attempt to compensate for his involvement in the war effort, “the technology of death” [4]. He left us with uncompleted work, but he opened wide avenues of research such as bio-inspired systems.

234 Artiﬁcial Life Volume 4, Number 3

References 1. Aspray, W. (1990). John von Neumann and the origins of modern computing. Cambridge, MA: MIT Press. 2. Aspray, W., & Burks, A. (1986). Papers of John von Neumann on computing and computer theory. Charles Babbage Institute Reprints, series for the History of Computing (vol. 12). 3. Goldstine, H. H. (1972). The computer from Pascal to von Neumann. Princeton, NJ: Princeton University Press. 4. Heims, S. J. (1980). John von Neumann and Norbert Weiner—From mathematics to the technologies of life and death. Cambridge, MA: MIT Press. 5. Mange, D., & Tomassini, M. (1998). Bio-inspired computing machines: Towards novel computational architectures. Lausanne, Switzerland: Presses Polytechniques et Universitaires Romandes. 6. Sipper, M. (1998). Fifty years of research on self-replication: An overview. Artiﬁcial Life, 4, 237–257. 7. von Neumann, J. (1996). Theory of self-reproducing automata. Urbana: University of Illinois Press. Edited and completed by A. W. Burks.

Note: More than 3,000 references on John von Neumann are available on the Web.

Artiﬁcial Life Volume 4, Number 3 235

Downloaded from http://www.mitpressjournals.org/doi/pdf/10.1162/106454698568567 by guest on 30 September 2021