Chapter 7: The Grand Unified Theory of Computation slides © 2017, David Doty ECS 220: Theory of Computation based on “The Nature of Computation” by Moore and Mertens - Great unifications in science Newton: apples and planets follow the same laws of motion and gravity Maxwell: electricity and magnetism are part of a single electromagnetic field, and its oscillations are what we call light Grand Unified Theory of Physics: (TODO: combine gravity with quantum mechanics) Turing (and others): all powerful models of computation known at the time (e.g., general recursive functions, λ-calculus, Turing machines) have equivalent power Chapter 7 2 Computing with steam “I wish to God that these calculations had been executed by steam!” -- Charles Babbage (while checking astronomical tables and finding errors) Any “well-behaved” function f:ℝ→ℝ has a Taylor series expansion around a ∈ ℝ 푓′ 푎 푓′′ 푎 푓′′′ 푎 푓′′′′ 푎 푓 푥 = 푓 푎 + 푥 − 푎 + 푥 − 푎 2 + 푥 − 푎 3 + 푥 − 푎 4 + … 1! 2! 3! 4! cos(x) is approximated within 0.02 in the interval [0,π/2] by the 4th order Taylor expansion: 푥2 푥4 cos 푥 ≈ 1 − + 2 24 7.1: Babbage's Vision and Hilbert's Dream 3 Method of finite differences if p(x) is a degree n polynomial, then Δp(x) = p(x+1) – p(x) is a degree n-1 polynomial x 0 1 2 3 4 5 6 7 8 p(x) = x3 + 2x2 – x + 1 1 3 15 43 93 171 283 435 633 Δp(x) = p(x+1) – p(x) = 3x2 + 7x + 2 2 12 28 50 78 112 152 198 250 ΔΔp(x) = Δp(x+1) – Δp(x) = 6x + 10 10 16 22 28 34 40 46 52 58 ΔΔΔp(x) = ΔΔp(x+1) – ΔΔp(x) = 6 6 6 6 6 6 6 6 6 6 7.1: Babbage's Vision and Hilbert's Dream 4 The Difference Engine (mid 1800’s) original design: replica: detailed explanation of mechanism: https://www.youtube.com/watch?v=PFMBU17eo_4 7.1: Babbage's Vision and Hilbert's Dream 5 7.1: Babbage's Vision and Hilbert's Dream 6 The analytical engine (hypothetical design) • Babbage: “The whole of arithmetic now appeared within the grasp of mechanism. A vague glimpse even of an Analytical Engine at length opened out, and I pursued with enthusiasm the shadowy vision.” • store (memory): 1000 50-digit integers • mill (CPU): addition, subtraction, multiplication, division • programmability: punched cards used by Jacquard Loom • variable cards: copy number into mill • operation card: instruction for mill to execute • combinatorial cards: if-then, goto, for loop 7.1: Babbage's Vision and Hilbert's Dream 7 The analytical engine: more than number crunching • Ada Lovelace: “Again, it might act upon other things besides number, were objects found whose mutual fundamental relations could be expressed by those of the abstract science of operations… Supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.” • i.e., computation isn’t about numbers necessarily, but general symbol manipulation algorithmically-composed music: https://www.youtube.com/watch?v=Cbb08ifTzUk 7.1: Babbage's Vision and Hilbert's Dream 8 The analytical engine: more than number crunching • Ada Lovelace wrote a program for the Analytical Engine with two nested for loops to calculate Bernoulli numbers • Machines were not precise enough in the 1800s to enable the construction of a reliable Analytical Engine • The idea of a universally programmable computer came up again independently in the 1930s, to explore a much more abstract question about the foundations of mathematics. 7.1: Babbage's Vision and Hilbert's Dream 9 The foundations of mathematics • Hilbert’s problems (1900) • 23 problems presented in 1900 to the International Congress of Mathematicians • Hilbert’s 10th problem: “Specify a procedure which, in a finite number of operations, enables one to determine whether or not a given Diophantine equation [a polynomial equation with integer coefficients] with an arbitrary number of variables has an integer solution.” • Note: he’s not bothering even to ask whether one exists… such optimism! • Even more optimism in 1928: the Entscheidungsproblem (“the decision problem”) • “The Entscheidungsproblem is solved if one knows a procedure that allows one to decide the validity of a given logical expression by a finite number of operations.” • e.g., decide if this is true: Ǝx,y,z,n ∈ ℕ \ {0}: (n ≥ 3) Λ (xn + yn = zn) • Subquestion for both Hilbert’s 10th problem and the Entscheidungsproblem: what exactly is a “procedure”? 7.1: Babbage's Vision and Hilbert's Dream 10 The foundations of mathematics • Goal: axiomatic foundation for mathematics: reduce all of mathematics to set theory and logic, creating a formal system powerful enough to prove all mathematical facts • Other mathematical objects can be defined using only sets: ℕ: ordered pairs: sequences: 0 = {} = {} (a,b) is defined as { {a}, {a,b} } [x0, x1, x2, … ] is defined by 1 = {0} = {{}} the function f(n) = xn 2 = {1} = {{{}}} functions: ℝ: 3 = {2} = {{{{}}}} f(n) = n2 is defined by its graph a pair of sequences of … { (n,n2) | n ∈ ℕ } digits [3] and [1,4,1,5,9,…] 7.1: Babbage's Vision and Hilbert's Dream 11 Russell’s paradox • One stumbling block: what objects exactly can “sets” contain? • other sets? • themselves? i.e., can we have a set S such that S ∈ S? • Define R = {S: S ∉ S}. Then R ∈ R ⇔ R ∉ R. • Solution: restricted set comprehension… one must define a larger set from which elements are being taken, i.e., for some existing set A we can write R = {S ∈ A: S ∉ S} 7.1: Babbage's Vision and Hilbert's Dream 12 Zermelo-Fraenkel set theory axioms Axiom 1. (Extensionality) If two sets have the same members, then they are equal: ∀x ∀y [∀z (z ∈ x ⇔ z ∈ y) ⇒ x = y] Axiom 2. (Comprehension) Given any set z and any property ϕ, there is a subset of z consisting of those elements of z with the property ϕ:, for any formula ϕ with free variables among x, z, w1, …, wn we have an axiom ∀z ∀w1 … ∀wn ∃y ∀x (x ∈ y ⇔ x ∈ z ∧ ϕ) Axiom 3. (Pairing) For any sets x, y there is a set which has them as members: ∀x ∀y ∃z (x ∈ z ∧ y ∈ z) Axiom 4. (Union) For any family F of sets, we can form a new set A which has as elements all elements which are in at least one member of F (maybe A has even more elements): ∀F ∃A ∀Y ∀x (x ∈ Y ∧ Y ∈ F ⇒ x ∈ A) Axiom 5. (Power set) For any set x, there is a set which has as elements all subsets of x, and again possibly has more elements: ∀x ∃y ∀z (z ⊆ x ⇒ z ∈ y) Axiom 6. (Infinity) There is a set that has infinitely many elements: ∃x [∅ ∈ x ∧ ∀y ∈ x (y ∪ {y} ∈ x)] Axiom 7. (Replacement) If a function has domain a set, then its range is also a set: For each formula with free variables among x, y, A, w1, …, wn, the following is an axiom. ∀A ∀w1 … ∀wn [∀x ∈ A ∃!y ϕ ⇒ ∃Y ∀x ∈ A ∃y ∈ Y ϕ] Axiom 8. (Foundation) Every nonempty set x has a member y which has no elements in common with x: ∀x [x ≠ ∅ ⇒ ∃y ∈ x (x ∩ y = ∅)] Axiom 9. (Choice) Allows one to pick out elements from each of an infinite family of sets: Gives a way to start with one ∀A [∀x ∈ A (x ≠ ∅) ∧ ∀x ∈ A ∀y ∈ A (x ≠ y ⇒ x∩y = ∅) ⇒ ∃B ∀x ∈ A ∃!y (y ∈ x ∧ y ∈ B) simple set, and then build other sets from that. Taken from 7.1: Babbage's Vision and Hilbert's Dream 13 http://euclid.colorado.edu/~monkd/m6730/gradsets02.pdf Computation: building up from simple parts • General theme: define some “atomic” operations, and define “computable” operations to be any way of composing them. • λ-calculus (Church) • general recursive functions (Gödel, Herbrand) • generalization of what’s now called “primitive recursive functions” (Skolem) • Turing machines (Turing) • All these definitions are equivalent. Church-Turing Thesis: these models capture anything that could “be reasonably called a computation” • Turing, 1936: Halting problem is undecidable • Hence the Entscheidungsproblem is undecidable, since the statement “This Turing machine halts” can easily be encoded as a mathematical theorem. • textbook: “Hilbert may have found the phenomenon of undecidability disappointing, but we find it liberating. Universal computation is everywhere, even in seemingly simple mathematical questions. It lends those questions an infinite richness, and proves that there is no mechanical procedure that can solve them once and for all. Mathematics will never end.” 7.1: Babbage's Vision and Hilbert's Dream 14 If programs can run other programs, some programs must be non-halting • Operating systems/Web browsers with Javascript • Any decent programming language has an interpreter, i.e., a universal program: • U(P,x) = P(x) • Now consider defining Boolean-returning program V that, on input P, runs P(P) and negates the answer, so V(P) = !P(P) • What is the output of V(V)? • Not well-defined: i.e., V(V) doesn’t halt • Any programming language powerful enough to write an interpreter for itself must contain programs that don’t halt • Universality implies non-halting programs 7.2: Universality and Undecidability 15 Relative sizes of sets • Note that for all finite sets A,B, |A| ≥ |B| ⇔ ∃ onto function f:A→B • Let’s take this as a definition of |A| ≥ |B| for any sets A,B • |{nonnegative even integers}| ≥ |ℕ|? • [0,1] ≥ |ℝ|? • P(ℕ) ≥ [0,1]? why? • |ℕ| ≥ |P(ℕ)|? • Theorem (Cantor, 1874): |ℕ| < |P(ℕ)|, i.e.,|ℕ| ≤ |P(ℕ)| and |ℕ| ≥ |P(ℕ)| 7.2: Universality and Undecidability 16 Diagonalization Theorem (Cantor, 1874): No function f:ℕ→P(ℕ) is onto, i.e., for every f:ℕ→P(ℕ), there exists S ⊆ ℕ such that, for all n ∈ ℕ, f(n) ≠ S.