Theory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and tragic death Nathan Brunelle Department of Computer Science University of Virginia www.cs.virginia.edu/~njb2b/theory I’ll be Absent Next Week (But you shouldn’t be) Tuesday: Jack Wadden Thursday: Robbie Hott Micron Automata Processor P and NP Oracles • Originated in Turing’s Ph.D. thesis • Named after the “Oracle of Apollo” at Delphi, ancient Greece • Black-box subroutine / language • Can compute arbitrary functions • Instant computations “for free” • Can greatly increase computation power of basic TMs E.g., oracle for halting problem Turing Machines with Oracles • A special case of “hyper-computation” • Allows “what if” analysis: assumes certain undecidable languages can be recognized • An oracle can profoundly impact the decidability & tractability of a language • Any language / problem can be “relativized” WRT an arbitrary oracle • Undecidability / intractability exists even for oracle machines! Theorem [Turing]: Some problems are still not computable, even by Turing machines with an oracle for the halting problem! Reaching a Contradiction * Assume there exists some TM H that decides ATM*. O is a ATM “Oracle” M* is Relativized to O * * H* is Relativized to O Define D (<M>) = Construct a TM that: D* is Relativized to O Outputs the opposite of the result of simulating H *on input (M,* <M>)* If M accepts its own description <M>* , D*(< M>)* rejects. If M rejects its own description <M>* , D(<M>)* accepts. What happens if we run D* on its own description, <D>* ? If D* accepts its own description <D*> , D(<* D*>) rejects. substituting If D* rejects its own description <D>* , D*(< D*>) accepts. D for M… Students of Turing Degrees Alonzo Church: • Turing (1937); studied by Post (1944) and Kleene (1954) • Quantifies the non-computability (i.e., algorithmic unsolvability) of (decision) problems and languages • Some problems are “more unsolvable” than others! Alan Turing 1912-1954 H* Georg Cantor 1845-1918 Emil Post • Defines computation 1897-1954 “relative” to an oracle. • “Relativized computation” H - an infinite hierarchy! H • A “relativity theory of computation”! Ø Stephen Kleene Turing degree 2 Turing degree 1 Turing degree 0 1909-1994 Students of Turing Degrees Alonzo Church: • Turing degree of a set X is the set of all Turing-equivalent (i.e., mutually-reducible) sets: an equivalence class [X] • Turing degrees form a partial order / join-semilattice Alan Turing • [0]: the unique Turing degree containing all computable sets 1912-1954 • For set X, the “Turing jump” operator X’ is the set of indices of oracle TMs which halt when using X as an oracle • [0’]: Turing degree of the halting problem H; [0’’]: Turing degree of the halting problem H* for TMs with oracle H. Emil Post 1897-1954 Turing jump Turing Stephen Kleene jump 1909-1994 Students of Turing Degrees Alonzo Church: • Each Turing degree is countably infinite (has exactly 0 sets) • There are uncountably many (20) Turing degrees • A Turing degree X is strictly smaller than its Turing jump X’ Alan Turing • For a Turing degree X, the set of degrees smaller than X is 1912-1954 countable; set of degrees larger than X is uncountable (20) • For every Turing degree X there is an incomparable degree (i.e., neither X Y nor Y X holds). • There are 20 pairwise incomparable Turing degrees Emil Post • For every degree X, there is a degree D strictly between X 1897-1954 and X’ so that X < D < X’ (there are actually 0 of them) The structure of the Turing degrees Turing semilattice is extremely complex! jump Turing Stephen Kleene jump 1909-1994 The Extended Chomsky Hierarchy S 2 * Decidable Presburger arithmetic EXPSPACE ? H EXPTIME H* H PSPACE Turing Context sensitive LBA degrees =RE Go NP QBF P anbncn SAT R Context-free ww n n complete complete Det. CF a b - complete complete complete complete - - Regular a* complete - Finite {a,b} NP PSPACE EXPSPACE EXPTIME Recognizable Recognizable Not Recognizable Recognizable Not Not finitely describable finitely Not Complexity Classes • 9*9=81 • 99*99*99= 970299 • 999*999*999*999= 996005996001 =1012 • Number of particles in the universe: about a googol (10100) • After just 10 iterations: 10110 • The scientists owe Ali G an apology Today - End Undecidable Decidable Tractable: “Decidable in a reasonable amount of time and space” Computability Complexity Undecidable Intractable Decidable Tractable 1960s – 2150? ~1800s – 1960s 1960s: Hartmanis and 1900: Hilbert’s Problems Stearns: Complexity class 1936: Turing’s Computable Numbers 1971: Cook/Levin, Karp: P=NP? 1957: Chomsky’s Syntactic Structures 1976: Knuth’s O, Ω, Θ (Mostly) “Dead” field Very Open and Alive Complexity Classes • Computability Classes: sets of problems (languages) that can be solved (decided/recognized) by a given machine • Complexity Classes: sets of problems (languages) that can be solved (decided) by a given machine (usually a TM) within a limited amount of time or space How many complexity classes are there? Infinitely many! “Languages that can be decided by some TM using less than 37 steps” is a complexity class Order Notation • O( f ), ( f ), o( f ), ( f ) • These notations define sets of functions – Generally: functions from positive integer to real • We are interested in how the size of the outputs relates to the size of the inputs Big O • Intuition: the set O(f) is the set of functions that grow no faster than f – More formal definition coming soon • Asymptotic growth rate – As input to f approaches infinity, how fast does value of f increase – Hence, only the fastest-growing term in f matters: O(12n2 + n) O(n3) O(n) O(63n + log n – 423) Formal Definition f O (g) means: There are positive constants c and n0 such that f(n) cg(n) for all values n n0. O Examples f (n) O (g (n)) means: there are positive constants c and n0 such that f(n) cg(n) for all values n n0. 2 x O (x )? Yes, c = 1, n0=2 works fine. Yes, c = 11, n =2 works fine. 10x O (x)? 0 No, no matter what c and n 2 0 x O (x)? we pick, cx2 > x for big enough x Lower Bound: (Omega) f(n) is (g (n)) means: There are positive constants c and n0 such that f (n) cg(n) for all n n0. Difference from O , this was Theta (“Order of”) • Intuition: the set (f ) is the set of functions that grow as fast as f • Definition: f (n) (g (n)) if and only if both: 1. 푓 (푛) ϵ 푂 (푔 (푛)) and 2. 푓 (푛) ϵ 훺(푔 (푛)) (품(풏)) = 푶(품(풏)) ∩ 휴(품 풏 ) – Note: we do not have to pick the same c and n0 values for 1 and 2 • When we say, “f is order g” that means f (n) (g (n)) Summary • Big-O: there exist c, n0 > 0 such that f(n) cg(n) for all n n0. • Omega (): there exist c, n0 > 0 s.t. f(n) cg(n) for all n n0. • Theta (): both O and are true When you were encouraged to use Big-O in cs201/cs216 to analyze the running time of algorithms, what should you have been using? Resource-Bounded Computation Previously: can something be done? Now: how efficiently can it be done? Goal: conserve computational resources: Time, space, other resources? Resource-Bounded Computation Def: L is decidable within time O(t(n)) if some TM M that decides L always halts on all w* within O(t(|w|)) steps / time. Def: L is decidable within space O(s(n)) if some TM M that decides L always halts on all w* while never using more than O(s(|w|)) space / tape cells. Complexity Classes Def: DTIME(t(n))={L | L is decidable within time O(t(n)) by some deterministic TM} Def: NTIME(t(n))={L | L is decidable within time O(t(n)) by some non-deterministic TM} Def: DSPACE(s(n))={L | L decidable within space O(s(n)) by some deterministic TM} Def: NSPACE(s(n))={L | L decidable within space O(s(n)) by some non-deterministic TM} Examples of Space & Time Usage n n Let L1={0 1 | n>0}: For 1-tape TM’s: 2 L1 DTIME(n ) L1 DSPACE(n) L1 DTIME(n log n) For 2-tape TM’s: L1 DTIME(n) L1 DSPACE(log n) Examples of Space & Time Usage Let L2=S* L2 DTIME(n) Theorem: every regular language is in DTIME(n) L2 DSPACE(1) Theorem: every regular language is in DSPACE(1) L2 DTIME(1) Let L3={w$w | w in S*} 2 L3 DTIME(n ) L3 DSPACE(n) L3 DSPACE(log n) .