Theory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and tragic death
Nathan Brunelle
Department of Computer Science
University of Virginia
www.cs.virginia.edu/~njb2b/theory
I’ll be Absent Next Week (But you shouldn’t be)
Tuesday: Jack Wadden Thursday: Robbie Hott Micron Automata Processor P and NP Oracles • Originated in Turing’s Ph.D. thesis • Named after the “Oracle of Apollo” at Delphi, ancient Greece • Black-box subroutine / language • Can compute arbitrary functions • Instant computations “for free” • Can greatly increase computation power of basic TMs E.g., oracle for halting problem Turing Machines with Oracles
• A special case of “hyper-computation” • Allows “what if” analysis: assumes certain undecidable languages can be recognized • An oracle can profoundly impact the decidability & tractability of a language • Any language / problem can be “relativized” WRT an arbitrary oracle • Undecidability / intractability exists even for oracle machines!
Theorem [Turing]: Some problems are still not computable, even by Turing machines with an oracle for the halting problem! Reaching a Contradiction * Assume there exists some TM H that decides ATM*. O is a ATM “Oracle” M* is Relativized to O * * H* is Relativized to O Define D (
• “Relativized computation” H - an infinite hierarchy! H
• A “relativity theory of computation”! Ø Stephen Kleene Turing degree 2 Turing degree 1 Turing degree 0 1909-1994 Students of Turing Degrees Alonzo Church: • Turing degree of a set X is the set of all Turing-equivalent (i.e., mutually-reducible) sets: an equivalence class [X] • Turing degrees form a partial order / join-semilattice Alan Turing • [0]: the unique Turing degree containing all computable sets 1912-1954 • For set X, the “Turing jump” operator X’ is the set of indices of oracle TMs which halt when using X as an oracle • [0’]: Turing degree of the halting problem H; [0’’]: Turing degree of the halting problem H* for TMs with oracle H. Emil Post 1897-1954
Turing jump
Turing Stephen Kleene jump 1909-1994 Students of Turing Degrees Alonzo Church:
• Each Turing degree is countably infinite (has exactly 0 sets) • There are uncountably many (20) Turing degrees • A Turing degree X is strictly smaller than its Turing jump X’ Alan Turing • For a Turing degree X, the set of degrees smaller than X is 1912-1954 countable; set of degrees larger than X is uncountable (20) • For every Turing degree X there is an incomparable degree (i.e., neither X Y nor Y X holds). • There are 20 pairwise incomparable Turing degrees Emil Post • For every degree X, there is a degree D strictly between X 1897-1954
and X’ so that X < D < X’ (there are actually 0 of them)
The structure of the Turing degrees Turing semilattice is extremely complex! jump Turing Stephen Kleene jump 1909-1994
2 ? S
Not finitely describable *
degrees Turing H* H
Not Recognizable The Extended Hierarchy Chomsky
H
Recognizable
Decidable EXPSPACE EXPSPACE-complete =RE EXPTIME EXPTIME-complete Go PSPACE
PSPACE-complete QBF
Context sensitive
NP-complete SAT Presburgerarithmetic NP P Context
Det.CF
Regular Finite - free
LBA a
{a,b}
ww n a
b n a* n b
c R n
n
Complexity Classes • 9*9=81 • 99*99*99= 970299 • 999*999*999*999= 996005996001 =1012 • Number of particles in the universe: about a googol (10100) • After just 10 iterations: 10110 • The scientists owe Ali G an apology
Today - End Undecidable
Decidable
Tractable: “Decidable in a reasonable amount of time and space” Computability Complexity
Undecidable Intractable
Decidable Tractable
1960s – 2150? ~1800s – 1960s 1960s: Hartmanis and 1900: Hilbert’s Problems Stearns: Complexity class 1936: Turing’s Computable Numbers 1971: Cook/Levin, Karp: P=NP? 1957: Chomsky’s Syntactic Structures 1976: Knuth’s O, Ω, Θ
(Mostly) “Dead” field Very Open and Alive Complexity Classes • Computability Classes: sets of problems (languages) that can be solved (decided/recognized) by a given machine • Complexity Classes: sets of problems (languages) that can be solved (decided) by a given machine (usually a TM) within a limited amount of time or space
How many complexity classes are there? Infinitely many! “Languages that can be decided by some TM using less than 37 steps” is a complexity class Order Notation • O( f ), ( f ), o( f ), ( f ) • These notations define sets of functions – Generally: functions from positive integer to real • We are interested in how the size of the outputs relates to the size of the inputs Big O
• Intuition: the set O(f) is the set of functions that grow no faster than f – More formal definition coming soon • Asymptotic growth rate – As input to f approaches infinity, how fast does value of f increase – Hence, only the fastest-growing term in f matters: O(12n2 + n) O(n3) O(n) O(63n + log n – 423) Formal Definition f O (g) means:
There are positive constants c and n0 such that f(n) cg(n)
for all values n n0. O Examples f (n) O (g (n)) means: there are positive constants c and n0 such that f(n) cg(n) for all values n n0.
2 x O (x )? Yes, c = 1, n0=2 works fine.
Yes, c = 11, n =2 works fine. 10x O (x)? 0 No, no matter what c and n 2 0 x O (x)? we pick, cx2 > x for big enough x Lower Bound: (Omega) f(n) is (g (n)) means:
There are positive constants c and n0 such that f (n) cg(n)
for all n n0.
Difference from O , this was Theta (“Order of”) • Intuition: the set (f ) is the set of functions that grow as fast as f • Definition: f (n) (g (n)) if and only if both: 1. 푓 (푛) ϵ 푂 (푔 (푛)) and 2. 푓 (푛) ϵ 훺(푔 (푛)) (품(풏)) = 푶(품(풏)) ∩ 휴(품 풏 )
– Note: we do not have to pick the same c and n0 values for 1 and 2 • When we say, “f is order g” that means f (n) (g (n)) Summary
• Big-O: there exist c, n0 > 0 such that f(n) cg(n) for all n n0.
• Omega (): there exist c, n0 > 0 s.t. f(n) cg(n) for all n n0. • Theta (): both O and are true
When you were encouraged to use Big-O in cs201/cs216 to analyze the running time of algorithms, what should you have been using? Resource-Bounded Computation Previously: can something be done? Now: how efficiently can it be done?
Goal: conserve computational resources: Time, space, other resources?
Resource-Bounded Computation
Def: L is decidable within time O(t(n)) if some TM M that decides L always halts on all w* within O(t(|w|)) steps / time.
Def: L is decidable within space O(s(n)) if some TM M that decides L always halts on all w* while never using more than O(s(|w|)) space / tape cells. Complexity Classes
Def: DTIME(t(n))={L | L is decidable within time O(t(n)) by some deterministic TM}
Def: NTIME(t(n))={L | L is decidable within time O(t(n)) by some non-deterministic TM}
Def: DSPACE(s(n))={L | L decidable within space O(s(n)) by some deterministic TM}
Def: NSPACE(s(n))={L | L decidable within space O(s(n)) by some non-deterministic TM} Examples of Space & Time Usage n n Let L1={0 1 | n>0}:
For 1-tape TM’s: 2 L1 DTIME(n )
L1 DSPACE(n)
L1 DTIME(n log n)
For 2-tape TM’s:
L1 DTIME(n)
L1 DSPACE(log n) Examples of Space & Time Usage
Let L2=S*
L2 DTIME(n) Theorem: every regular language is in DTIME(n)
L2 DSPACE(1) Theorem: every regular language is in DSPACE(1)
L2 DTIME(1)
Let L3={w$w | w in S*}
2 L3 DTIME(n )
L3 DSPACE(n)
L3 DSPACE(log n)