RecapRecap Decision Problems: Problems that take a yes/no answer Examples: ◦ Language Recognition: x Is a given string a member of language ? ◦ Graph properties: x Does a given graph have a Hamiltonian cycle? ◦ Language equality: x Do two given NFAs recognize the same language? ◦ Questions about Turing Machines: x Does the given TM halt when executed on the given string? RecapRecap A decision problem is decidable (solvable, recursive) if ∃ a TM that a) halts on every input, and b) always gives the right answer A decision problem is partially decidable (partially solvable, recursively enumerable) if ∃ a TM that a) halts/accepts on every YES instance* (may loop on NO instances), and b) always gives the right answer, i.e., does not accept any no instances

*Note: a "YES instance" ("NO instance") is an instance of a problem for which the correct answer is "YES" ("NO") LanguageLanguage RecognitionRecognition asas paradigmparadigm y Any decision problem can be cast as a language recognition problem: 1. Define a way to encode problem instances as strings over some finite alphabet 2. Define the language L to be the set of strings that encode instances that have a “yes” answer y Thus: A method of recognizing L can be used to solve instances of the problem ◦ Given an instance i, encode it as a string s ◦ Give s to the recognizer ◦ Answer “yes” if recognizer says s ∈ L, else “no” y The point: language recognition gives us a way to talk about the “hardness” of kinds of problems Example:TravelingExample:Traveling SalesmanSalesman The problem: Given N cities, a maximum cost B, and the one- way airfare between each pair of cities, is there a tour that goes to each city exactly once and costs at most B? Instance: N, B, list of N2 costs (N of them = 0)

DEN ORD SFO ATL MSP DEN 0 267 129 558 386

N=5 ORD 267 0 678 777 357 B=1800 SFO 129 678 0 598 462 ATL 558 777 598 0 433 MSP 386 357 462 433 0 Example:TravelingExample:Traveling SalesmanSalesman Instance encoded as string over {0,1,#}: N B DEN-ORD 101#111000010000#0#100001011#10000001#100 0101110#110000010#100001011#0#... Language = all strings corresponding to instances having tours with total cost ≤ B DEN ORD SFO ATL MSP DEN 0 267 129 558 386

N=5 ORD 267 0 678 777 357 B=1800 SFO 129 678 0 598 462 ATL 558 777 598 0 433 MSP 386 357 462 433 0 ClassesClasses ofof LanguagesLanguages y We have discussed classes of languages that require machines with increasingly sophisticated capabilities y Each successive class contains all the languages in the earlier classes y Let's review them ◦ order: simplest to most complex ◦ also "smallest" to "largest" RegularRegular LanguagesLanguages

y Example: strings with no b after an a y Defined by ◦ Regular expressions a*b* ◦ Regular grammars S → aS | B | Λ B → bB | Λ y Accepted by ◦ Deterministic Finite Automata (DFAs) ◦ Nondeterministic Finite Automata (NFAs) DeterministicDeterministic ContextContext--FreeFree LanguagesLanguages y Example: {anbn | n ∈ N} y Defined by ◦ LR(k) Grammars (we didn’t discuss these) S → aSb | Λ y Accepted by ◦ Deterministic Pushdown Automata (DPDAs) ContextContext--FreeFree LanguagesLanguages y Example: palindromes over {a,b} y Defined by ◦ Context-Free Grammars S → aSa | bSb | a | b | Λ y Accepted by ◦ [Nondeterministic] Pushdown Automata (PDAs) ContextContext--SensitiveSensitive LanguagesLanguages y Example: {anbncn | n ≥ 1} y Defined by ◦ Grammars with productions of the form v → w where 1 ≤ |v| ≤ |w| x Note: such languages cannot contain Λ y Example: ◦ S → abc | aAbc ◦ A → abC | aAbC ◦ Cb → bC ◦ Cc → y Accepted by ◦ Linear-Bounded Automata (LBAs) x Nondeterministic TMs that use no more than n+2 cells of the tape, where n = size of the input string (see Example 13.2 in text) RecursivelyRecursively EnumerableEnumerable LanguagesLanguages

n y Example: {a | Mn(n) halts } y Defined by ◦ Unrestricted Grammars y Accepted by ◦ Nothing (not decidable) y Can be enumerated by some TM ArbitraryArbitrary LanguagesLanguages

n y Example: {a | Mn halts on every input } y Not defined by any grammar ◦ “Non-grammatical” y Not recursively enumerable ◦ No TM can enumerate the language COMPLEXITYCOMPLEXITY CLASSESCLASSES TheThe ConceptConcept ofof ““ComplexityComplexity”” y Needed: a way to talk about how “hard” a (solvable!) problem is ◦ How to quantify hardness? ◦ Focus on decidable problems x Solution = : a Turing Machine that always halts y Intuition: the harder a problem, the more costly it is to solve ◦ Time cost: number of steps required ◦ Space cost: maximum number of tape cells used y Intuition: cost should be proportional to the size of the instance ◦ Larger instances should cost more before we consider them “hard” x Example: Expect TSP instance with 100 cities is costlier than one with 5 cities – but it's the same problem! DeterministicDeterministic TimeTime ComplexityComplexity y A deterministic Turing Machine’s is the worst-case number of instructions it executes before halting, expressed as a function of input size ◦ Note: input is assumed to be efficiently encoded y “TM M has time complexity f(n)” means f(n) = max(number of steps M takes with s as input) s: input of size n y Example: Add-1 TM has time complexity 2n+3 NondeterministicNondeterministic TimeTime ComplexityComplexity

A nondeterministic Turing Machine’s time complexity is f(n) if f(n) is the maximum number of instructions the TM executes in any of its possible computations on any input of length n. DeterministicDeterministic SpaceSpace ComplexityComplexity y A deterministic Turing Machine’s space complexity is the worst-case number of tape cells it visits, expressed as a function of input size y “TM M has time complexity f(n)” means f(n) = max(number of nonblank cells on tape with s as input) s: input of size n y Example: Add-1 TM has space complexity n+2 NondeterministicNondeterministic SpaceSpace ComplexityComplexity

The space complexity of a nondeterministic Turing Machine, each of whose possible computations halts, is the maximum number of tape cells it scans on any computation for an input of a given length Which is greater – Time or Space?

y For every TM: Time complexity ≥ Space complexity ... because it takes at least one step to write one cell on the tape ExpressingExpressing ComplexityComplexity

y Typically, we use “big-Oh” notation for complexity ◦ Ignores constant and lower-order factors ◦ “M’s time complexity is O(n)” instead of "M's time complexity is 2n+1" y Recall from elsewhere (CS 215): ◦ f(x) = O(g(x)) means

◦ ∃x0 ∃c ∀x (x > x0 → f(x) < cg(x))

c·g(x) f(x) ExpressingExpressing ComplexityComplexity

y Big-Oh notation hides constant factors y In discussing hardness we typically use even coarser measures: ◦ Linear cost: O(n) ◦ Polynomial cost: O(p(n)) where p(n) is some polynomial (we don't care which) ◦ Exponential cost: O(Cn) for some constant C (we don't care what) y Example: Add-1 TM has linear cost Polynomial Cost ≈ Tractability A problem is considered tractable if: There exists a deterministic TM that solves the problem (i.e., always gives the correct answer, always halts) and has polynomial time and space costs

Note: there might be lots of TMs that have higher time or space costs The hardness of the problem is determined by the least-cost TM that solves it Complexity Classes

y 3 important classes of problems: ◦ P: problems for which a deterministic polynomial-time solution exists ◦ NP: problems for which a nondeterministic polynomial-time solution exists ◦ PSPACE: problems for which a deterministic polynomial-space solution exists y These classes define the boundaries of what is considered to be tractable The Class P

y Problems in P can be solved in polynomial time (therefore polynomial space) by a deterministic TM y Examples ◦ Finding a name in a list – O(n) ◦ Sorting a list of names – O(n log n) ◦ Finding all the shortest paths in a graph – O(n3) ◦ Recognizing a – O(n) The Class NP

y Problems in NP can be solved in polynomial time (and space) by a nondeterministic TM ◦ NP: "Nondeterministic Polynomial-time" y Two ways to understand this: ◦ Think of the NTM as executing all of its possible computations in parallel x Accept as soon as any of them accepts x Reject when all of them reject ◦ Think of the NTM as magically "guessing" which a computation that will correctly accept, and carrying it out The Guess-and-Check Model

y Another way of thinking about it (per Hein): ◦ Guess a solution (in one step) S ◦ Run a deterministic algorithm to check (in polynomial time) whether S actually is a solution y Example: TSP ◦ Guess a tour (say, ORD-DEN-MSP-SFO-ATL) ◦ Check whether its cost exceeds the bound by adding the costs of the legs The Class PSPACE

y Problems in PSPACE can be solved in polynomial space (but not necessarily polynomial time) by a deterministic TM ◦ PSPACE: "Polynomial space" y Example: Quantified Boolean Formulas ◦ QBF: a logical formula of the form

Q1x1Q2x2...Qnxn E where each Q is either ∀ or ∃, and E is a expression that uses only ∧, ∨, ¬ and parentheses ◦ Problem: given a QBF, is its value true? Relationships

y Clearly P ⊂ NP ⊂ PSPACE y Nobody knows whether the containment is proper y That is, no one has ever been able to show there exists a problem in either NP – P or PSPACE – NP y On the other hand, nobody has ever proved NP = P or NP = PSPACE!