Review • Languages and Grammars – Alphabets, strings, languages • Regular Languages – Deterministic Finite and Nondeterministic Automata – Equivalence of NFA and DFA Regular Expressions CS 301 - Lecture 23 – Regular Grammars – Properties of Regular Languages – Languages that are not regular and the pumping lemma Recursive and Recursively • Context Free Languages – Context Free Grammars – Derivations: leftmost, rightmost and derivation trees Enumerable Languages – Parsing and ambiguity – Simplifications and Normal Forms – Nondeterministic Pushdown Automata – Pushdown Automata and Context Free Grammars Fall 2008 – Deterministic Pushdown Automata – Pumping Lemma for context free grammars – Properties of Context Free Grammars • Turing Machines – Definition, Accepting Languages, and Computing Functions – Combining Turing Machines and Turing’s Thesis – Turing Machine Variations – Universal Turing Machine and Linear Bounded Automata – Today: Recursive and Recursively Enumerable Languages Definition: A language is recursively enumerable Recursively Enumerable if some Turing machine accepts it and Recursive Languages 1 Let L be a recursively enumerable language Definition: A language is recursive and M the Turing Machine that accepts it if some Turing machine accepts it and halts on any input string For string w : if w∈ L then M halts in a final state In other words: if w∉ L then M halts in a non-final state A language is recursive if there is or loops forever a membership algorithm for it Let L be a recursive language and M the Turing Machine that accepts it Turing acceptable languages For string w : and Enumeration Procedures if w∈ L then M halts in a final state if w∉ L then M halts in a non-final state 2 We will prove: Theorem: (weak result) • If a language is recursive then if a language L is recursive then there is an enumeration procedure for it there is an enumeration procedure for it (strong result) • A language is recursively enumerable if and only if there is an enumeration procedure for it Proof: If the alphabet is { a , b } then ~ M can enumerate strings as follows: Enumeration Machine a b ~ M M aa ab ba bb aaa Enumerates all aab Accepts L strings of input alphabet ...... 3 Enumeration procedure Example: L = {b,ab,bb,aaa,....} ~ Enumeration Repeat: M L(M ) Output ~ M generates a string w a b b b M checks if w∈ L aa w ab ab ab YES: print to output ba NO: ignore w bb bb bb aaa aaa aaa aab End of Proof ...... ...... ...... Theorem: Proof: if language L is recursively enumerable then Enumeration Machine there is an enumeration procedure for it ~ M M Enumerates all Accepts L strings of input alphabet 4 NAIVE APPROACH If the alphabet is { a , b } then ~ Enumeration procedure M can enumerate strings as follows: ~ Repeat: M generates a string w a b M checks if w∈ L aa ab YES: print w to output ba NO: ignore w bb aaa aab Problem: If w ∉ L machine M may loop forever BETTER APPROACH ~ ~ Generates third string w M Generates first string w1 M 3 M executes first step on w1 M executes first step on w3 second step on w2 ~ M Generates second string w2 third step on w1 M executes first step on w2 And so on............ second step on w1 5 w1 w2 w3 w4 If for any string w i 1 1 1 1 machine M halts in a final state Step then it prints w i on the output in 2 2 2 2 string 3 3 3 3 End of Proof Theorem: Proof: Input Tape w If for language L there is an enumeration procedure Machine that then L is recursively enumerable accepts L Enumerator Compare for L 6 Turing machine that accepts L We have proven: For input string w A language is recursively enumerable Repeat: if and only if • Using the enumerator, there is an enumeration procedure for it generate the next string of L • Compare generated string with w If same, accept and exit loop End of Proof We will prove: Non Recursively Enumerable 1. There is a specific language which is not recursively enumerable Recursively Enumerable (not accepted by any Turing Machine) 2. There is a specific language Recursive which is recursively enumerable but not recursive 7 Some Languages Are Not Definition: A set is uncountable Recursively Enumerable if it is not countable Proof Using Uncountable Sets Theorem: Proof: Let S be an infinite countable set Since S is countable, we can write S S = {s1,s2,s3,…} The powerset 2 of S is uncountable Elements of S 8 We encode each element of the power set Elements of the powerset have the form: with a binary string of 0’s and 1’s {s1,s3} Encoding Powerset s s s s element 1 2 3 4 {s5,s7,s9,s10} {s1} 1 0 0 0 …… {s2,s3} 0 1 1 0 {s1,s3,s4} 1 0 1 1 Let’s assume (for contradiction) Powerset Encoding that the powerset is countable. element t1 1 0 0 0 0 t2 1 1 0 0 0 Then: we can enumerate the elements of the powerset t3 1 1 0 1 0 t4 1 1 0 0 1 9 t1 1 0 0 0 0 t 1 1 0 0 0 Take the powerset element 2 whose bits are the complements t3 1 1 0 1 0 in the diagonal t4 1 1 0 0 1 New element: 0011… (birary complement of diagonal) The new element must be some ti Since we have a contradiction: of the powerset S The powerset 2 of S is uncountable However, that’s impossible: from definition of ti the i-th bit of t i must be the complement of itself Contradiction!!! 10 An Application: Languages Example Alphabet : {a,b} Example Alphabet : {a,b} The set of all Strings: The set of all Strings: S ={a,b}* ={λ,a,b,aa,ab,ba,bb,aaa,aab,…} S ={a,b}* ={λ,a,b,aa,ab,ba,bb,aaa,aab,…} infinite and countable infinite and countable A language is a subset of S : L = {aa,ab,aab} Languages: uncountable Example Alphabet : {a,b} L1 L2 L3 Lk The set of all Strings: S ={a,b}* ={λ,a,b,aa,ab,ba,bb,aaa,aab,…} M1 M2 M3 ? infinite and countable Turing machines: countable The powerset of S contains all languages: S 2 ={{λ},{a},{a,b}{aa,ab,aab},…} There are more languages L1 L2 L3 L4 than Turing Machines uncountable 11 Conclusion: Languages not accepted by Turing Machines Lk There are some languages not accepted by Turing Machines Languages Accepted by Turing Machines (These languages cannot be described by algorithms) We want to find a language that is not Recursively Enumerable A Language which is not Recursively Enumerable This language is not accepted by any Turing Machine 12 Consider alphabet {a} Consider Turing Machines that accept languages over alphabet {a} Strings: a, aa, aaa, aaaa, … 1 2 3 4 a a a a They are countable: M1, M 2, M3, M 4, … Example language accepted by Mi 1 2 3 4 a a a a L(Mi ) = {aa,aaaa,aaaaaa} L(M ) 0 1 0 1 2 4 6 1 L(Mi ) = {a ,a ,a } L(M2) 1 0 0 1 Alternative representation L(M3) 0 1 1 1 1 2 3 4 5 6 7 a a a a a a a L(M4) 0 0 0 1 L(Mi ) 0 1 0 1 0 1 0 13 1 2 3 4 a a a a Consider the language L(M1) 0 1 0 1 i i 1 0 0 1 L = {a : a ∈ L(Mi )} L(M2) L(M3) 0 1 1 1 L(M4) 0 0 0 1 L consists from the 1’s in the diagonal L = {a3,a4,…} 1 2 3 4 a a a a Consider the language L L(M1) 0 1 0 1 i i L(M ) 1 0 0 1 L = {a : a ∈ L(Mi )} 2 L(M ) 0 1 1 1 i i 3 L = {a : a ∉ L(Mi )} L(M4) 0 0 0 1 L consists of the 0’s in the diagonal L = {a1,a2,…} 14 Theorem: Proof: Language L is not recursively enumerable Assume for contradiction that L is recursively enumerable There must exist some machine M k that accepts L L(M k ) = L 1 2 3 4 1 2 3 4 a a a a a a a a L(M1) 0 1 0 1 L(M1) 0 1 0 1 L(M2) 1 0 0 1 L(M2) 1 0 0 1 L(M3) 0 1 1 1 L(M3) 0 1 1 1 L(M4) 0 0 0 1 L(M4) 0 0 0 1 1 a ∈ L(Mk ) M ≠ M Question: Mk = M1? Answer: k 1 1 a ∉ L(M1) 15 1 2 3 4 1 2 3 4 a a a a a a a a L(M1) 0 1 0 1 L(M1) 0 1 0 1 L(M2) 1 0 0 1 L(M2) 1 0 0 1 L(M3) 0 1 1 1 L(M3) 0 1 1 1 L(M4) 0 0 0 1 L(M4) 0 0 0 1 2 a ∈ L(Mk ) M ≠ M Question: Mk = M2 ? Answer: k 2 2 a ∉ L(M2) 1 2 3 4 1 2 3 4 a a a a a a a a L(M1) 0 1 0 1 L(M1) 0 1 0 1 L(M2) 1 0 0 1 L(M2) 1 0 0 1 L(M3) 0 1 1 1 L(M3) 0 1 1 1 L(M4) 0 0 0 1 L(M4) 0 0 0 1 3 a ∉ L(Mk ) M ≠ M Question: Mk =M 3 ? Answer: k 3 3 a ∈ L(M3) 16 Therefore, the machine M k cannot exist Similarly: M k ≠ Mi for any i Because either: Therefore, the language L i i is not recursively enumerable a ∈ L(M k ) a ∉ L(M k ) or i i a ∉ L(Mi ) a ∈ L(Mi ) End of Proof Observation: Non Recursively Enumerable L There is no algorithm that describes L Recursively Enumerable (otherwise L would be accepted by Recursive some Turing Machine) 17 What’s Next • Read – Linz Chapter 1,2.1, 2.2, 2.3, (skip 2.4), 3, 4, 5, 6.1, 6.2, (skip 6.3), 7.1, 7.2, 7.3, (skip 7.4), 8, 9, 10, 11 – JFLAP Chapter 1, 2.1, (skip 2.2), 3, 4, 5, 6, 7, (skip 8), 9, (skip 10), 11 • Next Lecture Topics From 11.1, 11.2, 11.3, and 11.4 – More Recursive Languages, Unrestricted Grammars, Context Sensitive Grammars, and the Chomsky Hierarchy • Quiz 3 in Recitation on Wednesday 11/12 – Covers Linz 7.1, 7.2, 7.3, (skip 7.4), 8, and JFLAP 5,6,7 – Closed book, but you may bring one sheet of 8.5 x 11 inch paper with any notes you like.