Homework

• Homework #5 Turing Machines – Returned today • Homework #7 (due Thursday) – Help after lecture And the languages they handle

Before We Start Languages

• Any questions? • Is that your final answer? – What is a language? – What is a class of languages?

Now our picture looks like

Context Free Languages • A Machine consists of: – A state machine Deterministic Context Free Languages – An input tape – A movable r/w tape head Regular Languages • A move of a Turning Machine

Finite – Read the character on the tape at the current position of Languages the tape head – Change the character on the tape at the current position of the tape head – Move the tape head We’ going to start to look at – Change the state of the machine based on current state languages out here and character read

1 Turing Machine Turing Machine

• Tape that holds • In the original Turing Machine Input tape (input/memory) ... character string – The read tape was infinite in both directions Read head • Movable tape head that – The text describes a semi-infinite TM where reads and writes • The tape is bounded on the left and infinite on the State character right Machine • It can be shown that the semi-infinite TM is • Machine that changes (program) equivalent to the basic TM. state based on what is – As not to confuse, I’ll follow the conventions in read in the book…(as much as that bothers me!)

Turing Machines Turing Machines

• About the input tape • About the machine states – Bounded to the left, infinite to the right – A Turing Machine does not have a of – All cells on the tape are originally filled with a accepting states special “blank” character ∆ – Instead, each TM has a special halting state, h. – Tape head is read/write • Once in the halting state, the machine halts. – Tape head can not move to the left of the start – Unlike PDAs, the basic TM is deterministic! of the tape • If it tries, the machine “crashes”

Turing Machines Turing Machines

• Let’s formalize this • Transition function: – A Turing Machine M is a 5-tuple: – δ: Q x (Γ∪{ ∆ }) → (Q ∪ {h}) x (Γ∪{ ∆ }) x {R, L, Σ Γ δ S} – M = (Q, , , q0, ) where • Q = a finite set of states (assumed not to contain the halting state h) – Input: • Σ = input alphabet (strings to be used as input) • Current state • Γ = tape alphabet (chars that can be written onto the tape. • Tape read at current position of tape head Includes symbols from Σ) – Output: • Both Σ and Γ are assumed not to contain the “blank” symbol • State in which to move the machine (can be the halting state) • δ = transition function • Tape symbol to write at the current position of the tape head (can be the “blank” symbol) • Direction in which to move the tape head (R = right, L = left, S= stationary)

2 Turing Machine Turing Machine

• Accepting a string • Running a Turing Machine – A string x is accepted by a TM, if – The execution of a TM can result in 4 possible • Starting in the initial configuration cases: • With x on the input tape • The machine “halts” (ACCEPT) • The machine eventually ends up in the halting state. • The machine has nowhere to go (REJECT) –I.e. • The machine “crashes” (REJECT) ∆ * • The machine goes into an “infinite loop” (REJECT •(q0, x) a (h, xay) – for x,y ∈(Γ∪{∆})*, a ∈(Γ∪{∆}) but keeps us guessing!)

Turing Machine Computation with Turing Machines

• Language accepted by a TM • Computing functions with TMs – The language accepted by a TM is the set of all – The result of the function applied to an input input strings x on which the machine halts. string x, will be left on the tape when the machine halts. – Functions with multiple arguments can be placed on the input tape with arguments separated by blanks. • Questions?

Computation with Turing Machines Computation with Turing Machines

• Computing functions with TMs • Computing functions with TMs – Formally, – Formally (with multiple arguments) Σ Γ δ Σ Γ δ • Let T = (Q, , , q0, ) be a TM and let f be a • Let T = (Q, , , q0, ) be a TM and let f be a partial function on Σ*. We say T computes f if for partial function on (Σ*) k. We say T computes f if ∈Σ* every x where f is defined: for every (x1, x2. …, xk) where f is defined: ∆ * ∆ ∆ ∆ ∆ * ∆ –(q0, x) a (h, f(x)) –(q0, x1 x2… xk) a (h, f(x1, x2. …, xk)) • And for every other x, T fails to halt on input x. • And for every other k-tuple, T fails to halt on input x.

3 Computation with Turing Machines TMs and languages

• Characteristic function of a set: • Two classes of languages that involve TMs: – For any language L, the characteristic function is defined as: – Languages, L accepted by TM •1 if x ∈ L • Halts on x ∈L, “rejects” on x ∉L • 0 otherwise – Languages, L recognized by a TM – If a TM computes the characteristic function of a • TM computes the characteristic function of L language, it will always halt with either a 1 or 0 left on • Halts on all x the tape • Give yes/no answer as to whether x ∈L • This is stronger than saying that x is accepted by the TM since if x ∉ L, for a TM that accepts L, there is no guarantee that the the machine will halt.

TMs and languages TMs and languages

• Two classes of languages that involve TMs: • First observation: – A language is recursively enumerable if there is – Every recursive language is also recursively a TM that accepts it enumerable – A language is recursive if there is a TM that • Modify the TM that recognizes the language so that recognizes it. it goes into a “nowhere state” just before placing the 0 on the tape indicating that a string is not in the language.

TMs and languages TMs and languages

• Game plan Is there something out 1. Show that there exists a language that is not Recursively here recursively enumerable Enumerable 2. Show that there exists a language that is recursively enumerable but not recursive. Recursive Is there something in here

4 A language that is not recursively enumerable

• Sorry, no pumping Lemma for TMs! • Universal Turing Machine

–A TM Tu that takes as input, an encoded version of another TM M’ and an input to M’. • Instead we introduce the notion of a – M will simulate the processing of M’

Universal Turning Machine • If M’ halts on input x, Tu will halt • If M’ rejects on input x,Tu will reject.

–Tu is a general purpose TM • M’ is a program run on the general purpose TM

Universal Turing Machine Universal Turing Machine

• Encoding scheme • Encoding scheme – We need a way to encode a TM so that it can be – Associate states, symbols, and directions with provided as input to T u binary numbers: – Define sets: • States: • = {q , q , … } = set of all possible states that may appear in Q 1 2 → any TM – s(h) 0 → i+1 – s(qi) 0 • S = {a1, a2, a3, …} = set of all possible tapes symbols that can be written on any TM. • Symbols –So for any TM –s(∆) → 0 ⊆ Γ⊆ → i+1 •Q Q and S – s(ai) 0

Universal Turing Machine Universal Turing Machine

• Encoding scheme • Encoding scheme – Associate states, symbols, and directions with – Encoding transitions: binary numbers: • Encode each state, symbol, direction • Directions: • Separate by 1’a – s(S) → 0 – The transition δ (p,a) = (q, b, D) can be → – s(L) 00 encoded as: – s(R) → 000 • s(p)1s(a)1s(q)1s(b)1s(D)

5 Universal Turing Machine Universal Turing Machine

• Encoding • The input to Tu will be – Encoding for a TM – an encoded TM and • Encode the start state – a string x to run on that TM • Encode each of it’s moves – We encode x by individually encoding the • Separate by 1’s characters of x (separated by 1’s) – e(T) = s(q)1e(m )1e(m )1…1e(m )1 1 2 k •x = z1z2…zn

• e(x) = s(z1)1s(z2)1…1s(zn)1

Universal Turing Machine Universal Turing Machine

• Encoding Scheme • Encoding example – Finally, the description of the TM and the – States •q– state 1 encoding of the string to run on the TM will be 0 • p – state 2 separated by 2 1’s • s – state 3 • Input to Tu – Symbols – e(M)11e(x) • a – symbol 1 • b – symbol 2

Universal Turing Machine Universal Turing Machine

• Encoding example • Encoding scheme – States • s (h) = 0 – The actual scheme isn’t really important (in •s(q) = 00 0 fact, there are many) • s(p) = 000 • s(r) = 0000 – What is important is that we can encode a TM – Symbols and it’s input as a binary string. •s(∆) = 0 • s(a) = 00 • s(b) = 000 001 00101000101 000 – Directions • s(S) → 0 •s(L) → 00 • s(R) → 000

6 Universal Turing Machine Universal Turing Machine

• For the mathematically inclined… • How it works – The set of all TMs is countably infinite –3 tape TM • We can define a one-to-one and onto function • Tape 1 – Input and output tape between the set of TMs and the set of natural • Tape 2 – Working tape numbers { 0, 1, 2, …} • Tape 3 – encoded form of the state that machine being simulated is in

– Note that a 3 tape TM is equivalent to a “basic” TM.

Universal Turing Machine Universal Turing Machine

• How it works • On a single move, • How it works ... – The symbol on each – When we start ∆e(T)11e(x) is on tape 1 tape is read – Step 1: Move e(x) from tape 1 to tape 2 and ... – The symbol on each tape is written delete from tape 1 ∆ ... – The tape head for each • Tape 2: 01e(x) Read head(s) tape is moved – Step 2: Copy encoded initial state of T to tape 3 independently State and delete from tape 1 Machine • Tape 3: s(q0) (program)

Universal Turing Machine Universal Turing Machine

• How it works • How will the machine finish? – Step 3: Now simulation begins – The TM being simulated has nowhere to go. •Twill never find a suitable transition • “read” and replace character on tape 2 u – The TM being simulated “crashes” • “read” state on tape 3 • Can arrange for Tu to crash • “find” a transition in the encoded machine on tape 1. – The TM being simulated goes into an infinite loop • Replace destination state onto tape 3 •Tu will continually simulate these moves • Replace character on tape 2. – The TM being simulated halts rd • Move head on tape 2 appropriately •Tu will halt with 0 on it’s 3 tape

•Tu will copy the contents of Tape 2 to Tape 1

7 Universal Turing Machine Back to our problem

• Reality check: What have we done? • Show that there exists a language that is not – Defined: recursively enumerable • A means to encode a TM as a binary string – The encoding provides each TM with a unique “serial” number. • A Universal TM (T ) that takes as input: u • That “serial number” can be considered the decimal equivalent – An encoded TM, M of the binary encoding of the TM – An encoded input string, x •Mi = the TM whose code is wi , i represented in binary – That will simulate the running of x on M •L(Mi) is the language accepted by Mi

•Note: if wi is not a valid TM encoding, we define Mi to be the ∅ TM with 1 state and no transitions. In this case L(Mi) = .

Back to our problem Diagonalization Language

• So far… • Diagonalization Language (Ld) ∉ – For i=0, 1, 2, … – Set of all strings wi such that wi L(Mi) •wi is the binary representation of i – All strings w that are not accepted by the TM

•Mi is the TM that has wi as it’s encoding with w as it’s encoding. •L(Mi) is the language accepted by Mi – All encodings for TMs that don’t accept their own encoding when input

Diagonalization Language Diagonalization Language

• Why is it called diagonalization? •Ld is the complement of the diagonal – Consider all pairs (i,j). • i represents TM “serial numbers” • j represents decimal representations of binary string • (i, j) represents if j is accepted by the TM with “serial number” i.

8 The punch line A language that is not recursively enumerable

•Ld is not recursively enumerable • Questions? – Proof by contradiction

• Assume Ld was recursively enumerable then there exists a TM, M that accepts Ld. Let M have “serial number” i, for some i.

–Ld = L (Mi)

• Now ask: is wi in Ld?

– If yes, then Mi will accept wi. This contradicts the definition of Ld. – If no, then Mi will not accept wi. In this case, by the definition ∈ of Ld, w Ld. Can’t both be in Ld and not in Ld. – Contradiction!

•Ld must not be recursively enumerable.

For the mathematically For the mathematically inclined… inclined… • A language that is not recursively enumerable • A language that is not recursively enumerable – It can be shown that the power set 2S of a set (I.e. the – Countably infinite sets are smaller than uncountably set of all of S) is infinitely uncountable. infinite sets. – More specifically, the power set of the set of strings – The set of TMs is countable composed of symbols from an alphabet Σ (what is a – The set of languages is uncountable. language again?) is infinitely uncountable. – There are just not enough TMs to go around! – This power set is merely the set of all languages over Σ. – There are more languages than TMs, thus there must be some languages that have no TMs that accept them.

RE but not recursive Complements of Recursive Languages

• 2. Show that there exists a language that is • If L is recursive, then L’, its complement is also recursively enumerable but not recursive. recursive – If L is recursive, there exists a TM, M, that, on a given input x • Always halts • Helps to show a nice fact about the • Will leave 1 on the tape if x ∈ L complements of recursive languages. • Will leave 0 on the tape if x ∉ L. – Build a new TM, M’ that simulates M except • Will leave 0 on the tape if x ∈ L • Will leave 1 on the tape if x ∉ L. – M’ accepts L’

9 Universal Language Universal Language

• Universal Language (Lu) •Is Lu recursively enumerable? ∈ ∈ – Is there a TM that will always halt for wi L(Mi) – Set of all strings wi such that wi L(Mi) – Yes! The Universal TM, T – All strings w that are accepted by the TM with u w as it’s encoding. – Running T on w 11w – All encodings for TMs that do accept their u i i • Will halt if wi is accepted by the TM with encoding wi encoding when input • Will reject otherwise.

–Lu is recursively enumerable.

•Is Lu recursively enumerable?

Universal Language TMs and languages

•Is Lu recursive? • Assume that it is: Is there something out – Then Lu’ is also recursive ∈ Recursively here –Lu = Set of all strings wi such that wi L(Mi) ∉ –Lu’ = Set of all strings wi such that wi L(Mi) Enumerable –= Ld – We just showed L not to be recursively enumerable. Recursive d Is there YES – Ld – Since all recursive languages are also recursively something in enumerable, then Ld is certainly not recursive. here – Contradiction! YES – Lu –Lu must not be recursive!

Summary But what about this?

• Recursive vs. Recursively Enumerable Context Free Languages • Universal TM Deterministic Context Free Languages • Language that is not RE Regular Languages – Diagonalization Language Finite • Language that is RE but not recursive Languages – Universal language.

How does this relate to Recursive/RE Languages?

10 But what about this?

• Save for next time.

11