Turing Machines COMP 455 – 002, Spring 2019
Jim Anderson (modified by Nathan Otterness) 1 Turing Machines
We want to study computable functions, or algorithms. In particular, we will look at algorithms for answering certain questions. ❖A question is decidable if and only if an algorithm exists to answer it. Example question: Is the complement of an arbitrary Such an algorithm CFL also a CFL? does exist for any ❖This question is undecidable—there is no algorithm specific CFL. that takes an arbitrary CFL and outputs “yes” if the complement is a CFL and “no” otherwise.
Jim Anderson (modified by Nathan Otterness) 2 Undecidability Example
We can give an informal “proof” showing that an undecidable problem exists. Let’s consider: The “Hello World” problem: Given a program 푃 and an input to that program 퐼, print “yes” if 푃 prints “hello world” when run with input 퐼 and “no” otherwise. ❖푃 can be any program—including one that is very convoluted!
Jim Anderson (modified by Nathan Otterness) 3 Undecidability Example Suppose a program, 퐻, exists that solves the “hello world” problem:
푃 “yes” 퐻 퐼 “no”
Now consider another program, 퐻1, which is the same as 퐻, but prints “hello world” instead of “no”: 푃 “yes” 퐻1 퐼 “hello world”
Jim Anderson (modified by Nathan Otterness) 4 Undecidability Example
푃 “yes” 퐻 퐼 1 “hello world”
Next, construct a program 퐻2 that is identical to 퐻1 but only takes a single input, 푃, that it uses as both 푃 and 퐼 in 퐻1: “yes” 푃 퐻 2 “hello world”
Another way to “yes” represent 퐻2: 푃 퐻 1 “hello world”
Jim Anderson (modified by Nathan Otterness) 5 Undecidability Example
Now consider what happens when we run 퐻2 with itself as input: “yes” 퐻 퐻 2 2 “hello world” We have reached a paradox:
Suppose 퐻2 prints “hello world” when fed 퐻2 as input. This means that 퐻2 must also print “yes” when fed 퐻2 as input!
Jim Anderson (modified by Nathan Otterness) 6 Turing Machines
To be able to rigorously do proofs like the “hello world” example, we need a formal model for defining computable functions. The Turing Machine has become the accepted model for formalizing functions.
Jim Anderson (modified by Nathan Otterness) 7 Turing Machines
A TM can be defined by a 7-tuple 푄, Σ, Γ, 훿, 푞0, 퐵, 퐹 ❖푄: A finite set of states. A TM’s input starts out on the ❖Γ: The tape alphabet. “tape”, so the tape alphabet ❖Σ: The input alphabet. Σ ⊂ Γ. must include the input alphabet. ❖퐵: The blank tape symbol. (퐵 ∈ Γ, and 퐵 ∉ Σ) ❖훿: The next move function: 푄 × Γ → 푄 × Γ × 퐿, 푅 .
❖푞0: The start state. 훿 returns a new state, 훿 takes a state a new tape symbol, ❖퐹: The set of final states. and tape symbol and whether to move left or right. Jim Anderson (modified by Nathan Otterness) 8 Turing Machines
Conceptually, a Turing Machine looks like this: All of the rest of the tape initially Non-blank tape symbols is filled with “blank” symbols. (input symbols, etc.)
… 퐵 퐵 푋1 … 푋푖 … 푋푛 퐵 퐵 … Tape head The “finite control” just refers to the part of the TM Finite Control that keeps track of the state. Initially, the input starts out on the tape, and the tape head starts at the leftmost input symbol. Jim Anderson (modified by Nathan Otterness) 9 Example: Turing Machine Accepting 0푛1푛 Input string: 0011 … 퐵 퐵 0 0 1 1 퐵 퐵 퐵 …
Finite Control State = 푞0
Initial configuration: In start state, at leftmost input symbol on the tape.
Jim Anderson (modified by Nathan Otterness) 10 Example: Turing Machine Accepting 0푛1푛
… 퐵 퐵 푋0 푋0 1푌 1푌 퐵 퐵 퐵 …
Finite Control State =푞0 푞120 푞120 푞3 푞4
A Turing Machine halts
when it can’t make any more moves.
Jim Anderson (modified by Nathan Otterness) 11 Example: Formal Notation
푄, Σ, Γ, 훿, 푞0, 퐵, 퐹 , where:
푄 = 푞0, 푞1, 푞2, 푞3, 푞4 ,
Σ = 0, 1 , Γ = 0, 1, 푋, 푌, 퐵 ,
퐹 = 푞4 , And 훿 is defined on the following slide…
Jim Anderson (modified by Nathan Otterness) 12
Example: Formal Notation
훿 is as follows:
❖ 훿 푞0, 0 = 푞1, 푋, 푅 : Change the leftmost 0 to X ❖ 훿 푞0, 푌 = 푞3, 푌, 푅 : There are no more 0s ❖ 훿 푞 , 0 = 푞 , 0, 푅 : Skip over 0s 1 1 “Seek” to the leftmost 1. ❖ 훿 푞1, 푌 = 푞1, 푌, 푅 : Skip over Ys ❖ 훿 푞1, 1 = 푞1, 푌, 퐿 : Change the leftmost 1 to Y ❖ 훿 푞2, 0 = 푞2, 0, 퐿 : Skip over 0s “Seek” to the ❖ 훿 푞 , 푌 = 푞 , 푌, 퐿 : Skip over Ys 2 2 leftmost 0. ❖ 훿 푞2, 푋 = 푞0, 푋, 푅 : Move right of the rightmost X ❖ 훿 푞3, 푌 = 푞3, 푌, 푅 : Make sure there are no more 1s ❖ 훿 푞3, 퐵 = 푞4, 퐵, 푅 : There were no more 1s, accept.
Jim Anderson (modified by Nathan Otterness) 13 Instantaneous Descriptions for TMs
As with a PDA, we can also give an instantaneous description (ID) for a Turing Machine. This indicates both the location of the An ID for a TM has the following form: 훼1푞훼2. tape head and the ❖푞 corresponds to both the state of the TM and current state (푞). the position of the tape head (푞 is written directly before the tape symbol the head is on).
❖훼1훼2 = the tape’s current contents, and only We can’t write an contains the non-blank portion, except in cases infinite number of cells like 훼퐵퐵퐵푞 or 푞퐵퐵퐵훼. without either infinite input or an infinite The depicted portion of the tape is always finite. number of moves. Jim Anderson (modified by Nathan Otterness) 14 Notation for TM Moves
Left moves. Suppose 훿 푞, 푋푖 = 푝, 푌, 퐿 . Then, ❖푋1 … 푋푖−1푞푋푖 … 푋푛├ 푋1 … 푋푖−2푝푋푖−1푌 … 푋푛. 푀 ❑The tape head moved from 푋푖 to 푋푖−1. ❑The state changed from 푞 to 푝. ❑푋푖 was replaced by 푌. We moved left past the start of ❖Special case where 푖 = 1: the tape content, so we need to show the extra blank symbol (퐵). ❑푞푋1푋2 … 푋푛├ 푝퐵푌푋2 … 푋푛 푀 We replaced the rightmost tape ❖Special case where 푖 = 푛 and 푌 = 퐵: symbol (푋푛) with a 퐵, so we no ❑푋1 … 푋푛−1푞푋푛├ 푋1 … 푋푛−2푝푋푛−1 longer include it in the ID. 푀
Jim Anderson (modified by Nathan Otterness) 15 Notation for TM Moves
Right moves. Suppose 훿 푞, 푋푖 = 푝, 푌, 푅 . Then, ❖푋1 … 푋푖−1푞푋푖 … 푋푛├ 푋1 … 푋푖−1푌푝푋푖+1 … 푋푛. 푀 ❑The tape head moved from 푋푖 to 푋푖+1. ❑The state changed from 푞 to 푝. ❑푋푖 was replaced by 푌. The 퐵 is included here just ❖Special case where 푖 = 푛: to make it clear what the tape head is pointing at. ❑푋1 … 푋푛+1푞푋푛├ 푋1 … 푋푛−1푌푝퐵. 푀 ❖Special case where 푖 = 1 and 푌 = 퐵:
❑푞푋1 … 푋푛├ 푝푋2 … 푋푛. 푀
Jim Anderson (modified by Nathan Otterness) 16 The Language Defined by a TM
The language accepted by a Turing Machine 푀 is defined as: ∗ ∗ 퐿 푀 ≡ {푤 | 푤 ∈ Σ and 푞0푤 ├ 훼1푝훼2 for some 푝 ∈ 퐹, and 푀 ∗ 훼1, 훼2 ∈ Γ }. Assumption: If 푀 accepts 푤, 푀 halts. Notation used for moves (similar to a PDA): 푖 ∗ 푖 ∗ ❖ ├, ├, ├, ├, ├. 푀 푀 ❑The 푖 indicates “exactly 푖 moves” ❑The ∗ indicates “any number of moves” ❑The (optional) 푀 below the ├ indicates that TM 푀 made the moves.
Jim Anderson (modified by Nathan Otterness) 17 Recursively Enumerable Languages
A language that is accepted by a Turing Machine is called recursively enumerable (RE). If a string 푤 is in 퐿 푀 , then 푀 eventually halts and accepts 푤. If 푤 ∉ 퐿 푀 , then one of two things can happen: ❖푀 halts without accepting 푤, or ❖푀 never halts.
Jim Anderson (modified by Nathan Otterness) 18 Recursive Languages
A language that is accepted by a TM that halts on all inputs is called recursive. This is the accepted formal definition of “algorithm”. ❖For a recursive language, we can always algorithmically determine membership—but we can’t necessarily do this for RE languages. A problem that is solved by a TM that always halts is called decidable. ❖We will discuss decidable problems in much more detail later!
Jim Anderson (modified by Nathan Otterness) 19 TM Construction Technique: Extra Storage
Storage in the finite control: You can keep track of a finite amount of extra data by incorporating it into the state of a TM. Example: Keep track of an additional symbol: … … … … … If the “extra data” can be 퐴 or 퐵, and the “state” can be 푞0 or 푞1, then the actual Finite Control states of the TM can be State = 푞0 Extra data = 퐴 {[푞0, 퐴], [푞1, 퐴], [푞0, 퐵], [푞1, 퐵]}.
Jim Anderson (modified by Nathan Otterness) 20 Storage in the Finite Control: Example
Example: Scan the input and ensure the first symbol doesn’t appear a second time.
푀 = 푄, 0, 1 , 0, 1, 퐵 , 훿, 푞0, 퐵 , 퐵, 퐹
푄 = 푞0, 푞1 × 0, 1, 퐵
퐹 = 푞1, 퐵
► 훿([푞0, 퐵], 0) = ([푞1, 0], 0, 푅) ► 훿([푞0, 퐵], 1) = ([푞1, 1], 1, 푅) ► 훿([푞1, 0], 1) = ([푞1, 0], 1, 푅) ► 훿([푞1, 1], 0) = ([푞1, 1], 0, 푅) ► 훿([푞1, 0], 퐵) = ([푞1, 퐵], 0, 퐿) ► 훿([푞1, 1], 퐵) = ( 푞1, 퐵 , 0, 퐿)
Note: This doesn’t change the fact that the TM has a finite number of states!
Jim Anderson (modified by Nathan Otterness) 21 Storage in the Finite Control: Example
훿 푞0, 퐵 , 0 = 푞1, 0 , 0, 푅 훿 푞 , 0 , 1 = 푞 , 0 , 1, 푅 If the first symbol 1 1 훿 푞 , 0 , 퐵 = 푞 , 퐵 , 0, 퐿 was a 0, “store” a 0. 1 1 훿 푞0, 퐵 , 1 = 푞1, 1 , 1, 푅 훿 푞1, 1 , 0 = 푞1, 1 , 0, 푅 훿 푞1, 1 , 퐵 = 푞1, 퐵 , 0, 퐿
If the first symbol was a 1, “store” a 1.
This construction may result in unreachable states.
Jim Anderson (modified by Nathan Otterness) 22 TM Construction Technique: Multiple Tracks
… 퐵 ¢ 1 0 1 1 $ 퐵 퐵 … … 퐵 퐵 퐵 퐵 0 1 퐵 퐵 퐵 … … 퐵 퐵 0 0 0 0 0 퐵 퐵 …
Finite Control In the above figure, the “real” input is between the ¢ and $ on the first track; the other tracks are used for scratch storage. Tape symbols are tuples, i.e., ¢, 퐵, 퐵 , 1, 퐵, 0 , $, 퐵, 0 , etc. This doesn’t change the fact that there are a finite number of possible tape symbols!
Jim Anderson (modified by Nathan Otterness) 23 Multiple Tracks: Example
We will use multiple tracks and storage in the finite control to write a TM that accepts the ∗ language 퐿 = 푤푐푤 | 푤 ∈ 퐚 + 퐛 . Any string 푤, followed by a “c”, This TM will work by “checking off” followed by a second copy of 푤. corresponding symbols in each copy of 푤.
Jim Anderson (modified by Nathan Otterness) 24 Multiple Tracks: Example
Defining a TM accepting 퐿 = 푤푐푤 | 푤 ∈ 퐚 + 퐛 ∗ , using two tracks and storing an extra symbol in the finite control.
푀 = 푄, Σ, Γ, 훿, 푞0, 퐵, 퐹
푄 = 푞, 푑 | 푞 ∈ 푞1, … , 푞9 , 푑 ∈ 푎, 푏, 퐵 Σ = 퐵, 푑 | 푑 ∈ 푎, 푏, 푐 Γ = 푋, 푑 | 푋 ∈ 퐵,∗ , 푑 ∈ 푎, 푏, 푐, 퐵
푞0 = 푞1, 퐵
퐹 = 푞9, 퐵 퐵 = 퐵, 퐵
Jim Anderson (modified by Nathan Otterness) 25 Multiple Tracks: Example
Defining a TM accepting 퐿 = 푤푐푤 | 푤 ∈ 퐚 + 퐛 ∗ , using two tracks and storing an extra symbol in the finite control. This TM stores one additional 푀 = 푄, Σ, Γ, 훿, 푞0, 퐵, 퐹 symbol, 푑, in its finite control, along with the “state”. 푄 = 푞, 푑 | 푞 ∈ 푞1, … , 푞9 , 푑 ∈ 푎, 푏, 퐵 Σ = 퐵, 푑 | 푑 ∈ 푎, 푏, 푐 Γ = 푋, 푑 | 푋 ∈ 퐵,∗ , 푑 ∈ 푎, 푏, 푐, 퐵 The additional symbol that this TM stores in the finite 푞 = 푞 , 퐵 0 1 control can be 푎, 푏, or 퐵. This TM has 9 퐹 = 푞 , 퐵 9 logical “states” 퐵 = 퐵, 퐵
Jim Anderson (modified by Nathan Otterness) 26 Multiple Tracks: Example
Defining a TM accepting 퐿 = 푤푐푤 | 푤 ∈ 퐚 + 퐛 ∗ , using two tracks and storing an extra symbol in the finite control.
푀 = 푄, Σ, Γ, 훿, 푞0, 퐵, 퐹 The actual input string we care about 푄 = 푞, 푑 | 푞 ∈ 푞1, … , 푞9 , 푑 ∈ 푎, 푏, 퐵 consists of 푎s, 푏s, or 푐s. For example, 퐵, 푎 Σ = 퐵, 푑 | 푑 ∈ 푎, 푏, 푐 the symbol corresponds to a symbol 푎 in an input string. Γ = 푋, 푑 | 푋 ∈ 퐵,∗ , 푑 ∈ 푎, 푏, 푐, 퐵
푞0 = 푞1, 퐵 The tape has two tracks, so the input “symbols” simply 퐹 = 푞9, 퐵 initialize corresponding cells in 퐵 = 퐵, 퐵 one of the tracks as blank. Jim Anderson (modified by Nathan Otterness) 27 Multiple Tracks: Example
Defining a TM accepting 퐿 = 푤푐푤 | 푤 ∈ 퐚 + 퐛 ∗ , using two tracks and storing an extra symbol in the finite control.
푀 = 푄, Σ, Γ, 훿, 푞0, 퐵, 퐹
푄 = 푞, 푑 | 푞 ∈ 푞1, … , 푞9 , 푑 ∈ 푎, 푏, 퐵 Σ = 퐵, 푑 | 푑 ∈ 푎, 푏, 푐 The second track contains either input symbols or 퐵. Γ = 푋, 푑 | 푋 ∈ 퐵,∗ , 푑 ∈ 푎, 푏, 푐, 퐵
푞0 = 푞1, 퐵 The first track in the tape contains a ∗ to 퐹 = 푞9, 퐵 indicate if the input symbol in the 퐵 = 퐵, 퐵 second track has been “checked off”. Jim Anderson (modified by Nathan Otterness) 28 Multiple Tracks: Example To avoid having to write several similar rules, let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 . 훿 =
► 훿 푞1, 퐵 , 퐵, 푑 = 푞2, 푑 , ∗, 푑 , 푅 ► 훿 푞5, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 ► 훿 푞5, 퐵 , ∗, 푑 = 푞7, 퐵 , ∗, 푑 , 푅 ► 훿([푞2, 푑], 퐵, 푒 ) = ([푞2, 푑], 퐵, 푒 , 푅) ► 훿([푞2, 푑], [퐵, 푐]) = ([푞3, 푑], [퐵, 푐], 푅) ► 훿 푞6, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 ► 훿 푞6, 퐵 , ∗, 푑 = 푞1, 퐵 , ∗, 푑 , 푅 ► 훿 푞3, 푑 , ∗, 푒 = 푞3, 푑 , ∗, 푒 , 푅 ► 훿 푞3, 푑 , 퐵, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿 ► 훿 푞7, 퐵 , 퐵, 푐 = 푞8, 퐵 , 퐵, 푐 , 푅
► 훿 푞4, 퐵 , ∗, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿 ► 훿 푞8, 퐵 , ∗, 푑 = 푞8, 퐵 , ∗, 푑 , 푅 ► 훿 푞4, 퐵 , 퐵, 푐 = 푞5, 퐵 , 퐵, 푐 , 퐿 ► 훿 푞8, 퐵 , 퐵, 퐵 = 푞9, 퐵 , 퐵, 퐵 , 퐿
Jim Anderson (modified by Nathan Otterness) 29 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
► 훿 푞1, 퐵 , 퐵, 푑 = 푞2, 푑 , ∗, 푑 , 푅 ► Start off by storing the symbol you’re looking at in the “finite control” storage, checking off the symbol, and We’re looking at Store 푑 into the starting to move right. input symbol 푑 in finite-control ❖ The goal will be to make sure the the second track. storage. first “unchecked” symbol in the right-hand string matches the symbol stored in the finite control.
Jim Anderson (modified by Nathan Otterness) 30 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
► 훿 푞1, 퐵 , 퐵, 푑 = 푞2, 푑 , ∗, 푑 , 푅 ► Start off by storing the symbol you’re looking at in the “finite control” storage, checking off the symbol, and Previously, this starting to move right. symbol was not So we’ll ❖ The goal will be to make sure the checked off… check it off. first “unchecked” symbol in the right-hand string matches the symbol stored in the finite control.
Jim Anderson (modified by Nathan Otterness) 31 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
The point of state 푞2 is just to move right until we ► 훿 푞 , 퐵 , 퐵, 푑 = 푞 , 푑 , ∗, 푑 , 푅 1 2 reach the right-hand string.
► 훿([푞2, 푑], 퐵, 푒 ) = ([푞2, 푑], 퐵, 푒 , 푅) ► 훿([푞2, 푑], [퐵, 푐]) = ([푞3, 푑], [퐵, 푐], 푅) We know we’ve reached the right-hand string after ► 훿 푞3, 푑 , ∗, 푒 = 푞3, 푑 , ∗, 푒 , 푅 passing the input symbol 푐. ► 훿 푞3, 푑 , 퐵, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿
► 훿 푞4, 퐵 , ∗, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿 ► 훿 푞4, 퐵 , 퐵, 푐 = 푞5, 퐵 , 퐵, 푐 , 퐿
Jim Anderson (modified by Nathan Otterness) 32 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
► 훿 푞1, 퐵 , 퐵, 푑 = 푞2, 푑 , ∗, 푑 , 푅 The point of state 푞3 is to move right until we reach the first ► 훿([푞 , 푑], 퐵, 푒 ) = ([푞 , 푑], 퐵, 푒 , 푅) 2 2 unchecked symbol in the right- ► 훿([푞2, 푑], [퐵, 푐]) = ([푞3, 푑], [퐵, 푐], 푅) hand string.
► 훿 푞3, 푑 , ∗, 푒 = 푞3, 푑 , ∗, 푒 , 푅 ► 훿 푞3, 푑 , 퐵, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿 Once this symbol is reached, check it off. (The TM will die here ► 훿 푞4, 퐵 , ∗, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿 if the symbol doesn’t match the ► 훿 푞 , 퐵 , 퐵, 푐 = 푞 , 퐵 , 퐵, 푐 , 퐿 4 5 one in the finite-control storage.)
Jim Anderson (modified by Nathan Otterness) 33 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
► 훿 푞1, 퐵 , 퐵, 푑 = 푞2, 푑 , ∗, 푑 , 푅
► 훿([푞2, 푑], 퐵, 푒 ) = ([푞2, 푑], 퐵, 푒 , 푅) ► 훿([푞2, 푑], [퐵, 푐]) = ([푞3, 푑], [퐵, 푐], 푅) The point of state 푞 is to ► 훿 푞 , 푑 , ∗, 푒 = 푞 , 푑 , ∗, 푒 , 푅 4 3 3 move left until we reach ► 훿 푞3, 푑 , 퐵, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿 the left-hand string again.
► 훿 푞4, 퐵 , ∗, 푑 = 푞4, 퐵 , ∗, 푑 , 퐿 ► 훿 푞 , 퐵 , 퐵, 푐 = 푞 , 퐵 , 퐵, 푐 , 퐿 4 5 Enter state 푞5 after passing the symbol 푐.
Jim Anderson (modified by Nathan Otterness) 34 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
In 푞5, we are looking at the last symbol in the left-hand ► 훿 푞5, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 string. If it’s not checked off, ► 훿 푞5, 퐵 , ∗, 푑 = 푞7, 퐵 , ∗, 푑 , 푅 go to state 푞6. ► 훿 푞6, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 If the last symbol in the left- ► 훿 푞6, 퐵 , ∗, 푑 = 푞1, 퐵 , ∗, 푑 , 푅 hand string was checked off already, go to 푞7 instead. ► 훿 푞7, 퐵 , 퐵, 푐 = 푞8, 퐵 , 퐵, 푐 , 푅
► 훿 푞8, 퐵 , ∗, 푑 = 푞8, 퐵 , ∗, 푑 , 푅 ► 훿 푞8, 퐵 , 퐵, 퐵 = 푞9, 퐵 , 퐵, 퐵 , 퐿
Jim Anderson (modified by Nathan Otterness) 35 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
In 푞6, keep moving left until we find the leftmost ► 훿 푞5, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 checked-off symbol in the ► 훿 푞5, 퐵 , ∗, 푑 = 푞7, 퐵 , ∗, 푑 , 푅 left-hand string. ► 훿 푞6, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 Move right from this ► 훿 푞6, 퐵 , ∗, 푑 = 푞1, 퐵 , ∗, 푑 , 푅 symbol to get to the next unchecked symbol in the ► 훿 푞7, 퐵 , 퐵, 푐 = 푞8, 퐵 , 퐵, 푐 , 푅 left-hand string, and repeat ► 훿 푞 , 퐵 , ∗, 푑 = 푞 , 퐵 , ∗, 푑 , 푅 the process from state 푞1. 8 8 ► 훿 푞8, 퐵 , 퐵, 퐵 = 푞9, 퐵 , 퐵, 퐵 , 퐿
Jim Anderson (modified by Nathan Otterness) 36 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
► 훿 푞5, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 ► 훿 푞5, 퐵 , ∗, 푑 = 푞7, 퐵 , ∗, 푑 , 푅 In state 푞7, everything in the left-hand string has ► 훿 푞6, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 been checked off, so we’re ► 훿 푞6, 퐵 , ∗, 푑 = 푞1, 퐵 , ∗, 푑 , 푅 getting ready to make sure everything in the right- ► 훿 푞7, 퐵 , 퐵, 푐 = 푞8, 퐵 , 퐵, 푐 , 푅 hand string is checked off, too. ► 훿 푞8, 퐵 , ∗, 푑 = 푞8, 퐵 , ∗, 푑 , 푅 ► 훿 푞 , 퐵 , 퐵, 퐵 = 푞 , 퐵 , 퐵, 퐵 , 퐿 State 푞7’s main purpose is 8 9 to move right past the 푐.
Jim Anderson (modified by Nathan Otterness) 37 Multiple Tracks: Example (Let 푑 ∈ 푎, 푏 and 푒 ∈ 푎, 푏 .)
► 훿 푞5, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 ► 훿 푞5, 퐵 , ∗, 푑 = 푞7, 퐵 , ∗, 푑 , 푅 In state 푞8, move right while checking that everything in ► 훿 푞6, 퐵 , 퐵, 푑 = 푞6, 퐵 , 퐵, 푑 , 퐿 the right-hand string is ► 훿 푞6, 퐵 , ∗, 푑 = 푞1, 퐵 , ∗, 푑 , 푅 checked off (if any input
symbol is not checked off, ► 훿 푞7, 퐵 , 퐵, 푐 = 푞8, 퐵 , 퐵, 푐 , 푅 execution will die). ► 훿 푞8, 퐵 , ∗, 푑 = 푞8, 퐵 , ∗, 푑 , 푅 If we made it past all the input ► 훿 푞8, 퐵 , 퐵, 퐵 = 푞9, 퐵 , 퐵, 퐵 , 퐿 symbols without dying, go to Side note: we already accept here, so it state 푞9 (which is accepting). doesn’t matter whether we move L or R now. Jim Anderson (modified by Nathan Otterness) 38 Multiple Tracks: Example
Here’s the TM as a
transition diagram:
Jim Anderson (modified by Nathan Otterness) 39 TM Technique: Shifting Symbols Over
You may need to create extra space in the middle of a string of symbols in a TM. This can be accomplished by holding a small buffer of symbols in the finite control. Example: Shifting a string of nonblank symbols over by two spaces.
❖States will be of the form 푞, 퐴1, 퐴2 , where 푞 is 푞1 or 푞2 and 퐴1 and 퐴2 are in Γ.
Jim Anderson (modified by Nathan Otterness) 40 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols over by two spaces:
► 훿 푞1, 퐵, 퐵 , 퐴1 = 푞1, 퐵, 퐴1 , 푋, 푅 , 퐴1 ∈ Γ − 퐵, 푋 ► 훿 푞1, 퐵, 퐴1 , 퐴2 = 푞1, 퐴1, 퐴2 , 푋, 푅 , 퐴1 and 퐴2 ∈ Γ − 퐵, 푋
► 훿 푞1, 퐴1, 퐴2 , 퐴3 = 푞1, 퐴2, 퐴3 , 퐴1, 푅 , 퐴1, 퐴2, and 퐴3 ∈ Γ − 퐵, 푋
► 훿 푞1, 퐴1, 퐴2 , 퐵 = 푞1, 퐴2, 퐵 , 퐴1, 푅 , 퐴1 and 퐴2 ∈ Γ − 퐵, 푋 ► 훿 푞1, 퐴1, 퐵 , 퐵 = 푞2, 퐵, 퐵 , 퐴1, 퐿 , 퐴1 ∈ Γ − 퐵, 푋
► 훿 푞2, 퐵, 퐵 , 퐴 = 푞2, 퐵, 퐵 , 퐴, 퐿 , 퐴 ∈ Γ − 퐵, 푋
Jim Anderson (modified by Nathan Otterness) 41 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols over by two spaces:
► 훿 푞1, 퐵, 퐵 , 퐴1 = 푞1, 퐵, 퐴1 , 푋, 푅 , 퐴1 ∈ Γ − 퐵, 푋 ► 훿 푞1, 퐵, 퐴1 , 퐴2 = 푞1, 퐴1, 퐴2 , 푋, 푅 , 퐴1 and 퐴2 ∈ Γ − 퐵, 푋
Start by “shifting” the tape 퐴1 and 퐴2 can’t be 퐵. For simplicity, symbols 퐴1 and 퐴2 into the also assume that they can’t be 푋. finite-control buffer.
We will write the symbol 푋 into the “extra space” we are adding.
Jim Anderson (modified by Nathan Otterness) 42 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols over by two spaces:
► 훿 푞1, 퐵, 퐵 , 퐴1 = 푞1, 퐵, 퐴1 , 푋, 푅 , 퐴1 ∈ Γ − 퐵, 푋 ► 훿 푞1, 퐵, 퐴1 , 퐴2 = 푞1, 퐴1, 퐴2 , 푋, 푅 , 퐴1 and 퐴2 ∈ Γ − 퐵, 푋
► 훿 푞1, 퐴1, 퐴2 , 퐴3 = 푞1, 퐴2, 퐴3 , 퐴1, 푅 , 퐴1, 퐴2, and 퐴3 ∈ Γ − 퐵, 푋
Once the buffer is full, shift subsequent new symbols into the buffer while replacing them with the oldest symbol in the buffer.
Jim Anderson (modified by Nathan Otterness) 43 TM Technique: Shifting Symbols Over
Here are the relevant moves for shifting symbols over by two spaces:
► 훿 푞1, 퐵, 퐵 , 퐴1 = 푞1, 퐵, 퐴1 , 푋, 푅 , 퐴1 ∈ Γ − 퐵, 푋 ► 훿 푞1, 퐵, 퐴1 , 퐴2 = 푞1, 퐴1, 퐴2 , 푋, 푅 , 퐴1 and 퐴2 ∈ Γ − 퐵, 푋
► 훿 푞1, 퐴1, 퐴2 , 퐴3 = 푞1, 퐴2, 퐴3 , 퐴1, 푅 , 퐴1, 퐴2, and 퐴3 ∈ Γ − 퐵, 푋
► 훿 푞1, 퐴1, 퐴2 , 퐵 = 푞1, 퐴2, 퐵 , 퐴1, 푅 , 퐴1 and 퐴2 ∈ Γ − 퐵, 푋 ► 훿 푞1, 퐴1, 퐵 , 퐵 = 푞2, 퐵, 퐵 , 퐴1, 퐿 , 퐴1 ∈ Γ − 퐵, 푋 Finally, after you encounter blank ► 훿 푞2, 퐵, 퐵 , 퐴 = 푞2, 퐵, 퐵 , 퐴, 퐿 , 퐴 ∈ Γ − 퐵, 푋 symbols, shift the remaining After shifting everything, state 푞 causes the 2 symbols out of the buffer. TM to move left to the newly added space. Jim Anderson (modified by Nathan Otterness) 44 TM Technique: Shifting Symbols Over
… 퐵 퐵 0 0푋 1푋 1 퐵 퐵 퐵 퐵 …
퐴1, 퐴2, 퐴3, 퐴 ∉ 푋, 퐵 : Finite Control 훿 푞 , 퐵, 퐵 , 퐴 = 푞 , 퐵, 퐴 , 푋, 푅 푞 1 1 1 1 State = 12 훿 푞1, 퐵, 퐴1 , 퐴2 = 푞1, 퐴1, 퐴2 , 푋, 푅 Buffer = 퐵 , 퐵 훿 푞1, 퐴1, 퐴2 , 퐴3 = 푞1, 퐴2, 퐴3 , 퐴1, 푅 훿 푞1, 퐴1, 퐴2 , 퐵 = 푞1, 퐴2, 퐵 , 퐴1, 푅 훿 푞1, 퐴1, 퐵 , 퐵 = 푞2, 퐵, 퐵 , 퐴1, 퐿 Done! 훿 푞2, 퐵, 퐵 , 퐴 = 푞2, 퐵, 퐵 , 퐴, 퐿
Jim Anderson (modified by Nathan Otterness) 45 TM Technique: Subroutines
TMs can simulate subroutines, even including parameter-passing and recursion. Example: Multiplication using a “copy” subroutine ❖Given 0푚10푛1 as input, produce 0푚푛 as output. ❖This can be done by “copying” 푛 0s 푚 times.
Jim Anderson (modified by Nathan Otterness) 46 TM Technique: Subroutines
Here’s how the overall TM will work: 1. While processing, the tape will contain a string of the form 0푖10푛0푘푛 for some 푘. 2. In one “iteration”, we will change a 0 in the first group of 푖 0s to 퐵 and append 푛 0s to the last group of 0s. ❖ This will require copying the group of 푛 0s to the end of the string. 3. Eventually, there will be no more 0s at the start of the string, and the TM can delete the 10푛1, leaving only 0푛푚 on the tape.
Jim Anderson (modified by Nathan Otterness) 47 TM Technique: Subroutines
The key component of our “multiplication” TM is this subroutine, copy, defined in the following diagram:
Jim Anderson (modified by Nathan Otterness) 48 TM Technique: Subroutines
We can “plug in” the copy subroutine where it is needed in a larger TM:
The “copy” subroutine goes here.
Jim Anderson (modified by Nathan Otterness) 49 TM Technique: Subroutines
3 2 4 0 1q10 10 3 ├ 0 1Xq2010000 … Example move ├ 031X0q 10000 3 sequence for the 2 ├ 0 1XX100000q2 ├ 031201q 0000 3 copy routine for 2 ├ 0 1XX10000q300 an input where … … 푚 = 3, 푛 = 2, 3 3 ├ 0 1X010000q2 ├ 0 1Xq3X1000000 and 푖 = 4: 3 3 ├ 0 1X01000q300 ├ 0 1XXq11000000 3 3 ├ 0 1X0100q3000 ├ 0 1Xq4X1000000 3 … ├ 0 1q4X01000000 3 3 ├ 0 1q3X0100000 ├ 0 q41001000000 3 3 ├ 0 1Xq10100000 ├ 0 1q5001000000
Jim Anderson (modified by Nathan Otterness) 50 TM Extensions: Multiple Tapes A multi-tape TM can be conceptualized like this: … … … … … … … … …
푘 tapes … … … … … … … … …
… … … … … … … … …
Each tape is processed by Finite Control an independent tape head.
Jim Anderson (modified by Nathan Otterness) 51 TM Extensions: Multiple Tapes
In a multi-tape TM, each move depends on the state of the finite control and 푘 “current” tape symbols. During a move, a multi-tape TM can: 1. Change state. 2. Print a new symbol on each cell scanned by a tape head. 3. Move each tape head left, right, or stationary. We assume the input is initially on the first tape.
Jim Anderson (modified by Nathan Otterness) 52 Single-Tape vs. Multi-Tape TMs
Theorem 8.9: If 퐿 is accepted by a multi-tape TM, then it is accepted by a single-tape TM. Proof: Let 푀 be a multi-tape TM with 푘 tapes. We can construct a single-tape TM, 푀′ with 2푘 tracks that accepts 퐿 푀 .
Jim Anderson (modified by Nathan Otterness) 53 Single-Tape vs. Multi-Tape TMs
푀′’s (single) multi-track tape looks like this: Tape 1 of 푴 X11 X12 X1i Head for tape 1 B B # Tape 2 of 푴 X21 X22 X2j We use the # in these Head for tape 2 B B # tracks to indicate where the tape heads are. Tape 풌 of 푴 Xk1 Xk2 Xkn Head for tape 풌 B B #
The state of 푀′ includes: ►The state of 푀. ►The number of head markers to the right of the tape head. ►The tape symbols at each head marker. ►The scan direction.
Jim Anderson (modified by Nathan Otterness) 54 Single-Tape vs. Multi-Tape TMs
Tape 1 of 푴 X11 X12 X1i Head for tape 1 B B # Tape 2 of 푴 X21 X22 X2j Head for tape 2 B B # Keeping the number of head markers to the left of 푀′’s Tape 풌 of 푴 Xk1 Xk2 Xkn head makes it possible to Head for tape 풌 B B # know when all of the head markers have been visited. For 푀′ to simulate a move of 푀: 1. Starting at the leftmost head marker, move right and store the symbol at each head marker. 2. Determine the move 푀 would make. 3. Move left and update the tape symbols and head markers. 4. Change the state as indicated by the move that 푀 would make.
Jim Anderson (modified by Nathan Otterness) 55 Single-Tape vs. Multi-Tape TMs
Tape 1 of 푴 X11 X12 X1i Head for tape 1 B B # Tape 2 of 푴 X21 X22 X2j Proof (abridged): Head for tape 2 B B # After 푛 moves in 푀, the head markers in 푀′ can be at most 2푛 cells apart. This means that Tape 풌 of 푴 X X X k1 k2 kn simulating a single move in 푀 Head for tape 풌 B # B requires 푂 푛 moves in 푀′.
▶ Theorem 8.10: 푀′ takes 푂 푛2 moves to simulate 푛 moves of 푀.
▶ Since a multi-tape TM is just as powerful as a single- tape TM, we can use whichever is more convenient. Jim Anderson (modified by Nathan Otterness) 56 TM Extensions: Nondeterministic TMs
The original definition we discussed was for a deterministic TM (DTM). A nondeterministic TM (NDTM) can “choose” between multiple possible moves for any combination of state and tape symbol(s). Move function for a nondeterministic TM with 푘 tapes: ❖훿 maps 푄 × Γ푘 to subsets of 푄 × Γ × 퐿, 푅, 푆 푘.
Jim Anderson (modified by Nathan Otterness) 57 TM Extensions: Nondeterministic TMs
Example move function for a nondeterministic single-tape TM, where Γ = 0, 1, 퐵 :
훿 푞0, 0 = 푞1, 퐵, 푅 , 푞2, 1, 퐿
Jim Anderson (modified by Nathan Otterness) 58 Nondeterministic TMs
Note: The book reasons Theorem 8.11: If 퐿 is accepted by a single-tape about a multi-tape NDTM here, instead. NDTM 푀1, then 퐿 is accepted by some DTM 푀2. Let 푑 be the maximum number of nondeterministic choices 푀1 can make at any given move.
푀2 will systematically try all nondeterministic possibilities.
푀2 will have three tapes.
Jim Anderson (modified by Nathan Otterness) 59 Nondeterministic TMs
How 푀2 works: Tape 1 holds the input. Tape 2 contains a sequence of digits from 1 to 푑, generated systematically. Each sequence dictates a sequence of choices. e.g., ❖ 1 , 2 , 3 , … , 푑 , 1,1 , 1, 2 , 2, 1 , … , 푑, 푑 , 1,1,1 , 1,1,2 , 1, 2, 1 , … , 푑, 푑, 푑 , … Tape 3 contains a scratch copy of the input.
Jim Anderson (modified by Nathan Otterness) 60 Nondeterministic TMs
How 푀2 works (continued): 1. Generate the next sequence on tape 2 2. Copy tape 1 (input) to tape 3 (scratch copy)
3. Simulate 푀1 on tape 3, making choices according to the sequence on tape 2.
4. If 푀1 accepts, then accept; 5. Otherwise, go to step 1.
Jim Anderson (modified by Nathan Otterness) 61 Nondeterminism and Time Complexity
Definition: A NDTM 푀 is of time complexity 푇 푛 if for every accepted string of length 푛, some sequence of at most 푇 푛 moves leading to an accepting state exists.
Theorem: If 퐿 is accepted by a single-tape NDTM 푀1 with time complexity 푇 푛 , then 퐿 is accepted by some 푇 푛 DTM 푀2 with time complexity 푂 푐 , for some constant 푐.
Proof: If 푀1 in the previous construction can accept in at most 푇 푛 moves, then 푀2 can accept in at most 푂 푇 푛 푑 + 1 푇 푛 moves.
Jim Anderson (modified by Nathan Otterness) 62 Restricted TMs: Semi-infinite Tape
In a TM with a semi-infinite tape, the tape is infinite only to the right of the starting position. ❖Assume that trying to move left from the leftmost cell causes execution to die. Theorem 8.12: (reworded) 퐿 is recognized by a TM with a two-way infinite tape if and only if it is recognized by a TM with a semi-infinite tape.
Jim Anderson (modified by Nathan Otterness) 63 Restricted TMs: Semi-infinite Tape
Theorem 8.12: 퐿 is recognized by a TM with a two-way infinite Proof of Theorem 8.12: tape if and only if it is recognized “If”: by a TM with a semi-infinite tape. It’s pretty straightforward to simulate a semi-infinite tape TM using one with a two-way infinite tape. ❖Mark the tape cell to the left of the starting position with a special symbol, and halt without accepting if that symbol is ever encountered.
Jim Anderson (modified by Nathan Otterness) 64 Restricted TMs: Semi-infinite Tape
Proof (continued), “Only if”:
Let 푀2 = 푄2, Σ2, Γ2, 훿2, 푞2, 퐵, 퐹2 be a two-way TM.
Construct a one-way TM 푀1 to simulate 푀2.
The tape of 푀1 will have two tracks.
Tape for 푀2: … 퐴−2 퐴−1 퐴0 퐴1 퐴2 …
퐴 퐴1 퐴 … Tape for 푀1: 0 2 The ¢ helps us ¢ 퐴−1 퐴−2 … identify the
Jim Anderson (modified by Nathan Otterness) leftmost cell. 65 Restricted TMs: Semi-infinite Tape
Tape for 푀2: … 퐴−2 퐴−1 퐴0 퐴1 퐴2 …
퐴 퐴1 퐴 … Tape for 푀1: 0 2 ¢ 퐴−1 퐴−2 …
Formally, 푀1 = 푄1, Σ1, Γ1, 훿1, 푞1, 퐵, 퐹1 . ❖푄1 contains all 푞, 푈 and 푞, 퐷 for all 푞 ∈ 푄2. ❑We’re storing 푈 or 퐷 in the finite control to remember whether 푀1 is on the upper “푈”, or lower “퐷” track.
❑푄1 also contains 푞1 by itself. ❖Γ1 contains all 푋, 푌 , where 푋 ∈ Γ2 and 푌 ∈ Γ2 ∪ ¢ .
Jim Anderson (modified by Nathan Otterness) 66 Restricted TMs: Semi-infinite Tape
Tape for 푀2: … 퐴−2 퐴−1 퐴0 퐴1 퐴2 …
퐴 퐴1 퐴 … Tape for 푀1: 0 2 ¢ 퐴−1 퐴−2 …
Formally, 푀1 = 푄1, Σ1, Γ1, 훿1, 푞1, 퐵, 퐹1 .
❖Σ1 contains all 푎, 퐵 , where 푎 ∈ Σ2.
❖퐹1 is 푞, 푈 , 푞, 퐷 | 푞 ∈ 퐹2 . ❖퐵 is 퐵, 퐵 .
Jim Anderson (modified by Nathan Otterness) 67 Restricted TMs: Semi-infinite Tape
Tape for 푀2: … 퐴−2 퐴−1 퐴0 퐴1 퐴2 …
퐴 퐴1 퐴 … Tape for 푀1: 0 2 ¢ 퐴−1 퐴−2 …
The definition of 훿1:
훿1 푞1, 푎, 퐵 = 푞, 푈 , 푋, ¢ , 푅 if 훿2 푞2, 푎 = 푞, 푋, 푅
❖This is the case when the first move of 푀2 is right.
훿1 푞1, 푎, 퐵 = 푞, 퐷 , 푋, ¢ , 푅 if 훿2 푞2, 푎 = 푞, 푋, 퐿
❖This is the case when the first move of 푀2 is left.
Jim Anderson (modified by Nathan Otterness) 68 Restricted TMs: Semi-infinite Tape
Tape for 푀2: … 퐴−2 퐴−1 퐴0 퐴1 퐴2 …
퐴 퐴1 퐴 … Tape for 푀1: 0 2 ¢ 퐴−1 퐴−2 …
The definition of 훿1 (continued):
For all 푋, 푌 ∈ Γ1, with 푌 ≠ ¢, and 퐴 = 퐿 or 푅: If we’re in the lower track in 푀1 we need to move in 훿1 푞, 푈 , 푋, 푌 = 푝, 푈 , 푍, 푌 , 퐴 if 훿2 푞, 푋 = 푝, 푍, 퐴 . the opposite direction from ❖ This simulates 푀2 on the upper track. how 푀2 would move.
훿1 푞, 퐷 , 푋, 푌 = 푝, 퐷 , 푋, 푍 , 퐴 if 훿2 푞, 푌 = 푝, 푍, 퐴ҧ
❖ This simulates 푀2 on the lower track.
Jim Anderson (modified by Nathan Otterness) 69 Restricted TMs: Semi-infinite Tape
Tape for 푀2: … 퐴−2 퐴−1 퐴0 퐴1 퐴2 …
퐴 퐴1 퐴 … Tape for 푀1: 0 2 ¢ 퐴−1 퐴−2 …
The definition of 훿1 (continued):
훿1 푞, 푈 , 푋, ¢ = 훿1 푞, 퐷 , 푋, ¢ = 푝, 퐶 , 푌, ¢ , 푅 , if 훿2 푞, 푋 = 푝, 푌, 퐴 , where 푈 if 퐴 = 푅 퐶 = ቊ 퐷 if 퐴 = 퐿
This allows 푀1 to switch tracks if it is currently at the leftmost cell.
Jim Anderson (modified by Nathan Otterness) 70 Restricted TMs: Two-Stack Machine
Imagine a PDA that can make decisions and modify two stacks rather than one. This is called a two-stack machine. Theorem 8.13: A two-stack machine can simulate a TM. Proof sketch:
TM tape content: … 퐴1 퐴2 퐴3 퐴4 퐴5 퐴6 …
Tape head
퐴3 퐴4 Store tape contents in 퐴 two stacks like this: 퐴2 5 See Section 8.5.2 of the book for 퐴1 퐴6 a much more detailed proof.
Jim Anderson (modified by Nathan Otterness) 71 Restricted TMs: Counter Machines
A counter machine can be thought of as a multi-stack PDA, with only two stack symbols and some restrictions: ❖ 푍0, serving as the “bottom of stack marker” ❖ 푋
❖ Initially, only 푍0 is on each stack. ❖ Only 푋’s can be pushed or popped from the stack. Essentially, each stack is a counter. The PDA can only do three things with each stack: ❖ Increase the “counter” by pushing more 푋’s. ❖ Decrement the “counter” by popping a single 푋.
❖ Check if the “counter” is 0 by seeing if 푍0 is on top of the stack.
Jim Anderson (modified by Nathan Otterness) 72 Restricted TMs: Three-Counter Machines
Theorem 8.14: Every RE language is accepted by a three-counter machine. Proof: We will show that one stack can be simulated by two counters, the second of which is a “scratch” counter used for calculations. When simulating two stacks, we can then use the same scratch counter, so three counters in total are sufficient.
Jim Anderson (modified by Nathan Otterness) 73 Restricted TMs: Three-Counter Machines
We now need to show how to simulate one stack using two counters. Suppose the stack alphabet contains 푟 − 1 symbols. We can assume these symbols are denoted 1, … , 푟 − 1.
Store the stack contents 푋1푋2 … 푋푛 (where 푋1 is the top of the stack) as a base-푟 number: 푛−1 푛−2 푋푛푟 + 푋푛−1푟 + … + 푋2푟 + 푋1. Example: Assume 푟 = 10 and the stack contains the symbols 9, 2, 5, and 3 (9 is on top). The counter value associated with this stack is simply 3259.
Jim Anderson (modified by Nathan Otterness) 74 Restricted TMs: Three-Counter Machines
We now need to show how to simulate stack operations using a counter. Pop the stack: Replace the counter value 푖 by 푖/푟. The remainder is the old top-of-stack symbol, which can be stored in the finite control. 3259 3259 ❖Example: popping 3259. = = 325, with 푟 10 remainder 9.
Jim Anderson (modified by Nathan Otterness) 75 Restricted TMs: Three-Counter Machines
Push 푋 onto the stack: Replace the counter value 푖 by 푖푟 + 푋. ❖Example: pushing a 6 onto 3259. 3259푟 + 6 = 3259 ∗ 10 + 6 = 32596. Replace 푋 by 푌 on top of the stack: If 푌 > 푋, then increment 푖 by 푌 − 푋. If 푌 < 푋, then decrement it by 푋 − 푌.
Note that these operations (including pop) only involve constant values (based on the constant 푟).
Jim Anderson (modified by Nathan Otterness) 76 Restricted TMs: Three-Counter Machines
But how do we multiply or divide when we can only check if a counter is 0 or nonzero? To multiply a counter by 푟 using a scratch counter: scratch := 0; while count ≠ 0: count := count – 1; This algorithm only adds scratch := scratch + r; or subtracts constant /* If count was initially 푖, then scratch is now 푖푟. values, and only needs to So now, copy scratch back to count. */ check if counters are 0. while scratch ≠ 0: count := count + 1; scratch := scratch – 1;
Jim Anderson (modified by Nathan Otterness) 77 Restricted TMs: Three-Counter Machines To divide a counter by 푟 using a scratch counter and a bounded value 푘 (푘 ≤ 푟, and 푟 is constant, so we can keep track of 푘 using a finite number of states): scratch := 0; while count ≠ 0: k := 0; while k ≠ r and count ≠ 0: count := count – 1; k := k + 1; if k = r: scratch := scratch + 1; /* If count was initially 푖, then scratch is now 푖/푟 and k is the remainder.*/ while scratch ≠ 0: count := count + 1; scratch := scratch – 1; Jim Anderson (modified by Nathan Otterness) 78 Restricted TMs: Two-Counter Machines
We have now shown that we can simulate a single stack using two counters, and two stacks using three counters, but we can go farther! Theorem 8.15: Every RE language is accepted by a two- counter machine.
Proof: We will show that we can simulate three counters using two counters. If 푖, 푗, and 푘 are the three counts we want to track, we can store them in a single integer 푛 = 2푖3푗5푘.
Jim Anderson (modified by Nathan Otterness) 79 Restricted TMs: Two-Counter Machines
(From previous slide) If 푖, 푗, and 푘 are the three counts we want to track, we can store them in a single integer 푛 = 2푖3푗5푘. To increment 푖: Replace 푛 by 2푛. To decrement 푖: Replace 푛 by 푛/2. To test if 푖 is 0: Check if 푛 is not divisible by 2. The other counters are similar (e.g. increment 푗 by replacing 푛 with 3푛 and increment 푘 by setting 푛 = 5푛, etc.). We have already seen how to multiply and divide by constant values using a second “scratch” counter. Note that this construction works because 2, 3, and 5 are relatively prime.
Jim Anderson (modified by Nathan Otterness) 80 Restricted TMs: Summary
We have shown that the following machines are equally powerful in terms of languages they can identify: ❖ Nondeterministic TMs ❖ Multi-tape TMs ❖ Multi-track TMs ❖ Single-tape TMs ❖ Semi-infinite tape TMs ❖ Two-stack machines ❖ Three-counter machines ❖ Two-counter machines
Jim Anderson (modified by Nathan Otterness) 81 TMs vs. “Real” Computers
We want to show that anything a TM can do is possible using a computer, and vice versa. Designing a computer that simulates a TM is easy: just store the transition function in a table, which tells a program what to do next. One “gotcha”: a TM has infinite memory. ❖ The book says we can just continually add new disks to a TM when old ones are full. ❖ What if we have TM that uses more tape cells than atoms in the universe? (Clearly the book’s approach isn’t a true Turing Machine.) ❖ My take: we can’t simulate infinite memory, but we will also never have a practical application that requires it.
Jim Anderson (modified by Nathan Otterness) 82 Simulating a Computer using a TM We will consider a simplified model of a computer called a random access machine (RAM):
A potentially infinite number of values stored in memory: A finite number of registers:
푣1 Both memory and Register 1 registers can store any 푣 2 integer; even Register 2 indefinitely large ones. 푣3 …
푣4 Program Counter … Program “code” and data is stored in memory. Jim Anderson (modified by Nathan Otterness) 83 Random Access Machines
In a RAM, a “program” is a list of instructions. ❖Instructions are like what you’d find on a “real” computer, and consist of an opcode and an address. ❖Instructions are encoded as a single integer (e.g. with some bits representing the opcode and the remaining bits representing the argument). With this model, you can have basic instructions like LOAD, STORE, ADD, etc., like an assembly language.
Jim Anderson (modified by Nathan Otterness) 84 Simulating a Random Access Machine
Theorem: A TM can simulate a RAM, provided that the TM can simulate each individual RAM instruction. For example, you can’t have a RAM instruction that solves the halting problem. Proof sketch: We will use multiple tapes to store the RAM’s memory and register values in a TM.
Jim Anderson (modified by Nathan Otterness) 85 Simulating a Random Access Machine
One tape in our TM holds memory values that have been written.
❖The format of this tape is #0*푣0#1*푣1#10*푣2#... ❖The # and * are simply delimiters around the “address” of each value. Assume that each 푣푖 is stored in binary. Use one additional tape for each register.
Jim Anderson (modified by Nathan Otterness) 86 Simulating a Random Access Machine
Overview of the TM’s behavior for simulating the RAM: 1. Read the current PC value 푖 from the tape holding the “program counter” register. 2. Scan the memory tape for the “address” #푖*. Halt if it isn’t found. 3. Decode the instruction opcode and address from the memory value at location 푖. 4. Perform the operation indicated by the opcode and address. 5. Repeat from step 1.
Jim Anderson (modified by Nathan Otterness) 87 Simulating a Random Access Machine
Possible behavior for a couple common “instructions”: If the instruction indicates to “ADD” a value at address 푗 to a register: 1. Scan memory for #푗*, and add the value found afterwards to a register. 2. Add 1 to the program counter register to advance to the next instruction. If the instruction indicates to “GOTO” address 푗: 1. Copy the address 푗 to the program counter register. (Other instructions can be simulated in a similar manner)
Jim Anderson (modified by Nathan Otterness) 88 Running Time of the RAM Simulation
The previous simulation will run in polynomial time provided that a couple restrictions hold: First, the RAM can’t have an instruction that takes exponential time to compute. ❖For example, you can’t have a “solve the traveling salesman problem” instruction. ❖This isn’t a big restriction, because no real computer will have instructions like this anyway.
Jim Anderson (modified by Nathan Otterness) 89 Running Time of the RAM Simulation
(Continued list of restrictions for this simulation) Second, we can’t have instructions that “drastically” increase the length of some number. ❖ Example: If integer values are unrestricted, then starting with a value 2, applying a hypothetical “multiply a value 푛 by itself” instruction 푛 times would lead to a value of 22 , which takes 2푛+1 bits to represent. Just writing these down takes exponential time, so we don’t allow instructions like this. ❖ In reality, this is also a mild restriction. In real computers, the lengths of individual numbers are limited.
Jim Anderson (modified by Nathan Otterness) 90 Running Time of the RAM Simulation
Theorem 8.17: If a computer has (1) only instructions that increase the length of a number by 1 bit and (2) has only instructions that a multi-tape TM can perform on numbers of length 푘 in 푂 푘2 steps, then a TM exists that can simulate 푛 steps of the computer in 푂 푛3 moves. Notes: (1) rules out multiplication, which can double the length of a number. However, multiplication can be simulated in polynomial time by repeatedly adding, which is allowed. The 푂 푘2 bound in (2) was selected because it seems sufficient for most “reasonable” instructions.
Jim Anderson (modified by Nathan Otterness) 91 Running Time of the RAM Simulation
We won’t prove Theorem 8.17 in detail. However, If individual instructions can be simulated in polynomial time, we only need to worry about spending an exponential amount of time scanning the “memory” tape. Initially, the “memory” tape holds the program to execute and input data, all of which starts at a “constant” length. Restriction (1) ensures that the tape’s length remains polynomial in 푛 (after executing 푛 RAM instructions). See book Section 8.6.3 for the proof.
Jim Anderson (modified by Nathan Otterness) 92 Final Comment on RAM Simulation
We know from before that a single-tape TM can simulate a multi-tape TM in polynomial time. So, even a single-tape TM can simulate a RAM computation in polynomial time.
Jim Anderson (modified by Nathan Otterness) 93