Algorithms & Models of Computation CS/ECE 374, Fall 2017

Part I Non-deterministic Finite Automata (NFAs) NFA Introduction Lecture 4 Thursday, September 7, 2017

Sariel Har-Peled (UIUC) CS374 1 Fall 2017 1 / 39 Sariel Har-Peled (UIUC) CS374 2 Fall 2017 2 / 39

Non-deterministic Finite State Automata (NFAs) NFA behavior

0,1 0,1

0 0 1 qε q0 q00 qp ε

Differences from DFA Machine on input string w from state q can lead to of states From state q on same letter a ∈ Σ multiple possible states (could be empty) No transitions from q on some letters From qε on 1 ε-transitions! From qε on 0 Questions: From q0 on ε Is this a “real” machine? From qε on 01 What does it do? From q00 on 00

Sariel Har-Peled (UIUC) CS374 3 Fall 2017 3 / 39 Sariel Har-Peled (UIUC) CS374 4 Fall 2017 4 / 39 NFA acceptance: informal NFA acceptance: example

0,1 0,1

0 0 1 qε q0 q00 qp ε

Is 01 accepted? Informal definition: An NFA N accepts a string w iff some accepting state is reached by N from the start state on input w. Is 001 accepted? Is 100 accepted? The language accepted (or recognized) by a NFA N is denote by Are all strings in 1∗01 accepted? L(N) and defined as: L(N) = {w | N accepts w}. What is the language accepted by N? Comment: Unlike DFAs, it is easier in NFAs to show that a string is accepted than to show that a string is not accepted.

Sariel Har-Peled (UIUC) CS374 5 Fall 2017 5 / 39 Sariel Har-Peled (UIUC) CS374 6 Fall 2017 6 / 39

Simulating NFA Formal Tuple Notation Example the first a,b a,b Definition A non-deterministic finite automata (NFA) N = (Q, Σ, δ, s, A) is a b a b a five tuple where (N1) A B C D E Q is a finite set whose elements are called states, Run it on input ababa. Σ is a finite set called the input alphabet, Idea: Keep track of the states where the NFA might be at any given δ : Q × Σ ∪ {ε} → P(Q) is the transition function (here time. a,b a,b P(Q) is the power set of Q), t =0: s ∈ Q is the start state, a b a b A ⊆ Q is the set of accepting/final states. A B C D E Remaining input: ababa. δ(q, a) for a ∈ Σ ∪ {ε} is a of Q — a set of states. a,b a,b t =1: a b a b A B C D E Sariel Har-Peled (UIUC) CS374 7 Fall 2017 7 / 39 Sariel Har-Peled (UIUC) CS374 8 Fall 2017 8 / 39 Remaining input: baba. a,b a,b t =2: a b a b A B C D E Remaining input: aba. a,b a,b t =3: a b a b A B C D E Remaining input: ba. a,b a,b t =4: a b a b A B C D E Remaining input: a. a,b a,b t =5: a b a b A B C D E Remaining input: ε. Accepts: ababa. Reminder: Power set Example For a set Q its power set is: P(Q) = 2Q = {X | X ⊆ Q} is the 0,1 set of all of Q. 0,1 0 0 1 qε q0 q00 qp ε Example Q = {1, 2, 3, 4} Q = {qε, q0, q00, qp}   Σ = {0, 1}  {1, 2, 3, 4} ,    δ  {2, 3, 4} , {1, 3, 4} , {1, 2, 4} , {1, 2, 3} ,  P(Q) = {1, 2} , {1, 3} , {1, 4} , {2, 3} , {2, 4} , {3, 4} , s = qε    {1} , {2} , {3} , {4} ,  A = {qp}  {} 

Sariel Har-Peled (UIUC) CS374 9 Fall 2017 9 / 39 Sariel Har-Peled (UIUC) CS374 10 Fall 2017 10 / 39

Example Extending the transition function to strings Transition function in detail... 1 NFA N = (Q, Σ, δ, s, A) 2 δ(q, a): set of states that N can go to from q on reading a ∈ Σ ∪ {ε}. 3 Want transition function δ∗ : Q × Σ∗ → P(Q) 4 δ∗(q, w): set of states reachable on input w starting in state q. δ(q0, ε) = {q0, q00} δ(qε, ε) = {qε} δ(q0, 0) = {q00} δ(qε, 0) = {qε, q0} δ(q0, 1) = {} δ(qε, 1) = {qε} δ(q00, ε) = {q00} δ(qp, ε) = {qp}

δ(q00, 0) = {} δ(qp, 0) = {qp}

δ(q00, 1) = {qp} δ(qp, 1) = {qp}

Sariel Har-Peled (UIUC) CS374 11 Fall 2017 11 / 39 Sariel Har-Peled (UIUC) CS374 12 Fall 2017 12 / 39 Models of Computation Lecture ￿: Nondeterministic Automata [Fa’￿￿] the NFA somehow chose a path to an accept state still exist. One slight disadvantage of this metaphor is that if an NFA reads a string that is not in its language, it destroys all universes.

Proofs/oracles. Finally, we can treat NFAs not as a mechanism for computing something, but as a mechanism for verifying proofs. If we want to prove that a string w contains one of the suffixes 00 or 11, it suffices to demonstrate a single walk in our example NFA that starts at s and ends at c, and whose edges are labeled with the symbols in w. Equivalently, whenever the NFA faces a nontrivial choice, the prover can simply tell the NFA which state to move to next. This intuition can be formalized as follows. Consider a deterministic finite state machine whose input alphabet is the product ⌃ ⌦ of an input alphabet ⌃ and an oracle alphabet ⌦. Equivalently, we can imagine that this DFA⇥ reads simultaneously from two strings of the same length: the input string w and the oracle string !. In either formulation, the transition function has the form : Q (⌃ ⌦) Q. As usual, this DFA accepts the pair (w, !) (⌃ ⌦)⇤ if and ⇥ ⇥ ! 2 ⇥ only if ⇤(s, (w, !)) A. Finally, M nondeterministically accepts the string w ⌃⇤ if there is 2 2 an oracle string ! ⌦⇤ with ! = w such that (w, !) L(M). 2 | | | | 2 ￿.￿ "-Transitions It is fairly common for NFAs to include so-called "-transitions, which allow the machine to change stateExtending without reading the an transition input symbol. function An NFA with to" strings-transitions accepts a string w Extending the transition function to strings a1 a2 a3 a` if and only if there is a of transitions s q1 q2 q` where the final ! ! ! ··· ! state q` isDefinition accepting, each ai is either " or a symbol in ⌃, and a1a2 a` = w. Definition ··· For example,For NFA considerN = the (Q following, Σ, δ, s, NFAA) and withq"-transitions.∈ Q the reach (For this(q) example,is the set we indicate For NFA N = (Q, Σ, δ, s, A) and q ∈ Q the reach(q) is the set the "-transitionsof all states using large that redq can arrows; reach we using won’t only normallyε-transitions. do that.) This NFA deliberately has of all states that q can reach using only ε-transitions. more "-transitions than necessary.

0 0 Definition a b c ∗ ∗ ε Inductive definition of δ : Q × Σ → P(Q): ε ∗ 1,0 s ε ε g 1,0 if w = ε, δ (q, w) = reach(q)

ε ε if w = a where a ∈ Σ d e f δ∗(q, a) = ∪ (∪ reach(r)) 1 1 p∈reach(q) r∈δ(p,a) An NFA with "-transitions if w = ax, ∗ ∗ δ (q, w) = ∪p∈reach(q)(∪r∈δ(p,a)δ (r, x)) The NFA starts as usual in state s. If the input string is 100111, the the machine might non-deterministically choose the following transitions and then accept.

1 " " 0 0 " 1 1 " 1 " " s Sariels Har-Peledd (UIUC) a b cCS374d 13e f e f Fall 2017c 13g / 39 Sariel Har-Peled (UIUC) CS374 14 Fall 2017 14 / 39 ! ! ! ! ! ! ! ! ! ! ! ! More formally, the transition function in an NFA with "-transitions has a slightly larger Q domain : Q (⌃ " ) 2 . The "-reach of a state q Q consists of all states r that satisfy one of theFormal following⇥ [ { conditions:definition} ! of language accepted2 by N Example • either r = q, Definition • or r (q0, ") for some state q0 in the "-reach∗ of q. A2 string w is accepted by NFA N if δN (s, w) ∩ A 6= ∅. In other words, r is in the "-reach of q if there is a (possibly empty) sequence of "-transitions leading fromDefinitionq to r. For example, in the example NFA above, the "-reach of state f is a, c, d, f , g . { } The language L(N) accepted by a NFA N = (Q, Σ, δ, s, A) is What is: ￿ {w ∈ Σ∗ | δ∗(s, w) ∩ A 6= ∅}. δ∗(s, ) δ∗(s, 0) Important: Formal definition of the language of NFA above uses δ∗ δ∗(c, 0) and not δ. As such, one does not need to include ε-transitions ∗ when specifying δ, since δ∗ takes care of that. δ (b, 00)

Sariel Har-Peled (UIUC) CS374 15 Fall 2017 15 / 39 Sariel Har-Peled (UIUC) CS374 16 Fall 2017 16 / 39 Another definition of computation Why non-determinism?

Definition Non-determinism adds power to the model; richer programming w language and hence (much) easier to “design” programs q −→N p: State p of NFA N is reachable from q on w ⇐⇒ Fundamental in theory to prove many theorems there exists a sequence of states r0, r1,..., rk and a sequence x1, x2,..., xk where xi ∈ Σ ∪ {ε}, for each i, such that: Very important in practice directly and indirectly

r0 = q, Many deep connections to various fields in Computer Science and Mathematics for each i, ri+1 ∈ δ(ri , xi+1),

rk = p, and Many interpretations of non-determinism. Hard to understand at the outset. Get used to it and then you will appreciate it slowly. w = x1x2x3 ··· xk .

Definition n o ∗ w δ N(q, w) = p ∈ Q q −→N p .

Sariel Har-Peled (UIUC) CS374 17 Fall 2017 17 / 39 Sariel Har-Peled (UIUC) CS374 18 Fall 2017 18 / 39

DFAs and NFAs

Every DFA is a NFA so NFAs are at least as powerful as DFAs. Part II NFAs prove ability to “guess and verify” which simplifies design and reduces number of states Easy proofs of some closure properties Constructing NFAs

Sariel Har-Peled (UIUC) CS374 19 Fall 2017 19 / 39 Sariel Har-Peled (UIUC) CS374 20 Fall 2017 20 / 39 Example Example

Strings that represent decimal numbers. {strings that contain CS374 as a substring} {strings that contain CS374 or CS473 as a substring} {strings that contain CS374 and CS473 as substrings} ε, – 0 . 0,1,2,...,9 0 1 2 3 5

1,2,...,9 . 0,1,2,...,9 4

0,1,2,...,9

Sariel Har-Peled (UIUC) CS374 21 Fall 2017 21 / 39 Sariel Har-Peled (UIUC) CS374 22 Fall 2017 22 / 39

Example A simple transformation

Lk = {bitstrings that have a 1 k positions from the end} Theorem For every NFA N there is another NFA N 0 such that L(N) = L(N 0) and such that N 0 has the following two properties: N 0 has single final state f that has no outgoing transitions The start state s of N is different from f

Sariel Har-Peled (UIUC) CS374 23 Fall 2017 23 / 39 Sariel Har-Peled (UIUC) CS374 24 Fall 2017 24 / 39 Closure properties of NFAs Are the class of languages accepted by NFAs closed under the following operations? Part III union intersection Closure Properties of NFAs Kleene star complement

Sariel Har-Peled (UIUC) CS374 25 Fall 2017 25 / 39 Sariel Har-Peled (UIUC) CS374 26 Fall 2017 26 / 39

Closure under union Closure under concatenation Theorem Theorem

For any two NFAs N1 and N2 there is a NFA N such that For any two NFAs N1 and N2 there is a NFA N such that L(N) = L(N1) ∪ L(N2). L(N) = L(N1)·L(N2).

q1 f1 N1 q1 f1 q2 f2 N1 N2

q2 f2 N2

Sariel Har-Peled (UIUC) CS374 27 Fall 2017 27 / 39 Sariel Har-Peled (UIUC) CS374 28 Fall 2017 28 / 39 Closure under Kleene star Closure under Kleene star Theorem Theorem ∗ ∗ For any NFA N1 there is a NFA N such that L(N) = (L(N1)) . For any NFA N1 there is a NFA N such that L(N) = (L(N1)) .

ε

q1 f1 N1

q1 f1 N1

Does not work! Why?

Sariel Har-Peled (UIUC) CS374 29 Fall 2017 29 / 39 Sariel Har-Peled (UIUC) CS374 30 Fall 2017 30 / 39

Closure under Kleene star Theorem ∗ For any NFA N1 there is a NFA N such that L(N) = (L(N1)) . Part IV

ε q0 q f NFAs capture Regular Languages 1 N1 1

ε

Sariel Har-Peled (UIUC) CS374 31 Fall 2017 31 / 39 Sariel Har-Peled (UIUC) CS374 32 Fall 2017 32 / 39 Regular Languages Recap NFAs and

Regular Languages Regular Expressions Theorem For every regular language L there is an NFA N such that ∅ regular ∅ denotes ∅ L = L(N). {} regular  denotes {} {a} regular for a ∈ Σ a denote {a} Proof strategy: R1 ∪ R2 regular if both are r1 + r2 denotes R1 ∪ R2 For every r show that there is a NFA N such R1R2 regular if both are r1r2 denotes R1R2 that L(r) = L(N) ∗ ∗ ∗ R is regular if R is r denote R Induction on length of r

Regular expressions denote regular languages — they explicitly show the operations that were used to form the language

Sariel Har-Peled (UIUC) CS374 33 Fall 2017 33 / 39 Sariel Har-Peled (UIUC) CS374 34 Fall 2017 34 / 39

NFAs and Regular Language NFAs and Regular Language

For every regular expression r show that there is a NFA N such For every regular expression r show that there is a NFA N such that L(r) = L(N) that L(r) = L(N) Induction on length of r Induction on length of r Base cases: ∅, {ε}, {a} for a ∈ Σ. Inductive cases:

r1, r2 regular expressions and r = r1 + r2. By induction there are NFAs N1, N2 s.t L(N1) = L(r1) and L(N2) = L(r2). We have already seen that there is NFA N s.t L(N) = L(N1) ∪ L(N2), hence L(N) = L(r) r = r1·r2. Use closure of NFA languages under concatenation ∗ r = (r1) . Use closure of NFA languages under Kleene star

Sariel Har-Peled (UIUC) CS374 35 Fall 2017 35 / 39 Sariel Har-Peled (UIUC) CS374 36 Fall 2017 36 / 39 Example Example

(ε+0)(1+10)* ε (1+10) * (ε+0) (1+10)* 0

ε ε 10 * (1+10) * 0 1 0 ε 1 0* 0 1

Sariel Har-Peled (UIUC) CS374 37 Fall 2017 37 / 39 Sariel Har-Peled (UIUC) CS374 38 Fall 2017 38 / 39

Example ε 1 0* 0 1

ε 1 3 0 ε 0 1 2 4 0 1 ε

Final NFA simplified slightly to reduce states Sariel Har-Peled (UIUC) CS374 39 Fall 2017 39 / 39