Regular Languages and Finite State Automata Data structures and algorithms for Computational Linguistics III
Çağrı Çöltekin [email protected]
University of Tübingen Seminar für Sprachwissenschaft
Winter Semester 2019–2020 Introduction DFA NFA Regular languages Minimization Regular expressions Why study finite-state automata?
• Unlike some of the abstract machines we discussed, finite-state automata are efficient models of computation • There are many applications – Electronic circuit design – Workflow management – Games – Pattern matching – … But more importantly ;-) – Tokenization, stemming – Morphological analysis – Shallow parsing/chunking – …
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 1 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Finite-state automata (FSA)
• A finite-state machine is in one of a finite-number of states in a giventime • The machine changes its state based on its input • Every regular language is generated/recognized by an FSA • Every FSA generates/recognizes a regular language • Two flavors: – Deterministic finite automata (DFA) – Non-deterministic finite automata (NFA) Note: the NFA is a superset of DFA.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 2 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions DFA as a graph
b
state • States are represented as nodes 1 b • Transitions are shown by the edges, transition labeled with symbols from an alphabet 0 a • One of the states is marked as the initial state initial state • a Some states are accepting states 2
accepting state
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 3 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions DFA: formal definition
Formally, a finite state automaton, M, is a tuple (Σ, Q, q0, F, ∆) with Σ is the alphabet, a finite set of symbols Q a finite set of states
q0 is the start state, q0 ∈ Q F is the set of final states, F ⊆ Q ∆ is a function that takes a state and a symbol in the alphabet, and returns another state (∆ : Q × Σ → Q)
At any given time, for any input, a DFA has a single well-defined action to take.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 4 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions DFA: formal definition an example
b
Σ = {a, b} 1 Q = {q0, q1, q2} b
q0 = q0 F = {q } 2 0 a ∆ = {(q0, a) → q2, (q0, b) → q1, (q1, a) → q2, (q1, b) → q1} a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 5 / 56 – In that case, when we reach a dead end, recognition fails
• To make all transitions well-defined, we can add a sink (or error) state • For brevity, we skip the explicit error state
Introduction DFA NFA Regular languages Minimization Regular expressions Another note on DFA
b
• Is this FSA deterministic? 1 b
a, b 0 3 a
a, b a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 6 / 56 – In that case, when we reach a dead end, recognition fails
• For brevity, we skip the explicit error state
Introduction DFA NFA Regular languages Minimization Regular expressions Another note on DFA error or sink state
b
• Is this FSA deterministic? 1 • To make all transitions well-defined, b we can add a sink (or error) state a, b 0 3 a
a, b a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 6 / 56 – In that case, when we reach a dead end, recognition fails
Introduction DFA NFA Regular languages Minimization Regular expressions Another note on DFA error or sink state
b
• Is this FSA deterministic? 1 • To make all transitions well-defined, b we can add a sink (or error) state • For brevity, we skip the explicit error a, b state 0 3 a
a, b a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 6 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Another note on DFA error or sink state
b
• Is this FSA deterministic? 1 • To make all transitions well-defined, b we can add a sink (or error) state • For brevity, we skip the explicit error a, b state 0 3 a – In that case, when we reach a dead end, recognition fails a, b a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 6 / 56 3 3 3
Introduction DFA NFA Regular languages Minimization Regular expressions DFA: the transition table
transition table b symbol a b 1 →0 2 1 b 1 2 1 state *2 ∅ ∅ a, b 0 3 a
a, b → a marks the start state 2 * marks the accepting state(s)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 7 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions DFA: the transition table
transition table b symbol a b 1 →0 2 1 b 1 2 1 state *2 3 3 a, b 3 3 3 0 3 a
a, b → a marks the start state 2 * marks the accepting state(s)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 7 / 56 • What is the complexity of the algorithm? • How about inputs: – bbbb – aa
Introduction DFA NFA Regular languages Minimization Regular expressions DFA recognition
1. Start at q0 2. Process an input symbol, move accordingly b 3. Accept if in a final state at the end of the input b 1
0 a
a 2
Input: b b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 8 / 56 • What is the complexity of the algorithm? • How about inputs: – bbbb – aa
Introduction DFA NFA Regular languages Minimization Regular expressions DFA recognition
1. Start at q0 2. Process an input symbol, move accordingly b 3. Accept if in a final state at the end of the input b 1
0 a
a 2
Input: b b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 8 / 56 • What is the complexity of the algorithm? • How about inputs: – bbbb – aa
Introduction DFA NFA Regular languages Minimization Regular expressions DFA recognition
1. Start at q0 2. Process an input symbol, move accordingly b 3. Accept if in a final state at the end of the input b 1
0 a
a 2
Input: b b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 8 / 56 • What is the complexity of the algorithm? • How about inputs: – bbbb – aa
Introduction DFA NFA Regular languages Minimization Regular expressions DFA recognition
1. Start at q0 2. Process an input symbol, move accordingly b 3. Accept if in a final state at the end of the input b 1
0 a
a 2
Input: b b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 8 / 56 • What is the complexity of the algorithm? • How about inputs: – bbbb – aa
Introduction DFA NFA Regular languages Minimization Regular expressions DFA recognition
1. Start at q0 2. Process an input symbol, move accordingly b 3. Accept if in a final state at the end of the input b 1
0 a
a 2
Input: b b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 8 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions DFA recognition
1. Start at q0 2. Process an input symbol, move accordingly b 3. Accept if in a final state at the end of the input b 1
0 a
• What is the complexity of the a 2 algorithm? • How about inputs: – bbbb – aa Input: b b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 8 / 56 • Can you draw a simpler DFA for the same language? • Draw a DFA recognizing strings with even number of ‘a’s over Σ = {a, b}
Introduction DFA NFA Regular languages Minimization Regular expressions A few questions
• What is the language recognized by b this FSA? b 1
0 a
a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 9 / 56 • Draw a DFA recognizing strings with even number of ‘a’s over Σ = {a, b}
Introduction DFA NFA Regular languages Minimization Regular expressions A few questions
• What is the language recognized by b this FSA? • Can you draw a simpler DFA for the b 1 same language? 0 a
a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 9 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions A few questions
• What is the language recognized by b this FSA? • Can you draw a simpler DFA for the b 1 same language? • Draw a DFA recognizing strings 0 a with even number of ‘a’s over { , } Σ = a b a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 9 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Non-deterministic finite automata Formal definition
A non-deterministic finite state automaton, M, is a tuple (Σ, Q, q0, F, ∆) with Σ is the alphabet, a finite set of symbols Q a finite set of states
q0 is the start state, q0 ∈ Q F is the set of final states, F ⊆ Q ∆ is a function from (Q, Σ) to P(Q), power set of Q (∆ : Q × Σ → P(Q))
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 10 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions An example NFA a,b transition table a,b a,b 1 symbol a b 0 a →0 0,1 0,1 1 1,2 1 a,b 2 state *2 0,2 0
a • We have nondeterminism, e.g., if the first input is a, we need to choose between states 0 or 1 • Transition table cells have sets of states
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 11 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Dealing with non-determinism
• Follow one of the links, store alternatives, and backtrack on failure • Follow all options in parallel • Use dynamic programming (e.g., as in chart parsing)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 12 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b
1. Start at q0 a,b a,b 1 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q0, 1)
(q1, 1) 1. Start at q0 a,b a,b 1 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q0, 1)
(q1, 1) 1. Start at q0 a,b a,b 1 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q0, 2)
(q1, 2) 1. Start at q0 a,b a,b 1 (q1, 1) 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q0, 2)
(q1, 2) 1. Start at q0 a,b a,b 1 (q1, 1) 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q0, 3)
(q1, 3) 1. Start at q0 a,b a,b 1 (q1, 2) 2. Take the next input, place all (q1, 1) possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q0, 3)
(q1, 3) 1. Start at q0 a,b a,b 1 (q1, 2) 2. Take the next input, place all (q1, 1) possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q1, 3)
(q1, 2) 1. Start at q0 a,b a,b 1 (q1, 1) 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q1, 3)
(q1, 2) 1. Start at q0 a,b a,b 1 (q1, 1) 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q1, 2)
(q1, 1) 1. Start at q0 a,b a,b 1 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q1, 2)
(q1, 1) 1. Start at q0 a,b a,b 1 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q2, 3)
(q1, 3) 1. Start at q0 a,b a,b 1 (q1, 1) 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q2, 3)
(q1, 3) 1. Start at q0 a,b a,b 1 (q1, 1) 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search (with backtracking) Agenda a,b (q1, 3)
(q1, 1) 1. Start at q0 a,b a,b 1 2. Take the next input, place all possible actions to an agenda 0 a 3. Get the next action from the agenda, act a,b 2 4. At the end of input Accept if in an accepting state Reject not in accepting state & agenda a empty Backtrack otherwise Input: a b a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 13 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition as search summary
• Worst time complexity is exponential – Complexity is worse if we want to enumerate all derivations • We used a stack as agenda, performing a depth-first search • A queue would result in breadth-first search • If we have a reasonable heuristic A* search may be an option • Machine learning methods may also guide finding a fast or the best solution
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 14 / 56 Note: the process is deterministic, and finite-state.
Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition parallel version a,b 1. Start at q0 2. Take the next input, mark all possible a,b a,b 1 next states 3. If an accepting state is marked at the end 0 a of the input, accept
a,b 2
a
Input: a b a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 15 / 56 Note: the process is deterministic, and finite-state.
Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition parallel version a,b 1. Start at q0 2. Take the next input, mark all possible a,b a,b 1 next states 3. If an accepting state is marked at the end 0 a of the input, accept
a,b 2
a
Input: a b a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 15 / 56 Note: the process is deterministic, and finite-state.
Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition parallel version a,b 1. Start at q0 2. Take the next input, mark all possible a,b a,b 1 next states 3. If an accepting state is marked at the end 0 a of the input, accept
a,b 2
a
Input: a b a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 15 / 56 Note: the process is deterministic, and finite-state.
Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition parallel version a,b 1. Start at q0 2. Take the next input, mark all possible a,b a,b 1 next states 3. If an accepting state is marked at the end 0 a of the input, accept
a,b 2
a
Input: a b a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 15 / 56 Note: the process is deterministic, and finite-state.
Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition parallel version a,b 1. Start at q0 2. Take the next input, mark all possible a,b a,b 1 next states 3. If an accepting state is marked at the end 0 a of the input, accept
a,b 2
a
Input: a b a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 15 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA recognition parallel version a,b 1. Start at q0 2. Take the next input, mark all possible a,b a,b 1 next states 3. If an accepting state is marked at the end 0 a of the input, accept
a,b 2 Note: the process is deterministic, and a finite-state.
Input: a b a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 15 / 56 a,b
a b NFA: 0 1 2
b a a a DFA: 0 1 b 2
b
Introduction DFA NFA Regular languages Minimization Regular expressions An exercise Construct an NFA and a DFA for the language over Σ = {a, b} where all sen- tences end with ab.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 16 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions An exercise Construct an NFA and a DFA for the language over Σ = {a, b} where all sen- tences end with ab.
a,b
a b NFA: 0 1 2
b a a a DFA: 0 1 b 2
b Ç. Çöltekin, SfS / University of Tübingen WS 19–20 16 / 56 b
1 a
0 b a
2
a
Introduction DFA NFA Regular languages Minimization Regular expressions One more complication: ϵ transitions • An extension of NFA, ϵ-NFA, allows moving without consuming an input symbol, indicated by an ϵ-transition (sometimes called a λ-transition) • Any ϵ-NFA can be converted to an NFA b
1 a
0 ϵ
2
a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 17 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions One more complication: ϵ transitions • An extension of NFA, ϵ-NFA, allows moving without consuming an input symbol, indicated by an ϵ-transition (sometimes called a λ-transition) • Any ϵ-NFA can be converted to an NFA b b
1 1 a a
0 ϵ 0 b a
2 2
a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 17 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions ϵ-transitions need attention
3 b b ϵ b,ϵ a a a 0 1 2 4
• How does the (depth-first) NFA recognition algorithm we described earlier work on this automaton? • Can we do without ϵ transitions?
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 18 / 56 b
a
– ϵ-closure(q0) = {q0} { , } – ϵ-closure(q1) = q1 q2 ϵ b – ϵ-closure(q2) = {q2} • Replace each arc to each state with a arc(s) to all states in the ϵ-closure of the state a
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal
b
• We start with finding the ϵ-closure of all states a 1
0 ϵ
2
a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 19 / 56 b
a
{ , } – ϵ-closure(q1) = q1 q2 ϵ b – ϵ-closure(q2) = {q2} • Replace each arc to each state with a arc(s) to all states in the ϵ-closure of the state a
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal
b
• We start with finding the ϵ-closure of all states a 1 – ϵ-closure(q0) = {q0} 0 ϵ
2
a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 19 / 56 b
a
ϵ b – ϵ-closure(q2) = {q2} • Replace each arc to each state with a arc(s) to all states in the ϵ-closure of the state a
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal
b
• We start with finding the ϵ-closure of all states a 1 – ϵ-closure(q0) = {q0} { , } – ϵ-closure(q1) = q1 q2 0 ϵ
2
a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 19 / 56 b
a
ϵ b
• Replace each arc to each state with a arc(s) to all states in the ϵ-closure of the state a
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal
b
• We start with finding the ϵ-closure of all states a 1 – ϵ-closure(q0) = {q0} { , } – ϵ-closure(q1) = q1 q2 0 ϵ – ϵ-closure(q2) = {q2} 2
a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 19 / 56 b
a
ϵ
a
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal
b
• We start with finding the ϵ-closure of all states a 1 – ϵ-closure(q0) = {q0} { , } – ϵ-closure(q1) = q1 q2 0 ϵ b – ϵ-closure(q2) = {q2} • Replace each arc to each state with a 2 arc(s) to all states in the ϵ-closure of the state a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 19 / 56 ϵ∗ 0,1,2 0,3 1,3 1,2 3 1,3 2 3 ∅ 3 3 1
symbol symbol a b ϵ a b →0 0 ∅ 1 ⇒ →0 1 ∅ 1,3 2 1
state 2 3 ∅ ∅ 2 *3 3 1 ∅ *3
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ 0 1 ϵ a,bb b a,b 2 3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 0,3 1,3 3 1,3 3 ∅ 3 1
symbol ϵ∗ a b 0,1,2 ⇒ →0 1,2 1 2 2 3 *3
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol 0 1 a b ϵ ϵ →0 0 ∅ 1 a,bb b 1 ∅ 1,3 2 a,b
state ∅ ∅ 2 3 2 3 *3 3 1 ∅ a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 0,3 1,3 3 1,3 3 ∅ 3 1
symbol a b ⇒ →0 1,2 1 2 2 3 *3
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol ∗ 0 1 a b ϵ ϵ ϵ →0 0 ∅ 1 0,1,2 a,bb b 1 ∅ 1,3 2 a,b
state ∅ ∅ 2 3 2 3 *3 3 1 ∅ a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 0,3 1,3 3 1,3 3 ∅ 3 1
symbol a b ⇒ →0 1 2 3 *3
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol ∗ 0 1 a b ϵ ϵ ϵ →0 0 ∅ 1 0,1,2 a,bb b 1 ∅ 1,3 2 1,2 a,b
state ∅ ∅ 2 3 2 2 3 *3 3 1 ∅ a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 0,3 1,3 3 1,3 3 ∅ 3 1
symbol a b ⇒ →0 1 2 *3
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol ∗ 0 1 a b ϵ ϵ ϵ →0 0 ∅ 1 0,1,2 a,bb b 1 ∅ 1,3 2 1,2 a,b
state ∅ ∅ 2 3 2 2 3 *3 3 1 ∅ 3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 0,3 1,3 3 1,3 3 ∅ 3 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 a,bb b 1 ∅ 1,3 2 1,2 1 a,b
state ∅ ∅ 2 3 2 2 2 3 *3 3 1 ∅ 3 *3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 1,3 3 1,3 3 ∅ 3 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 a,bb b 1 ∅ 1,3 2 1,2 1 a,b
state ∅ ∅ 2 3 2 2 2 3 *3 3 1 ∅ 3 *3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 3 1,3 3 ∅ 3 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 a,b
state ∅ ∅ 2 3 2 2 2 3 *3 3 1 ∅ 3 *3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 1,3 3 ∅ 3 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 3 a,b
state ∅ ∅ 2 3 2 2 2 3 *3 3 1 ∅ 3 *3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 3 ∅ 3 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 3 1,3 a,b
state ∅ ∅ 2 3 2 2 2 3 *3 3 1 ∅ 3 *3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 ∅ 3 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 3 1,3 a,b
state ∅ ∅ 2 3 2 2 3 2 3 *3 3 1 ∅ 3 *3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 3 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 3 1,3 a,b
state ∅ ∅ ∅ 2 3 2 2 3 2 3 *3 3 1 ∅ 3 *3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 1
Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 3 1,3 a,b
state ∅ ∅ ∅ 2 3 2 2 3 2 3 *3 3 1 ∅ 3 *3 3 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 3 1,3 a,b
state ∅ ∅ ∅ 2 3 2 2 3 2 3 *3 3 1 ∅ 3 *3 3 1 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions ϵ removal a(nother) solution with the transition table
transition table a b bϵ symbol symbol ∗ 0 1 a b ϵ ϵ a b ϵ →0 0 ∅ 1 0,1,2 ⇒ →0 0,3 1,3 a,bb b 1 ∅ 1,3 2 1,2 1 3 1,3 a,b
state ∅ ∅ ∅ 2 3 2 2 3 2 3 *3 3 1 ∅ 3 *3 3 1 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 20 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions NFA–DFA equivalence
• The language recognized by every NFA is recognized by some DFA • The set of DFA is a subset of the set of NFA (a DFA is also an NFA) • The same is true for ϵ-NFA • All recognize/generate regular languages • NFA can automatically be converted to the equivalent DFA
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 21 / 56 – and a not-so-quick one
1. Construct (draw) an NFA for the language over Σ = {a, b}, such that 4th symbol from the end is an a a,b a a,b a,b a,b 0 1 2 3 4 2. Construct a DFA for the same language
A quick exercise
Introduction DFA NFA Regular languages Minimization Regular expressions Why do we use an NFA then? • NFA (or ϵ-NFA) are often easier to construct – Intuitive for humans (cf. earlier exercise) – Some representations are easy to convert to NFA rather than DFA, e.g., regular expressions • NFA may require less memory (fewer states)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 22 / 56 – and a not-so-quick one
a,b a a,b a,b a,b 0 1 2 3 4 2. Construct a DFA for the same language
Introduction DFA NFA Regular languages Minimization Regular expressions Why do we use an NFA then? • NFA (or ϵ-NFA) are often easier to construct – Intuitive for humans (cf. earlier exercise) – Some representations are easy to convert to NFA rather than DFA, e.g., regular expressions • NFA may require less memory (fewer states) A quick exercise
1. Construct (draw) an NFA for the language over Σ = {a, b}, such that 4th symbol from the end is an a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 22 / 56 – and a not-so-quick one
2. Construct a DFA for the same language
Introduction DFA NFA Regular languages Minimization Regular expressions Why do we use an NFA then? • NFA (or ϵ-NFA) are often easier to construct – Intuitive for humans (cf. earlier exercise) – Some representations are easy to convert to NFA rather than DFA, e.g., regular expressions • NFA may require less memory (fewer states) A quick exercise
1. Construct (draw) an NFA for the language over Σ = {a, b}, such that 4th symbol from the end is an a a,b a a,b a,b a,b 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 22 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Why do we use an NFA then? • NFA (or ϵ-NFA) are often easier to construct – Intuitive for humans (cf. earlier exercise) – Some representations are easy to convert to NFA rather than DFA, e.g., regular expressions • NFA may require less memory (fewer states) A quick exercise – and a not-so-quick one
1. Construct (draw) an NFA for the language over Σ = {a, b}, such that 4th symbol from the end is an a a,b a a,b a,b a,b 0 1 2 3 4 2. Construct a DFA for the same language
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 22 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Determinization the subset construction
Intuition: remember the parallel NFA recognition. We can consider an NFA being a deterministic machine which is at a set of states at any given time.
• Subset construction (sometimes called power set construction) uses this intuition to convert an NFA to a DFA • The algorithm can be modified to handle ϵ-transitions (or we can eliminate ϵ’s as a preprocessing step)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 23 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions The subset construction by example transition table with subsets
a,b symbol a b a,b a,b 1 ∅ ∅ ∅ → {0}{0, 1}{0, 1} { }{ }{ } 0 a 1 1, 2 1 * {2}{0, 2}{0} { }{ }{ } 2 0, 1 0, 1, 2 0, 1 a,b * {0, 2}{0, 1, 2}{0, 1} * {1, 2}{0, 1, 2}{0, 1} a * {0, 1, 2}{0, 1, 2}{0, 1}
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 24 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions The subset construction by example transition table with subsets
a,b symbol a b a,b a,b 1 ∅ ∅ ∅ → {0}{0, 1}{0, 1} { }{ }{ } 0 a 1 1, 2 1 * {2}{0, 2}{0} { }{ }{ } 2 0, 1 0, 1, 2 0, 1 a,b * {0, 2}{0, 1, 2}{0, 1} * {1, 2}{0, 1, 2}{0, 1} a * {0, 1, 2}{0, 1, 2}{0, 1}
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 24 / 56 Do you remember the set of states marked during parallel NFA recognition?
Introduction DFA NFA Regular languages Minimization Regular expressions The subset construction by example: the resulting DFA
transition table without useless/inaccessible states
symbol a b → {0}{0, 1}{0, 1} {0, 1}{0, 1, 2}{0, 1} * {0, 1, 2}{0, 1, 2}{0, 1}
b a a,b 0 01 012 a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 25 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions The subset construction by example: the resulting DFA
transition table without useless/inaccessible states
symbol a b → {0}{0, 1}{0, 1} {0, 1}{0, 1, 2}{0, 1} * {0, 1, 2}{0, 1, 2}{0, 1}
b a a,b 0 01 012 a b Do you remember the set of states marked during parallel NFA recognition?
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 25 / 56 • What language do they recognize?
Introduction DFA NFA Regular languages Minimization Regular expressions The subset construction by example: side by side NFA DFA
a,b b
a,b a,b 1 a,b 1
0 a 0 b a
a 2 a a,b 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 26 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions The subset construction by example: side by side NFA DFA
a,b b
a,b a,b 1 a,b 1
0 a 0 b a
a 2 a a,b 2
• What language do they recognize?
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 26 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions The subset construction wrapping up
• In worst case, resulting DFA has 2n nodes • Worst case is rather rare, number of nodes in an NFA and the converted DFA are often similar • In practice, we do not need to enumerate all 2n subsets • We’ve already seen a typical problematic case: a,b a a,b a,b a,b 0 1 2 3 4 • We can also skip the unreachable states during subset construction
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 27 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Yet another exercise
Determinize the following automaton
a,b
a b NFA: 0 1 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 28 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Regular languages: definition
A regular grammar is a tuple G = (Σ, N, S, R) where Σ is an alphabet of terminal symbols N are a set of non-terminal symbols S is a special ‘start’ symbol ∈ N R is a set of rewrite rules following one of the following patterns (A, B ∈ N, a ∈ Σ, ϵ is the empty string) Left regular Right regular
1. A → a 1. A → a 2. A → Ba 2. A → aB 3. A → ϵ 3. A → ϵ
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 29 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Regular languages: another definition
A language is regular if there is an FSA that recognizes it
• We denote the language recognized by a finite state automaton M, as L(M) • The above definition reformulated: if a language L is regular, there is a DFA M, such that L(M) = L • Remember: any NFA (with or without ϵ transitions) can be converted to a DFA
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 30 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Some operations on regular languages (and FSA)
L1L2 Concatenation of two languages L1 and L2: any sentence of L1 followed by any sentence of L2 ∗ L Kleene star of L: L concatenated by itself 0 or more times LR Reverse of L: reverse of any string in L ∗ ∗ L Complement of L: all strings in ΣL except the ones in L (ΣL − L) L1 ∪ L2 Union of languages L1 and L2: strings that are in any of the languages
L1 ∩ L2 Intersection of languages L1 and L2: strings that are in both languages
Regular languages are closed under all of these operations.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 31 / 56 Odd number of a’s over {a, b}. Odd number of b’s over {a, b}.
We will use these languages and automata for demonstration.
Introduction DFA NFA Regular languages Minimization Regular expressions Two example FSA what languages do they accept?
L L1 = L(M1) L2 = (M2)
M1 b b M2 a a a b 0 1 0 1 a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 32 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Two example FSA what languages do they accept?
L L1 = L(M1) L2 = (M2)
M1 b b M2 a a a b 0 1 0 1 a b Odd number of a’s over {a, b}. Odd number of b’s over {a, b}.
We will use these languages and automata for demonstration.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 32 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Concatenation
L1 L2 b b a a a b 0 1 0 1 a b
L1L2 b b a a a b ϵ 0 1 2 3 a b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 33 / 56 • What if there were more than one accepting states?
Introduction DFA NFA Regular languages Minimization Regular expressions Kleene star
∗ L1 L1
b b b b a a 0 1 0 1 a a ϵ
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 34 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Kleene star
∗ L1 L1
b b b b a a 0 1 0 1 a a ϵ
• What if there were more than one accepting states?
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 34 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Reversal
R L1 L1
b b b b a a 0 1 0 1 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 35 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Complement
L1 L1
b b b b a a 0 1 0 1 a a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 36 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Union
L1 ∪ L2
b b a
ϵ 01 11 a a a 0’ b
ϵ 02 12 b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 37 / 56 …or
L1 ∩ L2 = L1 ∪ L2
Introduction DFA NFA Regular languages Minimization Regular expressions Intersection a a L2 b 0 1 b
L1 b b 1 10 11 b
a a a a a a
b b 0 00 01 b L1 ∩ L2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 38 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Intersection a a L2 b 0 1 b
L1 b b 1 10 11 …or b
L1 ∩ L2 = L1 ∪ L2 a a a a a a
b b 0 00 01 b L1 ∩ L2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 38 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Closure properties of regular languages
• Since results of all the operations we studied are FSA: Regular languages are closed under – Concatenation – Kleene star – Reversal – Complement – Union – Intersection
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 39 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Is a language regular? — or not
• To show that a language is regular, it is sufficient to find an FSA that recognizes it. • Showing that a language is not regular is more involved • We will study a method based on pumping lemma
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 40 / 56 • Any FSA generating an infinite language has to have a loop (application of recursive rule(s) in the grammar) • Part of every string longer than some number will include repetition of the same substring (‘cklm’ above)
Introduction DFA NFA Regular languages Minimization Regular expressions Pumping lemma intuition
l m k a b c d e
• What is the length of longest string generated by this FSA?
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 41 / 56 • Any FSA generating an infinite language has to have a loop (application of recursive rule(s) in the grammar) • Part of every string longer than some number will include repetition of the same substring (‘cklm’ above)
Introduction DFA NFA Regular languages Minimization Regular expressions Pumping lemma intuition
l m k a b c d e
• What is the length of longest string generated by this FSA?
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 41 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Pumping lemma intuition
l m k a b c d e
• What is the length of longest string generated by this FSA? • Any FSA generating an infinite language has to have a loop (application of recursive rule(s) in the grammar) • Part of every string longer than some number will include repetition of the same substring (‘cklm’ above)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 41 / 56 v
l m k a b c d e
u w
Introduction DFA NFA Regular languages Minimization Regular expressions Pumping lemma definition For every regular language L, there exist an integer p such that a string x ∈ L can be factored as x = uvw, • uviw ∈ L, ∀i ⩾ 0 • v ≠ ϵ • |uv| ⩽ p
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 42 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Pumping lemma definition For every regular language L, there exist an integer p such that a string x ∈ L can be factored as x = uvw, • uviw ∈ L, ∀i ⩾ 0 • v ≠ ϵ • |uv| ⩽ p v
l m k a b c d e
u w
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 42 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions How to use pumping lemma
• We use pumping lemma to prove that a language is not regular • Proof is by contradiction: – Assume the language is regular – Find a string x in the language, for all splits of x = uvw, at least one of the pumping lemma conditions does not hold • uviw ∈ L (∀i ⩾ 0) • v ≠ ϵ • |uv| ⩽ p
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 43 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Pumping lemma example prove L = anbn is not regular • Assume L is regular: there must be a p such that, if uvw is in the language 1. uviw ∈ L (∀i ⩾ 0) 2. v ≠ ϵ 3. |uv| ⩽ p • Pick the string apbp • For the sake of example, assume p = 5, x = aaaaabbbbb • Three different ways to split
|{z}a |aaa{z} |abbbbb{z } violates 1 u v w |aaaa{z } |{z}ab |bbbb{z } violates 1 & 3 u v w |aaaaab{z } |{z}bbb |{z}b violates 1 & 3 u v w
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 44 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions DFA minimization
• For any regular language, there is a unique minimal DFA • By finding the minimal DFA, we can also prove equivalence (or not) of different FSA • In general the idea is: – Throw away unreachable states (easy) – Merge equivalent states • There are two well-known algorithms for minimization: – Hopcroft’s algorithm: find and eliminate equivalent states by partitioning the set of states – Brzozowski’s algorithm: ‘double reversal’
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 45 / 56 a, b b, c
c
a
The edges leaving the group of nodes are identical. Their right languages are the same.
Introduction DFA NFA Regular languages Minimization Regular expressions Finding equivalent states Intuition
c c b 2 a 4 c a c 0 b b c b 5 c a b a 1 a 3 b
a
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 46 / 56 a, b b, c
c
a
Introduction DFA NFA Regular languages Minimization Regular expressions Finding equivalent states Intuition
c c b 2 a 4 c a c 0 b b c b 5 c a b a 1 a 3 b
a
The edges leaving the group of nodes are identical. Their right languages are the same.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 46 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Finding equivalent states Intuition a, b b, c c c b 2 a 4 c a c 0 b b c b 5 c a b a 1 a 3 b a a
The edges leaving the group of nodes are identical. Their right languages are the same.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 46 / 56 • Accepting & non-accepting states form a partition
Q2 = {0, 1, 2, 3}, Q2 = {4, 5} a • If any two nodes go to different sets for any of the symbols split b b 2 • Q1 = {0, 3}, Q3 = {1}, Q4 = {2}, Q2 = {4, 5} a 03 1 a b • Stop when we cannot split any of the sets, merge the indistinguishable states a 45
b
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning a a b 0 1 2 a a b b a a b
3 4 b 5 b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 47 / 56 a • If any two nodes go to different sets for any of the symbols split b b 2 • Q1 = {0, 3}, Q3 = {1}, Q4 = {2}, Q2 = {4, 5} a 03 1 a b • Stop when we cannot split any of the sets, merge the indistinguishable states a 45
b
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning a a b 0 1 2 a a b b a a b • Accepting & non-accepting states form a partition 3 4 b 5 Q2 = {0, 1, 2, 3}, Q2 = {4, 5} b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 47 / 56 a b b 2 • Q1 = {0, 3}, Q3 = {1}, Q4 = {2}, Q2 = {4, 5} a 03 1 a b • Stop when we cannot split any of the sets, merge the indistinguishable states a 45
b
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning a a b 0 1 2 a a b b a a b • Accepting & non-accepting states form a partition 3 4 b 5 Q2 = {0, 1, 2, 3}, Q2 = {4, 5} b • If any two nodes go to different sets for any of the symbols split
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 47 / 56 a b b 2 a 03 1 a b • Stop when we cannot split any of the sets, merge the indistinguishable states a 45
b
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning a a b 0 1 2 a a b b a a b • Accepting & non-accepting states form a partition 3 4 b 5 Q2 = {0, 1, 2, 3}, Q2 = {4, 5} b • If any two nodes go to different sets for any of the symbols split
• Q1 = {0, 3}, Q3 = {1}, Q4 = {2}, Q2 = {4, 5}
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 47 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning a a b 0 1 2 a a b b a a b • Accepting & non-accepting states form a partition 3 4 b 5 Q2 = {0, 1, 2, 3}, Q2 = {4, 5} b a • If any two nodes go to different sets for any of the symbols split b b 2 • Q1 = {0, 3}, Q3 = {1}, Q4 = {2}, Q2 = {4, 5} a 03 1 a b • Stop when we cannot split any of the sets, merge the indistinguishable states a 45
b
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 47 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b
3 4 b 5 1 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b
3 4 b 5 1 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b
3 4 b 5 1 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b 1 3 4 b 5 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b 1 3 4 b 5 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b 1 3 4 b 5 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b 1 3 4 b 5 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b 1 3 4 b 5 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b 1 3 4 b 5 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 a b b 2 a 03 1 a b
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b 1 3 4 b 5 b 2 3 4 5 0 1 2 3 4
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 • The algorithm can be improved by choosing which cell to visit carefully
Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b
3 4 b 5 1 2 b a 3 b b 2 4 a 5 03 1 a b 0 1 2 3 4
a 45 • Merge indistinguishable states
b Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Minimization by partitioning tabular version a a b • Create a state-by-state table, mark distinguishable , , , 0 1 2 pairs: (q1, q2) such that (∆(q1 x) ∆(q2 x)) is a distinguishable pair for any x ∈ Σ a a b b a a b
3 4 b 5 1 2 b a 3 b b 2 4 a 5 03 1 a b 0 1 2 3 4
a 45 • Merge indistinguishable states • The algorithm can be improved by choosing which b cell to visit carefully Ç. Çöltekin, SfS / University of Tübingen WS 19–20 48 / 56 b 1 3 )
M b a, b ( a r
0 a 2 ∅ ∅ ))) )) M M (
( a b a b b
b r r ( ( d d ( 01 a 2 r 01 a 2 b d(r(d(r(M)))) a a, b 01 2 ∅
Introduction DFA NFA Regular languages Minimization Regular expressions Brzozowski’s algorithm double reverse (r), determinize (d) b 1 3 a, b M b a
0 a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 49 / 56 ∅ ∅ ))) )) M M (
( a b a b b
b r r ( ( d d ( 01 a 2 r 01 a 2 b d(r(d(r(M)))) a a, b 01 2 ∅
Introduction DFA NFA Regular languages Minimization Regular expressions Brzozowski’s algorithm double reverse (r), determinize (d) b 1 3 b 1 3 )
a, b M a, b M b b a ( a r
0 a 2 0 a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 49 / 56 ∅ ))) M ( a b b r ( d ( r 01 a 2 b d(r(d(r(M)))) a a, b 01 2 ∅
Introduction DFA NFA Regular languages Minimization Regular expressions Brzozowski’s algorithm double reverse (r), determinize (d) b 1 3 b 1 3 )
a, b M a, b M b b a ( a r
0 a 2 0 a 2 ∅ )) M
( a b b r ( d 01 a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 49 / 56 b d(r(d(r(M)))) a a, b 01 2 ∅
Introduction DFA NFA Regular languages Minimization Regular expressions Brzozowski’s algorithm double reverse (r), determinize (d) b 1 3 b 1 3 )
a, b M a, b M b b a ( a r
0 a 2 0 a 2 ∅ ∅ ))) )) M M (
( a b a b b
b r r ( ( d d ( 01 a 2 r 01 a 2
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 49 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Brzozowski’s algorithm double reverse (r), determinize (d) b 1 3 b 1 3 )
a, b M a, b M b b a ( a r
0 a 2 0 a 2 ∅ ∅ ))) )) M M (
( a b a b b
b r r ( ( d d ( 01 a 2 r 01 a 2 b d(r(d(r(M)))) a a, b 01 2 ∅
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 49 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Minimization algorithms final remarks
• There are many versions of the ‘partitioning’ algorithm. General idea is to form equivalence classes based on right-language of each state. • Partitioning algorithm has O(n log n) complexity • ‘Double reversal’ algorithm has exponential worst-time complexity • Double reversal algorithm can also be used with NFAs (resulting in the minimal equivalent DFA – NFA minimization is intractable) • In practice, there is no clear winner, different algorithms run faster on different input
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 50 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Regular expressions • Another way to specify a regular language (RL) is use of regular expressions (RE) • Every RL can be expressed by a RE, and every RE defines a RL • A RE x defines a RL L(x) • Relations between RE and RL – L(∅) = ∅, – L(a|b) = L(a) ∪ L(b) – L(ϵ) = ϵ, (some author use the notation a+b, – L(a) = a we will use a|b as in many practical – L(ab) = L(a)L(b) implementations) – L(a*) = L(a)∗ where, a, b ∈ Σ, ϵ is empty string, ∅ is the language that accepts nothing (e.g., ∗ ∗ Σ − Σ ) • Note: no standard complement operation in RE
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 51 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Regular some extensions
• Kleene star (a*), Concatenation (ab) and union (a|b) are the common operations • Parentheses can be used to group the sub-expressions. Otherwise, the priority of the operators as specified above a|bc* = a|(b(c*)) • In practice some short-hand notations are common
– . = (a1|...|an), – [^a-c] = . - (a|b|c) for Σ = {a ,..., a } 1 n – \d = (0|1|...|8|9) – a+ = aa* – [a-c] = (a|b|c) – … • And some non-regular extensions, like (a*)b\1 (sometimes the term regexp is used for expressions with non-regular extensions)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 52 / 56 = a(ϵ|b*) = ab*
a|ab* = aϵ|ab*
An exercise Simplify a|ab*
Introduction DFA NFA Regular languages Minimization Regular expressions Some properties of regular expressions Kleene algebra These identities are often used to simplify regular expressions. • ϵu = u • u(v|w) = uv|uw • ∅u = ∅ • • u(vw) = (uv)w (u|v)* = (u*|v*)* • ∅* = ϵ • ϵ* = ϵ • (u*)* = u* • u|v = v|u • u|u = u • u|∅ = u • u|ϵ = u • u|(v|w) = (u|v)|w
Note: most of these follow from set theory, and some can be derived from others.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 53 / 56 = a(ϵ|b*) = ab*
a|ab* = aϵ|ab*
Introduction DFA NFA Regular languages Minimization Regular expressions Some properties of regular expressions Kleene algebra These identities are often used to simplify regular expressions. • ϵu = u • u(v|w) = uv|uw • ∅u = ∅ • • u(vw) = (uv)w (u|v)* = (u*|v*)* • ∅* = ϵ • ϵ* = ϵ An exercise • (u*)* = u* Simplify a|ab* • u|v = v|u • u|u = u • u|∅ = u • u|ϵ = u • u|(v|w) = (u|v)|w
Note: most of these follow from set theory, and some can be derived from others.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 53 / 56 = a(ϵ|b*) = ab*
Introduction DFA NFA Regular languages Minimization Regular expressions Some properties of regular expressions Kleene algebra These identities are often used to simplify regular expressions. • ϵu = u • u(v|w) = uv|uw • ∅u = ∅ • • u(vw) = (uv)w (u|v)* = (u*|v*)* • ∅* = ϵ • ϵ* = ϵ An exercise • (u*)* = u* Simplify a|ab* • u|v = v|u a|ab* = aϵ|ab* • u|u = u • u|∅ = u • u|ϵ = u • u|(v|w) = (u|v)|w
Note: most of these follow from set theory, and some can be derived from others.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 53 / 56 = ab*
Introduction DFA NFA Regular languages Minimization Regular expressions Some properties of regular expressions Kleene algebra These identities are often used to simplify regular expressions. • ϵu = u • u(v|w) = uv|uw • ∅u = ∅ • • u(vw) = (uv)w (u|v)* = (u*|v*)* • ∅* = ϵ • ϵ* = ϵ An exercise • (u*)* = u* Simplify a|ab* • u|v = v|u a|ab* = aϵ|ab* • u|u = u = a(ϵ|b*) • u|∅ = u • u|ϵ = u • u|(v|w) = (u|v)|w
Note: most of these follow from set theory, and some can be derived from others.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 53 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Some properties of regular expressions Kleene algebra These identities are often used to simplify regular expressions. • ϵu = u • u(v|w) = uv|uw • ∅u = ∅ • • u(vw) = (uv)w (u|v)* = (u*|v*)* • ∅* = ϵ • ϵ* = ϵ An exercise • (u*)* = u* Simplify a|ab* • u|v = v|u a|ab* = aϵ|ab* • u|u = u = a(ϵ|b*) ∅ • u| = u = ab* • u|ϵ = u • u|(v|w) = (u|v)|w
Note: most of these follow from set theory, and some can be derived from others.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 53 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Converting regular expressions to FSA
Converting to NFA is easy: ab a b • For more complex expressions, one can 0 2 3 replace the paths for individual symbols a* a with corresponding automata • Using ϵ transitions may be ease the task 0 • The reverse conversion (from automata to regular expressions) is also easy: a|b b – identify the patterns on the left, collapse a paths to single transitions with regular 0 2 3 expressions Note the similarity with operations on regular languages discussed earlier.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 54 / 56 An exercise: simplify the resulting regular expressions
Introduction DFA NFA Regular languages Minimization Regular expressions Converting FSA to regular expressions Converting an FSA to a regular expression is also easy:
(b|bb)(ab)∗b b(ab)∗aaa(ab|b)(∗abb )∗b|b b a ab a b ∗ | ∗| | ∗ | ∗ | ∗ | ∗ a ((b bb)b(abbb()bbbbba)()(abbb()abb)baaa b(ab) b b) 0 1 2 3 b a aa
a a(ab)∗aa ba
• The general idea: remove (intermediate) states, replacing edge labels with regular expressions
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 55 / 56 An exercise: simplify the resulting regular expressions
Introduction DFA NFA Regular languages Minimization Regular expressions Converting FSA to regular expressions Converting an FSA to a regular expression is also easy:
(b|bb)(ab)∗b b(ab)∗aaa(ab|b)(∗abb )∗b|b b a ab a b ∗ | ∗| | ∗ | ∗ | ∗ | ∗ a ((b bb)b(abbb()bbbbba)()(abbb()abb)baaa b(ab) b b) 0 1 2 3 b a aa
a a(ab)∗aa ba
• The general idea: remove (intermediate) states, replacing edge labels with regular expressions
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 55 / 56 An exercise: simplify the resulting regular expressions
Introduction DFA NFA Regular languages Minimization Regular expressions Converting FSA to regular expressions Converting an FSA to a regular expression is also easy:
(b|bb)(ab)∗b b(ab)∗aaa(ab|b)(∗abb )∗b|b b a ab a b ∗ | ∗| | ∗ | ∗ | ∗ | ∗ a ((b bb)b(abbb()bbbbba)()(abbb()abb)baaa b(ab) b b) 0 1 2 3 b a aa
a a(ab)∗aa ba
• The general idea: remove (intermediate) states, replacing edge labels with regular expressions
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 55 / 56 An exercise: simplify the resulting regular expressions
Introduction DFA NFA Regular languages Minimization Regular expressions Converting FSA to regular expressions Converting an FSA to a regular expression is also easy:
(b|bb)(ab)∗b b(ab)∗aaa(ab|b)(∗abb )∗b|b b a ab a b ∗ | ∗| | ∗ | ∗ | ∗ | ∗ a ((b bb)b(abbb()bbbbba)()(abbb()abb)baaa b(ab) b b) 0 1 2 3 b a aa
a a(ab)∗aa ba
• The general idea: remove (intermediate) states, replacing edge labels with regular expressions
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 55 / 56 An exercise: simplify the resulting regular expressions
Introduction DFA NFA Regular languages Minimization Regular expressions Converting FSA to regular expressions Converting an FSA to a regular expression is also easy:
(b|bb)(ab)∗b b(ab)∗aaa(ab|b)(∗abb )∗b|b b a ab a b ∗ | ∗| | ∗ | ∗ | ∗ | ∗ a ((b bb)b(abbb()bbbbba)()(abbb()abb)baaa b(ab) b b) 0 1 2 3 b a aa
a a(ab)∗aa ba
• The general idea: remove (intermediate) states, replacing edge labels with regular expressions
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 55 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Converting FSA to regular expressions Converting an FSA to a regular expression is also easy:
(b|bb)(ab)∗b b(ab)∗aaa(ab|b)(∗abb )∗b|b b a ab a b ∗ | ∗| | ∗ | ∗ | ∗ | ∗ a ((b bb)b(abbb()bbbbba)()(abbb()abb)baaa b(ab) b b) 0 1 2 3 b a aa
a a(ab)∗aa ba
• The general idea: remove (intermediate) states, replacing edge labels with regular expressions An exercise: simplify the resulting regular expressions
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 55 / 56 Next: • Finite state transducers (FSTs) • Applications of FSA and FSTs
Introduction DFA NFA Regular languages Minimization Regular expressions Wrapping up • FSA and regular expressions express regular languages • FSA have two flavors: DFA, NFA (or maybe three: ϵ-NFA) • DFA recognition is linear • Any NFA can be converted to a DFA (in worst case number of nodes increase exponentially) • Regular languages and FSA are closed under – Concatenation – Reversal – Kleene star – Union – Complement – Intersection • Every FSA has a unique minimal DFA
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 56 / 56 Introduction DFA NFA Regular languages Minimization Regular expressions Wrapping up • FSA and regular expressions express regular languages • FSA have two flavors: DFA, NFA (or maybe three: ϵ-NFA) • DFA recognition is linear • Any NFA can be converted to a DFA (in worst case number of nodes increase exponentially) • Regular languages and FSA are closed under – Concatenation – Reversal – Kleene star – Union – Complement – Intersection • Every FSA has a unique minimal DFA Next: • Finite state transducers (FSTs) • Applications of FSA and FSTs
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 56 / 56 References References / additional reading material
• Hopcroft and Ullman (1979, Ch. 2&3) (and its successive editions) covers (almost) all topics discussed here • Jurafsky and Martin (2009, Ch. 2) • Other textbook references include: – Sipser (2006) – Kozen (2013)
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 A.1 References References / additional reading material (cont.)
Hopcroft, John E. and Jeffrey D. Ullman (1979). Introduction to Automata Theory, Languages, and Computation. Addison-Wesley Series in Computer Science and Information Processing. Addison-Wesley. isbn: 9780201029888. Jurafsky, Daniel and James H. Martin (2009). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. second. Pearson Prentice Hall. isbn: 978-0-13-504196-3. Kozen, Dexter C. (2013). Automata and Computability. Undergraduate Texts in Computer Science. Berlin Heidelberg: Springer. Sipser, Michael (2006). Introduction to the Theory of Computation. second. Thomson Course Technology. isbn: 0-534-95097-3.
Ç. Çöltekin, SfS / University of Tübingen WS 19–20 A.2