Theory of Computation- Lecture Notes Michael Levet August 3, 2020 Contents 1 Mathematical Preliminaries 3 1.1 Set Theory . .3 1.2 Relations and Functions . .4 1.2.1 Functions . .4 1.2.2 Equivalence Relations . .6 1.3 Proof by Induction . .8 1.3.1 A Brief Review of Asymptotics . 11 1.4 Combinatorics and Graph Theory . 12 1.4.1 Basic Enumerative Techniques . 12 1.4.2 Combinatorial Proofs . 15 1.4.3 Graph Theory . 16 1.5 Number Theory . 20 1.6 Russell's Paradox and Cantor's Diagonal Argument . 24 2 Automata Theory 25 2.1 Regular Languages . 25 2.2 Finite State Automata . 26 2.3 Converting from Regular Expressions to -NFA........................... 31 2.4 Algebraic Structure of Regular Languages . 33 2.5 DFAs, NFAs, and -NFAs ....................................... 34 2.6 DFAs to Regular Expressions- Brzozowski's Algebraic Method . 37 2.7 Pumping Lemma for Regular Languages . 41 2.8 Closure Properties . 42 2.9 Myhill-Nerode and DFA Minimization . 44 3 More Group Theory (Optional) 48 3.1 Introductory Group Theory . 48 3.1.1 Introduction to Groups . 48 3.1.2 Dihedral Group . 50 3.1.3 Symmetry Group . 53 3.1.4 Group Homomorphisms and Isomorphisms . 55 3.1.5 Group Actions . 56 3.1.6 Algebraic Graph Theory- Cayley Graphs . 58 3.1.7 Algebraic Graph Theory- Transposition Graphs . 60 3.2 Subgroups . 61 3.2.1 Cyclic Groups . 64 3.2.2 Subgroups Generated By Subsets of a Group . 66 3.2.3 Subgroup Poset and Lattice (Hasse) Diagram . 66 3.3 Quotient Groups . 69 3.3.1 Introduction to Quotients . 69 3.3.2 Normal Subgroups and Quotient Groups . 70 3.3.3 More on Cosets and Lagrange's Theorem . 73 3.3.4 The Group Isomorphism Theorems . 76 3.3.5 Alternating Group . 79 3.3.6 Algebraic Graph Theory- Graph Homomorphisms . 81 1 3.3.7 Algebraic Combinatorics- The Determinant . 84 3.4 Group Actions . 84 3.4.1 Conjugacy . 84 3.4.2 Automorphisms of Groups . 88 3.4.3 Sylow's Theorems . 88 3.4.4 Applications of Sylow's Theorems . 91 3.4.5 Algebraic Combinatorics- P´olya Enumeration Theory . 93 4 Turing Machines and Computability Theory 93 4.1 Standard Deterministic Turing Machine . 93 4.2 Variations on the Standard Turing Machine . 96 4.3 Turing Machine Encodings . 98 4.4 Chomsky Heirarchy and Some Decidable Problems . 98 4.5 Undecidability . 101 4.6 Reducibility . 102 5 Complexity Theory 103 5.1 Time Complexity- P and NP ...................................... 104 5.2 NP-Completeness . 106 5.3 More on P and P-Completeness . 111 5.4 Closure Properties of NP and P .................................... 114 5.5 Structural Proofs for NP and P ..................................... 114 5.6 Ladner's Theorem . 114 5.6.1 Russell Impagliazzo's Proof of Ladner's Theorem . 117 5.7 PSPACE .................................................. 119 5.8 PSPACE-Complete . 119 2 1 Mathematical Preliminaries 1.1 Set Theory Definition 1 (Set). A set is collection of distinct elements, where the order in which the elements are listed does not matter. The size of a set S, denoted jSj, is known as its cardinality or order. The members of a set are referred to as its elements. We denote membership of x in S as x 2 S. Similarly, if x is not in S, we denote x 62 S. Example 1. Common examples of sets include the set of real numbers R; the set of rational numbers Q, and + + + the set of integers Z. The sets R ; Q and Z denote the strictly positive elements of the reals, rationals, + and integers respectively. We denote the set of natural numbers N = f0; 1;:::g. Let n 2 Z and denote [n] = f1; : : : ; ng. We now review several basic set operations, as well as the power set. It is expected that students will be familiar with these constructs. Therefore, we proceed briskly, recalling definitions and basic examples intended solely as a refresher. Definition 2. Set Union Let A; B be sets. Then the union of A and B, denoted A [ B is the set: A [ B := fx : x 2 A or x 2 Bg Example 2. Let A = f1; 2; 3g and B = f4; 5; 6g. Then A [ B = f1; 2; 3; 4; 5; 6g. Example 3. Let A = f1; 2; 3g and B = f3; 4; 5g. So A [ B = f1; 2; 3; 4; 5g. Recall that sets do not contain duplicate elements. So even though 3 appears in both A and B, 3 occurs exactly once in A [ B. Definition 3. Set Intersection Let A; B be sets. Then the intersection of A and B, denoted A \ B is the set: A \ B := fx : x 2 A and x 2 Bg Example 4. Let A = f1; 2; 3g and B = f1; 3; 5g. Then A \ B = f1; 3g. Now let C = f4g. So A \ C = ;. Definition 4 (Symmetric Difference). Let A; B be sets. Then the symmetric difference of A and B, denoted A4B is the set: A4B := fx : x 2 A or x 2 B, but x 62 A \ Bg Example 5. Let A = f1; 2; 3g and B = f1; 3; 5g. Then A4B = f2; 5g. For our next two definitions, we let U be our universe. That is, let U be a set. Any sets we consider are subsets of U. Definition 5 (Set Complementation). Let A be a set contained in our universe U. The complement of A, denoted AC or A, is the set: A := fx 2 U : x 62 Ag Example 6. Let U = [5], and let A = f1; 2; 4g. Then A = f3; 5g. Definition 6 (Set Difference). Let A; B be sets contained in our universe U. The difference of A and B, denoted A n B or A − B, is the set: A n B = fx : x 2 A and x 62 Bg Example 7. Let U = [5], A = f1; 2; 3g and B = f1; 2g. Then A n B = f3g. Remark: The Set Difference operation is frequently known as the relative complement, as we are taking the complement of B relative to A rather than with respect to the universe U. Definition 7 (Cartesian Product). Let A; B be sets. The Cartesian product of A and B, denoted A × B, is the set: A × B := f(a; b): a 2 A; b 2 Bg Example 8. Let A = f1; 2; 3g and B = fa; bg. Then A × B = f(1; a); (1; b); (2; a); (2; b); (3; a); (3; b)g. 3 Definition 8 (Power Set). Let S be a set. The power set of S, denoted 2S or P(S), is the set of all subsets of S. Formally: 2S := fA : A ⊂ Sg Example 9. Let S = f1; 2; 3g. So 2S = f;; f1g; f2g; f3g; f1; 2g; f1; 3g; f2; 3g; f1; 2; 3gg. Remark: For finite sets S, j2Sj = 2jSj; hence, the choice of notation. Definition 9 (Subset). Let A; B be sets. A is said to be a subset of B if for every x 2 A, we have x 2 B as well. This is denoted A ⊂ B (equivocally, A ⊆ B). Note that B is a superset of A. Example 10. Let A = [3];B = [6];C = f2; 3; 5g. So we have A ⊂ B and C ⊂ B. However, A 6⊂ C as 1 62 C; and C 6⊂ A, as 5 62 A. Remark: Let S be a set. The subset relation forms a partial order on 2S. To show two sets A and B are equal, we must show A ⊂ B and B ⊂ A. We demonstrate how to prove two sets are equal below. Proposition 1.1. Let A = f6n : n 2 Zg;B = f2n : n 2 Zg;C = f3n : n 2 Zg. So A = B \ C. Proof. We first show that A ⊂ B \ C. Let n 2 Z. So 6n 2 A. We show 6n 2 B \ C. As 2 is a factor of 6, 6n = 2 · (3n) 2 B. Similarly, as 3 is a factor of 6, 6n = 3 · (2n) 2 C. So 6n 2 B \ C. We now show that B \ C ⊂ A. Let x 2 B \ C. Let n1; n2 2 Z such that x = 2n1 = 3n2. As 2 is a factor of x and 3 is a factor of x, it follows that 2 · 3 = 6 is also a factor of x. Thus, x = 6n3 for some n3 2 Z. So x 2 A. Thus, B \ C ⊂ A. Thus, A = B \ C, as desired. Proposition 1.2. Let A; B; C be sets. Then A × (B [ C) = (A × B) [ (A × C). Proof. Let (x; y) 2 A × (B [ C). If y 2 B, then (x; y) 2 (A × B). Otherwise, y 2 C and so (x; y) 2 (A × C). Thus, A × (B [ C) ⊂ (A × B) [ (A × C). Now let (d; f) 2 (A × B) [ (A × C). Clearly, d 2 A. So f must be in either B or C. Thus, (d; f) 2 A × (B [ C), which implies (A × B) [ (A × C) ⊂ A × (B [ C). We conclude that A × (B [ C) = (A × B) [ (A × C). 1.2 Relations and Functions Definition 10 (Relation). Let X be a set. A k-ary relation on X is a subset R ⊂ Xk. Example 11. The notion of equality = over R is the canonical example of a relation. It is perhaps the most well-known instance of an equivalence relation, which will be discussed later. Intuitively, a k-ary relation R contains k-tuples of elements from X that share common properties.