Theory of Computation Class Notes1
Total Page:16
File Type:pdf, Size:1020Kb
Theory of Computation Class Notes1 1based on the books by Sudkamp and by Hopcroft, Motwani and Ullman ii Contents 1 Introduction 1 1.1 Sets . 1 1.2 Functions and Relations . 3 1.3 Countable and uncountable sets . 5 1.4 Proof Techniques . 5 2 Languages and Grammars 9 2.1 Languages . 9 2.2 Regular Expressions . 12 2.3 Grammars . 13 2.4 Classification of Grammars and Languages . 18 2.5 Normal Forms of Context-Free Grammars . 19 2.5.1 Chomsky Normal Form (CNF) . 19 2.5.2 Greibach Normal Form (GNF) . 19 3 Finite State Automata 21 iii iv CONTENTS List of Figures 2.1 Derivation tree . 15 v vi LIST OF FIGURES Chapter 1 Introduction 1.1 Sets A set is a collection of elements. To indicate that x is an element of the set S, we write x S. The statement that x is not in S is written as x = S. A set is specified by enclosing some description2 of its elements in curly braces; for example, the2set of all natural numbers 0; 1; 2; is denoted by · · · = 0; 1; 2; 3; : N f · · ·g We use ellipses (i.e.,: : :) when the meaning is clear, thus n = 1; 2; 3; ; n represents the set of all natural numbers from 1 to n. J f · · · g When the need arises, we use more explicit notation, in which we write S = i i 0; i is even f j ≥ g for the last example. We read this as \S is the set of all i, such that i is greater than zero, and i is even." Considering a \universal set" U; the complement S¯ of S is defined as S¯ = x x U x = S f j 2 ^ 2 g The usual set operations are union ( ), intersection ( ), and difference( ), defined as [ \ − S S = x x S x S 1 [ 2 f j 2 1 _ 2 2g S S = x x S x S 1 \ 2 f j 2 1 ^ 2 2g S S = x x S x = S 1 − 2 f j 2 1 ^ 2 2g The set with no elements, called the empty set is denoted by . It is obvious that ; S = S = S [ ; − ; S = \ ; ; ¯ = U ; S¯ = S A set S1 is said to be a subset of S if every element of S1 is also an element of S. We write this as S S 1 ⊆ If S1 S, but S contains an element not in S1, we say that S1 is a proper subset of S; we write this as ⊆ S S 1 ⊂ 1 2 CHAPTER 1. INTRODUCTION The following identities are known as the de Morgan's laws, 1. S S = S S ; 1 [ 2 1 \ 2 2. S S = S S ; 1 \ 2 1 [ 2 1. S S = S S ; 1 [ 2 1 \ 2 x S S 2 1 [ 2 x U and x = S S , 2 2 1 [ 2 x U and (x S or x S ) (def:union) , 2 : 2 1 2 2 x U and ( (x S ) and (x S )) (negation of disjunction) , 2 : 2 1 : 2 2 x U and (x = S and x = S ) , 2 2 1 2 2 (x U and x = S ) and (x U and x = S ) , 2 2 1 2 2 2 (x S and x S ) (def:complement) , 2 1 2 2 x S S (def:intersection) , 2 1 \ 2 If S1 and S2 have no common element, that is, S S = ; 1 \ 2 ; then the sets are said to be disjoint. A set is said to be finite if it contains a finite number of elements; otherwise it is infinite. The size of a finite set is the number of elements in it; this is denoted by S (or #S). A set may have many subsets. The set of all subsets of a setj Sj is called the power set of S and is denoted by 2S or P (S). Observe that 2S is a set of sets. Example 1.1.1 If S is the set 1; 2; 3 , then its power set is f g 2S = ; 1 ; 2 ; 3 ; 1; 2 ; 1; 3 ; 2; 3 ; 1; 2; 3 f; f g f g f g f g f g f g f gg Here S = 3 and 2S = 8. This is an instance of a general result, if S is finite, then j j j j 2S = 2jSj j j Proof: (By induction on the number of elements in S). Basis: S = 1 2S = ; S 2S = 21 = 2 j j ) f; g ) j j Induction Hypothesis: Assume the property holds for all sets S with k elements. Induction Step: Show that the property holds for (all sets with) k + 1 elements. Denote Sk = y ; y ; : : : ; yk +1 f 1 2 +1g = Sk yk [ f +1g 1.2. FUNCTIONS AND RELATIONS 3 where Sk = y ; y ; y ; : : : ; yk f 1 2 3 g Sk+1 Sk 2 = 2 yk [ f +1g y ; yk y ; yk : : : yk; yk [ f 1 +1g [ f 2 +1g [ [ f +1g[ x;y Sk x; y; yk : : : [ 2 f +1g [ Sk [ +1 2Sk has 2k elements by the induction hypothesis. Sk+1 k The number of sets in 2 which contain yk+1 is also 2 . Consequently 2Sk+1 = 2 2k = 2k+1: j j ∗ A set which has as its elements ordered sequences of elements from other sets is called the Cartesian product of the other sets. For the Cartesian product of two sets, which itself is a set of ordered pairs, we write S = S S = (x; y) x S ; y S 1 × 2 f j 2 1 2 2g Example 1.1.2 Let S = 1; 2 and S = 1; 2; 3 . Then 1 f g 2 f g S S = (1; 1); (1; 2); (1; 3); (2; 1); (2; 2); (2; 3) 1 × 2 f g Note that the order in which the elements of a pair are written matters; the pair (3; 2) is not in S S . 1 × 2 Example 1.1.3 If A is the set of throws of a coin, i.e., A = head,tail , then f g A A = (head,head),(head,tail),(tail,head),(tail,tail) × f g the set of all possible throws of two coins. The notation is extended in an obvious fashion to the Cartesian product of more than two sets; generally S S Sn = (x ; x ; ; xn) xi Si 1 × 2 × · · · × f 1 2 · · · j 2 g 1.2 Functions and Relations A function is a rule that assigns to elements of one set (the function domain) a unique element of another set (the range). We write f : S S 1 ! 2 to indicate that the domain of the function f is a subset of S1 and that the range of f is a subset of S2. If the domain of f is all of S1, we say that f is a total function on S1; otherwise f is said to be a partial function on S1: 1. Domain f = x S (x; y) f; for some y S = Df f 2 1 j 2 2 2g 2. Range f = y S (x; y) f; for some x S = Rf f 2 2 j 2 2 1g 4 CHAPTER 1. INTRODUCTION 3. The restriction of f to A S ; f = (x; y) f x A ⊆ 1 jA f 2 j 2 g 4. The inverse f −1 : S S is (y; x) (x; y) f 2 ! 1 f j 2 g 5. f : S S is called a function on S 1 ! 1 1 6. If x Df then f is defined at x; otherwise f is undefined at x; 2 7. f is a total function if Df = S1. 8. f is a partial function if Df S ⊆ 1 9. f is an onto function or surjection if Rf = S . If Rf S then f is a function from S (Df ) 2 ⊆ 2 1 into S2 10. f is a one to one function or injection if (f(x) = z and f(y) = z) x = y ) 11. A total function f is a bijection if it is both an injection and a surjection. A function can be represented by a set of pairs (x ; y ); (x ; y ); ; where each xi is an f 1 1 2 2 · · · g element in the domain of the function, and yi is the corresponding value in its range. For such a set to define a function, each xi can occur at most once as the first element of a pair. If this is not satisfied, such a set is called a relation. A specific kind of relation is an equivalence relation. A relation denoted r on X is an equivalence relation if it satisfies three rules, the reflexivity rule: (x; x) r x 2X 8 2 the symmetry rule: (x; y) r then (y; x) r 2 x; y X 2 8 2 and the transitivity rule: (x; y) r; (y; z) r then (x; z) r 2 x; y2; z X 2 8 2 An equivalence relation on X induces a partition on X into disjoint subsets called equivalence classes Xj; j Xj = X; such that elements from the same class belong to the relation, and any two elements taken[from different classes are not in the relation. Example 1.2.1 The relation congruence mod m (modulo m) on the set of the integers . i = j mod m if i j is divisible by m; is partitioned into m equivalenceZ classes: − Z ; 2m; m; 0; m; 2m; ; 2{·m· +· 1−; m−+ 1; 1; m + 1; 2·m· ·g+ 1; {· · · ; −2m + 2; −m + 2; 2; m + 2; 2m + 2; · · · g {· · · − − · · · g ; m 1; 1; m· · · 1; 2m; 3m 1; {· · · − − − − − · · · g 1.3.