Introduction to the theory of complexity Daniel Pierre Bovet Pierluigi Crescenzi The information in this book is distributed on an “As is” basis, without warranty. Although every precaution has been taken in the preparation of this work, the authors shall not have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. First electronic edition: June 2006 Contents 1 Mathematical preliminaries 1 1.1 Sets, relations and functions 1 1.2 Set cardinality 5 1.3 Three proof techniques 5 1.4 Graphs 8 1.5 Alphabets, words and languages 10 2 Elements of computability theory 12 2.1 Turing machines 13 2.2 Machines and languages 26 2.3 Reducibility between languages 28 3 Complexity classes 33 3.1 Dynamic complexity measures 34 3.2 Classes of languages 36 3.3 Decision problems and languages 38 3.4 Time-complexity classes 41 3.5 The pseudo-Pascal language 47 4 The class P 51 4.1 The class P 52 4.2 The robustness of the class P 57 4.3 Polynomial-time reducibility 60 4.4 Uniform diagonalization 62 5 The class NP 69 5.1 The class NP 70 5.2 NP-complete languages 72 v vi 5.3 NP-intermediate languages 88 5.4 Computing and verifying a function 92 5.5 Relativization of the P 6= NP conjecture 95 6 The complexity of optimization problems 110 6.1 Optimization problems 111 6.2 Underlying languages 115 6.3 Optimum measure versus optimum solution 117 6.4 Approximability 119 6.5 Reducibility and optimization problems 125 7 Beyond NP 133 7.1 The class coNP 134 7.2 The Boolean hierarchy 140 7.3 The polynomial hierarchy 142 7.4 Exponential-time complexity classes 150 8 Space-complexity classes 156 8.1 Space-complexity classes 157 8.2 Relations between time and space 157 8.3 Nondeterminism, determinism and space 160 8.4 Nondeterminism, complement and space 162 8.5 Logarithmic space 164 8.6 Polynomial space 171 9 Probabilistic algorithms and complexity classes 178 9.1 Some probabilistic algorithms 179 9.2 Probabilistic Turing machines 188 9.3 Probabilistic complexity classes 194 10 Interactive proof systems 203 10.1 Interactive proof systems 204 10.2 The power of IP 208 10.3 Probabilistic checking of proofs 214 11 Models of parallel computers 221 11.1 Circuits 222 11.2 The PRAM model 230 11.3 PRAM memory conflicts 232 11.4 A comparison of the PRAM models 233 11.5 Relations between circuits and PRAMs 238 11.6 The parallel computation thesis 240 vii 12 Parallel algorithms 246 12.1 The class NC 247 12.2 Examples of NC problems 250 12.3 Probabilistic parallel algorithms 256 12.4 P-complete problems revisited 259 Preface The birth of the theory of computational complexity can be set in the early 1960s when the first users of electronic computers started to pay increasing attention to the performances of their programs. As in the theory of computation, where the concept of a model of computation had led to that of an algorithm and of an algo- rithmically solvable problem, similarly, in the theory of computational complexity, the concept of resource used by a computation led to that of an efficient algorithm and of a computationally feasible problem. Since these preliminary stages, many more results have been obtained and, as stated by Hartmanis (1989), ‘the systematic study of computational complexity theory has developed into one of the central and most active research areas of computer science. It has grown into a rich and exciting mathematical theory whose development is motivated and guided by computer science needs and technological advances.’ The aim of this introductory book is to review in a systematic way the most significant results obtained in this new research area. The main goals of compu- tational complexity theory are to introduce classes of problems which have similar complexity with respect to a specific computation model and complexity measure, and to study the intrinsic properties of such classes. In this book, we will follow a balanced approach which is partly algorithmic and partly structuralist. From an algorithmic point of view, we will first present some ‘natural’ problems and then illustrate algorithms which solve them. Since the aim is merely to prove that the problem belongs to a specific class, we will not always give the most efficient algorithm and we will occasionally give preference to an algorithm which is simpler to describe and analyse. From a structural point of view, we will be concerned with intrinsic properties of complexity classes, including relationships between classes, implications between several hypotheses about complexity classes, and identification of structural prop- erties of sets that affect their computational complexity. The reader is assumed to have some basic knowledge of theory of computation (as ix x Preface taught in an undergraduate course on Automata Theory, Logic, Formal Languages Theory, or Theory of Computation) and of programming languages and techniques. Some mathematical knowledge is also required. The first eight chapters of the book can be taught on a senior undergraduate course. The whole book together with an exhaustive discussion of the problems should be suitable for a postgraduate course. Let us now briefly review the contents of the book and the choices made in selecting the material. The first part (Chapters 1-3) provide the basic tools which will enable us to study topics in complexity theory. Chapter 1 includes a series of definitions and notations related to classic mathematical concepts such as sets, relationships and languages (this chapter can be skipped and referred to when needed). Chapter 2 reviews some important results of computability theory. Chapter 3 provides the basic tool of complexity theory: dynamic complexity measures are introduced, the concept of classes of languages is presented, the strict correspondence between such classes and decision problems is established, and techniques used to study the properties of such classes are formulated. The second part (Chapters 4-8) studies, in a detailed way, the properties of some of the most significant complexity classes. Those chapters represent the ‘heart’ of complexity theory: by placing suitable restrictions on the power of the computation model, and thus on the amount of resources allowed for the computation, it becomes possible to define a few fundamental complexity classes and to develop a series of tools enabling us to identify, for most computational problems, the complexity class to which they belong. The third part (Chapters 9-10) deals with probabilistic algorithms and with the corresponding complexity classes. Probabilistic Turing machines are introduced in Chapter 9 and a few probabilistic algorithms for such machines are analysed. In Chapter 10, a more elaborate computation model denoted as interactive proof system is considered and a new complexity class based on such a model is studied. The last part (Chapters 11 and 12) is dedicated to the complexity of parallel computations. As a result of advances in hardware technology, computers with thousands of processors are now available; it thus becomes important, not only from a theoretical point of view but also from a practical one, to be able to specify which problems are best suited to be run on parallel machines. Chapter 11 describes in detail a few important and widely differing models of parallel computers and shows how their performance can be considered roughly equivalent. Chapter 12 introduces the concept of a problem solvable by a fast parallel algorithm and the complementary one of a problem with no fast parallel algorithm, and illustrates examples of both types of problems. While selecting material to be included in the book, we followed a few guidelines. First, we have focused our attention on results obtained in the past two decades, mentioning without proof or leaving as problems some well-known results obtained in the 1960s. Second, whenever a proof of a theorem uses a technique described in a previous proof, we have provided an outline, leaving the complete proof as a Preface xi problem for the reader. Finally, we have systematically avoided stating without proof specialistic results in other fields in order to make the book as self-contained as possible. Acknowledgements This book originated from a course on Algorithms and Complexity given at the University of Rome ‘La Sapienza’ by D.P. Bovet and P. Crescenzi since 1986. We would like to thank the students who were exposed to the preliminary versions of the chapters and who contributed their observations to improve the quality of the presentation. We also would like to thank R. Silvestri for pointing out many corrections and for suggesting simpler and clearer proofs of some results. Chapter 1 Mathematical preliminaries In this chapter some preliminary definitions, notations and proof techniques which are going to be used in the rest of the book will be introduced. 1.1 Sets, relations and functions Intuitively, a set A is any collection of elements. If a, b and c are arbitrary elements, then the set A consisting of elements a, b and c is represented as A = {a, b, c}. A set cannot contain more than one copy or instance of the same element; furthermore, the order in which the elements of the set appear is irrelevant. Thus we define two sets A and B as equal (in symbols, A = B) if every element of A is also an element of B and vice versa. Two sets A and B are not equal (in symbols, A 6= B) when A = B does not hold true. The symbols ∈ and 6∈ denote, respectively, the fact that an element belongs or does not belong to a set.