The Complexity of Decision Problems in Automata Theory and Logic by Larry J. Stockmeyer ABSTRACT The inherent computational complexity of a variety of decision problems in mathematical logic and the theory of automata is analyzed in terms of Turing machine time and space and in terms of the complexity of Boolean networks. The problem of deciding whether a star-free expression (a variation of the regular expressions of Kleene used to describe languages accepted by finite automata) defines the empty set is shown to require time and space exceeding any composition of.functions exponential in the length of expressions. In particular, this decision problem is not elementary- recursive in the sense of Kalmar. The emptiness problem can be,reduced efficiently to decision problems for truth or satisfiability of sentences in the first order monadic theory of (N,<), the first order theory of linear orders, and the first order theory of two successors and prefix, among others. It follows that the decision problems for these theories are also not elementary-recursive. The number of Boolean operations and hence the size of logical circuits required to decide truth in several familiar logical theories of sentences only a few hundred characters long is shown to exceed the number of protons required to fill the known universe. The methods of proof are analogous to the arithmetizations and reducibility arguments of recursive function theory. Keywords: computational complexity, decision procedure star-free, Turing machine AM.(MOS) Subject Classification Scheme (1970) primary 68A20, 02G05 secondary 68A40, 94820 Table of Contents 1. Introduction 2, The Model of Computation 2.1 The Basic Model 2.2 A Technically Useful Model 3. Efficient Reduqibility 3,1 Definitions 3.2 Applications to Complexity Bounds 3.3 Other Applications 4. Regular-Like Expressions 4.1 Expressions With Squaring 4.2 Expressions With Complementation 4.3 (deleted) 4.4 Expressions Over a One-Letter Alphabet 5. Nonelementary Logical Theories 6. Complexity of Finite Problems 6.1 Second Order Theory of Successor 6.2 First Order Integer Arithmetic 7. Conclusion Bibliography Appendix I, Notation Appendix 11. Some Properties of logspace List of Figures Figure 4.1: E2 "matches" a word w Figure 6.1: P, B, and d Figure 6.2: Illustrating the proof . of Leuuna 6.5.2 (i) and (ii) 19 6 Figure 6.3: I and J "code" a circuit 19 8 Figure 6.4: The circuit Co Chapter 1. Introduction One major goal of computational complexity is to achieve the ability to characterize precisely the amount of computational resource needed to solve given computational problems or classes of problems. Two important kinds of computational resource are time and space, respectively the number of basic computational steps and the amount of memory used in solving the problem. The complexity of a particular problem can be characterized by upper and lower bounds on computatihal resources sufficient to solve the problem. Upper bounds are usually established by exhibiting a specific algorithm which solves the problem and whose time and/or space complexity can be bounded from above. Much progress has been made on this positive side of the complexity question. Many clever and efficient algorithms have been devised for performing a wide variety of computational tasks (cf. D.E. Knuth, ---The Art of Computer Programming). However the progress made on the negative side of the question has been less striking. In order to establish a lower bound or. the complexity of a particulzr problem, one must show that some mfnimum amount of resource (time or space) is always required no matter which of the infinitely many possible algorithms is used or how cleverly one writes the algorithm to solve the problem. It is this latter side of the complexity question which we address in thispaper . Although lower bound resuits are negative in nature, they have the value that they enable one to cease lookicg for efficient algorithm when none exist. Also, the exhibition of specific problems or classes of problems which are provably difficult may give insight into the "reasons" for their difficulty, and these "reasons" and proofs of difficulty may provide clues for reformulating the problems so that in revised form they become tractable. Let us now sketch a bit more precisely what we mean by "computational i- problem" and "algorithm" . Many computational problems can be viewed as problems of function evaluation. In particular, consider functions mapping strings of symbols to strings of symbols. As a concept of "algorithm" we could choose any one of a variety of universal computer models. For definiteness we choose the well-known Turing machine model. A Turing machine M computes the function f if M, when started with any string x on its tape, eventually halts with f(x) on its tape. The time ~,ndspace used by M on input x are respectively the number of basic steps executed and the number of tape squares visited by M before halting when started on input x. In general, the time and space will vary depending on the particular input x. One simplification which is commonly made is to measure the time and space solely as a function of the length of the input string. Note that some functions can be complex for a reason which sheds little light on the question of inherent difficulty; namely, a function can be computed no faster than the time required to print the value of '~om~letedefinitions appear in the main text. the function. For example, consider the function which, for any positive integer m, maps the binary representation of m to the binary representation of 2m. Any algorithm which computes this function uses at least 2" steps on many inputs of length n for all n, these steps being required to print the answer consisting of a one followed by as many as 2n-l zeroes. We avoid these cases by considering only functions whose value is always 0 or 1. The problem of computing such a 0-1 valued function f can be viewed as the problem of recognizing the set of inputs which f maps to 1. For example, we may wish to recognize the set of all strings which code true sentences of some decidab'le logical theory. When such a "set recognition" or "decision" problem is shown to n require time 2 on inputs of length n for infinitely many n, we conclude that there is something inherently complex about the set itself; that is, n 2 steps must be spent in deciding what to answer, not in printing the answer. Some information is known concerning the complexity of set recognition problems. There are known to be sets whose recognition problems are recursive yet "arbitrarily" complex [Rab60]. Let T(n) and S(n) be any recursive functions from positive integers to positive integers. Well-known diagonalization arguments imply the existence of a recursive set such that any algorithm recognizing Ah %ard ard requires at least time T(n) and space S(n) on all inputs of length n for all sufficiently large n. It is also possible to construct arbitrarily difficult recursive problems by considering "bounded" versions of undecidable problems. The "bound" implies decidability, but the problem can be made arbi- trarily complex by making the "bound" arbitrarily large. For example, Blum [B166] and Jeroslow [Jer72] consider a bounded version of the halting problem, and Ehrenfeucht [Ehr72] considers a bounded version of the first order theory of integer arithmetic. One might animadvert that sets such as above are not "natural" %ard in the sense that they were explicitly constructed to be difficult to recognize. Informally, by "natural" computational problem we mean one which has arisen previously in the mathematical literature (excluding complexity theory); for example, decision problems drawn from logic and automata theory, word problems in algebra, etc. Under even this weak view of "natural", there are few examples of natural recursive set recognition problems whose time complexity has been shown to necessarily grow faster than linearly in the length of the input. Excluding "diagonalization" and "bounded undecidable" problems, then prior to the research described here (and related work by Meyer [Mey73], Fischer and Rabin [FR74], and Hunt [Hun73b]) we h-ow of no examples of natural recursive set recognition problems whose time complexity had been shown to necessarily grow more than polynomially or whose space complexity had been shown to grow more than linearly in the length of the input. We now outline the remainder of this paper. Chapters 2 and 3 are devoted mainly to definitions of key concepts and descriptions of the technical machinery to be used in proving the results of Chapters 4 and 5. Chapter 2 defines our formal model of "algorithm" for set recognition and function computation. This model is a slight variant of the well-known Turing machine. Known facts concerning the model which are relevant to the sequel are also stated. Chapter 3 defines the concept of "efficient reducibility". This concept was first formally defined by Cook [Co7la], though its significance was emphasized earlier by Meyer and McCreight [MM71]. Speaking informally for the moment, we say that a set A is efficiently reducible to a set B, written A S B, if there is an efficiently ef f computable function f such that any question of the form "Is x in A?" has the same answer as the question "Is f(x) in B?". Instead of being precise about what is meant by f being "efficiently computable", let us for the moment just assume that the time and space required to compute f is very small compared to the minimum time required to recognize A or B. Now given an algorithm M which recognizes B, one can construct an algorithm M' which recognizes A as follows.