Iowa State University Capstones, Theses and Retrospective Theses and Dissertations Dissertations 1994 The complexity and distribution of computationally useful problems David W. Juedes Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/rtd Part of the Computer Sciences Commons, and the Mathematics Commons Recommended Citation Juedes, David W., "The ompc lexity and distribution of computationally useful problems " (1994). Retrospective Theses and Dissertations. 10613. https://lib.dr.iastate.edu/rtd/10613 This Dissertation is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Retrospective Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. , UMl MICROFILMED 1994 INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMl films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back of the book. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6" x 9" black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order. University Microfilms international A Bell & Howell Information Company 300 North Zeeb Road. Ann Arbor, Ml 48106-1346 USA 313/761-4700 800/521-0600 Order Number 9424227 The complexity and distribution of computationally useful problems Juedes, David W., Ph.D. Iowa State University, 1994 UMI 300 N. ZeebRcl. Ann Arbor, MI 48106 The complexity and distribution of computationally useful problems by David W. Juedes A Dissertation Submitted to the Graduate Faculty in Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Department: Computer Science Major: Computer Science Approved; Signature was redacted for privacy. In Cliarge of Major ^ Signature was redacted for privacy. For the Major Departmipnt Signature was redacted for privacy. For the Graduate College Iowa State University Ames, Iowa 1994 ii TABLE OF CONTENTS ACKNOWLEDGEMENTS v CHAPTER 1. INTRODUCTION 1 1.1 Complexity in Nature 2 1.2 Accessible Information and Computational Usefulness 5 1.3 Overview 8 CHAPTER 2. FUNDAMENTAL CONCEPTS FROM COMPLEX­ ITY THEORY 13 2.1 Basic Notation and Terminology 13 2.2 Models of Computation and Complexity Classes 15 2.3 Reducibilities and Hardness 18 2.4 Measure and Category 20 2.4.1 Resource-bounded Measure 21 2.4.2 Baire Category 29 2.5 Algorithmic Information and Randomness 33 2.5.1 Resource-bounded Kolmgorov Complexity 34 2.5.2 Algorithmic Information Theory 35 2.5.3 Algorithmic Randomness 46 iii CHAPTER 3. COMPUTATIONAL DEPTH 48 3.1 Strong Computational Depth 49 3.2 Weak Computational Depth 64 CHAPTER 4. THE STRUCTURE OF EXPONENTIAL TIME . 67 4.1 The Distribution of Uniform Complexity 68 4.2 The Complexity of Weakly Hard Problems: Lower Bounds 74 4.3 The Measure of Degrees 76 4.4 The Complexity of Hard Problems: Upper Bounds 82 CHAPTER 5. THE STRUCTURE OF EXPONENTIAL SPACE . 87 5.1 The Distribution of Nonuniform Complexity 88 5.1.1 Kolmogorov Complexity 89 5.1.2 Nonuniform Complexity Cores 95 5.1.3 Incompressibility by Nonuniform Reductions 98 5.2 Nonuniform Complexity of Weakly Hard Problems: Lower Bounds . 103 5.3 Nonuniform Complexity of Hard Problems: Upper Bounds 109 5.4 Distribution of Hardness and Measure of Degrees 115 CHAPTER 6. WEAKLY COMPLETE PROBLEMS 120 6.1 Martingale Diagonalization 121 6.2 Weakly Complete Problems are Not Rare 130 CHAPTER 7. CONCLUSION 137 BIBLIOGRAPHY 140 iv LIST OF FIGURES Figure 1.1: The dependency relation among the chapters 12 Figure 2.1: A self-delimiting Turing machine 36 Figure 2.2: The Turing Machine M used in the proof of Lemma 2.10. 43 Figure 3.1: The Turing machine M' used in the proof of Lemma 3.5. 57 Figure 3.2: A classification of binary sequences. RAND has measure 1, while wkDEEP — strDEEP is comeager 66 Figure 5.1: An algorithm that computes X/i=„ 97 Figure 5.2: The algorithm for Ma in the proof of Theorem 5.13 108 Figure 5.3: The machine M Ill Figure 5.4: The construction of an <^-complete language with high KS a.e 115 V ACKNOWLEDGEMENTS First and foremost I wish to thank my advisor, Jack Lutz, for his constant pa­ tience and support, and his many timely suggestions. I am forever grateful to him for introducing me to structural complexity theory and for always being a good friend. I also wish to thank my many friends and colleagues at Iowa State University who made my years there so interesting and enjoyable, especially Amy Ackerberg, Josef Breutzmann, Ann Grimm, Nicole Lang, James Lathrop, Kesig Lee, David Martin, Susan Mayer (Stapleton), Sunil Nair, Linda Null, Hitendra Patel, Susan Wahls, Tim­ othy Wahls, and Jihoon Yang. I also wish to thank my Ph.D. committee, Daniel Ashlock, David Fernandez-Baca, Jonathan Smith, and Giora Slutzki, for their input and suggestions on my work. Finally, I wish to thank my parents, brother, and sisters for their support during my long years at school. I will never forget the life and love that they have given me. I gratefully acknowledge Applied Communications, Inc. and the Arthur Collins Foundation for providing support for my graduate education through their generous scholarships. My thesis research was also supported in part by National Science Foun­ dation grants CCR-8809238 and CCR-9157382, with matching funds from Rockwell International, Microware Systems Corporation, and Amoco Foundation. Much of the work contained here was performed jointly with Jack Lutz and vi James Lathrop. I am eternally grateful for their contributions to this work and for their permission to include this work here. 1 CHAPTER 1. INTRODUCTION What do a sequence of DNA, an encyclopedia, and the UNIX operating system have in common? All three of these objects contain large amounts of useful informa­ tion, organized in such a fashion as to be readily available to biological, intellectual, or computational processes. Similarly, the solutions of certain natural decision prob­ lems, such as the halting problem or the boolean satisfiability problem, contain large amounts of useful information about computation that is readily available to effi­ cient computational processes. Here we search for the common thread among these objects. This dissertation investigates classes of problems that are computationally useful. In particular, we investigate the complexity and distribution of problems that contain large amounts of useful information about computation. The main results of this dissertation are of the following three general types. (1) Useful problems contain highly organized information. (2) Very useful problems are so highly organized that they are unusually simple and hence rare. (3) Useful problems are, as a whole, not rare and thus are not necessarily simple. Our main result of type (1) has its roots in the search for a measure that accu­ 2 rately reflects the complexity of natural objects. Our results of type (2) and (3) are motivated by the study of intractability in structural complexity theory. We first ex­ amine our main result of type (1). We begin with a brief digression on the complexity of objects in nature. 1.1 Complexity in Nature The spectrum of complexity in nature ranges from the exceedingly simple to the exceedingly complex. On the low end of the spectrum lie very simple objects: perfect crystals, pure liquids, gases, etc. On the high end of the spectrum lie very complex objects: DNA, human beings, societies, etc. In most cases, objects at the high end of the complexity spectrum are the result of the action, over time, of some complex adaptive system. The best example of this phenomenon is the amazing variety of life on earth. Each organism on this planet, including ourselves, is the result of the action of evolution over a span of billions of years. In the light of these phenomena, we are left wonder about the origin, nature, and driving principles of this vague notion of natural complexity. Recently [82, 90], there has been a concerted effort among scientists from various disciplines to uncover the nature and origins of complexity. Part of that investigation has concentrated on the complexity of the binary representations of physical and mathematical objects. This part of the investigation has its origins in algorithmic information theory. Algorithmic information theory provides a rigorous, quantitative measure of the information content of individual binary strings and binary sequences. Briefly, the algorithmic information content of an individual binary string is the length of the 3 shortest complete description of that string. (Algorithmic information theory was developed through the work of SolomonofF [87], Kolmogorov [44, 45, 46], Chaitin [18,19,20, 22], Martin-Lof [70, 71], Levin [48, 49,50, 51,52, 53,92], Schnorr [83], Gacs [28], Shen' [84, 85], and many others.