Computability and Complexity
Total Page:16
File Type:pdf, Size:1020Kb
Computability and Complexity Lecture Notes Herbert Jaeger, Jacobs University Bremen Version history Jan 30, 2018: created as copy of CC lecture notes of Spring 2017 Feb 16, 2018: cleaned up font conversion hickups up to Section 5 Feb 23, 2018: cleaned up font conversion hickups up to Section 6 Mar 15, 2018: cleaned up font conversion hickups up to Section 7 Apr 5, 2018: cleaned up font conversion hickups up to the end of these lecture notes 1 1 Introduction 1.1 Motivation This lecture will introduce you to the theory of computation and the theory of computational complexity. The theory of computation offers full answers to the questions, • what problems can in principle be solved by computer programs? • what functions can in principle be computed by computer programs? • what formal languages can in principle be decided by computer programs? Full answers to these questions have been found in the last 70 years or so, and we will learn about them. (And it turns out that these questions are all the same question). The theory of computation is well-established, transparent, and basically simple (you might disagree at first). The theory of complexity offers many insights to questions like • for a given problem / function / language that has to be solved / computed / decided by a computer program, how long does the fastest program actually run? • how much memory space has to be used at least? • can you speed up computations by using different computer architectures or different programming approaches? The theory of complexity is historically younger than the theory of computation – the first surge of results came in the 60ties of last century. It grounds in the theory of computation. In comparison to the theory of computation, the theory of complexity is itself much more complex, is haunted by difficult unsolved fundamental questions, and is a highly active field of research. Why should you want to know about either of the two? There are three sorts of reasons. First, a philosophical reason. Being (or becoming) computer scientists, you should (want to) know what computation is. Just out of curiosity, just like a physicist should have sharp, rich and immediate intuitions about energy, mass or spacetime. We will use the largest part of today's lecture to outline the philosophical/mathematical history of how some of the most famous thinkers of mankind tried to come to terms with "computation". Second, a social reason. If you are a computer science professional, you will talk to other computer science professionals over lunch or in cozy conference lounges. Your colleagues will have learnt all that is standardly learnt about computation and complexity. A lecture on computation and complexity is taught in every CS program on every habitable planet in the universe. Just in order to be accepted by your colleagues, you need to know how to keep a conversation running that starts like, "I have a problem that seems undecidable to me...", or "no chance to solve this task in a thousand years of runtime... ", or "smells like NP-complete, I don't envy you, ha-ha..." (very funny). Third, some practical reasons: 2 1. The theory of computation has led to a mathematical formalism called lambda calculus that in turn has spawned a family of pragramming languages called functional programming languages. The best known among them are probably LISP, the workhorse language for many applications in artificial intelligence, and ML (you know it from your GenCS lecture). 2. In your future career as professional computer scientist you might be asked to write computer programs that solve tasks like the following: • Given an electronic ciruit, find a VLSI chip layout that minimizes connection wiring length. • Given a microprocessor program for an airplane autopilot, prove that the program will never stall (the certification-of-airworthiness authorities will want such a proof). • Given the daily assignment of customers to serve in different cities, schedule a fleet of delivery trucks such that effective mileage is minimized. • Given two DNA sequences, decide whether they share some subsequence, possibly with omissions and insertions within the shared subsequence. Three among these four problems are practically unsolvable, and one is (comparably) easy. Unfortunately, the autopilot certification task is not the simple one – and in fact, passenger airplanes have crashed because of onboard computer program failure. In this lecture you will • learn what “practically unsolvable” means (roughly, it means NP-complete), • get a hands-on feeling for detecting a practically unsolvable problem when you see it (by studying numerous examples), • learn about ways to attack "practically unsolvable" problems (by resorting to a slightly different but solvable problem; by finding approximate solutions; or by finding the correct solution with high probability). In sum, a training in computational complexity will prepare you for the diagnosis of, and the coping with, computationally complex tasks. 3. The theory of complexity (together with number theory) is at the core of modern cryptography techniques – and encrypting data is essential for safe communication protocols. In this sense, the future commercial impact of web-based technology hinges on (advanced) results from the theory of complexity. 1.2 Overview This is the planned contents for this course. Parts marked by * are optional; we will enter them as time admits. 1. History of "computation" 2. Our workhorse: Turing machines 3. Computability and decidability, computing functions vs. deciding languages 4. Recursive functions 5. Lambda calculus 6. Functional programming 7. What "problems" are in the perspective of complexity theory 8. Complexity classes 3 9. NP-complete problems: your bread and butter as a computer scientist 10. *Logical characterization of complexity In parts 1. through 6., I used many sources and recommend to rely mainly on these lecture notes. For parts 2. and 3., the textbook Introduction to Automata Theory, Languages, and Computation by Hopcroft, Motwani and Ullman will be helpful as an additional reference, but I will not follow it too closely. In 7. through 11., I will largely follow the book Computational Complexity by Christos H. Papadimitriou – a standard textbook. These lecture notes are intended to be a fully self-contained course companion, and the material covered herein is what you need to know for exams. 4 Part 1: Computation 2. A short history of "computation" Philosophers and mathematicians struggled with the question what "computation" is for over two thousand years. You are in the happy historical condition that today this question is deemed understood and even answered. In the beginning, computation was not called computation. Instead, the question first appeared in the form of what is logical reasoning. Roughly speaking, logical reasoning is the art of arriving at a conclusion from some assumptions in a step-by-step fashion, executing justifiable rules of argumentation for every step of the argumentation chain. Only today we understand that this is largely the same thing as performing a computation – which also, on a digital von-Neumann computer such as your PC, proceeds by executing "correct" instructions in step-by-step fashion. The theory of computation departed from logics only quite recently, roughly since 130 years. In this little overview we will cover both the "ancient" and the more modern developments. A handy overview1 of this philosophical history of logic and computation can be found at the Website of theoretical computer scientist Grant Malcolm2 – I have taken some inspirations for the following overview from it, as well as from the magnificent collection of mathematician's biographies3 maintained by John J O'Connor4 and Edmund F Robertson5 at the University of St. Andrews, Scotland. Aristotle6 (384-322 B.C.) is, in many respects, the original inventor of the natural sciences. He emphasized rational reasoning over myth and mystics and painstakingly described and catalogued natural phenomena, especially animals and plants. He also investigated what correct argumentation (as opposed to mere rhetoric and "sophisms") is and found formal reasoning rules called syllogisms. The most famous syllogism is 1. All humans are mortal. 2. All Greeks are humans. 3. Therefore: All Greeks are mortal. Syllogisms describe "logical" derivations of one conclusion (3.) from two premises (1., 2.). When the premises are true, the conclusion is true, too – because of the logical form of the argument itself. Aristotle presented and systematicised 19 syllogisms and claimed "that every deductive argument can be expressed as a series of syllogistic inferences. That the argument is unconvincing masks the fact that simply by raising the problem, Aristotle earns the right to be considered not only the father of logic, but also the (grand)father of metalogic." (Jonathan Lear, Aristotle and Logical Theory. Cambridge: Cambridge University Press, 1980.). Euclid7 (330? – 275? B.C.) invented what is today understood by mathematicians as mathematics. He used sequences of strict arguments to derive true (geometrical) theorems from axioms which were evidently true. You can still buy Euclid's book "The Elements" at Amazon, which is remarkable in many ways (consider, for instance, that the webserver 1 http://www.csc.liv.ac.uk/~grant/Teaching/COMP317/logic.html 2 http://www.csc.liv.ac.uk/~grant/index.html 3 http://www-history.mcs.st-and.ac.uk/ 4 http://www-history.mcs.st-andrews.ac.uk/%7Ejohn 5 http://www-history.mcs.st-andrews.ac.uk/%7Eedmund 6 http://www-history.mcs.st-andrews.ac.uk/history/Mathematicians/Aristotle.html 7 http://www-history.mcs.st-andrews.ac.uk/history/Mathematicians/Euclid.html 5 software of Amazon runs by the same principles as the arguments in Euclid's book – not from axioms to theorems but from your typed-in commands to the billing message). Middle ages in Europe: academic thinking – both subjects and methods – was rigidly ruled by the ideology of scholasticism8, very much a re-brew of Aristotelian philosophy.