DOCSLIB.ORG

The “Complexity” Class NP

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The “Complexity” Class NP Lecture 27: The Grand Finale The“complexity”classNP ✦ Agenda for the final class: ✦ Definition: NP is the set of all problems for which a given ➭ P, NP, and NP-completeness candidate solution can be tested in polynomial time ➭ The NP =? P problem ➭ Suppose someone gives you a solution – can it be tested in ➧ Major extra-credit problem polynomial time? (testing is easier than solving it) ➭ Final Review ➧ Summary of what you’ve learned in this course ✦ Example of a problem in NP: ➭ End-of-quarter M.o.M. party ➭ Our new friend, the Hamiltonian circuit problem:Whyis ➧ Munchonmunchiesasyouleave… it in NP? ➧ Given a candidate path, can test in linear time if it is a Hamiltonian circuit – just check if all vertices are visited exactly once in the candidate path (except start/finish vertex) R. Rao, CSE 373 1 R. Rao, CSE 373 3 From Last Time: The “complexity” class P Why NP? ✦ The set P is defined as the set of all problems that can be ✦ NP stands for Nondeterministic Polynomial time solved in polynomial worse case time ➭ Why “nondeterministic”? Corresponds to algorithms that can ➭ Also known as the polynomial time complexity class – search all possible solutions in parallel and pick the correct contains problems whose time complexity is O(Nk)forsomek one ! each solution can be checked in polynomial time ➭ Nondeterministic algorithms don’t exist – purely theoretical ✦ Examples of problems in P: searching, sorting, topological idea invented to understand how hard a problem could be sort, single-source shortest path, Euler circuit, etc. ✦ Examples of problems in NP: ➭ Hamiltonian circuit: Given a candidate path, can test in linear time if it is a Hamiltonian circuit ➭ Sorting: Can test in linear time if a candidate ordering is sorted ➭ Sorting is also in P. Are any other problems in P also in NP? R. Rao, CSE 373 2 R. Rao, CSE 373 4 More revelations about NP NP-complete problems ✦ Sorting is in P. Are any other problems in P also in NP? ✦ The “hardest” problems in NP are called NP-complete ➭ YES! All problems in P are also in NP ! P ⊆ NP (NPC) problems ➧ If you can solve a problem in polynomial time, can ✦ Why “hardest”? A problem X is NP-complete if: definitely verify a solution in polynomial time 1. XisinNPand 2. any converted to ✦ Question: Are all problems in NP also in P? problem Y in NP can be Xin ⊆ polynomial time, such that solving X also provides a ➭ Is NP P? solution for Y ! Can use algorithm for X as a subroutine to solve Y ✦ Thus, if you find a poly time algorithm for just one NPC problem, all problems in NP can be solved in poly time ✦ Example: The Hamiltonian circuit problem can be shown to be NP-complete (not so easy to prove!) R. Rao, CSE 373 5 R. Rao, CSE 373 7 Your chance to win a Turing award: P = NP? P, NP, and Exponential Time Problems ✦ Nobody knows whether NP ⊆ P ✦ All algorithms for NP- ➭ Proving or disproving this will bring you instant fame! complete problems so far have EXPTIME tended to run in nearly ✦ It is generally believed that P ≠ NP i.e. there are problems in NPC NP that are not in P exponential worst case time ➭ But this doesn’t mean fast ➭ But no one has been able to show even one such problem NP sub-exponential time ✦ A very large number of problems are in NP (such as the algorithms don’t exist! Not Hamiltonian circuit problem) proven yet… P ➭ No one has found fast (polynomial time) algorithms for these ✦ Diagram depicts relationship problems betweenP,NP,andEXPTIME ➭ On the other hand, no one has been able to prove such algorithms don’t exist (i.e. that these problems are not in P)! (class of problems that require It is believed that exponential time to solve) P ≠ NP ≠ EXPTIME R. Rao, CSE 373 6 R. Rao, CSE 373 8 The “graph” of NP-completeness TSP is NP-complete! ✦ Cook first showed (in 1971) ✦ We can show TSP is also NPC if we can convert any input that satisfiability of Boolean forHCtoaninputforTSPinpolytime.Here’soneway: formulas (SAT) is NP-complete ✦ Hundreds of other problems 1 B C B C (from scheduling and databases 1 1 to optimization theory) have HC G TSP since been shown to be NPC G 111 1 ✦ How? By showing an algorithm D E 2 D 1 E 2 that converts a known NPC problem to your pet problem in poly time ! then, your problem is also NPC! This graph has a Hamiltonian circuit iff this fully-connected graph has a TSP cycle of total cost ≤ K, where K = |V|(here,K=5) R. Rao, CSE 373 9 R. Rao, CSE 373 11 Showing NP-completeness: An example Coping with NP-completeness 4 ✦ Consider the Traveling B C ✦ Given that it is difficult to find fast algorithms for NPC Salesperson (TSP) Problem: 1 Cycle problems, what do we do? Given a fully connected, weighted 3 2 with cost graph G = (V,E), is there a cycle 1 ≤ 8 ! ✦ Alternatives: that visits all vertices exactly once 3 BDCEB 1. Dynamic programming: Avoid repeatedly solving the same and has total cost ≤ K? D E subproblem – use table to store results (see Chap. 10) 2. Settle for algorithms that are fast on average: Worst case still ✦ TSPisinNP(why?) B C takes exponential time, but doesn’t occur very often ✦ Can we show TSP is NP- Convert 3. Settle for fast algorithms that give near-optimal solutions:In complete? G TSP, may not give the cheapest tour, but maybe good enough ➭ Hamiltonian Circuit (HC) is NPC to input ➭ Can show TSP is also NPC if we for TSP 4. Try to get a “wimpy exponential” time algorithm: It’s okay if can convert any input for HC to running time is O(1.00001N) – bad only for N > 1,000,000 an input for TSP in poly time D E R. Rao, CSE 373 Input for HC 10 R. Rao, CSE 373 12 Yawn…What does all this have to do with data structures and programming? Final Review: What you need to know ✦ Top 5 reasons to know and understand NP-completeness: ✦ Basic Math N N(N +1) ∑i = ➭ Logs, exponents, summation of series i=1 2 5. What if there’s an NP-completeness question in the final? + ➭ Proof by induction N AN 1 −1 ∑ Ai = = A − 4. When you are having a tough time programming a fast ✦ Asymptotic Analysis i 0 1 algorithm for a problem, you could show it is NP-complete ➭ Big-oh, little-oh, Theta and Omega 3. When you are having a tough time programming a fast ➭ Know the definitions and how to show f(N) is big-oh/little- oh/Theta/Omega of (g(N)) algorithm for a problem, you could just say it is NPC (and ➭ How to estimate Running Time of code fragments many will believe you (yes, it’s a sad state of affairs)) ➧ E.g. nested “for” loops 2. When you are at a cocktail party, you can impress your ✦ Recurrence Relations friends with your profound knowledge of NP-completeness ➭ Deriving recurrence relation for run time of a recursive 1. Make money with new T-shirt slogan: “And God said: P=NP” function ➭ Solving recurrence relations by expansion to get run time R. Rao, CSE 373 13 R. Rao, CSE 373 15 Final Review: What you need to know ✦ Lists, Stacks, Queues ➭ Brush up on ADT operations – Insert/Delete, Push/Pop etc. ➭ Array versus pointer implementations of each data structure Final Review ➭ Header nodes, circular, doubly linked lists ✦ Trees ➭ Definitions/Terminology: root, parent, child, height, depth etc. (“We’ve covered way too much in this course… ➭ Relationship between depth and size of tree ➧ What do I really need to know?”) Depth can be between O(log N) and O(N) for N nodes R. Rao, CSE 373 14 R. Rao, CSE 373 16 Final Review: What you need to know Final Review: What you need to know ✦ Binary Search Trees ✦ Sorting Algorithms: Know run times and how they work ➭ How to do Find, Insert, Delete ➭ Elementary sorting algorithms and their run time ➧ Bad worst case performance – could take up to O(N) time ➧ Bubble sort, Selection sort, Insertion sort ➭ AVL trees ➧ ➭ Shellsort – based on several passes of Insertion sort Balance factor is +1, 0, -1 ➧ ➧ Know single and double rotations to keep tree balanced Increment Sequence ➧ All operations are O(log N) worst case time ➭ Heapsort – based on binary heaps (max-heaps) ➭ Splay trees – good amortized performance ➧ BuildHeap and repeated DeleteMax’s ➧ AsingleoperationmaytakeO(N)timebutinasequenceof ➭ Mergesort – recursive divide-and-conquer, uses extra array operations, average time per operation is O(log N) ➭ Quicksort – recursive divide-and-conquer, Partition in-place ➧ Every Find, Insert, Delete causes accessed node to be ➧ fastest in practice, but O(N2) worst case time moved to the root ➧ Pivot selection – median-of-three works best ➧ Know how to zig-zig, zig-zag, etc. to “bubble” node to top ➭ Know which of these are stable and in-place ➭ B-trees: Know basic idea behind Insert/Delete ➭ Lower bound on sorting, bucket sort, and radix sort R. Rao, CSE 373 17 R. Rao, CSE 373 19 Final Review: What you need to know Final Review: What you need to know ✦ Priority Queues ✦ Disjoint Sets and Union-Find ➭ Binary Heaps: Insert/DeleteMin, Percolate up/down ➭ Up-trees and their array-based implementation ➧ Array implementation ➭ Know how Union-by-size and Path compression work ➧ BuildHeap takes only O(N) time (used in heapsort) ➭ No need to know run time analysis – just know the result: ➭ Binomial Queues: Forest of binomial trees with heap order ➧ Sequence of M operations with Union-by-size and P.C.
Load more

Recommended publications
  • Complexity Theory Lecture 9 Co-NP Co-NP-Complete
    Complexity Theory 1 Complexity Theory 2 co-NP Complexity Theory Lecture 9 As co-NP is the collection of complements of languages in NP, and P is closed under complementation, co-NP can also be characterised as the collection of languages of the form: ′ L = x y y <p( x ) R (x, y) { |∀ | | | | → } Anuj Dawar University of Cambridge Computer Laboratory NP – the collection of languages with succinct certificates of Easter Term 2010 membership. co-NP – the collection of languages with succinct certificates of http://www.cl.cam.ac.uk/teaching/0910/Complexity/ disqualification. Anuj Dawar May 14, 2010 Anuj Dawar May 14, 2010 Complexity Theory 3 Complexity Theory 4 NP co-NP co-NP-complete P VAL – the collection of Boolean expressions that are valid is co-NP-complete. Any language L that is the complement of an NP-complete language is co-NP-complete. Any of the situations is consistent with our present state of ¯ knowledge: Any reduction of a language L1 to L2 is also a reduction of L1–the complement of L1–to L¯2–the complement of L2. P = NP = co-NP • There is an easy reduction from the complement of SAT to VAL, P = NP co-NP = NP = co-NP • ∩ namely the map that takes an expression to its negation. P = NP co-NP = NP = co-NP • ∩ VAL P P = NP = co-NP ∈ ⇒ P = NP co-NP = NP = co-NP • ∩ VAL NP NP = co-NP ∈ ⇒ Anuj Dawar May 14, 2010 Anuj Dawar May 14, 2010 Complexity Theory 5 Complexity Theory 6 Prime Numbers Primality Consider the decision problem PRIME: Another way of putting this is that Composite is in NP.
    [Show full text]
  • The Shrinking Property for NP and Conp✩
    Theoretical Computer Science 412 (2011) 853–864 Contents lists available at ScienceDirect Theoretical Computer Science journal homepage: www.elsevier.com/locate/tcs The shrinking property for NP and coNPI Christian Glaßer a, Christian Reitwießner a,∗, Victor Selivanov b a Julius-Maximilians-Universität Würzburg, Germany b A.P. Ershov Institute of Informatics Systems, Siberian Division of the Russian Academy of Sciences, Russia article info a b s t r a c t Article history: We study the shrinking and separation properties (two notions well-known in descriptive Received 18 August 2009 set theory) for NP and coNP and show that under reasonable complexity-theoretic Received in revised form 11 November assumptions, both properties do not hold for NP and the shrinking property does not hold 2010 for coNP. In particular we obtain the following results. Accepted 15 November 2010 Communicated by A. Razborov P 1. NP and coNP do not have the shrinking property unless PH is finite. In general, Σn and P Πn do not have the shrinking property unless PH is finite. This solves an open question Keywords: posed by Selivanov (1994) [33]. Computational complexity 2. The separation property does not hold for NP unless UP ⊆ coNP. Polynomial hierarchy 3. The shrinking property does not hold for NP unless there exist NP-hard disjoint NP-pairs NP-pairs (existence of such pairs would contradict a conjecture of Even et al. (1984) [6]). P-separability Multivalued functions 4. The shrinking property does not hold for NP unless there exist complete disjoint NP- pairs. Moreover, we prove that the assumption NP 6D coNP is too weak to refute the shrinking property for NP in a relativizable way.
    [Show full text]
  • NP-Completeness: Reductions Tue, Nov 21, 2017
    CMSC 451 Dave Mount CMSC 451: Lecture 19 NP-Completeness: Reductions Tue, Nov 21, 2017 Reading: Chapt. 8 in KT and Chapt. 8 in DPV. Some of the reductions discussed here are not in either text. Recap: We have introduced a number of concepts on the way to defining NP-completeness: Decision Problems/Language recognition: are problems for which the answer is either yes or no. These can also be thought of as language recognition problems, assuming that the input has been encoded as a string. For example: HC = fG j G has a Hamiltonian cycleg MST = f(G; c) j G has a MST of cost at most cg: P: is the class of all decision problems which can be solved in polynomial time. While MST 2 P, we do not know whether HC 2 P (but we suspect not). Certificate: is a piece of evidence that allows us to verify in polynomial time that a string is in a given language. For example, the language HC above, a certificate could be a sequence of vertices along the cycle. (If the string is not in the language, the certificate can be anything.) NP: is defined to be the class of all languages that can be verified in polynomial time. (Formally, it stands for Nondeterministic Polynomial time.) Clearly, P ⊆ NP. It is widely believed that P 6= NP. To define NP-completeness, we need to introduce the concept of a reduction. Reductions: The class of NP-complete problems consists of a set of decision problems (languages) (a subset of the class NP) that no one knows how to solve efficiently, but if there were a polynomial time solution for even a single NP-complete problem, then every problem in NP would be solvable in polynomial time.
    [Show full text]
  • Computational Complexity: a Modern Approach
    i Computational Complexity: A Modern Approach Draft of a book: Dated January 2007 Comments welcome! Sanjeev Arora and Boaz Barak Princeton University [email protected] Not to be reproduced or distributed without the authors’ permission This is an Internet draft. Some chapters are more finished than others. References and attributions are very preliminary and we apologize in advance for any omissions (but hope you will nevertheless point them out to us). Please send us bugs, typos, missing references or general comments to [email protected] — Thank You!! DRAFT ii DRAFT Chapter 9 Complexity of counting “It is an empirical fact that for many combinatorial problems the detection of the existence of a solution is easy, yet no computationally efficient method is known for counting their number.... for a variety of problems this phenomenon can be explained.” L. Valiant 1979 The class NP captures the difficulty of finding certificates. However, in many contexts, one is interested not just in a single certificate, but actually counting the number of certificates. This chapter studies #P, (pronounced “sharp p”), a complexity class that captures this notion. Counting problems arise in diverse fields, often in situations having to do with estimations of probability. Examples include statistical estimation, statistical physics, network design, and more. Counting problems are also studied in a field of mathematics called enumerative combinatorics, which tries to obtain closed-form mathematical expressions for counting problems. To give an example, in the 19th century Kirchoff showed how to count the number of spanning trees in a graph using a simple determinant computation. Results in this chapter will show that for many natural counting problems, such efficiently computable expressions are unlikely to exist.
    [Show full text]
  • Notes on Space Complexity of Integration of Computable Real
    Notes on space complexity of integration of computable real functions in Ko–Friedman model Sergey V. Yakhontov Abstract x In the present paper it is shown that real function g(x)= 0 f(t)dt is a linear-space computable real function on interval [0, 1] if f is a linear-space computable C2[0, 1] real function on interval R [0, 1], and this result does not depend on any open question in the computational complexity theory. The time complexity of computable real functions and integration of computable real functions is considered in the context of Ko–Friedman model which is based on the notion of Cauchy functions computable by Turing machines. 1 2 In addition, a real computable function f is given such that 0 f ∈ FDSPACE(n )C[a,b] but 1 f∈ / FP if FP 6= #P. 0 C[a,b] R RKeywords: Computable real functions, Cauchy function representation, polynomial-time com- putable real functions, linear-space computable real functions, C2[0, 1] real functions, integration of computable real functions. Contents 1 Introduction 1 1.1 CF computablerealnumbersandfunctions . ...... 2 1.2 Integration of FP computablerealfunctions. 2 2 Upper bound of the time complexity of integration 3 2 3 Function from FDSPACE(n )C[a,b] that not in FPC[a,b] if FP 6= #P 4 4 Conclusion 4 arXiv:1408.2364v3 [cs.CC] 17 Nov 2014 1 Introduction In the present paper, we consider computable real numbers and functions that are represented by Cauchy functions computable by Turing machines [1]. Main results regarding computable real numbers and functions can be found in [1–4]; main results regarding computational complexity of computations on Turing machines can be found in [5].
    [Show full text]
  • Lecture 9 1 Interactive Proof Systems/Protocols
    CS 282A/MATH 209A: Foundations of Cryptography Prof. Rafail Ostrovsky Lecture 9 Lecture date: March 7-9, 2005 Scribe: S. Bhattacharyya, R. Deak, P. Mirzadeh 1 Interactive Proof Systems/Protocols 1.1 Introduction The traditional mathematical notion of a proof is a simple passive protocol in which a prover P outputs a complete proof to a verifier V who decides on its validity. The interaction in this traditional sense is minimal and one-way, prover → verifier. The observation has been made that allowing the verifier to interact with the prover can have advantages, for example proving the assertion faster or proving more expressive languages. This extension allows for the idea of interactive proof systems (protocols). The general framework of the interactive proof system (protocol) involves a prover P with an exponential amount of time (computationally unbounded) and a verifier V with a polyno- mial amount of time. Both P and V exchange multiple messages (challenges and responses), usually dependent upon outcomes of fair coin tosses which they may or may not share. It is easy to see that since V is a poly-time machine (PPT), only a polynomial number of messages may be exchanged between the two. P ’s objective is to convince (prove to) the verifier the truth of an assertion, e.g., claimed knowledge of a proof that x ∈ L. V either accepts or rejects the interaction with the P . 1.2 Definition of Interactive Proof Systems An interactive proof system for a language L is a protocol PV for communication between a computationally unbounded (exponential time) machine P and a probabilistic poly-time (PPT) machine V such that the protocol satisfies the properties of completeness and sound- ness.
    [Show full text]
  • Simple Doubly-Efficient Interactive Proof Systems for Locally
    Electronic Colloquium on Computational Complexity, Revision 3 of Report No. 18 (2017) Simple doubly-efficient interactive proof systems for locally-characterizable sets Oded Goldreich∗ Guy N. Rothblumy September 8, 2017 Abstract A proof system is called doubly-efficient if the prescribed prover strategy can be implemented in polynomial-time and the verifier’s strategy can be implemented in almost-linear-time. We present direct constructions of doubly-efficient interactive proof systems for problems in P that are believed to have relatively high complexity. Specifically, such constructions are presented for t-CLIQUE and t-SUM. In addition, we present a generic construction of such proof systems for a natural class that contains both problems and is in NC (and also in SC). The proof systems presented by us are significantly simpler than the proof systems presented by Goldwasser, Kalai and Rothblum (JACM, 2015), let alone those presented by Reingold, Roth- blum, and Rothblum (STOC, 2016), and can be implemented using a smaller number of rounds. Contents 1 Introduction 1 1.1 The current work . 1 1.2 Relation to prior work . 3 1.3 Organization and conventions . 4 2 Preliminaries: The sum-check protocol 5 3 The case of t-CLIQUE 5 4 The general result 7 4.1 A natural class: locally-characterizable sets . 7 4.2 Proof of Theorem 1 . 8 4.3 Generalization: round versus computation trade-off . 9 4.4 Extension to a wider class . 10 5 The case of t-SUM 13 References 15 Appendix: An MA proof system for locally-chracterizable sets 18 ∗Department of Computer Science, Weizmann Institute of Science, Rehovot, Israel.
    [Show full text]
  • Lecture 10: Space Complexity III
    Space Complexity Classes: NL and L Reductions NL-completeness The Relation between NL and coNL A Relation Among the Complexity Classes Lecture 10: Space Complexity III Arijit Bishnu 27.03.2010 Space Complexity Classes: NL and L Reductions NL-completeness The Relation between NL and coNL A Relation Among the Complexity Classes Outline 1 Space Complexity Classes: NL and L 2 Reductions 3 NL-completeness 4 The Relation between NL and coNL 5 A Relation Among the Complexity Classes Space Complexity Classes: NL and L Reductions NL-completeness The Relation between NL and coNL A Relation Among the Complexity Classes Outline 1 Space Complexity Classes: NL and L 2 Reductions 3 NL-completeness 4 The Relation between NL and coNL 5 A Relation Among the Complexity Classes Definition for Recapitulation S c NPSPACE = c>0 NSPACE(n ). The class NPSPACE is an analog of the class NP. Definition L = SPACE(log n). Definition NL = NSPACE(log n). Space Complexity Classes: NL and L Reductions NL-completeness The Relation between NL and coNL A Relation Among the Complexity Classes Space Complexity Classes Definition for Recapitulation S c PSPACE = c>0 SPACE(n ). The class PSPACE is an analog of the class P. Definition L = SPACE(log n). Definition NL = NSPACE(log n). Space Complexity Classes: NL and L Reductions NL-completeness The Relation between NL and coNL A Relation Among the Complexity Classes Space Complexity Classes Definition for Recapitulation S c PSPACE = c>0 SPACE(n ). The class PSPACE is an analog of the class P. Definition for Recapitulation S c NPSPACE = c>0 NSPACE(n ).
    [Show full text]
  • Glossary of Complexity Classes
    App endix A Glossary of Complexity Classes Summary This glossary includes selfcontained denitions of most complexity classes mentioned in the b o ok Needless to say the glossary oers a very minimal discussion of these classes and the reader is re ferred to the main text for further discussion The items are organized by topics rather than by alphab etic order Sp ecically the glossary is partitioned into two parts dealing separately with complexity classes that are dened in terms of algorithms and their resources ie time and space complexity of Turing machines and complexity classes de ned in terms of nonuniform circuits and referring to their size and depth The algorithmic classes include timecomplexity based classes such as P NP coNP BPP RP coRP PH E EXP and NEXP and the space complexity classes L NL RL and P S P AC E The non k uniform classes include the circuit classes P p oly as well as NC and k AC Denitions and basic results regarding many other complexity classes are available at the constantly evolving Complexity Zoo A Preliminaries Complexity classes are sets of computational problems where each class contains problems that can b e solved with sp ecic computational resources To dene a complexity class one sp ecies a mo del of computation a complexity measure like time or space which is always measured as a function of the input length and a b ound on the complexity of problems in the class We follow the tradition of fo cusing on decision problems but refer to these problems using the terminology of promise problems
    [Show full text]
  • User's Guide for Complexity: a LATEX Package, Version 0.80
    User’s Guide for complexity: a LATEX package, Version 0.80 Chris Bourke April 12, 2007 Contents 1 Introduction 2 1.1 What is complexity? ......................... 2 1.2 Why a complexity package? ..................... 2 2 Installation 2 3 Package Options 3 3.1 Mode Options .............................. 3 3.2 Font Options .............................. 4 3.2.1 The small Option ....................... 4 4 Using the Package 6 4.1 Overridden Commands ......................... 6 4.2 Special Commands ........................... 6 4.3 Function Commands .......................... 6 4.4 Language Commands .......................... 7 4.5 Complete List of Class Commands .................. 8 5 Customization 15 5.1 Class Commands ............................ 15 1 5.2 Language Commands .......................... 16 5.3 Function Commands .......................... 17 6 Extended Example 17 7 Feedback 18 7.1 Acknowledgements ........................... 19 1 Introduction 1.1 What is complexity? complexity is a LATEX package that typesets computational complexity classes such as P (deterministic polynomial time) and NP (nondeterministic polynomial time) as well as sets (languages) such as SAT (satisfiability). In all, over 350 commands are defined for helping you to typeset Computational Complexity con- structs. 1.2 Why a complexity package? A better question is why not? Complexity theory is a more recent, though mature area of Theoretical Computer Science. Each researcher seems to have his or her own preferences as to how to typeset Complexity Classes and has built up their own personal LATEX commands file. This can be frustrating, to say the least, when it comes to collaborations or when one has to go through an entire series of files changing commands for compatibility or to get exactly the look they want (or what may be required).
    [Show full text]
  • A Short History of Computational Complexity
    The Computational Complexity Column by Lance FORTNOW NEC Laboratories America 4 Independence Way, Princeton, NJ 08540, USA [email protected] http://www.neci.nj.nec.com/homepages/fortnow/beatcs Every third year the Conference on Computational Complexity is held in Europe and this summer the University of Aarhus (Denmark) will host the meeting July 7-10. More details at the conference web page http://www.computationalcomplexity.org This month we present a historical view of computational complexity written by Steve Homer and myself. This is a preliminary version of a chapter to be included in an upcoming North-Holland Handbook of the History of Mathematical Logic edited by Dirk van Dalen, John Dawson and Aki Kanamori. A Short History of Computational Complexity Lance Fortnow1 Steve Homer2 NEC Research Institute Computer Science Department 4 Independence Way Boston University Princeton, NJ 08540 111 Cummington Street Boston, MA 02215 1 Introduction It all started with a machine. In 1936, Turing developed his theoretical com- putational model. He based his model on how he perceived mathematicians think. As digital computers were developed in the 40's and 50's, the Turing machine proved itself as the right theoretical model for computation. Quickly though we discovered that the basic Turing machine model fails to account for the amount of time or memory needed by a computer, a critical issue today but even more so in those early days of computing. The key idea to measure time and space as a function of the length of the input came in the early 1960's by Hartmanis and Stearns.
    [Show full text]
  • Subgrouping Reduces Complexity and Speeds up Learning in Recurrent Networks
    638 Zipser Subgrouping Reduces Complexity and Speeds Up Learning in Recurrent Networks David Zipser Department of Cognitive Science University of California, San Diego La Jolla, CA 92093 1 INTRODUCTION Recurrent nets are more powerful than feedforward nets because they allow simulation of dynamical systems. Everything from sine wave generators through computers to the brain are potential candidates, but to use recurrent nets to emulate dynamical systems we need learning algorithms to program them. Here I describe a new twist on an old algorithm for recurrent nets and compare it to its predecessors. 2 BPTT In the beginning there was BACKPROPAGATION THROUGH TUvffi (BPTT) which was described by Rumelhart, Williams, and Hinton (1986). The idea is to add a copy of the whole recurrent net to the top of a growing feedforward network on each update cycle. Backpropa­ gating through this stack corrects for past mistakes by adding up all the weight changes from past times. A difficulty with this method is that the feedforward net gets very big. The obvious solution is to truncate it at a fixed number of copies by killing an old copy every time a new copy is added. The truncated-BPTT algorithm is illustrated in Figure 1. It works well, more about this later. 3RTRL It turns out that it is not necessary to keep an ever growing stack of copies of the recurrent net as BPTT does. A fixed number of parameters can record all of past time. This is done in the REAL TI!\.1E RECURRENT LEARNING (RTRL) algorithm of Williams and Zipser (1989).
    [Show full text]