1/22/2009
CS 373 What is computable? • Examples: – check if a number n is prime Theory of Computation – compute the product of two numbers – sort a list of numbers Spring 2009 – find the maximum number from a list • Hard but computable: Sariel Har Peled – Given a set of linear inequalities, maximize a linear [email protected] function Eg. maximize 5x+2y Madhusudan Parthasarathy ( Madhu ) 3x+2y < 53 [email protected] x < 32 5x – 9y > 22
Theory of Computation Theory of Computation Primary aim of the course: - What can be computed? • What is “computation”? Computability - Can a computer solve any problem, given enough time and disk-space? • Can we define computation without refering to a modern computer? - How fast can we solve a problem? • Can we define, mathematically, a computer? Complexity How little disk-space can we use (yes, Turing machines) to solve a problem
• Is computation definable independent of present-day engineering limitations, understanding of physics, etc.? What problems can we solve given Automata really very little space? • Can a computer solve any problem, given enough time and (constant space) disk-space? Or are they fundamental limits to computation?
In short, understand the mathematics of computation
Theory of Computation Theory of Computation What problems can a computer solve? What problems can a computer solve? Not all problems!!! Even checking whether a C program will Computability Eg. Given a C-program, we cannot Computability halt/terminate is not possible! check if it will not crash! input n; Verification of correctness of programs assume n>1; No one knows is hence impossible! Complexity Complexity while (n !=1) { whether this if (n is even) terminates on (The woe of Microsoft!) n := n/2; on all inputs! else n := 3*n+1; Automata Automata }
17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1.
1 1/22/2009
Theory of Computation Theory of Computation
How fast can we compute a function? Computability How much space do we require? Computability
• Polynomial time computable • Non-det Poly Time (NP) • Approximation, Randomization Complexity Complexity
Functions that cannot be computed fast: Machines with finite memory:-- • Applications to security traffic signals, vending machines • Encrypt fast, hardware circuits Automata • Decryption cannot be done fast Automata Tractable. • RSA cryptography, web applications Applications to searching, verification of hardware, etc.
I Theory of Computation N Theory of Computation C R What can we compute? E Most general notions of computability A Computability Uncomputable functions S Turing machines I N What can we compute fast? G Complexity Faster algorithms, polynomial time Context free Problems that cannot be solved fast: C * Cryptography O . languages M P What can we compute with very little space? Automata: L Constant space (+stack) Foundations of computing Automata E Automata * String searching, language parsing, Mathematical methods of argument X hardware verification, etc. Simple setting I T Y
I I N Theory of Computation N Theory of Computation C C R R E E Turing machines (1940s): A Turing machines A Turing machines The most general notion of computing S S The Church Turing thesis I I Limits to computing: N N Uncomputable functions G G
C Context free C Context free O . languages O . languages Context free languages Motivation from mathematics: M M Grammars, parsing • Can we solve any mathematical question P P Machines with stack methodically ? L L Still a simple setting; but infinite state • Godel’s theorem: NO! E Automata E Automata • “Even the most powerful machines X X cannot solve some problems.” I I T T Y Y
2 1/22/2009
Theory of Computation Kurt Gödel I N C • Logician extraordinaire R Turing machines: E Turing machines Weeks 13--15 • Hilbert, Russel, etc. tried to A S formalize mathematics I • “Incompleteness theorem” (1931) N – Cannot prove consistency of G Context free arithmetic formally . languages Context-free languages: C Weeks 9-12 – Consequence: unprovable theorems O M P Since proofs computation, Automata Automata theory: L Weeks 2 thro’ 8 non computability was established E Kurt Godel: 1906 - 1978 X Mathematical techniques: I Week 1 T Y
Alonzo Church Alan Turing
First notions of computable functions • “father of computer science” • Defined the first formal notion of a computer (Turing machine) in 1936: First language for programs “On Computable Numbers, with an Application to the Entscheidungsproblem ” lambda calculus • Proved uncomputable functions formal algebraic language exist for computable functions • Church Turing thesis: all real world computable • functions are Turing m/c computable Alan Turing: 1912 - 1954 Alonzo Church: • Cryptanalysis work breaking Enigma in WW II 1903 - 1995
Noam Chomsky Automata theory
• Linguist ; introduced the notion of formal • Automata: machines with finite memory languages arguing generative grammars are at the base of natural languages • “Finite Automata and Their Decision Problem” • Hierarchy of formal languages that - Rabin and Scott (1959) coincides with computation • Eg. Context free grammars capture • Introduced nondeterministic automata most skeletons of prog. languages and the formalism we still use today Noam Chomsky: 1928- “Logical Structure of Linguistic Theory” (1957) • Initial motivation: modeling circuits • Turing Award (1976)
3 1/22/2009
I N Theory of Computation Goals of the course C R E Turing: 1931 • To understand the notion of “computability” A S Turing machines • Inherent limits to computability I N • The tractability of weaker models of computation G • The relation of computability to formal languages Chomsky: 1957 C Context free O . languages M • Mathematics of computer science P L – Rigor E Automata Rabin-Scott: 1959 – Proofs X I T Y
A result you would know at the end... Textbook
Michael Sipser • Proving that it is impossible to check if a C program will halt. Introduction to Theory of Computation (2 nd ed; 1 st ed may be ok) • Formal proof!
I will announce chapter • No computer *ever* will solve this problem readings that you must (not even a quantum computer) read before class.
Course logistics Teaching assistants • Section 1: Tu/Thu 11:00 12:15 Sariel Har Peled Section 2: Tu/Thu 12:30 13:45 Madhusudan Parthasarathy. Lectures in SC 1105. • Micah Hodosh [email protected] ,
• Discussion sections (all in SC 1111): by TAs • Aparna Sundar [email protected] , Tue 2:00 PM 2:50 PM • Reza Zamani [email protected] Tue 3:00 PM 3:50 PM Tue 4:00 PM 4:50 PM Wed 4:00 PM 4:50 PM
• Announcements (homework posting announcements, discussions, corrections/clarifications): Newsgroup: class.cs373
4 1/22/2009
Problem Sets Grading
• Homeworks every week; handed out on Thursday, • Two midterms 20% each due in class by 12:30pm on Thursday.
• Write each problem on a separate sheet of paper. • Final exam 30% (allows distributed grading)
• Homework can be done in groups of at most three people. • Homework/Quizzes 25%
• However, each student must hand in their own homework (least scored HW not counted) (no group submissions; must clearly write your group members)
• There may be additional “quizzes” (15min tests) at discussion • Attendance to sections as well. discussion sections 5%
Curve My lectures • Raw numerical scores tend to run low in theory classes; letter grades will primarily be decided based on relative • I will use a tablet PC; all class lecture slides will ranking within the class. be posted online. Class Percentile Grade 95 % A+ • Additional resources (on course webpage) 85 % A – Sariel’s and Margaret’s lecture notes (Sp08) 80 % A 70 % B+ – Lecture notes (slides) from Fall’08 60 % B 50 % B – Review notes on main results you should 40 % C+ learn/know (by me) 30 % C 20 % C – Old homeworks/solutions online 15 % D+ 10 % D – Probably too many resources!.... 5 % D < 5 % F
Honors? How to do well
• Honors students will do extra problems • This is essentially a math course: – you must learn the concepts well; if you don’t there’s almost no and a project. chance of success – if you do learn the concepts, there is very little else (facts, etc.) to learn; you can do really well! • Please contact me after class if you intend – You must do problems. There’s no replacement for this. – Attending lectures is highly adviced! taking this course as an honors course. • It will be very hard to learn the concepts by yourself or from textbook.
– Don’t postpone learning; you will not be able to “make up” later. Topics get quickly hard. – Come regularly to discussion sections; you will learn a lot by working out problems and learn from fellow students
5 1/22/2009
How to do well
• Come to office hours!! – We are here to help you learn and do well. • A new plan: – We will categorize homework problems, exam problems into various categories: • Machine construction, Proofs, Notation, Conceptual understanding, etc. • We will tell you how well you do in each category – We will also, at midterms, try to estimate how well you are doing and your projected grade. This will help you gauge your grade and overcome the panic of looking at low scores!
6