sign in sign up
<<
Home , Formal language

, Automata, Computation 1 Administrivia

Klaus Sutner

Carnegie Mellon University  FLAC

Fall 2019

Web & Piazza 3 Piazza 4

. . . is great It’s a good way to share information and get answers without http://www.cs.cmu.edu/~flac terrible delays.

. . . sucks http://piazza.com/ 15-453 It can be used to avoid work by asking lots of silly questions, and ⇒ relying on others to do all the heavy lifting.

This is an upper lever , don’t play games.

Also, don’t repost the same question a dozen times.

Course Staff 5 Course Material 6

Prof: Klaus Sutner, [email protected], No textbook, but lots of material on the web. http://www.cs.cmu.edu/~sutner If you absolutely want to have a textbook: TAs: Paul G¨olz, [email protected] M. Sipser Introduction to the of Computation Cengage India, 2014 Course secretary: Rosie Battenfelder, [email protected] Sipser 7 Learning Style 8

The book is based on a great idea: for a complicated proof, first present The topics covered in this course translate into quite a bit of a fairly informal sketch, outlining the main ideas. material. Allocate enough time to deal with this material.

Then get down into the weeds and produce a formal . Read the notes, search the net, go to the library, talk to each other, talk to us.

Again, great idea, but in reality the difference between the informal and Post on piazza (but make sure to read previous posts first). the formal argument is a bit anemic. You really get a sketch, and then a detailed sketch. I know this is an obnoxious thing to say at this point, One of the desired outcomes of this course is that you know where but in the back of your mind always think about a prover. to find more information should you ever need it (the lifelong learning meme).

Samuel Beckett 9 Bureaucracy 10

Ever tried. Ever failed. The usual testing: homeworks 50% • midterm (in-house) 20% • No matter. Try Again. Fail again. Fail better. final 30% •

Worstward Ho! There are no makeups; if you miss some assessment you can sit for an oral exam.

Bureaucracy II 11 Preserving TA Sanity 12

Typeset your solutions to the homework and submit pdf on afs Midterm is in-house, 80 minutes on Oct 19. (more on this on Piazza).

Final will be a project (think 8 pages), more later. If you have extensive conversations with other students about a HW, mention them as “collaborators” in your submission. Homework is obviously critically important. It’s 50% of your grade, so I trust you to do the work yourself. If you use a computer program in your homework, make sure to I hate having to play policeman. it properly (but do not hand in 50 pages of ). Lateness 13 Cooperation 14

You have a total of 6 (six) late days at your disposal; use prudently. Lectures will be warm and friendly. Make sure to be an active participant – FLAC is not a spectator sport. A late day is a discrete atom, with no smaller parts. You are strongly encouraged to talk about the course material to Mention lateness in the header of your HW. each other, the course staff and other students.

I reserve the right to give you no credit at all for work submitted This includes discussions of homework problems. beyond the allotted time.

Limits to Cooperation 15 More on Limits 16

Think of this as being able to copyright your solution: you may have However, even after ample consultation, the work you submit must had conversations about it with other during the problem solving be written entirely by yourself. phase, but the actual work is solely yours. No one should be in the room when you start writing things up. List all your “consultants” on the first page of your homework. We will provide a convenient template. Needless to say, you have to be able to explain all the details of your solution at any time. To avoid problems with originality, do not take notes when discussing homework problems. Don’t even think about copy & paste (from each other or the web), file sharing, clairvoyance, telepathy, . . . If you write on a board, erase everything in the end. If you have any questions about policy issues talk to the course staff, preferably some time before you get into trouble.

Yet More on Limits 17 Email 18

Some of you may still remember email (a medieval communication tool, And, of course, all the official university policies apply. predating social networks). If you decide to get in touch via email, use

Subject line: [FLAC] will miss midterm

http://www.cmu.edu/academic-integrity/index.html or some such. I filter rather aggressively, make sure to have the [FLAC] tag. Wellness 19 Wellness II 20

Take care of yourself. Do your best to maintain a healthy lifestyle this semester by eating well, exercising, avoiding drugs and alcohol, getting enough sleep and taking some time to relax. This will help you achieve your goals and cope with stress. If you or someone you know is feeling suicidal or in danger of self-harm, All of us benefit from support during times of struggle. There are many call someone immediately, day or night: helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support CaPS: 412-268-2922 sooner rather than later is almost always helpful. Re:solve Crisis Network: 888-796-8226 If the situation is life threatening, call the police If you or anyone you know experiences any academic stress, difficult life On campus: CMU Police: 412-268-2323 events, or feelings like anxiety or depression, we strongly encourage you Off campus: 911 to seek support. Counseling and Psychological Services (CaPS) is here to help: call 412-268-2922 and visit their website at http://www.cmu.edu/counseling/. Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.

Formal Languages, Automata, Computation 22

This is the official course title for 15-453.

 Administrivia Mostly a historical artifact, a better title would be

CAFL: Computation, Automata, Formal Languages 2 FLAC

We’ll start with the general , then dive all the way down to finite machines, and then talk a bit about the and complexity theory.

The Computational Universe 23 Theory First 24

This may sound like just a whole bunch of math. A (ironic?) fact of history:

It is, but this is a CS course: we don’t live in the -theory universe of The theory of computation predates computers. math, we live in the computational universe. In fact, theory was closely connected to problems in the What matters to us is not just the pure mathematical theory, but its foundations of in the 1930s, there simply was no computer computational meaning. In particular we want to emphasize . science at the time. Bear this in mind, otherwise things may occasionally seem a bit bizarre. As you will see, this makes life much more interesting, but also quite challenging: we need our to be constructive and As it turns out, is highly relevant to computer computational, not just logically correct. science, but one needs a bit of background to see why. Early History 25 Eratosthenes 26

Around 240 BCE, error may have been as low as 1%. Also calculated diameter of sun, not as accurate (27 times Earth, in reality 109). Early mathematics was very much focused on computation (Plimpton 322, about 1800 BCE).

Beginnings of Abstraction 27 Full Abstraction 28

Things changed when the field matured. For example, the main accomplishments in mathematics in the 19th century can be summarized like so:

complex variables

abstract groups

While such “top ten” lists are often contentious, this one is fairly uncontroversial.

Note that computation is essentially absent, the level of abstraction is rather high (logical depth).

Even More Abstraction 29 What Could Go Wrong? 30

Despite all the progress, in the last third of the 19th century a number of annoying problems came to light.

If Gauss says he has proved something, it seems very probable to me; if Cauchy says so, it is about as likely as not; if Dirichlet says so, it is certain. Carl Gustav Jacob Jacobi

For example, seemingly intuitive notions in analysis such as continuity and differentiability are much more subtle than they might seem. It is very, very easy to trip up. Levels of Trouble 31 Weierstrass Monster 32

f(x) = bn cos(anπx) n 0 X≥ paradox para doxa; outside of the doctrine, a misalignment with

common sense 2

1.5 antinomy anti nomos; against the law, not just a misalignment, but 1 something offensive, in need of correction 0.5

contradiction contra dicere; to speak against, a direct counterargument, 0.5 1 1.5 2 2.5 3

all hell breaks loose (aka inconsistency) -0.5

-1

-1.5 This hierarchy is non-standard and entirely informal, don’t over-interpret.

For 0 < b < 1 and ab > 1 + 3/2π, this is continuous but nowhere differentiable. A mild paradox.

Cantor’s Work 33 More Trouble in Paradise 34

Burali-Forti The unit square has the same size as the unit interval. The set of all ordinal numbers. There are more reals than rationals. Russell There is an infinite hierarchy of increasing infinities. S = x x / x { | ∈ } K¨onig,Richard Paradoxical, no more. The least not definable by at most 100 words.

Remarkably, Cantor was lead to his discoveries by studying Fourier analysis, a relentlessly applied part of mathematics. Some actual antinomies, but perhaps not quite at the level where the whole system needs to come crashing down.

Church 35 Bad Proofs 36

The Four Color Theorem was first conjectured in 1852 by F. Guthrie: any Church’s first attempt at the λ-calculus allows for a term planar can be colored with just four colors.

k = λx.( (xx)) A. Kempe published a “proof” in 1879, but an error was found by P. ¬ Heawood in 1890. Heawood gave a correct proof of the weaker Five But then we have the “Kleene-Rosser paradox” Color Theorem.

kk = (kk) An alternative “proof” was given in 1880 by P. G. Tait, but his argument ¬ too had a bug, discovered in 1891 by J. Petersen. This seems to be fatal, and requires a complete restart (typed λ-calculus). The first major step forward came in 1969 when H. Heesch managed to show that the proof could be reduced to checking finitely many cases. It is easier than you may think to construct an inconsistent system. This opened the door for a computer-assisted proof: Appel and Haken 1976, Gonthier 2006. Grundlagenkrise 37 Hilbert and Poincar´e 38

Adding up all these little insults and injuries to mathematical thought ultimately added up to a major crisis: it was no longer clear that mathematics was the ultimate standard for reliability and precision, the queen of the sciences that dwarfed all other efforts.

To be sure, most mathematicians blithely ignored these issues, they were more than happy to apply a highly developed machinery that could apparently dismember problems of any complexity. Math worked, and who cares about philosophical objections?

Blowback 39 Hilbert’s Program 40

. . . dispose of the foundational questions in mathematics as such once and for all. Formalize mathematics and concoct a finite set of Formerly, when one invented a new function, it was to fur- that are strong enough to prove all of mathematics ther some practical purpose; today one invents them in order () and show that the system is consistent; by to make incorrect the reasoning of our fathers, and nothing strictly finitary means. Also show that statements about more will ever be accomplished by these inventions. “ideal objects” can be proven in the system, without using ideal objects. H. Poincar´e D. Hilbert, 1929

Poincar´eand Hilbert were arguably the two top mathematicians around 1900; it is remarkable that their attitude towards foundational issues was so diametrically opposed.

Hilbert’s 41 Rebuilding Foundations 42

In a nutshell, Hilbert’s project seems to require at least the following The Entscheidungsproblem is solved when one knows a pro- ingredients: cedure by which one can decide in a finite number of oper- ations whether a given logical is generally valid or is satisfiable. The solution of the Entscheidungsproblem Formalization of mathematics. This requires in particular a formal is of fundamental importance for the theory of all fields, the logical system: a formal , rules of , notion of proof, theorems of which are at all capable of logical development systems that describe our domains of discourse. from finitely many axioms. A completeness proof that shows that the system can prove all true D. Hilbert, W. Ackermann assertions. Grundz¨ugeder theoretischen Logik, 1928 A consistency proof that shows that the system cannot prove a contradiction (soundness of the system).

A decision that determines whether assertions in the In modern terminology: Hilbert wanted a decision algorithm, more or less system are true or false. for all of mathematics. The Trouble with Formal Systems 43 (1848-1925) 44

They don’t fit very well with the way actual humans conjecture, argue, think and reason.

Every little detail of an argument has to be spelled out with mind-numbing precision. As a result, proofs get to be very, very long, and are more or less incomprehensible (to humans, at least).

As a consequence, most mathematicians happily ignore and formal systems. But note that writing a computer program is not much different from writing a : the same, utterly annoying level of precision and detail is required.

Frege’s Books 45 Frege in Attack Mode 46

He criticizes others (Dedekind, Peano) for the lack of precision in their work. Begriffsschrift 1879 Introduces a surprisingly modern and careful system It cannot be demanded that everything be proved, because of logic (except for notation). The first real step forward that is impossible; but we can require that all since Leibniz. used without proof be expressly declared as such, so that we can see distinctly what the whole structure rests upon. Grundlagen 1884 Critique of previous foundational efforts, outline of his After that we must try to diminish the number of primitive own project. laws as far as possible, by proving everything that can be proved. Furthermore, I demand–and in this I go beyond Grundgesetze 1893/1903 Develops theory of arithmetic and analysis in Euclid–that all methods of inference employed be specified his system. Ruined by Russel’s discovery. in advance.

This is from “Grundgesetze der Arithmetik I,” published in 1893.

Alas . . . 47 Principia Mathematica 48

Q(y) The next large-scale attempt to provide a formal foundation for a large P (x, y) fragment of mathematics was undertaken by by Russell and Whitehead, Q(x) leading to publication of three volumes 1910–1913. b a Q(a) P (b, a) Q(b) PM was a remarkable effort and absolutely groundbreaking. Alas, both Q(y) their notation system and their ontological assumptions were not P (x, y) sustainable. Russell later complained that his intellect a Q(a) P (x, a) “. . . never quite recovered from the strain of writing it.”

From Frege’s 1879 “Begriffsschrift” (concept script). Principia Mathematica 49 Progress Surrounding Hilbert’s Program 50

1889 Peano gives an axiomatization of basic .

1899 Hilbert gives an axiomatization of basic geometry.

1910s Russell and Whitehead build a fairly comprehensive of mathematics based on types.

1918 Bernays shows that propositional is sound and complete.

1929 G¨odelshows that first-order logic is sound and complete.

A Catastrophe 51 Computation 52

One key idea in G¨odel’sproof is arithmetization: one can express The Incompleteness Theorem was announced by G¨odelon October 7, formulae and proofs, plus all necessary manipulations, in terms of 1930, at a conference in K¨onigsberg, the “First International Conference arithmetic. (The other is the Liar’s Paradox.) on the Philosophy of Mathematics”:

The operations on numbers representing formulae are all easily There is a of number theory such that Peano arith- computable. metic proves neither the formula nor its negation.

Since they are easily computable, they can be handled by any formal Since either the formula or its negation must be true, we are missing out system that is is expressive enough to cover some basic aspects of on a true statement of arithmetic. arithmetic.

It seems that the only person in the audience who understood what was going on was von Neumann. Hilbert was in K¨onigsberg, he gave his “werden wissen” speech the next day, but apparently he did not attend Note the irony: computability helps to demolish Hilbert’s dream, which G¨odel’slecture. basically says everything is computable.

Computer Science 53 Sequencing 54

“The standard of correctness and completeness necessary to get a computer program to work at all is a couple of orders We will follow the historical chain of events: of magnitude higher than the mathematical community’s standard of valid proofs.”

W. Thurston 1930s Computability theory Notices AMS 1994 1950s

1970s Complexity theory

To a programmer, all this may sound oddly familiar: mind-numbing precision is also required in writing programs. In fact, much of the computability/complexity/automata machinery that is relevant for the Why? There are good reasons things happened in this particular order. Grundlagenkrise is also directly relevant to CS. Computability 55 Turing’s Machines 56

There are several problems we need to tackle:

Give a precise and robust definition of computability.

Provide evidence that our definition adequately reflects the intuitive notion of computability.

Study the basic properties of computability and prove the main results in the area.

Develop tools to show that some problems are computationally solvable or fail to be so solvable.

Do the same for a theory of tractable problems and practical computation.