Graph Coloring and Machine Proofs in Computer Science, 1977-2017

Andrew W. Appel Princeton University

Hales60 conference, Pittsburgh, PA, June 2018

1 National Science Foundation Expedition in Computing

Andrew Lennart Stephanie Benjamin Steve Adam Zhong Appel Beringer Weirich Pierce Zdancewic Chlipala Shao

Princeton Penn MIT Yale Deep Specifications are RICH, FORMAL, 2-SIDED, and LIVE

describe complex RICH behaviors in detail

in notation with a FORMAL clear semantics

connected to both 2-SIDED implementations & clients

machine-checked LIVE connection to implementations

3 Verifiable C a foundationally sound higher-order impredicative concurrent separation logic in Coq

with Andrew W. Appel Lennart Beringer Qinxiang Cao Josiah Dodds Aquinas Hobor William Mansky et alia Let’s prove programs correct! Let’s prove C programs correct!

That’s preposterous! Surely, one should write pure functional programs in a language with a clean proof theory, and prove those correct!

True, for most applications. But some applications have specific demands (latency, performance, API compatibility) that require an industry-compatible low-level language. And how do you implement the runtime system for your beautiful pure functional language? Let’s prove C programs correct!

Need Hoare logic or abstract interp. We choose Hoare logic.

7 Our Hoare logic should be Separation Logic. Need higher-order logic that use pointer modular data structures Let’s prove C programs correct!

Need Hoare logic or abstract interp. Not just “safe.” We choose So we need to reason Hoare logic. in the mathematics of the application domain. ∴ Assertion language must be an expressive general-purpose logic; we choose Coq.

8 Higher-order logic

First-order logic: quantification at base types

∀ x : ℕ . x>3 → x>2

Higher-order logic: quantification over predicates ∀ 푃 : (ℕ →Prop) . 푃(x+2) → 푃(1+x+1)

9 Our Hoare logic should be Separation Logic. Need impredicative higher-order logic object- oriented that use pointer modular data structures Let’s prove C programs correct!

10 Impredicative quantification

First-order logic: quantification at base types

∀ x : ℕ . x>3 → x>2

Higher-order logic: quantification over predicates ∀ 푃 : (ℕ →Prop) . 푃(x+2) → 푃(1+x+1)

Impredicative: when 푃 can be instantiated by quantified formulas

11 Impredicative quantification with contravariance

C programs have: A program logic for C • function pointers needs higher-order • Pthreads lock pointers features such as impredicative • data abstraction quantification • dynamic alloc/dealloc • void *

12 Function pointers To specify this in logic: struct object { void (*move) (void *self); } contravariant quantification over a predicate void make (void (*f)(void *self); which might itself struct object *p) be instantiated by { But this leads to paradoxes! a quantified predicate. p->move =Impredicativity f; with } contravariance is not sound!

That’s so 20th-century! In the 21st century, just use a logic of approximation with the Löb operator, which we call “later” ⊳. Prove soundness with a step-indexed model.

An Indexed Model of Recursive Types for Foundational Proof-Carrying Code, by Appel & McAllester, TOPLAS 2001. A Stratified Semantics of General References Embeddable in Higher-Order Logic, by Ahmed, Appel, & Virga, LICS 2002. An Indexed Model of Impredicative Polymorphism and Mutable References, by Ahmed, Appel, & Virga, 2003. Semantics of Types for Mutable State, by Amal Ahmed, Ph.D. Thesis 2004. A Very Modal Model of a Modern, Major, General Type System, by Appel, Melliès, Richards, & Vouillon, POPL 2007. Logical Step-Indexed Logical Relations, by Dreyer, Ahmed, & Birkedal, LICS 2009. A Theory of Indirection via Approximation, by Hobor, Dockins, & Appel, POPL 2010. Step-indexed Kripke models over recursive worlds, by Birkedal, Reus, Schwinghammer, Støvring, Thamsborg, & Yang, POPL 2011. ... and many more: just google-scholar “step-indexed author:Birkedal” 13 Our Hoare logic should be Separation Logic. Need impredicative higher-order logic object- oriented that use pointer modular data structures Let’s prove C programs correct!

concurrent Need Hoare logic or abstract interp. Not just “safe.” need Concurrent We choose So we need to reason Separation Logic Hoare logic. in the mathematics of the application domain. ∴ Assertion language must be an expressive general-purpose logic; we choose Coq.

14 Let’s prove concurrent, modular, object-oriented C programs that use pointers, correct!

∴ Use a higher-order impredicative concurrent separation logic embedded in Coq.

But such a logic would be far too complex to trust its soundness! ∴ Prove it sound, in Coq, with respect to the operational semantics of C, But the operational using step-indexed semantic model. semantics of C is far too complex to trust ∴ Prove sound w.r.t. the CompCert as a definition! Clight operational semantics, with

respect to which CompCert is proved correct. 15 Let’s prove concurrent, modular, object-oriented C programs that use pointers, correct!

∴ Use a foundationally sound higher-order impredicative concurrent separation logic embedded in Coq. But such a logic would be too complex for users to actually use! ∴ Provide a proof automation system, in Coq’s lemma and tactic language, to help users apply the program logic to their programs.

16 VST-Floyd interactive prover

Verifiable C program logic soundness proof CompCert verified C compiler

17 Conclusion

Let’s prove C programs correct!

Oh, all right. 1955, in Michigan . . .

Kenneth Appel finishes his two-year army service, starts PhD in mathematics at the University of Michigan . . .

takes a computer programming course, gets summer job at Douglas Aircraft programming air defense simulations

1972, Urbana, Illinois

. . . teaches his 12-year-old son how to program a computer (and how to prove by diagonalization that there are more reals than integers)

19 Graph Coloring and Machine Proofs in Computer Science, 1977-2017

Kenneth Appel 1932-2013

Wolfgang Haken 1928-

20 Photo 1976 Can it really be a proof if you can’t check it by machine?

21 Alfred B. Kempe, 1849-1922 Barrister of ecclesiastical law; mathematician

In 1876, Kempe’s Universality Theorem: for an arbitrary algebraic plane curve, a linkage can be constructed that draws the curve.

Oops! There was a bug in the proof.

Finally proved in 2002 by Michael Kapovich and John J. Millson 22 Alfred B. Kempe, 1849-1922 Barrister of ecclesiastical law; mathematician

In 1879, proof of the 4-color theorem: every planar graph can be colored using at most 4 colors.

(Any nodes connected f by an edge must have different colors.) e j k b m h d c g

23 Alfred B. Kempe 1879

6-color theorem: Every planar graph is 6-colorable.

5-color theorem: Every planar graph is 5-colorable.

4-color theorem: Every planar graph is 4-colorable.

Percy J. Heawood found a bug in the proof, 1890 24 Alfred B. Kempe 1879

6-color theorem: Every planar graph is 6-colorable.

Proof: 1. Every planar graph has at least one node of degree <6

(by Euler’s polyhedron formula): V−E+F = 2, average degree < 6

2. If you remove one node from a planar graph, what remains is a planar graph. 3. This leads to an algorithm for coloring graphs . . .

25 Kempe’s graph-coloring algorithm

To 6-color a planar graph:

1. Every planar graph has at least one vertex of degree ≤ 5. 2. Remove this vertex. 3. Color the rest of the graph with a recursive call to Kempe’s algorithm. 4. Put the vertex back. It is adjacent to at most 5 vertices, which use up at most 5 colors from your “palette.” Use the 6th color for this vertex.

26 Example: 6-color this graph

j b m k d h c

27 Example: 6-color this graph

j b m k d h c

g This node has degree < 6 ; remove it! 28 Example: 6-color this graph

Now, by induction, suppose we could color the rest of the graph f

j b m k d h c

29 Now, color the residual graph

Now, by induction, suppose we could color the rest of the graph f

We can surely e find a color for c j b m k d h c

g Find a color for this node that’s not already used in an adjacent node 30 Put back the node c, and color it

Why did this work? Because when we removed each node, at that time it had degree < 6. f So when we put it back, it’s adjacent to at most 5 already-colored nodes. e

j b m k d h c

31 Kempe’s 4-coloring algorithm

To 4-color a planar graph: 1. Find a vertex of degree ≤ 5 (there must be one) 2. Remove this vertex. 3. Color the rest of the graph with a recursive call to Kempe’s algorithm. 4. Put the vertex back. These cases: easy; you can find a color not used by an adjacent node.

This case: use the method of “Kempe chains”

This case . . . 32 Kempe chains

Suppose you are 4-coloring this graph:

b f u j d k

k g h

? 33 Kempe’s 4-coloring algorithm

These cases: easy

This case: use “Kempe chains”

This case: use “simultaneous Kempe chains” 34 Kempe’s 4-coloring algorithm

These cases: easy

This case: use “Kempe chains”

“simultaneous Kempe chains” 35 Heawood 1890 Kempe 1879 Kempe 1879 6-color thm 5-color thm Every planar graph Every planar graph contains at least 1 of contains at least 1 of these configurations: these configurations:

“reduce”: Replace that “reduce”: Replace that configuration with a configuration with a smaller config., color smaller config., color the remaining graph, the remaining graph, put the node back, put the node back, you can find a color you can find a color for the node! for the node! 36 Unavoidable sets

Illinois Journal of Mathematics 1976 (received 1974) 37 Wernicke, Franklin, 4-color thm Lebesgue, Heesch Every planar graph contains at least 1 of “unavoidable set” these configurations:

? of “reducible ? ? configurations” ? ? ? ? would prove ? ? the 4-color ? Heinrich Heesch theorem 1906-1995 “reduce”: Replace that ~1970: [paraphrase] configuration with a I estimate that computers smaller config., color will be powerful enough someday, the remaining graph, to find an unavoidable set of put the config back, perhaps 10,000 reducible you can recolor the configurations graph by Kempe chains! 38 Appel and Haken

1972-1974: Let’s use computers to analyze unavoidable sets, and estimate, (1)how many configurations might be in an unavoidable set of reducible configurations? (2)in what year will future computers be fast enough to calculate this?

39 40 Appel and Haken

1974:1974: andand thethe estimateestimate is,is, (1)(1) aboutabout 20002000 configurationsconfigurations (2)in the year 1972!

IBM System/370 Model 168, 1972 41 Appel and Haken and Koch

1974-1976: Calculate (1) an unavoidable set of 1900 configs (using a version of Heesch’s “discharging” procedure) (2) reducibility proofs for each config., using various reducibility algorithms (implemented with the assistance of C.S. PhD student John Koch)

42 Teletype model ASR-33 110 bits per second

43 Mathematical Games

44 Kempe 1879 Kempe 1879 Appel and Haken 1976 6-color thm 5-color thm 4-color thm Every planar graph Every planar graph Every planar graph contains at least 1 of contains at least 1 of contains at least 1 of these configurations: these configurations: these configurations:

(degree 4) (degree 1)

(a degree 5 node) (and 1900 more) “reduce”: Replace that “reduce”: Replace that “reduce”: Replace that configuration with a configuration with a configuration with a smaller config., color smaller config., color smaller config., color the remaining graph, the remaining graph, the remaining graph, put the node back, put the config back, put the config back, you can find a color you can recolor the you can recolor the for the node! graph! graph! 45 Math department postage meter July 22, 1976

46 My own contribution to the 4CT proofreading: None.

Dorothea Haken Laurel Appel Blostein (1962-2013) (1959-) Adjunct Assoc. Prof. Professor of C.S. of Biology Queens University Wesleyan University

two “[with] five of their children … Dorothea and Armin Haken, and Laurel, Peter, and Andrew Appel, they set to work [proofreading configurations from computer printouts]”

Robin Wilson, 2002 47 Which part don’t you believe?

“Haken’s son Armin, by then a graduate student at … Berkeley, gave a lecture on the four-colour problem…. At the end, the audience split into two groups: the over- forties could not be convinced that a proof by computer was correct, while the under-forties could not be convinced that a proof containing 700 pages of hand calculations could be correct.”

Princeton University Press 2002 48 One history

1850 Guthrie

1880 Kempe Heawood “unavoidable set” 1904 Wernicke “reducible” 1920 Birkhoff

1960 Heesch “discharging” to compute unavoidable set 1976 Appel, Haken “Every planar map is 4-colorable” Robertson, Sanders, 1996 Seymour, Thomas improved proof, same basic recipe 2006 Gonthier 49 One history Another history 1700 Leibniz Guthrie 1850 1850 Babbage Can we mechanize mathematics? 1880 Kempe 1920 Hilbert Heawood 1930 Gödel ProofCan we checking: mechanize yes Turing Proving:mathematics? not quite 1904 Wernicke

1920 Birkhoff

1960 In particular, a short Heesch theorem statement might have a very 1976 Appel, Haken long proof. Robertson, Sanders, 1996 Seymour, Thomas Yes, we noticed!

2006 Gonthier 50 One history Another history 1700 Leibniz Guthrie 1850 1850 Babbage Can we mechanize mathematics? 1880 Kempe 1920 Hilbert Heawood 1930 Gödel ProofCan we checking: mechanize yes Turing Proving:mathematics? not quite 1904 Wernicke 1950 von Neumann Let’s build 1920 Birkhoff 1960 IBM those computers!

1960 Heesch Thank you! 1976 Appel, Haken Robertson, Sanders, 1996 Seymour, Thomas

2006 Gonthier 51 One history Another history 1700 Leibniz Guthrie 1850 1850 Babbage Can we mechanize mathematics? 1880 Kempe 1920 Hilbert Heawood 1930 Gödel ProofCan we checking: mechanize yes Turing Proving:mathematics? not quite 1904 Wernicke 1950 von Neumann Let’s build 1920 Birkhoff 1960 IBM those computers!

1960 Optimizing John Robin Proof Heesch compilers Cocke 1970s Milner Assistants 1976 Appel, Haken Robertson, Sanders, 1996 Seymour, Thomas

2006 Gonthier 52 One history Another history 1700 Leibniz Guthrie 1850 1850 Babbage 1880 Kempe 1920 Hilbert Heawood 1930 Gödel Turing 1904 Wernicke 1950 von Neumann 1920 Birkhoff 1960 IBM

1960 John Heesch Cocke 1925-2002 IBM Research 1976 Appel, Haken Robertson, Sanders, 1996 Seymour, Thomas

2006 Gonthier 53 Register Allocation

Procedure P (k, j) registers memory g := mem[j+12] r1 g 0 k h := k-1 r2 j 1 h f := g∗h r3 f 2 m e := mem [j+8] r4 e 3 b m := mem[j+16] 4 c b := mem[f] 5 d c := e+8 6 7 d := c 8 k := m+4 9 j := b 10 return (d, k, j) 11 12 13 14 . 54 . One history Another history 1700 Leibniz Guthrie 1850 1850 Babbage 1880 Kempe 1920 Hilbert Heawood 1930 Gödel Turing 1904 Wernicke 1950 von Neumann 1920 Birkhoff 1960 IBM

1960 John Heesch Cocke 1925-2002 IBM Research 1976 Appel, Haken 1977: Robertson,Hmm, this Sanders, 4-color 1996 Seymour,theorem is Thomas interesting. John, ask Gregory to try Kempe’s coloring algorithm 2006 Ashok Chandra Gregory Chaitin in theGonthier register allocator (1948-2014) (1947-) of our compiler. 55 One of the most influential papers in all of computer science I was recruited to do a coloring register allocator by John Cocke, IBM's greatest computer architect, who needed it for his RISC project. He mentioned that Ashok K. Chandra, also at IBM Research at that time, had suggested recursively reducing the graph by eliminating vertices of degree less than the number of available colors, as just one possible component of a coloring algorithm.

I certainly remember the spectacular work your father did with Haken ... I heard Haken give a talk on their proof soon after they had done it. But the details of the proof escaped me. That was more Chandra's area of interest; mine is information theory. Gregory Chaitin

56 Register Allocation Chaitin et al. 1981

Live ranges Interferences Interference Procedure P (k, j) (some not shown) Graph g := mem[j+12] k h := k-1 g h f := g∗h j e := mem [j+8] f m := mem[j+16] e b := mem[f] c := e+8 c m d := c b k := m+4 j := b d k return (d, k, j) j

figure 11.1 from Modern Compiler Implementation in ML, Andrew W. Appel, Cambridge University Press 1998 57 Chaitin’s Heuristic hack of Kempe’s algorithm To mostly K-color a graph (whether planar or not!)

Is there a vertex of degree < K ? If so: Remove this vertex. Color the rest of the graph with a recursive call to the algorithm. Put the vertex back. It is adjacent to at most K-1 vertices. They use (among them) at most K-1 colors. That leaves one of your colors for this vertex. If not: Remove this vertex. Color the rest of the graph with a recursive call. Put the vertex back. It is adjacent to ≥ K vertices. How many colors do these vertices use among them? If < K : there is an unused color to use for this vertex If ≥ K: 58 Briggs’s version of Chaitin’s Heuristic hack of Kempe’s algorithm To mostly K-color a graph (whether planar or not!)

Is there a vertex of degree < K ? If so: Remove this vertex. Color the rest of the graph with a recursive call to theWhat? algorithm. Put the vertex back. It is adjacent to at mostAre Kwe-1 vertices.allowed Theyto do use that? (among them) at most K-1 colors. That leaves one of your colors for this vertex. If not: Yes! Remove this vertex. This is an algorithm to Color the rest of the graph with a recursive“mostly call. K-color” a graph. Put the vertex back. It is adjacent to ≥ K vertices. How many colors do these vertices use among them? If < K : there is an unused color to use for this vertex If ≥ K: leave this vertex uncolored. 59 Example: 3-color this graph

j b m k d h c

Stack: 60 Example: 3-color this graph

j b m k d h c

g This node has degree < 3 ; remove it! Stack: 61 Example: 3-color this graph

j b m k d h c

Push node c on Stack: c the stack 62 Example: 3-color this graph

Removing c lowers the degree f of nodes b and m; that will be helpful later! e

j b m k d h c

Stack: c 63 Example: 3-color this graph

e This node has degree < 3 ; j b m remove it! k d h

Stack: c 64 Example: 3-color this graph

e This node has degree < 3 ; j b m remove it! k d h

Stack: h c 65 Example: 3-color this graph

j b m This node k has degree < 3 ; d remove it!

Stack: h c 66 Example: 3-color this graph

No node has degree < 3

Pick a node arbitrarily, f remove it, and push it on the stack e

j b m k d

Stack: g h c 67 Example: 3-color this graph

j b m k d

Stack: k g h c 68 Example: 3-color this graph

j b m

d This node has degree < 3 ; remove it!

Stack: k g h c 69 Example: 3-color this graph

e j This node b m has degree < 3 ; remove it!

Stack: d k g h c 70 Example: 3-color this graph

This node f has degree < 3 ; remove it! e

b m

Stack: j d k g h c 71 Example: 3-color this graph

e This node has degree < 3 ; remove it! b m

Stack: f j d k g h c 72 Example: 3-color this graph

This node b m has degree < 3 ; remove it!

Stack: e f j d k g h c 73 Example: 3-color this graph

This node m has degree < 3 ; remove it!

Stack: b e f j d k g h c 74 Example: 3-color this graph

Stack: m b e f j d k g h c 75 Now, color the nodes in stack order

Find a color for this node that’s not already used in an adjacent node used in an adjacent node f

j b m k d h c

Stack: m b e f j d k g h c 76 Now, color the nodes in stack order