Combinatorial Optimization Top Ten List Discrete Mathematics 2000 as selected by William R. Pulleyblank

T. J. Watson Research Center IBM Corporation Yorktown Heights, NY Euler's Theorem 1736

Theorem: A graph has a Euler tour if and only if there are zero or two nodes of odd degree

L. Euler, Solutio Problematis ad Geometriam Situs Pertinentis, Commentarii Academiae Scientiarum Imperalis Petropolitanae 8 (1736) 128-140.

" As far as the problem of the seven bridges of Koenigsberg is concerned, it can be solved by making an exhaustive list of all possible routes, and then finding whether or not any route satisfies the conditions of the problem. Because of the number of possibilities, this method of solution would be too difficult and laborious, and in other problems with more bridges, it would be impossible..." " So whatever arrangement may be proposed, one can easily determine whether or not a journey can be made, crossing each bridge once, by the following rules: If there are more than two areas to which an odd number of bridges lead, then such a journey is impossible. If, however, the number of bridges is odd for exactly two areas, then the journey is possible if it starts in either of these areas. If, finally, there are no areas to which an odd number of bridges leads, then the required journey can be accomplished starting from any area. " Annalen

Algorithm first published 137 years later

C. Hierholzer, Ueber die Moeglichkeit, einen Linienzug ohne Wiederholung und ohne Unterbrechnung zu umfahren, Mathematische 6 (1873), 30-32.

- start at an odd degree node; - walk until you are stuck * what's left has no odd degree nodes - solve the remaining problem and splice in the answer.

see N.L. Biggs, E.K.Lloyd, R.J.Wilson, Graph Theory 1736-1936, Oxford University Press, 1976. Chinese Postman problem

Guan Meigu, Graphic Programming using odd and even points, Chinese Math. 1 (1962) 273-277

The streets were given lengths, and any (even) number of odd degree intersections was permitted. The problem now became:

Find the shortest tour traversing each street at least once.

Solved polynomially in J. Edmonds and E.L. Johnson, Matching, Euler Tours and the Chinese Postman, Mathematical Programming 5 (1973) 88-124. Many extensions - one in particular -

P.D. Seymour, The with the max-flow min-cut property, J. Combin. Theory Ser. B 23 (1977) 189-222. Herbert J. Ryser, Combinatorial Mathematics, The Mathematical Association of America, 1963.

" Combinatorial mathematics cuts across the many subdivisions of mathematics, and this makes a formal definition difficult. ... Two general types of problems appear throughout the literature. In the first the existence of the prescribed configuration is in doubt, and the study attempts to settle this issue. These we call existence problems. In the second the existence of the configuration is known, and the study attempts to determine the number of configurations or the classification of these configurations according to types. These we call enumeration problems. "

March 1971 -- Second Louisiana Conference on Combinatorics, Graph Theory and Computing, LSU, Baton Rouge, Ryser gave an invited address and included a third type of problem:

How do you efficiently find or construct the configurations? Max-flow Min-cut Theorem 1956

Given a directed graph G = (V, E),with finite arc

capacities ( uj : j? E), and designated source and sink nodes s and z, the maximum value of an s - z flow is the minimum capacity of an s - z cut.

L.R. Ford Jr. and D.R. Fulkerson, Maximal flow through a network, Canadian J. of Math. 8 (1956) 399-404.

A. Kotzig, "Suvislost' a Pravidelina Suvislost Konecnych Grafor", Bratislava: Vysoka Skola Economicka (1956).

P. Elias, A. Feinstein and C.E. Shannon, A note on the maximum flow through a network, IRE Transactions on Information Theory IT2 (1956) 117-119. Equivalent Theorems:

Menger's Theorem (1927) (edge or node disjoint paths) Dilworth's Theorem (1950) (antichains in partially ordered sets) P. Hall's Theorem (1935) (Systems of Discrete Representatives) Koenig's Theorem (1931) (maximum matching and minimum cover in a bipartite graph) Hoffman's Circulation Theorem (1960) Gale's Supply Demand Flow (1957) ...

These were all linked by total unimodularity - Hoffman and Kruskal (1956) Write down a "natural" formulation, then there will be an integral optimal solution ... and some extensions ...

Flow augmenting paths Dinit (1970) and Edmonds-Karp (1972) - shortest flow augmenting paths give polynomial performance Preflow-push algorithms Generalized flows Multicommodity flows ...

Seymour'sCombin. Theory Theorem Ser. (1980) B : [A good characterization of total unimodularity] - a is totally unimodular if and only if it can be composed from network matrices, their transposes, and two exceptions.

P.D. Seymour, Decomposition of regular matroids, J. of 28 (1980) 305-359. Maths.Nonbipartite matching algorithm and polyhedral characterization 1965.

Jack Edmonds, Paths, trees and flowers, Canadian J. of 17 (1965) 449-467.

There exists a polynomially bounded algorithm which finds a maximum matching in a nonbipartite graph.

Generalized the Hungarian method of Kuhn 1955 and Munkres 1957 for bipartite graphs

Alternating paths not enough

You shrink the odd sets! Paths, trees and flowers (Edmonds 1965) -

2. Digression

... I am claiming, as a mathematical result, the existence of a good algorithm for finding a maximum cardinality matching in a graph. ... an algorithm whose difficulty grows only algebraically with the size of the graph...

...For practical purposes the difference between algebraic and exponential order is often more crucial than the difference between finite and non-finite. also, Cobham (1965) Foundation of Polyhedral Combinatorics

The convex hull of the incidence vectors x of the matchings of a nonbipartite graph is defined by

xj m 0 for all j ? E, ?@ 5 (xj : j (v)) [ 1 for all v ?V, ? 5 (xj : j E(S)) [ (|S| -1)/2 for all odd cardinality S <V.

J. Edmonds, Maximum matching and a polyhedron with 0-1 vertices, J. Res. Nat. Bureau of Standards B 69 (1965) 125-130. Extensions:

bidirected capacitated weighted matchings - includes network flows, edge covers and natchings

parity constraints - includes T-joins, cuts and postman problems

stable sets in claw free graphs

path systems Intersection Theorem 1970

Matroids had been introduced by Hassler Whitney in 1935 as an abstraction of .

A family F of (so called independent) subsets of a set E forms a matroid if 1. all subsets of an independent set are independent; 2. for any A < E, all maximal independent subsets of A have the same cardinality - denoted by r(A).

(If 1. is satisfied, then F is called an independence system.)

Examples - edge sets of forests in a graph, linearly independent sets of columns of a matrix, systems of distinct representatives of a set family, ... Matroid Optimization Problem: Let each element e c E have a weight we. Find an independent set I for which 5

(we: ec I) is maximized.

Theorem (Rado 1957, Edmonds 1970): An independence system is a matroid if and only if the solves the Matroid Optimization Problem, for any vector w of weights.

Theorem (Edmonds 1970): The incidence vectors x of the independent sets of a matroid are the vertices of the polyhedron defined by

xe m 0 for all ec E,

5(xe: e? A) [ r(A) for all A ` E. Theorem (Edmonds(1970):

Let M1 = (E, F1) and M2 = (E, F2) be matroids on a set E. Then

maximum |I| = minimum {r1(S)+r2(E\S)}. c 3 ` I F1 F2 S E Weighted Intersection Theorem (Edmonds 1970):

Let M1 = (E, F1) and M2 = (E, F2) be matroids on a set E. Let each element e c E have weight we. Then x is the incidence vector of a maximum weight set independent in both matroids if and only if x is a basic optimal solution to the linear program maximize wx subject to

xe m 0 for all ec E,

5(xe: e? A) [ r1(A) for all A ` E,

5(xe: e? A) [ r2(A) for all A ` E. Consequences: Koenig's Theorem for maximum matchings in bipartite graphs;

Edmonds' Matroid Partition Theorem

Perfect's Theorem on Systems of Common Representatives

Extensions: Polymatroids and polymatroid intersection

Submodular and supermodular functions

Unification with nonbipartite matching: Matroid parity problem, or matroid matching - cardinality case for linear matroids (Lovasz 1980) Cook's Theorem 1971

If there exists a polynomially bounded algorithm for satisfiability, then if a class of decision problems has a polynomial length certificate for a "YES" answer, then there is a polynomial time algorithm to solve any decision problem in the class.

S.A. Cook, The complexity of theorem-proving procedures, Proc. 3rd Ann. ACM Symp. on Theory of Computing, Association for Computing Machinery, New York (1971).

NP - problems having short "YES" certificates - polynomially solvable on nondeterministic Turing machines P - problems solvable in polynomial time Cook's Theorem: SATISFIABILITY is NP-complete - a polynomial algorithm for SATISFIABILITY implies P = NP is P = NP ? R. M. Karp 1972 Reducibility among combinatorial problems in R.E. Miller and J.W. Thatcher (eds.), Complexity of Computer Computations, Plenum Press, New York, 85-103.

Showed that many well known problems were NP-complete - Hamiltonian circuit; large independent set of nodes; scheduling problems; 3-d matching ...

M.R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, Freeman, San Francisco, 1979

* More than 300 NP-complete problems Conventional Wisdom: P CNP.

Garey and Johnson - 12 open problems:

1. Graph Isomorphism open 2. Subgraph homeomorphism for a fixed graph P 3. Graph genus NP-complete 4. Chordal graph completion NP-complete 5. Chromatic Index NP-complete 6. parity P 7. Partial Order Dimension NP-complete 8. Precedence constrained 3-processor sched. open 9. Linear Programming P 10. Total Unimodularity P 11. Composite number open 12. Minimum length triangulation open Dantzig, Fulkerson and Johnson - solution of the 49 city problem - 1954

G. Dantzig, R. Fulkerson and S. Johnson, "Solution of a large-scale traveling salesman problem", Operations Research 2, (1954), 393-410.

Optimal solution of a 49 city traveling salesman problem using cutting planes

... and the race was on Largest solved TSP problems

Size Researchers Year 49 Nodes Dantzig, Fulkerson, and Johnson 1954 60 Nodes Held and Karp 1970 80 Nodes Helbig, Hansen, and Krarup 1974 100 Nodes Camerini, Fratta, and Maffiol 1975 120 Nodes Groetschel 1977 120 Nodes Padberg and Hong 1980 318 Nodes Crowder and Padberg 1980 532 Nodes Padberg and Rinaldi 1987 666 Nodes Groetschel and Holland 1988 1002 Nodes Padberg and Rinaldi 1988 2392 Nodes Padberg and Rinaldi 1988 3038 Nodes Applegate, Bixby, Chvatal, and Cook 1992 4461 Nodes Applegate, Bixby, Chvatal, and Cook 1993 7397 Nodes Applegate, Bixby, Chvatal, and Cook 1994 13509 Nodes Applegate, Bixby, Chvatal, and Cook 1998 Size of largest TSP solved to provable optimality number of cities 100000

10000

1000

100

10 1950 1960 1970 1980 1990 2000 Held and Karp's Subgradient relaxation of the TSP - 1970, 71

Start with a minimal formulation of the TSP: 5 minimize dij xij

subject to xij ? {0,1} for all i, j 5 xij = 2 for all j i Lg _ 5 xij [ |S| - 1 for all S V. i, j cs - remove the degree constraints!

minimize 5 5 5 dij xij - pj(2 - x ij ) j i

subject to xij ? {0,1} for all i, j

5 LC _ xij [ |S| - 1 for all S V. i, j cs

M. Held and R.M. Karp, The traveling salesman problem and minimum spanning trees, Operations Research 18 (1970) 1138-1162.

------, The traveling salesman problem and minimum spanning trees: Part II, Mathematical Programming 1 (1971) 6-25.

The Volume Algorithm

Anbil and Barahona (1998) showed how subgradient methods could be extended to yield a solution to the linear program which is "almost" feasible and "almost" optimal.

Essentially, they add a weighted average of the (infeasible) primal solutions produced by a subgradient method.

this followed earlier work on approximate "almost feasible" by Plotkin, Shmoys, and Tardos; Grigoriadis and Khachiyan

F. Barahona and R. Anbil, The volume algorithm: producing primal solutions with a subgradient method, Mathematical Programming, Ser. A 87, 2000, 385-399. The Volume Algorithm - producing good primal solutions. Lagrangian Methods and The Volume Algorithm (continued)

# # # Primal- Dual Volume Rows Columns Elements Dual Simplex plus Interior Dual Simplex 2504 53226 553148 1341 3747 320

2991 46450 502338 1984 6928 923

4810 95933 1009283 4165 25917 2576 Lin Kernighan Algorithm 1973

Given a feasible solution to a Traveling Salesman problem, improve it by local exchanges

Local search: alternatively removes and adds edges to a tour until a better tour is found or it gives up. Start from a random tour

Chained Lin Kernighan (Martin and Otto 1996) - random 4 - Interchange

o o o o o o o o D.S. Johnson, J.L. Bentley, L.A. McGeoch and E.E. Rothberg 1997

L-K produces tours 2% over opt. Chained L-K produces tours 1% over opt.

Can be implemented to run very quickly

10,000 node Euclidean TSP on average workstation - 9.7 seconds

L-K developed into local search activity - , , Genetic Algorithms, ... Optimization = Separation 1980, 1981

The Ellipsoid Algorithm (L.G. Khachiyan 1979, 1980) operated by testing the centre x of an ellipsoid for membership in a polyhedron P. If x is not in P, this is shown by producing a violated defining inequality. Then the ellipsoid is updated. The entire process runs in polynomial time.

Theorem (Karp and Papadimitriou 1980, Padberg and Rao 1981, Groetschel, Lovasz, Schrijver 1981): Optimization and Separation are polynomially equivalent Applications:

1. Submodular function minimization (Groetschel, Lovasz, Schrijver 1981)

2. Maximum weight stable sets in perfect graphs (Groetschel, Lovasz, Schrijver 1981) **

3. Separation for odd cuts (Padberg, Rao 1982)

4. Separation for matroid polyhedra (Cunningham 1984) The Shannon capacity_ of the pentagon is -5 1979

L. Lovasz, On the Shannon capacity of a graph, IEEE Transactions on Information Theory IT-25 (1979) 1-7. =(G) is the stability number - the maximum number of pairwise nonadjacent nodes.

Let G be a graph whose nodes are letters in an alphabet and where adjacency means that letters can be confused. The maximum number of 1 letter messages that can be sent without confusion is =(G).

Let =(Gk ) denote the maximum number of k-letter messages which can be sent without danger of confusion. (Confusion occurs if two words are identical or confusable in all positions.) C5 - the pentagon a e b =(C5) = 2

d c aa a bc =(C 2) = 5 5 e b ce db d c ed The Shannon Capacity, 3(G),is defined by k _ k _ 3(G) = sup - =(Gk) = lim - =(Gk). k J  k 8

Lovasz(1979) introduced the concept of an orthonormal representation of a graph G=(V,E) :

a set of |V| vectors ui such that ||ui||=1 for all i T and ui uj = 0 for all pairs i,j of nonadjacent vertices. Orthonormal representation constraint:

Let u be an orthonormal representation of G=(V,E). For any c ? IR n with ||c|| = 1, the incidence vector x of any stable set S satisfies

T 2 5 (c ui) xi [ 1. i ? V

V TH(G) = { x c : x m 0 and x satisfies the orthonormal IR n representation constraints for all c c IR }.

Note: TH(G) is not polyhedral

J(G) = max{1Tx : x ? TH(G)}. Theorem: (Lovasz 1979): Q(G) [ J(G)

plus, properties of J(G) and how to compute and estimate for a general graph G. _ Theorem (Lovasz 1979): J(C5) [ - 5. _ _ - - 5 [ Q(C5) [ J(C5) [ 5.

and where did it lead? Groetschel, Lovasz, Schrijver (1981, 1988):

J(G, w) = max{wTx : x ? TH(G)}.

Theorem: TH(G) is a polytope if and only if G is a perfect graph.

Theorem: TH(G) = STAB(G) if and only if G is perfect.

Theorem: The weak optimization problem for TH(G) can be solved in polynolmial time.

Corollary: Both the weighted stable set problem and weighted problem for perfect graphs can be solved in polynomial time.

but we still cannot recognize a perfect graph in polynomial time. .878 for the max cut problem 1994

Given an undirected graph G=(V,E) with edge weights we: e ? E, partition V into sets V1 and

V2 so that the weights of the edges between the two sets is maximized.

NP-hard, Sahni and Gonzalez (1976) gave a 1/2 approximation algorithm. M.X. Goemans and D.P. Williamson, 0.878-approximation algorithm for MAX-CUT and MAX-2SAT, Proceedings of the 26th Annual ACM Symposium on the Theory of Computing (1994) 422-431.

Formulated the problem as a quadratic {+1, -1} problem;

Relaxed it to a problem;

Solved polynomially, using GLS 81, 87.

Used randomized rounding to get good expected solution,

Derandomized Top Ten List 1. Euler's Theorem 1736 2. Max-flow Min-cut Theorem 1956 3. Edmonds' matching algorithm & polyhedron 1965 4. Edmonds' matroid intersection 1965 5. Cook's Theorem 1971 6. Dantzig Fulkerson and Johnson 49 cities 1954 Held & Karp relaxation of the TSP 1970, 71 7. Lin Kernighan local search for the TSP 1973 8. Optimization = Separation 1981 9. Lovasz's Shannon Capacity of pentagon 1979 10. Goemans Williamson .878 approx for MAX CUT 1994 a reference ...