Optimization WS 13/14: , by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16:38 2001 Linear programming Optimization Problems • General optimization problem maxfz(x) j fj (x) ≤ 0,x 2 Dg or minfz(x) j fj (x) ≤ 0,x 2 Dg n where D ⊆ R ,fj : D ! R, for j = 1,...,m, z : D ! R. • Linear optimization problem T ≤ n n m×n m maxfc x j Ax ≥= b,x 2 R g, with c 2 R ,A 2 R ,b 2 R • Integer optimization problem: x 2 Zn • 0-1 optimization problem: x 2 f0,1gn Feasible and optimal solutions • Consider the optimization problem maxfz(x) j fj (x) ≤ 0,x 2 D,j = 1,...,mg ∗ n ∗ • A feasible solution is a vector x 2 D ⊆ R such that fj (x ) ≤ 0, for all j = 1,...,m. • The set of all feasible solutions is called the feasible region. • An optimal solution is a feasible solution such that ∗ z(x ) = maxfz(x) j fj (x) ≤ 0,x 2 D,j = 1,...,mg. • Feasible or optimal solutions – need not exist, – need not be unique. Transformations • minfz(x) j x 2 Sg = max{−z(x) j x 2 Sg. • f (x) ≥ a if and only if −f (x) ≤ −a. • f (x) = a if and only if f (x) ≤ a ^ −f (x) ≤ −a. Lemma Any linear programming problem can be brought to the form maxfcT x j Ax ≤ bg or maxfcT x j Ax = b,x ≥ 0g. T T 0 0 Proof: a) a x ≤ b a x + x = b,x ≥ 0 (slack variable) + − + − b) x free x = x − x , x ,x ≥ 0. Practical problem solving 2002 Linear Programming, by Alexander Bockmayr, 9. Dezember 2013, 16:38 1. Model building 2. Model solving 3. Model analysis Example: Production problem A firm produces n different goods using m different raw materials. • bi : available amount of the i-th raw material • aij : number of units of the i-th material needed to produce one unit of the j-th good • cj : revenue for one unit of the j-th good. Decide how much of each good to produce in order to maximize the total revenue decision variables xj . Linear programming formulation max c1x1 + ··· + cnxn w.r.t. a11x1 + ··· + a1nxn ≤ b1, . am1x1 + ··· + amnxn ≤ bm, x1, ... ,xn ≥ 0. In matrix notation: maxfcT x j Ax ≤ b,x ≥ 0g, m×n m n n where A 2 R ,b 2 R ,c 2 R ,x 2 R . Geometric illustration max x1 + x2 w.r.t. x1 + 2x2 ≤ 3 2x1 + x2 ≤ 3 x1 , x2 ≥ 0 Polyhedra • Hyperplane H = fx 2 Rn j aT x = bg, a 2 Rn n f0g,b 2 R • Closed halfspace H = fx 2 Rn j aT x ≤ bg Linear Programming, by Alexander Bockmayr, 9. Dezember 2013, 16:38 2003 n m×n m • Polyhedron P = fx 2 R j Ax ≤ bg,A 2 R ,b 2 R • Polytope P = fx 2 Rn j Ax ≤ b, l ≤ x ≤ ug,l,u 2 Rn • Polyhedral cone P = fx 2 Rn j Ax ≤ 0g The feasible set n P = fx 2 R j Ax ≤ bg of a linear optimization problem is a polyhedron. Vertices, Faces, Facets • P ⊆ H,H \ P 6= 0/ (Supporting hyperplane) • F = P \ H (Face) • dim(F) = 0 (Vertex) • dim(F) = 1 (Edge) • dim(F) = dim(P) − 1 (Facet) • P pointed: P has at least one vertex. Illustration Rays and extreme rays • r 2 Rn is a ray of the polyhedron P if for each x 2 P the set fx + lr j l ≥ 0g is contained in P. • A ray r of P is extreme if there do not exist two linearly independent rays r 1,r 2 of P 1 1 2 such that r = 2 (r + r ). 2004 Linear Programming, by Alexander Bockmayr, 9. Dezember 2013, 16:38 Hull operations • x 2 Rn is a linear combination of x1,...,xk 2 Rn if 1 k x = l1x + ··· + lk x , for some l1,...,lk 2 R. • If, in addition 8 9 8 9 < l1,...,lk ≥ 0, = < conic = l1 + ··· + lk = 1, x is a affine combination. : l1,...,lk ≥ 0, l1 + ··· + lk = 1, ; : convex ; • For S ⊆ Rn,S 6= 0/, the set lin(S) (resp. cone(S),aff(S),conv(S)) of all linear (resp. conic, affine, convex) combinations of finitely many vectors of S is called the linear (resp. conic, affine, convex) hull of S. Outer and inner descriptions • A subset P ⊆ Rn is a H-polytope, i.e., a bounded set of the form Outer n m×n m P = fx 2 R j Ax ≤ bg, for some A 2 R ,b 2 R . if and only if it is a V -polytope, i.e., Inner n P = conv(V ), for some finite V ⊂ R • A subset C ⊆ Rn is a H-cone, i.e., Outer n m×n C = fx 2 R j Ax ≤ 0g, for some A 2 R . if and only if it is a V -cone, i.e., Inner n C = cone(Y ), for some finite Y ⊂ R Minkowski sum • X,Y ⊆ Rn • X + Y = fx + y j x 2 X,y 2 Y g (Minkowski sum) + = Main theorem for polyhedra A subset P ⊆ Rn is a H-polyhedron, i.e., Outer n m×n m P = fx 2 R j Ax ≤ bg, for some A 2 R ,b 2 R . if and only if it is a V -polyhedron, i.e., Inner n P = conv(V ) + cone(Y ), for some finite V ,Y ⊂ R Linear Programming, by Alexander Bockmayr, 9. Dezember 2013, 16:38 2005 Theorem of Minkowski • For each polyhedron P ⊆ Rn there exist finitely many points p1,...,pk in P and finitely many rays r 1,...,r l of P such that P = conv(p1,...,pk ) + cone(r 1,...,r l ). • If the polyhedron P is pointed, then p1,...,pk may be chosen as the uniquely determined vertices of P, and r 1,...,r l as representatives of the up to scalar multiplication uniquely determined extreme rays of P. • Special cases – A polytope is the convex hull of its vertices. – A pointed polyhedral cone is the conic hull of its extreme rays. Application: Metabolic networks ¦ ¦ ¦ ¦ § § § § Reversible reaction ¦ ¦ ¦ ¦ § § § Internal metabolite § X3 ¦ ¦ ¦ ¦ § § § § "System boundary" 3 F X12 External metabolite 12 11 5 B D ¤ ¤ ¤ ¥ ¥ ¥ 9 ¨ ¨ ¨ © © 1 © ¤ ¤ ¤ ¥ ¥ ¥ A 28 10 ¨ ¨ ¨ © © X1 © ¤ ¤ ¤ ¥ ¥ ¥ G X10 ¨ ¨ ¨ © © © C 6 E External reaction 4 7 ¡ ¡ ¡ ¢ ¢ ¢ ¢ £ £ £ £ ¡ ¡ ¡ ¢ ¢ ¢ ¢ £ £ £ £ X7 Internal reaction ¡ ¡ X4 ¡ ¢ ¢ ¢ ¢ £ £ £ Irreversible reaction £ Stoichiometric matrix • Metabolites (internal) rows • Biochemical reactions columns A 0 ... −k ... 1 B B 0 C B C C B −l C B C j D B 0 C kA + lC −! mE + nH B C E B m C B C F B 0 C B C G @ 0 A H ... n ... Example network F 3 12 11 5 B D 9 1 A 28 10 G C 6 E 4 7 2006 Linear Programming, by Alexander Bockmayr, 9. Dezember 2013, 16:38 0 1 −10000000000 1 B 0 1 −1 0 −1 0 0 0 0 0 0 0 C B C B 0 1 0 −1 0 −1 0 0 0 0 0 0 C B C S = B 0 0 0 0 1 0 0 1 −1 0 −1 0 C. B C B 0 0 0 0 0 1 −1 −1 0 0 0 0 C B C @ 00000000001 −1 A 0 0 0 0 0 0 0 0 1 −1 0 0 Flux cone • Flux balance: Sv = 0 • Irreversibiliy of some reactions: vi ≥ 0,i 2 Irr. • Steady-state flux cone n C = fv 2 R j Sv = 0,vi ≥ 0, for i 2 Irrg 1 k 1 k • Metabolic network analysis find p ,...,p 2 C with C = conefp ,...,p g. Simplex Algorithm: Geometric view Linear optimization problem T n maxfc x j Ax ≤ b,x 2 R g (LP) Simplex-Algorithm (Dantzig 1947) 1. Find a vertex of P. 2. Proceed from vertex to vertex along edges of P such that the objective function z = cT x increases. 3. Either a vertex will be reached that is optimal, or an edge will be chosen which goes off to infinity and along which z is unbounded. Basic solutions m×n • Ax ≤ b, A 2 R ,rank(A) = n. • M = f1,...,mg row indices, N = f1,...,ng column indices • For I ⊆ M,J ⊆ N let AIJ denote the submatrix of A defined by the rows in I and the columns in J. • I ⊆ M,jIj = n is called a basis of A iff AI∗ = AIN is non-singular. −1 • In this case, AI∗ bI , where bI is the subvector of b defined by the indices in I, is called a basic solution. −1 • If x = AI∗ bI satisfies Ax ≤ b, then x called a basic feasible solution and I is called a feasible basis. Linear Programming, by Alexander Bockmayr, 9. Dezember 2013, 16:38 2007 Algebraic characterization of vertices Theorem Given the non-empty polyhedron P = fx 2 Rn j Ax ≤ bg, where rank(A) = n, a vector v 2 Rn is a vertex of P if and only if it is a basic feasible solution of Ax ≤ b, for some basis I of A. For any c 2 Rn, either the maximum value of z = cT x for x 2 P is attained at a vertex of P or z is unbounded on P. Corollary P has at least one and at most finitely many vertices. Remark In general, a vertex may be defined by several bases. Simplex Algorithm: Algebraic version • Suppose rank(A) = n (otherwise apply Gaussian elimination). −1 • Suppose I is a feasible basis with corresponding vertex v = AI∗ bI .