10 - 1 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 10. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quanti er elimination Feasibility of semialgebraic sets Real elds and inequalities The real Nullstellensatz The Positivstellensatz Example: Farkas lemma Hierarchy of certi cates Boolean minimization and the S-procedure Exploiting structure 10 - 2 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Basic Semialgebraic Sets
The basic (closed) semialgebraic set de ned by polynomials f1, . . . , fm is
n x R fi(x) 0 for all i = 1, . . . , m ∈ | n o Examples
The nonnegative orthant in Rn The cone of positive semide nite matrices Feasible set of an SDP; polyhedra and spectrahedra Properties
If S1, S2 are basic closed semialgebraic sets, then so is S1 S2; i.e., the class is closed under intersection ∩ Not closed under union or projection 10 - 3 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Semialgebraic Sets Given the basic semialgebraic sets, we may generate other sets by set the- oretic operations; unions, intersections and complements.
A set generated by a nite sequence of these operations on basic semial- gebraic sets is called a semialgebraic set.
Some examples:
The set n S = x R f(x) 0 ∈ | is semialgebraic, where denotesn <, , =, =. o 6 In particular every real variety is semialgebraic. We can also generate the semialgebraic sets via Boolean logical oper- ations applied to polynomial equations and inequalities 10 - 4 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Semialgebraic Sets Every semialgebraic set may be represented as either an intersection of unions m pi n S = x R sign fij(x) = aij where aij 1, 0, 1 ∈ | ∈ { } i\=1 j[=1 n o a nite union of sets of the form n x R fi(x) > 0, hj(x) = 0 for all i = 1, . . . , m, j = 1, . . . , p ∈ | n o in R, a nite union of points and open intervals Every closed semialgebraic set is a nite union of basic closed semialgebraic sets; i.e., sets of the form
n x R fi(x) 0 for all i = 1, . . . , m ∈ | n o 10 - 5 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Properties of Semialgebraic Sets If S , S are semialgebraic, so is S S and S S 1 2 1 ∪ 2 1 ∩ 2 The projection of a semialgebraic set is semialgebraic The closure and interior of a semialgebraic sets are both semialgebraic Some examples:
Sets that are not Semialgebraic Some sets are not semialgebraic; for example
the graph (x, y) R2 y = ex ∈ | the in nite staircase (x, y) R2 y = x ∈ | b c the in nite grid Zn 10 - 6 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Tarski-Seidenberg and Quanti er Elimination
Tarski-Seidenberg theorem: if S Rn+p is semialgebraic, then so are x Rn y Rp (x, y) S (closure under projection) ∈ | ∃ ∈ ∈ x Rn y Rp (x, y) S (complements and projections) ∈ | ∀ ∈ ∈ i.e., quanti ers do not add any expressive power
Cylindrical algebraic decomposition (CAD) may be used to compute the semialgebraic set resulting from quanti er elimination 10 - 7 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Feasibility of Semialgebraic Sets Suppose S is a semialgebraic set; we’d like to solve the feasibility problem
Is S non-empty?
More speci cally, suppose we have a semialgebraic set represented by poly- nomial inequalities and equations
n S = x R fi(x) 0, hj(x) = 0 for all i = 1, . . . , m, j = 1, . . . , p ∈ | n o
Important, non-trivial result: the feasibility problem is decidable. But NP-hard (even for a single polynomial, as we have seen) We would like to certify infeasibility 10 - 8 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Certi cates So Far The Nullstellensatz: a necessary and su cient condition for feasibility of complex varieties
n x C hi(x) = 0 i = 1 ideal h1, . . . , hm ∈ | ∀ ∅ ⇐⇒ ∈ { } n o
Valid inequalities: a su cient condition for infeasibility of real basic semialgebraic sets
n x R fi(x) 0 i = = 1 cone f1, . . . , fm ∈ | ∀ ∅ ⇐ ∈ { } n o
Linear Programming: necessary and su cient conditions via duality for real linear equations and inequalities 10 - 9 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Certi cates So Far
Degree Field Complex Real \
Linear Range/Kernel Farkas Lemma Linear Algebra Linear Programming
Polynomial Nullstellensatz ???? Bounded degree: LP ???? Groebner bases
We’d like a method to construct certi cates for polynomial equations over the real eld 10 - 10 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Real Fields and Inequalities If we can test feasibility of real equations then we can also test feasibility of real inequalities and inequations, because
inequalities: there exists x R such that f(x) 0 if and only if ∈ 2 2 there exists (x, y) R such that f(x) = y ∈
strict inequalities: there exists x such that f(x) > 0 if and only if 2 2 there exists (x, y) R such that y f(x) = 1 ∈
inequations: there exists x such that f(x) = 0 if and only if 6 2 there exists (x, y) R such that yf(x) = 1 ∈ The underlying theory for real polynomials called real algebraic geometry 10 - 11 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Real Varieties
The real variety de ned by polynomials h1, . . . , hm R[x1, . . . , xn] is ∈ n h1, . . . , hm = x R hi(x) = 0 for all i = 1, . . . , m VR{ } ∈ | We’d like to solve the feasibility problem; is h , . . . , hm = ? VR{ 1 } 6 ∅
We know
Every polynomial in ideal h , . . . , hm vanishes on the feasible set. { 1 } The (complex) Nullstellensatz: 1 ideal h , . . . , hm = h , . . . , hm = ∈ { 1 } ⇒ VR{ 1 } ∅ But this condition is not necessary over the reals 10 - 12 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 The Real Nullstellensatz Recall is the cone of polynomials representable as sums of squares.
Suppose h1, . . . , hm R[x1, . . . , xn]. ∈
1 + ideal h , . . . , hm h , . . . , hm = ∈ { 1 } ⇐⇒ VR{ 1 } ∅
Equivalently, there is no x Rn such that ∈ hi(x) = 0 for all i = 1, . . . , m if and only if there exists t1, . . . , tm R[x1, . . . , xn] and s such that ∈ ∈ 1 = s + t h + + tmhm 1 1 10 - 13 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Example Suppose h(x) = x2 + 1. Then clearly h = VR{ } ∅ We saw earlier that the complex Nullstellensatz cannot be used to prove emptyness of h VR{ }
But we have 1 = s + th with s(x) = x2 and t(x) = 1 and so the real Nullstellensatz implies h = . VR{ } ∅
The polynomial equation 1 = s + th gives a certi cate of infeasibility. 10 - 14 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 The Positivstellensatz We now turn to feasibility for basic semialgebraic sets, with primal problem
n Does there exist x R such that ∈ f (x) 0 for all i = 1, . . . , m i hj(x) = 0 for all j = 1, . . . , p
Call the feasible set S; recall
every polynomial in cone f , . . . , fm is nonnegative on S { 1 } every polynomial in ideal h , . . . , hp is zero on S { 1 } The Positivstellensatz (Stengle 1974)
S = 1 cone f , . . . , fm + ideal h , . . . , hm ∅ ⇐⇒ ∈ { 1 } { 1 } 10 - 15 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Example 3
Consider the feasibility problem 2 2 1 S = (x, y) R f(x, y) 0, h(x, y) = 0 ∈ | 0 where −1
−2 f(x, y) = x y2 + 3 −3 2 h(x, y) = y + x + 2 −4 −4 −3 −2 −1 0 1 2 By the P-satz, the primal is infeasible if and only if there exist polynomials s1, s2 and t R[x, y] such that ∈ ∈ 1 = s + s f + th 1 2 A certi cate is given by
s = 1 + 2 y + 3 2 + 6 x 1 2, s = 2, t = 6. 1 3 2 6 2 10 - 16 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Explicit Formulation of the Positivstellensatz The primal problem is
n Does there exist x R such that ∈ f (x) 0 for all i = 1, . . . , m i hj(x) = 0 for all j = 1, . . . , p
The dual problem is
Do there exist ti R[x1, . . . , xn] and si, rij, . . . such that ∈ ∈ 1 = h t + s + s f + r f f + i i 0 i i ij i j i i i=j X X X6
These are strong alternatives 10 - 17 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Testing the Positivstellensatz
Do there exist ti R[x1, . . . , xn] and si, rij, . . . such that ∈ ∈ 1 = t h + s + s f + r f f + i i 0 i i ij i j i i i=j X X X6
This is a convex feasibility problem in t , s , r , . . . i i ij To solve it, we need to choose a subset of the cone to search; i.e., the maximum degree of the above polynomial; then the problem is a semide nite program This gives a hierarchy of syntactically veri able certi cates The validity of a certi cate may be easily checked; e.g., linear algebra, random sampling Unless NP=co-NP, the certi cates cannot always be polynomially sized. 10 - 18 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Example: Farkas Lemma
The primal problem; does there exist x Rn such that ∈ Ax + b 0 Cx + d = 0 T T Let fi(x) = ai x + bi, hi(x) = ci x + di. Then this system is infeasible if and only if 1 cone f , . . . , fm + ideal h , . . . , hp ∈ { 1 } { 1 } Searching over linear combinations, the primal is infeasible if there exist 0 and such that T (Ax + b) + T (Cx + d) = 1 Equating coe cients, this is equivalent to
T A + T C = 0 T b + T d = 1 0 10 - 19 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Hierarchy of Certi cates Interesting connections with logic, proof systems, etc. Failure to prove infeasibility (may) provide points in the set. Tons of applications: optimization, copositivity, dynamical systems, quantum mechanics... 10 - 20 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Special Cases Many known methods can be interpreted as fragments of P-satz refutations. LP duality: linear inequalities, constant multipliers. S-procedure: quadratic inequalities, constant multipliers Standard SDP relaxations for QP. The linear representations approach for functions f strictly positive on the set de ned by f (x) 0. i f(x) = s + s f + + snfn, s 0 1 1 i ∈ Converse Results Losslessness: when can we restrict a priori the class of certi cates? Some cases are known; e.g., additional conditions such as linearity, per- fect graphs, compactness, nite dimensionality, etc, can ensure speci c a priori properties. 10 - 21 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Example: Boolean Minimization
xT Qx x2 1 = 0 i A P-satz refutation holds if there is S 0 and Rn, ε > 0 such that ∈ n ε = xT Sx + xT Qx + (x2 1) i i Xi=1 which holds if and only if there exists a diagonal such that Q , = trace ε. The corresponding optimization problem is
maximize trace subject to Q is diagonal 10 - 22 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Example: S-Procedure
The primal problem; does there exist x Rn such that ∈ xT F x 0 1 xT F x 0 2 xT x = 1
We have a P-satz refutation if there exists 1, 2 0, R and S 0 such that ∈
1 = xT Sx + xT F x + xT F x + (1 xT x) 1 1 2 2 which holds if and only if there exist , 0 such that 1 2 F + F I 1 1 2 2 Subject to an additional mild constraint quali cation, this condition is also necessary for infeasibility. 10 - 23 The Positivstellensatz P. Parrilo and S. Lall, CDC 2003 2003.12.07.02 Exploiting Structure What algebraic properties of the polynomial system yield e cient compu- tation? Sparseness: few nonzero coe cients. Newton polytopes techniques Complexity does not depend on the degree Symmetries: invariance under a transformation group Frequent in practice. Enabling factor in applications. Can re ect underlying physical symmetries, or modelling choices. SOS on invariant rings Representation theory and invariant-theoretic techniques. Ideal structure: Equality constraints. SOS on quotient rings Compute in the coordinate ring. Quotient bases (Groebner)