The Relation Between Polynomial Calculus, Sherali-Adams, and Sum-Of-Squares Proofs

Electronic Colloquium on Computational Complexity, Report No. 154 (2017)

The Relation between Polynomial Calculus, Sherali-Adams, and Sum-of-Squares Proofs

Christoph Berkholz Humboldt-Universit¨atzu Berlin [email protected]

October 12, 2017

Abstract We relate different approaches for proving the unsatisfiability of a system of real polynomial equations over Boolean variables. On the one hand, there are the static proof systems Sherali- Adams and sum-of-squares (a.k.a. Lasserre), which are based on linear and semi-definite programming relaxations. On the other hand, we consider polynomial calculus, which is a dynamic algebraic proof system that models Gröbner basis computations. Our first result is that sum-of-squares simulates polynomial calculus: any polynomial calculus refutation of degree d can be transformed into a sum-of-squares refutation of degree 2d and only polynomial increase in size. In contrast, our second result shows that this is not the case for Sherali-Adams: there are systems of polynomial equations that have polynomial calculus refutations√ of degree 3 and polynomial size, but require Sherali-Adams refutations of degree Ω( n/ log n) and exponential size.

1 Introduction

The area of proof complexity was founded in [6] and studies the complexity of proofs for co-NP complete problems. Traditionally, one considers proof systems for proving the unsatisﬁability of (or refuting) a propositional formula in conjunctive normal form. If one faces a proof system, there are two important questions to ask:

1. Does the system always produce proofs of polynomial size?

2. How strong is the system compared to other proof systems?

If the answer to the first question is yes, in which case the system is called p-bounded, then NP = co-NP. Therefore, it is conjectured that no proof system is p-bounded and this has been proven for a number of weak proof systems. For the second question, one considers the notion of polynomial simulation: A proof system P polynomial simulates a proof system Q if for every Q-proof of size S there is a P-proof of size poly(S). Nowadays, a large part of proof complexity focuses on weak proof systems, for which the first question has already been answered negatively. One reason for this is that they often model algorithms for solving hard problems and understanding the complexity of proofs might shed light on the complexity of algorithmic approaches that implicitly or explicitly search for proofs in the underlying proof system. The (semi-)algebraic proof systems we consider in this paper also fall into this category and are used to prove the unsatisfiability of a system F of 1 real polynomial equations fi = 0 over n Boolean variables xj ∈ {0, 1}. On the one hand, we consider polynomial calculus, which is a dynamic algebraic proof system that allows to derive new

1Note that this subsumes the problem of refuting 3-CNF formulas, because a clause x ∨ y ∨ z can be encoded as polynomial equation (1 − x)y(1 − z) = 0.

1 ISSN 1433-8092 polynomial equations that follow from F line-by-line. This proof system was introduced in [5] to model Gröbnerbasis computations and proofs of degree d (where the degree of all polynomials in the derivation is bounded by d) can be found in time nO(d) by a bounded-degree variant of the Gröbnerbasis algorithm. On the other hand, we consider the semi-algebraic proof system Sherali-Adams and the stronger sum-of-squares proof system. They are based on the linear and semi-definite programming hierarchies of Sherali-Adams [19] and Lasserre [13] and can be used to prove the unsatisfiability of a system of polynomial equations and inequalities. Proofs of degree d can be found algorithmically by solving a linear program (for Sherali-Adams) or a semi-definite program (for sum-of-squares) of size nO(d). Contrary to polynomial calculus, both systems are static in the sense that they provide the whole proof at once. In order to compare these semi-algebraic proof systems with polynomial calculus, we first remark that it is known that both systems cannot be simulated by polynomial calculus. A simple Pn example is the linear equation i=1 xi = n + 1, which has a refutation of linear size and degree 2 in Sherali-Adams and sum-of-squares, but requires polynomial calculus refutations of degree Ω(n) and size 2Ω(n) [11]. Our first theorem states that sum-of-squares is strictly stronger than polynomial calculus. Theorem 1.1. Let F be a system of polynomial equations over the reals. If F has a polynomial calculus refutation of degree d and size S, then it has a sum-of-squares refutation of degree 2d and size poly(S). For the author of this paper, this theorem was highly unexpected. In fact, there has been some evidence that the converse might be true. First, in the non-Boolean setting there are systems of equations that are easier to refute for polynomial calculus than for sum-of-squares [10] (see Section 2.4 for a discussion). Second, even for systems of polynomial equations over Boolean variables, separations of polynomial calculus from its static version Nullstellensatz were known [4]. Since sum-of-squares extends Nullstellensatz, it follows that the semi-definite lifts in the sum- of-squares/Lasserre hierarchy are necessary for “flattening” a dynamic polynomial calculus proof into a static one, although polynomial calculus is a purely algebraic system without semi-definite components. Our second theorem concerns the question whether the weaker Sherali-Adams linear programming hierarchy is already able to simulate polynomial calculus. Here we have a negative answer (that we would have expected for sum-of-squares as well).

Theorem 1.2. There is a system F of polynomial equations over R[x1, . . . , xn] such that: 1. F has a polynomial calculus refutation of degree 3 and size O(n2). √ √ 2. Every Sherali-Adams refutation of F has degree Ω( n/ log n) and size 2Ω( n/ log n). The lower bound is based on a modiﬁed version of the pebbling contradictions. The original pebbling contradictions have already been used to separate Nullstellensatz degree from polynomial calculus degree [4], but it turns out that they are easy to refute in Sherali-Adams. To obtain contradictions that are hard for Sherali-Adams (and still easy for polynomial calculus), we apply a substitution trick twice: ﬁrst to show that the resulting contradiction requires high degree in Sherali-Adams and second to obtain a size lower bound from a degree lower bound. We believe that both techniques are also helpful for future lower bound arguments for static proof systems.

Acknowledgements Part of this work was done during the Oberwolfach workshop 1733 Proof and complexity and Beyond and the author acknowledges helpful discussions with Albert Atserias, Edward Hirsch, Massimo Lauria, Joanna Ochremiak, and Iddo Tzameret on Theorem 1.1. The author thanks Edward Hirsch for providing helpful comments on an earlier version the paper. We acknowledge the ﬁnancial support by the German Research Foundation DFG under grant SCHW 837/5-1.

2 2 Proof Systems

For this section we fix a system of real polynomial equations F = {f1 = 0, . . . , fm = 0} and a system of polynomial inequalities H = {h1 > 0, . . . , hs > 0} over variables x1, . . . , xn. As it is common in propositional proof complexity, we focus on the special case of polynomial equations (and inequalities) over Boolean variables and consider the task of proving that a system of polynomial equations (and/or inequalities) has no 0/1-solution. To enforce Boolean variables, 2 the axioms xj = xj are always included in the proof systems. In Section 2.4 we briefly discuss non-Boolean variants. Algebraic proof systems are used for proving the unsatisfiability of a system of multivariate polynomial equations over some field F. As we focus on real polynomials we set F = R, unless mentioned otherwise. Semi-algebraic proof systems are used to prove the unsatisfiability of a system of polynomial equations and/or polynomial inequalities (in this setting the polynomials are always real).

2.1 Algebraic Proof Systems: Nullstellensatz and Polynomial Calculus

Nullstellensatz [2] is a static algebraic proof system that is based on Hilbert’s Nullstellensatz. A Nullstellensatz proof of f = 0 from F is a sequence of polynomials (g1, . . . , gm; q1, . . . , qn) such that m n X X 2 gifi + qj(xj − xj) = f. (1) i=1 j=1

Note that the proof is sound in the sense every 0/1-assignment that satisfies F also satisfies f = 0. The degree of the Nullstellensatz proof is max {deg(gifi): i ∈ [m]} ∪ {deg(hj) + 2 : j ∈ [n]} . The size of the derivation is the sum of the sizes of the binary encoding of the polynomials f, gifi, 2 qj(xj − xj), each represented as a sum of monomials. A Nullstellensatz refutation of F is a proof of −1 = 0 from F, in which case F is unsatisfiable (i. e., has no 0/1-solution). The Nullstellensatz system is also complete: If F is an unsatisfiable system of multi-linear polynomials, then it has a refutation of degree at most n. Nullstellensatz is a static (or one-shot) proof system, as it provides the whole proof at once. The dynamic version of Nullstellensatz is polynomial calculus (PC) [5]. It consists of the following derivation rules for polynomial equations (fi = 0) ∈ F, polynomials f, g, variables xj, and numbers a, b ∈ R:

f = 0 g = 0 f = 0 , 2 , , . (2) fi = 0 xj − xj = 0 xjf = 0 ag + bf = 0

A polynomial calculus derivation of f = 0 from F is a sequence (r1 = 0, . . . , rL = 0) of polynomial equations that are iteratively derived using the rules (2) and lead to f = rL = 0. The degree of a derivation is the maximum degree of the polynomials in the derivation and the size is the sum of the sizes of the binary encoding of the polynomials in the derivation. A polynomial calculus refutation is a derivation of −1 = 0. It is straightforward to check that polynomial calculus simulates Nullstellensatz: If F has a Nullstellensatz refutation of degree d and size N, then it has a polynomial calculus refutation of degree d and size polynomial in N. In both systems proofs of bounded degree d can be found in time nO(d): for Nullstellensatz the coeﬃcients of the polynomials can be computed by solving a system of linear equations of size nO(d), and for polynomial calculus this can be done by using a bounded degree variant of the Gr¨obnerbasis algorithm [5].

3 2.2 Semi-algebraic proof systems: Sherali-Adams, Sum-of-Squares, Positivstel- lensatz Sherali-Adams is a static proof system that models the Sherali-Adams lift-and-project hierarchy of linear programming relaxations [19]. It can also be viewed as an extension of the Nullstel- lensatz system. A Sherali-Adams proof of f > 0 from (F, H) is a sequence of polynomials (g1, . . . , gm; q1, . . . , qn; p0, . . . , ps) such that

m n s X X 2 X gifi + qj(xj − xj) + p0 + p`h` = f, (3) i=1 j=1 `=1

P ` Q Q and where every p` has the form p` = A,B aA,B j∈A xj j∈B(1 − xj) with non-negative ` 2 n n coefficients aA,B. Note that the polynomials p` : R → R are positive in [0, 1] and hence the proof is sound in the sense every 0/1-assignment that satisfies F and H also satisfies f > 0. The degree (sometime called rank) of a Sherali-Adams proof is the maximum degree of the polynomials 2 gifi, qj(xj − xj), p0, p`h` and the size is the sum of the sizes of their encoding. A Sherali-Adams refutation of (F, H) is a proof of −1 > 0 from (F, H). Note that every Nullstellensatz refutation of F is a Sherali-Adams refutation of (F, ∅) by choosing p0 = 0. Sum-of-squares (SOS) is a semi-algebraic proof system that extends Nullstellensatz and Sherali-Adams. It models the Lasserre hierarchy of semi-definite programming relaxations [13], for which reason it is sometimes called Lasserre, and also builds on Putinar’s Positivstellensatz [18]. The difference to Sherali-Adams is that the positive polynomials p` are now sums of squares. Formally, a sum-of-squares proof of f > 0 from (F, H) is a sequence of polynomials (g1, . . . , gm; q1, . . . , qn; p0, . . . , ps) such that

m n s X X 2 X gifi + qj(xj − xj) + p0 + p`h` = f, (4) i=1 j=1 `=1

Pt` 2 and where every p` has the form p` = c=1(p`,c) (and is encoded as such) for arbitrary polynomials p`,c (in standard monomial form). Again, the degree of a proof is the maximum 2 degree of the polynomials gifi, qj(xj − xj), p0, p`h`, the size is the sum of the sizes of their encoding. A sum-of-squares refutation is a proof of −1 > 0. It is not hard to see that the positive P Q Q polynomials p = A,B aA,B j∈A xj j∈B(1 − xj) in the Sherali-Adams proof system have a sum-of-squares proof (from F = H = ∅) of degree |A| + |B| + 1 and size poly(p). It immediately follows that sum-of-squares simulates Sherali-Adams.

Lemma 2.1. If (F, H) has a Sherali-Adams refutation of degree d and size N, then it has a sum-of-squares refutation of degree d + 1 and size poly(N).

Another semi-algebraic system that is related to sum-of-squares is Positivstellensatz. It builds on Stengle’s Positivstellensatz (independently proven by Krivine [12] and Stengle [20]), which has also been used to define a hierarchy of relaxations, see [17]. Our definition of the Positivstellensatz proof system follows the one introduced in [10], a different way of formalising Stengle’s Positivstellensatz as a proof system (without focusing on complexity) was presented in [14]. We remark that Stengle’s Positivstellensatz and the Positivstellensatz proof system as defined 2 in [10] do not necessarily include the Boolean axioms xj − xj and also work for polynomials over non-Boolean variables. To be precise, we will call the system that is named “Positivstellensatz” in [10] “non-Boolean Positivstellensatz” in this paper (see Section 2.4). To define the proof system, we consider for the system of polynomial inequalities H = {h1 > 0, . . . , hs > 0} the system Q Hb = { `∈I h` > 0 : I ⊆ [s]}, which extends H by taking products of polynomial inequalities.

2 We assume that the p` are explicitely provided in this form, whereas gi and qj are arbitrary polynomials encoded in the standard way as a sum of monomials.

4 Clearly, (F, H) is satisfiable if and only if (F, Hb) is satisfiable. A Positivstellensatz proof of f > 0 from (F, H) is a sum-of-squares proof of f > 0 from (F, Hb). Note that on systems of polynomial equations (where H = ∅) sum-of-squares and Positivstellensatz are the same. One way of combining polynomial calculus with semi-algebraic proof systems is as follows. Note that a Sherali-Adams, sum-of-squares, or Positivstellensatz proof of f > 0 can be decomposed to X g + p0 + p`h` = f, (5) ` m n X X 2 gifi + qj(xj − xj) = g, (6) i=1 j=1 where (6) is a Nullstellensatz proof of g = 0. By replacing this Nullstellensatz proof of g = 0 with a polynomial calculus proof of g = 0, we obtain dynamic versions of the static semi-algebraic proof systems. The dynamic version of Positivstellensatz is called Positivstellensatz calculus and was also introduced in [10]. However, the proof of Theorem 1.1 (in particular Lemma 3.1) implies that Positivstellensatz and Positivstellensatz calculus can simulate each other. Corollary 2.2. If (F, H) has a Positivstellensatz calculus refutation of degree d and size S, then it has a Positivstellensatz refutation of degree 2d and size poly(S). Proof. By definition, a Positivstellensatz calculus refutation of (F, H) is a polynomial calculus P derivation of −1 − p0 − ` p`h` from F, where h` ∈ Hb. By Lemma 3.1, there is a degree-2d, size poly(S) sum-of-squares proof of non-negativity of

P 2 2 P P P − −1 − p0 − ` p`h` = −1 − 2p0 − p0 − (2 + 2p0) ` p`h` − ` `0 p`p`0 h`0 h` , (7)

from (F, Hb), which in turn is a Positivstellensatz refutation of (F, H). For completeness, we mention that there are also dynamic semi-algebraic proof systems that are based on the Lov´asz-Schrijver lift-and-project method [15] and where one can infer polynomial inequalities line-by-line (see [9] for an overview). These systems are, however, much stronger and somewhat diﬀerent from the proof systems considered in this paper.

2.3 Twin variables In all the proof systems mentioned above, it might be useful to introduce twin variables: for ¬ every variable xj one has available the formal variable xj that expresses its “negation” 1 − xj. ¬ To ensure that they are complementary, the additional polynomial equality xj + xj = 1 is always present in F. Except for Sherali-Adams this does not change the definition of the proof systems, as it only affects the input encoding. For Sherali-Adams with twin variables, it is additionally P ` Q Q ¬ assumed that every p` has now the form p` = A,B aA,B j∈A xj j∈B xj [7]. Note that inclusion of twin variables does not affect the degree of a refutation, but it might Q Θ(n) affect the size, as for example the polynomial j∈[n](1 − xj), which has size 2 , can be more Q ¬ succinctly expressed as j∈[n] xj , which is of size Θ(n). We are, however, not aware of any formal separation of (semi-)algebraic proof systems with and without twin variables with respect to proof size. Twin variables are particularly useful when encoding CNF formulas into polynomial equations. It is known that polynomial calculus with twin variables, which is called polynomial calculus resolution (PCR) [1], can polynomially simulate the resolution calculus [5, 1]. The same is true for Sherali-Adams [7] and hence sum-of-squares, but not for Nullstellensatz3. Remark 2.3. Theorem 1.1 and Theorem 1.2 remain true in the presence of twin variables.

3This essentially follows from the degree lower bounds in [4] and Lemma 4.8.

5 2.4 The non-Boolean case It is also conceivable to consider (semi-)algebraic proof systems over non-Boolean variables. In 2 this case the additional Boolean axioms xj − xj = 0 are omitted in the definitions (formally, we require that qj = 0 in the above definitions). Note that there is no meaningful non-Boolean variant of the Sherali-Adams proof system, as its correctness (specifically, the non-negativity of the polynomials p`) crucially depends on the fact that all variables are between 0 and 1. However, non-Boolean variants of Nullstellensatz, polynomial calculus, sum-of-squares, and Positivstellensatz are still sound proof systems. It follows from Stengle’s Positivstellensatz [20], that Positivstellensatz is also refutational complete in this setting. For sum-of-squares this does only hold if we put additional requirements on F ∪ H (being Archimedian [18]). Non-Boolean Nullstellensatz and polynomial calculus are only complete over algebraically closed fields (such as the complex numbers). We remark that in these systems it is no longer the case that every unsatisfiable multi-linear system of equations over n variables has a refutation of degree n: for example, the so-called ts 2 2 2 telescopic system Fn := {yx1 = 1, x1 = x2, x2 = x3, . . . , xn−1 = xn, xn = 0} requires exponential refutation degree in Nullstellensatz [3] and sum-of-squares [10]. Moreover, the same example shows that the simulation of polynomial calculus by sum-of-squares (Theorem 1.1) does not hold in the non-Boolean case:

ts Theorem 2.4 ([10]). Let Fn be the telescopic system as deﬁned above.

ts O(n) 1. Fn has a non-Boolean Nullstellensatz (hence sum-of-squares) refutation of degree 2 .

ts 2. Fn has a non-Boolean polynomial calculus refutation of degree O(n).

ts Ω(n) 3. Every non-Boolean sum-of-squares refutation of Fn has degree 2 .

3 Sum-of-Squares Simulates Polynomial Calculus

This section is dedicated to the proof of Theorem 1.1. Let us fix an unsatisfiable system of polynomial equations F = {f1 = 0, . . . , fm = 0}. Let (r1 = 0, . . . , rL = 0) be a polynomial calculus derivation of rL = 0 from F of degree d and size S. Let a be the minimal integer such −1 2 that every coefficient c in the proof satisfies a 6 4c 6 a. Hence, the largest encoding size of coefficient is Θ(log a). Theorem 1.1 follows immediately from the following inductive lemma.

Lemma 3.1. There are polynomials q1, . . . , qL and p1, . . . , pL of size at most poly(S) such that −Lb Lb for every Lb 6 L there are nonnegative coeﬃcients a 6 ai, b`, c` 6 a , such that

m Lb Lb X X X (−a f )f + b q (x2 − x ) + c p2 = −(r )2 (8) i i i ` ` j` j` ` ` Lb i=1 `=1 `=1 is a sum-of-squares proof of −(r )2 0 of degree 2d. Lb > Proof. First note that (8) is indeed a sum-of-squares proof of the form (4) since

Lb n X X X b q (x2 − x ) = b q (x2 − x ) (9) ` ` j` j` ` ` j j `=1 j=1 `: j`=j

2 √ 2 and c`p` = ( c`p`) (as we require c` > 0). Although we shall ﬁrst provide the polynomials q` and p`, we just assume that we have already done so and postpone their deﬁnition for ease of exposition. The proof is now by induction on Lb and we do a case analysis on the four types of derivation rules (2). First suppose that r = f is an axiom from F. Then we can easily derive Lb i

6 −(r )2 in sum-of-squares by defining p = q := 0, setting a to 1 and all other coefficients to 0. Lb Lb Lb i The case of a Boolean axiom r = x2 − x is also simple. We define q := −(x2 − x ) as well as Lb j j Lb j j p := 0, set b to 1 and all other coefficients to 0 in order to derive −(r )2. Lb Lb Lb Now suppose that r = x 0 r 0 is obtained by multiplying a previously derived polynomial r 0 Lb j L L 0 (for some L < L) by a variable xj0 . By induction assumption we have a sum-of-squares proof of 2 −(rL0 ) > 0 of degree 2d: m L0 L0 X X 2 X 2 2 (−a f )f + b q (x − x ) + c p = −(r 0 ) . (10) i i i ` ` j` j` ` ` L i=1 `=1 `=1 2 Now we want to turn this proof into a proof of −(xj0 rL0 ) > 0. Of course, we could do this by 2 just multiplying everything by xj0 . However, this would increase the degree of the refutation to 2d + 2! Instead, we use the sum of squares polynomials in order to simulate the multiplication rule in polynomial calculus without increasing the degree. We define p := r 0 − x 0 r 0 as well as Lb L j L 2 q := −2(r 0 ) and observe that Lb L 2 2 2 2 2 2 2 2 2 (p ) + q · (x 0 − x 0 ) = (r 0 ) − 2x 0 (r 0 ) + x (r 0 ) − 2x 0 (r 0 ) + 2x 0 (r 0 ) (11) Lb Lb j j L j L j L j L j L 2 2 = (rL0 ) − (xj0 rL0 ) . (12) 2 By adding them to (10) we derive −(xj0 rL0 ) > 0 without increasing the degree. Formally, we define j := j0, set b = c = 1 and obtain Lb Lb Lb

m Lb Lb X X 2 X 2 2 2 (−a f )f + b q (x − x ) + c p = −(x 0 r 0 ) = −(r ) . (13) i i i ` ` j` j` ` ` j L Lb i=1 `=1 `=1

The remaining case is derivation of r = a · r 0 + b · r 00 for a, b ∈ as a linear combination of Lb L L R two previously derived polynomials rL0 and rL00 . By induction assumption we have m L0 L0 X 0 X 0 2 X 0 2 2 (−a f )f + b q (x − x ) + c p = −(r 0 ) and (14) i i i ` ` j` j` ` ` L i=1 `=1 `=1 m L00 L00 X 00 X 00 2 X 00 2 2 (−a f )f + b q (x − x ) + c p = −(r 00 ) . (15) i i i ` ` j` j` ` ` L i=1 `=1 `=1 2 2 2 2 2 Our goal is to devise a sum-of-squares proof of −(r ) = −a (r 0 ) − 2ab · r 0 r 00 − b (r 00 ) . For Lb L L L L 2 this we deﬁne p := a·r 0 −b·r 00 and q := 0. To derive −(r ) , we multiply the sum-of-squares Lb L L Lb Lb proof (14) by 2a2, multiply (15) by 2b2, and then add both proofs together with (p )2. More Lb 2 0 2 00 2 0 2 00 2 0 2 00 precisely, we set ai = 2a ai + 2b ai for all i ∈ [m]; b` = 2a b` + 2b b` , c` = 2a c` + 2b c` for all ` max(L0,L00); c = 1 and set the remaining coeﬃcients to 0. Then we obtain 6 Lb

m Lb Lb X X 2 X 2 2 2 2 2 2 (−a f )f + b q (x − x ) + c p = −2a (r 0 ) − 2b (r 00 ) + (p ) (16) i i i ` ` j` j` ` ` L L Lb i=1 `=1 `=1 2 2 2 2 = −a (rL0 ) − b (rL00 ) − 2ab · rL0 rL00 (17) = −(r )2 (18) Lb 2 2 −1 1 By the deﬁnition of a, the factors 2a and 2b are bounded by 2a and 2 a from below and −Lb+1 0 0 0 00 00 00 Lb−1 above. Since by induction assumption we have a 6 ai, b`, c`, ai , b` , c` 6 a , it follows that −Lb Lb a 6 ai, b`, c` 6 a . This concludes the proof of Lemma 3.1. Proof of Theorem 1.1. The theorem follows immediately from Lemma 3.1, since every degree-d polynomial calculus derivation of −1 = 0 can be transformed into a degree-2d sum-of-squares proof of non-negativity of −(−1)2 = −1. By the requirements in the Lemma the size of the sum-of-squares proof is poly(S).

7 4 Sherali-Adams does not Simulate Polynomial Calculus

The system of polynomial equations that separates Sherali-Adams from polynomial calculus (Theorem 1.2) is a variant of the pebbling contradictions, which are unsatisﬁable propositional formulas that are based on the black pebble game. These formulas and their variants have found several applications in propositional proof complexity. For an in-depth treatment of the history and some of the applications of pebbling in proof complexity we refer the reader to the survey [16]. Let us ﬁx some notation. In a directed graph G = (V,E) we let N −(v) = {u :(u, v) ∈ E} be the set of incoming and N +(v) = {w :(v, w) ∈ E} be the set of outgoing neighbours of a vertex v ∈ V . The vertex sets S = {v : N −(v) = ∅} and T = {v : N +(v) = ∅} are called the sources and the sinks of G.A circuit is a directed acyclic graph G with a unique sink t and where every non-source vertex v ∈ V \ S has two incoming neighbours. The (black) pebble game is a one-player game played on a circuit G = (V,E). The player has available a pool of P pebbles and the game proceeds by placing and removing pebbles on the vertices of G. In each round the player can do one of the following moves: 1. place a pebble on a sink vertex s ∈ S, 2. place a pebble on w ∈ V \ S if there are pebbles on both vertices in N −(w), or 3. remove an arbitrary pebble. The player wins the game when he places a pebble on the sink node t. It is obvious, that the player can always win the game with |V | pebbles and the (black) pebbling price Peb(G) 6 |V | is the minimal number P such that the player wins the black pebble game on G with P pebbles. For our lower bounds we will consider circuits G with high pebbling price. Theorem 4.1 ([8]). For every large enough n there is a circuit G with n vertices and Peb(G) = Ω(n/ log n).

The pebbling contradiction FG for a circuit G = (V,E) is the system of polynomial equations over Boolean variables {xv : v ∈ V } that contains the following equations:

xs = 1, for all s ∈ S, (19) − xuxv = xuxvxw, for all w ∈ V \ S and N (w) = {u, v}, and (20)

xt = 0, for the sink t. (21)

It is easy to see that this system is unsatisﬁable. Moreover, we remark that FG is the standard encoding of the CNF pebbling contradiction, which contains clauses xs, xu ∨ xv ∨ xw, and xt. As this CNF can be easily refuted in resolution using unit propagation, it follows that this system is easy to refute in any proof system that simulates resolution, such as polynomial calculus, Sherali-Adams, and sum-of-squares. For later reference, the next lemma formulates this claim for polynomial calculus.

Lemma 4.2. FG has a polynomial calculus refutation of degree 3 and size O(n) for any n-vertex circuit G. Proof. For a vertex v ∈ V let dist(v) be the smallest distance to a source vertex in S. By induction on dist(v) we derive the equation xv = 1. If dist(v) = 0, then this equation is an axiom. For the induction step let w be a vertex with incoming neighbours N −(w) = {u, v} and assume that (a) xu = 1 as well as (b) xv = 1 was already derived. We also have the axiom (c) xuxv = xuxvxw available. Multiplying (a) with xv gives (d) xuxv = xv and by a linear combination with (b) we get (e) xuxv = 1. Multiplying (e) with xw results in (f) xuxvxw = xw. Now we can derive xw = 1 by a linear combination of (c), (e), and (f). The lemma follows since from xt = 1 and the axiom xt = 0 we can derive −1 = 0.

8 In [4] it was shown that every Nullstellensatz refutation of FG requires degree Peb(G) and hence this system separates Nullstellensatz degree from polynomial calculus degree. However, it is not hard to construct a Nullstellensatz refutation of FG that has size poly(n). Therefore, this example does not separate both systems with respect to proof size. Moreover, as mentioned before, this system is also easy for Sherali-Adams (with respect to size and degree). To prove our separation theorem between Sherali-Adams and polynomial calculus, we modify the formula a bit in order to make it hard for Sherali-Adams, while at the same time it remains easy for polynomial calculus. We do this by substituting for every variable xv the sum of fresh variables according to the following deﬁnition.

Deﬁnition 4.3. Let F be a set of polynomial equations over variables x1,..., xn and k > 1. The system F[+k] is obtained from F be replacing every variable xi in every f ∈ F by the sum xi,1 + ··· + xi,k of k new variables and including the additional polynomial equations xi,`xi,`0 = 0 0 for all i ∈ [n] and 1 6 ` < ` 6 k. The following lemma shows that after substitution the system remains easy to refute in polynomial calculus. Lemma 4.4. Let F be a set of polynomial equations and suppose there is a polynomial calculus refutation of F of degree d and size S. Then F[+k] has a polynomial calculus refutation of degree d and size O(kdS).

Proof. We obtain the new proof by substituting all variables xi by xi,1 + ··· +xi,k and expand the polynomials to monomial form (this increases the size by a factor of kd). It remains to check that the substituted equations form a polynomial calculus refutation of F[+k]. It is clear that a former derivation of an axiom f ∈ F is now a derivation of an substituted axiom from F[+k]. A derivation 2 P 2 P of a Boolean axiom xi = xi translates to ( `∈[k] xi,`) = `∈[k] xi,`, which can be derived using 2 the Boolean axioms xi,` = xi,` and the additional equations xi,`xi,`0 = 0 (see Deﬁnition 4.3). The substituted variant of a linear combination of two previously derived polynomials f, g is just the linear combination of the substituted versions of f and g. Multiplication by a variable xj to a P polynomial in the original proof translates to multiplying by `∈[k] xj,`, which can be simulated by k separate multiplications of xj,1, . . . , xj,k and subsequent addition steps. To obtain a system of equations that is hard for Sherali-Adams and easy for polynomial calculus we apply two substitution steps to the formula FG for circuits from Theorem 4.1. First we prove that every refutation of FG[+n] in Sherali-Adams requires degree d = Peb(G). In the second step we show that a degree d lower bound for an arbitrary instance F translates to a Ω(d) 2 size lower bound for F[+2]. Together we obtain that FG[+n][+2] requires high degree and size in Sherali-Adams. We will use a common approach for proving lower bounds in static proof systems and deﬁne a solution for the “dual” system.

Deﬁnition 4.5. A mapping D : R[x1, . . . , xn] → R is a d-evaluation if it satisﬁes the following conditions.

(D1) D is linear: D(af + bg) = aD(f) + bD(g) for all f, g ∈ R[x1, . . . , xn]

Q dj Q (D2) D is multi-linear: D( j xj ) = D( j xj)

(D3) D(f · fi) = 0 for every axiom fi ∈ F and f ∈ R[x1, . . . , xn] with deg(f) 6 d − deg(fi) Q Q (D4) D j∈A xj j∈B(1 − xj) > 0 for all A, B ⊆ [n] with |A ∪ B| 6 d. It is not hard to verify that the existence of a d-evaluation implies that there is no Sherali- Adams refutation of degree d: suppose for contradiction that there is a Sherali-Adams refutation of degree d of the form m n X X 2 gifi + qj(xj − xj) + p0 = −1, (22) i=1 j=1

9 P 0 Q Q with p0 = A,B aA,B j∈A xj j∈B(1 − xj). Now we apply D to both sides of the equation. 2 From (D3) it follows that D(gifi) = 0, from (D2) we obtain D(qj(xj − xj)) = 0, and from (D4) it follows that D(p0) > 0. By linearity (D1) the left hand side is evaluated to something non-negative, whereas on the right-hand side we have D(−1) = −1. Due to the multi-linearity (D2) the lower bound technique actually proves something stronger. The ml-degree of a polynomial is the degree of its multi-linearisation, i. e., the maximum number of distinct variables in a monomial. We immediately get the following lemma.

Lemma 4.6. If a system of multi-linear equations F has a d-evaluation D, then there is no Sherali-Adams refutation of F that has ml-degree 6 d.

The next lemma is proven by constructing a d-evaluation.

Lemma 4.7. Let G be a circuit with n vertices. Every Sherali-Adams refutation of FG[+k] requires ml-degree at least min(Peb(G), k/2).

Proof. Let d < min(Peb(G), k/2) and suppose for contradiction that there is a Sherali-Adams refutation of ml-degree d. By Lemma 4.6 it suffices to define an operator D that satisfies (D1)– (D4). We start by defining D on multi-linear terms. We call a multi-linear term inconsistent, Q if it contains two distinct variables xv,` and xv,`0 for some v ∈ V . If g = (v,`)∈I xv,` is an Q Q inconsistent term, we define D(g) := 0. Otherwise, g = (v,`)∈I xv,` = u∈U xu,ù and the value V of D(g) := De(U) will only depend on the set U ⊆ V . To define the mapping De : 2 → R, we say that U ⊆ V is reachable, if the player has a strategy in the black pebble game with d pebbles to reach a position where exactly the vertices in U are pebbled. The mapping is now defined as follows. ( 1 |U|, if U is reachable, De(U) := k (23) 0, otherwise.

We extend the definition of D to all polynomials by (multi-)linearity. Note that this completes the definition of D and immediately satisfies (D1), (D2), as well as (D3) for the axioms xi,`xi,`0 = 0 introduced by Definition 4.3. To verify (D4), we have to show that D(p) > 0 for every polynomial Q Q p = (v,`)∈I xv,` (v,`)∈J (1 − xv,`) of degree at most d. First note that if I ∩ J 6= ∅, then D(p) = 0 since the mapping D satisfies (D2). Therefore, we may assume that p is multi- Q linear when multiplied out to monomial form. If (v,`)∈I xv,` is either inconsistent or it is consistent and defines a non-reachable position, then D(p) = 0 and we are done. Otherwise, Q −|I| D( (v,`)∈I xv,`) = k and we get   |I| 1 X Y D(p) = + (−1)|K| D x k  v,` ∅6=K⊆J (v,`)∈K∪I |I| |I|+|K| 1 X 1 − > k k ∅6=K⊆J   |I| |J| z 1 X |J| 1 = 1 − . k  z k  z=1

Because we have have |J| 6 d < k/2 it follows that

|J| z |J| z z ∞ X |J| 1 X k 1 X < < 2−z = 1. z k 2 k z=1 z=1 z=1

10 Hence, D(p) > 0 and property (D4) is proven. It remains to verify (D3) for all three types of substituted axioms. For every multi-linear term g we need to check:

Pk D g · `=1 xs,` = D(g) , (24) Pk Pk Pk Pk Pk D g · `=1 xu,` `=1 xv,` = D g · `=1 xu,` `=1 xv,` `=1 xw,` , (25) Pk D g · `=1 xt,` = 0, (26)

where s ∈ S is a source, w ∈ V \ S with N −(w) = {u, v}, and t is the sink. First suppose that g is either inconsistent or deﬁnes a position U that is not reachable. In both cases everything Q above evaluates to 0. Hence, let g = u∈U xu,`u for a reachable vertex set U. In the case of (24), we have |U| 6 d − 1. If s ∈ U, then D(xs,`s g) = D(g), since we satisfy (D2). Since the other summands xs,`g are inconsistent for ` 6= `s, they evaluate to 0 and the equality (24) holds. Now assume that s∈ / U. We have that U ∪ {s} is reachable as well, since the player has at least one pebble remaining and can place it on the source s. It follows that Pk 1 |U|+1 1 |U| D g · `=1 xs,` = k · ( k ) = ( k ) = D(g). Checking (25) for non-source vertices w with N −(w) = {u, v} is similar. Here we have |U| 6 d − 3 and by the rules of the game we know that U ∪ {u, v} is reachable if and only if U ∪ {u, v, w} is reachable. Hence, if U ∪ {u, v} is not reachable, both sides evaluate to 0. Otherwise, by a case analysis on the shape of U ∩ {u, v, w}, one can easily verify that both sides 1 |U| evaluate to ( k ) . For the source vertex t, note that since d < Peb(G), no position that contains t is reachable. Hence, D(gxt,`) = 0 for all ` ∈ [k] and the equality (26) holds. This concludes the proof of the lemma.

Lemma 4.7 (together with Theorem 4.1 and Lemma 4.2) already provides a separation between degree in Sherali-Adams and polynomial calculus. To separate the proof size we need the following lifting lemma.

Lemma 4.8. Let F be a system of multi-linear polynomial equations and let P be one of the proof systems Nullstellensatz, Sherali-Adams, or sum-of-squares. If every P-refutation of F has d ml-degree at least d, then every P-refutation of F[+2] has ml-degree at least d and size Ω(2 ).

Proof. Let F = {f1 = 0, . . . , fm = 0} over variables {x1, . . . , xn} and consider a P-refutation

m n 2 X 0 X X 2 gifi + qj,`(xj,` − xj,`) + p0 = −1 (27) i=1 j=1 `=1

0 0 of F[+2] = {f1 = 0, . . . , fm = 0} ∪ {x1,1x1,2 = 0, . . . , xn,1xn,2 = 0}. Suppose that this refutation d−1 d−1 has size 2 and let L 6 2 be the total number of large monomials of ml-degree > d in 2 the refutation (i. e. in the polynomials gifi, qj(xj − xj), and p0). We consider the set Γ of all restrictions that set for every j exactly one of the variables xj,1 and xj,2 to 0 and leaves the other variable unset. It follows that for every γ ∈ Γ the set of equations Fγ that results from F[+2] by restricting the variables according to γ agrees with F (modulo renaming the variables). Moreover, by applying the restriction to (27) one obtains a P-refutation of Fγ. It remains to argue that if L is too small, then choosing a restriction γ ∈ Γ uniformly at random might end up with a refutation of Fγ of ml-degree < d, contradicting the assumption. This follows by a simple union bound argument. First note that the probability that a monomial of ml-degree > d 1 d 4 is not set to 0 by a restriction γ is 6 ( 2 ) . Furthermore, the probability that the restricted

4This claim does only hold for “ml-degree” and not for “degree” and that’s why we consider this notion.

11 sum-of-squares

polynomial calculus Sherali-Adams

Nullstellensatz

Figure 1: Relation between the proof systems. An arrow P → Q indicates that a proof in system P of degree d and size S can be converted into a proof in system Q of degree O(d) and size poly(S). Whenever there is no irreﬂexive arrow, it is known that the simulation does not hold.

refutation has ml-degree < d is bounded by the probability that at least one large monomial does not vanish. Since we have

1 d 1 Pγ∈Γ[ex. monomial of ml-degree > d that does not vanish] 6 L · 2 6 2 , (28) which is bounded away from 1, the lemma follows.

By combining Lemma 4.7 and Lemma 4.8 we can now prove Theorem 1.2.

Proof of Theorem 1.2. Let G be a circuit from Theorem 4.1 on k vertices. By Lemma 4.7 we obtain that FG[+k] requires Sherali-Adams refutations of ml-degree Ω(k/ log k). By Lemma 4.8 it follows that every Sherali-Adams refutation of FG[+k][+2] requires ml-degree (and hence degree) Ω(k/ log k) and size 2Ω(k/ log k). On the other hand, Lemma 4.2 combined with Lemma 4.4 4 shows that FG[+k][+2] has a polynomial calculus refutation of degree 3 and size O(k ). Since 2 FG[+k][+2] has n = 2k variables, the theorem follows.

5 Conclusions

We compared the static semi-algebraic proof systems Sherali-Adams and sum-of-squares with polynomial calculus, a dynamic algebraic proof system. The main results show that sum-of- squares simulates polynomial calculus (Theorem 1.1), while Sherali-Adams is not able to do so (Theorem 1.2). The relations between the proof systems considered in this paper are described in Figure 1. One open question concerns the separation between polynomial calculus and Sherali-Adams. Note that the pebbling contradiction FG that separates polynomial calculus degree from Null- stellensatz degree is a system of polynomial equations that encodes a CNF formula. This is no longer the case for the substituted formula FG[+k][+2] that separates polynomial calculus from Sherali-Adams, and encoding FG[+k][+2] as a CNF blows up its size exponentially. It would therefore be nice to know whether there is a separating CNF. Note that such a CNF would have to be hard for resolution as well, which is not the case for the substituted variants of the pebbling contradictions (that are in conjunctive normal form) considered in the literature (see [16]).

References

[1] Michael Alekhnovich, Eli Ben-Sasson, Alexander A. Razborov, and Avi Wigderson. Space complexity in propositional calculus. SIAM J. Comput., 31(4):1184–1211, 2002. URL: https://doi.org/10.1137/S0097539700366735, doi:10.1137/S0097539700366735.

[2] Paul Beame, Russell Impagliazzo, Jan Kraj´ıˇcek,Toniann Pitassi, and Pavel Pudl´ak.Lower bounds on hilbert’s nullstellensatz and propositional proofs. Proceedings of the London Mathe- matical Society, s3-73(1):1–26, 1996. URL: http://dx.doi.org/10.1112/plms/s3-73.1.1, doi:10.1112/plms/s3-73.1.1.

12 [3] W. Dale Brownawell. Bounds for the degrees in the nullstellensatz. Annals of Mathematics, 126(3):577–591, 1987. URL: http://www.jstor.org/stable/1971361.

[4] Josh Buresh-Oppenheim, Matthew Clegg, Russell Impagliazzo, and Toniann Pitassi. Homoge- nization and the polynomial calculus. Computational Complexity, 11(3-4):91–108, 2002. URL: https://doi.org/10.1007/s00037-002-0171-6, doi:10.1007/s00037-002-0171-6.

[5] M. Clegg, J. Edmonds, and R. Impagliazzo. Using the Groebner basis algorithm to ﬁnd proofs of unsatisﬁability. In Proceedings of the 28th annual ACM symposium on Theory of computing, pages 174–183, 1996.

[6] Stephen A. Cook and Robert Reckhow. The relative eﬃciency of propositional proof systems. Journal of Symbolic Logic, 44(1):36–50, March 1979.

[7] Stefan S. Dantchev, Barnaby Martin, and Mark Nicholas Charles Rhodes. Tight rank lower bounds for the sherali-adams proof system. Theor. Comput. Sci., 410(21-23):2054–2063, 2009. URL: https://doi.org/10.1016/j.tcs.2009.01.002, doi:10.1016/j.tcs.2009. 01.002.

[8] John R. Gilbert and Robert Endre Tarjan. Variations of a pebble game on graphs. Tech- nical Report STAN-CS-78-661, Stanford University, 1978. Available at http://infolab. stanford.edu/TR/CS-TR-78-661.html.

[9] Dima Grigoriev, Edward A. Hirsch, and Dmitrii V. Pasechnik. Complexity of semi-algebraic proofs. Moscow Mathematical Journal, 2(4):647–679, 2002. URL: http://www.ams.org/ distribution/mmj/vol2-4-2002/abst2-4-2002.html.

[10] Dima Grigoriev and Nicolai Vorobjov. Complexity of null- and positivstellensatz proofs. Annals of Pure and Applied Logic, 113(1–3):153–160, 2001.

[11] Russell Impagliazzo, Pavel Pudl´ak,and Jir´ıSgall. Lower bounds for the polynomial calculus and the gr¨obnerbasis algorithm. Computational Complexity, 8(2):127–144, 1999. URL: https://doi.org/10.1007/s000370050024, doi:10.1007/s000370050024.

[12] J. L. Krivine. Anneaux préordonnés. Journal d’Analyse Mathématique, 12(1):307–326, Dec 1964. URL: https://doi.org/10.1007/BF02807438, doi:10.1007/BF02807438.

[13] J. B. Lasserre. Global optimization with polynomials and the problem of moments. SIAM Journal on Optimization, 11(3):796–817, 2001. doi:10.1137/S1052623400366802.

[14] H. Lombardi, N. Mnev, and M.-F. Roy. The positivstellensatz and small deduction rules for systems of inequalities. Math. Nachr., 181:245–259, 1996.

[15] LászlóLovászand Alexander Schrijver. Cones of matrices and set-functions and 0-1 optimization. SIAM Journal on Optimization, 1(2):166–190, 1991. URL: https://doi.org/ 10.1137/0801013, doi:10.1137/0801013.

[16] Jakob Nordstr¨om.Pebble games, proof complexity, and time-space trade-oﬀs. Logical Methods in Computer Science, 9(3), 2013. URL: https://doi.org/10.2168/LMCS-9(3:15)2013, doi:10.2168/LMCS-9(3:15)2013.

[17] P. Parrilo. Structured Semideﬁnite Programs and Semialgebraic Geometry Methods in Robustness and Optimization. PhD thesis, California Institute of Technology, 2000.

[18] Mihai Putinar. Positive polynomials on compact semi-algebraic sets. Indiana Univer- sity Mathematics Journal, 42(3):969–984, 1993. URL: http://www.jstor.org/stable/ 24897130.

13 [19] H. D. Sherali and W. P. Adams. A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems. SIAM Journal on Discrete Mathematics, 3(3):411–430, 1990.

[20] Gilbert Stengle. A nullstellensatz and a positivstellensatz in semialgebraic geometry. Mathe- matische Annalen, 207(2):87–97, Jun 1974. URL: https://doi.org/10.1007/BF01362149, doi:10.1007/BF01362149.

ECCC ISSN 1433-8092 14 https://eccc.weizmann.ac.il