Proof Verification and Hardness of Approximation Problems

Sanjeev Arora∗ Carsten Lund† Rajeev Motwani‡ Madhu Sudan§ Mario Szegedy¶

Abstract that finding a solution within any constant factor of optimal is also NP-hard. Garey and Johnson The class PCP(f(n), g(n)) consists of all languages [GJ76] studied MAX-CLIQUE: the problem of find- L for which there exists a polynomial-time probabilistic ing the largest clique in a graph. They showed oracle machine that uses O(f(n)) random bits, queries that if a polynomial time algorithm computes MAX- O(g(n)) bits of its oracle and behaves as follows: If CLIQUE within a constant multiplicative factor then x ∈ L then there exists an oracle y such that the ma- MAX-CLIQUE has a polynomial-time approximation chine accepts for all random choices but if x 6∈ L then scheme (PTAS) [GJ78], i.e., for any c > 1 there ex- for every oracle y the machine rejects with high proba- ists a polynomial-time algorithm that approximates bility. Arora and Safra very recently characterized NP MAX-CLIQUE within a factor of c. They used graph as PCP(log n, (log log n)O(1)). We improve on their products to construct “gap increasing reductions” that result by showing that NP = PCP(log n, 1). Our re- mapped an instance of the clique problem into an- sult has the following consequences: other instance in order to enlarge the relative gap of 1. MAXSNP-hard problems (e.g., metric TSP, the clique sizes. Berman and Schnitger [BS92] used MAX-SAT, MAX-CUT) do not have polynomial the ideas of increasing gaps to show that if the clique time approximation schemes unless P=NP. size of graphs with bounded co-degree (degree of the complement) does not have a randomized PTAS then 2. For some  > 0 the size of the maximal clique in there is an  > 0 such that MAX-CLIQUE cannot a graph cannot be approximated within a factor of be approximated within a factor of n in randomized  n unless P=NP. polynomial-time. Yet for the most part, not much could be said for a wide variety of problems until very recently. A con- 1 Introduction nection between two seemingly unrelated areas within theoretical computer science, established by Feige et The notion of NP-completeness [Coo71, Kar72, al. [FGLSS91], led to surprisingly strong hardness re- Lev73] has been used since the early seventies to show sults for approximating optimization problems. Feige the hardness of finding optimum solutions for a large et al. exploited a recent characterization of multi- variety of combinatorial optimization problems. The prover interactive proof systems by Babai, Fortnow apparent intractability of these problems motivated and Lund [BFL91] to obtain intractability results for the search for approximate solutions to these hard op- approximating MAX-CLIQUE under the assumption timization problems. For some problems this effort that NP 6⊆ DTIME(nO(log log n)). gave good approximation algorithms, but for others it Recently Arora and Safra [AS92] improved on this seemed hard even to find near optimal solutions. by showing that it is NP-hard to approximate MAX- The task of proving hardness of the approxima- CLIQUE within any constant factor (and even within tion versions of such problems met with limited suc- log n/(log log n)O(1) cess. For the traveling salesman problem without tri- a factor of 2 ). Their solution builds angle inequality Sahni and Gonzalez [SG76] showed on and further develops the techniques of Feige et al. and yields an elegant new characterization of the class ∗ Computer Science Division, U. C. Berkeley, Berkeley, CA NP in terms of probabilistically checkable proofs. 94720. Supported by NSF PYI Grant CCR 8896202. †AT&T Bell Labs, Murray Hill, NJ 07974. Relying on their work we further develop these tech- ‡Department of Computer Science, Stanford University, niques and show that there is no PTAS for a large Stanford, CA 94305. Supported by NSF Grant CCR-9010517, number of combinatorial optimization problems un- and grants from Mitsubishi and OTL. less P=NP. These problems, like traveling salesman §Computer Science Division, U. C. Berkeley, Berkeley, CA 94720. Supported by NSF PYI Grant CCR 8896202. problem with triangle inequality, minimal steiner tree, ¶AT&T Bell Labs, Murray Hill, NJ 07974. maximum directed cut, shortest superstring, etc. be- long to the class of MAXSNP-hard problems, defined A problem opt is MAXSNP-hard (complete) if ev- by Papadimitriou and Yannakakis [PY91] in terms of ery problem in MAXSNP L-reduces to it (and opt ∈ logic and reductions that preserves approximability. MAXSNP ). Papadimitriou and Yannakakis showed Our result also improves the parameters for the that if any of the MAXSNP-hard problems have a MAX-CLIQUE result of Arora and Safra. We show, PTAS then every problem in MAXSNP has a PTAS. that there is an  > 0 such that approximating MAX- MAX-3SAT is a typical MAXSNP-complete prob- CLIQUE within a factor of n in NP-hard. lem: Given a conjunction ϕ of clauses, each a disjunc- In the next section we elaborate on the definition tion of 3 variables or their negation, maximize the the of the class MAXSNP and then move on to give back- number of satisfied clauses over all possible assign- ground on probabilistically checkable proofs and for- ments to the variables of ϕ. mulate our main lemma, which is a characterization of The following problems are also MAXSNP- NP that improves on the one in [AS92]. Then we re- complete or hard: MAX-SAT; MAX-2SAT; INDE- late this characterization to the non-approximability PENDENT SET-B; NODE COVER-B; MAX-CUT; of MAXSNP. The rest of the paper will be devoted to MAX-DIRECTED CUT; METRIC TSP; STEINER the proof of the main lemma. TREE; SHORTEST SUPERSTRING; MULTIWAY CUTS; THREE DIMENSIONAL MATCHING. The 1.1 The class MAXSNP exact definitions can be found in [PY91, PY92, BP89, BJLTY91, DJPSY92, Kan91]. Given a maximization problem, formulated as optF (x) = maxy F (x, y), an approximation algorithm 1.2 Probabilistically checkable proofs A is said to achieve worst-case performance ratio α(n) −1 if for every input x : F (x, A(x)) ≥ α(n) optF (x), The notion of probabilistically checkable proofs where n is the size of x. (PCP) was introduced by Arora and Safra [AS92], In 1988 Papadimitriou and Yannakakis [PY91] us- as a slight variation of the notions of randomized or- ing Fagin’s definition of NP [Fag74] observed that acle machines due to Fortnow, Rompel and Sipser there is an approximation algorithm which has con- [FRS88] and transparent proofs due to Babai, Fort- stant performance ratio for any maximization problem now, Levin and Szegedy [BFLS91]. All these mod- that is defined by a quantifier free first order formula els are in turn variations of interactive proof systems ϕ as [Bab85, GMR89] and multiprover interactive proof systems [BGKW88]. optϕ(X) = max : |{z|ϕ(S, X, z)}| , (1) S Definition 1.3 (Arora-Safra [AS92]) A language where z is a vector of constant number of first order L is in PCP(f(n), g(n)) if there is polynomial-time variables. randomized oracle machine M y(r, x) which works as Definition 1.1 (Papadimitriou-Yannakakis) An follows: optimization problem opt(x) belongs to MAXSNP if there is a polynomial-time algorithm that encodes in- 1. It takes input x and a (random) string r of length O(f(n)), where n = |x|. put x into X in 1 such that opt(x) becomes optϕ(X). They further considered the following type of “con- 2. Generates a query set Q(r, x) = {q1, . . . , qm} of stant gap preserving” reductions: size m = O(g(n)). Definition 1.2 (Papadimitriou-Yannakakis) Let 0 opt and opt be two optimization problems defined by 3. Reads the bits yq1 , . . . , yqm . 0 0 functions F and F . We say that opt L-reduces to opt 4. Makes a polynomial-time computation using the if there exist two polynomial time algorithms f and g y data r, x and yq1 , . . . , yqm and outputs M (r, x) ∈ and constants α, β > 0 such that for each instance x {0, 1}. of opt: Moreover the following acceptance conditions hold for 0 0 1. Algorithm f produces an instance x of opt such some δ > 0 and for all x: that opt0(x0) ≤ αopt(x). 1. If x ∈ L then there exists a y such that for every 0 2. For any y algorithm g outputs a y with the prop- r we have M y(r, x) = 1. erty: 2. If x 6∈ L then for every y we have 0 0 0 0 y |F (x, y) − opt(x)| ≤ β|F (x , y ) − opt(x )|. P robr(M (r, x)) = 0) ≥ δ. Observe that the definition does not say anything Proof checking itself is a maximization problem about the length of y, but it is easy to see that from the point of view of the prover: the task is we can assume without loss of generality that |y| = to maximize the chance of acceptance of the verifier. 2O(f(n))g(n). This definition is a generalization of NP More formally, let M y(x, r) be a probabilistic oracle since it is clear that NP = PCP(0, nO(1)). machine and let R be the set of all possible values of Theorem 1 (Arora-Safra [AS92]) r for a given parameter n. The prover’s intention is NP = PCP(log n, (log log n)O(1)). to solve the following optimization problem: We improve on the second parameter: y opt(x) = maxy|{r ∈ R| M (x, r) = 1}|. (2) Theorem 2 NP = PCP(log n, 1). In this paper we prefer to use the following reformu- Much of the paper will be devoted to the proof lation of this problem: of this theorem. The connection to clique approx- imations, of Feige et al., shows that if NP = y opt(x) = max P robrM (x, r). (3) PCP(log n, g(n)), then the clique size cannot be ap- y proximated within a factor of 2Ω(log n/g(n)) unless For fixed values of r and x the value of M y(x, r) can P=NP. Thus Theorem 2 has the corollary: be computed by a circuit, which takes as many input Corollary 3 There is an  > 0 such that to approxi- bits from y as the size of the query set. Let us denote mate MAX-CLIQUE within a factor of n is NP-hard. this circuit by Cx,r. Equation 3 can now be rewritten as 1.3 Related Areas opt(x) = max P robr(Cx,r(y) = 1) (4) y The results in this paper borrow significantly from If M is a PCP (f(n), g(n)) machine for a language L results in the area of self-testing/self-correcting of pro- then there exists a δ such that opt(x) is 1 if x ∈ L and grams (see [BLR90], [Rub90]). The areas of self- opt(x) ≤ 1 − δ if x 6∈ L. testing/correcting are closely connected to the areas Conversely, if Cx,r(y) is a family of circuits such of error-detection/correction in coding theory. In par- that ticular, we observe that results from the former area 1. |r| = O(f(n)); can be interpreted as yielding very efficient random- ized error-detecting and error-correcting schemes for 2. given x and r, the circuit Cx,r can be built in some well known codes. In Section 4 we use the “lin- polynomial size; earity tester” of Blum, Luby and Rubinfeld [BLR90] as an efficient error-detection scheme for the Hadamard 3. Cx,r(y) feeds from at most g(n) bits of y; Codes and this plays a crucial role in our proof. Later 4. for every x the maximization problem correspond- in Section 7 we use the “low-degree test” of Rubin- ing to Equation 4 either has solution 1 or it has feld and Sudan [RS92], with its improved efficiency solution less than 1 − δ for some fixed δ, due to a technical lemma from [AS92], as an efficient mechanism to test Reed Solomon Codes. then a probabilistic oracle machine can be built that Other ingredients in our proof borrow from work recognizes the language x : opt(x) = 1. done in “parallelizing” the MIP=NEXPTIME proto- Definition 2.1 We say that an optimization problem col [LS91],[FL92]. The result described in Section 7 opt is well behaved if there is a δ such that for every uses ideas from their work. x either opt(x) = 1 or opt(x) ≤ 1 − δ. Our observation above can be stated so that there is a one to one correspondence between well behaved 2 PCP and MAXSNP optimization problems, where the underlying family of circuits (parameterized by x and r; |r| = f(|x|)) The methods of [FGLSS91] and [AS92] have been is polynomially constructible and the problems in the applied so far only to the clique approximation prob- class PCP(f(n), g(n)), where g(n) is determined by lem. Here we show that the whole class of optimiza- the upper bound imposed upon the input size of cir- tion problems can be handled in the same vein. The cuits Cx,r where |x| = n. results of this section are due to independent observa- Let M y(x, r) be a machine PCP machine with con- tions of Mario Szegedy and Madhu Sudan and appear stant size query sets that recognizes a language L. in [AMSS92]. Then the size of the members of the the corresponding family of circuits {Cx,r|x, r} is bounded, since their Definition 3.1 A well behaved optimization problem

f(n) input size is bounded. To maximize the number of opt(x) = maxy P robr∈U {0,1} (Cx,r(y) = 1) is re- satisfiable constant size circuits each feeding from the stricted with parameters (f(n), g(n)) if same input is in MAX SNP. This can be seen or by 1. We can partition y into a disjoint union of seg- reducing the problem to MAX-3SAT or directly (due ments y , where each segment has size O(g(n)) to Yannakakis): i and that each circuit C depends only on a con- The input structure consists of a (k + 1)-ary re- x,r k stant number of such segments. lation A and 2 unary relations Bv indexed by the O(1) bit-vectors v of length k. A tuple (r, i1, ..., ik) is in A 2. Each circuit Cx,r has size (g(n)) . if and only if Cx,r inputs bits i1, ..., ik on this choice (thus, A contains one tuple for each choice). A rela- The class of languages recognized by such optimization problems we call OPT (f(n), g(n)). tion Bv contains a random choice r if and only if the We remark that PCP (f(n), g(n)) ⊇ OPT (f(n), bit vector v satisfies Cx,r. The problem is to maximize over all unary relations g(n)) is straightforward, but the other direction is true (sets) S (these are the sets of 1 bits in y) the number but not so straightforward. In Sections 4 and 7, we work towards the devel- of tuples (r, y1, ..., yk) that make the following formula true: opment of such restricted proof systems. The results of these sections show that NP ⊂ OPT (poly(n), 1)

A(r, y1, ..., yk) ∧ {[B11..1(r) ∧ S(y1) ... ∧ S(yk)] ∨ ... (Theorem 5) and NP ⊂ OPT (log n, polylog n) (Theo- rem 8). Theorems 9 and 10 show that the recursion

∨ [B0..0(r) ∧ ¬S(y1)... ∧ ¬S(yk)]} idea applies to these proof systems, and in particular shows the following: When |r| = f(n), the size of the instance {Cx,r|r} is 2|r| = 2O(f(n). Taking the advantage of the gap δ 1. OPT (f(n), g(n)) ⊂ OPT (f(n) + O(log g(n)) between the optimal value opt(x) for those instances , (log g(n))O(1)) and x that are in L versus those that are not in L, an 2. OPT (f(n), g(n)) ⊂ OPT (f(n) + (g(n))O(1), 1). algorithm could solve the membership problem for L, which approximates the opt(x) close enough to 1. This allows us to conclude Thus we conclude: that NP ⊂ OPT (log n, polyloglog n) (by composing Theorem 4 If MAX SNP complete problems have two OPT (log n, polylog n) proof systems) and then PTAS, then PCP (f(n), 1) is in DTIME(2O(f(n))). by composing this system with the OPT (poly(n), 1) proof system we obtain OPT (log n, 1) proof system for NP. 3 Outline of the Proof of the Main Theorem 4 A PCP machine for NP with con- Feige et al [FGLSS91] elaborated on the techniques stant number of queries of Babai, Fortnow and Lund [BFL91] to show NP ⊂ PCP(log n loglog n, log n loglog n). The log n loglog n In this section we prove: barrier stood in the way of a full characterization of Theorem 5 NP ⊆ PCP(poly(n), 1). NP by PCP. Arora and Safra [AS92] introduced a new When contrasted with PCP(log n, 1) = NP the technique – recursive proof checking – that enabled lemma may appear weak, nevertheless a slight vari- them to characterize NP as PCP (log n, log log n) ation of the construction is an essential ingredient of [AS92]. In this section we translate this technique into our proof of Theorem 2. our optimization problem scenario. Among the tech- We construct for 3SAT a M y(r, x) as in Definition nicalities we build on data encoding stands out, and 1.3. For every x let us turn the 3-CNF formula repre- it is also a major motif in [BFL91, BFLS91, AS92]. sented by x into an arithmetic circuit Cx over GF (2) Our proof of Theorem 2 is arrived at by the follow- with l = O(|x|) gates. Look at x as a Boolean formula ing basic steps. We observe that the recursion idea of and use DeMorgan’s laws to eliminate the OR gates [AS92] can be generalized to apply to any arbitrary and add gates such that the fanin for every gate is at proof system provided it satisfies certain restrictions. most 2. Thereafter replace every AND gate by a mul- We shall consider a restricted type of the optimization tiplication gate and every negation gate with a gate problems discussed in Section 2. that computes the function x 7→ 1 − x. If Cx is satisfiable, the successful setting of the in- distribution D.” P,Q,R,S,T,U,Z,L1,L2 are generated put as well as the values of the gates for this input set- in an obvious manner; V is generated from a random l
ting will be encoded by the prover into an exponential seed λ. Altogether 6(1 + l + 2 ) + 2l + (l + 1) ran- size table. The table will also facilitate the lookup of dom bits are used. All the computations can be done any quadratic expression of these values. If the table in polynomial time. The number of query bits is 13: errs in more than 1% of the cases, the checking will q1 = P ; ... ; q13 = U + V . We are yet to prove that reveal it with fixed positive probability. Lookups will the acceptance conditions hold. be error corrected by random reduction. Looking at If x ∈ L then there is an assignment A to the an appropriate random quadratic expression will re- variables of x that x(A) = 1. Evaluate Cx on A. veal if the circuit makes a computational error. The We define γi as the value of the gate gi under this checks are performed simultaneously. Only constantly evaluation. Clearly Pi(γ) = 0 for 1 ≤ i ≤ l, and many bits of the table will be checked. Pl+1(γ) = 0, where γ = (γ1, γ2, . . . , γl). The latter Introduce variables g1, . . . , gl corresponding to the because Cx(A) = x(A) = 1. gates of Cx. We assume that gl corresponds to the Define y by yP = P (γ) for P ∈ P2. output gate. We define: I4 holds for every r, since

l X yU + yU+V = U(γ) + (U + V )(γ) = P1 = { cigi | ci ∈ GF (2) for 1 ≤ i ≤ l}; l+1 i=1 X V (γ) = λiPi(γ). l l X X i=1 P2 = {c0 + cigi + cijgigj | ci, cij ∈ GF (2)}. The latter sum evaluates to 0 since γ was defined i=1 1≤i

P robV ∈ P (L(V ) = 0) ≥ 0.95. (9) D 2 In this section we reduce the size of the circuits that We claim that L(gi)L(gj) = L(gigj) for all 1 ≤ define well behaved optimalization problem. We will i, j ≤ l. Consider the matrices Y and Z defined by heavily use ideas in [BFLS91, AS92]. Y (i, j) = L(gi)L(gj) and Z(i, j) = L(gigj). Let the vector v = v(L1) be the vector of the coefficients of 5.1 Encoding schemes the linear function L1. Similarly w = w(L2) is the T vector of the coefficients of L2. L(L1)L(L2) = v Y w T An encoding scheme E = {En} is a polynomial and L(L1L2) = v Zw. If the linear forms L1 and n time computable sequence of functions En : {0, 1} → L2 are drawn randomly and independently from P1 {0, 1}p(n), where p(n) is a function that is polynomial then the vectors v and w are drawn randomly and l in n. Moreover there is a δ > 0 such that for every n if independently from GF (2) . x and x0 are two different strings of length n then the We now use the argument of Freivald [Fre79] to hamming distance dist(x, x0) between E(x) and E(x0) show that Y = Z. It is easily verified that if Y 6= is at least δn. An encoding scheme can be constructed Z then with probability at least 0.5, vT Y 6= vT Z. n n using the Reed-Solomon codes [MS81]. For an encod- Moreover, for vectors α, β ∈ Z2 and w∈RZ2 , if α 6= −1 T T ing scheme we define the decoding E (z) as the x β then with probability at least 0.5, α w 6= β w. that minimizes dist(E(x), z). It follows that for random v and w, if Y 6= Z then T T with probability at least 0.25, v Y w 6= v Zw. Thus 5.2 Circuit verification from Inequality 7 we get that Y = Z. This in turn vindicates our claim. The theorem of proof verification in [BFLS91] we The values L(g ) for i = 1, . . . , l and L(g g ) = i i j turn into circuit verification. L(gi)L(gj) determine the value of L(P ) for any P ∈ P by linearity. Let us denote L(g ) by γ . Then Theorem 8 Let C(y) be an arbitrary circuit. Then 2 i i 0 00 L(P ) is expressed as P (γ) (note that we here use that a polynomial size family of circuits Cr(y , y ) can be L(1) = 1). computed in polynomial-time from C and r with the properties: Now, since x is not satisfiable, L(Pi) = Pi(γ) 6= 0 for at least one of 1 ≤ i ≤ l + 1. But then 1. Each circuit has size (log n)O(1) and each of them depends only on a constant number of input bits P robV ∈D P2 (L(V ) = 0) = from y0.

P robV ∈D P2 (V (γ) = 0) = 00 l+1 2. If Cx(y) = 1, then there exists a y such that X 00 l P robr(Cr(E(y), y ) = 1) = 1. P robλ∈U GF (2) ( λiPi(γ) = 0) = 0.5, i=1 0 00 3. If P robr(Cr(y , y ) = 1) ≥ 1 − δ then which contradicts Inequality 9. 2 C(E−1(y0)) = 1. Later we need the following interpretation of the theorem: The theorem can be generalized to circuits with Theorem 7 Let C(y) be a circuit, where |y| = n. We constant number of inputs y1, y2, . . . , yc: 0 can construct a family Fc = {Cr(y )|r ∈ R} of circuits Theorem 9 Let C(y , y , . . . , y ) be an arbitrary cir- 0 1 2 c each of size at most O(1) where y is identified with cuit. Then a polynomial size family of circuits P2 such that 0 0 00 Cr(y1, . . . , yc, y ) can be computed in polynomial-time n 1. |R| = 2O(2 ); from C and r with the properties: 2. If C(y) = 1 then there is a setting of y0 such that 1. Each circuit has size (log n)O(1) and each of them

0 depends only on a constant number of input bits (a) P robr(Cr(y ) = 1) = 1 0 from yi for all i = 1, 2, . . . , c. (b) P robQ∈P2 (yQ + yQ+gi = gi(yi)) = 1, where 00 g(y) is the value of gate g for input y. 2. If Cx(y) = 1, then there exists a y such that P rob (C (E(y ),...,E(y ), y00) = 1) = 1. 0 r r 1 c 3. If P robr(Cr(y ) = 1) ≥ 1 − δ then there is a y 0 00 such that C(y) = 1 and for every gate gi of C 3. If P robr(Cr(y , y ) = 1) ≥ 1 − δ then −1 0 −1 P robQ∈P2 (yQ + yQ+gi = gi(yi)) ≥ 1 − 4δ. C(E (y1),...,E (yc)) = 1. Theorem 10 In Theorem 9 we can also assume that Assume now that opt(x) ≤ 1 − δ. Let us denote y00 in segmented in such a way that the corresponding the set of all possible random strings r by R. Let 0 −1 0 family of circuits meets the the segmentation require- y be arbitrary. Let yi = E (yi) (1 ≤ i ≤ l). ment in Definition 3.1. Let us denote by B the set of those r’s for which The proof of this theorem is the topic of Section 7. Cr,x(y1, y2, . . . , yc) = 0. By our assumption |B|/|R| ≤ 1 − δ0. We have 5.3 The reduction step 0 P robr,r0 (Cx,r,r0 (y ) = 1) =

We show that we can take a well behaved optimiza- 0 X P robr0 (Cx,r,r0 (y ) = 1) tion problem and use the circuit verification results ≤ |R| from the previous section and obtain another well be- r∈R haved optimization problem that recognizes the same |B|(1 − δ) + (|R| − |B|) language but where the circuits are much smaller. The ≤ 1 − δδ0. 2 basic idea is to use the circuit verification to verify |R| that the circuits describing the optimization problem is satisfied. Theorem 11 OPT (f(n), g(n)) ⊂ OPT (f(n) + 6 The final verification O(log g(n)), (log g(n))O(1)). Theorem 12 Let E be an arbitrary encoding scheme Proof: Let opt(x) = max P rob (C (y) = 1) be a y r x,r with parameter p(n). Let C(y , y , . . . , y ) be an ar- well behaved and restricted optimization problem that 1 2 c bitrary circuit of size n. Then there is a family recognizes a language in OPT (f(n), g(n)). We con- F = {C |r ∈ R} of circuits on inputs y0 , . . . , y0 , y00, struct another opt0 as follows. Fix x and r. Without r 1 c δ > 0 such that: loss of generality for notational simplicity we assume 0 00 O(1) that the circuit Cx,r(y , y ) relies only on segments 1. |R| = 2n . y1, . . . , yc. Consider the family of circuits Fx,r = 0 0 00 0 {Cx,r,r0 (y1, . . . , yc, yr )|r )} given by Theorem 10. 2. Each circuit has size O(1) and can be constructed S in polynomial time given x and r. The set of circuits r Fx,r is such that 1. each circuit can be computed from x, r and r0 in 3. If C(y) = 1, then there exists a y00 such that 00 polynomial time; P robr(Cr(E(y1),...,E(yc), y ) = 1) = 1.

O(1) 0 0 00 2. each circuit has size log g(n); 4. If P robr(Cr(y1, . . . , yc, y ) = 1) ≥ 1 − δ then C(E−1(y0 ),...,E−1(y )) = 1, where δ > 0 is 3. the new input set y0 = S y0 ∪ S y can be seg- 1 c i i r r some constant depending only on c. mented in the following way: each yr is segmented 0 as required in Theorem 10 and the segment size Proof Let C (y1, y2, . . . , yc) be such a circuit that we is logO(1) g(n). Let us decompose the first union obtain from C by adding extra gates to it to compute 0 (each yi) into segments of size 1. As it is required all the bits of each E(yi) for i = 1 . . . c. We denote by th in Theorem 10, each Cx,r,r0 takes only a constant gi,j the gate that computes the j bit of E(i)(E(i, j)). S 0 0 number of bits from i yi and a constant number Build a circuit verification scheme for C as in Theo- S 00 of segments from r yr. rem 7. We identify y with the bits of this verification scheme. Let each y0 have length p(|y |). Now flip a We have to argue now that the optimization prob- i i coin. lem

0 0 1. If tail, then by further coin flips choose a circuit opt (x) = max P robr,r0 (Cx,r,r0 (y ) = 1) (10) 0 y from the verification scheme for C . is well behaved and the underlying language is the 2. If head, then pick randomly a number between 1 0 same as for opt. and c, and compare a randomly chosen bit yi[j] 0 0 00 Assume opt(x) = 1. then if we set each yi to E(yi) of yi with the value that the scheme y gives for for (1 ≤ i ≤ l), where yi (1 ≤ i ≤ l) is a solution of E(i, j). To compute this bit we have to look at yQ opt(x), then by Theorem 7 for every r there exists a and ygi,j +Q, where Q is a random element of P2. 00 0 0 yr such that under this setting α of y all the circuits The check yQ + ygi,j +Q = yi[j] can be computed Cx,r,r0 output 1. Thus P robr(Cx,r,r0 (α) = 1) = 1. by a constant size circuit Ci,j,Q. d Family F is a union of {Ci,j,Q|1 ≤ i ≤ c; 1 ≤ j ≤ E(y) is a function E(y): GF (p) → GF (p), 0 p(|yi|); Q ∈ P2} and the circuits of the circuit verifi- where d = dlog n/ log log ne and p is a prime in cation scheme for C0. The multiplicity of each circuit [(k log n)c, 2(k log n)c] for some constant c. Let I = in the family is proportional to the probability with {1, 2,... dlog ne} ⊂ GF (p). Since |Id| ≥ n we can which it occurs in the above checking procedure. We use Id as indexes of the bits of y. Thus we can look leave the easy proof to the reader that the scheme at y as a function from Id to {0, 1}. Given a func- satisfies the conditions of the theorem. 2 tion y : Id → {0, 1} there exists a unique multivariate Using the recursion idea (Theorem 11) and the polynomial E(y) over GF (p) that for all α ∈ Id agrees above circuit verification procedure we obtain: with y and the degree of any variable is bounded by Theorem 13 OPT (f(n), g(n)) ⊂ OPT (f(n) + |I|−1. Note that the “total” degree of E(y) is at most (g(n))O(1), 1). d|I|. The new stringy ˜ contains three tables:

d 7 Segmentation • A table Y indexed by GF (p) , where segment α contains E(y)(α).

The major obstacle is that the proof y can not be l 2 • A table Ylines that is indexed by (GF (p) ) , segmented in such a way that the verifier only accesses where segment (α, β) contains the polynomial a constant number of segments. We overcome this by E(y)(tα + (1 − t)β). showing that we can transform an unsegmented proof into a segmented proof where the verifier only accesses • A table T that is indexed by {0, 1}t × GF (p)d. a constant number of segments. Let Qr = {q1, q2, . . . , qk}. The segment (r, α) Lemma 14 For any polynomial-time computable col- contains a polynomial pr,α of degree at most lection (the old query set) k(|I| − 1)d such that pr,α(0) = E(y)(α), pr,α(1) = E(y)(q1), . . . , pr,α(k) = E(y)(qk). t {Qr ⊂ {1, 2, . . . , n}|r ∈ {0, 1} } The pr,α exists, since it is possible to construct, then there exist a polynomial-time computable family by interpolation, a parameterized curve Cr,α : l Fp → F such that Cr,α(0) = r, Cr,α(1) = of circuits Dr,r0 and a δ > 0 such that for every string p y such that |y| = n there exists a string y˜ such that: q1,...,Cr,α(k) = qk and such that each coordi- nate function of Cr,α is a polynomial of degree k. 1. We can partition y˜ into segments y˜1, y˜2,..., y˜m˜ Now pr,α(t) = E(Y )(Cr,α(t)). all of length l = O(k log2 n), where k = 0 0 d 0 maxr |Qr|. Let r = (α, β, t, t ) where α, β, ∈ GF (p) and t, t ∈ {k+1, . . . , p−1}. The circuit Dr,r0 checks that Y [tα+ 2. Each circuit has size lO(1) and has inputs from 0 0 (1 − t)β] = Ylines[α, β](t), T [(r, α)](t ) = Y [Cr,α(t )]. only a constant number of segments. The cir- If any of the checks fail then Dr,r0 rejects otherwise it’s cuit Dr,r0 outputs either reject or a string of ith output for i = 1, . . . , k will be the least significant 0 length |Qr|. Furthermore r has length O(log n + bit of T [(r, α)](i). log k log n log log n ). Note that given these definition it follows that

0 0 0 3. For all r, r we have that Dr,r (˜y) = y[Qr], where ∀r, r : Dr,r0 (˜y) = y[Qr]. y[Qr] is the substring of y indexed by Qr. Thus we only need to prove the 4th claim, hence m˜ 4. For all z ∈ {0, 1} if there exists an r0 such that assume z = (Y,Ylines,T ) is a string and for some r0 we have that 0 0 P robr (Dr0,r (z) = reject) ≤ δ

P rob 0 (D 0 (z) 6= reject) ≥ 1 − δ. then there exists a y such that for all r: r r0,r This implies that P robr0 (Dr,r0 (z) 6∈ {reject, y[Qr]}) ≤ 3δ

P rob(α,β,t)(Y [tα + (1 − t)β] 6= Ylines[α, β](t)) < δ. Proof: The proof relies on the low degree code E [BFL91, BFLS91] and a technical lemma about check- As in Section 4 we need a code checking lemma. ing E [RS92]. This one is due to Rubinfeld and Sudan [RS92]. Lemma 15 (Rubinfeld-Sudan [RS92]) Let g : show that unless P = NP there is no constant ratio d GF (p) 7→ GF (p) be a function and let glines be approximation algorithm for longest paths. It is con- a function that given α, β ∈ GF (p)d determines a jectured that this problem is as hard to approximate polynomial pα,β of degree at most k such that p = as clique or chromatic number. O((kd)c), where c is some universal constant, and

P robα,β,t(g(tα + (1 − t)β) 6= glines(α, β)(t)) ≤ δ Acknowledgments then there exists a multivariate polynomial f of degree We acknowledge the contribution of Muli Safra. at most k such that P robα(g(α) 6= f(α)) ≤ 2δ. Furthermore we would like to thank the contribu- Thus we can conclude that there exists a polyno- tions of Yossi Azar, Manuel Blum, Tomas Feder, mial g such that Joan Feigenbaum, Lance Fortnow, Magnus Halldors- son, David Karger, Moni Naor, Steven Phillips, Umesh P robα(Y [α] 6= g(α)) < 2δ. (11) Vazirani and Mihalis Yannakakis.

d Let yi be the least significant bit of g(i), where i ∈ I (remember that we index the bits in a n bit string by Id). References Let r be an arbitrary random string. If T [(r, α)] 6= [AMSS92] S. Arora, R. Motwani, S. Safra and M. Sudan. g(Cr,α) then for at most k(|I| − 1)d values of t we PCP and Approximation Problems. Unpublished note, have that T [(r, α)](t) = g(Cr,α(t)). Note that Cr,α(t) 1992. d is uniformly distributed over GF (p) when α, t ∈U [AS92] S. Arora and S. Safra. Probabilistic Checking GF (p) × {k + 1, . . . , p − 1}. Thus of Proofs: A New Characterization of NP. In Proc. 33rd IEEE Symp. on Foundations of Computer Science, P robr0 (Dr,r0 (z) 6∈ {reject, y[Qr]}) 1992. = P robr0 (T [(r, α)] 6= g(Cr,α) and [Azar92] Y. Azar. Personal Communication. 1992. [Bab85] L. Babai. Trading group theory for randomness. T [(r, α)](t) = Y [Cr,α(t)]) In Proc. 17th ACM Symp. on Theory of Computing, ≤ P robr0 (T [(r, α)] 6= g(Cr,α) and pages 421–420, 1985. T [(r, α)](t) = g[Cr,α(t)]) + 2δ [BFL91] L. Babai, L. Fortnow, and C. Lund. Non- k(|I| − 1)d deterministic Exponential Time has Two-Prover Inter- ≤ + 2δ < 3δ 2 active Protocols. Computational Complexity, vol. 1, p − k − 1 pages 3–40, 1991. [BFLS91] L. Babai, L. Fortnow, L. Levin, and M. Szegedy. Checking Computations in Polylogarithmic Time. In 8 Further Work Proc. 23rd ACM Symp. of Theory of Computing, pages 21–31, 1991. Several questions remain open. There is a big gap [BGKW88] M. Ben-Or, S. Goldwasser, J. Kilian and between the (negligible) constants in the hardness re- A. Wigderson. Multi-prover interactive proofs: How sults and the approximation ratio currently achievable to remove intractability. In Proc. 20th ACM Symp. on for the MAXSNP problems. For example, MAX SAT Theory of Computing, pages 113–131, 1988. can only be approximated to a ratio of 4/3 [Yan92], [BJLTY91] A. Blum, T. Jiang, M. Li, J. Tromp and while MAX CUT and vertex cover can only be ap- M. Yannakakis. Linear Approximation of Shortest Su- proximated to a ratio of 2 [GJ79, Mot92]. perstrings. In Proc. 23rd ACM Symp. on Theory of The recent results of Lund and Yannakakis [LY92] Computing pages 328–336, 1991. showed that the chromatic number is as hard to ap- [BLR90] M. Blum, M. Luby, R. Rubinfeld. Self- Testing/Correcting with Applications to Numerical proximate as the clique and thus solce a long-standing Problems. In Proc. 22th ACM Symp. on Theory of Com- open problem. They also show that the logarithmic puting, pages 73–83, 1990. ratio achievable for the set cover problem is essentially [BP89] M. Bern and P. Plassmann. The Steiner problem the best possible. Can the former result be extended with edge lengths 1 and 2. Information Processing Let- to the case of 3-colorable graphs? ters, vol. 32, pages 171–176, 1989. Another open problem is that of approximating the [BS92] P. Berman and G. Schnitger. On the Complexity longest path in a graph. Based on our results, Karger, of Approximating the Independent Set Problem. Infor- Motwani and Ramkumar and Azar [KMR92, Azar92] mation and Computation, vol. 96, pages 77–94, 1992. [Coo71] S. A. Cook. The complexity of theorem-proving [Lev73] L. Levin. Universal’ny˘ıe pereborny˘ıe zadachi procedures. In Proc. 3rd ACM Symp. on Theory of (Universal search problems, in Russian), In Problemy Computing, pages 151–158, 1971. Peredachi Informatsii, vol. 9, pages 265–266, 1973. A [DJPSY92] E. Dahlhaus, D. S. Johnson, C. H. Papa- corrected English translation appears in an appendix to ditriou, P. D. Seymour and M. Yannakakis. The com- Trakhtenbrot [Tra84] plexity of multiway cuts. In Proc. 24th ACM Symp. on [LFKN90] C. Lund, L. Fortnow, H. Karloff, and N. Nisan. Theory of Computing, pages 241–251, 1992. Algebraic methods for interactive proof systems. In [Fag74] R. Fagin. Generalized First-Order Spectra and Proc. 22nd ACM Symp. on Theory of Computing, pages Polynomial-time Recognizable Sets. In Richard Karp 2–10, 1990. (ed.), Complexity of Computer Computations, AMS, [LS91] D. Lapidot and A. Shamir. Fully Parallelized Multi 1974. Prover Protocols for NEXP-time. In Proc. 32nd IEEE [Fre79] R. Freivalds. Fast Probabilistic Algorithms. Symp. on Foundations of Computer Science, pages 13– In Mathematical Foundations of Computer Science. 18, 1991. Springer-Verlag Lecture Notes in Computer Science, No. [LY92] C. Lund and M. Yannakakis. On the Hardness 74, pages 57–69, 1979. of Approximating Minimization Problems. Manuscript, 1992. [FGLSS91] U. Feige, S. Goldwasser, L. Lov´asz,S. Safra, and M. Szegedy. Approximating clique is almost NP- [MS81] F. MacWilliams and N. Sloane. The Theory of complete. In Proc. 32nd IEEE Symp. on Foundations Error-Correcting Codes. North-Holland, 1981. of Computer Science, pages 2–12, 1991. [Mot92] R. Motwani. Lecture Notes on Approximation Al- [FL92] U. Feige and L. Lovasz. Two-Prover One-Round gorithms. Technical Report, Dept. of Computer Science, Proof Systems: Their Power and Their Problems. In Stanford University (1992). Proc. of 24th ACM Symp. on Theory of Computing, [PY91] C. H. Papadimitriou and M. Yannakakis. Op- pages 733–744, 1992. timization, Approximation, and Complexity Classes. Journal of Computer and System Sciences, vol. 43, [FRS88] L. Fortnow, J. Rompel and M. Sipser. On the pages 425–440, 1991. power of multi-prover interactive protocols. In Proc. 3rd IEEE Symp. on Structure in Complexity Theory, pages [PY92] C. H. Papadimitriou and M. Yannakakis. The 156–161, 1988. traveling salesman problem with distances one and two, Mathematics of Operations Research, to appear. [GJ76] Michael R. Garey and David S. Johnson. The com- [Rub90] R. Rubinfeld. A Mathematical Theory of Self- plexity of near-optimal graph coloring. J. of the ACM, Checking, Self-Testing and Self-Correcting Programs. vol 23, pages 43-49, 1976. Ph.D. Thesis, Computer Science Department, U.C. [GJ78] Michael R. Garey and David S. Johnson. “Strong” Berkeley, 1990. NP-completeness results: motivation, examples and Im- [RS92] R. Rubinfeld, and M. Sudan. Testing Polynomial plications. J. of the ACM, vol 25, pages 499-508, 1978. Functions Efficiently and over Rational Domains. In [GJ79] Michael R. Garey and David S. Johnson. Comput- Proc. 3rd Annual ACM-SIAM Symp. on Discrete Algo- ers and Intractability: A Guide to the Theory of NP- rithms, pages 23–32, 1992. Completeness. W. H. Freeman, 1979. [Sha90] A. Shamir. IP=PSPACE. In Proc. 22nd ACM [GLRSW91] P. Gemmell, R. Lipton, R. Rubinfeld, M. Su- Symp. on Theory of Computing, pages 11–15, 1990. dan, and A. Wigderson. Self-Testing/Correcting for [SG76] S. Sahni and T. Gonzalez. P-complete approxima- Polynomials and for Approximate Functions. In Proc. tion problems. JACM, vol. 23, pages 555–565, 1976. 23th ACM Symp. on Theory of Computing, pages 32–42, [Tra84] B. A. Trakhtenbrot. A survey of Russian ap- 1991. proaches to Perebor (brute-force search) algorithms. In [GMR89] S. Goldwasser, S. Micali and C. Rackoff. The Annals of the History of Computing vol. 6, pages 384– Knowledge Complexity of Interactive Proof Systems. 400, 1984. SIAM Journal on Computing, vol. 18, pages 186–208, [Yan92] M. Yannakakis. On the Approximation of Max- 1989. imum Satisfiability. In Proc. 3rd Annual ACM-SIAM [Kan91] V. Kann. Maximum bounded 3-dimensional Symp. on Discrete Algorithms, pages 1–9, 1992. matching is MAXSNP-complete. Information Process- ing Letters, vol. 37, pages 27-35, 1991. [Kar72] R.M. Karp. Reducibility among combinatorial problems. In Raymond E. Miller and James W. Thatcher, editors, Complexity of Computer Computa- tions, pages 85–103. Plenum Press, 1972. [KMR92] D. Karger, R. Motwani and G. D. S. Ramku- mar. On Approximating the Longest Path in a Graph. Manuscript, 1992.