Probabilistic Checking of Pro ofs

and

Hardness of Approximation Problems

Sanjeev Arora

CSTR

Revised version of a dissertation submitted at

CS Division UC Berkeley in August

Princeton techrep orts are available in p ostscript form via anonymous ftp from ftpcsprincetonedu

InterNet address You may log in as anonymous and giveyour email ad

dress as password Change to the directory reports The le README gives details and the

le INDEX lists the numb ers and the titles of all rep orts This rep ort is numb ered

Hardcopies of this rep ort are available for a charge of only US dollar checks

accepted from the following address

Technical Rep orts

Department of Computer Science

Princeton University

Olden Street Princeton NJ

Probabilistic Checking of Pro ofs

and

Hardness of Approximation Problems

c

Copyright

by

Sanjeev Arora

The dissertation committee consisted of

Professor Umesh V Vazirani Chair

Professor Richard M Karp

Professor John W Addison

The authors current address

Department of Computer Science

Olden St

Princeton University

Princeton NJ email aroracsprincetonedu

Tomyparents

and

members of my family

including the ones who just joined

Contents

Intro duction

This dissertation

Hardness of Approximation

A New Characterization of NP

Knowledge assumed of the Reader

Old vs New Views of NP

The Old View Co okLevin Theorem

Optimization and Approximation

A New View of NP

The PCP Theorem Connection to Approximation

History and Background

PCPAnOverview

Co des

Pro of of the PCP Theorem an Outline

History and Background

A Pro of of the PCP Theorem

Polynomial Co des and Their Use

Algebraic Pro cedures for Polynomial Co des

An Application Aggregating Queries

AVerier Using O log n Random Bits

A Less EcientVerier

Checking Split Assignments

AVerier using O query bits

Checking Split Assignments

The Algebraic Pro cedures

SumCheck

Pro cedures for the Linear Function Co de

Pro cedures for General Polynomial Co de

The Overall Picture

HistoryAttributions iii

The Lowdegree Test

The Bivariate Case

Correctness of the Lowdegree Test

Discussion

History

Hardness of Approximations

The Canonical Problems

GapPreserving Reductions

MAXSNP

Problems on Lattices Co des Linear Systems

The Problems

The Results

Signicance of the Results

A Set of Vectors

Reductions to NV and others

Hardness of Approximating SV

Proving n approximations NPhard

MAXSATISFY

Other Inapproximability Results A Survey

Historical NotesFurther Reading

eriers that make queries PCP V

Hardness of Approximating Lab el Cover

Hardness of SV

Unifying Lab elCover and MAXSAT

Applications of PCP Techniques

Strong Forms of the PCP Theorem

The Applications

Exact Characterization of Nondeterministic Time Classes

Transparent Math Pro ofs

Checking Computations

Micalis Certicates for VLSI Chips

Characterization of PSPACE Condon et al

Probabilistically Checkable Co des

Kilians ZK arguments

Khanna et als Structure Theorem for MAXSNP

The Hardness of nding Small Cliques

Op en Problems

Hardness of Approximations

Proving hardness where no results exist

Improving existing hardness results

Obtaining Logical Insightin to Approximation Problems

Op en Problems connected with PCP techniques

Do es the PCP theorem have a simpler pro of

A Library of Useful Facts

List of Figures

Tableau and window The windowshows the nite control initially in state

q and reading a The control overwrites the with a moves one cell to

right and changes state to q

Verier in the denition of PCP

a Pro of that V exp ects b Pro of that V exp ects Shaded area in

new

represents the assignment split into Q parts that corresp onds to V s

random seed r

m

A function from h to F can b e extended to a p olynomial of degree mh

A table of partial sums may b e conceptualized as a tree of branching factor

q The Sumcheckfollows a random path down this tree

How to enco de an assignment so that the PCPlog n verier accepts with

probability

Lab el Cover instance for formula x x x x x x The

symb ols on edge e representmap

e

Figure showing the epro jections of vectors V and V in the vector

v a v a

set where e v v

vii

viii

Acknowledgements

Imust acknowledge my great debt to my advisor Umesh Vazirani for the nurturing and

advising the sake and the dinners he has provided me over the last four years My only

regret is that I never to ok squash lessons from him even though I kept planning to

The theory group at Berkeley wasawonderful enviroment in which to b e a graduate

student Thanks to Dick Karp Manuel Blum Mike LubyRaimund Seidel Gene Lawler

and Abhiram Ranade for creating the environment Dicks phenomenally clear mind proved

a go o d resource on many o ccasions Manuel and Mike got me thinking ab out manyofthe

problems up on which this dissertation is based Some of Mikes unpublished observations

were central to guiding my dissertation research in the right direction

Iwould like to thank my colleagues with whom I did this research Sp ecial thanks go

to Madhu Sudan and Muli Safra for their patience while collab orating with me in my early

gradscho ol days I learnt a lot from them I also thank Carsten Lund Jacques Stern Laci

Babai Mario Szegedy Ra jeev Motwani and Z Sweedyk for collab orating with me

Thanks to John Addison for serving on my dissertation committee and giving me lots

of comments in his usual precise style

Thanks to all my fellowstudents in the department and the p ostdo cs at ICSI for talks

TGIFs happy hours and bullsessions

Thanks also to all the p eople at MIT who got me interested in theoretical computer

science and softball during my undergraduate years I am esp ecially grateful to Bruce

Maggs Tom Leighton Mike Sipser Mauricio Karchmer Richard Stanley Charles Leiserson

and David Shmoys

There are many others who over the years have help ed me develop as a p erson and

a researcher I will not attempt to list their names for fear that anysuch list will b e

incomplete but I express my thanks to them all

My deep est gratitude is to my family for instilling a love of knowledge in me I hop e

my father is not to o dismayed that I am contributing two more professors to the family

FinallyIwould like to thank Silvia She was clearly my b est discovery at Berkeley

Chapter

Intro duction

The study of the diculty or hardness of computational problems has two parts The

theory of algorithms is concerned with the design of ecient algorithmsin other w ords

with proving upp er b ounds on the amount of computational resources required to solve

a sp ecic problem Complexity theory is concerned with proving the corresp onding lower

b ounds Our work is part of the second endeavormore sp ecically the endeavor to prove

problems computationally dicult or hard

Despite some notable successes lower b ound research is still in a stage of infancy

progress on its op en problems has b een slow Central among these op en problems is the

question whether P NP In other words is there a problem that can b e solved in

p olynomial time on a nondeterministic Turing Machine but cannot b e solved in p olynomial

time deterministically The conjecture P NP is widely b elieved but currently our chances

of proving it app ear slim

If we assume that P NP however then another interesting question arises given

any sp ecic optimization problem of interest is it in P or not Complexity theory has

had remarkable success in answering such questions The theory of NPcompleteness due

to Co ok Levin and Karp allows us to prove that explicit problems are not in P assuming

P NP The main idea is to prove the given problem NPhard that is to give a p olynomial

time reduction from instances of any NP problem to instances of the given problem If an

NPhard problem were to have a p olynomialtime algorithm so w ould every NP problem

whichwould contradict the assumption P NP Hence if P NP then an NPhard

problem has no p olynomialtime algorithm To put it dierently an NPhard problem is

no easier than any other problem in NP

The success of the theory of NP completeness lay in the unityitbrought to the study of

computational complexity a wide array of optimization problems arising in practice and

which had hitherto deed all eorts of algorithm designers to nd ecient algorithms were

proved NPhard in one swo op using essentially the same kind of reductions For a list of

NPhard problems c see the survey by Garey and Johnson GJ

CHAPTER INTRODUCTION

But one ma jor group of problems seemed not to t in the framework of NPcompleteness

approximation problems Approximating an NPhard optimization problem within a factor

c means to compute solutions whose cost is within a multiplicative factor c of the cost

of the optimal solution Such solutions would suce in practice if c were close enough

to For some NPhard problems weknowhow to compute such solutions in p olynomial

timefor most it seemed that ev en approximation was hard at least a substantial b o dy

of research failed to yield any ecient approximation algorithms However it was not clear

howtoprove the hardness of approximation using the Co okKarpLevin technique We

elab orate on this p oint in Chapter A recentresultbyFeige et al FGL provided a

breakthrough by using algebraic techniques to show that approximating the clique problem

is hard These algebraic techniques are derived from recentworkoninteractive pro ofs and

program checking see Section

This dissertation

We use techniques similar to those in the ab ovementioned pap er FGL to prove

the hardness of many other approximation problems We also prove a new probabilistic

characterization of the class NP The results in this dissertation are from three pap ers

ALM ABSS and some previously unpublished observations AS

Hardness of Approximation

We exhibit many NPhard optimization problems for which approximation for a range of

values of the factor c is NPhard In other words approximating the problem is no easier

than solving it exactly at least as far as p olynomialtime solvability is concerned Some

of the imp ortant problems to which our result applies are the following

Clique and Indep endent Set We show that approximating these problems within any

constant factor is NPhard AS Further in ALM we show that some p ositive

constant exists such that approximating these problems within a factor of n n

numberofvertices in the graph is NPhard A weaker hardness result was known

earlier namely that if these problems can b e approximated within any constant factor

in p olynomial time then all NP problems can b e solved deterministically in time

O log log n

n FGL

MAXSNPhard problems The class MAXSNP of optimization problems was identi

ed byPapadimitriou and Yannakakis PY and the class of problems hard for

this class include vertexcover metric TSP shortest sup erstring and others We show

c such that that for every MAXSNPhard problem there is some xed constant

approximating the problem within a factor c is NPhard ALM

Optimization problems on lattices co des and linear systems Weshow the hard

ness of approximating many problems includin g the wellknown Nearest Lattice Vector

THIS DISSERTATION

and the Nearest Co deword problems ABSS A hardness result is also obtained for

aversion of the Shortest Lattice Vector problem namely the version using the

norm We note that tightening the result would prove the hardness of exact

optimization in the norm a longstanding op en problem

A selfcontained description of the ab ove hardness results app ears in Chapter That

chapter also attempts to unify all known hardness results More sp ecicallyitintro duces

two canonical problems and indicates how reductions from them can b e used to prove all

known hardness results For some problems however this approach yields inapproximability

results not as strong as those provable otherwise Chapter discusses a p ossible approach

to remedy this diculty

A New Characterization of NP

Our results on hardness of approximation rely on a new typ e of reduction whose main feature

is that it acts globally unlike classical reductions which p erform lo cal transformations

Another way to view this new reduction is as a new denition of NP According to this

denition the class NP contains exactly those languages for whichmemb ership pro ofs can

be checked by a probabilistic verier that uses O log n random bits and examines O bits

in the pro of AS ALM Please see Chapter for a more careful statement The

equivalence b etween the new and the old denitions of NP is the sub ject of the socalled

the PCP Theorem whose pro of uses algebraic techniques partially derived from previous

work An outline of the pro of app ears in Chapter and details app ear in Chapters and

At the end of eachchapter a brief section gives p ointers to other literature and a

historical account of the the development of the ideas of that chapter

To the b est of our knowledge this dissertation represents the rst selfcontained exp o

sition of the entire pro of of the PCP Theorem incorp orating all necessary lemmas from the

pap ers AS ALM and other previous work For other almost complete exp osi

tions we refer the reader to Sud ALM However the exp osition in Sud takes

a dierent viewp oint Its main results concern program checking and the PCP theorem is

derived as a corollary to those results

We feel that in the long run the algebraic techniques used in proving the PCP theorem

will nd many other applications To some extent this has already happ ened and in

Chapter we include a brief survey of some of the recentlydiscovered applications Many

applications are due to other researchers

Finally Chapter contains a list of op en questions ab out b oth hardness of approximat

ion and the algebraic techniques used in earlier chapters One imp ortant op en question is

whether pro of of the PCP theorem can b e simplied Chapter discusses this question as

well

CHAPTER INTRODUCTION

Knowledge assumed of the Reader

This dissertation has b een written as a survey for the nonsp ecialist We assume only famil

iarity with Turing Machines and standard conventions ab out them asymptotic notation

and p olynomial time and NPcompleteness For an intro duction to all these see GJ A

list of assumed algebraic facts with brief pro ofs app ears in App endix A However most

readers should nd that they can understand most of the dissertation on the basis of just

the following mantra

A nonzero univariate polynomial of degree d has at most d roots in a eld

Chapter

Old vs New Views of NP

Let b e a nite set called the alphabet A language L is a set of nitesized strings over

ie L Byastraightforward equivalent of the classical denition a language L

is in NP i there is a p olynomial time deterministic Turing Machine M and a p ositive

number c such that for any x in

c

x L i y jy j jxj st M accepts x y

String y is variously called witness nondeterministic guess or memb ership pro of

We prefer the term membership proof Machine M is called the verier

As is standard we assume f g

Example ACNFformula in b o olean variables s s is of the form

n

m

w w w

i i i

i

where each w is a literal ie either s or s for some k The subformula w w w

i k k i i i

j

is called a clause The formula is satisable if there is an assignmenttothes s which makes

i

all clauses true

Let SAT b e the set of satisable CNF formulae By enco ding formulae with s and

s in some canonical fashion we can consider SATasalanguage It is in NP since a

satisfying assignment constitutes a memb ership pro of that can b e checked in p olynomial

time

The Old View Co okLevin Theorem

Co ok Co o and indep endently Levin Lev showed that SAT is NPcomplete

More sp ecically given any NPlanguage L and input x they gave a p olynomial time

CHAPTER OLD VS NEW VIEWS OF NP

xy

01q,0 window

01q’,1

Figure Tableau and window The window shows the nite control initially in state q

and reading a The control overwrites the with a moves one cell to right and changes

state to q

construction of a CNF formula that is satisable i x L The Co okLevin result

xL

underlies classical work on NPcompleteness since most other problems are proven NP

complete by doing reductions from SAT We briey recall the textb o ok version of their

construction for further details see GJ

How can a generic reduction b e given from all NP languages to SAT After all the

notion of memb ership pro of diers widely for dierent NP languages Co ok and Levin

noticed that a single notion or format suces for all languages The pro of can b e the

tableau of an accepting computation of the p olynomialtime verier Further in this format

the pro of is correct i it satises some lo cal constraints

A tableau is a dimensional transcript of a computation If the computation ran for

T steps the tableau is a dimensional table that has T lines of T entries each where j th

entry in line i contains the following information i the contents of the j th cell of the

tap e at time iand ii whether or not the nite con trol was in that cell or not and if so

what state it was in see Figure in GJ

Let L b e an NPlanguage and M be the verier for L A tableau is valid for M if each

step of the computation followed correctly from the previous step and the verierisinan

accept state in the last line

A lo ok at the denition in shows that an input x is in L i

valid tableau for M with rst line x y for a string y of a suitable size

Given a tableau here is howtocheck that it satises the conditions in a Check

bitbybit that the rst line contains x b Check that M is in an accept state in the last

line And nally c check that the computation was done correctly at each step

But to checkcwe only need to check that the nite control of M op erated correctly at

each step and that every tap e cell not in the immediate neighb orho o d of the nite control

OPTIMIZATION AND APPROXIMATION

do es not change its contents in that step

Thus whether or not the tableau passes the checks a b and c dep ends up on

whether or not its entries satisfy some lo cal constraints In fact it is easy to see that

the computation is done correctly overall i all neighb orho o ds in the tableau lo ok

correct This neighb orho o d is sometimes called a window See Figure

Here is how Co ok and Levin constructed a CNF formula that is satisable i there is

a tableau that passes the checks a b and c For i j T represent the contents of

the the j th entry of the ith line by O b o olean variables For each window express its

correctness by a b o olean function of the variables corresp onding to the cells in that window

Rewrite this function using clauses of size

The overall formula is the of the formulae expressing checks a b and c For

instance the formula expressing c is

Formula expressing correctness of window around j th cell of ith line

ij T

Example We give some details of the CookLevin construction see GJ for further

details Assume the machines alphab et is f g The corresp onding formula has for each

i j T the variables z y and for each state q in the nite control a variable s The

ij ij ij q

interpretation to z y b for b f g is At time i the the j th cell contains bit b

ij ij

And z y means the cell has a blank symb ol The interpretation to s b eing set to

ij ij ij q

true is At time i the j th cell contains the Turing Machine head and the nite control is

in state q Then heres how to express that if the nite control is in cell j at time i then

it cannot b e in multiple states

s s

q q ij q ij q

ximation Optimization and Appro

With most wellknown NP languages we can asso ciate a natural optimization problem The

problem asso ciated with SAT is MAXSAT For a CNF formula and assignment A

let valA denote the numb er of clauses in satised by A

Denition MAXSAT This is the following problem

Input CNF formula

Output OPT which is max fvalA A is an assignmentg

Clearly MAXSAT is NPhard As mentioned in the intro duction a natural waytodeal

with NPhardness is to compute approximate solutions

CHAPTER OLD VS NEW VIEWS OF NP

Denition For a rational number c an algorithm is said to compute capproxi

mations to MAXSATifgiven any input its output is an assignment B such that

OPT

valB OPT

c

This dissertation addresses the following question For what values of c can capproxi

mations to MAXSAT b e computed in p olynomial time For c this was known

to b e p ossible Yan see also GW The algorithm for c is actually quite

straightforward Whether or not the same was true for every xed constant cwas not

known

The Co okLevin reduction do es not rule out the existence of p olynomialtime algorithms

that compute capproximations for every xed c Recall that the CNF formulae it

pro duces always represent tableaus For suchformulae weshowhow to satisfy a fraction

T of the clauses in p olynomial time where T is the numb er of lines in the tableau

Construct in p olynomial time an invalid tableau that starts o by representing a valid

computation on x y for some string y but then switches the computation to some

trivial accepting computation In such a tableau all the windows lo ok correct except

those in the line where the switchwas made Nowinterpret the tableau as an assignmentto

the corresp onding CNF formula The assignment satises all the clauses except the ones

ords all but T fraction of the clauses Since T corresp onding to a single linein other w

is smaller than any constant we conclude that in the Co okLevin instances of MAXSAT

optimization is hard but computing capproximations for any xed c is easy

Furthermore the known reductions from SAT to other optimization problems trans

form SAT instances in a lo cal fashion namelyby using gadgets to represent clauses and

variables When p erformed on the Co okLevin instances of SAT such lo cal transforma

tions yield instances of the other problem in which optimization is hard but capproximation

is easy for every c

Thus it b ecomes clear that a new typ e of NPcompleteness reduction is called for to

prove the hardness of approximations

A New View of NP

In this section we state a new probabilistic denition of NP It is based up on a new

complexity class PCP whose name abbreviates Probabilistically Checkable Pro ofs

Let a verier b e a p olynomialtime probabilistic Turing Machine containing an input

tap e a work tap e a source of random bits and a readonly tap e called the proof string

and denoted as see Figure The machine has random access to the worktap e

contains a sp ecial addressing portion on which M can write the address of a lo cation in

and then read just the bit in that lo cation The op eration of reading a bit in is called a

query

A NEW VIEW OF NP

INPUT x PROOF

V (FINITE CONTROL) RANDOM STRING τ

WORK−TAPE

Figure Verier in the denition of PCP

The source of random bits generates at most one bit p er step of the machines computa

tion Since the machine uses only a nite sequence of those bits we can view the sequence

as an additional input to the machine called the random string

Denition Averier is r nqnrestricted if on each input of size n it uses at most

O r n random bits for its computation and queries at most O q n bits of the pro of

In other words an r nqnrestricted verier has two asso ciated integers c k The

random string has length cr n The verier op erates as follows on an input of size nIt

reads the random string computes a sequence of kqn lo cations i i and

kqn

queries those lo cations in Dep ending up on what these bits were it accepts or rejects

Dene M x tobe if M accepts input x with access to the pro of using a string

of random bits and otherwise

Denition Averier M can probabilistical ly check membership proofs for language L

if

For every input x in L there is a pro of that causes M to accept for every random

x

string ie with probability

For any input x not in Levery pro of is rejected with probability at least

PrM x

Note that we are restricting the verier to query the pro of nonadaptively the lo cations it queries in

dep end only up on its random string In contrast the original denition of PCP AS allowed the

verier base its next query on the bits it had already read from We include nonadaptiveness as part of

the denition b ecause many applicatio ns require it

CHAPTER OLD VS NEW VIEWS OF NP

Note The choice of probability in the second part is arbitrary By rep eating the

veriers program O times and rejecting if the verier rejects even once the probability

of rejection in the second part can b e reduced to any arbitrary p ositive constant Thus

we couldve used any constant less than instead of

Denition A language L is in PCPr nqn if there is an r nqnrestricted

verier M that can check memb ership pro of for L

Note that NP PCP polyn since PCP polyn is the set of languages for which

memb ership pro ofs can b e checked in deterministic p olynomialtime which is exactly NP

according to Denition In Chapter we will prove the following result

Theorem PCP Theorem NP PCPlog n

Next we prove the easier half of the PCP Theorem PCPlog n NP Observe that

O log n O

when the input has size n a log n restricted verier has n choices for its

random string Further once we x the random string the veriers decision is based up on

O bits in the pro of More formallywe can state the following lemma

Lemma Let language L beinPCPlog n Thenthereareintegers c d k such

c d

that for every input x thereare n boolean functions f f in n boolean variables where

n is the size of x with the fol lowing properties

Each f is a function of only k variables and its truthtable can becomputedin

i

polynomial time given x i

If input x is in L there is an assignment to the boolean variables that makes every f

i

evaluate to true

If x L then no assignment to the boolean variables makes morethan the f s

i

true

c log n c

Pro of Let the verier for L use c log n random bits Note that it has at most n

dierent p ossible runs and in each run it reads only O bits in the pro ofstring Hence

wlog we can assume that the number of bits in any provided pro ofstring is at most

c d

O n For concreteness assume this number is n for some integer d

For any b o oleanvalued variables y y the set of p ossible assignments to y y

d d

n n

is in onetoone corresp ondence with the set of p ossible pro ofstrings We assume wlog

that the pro ofstring is an assignment to the variables y y y

d

n

John Addison has p ointed out an unintended pun in this result In descriptive set theoryPCC for any

class C would b e the pro jections of sets that are complements of sets of C For C the complexity class

P this would refer to NP since complement of P is P and pro jection of P is NP

THE PCP THEOREM CONNECTION TO APPROXIMATION

c log n

Fixing the veriers random string to r f g xes the sequence of lo cations

that it will examine in the pro of Let this sequence have size k O Let the sequence

of lo cations b e i r i r The veriers decision dep ends only up on the assignments

k

to y y Dene a b o olean function on k bits f asf b b true i

r r k

i r i r

k

the verier accepts when the assignment to the sequence of variables y y is

i r i r

k

b b Since the verier runs in p olynomial time we can compute the truth table of f

k r

k

in p olynomial time by going through all p ossible values of b b and computing the

k

veriers decision on each sequence

o n

c log n

c

dened in this fashion By def Consider the set of n functions f r f g

r

inition of PCPlog n when the input is in the language there is an assignment to the

y y that makes all functions in this set evaluate to true Otherwise no assignment

makes more than of them evaluate to true

Corollary PCPlog n NP

Pro of Let L PCPlog n and x b e an input of size n Construct in p oly n time the

d

functions of Lemma Clearly x L i there exists an assignment to the n variables

that makes all the f s evaluate to TRUE Such an assignment constitutes a memb ership

i

pro of that x L and can b e checked in p oly ntime

The PCP Theorem Connection to Approximation

This section shows that the characterization of NP as PCPlog n allows us to dene a

new format for memb ership pro ofs for NP languages The format is more robust in a sense

explained b elow than the tableau format of Co ok and Levin and immediately suggests

a new waytoreduceNPtoMAXSAT This new reduction shows that approximating

MAXSAT is NPhard

Let L be any NP language Since NP PCPlog n Lemma holds for it Let

x b e an input Use the set of functions given by Lemma to dene a new format for

memb ership pro ofs for L the pro of is a b o olean assignment ie a sequence of bits that

makes all of f f evaluate to true

Then follow the Co okLevin construction closely Think of each f as representing a

i

correctness condition for a set of k bitp ositions in the pro ofth us this set of k lo cations

plays a role analogous to that of a window in the tableau The statement of Lemma

implies that if x L then half the windows areincorrect Tocontrast the new format with

the tableau format recall that when x L there exist tableaus in which almost all the

windows lo ok correct In this sense the new format is more robust

Formally the reduction consists in writing a SAT formula that represents all the f s

i

The gap of a factor b etween the fraction of f s that can b e satised in the two cases

i

translates into a gap b et ween the fraction of clauses that can b e satised in the two case

CHAPTER OLD VS NEW VIEWS OF NP

The pro of of the following corollary formalizes the ab ove description

Corollary Thereisaconstant such that computing approximations to

MAXSAT is NPhard

Pro of Let L b e an NPcomplete language and x b e an input of size n Assuming NP

PCPlog n Lemma applies to LWe use notation from that lemma Let y y

d

n

c

b e the set of b o olean variables and ff i n g the collection of functions corresp onding

i

to x

Consider a function f from this collection Let it b e a function of variables y y

i i i

k

k

Then f can b e expressed as a conjunction of clauses in these variables each of size at

i

most k Let C C denote these clauses From nowonwe use the terms k clause

k

i

i

and clause to talk ab out clauses of size k and resp ectively

Then the k CNF formula

c k

n

C

ij

i j

is satisable i x L Also if x L then every assignment fails to satisfy half the f s

i

eachofwhich yields an unsatised k clause So if x L the fraction of unsatised clauses

is at least which is some xed constant

k

To obtain a CNF formula rewrite every k clause as a conjunction of clauses of size

as follows For a k clause l l l the l s are literals write the formula

k i

k

z l z l l z l l z

t t t k k k

t

where z z are new variables which are not to b e used again for anyotherk clause

k

Clearly a xed assignmenttol l satises the original k clause i there is a further

k

assignmenttoz z that satises the formula in

k

Thus the formula of has b een rewritten as a CNF formula that is satisable i

x LFurther if x Levery unsatised k clause in yields an unsatised clause in

the new formula so the fraction of unsatised clauses is at least

k

k

Hence the lemma has b een proved for the value of given by

k

k

As shown in Chapter Corollary implies similar hardness results for a host of other problems

HISTORY AND BACKGROUND

Where the gap came from The gap of a factor of in the ab ove reduction came from

two sources the gap versus in the fraction of satisable f s in Lemma and the

i

fact that each f involves O variables But recall that each f just represents a p ossible

i i

run of the log n restricted verier for the language Thus the description of each f

i

dep ends up on the construction of the verier Unfortunately the only known construction

of this verier is quite involved It consists in dening a complicated algebraic ob ject which

exists i the input is in the language The verier exp ects a memb ership pro of to contain

a representation of this ob ject In each of its runs the verier examines a dierentpartof

this provided ob ject Thus the function f representing that run is a correctness condition

i

for that part of the ob ject

For details on the algebraic ob ject we refer the reader to the next chapter A detail

worth mentioning is that each part of the ob ject and thus the denition of each f

i

dep ends up on every input bit This imparts our reduction a global structure In contrast

classical NPcompleteness reductions usually p erform lo cal transformations of the input

History and Background

Approximability The question of approximabilit y started receiving attention so on af

ter NPcompleteness was discovered JohSG See GJ for a discussion Muchof

the work attempted to discover a classication framework for optimization and aproxima

tion problems analogous to the framework of NPcompleteness for decision problems See

ADPADP AMSP for some of these attempts The most successful attempt was

due to Papadimitriou and Yannakakis who based their classication around a complexity

class they called MAXSNP see Chapter They proved that MAXSAT is complete

for MAXSNP in other words any unapproximability result for MAXSAT transfers au

tomatically to a host of other problems The desire to provesuch an unapproximability

result motivated the discovery of the PCP theorem

Ro ots of PCP The ro ots of the denition of PCP sp ecically the fact that the verier is

randomized go back to the denition of Interactive Pro ofs Goldwasser Micali and Racko

GMR and ArthurMerlin games BabaiBab see also BM Two complexity

classes arise from their denitions IP and AM resp ectively Both feature a p olynomial time

verier interacting with an allp owerful adversary called the prover who has to convince

the verier that the given input is in the language The dierence b etween the two classes

is that in the denition of IP the prover cannot read the veriers random string Early

results ab out these classes eshed out their prop erties including the surprising fact that

they are the same class GS see also GMS

The next step involved the in vention of multiprover interactive pro ofs and the asso ci

ated complexity class MIP by BenOr Goldwasser Killian and Wigderson BGKW

Here the single allp owerful prover in the IP scenario is replaced bymany allp owerful

provers who cannot communicate with one another during the proto col Again the mo

CHAPTER OLD VS NEW VIEWS OF NP

tivation was cryptography although so on Fortnow Romp el and Sipser FRS analyzed

the new mo del from a complexitytheoretic viewp oint They proved that the class MIP is

exactly the class of languages for whichmemb ership pro ofs can b e checked by a probabilis

tic p olynomial time verier that has random access to the pro of Since the probabilistic

verier can access over all choices of its random seed a pro of of exp onential size it follows

that MIP NEXPTIME Recall that NEXPTIME is the exp onential analogue of NP It

contains the set of languages that can b e accepted by a nondeterministic Turing Machine

that runs in exp onential time Since it seemed clear that MIP was quite smaller than

NEXPTIME the statementMIP NEXPTIME was considered unsatisfactorily weak A

similar situation prevailed for IP where the analogous statement IP PSPACE proved

implicitly byPapadimitriou in Pap was considered quite weak

The next development in this area came as a big surprise Techniques from program

checking due to Blum and Kannan BK Lipton Lip and Blum Luby and Rubin

feld BLR as well as some new ideas ab out how to represent logical formulae with

p olynomials Babai and FortnowBF Fortnow Lund Karlo and Nisan LFKN

and Shamir Sha were used to show that IP PSPACELFKN Sha and MIP

NEXPTIMEBabai Fortnow and LundBFL These connections b etween traditional

traditional complexity classes were proved using novel algebraic technques some of and non

whichwillbecovered later in this b o ok

Emergence of PCP The characterization of MIP from FRS and the result MIP

NEXPTIME together imply that NEXPTIME is exactly the set of languages for which

memb ership pro ofs can b e checked by a probabilistic p olynomial time verier Sucha

surprising result led to some thinking ab out NP as well sp ecically the pap ers of Babai

Fortnow Levin and Szegedy BFLS and Feige Goldwasser Lovasz Safra and Szegedy

FGL Although only the latter talked explicitly ab out NP the former dealt with

checking nondeterministic computationsincluding NP computations as a sub case their

techniques were actually scaling down the MIPNEXPTIME result The pap er FGL

implicitly dened a hierarchy of complexity classes unnamed there but whichwe can call

C Their class C tn is identical to the class PCPtntn as dened in this chapter

and their main result was that NP C log n log log n

What caused great interest in FGL was their corollary If the clique number of a

graph can b e appro ximated within any xed constant factor then all NP problems can b e

O log log n

solved deterministically in time n Computing the clique numberisawellknown

NPcomplete problem Kar They showed how to reduce every problem in PCPlog n

log log n and as a sp ecial sub case every problem in NP to the Clique problem in

suchaway that the gap probabilityversus probability used in the denition of

PCP translates into a gap in the clique number

To prove that NP C log n seemed to b e the next logical step and for two reasons

First this would imply that approximating the clique numb er is NPhard Second since

C log n is trivially a sub class of NP such a result would imply a new characterization of

NP namelyNPC log n

HISTORY AND BACKGROUND

This step was taken in the pap er Arora and Safra AS Somewhat curiously

we found that although the numb er of random bits used by the verier cannot b e sub

logarithmic or else P NPsee AS there w as no such restriction on the number of

querybits Hence we dened the class PCP denition with two parameters instead

of the single parameter used in FGL Weshowed that NP PCPlog n log log n

At this p oint several p eople for example the authors of AMS realized that prov

ing NP PCPlog n would prove the inapproximability of MAXSAT This result was

actually obtained in the pap er Arora Lund Motwani Sudan and Szegedy ALM

Owing to its great dep endence up on AS the pro of of the PCP Theorem is often at

tributed to jointly to ALM AS

Among other pap ers that were inuential in the ab ovedevelopments were those by

Beaver and Feigenbaum BF Lapidot and Shamir LS Rubinfeld and Sudan

RS and Feige and Lovasz FL Their contributions will b e describ ed in appro

priate places later

ve discovered other probabilistic char Other characterizations of NP Researchers ha

acterizations of NP One such result implicit in Liptons pap er Lip says that NP is

exactly the set of languages for whichmemb ership pro ofs can b e checked by a probabilistic

logspace verier that uses O log n random bits and makes just one sweep say lefttoright

over the pro ofstring Condon and Ladner CL further strengthened Liptons charac

terization Even more interestingly Condon Con then used the result of CL to

show the hardness of approximating the maxword problem This unapproximability result

app eared somewhat b efore and was indep endent of the more wellknown FGL pap er

An older characterization of NP in terms of sp ectra of secondorder formulae is due to

FaginFag His result since it involves no notion of computation is an interesting al

ternative viewp oint of NP It has motivated the denition of many classes of approximation

problems includin g MAXSNP

The above account of the evolution of PCP has been kept brief for more details refer

to the surveys by Babai Bab and Johnson Joh Arecent survey by Goldreich

Gol describes the known variations on probabilistic proof systems and how they are

usedincryptography

ects of the above theory Rubinfelds Three existing dissertations describe various asp

Rub describes the theory of program checking Lunds Lun describes the sur

prising results on interactive proofs and Sudans Sud describes program checking for

algebraic programs and its connection to the PCP Theorem

CHAPTER OLD VS NEW VIEWS OF NP

Chapter

PCP An Overview

The PCP theorem Theorem states merely the existence of a verier for any NP prob

lem Our pro of of the theorem is quite constructive wegive an explicit program for the

verier as well as explicit descriptions of what the pro of must contain in order to b e ac

cepted with probability

Details come later but let us rst face a fundamental problem to b e solved How can

the verier recognize the pro of as valid or invalid after examining only O bits in it At

the very least an invalid pro of must dier from a valid one on a large fraction of bits so

that they app ear dierent to a randomized test that examines very few bits in them This

consideration alone suggests using the theory of errorcorrecting codes Although wedo

not need much of the classical machinery of co ding theory some of its terminology is very

useful

Co des

m

Let b e an alphab et of symb ols and m an integer Let a word b e a string in The

distance between twowords is the fraction of co ordinates in which they dier This distance

function is the wellknown Hamming metric but scaled to lie in For let

Ball y denote the set of words whose distance to y is less than

m

de C is a subset of ev ery word in C is called a codewordAword y is close to A co

C if there is a co deword in Ball y Wesay just close when C is understo o d from the

context

The minimum distance of C denoted is the minimum distance b etween twocode

min

words Note that if word y is close there is exactly one co deword z in Ball y

min min

for if z is another suchcodeword then by the triangle inequality z z z y

y z whichisacontradiction Let Ny denote this nearest co deword to y

min

Acode C is useful to us as an enco ding ob ject using it we can enco de strings of bits

CHAPTER PCPANOVERVIEW

k

k

For anyinteger k such that jCjlet b e a onetoone map from f g to C such

a map clearly existsin our applications it will b e dened in an explicit fashion For

k k

x f g we call the co deword xan encoding of x Note that x y f g wehave

xy We emphasize that need not b e onto that is is not dened for

min

all co dewords

Example Let F b e the eld GFq and h an integer less than q Consider F as an

h

h

F as the following set of size jFj alphab et and dene a co de C

h h h

X X X

i i i

a b a b a b a a F

i i i h

h

i i i

where b b are distinct elements of F Note that a co deword is the sequence of values

h

P

h

i

taken by some p olynomial a x at these h p oints Since two distinct p olynomials of

i

i

degree h agree on at most h points the minimum distance b etween anytwo co dewords in

C is at at least hh

What is the alphab et size q jFj as a function of the co desize jC jLetN denote

h q

the number of codewords that is q We assumed that qhsowehaveq N

log N

Hence q

log log N

h

h

Heres one way to dene f g F For a a f g dene

h

h h h

X X X

i i i

a b a b a b a a

i i i h

h

i i i

Pro of of the PCP Theorem an Outline

We know of no simple direct pro of of the PCP Theorem Theorem The only known

pro of the one presented here uses dierentveriers which are combined in a hierarchical

construction using Lemma There is a tradeo b etween the numb er of randombits

and querybits used by the twoveriers which the construction exploits To enable this

w construction we require the veriers to b e in a certain normal form which is describ ed b elo

Moreover we asso ciate a new parameter with a verier namely decision time which is the

chief parameter of interest in the hierarchical construction

Recall from the description b efore Denition that a veriers op eration maybe

viewed as having three stages The rst stage reads the input and the random string and

decides what lo cations to examine in The second stage reads symb ols from onto the

worktap e The third stage decides whether or not to accept

Denition The decision time of a verier is the time taken by the third stage

PROOF OF THE PCP THEOREM AN OUTLINE

Next we describ e the normal form For ease we describ e veriers for the language SAT

Since SAT is NPcomplete veriers for other NP languages are trivial mo dications of

the verier for SAT Denote by the input CNF formula and by n the number of

n

its variables Identify in the obvious way the set of strings in f g with the set of

assignments to variables of Finally let denote the provided pro ofstring

Denition Averier for SATisin normal form if it satises the following prop erties

Has a certain alphab et The verier exp ects the pro of to b e a string over a certain

alphab et say the size of may dep end up on the input size n A query of a

verier involves reading a symb ol of and not just a bit

Can check assignments that are split into many parts The verier has a

sp ecial subroutine for eciently checking pro ofs of a very sp ecial form Let p be any

given p ositiveinteger The subroutine b ehaves as follows

ver the alphab et with The co de i It denes an asso ciated co de C o

min

n

p

to co dewords has an asso ciated onetoone map from strings in f g

ii It exp ects to have the sp ecial form

a a a

p

concatenation of strings where is a string that is supp osed to show that

the string a a is a satisfying assignment

p

More formallywesay that the subroutine can check assignments split into p

parts if it has the following b ehavior on pro ofs of the form z z where

p

z s and are strings over alphab et

i

If z z are co dewords such that z z is a satisfying

p p

assignment then there is a such that

Pr subroutine accepts z z

p

If i i p such that z is not close then for all

i min

Pr subroutine accepts z z

p

If all z s are close but N z N z is not a satisfying

i min p

assignment where N z iscodeword nearest to z then again for all

i i

Pr subroutine accepts z z

p

Nowwe mo dify Denition to make it meaningful for a verier in normal form

CHAPTER PCPANOVERVIEW

Denition Averier in normal form is r nqntnrestricted if on inputs of

size n it uses O r n random bits reads O q n symb ols from and has decision time

p oly tn While checking assignments split into p parts it reads O p q n symb ols

Note The parameter q n refers to the number of symb ols that is elements of the

alphab et read from Thus the number of bits of information read from the pro of is

O q nlogjj Wecho ose not to make jj a parameter since there is already an implicit

upp erb ound for it in terms of the ab ove parameters Realize that the decision time includes

the time to pro cess O q nlogjj bits of information so O q nlogjj polytn This

upp erb ound is go o d enough for our purp oses

Nowwe describ e a general technique to reduce the decision time and in the pro cess

the numb er of bits of information read from the pro of

Lemma Comp osition Lemma Let V and V be normalform veriers for SAT

that are RnQnTnrestrictedand r nqntnrestrictedrespectively Then

there is normal form SAT verier that is Rnr nQnq ntnrestricted where

r nr polyT n q nq polyT nand t n tpolyT n

Note Whenever we use this lemma b oth Qnqnare O Then the three veriers re

sp ect the b ounds Rn Tn r n tn and Rnr p olyT n tp olyT n

resp ectively Think of t as b eing some slowlygrowing function like log Then the decision

time of the new verier is at most log p oly T n O logT n an exp onential improve

mentover T n

We will prove the Comp osition Lemma at the end of this section First we outline how

it is used to prove the PCP Theorem The essential ingredients are Theorem and

whichwillbeproved in Chapter

Theorem SAT has a log n log nrestricted normal form verier

Although the ab oveverier reads only O symb ols in the pro of the numb er of bits it

reads is p oly log n We use the Comp osition Lemma to improveit

Corollary SAT has a normalform verier that is log n log log nrestricted

Pro of Use the verier of theorem to play the role of b oth V and V in the Comp osition

Lemma

Since the verier of Corollary is in normal form we could apply the Comp osition

Lemma again the reader may wish to calculate the b est PCP result obtained this way

Instead we use a new verier to terminate the comp osition It requires a huge number of

random bits but is really ecient with its queries

PROOF OF THE PCP THEOREM AN OUTLINE

Theorem SAT has a normalform verier that is n restricted and uses the

alphabet f g

Nowwe can prove the following theorem a strong form of the PCP theorem

Theorem There exists a normal form verier for SATthatis log n restricted

and uses the alphabet f g

Pro of Use the veriers of corollary and theorem as V and V resp ectively in the

Comp osition Lemma

The verier of Theorem likeallourveriers is a verier for SAT By examining

its p erformance not only do we conclude that NP PCPlog n but also that the verier

in question is in normal form For this and other strong forms of the PCP theorem see

Chapter

Our outline of the pro of of the PCP Theorem is complete Wehave shown that it suces

to prove the Comp osition Lemma whichwe will do now and Theorems and which

we will do in chapter

Pro of Of Composition Lemma The ideas underlying the comp osition are simple Once

V its decision dep ends up on a very small we x the random string of the rst verier

p ortion of the pro of string and is computed in very little time namely the decision time

The Co okLevin Theorem implies that a tinyCNFformula describ es whether the decision

is an accept or a reject Wemodifyverier V to use verier V which is in normal form

tocheck that the ab ovementioned p ortion of the pro of is a satisfying assignment to this

tiny SATformula Doing this involves using V s abilitytocheck split assignments and

requires that relevant p ortions of the pro ofstring b e present in an enco ded form using V s

enco ding

Nowwe provide details Let n b e the size of the input given to verier V and let

Q R T denote resp ectively the numb er of queries made by the verier the number of

random bits it uses and its decision time The hyp othesis of the lemma implies that

Q R T are O Qn O Rn and p oly T n resp ectively

R

Let the random string of verier V b e xed to r f g This xes the sequence of

lo cations in pro of that the verier will examine Let i i i denote these lo cations

Q

Strictly sp eaking we should express the dep endence up on r explicitly and denote these

by i r i r etc The decision pro cess of V is a computation that runs in time T

using an input i i i where j denotes the symbol in the j th lo cation

Q

of pro of and denotes concatenation of strings The strong form of the Co okLevin

theorem Fact A implies we can write down a SATformula of size p oly T such that

r

V accepts i

y st i i i y is a satisfying assignmentfor

r Q r r

CHAPTER PCPANOVERVIEW

Here we are thinking of eachsymbol i as b eing represented by a string of bits

j

Tomake our description cleaner we assume from nowonthatcontains a sequence of

additional lo cations one for eachchoice of the random string r The r th lo cation supp osedly

contains y Further we assume that V when using the random string r makes a separate

r

query to this lo cation to read y Let i b e the index of this lo cation Then V accepts

r Q

using the random string r i

i i i is a satisfying assignmentfor

Q r

But there is a very ecientway to determine the truth of a statementlike the one in

Use V s subroutine for checking assignments split into Q parts Of course this requires

the structure of to b e somewhat dierent The symbol in each lo cation of is now

from bitstrings to co dewords required to b e enco ded using V s map

The next page contains the program of V the new verier obtained by the comp osition

It uses V to check assignments for that are split into Q parts We assume without

r

loss of generalityby padding with irrelevant bits that the string y in has the

r

same length as eachj So these Q parts have equal size Let m denote the number

of variables in Note m polyT Let denote V s map from bitstrings of size

r

mQ to co dewords and let R be the numb er of random bits it uses while checking the

split assignment Note By the hyp othesis of the lemma R O r m For convenience

we assume by rep eating the program of V some O log times that when the input

formula is not satisable V rejects with probabilit y at least instead of just as

required by denition of a verier is a small enough constant say

Complexity We analyze V s eciency Let m denote j j The verier uses R R

r

R r m random bits whichisO Rnr p olyT n Further the numb er of queries it

makes and its decision time are exactly those of V s subroutine Since V is r nqntn

restricted its subroutine makes Q q m queries while checking assignments split into

Q parts and has decision time tm We conclude that V is R r m Q

q mtmrestricted But m j j polyT n so the parameters for V are as claimed

r

Program of V

Given Table with the same numb er of lo cations as

new

R

Table with entries

R R

Pick a random r f g and a random r f g

Use V to generate lo cations i r i r and SATformula

Q r

Run V s subroutine using random string r tocheck that the

pro of z z enco des a satisfying assignmentfor

Q r r

where z is the entry i r and the entry r

j new j r

ACCEPT i V s subroutine accepts

PROOF OF THE PCP THEOREM AN OUTLINE

ΠΠ new i (r) i (r) i (r) i (r) 1 Q+1 1 Q+1

Γ r

( a ) ( b )

Figure a Pro of that V exp ects b Pro of that V exp ects Shaded area in

new

represents the assignment split into Q parts that corresp onds to V s random seed r

Correctness Showing that V is a correct verier for SAT consists in two parts In b oth

parts let a denote the string i r i r ands denote the string i r

r Q r

i r It will b e clear from the context what refers to

Q

First supp ose the input is satisable Then there is a pro of that V accepts with

probabilityWe show that there exists a pro of which V accepts with probability

new

Since V accepts with probabilitywehave

Pr a is a satisfying assignmentfor

r r

R

r f g

where is the SAT formula representing the veriers computation Denition ab out

r

R

what it means to check assignments split into Q parts implies that for every r f g

there is a string such that

r

Pr V s subroutine accepts s using r as a random string

r r

R

r fg

Construct the desired new pro of as follows Let b e a table with the

new new

same numb er of lo cations as whose j th lo cation contains the co deword j Let

R

R

b e a table of lo cations viewed as b eing in onetoone corresp ondence with f g For

R

r f g let the lo cation r of contain the string dened in Equation The

r

R R

decision of the new verier V when it picks r from f g and r from f g is just the

decision of V s subroutine on the pro ofstring s Our construction insures that the

r r

subroutine accepts irresp ectiveofrr Hence

Pr V accepts

new

rr

Nowwe turn to the second half is unsatisable Let b e any pro of and let

new

p b e the probability with which V accepts it We show that p

CHAPTER PCPANOVERVIEW

First mo dify the pro ofstring as follows replace eachentry of by the co deword

new

nearest to it or a co deword nearest to it if there is more than one choice Call this new

table Then V must accept with probability at least p For V accepts

new new

i V s subroutine for checking split assignments accepts The subroutine accepts with

probability no more than if even one of the parts in the enco ding of the split assignment

is not close Hence turning the entries of into co dewords as ab ovecanlower the

min new

probability of acceptance byatmost

Next weturn into a pro of that the original verier V accepts with almost the

new

same probabilityasV do es Dene by making its j th entry the preimage of the j th entry

j note if is not dened use an arbitrary of that is j

new new new

symb ol instead

Let b e the probability with which V rejects Since is unsatisable is more

R

than That is to say is the fraction of r f g for which

a do es not satisfy Pr

r r

R

r fg

Let r b e one of the choices of the random string for which a do es not satisfy That

r r

is i r i r do es not satisfy Since s is i r i r

Q r r Q

wehave

V s subroutine accepts s using the random string r Pr

r

R

r fg

irresp ectiveof In particular V s subroutine accepts the pro of s r with probability

r

at most Hence wehave

Pr V accepts

new

rr

But we assumed that V accepts with probability is at least p Therefore we

new

have

p

which implies p Thus the claim of correctness is proved

History and Background

The rst PCPtyp e verier app ears in the result MIP NEXPTIME BFL When

Babai et al BFLS scaled down that result to NP they noticed that their PCP

typ e verier for NP has a low decision time Actually their exact result was somewhat

stronger This observation motivated the work in AS where the Comp osition Lemma

HISTORY AND BACKGROUND

is implicit The use of largedistance co des in the denition of the normal form verier was

also motivated by a similar situation in BFLS Comp osition was termed Recursion in

AS b ecause in that pap er veriers were comp osed only with themselves Full use of the

lemma as describ ed in this section was made in ALM where Theorems and

were proven The b est verier of ASwas log n log log n log log n restricted

The idea of comp osing veriers has b een crucial in more recentdevelopments in the area of

probabilistically checkable pro ofs sp ecically in constructing more ecient PCPs

CHAPTER PCPANOVERVIEW

Chapter

A Pro of of the PCP Theorem

This chapter contains a pro of of the PCP theorem As p er the outline in Chapter it

suces to construct the veriers of Theorems and First wegive a brief overview

of the techniques used

Underlying the description of b oth veriers is an algebraic representation of a SAT

formula The representation uses a simple fact every assignment can b e enco ded as a

multivariate p olynomial that takes values in a nite eld see Section A p olynomial

that enco des a satisfying assignment is called a satisfying polynomial Just as a satisfying

b o olean assignment can b e recognized bychecking whether or not it makes all the clauses

of the SATformula true a satisfying p olynomial can b e recognized bychecking whether

it satises some set of equations involving the op erations and of the nite eld

Each of our veriers exp ects the pro of to contain a p olynomial plus some additional

information showing that this p olynomial is a satisfying p olynomial in other words it

satises the ab ovementioned set of equations The verier checks this information using

some algebraic pro cedures connected with p olynomials These pro cedures are describ ed in

a blackb ox fashion in Section and in full detail in Section The blackb oxde

scription should suce to understand Sections and in whichweprove Theorems

and resp ectively

All results in this chapter are selfcontained except Theorem ab out the p erformance

of the lowdegree test whose pro of takes up Chapter

Throughout the chapter denotes the SATformula for which pro ofs are b eing checked

We use n to denote b oth the numb er of clauses and the numberofvariables We defend

this usage on the grounds that the number of variables and clauses could b e made equal by

adding irrelevantvariables which do not app ear in any clause to the set of variables

A note on error probabilities While describing veriers we write in parenthesis

and in italics the conditions that a pro of must satisfy in the go o d case ie the case

when is satisable The reader maywishtocheck that there exists a pro of meeting

those conditions and which is therefore accepted with probability Upp erb ounding the

CHAPTER APROOF OF THE PCP THEOREM

probability of rejection in the bad case when is not satisable by is more dicult

and requires pro of

The overall picture After reading the pro of a lo ok at Figure might help the

reader recall all imp ortant steps

Polynomial Co des and Their Use

Let F b e the nite eld GFq and k d be integers A k variate polynomial of degree d

j j j

k

over F is a sum of terms of the form ax x x where a F and integers j j

k

k

k

satisfy j j dLetFx x b e the set of functions from F to F that can

k d k

b e describ ed by a p olynomial of total degree at most d

We will b e interested in representations of p olynomials by valueAk variate p olynomial

k

k

k

denes a function from F to F so it can b e expressed by j Fj q values In this

k

representation a k variate p olynomial or any function from F to F for that matter is a

k

word of length q over the alphab et F

Denition The code of k variate polynomials of degree d or just polynomial code

k

q

when k d are understo o d from context is the co de F x x inF

d k

k

Note that the distance b etween twowords is the fraction of p oints in F they disagree

on The following lemma for a pro of see App endix A shows that the p olynomial co de has

large minimum distance

Fact Schwartz Two distinct polynomials in F x x disagreeonatleast

d k

k

dq fraction of points in F

k

F Fisa close function say In our applications d q Thus if f

then the p olynomial that agrees with it in at least fraction of the p oints is unique

In fact no other p olynomial describ es f in more than even dq fraction of the p oints

We will often let denote a suitably less constant say

e

Denition Foraclose function f the unique nearest p olynomial is denoted by f

Polynomials are useful to us as enco ding ob jects We dene b elow a canonical way due

to BFLS to enco de a sequence of bits with a p olynomial For convenience we describ e a

more general metho d that enco des a sequence of eld elements with a p olynomial Enco ding

a sequence of bits is a subcase of this metho d since F

The use of F ab ove should not b e confused with the practice in some algebra texts of using F as a

d q

shorthand for GF q

POLYNOMIAL CODES AND THEIR USE

m [ 0, h ] m

F

m

Figure A function from h to F can b e extended to a p olynomial of degree mh

Let h be an integer such that the set of integers h is a subset of the eld F Readers

uncomfortable with this notation can think of hasany subset of the eld F that has

size h

m

b

Theorem For every function s h Fthere is a function s F x x

mh m

m

b

such that sy s y for al l y h

m

Pro of For u u u h let L b e the p olynomial dened as

m u

m

Y

L x x l x

u m u i

i

i

where l is the unique degreeh p olynomial in x that is at x u and at x h nfu g

u i i i i i

i

u and at all is at That l x exists follows from Fact A Note that the value of L

u u i

i

m

the other p oints in h Also its degree is mh

b

Now dene the p olynomial s as

X

b

s x x sx x u L

u m m

m

uh

Example Let m h Given any function f Fwe can map it to a

b

bivariate degree p olynomial f asfollows

b

f x x x x f x x f

x x f x x f

m

Denition For a function s h F a polynomial extension of s is a function

b

s F x x that satises

mh m

m

b

s y sy y h note the extension need not b e unique

CHAPTER APROOF OF THE PCP THEOREM

The enco ding Let h b e an integer such that h F and l an integer such that

l

m

l h for some integer m Dene a onetoone map from F to F x x in

mh m

other words from sequences of l eld elements to p olynomials in F x x as follows

mh m

m

m

Identify in some canonical way the set of integers flg and the set h F For

instance identify the integer i flg with its mdigit representation in base h

m

Thus a sequence s of l eld elements may b e viewed as a function s from h to F

b

s of this function This map is onetoone Map the sequence s to any p olynomial extension

b

b

b ecause if p olynomials f and g are the same then they agree everywhere and in particular

m

on h which implies f g

The inverse map of the ab ove enco ding is obvious A p olynomial f F x x is

mh m

m m

the p olynomial extension of the function r h F dened as r x f x x h

m

m m

Note that we enco de sequences of length l h by sequences of length jFj q

In later applications this increase in size is not to o much The applications dep end up on

some algebraic pro cedures to work correctly for which it suces to take q polyh Then

m O m

q is h polyl Hence the increase in size is p olynomially b ounded

Next we indicate howwe will use the ab ove enco ding

Denition Let b e a CNF formula with n variables and F b e a eld A sequence of

eld elements a a a F is said to represent a satisfying assignment if the following

n

partial b o olean assignment makes all clauses in true If a resp a let the

i i

ith variable b e false resp true and otherwise do not assign a value to the ith variable

Akey concept connected with our veriers is that of a satisfying polynomial

Denition Let b e a CNF formula with n variables and F b e a eld A satisfying

polynomial of is a p olynomial extension for any appropriate choice of parameters m and

h as ab ove of a sequence of eld elements that represents a satisfying assignmentto

As noted ab ove when the parameters m h jFj are chosen appropriately the satisfying

p olynomial can b e represented by p oly n bits

harac Representing clauses by equations We give a simple algebraic condition to c

terize sequences of eld elements that represent satisfying assignments Our veriers will

use these conditions to check whether or not a given p olynomial is a satisfying p olynomial

Lemma Let be a SAT instancewith n variables and n clauses Let F beany

eld and let X X be formal variables taking values over F Thereisasetofn cubic

n

equations fp X X X i ng such that a sequence a a Frepresents

i i i i n

a satisfying assignment i

p a a a i n

i i i i

POLYNOMIAL CODES AND THEIR USE

The set of equations can beconstructedinpolyn time

Pro of Let y y be the variables of Arithmetize each clause of as follows For

n

i n asso ciate the eld variable X with the b o olean variable y Ineach clause

i i

replace y by X the op eration by multiplication over F and y by X Thus

i i i i

for example clause y y y is replaced by the cubic equation X X X

i j k i j k

X Note that the expression on the left hand side of this equation is is i X or

j i

or X Thatistosayvalues of X X X that satisfy the equation corresp ond in a

k i j k

natural way to b o olean values of y y y that make the clause true

i j k

If the variables involved in the ith clause are y y y the arithmetization ab ove

i i i

yields a cubic equation p X X X for this clause Thus wegetn equations

i i i i

one p er clause such that the assignment X a X a satises all of them i the

n n

corresp onding b o olean assignmenttoy y satises

n

Algebraic Pro cedures for Polynomial Co des

In this section wegive a blackb ox description of some algebraic pro cedures concerning

p olynomial co des for details of howtheyw ork refer to Section First we explain how

averier uses them

The verier denes using the p olynomial extension enco ding from Denition a

mapping from b o olean assignments to p olynomials of degree d for some suitable d and

exp ects the pro of to contain one such p olynomial Recall that p olynomials are represented

m

byvalue so the pro of actually contains some table of values f F FHow can the

verier check that f F x x Our rst pro cedure the test for closeness allows

d m

it to do almost that By lo oking at very few values of f the pro cedure determines whether

or not f is close where is some suitably small constant So supp ose f is indeed found

e

to b e close Our second pro cedure can reconstruct values of f the p olynomial closest to

f atany desired p oints Together the two pro cedures allow the verier to assume for all

practical purp oses that f is exactly a p olynomial

Actually we describ e two pairs of pro cedures The rst pair is somewhat sp ecialized

and works only for a sp ecial p olynomial co de called the linear function code which is the

co de F x x whereFisGF the eld of two elements and degree d is This

d m

co de will b e used in Section

Pro cedure Pro cedures for the linear function co de Let F b e the eld GF

m

i Test for closeness Given any function f F Fand the pro cedure tests

f for closeness by examining only O values of f Iff is a co deword the pro cedure

accepts with probabilityIff is not close it rejects with probability

m m

e

ii Reconstructing Values of f Given any close function f F F and b F

e

this pro cedure outputs f b with probability at least It reads the value of f at only

CHAPTER APROOF OF THE PCP THEOREM

p oints

Complexity The pro cedures examine f in O random lo cations which assuming is

m

a constant requires O log jFj O m random bits Apart from the time taken to read

these values the pro cedures p erform only O op erations in GF which takes only O

time

Next we describ e the pro cedures for p olynomial co des of degree higher than Two

dierences should b e noted First the pro cedures for general degree d require additional

information in the form of a separate table T We emphasize that the correctness of the

pro cedure do es not dep end up on the contents of the tableb efore using an y information

from the table the pro cedure rst checks probabilistically that it is correct Nevertheless

reading information from the table helps eciencysincechecking that it is correct is easier

than generating it from scratch Second the total number of entries read by the pro cedures

is some constant indep endent of the degree d and numberofvariables m Even more

signicantly in part ii the number of entries read is indep endentofc the number of

e

p oints at which the pro cedure is constructing values of f The fact that b oth pro cedures

examine only O entries in the tables will b e crucial for our strongest constructions

Pro cedure Pro cedures for the general p olynomial co de Let F b e the eld

GFq

m

i Test for closeness Given f F F a number d such that qd and and table

T

If f isacodeword there exists a table T such that the pro cedure accepts with probability

If f is not close to F x x the pro cedure rejects with probability at least

d m

irresp ectiveofT The pro cedure reads O entries from T and the same number of

values of f

e

ii Reconstructing Values of f Letc d be integers satisfying cd q

m m

and a table Given a close function f F F a sequence of c p oints z z F

c

T

The pro cedure reads values of f andentry from T Iff is a co deword there exists a

e e

table T such that the pro cedure always outputs the correct values of f z f z and

c

p

irresp ective in particular never outputs REJECT But otherwise with probability

e e

of T the pro cedure either outputs REJECT or correct values of f z f z

c

Complexity The rst pro cedure runs in time p oly m d log the second in time

m

p oly m d c Randomness is required only to generate O elements of F so only

O m log jF j random bits are needed

Note Whenever we use Pro cedure the function f is supp osed to represent a sequence

of n bits The eld size the degree and the number of variables have b een carefully chosen

m

so that jFj p oly n Thus the pro cedures require O m log jFjO log n random bits

Also the degree the number of variables and c the number of points in Pro cedure ii

is p oly log n so b oth the running time and the size of the table entries are p oly log n

POLYNOMIAL CODES AND THEIR USE

An Application Aggregating Queries

As an immediate application of Pro cedure we prove a general result ab out howto

mo dify anyverier so that all its queries to the pro of get aggregated into O queries The

mo dication causes a slight increase in the numb er of random bits and the alphab et size

Lemma Let L be any language For every normal form verier V for L that uses

O r n random bits and has decision time polytn there is a normal form verier V for

log tn

L that is r n tnr nrestricted

log r n

Pro of For starters we ignore the prop erty of normal form veriers having to do with

checking assignments that are split into many parts

is that the pro of is now supp osed The main idea in the construction of the new verier V

to b e in a dierent format It contains a p olynomial extension of a string of bits that the old

verier V would have accepted as a pro of with probability What enables V to bunch

up its queries to a pro of in this format is the magical ability of Pro cedure ii to extract

manyvalues of a provided p olynomial by reading only O entries in some accompanying

tables

Nowwe state the construction more precisely Let us x an input x of size n Let R

and t stand for the numb er of random bits and the decision time of V resp ectivelyNote

that R O Rnt p olytn The numb er of bits of information that V can read and

pro cess is upp erb ounded by the decision time tWe will assume wlog that exactly t bits

R

are read in each run Since there are only dierent p ossible runs one for eachchoice

of the random string and in each run exactly t bits are read wemay assume that every

N

R

provided pro ofstring has size N where N t Let us call a string in f g as perfect

if V accepts it with probability

j

Assume by allowing the pro of to contain unnecessary bits if necessary that N l

for some integers j l Let F b e a eld such that l Fwe sp ecify the values of the

N

paramaters later Recall from Denition that every string s in f g in other words

b

s is a pro of that the old verier can check has a p olynomial extension s in F x x

jl l

l

b

is exactly s The sequence of values that s takes on j

The new verier V exp ects as a pro of that the input is in the language a function that

l

is a p olynomial extension of a p erfect string But given such a function say g F F how

to check that it represents a p erfect string First the verier checks using Pro cedure

i that g is close for some small enough Thenitmust check whether the sequence of

l

e e

values of g on j is a p erfect string Admittedly the sequence of values of g is a string

of eld elements and not of bits but we can view it as a string of bits byinterpreting the

zero of the eld as the bit and every nonzero as the bit To do this V runs the old

verier V on the sequence and accepts i V rejects Since V queries t bits while checking

e

a pro of the new verier V needs to reconstruct the values taken by g at some t p oints

j

in l It uses Pro cedure ii for this purp ose and therefore needs to read only O

entries in some provided table

CHAPTER APROOF OF THE PCP THEOREM

Our informal description of V is complete A more formal description app ears on the

next page Next we prove that V is a correct verier for language L

If x L there is a p erfect string When V is provided with a p olynomial extension

of this string along with all the prop er tables required by Pro cedure it accepts with

probability

Program of V

j

Given f F F tables P and P P

R

i Use Pro cedure i and table P to check that f is close

if the pro cedure fails

f is not a polynomial

output REJECT and exit

R

ii Pick a random r f g

Compute the sequence of t lo cations in N that

V would examine in a pro of using r as random string

As indicatedabove these locations may be

j

viewedaspoints in l

Use Table P and Pro cedure ii to reconstruct

r

e

the values of f on these t lo cations

As indicatedabove these values may be

viewed as bits

If the Pro cedure rejects

REJECT

else

simulate V s computation on these t bits

and ACCEPT i it accepts

exit

We show that when x L then V rejects every pro of with probability at least

p

beany pro of given to V Iff is not close part i of Let f P P P

R

the program rejects with probability at least So assume wlog that f is close

e

Let b e the string of bits whose extension is f Sincex L the old verier V must reject

with probability at least But the new verier merely simulates V on except that

the simulation may sometimes b e erroneous if Pro cedure ii fails to pro duce the correct

bits of The probability that Pro cedure ii outputs an erroneous answer is at most

p

since f is close Thus the probability that the new verier V rejects while

p

simulating V is at least

Hence wehave shown the correctness of V asaverier for language L

A VERIFIER USING O LOG N RANDOM BITS

Complexity The prop erties of Pro cedure imply that V reads only O entries from

the provided tables By viewing eachtableentry as a symb ol in some alphab et we conclude

that V reads only O symb ols from the pro of Nowwe sp ecify the minimum eld size

R

that makes everything else work out Recall that N t Assume l p oly log N Since

j

l N this means j logNlog log N Let eld size q b e the larger one of p oly jl

and p oly jT This is the minimum we need for Pro cedure to work as claimed

The decision time of V is Decision time of V Time for pro cedure whichis

T p oly jlT polyR T The amount of randomness it uses is

R log T

j

O R R O log jFj O R j log T

log R

Thus V resp ects the claimed b ounds

Checking assignments split into p parts The mo dication is the same That is to

say V exp ects each of the p split parts to b e represented by its p olynomial extension along

with tables similar to the ones ab ove

AVerier Using O log n Random Bits

In this section we prove Theorem one of the two theorems wewanted to provein

this chapter One essential ingredient is the following lemma whose pro of app ears in

Section

Lemma There exists a normal form verier for SATthatis log n polylog n log n

restricted

Theorem follows easily from this lemma by using the general pro cedure for aggre

gating queries into O queries

Pro of Theorem The verier of Lemma fails the requirements of Theorem in

ever Lemma just one way it reads p oly log nsymb ols in the pro of instead of O How

allows us to aggregate the queries of this verier and replace them with O queries A

quick lo ok at the accompanying blowup in decision time and amount of randomness shows

that it is not to o much

Using Lemma on the verier of Lemma we obtain a log n log nrestricted

verier

CHAPTER APROOF OF THE PCP THEOREM

A Less EcientVerier

This section contains a pro of of Lemma The exp osition of the pro of uses a mix of ideas

from BFL BFLS FGL

Let b e an instance of SAT and F a nite eld The verier will use the fact see

Denition that every assignment can b e enco ded by its p olynomial extension

Denition A satisfying polynomial for SAT instance is a p olynomial extension of

a satisfying assignment for Note see Denition for a moredetailed denition

P

k k

Denition The sum of a function g F FonasetS F is the value g x

xS

The verier exp ects the pro of to contain a satisfying p olynomial for some sp ecied

choice of parameters m h Using the pro cedure describ ed in Section the verier

e

checks that the provided function f is close Next it has to check that f the p olynomial

nearest to f is a satisfying p olynomial The verier reduces this question probabilistically

using Lemma to whether or not a certain related p olynomial P sums to on a certain

i

edifitisofthetyp e l for some nicely b ehaved set S A set S is nicely b ehav

integers l i The Sumcheck pro cedure Pro cedure can eciently verify the sum of a

p olynomial on a nicely b ehaved set The verier uses this pro cedure to verify that the

sum of p olyomial P on S is if it is not the v erier rejects While doing the Sumcheck

e

the verier needs to reconstruct values of f which is also easy to do using the the pro cedure

in Section

Nowwe set out the parameters used in the rest of this section Assume m h are integers

m

suchthath O log n n h if n is not of this form we add unnecessary

variables and clauses until it is Note that this means m log nlog log n Finally

m

let F GFq b e a eld of size p oly h Then a function f F F is represented by

m O m

q h polynvalues

The following lemma describ es an algebraic condition that characterizes a satisfying

p olynomial

Lemma Algebraic View of SAT Given A F x x thereisapoly

mh m

A A

nomial time constructible sequenceofpolyn polynomials P P F x x

mh m

such that

A A m

If A is a satisfying polynomial for then each of P P sums to on h

Otherwise at most th of them do

A

For each i evaluating P at any point requires the values of A at points

i

Pro of In Lemma we replaced clauses of by cubic equations A sequence of eld

ts satises these equations i it represents a satisfying assignmentto In this elemen

lemma we replace that set of equations by a more compactly stated algebraic condition

A VERIFIER USING O LOG N RANDOM BITS

m m

Since h n the cub e h has n points By denition p olynomial A is

m

a satisfying p olynomial i the sequence of values Av v h ordered in some

canonical way represents a satisfying assignment in other words satisfy the set of cubic

equations in Lemma

m m

For j let c v b e the function from h h to f g such that

j

th

variable in clause c and otherwise Similarly let s cbea c v if v is the j

j j

th

m

function from h to f g such that s c if the j variable of clause c is unnegated

j

and otherwise The following is a restatement of the set of equations in Lemma

m

A is a satisfying p olynomial i for every clause c h and every triple of variables

m

v v v h wehave

Y

c v s c Av

j j j j

j

that is to sayi

Y

c b

c v s c Av

j j j j

j

where in the previous condition wehave replaced functions and s app earing in condition

j j

m m

c b

by their p olynomial extensions F Fands F F resp ectively Con

j j

ditions and are equivalent b ecause by denition a p olynomial extension takes

m m

the same values on the underlying cub e whichish for s and h for asthe

j j

original function

m

Let g F F a p olynomial in F x x b e dened as

A mh m

Y

b c

z w w w z w s z Aw g

j j j j A

j

where eachof z w w w is a vector of m variables z x x w x x

m m m

w x x w x x

m m m m

Then wemay restate condition as A is a satisfying p olynomial i

m

g isatevery p ointofh

A

in other words i for every p olynomial R in the zerotester family we will construct in

i

Lemma

m

R g sums to on h

i A

Further if the condition in is false the statement of Lemma implies that the

condition is false for at least of the p olynomials in the zerotester family

n o

A A

Now dene the desired family of p olynomials P by P

A

z w w w R z w w w g z w w w P

i A i

CHAPTER APROOF OF THE PCP THEOREM

A

where R is the ith memb er of the zerotester family Note that P F x x

i mh m

i

A

Further evaluating P at anypoint requires the value of g at one p oint whichby

A

i

insp ecting requires the value of A at three p oints

Thus the claim is proved

Constructibility The construction of the p olynomial extension in the the pro of of The

c b

orem is eective We conclude that the functions s can b e constructed in p olyn

j j

time

Thus assuming Lemma Lemma has b een proved

The following lemma concerns a family of p olynomials that is useful for testing whether

j

or not a function is identically zero on the cub e h for anyintegers h j

Lemma Zerotester Polynomials BFLSFGL There exists a family of

m O m

R R g in F x x such that if f h Fisany q polynomials f

mh m

function not identical ly thenif R is chosen randomly from this family

X

Ry f y Pr

m

y h

O m

This family is constructible in q time

Pro of In this pro of we will use the symbols h to denote b oth integers in fhg

and eld elements We use b oldface to denote the latter use Thus F for example

For nowlet t t b e formal variables later we give them values Consider the

m

following degree h p olynomial in t t

m

m

Y X

i

j

t f i i i

j

m

j

i i i h

m

m

This p olynomial is the zero p olynomial i f is identically on h Further if it

is not the zero p olynomial then its ro ots constitute a fraction no more than hmq of all

m

p oints in F Assume that this fraction is less than

m

We prove the lemma by constructing a family of q p olynomials

fR b b Fg such that

b b m

m

X

R i i i f i i i

b b

m

m m

i i i h

m

A VERIFIER USING O LOG N RANDOM BITS

i b b is a ro ot of the p olynomial in

m

Denote by I x the univariate degreeh p olynomial in x whose values at h F

t i i

i

h

are t t resp ectively such a p olynomial exists see Fact A

i

i

Let g b e the following p olynomial in variables x x t t

m m

m

Y

g t t x x I x

m m t i

i

i

Note that

X

f i i i g t t i i i

m

m m

i i i h

m

m

X Y

i

j

f i i i t

j

m

j

i i i h

m

Now dene R as the p olynomial obtained by substituting t b t b

b b m m

m

in g

R x x g b b x x

b b m m m

m

This family of p olynomials clearly satises the desired prop erties

p

Example We write a p olynomial g for checking the sums of functions on h for

h and p

g t t x x x t x t

Hence wehave

X

f x x g t t x x f f t f t f t t

x x

Clearly the p olynomial in is nonzero if any of its four terms is nonzero

Nowwegive a blackb ox description of Sumcheck a pro cedure that checks the sums of

l

p olynomials on the cub e p for some integers p l Section describ es the pro cedure

more completely

Pro cedure SumCheck Let F GFq andd l be integers satisfying dl q

F and a table T Given B F y y p Favalue c

d l

l

If the sum of B on p is not c the pro cedure rejects with probabilityatleast But

if the sum is c there is a table T such that the pro cedure accepts with probability

l

Complexity The pro cedure uses the value of B at one random p ointinF and read another

O ld log jFj bits from the pro of It runs in time p oly l d log jFj

CHAPTER APROOF OF THE PCP THEOREM

Nowwe prove Lemma

Pro of Of Lemma For nowwe ignore the asp ect of normal form veriers having to

do with checking split assignments

m

Our verier exp ects the pro of to contain a function f F F and a set of tables

describ ed b elow In a goodproof f is a satisfying polynomial

One of these tables allows the verier to check that f is close using Pro cedure

i

Another set of tables allows the verier to p erform a Sumcheckoneach p olynomial

e e

f f

constructed in Lemma concerning the algebraic view of P in the family P

SAT

e

Another set of tables allows the verier to reconstruct the value of f at any three p oints

e e

The verier works as follows It checks that f is close Then to checks that f is

a satisfying p olynomial it uses the algebraic view of SATItusesO log n random bits

to select a p olynomial P uniformly at random from the family of Lemma and uses the

m

Sumcheck Pro cedure to check that P sums to on h Wehave describ ed the

verication as a sequence of steps but actually no step requires results from the previous

steps so they can all b e done in parallel

The Sumcheck requires the value of the selected p olynomial P at one p oint whichby

e

the statement of Lemma requires values of f at p oints The verier reconstructs these

three values using Pro cedure ii and the appropriate table in the pro of

Correctness Supp ose is satisable The verier clearly accepts with probabilityany

pro of containing the p olynomial extension of a satisfying assignment as well as prop er

tables required by the various pro cedures

Now supp ose is not satisable If f is not close the verier rejects with probability

at least So assume wlog that f is close The verier can accept only if one of the

m

three events happ ens i The selected p olynomial P sumstoonh By Lemma

this event can happ en with probabilityatmost ii P do es not sum to but the

Sumcheck fails to detect this The probability of this even t is upp erb ounded by the error

probability of the Sumcheck whichisO mhq o iii Pro cedure ii pro duces

p

e

erroneous values of f The probability of this event is upp erb ounded by

To sum up if is not satisable the probability that the verier accepts is at most

o which is less than By running it O times the probability can b e

reduced b elow

Complexity By insp ecting the complexity of the Sumcheck the lowdegree test and the

m

test for closeness we see that the verier needs only log jFj O log n random bits for its op eration

A VERIFIER USING O LOG N RANDOM BITS

Toachievealow decision time the verier has to do things in a certain order In its

rst stage it reads the input selects the ab ovementioned p olynomial P and constructs P

as outlined in the pro of of Lemma All this takes p oly n time and do es not involve

reading the pro of The rest of the verication requires reading the pro of and consists of the

following pro cedures the test for closeness the Sumcheck and the reconstruction of

e

three values of f All these pro cedures run in time p olynomial in the degree h the number

of variables O m and the log of the eld size log jFjWechose these parameters to b e

p oly log n Hence the decision times is p oly log n and so is the alphab et size and the

numb er of queries

To nish our claim that the verier is in normal form wehave to showthatitcancheck

split assignments We do this next

Checking Split Assignments

We show that the verier of Lemma can check assignments that are split into k parts

for anypositiveinteger k

Recall from Denition that in this setting the verier denes an enco ding metho d

and exp ects the pro of to b e of the form S S where is some information

k

that allows an ecientcheckthatS S is a satisfying assignment concatenation

k

of strings

In this case we assume the SAT instance has nk variables split into k equal blo cks

y y y y

n nk

nk

m

Let n h with m h the same as in the pro of of Lemma The verier uses

n

the enco ding that maps strings in f g to their degreemh p olynomial extensions In

m

other words the pro of is required to contain k functions f f F F Part has to

k

contain a set of tables In a goodproof f f are mvariate extensions of assignments

k

to the k blocks such that the overal l assignment is a satisfying assignment

While checking such a pro of the verier follows the program of Lemma quite closely

It rst checks that each f f is close Then using a mo dication of the algebraic

k

f f

view of SAT Lemma it reduces the question of whether or not f f together

k

represent a satisfying assignment to a single Sumcheck The mo dication is the following

Corollary Given A A F x x thereisasequenceofpolynomials

k mh m

A A A A

k k

such that P P F x x

mh m

A A A A

k k

If A A together represent a satisfying assignment then al l of P P

k

m

sum to on h and otherwise at most of them do

A A

k

at any point can beconstructedfrom the values of A A A the value of P

k

i

at points each

CHAPTER APROOF OF THE PCP THEOREM

m m

Pro of For j and i k let p b e a function from h h to f g

ij

such that p c v is i y is the j th variable in clause c Note p is some kind

ij ij

inv

of multiplexor function

P

k

c

Use the same arithmetization as in lemma except replace Av by p v A v

j ij j i j

i

AVerier using O query bits

restricted normal form verier Nowweprove Theorem ab out the existence of a n

for SAT In this section G denotes the eld GF and denote op erations in G We

will often write x y as xy where this causes no confusion Let denote a CNF formula

that is the veriers input and let n b e the number of variables and clauses in it

k

A function f G G is called linear if for some a a G it can b e describ ed as

k

P

k

a x Since each co ecient a can take only twovalues the set of n f x x

i i i k

i

variate linear functions is in onetoone corresp ondence with the set of p ossible assignments

to

P

n

a x is called a satisfying linear function for if Denition A linear function

i i

i

its co ecients a a constitute a satisfying assignmentto where we view G and

n

G as the b o olean values T F resp ectively

The verier exp ects the pro of to contain a satisfying linear function plus some other

information As in Section we assume that functions are represented by a table of their

n

n

values at all p oints in the eld A satisfying linear function is represented by jGj

values

In its basic outline this verier resembles the one in Lemma First it checks that

the provided function f isclose to the set of linear functions Then it writes down a

sequence of algebraic conditions whichcharacterize satisfying linear functions and checks

e

that f the linear function closest to f meets these conditions Instead of the Sumcheck

it uses Fact

The following lemma describ es the set of algebraic conditions

Lemma Let beaSAT instancewith n variables and n clauses and let X X

n

be variables taking values over GF Thereisapolyn time construction of n cubic

P

n

X X X i ng such that any linear function a x is a equations fp

i i i i i i

i

satisfying linear function for i

p a a a i n

i i i i

n

There is no problem with the pro of b eing of size Recall that the verier has random access to the

n

pro of string so it requires only O log O ntimetoaccessany bit in this pro of

A VERIFIER USING O QUERY BITS

Pro of Identical to that of Lemma

The following fact ab out a nonzero vector of bits forms the basis of the veriers prob

abilistic tests

Fact Let c c be elements of G that are not al l zeroThenforr r picked

k k

randomly and independently from G

X

Pr c r

i i

r r

k

i

P

Pro of Assume wlog that c After r r have b een picked the sum c r

k k i i

i

is still equally likely to b e or

We describ e an application of the ab ove fact We need the following denition

Denition Let b c b e the following linear functions in k and l variables resp ectively

P P

k l

b x and c x Thetensor product of b and c denoted b c is the following linear

i i i i

i j

P P

k l

function in kl variables b c z

i j ij

i j

es a pro cedure for testing given linear functions The following lemma implicitly giv

b c d whether or not d is the tensor pro duct of b and c The probabilistic test app earing

in the statement of the lemma requires one value each of all three functions

LemmaTesting for Tensor Pro duct Let b c d be the fol lowing linear functions

P P P P

k l k l

in k l and kl variables respectively b x c x and d z If d b c

i i i i ij ij

i j i j

then for u u and v v chosen randomly from G

k l

X X X

b u c v Pr d u v

i i j j ij i j

u v

i j ij

where the indices i j take values in k and l respectively

Pro of Consider the two k l matrices M N dened as M d andN b c

ij i j

ij ij

k

b

If d b c then M N Letavector u u u b e picked randomly from G

k

Fact implies that with probability at least

b b

uM uN

b b

where uM stands for the pro duct of vector u with the matrix M as dened normally

In normal mathematical usage d would b e called the linear function whose sequence of co ecients is

the tensor pro duct of the sequence of co ecients of b with the sequence of co ecients of c

CHAPTER APROOF OF THE PCP THEOREM

l

b

Now let vector v v v b e also picked randomly from G Fact implies that

l

with probability at least

T T

b b b b

u M v u N v

Hence the lemma is proved

Nowwe prove Theorem

Pro ofTheorem The verier exp ects the pro of to contain tables of three functions

n n n

f G G g G G and h G GIn a goodproof f is a satisfying linear

function and g f f h f g For ease of exp osition we describ e the verication as

consisting of a sequence of three steps However the verier can actually do all three in

parallel since no step uses the results of the previous step

Step The verier runs Pro cedure i to check f g h for closeness and rejects

outright if that pro cedure rejects anyoff g h

Pro cedure i reads only O values eachoff g h and rejects with high probability

if either of the three is not close Assume for arguments sake that the probability that

e e

e

it rejects is not high Then all three functions are close Let f g h b e the linear

functions closest to f g h resp ectively

e e

e

Step The verier runs a test that rejects with high probability if either g f f or

e e

e

h f g

The statement of Lemma implicitly shows howtodoStepFor instance to check

e e

e

g f f the verier can rep eat the following test O times Pick random u u

n

and v v and c heck that

n

e e

e

g u v f u u f v v

i j n n

where u v is used as shorthand for a vector of length n whose i j co ordinate for

i j

i j nis u v

i j

e

e

For this test the verier needs to reconstruct values of f at p oints and that of g at

p oint It uses Pro cedure ii to do this while reading the values of f g h at only O

p oints in the pro cess

Assume for arguments sake that Step do es not reject with high probability Then

P

e e e e e

e e

g f f and h f g In other words if the f a x then

i i

i

X

e

g y a a y and

ij i j ij

ij

X

e

hz a a a z

ij k i j k ij k ijk

A VERIFIER USING O QUERY BITS

where i j k n

e

Step The verier runs a test that rejects with high probabilityiff is not a satisfying linear

function that is its co ecients a a do not satisfy condition in Lemma

n

n

The verier picks a random vector r r G and checks that

n

n

X

r p a a a

i i i i i

i

i fng is not a a Supp ose a a are such that the nbit vector p a

i i n i i

the zero vector Then Fact implies that the test rejects with probability

P

n

But how can the verier compute the sum r p a a a Note that the sum

i i i i i

i

is a linear combination of the p s which since the p s are cubic can b e expressed as the

i i

P P P

sum of one value each of the functions a x a a y and a a a z In other

i i i j ij i j k ij k

i ij ijk

e e

e

words the verier only needs to reconstruct one value eachoff g h resp ectively This is

easy

This nishes the description of the verier

Complexity Steps and require reading only O bits from the pro of They require

n

O log jGj random bits in other words O n random bits

Correctness Supp ose form ula is satisable It is clear that Steps and never reject

a pro of containing a satisfying linear extension and its tensor pro ducts

Now supp ose is not satisable and so there is no satisfying linear function either

One of the following must b e true one of f g h is not close or the tensorpro duct

prop erty is violated or the condition in is violated In each case one of Steps or

rejects with probability at least

We still havetoshowhow the verier can check assignments split into many parts as

required by Denition Weshow this next

Checking Split Assignments

Wesketchhow the verier of Theorem can also check assignments split into many parts

Recall from Denition that in this setting the verier denes an enco ding metho d

and exp ects a pro of to b e of the form S S where is some information

k

that allows an ecientcheck that S S is a satisfying assignment concatenation

k

to k equalsized In this case we assume the SAT instance has nk variables split in

blo cks y y y y

n nk

nk

CHAPTER APROOF OF THE PCP THEOREM

n

The verier uses the enco ding that maps a string a a f g to the linear

n

P

n

function a x In other words the pro of is required to contain k functions f f

i i k

i

m

F FPart contains a set of tables In a goodproof f is a linear function representing

i

an assignment to the variables in the ith block such that the overal l assignment represented

by f f is a satisfying assignment Part contains f f and f f f where f is

k

the linear function denedbelow

f are close Supp ose the The verier uses Pro cedure to check that f

k

nk

pro cedure do es not reject Then the verier denes a function f G Gas

k

X

e

f x x f x x

i ni nk

ni

i

Clearly f is a linear function and its nk co ecients form a sequence that is the concatenation

f f

of the sequences formed by the co ecients of f and f Now the verier uses f in

k

Steps and exactly as b efore Clearlyiff is not a satisfying linear function the verier

rejects with high probability

From the denition of f it should b e clear that only O values eachoff f f

k

need to b e read

The Algebraic Pro cedures

Nowwe describ e in some detail the Pro cedures and Throughout this section

F denotes a eld A pro cedure maywork only for certain eld sizes in which case we will

sp ecify these sizes explicitly

SumCheck

We describ e Pro cedure the SumChec k The inputs to the pro cedure consist of a degree

d p olynomial B in l variables a set H F and a value c F The pro cedure has to verify

l

l

that the sum of the values of B on the subset H of F is c It will need in addition to the

table of values of B and the integers l and d an extra table We describ e rst what the

table must contain

For a F let us denote by B a y y the p olynomial obtained from B by xing

l

the rst variable to a

When wexallvariables of B but one we get a univariate p olynomial of degree at most

d in the unxed variable It follows that the sum

X

B a a yy y

i i i l

y y H

i l

THE ALGEBRAIC PROCEDURES

for i st i land a a F is a degreed univariate p olynomial in the variable

i

y We denote this sum by B y For i the notation B y do es not make

i a a i a

i

sense so we use the notation B y instead

Example The univariate p olynomial B y is represented by d co ecients When

we substitute y a in this p olynomial we get the value B a which by denition is the

sum of B on the following subcub e

a and x x H g fx x x

l l

l

Alternativelywe can view the value B a as the sum of B a y y onH

l

Thus B y is a representation of q jFj sums using d co ecients Supp ose f y

is another degreed univariate p olynomial dierentfromB y Then the two p olynomials

agree at no more than d p oints Hence for q d values of a the value f ais not the sum

l

of B a y y onH This observation is useful in designing the Sumchec k

l

Denition A table of partial sums is any table containing for every i i l and

every a a F a univariate p olynomial g y of degree dTheentire table is

i a a i

i

denoted by g

Nowwe describ e Pro cedure It exp ects the table T to b e a table of partial sums

In a goodproof the table contains the set of polynomials dened in

SumCheck

Inputs B F x x c F

d l

l

ToVerify Sum of B on H is c

Given Table of partial sums g

currentvalue c

Pickrandoma a F

l

For i to l do

P

if currentvalue g y

a a i

y H

i

i

output REJECT exit

else

currentvalue g a

a a i

i

If g a a a B a a

l l l

output REJECT

else

output ACCEPT

Complexity The pro cedure needs l log q random bits to generate elements a a ran

l

domly from F It needs the value of B at one p oint namelya a In total it

l

CHAPTER APROOF OF THE PCP THEOREM

reads l entries from the table of partial sums where eachentry is a string of size at most

d logq It p erforms O ldh eld op erations where h jH j Therefore the running

time is p oly ldh log q

l

Correctness Supp ose B sums to c on H The pro cedure clearly accepts with probability

the table of partial sums containing the univariate p olynomials B B y etc dened

a

in

Supp ose B do es not sum to c The next lemma shows that the pro cedure rejects with

high probability

l

Lemma B F x x c F if B does not sum to c on H then

d l

dl

Prthe Sumcheck outputs RE J E C T

q

regard less of what the table of partial sums contains

Pro of The pro of is by induction on the numberofvariables l Such an induction works

b ecause the Sumcheckisessentially a recursive pro cedure it randomly reduces the problem

of checking the sum of a p olynomial in l variables to checking the sum of a p olynomial in

l variables

To see this view the table of partial sums as a tree of branching factor q see Fig

ure The p olynomial g y is stored at the ro ot of the tree and the set of p olynomials

fg y a Fg are stored on the children of the ro ot and so on

a

The rst step in the Sumcheckveries that that the sum of the values taken by g on

l

the set H is c Supp ose the given multivariate p olynomial B do es not sum to c on H

Then the sum of the values taken by B on H is not c and the rst step can succeed only

if g B But if g B then as observed in Example g a B a for q d vals of

a in F That is to sayforq d values of a the value g ais not the sum of B a y y

l

l

on H Since d q it suces to picks a value for y randomly out of F say a and check

l

recursively that B a y y sums to g a onH Note While checking the sum

l

l

a y y on H the recursivecallmust use as the table of partial sums the of B

l

sequence of p olynomials stored in the a th subtree of the ro ot This is exactly what the

remaining steps of the Sumcheck do In this sense the Sumcheck is a recursive pro cedure

Nowwe do the inductive pro of

Base case l Thisiseasy since B y is a univariate p olynomial and B B

The table contains only one p olynomial g Ifg B then g do esnt sum to c either and

is rejected with probabilityIfg B then the two disagree in at least q d p oints

Therefore Pr g a B a dq

a

Inductive Step Supp ose the assumption is true for all p olynomials in l variables

Now there are two cases

THE ALGEBRAIC PROCEDURES

g ε (y 1 )

g (y ) a 1 2

g (y ) a1a2 3

Branching Factor of Tree = | F |

g (y ) ... a l

a 1 l −1

Figure A table of partial sums may b e conceptualized as a tree of branching factor q

The Sumcheckfollows a random path down this tree

Case i g B In this case

X X

g y B y c

y H y H

so the pro cedure will REJECT rightaway ie with probability So the inductive step is

complete

Case ii g B In this case as observed in Example for q d vals of a

g a B a

l

By the inductive Let a Fbesuch that g a is not the sum of B a y y onH

l

assumption no table of partial sums can convince the Sumcheck with probabilitymore

l

than dl q that g a is the sum of B a y y onH In particular the table

l

of partial sums stored in the subtree ro oted at the a th child of the ro ot cannot make the

Sumcheck accept with probability more than dl q Since this is true for q d values

q d

dl q dl q of a the overall probability of rejection is at least

q

In either case the inductive step is complete

Pro cedures for the Linear Function Co de

In this section F denotes the eld GF Our rst pro cedure checks whether or not a given

m

function f F Fis close to a linear function It uses the following obvious prop erty

CHAPTER APROOF OF THE PCP THEOREM

m

of linear functions A function h F F is linear i for every pair of p oints y y

m

m

and z z inF it satises

m

hy z y z hy y hz z

m m m m

The only if part of the statement is easythe if part follo ws from Fact A in the

app endix

The pro cedure uses a stronger version of the ab ove statement if h satises the prop erty

in for most pairs of mtuples then h is close for some small

Test for closeness Pro cedure i

m

Given f F F where F GF

rep eat times

m

Pick p oints y z randomly from F

if f y f z f y z

Note on the left is addition mod and

that on the right is componentwise addition mod

exit and REJECT

exit and ACCEPT

Complexity The test requires m random bits and reads values of f

Correctness Note that if f F x x then the test accepts with probability

m

According to the contrap ositive to the next lemma if f is not close then the basic step

in the test fails with probability at least Hence after rep eating the basic step times

the test rejects with probability close to e

m

Theorem BLR Let F GF and f be a function from F to F such that

m

when we pick y z randomly from F

Prf y f z f y z

where Then f is close to some linear function

Pro of The pro of consists in three claims

m

or every p oint b F there is a value g b f g such that Claim F

Pr f w b f w g b

m

w F

THE ALGEBRAIC PROCEDURES

m

Proof Let b F Denote by p the probabilityPr f w b f w where w is picked

w

m

uniformly at random in F Dene random variables v v taking values in F as follows

m m

Pickpoints y z F randomly and indep endently from F and let v f y b f y

and v f z b f z Clearly v v are indep endent random variables that takevalue

p The probability of the eventv v with probability p and with probability

is exactly p p Weshow that actually this event happ ens with probability at least

whence it follows that either p or p Ifp setting g bto

fullls the requirements of the lemmain the other case setting g b to do es

Note that and are the same over GF so

v v f y b f y f z b f z

f z y b f y b f z y b f z b f y f z

Further y b and z b are indep endently chosen random p oints Hence the probability that

each of the following twoevents happ ens is at least f z y b f y b f z

and f z y b f z b f y So the probability that they b oth happ en is at least

that is

Prf z y b f y b f z y b f z b f y f z

Thus Prv v which nishes the pro of of Claim

Claim The function g constructed in Claim agrees with f in at least fraction of

m

b in F

m

Proof Let b e the fraction of p oints b F such that f b g b

m

Pick y z randomly from F and denote by A the eventf y z g y z and by B

the eventf y f y f y z Note that A and B need not b e indep endent However

the hyp othesis of the theorem implies that PrB Further our assumption was that

PrA Now note that

PrB PrB APrB A

PrA PrB j A

where the last line uses the following implication of Claim

Prf y f z f y z jf y z g y z

But as we observed PrB Hence This nishes the pro of of Claim

Claim Function g is linear that is

m

a b F g a b g b g a

CHAPTER APROOF OF THE PCP THEOREM

m

Proof Fix arbitrary p oints a b F To prove g a b g ag b it suces to prove the

m

existence of p oints y z F such that each of the following is true i f b a y z f y

z g a b ii f b a y z f a y z g b and iii f a y z f y z f a

m

For if i ii and iii are true for any y z F then

g b a f b a y z f y z

f b a y z f a y z f a y z f y z

g b g a

Weprove the existence of the desired y z in a probabilistic fashion Cho ose y z inde

m

p endently at random from F The probability that any of i ii and iii is true is by

and so the probability that all three are true is at least Claim at least

Since the probability is strictly more than that we obtain a pair y z satisfying all

the conditions of the claim It follows that the desired pair y z exists This proves Claim

Finally note that Claims and imply together with the fact in Equation that

f is close

Nowwe describ e the other pro cedure connected with the linear function co de

e

Pro ducing a value of f Pro cedure ii

m

Given f F F that is closeF GF

m

Point b F

m

Pick random p oint y in F

output f y b f y

Complexity The pro cedure uses m random bits and reads values of f

e e

Correctness If f is a linear function then f f and Pr f y b f y f b

y

Now supp ose f is just close to some linear function The following lemma shows that

the pro cedure works correctly

e

Lemma Pr f y b f y f b

y

m

Pro of Both y and y b are uniformly distributed in F although they are not indep en

dent hence

e e

Prf y b f y b Prf y f y and

THE ALGEBRAIC PROCEDURES

e e e e

Since f b f b y f y we conclude that Prf b y f y f b

Pro cedures for General Polynomial Co de

Both pro cedures are randomizedthey use randomness only to pic k O p oints uniformly at

m

random from F This requires O m log jFj random bits Each pro cedure also requires that

some table b e provided in addition to f The entries in the table havesizepolyd m log q

and the pro cedure reads O entries from the table

m

Pro cedure i the Lowdegree Test veries that a given function f F Fis

close to a degreed p olynomial It uses the notion of a line

m

Denition A line in F is a set q p oints with a parametric representation of the form

m

fu t u t Fg for someu u F Thesymb ol denotes comp onentwise addition

of twovectors of length m Note that it is identical to the line fu t cu t Fg for any

c F nfg Our convention is to x one of the representations as canonical

m

Denition Let l b e a line in F whose canonical representation is fu t u t Fg

and g F F b e a function of one variable tThevalue producedbyg at the point u a u

of l is g a

Note that if a function f is in F x x then the values of f on anylineare

d m

describ ed by a univariate degreed p olynomial in parameter tWe illustrate this fact with

an example

Example Let f F x x b e a bivariate p olynomial of degree dened as f x x

Fas x x x Consider the line fa a t b b t Fg Dene a function h F

hta b ta b t a b t

It is a univariate p olynomial of degree and describ es f at every p oint on the line For

instance the value pro duced by h at the p ointa a is h a a a f a a

m

The Lowdegree Test picks lines uniformly at random from all lines in F and checks

howwell the restriction of f is describ ed by a univariate p olynomial For purp oses of

eciency it requires that a table T b e provided along with f supp osedly containing for

each line the b est univariate degree d p olynomial describing f on that line It p erforms many

wing trial Pick a line uniformly at random and a p oint uniformly at rep etitions of the follo

random on this line Read the univariate p olynomial provided for the line in the table and

check whether it describ es f at the p oint

CHAPTER APROOF OF THE PCP THEOREM

Lowdegree Test Pro cedure i

m

Inputs f F F

ToVerify f is close to F x x

d m

Given A table T containing for each line l a univariate

degree d p olynomial P

l

Pick k random lines l l and a

k

random p ointoneach of these lines z l z l

R k R k

Read P P and f z fz

l l k

k

If for i k P correctly describ es f at z

l i

i

ACCEPT

else

REJECT

Complexity Generating a random line requires picking the line parametersu u randomly

m

from F This requires only m log q random bits Also the test reads only values of

f and the same numb er of line p olynomials

Correctness Clearlyif f is a degreed p olynomial then by making the table T contain the

m

univariate p olynomials that describ e the lines in F the test can b e made to accept with

probability The next theorem shows that if f is not close then the test rejects with

high probability irresp ective of the contents of the table

Theorem Let the eld size q be d mIff is not close for then the

lowdegreetestaccepts with probability

We defer the pro of of Theorem until Chapter

m

Next we describ e Pro cedure ii This pro cedure given a close function f F

is a suciently small constant and a sequence of l points z z recovers the F where

s

e e

values f z f z The pro cedure sometimes pro duces erroneous output with some

s

small probability The pro cedure uses the notion of a curve whose denition generalizes

that of a line

m

Denition A degreek curve in F is a set of q points with a parametric represen

tation of the form

n o

k

u t u t u t F

k

m

whereu u u F the symb ol denotes comp onentwise addition of twovectors of

k

length m Note that it is identical to the curve

n o

k k

u t cu t c u t F

k

THE ALGEBRAIC PROCEDURES

for any c F nfg Our convention is to x one of the representations as canonical

m

Denition Let C b e a degreek curveinF whose canonical representation is

n o

k

u t u t u t F andg F F b e a function of one variable t The

k

k

value producedbyg at the point u a u a u of C is g a

k

Note that if a function f is in F x x then the values of f on any degreek curve

d m

are describ ed by a univariate degreedk p olynomial in parameter t

In describing the pro cedure we assume that elements of eld F are ordered in some

canonical fashion Thus we can talk ab out the ith p oint of a curve for any p ositiveinteger

i less than q Furthermore when we refer to the set of p oints on a curve weare

actually referring to a multiset since a curve could pass through the same p oint more than

once Having claried this we observe next that a degree k curve is xed once weknowits

rst k points

m

Fact For any set of k points z z F where therecould berepetitions

k

among z z there is a unique degree k curve whose rst k points are z z

k k

n o

m

k

Pro of To sp ecify a degreek curve u t u t u t F in F we need to

k

sp ecify its co ecientsu u whicharemtuples over F Clearly an equivalent sp eci

k

cation consists of m univariate p olynomials g g in the curve parameter t Each g

m i

m

has degree at most k and the tth p ointonC is g tg t F

m

Fact A App endix A implies that once we x the rst k p oints of C the p olynomials

g g are uniquely determined Hence the curve is also uniquely determined

m

m

Denition For any p oints z z in F let P z z denote the set of

k k

degreek curves whose rst k points are z z

k

e

Let z z b e the p oints at which the pro cedure has to reconstruct values of f Since

s

m

s points x a degreec curve the numb er of curves in P z z is q The

s

pro cedure requires a table containing for each such curve a degreesd p olynomial that b est

describ es f on the curve It picks a curve uniformly at random from the set and reads the

p olynomial corresp onding to it in the table It checks that the p olynomial agrees with f

on a random p oint on the curve As we will show this provides go o d condence that the

e

p olynomial correctly describ es f and therefore the pro cedure outputs the values taken by

this p olynomial at z z

s

CHAPTER APROOF OF THE PCP THEOREM

e

Extracting s values of f Pro cedure ii

m

Input A function f F F that is close to

m

F x x and c points z z F

d m s

e e e

Aim To obtain f z f z f z

s

Given A table T containing for each degreec curve C

passing through z z a univariate p olynomial T in

s

C

the parameter t of degree ds

Pick a degrees curve C randomly from P z z

s

Pickapoint y randomly from the set of p oints in C

Lo ok up the p olynomial T t and the value f y

C

If T correctly describ es f at y

C

then

output the values pro duced by T tatz z

s

C

e e

as f z f z

s

else

REJECT

Note that if f F x x and the table contains curve p olynomials that de

d m

e

scrib e f correctly the pro cedure will never output REJECT and correctly output f z

e e

f z f z

s

Lemma If f is close then

p

e e

Prthe procedure outputs values that arenotf z f z o

s

no matter what the table of curve polynomials contains

e

Pro of For a curve C let us denote by f j t the univariate p olynomial in t of degree sd

C

e

that describ es f on p oints in C The only case in which our pro cedure can output incorrect

e

values of f is when it picks a curve C such that the p olynomial T t provided in the table

C

e

is dierent from f j t We will show that on most curves such an incorrect T t do es not

C C

describ e f on most p oints of the curve Hence the pro cedure outputs REJECT with high

probability when it compares the values of T t and f on random p oint y of the curve

C

First we state a claim whichwe will prove later

p

For at least fraction of curves in P z z the following Claim Let

s

is true Let C denote the curve Then

e

f j t describ es f at fraction of p oints in C

C

Let C b e a curve that satises the condition in Recall that two dierent degreeds

e

univariate p olynomials agree at no more than ds points in FHenceiff j describ es f on C

THE OVERALL PICTURE

fraction of p oints in the curve then every other univariate p olynomial of degree d describ es

e

f in no more than dsq fraction of p oints In particular supp ose T f j Then

C C

p

T describ es f at no more than cdq dsq fraction of p oints in C

C

e

Hence an upp erb ound on the probability of outputting incorrect values of f is

p

ds

Pr C do es not satisfy condition

C

q

p p

ds

which is at most o

q

Hence the lemma is proved Nowwe prove the claim

m

e

Pro of of the claim Let S F bethesetofpoints where functions f and f disagree

m

Then S constitutes a fraction at most of F Lemma A implies that when we picka

curve randomly from P z z the exp ected fraction of p oints on the curve that

k

jS j

are in S is which is at most The Averaging Principle Fact A implies that the

m

F

j j

fraction of curves on which

p

fraction of p oints of the curvearein S more than

p

is no more than

This proves the claim and hence Lemma

The Overall Picture

Our pro of of the PCP Theorem consists in dening twoveriers namely the ones in

Theorems and and comp osing them using the Comp osition Lemma to construct

a log n restricted verier

While describing eachverier we describ ed how to enco de a satisfying assignment such

that the verier accepts the enco ding with probability When we comp ose twoveriers

their asso ciated enco ding schemes get comp osed as well where comp osition has to b e

dened appropriately using the construction in the pro of of the Comp osition Lemma

In other words our pro of of the PCP Theorem basically consists of a complicated

denition of an enco ding scheme itself dened as a comp osition of other schemes Figure

gives a birdseye view of dierent steps in the enco ding scheme Each step corresp onds to

a dierentverier Next to eachverier wehave written down the parameters asso ciated

with it the numb er of random bits the numb er of queries and the decision time

HistoryAttributions

The techniques in this section are inspired by the phenomenon of randomselfreducibility

n

A function f on domain say f g is said to exhibit this phenomenon if the computation

CHAPTER APROOF OF THE PCP THEOREM

of f on a worstcase input x can b e reduced in p olynomial time to the computation of f

on a small numb er of inputs that are uniformly not necessarily indep endently distributed

n

in f g

The phenomenon in some sense underlies cryptography For example the pseudo

random generator of BM uses the fact that the discrete log problem is random self

reducible The term randomselfreducible app ears to have b een explicitly dened rst in

AFK see also FKN Manyavors of randomselfreducibil ityhave since b een de

ned and studied FFFei

Blum BK and Lipton Lip rephrased the rsr prop erty as follows Given

any program that computes f on most inputs a randomized algorithm can recover the

value of f at an arbitrary input x This observation was the starting p oint for the theory

of selftestingselfcorrecting programs BK Lip BLR Most functions to which

this theory seemed to apply were algebraic in nature see Yao for some nonalgebraic

examples though

A new insightwas made in BF Sha LFKN it is p ossible to represent logical

formulae using low degree p olynomials But recall that dierent classes of logical formulae

are complete for complexity classes like PSPACE NEXPTIME NP etc This at once

suggests that randomized techniques from programtestingcorrecting should b e applicable

to the study of conventional complexit y classes The results in this chapter are precisely of

this character

Techniques of this chapter attributions The rst algebraic representation of SAT

is due to BFL That representation using multilinear p olynomials has a problem the

eldsize and numberofvariables required to represent a string of n bits are such that

m

generating a random p oint in the space F requires more than O log n random bits The

p olynomial extension used in this chapter do es not suer from this problem It is due to

BFLS The Sumcheck pro cedure Pro cedure is due to LFKN

Thus Lemma could b e proved with minor mo dications of the ab ovementioned

results although no previous pap er had proved it explicitly b efore AS All other results

in this chapter except the Lowdegree Test and the idea of checking split assignments are

from ALM The idea of checking split assignments is from AS The history of the

Lowdegree Test will b e covered in the next section

The discovery of the pro of of Lemma was inuenced by the paral lelization pro cedures

of LS FL although the ideas used in this lemma can b e traced back to BF

LFKN The design of the verier of Section owes much to existing examples of

selftestingcorrecting programs from BLR Fre

HISTORYATTRIBUTIONS

Polynomial Extension + Some Tables Satisfying Assignment

(Bunching up a a . . . a 1 2 n queries )

( log n, polylog n, log n )

( log n, 1, 1 ) (View as ( log n, 1, log n ) single . . . table) ......

( log n, 1, loglog n )

(Compose (Compose with verifier . .

with itself) O(1) bit verifier )

Figure How to enco de an assignment so that the PCPlog n verier accepts with

probability

CHAPTER APROOF OF THE PCP THEOREM

Chapter

The Lowdegree Test

This chapter contains a pro of of the correctness of the Lowdegree Test sp ecically a pro of

of Theorem

Let d b e an arbitrary integer that is xed for the rest of the chapter Let F b e the

nite eld GFq where q might dep end on dWecontinue to use F x x the co de

d m

of mvariate p olynomials of degree d whichwas dened in Section We will also use

m

prop erties of lines in F whichwere dened in Section Denitions and

We remind the reader of the following observation from Section sp ecically Ex

ample If a function f is in F x x then the values of f on any line are describ ed

d m

by a univariate degreed p olynomial in the line parameter tTheLowdegree Test is based

on a strong contrap ositive of that observation If on most lines most values of f are

describ ed by a univariate p olynomial of degree d then f itself is close to F x x for

d m

some small To state this contrap ositive more precisely we need to dene some concepts

f

m

Denition Let f F F b e a function and l b e a line The symbol P denotes the

l

univariate degree d p olynomial in the line parameter t that describ es f on more p oints of l

than any other degree d p olynomial We arbitrarily break ties among dierent p olynomials

that describ e f equally well on the line

f

l instead of Note i To make the degree d explicit wecouldhave used the symbol P

d

f

P But the degree d can always b e inferred from the context ii Supp ose line l is such

l

f

that p olynomial P describ es f on more than dq fraction of p oints in l Then no

l

other univariate p olynomial can do b etter For if any other p olynomial do es as well as

f

P then there is a set of at least d p oints on which they b oth describ e f But if two

l

degree d p olynomials agree on d points they are the same In other words for sucha

f

line l the b est line p olynomial P is uniquelydened

l

Nowweintro duce some notation Let S be any nite set and let b e a realvalued

function dened on S The average value of on S is denoted byE x We justify

xS

CHAPTER THE LOWDEGREE TEST

this notation on the grounds that if we pick x uniformly at random from S the exp ectation

of x is exactly the average value of on S When S is clear from context we drop S

and use E x

x

m

Denition Let f F F b e a function and l b e a line The success rate of f on l

f

denoted l is dened as

f

f

l fraction of p oints on l where P describ es f

l

The success rate of f is dened as the average success rate of f among all lines that is as

m

f

E l where L is the set of lines in F

lL

The following theorem is the precise statemen t of the contrap ositivementioned ab ove It is

proved in Section A crucial comp onent in its pro of is Lemma

m

Theorem Let d m beintegers satisfying d mq Every function f F F

whose success rate is for isclose to F x x

d m

The pro of can b e tightened so that it works for larger Whether it works for

say is op en

Nowwe provethattheLowdegree Test works as claimed

m

Pro of Of Theorem Letf F F b e a function and T b e a table of line

p olynomials provided to the Lowdegree Test Let p denote the success probability of the

T

following trial Pick a random line l and a random p oin t y l Call the trial a success if

the p olynomial provided for l in table T correctly describ es f at x

By denition the success rate of f is the maximum over all tables T ofp Hence if f

T

is not close where then Theorem implies that p forevery table

T

T

Recall that the Lowdegree Test p erforms rep etitions of the ab ove trial and accepts

i all succeed Now supp ose f is not close After rep eating the trial times the test

will reject with probability at least Thus Theorem is proved

The pro of of Theorem relies on the following Lemma which asserts the truth of

the theorem statement when the number of variables mis The lemma is proved in

Section

Lemma Let integer d be such q d Every g F F whose success rate is

p

for is close

THE BIVARIATE CASE

Convention Atypical lemma statement in this chapter says If the hyp othesis H holds

foranumber where is small enough then the conclusion C holds for the number c

p

where c is explictly dened in terms of In Lemma for example c is When

applying the lemma later we will need to make c smaller than some suitably small constant

whichwe can do by making some other suitably small constant Having claried this

from nowonwe will never state explicit values of such constants

The Bivariate Case

This section contains a pro of of Lemma

m

For an element a F let the the row fy bg b e the line of p oints in F whose second

co ordinate is b ie the set fx b x Fg The column fx ag for a F is dened

f

similarlyWe often use the shorthand row b for row fy bg As usual P is the

fy bg

the degreed p olynomial in x that b est describ es f in the row fy bgFor claritywe

denote it by R Likewise C denotes a degreed p olynomial in y that b est describ es f

b a

on the column fx ag The following lemma says that if on most p oints in F the row and

column p olynomial describ e f correctly then f is close for some small At the end of

this section we derive Lemma easily from this lemma

Lemma Let f F Fbe a function Suppose at more than fraction of points

a b F we have

f a b R a C b

b a

p

close to a polynomial of bidegree d d where is smal l enough Then f is

Let a function b e called a rational polynomial if it can b e expressed as hg where h and

g are p olynomials

We divide our goal of proving Lemma into two subgoals

Subgoal Show that thereare bivariate polynomials h g with fairly low degree such that

on most points in F function f agrees with the rational polynomial hg

Subgoal Show that the rational polynomial hx y g x y obtainedfrom Subgoal is a

bivariate polynomial of degree d

Why is Subgoal realistic Using the hyp othesis of Lemma and an averaging

argument we can show that f s restriction to most rows and columns is close for some

small Further the statement of Subgoal says that f and hg agree on most p oints

eraging shows that the restriction of hg on most rows and columns is Another use of av

close for some A simple argument using Euclids division algorithm for p olynomials

CHAPTER THE LOWDEGREE TEST

will then show that hg is a bivariate p olynomial of degree dFor details see the pro of of

Lemma

Why is Subgoal realistic We give a motivating example from the univariate case

Example BW Let k be an integer much bigger than d Let a b a b

a b b e a sequence of pairs of the typ e p oint value where the p oints a are in F

k k i

and so are the values b Further supp ose there is some univariate p olynomial r of degree

i

d that describ es th of the pairs that is

k

values of i f kg r a b for

i i

k k

We showhow to construct univariate p olynomials c e of degree at most d and

resp ectivelysuch that

c a b ea i fkg

i i i

In other words we construct a rational p olynomial cxex that describ es the entire

sequence of pairs

Let e b e a nonzero univariate p olynomial of degree k that is on the set fa r a b g

i i i

Fact A implies that such a p olynomial e exists Then wehave

ea r a b ea i fkg

i i i i

By dening c as cx exr x were done

Let us consider the relevance of Example to Subgoal Recall that the hyp othesis of

Lemma says that at a typical p ointinF therow p olynomial the column p olynomial

and f all agree Pick k typical columns say a a Pickatypical row fy cg

k

Denote by b b b the values taken by the column p olynomials C y C y

k a a

k

resp ectively in this row that is b is C c Since wechose typical columns and row the

i a

i

row p olynomial R should describ e most of these values correctly For arguments sake

c

supp ose it describ es th of the values that is

k

values of i f kg R a b for

c i i

Then the construction of Example applies to the sequence of pairs a b a b

a b and we can construct a lowdegree rational p olynomial cxex that describ es all

k k

of b b

k

But there is an added twist The values b b are not any arbitrary elements of

k

F Each is itself the value of the corresp onding column p olynomial which is a degree d

p olynomial in a separate parameter y Hence a more prop er way to view the b s in this case

i

is as elements of Fy the domain of p olynomials in the formal variable y This viewp oint

along with the fact that a go o d fraction of rows are typical allows us to nd a go o d

description of f in terms of a rational bivariate p olynomial

Nowwe state the results precisely First wemake explicit the algebraic ob ject that

underlies Example an overdetermined system of linear equations

THE BIVARIATE CASE

Lemma Let fa b a b g beasequenceof d point value pairs for

d d

which there is a univariate degreed polynomial r such that

r a b for d values of i f dg

i i

Then the fol lowing system of equations over F in the variables c c e e

d d

has a nontrivial solution

d d

c c a c a b e e a e a

d d

d d

c c a c a b e e a e a

d d

d d

e a c c a c a b e e a

d d d d d

d d

Pro of As in Example construct p olynomials c e of degree d and d resp ectively

such that

ca ea b i f dg

i i i

Wlog we assume that at least one of the p olynomials e c is not the zero p olynomial

P P

d d

i j

Nowlet e and c b e expressed as ex e x andcx e x resp ectively

i j

i j

Then c c e e is a nontrivial solution to the given linear system

d d

verdetermined it consists Note The linear system in the statement of Lemma is o

of d equations in d variables Represent the system in standard form as A x

where x is the vector of variables c c c e e The system has a non

d d

trivial solution Hence Fact Ai of the App endix implies that the determinantofevery

d d submatrix of A is

Denition If l and m are integers a bivariate p olynomial g in the variables x and y

has bidegree l m if its degree in x and y is l and m resp ectively

The following Lemma achieves Subgoal

Lemma Let f be the function of Lemma Then therearepolynomials g x y

hx y of bidegree d d d d respectively such that hx y g x y describes f

p

in fraction of the points where

Pro of The hyp othesis of Lemma implies that in the average column the column

p olynomial describ es f and the row p olynomials in a fraction of the the p oints

Since d ojFj an averaging argument shows the following There are d columns

a and a suchthatfori d the column p olynomial C y describ es f at

d a

i

more than o fraction of the p oints in the column a i

CHAPTER THE LOWDEGREE TEST

For counting purp oses weputa at every p oint in these columns at which the row and

column p olynomials agree Let y be a row that has at least d s Then the sequence

of pairs a C y a C y satisfy the hyp othesis of Lemma Hence the

a d a

d

following system of equations has a solution

d d

c c a c a C y e e a e a

d a d

d d

c c a c a C y e e a e a

d a d

d d

c c a c a C y e e a e a

d d a d d

d d

d

Represent the system in standard form as A x where x is the vector of variables

c c c e e and A is the d d matrix

d d

d d

a a a C y a C y a C y

a a a

B C

d d

a a a C y a C y a C y

B C

a a a

B C

B C

A

d d

a a a C y a C y a C y

d a d a a

d d d d d d

We taketwo dierent views of this system In the rst view it is a family of q separate

has a solution systems one for eachvalue of y in F The system corresp onding to y b

i the number of s in the rowisd Averaging implies there are q such

values of y Further up on substituting anysuchvalue of y in A the determinantofevery

d d submatrix of A b ecomes See the note following Lemma

In the second view A is the matrix of a system of equations over the domain Fy

the domain of p olynomials in the formal variable y We show that we can nd a non

trivial solution to c c e e in this domain More sp ecicallywe show that

d d

the determinantofevery d d submatrix of A is the zero p olynomial of

Fy and then use Fact A part

Let B be anyd d submatrix of A Fact A implies that detB

By the determinantofB is a p olynomial of degree at most d in the entries of B

insp ection eachentry of B is a p olynomial in y of degree at most d Therefore detB

is a p olynomial in y of degree at most d d Either detB is the zero p olynomial

or it is has at most d d ro ots But as already noted the determinantofevery

d d submatrix of A b ecomes for at least q values of y Since

d dwe conclude detB is the zero p olynomial q

Wehave shown that the system has a nontrivial solution in the domain Fy Fur

ther Fact A part implies that a nontrivial solution can b e found in which eachof

c c e e is itself a p olynomial of degree d in the entries of A In other

d d

words each e and c is a p olynomial in y of degree d d Let c c e e

i i d d

THE BIVARIATE CASE

F y besuch a solution Dene the bivariate p olynomials h g as

d

d d

X X

i i

hx y c y x and g x y e y x

i i

i i

Clearly h g have bidegree d d and d d resp ectively

It only remains to show the following

p

Claim The rational p olynomial hg describ es f in fraction of p oints in F

To this end we prove Claim

p

Claim A o fraction of rows satisfy all the following prop erties i The

restriction of g to this row is not the zero p olynomial in Fx ii The rowcontains at least

p

d s iii The row p olynomial describ es f at a fraction of p oints of the row

First we show that Claim implies Claim

Let fy bg be a row that satises conditions i and ii in the statement of Claim

Then the univariate p olynomial hx b R xg x b has degree d but more than d

b

ro ots Hence the p olynomial is identically zero in other words hx bg bx R x

b

p

Now if the row also satises iii then hg describ es f on at least fraction of p oints

in this row

p p

Thus Claim implies that hg describ es f on at least o

p

Hence Claim follows fraction of p oints in F This fraction is at least

Nowwe prove Claim

p

Averaging shows that at most fraction of the rows fail to satisfy iii

The p olynomial g x y is nonzero and its degree in y is at most d Hence at most

d rows fail to satisfy i

Finally in each of columns a a a o fraction of p oints contain s

d

Averaging shows that the fraction of rows that fail ii is at most that is at most

p

Hence the fraction of rows that satisfy all of i ii and iii is at least d q

Since d q o Claim follows

Next wemovetowards Subgoal

Lemma If rst are univariate polynomials in x of degree l m n respectively and

r x sx tx for max fl m ng values of x

CHAPTER THE LOWDEGREE TEST

then r s t

Pro of The univariate p olynomial r s t has degree max fl m ng If it has more than

max fl m ng ro ots it is identically zero

Note Let rs b e univariate p olynomials of degree l m resp ectively Assume max fl mg is

q for some Lemma implies that the rational p olynomial rs can only exhibit

two b ehaviors Either it is a univariate p olynomial of degree l m or else it is not even

close to any univariate p olynomial of degree l m

The following lemma achieves Subgoal

Lemma If h g arepolynomials obtainedinLemma then hg is a bivariate poly

nomial of bidegree d d

Pro of The pro of consists in three claims

hg on most rows and columns is a univariate The rst claim is that the restriction of

p olynomial of degree d This uses two observations First the restriction of hg to most

rows and columns describ es f quite well and is therefore a close for some small Second

Lemma implies that if the restriction of hg on anyrow or column is close then the

restriction is a univariate p olynomial

Claim For at least the elements a F hx ag x aisclose and hence by

Lemma is in F x

d

p p

Indeed byaveraging in at least of the rows f is close Let b e as in

p p

Lemma Then in at least fraction of the rows hx y g x y describ es f in

p

p

fraction of p oints Hence in fraction of rows the restriction of hx y g x y

p p

p

p

close Since so Claim is proved is

Let k l stand for d d resp ectively so that the bidegrees of g h are l k and l d k

resp ectively

The second claim is that the degree of hg in x is d

Claim Assuming Claim there exists a p olynomial sy of degree dk such that hsg

is a p olynomial of bidegree d d

P

l

i

Assume wlog that h g have no common factor Represent g x y hx y as p y x

i

i

P

ld

i

and s y x resp ectively where p y p y and s y s y are univariate

i l ld

i

p olynomials of degree k Using Euclidean pseudodivision for p olynomials see Knu

p we obtain p olynomials q x y r x y of bidegree d k dl and l k dl

resp ectively such that

d

p y hx y q x y g x y r x y l

CORRECTNESS OF THE LOWDEGREE TEST

For every a Fsuch that g x ajhx aitmust b e that r x a is a zero p olynomial

By Claim at least the p oints a Fhave this prop erty Hence r a x is the zero

p olynomial for at least qvalues of a which is more than ld k the degree of r in y

d

Hence r is identically and p y hx y q x y g x y By using sy as a shorthand

l

d

for p y wegetg jh s and Claim is proved

l

hg has degree d in b oth x and y This proves the Lemma The next claim is that

Claim Assuming Claim hx y g x y has degree at most d in x y

P

d

i

Thus far we only know that hx y g x y has the form x t y where t y isa

i i

i

rational p olynomial whose degree in the numerator and denominator is at most d

But the restriction of hg to more than d columns is close by applying the

reasoning of Claim to columns instead of rows By Lemma these restrictions are

degree d p olynomials It follows byinterp olation that each t y is a degreed p olynomial

i

Similarlywe can argue that the degree in x is also dThus Claim is proved

Pro ofOf Lemma We show that every bivariate function with success rate ie

one satisfying the conditions of Lemma satises the conditions of Lemma with

o This is seen as follows

Break up all lines into equivalence classes with a class containing only lines parallel to

each other See Fact A for a denition of paral lelthe denition is the ob vious one An

averaging argumentshows that there are two distinct classes in which the average success

rate is at least o Now rotate co ordinates to make these two directions the

x and y directions By construction the b est p olynomials describing the rows describ e

f in a fraction o of p oints in F and so do the b est p olynomials describing

the columns Hence the fraction of p oints in F on which both the row and the column

p

close p olynomial describ e f is at least o Lemma implies now that f is

to a bivariate p olynomial of bidegree d d Wehavetoshow that it is close to a p olynomial

of total degree d But the fact that the successrate is implies that the restriction of f

to fraction of lines is at least close to a univariate p olynomial of degree d Hence

p

Fact A implies that the p olynomial that f is close to actually has total degree d

p p p

we conclude that f is close to a bivariate p olynomial of total Since

degree d Hence the lemma has b een proved

Correctness of the Lowdegree Test

This section is devoted to proving Theorem

m

Denition A planeinF is a set of q points that has a parametric representation

m

of the form fu t v t w t t Fg for someu v w F Note that it is identical

CHAPTER THE LOWDEGREE TEST

to the set fu t c v t c w t t Fg for any c c F nfg Our convention is to

x one representation as the canonical one

m

Given a bivariate p olynomial h in the parameters t t and a function f F F

wesay that h describes f at the point u a v a w of the ab ove plane i ha a

a v a w f u

Denition Let C b e a plane and L b e the set of lines in the plane For a function

C

m

f F F the success rate of f in C is

f

E l

lL

C

in other words the average success rate of f among lines in the plane

The general idea of the pro of is that if the overall success rate of f is high then symmetry

m

considerations imply that f has high success rate in almost every plane in F But the

restriction of f to a plane is a bivariate function so if the success rate in a plane is high

this bivariate function by the bivariate case of the theorem namely Lemma is close

m

for some small Hence we conclude that for almost every plane in F there is a bivariate

p olynomial in the plane parameters t t that describ es f almost everywhere in the plane

This implies some very strong conditions on f namely the statement of Lemma which

in turn imply that f itself is welldescrib ed byanmvariate p olynomial

Throughout our symmetrybased arguments use twoobvious facts i Every two p oints

m m

in F together determine a unique line that contains them ii Every line in F and every

p oint outside it together determine a unique plane that contains them

To state our calculations cleanly weintro duce some simplication in notation Wex

the letter f to denote the function from the statement of Theorem whose success rate

is for some small enough Thenwe can make statements like the line l is close

instead of the restriction of f to line l is close and the value pro duced byline l at

f

at the p oint b l and so p oint b l instead of value pro duced by line p olynomial P

l

on

Also wemake Lemma more userfriendly and use the following lo ose form

Lo ose form of Lemma If the success rate of f in a plane is at least for some

close to a bivariate polynomial smal l enough then the restriction of f to the plane is

Note We justify the lo ose form on the grounds that wearenotkeeping track of constants

anyway Thus the constant in the conclusion of the lo ose form is not fundamentally

p

dierent from the constant in the correct conclusion

Nowwe can cleanly state an interesting prop erty of planes with high success rate

m

Denition Let b be a pointinF Two lines that pass through b are said to b e

coherent at b if they pro duce the same value at bLetS b e a set of lines which all pass

CORRECTNESS OF THE LOWDEGREE TEST

through b The set is ccoherent at b for some number c if it is coherentatb and

in addition every line in S is cclose

We will show in Lemma that a high success rate implies that for every p oint most

lines passing through it are coherent As a rst step weshow this fact for the bivariate

case In the following lemma the reader may wish to skip reading part ii for now and

return to it later

Lemma Suppose plane C has success rate for and b is any point

in C Then the fol lowing holds

p p

coherent set of lines at b which contains fraction of al l lines in C i Thereisa

that pass through b

ii Let g be the bivariate polynomial that best describes the values of f in this plane and

let it produce the value g at point b Then the value producedat b by the coherent lines in

b

part i is also g

b

Pro of If the success rate of plane C is then Lemma shows that there is a

unique bivariate p olynomial g that describ es f on fraction of p oints in C nfbg Let

g denote the value pro duced by g at b

b

Let S b e the lines of C that pass through bWe show that the desired coherent set

S S is the one dened as

p

S l S g describ es f on at least fraction of p oints of l

First weshow the set is coherentatbLetl b e a line in S Since the restrictions of g

p p

fraction of p oints on l and dq itfollows that the and f agree in

restriction of g to the line is also the b est univariate p olynomial describing l In particular

f

pro duces the value g at b Since this is true for every the line p olynomial for the line P

b

l

line l S itfollows that the lines in S are coherentatbFurther the denition of S also

p p

implies that each line in S is close Hence S is coherentatbTo complete the

p

jS j pro of of part i we show that jS j

Every p ointinC nfbg is connected to b by a unique line Hence the lines in S partition

C nfbgFor line l S denoteby l the fraction of p oints on l where f and g disagree

p p

Then E l Averaging shows that l for at least fraction

lS

p

of lines l S Hence jS j jS j and part i is proved

Part ii follows from an observation in the rst para of the pro of namely that all lines

in S pro duce the value g at b

b

Note Why is Lemma signicant Recall the denition of success rate it is the exp ec

tation

f

describ es f E fraction of p oints in l where P

l

l

m

where l is picked randomly from lines in F Symmetry implies the the following exp ectation

is also the success rate

m

E among lines intersecting x the fraction of those that describ e f at x

xF

CHAPTER THE LOWDEGREE TEST

Hence high success rate implies that for most p oints x most lines passing through

x pro duce the same value at x namely f x In other words most p oints havean

asso ciated coherent set of lines and the set is large Lemma says that at least in the

bivariate case this is true for al l p oints x and not just for most x Lemma will show

that an analogous statement holds for the mvariate case as well

First we p oint out the following symmetrybased fact

m

Lemma Let p denote the success rate of f and let b be a xedpoint in F Let C be

the set of planes that contain bThenwehave

E success rate of f in C p o

C C

Pro of Recall the denition of success rate it is the exp ected success rate of a line picked

randomly from among all lines But note that only a o fraction of all lines pass through

b Hence the exp ectation is almost unchanged if we pic k the line randomly from among

all lines that do not pass through bLetq b e this new exp ectation As we just argued

q p o

Nowwe use symmetry to obtain an alternative denition of q Every line outside b

determines a unique plane containing itself and b All planes containing b contain the same

numb er of lines that dont contain bSoq is also the exp ected success rate of a line picked

as follows First pick a random plane C C and then randomly pick a line in C that do es

not contain b But the new exp ectation is exactly the one o ccuring in the lemma statement

so wehaveproved the lemma

m

Lemma Main Lemma Let f F F have success rate where is smal l

m

enough and let b be any point in F Thenthereisa coherent set at b that contains a

p

of al l lines passing through bwhere fraction at least

m

Pro of Let b F andL b e the set of lines passing through b Let L L b e maximal

coherent subsets of L satisfying i and ii Maximality implies that the L s are mutually

i

disjoint Denote by the fraction jL j jLj and by the largest of the s

i i i

Let the term pair refer to a pair of lines l l where l l are distinct lines in LWe

call the pair l l agreeable if they are b oth close and further their line p olynomials

pro duce the same value at b Notice that b oth the lines in an agreeable pair must come

from the same subset out of L L Hence the fraction of pairs that are agreeable is at

most

X X

jL j

i

i

jLj

i i

CORRECTNESS OF THE LOWDEGREE TEST

We will show that at least of all pairs are agreeable which implies thus

proving the Lemma

Tocount the fraction of agreeable pairs we use symmetry Since each pair denes a

unique plane containing b and since each plane containing b contains the same number of

pairs the total number of pairs is

Numb er of planes containing b Numb er of pairs p er plane

Lemma implies that the average success rate of planes passing through b is

o Averaging shows that at least fraction of them have success rate at

least LetC b e a plane that has success rate Lemma implies that of all

lines in C that pass through b at least a fraction are coherent It follows that the

p

which is at least fraction of pairs in C that are agreeable is at least

Wehaveshown that in fraction of planes containing b the fraction of pairs

in the plane that are agreeable is Hence the fraction of pairs overall that are

agreeable is at least which is at least Thus the Lemma has b een

proved

m

b

Denition Let a function f F F b e dened as

b

f x ma jority fvalue of P at xg

l

lx

Note Lemma implies that this majority is an overwhelming majority

b

The following lemma shows that f coincides with f almost everywhere

m

Lemma Let f F Fbe a function with succ ess rate where is a suciently

n o

m

b

smal l constant The set of points x F f x f x constitutes a fraction at most

of al l points

b

Pro of Let t denote the fraction of p oints x such that f x f x

As noted ab ove one way to view the success rate is as

f

describ es f at x PrP

l

xl

where x is picked randomly from among all p oints and l is picked randomly from the set of

lines containing x

m

b

Let x F be such that f x f x Lemma implies that function f is describ ed

p

Therefore an upp er at x by at most a fraction of the lines through xwhere

b ound on the the success rate is t t Since the success rate is at least we

havet t which implies t

CHAPTER THE LOWDEGREE TEST

b

f

denote the the univariate p olynomial that b est describ es values of For any line l let P

l

b

f on l

b

f

b

The following lemma shows that on every line l the p olynomial P describ es f on every

l

p ointofl

b

f

b

at b is f b Lemma For every line l and every point b l the value producedbyP

l

Pro of Let l be a line and b l b e a p oint

Let y be anypointonl Wesay a line l y is nicefor y if l is in y s coherent set

whose existence is proved in Lemma In other words l is close and pro duces the

b

value f y at y Let C b e a plane containing l Wesay C is goodfor y if among all lines in

C that pass through y afraction are nice for y

Claim There is a plane C containing l that is goodforbandgood also for a fraction

of points on l

First weshowhow to complete the pro of of the lemma using the Claim Let C b e the

plane mentioned in the claim Fact A part iii implies that line l intersects almost

all the lines in C more sp ecically a fraction O q whichis o Since plane

close is C is go o d for fraction of p oints in l the fraction of lines in C that are

o Hence the success rate of C is at least which

we assume is more than Lemma implies there exists a bivariate p olynomial h that

describ es f on fraction of p oints of the plane Let z l be a pointandh denote the

z

value that h pro duces at z Since the success rate in plane C is high Lemma partii

implies that z has an asso ciated set of coherent lines in C each of which pro duces the value

h at z Now supp ose z is one of the p oints for which C is go o d By denition of go o d

z

b

most lines that pass through z pro duce the value f at z But we know that most lines lie

b

in the ab ovementioned coherent set for z Hence the values f z and h must b e identical

z

b

In other words for fraction of p oints on l including bwehave f z h Since

z

b

f

b

f is describ ed by h on fraction of p oints of l the line p olynomial P must b e the

l

b

f

restriction of h to l In particular the value pro duced by P at b must b e h But as noted

b

l

b

h f b Hence the Lemma is proved

b

Next we prove the Claim We prove the existence of the desired plane probabilistically

Let C b e the set of planes that contain l Wepick C randomly from C Lety b e a p ointin

l We upp erb ound the probability that C is not go o d for y

Dene the random variable I to b e the fraction of lines among all lines of C that

C

contain y that are nice for y Recall that every line l y that is not l determines a unique

plane together with l Symmetry implies that every plane C containing l contains the same

numb er of lines l y Therefore wehave

E I

C

C C

DISCUSSION

p p

Assume that Hence if plane C is not go o d for y then I The

C

Averaging Principle implies that

p p

Pr C is not go o d for y Pr I

C

C C C

Hence the probability that C fails the conditions of the claim is at most

PrC is not go o d for b PrC is not nice for fraction of p oints of l

C C

p

p

p p

Assuming the probability that C fails the conditions of the claim

is less than Hence there exists a plane meeting all the conditions

Nowwe prove Theorem the main theorem of this chapter

b

Pro ofOf Theorem We dened a function f in Denition Lemma shows that

b

f

b

on every line the restriction of f is describ ed exactly by the degreed p olynomial P Using

l

b

Fact A in the app endix we conclude that f F x x But Lemma shows

d m

b

that f agrees with f in fraction of p oints It follows that f is close

Discussion

The results in this chapter comprise two parts One part given in Section shows that

the correctness of the Lowdegree Test follows from the correctness of the bivariate case of

the test in other words from Lemma This part is due to Rubinfeld and SudanRS

although our pro of is somewhat dierent in the way it uses symmetry arguments The other

part of the chapter Section shows the correctness of the bivariate case The pro of of

the bivariate case is a consequence of a more general multivariate result in AS which

is stated in the next section The pro of of the bivariate case as given here uses a little

mo dication due to Sud

No simpler pro of of the general result of AS is known But a somewhat simpler pro of

for the bivariate case app ears in PS More imp ortantly that pap er lowers the eld size

required in the lemma to O dan impro vementover O d as required by our pro of

Many p eople have conjectured the following although a pro of still eludes us

m

Conjecture Let jF j polydIff F F is a function whose success rate is p

p

for p d jF jthenf is pclose to F x x

d m

CHAPTER THE LOWDEGREE TEST

If the eld size jFj is allowed to b e exp onential in the degree the previous conjecture is

true Wedonotgive the pro of here

History

Improvements to the Lowdegree Test have accompanied most advances in the theory of

probabilistically checkable pro ofs The rst such test the multilinearity test enabled the

result MIP NEXPTIME BFL Subsequent tests given in BFLS FGL were

crucial for scaling down the result for NEXPTIME to NP An improvement in the e

ciency of those tests was in turn used to prove a new characterization of NP in terms of

PCP AS Finally the discovery of the most ecient test the one in this section led

to the discovery of the PCP Theorem ALM

Actually the ab ove list of tests includes two kinds of lowdegree tests The rst kind

upp erb ounds the the degree of the given function in each variable The second kind and

the one in this chapter is of this kind upp erb ounds the total degree

The tests in BFL BFLS FGL AS are of the rst kind A particu

larly b eautiful exp osition of this typ e of test She was never published although a

later version app ears in FSH Lowdegree tests of the rst kind consist in estimat

ing the success rate of the given function on axisparal lel lines A line is called axis

parallel if for some i fmg and a a a a F it is of the form

i i m

fa a x a a x Fg Note on the other hand that the test in this

i i i m i

chapter estimates the success rate on al l lines The strongest result ab out lowdegree tests

of the rst kind is due to AS

m

Theorem AS Let f F Fbe a function If the success rate of f on axis

paral lel lines is O m then f is close to a polynomial in x x whose degree

m

in each variable is at most d

Note that Lemma whichwas used in our pro of of the bivariate case is equivalent

e theorem to the sp ecial sub case m of the ab ov

Lowdegree tests of the second kind arose in program checking The earliest such test

app ears in BLR where it is called the multilinearity test Subsequent tests app earing

in GLR RS work for higher degree In fact these latter tests are identical to the

one used in this chapter but their analysis was not as go o d it could not showthatthetest

works when the success rate is less than O d

The idea of combining the work of AS and RS to obtain the lowdegree test of

this chapter is due to ALM This combination allows the test to work even when the

success rate is a constant indep endent of the degree To estimate that the success rate is

some high enough constant the test needs to only pick O random lines and therefore

reads only O chunks of information in the provided tables This prop erty plays an

imp ortant part in the pro of of the PCP Theorem Sp ecicall y suchalowdegree test enables

DISCUSSION

queries to b e aggregated see Section a prop erty crucial in constructing the verier

in Theorem

The algebraic ideas used in the pro of of Lemma were originally inspired by a Lemma

of BW as used in GS

CHAPTER THE LOWDEGREE TEST

Chapter

Hardness of Approximations

In Chapter weproved the following consequence of the PCP Theorem There is a constant

such that if MAXSAT can b e approximated within a factor of then P NP

Corollary Similar hardness results are now known for a host of other optimization

problems namely that if they can b e approximated within some explicit factor where

the exact factor in question dep ends up on the problem in p olynomial time then some

wellknown complexitytheoretic conjecture is false

The reductions used to prove this b o dy of results go via either Prover Round pro ofs

for NP a discussion on Prover pro ofs app ears in Chapter or the PCP theorem Often

the b est reduction for a given problem ie one that proves the strongest inapproximability

result for the problem uses techniques sp ecic to that problem and sometimes also details

of the construction of the PCP verier

Quite understandably this new way of doing NPcompleteness reductions is generally

p erceived as not very userfriendly Traditionally NPcompleteness reductions which

prove NPhardness only of exact optimization are far simpler each one is based in a

simple way up on one of a few canonical NPcomplete problems The numb er of canonical

problems in GJissix

In this chapter we attempt to identify problems that can serve as canonical problems

for proving inapproximability results The aim is to to derive all known inapproximability

results using as few assumptions in other words as few canonical problems as p ossible We

nd that two canonical problems suce The rst is a version of MAXSAT The second is

a new problem Lab el Cover dened expressly for the purp ose of proving inapproximability

Note that reductions from these two cannot for ev ery given problem prove the strongest

p ossible inapproximability results for that problem But the reductions always prove results

that are in the right ballpark for instance the factor of approximation they prove hard

is not much smaller than the b est that can b e proven otherwise In some cases reductions

from Lab el Cover prove approximation to b e only almostNPhard this term is dened in

Denition whereas direct reductions using the PCP verier would prove it NPhard

We could partially remedy this latter state of aairs by including a third less natural

CHAPTER HARDNESS OF APPROXIMATIONS

canonical problem in our list but wecho ose not do so The hop e is that further progress

in the construction of PCP likeveriers Conjecture says precisely what else needs to

b e proved will allow NPhardness to b e proved using Lab el Cover

We rst dene a few terms that app ear throughout this chapter

Denition Let b e an optimization problem involving maximization resp mini

mization and let OP T I denote the value resp cost of the optimum solution on input

I For a rational number c an algorithm is said to capproximate if for every input

I the algorithm pro duces a solution that has value at least OP T I c resp has cost at

most c OP T I

Note Making c larger makes the algorithms task easier the set of acceptable solutions it

can output gets enlarged In many applications it makes sense to allow c to grow with the

input size since xing c to b e a constant seems to rule out p olynomial time approximation

algorithms For example it makes sense to let c b e log nwheren is the input size Nowwe

dene one such factor that often crops up in inapproximability results

umber in the interval Denition An approximation factor is large if for some xed n

log n

it is at least where n input size

Denition A computational problem is almostNPhard if a p olynomial time algo

polylog n

rithm for it can b e used to solveevery NP problem in time n

Note People b elieve that not only is NP P but also that there are problems in NP whose

p olylog n

solution requires more than n time Hence proving a problem is almostNPhard

provides a go o d reason to b elieve that it will not have p olynomialtime algorithms

Organization of the Chapter Section describ es our canonical problems and their

prop erties Section denes the gappreserving reduction an essential ingredient of inap

proximability results An example of a gappreserving reduction is given using the CLIQUE

problem Section discusses MAXSNP a class of optimization problems and shows that

approximating MAXSNPhard problems is NPhard Section shows the inapproxima

bility of a host of problems having to do with lattices co des and linear systems The

reductions to these problems use the canonical problem Lab el Cover Section shows

how to exhibit the NPhardness of n approximation where is some xed p ositive con

stant Section contains a survey of known inapproximability results and indicates how

they can b e proved using our canonical problems

The Canonical Problems

Our reductions use two canonical problems MAXSAT and Lab el Cover The latter

comes in twoversions one in volving maximization and the other minimization The two

THE CANONICAL PROBLEMS

versions are not really indep endent since a form of weak duality links them Lemma

However wewould liketokeep the twoversions separate and for the obvious reason

proving the inapproximability of minimization problems as we do in Section requires

us to have a minimization problem in our list of canonical problems

Denition MAXSAT This is the version of MAXSAT in whichno

variable app ears in more than clauses Note has no sp ecial signicance

other than that it is a convenient constant

LABEL COVER The input consists of i A regular bipartite graph G V V E

ii An integer N in unaryWe think of anyinteger in Nasa label iii For each

edge e E a partial function N N

e

A label ling has to asso ciate a set of lab els with every vertex in V V It is said to

cover an edge e u v where u V v V if for every lab el a assigned to v there

is some lab el a assigned to v such that a a

e

Maximization Version Using a lab elling that assigns lab el p er vertex maximize the

numberofcovered edges

Minimization Version Using more than lab el p er vertex if necessarycover all the

edges Minimize the cost of doing this where the cost is dened as

X

numb er of lab els assigned to v

v V

that is the total numb er of lab els counting multiplicities assigned to vertices in V

Convention To state results cleanly the value of the optimum will b e stated as a

ratio In the maximization version of Lab el Cover this ratio is the fraction of edges

covered In the minimization version this ratio is average numb er of lab els used p er

vertex in V that is

cost of lab elling

jV j

Also the careful reader will realize that in the min version an optimum lab elling

never needs to assign more than one lab el to anyvertex of V We do not make this

a part of the denition b ecause it makes our reductions more dicult to describ e

The Lab el Cover problem was implicit in LY and was dened in ABSS Although

an ungainly problem at rst sight it is quite useful in reductions We rst give some

examples to clarify the denition of the problem We currently dont see a way to simplify

its denition since all asp ects of the denition seem to play a role when we do reductions

For our purp oses a bipartite graph is regular if for some integers d d every vertex on the left resp

right has degree d resp d

CHAPTER HARDNESS OF APPROXIMATIONS

{0,1} 1 * * −−−−> 1 . x {0,7} \ {001} 1 0 * * −−−−> 0

_ (x1 V x 2 V x 3) * 1 * −−−−> 1, * 0 * −−−−> 0

. x {0,1} 2 _ _ ( x 12 V x V x 3)

{0,7} \ {110} * *1 −−−> 1 . x {0,1}

* * 0 −−−> 0 3

Figure Lab el Cover instance for formula x x x x x x The symbols

on edge e representmap

e

Example SAT is reducible to Lab el Cover max version Given a SAT instance

dene a Lab el Cover instance as follows Let V have one vertex for each clause and V

haveavertex for every variable let adjacency corresp onds to the variable app earing in the

clause whether negated or unnegated For the momentwe ignore the regularity condition

and leave it to the reader to x The set of lab els is where the signicance of the

numb er of lab els is that it is the numb er of p ossible assignments to the variables in

a clause We denote the lab els in binaryFor a vertex in V if the corresp onding clause

b b as the assignment involves variables x x x the reader should think of a lab el b

i j k

x b x b x b For a vertex in V say one corresp onding to variable x the set

i j k i

of admissible lab els is f gto b e though tofasassignments and resp ectively to

that variable

The edge function is describ ed as follows Supp ose e is the edge u v where u V

e

corresp onds to clause x x x and v V corresp onds to variable x The partial

function for this edge has the domain nfg in other words the assignments

e

to x x x that satisfy the clause Every lab el of the typ e x x x means

anything is mapp ed under to and every lab el of the typ e x x x

e

is mapp ed to

Figure shows the ab ove transformation as applied to the formula x x x

x x x The gure uses the shorthand and for resp ectively for the

vertices on right

Since eachvertex is allowed only lab el the lab els on the righthandsizevertices

consitute a b o olean assignmenttox x The lab el on a lefthand vertex also constitute

THE CANONICAL PROBLEMS

an assignmenttothevariables in the corresp onding clause The edge joining a clausevertex

andavariablevertex is covered i it is assigned the same value by b oth assignments

Clearly if all edges are covered then the assignment is a satisfying assignment Con

versely if there is no satisfying assignment some edge is left uncovered by all lab ellings

Actually we can make a stronger statement which will b e useful later Any lab elling

that covers a fraction of edges yields an assignment that satises fraction of the

clauses

Example Hitting Set is a subcase of Lab el Cover min version Recall that the

input to Hitting Set is a set U and some sets S S U The output is the smallest

K

subset of U that has a nonemptyintersection with all of S S This problem is the

K

dual of setcover

Hitting set is a trivial reformulation of the following subcase of Lab el Cover min ver

sion there is just one vertex on the left that is jV j and for all edges e the map

e

has exactly one element in its imageset Note in particular that eachvertex v V has a

unique edge inciden t to it say e and has an imageset of size

v e

v

Given such an instance of Lab el Cover dene U to b e N Dene the set S U as

v

fa label a is a preimage of g Solving hitting set gives us the smallest set S N

e

v

such that S S is nonempty for each v V Then S is the solution to Lab el Cover since

v

it is the smallest set of lab els that must b e assigned to the lone vertex in V such that all

edges are covered

A similar transformation works in the other direction from Hitting Set to Lab el Cover

The following weak duality implicit in LY links the max and min versions of Lab el

Cover Note how our convention ab out stating costsvalues as ratios allows a clean state

ment

Lemma For any instance I of Label Cover if OP T I and OP T I are respec

max min

tively the optimum value in the max version and the minimum cost in the min version

then

OP T I I

max

OP T I

min

Pro of Consider the solution of the minimization version namely a lab elling using on an

average OP T I lab els p er vertex in V and covering all the edges For anyvertex u V

min

let it assign n lab els to this vertex Then by denition of cost

u

X

n OP T I jV j

u min

uV

We randomly delete all lab els for eachvertex in V V except one This gives a

lab elling that assigns only lab el p er vertex in other words a candidate solution for the

CHAPTER HARDNESS OF APPROXIMATIONS

maximization version We claim that the probability that any particular edge u v is still

covered is n For the original lab ellin g covered u v for ev ery lab el a it assigned to

u

v it assigned some preimage a thatwas assigned to u The probability that this

e

preimage survives is n

u

In the new randomly constructed lab elling the exp ected numb er of edges still left

covered is at least

X

n

u

euv

Since eachvertex in V has the same degree say d the numb er of edges jE jis d jV j and

the ab ove exp ectation can b e rewritten as

X X

d jV j

P

d d

n n n

u u u

uV

uV uV

jV j jE j

d

OP T I jV j OP T I

min min

P

The crucial fact used ab ove is that the sum n is minimized when all n are equal

u u

u

Thus wehaveshown a randomized way to construct a candidate solution to the max ver

sion such that the exp ected fraction of edges covered is at least OP T I It follows

min

that there must exist a candidate solution that covers this many edges Hence wehave

proved

OP T I

max

OP T I

min

Note Lemma uses the fact that the bipartite graph in the Lab el Cover instance is

regular This is the only place where we need this factw e do not need it for our reductions

Nowwegive the hardness results for our canonical problems

Theorem There is a xedpositive constant for which thereispolynomial time re

duction from any NP problem to MAXSAT such that YES instances map to satisable

formulae and NO instances map to formulae in which less than fraction of clauses can

be satised

Pro of In Corollary we describ ed such a reduction for MAXSAT For some xed

constant it satises the prop erty that YES instances map to satisable formulae and NO

instances to formulae in which no more than fraction of the clauses can b e satised

We showhowtochange instances of MAXSATinto instances of MAXSAT without

changing the gap of bymuch The reduction is a slight simplication of the one in PY

although their reduction yields instances of MAXSAT and uses weaker expanders

We need the following result ab out explicit constructions of expander graphs LPS

There is a pro cedure that for every integer k can construct in p oly k time a regular

THE CANONICAL PROBLEMS

graph G on k vertices such that for every set S of size at most k there are more than

k

S jS j edges b etween S and its complement

Let the instance of SAThave n variables y y andm clauses Let m denote

n i

P

the numb er of clauses in whichvariable y app ears Let N denote the sum m Sincea

i i

i

clause contains at most variables N m

For eachvariable do the following If the variable in question is y replace it with m

i i

j

new variables y y Use the j th new variable y in place of the j th o ccurence of y

i

i i i

m

i

Next to ensure that the optimal assignment assigns the same value to y y add the

i

i

is an edge of the expander following m new clauses For each j l m suchthatj l

i i

j j

l l

Together this pair just says and y y G add a pair of new clauses y y

m

i i i

i i

j j

l l

and y y an assignmen t satises the pair i it assigns the same value to y y

i i

i i

Hence the new formula contains N new clauses and m old clauses Eachvariable o ccurs

in exactly new clauses and old clause If the old formula was satisable so is the new

formula Next we show that if every assignment satises less than fraction of clauses

in the old formula then no assignment satises less than fraction of clauses in the

new formula

Consider an optimal assignment to the new formula namely one that satises the

maximum numb er of clauses We claim that it satises all new clauses For supp ose it

do es not satisfy a new clause corresp onding to y Then it do es not assign the same value to

i

m

i

all of y y Divide up these m v ariables into twosets S and S according to the value

i

i

i

they were assigned One of these sets has size at most m sa yitis S In the expander

i

G consider the set of jS j vertices corresp onding to vertices in S Expansion implies there

m

i

are at least jS j edges leaving this set Each such edge yields an unsatised new clause

Hence by ipping the value of the variables in S we can satisfy at least jS j clauses

that werent satised b efore and p ossibly stop satisfying the at most jS j old clauses that

contain these variables The net gain is still at least Therefore our assumption that the

assignmentwe started with is optimal is contradicted

We conclude that i fng the optimal assignment assigns identical values to

m

i

the dierentcopiesy y of y Now supp ose no assignment could satisfy more than

i

i

i

m clauses in the original formula Then in the new formula no assignment can satisfy

mwe see that the fraction of unsatised more than N m clauses Since N

clauses is at least Hence the theorem has b een proved for anyvalue of

mm

that is less than

The following corollaries are immediate

Corollary Thereisan such that nding approximations to MAX

SAT is NPhard

CHAPTER HARDNESS OF APPROXIMATIONS

Corollary Finding approximations to Label Cover max version is NP

hard where is the same as in Corol lary

Pro of Use the reduction from SAT to Lab el Cover in Example As observed there

any lab elling that covers a fraction p of the edges for some p yields an assignment that

satises p of the clauses

Approximating Lab el Cover even quite weakly is also hard

polylog n

Theorem Let beanylarge factor as dened in Denition Thereisa n

time reduction from any NP problem to Label Cover max version such that YES instances

map to instances in which the optimum value is and NO instances map to instances in

which the optimum value is less than

Pro of Proved in Chapter

Hence wehave the following corollary

Corollary Approximating Label Cover max version within any large factor is almost

NPhard

The following theorem gives the hardness of approximating the min version of lab el

cover

Theorem Approximating the minimization version of Label Cover within any large

polylog n

factor is almostNPhard Morespecical ly for every large factor thereisan

time reduction from NP to Label Cover min version that maps YES instances to instances

in which optimum cost is and NO instances to instances in which the optimum cost is

more than

Pro of Consider the instances of Lab el Cover arising out of the reduction in Theorem

by using the same value of The optimum value of the max version is either or less

than When the optimum cost is there is a lab elling that uses lab el p er vertex

and and covers all the edges hence the optimum cost of the min version is When the

optimum value in the max version is less than Lemma implies the optimum cost

of the min version is at least

GAPPRESERVING REDUCTIONS

GapPreserving Reductions

In Theorems and weproved that approximating our canonical problems is hard To

extend these inapproximability result to other problems we need to do reductions carefully

so that they are gappreserving

Denition Let and be two maximization problems and A gap

preserving reduction with parameters c c from to is p olynomialtime algo

rithm f For each instance I of algorithm f pro duces an instance I f I of The

optima of I and I say OP T I andOP T I resp ectively satisfy the following prop erty

c and if OP T I c then OP T I c if OP T I c then OP T I

Example For a CNF formula dene OPT as the maximum fraction of clauses

that can b e satised by an assignment Recall the expanderbased reduction from MAX

SAT to MAXSAT in Theorem It maps formulae in which OP T to formulae

in which OP T Further it maps formulae in whichOPT to formulae in which

OPT

Hence the reduction is gappreserving with parameters

for every p ositive fraction

Comments on Denition Supp ose there is a p olynomial time reduction from

NP to such that YES instances are mapp ed to instances of of cost c and NO

instances to instances of cost c Then the reduction of Lemma if it exists implies

that nding approximations to is NPhard Like most known reductions ours will

also map solutions to solutions in an obvious wayFor instance given a solution to I of

cost c a solution to I of cost c can b e pro duced in p olynomial time But wekeep this

asp ect out of the denition for simplicity The ab ove denition can b e mo died in an

obvious way when one or b oth of the optimization problems involve minimization A

gappreserving reduction since its niceness namely equation holds only on a partial

domain is a weaker notion than the Lreduction intro duced in PY where niceness

has to b e maintained on al l instances of An Lreduction coupled with an approximation

algorithm for yields an approximation algorithm for The previous statement is false

for a gappreserving reduction On the other hand for exhibiting merely the hardness of

approximation it suces and is usually easier to nd gappreserving reductions Indeed

for some of the problems we will cover later Lreductions are not known to exist

Often our gap preserving reductions work with values of the factors that are func

tions of the input length The reduction in the following result is in this vein Let CLIQUE

b e the problem of nding the largest clique ie a vertexinduced subgraph that is a com

plete graph in the given graph

CHAPTER HARDNESS OF APPROXIMATIONS

log n

Theorem For every nding approximations to CLIQUE is almost

NPhard

Pro of We give a gappreserving reduction from Lab el Cover max version to CLIQUE

such that the optimum in the CLIQUE instance is exactly the optimum value of the Lab el

Cover instance In other words for every c the reduction satises the parameters

c c Since Lab el Cover is hard to approximate within any large factor Lemma

it follows that so is CLIQUE

Let E V V N b e an instance of Lab el Cover

For an edge e and lab els a a letalabel ling scenario or scenario for short b e a triple

e a a such that a a Ife u v we think of the scenario as assigning lab el

e

a to vertex u and lab el a to v Clearlyany lab elling that assigned lab els this waywould

cover e Two scenarios are inconsistent with each other if the two edges involved share a

vertex and the scenarios assign dierent lab els to this shared vertex Togive an example

the scenarios u xa a and u y a b are inconsistent regardless of what x a y b are

since one scenario assigns lab el a to u while the other assigns a

Nowwe construct a graph G V E using the given instance of Lab el Cover

The set of vertices in G is the set of lab ellin g scenarios Two dierent scenarios are

adjacentin G i they are not inconsistent

Claim Size of largest clique in G Numb er of edges covered by optimum lab elcover

Theproofisintwo parts

For any lab elling of V V that covers K edges take the set of scenarios o ccuring at

the covered edges All the scenarios in this set are consistent with one another so they

form a clique in G The clique has size K

For any clique S V of size K construct as follows a lab elling that assigns lab els to a

subset of vertices and covers K edges Clearly such a partial lab ellin g can b e extended to

a complete lab ellin g that covers at least K edges Notice that no two scenarios in a clique

can b e inconsistent with one another Hence for anyvertex in V V notwo scenarios

present in the clique assign dierent lab el to that vertex Now assign to eachvertex any

lab el if one exists that gets assigned to it by a scenario in the clique This denes a

partial lab elling that covers every edge e which app ears in a scenario in S The number

of such edges is K

Note The graph G pro duced by this reduction has sp ecial structure it is a union of jE j

indep endent sets for any edge e u v the two distinct vertices e a a and e a a

must have either a a ora a and so are not connected in G Further the size of

the largest clique is either jE j or jE j This prop erty of the graph G is useful in doing a

further reduction to Chromatic Numb er see Section

MAXSNP

MAXSNP

NPhard optimization problems exhibit a vast range of b ehaviors when it comes to approxi

mation Papadimitriou and Yannakakis PY identied a large subclass of them that

exhibit the same b ehavior The authors dened a class of optimization problems MAXSNP

as well as a notion of completeness for this class Roughly sp eaking a MAXSNPcomplete

problem b ehaves just like MAXSAT in terms of approximability This made MAXSAT

a plausible candidate problem to prove hard to approximate and in particular motivated

the discovery of the PCP theorem

MAXSNP contains constraintsatisfaction problems where the constraints are lo cal

More formally the constraints are denable using a quantierfree prop ositional formula

The goal is to satisfy as many constraints as p ossible

Denition A maximization problem is in MAXSNP if given an instance I we can

x with the in p olynomial time write a structure G and a quantierfree formula G S

following prop erties Formula involves the relations in G a relation symbol S not part

of G and a vector of variables x where eachvariable in x takes values over the universe of

G

The value of the optimum solution on instance I OPTI is given by

x G S x TRUEgj OPTI max jf

S

Note The ab ove denition is inspired byFagins mo deltheoretic characterization of

NP Fag and an explanation is in order for those unfamiliar with mo del theory G

consists of a sequence of relations of xed arityover some universe U IfU n then G

k

implicitly denes an input of size O n where k is the largest arity of a relation in G

The role of S is that of a nondeterministic guess

Example Let MAXCUT b e the problem of partitioning the vertexset of an undi

rected graph into two parts such that the numb er of edges crossing the partition is max

imised Heres how to see it is in MAXSNP The universe is the vertexset of the graph

Let G consist of E a binary relation whose interpretation is adjacency Let S b e a unary

relation interpreted as one side of the cut and E S u v E u v S u S v

Both MAXSAT and MAXSAT are also in MAXSNPFor every MAXSNP

problem there is some constant c such that the problem can b e capproximated in

p olynomial time PY The smallest suchvalue of c for MAXCUT is approximately

GW for example

There is a notion of completeness in the class MAXSNP According to the original

denition a MAXSNP problem is complete for the class if every MAXSNP problem can

b e reduced to it using an Lreduction We are interested in hardness of approximation

CHAPTER HARDNESS OF APPROXIMATIONS

For us it will suce to consider a problem MAXSNPhard if every MAXSNP problem

has a gappreserving reduction to it with parameters some absolute constants greater

than See the note following Denition to see why the original denition is more

restrictive Examples of MAXSNPhard problems include MAXCUT MAXSAT

and many others See Section for a partial list From Theorem the following is

immediate

Corollary For every MAXSNPhardproblem there exists some c such that

nding capproximations to it is NPhard

Problems on Lattices Co des Linear Systems

This section contains inapproximability results for a large set of NPhard functions All

the results involve gappreserving reductions from Lab el Cover The functions considered

include wellknown minimum distance problems for integral lattices and linear co des as well

as the problem of nding a largest feasible subsystem of a system of linear equations or

inequalities over Q In this section n denotes the length of the input and m the dimension

of the lattice co de etc under consideration

The Problems

k

An integral lattice Lb b in R generated by the basis fb b g is the set of all

m m

P

m

k

linear com binations b where the fb g is a set of indep endentvectors in Z and

i i i

i

Z

i

Denition Shortest Vector Problem in norm SV Givenabasis fb b g

p p m

nd the shortest nonzerovector in norm in Lb b

p m

Denition Nearest Vector Problem in norm NV Given a basis fb b g

p p m

k

andapoint b Q where b ndthenearest vector in norm in Lb b to

p m

b

Next we dene three other problems all of which in one way or another involve distances

of vectors

Denition Nearest Co deword INPUT An m k matrix A over GF and

k

avector y GF

m

OUTPUT Avector x GF minimizing the Hamming distance b etween xA

and y

PROBLEMS ON LATTICES CODES LINEAR SYSTEMS

MaxSatisfy INPUT A system of k equations in m variables over Q

OUTPUT Size of the largest subset of equations that is a feasible system

MinUnsatisfy INPUT A system of k equations in m variables over Q

OUTPUT Size of The smallest set of equations whose removal makes the system

feasible

The next problem is a wellknown one in learning theory learning a halfspace in the

presence of malicious errors The problem arises in the context of training a p erceptron

a learning mo del rst studied by Minsky and Pap ert MP Rather than describing

the learning problem in the usual PAC settingVal we merely present the underlying

combinatorial problem

m

The input to the learner consists of a set of k points in R each lab elled with or

These should b e considered as p ositive and negative examples of a concept The learners

m

output is a hyp erplane a x b a x R b R The hyp othesis is said to correctly

classify a p oint marked resp if that p oint say y satises a yba ybresp

Otherwise it is said to misclassify the p oint

Finding a hyp othesis that minimizes the numb er of misclasscations is the open hemi

spheres problem which is NPhard GJ Dene the error of the algorithm as the number

of misclassications byitshyp othesis and the noise of the sample as the error of the b est

p ossible algorithm Let the failureratio of the algorithm b e the ratio of the error to noise

m

Denition Failure Ratio Input A set of k points in R each lab elled with

or

Output A hyp othesis that makes the ratio of error to noise

Note that to capproximate Failure Ratio means to output a hyp othesis whose failure ratio

is at most c

The Results

We prove the following results

Theorem Approximating eachofNV Nearest Codeword MinUnsatisfy and

p

FailureRatio i within any constant factor c is NPhard ii within any large

factor is almostNPhard

Approximating SV within any large factor is almostNPhard

ectorssystems with all entries Hence it follows We note that our reductions use only v

that approximation in those subcases is equally hard Part of Theorem is proved

CHAPTER HARDNESS OF APPROXIMATIONS

in Section Part follows easily from Lemma in conjunction with the hardness

result for Lab el Cover Theorem Lemma is proved in Section although an

imp ortant gadget used there is describ ed earlier in Section

Lemma For each of the problems in part of Theorem and every factor

there is some cost c and some polynomial time gappreserving reduction from Label Cover

min version to the given problem which has parameters c where for al l

p

p

the reductions except for NV whereitis

p

Note the values of the parameters more sp ecically the fact that the reduction maps

Lab el Cover instances with optimum cost at most to instances of the other problem with

optimum cost at most c By denition the cost of the min version cannot b e less than

So what the gappreserving reduction actually ensures is that Lab el Cover instances

with optimum cost exactly are mapp ed to instances with cost at least c This is an

example of a reduction for whichwedonotknow whether an Lreduction in the sense of

PY exists However the gappreserving reduction as stated ab ove is still go o d enough

for proving inapproximability in conjunction with the reduction stated in Theorem

since that other reduction quite conveniently it seems did map YES instances to Lab el

Cover instances with optimum cost exactly

For MaxSatisfy we will prove a stronger result in Section

Theorem Thereisapositive constant such that nding n approximations to Max

Satisfy is NPhard

Better Results For NEARESTCODEWORD and NV for all p wewe can prove

p

log n log n

almostNPhardness up to a factor instead of These results app ear in

ABSS Also in our reductions the number of v ariables dimensions input size etc are

p olynomially related so n could b e any of these

Previous or Indep endentWork Bruck and Naor BN haveshown the hardness of

approximating the NEARESTCODEWORD problem to within some factor Amaldi

and Kann AK have indep endently obtained results similar to ours for MAXSATISFY

and MINUNSATISFY

Signicance of the Results

We discuss the signicance of the problems we dened

Lattice Problems The SV problem is particularly imp ortant b ecause even relatively

m

p o or p olynomialtime approximate solutions to it within c LLL have b een used in a

PROBLEMS ON LATTICES CODES LINEAR SYSTEMS

host of applications including integer programming solving lowdensity subsetsum prob

lems and breaking knapsackbased co des LO simultaneous diophantine approximation

and factoring p olynomials over the rationals LLL and strongly p olynomialtime algo

rithms in combinatorial optimization FT For details and more applications esp ecially

to classical problems in the geometry of numb ers see the surveys byLovasz Lovor

Kannan Kan

Lovaszs celebrated lattice transformation algorithm LLL runs in p olynomial time

m

and approximates SV p within c A mo dication of this algorithm Bab yields

p

the same for NV Schnorr mo died the Lovasz algorithm and obtained for every

p

m

approximations within O in p olynomial time for these problems Sch

On the other hand Van Emde Boas showed that NV is NPhard for all p vEB

p

see Kan for a simpler pro of Lagarias showed that the shortest vector problem is NP

hard under the ie max norm But it is still an op en problem whether SV is NPhard

p

for any other p and sp ecically for p

While wedonotsolve this op en problem we obtain hardness results for the approximate

solutions of the known NPhard cases

Wemention that improving the large factors in either the NV or the SV result to

p

m m dimension would prove hardness of SV a long standing op en question The

p

reason is that approximating either SV or NV to within a factor m is reducible to

SV To see this for SV notice that the solutions in SV and SV are always within a

p

factor m of eachotherFor NV the implication follows from Kannans result Kan

p

d is reducible to SV that approximating NV within a factor

We also note that approximating NV within any factor greater than m is unlikely

to b e NPcomplete since Lagarias et al LLS haveshown that this problem lies in

NP coNP

Problems on Linear Systems Note that a solution to MAXSATISFY is exactly the

complement of a solution to MINUNSATISFY and therefore the two problems have the

same complexity Indeed it is known that b oth are NPhardthis is implicit eg in JP

However the same need not b e true for approximate solutions For instance vertexcover

and indep endentset are another complementary pair of problems and seem to dier

greatly in howwell they can b e approximated in p olynomial time Vertex cover can b e

c

approximated within a factor and indep endentset is NPhard up to a factor n for some

c FGL AS ALM

We nd it interesting that large factor approximation is hard for our two complementary

problems We do not knowofany other complementary pair with the same b ehavior

We knowofnogoodapproximation algorithms for any of these problems Kannan has

shown us a simple p olynomial time algorithm that uses Hellys theorem to approximate

MINUNSATISFY within a factor of m

CHAPTER HARDNESS OF APPROXIMATIONS

Failure Ratio That the failure ratio is hard to approximate has often b een conjectured

in learning theory HS We knowofnogoodapproximation algorithms A failure ratio

m can b e achieved by Kannans idea mentioned ab ove

A Set of Vectors

In this section we use Lab el Cover instances to dene a set of vectors that will b e used by

all the reductions

Let V V EN b e an instance of Lab el Cover min version Think of a lab elling

as a pair of functions P P where the set of lab els assigned to the vertex v V is

P v and the set of lab els assigned to the vertex v V is P v Let us call a lab elling

that covers every edge a total cover

First simplify the structure of the edge functions For a vertex v V prune as

e

follows the domains of the edge functions of eachedgecontaining it Restrict the set of

lab els that can b e assigned to v to contain only lab els a N such that for every edge

e incidenttov there is a lab el a suchthat a a Inotherwords prune so that

e e

the domain contains only those lab els a that can b e used to cover all edges incidenttov

Call suchalabel valid for v and call v a avalid pair

Allowing only valid lab els to b e assigned to vertices do es not hurt us The reduction

b eing gappreserving with parameters can assume that the total cover uses either

exactly lab el p er vertex or at least lab els p er vertex on average In the former case

each lab el used must b e valid And in the latter case restricting the set of p ossible vertex

lab el pairs to b e valid can only increase the minimum cost of a total cover

and a N and a The set of vectors contains a vector V for each v V

v a

vector V for each valid pair v a where v V and a N Note that any

v a

linear combination of these vectors implicitly denes a lab ellin g where lab el a is assigned

to vertex v i V has a nonzero co ecient in the combination

va

P

Denition Let x c V b e a linear combination of the vectors in the

v a v a

i i i i

set where the co ecients c are integers The label ling dened by the vector x denoted

v a

i i

x x x

P is the one that assigns to v V the set of lab els P v fa j c g P

v a

x

The set of lab els P v assigned to v V is dened similarly

Dene the vectors in the set as follows Eachhas jE j N co ordinates N consec

utive co ordinates for each edge e E Call the co ordinates corresp onding to e inavector

as its eprojection See Figure

For j N let u be a vector with N co ordinates in which the j th entry

j

is and all the other entries are With some abuse of notation well asso ciate the vector

with the lab el a N Let and b e the allzero vector and allone vectors u

a

resp ectively

PROBLEMS ON LATTICES CODES LINEAR SYSTEMS

V V [(v , a )] [(v , a )] 1 1 2 2

u i

0 0 . i th . 1 − u ua entry 1 b e − projection 2 N + 1 . ( b = Π (a ) ) . e 1 0 0

ε ε V v V

v1 1 2 2

Figure Figure showing the epro jections of vectors V and V in the vector

v a v a

set where e v v

For v V a N let the epro jection of the vector V be u if e is incident

a

v a

to v and otherwise

For eachvalid pair v a let the epro jection of the vector V be u if e is

v a a

e

ttov and otherwise inciden

Note that the epro jections of the vectors form a multiset comprised of exactly one copy

of the vector u for eachlabel a zero or more copies of the vector u and multiple

a a

copies of The following lemma says that a linear combination of vectors in the multiset

is i for some lab el a b oth the vectors u and u app ear in it

a a

Lemma Suppose some integer linear combination of the vectors fu ja Ng

a

o n

ja N u is Then there is some a N such that the coecients of both

a

in the linear combination are nonzero and u u

a a

Pro of The vectors fu u g are linearly indep endent and do not span Therefore if

N

X

c u d u

a a a a

a

then c d for all a Furthermore there exists an a for which these are not zero

a a

Corollary If x is a nontrivial linear combination of the vectors fV g and x

v a

i i

x x

then P P is a total cover

CHAPTER HARDNESS OF APPROXIMATIONS

Pro of For anyedge e u v the epro jections of the vectors fV g form a system

v a

i

describ ed in the hyp othesis of Lemma Note that the epro jections of the typ e u

a

b elong to the vector of V and the epro jection u belongtoavector V such

a

va ua

that a a The fact that the linear combination x assigns nonzero co ecients to

e

x x

to v and a for some a implies that P P assigns some lab el a and u b oth u

a a

x x

a to u such that a a Since this happ ens for every e lab ellin g P P covers all

e

edges and is a totalcover

Reductions to NV and others

Nowwe prove Lemma

Pro of Lemma First we show the reduction to the Nearest Lattice Vector problem

with the normresults for the other problems will follo w easily The main idea in the

n o

reduction is to use the set of vectors V from Section as part of the lattice

v a

i i

denition Thexedpointischosen in suchaway that all vectors that are near to it

n o

involveaninteger linear combination of the set V that is Thus vectors near to

v a

i i

the xed p oint are forced to dene a total cover as seen in Corollary

The basis vectors hav e jE j N jV jN co ordinates jV j N more than vectors

in the previous section

Let L b e the integer jE j N The xed p oint W willhavean L in each of the

rst jE j N dimensions and elsewhere

The basis of the lattice consists of the following vectors for every vector in the ab ove

o n

there is a basis vector W In the rst jE j N co ordinates the set V

v a v a

i i i i

e think of the last jV jN co ordinates as b eing identied vector W equals L V W

v b v b

i i i i

onetoone with a valid pair v a Then the co ordinate identied with v a contains

in W and in all other vectors

v a

Since corresp onding to every basis vector there is a unique co ordinate in which this

vector is but no other vector is the following claim is immediate

P

Claim Let x c W be a vector in the lattice Then jj W xjj

v a v a

i i i i

x x

where cost is b eing measured as usual as a ratio P jV jcost of P

Now let OPT b e the minimum cost of a total cover Weshow that every vector x in the

lattice satises jj W xjj min fL jV jOPT g Notice that eachentry of W x in the

rst jE j N dimensions is a sum of integer multiples of L If it isnt its magnitude

is L and so jj W xjj L On the other hand if all those entries are then by

x x

is a totalcover and so by the ab oveclaim jj W xjj jV j P Corollary P

OPT

PROBLEMS ON LATTICES CODES LINEAR SYSTEMS

Finally if there is a totalcover P P of cost then the following vector has length

jV j

P P

x W W W

v P v v P v

v V v V

Nowweshow the hardness of the other problems Let m b e the numb er of basis vectors

and U b e an integer such that the numb er of co ordinates is m U

NV with vectors Replace each of the last U co ordinates by a set of L new

ector had an L in the original co ordinate it has a in each of the new co ordinates If a v

L co ordinates and otherwise

p

p

c Other Finite Norms Changing the norm from to changes the gap from c to

p

hence the result claimed for norms also follows

p

l Norm See ABSS for details

obtained from the reduction to NV b NearestCo deword View the vectors b

m

Then the with vectors as generators of a binary co de Let the received message b e b

minimum distance of b to a co deword is exactly K in one case and c K in the other

again Eachvector has m L U b b MinUnsatisfy Consider the instance b

m

co ordinates This instance implicitly denes the following system of m L U equations in

m variables

X

b b

i

i

where the s are the variables and is the vector whose all co ordinates are The

i

structure of the problem assures us that if we satisfy the rst L U equations then the

nonzero variables must yield a setcover and then each nonzero variable gives rise to an

unsatised equation among the last m ones Thus the minimal numb er of equations that

must b e removed in order to yield a satisable system is K in one case and c K in the

other

Learning Halfspaces Notice that minimizing the failure ratio involves solving the fol

lowing problem Given a system of strict linear inequalities nd the smallest subset of

inequalities whose removal from the system makes it feasible Nowwetake the system of

equations in the MINUNSATISFY reduction and replace each equation bytwo inequalities

in the obvious way This do es not give a system of strict inequalities however it do es give

a gap in the numb er of inequations that mustberemoved in order to make the system

feasible The inequalities can b e made strict for this sp ecial case byintro ducing a new

variable and changing each inequation to at the same time

Itisnow easily seen that any solution to intro ducing L identical new inequations L

the largest feasible subsystem must have L which in turn forces the variables to b e

CHAPTER HARDNESS OF APPROXIMATIONS

Hardness of Approximating SV

Proving the correctness of our reduction to SV involves delving into the geometric struc

ture of Lab el Cover instances pro duced with a sp ecic pro of system namely the one due

to FeigeLovasz pro ofsystem FL Wedonotknow whether a gappreserving reduction

exists from Lab elCover

For the reduction to SV well need to prove the hardness of a related and not very

natural covering problem

Denition Let V V EN b e an instance of Lab el Cover min version Let

P P b e a lab elling and e v v b e an edge The edge is untouched by the lab elling

if P v andP v are empty sets It is cancel led if P v is empty P v is not empty

and for every b P v there is an b P q such that b oth e b a and e b a

are in for some a A

Denition A lab ellin g P P isapseudocover if it assigns a lab el at least one

vertex and every edge is either untouched cancelled or covered byit

Note that in a pseudocover the only case not allowed is that for some edge v v the set

of lab els P v is empty but P v is not

Denition The cost of a lab ellin g P P is

max fjP v j v V g

One of our main theorems is that approximating the minimum cost of a pseudocover

is hard

poly log n

Lemma For every large factor thereisan time reduction from any

NP language to instances of LabelCover such that YES instances map to instances which

have a pseudocover with cost and NO instances map to instances where every pseudo

cover has has cost at most

We indicate in Chapter how this lemma is proved Nowwe show the hardness of

approximating SV

p

Theorem For any large factor approximating SV within a factor is almost

NPhard

To prove the theorem we again use the vectors from Section

PROBLEMS ON LATTICES CODES LINEAR SYSTEMS

Lemma Let fV g be the set of vectors dened for the Label Cover instances of

v a

i i

Lemma constructedasinSection If x is a nontrivial linear combination of the

x x

vectors and x then P P is a pseudo cover

Pro of For anyedge e u v the epro jections of the vectors fV g form a system

v a

i

describ ed in the hyp othesis of Lemma Note that the epro jections of the typ e u

a

such b elong to the vector of V and the epro jection u belongtoavector V

a

ua va

that a a Since the set of vectors u are linearly indep endent it follows that for

e a

every linear combination of these vectors that is for every lab el a either b oth u and

a

u have co ecientorbothhave a nonzero co eecient Hence in terms of the

a

x x x x

lab elling P P dened by this combination if P assigns some lab el a to v then P

must assign a lab el a to u such that a a Since this happ ens for every e lab elling

e

x x

P P pseudocover

t

The reduction uses an Hadamard matrix ie a matrix suchthatH H I

H exists eg when is a p ower of cf Bol p

p

k Lemma Let z Z Ifz has at least k nonzero entries then jjH z jj

p

p p

Pro of The columns of k H form an orthonormal basis Hence jj H z jj jjz jj

Pro ofof Theorem We use the set of vectors from Lemma and extend them by

adding new co ordinates to get the basis set for our lattice

Let L b e the integer jE jjA j The vectors in the basis have jE j N jV jN

co ordinates each that is jV jN more than the vectors of Lemma For eachvector

V of that other set the basis containsavector W The basis also contains an

v a v b

i i i i

additional vector W that has L in each of the rst jE j N co ordinates and

elsewhere

As in the NV reduction W will equal L V in the rst jE j N co ordinates

v a v a

i i i i

The remaining jV jN co ordinates will b e viewed as blo cks of N co ordinates each asso ciated

with a v V We refer to entries in the blo ck asso ciated with v as v projection of the

vector

Wemay assume there exists a Hadamard matrix H for N With each lab el

a Nwe identify a unique column vector of H denoted h Thenthev pro jection

l a

of W is h if v v and if v v

a

va

Let OPT b e the minim um cost of a pseudocover

p

OP T Claim For anyvector x in the lattice jjxjj

Pro of The entry in any of the rst jE j jA j co ordinates is a sum of integer multiples

of L so if it is not its magnitude is L and hence OPT So all these entries must b e

CHAPTER HARDNESS OF APPROXIMATIONS

But then by Lemma we conclude that the lab elling dened by x is a pseudocover

p

OPT by and must therefore assign OPT lab els to some v V But then jjxjj

Lemma

Finally if OPT and P P achieves it then the following vector has norm

X X

W W W

v P v v P v

v V v V

Proving n approximations NPhard

In this section wegive some idea of howtoprove the NPhardness of n approximations

Original pro ofs of these results were more dicult and we present a simplication due to

AFWZ All the results could also b e proved in a weaker form using reductions from

Lab el Cover but that would prove only almostNPhardness of large factors although see

the op en problems in Chapter sp ecically Conjecture

We illustrate this prop Often problems in this class haveaselfimprovement property

erty with the example of clique

Denition Given graphs G V E and G V E their product G G

is the graph whose vertexset is the set V V and edgeset is

fu v u v u u E and v v E g

Example Let G b e the size of the largest clique in a graph It is easily checked

that G G G G

Now supp ose a reduction exists from SAT to clique such that the graph G pro duced

by the reduction has clique numb er either l or l dep ending on whether or not the

SAT formula was satisable In other words approximation of clique number is

NPhard We claim that as a consequence any constantfactor approximation is NPhard

k k k k k

Consider G thek th p ower of this graph Then G is either l or l by increasing

k

k enough the gap in clique numb ers can b e made arbitrarily large This is what

we mean by selfimprovement

k k

Note however that G has size n sok must remain O if the ab ove construction has

to work in p olynomial time

There exist other ways of dening graphpro ducts as well

PROVING N APPROXIMATIONS NP HARD

The rapid increase in problem size when using selfimprovementmay seem hard to

avoid Surprisingly the following combinatorial ob ject sp ecically as constructed in The

orem often allows us to do just that

Denition Let n b e an integer A k n booster is a collection S of subsets of

fng eachofsize k For every subset A fng the sets in the collection that

k k

are subsets of A constitute a fraction b etween and of all sets in S where

jAj

n

k

Convention When thequantity should b e considered to b e

Example The set of all subsets of fng of size k is a b o oster with This

n n

is b ecause for any A fng jAj n the fraction of sets contained in A is

k k

n

k k

whichis The problem with this b o oster is that its size is O n hence k must

k

be O for any use in p olynomial time reductions

The following theorem app ears in AFWZ Its pro of uses explicit constructions of

expander graphs GG

Theorem For any k O log n and an n k bo oster of size polyn can

beconstructedinpolyn time

Let G b e a graph on n vertices Using anyn k b o oster for any k we can

dene a booster product of G This is a graph whose vertices are the sets of the b o oster S

and there is an edge b etween sets S S S i u v S S either u v or fu v g is an

i j i j

edge in G

Lemma For any graph G and any k n booster the clique number of the booster

k k

product of G lies between G jSj and G jSj

Pro of Let A fng b e a clique of size Gingraph G Then the number of sets

k k

from S that are subsets of A is b etween G jS j G jSj Clearly all such

sets form a clique in the b o oster pro duct

Conversely given the largest clique B in the b o oster pro duct let A b e the union of all

sets in the clique Then A is a clique in G and hence must have size at most G The

b o oster prop erty implies that the size of B is as claimed

Theorem Approximating Clique within a factor n for some is NPhard

h is due to ALM weprovea weaker result Before proving Theorem whic

ab out a related problem Vertex Cover Let VC G denote the size of the minimum

min

vertex cover in graph G

CHAPTER HARDNESS OF APPROXIMATIONS

Theorem There exist xedconstants c and a polynomial time reduction from a

SAT instance to a graph with n vertices m O n edges and degree such that if is

satisable then VC cnandif is not satisable then VC cn

min min

Pro of Consider the transformation from MAXSAT to Vertex Cover in GJ p

Thedegreeofanyvertex is no more than the maximum numb er of clauses that a variable

app ears in in this case So in particular the numb er of edges is linear in the number

of vertices Further the reduction is gappreserving in the sense of Denition with

Hence bycombining this reduction with the hardness result for MAXSAT

Theorem we get the desired result

Corollary The statement in Theorem holds for Clique as wel l except the graph

produced by the reduction may not have linear size

Pro of In general for any graph G wehave see GJ again

Gn VC G

min

where G is the complement graph of G that is a graph with the same vertex set as G but

with edgeset fu v u v Gg

G is either cn or cn Let G b e the graph of Theorem Then

Thus there is a gap of nbetween the clique numb ers in the two cases This proves the

theorem Of course the numb er of edges in G is not linear in nit is n

Nowwe prove that n approximation to clique is NPhard

Pro ofOf Theorem Take the graph G from Corollary whichwe know has clique

numb er either cn or c n for some xed c

Now construct a n log n b o oster S using Theorem bycho osing c

say Construct the b o oster pro duct of G Lemma says the clique numb er is either

log n log n

c jS j or c jSj Hence the gap is now n for some and further

jS j p oly n so this gap is jS j for some

For a history of the various results on the Clique problem see the note at the end of

the section

Wemust emphasize that for some problems including chromatic numb er LYLY

the known hardness results for a factor n do not use a selfimprovement prop erty Instead

a direct reduction is given from instances of Clique obtained in Theorem and the

following prop erty ensured by a careful reduction and b o oster construction is crucial the

graph obtained from the clique reduction is r partite for some r and furthermore in one

of the cases the clique numb er is exactly r It is op en whether a reduction exists in the

absence of this prop erty

PROVING N APPROXIMATIONS NP HARD

Finally we note that reductions from Lab elCover are easier to describ e than the ab ove

reductions eg see the Clique result in Theorem and also have the ab ovementioned r

partiteness prop erty In other words we can prove the hardness of approximating Chromatic

Numb er in a somewhat easier fashion when we reduce from Lab el Cover This is one of the

reasons whywe prefer Lab elCover as a canonical problem However reductions from Lab el

Cover prove almostNPhardness instead of NPhardness and there is ro om for improvement

there

An op en problem Are all selfimprovable problems NPhard to approximate within

a factor n for some A p ossible exception might b e the Longest Path problem

KMR for whichwe do not know a b o oster construction analogous to the one given

ab ove for Clique

MAXSATISFY

This section proves the hardness of n approximations to MAXSATISFY Denition

Pro ofof Theorem We rst show the hardness of approximations and then use

selfimprovement to sho w hardness for n approximation just as for the Clique problem

We will reduce from the vertex cover instances of Theorem to a system of N

n m linear equations of whichatmostm n VC canbesimultaneously satisable

min

This implies a gap of N in the optimum in the two cases since m O n

For eachvertex i there is a variable x and an equation x For each edge fi j g

i i

there are equations

x x x x

i j i j

Notice that at most of the equations for each edge can b e satised simultaneously

Further to satisfy of these equations x and x must takeonvalues from f g and at

i j

least one must take the value

x s are We claim that the maximum numb er of equations are satised when all the

i

Supp ose under some assignment x is not Then note that the following resetting of

i

then set x and variables strictly increases the numb er of satised equations If x

i i

if x then set x Hence the optimum setting is a setting

i i

Now notice that under any optimal assignment the set of vertices fi x g constitutes

i

avertex cover For if not then there mustbebeanedge fi j g such that b oth x and x

i j

are Thus all three equations asso ciated with this edge are unsatised Resetting x

i

to will satisfy equations asso ciated with this edge and violate one equation x

i

whichwas previously satised Thus there is a a net gain of equation whichcontradicts

the assumption that the original assignmentwas optimal It follows that the optimum

assignment satises m n VC equations min

CHAPTER HARDNESS OF APPROXIMATIONS

SelfImprovement Supp ose wehave as ab ove N equations in which the answer to

MAXSATISFY is either OPT or OPT for some Let the equations b e written

as p p p Let k T be integers to b e sp ecied later The new set

N

of equations contains for every k tuple of old equations p p asetofT equations

i i

k

P

N

k

j

y where y TThus the numb er of equations is T p

i

j

j

k

Using the fact that a p olynomial of degree k has at most k ro ots it is easily seen that

OP T

then the numb er of equations that can b e satised in the two cases is either T or

k

N OP T

k

k T Bycho osing T N we see that the gap b etween the optima

k k

k

in the two cases is approximately

Now it should b e clear that instead of using the trivial b o oster namely the set of all

subsets of size k we can use the b o oster of Theorem Write down T equations for every

subset of k equations that form a set in the b o oster Use k logN c Thus the

NPhardness of N approximation follows

Other Inapproximability Results A Survey

This section contains a brief survey of other known inapproximability results All can

b e proved using reductions from MAXSAT and Lab el Cover Original pro ofs for

many used direct reductions from Prover Round pro of systems see Chapter for a

denition The Lab el Cover problem extracts out the the combinatorial structure of these

pro of systems so we can mo dify all those reductions easily to use only Lab el Cover Wedo

not go in to further details here

The hardness results using Lab el Cover are somewhat p eculiar They dont just use the

fact that Lab el Cover is hard to approximate but the following stronger result For Lab el

Cover max version it is hard to distinguish b etween instances in which the optimum is

exactly and those in which the optimum is at most where is a large factor see

Theorem For an example of such a p eculiar hardness result see the commentafter

Theorem

Problems seem to divide into four natural classes based up on the b est inapproximability

result we can prove for them

Class I This class contains problems for whichapproximation for some xed

is NPhard The value of may dep end up on the problem All the problems known

to b e in this class are MAXSNPhard as dened in Section Consequently

Corollary implies the ab ove inapproximability result for them The following is

a partial list of MAXSNPhard problems MAXSAT MAXSAT Indep en

dent Set Vertex Cover MaxCut all in PY Metric TSP PYb Steiner

Tree BP Shortest Sup erstring BJL Multiway CutsDJP and D

Matching Kan Many more continue to b e found Since ALM there have

b een improvements in the value of the constant for whic h the ab ove results are

OTHER INAPPROXIMABILITY RESULTS A SURVEY

known to hold Currently the constant is of the order of for most problems and

ab out for MAXSAT BGLR BS

Class I I This class contains problems for which log napproximation is hard Cur

rently it contains only Set Cover and related problems like Dominating Set A re

duction in LYshows that if there is a p olynomialtime algorithm that computes

O log log n

log napproximations for these then all NP problems can b e solved in n

time The reduction can b e trivially mo died to work with Lab el Cover It is also

known that any constant factor approximation is NPhard BGLRthis result can

also b e proved using Lab el Cover

Class I I I This class contains problems for which approximation is almostNPhard

where is a large factor large factors are dened in Denition These prob

lems may b e further divided into two sub classes based up on how inapproximability

is proved for them

Sub class I I Ia This contains problems for which inapproximability results are based

up on Lab el Cover Some of these problems are Nearest Lattice Vector Near

est Co deword MinUnsatisfy Learning Halfspaces in presence of error all in

ABSS and in Section Quadratic Programming FL BR and

an entire family of problems called MAX SubgraphL Y A problem is

in MAX subgraph if it involves computing for some xed nontrivial graph

prop erty that is closed under vertex deletion the largest induced subgraph of

the given graph that has prop erty A recent result of Raz see Section

improves the inapproximability result for these somewhat they are hard upto a

log n

factor for some

Sub class I I Ib This contains selfimprovable problems for whichwe do not knowof

a b o ostertyp e construction analogous to the one given for the Clique problem

Section The way to prove these results is to rst prove that

approximation is NPhard using reductions from MAXSAT and then

use selfimprovement see Example to get a hardness result for a factor

log n

for some This set of problems includes Longest Path KMR

and the Nearest Co deword problem ABSS although for the latter a more

direct reduction is given in Section The Lab el Cover Problem is also in this

class although with a weaker notion of selfimprovement see Section

Class IV This contains problems for which n approximation is NPhard for some

The class includes Clique and Indep endent Set ALM see Section Chro

Y MaxPlanarSubgraph the problem of computing the largest matic Number L

induced planar subgraph of a graph LY MaxSetPacking and constrained ver

sions of the problems in Karps original pap er the last two results are in Zuc

All these results are provable using MAXSAT

The lone problem that do es not t into the ab ove classication is the Shortest Vector

Problem using the norm The reduction to it outlined in Section uses in an

CHAPTER HARDNESS OF APPROXIMATIONS

intimate way the structure of the pro ofsystem in FL sp ecically the fact that the

proto col involves a higher dimensional geometry of lines and p oints

Finallywemention a recent result by Ran Raz Raz see Chapter for an intro duc

tion that allows the hardness of Lab el Cover to b e proved using just the hardness result for

MAXSAT However since we need the p eculiar hardness result for Lab el Cover that

was describ ed at the b eginning of this section we need to use something stronger than just

the fact that MAXSAT is hard to approximate We need Theorem For some xed

it is NPhard to distinguish b etween instances of MAXSAT that are satisable

and instances in whichevery assignment satises less than a fraction of the clauses

In particular Razs result implies that all known hardness results except the ab ove

mentioned version of the Shortest Lattice Vector can nowbederived from Theorem

But his pro of is very complicated so it seems prudent to just retain Lab el Cover as a

canonical problem in our list

Historical NotesFurther Reading

The lure of proving b etter inapproximability results for Clique has motivated many devel

opments in the PCP area The rst hardness result for Clique was obtained in FGL

NPhardness of constantfactor approximation was proved in AS Further as observed

rst in Zuc the constan t factor hardness result can b e improved to larger factors by

using a pseudorandom graphpro duct The result of AFWZ stated in Section is the

cleanest statementofsuch a construction The NPhardness of n approximation is due to

ALM although with a dierent reduction namely the one due to FGL plus the

idea of Zuc The connection b etween MAXSNP and Clique alb eit with a randomized

b o oster construction was rst discovered in BS

A result in Zuc shows the hardness of approximating the k times iterated log of the

clique numb er for any constant k

Free bits The constant in the Clique result has seen manyimprovements ALM

BGLR FKb BS The latest improvements center around the concept of free bits

FKb This is a new parameter asso ciated with the PCP verier that is upp erb ounded

by the numb er of query bits but is often eg in the verier we constructed much smaller

Improvements in this parameter lead directly to an improvement in the value of in the

Clique result As a result of many optimizations on this parameter Bellare et al have

p

napproximates Clique recently shown that if there is a p olynomialtime algorithm that

then every NP language has a sub exp onential randomized algorithm A related result says

p

napproximation to Clique is NPhard This result also implies that Chromatic Number

p

n Fur is hard to approximate upto a factor of

Finallywemust mention older inapproximability results that are not based up on PCP

These include a result on the hardness of approximating the unrestricted Travelling Sales

HISTORICAL NOTESFURTHER READING

man Problem SG and a result ab out an entire class of maximal subgraph problems

Yan

Further reading A recent unpublished survey by the author and Carsten Lund AL

provides a more comprehensive treatment of results on the hardness of approximations than

the one given here For a listing of optimization problems according to their approximation prop erties consult CK

CHAPTER HARDNESS OF APPROXIMATIONS

Chapter

PCP Veriers that make queries

The veriers referred to in the title of this chapter are b etter known as Prover Round

interactive pro of systems These systems constitute an alternative probabilistic setting

somewhat dierent from the PCP setting in which NP has b een studied Our reason for

renaming them is that as noted bymany researchers their denition BGKW FRS

sp ecializes that of PCP

We need them to prove the hardness of approximating Lab el Cover one of our two

canonical problems in Chapter For this reason our denition is geared to a careful study

of Lab el Cover and is therefore less general than the denition of Prover Round systems

Denition A restricted PCP verier inherits all the prop erties of the verier in the

denition of PCP see the description b efore Denition In addition it has the following

restrictions

Uses a certain alphab et The pro of has to b e a string in where is the

veriers alphab et dep ends up on the input size

Exp ects two tables in the pro of The pro of has to consist of two tables T and

T A certain length dep ending up on the input size is prescrib ed for eachtable

Makes two randomlydistributed queries The verier reads the symb ol in one

lo cation each in b oth T and T That lo cation in T resp T ischosen uniformly

at random from among all lo cations in T resp T

Exp ects T to conrm what T says Supp ose we x the veriers random string

and thus also the lo cations it queries in T and T For every choice of symbol a

there is at most one symbol a such that the verier accepts up on reading a in

T and a in T

Note Condition some kind of a regularity condition do es not require that the queries

to tables T and T come from indep endent distributions For example if b oth tables have

CHAPTER PCP VERIFIERS THAT MAKE QUERIES

t entries the following way of generating queries is legal since b oth queries are uniformly

distributed in t

Pick i randomly from t Query lo cation i in T and lo cation i mod t in

T

Denition For integer valued functions rsp a language L is in RPCPr nsnpn

if there is a restricted verier which on inputs of size n uses O r n random bits an al

O sn

phab et of size that is every symbol can be represented by O sn bitsand

satises

If input x L there is a pro of such that the verier accepts for every choice of

random string ie with probability

pn

If input x L the verier accepts no pro ofs with probability more than

Example We give an example of an RPCP verier to clarify the denition Hop efully

it will also motivate the connection to Lab el Cover since the veriers program is intimately

related to the Lab el Cover instance constructed in Example

Let L bealanguageinNPWe giveaRPCPverier for L that uses O log n random

bits examines bits in T and bit in T There is a xed p ositive constant indep endent

of the input size such that if an input is not in L then the verier rejects with probability

at least

Given any input x the verier reduces it to an instance of MAXSAT by using

the reduction of Theorem Assume every variable of app ears in exactly clauses and

every clause has exactly literals this can b e arranged

The verier exp ects the tables to b e structured as follows Table T has to contain for

each clause in a sequence of bits representing an assignment to the variables of this

clause and T to contain for eachvariable in a bit representing the assignment to this

v ariable The verier picks a clause uniformly at random from among all clauses of and

avariable uniformly at random out of the variables app earing in it It accepts i the

bits given for this clause in T satisfy the clause and if this assignment is consistentwith

the assignmenttothisvariable in T

If x Levery assignmentin T fails to satisfy a fraction of the clauses Hence the

verier rejects with probability at least The fact that the the queries are uniformly

distributed follows from the sp ecial structure of Also the verier satises Condition of

the denition since it accepts i the value assigned by T to a variable conrms the value

assigned by T to that variable

The following theorem represents the b est construction ab out restricted PCP veriers

A recent result by Raz describ ed later improves it

HARDNESS OF APPROXIMATING LABEL COVER

Theorem FL For al l integers k

k k k

NP RPCPlog n log n log n

Hardness of Approximating Lab el Cover

Lab el Cover captures exactly the underlying combinatorial structure of an RPCP verier

averier yields once the input has b een xed an instance of Lab el Cover Vertices of

the bipartite graph V V E represent lo cations in the tables T and T The set of edges

corresp onds to the set of p ossible random strings the verier can use The lab els corresp ond

to the veriers alphab et The edge functions represent the veriers decisions up on reading

pairs of lab els The regularity condition on the veriers queries ensures that the graph

thus pro duced is regular as required by denition of Lab el Cover Nowwe describ e this

construction

Fix an input x This xes the length of the tables T and T in the pro of say they are

umber of choices for n n resp ectivelythe size of the v eriers alphab et say N and the n

the random string say RWe identify symb ols in the alphab et with numb ers in N and

the set of p ossible random strings of the verier with with numb ers in R

Condition on the query to T b eing uniformly distributed implies that for each lo cation

q in T

R

jfr r Rand r causes q to b e queried gj

n

Similarlyeach lo cation in T is queried by Rn random strings

Note that xing the veriers random string to r xes the lo cations it queries in T and

T say q and q resp ectivelyIfV accepts using random seed r and reading a in lo cation

q and a in q we denote this by V ra a Condition in Denition implies

that

r Ra N there is a unique a N V ra a

j n jV j Nowwe construct the instance of lab elcover The graph V V E has jV

n Vertices in V V are identied to with lo cations in T and T Let the set of edges

E b e dened as

fu v u V v V and r R using which V queries these lo cations g

Thus we can identify E and R in a onetoone fashion and jE j R The condition in

Equation implies that the graph V V E is regular

Let the set of lab els b e N For e r u v dene the partial function

e

N Nas

a a i V ra a

e

CHAPTER PCP VERIFIERS THAT MAKE QUERIES

The condition in Equation implies that is welldened as a partial function

e

The construction can b e p erformed in time which is at most p olynomial in the running

time of the verier and R N and pro duces instances of size O RN

Lemma The optimum value of the max version of labelcover on the above instance

is exactly

jE j max fPrV accepts on input x aproofstringg

Pro of In this lemma lab elling refers to an assignment of lab el p er vertex in V V

The set of lab ellings is in to corresp ondence with the set of p ossible ways to construct

the tables T T For any edge e u v E if r R is the corresp onding random

string then lab els a a assigned to u v resp ectively cover e i V ra a Thus the

set of edges covered by a lab elling is exactly the set of random strings for which the verier

accepts the corresp onding tables T T

Theorem is a simple corollary to Theorem and Lemma

Pro of Of Theorem Let L NP Let V b e the verier in Theorem when k

k

log n

For any input x construct a lab elcover instance as ab ove using V IfT the

reduction runs in p oly T time and pro duces instances of size O T Lemma implies

that if x L there exists a lab elling covering all the edges and otherwise no lab elling covers

k k kk

log n log n log T log T

more than fraction of edges Since the gap is as

claimed

Hardness of SV

Recall that the hardness result for SV used Lemma In this section wegive some

idea of how this result is provedmore details app ear in ABSS

The main idea is that Lab el Cover instances pro duced in the ab ove reduction represent

m

the following algebraic ob ject Let m be an integer and F a eld Let lines in F b e dened

as in Section

V is the set of all lines In the bipartite graph V V E constructed by the reduction

m m

passing through p oints of a xed set S F where S contains an ane basis for F

m m

The other side V corresp onds to p oints in F An edge v v isinE if the p ointofF

represented by v lies on the line represented by v The set of lab els and the edge relations

also has a related algebraic description

e

The result is proved using an expansionlike prop erty of the graph V V E whichwe

dont state here

UNIFYING LABELCOVER AND MAXSAT

Unifying Lab elCover and MAXSAT

In this section we state a recent result of Ran Raz from Raz An immediate consequence

is that the inapproximability result for Lab el Cover can b e derived from the inapproximabil

ity result for MAXSAT Before this developmentweknewhowtoprove the hardness

of Lab el Cover only by using the result in FL a result that seemed indep endent of the

PCP Theorem

Raz proves the socalled paral lel repetition conjecture a longstanding conjecture from the

theory of interactive pro ofs We describ e only the consequence for restricted PCP veriers

Let V be any restricted verier using alphab et Let us dene V k the k th product of

k k

V asfollows Verier V k exp ects tables T T of size jT j and jT j resp ectively where

the set of lo cations in T is in to corresp ondence with the set of all p ossible k tuples of

k

lo cations in T and table T b ears a similar relation to T The alphab et of V k is

Verier V k p erforms k indep endentrunsofV except it bunches up the sequence of k

queries to T into a single query to T and the sequence of queries to T into a single query

to T It reads the k tuples of symb ols from these lo cations in T T and accepts i all k

runs of V would have accepted

For any xed input if there exist tables T T which V accepts with probability

then there clearly exist tables T T which V k accepts with probability

Claim Raz For a given input supp ose verier V accepts every pair of tables T T with

T probability less than p where p Then V k will accept every pair of tables T

ck

log N

with probability less than p where c is xed p ositive constant dep ending only up on

the verier and not the input size and N is the numb er of strings that V could giveas

answers

The pro of of this claim is very complicated

As an immediate consequence we can improve Theorem

Theorem For al l positive increasing functions k of the input size

NP RPCPk nlogn k nkn

Pro of Example implies that NP RPCPlog n Take the O k nth pro duct

of that verier Since the numb er of p ossible answers for the original verier is O actually

ck n

probability of incorrect acceptance b ecomes Hence NP RPCPk nlogn k nkn

Wesaw in Lemma that Lab el Cover captures ex Implications for Lab el Cover

actly the combinatorial structure of RPCP veriers Razs result implies that Lab elCover

CHAPTER PCP VERIFIERS THAT MAKE QUERIES

maxversion is selfimprovable under the following op eration

Given an instance V V EN of Lab el Cover max version dene its k th pro duct

k k k k

as follows The underlying graph is V V E where V denotes cartesian pro duct of

k

the set V with itself k times and the set of lab els is N The set of lab els is viewed

k

actually as N thesetof k tuples of lab els Thus a lab elling must now assign a k tuple

k k k

of lab els from Ntovertices in V V The new set of edgefunctions denoted

are dened as follows Let e be a k tuple of edges e e in the original graph Then

k

dene

k k k

a a a a a a

e

i the lab els satisfy

j j

j k a a

e

j

Razs result implies that if the value of the optimum in original instance is at most p

ck

then the value of the optimum in the k th pro duct of the instance is p In other words

so long as the original optimum is some constant less than the new optimum decreases

exp onentially as we increase k

We already gave a reduction from MAXSAT to Lab el Cover max version that

shows the hardness of approximation As a consequence of the ab ove result ab out

selfimprovement this reduction can b e mo died using the ab ovenotionofk th pro duct

to give a hardness result for approximation within large factors namely the hardness result

in Theorem as well

This leaves a tantalizing op en problem Is there a b o osterlike construction in the sense

of Section for Lab el Cover which can prove that n approximation is NPhard for some

xed A recent result byFeige and Killian suggests that such b o osters do not exist

Chapter

Applications of PCP Techniques

The techniques used to prove the PCP Theorem have found other applications b esides the

hardness of approximations This chapter surveys some such applications

Many applications rely up on stronger forms of the PCP Theorem We recall the setting

of the PCP theorem and then describ e alternate settings that o ccur in its stronger forms

The Basic Situation A probabilistic p olynomial time verier needs to decide whether its

input x satises a predicate where is computable in nondeterministic p olynomial

time The verier is given random access to a remote database that purp orts to show that

x In verifying the database the verier has to minimize the following two resources

the numb er of bits it examines in the database and the amount of randomness it uses

What minimum amounts as a function of input size n of the two resources do es it need

in order that the database have a reasonable chance of convincing it if x and a

negligible chance otherwise

According to the PCP theorem upp erb ounds on the resources are O log n random bits

and O query bits Further if these amounts could b e lowered anyfurtherthenPNP

AS Nowwe consider mo dications of the Basic Situation The rst mo dication

considers the complexity of constructing the database

Situation In the Basic Situation supp ose the input x is a CNF formula and the pred

icate is dened to b e i x is satisable Minimize the time required to construct the

database assuming the database constructor is already provided with a satisfying assign

ment Also giveaway to construct the database such that the verier can recov er any

desired bit of the satisfying assignment as eciently as p ossible

Situation Similar to the Basic Situation except that the predicate is computable

in nondeterministic time tn where tn p olyn Make the verier as ecientas

CHAPTER APPLICATIONS OF PCP TECHNIQUES

p ossible

Strong Forms of the PCP Theorem

We state stronger forms of the PCP Theorem that deal with the ab ovetwo situations

Theorem Strong Form In Situation the database constructor runs in time

polynFurther it can construct the database in a way such that it contains an encoding

of the assignment which the verier can decode bitbybit Decoding a bit of the assignment

requires examining O bits in the database The decoding algorithm is probabilistic

Pro of Sketch Recall the constructive nature of the pro of of the PCP Theorem To obtain

a database from a satisfying assignment one needs to construct the p olynomial extension of

the assignment and the tables required by the various comp onent pro cedures Lowdegree

Test the sumcheck the pro cedure that aggregates queries and so on A quick lo ok at the

descriptions of all these tables shows that they can b e constructed in time p oly n Also

the Comp osition step of Chapter is also quite constructive in nature Hence the rst half

of the Theorem is proved

Wegive an indirect pro of of the second half sp ecically the fact that the verier can

recover bits of the assignment from such a database A direct pro of using prop erties of

the p olynomial extension is also p ossible Wedonotgive it here

Recall that in our construction the database has two parts The rst contains an enco d

ing of the assignment The enco ding uses p olynomial extensions of bitstrings The second

tains information showing that the assignment satises the formula Now supp ose the con

verier wants to b e convinced that the assignment satises a second NPpredicate Then

only the second part needs to b e changed the database constructor just adds information

showing that the string enco ded by the rst part also satises

In particular the following predicate is computable in p olynomial time and therefore

is an NPpredicate YES on a string i the ith bit of the string is Denote this predicate

by

i

The verier stipulates that the second part of the database contain the following ad

ditional information for each bitp osition i in the assignment if the ithbitisaproof

that the assignment satises and otherwise a pro of that the assignment satises the

i i

negation of

i

If the verier feels the need to deco de the ith bit of the assignment it can check using

as the case may b e If the check succeeds then the ith O queries a pro of for or

i i

bit has eectively b een recovered

Polishchuk and Spielman PS have further strengthened Strong Form They

show that the size of the database is just O n where n is the running time of the

STRONG FORMS OF THE PCP THEOREM

nondeterministic computation that computes and is any p ositive constant

Next we state the result ab out Situation

Theorem Strong Form BFLS ALM In Situation the verier needs

O log tn random bits and O query bits Its running time is polylog tnn where

n is the size of the input xFurther the database continues to have the decoding property

mentionedinStrong Form

Pro of Sketch First we describ e the trivial mo dication to the pro of of the PCP Theo

rem that has all the right prop erties except the verier runs in time p olytn instead of

p oly log tnn

Using the obvious extension of the Co okLevin theorem do a reduction from predicate

which is computable in nondeterministic time tn to instances of SATofsizepolytn

Then use the verier of the PCP Theorem on such instances of SAT It uses O log tn

random bits queries O bits and runs in time p oly tn

Nowwe describ e howtomake the verier run in time p oly log tnn First notice

that the verier of the PCP Theorem uses p oly tn time solely b ecause of Lemma the

algebraic view of SAT All other subpro cedures used to dene the verier contribute only

time p oly log tn to the running time For instance the technique of aggregating queries

Lemma involves two simple algebraic pro cedures that run in time p olyd m where

m d are resp ectively the degree of and the number of variables in the p olynomial extensions

b eing used A similar statement holds for the sumcheck Recall that in all those cases m

and d are p oly log tn

Toimprove Lemma we need an idea from BFLS Use Levins Theorem Lev

to reduce the decision problem on input x to an instance of Tiling The Tiling problem asks

for square unitsized tile to b e put on eachvertex of a K K grid such that each tile is

one of a set of p ossible typ es and the set of tiles around each gridp ointlooksvalid The

rst line of the grid is already tiledthe nal tiling has to extend this The size K of the

desired tiling is p oly tn the num ber of allowable tiletyp es is c and the number of valid

neighborhoods allowed in the tiling is d where c and d are some constants indep endentof

n Levins reduction runs in time p oly n logtn An aside Levins Theorem follows

easily from the tableau viewp oint describ ed earlier in Chapter

Mo dify the verier of Lemma to work with the Tiling problem instead of SAT It

now exp ects the database to contain a p olynomial extension of a valid tiling Mo dify the

ideas of Lemma to pro duce an algebraic view of the tiling problem There is no need

to write the functions s etc of that Lemma now Instead there is a more direct way

j j

to write an algebraic formula that represents the set of valid neighb orho o ds We dont give

further details here

This allows the verier to run in time p oly t logn

CHAPTER APPLICATIONS OF PCP TECHNIQUES

Note This theorem was proved in BFLS in a somewhat weaker formtheir v erier

required p oly log tn random bits and as many query bits

The Applications

This section describ es various applications of the ab ove Strong Forms

Exact Characterization of Nondeterministic Time Classes

This application uses Strong Form

Let Ntimetn for tn polyn denote the set of languages computable in nondeter

ministic time tn The following is a restatementofStrongForm

Theorem Ntimetn PCP log tn tn polyn

This characterization generalizes the result MIP NEXPTIME in BFL which can

polyn

b e equivalently stated as Ntime PCPp oly n p olyn It also generalizes the

GL whose result could b e interpreted as saying that Ntimetn work of BFLS F

PCPp olylog tn p olylog tn

Transparent Math Pro ofs

In this section weshow that formal pro ofs in rst order logic can b e checked very eciently

by a probabilistic algorithm The algorithm needs to examine only a constantnumber of

bits in the pro of This application of Strong Forms and was suggested by Babai et al

BFLS

Wehave to restrict ourselves to reasonable axiomatic systems over rst order logic

These are axiomatic systems for whichaTuring Machine can check pro ofs in p olynomial

time More sp ecically given any alleged theorem T and a claimed pro of for it the Turing

Machine can determine in time p oly jT j jj whether or not the pro of is correct in the

system Most p opular axiomatic systems for instance the ZermeloFraenkel axioms are

reasonable They involve a constructible set of axioms and induction rules and checking

whether each step of a given pro of is deduced correctly from preceding steps involves a

simple syntactic check

For a reasonable axiomatic system let the term pr oofchecker refer to anyTuring Ma

chine probabilistic or deterministic that can check pro ofs in that system The following

theorem shows that the pro ofchecker can b e made quite ecient

THE APPLICATIONS

Theorem For any reasonable axiomatic system thereisaprobabilistic proofchecker

Turing Machine that given a theoremcandidate T andaproofcandidate runs in time

polyjT j log jj examines only O bits in the proof and satises the fol lowing

If theoremcandidate T has a proof in the system of size N then there exists a string

of size polyN such that the checker accepts T with probability

If T has no proofs the checker rejects T with probability at least where is

any string whatsover

The fol lowing two transformations can bedoneinpolynomial time a Transforming

any string that is accepted by the verier with probabilityatleast in partic

ular implies that T in this case must beatheorem to a proof in the axiomatic

system b Transforming any valid axiomatic proof to a proof that is acceptedbythe

checker with probability

Notes Condition shows a p olynomialtime equivalence b etween provability in the

classical sense in the axiomatic system and provability for our verier The advantage of

our system is that the running time of the verier grows as a p olynomial in the logarithm

of the pro ofsize Also only a constantnumb er of bits are examined in the pro ofstring

Pro of Use the wellknown connection b etween mathematical pro ofs and nondeterministic

Turing Machines For every reasonable system by denition there is a nondeterministic Tur

ing Machine that accepts the language fT n T is a theorem that has a pro of of size n g

Note the machine guesses a pro of of length n and then checks in p oly n time that it is

correct Using Theorem parts and of the theorem statement follow

F urther part a follows from the fact that the database is eciently constructible

Part b uses the deco ding prop erty mentioned in Strong Forms and Given a

database that is accepted with high probability the verier can deco de the original non

deterministic guess in this case a pro of of theorem T bit by bit

A philosophical problem needs to b e p ointed out here It may app ear that our con

struction has simplied the checking of math pro ofs since our checker needs to examine

only O bits in the pro of However in another sense the new checkers program is quite

complex At least the waywe proved the ab ove theorem the checker must write down a

SAT formula or set of acceptable tile typ es that expresses an axiomatic system This is

not an easy task and certainly not so for humans

Checking Computations

Strong Form also enables constructions of certicates for all nondeterministic compu

tations The certicates length is p olynomial in the running time of the computation

CHAPTER APPLICATIONS OF PCP TECHNIQUES

and checking it requires examining only O bits in the certicate This application of

PCPtyp e results was also suggested in BFLS

The situation wehave in mind is a restatement of Situation Supp ose a user has a

nondeterministic program P and an input x The user would liketoknow if the nondeter

ministic program has an accepting computation on the input

A database constructor with unlimited computational p ower say a sup ercomputer

can go through all p ossible branches of the nondeterministic program P nd an accepting

branch if one exists and change it into a database that the verier can checkby examining

only O bits in it This database can therefore b e viewed as an easily checkable certicate

that P accepts x

Of course as a sp ecial subcase the ab ove construction also applies to deterministic

programs More imp ortantlyinmany cases a complicated deterministic program can b e

replaced by a trivial but equivalent nondeterministic program Hence a certicate that the

nondeterministic version accepts is also a certicate for the more complicated deterministic

version We illustrate this p oint with an example

Example Supp ose we are given a number N There is a wellknown deterministic

O log log n

algorithm that checks in time n whether or not N is comp osite where n log N

But this algorithm is complicated and therefore wemighthav e reasons to mistrust any

software that claims to implementit

Now consider the following nondeterministic program Guess a number between and

N and accept i it is a nontrivial divisor of N This program runs in time O n orsoand

has an accepting branchiN is comp osite

Hence certicates for the nondeterministic program are much shorter and simpler

than those for the deterministic program

Note however that nding a certicate for the nondeterministic computation is equiv

alent to nding a divisor of the number N which is the celebrated factoring problem The

p

n

Hence by insisting that it wants to see only b est known algorithms for it run in time

certicates for the nondeterministic program the verier has made the task of the certicate

constructor much more dicult

Acaveat is in order Our technique do es not representawaytochecksoftware We

assumed throughout that software for the nondeterministic program P is reliable

Micalis Certicates for VLSI Chips

Micali Mic notes that the ab oveideaofchecking computations since it assumes that

software is reliable makes more sense in the context of checking VLSI chips Chips are

designed carefully in other words they are programmed with reliable software but the

fabrication pro cess mightintro duce bugs in the hardware Instead of testing the chip

THE APPLICATIONS

exhaustivelywe could require by adding the necessary hardware to the chip that it always

give us a certicate that its computation was done according to sp ecications However

inputoutput from chips is slow so eciency would b e terrible if we use the large certicates

describ ed ab ove since their size p olynomial in the running time of the computation Micali

shows how to hash down the certicate to p olylogarithmic size in a cryptographic fashion

By cryptographic hashing we mean that although it is p ossible to pro duce a fraudulent

certicate doing so takes a lot of time which the chip do es not have

Micalis idea works also with the weaker result ab out checking nondeterministic compu

tations that app ears in BFLS but eciency is b etter using the PCP theorem Also he

p oints out an interesting extension of the class IP Section

Characterization of PSPACE Condon et al

Condon Feigenbaum Lund and Shor CFLS give a new characterization of PSPACE

the set of languages accepted bymachines that run in p olynomial space Their result

PSPACE RP C D S log n uses the Strong Form of the PCP Theorem

RP C D S r n is a class of languages dened using a debate between two players

ers alternate in and where is just a source of indep endent random bits The play

setting down strings of bits on a debatetap e whichischecked at the end by a p olynomial

time verier who accepts or rejects The verier has random access to the debate tap e

Language L is in RP C D S r n if the verier on inputs of size n uses O r n random

bits examines O bits in the tap e and satises For all inputs in L there is an player

such that the debate is accepted with probability and for all inputs not in L the debate

is rejected with probability at least irresp ective of what the player do es

Shamirs result Sha implies that PSPACE RP C D S polyn Condon et al

obtain their improvementby stipulating that the player write down at the end of the

debate a certicate that is the database referred to in Strong Form showing that

the verier would have accepted the debate Recall that this means that the certicate

contains an enco ding using p olynomial extensions of the debate lots of other tables and

so on Strong Form implies that the verier can check this certicate by examining only

O bits in it Only one task remains howtoverify that the debate enco ded in sucha

certicate is the actual debate that to ok place In the CFLS pap er it is shown using

Shamirs result that the verier only needs to deco de O bits from the enco ded debate

the deco ding requires reading only O bits according to Strong Form and check them

o against the corresp onding bits in the actual debate

Condon et al use their characterization of PSPACE to show the hardness of approximat

ing some PSPACEhard problems

CHAPTER APPLICATIONS OF PCP TECHNIQUES

Probabilistically Checkable Co des

Recall the denition of a code from Section For nowwe restrict attention to co des on

the alphab et f g

Denition A family of Probabilistical ly Checkable Codes is a family of co des ie

with one co de for each co deword size whose minimum distance is some constant like

min

indenp endent of the co deword size Further the family has an asso ciated p olynomial

time randomized checker Givenaword of size n the checker uses O log n random bits

examines O bits in the word and has the following prop erties

If the word is a co deword then the checker accepts with probability

If the word is not close then the checker rejects with probability

min

Prop osition ALMSS For some c thereexistprobabilistical ly checkable codes

c

n

n

in strings f g that contain

We can use stronger versions of Denition none of which aect the validity of the

previous prop osition the co des have asso ciated co dingdeco ding algorithms that run in

p olynomial timethe v erier runs in time p oly log n instead of p oly na probabilistic

deco ding of any bit in a co deword requires examining only O bits in the word

min

and so on

We will not prove Prop osition here The construction uses techniques from the pro of

of the PCP Theorem Recall that the pro of of the PCP theorem consisted of a sequence of

enco ding schemes for assignments see Figure for a birdseye view The same sequence

of schemes works also for enco ding bitstrings

Kilians Communicationecient ZeroKnowledge Arguments

We will not attempt to give an exact denition of zeroknowledge arguments here Roughly

sp eaking the situation involves two parties b oth of whom run in probabilistic p olynomial

time and a SATformula One party has a satifying assignment to the formula and

wants to convince the other of this fact in suchaway that the other party do es not learn

even a single bit in the satisfying assignment

Proto cols for doing this are known but they require to o muchcommunication b etween

the parties Kilian Kil shows how to reduce the communication requirement His idea

heckable database of is to hash down in a cryptographic fashion the probabilistically c

Strong Form Recall that Micalis idea is similar

He needs something more than the Strong Form sp ecically the fact proved in

BFLS PS that the database given by Strong Form has size n where n is the

size of the SATformula

THE APPLICATIONS

Khanna et als Structure Theorem for MAXSNP

The class APX contains optimization problems which can b e approximated within some

constant factor in p olynomial time An op en question p osed in PYwas is APX con

tained in MAXSNP The answer turns out to dep end up on how MAXSNP is dened It

app ears that the denition intended in PY although never stated explicitly thus was

that every problem that has an approximationpreserving reduction to MAXSAT should

b e considered to b e in MAXSNP With this denition MAXSNP equals APX KMSV

The pro of is not to o dicult the PCP Theorem can b e used to give a reduction from any

APX problem to MAXSAT

The Hardness of nding Small Cliques

Given a graph of size nhow hard is it to determine whether or not it has a clique of size

log n

dlog ne The trivial algorithm based up on exhaustive search takes O n time Do es a

p olynomial time algorithm exist

This question was raised in PYa and is also related to the study of xed parameter

intractability DF

RecentlyFeige and Killian FKa related this question to another question ab out

traditional complexity classes They show that if a p olynomial time algorithm exists then

tn

Ntimetn Dtime for some small p ositive constant

They use the version of Strong Form due to PS see the note following Strong

Form Their idea is to do a reduction to clique using this strong form and then apply a

b o osterlike construction as in Section

e shown the same result without using the PCP Theorem Note Noam Nisan has sinc

CHAPTER APPLICATIONS OF PCP TECHNIQUES

Chapter

Op en Problems

We list twotyp es of op en problems The rst typ e contained in Section concern the

hardness of approximation The second typ e in Section concern PCP techniques

The latter term is an umbrella term for ideas like programchecking selftestingcorrecting

random selfreducibili ty and algebraic prop erties of p olynomials in other words the in

gredients of the new results in complexity theory

Finally in Section we discuss the following op en problem Do es the PCP Theorem

have a simpler pro of

Hardness of Approximations

There are two ma jor op en areas here First to show the inapproximability of problems for

which this is not known Second to improve existing inapproximability results

Proving hardness where no results exist

Despite great progress on proving hardness results for problems like clique chromatic num

b er set cover etc similar results elude us for the following problems

n o

P

Shortest Lattice Vector Given an integer lattice b Z the problem is to

i i i

i

nd a nonzero lattice vector whose norm is the smallest see Section It is op en

even whether exact optimization is NPcomplete The b est factor of approximation

O n log n

currently achievable in p olynomial time is Sch The b est inapproxima

bility result says that the version of the problem using the norm is almostNPhard

log n

upto a factor ABSS also Section Improving the result to a

p

n will prove hardness of the version as well since the optima in the two factor of

p

norms are related by a factor of n

CHAPTER OPEN PROBLEMS

The rigid structure of the lattice makes it dicult to come up with reductions The

geometric arguments used in the result may provide a hintonhow to pro ceed

Euclidean TSP This is the version of the Travelling Salesman Problem where p oints lie

in a Euclidean space Exact optimization is NPhard even when all p oints lie in a

plane GGJPap The b est factor of approximation currently achievable in

p olynomial time is Chr No inapproximability results exist It is known

however that Metric TSP the version in which the underlying space is a p ossibly

nonEuclidean metric space is MAXSNPhard PYb so as a consequence of

Theorem nding approximations is NPhard

The NPhardness of exact optimization in the Euclidean case is proved using the

NPcompleteness of planarSAT But MAXSAT restricted to planar instances has a

p olynomial time approximation scheme AKM which rules out the use of planar

SAT in proving hardness of approximation

Instances of SAT pro duced by the current pro of of the PCP theorem represent high

dimensional ob jects namely a geometry involving the p oints and lines of a log n

dimensional spacesee c hapter and also Lemma It seems dicult to do

reductions from these to planar TSP but p erhaps a reduction is p ossible to higher

dimensional TSP

Edgedeletion typ e problems This group of problems prop osed byYannakakis Yan

consists of any problem stated in the follo wing form for some prop erty T of graphs

that is closed under edgedeletion for example disconnectedness Remove the min

imum p ossible numb er of edges so that the remaining graph satises T

The following are twowellknown examples Graph bisection T there is a set of

connected comp onents which together include exactly nvertices and Minimum

feedback Arc Set T acyclicity

A series of pap ers starting with LR use owtechniques to approximate many

edgedeletion problems within factors likelogn or p oly log n For graph bisection

the approximate solution pro duces not an exact bisection but a split of the

vertex set

No go o d hardness results are knownin man y cases like graph bisection not even

MAXSNPhardness is known The following clean problem seems to b e a go o d can

didate to prove hard to approximate Given an instance of MAXSAT delete the

smallest numb er of clauses so as to make it satisable To see why this ts the edge

deletion framework and also an approximation algorithm for a related problem refer

to KARR GVY

Improving existing hardness results

There are twoways to improve existing hardness results First to base the result on

the assumption P NPmany results are currently based up on stronger assumptions

HARDNESS OF APPROXIMATIONS

Second to show that approximation within larger factors is also hard For most problems

there is a large gap b etween those factors of approximation known to b e NPhard and those

achievable in p olynomial time

Basing results on P  NP

One shortcoming of many existing hardness results is that they are based up on complexity

assumptions stronger than P NP This was also originally the case with the clique

problem FGL but as a result of AS we are now able to base that hardnes result

up on P NP So there is hop e that the same may b e p ossible with other problems

The reason for resorting to strong complexity assumptions is that many hardness results

involve reductions from Lab el Cover for a denition see Chapter Approximating Lab el

log n

Cover upto a factor of is known to b e only almostNPhard instead of NPhard

Approximating within constant factors is NPhard however Thus reductions from Lab el

Cover also prove almostNPhardness of large factor approximation On the other hand a

pro of that n approximation to Lab el Cover is NPhard for instance would immediately

make Lab el Cover much more useful as a canonical problem One way to provesuch a result

is to prove the following conjecture for a denition of RPCP see Denition

Conjecture For al l k O log n

NP RPCPlog n k k

The conjecture is true for the case k O FKb In general if conjecture

k

holds for any given k then approximating Lab el Cover within a factor is NPhard

Approximation Problems for which Conjecture implies NPhardness

SetCover upto a factor O log n For this result it suces that the conjecture b e true

for k O log log nLY or even that the conjecture b e true with O tables

instead of just BGLR

Lattice and other Problems An entire group of problems involving lattices systems

of linear equations and inequalities from section upto a factor of n for some

small

Vertex Deletion problems An entire family of problems upto a factor of n LY

Avery recent result byFeige and Killian suggests that the conjecture is hard to prove

CHAPTER OPEN PROBLEMS

In addition the longest path problem is currently known to b e NPhard to approximate

within any constant factor Assuming SAT cannot b e solved in sub exp onential time the

log n

factor can b e pushed up to KMR It is not clear how to base the second result

on P NP even if conjecture is true since the known reduction from MAXSAT

inherently blows up the size of the instance b eyond any p olynomial See also the note in

Section

Improving the Factors

The other waytoimproveknown hardness results would b e to prove that approximation is

hard even for larger factors The following is a list of some of the signicant problems

Clique and Indep endent Set Assuming NP problems cannot b e solved probabilistically

p

napproximation to Clique and Indep endent Set is imp os in sub exp onential time

sible in p olynomial time BS The b est p olynomialtime algorithms achievea

factor np oly log n Can weprove hardness for a factor n

Chromatic Number i The discussion from indep endent set applies to chromatic num

b er to o ii Given a colorable graph what is the least numb er of colors with which

we can color it in p olynomial time The b est algorithms use n colors KMS

The b est hardness result says that at least colors are needed if P NP KLS

Can b e improved to n for some small enough asmany b elieve A result by

A Blum shows that if coloring colorable graphs with even p oly log n colors is hard

then approximating Clique within a factor n is hard where is an arbitrarily small

p ositive constant A relevantfactherewhich Blum also uses in his result is that

chromatic numb er is selfimprovable as shown in LV

Classic MAXSNPhard problems These include vertex cover TSP with triangle in

equality MAXSAT etc The b est p olynomial time algorithms achieve approximation

factors of and resp ectivelyWe only knowthatapproximations

are hard for Can the hardness result b e improved A surprising development

in this area is the result of Go emans and Williamson GW where it is shown that

MAXSAT and MAXCUT two other MAXSNPhard problems with with classic

approximation and approximation algorithms resp ectively can b e approximated

within a factor b etter than

Obtaining Logical Insightinto Approximation Problems

In Section wegave a survey of known inapproximability results Problems seem to fall

into four main classes according to the factor of approximation which is provably hard to

achieve in p olynomial time

Why do problems fall into these four classes Is there a metho d at least at an intuitive

level for recognizing for a given problem which of these classes it falls in

OPEN PROBLEMS CONNECTED WITH PCP TECHNIQUES

No satisfactory answerstosuch questions are known any class except the rst Recall

that this class contains only MAXSNPhard problems

Further the edgedeletion problems seem to form a class of their own It would b e nice to

nd a complete problems in this class in other words a problem whose inapproximability

implies the inapproximabilityoftheentire class

Op en Problems connected with PCP techniques

The class PCPr nrn for r olog n The class PCPolog nolog n is contained

in NP but do es not cotain any NPcomplete problems if P NP AS A re

sult in FGL shows how to reduce the question of memb ership in a language in

O r n

PCPr nrn to an instance of Clique of size So the memb ership problem

seems to involve limited nondeterminism PYa but it is op en if there is an exact

characterization lurking there

Size of the pro of In the pro of of the PCP theorem what is the minimum size of the

pro of needed for SATformulae of size n In our pro ofs wew ere sloppy with the

numb ers but the b est size that achievable is n Sud A tighter construction

PS achieves size n Canweachieve size n p oly log n The size is imp ortant

for cryptographic applications Kil

Size of probabilisticall y checkable co des These co des were dened in Section

The b est construction PS achieves a constant minimum distance say

min

and enco des n bits with n bits Can the size of the enco ding b e reduced to

n p oly log n or b etter still to O n Shannons theorem says the size would b e O n

if we didnt imp ose the probabilistic checkability condition MS

Improving the lowdegree test Do es the lowdegree test work even for high error rates

In other words is Theorem in Chapter true even when the success rate is less

than say and jFj p oly d

Selfcorrection on p olynomials Can p olynomials b e selfcorrected for denition see

BLR in the presence of high errorrates A selfcorrector is a probabilistic pro

m

gram that is given a rational number p and a function f F F that is

pclosetoF x x The selfcorrectors task given an arbitrary p oint b

d m

m

in F is to pro duce g b in time p oly d p log jFj where g F x x isany

d m

p olynomial that agrees with f in a fraction p of the p oints Selfcorrectors are known

to exist for all p such that p for some The case p is op en

In general we seem to b e missing some crucial insightinto pclose functions for

p which is p ossibly why the previous problem is also op en

e Applications to cryptography Do the algebraic techniques of the PCP results hav

applications in cryptography and program checking For instance a key result in

cryptography is the construction of a hardcore bit for pseudorandom generation

CHAPTER OPEN PROBLEMS

GL At the heart of this result is a simple selfcorrector in the sense of BLR

for linear functions over GF these functions were also encountered in Section

Nowwe knowofmuch stronger results ab out p olynomials eg those in Chapter

What are the applications if any for cryptography One p ossible application could

b e pseudorandom generation in parallel a longstanding op en problem

Implications for complexity classes Wenowknow of PCPlikecharacterizations not

only for nondeterministic time classes Section but also for PSPACE and PH

see Section What are the implications of these new characterizations if any

Can new characterizations b e given for any other complexity classes say P or EXP

TIME

Do es the PCP theorem havea simpler pro of

We mentioned earlier while describing the overall picture of the pro of of the PCP theorem in

Section that central to our pro of of the PCP Theorem is a new way to enco de satisfying

assignments The enco ding uses p olynomials represented by value instead of by co ecient

Sp ecically it uses the fact that the set of lowdegree p olynomials when represented by

value form a co de of large minimum distance see Section The denition of the

enco ding is not simpleit in volves many steps where each step consists of dening a new

verier The veriers are comp osed at the end to give a nal verier and a nal enco ding

Figure gives a birdseye view of this pro cess

Must every pro of of the PCP Theorem b e this complicated There is no easy answer

In fact it is not even clear why the pro of needs to involve an enco ding pro cess In Claim

given b elow w e try to giveintuition whyFurther we try to explain at an intuitive level

why the enco ding must make use of errorcorrecting co des as ours did We actually argue

something stronger that the enco ding must use an ob ject very muchlike a probabilistically

checkable co de see Denition

We state two claims the rst rigorous and the second intuitive

The rst Claim makes the following intuition precise If a PCPlog n verier accepts

one string but rejects the other then the two strings must dier in many bits

Claim Every PCPlog n verier V hasanequivalent uniform forminwhichithas

the fol lowing property Thereisapositive integer C such that for every two proofstrings

and and any input x

j PrV accepts on input x PrV accepts on input xj C

where is the Hamming distancebetween and as denedinSection

Pro of Sketch Note that the verier will not give dierent answers on two strings unless

Our argument in this section is somewhat imprecise It is made more precise in Aro

DOES THE PCP THEOREM HAVE A SIMPLER PROOF

if it queries a bitp osition in which they dier In other words for any strings and

an upp er b ound on the quantity

j PrV accepts on input x PrV accepts on input xj

is given byPrV queries a bitp osition in which and dier Weshowhow to make

the queries of the verier uniformly distributed so that the ab ove upp er b ound can b e

given in terms of the distance b etween and

Let K polyn b e the size of the pro ofstring and q O b e the numb er of bits the

verier queries in a single run Consider the following probability distribution

p Pr V queries bitp osition i in the provided pro of in its rst query

i

We rst mo dify the verier so that this distribution b ecomes uniform and furthermore is

identical for all q queries Important The queries could becorrelated just their distribution

is uniform Then the claim follows since if

Prthe rst bit queried in and is dierent p

then

j PrV accepts on input x PrV accepts on input xj qp

Finally the uniformity of the ab ove distribution implies p By substituting

C q the claim is proved

Nowwe explain the mo dication to V to achieve uniformity First make the ab ove

distribution identical for all q queries by randomly scrambling the order in which V makes

its q queries recall V queries the pro of nonadaptively To do this the verier requires a

random p ermutation of a set of C elements in other words O random bits Now pick

a suciently large integer R p oly n Mo dify V so that it exp ects a pro of of size RK

containing bRp c copies of the ith bit of the old pro of

i

Whenever V reads the ith bit in its pro of the mo died verier will cho ose twobits

randomly from among all bRp c copies of this bit check that they have the same value and

i

if so use that common value

The distribution induced by this query pattern is almost uniform when

R Number of choices for the original veriers random string

Claim Intuitive Whenever we construct a PCPlog n verier V for SAT we

must implicitly dene for every SAT instance a onetomany map from assignments

for tosetsofproofstrings such that

CHAPTER OPEN PROBLEMS

If assignment A satises thereisaproofstring A such that

PrV on input accepts

If any proofstring satises

PrV on input accepts

then has a unique preimage under Further the preimage is a satisfying assign

ment

Justication The construction of suchaV is an implicit pro of of NPcompleteness of

the following language

L is a CNF formula and st PrV accepts on input

To see that this problem is in NP lo ok at the pro of of Corollary

All known NPcompleteness results map witnesses to witnesses in a onetomany fashion

We quote this as intuitive justication for our claim

Supp ose we b elieve in Claim Then weshowhow to use any uniform form PCPlog n

verier for SATtodeneacodeover the alphab et f g that is quite close to b eing

probabilistically checkable Fix the input Dene the co de as

f Prthe verier on input accepts g

The checker for this co de is the uniform form verier It accepts all co dewords with

probabilityConversely if it rejects anyword with probability more than then since

it examines only O bits in the word Claim implies that the word is not close for

some small enough constant

We cannot rigorously prove that the co de has minimum distance c for some xed c

indep endentof However we can prove it under the assumption that the pro ofstrings

be in the co de are in onetoone corresp ondence with satisfying assignments For let

two co dewords representing dierent satisfying assignments whose mutual distance is less

than cLet be anyword that that agrees with b oth of them in c fraction of p oints

Then the probability that the verier accepts is by Claim close to By Claim

must b e deco dable to a unique preimage This contradicts the assumption that

represented distinct satisfying assignments

The wellknown randomized reduction from NP to UNIQUESAT VV do es not map every witness

to a witness We do not consider this reduction a counterexample b ecause it succeeds with probabili ty less

than n which in the PCP context is negligible A deterministic or lowerror version of this reduction

would b e a valid counterexample though

DOES THE PCP THEOREM HAVE A SIMPLER PROOF

Conclusion Wehave tried to argue that constructing probabilistically checkable co des

PCCs is a precondition to proving the PCP Theorem Currently the only way to con

struct such co des involves a small mo dication of the pro of of the PCP Theorem just take

out the sumcheck part from the rst step of Figure We feel that a simpler construction

of PCCs will very likely yield a simpler pro of of the PCP Theorem

Finally note the following machineless analogue of a PCC in the spirit of Lemma

which replaced a log n restricted verier with a CNF formula Any PCC yields a

CNF formula in n variables such that for some constants c d i the set of satisfying

c

close assignments form a co de with minimum distance c ii the set of words that are not

satisfy fewer than d fraction of clauses Ideallywewant the formula to also satisfy

n

the condition that the numb er of satisfying assignments is at least for some

Currentlywe do not knowhow to prove the existence of such CNF formulae except

as a bypro duct of the PCP Theorem Hence an alternative pro of of existence saya

nonconstructive pro of would also yield fresh insightinto PCCs

CHAPTER OPEN PROBLEMS

Biblio graphy

ABSS S Arora L Babai J Stern and Z Sweedyk The hardness of approximate optima

in lattices co des and linear equations In Proc th IEEE Symp on Foundations of

Computer Science pages

ADP G Ausiello A DAtri and M Protasi On the structure of combinatorial problems and

structure preserving reductions In Proc th Intl Col l on Automata Languages and

Programming

ADP G Ausiello A DAtri and M Protasi Structure preserving reductions among convex

optimization problems Journal of Computer and System Sciences

AFK M Abadi J Feigenbaum and J Kilian On hiding information from an oracle Journal

of Computer and System Sciences

AFWZ N Alon U Feige A Wigderson and D Zuckerman Derandomized graph pro ducts

Manuscript

AK E Amaldi and V Kanna The complexity and approximability of nding maximum

feasible subsystems of linear relations Technical rep ort TR ORWP Dept of

Mathematics Swiss Federal Institute of Technology Lausanne

AKM S Arora S Khanna and R Motwani A PTAS for planar MAXSAT Manuscript

AL S Arora and C Lund Hardness of approximations Survey chapter to app ear in a b o ok

on Approximation Algorithms D Ho chbaum ed Available from the authors

ALM S Arora C Lund R Motwani M Sudan and M Szegedy Pro of verication and

intractabilityofapproximation problems In Proc r d IEEE Symp on Foundations of

Computer Science pages

AMS S Arora R Motwani M Safra M Sudan and M Szegedy PCP and approximation

problems Manuscript

AMSP G Ausiello A MarchettiSpaccamela and M Protasi Toward a unied approach for

the classication of NPcomplete optimization problems Theoretical Computer Science

Aro S Arora Reductions co des PCPs and inapproximability Unpublished manuscript

AS S Arora and S Safra Probabilistic checking of pro ofs A new characterization of NP

In Proc rd IEEE Symp on Foundations of Computer Science pages

BIBLIOGRAPHY

Bab L Babai Trading group theory for randomness In Proc th ACM Symp on Theory

of Computing pages

Bab L Babai On Lovaszs lattice reduction and the nearest lattice p oint problem Combi

natorica

Bab L Babai Transparent pro ofs and limits to approximations In Proceedings of the First

European Congress of Mathematicians Birkhauser

BF D Beaver and J Feigenbaum Hiding instances in multioracle queries In Proceedings

of the th Symp on Theoretical Aspects of Computing pages Lecture Notes in

Computer Science 

BF L Babai and L Fortnow Arithmetization a new metho d in structural complexity

theory Computational Complexity

BFL L Babai L Fortnow and C Lund Nondeterministic exp onential time has twoprover

interactive proto cols Computational Complexity

BFLS LBabaiLFortnow L Levin and M Szegedy Checking computations in p olyloga

rithmic time In Proc rdACM Symp on Theory of Computing pages

BGKW M Benor S Goldwasser J Kilian and A Wigderson Multi prov er interactive pro ofs

Howtoremoveintractability assumptions In Proc th ACM Symp on Theory of

Computing pages

BGLR M Bellare S Goldwasser C Lund and A Russell Ecientmultiprover interactive

pro ofs with applications to approximation problems In Proc th ACM Symp on

Theory of Computing pages

BJL A Blum T Jiang M Li J Tromp and M Yannakakis Linear approximation of

shortest sup erstrings In Proc rdACM Symp on Theory of Computing pages

BK M Blum and S Kannan Designing programs that check their work In Proc st ACM

Symp on Theory of Computing pages

BLR M Blum M Luby and R Rubinfeld Selftestingcorrecting with applications to nu

merical problems In Proc nd ACM Symp on Theory of Computing pages

BM M Blum and S Micali How to generate cryptographically strong sequences of pseudo

random bits SIAM Journal on Computing

BM L Babai and S Moran ArthurMerlin games a randomized pro of system and a

hierarchy of complexity classes Journal of Computer and System Sciences pages

BN J Bruck and M Naor The hardness of deco ding linear co des with prepro cessing IEEE

Tr ansactions on Inform Theory pages

Bol B Bollobas CombinatoricsCambridge University Press

BP M Bern and P Plassmann The Steiner problem with edge lengths and Information

Processing Letters

BR M Bellare and PRogaway The complexity of approximating nonlinear programs In

PM Pardalos editor Complexity of Numerical OptimizationWorld Scientic

Preliminary version IBM Research Rep ort RC March

BIBLIOGRAPHY

BS P Berman and G Schnitger On the complexityofapproximating the indep endentset

problem Information and Computation

BS M Bellare and M Sudan Improved nonapproximability results In Proc th ACM

Symp on Theory of Computing pages

BW E Berlekamp and L Welch Error correction of algebraic blo ck co des US Patent

Numb er

CFLS A Condon J Feigenbaum C Lund and P Shor Random debaters and the hardness of

approximating sto chastic functions In Proc of the th Structure in Complexity Theory

Conference pages Also available as DIMACS Techrep ort TR

Chr N Christodes Worst case analysis of a new heuristic for the travelling salesman

problem Technical rep ort Grad Scho ol of Industrial Optimization CarnegieMellon

University

CK P Crescenzi and V Kann A comp endium of NP optimization problems manuscript

CL A Condon and R Ladner On the complexity of space b ounded interactive pro ofs In

Proc th IEEE Symp on Foundations of Computer Science pages

Con A Condon The complexity of the maxword problem and the p ower of onewayinter

active pro of systems Computational Complexity

Co o S Co ok The complexity of theoremproving pro cedures In Proc rdACM Symp on

Theory of Computing pages

DF R Downey and M Fellows Fixedparameter tractability and completeness ii Com

pleteness for W Theoretical Computer Science to app ear

DJP E Dahlhaus D S Johnson C H Papadimitriou P D Seymour and M Yannakakis

The complexityofmultiway cuts In Proc th ACM Symp on Theory of Computing

pages

Fag R Fagin Generalized rstorder sp ectra and p olynomialtime recognizable sets In

Richard Karp editor Complexity of Computer Computations pages AMS

Fei J Feigenbaum Lo cally random reductions in interactive complexitytheoryInAd

vances in Computational Complexity pages American Mathematical So ciety

Providence DIMACS Series on Disc Maths and Theoretical CS volume

FF J Feigenbaum and L Fortnow On the randomselfreducibility of complete sets SIAM

Journal on Computing Prelim version in Pro c of the th Annual

Structure in Complexity Theory Conference

Approximating clique is FGL U Feige S Goldwasser L Lovasz S Safra and M Szegedy

almost NPcomplete In Proc nd IEEE Symp on Foundations of Computer Science

pages

FKa U Feige and J Kilian Towards sub exp onential algorithms for NP Manuscript Octob er

FKb U Feige and J Kilian Twoprover proto colslow error at aordable rates In Proc

th ACM Symp on Theory of Computing pages

FKN J Feigenbaum S Kannan and N Nisan Lower b ounds on randomselfreducibilityIn

Proceedings of the th Structure in Complexity Theory pages

BIBLIOGRAPHY

FL U Feige and L Lovasz Twoprover oneround pro of systems Their p ower and their

problems In Proc th ACM Symp on Theory of Computing pages

Fre R Freivalds Fast probabilistic algorithms In LNCS pages Springer Verlag

FRS L Fortnow J Romp el and M Sipser On the p ower of multiprover interactivepro

to cols In Proceedings of the rd Conference on Structure in Complexity Theorypages

FSH K Friedl A Shen and Z Hatsagi The lowdegree test In Proc th SIAM Symposium

on Discrete Algorithms

FT A Frank and E Tardos An application of simultaneous approximationincombinatorial

optimization In Proc th IEEE Symp on Foundations of Computer Sciencepages

Fur M Furer Improved hardness results for approximating the chromatic number

Manuscript

GG O Gabb er and Z Galil Explicit constructions of linear sized sup erconcentrators Jour

nal of Computer and System Sciences

GGJ MR Garey RL Graham and DS Johnson Some NP complete geometric problems

In Proc th ACM Symp on Theory of Computing pages

GJ M R Garey and D S Johnson Computers and Intractability a guide to the theory of

NPcompletenessWHFreeman

GL O Goldreich and LA Levin A hardcore predicate for all oneway functions In Proc

st ACM Symp on Theory of Computing pages

GLR P Gemmell R Lipton R Rubinfeld M Sudan and A Wigderson Self

testingcorrecting for p olynomials and for approximate functions In Proc rdACM

Symp on Theory of Computing pages

GMR S Goldwasser S Micali and C Racko The knowledge complexityofinteractive

pro ofs SIAM J of Computation

GMS O Goldreich Y Mansour and M Sipser Interactive pro of systems Provers that never

fail and random selection In Proc th IEEE Symp on Foundations of Computer

Science pages

Gol O Goldreich Probabilistic pro of systems Technical Rep ort RS Basic Research

in Computer Science Center of the Danish National ResearchFoundation September

eedings of the International Congress of Mathematicians To appear in the Proc

Birkhauser Verlag

GS S Goldwasser and M Sipser Private versus public coins in interactive pro of systems

In Proc th ACM Symp on Theory of Computing pages

GS P Gemmell and M Sudan Highly resilient correctors for p olynomials Information and

Computation

GVY N Garg VV Vazirani and M Yannakakis Approximate maxow minmulticut

theorems and their applications In Proc th ACM Symp on Theory of Computing pages

BIBLIOGRAPHY

GW M Go emans and D Williamson A approximation algorithm for MAXSATand

MAXCUT In Proc th ACM Symp on Theory of Computing pages

HS KU Ho egen and HU Simon Robust trainability of single neurons In Proceedings

of the ConferenceofLearning Theory pages

Joh D S Johnson Approximation algorithms for combinatorial problems Journal of Com

puter and System Sciences

Joh D S Johnson The NPcompleteness column an ongoing guide Journal of Algorithms

JP D S Johnson and F P Preparata The densest hemisphere problem Theoretical

Computer Science

Kan R Kannan Minkowskis convex b o dy theorem and integer programming Mathematics

of Operations Research

Kan V Kann On the approximability of NPcomplete optimization problems PhD thesis

Royal Institute of Technology Sto ckholm Sweden

Kar R M Karp Reducibility among combinatorial problems In Miller and Thatcher

editors Complexity of Computer Computations pages Plenum Press

KARR P Klein A Agarwal R Ravi and S Rao Approximation through multicommodity

ow In Proc st IEEE Symp on Foundations of Computer Science pages

Kil J Kilian A note on ecient zeroknowledge pro ofs and arguments In Proc th ACM

Symp on Theory of Computing pages

KLS S Khanna N Linial and S Safra On the hardness of approximating the chromatic

number In Proceedings of the nd Israel Symposium on Theory and Computing Systems

ISTCS pages IEEE Computer So ciety Press

KMR D Karger R Motwani and GDS Ramkumar On approximating the longest path in a

graph In Proceedings of Workshop on Algorithms and Data Structures pages

LNCS SpringerVerlag v

KMS D Karger R Motwani and M Sudan Graph coloring through semidenite program

ming In Proc th IEEE Symp on Foundations of Computer Science

KMSV S Khanna R Motwani M Sudan and U Vazirani Computational versus syntactic

views of approximability In Proc th IEEE Symp on Foundations of Computer

Science pages

Knu DE Knuth The Art of Computer Pro gramming vol AddisonWesley Reading

Mass

Lev L Levin Universalnye p ereb ornye zadachi universal search problems in Russian

Problemy Peredachi Informatsii

LFKN CLundLFortnow H Karlo and N Nisan Algebraic metho ds for interactive pro of

systems Journal of the ACM Octob er

Lip R Lipton Ecientchecking of computations In Proceedings of th STACS

LLL AK Lenstra HW Lenstra and L Lovasz Factoring p olynomials with rational co ef

cients Math Ann

BIBLIOGRAPHY

LLS J Lagarias HW Lenstra and CPSchnorr KorkineZolotarev bases and successive

minima of a lattice and its recipro cal lattice Combinatorica

LO JC Lagarias and AM Odlyzko Solving lowdensity subsetsum problems Journal of

the ACM

Lov L Lovasz Algorithmic theory of numbers graphs and convexity NSFCBMS Reg

Conference Series SIAM

LPS A Lub otzky R Phillips and P Sarnak Ramanujam graphs Combinatorica

LR T Leighton and S Rao An approximate maxow mincut theorem for uniform multi

commo dityow problems with applications to approximation algorithms In Proc th

IEEE Symp on Foundations of Computer Science pages

LS D Lapidot and A Shamir Fully parallelized multi prover proto cols for NEXPTIME

In Proc nd IEEE Symp on Foundations of Computer Science pages

Lun C Lund The Power of Interaction MIT Press Cambridge Mass

LV N Linial and U Vazirani Graphpro ducts and chromatic numb er In Proc th IEEE

Symp on Foundations of Computer Science pages

LY C Lund and M Yannakakis The approximation of maximum subgraph problems In

oceedings of International Col loquium on Automata Languages and Programming Pr

ICALP pages

LY Carsten Lund and Mihalis Yannakakis On the hardness of approximating minimization

problems Journal of the ACM

Mic S Micali CS Computationally Sound pro ofs In Proc th IEEE Symp on Founda

tions of Computer Science pages

MP M Minsky and S Pap ert Perceptrons

MS F J MacWilliams and N J A Sloane The Theoryoferrorcorrecting codes North

Holland Amsterdam

Pap C Papadimitriou The Euclidean travelling salesman problem is NP complete Theo

retical Computer Science

Pap C Papadimitriou Games against nature In Proc th IEEE Symp on Foundations

of Computer Science pages

PS A Polishchuk and D Spielman Nearly linear size holographic pro ofs In Proc th

ACM Symp on Theory of Computing

PY C Papadimitriou and M Yannakakis Optimization approximation and complexity

classes Journal of Computer and System Sciences

akis On limited nondeterminism and the complexity PYa C Papadimitriou and M Yannak

of the vc dimension In Proc th IEEE Structure in Complexity Theory Conference

PYb C Papadimitriou and M Yannakakis The traveling salesman problem with distances

one and two Mathematics of Operations Research

Raz R Raz A parallel rep etition theorem Manuscript

BIBLIOGRAPHY

RS R Rubinfeld and M Sudan Testing p olynomial functions eciently and over rational

domains In Proc rdAnnual ACMSIAM Symp on Discrete Algorithms pages

Rub R Rubinfeld A mathematical theory of selfchecking selftesting and selfcorrecting

Programs PhD thesis UC Berkeley

Sch CPSchnorr A hierarchy of p olynomialtime basis reduction algorithms In Proceedings

of ConferenceonAlgorithms Pecs Hungary pages NorthHolland

SG S Sahni and T Gonzalez Pcomplete approximation problems Journal of the ACM

Sha A Shamir IP PSP ACE Journal of the ACM Octob er

She A Shen Multilinearity test made easy Manuscript

Sud M Sudan Ecient checking of polynomials and proofs and the hardness of approximat

ion problems PhD thesis UC Berkeley

Val L G Valiant A theory of the learnable Communications of the ACM

vEB Pvan Emde Boas Another NPcomplete problem and the complexity of computing

short vectors in a lattice Technical Rep ort Math Inst Univ Amsterdam

VV L G Valiant and V V Vazirani NP is as easy as detecting unique solutions Theoretical

Computer Science

Yan M Yannakakis The eect of a connectivity requirement on the complexity of of maxi

mum subgraph problems Journal of the ACM

Yan M Yannakakis Edge deletion problems SIAM Journal of Computing

Yan M Yannakakis On the approximation of maximum satisabilityInProceedings of rd

Annual ACMSIAM Symposium on Discrete Algorithms pages

Yao ACC Yao Coherent functions and program checkers In Proc nd ACM Symp on

Theory of Computing pages

Zuc D Zuckerman Simulating BPP using a general weak random source In Proc nd

IEEE Symp on Foundations of Computer Science pages

Zuc D Zuckerman NPcomplete problems haveaversion thats hard to approximate In

ory Conf pages th Structure in Complexity The

BIBLIOGRAPHY

App endix A

Library of Useful Facts

We include pro ofs of some simple facts assumed in the thesis

Fact A Co okLevin Theorem Stronger Form For any language L NP thereis

axedconstant c such that given any input size n we can in time polyn construct a SAT

n

c

formula in the variables x x y y such that the an input b f g is in L

n n

i

c c

TRUE b b y y y y

n n n

where b b are the bits of b

n

Pro of Follows from an easy mo dication of the pro of of the standard version of the Co ok

Levin theorem

The following fact often go es by the name Markovs inequality

Fact A Averaging Principle Suppose the average of a set of numbers from is

Then i The fraction of them that aregreater than k is at most k ii The fraction

p p

of them that aregreater than is less than

Pro of i For if not then the average is more than k k This is a contradiction

p

Part ii follows by using k

In the rest of this section F denotes the eld GFq Some lemmas require q to b e

lowerb ounded by a function of some other parameters

Fact A For every set of k point value pairs fa b i k g where a b F

i i i i

and no point a is repeated in the list thereisauniquepolynomial px of degree k such

i

that

pa b

i i

APPENDIX A LIBRARY OF USEFUL FACTS

Pro of Let

Y

x a

j

L x

i

a a

i j

j i

b e the p olynomial that is at a and zero at all a for j i Then the desired p olynomial

i j

p is given by

X

px b L x

i i

ik

Uniqueness is easy to verify

Fact A Schwartz An mvariate polynomial of degree d is at no more than dq

m

fraction of points in F where q jFj

Pro of Proved by induction on mTruth for m is clear since a univariate degree d

p olynomial has at most d ro ots

A degreed p olynomial px x has a representation as

m

k

X

i

p x x A x

i m

i

where k d and each p x x is a nonzero m variate p olynomial of degree at

i m

most d i

By the inductivehyp othesis for at least d k q fraction of values of x x

m

p x x For any suchvalue of x x the expression in Equation A is a

k m m

degree k p olynomial in x and so is zero for at most k values of x

Hence the fraction of nonzero es of p is at least d k q kq dq

Fact A Let d q and f bean mvariate polynomial of degree at most d If its restric

tion on dq fraction of lines is dq close to a univariate degree k polynomial where

kd then the degreeof f is exactly k

Pro of A line is sp ecied as fu u t v v t Fg for some u u

m m m

v v F The restriction of f on such a line is given by

m

d

X

i

v t p u u v

m i m

i

where each p is a p olynomial in u u v v of total degree at most d In fact a

i m m

little thought shows that p is a function only of v v and is exactly the sum of those

d m

terms in f whose total degree is d

On any line where f s restriction is dq close to a univariate p olynomial of degree

k p p p are The hyp othesis says that this happ ens for dq fraction of the lines

d d k

Hence p must b e identically zero Thus f has no terms of total degree d Rep eating this

d

argument shows that it has no terms of total degree more than k

m

Fact A Suppose d q and f F F is a function whose restriction on every line is

described by a univariate degreed polynomial Then f F x x

d m

Pro of By induction on m The case m is trivial

Let m Let a a b e distinct p oints in F According to the inductivehyp othesis

d

a x x x x Fg the restriction of f on the m dimensional subspace f

i m m

is describ ed byanm variate p olynomial of degree d Let f denote this p olynomial

i

Let L b e the univariate p olynomial that is at a and at all a for j i Consider

i i j

the mvariate p olynomial g dened as

d

X

L x f x x

i i m

i

The restriction of g on any line that is parallel to the x axis that is a line of the

form ft b b t Fg is a degree d p olynomial in x This univariate p olynomial

m

describ es f when t a a a Hence it must b e the univariate p olynomial that

d

describ es f on the entire line

m

Since the set of lines parallel to the x axis intersects all p oints in F we conclude

that g f In particular f is a p olynomial of degree at most d But on every line it is

describ ed by a degree d univariate p olynomial So Fact A implies that f has degree d

Fact A Let A a beann n matrix where the entries a areconsideredas

ij ij

variables Then the determinant of A is a polynomial of degree n in the a s

ij

Pro of Follows from insp ection from the expression for the determinant

X Y

detA a sgn

i i

S in

n

where S is the set of all p ermutations of fng

n

The following fact is used in Section The reader maycho ose to read it bymentally

substituting Fy the set of p olynomials over eld F in a formal variable y in place of R

Fact A Cramers Rule Let A beanm n matrix whose entries come from an inte

gral domain RandmnLet A x be a system of m equations in n variables note

it is an overdetermined homogeneous system

APPENDIX A LIBRARY OF USEFUL FACTS

The system has a nontrivial solution i al l n n submatrices of A have determinant

If the system does have a solution then it has one of the type x u x u

n n

whereeach u is a sum of determinants of submatrices of A

i

Pro of Normally wewould solvesuch equations by identifying the largest nonsingular

submatrix of Asay B xing the variables that are not in B and multiplying b oth sides

by the inverse B of B Since the system is homogeneous it is easily seen that we can

also multiply b oth sides any matrix that is a scaling of B In this case wemultiply by

detB B But eachentry of detB B is itself a determinant of submatrices of B

ed Hence the claim is prov

The following lemma uses terms dened in Denition and It says that the

m

m

average curvein P x x hitsevery subset of F of size jFj in ab out jFj

k

p oints Recall that when wesay set of p oints on a curve we are actually talking ab out

amultiset for example the sequence x x below could have rep etitions

k

m

LemmaAWelldistribution Lemma for Curves Let x x F bepoints

k

m

not al l x s are the same and S F be any set Then the average of jC S j among al l

i

jS j

curves C P x x is

m

k

F

j j

Pro of Let C b e the set of curves of degree k whose rst k points are fx x g Consider

k

the following enumeration of elements of C

m

For every x F and j such that kj jFj the curveinC whose j th p ointis

x

m

This counts each curvein C and each x F exactly jFjk times

jS j

Hence the average fraction of p oints from S on a curvein C is

m

F

j j

Fact A Geometry of a plane The lines and points of the plane F have the fol

lowing properties i the number of lines is q q iiLet two lines be parallel to each

other if they do not intersect For any xed line thereare q other lines paral lel to it

iii Every line intersects q other lines

Pro of Every twopoints determine a unique line Every line has q points Hence the the

q q

numb er of lines is q q Thus i is proved

The set of lines that are parallel to a given xed line are mutually disjoint Eachhasq

p oints Hence their numb er is at most q q q q It is easily seen that the number

is exactly q Th us ii is proved

Finally a line intersects every line it is not parallel to The number of such lines using

i and ii is q q q whichisq Hence iii is proved

Index

denition SAT

p olynomialbased SAT

probabilistically checkable algebraic representation of

coherent lines MAXSAT

comp osition

MAXSAT

concatenation planar

cryptography

aggregating queries

curve

almost NPhard

of degree k

alphab et

decision time

approximation

degree

capproximation

of a p olynomial

history of

distance

APX

Hamming

argumentZero Knowledge

minimum

ArthurMerlin games

to a co de

assignment

close

split

Dominating Set

b o oster

duality

b o oster pro duct

edgedeletion problems

enco ding

canonical problems

expander graphs

certicate

for a computation

e

f p olynomial closest to f

for chips

failure ratio

checking split assignments

Chromatic Numb er

gap

clique

gappreserving reduction

clique number G

denition

CLIQUE problem

graph

hardness of approximating

b o oster pro duct

complement

close

pro duct

close

Graph Bisection

co des

Hamming distance based on linear functions

INDEX

Hitting Set MIP

mo del theory

inapproximability

Multiway Cuts

history of

Nearest Co deword

Indep endent Set

NEXPTIME

interactive pro ofs

normal form verier

multiprover

NP

IP

almost NPhard

Lab el Cover

denition of

denition

new characterization of

hardness of

NPcompleteness

lab elling

P

denition

PCP

pseudocover

nsnpn RPCPr

total cover

Theorem

large factor

PCP Theorem

lattice

strong forms

basis reduction

PCPr nqn

Nearest Lattice Vector

p erceptron

Shortest Lattice Vector

plane

p olynomial co de

learning theory

pro cedures for

line

p olynomial extension

linear function

p olynomials

satisfying

bidegree

linear function co de

functions close to

pro cedures for

degree

Longest Path

over elds

lowdegree test

p olynomial co de

p olynomial extensions Max subgraph

MAXSAT

pro cedures for

rational MAXSAT

MAXSAT

satisfying

zerotester family MAXCUT

MaxPlanarSubgraph

program

selftestingcorrecting MaxSatisfy

MaxSetPacking

checking

selftestingcorrecting MAXSNP

pro of classic problems

of memb ership completeness

probabilistically checkable denition of

pseudocover MinUnsatisfy

PSPACE Minimum Feedback Arc Set

INDEX

Quadratic Programming

random selfreducibili ty

reduction

gappreserving

Karp

Lreduction

RPCP

selfimprovement

Set Cover

Shortest Sup erstring

Steiner Tree

success rate

of a plane

of line

Sumcheck

SVP

p

symbol

tableau

tensor pro duct

Tiling

total cover

TSP

Euclidean

metric

verier

aggregating the queries of

comp osing veriers

normal form

NP

PCP

using O query bits

restricted

r nqnrestricted

r nqntnrestricted

RPCP

Vertex Cover

vertex deletion problems

zeroknowledge zerotester p olynomials