Chapter

Probabilistic Pro of Systems

A pro of is whatever convinces me

Shimon Even

The glory attached to the creativity involved in nding pro ofs makes us forget that

it is the less gloried pro cess of verication that gives pro ofs their value Conceptu

ally sp eaking pro ofs are secondary to the verication pro cess whereas technically

sp eaking pro of systems are dened in terms of their verication pro cedures

The notion of a verication pro cedure presumes the notion of computation and

furthermore the notion of ecient computation This implicit stipulation is made

explicit in the denition of NP where ecient computation is asso ciated with

deterministic p olynomialtime algorithms However as argued next we can gain a

lot if we are willing to take a somewhat nontraditional step and allow probabilistic

verication pro cedures

In this chapter we shall study three types of probabilistic pro of systems called

interactive proofs zeroknowledge proofs and probabilistic checkable proofs In each

of these three cases we shall present fascinating results that cannot b e obtained

when considering the analogous deterministic pro of systems

Summary The asso ciation of ecient pro cedures with deterministic

p olynomialtime pro cedures is the basis for viewing NPpro of systems

as the canonical formulation of pro of systems with ecient verica

tion pro cedures Allowing probabilistic verication pro cedures and

moreover ruling by statistical evidence gives rise to various types of

probabilistic pro of systems Indeed these probabilistic pro of systems

carry a probability of error which is explicitly b ounded and can b e

reduced by successive application of the pro of system yet they of

fer various advantages over the traditional deterministic and errorless

pro of systems

Randomized and interactive verication pro cedures giving rise to inter

active proof systems seem much more p owerful than their deterministic

CHAPTER PROBABILISTIC PROOF SYSTEMS

counterparts In particular such interactive pro of systems exist for any

set in P S P AC E coNP eg for the set of unsatised prop ositional

formulae whereas it is widely b elieved that some sets in coNP do not

have NPpro of systems ie NP coNP We stress that a pro of

in this context is not a xed and static ob ject but rather a randomized

and dynamic pro cess in which the verier interacts with the prover

Intuitively one may think of this interaction as consisting of questions

asked by the verier to which the prover has to reply convincingly

Such randomized and interactive verication pro cedures allow for the

meaningful conceptualization of zeroknowledge proofs which are of

great theoretical and practical interest esp ecially in cryptography

Lo osely sp eaking zeroknowledge pro ofs are interactive pro ofs that

yield nothing to the verier b eyond the fact that the assertion is

indeed valid For example a zeroknowledge pro of that a certain prop o

sitional formula is satisable do es not reveal a satisfying assignment to

the formula nor any partial information regarding such an assignment

eg whether the rst variable can assume the value true Thus

the successful verication of a zeroknowledge pro of exhibit an extreme

contrast b etween b eing convinced of the validity of a statement and

learning nothing else while receiving such a convincing pro of It turns

out that under reasonable complexity assumptions ie assuming the

existence of oneway functions every set in NP has a zeroknowledge

pro of system

NPpro ofs can b e eciently transformed into a redundant form that

oers a tradeo b etween the number of lo cations randomly exam

ined in the resulting pro of and the condence in its validity In par

ticular it is known that any set in NP has an NPpro of system that

supp orts probabilistic verication such that the error probability de

creases exp onentially with the number of bits read from the alleged

pro of These redundant NPpro ofs are called probabilistically checkable

proofs or PCPs In addition to their conceptually fascinating nature

PCPs are closely related to the study of the complexity of numerous

natural approximation problems

Introduction and Preliminaries

Conceptually sp eaking pro ofs are secondary to the verication pro cess Indeed

b oth in mathematics and in reallife pro ofs are meaningful only with resp ect to

commonly agreed principles of reasoning and the verication pro cess amounts to

checking that these principles were prop erly applied Thus these principles which

are typically taken for granted are more fundamental than any sp ecic pro of that

applies them that is the mere attempt to reason ab out anything is based on

commonly agreed principles of reasoning

The commonly agreed principles of reasoning are asso ciated with a verication

pro cedure that distinguishes prop er applications of these principles from improp er

ones A line of reasoning is considered valid with resp ect to such xed principles

and is thus deemed a pro of if and only if it pro ceeds by a prop er applications

of these principles Thus a line of reasoning is considered valid if and only if it is

accepted by the corresp onding verication pro cedure This means that technically

sp eaking pro ofs are dened in terms of a predetermined verication pro cedure

or are dene with resp ect to such a pro cedure Indeed this state of aairs is

b est illustrated in the formal study of pro ofs ie logic which is actually the

study of formally dened pro of systems The p oint is that these pro of systems are

dened often explicitly and sometimes only implicitly in terms of their verication

pro cedures

The notion of a verication pro cedure presumes the notion of computation This

fact explains the historical interest of logicians in computer science cf

Furthermore the verication of pro ofs is supp osed to b e relatively easy and hence

a natural connection emerges b etween verication pro cedures and the notion of

ecient computation This connection was made explicit by complexity theorists

and is captured by the denition of NP and NPpro of systems cf Denition

which targets all ecient verication pro cedures

Recall that Denition identies ecient verication pro cedures with de

terministic p olynomialtime algorithms and that it explicitly restricts the length

of pro ofs to b e p olynomial in the length of the assertion Thus verication is

performed in a number of steps that is polynomial in the length of the assertion

We comment that deterministic pro of systems that allow for longer pro ofs but

require that verication is ecient in terms of the length of the alleged pro of can

b e mo deled as NPpro of systems by adequate padding of the assertion

Indeed NPpro ofs provide the ultimate formulation of eciently veriable pro ofs

ie pro of systems with ecient verication pro cedures provided that one asso

ciates ecient pro cedures with deterministic p olynomialtime algorithms How

ever as we shall see we can gain a lot if we are willing to take a somewhat

nontraditional step and allow probabilistic p olynomialtime algorithms and in

particular probabilistic verication pro cedures

Randomized and interactive verication pro cedures seem much more p owerful

than their deterministic counterparts

Such interactive pro of systems allow for the construction of meaningful

zeroknowledge pro ofs which are of great conceptual and practical interest

NPpro ofs can b e eciently transformed into a redundant form that sup

p orts sup erfast probabilistic verication via very few random prob es into the

alleged pro of

In contrast traditional pro of systems are formulated based on rules of inference that seem

natural in the relevant context The fact that these inference rules yield an ecient verication

pro cedure is merely a consequence of the corresp ondence b etween pro cesses that seem natural

and ecient computation

CHAPTER PROBABILISTIC PROOF SYSTEMS

In all these cases explicit b ounds are imp osed on the computational complexity of

the verication pro cedure which in turn is p ersonied by the notion of a verier

Furthermore in all these pro of systems the verier is allowed to toss coins and

rule by statistical evidence Thus all these pro of systems carry a probability of

error yet this probability is explicitly b ounded and furthermore can b e reduced

by successive application of the pro of system

One imp ortant convention When presenting a pro of system we state all

complexity b ounds in terms of the length of the assertion to b e proved which is

viewed as an input to the verier Namely when we say p olynomialtime we

mean time that is p olynomial in the length of this assertion Indeed as will b ecome

evident this is the natural choice in all the cases that we consider Note that this

convention is consistent with the foregoing discussion of NPpro of systems

Notational Conventions We denote by poly the set of all integer functions

that are upp erb ounded by a p olynomial and by log the set of all integer functions

b ounded by a logarithmic function ie f log if and only if f n O log n

All complexity measures mentioned in this chapter are assumed to b e constructible

in p olynomialtime

Organization In Section we present the basic denitions and results regard

ing interactive pro of systems The denition of an interactive pro of systems is the

starting p oint for a discussion of zeroknowledge pro ofs which is provided in Sec

tion Section which presents the basic denitions and results regarding

probabilistically checkable pro ofs PCP can b e read indep endently of the other

sections

Prerequisites We assume a basic familiarity with elementary probability theory

see App endix D and randomized algorithms see Section

Interactive Pro of Systems

In light of the growing acceptability of randomized and interactive computations

it is only natural to asso ciate the notion of ecient computation with probabilistic

and interactive p olynomialtime computations This leads naturally to the notion

of an interactive pro of system in which the verication pro cedure is interactive and

randomized rather than b eing noninteractive and deterministic Thus a pro of

in this context is not a xed and static ob ject but rather a randomized dynamic

pro cess in which the verier interacts with the prover Intuitively one may think of

this interaction as consisting of questions asked by the verier to which the prover

has to reply convincingly

Recall that Denition refers to p olynomialtime verication of alleged pro ofs which in

turn must have length that is b ounded by a p olynomial in the length of the assertion

INTERACTIVE PROOF SYSTEMS

The foregoing discussion as well as the denition provided in Section

makes explicit reference to a prover whereas a prover is only implicit in the tradi

tional denitions of pro of systems eg NPpro of systems Before turning to the

actual denition we highlight and further discuss this issue as well as some other

conceptual issues

Motivation and Perspective

We shall discuss the various interpretations given to the notion of a pro of in dif

ferent human contexts and the attitudes that underly andor accompany these

interpretations This discussion is aimed at emphasizing that the motivation for

the denition of interactive pro of systems is not replacing the notion of a mathemat

ical pro of but rather capturing other forms of pro ofs that are of natural interest

We also discuss the roles of the prover and the verier in these settings and the

general notions of completeness and soundness

A static ob ject versus an interactive pro cess

Traditionally in mathematics a pro of is a xed sequence consisting of statements

that are either selfevident or are derived from previous statements via selfevident

rules Actually b oth conceptually and technically it is more accurate to substitute

the phrase selfevident by the phrase commonly agreed b ecause at the last

account selfevidence is a matter of common agreement In fact in the formal

study of pro ofs ie logic the commonly agreed statements are called axioms

whereas the commonly agreed rules are referred to as derivation rules We highlight

a key property of mathematics proofs these proofs are viewed as xed static

objects

In contrast in other areas of human activity the notion of a pro of has a

much wider interpretation In particular a pro of is not a xed ob ject but rather

a pro cess by which the validity of an assertion is established For example in the

context of Law standing a crossexamination by an opp onent who may ask tough

andor tricky questions is considered a pro of of the facts claimed by the witness

Likewise various debates that take place in daily life have an analogous p otential of

establishing claims and are then p erceived as pro ofs This p erception is quite com

mon in philosophical and p olitical debates and applies even in scientic debates

Needless to say a key property of such debates is their interactive dynamic

nature Interestingly the app ealing nature of such interactive pro ofs is reected

in the fact that they are mimicked in a rigorous manner in some mathemati

cal proofs by contradiction which emulate an imaginary debate with a p otential

generic skeptic

Another dierence b etween mathematical pro ofs and various forms of daily

pro ofs is that while the former aim at certainty the latter are intended only

for establishing claims beyond any reasonable doubt Arguably an explicitly b ounded

error probability as present in our denition of interactive pro of systems is an

extremely strong form of establishing a claim b eyond any reasonable doubt

CHAPTER PROBABILISTIC PROOF SYSTEMS

We also note that in mathematics pro ofs are often considered more imp ortant

than their consequence ie the theorem In contrast in many daily situations

pro ofs are considered secondary in imp ortance to their consequence These con

icting attitudes are wellcoupled with the dierence b etween written pro ofs and

interactive pro ofs If one values the pro of itself then one may insist on having it

archived whereas if one only cares ab out the consequence then the way in which

it is reached is immaterial

Interestingly the foregoing set of daily attitudes rather than the mathematical

ones will b e adequate in the current chapter where proofs are viewed merely as

a vehicle for the verication of the validity of claims This attitude gets to an

extreme in the case of zeroknowledge pro ofs where we actually require that the

pro ofs themselves b e useless b eyond b eing convincing of the validity of the claimed

assertion

In general we will b e interested in mo deling various forms of pro ofs that may

o ccur in the world fo cusing on pro ofs that can b e veried by automated pro cedures

These verication pro cedures are designed to check the validity of p otential pro ofs

and are oblivious of additional features that may app eal to humans such as b eauty

insightfulness etc In the current section we will consider the most general form

of pro of systems that still allow ecient verication

We note that the pro of systems that we study refer to mundane theorems eg

asserting that a specic prop ositional formula is not satisable or that a party sent

a message as instructed by a predetermined proto col We stress that the meta

theorems that we shall state regarding these pro of systems will b e proved in the

traditional mathematical sense

Prover and Verier

The wide interpretation of the notion of a pro of system which includes interactive

pro cesses of verication calls for the explicit introduction of two interactive players

called the prover and the verier The verier is the party that employs the

verication pro cedure which underlies the denition of any pro of system while

the prover is the party that tries to convince the verier In the context of static

or noninteractive pro ofs the prover is the transcendental entity providing the

pro of and thus in this context the prover is often not mentioned at all when

discussing the verication of alleged pro ofs Still explicitly mentioning p otential

provers may b e b enecial even when discussing such static noninteractive pro ofs

We highlight the distrustful attitude towards the prover which underlies any

pro of system If the verier trusts the prover then no pro of is needed Hence

whenever discussing a pro of system one should envision a setting in which the

verier is not trusting the prover and furthermore is skeptic of anything that the

prover says In such a setting the provers goal is to convince the verier while the

verier should make sure that it is not fo oled by the prover See further discussion

in x Note that the verier is trusted to protect its own interests by

employing the predetermined verication pro cedure indeed the asymmetry with

resp ect to who we trust is an artifact of our fo cus on the verication pro cess or

task In general each party is trusted to protect its own interests ie the verier

INTERACTIVE PROOF SYSTEMS

is trusted to protect its own interests but no party is trusted to protect the

interests of the other party ie the prover is not trusted to protect the veriers

interest of not b eing fo oled by the prover

Another asymmetry b etween the two parties is that our discussion fo cuses on

the complexity of the verication task and ignores as a rst approximation the

complexity of the proving task which is only discussed in x Note that this

asymmetry is reected in the denition of NPpro of systems that is verication

is required to b e ecient whereas for sets NP n P nding adequate pro ofs is

infeasible Thus as a rst approximation we consider the question of what can

b e eciently veried when interacting with an arbitrary prover which may b e

innitely p owerful Once this question is resolved we shall also consider the

complexity of the proving task indeed see x

Completeness and Soundness

Two fundamental prop erties of a pro of system ie of a verication pro cedure are

its soundness or validity and completeness The soundness prop erty asserts that

the verication pro cedure cannot b e tricked into accepting false statements In

other words soundness captures the veriers ability to protect itself from b eing

convinced of false statements no matter what the prover do es in order to fo ol

it On the other hand completeness captures the ability of some prover to con

vince the verier of true statements b elonging to some predetermined set of true

statements Note that b oth prop erties are essential to the very notion of a pro of

system

We note that not every set of true statements has a reasonable pro of system

in which each of these statements can b e proved while no false statement can b e

proved This fundamental phenomenon is given a precise meaning in results

such as Godels Incompleteness Theorem and Turings theorem regarding the un

decidability of the Halting Problem In contrast recall that NP was dened as the

class of sets having pro of systems that supp ort ecient deterministic verication

of written pro ofs This section is devoted to the study of a more lib eral notion

of ecient verication pro cedures allowing b oth randomization and interaction

Denition

Lo osely sp eaking an interactive pro of is a game b etween a computationally b ounded

verier and a computationally unbounded prover whose goal is to convince the

verier of the validity of some assertion Sp ecically the verier employs a proba

bilistic p olynomialtime strategy whereas no computational restrictions apply to

the provers strategy It is required that if the assertion holds then the verier

always accepts ie when interacting with an appropriate prover strategy On the

other hand if the assertion is false then the verier must reject with probability

no matter what strategy is b eing employed by the prover The error at least

probability can b e reduced by running such a pro of system several times

We formalize the interaction b etween parties by referring to the strategies that

CHAPTER PROBABILISTIC PROOF SYSTEMS

the parties employ A strategy for a party describ es the partys next move ie

its next message or its nal decision as a function of the common input ie

the aforementioned assertion its internal coin tosses and al l messages it has

received so far That is we assume that each party records the outcomes of its past

coin tosses as well as all the messages it has received and determines its moves

based on these Thus an interaction b etween two parties employing strategies

A and B resp ectively is determined by the common input denoted x and the

randomness of b oth parties denoted r and r Assuming that A takes the rst

A B

move and B takes the last move the corresp onding tround interaction transcript

on common input x and randomness r and r is where

A B t t i

Ax r and B x r The corresp onding nal decision

A i i B i

of A is dened as Ax r

A t

We say that a party employs a probabilistic p olynomialtime strategy if its next

move can b e computed in a number of steps that is polynomial in the length of

the common input In particular this means that on input common input x the

strategy may only consider a p olynomial in jxj many messages which are each of

p oly jxj length Intuitively if the other party exceeds an a priori p olynomial in

jxj b ound on the total length of the messages that it is allowed to send then the

execution is susp ended Thus referring to the aforementioned strategies we say

that A is a probabilistic p olynomialtime strategy if for every i and r

A i

the value of Ax r can b e computed in time p olynomial in jxj Again

A i

in prop er use it must hold that jr j t and the j js are all p olynomial in jxj

A i

Denition Interactive Pro of systems IP An interactive proof system for

a set S is a twoparty game between a verier executing a probabilistic p olynomial

time strategy denoted V and a prover that executes a computationally unbounded

strategy denoted P satisfying the fol lowing two conditions

Completeness For every x S the verier V always accepts after interacting

with the prover P on common input x

Soundness For every x S and every strategy P the verier V rejects with

after interacting with P on common input x probability at least

We denote by IP the class of sets having interactive proof systems

The error probability in the soundness condition can b e reduced by successive

applications of the pro of system This is easy to see in the case of sequential

rep etitions but holds also for parallel rep etitions see Exercise In particular

An alternative formulation refers to the interactive machines that capture the b ehavior of each

of the parties see eg Sec Such an interactive machine invokes the corresp onding

strategy while handling the communication with the other party and keeping a record of all

messages received so far

Needless to say the number of internal coin tosses fed to a p olynomialtime strategy must

also b e b ounded by a p olynomial in the length of x

We follow the convention of sp ecifying strategies for b oth the verier and the prover An

alternative presentation only sp ecies the veriers strategy while rephrasing the completeness

condition as follows There exists a prover strategy P such that for every x S the verier V

always accepts after interacting with P on common input x

INTERACTIVE PROOF SYSTEMS

rep eating the proving pro cess for k times reduces the probability that the verier

k

is fo oled ie accepts a false assertion to and we can aord doing so for any

k p oly jxj Variants on the basic denition are discussed in Section

The role of randomness Randomness is essential to the p ower of interactive

pro ofs that is restricting the verier to deterministic strategies yields a class of

interactive pro of systems that has no advantage over the class of NPpro of systems

The reason b eing that in case the verier is deterministic the prover can predict

the veriers part of the interaction Thus the prover can just supply its own

sequence of answers to the veriers sequence of predictable questions and the

verier can just check that these answers are convincing Actually we establish

that soundness error and not merely randomized verication is essential to the

p ower of interactive pro of systems ie their ability to reach b eyond NPpro ofs

Prop osition Suppose that S has an interactive proof system P V with no

soundness error that is for every x S and every potential strategy P the verier

V rejects with probability one after interacting with P on common input x Then

S NP

Pro of We may assume without loss of generality that V is deterministic by just

xing arbitrarily the contents of its randomtap e eg to the allzero string and

noting that b oth p erfect completeness and p erfect ie errorless soundness still

hold Thus the case of zero soundness error reduces to the case of deterministic

veriers

Now since V is deterministic the prover can predict each message sent by V

b ecause each such message is uniquely determined by the common input and the

previous prover messages Thus a sequence of optimal provers messages ie a

sequence of messages leading V to accept x S can b e predetermined without

interacting with V based solely on the common input x Hence x S if and only

if there exists a sequence of provers messages that make the deterministic V

accept x where the question of whether a sp ecic sequence of provers messages

makes V accept x dep ends only on the sequence and on the common input x

b ecause V tosses no coins that may aect this decision The foregoing condition

can b e checked in p olynomialtime and so a passing sequence constitutes an

NPwitness for x S It follows that S NP

Reection The moral of the reasoning underlying the pro of Prop osition is

that there is no point to interact with a party whose moves are easily predictable

b ecause such moves can b e determined without any interaction This moral repre

sents the provers p oint of view regarding interaction with deterministic veriers

As usual we do not care ab out the complexity of determining such a sequence since no

computational b ounds are placed on the prover

Recall that in the case that V is randomized its nal decision also dep ends on its internal

coin tosses and not only on the common input and on the sequence of provers messages In

that case the veriers own messages may reveal information ab out the veriers internal coin

tosses which in turn may help the prover to answer with convincing messages

CHAPTER PROBABILISTIC PROOF SYSTEMS

In contrast even an innitely p owerful party eg a prover may gain by inter

acting with an unpredictable party eg a randomized verier b ecause this in

teraction may provide useful information eg information regarding the veriers

coin tosses which in turn allows the prover to increase its probability of answer

ing convincingly Furthermore from the veriers p oint of view it is b enecial to

interact with the prover b ecause the latter is computationally stronger and thus

its moves may not b e easily predictable by the verier even in the case that they

are predictable in an information theoretic sense

The Power of Interactive Pro ofs

We have seen that randomness is essential to the p ower of interactive pro of systems

in the sense that without randomness interactive pro ofs are not more p owerful than

NPpro ofs Indeed the p ower of interactive pro of arises from the combination of

randomization and interaction We rst demonstrate this p oint by a simple pro of

system for a sp ecic coNPset that is not known to have an NPpro of system and

next prove the celebrated result IP P S P AC E which suggests that interactive

pro ofs are much stronger than NPpro ofs

A simple example

One day on the Olympus brighteyed Athena claimed that Nectar

p oured out of the new silvercoated jars tastes less go o d than Nec

tar p oured out of the older golddecorated jars Mighty Zeus who was

forced to introduce the new jars by the practically oriented Hera was

annoyed at the claim He ordered that Athena b e served one hundred

glasses of Nectar each p oured at random either from an old jar or from

a new one and that she tell the source of the drink in each glass To

everybo dys surprise wise Athena correctly identied the source of each

serving to which the Father of the Go ds resp onded my child you are

either right or extremely lucky Since all go ds knew that b eing lucky

was not one of the attributes of PallasAthena they all concluded that

the imp eccable go ddess was right in her claim

The foregoing story illustrates the main idea underlying the interactive pro of for

Graph NonIsomorphism presented in Construction Informally this interac

tive pro of system is designed for proving dissimilarity of two given ob jects in the

foregoing story these are the two brands of Nectar whereas in Construction

these are two nonisomorphic graphs We note that typically proving similarity

b etween ob jects is easy b ecause one can present a mapping of one ob ject to the

other that demonstrates this similarity In contrast proving dissimilarity seems

harder b ecause in general there seems to b e no succinct pro of of dissimilarity eg

clearly showing that a particular mapping fails do es not suce while enumerat

ing all p ossible mappings and showing that each fails do es not yield a succinct

pro of More generally it is typically easy to prove the existence of an easily veri

able structure in a given ob ject by merely presenting this structure but proving

INTERACTIVE PROOF SYSTEMS

the nonexistence of such a structure seems hard Formally membership in an

NPset is proved by presenting an NPwitness but it is not clear how to prove

the nonexistence of such a witness Indeed recall that the common b elief is that

coNP NP

Two graphs G V E and G V E are called isomorphic if there exists

a and onto mapping from the vertex set V to the vertex set V such that

fu v g E if and only if fv ug E This edge preserving mapping

in case it exists is called an isomorphism b etween the graphs The following

proto col sp ecies a way of proving that two graphs are not isomorphic while it is

not known whether such a statement can b e proved via a noninteractive pro cess

ie via an NPpro of system

Construction Interactive pro of for Graph NonIsomorphism

Common Input A pair of graphs G V E and G V E

Veriers rst step V The verier selects at random one of the two input

graphs and sends to the prover a random isomorphic copy of this graph

Namely the verier selects uniformly f g and a random permutation

from the set of permutations over the vertex set V The verier constructs

a graph with vertex set V and edge set

def

E ff u v g fu v g E g

and sends V E to the prover

Motivating Remark If the input graphs are nonisomorphic as the prover

claims then the prover should be able to distinguish not necessarily by an

ecient algorithm isomorphic copies of one graph from isomorphic copies of

the other graph However if the input graphs are isomorphic then a random

isomorphic copy of one graph is distributed identically to a random isomorphic

copy of the other graph

V E from the verier the Provers step Upon receiving a graph G

prover nds a f g such that the graph G is isomorphic to the input

graph G If both satisfy the condition then is selected arbitrarily

In case no f g satises the condition is set to The prover sends

to the verier

Veriers second step V If the message received from the prover equals

chosen in Step V then the verier outputs ie accepts the common

input Otherwise the verier outputs ie rejects the common input

The veriers strategy in Construction is easily implemented in probabilistic

p olynomialtime We do not known of a probabilistic p olynomialtime implemen

tation of the provers strategy but this is not required The motivating remark

justies the claim that Construction constitutes an interactive pro of system for

CHAPTER PROBABILISTIC PROOF SYSTEMS

the set of pairs of nonisomorphic graphs Recall that the latter is a coNP set

which is not known to b e in NP

The full p ower of interactive pro ofs

The interactive pro of system of Construction refers to a sp ecic coNPset that

is not known to b e in NP It turns out that interactive pro of systems are p owerful

enough to prove membership in any coNPset eg prove that a graph is not

colorable Thus assuming that NP coNP this establishes that interactive

pro of systems are more p owerful than NPpro of systems Furthermore the class

of sets having interactive pro of systems coincides with the class of sets that can b e

decided using a p olynomial amount of workspace

Theorem The IP Theorem IP P S P AC E

Recall that it is widely b elieved that NP is a proper subset of P S P AC E Thus

under this conjecture interactive pro ofs are more p owerful than NPpro ofs

Sketch of the Pro of of Theorem

We rst show that coNP IP by presenting an interactive pro of system for

the coNP complete set of unsatisable CNF formulae Next we extend this pro of

system to obtain one for the P S P AC E complete set of unsatisable Quantied

Bo olean Formulae Finally we observe that IP P S P AC E Indeed proving that

some coNP complete set has an interactive pro of system is the core of the pro of

of Theorem see Exercise

We show that the set of unsatisable CNF formulae has an interactive pro of

system by using algebraic metho ds which are applied to an arithmetic generaliza

tion of the said Boolean problem rather than to the problem itself That is in

order to demonstrate that this Bo olean problem has an interactive pro of system we

rst introduce an arithmetic generalization of CNF formulae and then construct

an interactive pro of system for the resulting arithmetic assertion by capitalizing

on the arithmetic formulation of the assertion Intuitively we present an iterative

pro cess which involves interaction b etween the prover and the verier such that in

each iteration the residual claim to b e established b ecomes simpler ie contains

one variable less This iterative pro cess seems to b e enabled by the fact that the

various claims refer to the arithmetic problem rather than to the original Bo olean

problem Actually one may say that the key p oint is that these claims refer to a

generalized problem rather than to the original one

In case G is not isomorphic to G no graph can b e isomorphic to b oth input graphs ie

b oth to G and to G In this case the graph G sent in Step V uniquely determines the bit

On the other hand if G and G are isomorphic then for every G sent in Step V the

number of isomorphisms b etween G and G equals the number of isomorphisms b etween G and

G It follows that in this case G yields no information ab out chosen by the verier and so

no prover may convince the verier with probability exceeding

INTERACTIVE PROOF SYSTEMS

Teaching note We devote most of the presentation to establishing that co NP IP

and recommend doing the same in class Our presentation fo cuses on the main ideas

and neglects some minor implementation details which can b e found in

The starting p oint We prove that coNP IP by presenting an interactive

pro of system for the set of unsatisable CNF formulae which is coNP complete

Thus our starting p oint is a given Bo olean CNF formula which is claimed to b e

unsatisable

Arithmetization of Bo olean CNF formulae Given a Bo olean CNF for

mula we replace the Bo olean variables by integer variables and replace the logical

op erations by corresp onding arithmetic op erations In particular the Bo olean val

ues false and true are replaced by the integer values and resp ectively

orclauses are replaced by sums and the top level conjunction is replaced by a

pro duct This translation is depicted in Figure Note that the Bo olean formula

Boolean arithmetic

variable values false true

connectives x and x and

nal values false true p ositive

Figure Arithmetization of CNF formulae

is satised resp unsatised by a sp ecic truth assignment if and only if evaluat

ing the resulting arithmetic expression at the corresp onding assignment yields

a p ositive integer value resp yields the value zero Thus the claim that the

original Bo olean formula is unsatisable translates to the claim that the summa

tion of the resulting arithmetic expression over all assignments to its variables

yields the value zero For example the Bo olean formula

x x x x x x x

is replaces by the arithmetic expression

x x x x x x x

and the Bo olean formula is unsatisable if and only if the sum of the corresp onding

arithmetic expression taken over all choices of x x x f g equals

Thus proving that the original Boolean formula is unsatisable reduces to proving

that the corresponding arithmetic summation evaluates to We highlight two

additional observations regarding the resulting arithmetic expression

The arithmetic expression is a low degree p olynomial over the integers sp ecif

ically its total degree equals the number of clauses in the original Bo olean formula

CHAPTER PROBABILISTIC PROOF SYSTEMS

For any Bo olean formula the value of the corresp onding arithmetic expression

m

for any choice of x x f g resides within the interval v where

n

v is the maximum number of variables in a clause and m is the number of

n

clauses Thus summing over all p ossible assignments where n v m

n m

is the number of variables yields an integer value in v

Moving to a Finite Field In general whenever we need to check equality

b etween two integers in M it suces to check their equality mo d q where

q M The b enet is that if q is prime then the arithmetic is now in a nite

eld mo d q and so certain things are nicer eg uniformly selecting a value

Thus proving that a CNF formula is not satisable reduces to proving an equality

of the following form

X X

x x mo d q

n

x x

n

where is a lowdegree multivariate p olynomial and q can b e represented using

O jj bits In the rest of this exp osition all arithmetic op erations refer to the

nite eld of q elements denoted GF q

Overview of the actual proto col stripping summations in iterations

Given a formal expression as in Eq we strip o summations in iterations

stripping a single summation at each iteration and instantiate the corresp onding

free variable as follows At the b eginning of each iteration the prover is supp osed

to supply the univariate p olynomial representing the residual expression as a func

tion of the single currently stripp ed variable By Observation this is a low

degree p olynomial and so it has a short description The verier checks that the

p olynomial say p is of low degree and that it corresp onds to the current value

say v b eing claimed ie it veries that p p v Next the verier ran

domly instantiates the currently free variable ie it selects uniformly r GFq

yielding a new value to b e claimed for the resulting expression ie the verier

computes v pr and exp ects a pro of that the residual expression equals v

The verier sends the uniformly chosen instantiation ie r to the prover and the

parties pro ceed to the next iteration which refers to the residual expression and

to the new value v At the end of the last iteration the verier has a closed form

expression ie an expression without formal summations which can b e easily

checked against the claimed value

th

A single iteration detailed The i iteration is aimed at proving a claim of

the form

X X

r r x x x v mo d q

i i i n i

x x

i n

We also use Observation which implies that we may use a nite eld with elements having

a description length that is p olynomial in the length of the original Bo olean formula ie log q

O v m

INTERACTIVE PROOF SYSTEMS

where v and r r and v are as determined in previous iterations

i i

th

The i iteration consists of two steps messages a prover step followed by a

verier step The prover is supp osed to provide the verier with the univariate

p olynomial p that satises

i

X X

def

p z r r z x x mo d q

i i i n

x x

i n

Note that mo dule q the value p p equals the lhs of Eq Denote by

i i

p the actual p olynomial sent by the prover ie the honest prover sets p p

i

i i

Then the verier rst checks if p p v mo d q and next uniformly

i

i i

selects r GFq and sends it to the prover Needless to say the verier will

i

reject if the rst check is violated The claim to b e proved in the next iteration is

X X

r r r x x v mo d q

i i i n i

x x

i n

def

where v p r mo d q is computed by each party

i i

i

Completeness of the proto col When the initial claim ie Eq holds

the prover can supply the correct p olynomials as determined in Eq and

this will lead the verier to always accept

Soundness of the proto col It suces to upp erb ound the probability that for

a particular iteration the entry claim ie Eq is false while the ending claim

th

ie Eq is valid Indeed let us fo cus on the i iteration and let v and

i

p b e as in Eq and Eq resp ectively that is v is the wrong value

i i

th

claimed at the b eginning of the i iteration and p is the p olynomial representing

i

the expression obtained when stripping the current variable as in Eq Let

p b e any p otential answer by the prover We may assume without loss of

i

is of lowdegree since v mo d q and that p p generality that p

i

i i i

otherwise the verier will denitely reject Using our hypothesis that the entry

claim of Eq is false we know that p p v mo d q Thus

i i i

p and p are dierent lowdegree p olynomials and so they may agree on very few

i

i

p oints if at all Now if the veriers instantiation ie its choice of a random r

i

do es not happ en to b e one of these few p oints ie p r p r mo d q then

i i i

i

the ending claim ie Eq is false to o b ecause the new value ie v is set

i

to p r mo d q while the residual expression evaluates to p r Details are left

i i i

i

as an exercise see Exercise

This establishes that the set of unsatisable CNF formulae has an interactive

pro of system Actually a similar pro of system which uses a related arithmeti

zation see Exercise can b e used to prove that a given formula has a given

number of satisfying assignment ie prove membership in the counting set

f k jf gj k g

CHAPTER PROBABILISTIC PROOF SYSTEMS

Using adequate reductions it follows that every problem in P has an interactive

pro of system ie for every R PC the set fx k jfy x y R gj k g is in

IP Proving that P S P AC E IP requires a little more work as outlined next

Obtaining interactive pro ofs for PSPACE the basic idea We present

an interactive pro of for the set of satised Quantied Bo olean Formulae QBF

which is complete for P S P AC E see Theorem Recall that the number of

quantiers in such formulae is unbounded eg it may b e p olynomially related to

the length of the input that there are b oth existential and universal quantiers

and furthermore these quantiers may alternate In the arithmetization of these

formulae we replace existential quantiers by summations and universal quantiers

by pro ducts Two diculties arise when considering the application of the foregoing

proto col to the resulting arithmetic expression Firstly the integral value of

the expression which may involve a big number of nested formal pro ducts is

only upp erb ounded by a doubleexp onential function in the length of the input

Secondly when stripping a summation or a pro duct the expression may b e a

p olynomial of high degree due to nested formal pro ducts that may app ear in the

remaining expression For example b oth phenomena o ccur in the following

expression

X Y Y

x y

n

x y y

n

P

n n

x x The rst diculty is easy to resolve by which equals

x

using the fact to b e established in Exercise that if two integers in M are

dierent then they must b e dierent mo dulo most of the primes in the interval

p olylog M Thus we let the verier selects a random prime q of length that

is linear in the length of the original formula and the two parties consider the

arithmetic expression reduced mo dulo this q The second diculty is resolved by

noting that P S P AC E is actually reducible to a sp ecial form of noncanonical QBF

in which no variable app ears b oth to the left and to the right of more than one

universal quantier see the pro of of Theorem or alternatively Exercise

It follows that when arithmetizing and stripping summations or pro ducts from

the resulting arithmetic expression the corresp onding univariate p olynomial is of

low degree ie at most twice the length of the original formula where the factor

Actually the following extension of the foregoing pro of system yields a pro of system for the

set of unsatised Quantied Bo olean Formulae which is also complete for P S P AC E Alterna

tively an interactive pro of system for QBF can b e obtained by extending the related pro of system

presented in Exercise

This high degree causes two diculties where only the second one is acute The rst diculty

is that the soundness of the corresp onding proto col will require working in a nite eld that

is suciently larger than this high degree but we can aord doing so since the degree is at

most exp onential in the formulas length The second and more acute diculty is that the

p olynomial may have a large ie exp onential number of nonzero co ecients and so the verier

cannot aord to read the standard representation of this p olynomial as a list of all nonzero

co ecients Indeed other succinct and eective representations of such p olynomials may exist

in some cases as in the following example but it is unclear how to obtain such representations

in general

INTERACTIVE PROOF SYSTEMS

of two is due to the single universal quantier that has this variable quantied on

its left and app earing on its right

IP is contained in PSPACE We shall show that for every interactive pro of

system there exists an optimal prover strategy that can b e implemented in p olynomial

space where an optimal prover strategy is one that maximizes the probability that

the prescrib ed verier accepts the common input It follows that IP P S P AC E

b ecause for every S IP we can emulate in p olynomial space all p ossible inter

actions of the prescrib ed verier with any xed p olynomialspace prover strategy

eg an optimal one

Prop osition Let V be a probabilistic polynomialtime verier strategy Then

there exists a polynomialspace computable prover strategy f that for every x

maximizes the probability that V accepts x That is for every P and every x it

holds that the probability that V accepts x after interacting with P is upperbounded

by the probability that V accepts x after interacting with f

Pro of Sketch For every common input x and any p ossible partial transcript of

the interaction so far the strategy f determines an optimal nextmessage for the

prover by considering all p ossible coin tosses of the verier that are consistent with

x Sp ecically f is determined recursively such that f x m if m maxi

mizes the number of outcomes of the veriers cointosses that are consistent with

x and lead the verier to accept when subsequent prover moves are determined

by f which is where recursion is used That is the veriers random sequence r

supp ort the setting f x m where if the following two

t t

conditions hold

r is consistent with x which means that for every i f tg it holds

that V x r

i i

r leads V to accept when the subsequent prover moves are determined by f

which means at termination ie after T rounds it holds that

V x r m

t t T

where for every i ft T g it holds that f x m

i t i i

and V x r m

i t t i

Thus f x m if m maximizes the value of E x R m where R is

f V

selected uniformly among the r s that are consistent with x and x r m

f V

indicates whether or not V accepts x in the subsequent interaction with f which

refers to randomness r and partial transcript m It follows that the value

f x can b e computed in p olynomialspace when given oracle access to f x

The prop osition follows by standard comp osition of spaceb ounded computations

ie allo cating separate space to each level of the recursion while using the same

space in all recursive calls of each level

For sake of convenience when describing the strategy f we refer to the entire partial tran

script of the interaction with V rather than merely to the sequence of previous messages sent

by V

CHAPTER PROBABILISTIC PROOF SYSTEMS

Variants and ner structure an overview

In this subsection we consider several variants on the basic denition of interactive

pro ofs as well as ner complexity measures This is an advanced subsection which

only provides an overview of the various notions and results as well as p ointers to

pro ofs of the latter

ArthurMerlin games aka publiccoin pro of systems

The veriers messages in a general interactive pro of system are determined arbi

trarily but eciently based on the veriers view of the interaction so far which

includes its internal coin tosses which without loss of generality can take place at

the onset of the interaction Thus the veriers past coin tosses are not necessarily

revealed by the messages that it sends In contrast in publiccoin pro of systems

aka ArthurMerlin pro of systems the veriers messages contain the outcome

of any coin that it tosses at the current round Thus these messages reveal the

randomness used towards generating them ie this randomness b ecomes public

Actually without loss of generality the veriers messages can b e identical to the

outcome of the coins tossed at the current round b ecause any other string that the

verier may compute based on these coin tosses is actually determined by them

Note that the pro of systems presented in the pro of of Theorem are of the

publiccoin type whereas this is not the case for the Graph NonIsomorphism pro of

system of Construction Thus although not all natural pro of systems are of

the publiccoin type by Theorem every set having an interactive pro of system

also has a publiccoin interactive pro of system This means that in the context of

interactive proof systems asking random questions is as powerful as asking clever

questions A stronger statement app ears at the end of x

Indeed publiccoin pro of systems are a syntactically restricted type of inter

active pro of systems This restriction may make the design of such systems more

dicult but p otentially facilitates their analysis and esp ecially when the analy

sis refers to a generic system Another advantage of publiccoin pro of systems is

that the veriers actions except for its nal decision are oblivious of the provers

messages This prop erty is used in the pro of of Theorem

Interactive pro of systems with twosided error

In Denition error probability is allowed in the soundness condition but not in

the completeness condition In such a case we say that the pro of system has p erfect

completeness or onesided error probability A more general denition allows an

error probability upp erb ounded by say in b oth the completeness and the

soundness conditions Note that sets having such generalized twosided error

interactive pro ofs are also in P S P AC E and thus by Theorem allowing two

sided error do es not increase the p ower of interactive pro ofs See further discussion

at the end of x

INTERACTIVE PROOF SYSTEMS

A hierarchy of interactive pro of systems

Denition only refers to the total computation time of the verier and thus

allows an arbitrary p olynomial number of messages to b e exchanged A ner

denition refers to the number of messages b eing exchanged also called the number

of rounds

Denition The roundcomplexity of interactive pro of

For an integer function m the complexity class IP m consists of sets having

an interactive proof system in which on common input x at most mjxj

messages are exchanged between the parties

S

def

IP m Thus For a set of integer functions M we let IP M

mM

IP IP poly

For example interactive pro of systems in which the verier sends a single message

that is answered by a single message of the prover corresp onds to IP Clearly

NP IP yet the inclusion may b e strict b ecause in IP the verier may toss

coins after receiving the provers single message Also note that IP coRP

Denition gives rise to a natural hierarchy of interactive pro of systems

where dierent levels of this hierarchy corresp ond to dierent growth rates of

the roundcomplexity of these systems The following results are known regarding

this hierarchy

A linear sp eedup see App endix F or and For every integer

function f such that f n for all n the class IP O f collapses to

the class IP f In particular IP O collapses to IP

The class IP contains sets that are not known to b e in NP eg Graph

NonIsomorphism see Construction However under plausible intractabil

ity assumptions IP NP see

If coNP IP then the PolynomialTime Hierarchy collapses see

It is conjectured that coNP is not contained in IP and consequently that inter

active pro ofs with an unbounded number of message exchanges are more p owerful

than interactive pro ofs in which only a b ounded ie constant number of messages

are exchanged

The class IP also denoted MA seems to b e the real randomized and yet

noninteractive version of NP Here the prover supplies a candidate p olynomial

size pro of and the verier assesses its validity probabilistically rather than

deterministically

An even ner structure emerges when considering also the total length of the messages sent

by the prover see

We count the total number of messages exchanged regardless of the direction of

communication

Note that the linear sp eedup cannot b e applied for an unbounded number of times b ecause

each application may increase eg square the timecomplexity of verication

CHAPTER PROBABILISTIC PROOF SYSTEMS

The IPhierarchy ie IP equals an analogous hierarchy denoted AM

that refers to publiccoin aka ArthurMerlin interactive pro ofs That is for

every integer function f it holds that AMf IP f For f it is also the

case that AMf AMO f actually the aforementioned linear sp eedup for

IP is established by combining the following two results

Emulating IP by AM see xF or IP f AMf

Linear sp eedup for AM see xF or AMf AMf

In particular IP O AM even if AM is restricted such that the verier

tosses no coins after receiving the provers message Note that IP AM

and IP AM are trivial We comment that it is common to shorthand

AM by AM which is indeed inconsistent with the convention of using IP as

shorthand of IP poly

The fact that IP O f IP f is proved by establishing an analogous result

for AM demonstrates the advantage of the publiccoin setting for the study

of interactive pro ofs A similar phenomenon o ccurs when establishing that the

IPhierarchy equals an analogous twosided error hierarchy see Exercise

Something completely dierent

We stress that although we have relaxed the requirements from the verication

pro cedure by allowing it to interact with the prover toss coins and risk some

b ounded error probability we did not restrict the validity of its assertions by

assumptions concerning the p otential prover This should b e contrasted with other

notions of pro of systems such as computationallysound ones see x in

which the validity of the veriers assertions dep ends on assumptions concerning

the p otential provers

On computationally b ounded provers an overview

Recall that our denition of interactive pro ofs ie Denition makes no ref

erence to the computational abilities of the p otential prover This fact has two

conicting consequences

The completeness condition do es not provide any upp er b ound on the com

plexity of the corresp onding proving strategy which convinces the verier to

accept valid assertions

The soundness condition guarantees that regardless of the computational

eort sp end by a cheating prover the verier cannot b e fo oled to accept

invalid assertions with probability exceeding the soundness error

Note that providing an upp erb ound on the complexity of the prescrib ed prover

strategy P of a sp ecic interactive pro of system P V only strengthens the claim

that P V is a pro of system for the corresp onding set of valid assertions We

stress that the prescrib ed prover strategy is referred to only in the completeness

INTERACTIVE PROOF SYSTEMS

condition and is irrelevant to the soundness condition On the other hand relax

ing the denition of interactive pro ofs such that soundness holds only for a sp ecic

class of cheating prover strategies rather than for all cheating prover strategies

weakens the corresp onding claim In this advanced section we consider b oth p os

sibilities

Teaching note Indeed this is an advanced subsection which is b est left for indep en

dent reading It merely provides an overview of the various notions and the reader is

directed to the chapters notes for further detail ie p ointers to the relevant literature

How p owerful should the prover b e

Supp ose that a set S is in IP This means that there exists a verier V that

can b e convinced to accept any input in S but cannot b e fo oled to accept any

input not in S except with small probability One may ask how p owerful should

a prover b e such that it can convince the verier V to accept any input in S

Note that Prop osition asserts that an optimal prover strategy for convincing

any xed verier V can b e implemented in p olynomialspace and that we cannot

exp ect any b etter for a generic set in P S P AC E IP b ecause the emulation of

the interaction of V with any optimal prover strategy yields a decision pro cedure

for the set Still we may seek b etter upp erb ounds on the complexity of some

prover strategy that convinces a sp ecic verier which in turn corresp onds to a

sp ecic set S More interestingly considering all p ossible veriers that give rise to

interactive pro of systems for S we ask what is the minimum p ower required from

a prover that satises the completeness requirement with resp ect to one of these

veriers

We stress that unlike the case of computationallysound pro of systems see

x we do not restrict the p ower of the prover in the soundness condition

but rather consider the minimum complexity of provers meeting the completeness

condition Sp ecically we are interested in relatively ecient provers that meet

the completeness condition The term relatively ecient prover has b een given

three dierent interpretations which are briey surveyed next

A prover is considered relatively ecient if when given an auxiliary input in

addition to the common input in S it works in probabilistic p olynomial

time Sp ecically in case S NP the auxiliary input maybe an NPpro of

that the common input is in the set Still even in this case the interac

tive pro of need not consist of the prover sending the auxiliary input to the

verier for example an alternative pro cedure may allow the prover to b e

zeroknowledge see Construction

This interpretation is adequate and in fact crucial for applications in which

such an auxiliary input is available to the otherwise p olynomialtime parties

Typically such auxiliary input is available in cryptographic applications in

which parties wish to prove in zeroknowledge that they have correctly con

ducted some computation In these cases the NPpro of is just the transcript

CHAPTER PROBABILISTIC PROOF SYSTEMS

of the computation by which the claimed result has b een generated and thus

the auxiliary input is available to the proving party

A prover is considered relatively ecient if it can b e implemented by a proba

bilistic p olynomialtime oracle machine with oracle access to the set S itself

Note that the prover in Construction has this prop erty and see also

Exercise

This interpretation generalizes the notion of selfreducibility of NPpro of sys

tems Recall that by selfreducibility of an NPset or rather of the corre

sp onding NPpro of system we mean that the search problem of nding an

NPwitness is p olynomialtime reducible to deciding membership in the set

cf Denition Here we require that implementing the prover strategy

in the relevant interactive pro of b e p olynomialtime reducible to deciding

membership in the set

A prover is considered relatively ecient if it can b e implemented by a prob

abilistic machine that runs in time that is p olynomial in the deterministic

complexity of the set This interpretation relates the timecomplexity of con

vincing a lazy p erson ie a verier to the timecomplexity of determining

the truth ie deciding membership in the set

Hence in contrast to the rst interpretation which is adequate in settings

where assertions are generated along with their NPpro ofs the current in

terpretation is adequate in settings in which the prover is given only the

assertion and has to nd a pro of to it by itself b efore trying to convince a

lazy verier of its validity

Computationalsoundness

Relaxing the soundness condition such that it only refers to relativelyecient ways

of trying to fo ol the verier rather than to all p ossible ways yields a fundamen

tally dierent notion of a pro of system Assertions proved in such a system are

not necessarily correct they are correct only if the p otential cheating prover do es

not exceed the presumed complexity limits As in x the notion of rela

tive eciency can b e given dierent interpretations the most p opular one b eing

that the cheating prover strategy can b e implemented by a nonuniform fam

ily of p olynomialsize circuits The latter interpretation coincides with the rst

interpretation used in x ie a probabilistic p olynomialtime strategy that

is given an auxiliary input of p olynomial length Sp ecically in this case the

soundness condition is replaced by the following computational soundness condition

that asserts that it is infeasible to fo ol the verier into accepting false statements

Formally

For every prover strategy that is implementable by a family of polynomial

size circuits fC g and every suciently long x f g n S the prob

n

ability that V accepts x when interacting with C is less than

jxj

ZEROKNOWLEDGE PROOF SYSTEMS

As in case of standard soundness the computationalsoundness error can b e re

duced by rep etitions We warn however that unlike in the case of standard sound

ness where b oth sequential and parallel rep etitions will do the computational

soundness error cannot always b e reduced by parallel rep etitions

It is common and natural to consider pro of systems in which the prover strate

gies considered b oth in the completeness and soundness conditions satisfy the same

notion of relative eciency Proto cols that satisfy these conditions with resp ect

to the foregoing interpretation are called arguments We mention that argument

systems may b e more ecient eg in terms of their communication complexity

than interactive pro of systems

ZeroKnowledge Pro of Systems

Standard mathematical pro ofs are b elieved to yield extra knowledge and not

merely establish the validity of the assertion b eing proved that is it is commonly

b elieved that go o d pro ofs provide a deep er understanding of the theorem b eing

proved At the technical level an NPpro of of membership in some set S N P n P

yields something ie the NPpro of itself that is hard to compute even when

assuming that the input is in S For example a coloring of a graph constitutes an

NPpro of that the graph is colorable but it yields information ie the coloring

that seems infeasible to compute when given an arbitrary colorable graph

A natural question that arises is whether or not proving an assertion always

requires giving away some extra knowledge The setting of interactive pro of systems

enables a negative answer to this fundamental question In contrast to NPpro ofs

which seem to yield a lot of knowledge zeroknowledge interactive pro ofs yield no

knowledge at all that is zeroknowledge proofs are both convincing and yet yield

nothing beyond the validity of the assertion being proved For example a zero

knowledge pro of of colorability do es not yield any information ab out the graph

eg partial information ab out a coloring that is infeasible to compute from

the graph itself Thus zeroknowledge pro ofs exhibit an extreme contrast b etween

b eing convincing of the validity of a assertion and teaching anything on top of

the validity of the assertion

Needless to say the notion of zeroknowledge pro ofs is fascinating eg since

it dierentiates pro ofverication from learning Still the reader may wonder

whether such a phenomenon is desirable b ecause in many settings we do care

to learn as much as p ossible rather than learn as little as p ossible However

in other settings most notably in cryptography we may actually wish to limit

the gain that other parties may obtained from a pro of and in particular limit

this gain to the minimal level of b eing convinced in the validity of the assertion

Indeed the applicability of zeroknowledge pro ofs in the domain of cryptography is

vast they are typically used as a to ol for forcing p otentially malicious parties to

b ehave according to a predetermined proto col without having them reveal their

own private inputs The interested reader is referred to discussions in xC

and xC and to detailed treatments in We also mention that in

addition to their direct applicability in Cryptography zeroknowledge pro ofs serve

CHAPTER PROBABILISTIC PROOF SYSTEMS

X X is true! ?? ? ! ?

! !

Figure Zeroknowledge pro ofs an illustration

as a go o d b enchmark for the study of various questions regarding cryptographic

proto cols

Teaching note We b elieve that the treatment of zeroknowledge pro ofs provided in

this section suces for the purp ose of a course in complexity theory For an extensive

treatment of zeroknowledge pro ofs the interested reader is referred to Chap

Denitional Issues

Lo osely sp eaking zeroknowledge pro ofs are pro ofs that yield nothing b eyond the

validity of the assertion that is a verier obtaining such a pro of only gains convic

tion in the validity of the assertion This is formulated by saying that anything that

can b e feasibly obtained from a zeroknowledge pro of is also feasibly computable

from the valid assertion itself The latter formulation follows the simulation

paradigm which is discussed next

A wider p ersp ective the simulation paradigm

In dening zeroknowledge pro ofs we view the verier as a p otential adversary

that tries to gain knowledge from the prescrib ed prover We wish to state that

no feasible adversary strategy for the verier can gain anything from the prover

b eyond conviction in the validity of the assertion The question addressed here

is how to formulate the no gain requirement

Let us consider the desired formulation from a wide p ersp ective A key ques

tion regarding the mo deling of security concerns is how to express the intuitive

requirement that an adversary gains nothing substantial by deviating from the

prescrib ed b ehavior of an honest user The answer is that the adversary gains noth

ing if whatever it can obtain by unrestricted adversarial behavior can be obtained

Recall that when dening a pro of system eg an interactive pro of system we view the

prover as a p otential adversary that tries to fo ol the prescrib ed verier into accepting invalid assertions

ZEROKNOWLEDGE PROOF SYSTEMS

within essential ly the same computational eort by a benign or prescrib ed behav

ior The denition of the b enign b ehavior captures what we want to achieve

in terms of security and is sp ecic to the security concern to b e addressed For

example in the context of zeroknowledge a benign behavior is any computation

that is based only on the assertion itself while assuming that the latter is valid

Thus a zeroknowledge pro of is an interactive pro of in which no feasible adversar

ial verier strategy can obtain from the interaction more than a b enign party

which b elieves the assertion can obtain from the assertion itself

The foregoing interpretation of gaining nothing means that any feasible ad

versarial b ehavior can b e simulated by a b enign b ehavior and thus there is no

gain in the former This line of reasoning is called the simulation paradigm and

is pivotal to many denitions in cryptography eg it underlies the denitions of

security of encryption schemes and cryptographic proto cols for further details see

App endix C

The basic denitions

We turn back to the concrete task of dening zeroknowledge Firstly we com

ment that zeroknowledge is a prop erty of some prover strategies actually more

generally zeroknowledge is a prop erty of some strategies Fixing any strategy

eg a prescrib ed prover we consider what can b e gained ie computed by an

arbitrary feasible adversary eg a verier that interacts with the aforementioned

xed strategy on a common input taken from a predetermined set in our case the

set of valid assertions This gain is compared against what can b e computed by an

arbitrary feasible algorithm called a simulator that is only given the input itself

The xed strategy is zeroknowledge if the computational p ower of these two

fundamentally dierent settings is essentially equivalent Details follow

The formulation of the zeroknowledge condition refers to two types of probabil

ity ensembles where each ensemble asso ciates a single probability distribution to

each relevant input eg a valid assertion Sp ecically in the case of interactive

pro ofs the rst ensemble represents the output distribution of the verier after

interacting with the sp ecied prover strategy P on some common input where

the verier is employing an arbitrary ecient strategy not necessarily the sp ecied

one The second ensemble represents the output distribution of some probabilistic

p olynomialtime algorithm which is only given the corresp onding input and do es

not interact with anyone The basic paradigm of zeroknowledge asserts that for

every ensemble of the rst type there exist a similar ensemble of the second type

The sp ecic variants dier by the interpretation given to the notion of similarity

The most strict interpretation leading to p erfect zeroknowledge is that similarity

means equality

Denition p erfect zeroknowledge oversimplied A prover strategy P

In the actual denition one relaxes the requirement in one of the following two ways The

rst alternative is allowing A to run for expected rather than strict p olynomialtime The

second alternative consists of allowing A to have no output with probability at most and

considering the value of its output conditioned on it having output at all The latter alternative

implies the former but the converse is not known to hold

CHAPTER PROBABILISTIC PROOF SYSTEMS

is said to be p erfect zeroknowledge over a set S if for every probabilistic polynomial

time verier strategy V there exists a probabilistic polynomialtime algorithm

A such that

P V x A x for every x S

where P V x is a random variable representing the output of verier V after

interacting with the prover P on common input x and A x is a random variable

representing the output of algorithm A on input x

We comment that any set in coRP has a p erfect zeroknowledge pro of system in

which the prover keeps silence and the verier decides by itself The same holds

for BPP provided that we relax the denition of interactive pro of system to allow

twosided error Needless to say our fo cus is on nontrivial pro of systems that is

pro of systems for sets outside of BPP

A somewhat more relaxed interpretation of the notion of similarity leading

to almostp erfect zeroknowledge aka statistical zeroknowledge is that similar

ity means statistical closeness ie negligible dierence b etween the ensembles

The most lib eral interpretation leading to the standard usage of the term zero

knowledge and sometimes referred to as computational zeroknowledge is that

similarity means computational indistinguishability ie failure of any ecient pro

cedure to tell the two ensembles apart Combining the foregoing discussion with

the relevant denition of computational indistinguishability ie Denition C

we obtain the following denition

Denition zeroknowledge somewhat simplied A prover strategy P is

said to be zeroknowledge over a set S if for every probabilistic polynomialtime

verier strategy V there exists a probabilistic polynomialtime simulator A

such that for every probabilistic polynomialtime distinguisher D it holds that

def

dn max fjPr D x P V x Pr D x A x jg

n

xS fg

is a negligible function We denote by ZK the class of sets having zeroknowledge

interactive proof systems

Denition is a simplied version of the actual denition which is presented in

App endix C Sp ecically in order to guarantee that zeroknowledge is preserved

under sequential comp osition it is necessary to slightly augment the denition by

providing V and A with the same value of an arbitrary p oly jxjbit long aux

iliary input Other denitional issues and related notions are briey discussed in

App endix C

That is d vanishes faster that the recipro cal of any p ositive p olynomial ie for every p ositive

def

p olynomial p and for suciently large n it holds that dn pn Needless to say dn

n

if S f g

ZEROKNOWLEDGE PROOF SYSTEMS

On the role of randomness and interaction It can b e shown that only

sets in BPP have zeroknowledge pro ofs in which the verier is deterministic see

Exercise The same holds for deterministic provers provided that we consider

auxiliaryinput zeroknowledge as in Denition C It can also b e shown that

only sets in BPP have zeroknowledge pro ofs in which a single message is sent see

Exercise Thus b oth randomness and interaction are essential to the non

triviality of zeroknowledge pro of systems For further details see Sec

Advanced Comment Knowledge Complexity Zeroknowledge is the lowest

level of a knowledgecomplexity hierarchy which quanties the knowledge revealed

in an interaction Sp ecically the knowledge complexity of an interactive pro of

system may b e dened as the minimum number of oraclequeries required in order

to eciently simulate an interaction with the prover See Sec for

references

The Power of ZeroKnowledge

When faced with a denition as complex and seemingly selfcontradictory as the

denition of zeroknowledge one should indeed wonder whether the denition can

b e met in a nontrivial manner It turns out that the existence of nontrivial

zeroknowledge pro ofs is related to the existence of intractable problems in NP

In particular we will show that if oneway functions exist then every NPset has a

zeroknowledge pro of system For the converse see Sec or But

rst we demonstrate the nontriviality of zeroknowledge by a presenting a simple

p erfect zeroknowledge pro of system for a sp ecic NPset that is not known to

b e in BPP In this case we make no intractability assumptions yet the result is

signicant only if NP is not contained in BPP

A simple example

A story not found in the Odyssey refers to the not so famous Labyrinth

of the Island of Aeaea The Sorceress Circe daughter of Helius chal

lenged go dlike Odysseus to traverse the Labyrinth from its North Gate

to its South Gate Canny Odysseus doubted whether such a path ex

isted at all and asked b eautiful Circe for a pro of to which she replied

that if she showed him a path this would trivialize for him the chal

lenge of traversing the Labyrinth Not necessarily clever Odysseus

replied you can use your magic to transp ort me to a random place in

the labyrinth and then guide me by a random walk to a gate of my

choice If we rep eat this enough times then Ill b e convinced that there

is a labyrinthpath b etween the two gates while you will not reveal to

me such a path Indeed wise Circe thought to herself showing

this mortal a random path from a random lo cation in the labyrinth to

Recall that any set in BPP has a trivial zeroknowledge twosided error pro of system in

which the verier just determines membership by itself Thus the issue is the existence of zero

knowledge pro ofs for sets outside BPP

CHAPTER PROBABILISTIC PROOF SYSTEMS

the gate he chooses will not teach him more than his taking a random

walk from that gate

The foregoing story illustrates the main idea underlying the zeroknowledge pro of

for Graph Isomorphism presented next Recall that the set of pairs of isomorphic

graphs is not known to b e in BPP and thus the straightforward NPpro of system

in which the prover just supplies the isomorphism may not b e zeroknowledge

Furthermore assuming that Graph Isomorphism is not in BPP this set has no

zeroknowledge NPpro of system Still as we shall shortly see this set do es have

a zeroknowledge interactive pro of system

Construction zeroknowledge pro of for Graph Isomorphism

Common Input A pair of graphs G V E and G V E

If the input graphs are indeed isomorphic then we let denote an arbitrary

isomorphism between them that is is a and onto mapping of the vertex

set V to the vertex set V such that fu v g E if and only if fv ug

E

Provers rst Step P The prover selects a random isomorphic copy of

G and sends it to the verier Namely the prover selects at random with

uniform probability distribution a permutation from the set of permutations

over the vertex set V and constructs a graph with vertex set V and edge set

def

E ff u v g fu v g E g

The prover sends V E to the verier

Motivating Remark If the input graphs are isomorphic as the prover claims

then the graph sent in Step P is isomorphic to both input graphs However

if the input graphs are not isomorphic then no graph can be isomorphic to

both of them

Veriers rst Step V Upon receiving a graph G V E from the

prover the verier asks the prover to show an isomorphism between G and

one of the input graphs chosen at random by the verier Namely the verier

uniformly selects f g and sends it to the prover who is supposed to

answer with an isomorphism between G and G

Provers second Step P If the message received from the verier equals

then the prover sends to the verier Otherwise ie the prover

def

sends ie the composition of on dened as v v

to the verier

Indeed the prover treats any as Thus in the analysis we shal l

assume without loss of generality that f g always holds

Veriers second Step V If the message denoted received from the

prover is an isomorphism between G and G then the verier outputs

otherwise it outputs

ZEROKNOWLEDGE PROOF SYSTEMS

The verier strategy in Construction is easily implemented in probabilistic

p olynomialtime If the prover is given an isomorphism b etween the input graphs as

auxiliary input then also the provers program can b e implemented in probabilistic

p olynomialtime The motivating remark justies the claim that Construction

constitutes an interactive pro of system for the set of pairs of isomorphic graphs

Thus we fo cus on establishing the zeroknowledge prop erty

We consider rst the sp ecial case in which the verier actually follows the

prescrib ed strategy and selects at random and in particular obliviously of the

graph G it receives The view of this verier can b e easily simulated by selecting

and at random constructing G as a random isomorphic copy of G via

the isomorphism and outputting the triple G Indeed even in this

case the simulator b ehaves dierently from the prescrib ed prover which selects

G as a random isomorphic copy of G via the isomorphism but its output

distribution is identical to the veriers view in the real interaction However

the foregoing description assumes that the verier follows the prescrib ed strategy

while in general the verier may adversarially select dep ending on the graph

G Thus a slightly more complicated simulation describ ed next is required

A general clarication may b e in place Recall that we wish to simulate the

interaction of an arbitrary verier strategy with the prescrib ed prover Thus this

simulator must dep end on the corresp onding verier strategy and indeed we shall

describ e the simulator while referring to such a generic verier strategy Formally

this means that the simulators program incorp orates the program of the corre

sp onding verier strategy Actually the following simulator uses the generic verier

strategy as a subroutine

Turning back to the sp ecic proto col of Construction the basic idea is that

simulator tries to guess and completes a simulation if its guess turns out to b e

correct Sp ecically the simulator selects f g uniformly hoping that the

verier will later select and constructs G by randomly p ermuting G and

thus b eing able to present an isomorphism b etween G and G Recall that the

simulator is analyzed only on yesinstances ie the input graphs G and G are

isomorphic The p oint is that if G and G are isomorphic then the graph G

do es not yield any information regarding the simulators guess ie Thus

but not on the value selected by the adversarial verier may dep end on G

which implies that Pr In other words the simulators guess ie

is correct ie equals with probability Now if the guess is correct then the

simulator can pro duce an output that has the correct distribution and otherwise

the entire pro cess is rep eated

Digest a few useful conventions We highlight three conventions that were

either used implicitly in the foregoing analysis or can b e used to simplify the

description of this andor other zeroknowledge simulators

Without loss of generality we may assume that the cheating verier strategy

is implemented by a deterministic p olynomialsize circuit or equivalently

Indeed this observation is identical to the observation made in the analysis of the soundness

of Construction

CHAPTER PROBABILISTIC PROOF SYSTEMS

by a deterministic p olynomialtime algorithm with an auxiliary input

This is justied by xing any outcome of the veriers coins and observing

that our uniform simulation of the various residual deterministic strategies

yields a simulation of the original probabilistic strategy

Without loss of generality it suces to consider cheating veriers that only

output their view of the interaction ie the common input their internal

coin tosses and the messages that they have received In other words it

suces to simulate the view that cheating veriers have of the real interaction

This is justied by noting that the nal output of any verier can b e obtained

from its view of the interaction where the complexity of the transformation

is upp erb ounded by the complexity of the veriers strategy

Without loss of generality it suces to construct a weak simulator that

pro duces output with some noticeable probability such that whenever an

output is pro duced it is distributed correctly ie similarly to the distri

bution o ccuring in real interactions with the prescrib ed prover

This is justied by rep eatedly invoking such a weak simulator p olynomially

many times and using the rst output pro duced by any of these invocations

Note that by using an adequate number of invocations we fail to pro duce

an output with negligible probability Furthermore note that a simulator

that fails to pro duce output with negligible probability can b e converted

to a simulator that always pro duces an output while incurring a negligible

statistic deviation in the output distribution

The full p ower of zeroknowledge pro ofs

The zeroknowledge pro of system presented in Construction refers to one sp e

cic NPset that is not known to b e in BPP It turns out that under reasonable

assumptions zeroknowledge can b e used to prove membership in any NPset In

tuitively it suces to establish this fact for a single NPcomplete set and thus we

fo cus on presenting a zeroknowledge pro of system for the set of colorable graphs

It is easy to prove that a given graph G is colorable by just presenting a

coloring of G and the same holds for membership in any set in NP but this NP

pro of is not a zeroknowledge pro of unless NP BPP In fact assuming NP

BPP graph colorability has no zeroknowledge NPpro of system Still as we

shall shortly see graph colorability do es have a zeroknowledge interactive pro of

system This pro of system will b e describ ed while referring to b oxes in which

information can b e hidden and later revealed Such b oxes can b e implemented

using oneway functions see eg Theorem

This observation is not crucial but it do es simplify the analysis by eliminating the need to

sp ecify a sequence of coin tosses in each invocation of the veriers strategy

Recall that a probability is called noticeable if it is greater than the recipro cal of some p ositive

p olynomial in the relevant parameter

ZEROKNOWLEDGE PROOF SYSTEMS

Construction Zeroknowledge pro of of colorability abstract description

The description refers to abstract nontransparent boxes that can be perfectly locked

and unlocked such that these boxes perfectly hide their contents while being locked

Common Input A simple graph G V E

Provers rst step Let be a coloring of G The prover selects a random

def

permutation over f g and sets v v for each v V

Hence the prover forms a random relabeling of the coloring The prover

sends to the verier a sequence of jV j locked and nontransparent boxes such

th

that the v box contains the value v

Veriers rst step The verier uniformly selects an edge fu v g E and

sends it to the prover

Motivating Remark The boxes are supposed to contain a coloring of the

graph and the verier asks to inspect the colors of vertices u and v Indeed

for the zeroknowledge condition it is crucial that the prover only responds

to pairs that correspond to edges of the graph

Provers second step Upon receiving an edge fu v g E the prover sends to

the verier the keys to boxes u and v

For simplicity of the analysis if the verier sends fu v g E then the prover

behaves as if it has received a xed or random edge in E rather than sus

pending the interaction which would have been the natural thing to do

Veriers second step The verier unlocks and opens boxes u and v and

accepts if and only if they contain two dierent elements in f g

The verier strategy in Construction is easily implemented in probabilistic

p olynomialtime The same holds with resp ect to the provers strategy provided

that it is given a coloring of G as auxiliary input Clearly if the input graph

is colorable then the verier accepts with probability when interacting with

the prescrib ed prover On the other hand if the input graph is not colorable

then any contents put in the b oxes must b e invalid with resp ect to at least one

Hence edge and consequently the verier will reject with probability at least

jE j

the foregoing proto col exhibits a nonnegligible gap in the accepting probabilities

b etween the case of colorable graphs and the case of noncolorable graphs To

increase the gap the proto col may b e rep eated suciently many times of course

using indep endent coin tosses in each rep etition

So far we showed that Construction constitutes a weak form of an in

teractive pro of system for Graph Colorability The p oint however is that the

prescrib ed prover strategy is zeroknowledge This is easy to see in the abstract

setting of Construction b ecause all that the verier sees in the real interac

tion is a sequence of b oxes and a random pair of dierent colors which is easy to

simulate Indeed the simulation of the real interaction pro ceeds by presenting a

sequence of b oxes and providing a random pair of dierent colors as the contents

CHAPTER PROBABILISTIC PROOF SYSTEMS

of the two b oxes indicated by the verier Note that the foregoing argument relies

on the fact that the b oxes indicated by the verier corresp ond to vertices that

are connected by an edge in the graph

This simple demonstration of the zeroknowledge prop erty is not p ossible in

the digital implementation discussed next b ecause in that case the b oxes are

not totally unaected by their contents but are rather aected yet in an indistin

guishable manner Thus the veriers selection of the insp ected edge may dep end

on the outside app earance of the various b oxes which in turn may dep end in

an indistinguishable manner on the contents of these b oxes Consequently we

cannot determine the b oxes contents after a pair of b oxes are selected and so the

simple foregoing simulation is inapplicable Instead we simulate the interaction as

follows

We rst guess at random which pair of b oxes corresp onding to an edge

the verier would ask to op en and place a random pair of distinct colors

in these b oxes and garbage in the rest Then we hand all b oxes to the

verier which asks us to op en a pair of b oxes corresp onding to an edge

If the verier asks for the pair that we chose ie our guess is successful

then we can complete the simulation by op ening these b oxes Otherwise we

try again ie rep eat Step with a new random guess and random colors

The key observation is that if the b oxes hide the contents in the sense that

a b oxs contents is indistinguishable based on it outside app earance then

our guess will succeed with probability approximately jE j Furthermore

in this case the simulated execution will b e indistinguishable from the real

interaction

Thus it suces to use b oxes that hide their contents almost p erfectly rather than

b eing p erfectly opaque Such b oxes can b e implemented digitally

Teaching note Indeed we recommend presenting and analyzing in class only the

foregoing abstract proto col It suces to briey comment ab out the digital implemen

tation rather than presenting a formal pro of of Theorem which can b e found

in or Sec

Digital implementation overview We implement the abstract b oxes re

ferred to in Construction by using adequately dened commitment schemes

Lo osely sp eaking such a scheme is a twophase game b etween a sender and a re

ceiver such that after the rst phase the sender is committed to a value and yet

at this stage it is infeasible for the receiver to nd out the committed value ie

the commitment is hiding The committed value will b e revealed to the receiver

in the second phase and it is guaranteed that the sender cannot reveal a value other

than the one committed ie the commitment is binding Such commitment

An alternative and more ecient simulation consists of putting random indep endent colors

in the various b oxes hoping that the verier asks for an edge that is prop erly colored The latter

event o ccurs with probability approximately provided that the b oxes hide their contents

almost p erfectly

ZEROKNOWLEDGE PROOF SYSTEMS

schemes can b e implemented assuming the existence of oneway functions as in

Denition see xC

Zeroknowledge pro ofs for other NPsets Using the fact that colorability

is NPcomplete one can derive from Construction zeroknowledge pro of sys

tems for any NPset Furthermore NPwitnesses can b e eciently transformed

into p olynomialsize circuits that implement the corresp onding prescrib ed zero

knowledge prover strategies

Theorem The ZK Theorem Assuming the existence of nonuniformly

hard oneway functions it holds that NP ZK Furthermore every S NP has

a computational zeroknowledge interactive proof system in which the prescribed

prover strategy can be implemented in probabilistic polynomialtime provided that

it is given as auxiliaryinput an NPwitness for membership of the common input

in S

The hypothesis of Theorem ie the existence of oneway functions seems un

avoidable b ecause the existence of zeroknowledge pro ofs for hard on the average

problems implies the existence of oneway functions and likewise the existence

of zeroknowledge pro ofs for sets outside BPP implies the existence of auxiliary

input oneway functions

Theorem has a dramatic eect on the design of cryptographic proto cols

see App endix C In a dierent vein we mention that under the same assumption

any interactive pro of can b e transformed into a zeroknowledge one This trans

formation however do es not necessarily preserve the complexity of the prover

Theorem The ultimate ZK Theorem Assuming the existence of non

uniformly hard oneway functions it holds that IP ZK

Lo osely sp eaking Theorem can b e proved by recalling that IP AMpoly

and mo difying any publiccoin proto col as follows the mo died prover sends com

mitments to its messages rather than the messages themselves and once the orig

inal interaction is completed it proves in zeroknowledge that the corresp onding

transcript would have b een accepted by the original verier Indeed the latter as

sertion is of the NP type and thus the zeroknowledge pro of system guaranteed

in Theorem can b e invoked for proving it

Reection The pro of of Theorem uses the fact that colorability is NP

complete in order to obtain a zeroknowledge pro ofs for any set in NP by using such

a proto col for colorability ie Construction Thus an NPcompleteness

result is used here in a p ositive way that is in order to construct something

rather than in order to derive a negative hardness result cf Section

Actually we should either rely on the fact that the standard Karpreductions are invertible

in p olynomial time or on the fact that the colorability proto col is actually zeroknowledge with

resp ect to auxiliary inputs as in Denition C

Historically the pro of of Theorem was probably the rst p ositive application of NP

completeness Subsequent p ositive uses of completeness results have app eared in the context of

CHAPTER PROBABILISTIC PROOF SYSTEMS

Perfect and Statistical ZeroKnowledge The foregoing results which refer

to computational zeroknowledge pro of systems should b e contrasted with the

known results regarding the complexity of statistical zeroknowledge pro of systems

Statistical zeroknowledge pro of systems exist only for sets in IP coIP and

thus are unlikely to exist for all NPsets On the other hand the class Statistical

ZeroKnowledge is known to contain some seemingly hard problems and turns

out to have interesting complexity theoretic prop erties eg b eing closed under

complementation and having very natural complete problems The interested

reader is referred to

Pro ofs of Knowledge a parenthetical subsection

Teaching note Technically sp eaking this topic b elongs to Section but its more

interesting demonstrations refer to zeroknowledge pro ofs of knowledge hence its cur

rent p ositioning

Lo osely sp eaking pro ofs of knowledge are interactive pro ofs in which the prover

asserts knowledge of some ob ject eg a coloring of a graph and not merely

its existence eg the existence of a coloring of the graph which in turn is equiv

alent to the assertion that the graph is colorable Note that the entity asserting

knowledge is actually the provers strategy which is an automated computing de

vice hereafter referred to as a machine This raises the question of what do we

mean by saying that a machine knows something

Abstract reections

Any standard dictionary suggests several meanings for the verb to know but these

are typically phrased with reference to the notion of awareness a notion which is

certainly inapplicable in the context of machines Instead we should lo ok for a

behavioristic interpretation of the verb to know Indeed it is reasonable to link

knowledge with the ability to do something eg the ability to write down whatever

one knows Hence we may say that a machine knows a string if it can output

the string But this seems as total nonsense to o a machine has a well dened

output either the output equals or it do es not so what can b e meant by saying

that a machine can do something

Interestingly a sound interpretation of the latter phrase do es exist Lo osely

sp eaking by saying that a machine can do something we mean that the machine

can b e easily modied such that it or rather its mo died version do es whatever

is claimed More precisely this means that there exists an ecient machine that

using the original machine as a blackbox or given its co de as an input outputs

whatever is claimed

Technically sp eaking using a machine as a blackbox seems more app ealing

when the said machine is interactive ie implements an interactive strategy

Indeed this will b e our fo cus here Furthermore conceptually sp eaking whatever

interactive pro ofs see the pro of of Theorem probabilistically checkable pro ofs see the pro of

of Theorem and the study of statistical zeroknowledge cf

ZEROKNOWLEDGE PROOF SYSTEMS

a machine knows or do es not know is its own business whereas what can b e

of interest and reference to the outside is whatever can b e deduced ab out the

knowledge of a machine by interacting with it Hence we are interested in pro ofs

of knowledge rather than in mere knowledge

A concrete treatment

For sake of simplicity let us consider a concrete question how can a machine prove

that it knows a coloring of a graph An obvious way is just sending the coloring

to the verier Yet we claim that applying the proto col in Construction ie

the zeroknowledge pro of system for Colorability is an alternative way of proving

knowledge of a coloring of the graph

The denition of a verier of know ledge of coloring refers to any p ossible

prover strategy and links the ability to extract a coloring of a given graph

from such a prover to the probability that this prover convinces the verier That is

the denition p ostulates the existence of an ecient universal way of extracting a

coloring of a given graph by using any prover strategy that convinces this verier

to accept this graph with probability or more generally with some noticeable

probability On the other hand we should no exp ect this extractor to obtain

much from prover strategies that fail to convince the verier or more generally

convince it with negligible probability A robust denition should allow a smo oth

transition b etween these two extremes and in particular b etween provers that

convince the verier with noticeable probability and those that convince it with

negligible probability Such a denition should also supp ort the intuition by which

the following strategy of Alice is zeroknowledge Alice sends Bob a coloring of

a given graph provided that Bob has successfully convinced her that he knows this

coloring We stress that the zeroknowledge prop erty of Alices strategy should

hold regardless of the pro ofofknowledge system used for proving Bobs knowledge

of a coloring

Lo osely sp eaking we say that a strategy V constitutes a verier for knowledge

of coloring if for any prover strategy P the complexity of extracting a coloring

of G when using P as a black b ox is inversely prop ortional to the probability

that V is convinced by P to accept the graph G Namely the extraction of the

coloring is done by an oracle machine called an extractor that is given access to

the strategy P ie the function sp ecifying the message that P sends in resp onse to

any sequence of messages it may receive We require that the expected running

time of the extractor on input G and oracle access to P be inversely related by

a factor p olynomial in jGj to the probability that P convinces V to accept G In

particular if P always convinces V to accept G then the extractor runs in exp ected

p olynomialtime The same holds in case P convinces V to accept with noticeable

probability On the other hand if P never convinces V to accept then nothing is

required of the extractor We stress that the latter sp ecial cases do not suce for

For simplicity the reader may consider graphs that have a unique coloring upto a rela

b eling In general we refer here to instances that have unique solution cf Section which

arise naturally in some cryptographic applications

Indeed one may consider also nonblackbox extractors

CHAPTER PROBABILISTIC PROOF SYSTEMS

a satisfactory denition see discussion in Sec

Pro ofs of knowledge and in particular zeroknowledge pro ofs of knowledge

have many applications to the design of cryptographic schemes and cryptographic

proto cols see eg These are enabled by the following general result

Theorem Theorem revisited Assuming the existence of nonuniformly

hard oneway functions any NPrelation has a zeroknowledge proof of know ledge

of a corresp onding NPwitnesses Furthermore the prescribed prover strategy

can be implemented in probabilistic polynomialtime provided it is given such an

NPwitness

Probabilistically Checkable Pro of Systems

Teaching note Probabilistically checkable pro of PCP systems may b e viewed as

a restricted type of interactive pro of systems in which the prover is memoryless and

resp onds to each verier message as if it were the rst such message This p ersp ective

creates a tighter link with previous sections but is somewhat contrived Indeed such

a memoryless prover may b e viewed as a static ob ject that the verier may query at

lo cations of its choice But then it is more app ealing to present the mo del using the

more traditional terminology of oracle machines rather than using and degenerating

the terminology of interactive machines or strategies

Probabilistically checkable pro of systems can b e viewed as standard determinis

tic pro of systems that are augmented with a probabilistic pro cedure capable of

evaluating the validity of the assertion by examining few lo cations in the alleged

pro of Actually we fo cus on the latter probabilistic pro cedure which in turn im

plies the existence of a deterministic verication pro cedure obtained by going over

all p ossible random choices of the probabilistic pro cedure and making the adequate

examinations

Mo deling such probabilistic verication pro cedures which may examine few

lo cations in the alleged pro of requires providing these pro cedures with direct access

to the individual bits of the alleged pro of so that they need not scan the pro of

bitbybit Thus the alleged pro of is a string as in the case of a traditional

pro of system but the probabilistic verication pro cedure is given direct access

to individual bits of this string

We are interested in probabilistic verication procedures that access only few

locations in the proof and yet are able to make a meaningful probabilistic verdict

regarding the validity of the al leged proof Sp ecically the verication pro cedure

should accept any valid pro of with probability but rejects with probability

at least any alleged pro of for a false assertion Such probabilistic verication

pro cedures are called probabilistically checkable proof PCP systems

The fact that one can meaningfully evaluate the correctness of pro ofs by

examining few lo cations in them is indeed amazing and somewhat counterintuitive

Needless to say such pro ofs must b e written in a somewhat nonstandard format

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

b ecause standard pro ofs cannot b e veried without reading them in full since a aw

may b e due to a single improp er inference In contrast pro ofs for a PCP system

tend to b e very redundant they consist of sup eruously many pieces of information

ab out the claimed assertion but their correctness can b e meaningfully evaluated

by checking the consistency of a randomly chosen collection of few related pieces

We stress that by a meaningful evaluation we mean rejecting alleged pro ofs of

false assertions with constant probability rather than with probability that is

inversely prop ortional to the length of the alleged pro of

The main complexity measure asso ciated with PCPs is indeed their query com

plexity Another complexity measure of natural concern is the length of the pro ofs

b eing employed which in turn is related to the randomness complexity of the

system The randomness complexity of PCPs plays a key role in numerous appli

cations eg in comp osing PCP systems as well as when applying PCP systems to

derive inapproximability results and thus we sp ecify this parameter rather than

the pro of length

Teaching note Indeed PCP systems are most famous for their role in deriving nu

merous inapproximability results see Section but our view is that the latter

is merely one extremely imp ortant application of the fundamental notion of a PCP

system Our presentation is organized accordingly

Denition

Lo osely sp eaking a probabilistically checkable pro of system consists of a probabilis

tic p olynomialtime verier having access to an oracle that represents an alleged

pro of in redundant form Typically the verier accesses only few of the oracle

bits and these bit p ositions are determined by the outcome of the veriers coin

tosses As in the case of interactive pro of systems it is required that if the asser

tion holds then the verier always accepts ie when given access to an adequate

oracle whereas if the assertion is false then the verier must reject with proba

no matter which oracle is used The basic denition of the PCP bility at least

setting is given in Part of the following denition Yet the complexity measures

introduced in Part are of key imp ortance for the subsequent discussions

Denition Probabilistically Checkable Pro ofs PCP

A probabilistically checkable proof system PCP for a set S is a probabilistic

polynomialtime oracle machine called verier and denoted V that satises

the fol lowing two conditions

Completeness For every x S there exists an oracle such that on

x

input x and access to oracle machine V always accepts x

x

Soundness For every x S and every oracle on input x and access

to oracle machine V rejects x with probability at least

We say that a probabilistically checkable proof system has query complexity

q N N if on any input of length n the verier makes at most q n oracle

CHAPTER PROBABILISTIC PROOF SYSTEMS

queries Similarly the randomness complexity r N N upperbounds the

number of coin tosses performed by the verier on a generic nbit long input

For integer functions r and q we denote by PCP r q the class of sets having

probabilistically checkable proof systems of randomness complexity r and query

complexity q For sets of integer functions R and Q

def

PCP R Q PCP r q

r R q Q

The error probability in the soundness condition of PCP systems can b e reduced

by successive applications of the pro of system In particular rep eating the pro cess

for k times reduces the probability that the verier is fo oled by a false assertion to

k

whereas all complexities increase by at most a factor of k Thus PCP systems

of nontrivial querycomplexity cf Section provide a tradeo b etween the

number of lo cations examined in the pro of and the condence in the validity of the

assertion

We note that the oracle referred to in the completeness condition of a PCP

x

system constitutes a pro of in the standard mathematical sense Indeed any PCP

system yields a standard pro of system with resp ect to a verication pro cedure

that scans all p ossible outcomes of V s internal coin tosses and emulates all the

corresp onding checks Furthermore the oracles in PCP systems of logarithmic

randomnesscomplexity constitute NPpro ofs see Exercise However the

oracles of a PCP system have the extra remarkable property of enabling a lazy

verier to toss coins take its chances and assess the validity of the pro of without

reading all of it but rather by reading a tiny p ortion of it Potentially this allows

the verier to examine very few bits of an NPpro of and even utilize very long

pro ofs ie of sup erp olynomial length

Adaptive versus nonadaptive veriers Denition allows the verier

to b e adaptive that is the verier may determine its queries based on the an

swers it has received to previous queries in addition to their dep endence on the

input and on the veriers internal coin tosses In contrast nonadaptive veriers

determine all their queries based solely on their input and internal coin tosses

P

q

i q

non Note that q adaptive binary queries can b e emulated by

i

adaptive binary queries We comment that most constructions of PCP systems

use nonadaptive veriers and in fact in many sources PCP systems are dened as

nonadaptive

Randomness versus pro of length Fixing a verier V we say that lo cation

i in the oracle is relevant to input x if there exists a computation of V on input

x in which lo cation i is queried ie there exists and such that on input

x randomness and access to the oracle the verier queries lo cation i The

eective proof length of V is the smallest function N N such that for every

input x there are at most jxj lo cations in the oracle that are relevant to x

As usual in complexity theory the oracle answers are binary values ie either or

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

We claim that the eective pro of length of any PCP system is closely related to

its randomness and query complexity On one hand if the PCP system has

randomnesscomplexity r and querycomplexity q then its eective proof length is

r q r

upperbounded by whereas a b ound of q holds for nonadaptive systems

see Exercise Thus PCP systems of logarithmic randomness complexity have

eective proof length that is polynomial and hence yield NPpro of systems On the

other hand in some sense the randomness complexity of a PCP system can b e

upp erb ounded by the logarithm of the eective length of the pro ofs employed

provided we allow nonuniform veriers see Exercise

On the role of randomness The PCP Theorem ie NP PCP log O

asserts that a meaningful probabilistic evaluation of pro ofs is p ossible based on

a constant number of examined bits We note that unless P NP such a

phenomena is imp ossible when requiring the verier to b e deterministic Firstly

r

q r

note that PCP O P holds as a sp ecial case of PCP r q Dtime

p oly see Exercise Secondly as shown in Exercise P NP implies that

NP is not contained in PCP olog olog Lastly assuming that not all NPsets

have NPpro of systems that employs pro ofs of length eg n n it follows

r n

that if q n n then PCP r q do es not contain NP see Exercise

again

The Power of Probabilistically Checkable Pro ofs

The celebrated PCP Theorem asserts that NP PCP log O and this result

is indeed the fo cus of the current section But b efore getting to it we make several

simple observations regarding the PCP Hierarchy

We rst note that PCP poly equals coRP whereas PCP poly equals

NP It is easy to prove an upp er b ound on the nondeterministic time complexity

of sets in the PCP hierarchy see Exercise

Prop osition upp erb ounds on the p ower of PCPs For every polynomially

r

bounded integer function r it holds that PCP r poly Ntime poly In

particular PCP log poly NP

The fo cus on PCP systems of logarithmic randomness complexity reects an inter

est in PCP systems that utilize pro of oracles of p olynomial length see discussion in

Section We stress that such PCP systems ie PCP log q are NPpro of

systems with a p otentially amazing extra prop erty the validity of the assertion

can b e probabilistically evaluated by examining a small p ortion ie q n bits

of the pro of Thus for any xed p olynomially b ounded function q a result of the

form

NP PCP log q

is interesting b ecause it applies also to NPsets having witnesses of length exceed

ing q Needless to say the smaller q the b etter The PCP Theorem asserts the

amazing fact by which q can b e made a constant

CHAPTER PROBABILISTIC PROOF SYSTEMS

Theorem The PCP Theorem NP PCP log O

Thus probabilistically checkable pro ofs in which the verier tosses only logarith

mically many coins and makes only a constant number of queries exist for every

set in NP This constant is essentially three see x Before reviewing the

pro of of Theorem we make a couple of comments

Ecient transformation of NPwitnesses to PCP oracles The pro of of

Theorem is constructive in the sense that it allows to eciently transform

any NPwitness for an instance of a set in NP into an oracle that makes the

PCP verier accept with probability That is for every NPwitness relation

R PC there exists a PCP verier V as in Theorem and a polynomialtime

computable function such that for every x y R the verier V always accepts the

xy

input x when given oracle access to the proof x y ie Pr V x

Recalling that the latter oracles are themselves NPpro ofs it follows that NPpro ofs

can b e transformed into NPpro ofs that oer a tradeo b etween the p ortion of the

pro of b eing read and the condence it oers Sp ecically for every if one is

willing to tolerate an error probability of then it suces to examine O log

bits of the transformed NPpro of Indeed as discussed in Section these

bit lo cations need to b e selected at random

The foregoing strengthening of Theorem oers a wider range of applica

tions than Theorem itself Indeed Theorem itself suces for negative

applications such as establishing the infeasibility of certain approximation prob

lems see Section But for p ositive applications see x typically

some user or a real entity will b e required to actually construct the PCPoracle

and in such cases the strengthening of Theorem will b e useful

A characterization of NP Combining Theorem with Prop osition we

obtain the following characterization of NP

Corollary The PCP characterization of NP NP PCP log O

Roadmap for the pro of of the PCP Theorem Theorem is a culmina

tion of a sequence of remarkable works each establishing meaningful and increas

ingly stronger versions of Eq A presentation of the full pro of of Theorem

is b eyond the scop e of the current work and is in our opinion unsuitable for a

basic course in complexity theory Instead we present an overview of the original

pro of see x as well as of an alternative pro of see x which was found

more than a decade later We will start however by presenting a weaker result

that is used in b oth pro ofs of Theorem and is also of indep endent interest

This weaker result see x asserts that every NPset has a PCP system with

constant querycomplexity alb eit with p olynomial randomness complexity that

is NP PCP poly O

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

Teaching note In our opinion presenting in class any part of the pro of of the PCP

Theorem should b e given low priority In particular presenting the connections b etween

PCP and the complexity of approximation should b e given a higher priority As for

relative priorities among the following three subsections we strongly recommend giving

x the highest priority b ecause it oers a direct demonstration of the p ower of

PCPs As for the two alternative pro ofs of the PCP Theorem itself our recommendation

dep ends on the intended goal On one hand for the purp ose of merely giving a taste

of the ideas involved in the pro of we prefer an overview of the original pro of provided

in x On the other hand for the purp ose of actually providing a full pro of we

denitely prefer the new pro of which is only outlined in x

Proving that NP PCP poly O

The fact that every NPset has a PCP system with constant querycomplexity

regardless of its randomnesscomplexity already testies to the p ower of PCP

systems It asserts that probabilistic verication of proofs is possible by inspecting

very few locations in a p otentially huge proof Indeed the PCP systems presented

next utilize exp onentially long pro ofs but they do so while insp ecting these pro ofs

at a constant number of randomly selected lo cations

We start with a brief overview of the construction We rst note that it suces

to construct a PCP for proving the satisability of a given system of quadratic

equations over GF b ecause this problem is NPcomplete see Exercise

For an input consisting of a system of quadratic equations with n variables the

oracle of this PCP is supp osed to provide the evaluation of all quadratic ex

pressions in these n variables at some xed assignment to these variables This

assignment is supp osed to satisfy the system of quadratic equations that is given as

input We distinguish two tables in the oracle the rst table corresp onding to all

n n

linear expressions and the second table to all quadratic expressions Each

table is tested for selfconsistency via a linearity test and the two tables are

tested to b e consistent with each other via a matrixequality test which utilizes

selfcorrection Finally we test that the assignment enco ded in these tables sat

ises the quadratic system that is given as input This is done by taking a random

linear combination of the quadratic equations that app ear in the quadratic system

and obtaining the value assigned to the corresp onding quadratic expression by the

aforementioned tables again via selfcorrection The key p oint is that each of the

foregoing tests utilizes a constant number of Bo olean queries and has time and

randomness complexity that is p olynomial in the size of the input Details follow

Teaching note The following text refers to notions such as the Hadamard enco ding

testing and selfcorrection which app ear in other parts of this work see eg xE

Section and x resp ectively While a wider p ersp ective provided in the

aforementioned parts is always useful the current text is selfcontained

Here and elsewhere we denote by GF the element eld

CHAPTER PROBABILISTIC PROOF SYSTEMS

The starting p oint We construct a PCP system for the set of satisable

quadratic equations over GF The input is a sequence of such equations over the

variables x x and the pro of oracle consist of two parts or tables which are

n

supp osed to provide information regarding some satisfying assignment

n

also viewed as an nary vector over GF The rst part denoted T is sup

p osed to provide a Hadamard enco ding of the said satisfying assignment that is

n

for every GF this table is supp osed to provide the inner pro duct mo d of

P

n

the nary vectors and ie T is supp osed to equal The second

i i

i

part denoted T is supp osed to provide all linear combinations of the values of

n

the s that is for every GF viewed as an nbyn matrix over GF

i j

P

Indeed T is contained in the value of T is supp osed to equal

ij i j

ij

T b ecause for any GF The PCP verier will use the two tables

for checking that the input ie a sequence of quadratic equations is satised by

the assignment that is enco ded in the two tables Needless to say these tables may

not b e a valid enco ding of any nary vector let alone one that satises the input

and so the verier also needs to check that the enco ding is close to b eing valid

We will fo cus on this task rst

Testing the Hadamard Co de Note that T is supp osed to enco de a linear

n

function that is there must exist some GF such that T

n

P

n

n

holds for every GF This can b e tested by selecting

i i n

i

n

uniformly GF and checking whether T T T

where denotes addition of vectors over GF The analysis of this natural

tester turns out to b e quite complex Nevertheless it is indeed the case that any

table that is far from b eing linear is rejected with probability at least

see Exercise where T is far from b eing linear if T disagrees with any linear

function f on more than an fraction of the domain ie Pr T r f r

r

By rep eating the linearity test for a constant number of times we may reject

each table that is far from b eing a co deword of the Hadamard Co de with

probability at least Thus using a constant number of queries the verier

n

rejects any T that is far from b eing a Hadamard enco ding of any GF

and likewise rejects any T that is far from b eing a Hadamard enco ding of

n

any GF We may thus assume that T resp T is close to the

Hadamard enco ding of some resp Needless to say this do es not mean

that equals the outer pro duce of with itself

n n

In the rest of the analysis we x GF and GF and denote the

n n

GF Hadamard enco ding of resp by f GF GF resp f

GF Recall that T resp T is close to f resp f

Selfcorrection of the Hadamard Co de Supp ose that T is close to a linear

m

function f GF GF ie Pr T r f r Then we can recover

r

the value of f at any desired p oint x by making two random queries to T

Note that resp is uniquely determined by T resp T b ecause every two dierent

m

linear functions GF GF agree on exactly half of the domain ie the Hadamard co de

has relative distance

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

m

Sp ecically for a uniformly selected r GF we use the value T x r T r

Note that the probability that we recover the correct value is at least b ecause

Pr T x r T r f x r f r and f x r f r f x by

r

linearity of f Needless to say for the function T cannot b e close to

two dierent linear functions Thus assuming that T is close to f resp

we may correctly recover ie with error probability T is close to f

at any desired p oint by making queries to T resp the value of f resp f

T This pro cess is called selfcorrection cf eg x r s r τ τ s .

A = = fτ (r) fτ (s)

Figure Detail for testing consistency of linear and quadratic forms

Supp ose that we are given access to Checking consistency of f and f

P

n n

GF GF where f f GF GF and f

i i

i

P

and that we wish to verify that for ev and f

i j

ij ij ij

ij

ery i j f ng In other words we are given a somewhat weird enco ding

and we wish to check whether of two matrices A and A

ij i j ij

ij

or not these matrices are identical It can b e shown see Exercise that if

A A then Pr r As r A s where r and s are uniformly distributed

rs

nary vectors Note that in our case where A and A it

i j ij ij

ij

P P

r s f r f s see Figure and r A s holds that r As

i i j j

i j

P P

r s where r s is the outerpro duct of s and r Thus for r s f

i j

ij

i j

r s we have Pr f r f s f

i j ij ij rs

ij

Recall however that we do not have direct access to the functions f and f

but rather to tables ie T and T that are close to these functions Still

at any desired p oint using selfcorrection we can obtain the values of f and f

with very high probability Actually when implementing the foregoing consistency

b ecause we use the values of f at test it suces to use selfcorrection for f

n

two indep endently and uniformly distributed p oints in GF ie r s but the

n

is required at r s which is not uniformly distributed in GF Thus value f

n

by selecting uniformly r s GF and we test the consistency of f and f

n

R GF and checking that T r T s T r s R T R

By rep eating the aforementioned selfcorrected consistency test for a constant

number of times we may reject an inconsistent pair of tables with probability at

least Thus in the rest of the analysis we may assume that

i j ij ij

ij

Indeed this fact follows from the selfcorrection argument but a simpler pro of merely refers

to the fact that the Hadamard co de has relative distance

CHAPTER PROBABILISTIC PROOF SYSTEMS

Checking that satises the quadratic system Supp ose that we are given

access to f and f as in the foregoing where in particular A key

observation is that if do es not satisfy a system of quadratic equations then

with probability it do es not satisfy a random linear combination of these

equations Thus in order to check whether satises the quadratic system which

is given as input we create a single quadratic equation by taking such a random

linear combination and check whether this quadratic equation is satised by

The punchline is that testing whether satises the quadratic equation Qx

Q Again the actual checking is implemented amounts to testing whether f

by using selfcorrection of the table T

This completes the description of the verier Note that this verier p erforms

a constant number of co deword tests for the Hadamard Co de and a constant

number of consistency and satisability tests where each of the latter involves self

correction of the Hadamard Co de Each of the individual tests utilizes a constant

number of queries ranging b etween two and four and uses randomness that is

quadratic in the number of variables and linear in the number of equations in the

input Thus the querycomplexity is a constant and the randomnesscomplexity

is at most quadratic in the length of the input quadratic system Clearly if

the input quadratic system is satisable by some then the verier accepts the

corresp onding tables T and T ie T f and T f with probability

On the other hand if the input quadratic system is unsatisable then any pair of

tables T T will b e rejected with constant probability by one of the foregoing

tests It follows that NP PCP q O where q is a quadratic p olynomial

Reection Indeed the actual test of the satisability of the quadratic system

that is given as input is facilitated by the fact that a satisfying assignment is

enco ded in the oracle in a very redundant manner which ts the nal test of

satisability But then the burden of testing moves to checking that this enco ding

is indeed valid In fact most of the tests p erformed by the foregoing verier are

aimed at verifying the validity of the enco ding Such a test of validity of enco ding

may b e viewed as a test of consistency b etween the various parts of the enco ding

All these themes are present also in more advanced constructions of PCP systems

Overview of the rst pro of of the PCP Theorem

The original pro of of the PCP Theorem Theorem consists of three main

conceptual steps which we briey sketch rst and further discuss later

Constructing a nonadaptive PCP system for NP having logarithmic ran

domness and polylogarithmic query complexity that is this PCP has the

desired randomness complexity and a very low but nonconstant query com

plexity Furthermore this pro of system has additional prop erties that enable

pro of comp osition as in the following Step

Constructing a PCP system for NP having polynomial randomness and con

stant query complexity that is this PCP has the desired constant query

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

complexity but its randomness complexity is prohibitingly high Indeed we

showed such a construction in x Furthermore this pro of system to o

has additional prop erties enabling pro of comp osition as in Step

The proof comp osition paradigm In general this paradigm allows to com

p ose two pro of systems such that the inner one is used for probabilistically

verifying the acceptance criteria of the outer verier The aim is to conduct

this comp osed verication using much fewer queries than the query com

plexity of the outer pro of system In particular the inner verier cannot

aord to read its input which makes comp osition more subtle than the term

suggests

Lo osely sp eaking the outer verier should b e robust in the sense that its

soundness condition guarantee that with high probability the oracle answers

are far from satisfying the residual decision predicate rather than merely

not satisfy it Furthermore the latter predicate which is welldened by

the nonadaptive nature of the outer verier must have a circuit of size

b ounded by a p olynomial in the number of queries The inner verier is

given oracle access to its input and is charged for each query made to it but

is only required to reject with high probability inputs that are far from b eing

valid and as usual accept inputs that are valid That is the inner verier

is actually a verier of proximity

Comp osing two such PCPs yields a new PCP for NP where the new pro of

oracle consists of the pro of oracle of the outer system and a sequence of

pro of oracles for the inner system one inner pro of p er each p ossible

randomtap e of the outer verier The resulting verier selects coins for

the outerverier and uses the corresp onding inner pro of in order to verify

that the outerverier would have accepted under this choice of coins Note

that such a choice of coins determines lo cations in the outer pro of that the

outerverier would have insp ected and the combined verier provides the

innerverier with oracle access to these lo cations which the innerverier

considers as its input as well as with oracle access to the corresp onding

inner pro of which the innerverier considers as its pro oforacle

Note that comp osing an outerverier of randomnesscomplexity r and query

complexity q with an innerverier of randomnesscomplexity r and query

complexity q yields a PCP of randomnesscomplexity r n r nr q n

and querycomplexity q n q q n b ecause q n represents the length

of the input oracle that is accessed by the innerverier Recall that the

outerverier is nonadaptive and thus if the innerverier is nonadaptive

resp robust then so is the verier resulting from the comp osition which is

imp ortant in case we wish to comp ose the latter verier with another inner

verier

In particular the pro of system of Step is comp osed with itself using r n

r n O log n and q n q n p oly log n yielding a PCP system for

Our presentation of the comp osition paradigm follows rather than the original presen

tation of

CHAPTER PROBABILISTIC PROOF SYSTEMS

NP of randomnesscomplexity r n r n r q n O log n and query

complexity q n q q n p oly log log n Comp osing the latter system used

as an outer system with the PCP system of Step yields a PCP system for

NP of randomnesscomplexity r n p oly q n O log n and querycomplexity

O thus establishing the PCP Theorem

A more detailed overview the plan The foregoing description uses two

nontrivial PCP systems and refers to additional prop erties such as robustness

and verication of proximity A PCP system of p olynomial randomnesscomplexity

and constant querycomplexity as p ostulated in Step is outlined in x We

thus start by discussing the notions of verifying proximity and b eing robust while

demonstrating their applicability to the said PCP Finally we outline the other

PCP system that is used ie the one p ostulated in Step

PCPs of Proximity Recall that a standard PCP verier gets an explicit input

and is given oracle access to an alleged pro of for membership of the input in a

predetermined set In contrast a PCP of proximity verier is given direct access

to two oracles one representing an input and the other being an al leged proof

and its queries to both oracles are counted in its querycomplexity Typically the

querycomplexity of this verier is lower than the length of the input oracle and

hence this verier cannot aord reading the entire input and cannot b e exp ected

to make absolute statements ab out it Indeed instead of deciding whether or not

the input is in a predetermined set the verier is only required to distinguish the

case that the input is in the set from the case that the input is far from the set

where far means b eing at relative Hamming distance at least or any other

small constant

For example consider a variant of the system of x in which the quadratic

system is xed and the verier needs to determine whether the assignment ap

p earing in the input oracle satises the said system or is far from any assignment

that satises it We use a pro of oracle is as in x and a PCP verier of

proximity that pro ceeds as in x and in addition p erform a proximity test to

verify that the input oracle is close to the assignment enco ded in the pro of oracle

Sp ecically the verier reads a uniformly selected bit of the input oracle and com

pares this value to the selfcorrected value obtained from the pro of oracle ie for

th

a uniformly selected i f ng we compare the i bit of the input oracle to the

i ni

selfcorrection of the value T obtained from the pro of oracle

Robust PCPs Comp osing an outer PCP verier with an inner PCP veri

er of proximity makes sense provided that the outer verier rejects in a robust

manner That is the soundness condition of a robust verier requires that with

probability at least the oracle answers are far from any sequence that is ac

ceptable by the residual predicate rather than merely that the answers are rejected

by this predicate Indeed if the outer verier is nonadaptive and robust then

Indeed in our applications the quadratic system will b e known to the inner verier

b ecause it is determined by the outer verier

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

it suces that the inner verier distinguish with the help of an adequate pro of

answers that are valid from answers that are far from b eing valid

For example if robustness is dened as referring to relative constant distance

which is indeed the case then the PCP of x as well as any PCP of con

stant query complexity is trivially robust However we will not care ab out the

robustness of this PCP b ecause we only use this PCP as an inner verier in pro of

comp osition In contrast we will care ab out the robustness of PCPs that are used

as outer veriers eg the PCP presented next

Teaching note Unfortunately the construction of a PCP of logarithmic randomness

and p olylogarithmic query complexity for NP involves many technical details Further

more obtaining a robust version of this PCP is b eyond the scop e of the current text

Thus the following description should b e viewed as merely providing a avor of the

underlying ideas

PCP of logarithmic randomness and p olylogarithmic query complexity

for NP We fo cus on showing that NP PCP f f for f n p oly log n

and the claimed result will follow by a relatively minor mo dication discussed

afterwards The pro of system underlying NP PCP f f is based on an arith

metization of CNF formulae which is dierent from the one used in x for

constructing an interactive pro of system for coNP We start by describing this

arithmetization and later outline the PCP system that is based on it

In the current arithmetization the names of the variables resp clauses of a

CNF formula are represented by binary strings of logarithmic in jj length and

a generic variable resp clause of is represented by a logarithmic number of new

variables which are assigned values in a nite eld F f g Indeed throughout

the rest of the description we refer to the arithmetic op erations of this nite eld

F which will have cardinality p oly jj The structure of the CNF formula

O log n

x x is represented by a Bo olean function C f g f g such

n

th th

that C if and only if for i the i literal in the

clause of has index which is viewed as a variable name augmented by

i i i

log jj log n

its sign Thus for every f g there is a unique f g

such that C holds Next we consider a multilinear extension

of C over F denoted that is is the unique multilinear p olynomial that

O log n O log n

agrees with C on f g F

Turning to the PCP we rst note that the verier can reduce the original SAT

instance to the aforementioned arithmetic instance that is on input a CNF

formula the verier rst constructs C and as in Exercise Part of the

log n

pro of oracle for this verier is viewed as function A F F which is supp osed

to b e a multilinear extension of a truth assignment that satises ie for every

log n th

f g n the value A is supp osed to b e the value of the variable

log jj

in such an assignment Thus we wish to check whether for every f g

it holds that

Y X

A

i

log n

i

fg

CHAPTER PROBABILISTIC PROOF SYSTEMS

th

where A is the value of the literal under the variable assignment A

log n

that is for where f g is a variable name and f g

indicates the literals type ie whether the variable is negated it holds that

th

A A A Thus Eq holds if and only if the

clause is satised by the assignment induced by A b ecause A must hold

for at least one of the three literals that app ear in this clause

As in x we cannot aord to verify all jj instances of Eq Fur

thermore unlike in x we cannot aord to take a random linear combination

of these jj instances either b ecause this requires to o much randomness For

tunately taking a pseudorandom linear combination of these equations is go o d

enough Sp ecically using an adequate eciently constructible smallbias prob

ability space cf x will do Denoting such a space of size p oly jj jF j

jj

and bias at most by S F we may select uniformly s s S and

jj

check whether

Y X

s A

i

i

fg

def

where log jj log n The smallbias prop erty guarantees that if A fails to

satisfy any of the equations of type Eq then with probability at least

taken over the choice of s s S it is the case that A fails to satisfy

jj

jj

Eq Since jS j p oly jj jF j rather that jS j we can select a sample

in S using O log jj coin tosses Thus we have reduced the original problem to

checking whether for a random s s S Eq holds

jj

Assuming for a moment that A is a lowdegree p olynomial we can probabilis

tically verify Eq by applying a summation test as in the interactive pro of

for coNP that is we refer to stripping the binary summations in iterations

where in each iteration the verier obtains a corresp onding univariate p olynomial

and instantiates it at a random p oint Indeed the verier obtains the relevant uni

variate p olynomials by making adequate queries which sp ecify the entire sequence

of choices made so far in the summation test Note that after stripping the

summations the verier endups with an expression that contains three unknown

values of A which it may obtain by making corresp onding queries to A The sum

mation test involves tossing log jFj coins and making O log jFj Bo olean

queries which corresp ond to queries that are each answered by a univariate p oly

nomial of constant degree over F and three queries to A each answered by an

element of F Soundness of the summation test follows by setting jF j O

where O log jj

Recall however that we may not assume that A is a multivariate p olynomial of

low degree Instead we must check that A is indeed a multivariate p olynomial of

log n

Note that for this there exists a unique triple f g such that

th

This triple enco des the literals app earing in the clause

and this clause is satised by A if and only if i st A

i

The query will also contain a sequence s s S selected at random by the verier

jj

and xed for the rest of the pro cess

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

low degree or rather that it is close to such a p olynomial and use selfcorrection

for retrieving the values of A which are needed for the foregoing summation test

Fortunately a lowdegree test of complexities similar to those of the summation

test do es exist and selfcorrection is also p ossible within these complexities Thus

using a nite eld F of p olylog n elements the foregoing yields NP PCP f f

def

for f n O log n log logn

To obtain the desired PCP system of logarithmic randomness complexity we

O log n

represent the names of the original variables and clauses by long sequences

log log n

over f log ng rather than by logarithmicallylong binary sequences This re

quires using low degree p olynomial extensions ie p olynomial of degree log n

rather than multilinear extensions We can still use a nite eld of p oly log n

O log n

O log log n random bits for the summation elements and so we need only

log log n

and lowdegree tests However the number of queries needed for obtaining the

answers in these tests grows b ecause now the p olynomials that are involved have

individual degree log n rather than constant individual degree This merely

log n

since the individ means that the querycomplexity increases by a factor of

log log n

ual degree increases by a factor of log n but the number of variables decreases by

def

a factor of log log n Thus we obtain NP PCP log q for q n O log n

Warning Robustness and PCP of proximity Recall that in order to use

the latter PCP system in comp osition we need to guarantee that it or a version

of it is robust as well as to present a version that is a PCP of proximity The

latter version is relatively easy to obtain using ideas as applied to the PCP of

x whereas obtaining robustness is to o complex to b e describ ed here We

comment that one way of obtaining a robust PCP system is by a generic application

of a randomnessecient parallelization of PCP systems cf which in

turn dep ends heavily on highly ecient lowdegree tests An alternative approach

cf capitalizes of the sp ecic structure of the summation test as well as on

the evident robustness of a simple lowdegree test

Reection The PCP Theorem asserts a PCP system that obtains simultane

ously the minimal p ossible randomness and query complexity up to a multiplica

tive factor assuming that P NP The foregoing construction obtains this

remarkable result by combining two dierent PCPs the rst PCP obtains loga

rithmic randomness but uses p olylogarithmically many queries whereas the second

PCP uses a constant number of queries but has p olynomial randomness complex

ity We stress that each of these two PCP systems is highly nontrivial and very

interesting by itself We also highlight the fact that these PCPs are combined us

ing a very simple comp osition metho d which refers to auxiliary prop erties such as

robustness and proximity testing

Advanced comment We comment that the comp osition of PCP systems that lack these

extra prop erties is p ossible but is far more cumbersome and complex In some sense this alterna

tive comp osition involves transforming the given PCP systems to ones having prop erties related

to robustness and proximity testing

CHAPTER PROBABILISTIC PROOF SYSTEMS

Overview of the second pro of of the PCP Theorem

The original pro of of the PCP Theorem fo cuses on the construction of two PCP

systems that are highly nontrivial and interesting by themselves and combines

them in a natural manner Lo osely sp eaking this combination via pro of comp o

sition preserves the go o d features of each of the two systems that is it yields

a PCP system that inherits the logarithmic randomness complexity of one sys

tem and the constant query complexity of the other In contrast the following

alternative pro of is fo cused at the amplication of PCP systems via a gradual

pro cess of logarithmically many steps We start with a trivial PCP system that

has the desired complexities but rejects false assertions with probability inversely

prop ortional to their length and in each step we double the rejection probability

while essentially maintaining the initial complexities That is in each step the

constant query complexity of the verier is preserved and its randomness complex

ity is increased only by a constant term Thus the pro cess gradually transforms

an extremely weak PCP system into a remarkably strong PCP system ie a PCP

as p ostulated in the PCP Theorem

In order to describ e the aforementioned pro cess we need to redene PCP sys

tems so to allow arbitrary soundness error In fact for technical reasons it is more

convenient to describ e the pro cess as an iterated reduction of a constraint satisfac

tion problem to itself Sp ecically we refer to systems of variable constraints

which are readily represented by lab eled graphs such that the vertices corresp ond

to nonBo olean variables and the edges are asso ciated with constraints

Denition CSP with variable constraints For a xed nite set an

instance of CSP consists of a graph G V E which may have parallel edges

and selflo ops and a sequence of variable constraints associated

e eE

with the edges where each constraint has the form f g The value

e

of an assignment V is the number of constraints satised by that is

the value of is jfu v E u v gj We denote by vltG

uv

standing for violation the fraction of unsatised constraints under the best possible

assignment that is

jfu v E u v gj

uv

vltG min

V

jE j

For various functions N we wil l consider the promise problem gapCSP

having instances as in the foregoing such that the yesinstances are ful ly satis

able instances ie vlt and the noinstances are pairs G for which

vltG jGj holds where jGj denotes the number of edges in G

fg

Note that SAT is reducible to gapCSP for m m see Exercise

fg

Our goal is to reduce SAT or rather gapCSP to gapCSP for some xed

c

nite and constant c The PCP Theorem will follow by showing a simple PCP

system for gapCSP see Exercise The relationship b etween constraint satis

c

faction problems and the PCP Theorem is further discussed in Section The

desired reduction of gapCSP to gapCSP is obtained by iteratively applying

m

the following reduction logarithmically many times

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

Lemma amplifying reduction of gapCSP to itself For some nite and

constant c there exists a polynomialtime computable function f such that for

every instance G of gapCSP it holds that G f G is an instance

of gapCSP and the two instances are related as fol lows

If vltG then vltG

vltG min vltG c

jG j O jGj

That is satisable instances are mapp ed to satisable instances whereas instances

that violate a fraction of the constraints are mapp ed to that violate at least

a min c fraction of the constraints Furthermore the mapping increases the

number of edges in the instance by at most a constant factor We stress that

b oth and consists of Bo olean constraints dened over

Pro of Outline Before turning to the pro of let us highlight the diculty that

it needs to address Sp ecically the lemma asserts a violation amplifying eect

ie Items and while maintaining the alphab et and allowing only a mo derate

increase in the size of the graph ie Item Waiving the latter requirements

allows a relatively simple pro of that mimics an augmented version of the parallel

rep etition of the corresp onding PCP Thus the challenge is signicantly decreasing

the size blowup that arises from parallel rep etition and maintaining a xed

alphab et The rst goal ie Item calls for a suitable derandomization and

indeed we shall use the Expander Random Walk Generator of Section

Those who read x may guess that the second goal ie xed alphab et

can b e handled using the pro of comp osition paradigm The rest of the overview

is intended to b e understo o d also by those who did not read Section and

x

The lemma is proved by presenting a threestep reduction The rst step is a

prepro cessing step that makes the underlying graph suitable for further analysis

eg the resulting graph will b e an expander The value of vlt may decrease

during this step by a constant factor The heart of the reduction is the second

step in which we increase vlt by any desired constant factor This is done by a

construction that corresp onds to taking a random walk of constant length on the

current graph The latter step also increases the alphab et and thus a p ost

pro cessing step is employed to regain the original alphab et by using any inner

PCP systems eg the one presented in x Details follow

We rst stress that the aforementioned and c as well as the auxiliary pa

rameters d and t to b e introduced in the following two paragraphs are xed

constants that will b e determined such that various conditions which arise in the

course of our argument are satised Sp ecically t will b e the last parameter to

For details see

Advanced comment The augmentation is used to avoid using the Parallel Rep etition

Theorem of In the augmented version with constant probability say half a consistency

check takes place b etween tuples that contain copies of the same variable or query

CHAPTER PROBABILISTIC PROOF SYSTEMS

b e determined and it will b e made greater than a constant that is determined by

all the other parameters

We start with the prepro cessing step Our aim in this step is to reduce the input

G of gapCSP to an instance G such that G is a dregular expander

graph Furthermore each vertex in G will have at least d selflo ops the

number of edges will b e preserved up to a constant factor ie jG j O jGj and

vltG vltG This step is quite simple essentially the original

vertices are replaced by expanders of size prop ortional to their degree and a big

dummy expander is sup erimp osed on the resulting graph see Exercise

The main step is aimed at increasing the fraction of violated constraints by a

suciently large constant factor The intuition underlying this step is that the

probability that a random tedge long walk on the expander G intersects a xed

set of edges is closely related to the probability that a random sample of t edges

intersects this set Thus we may exp ect such walks to hit a violated edge with

probability that is mint c where is the fraction of violated edges Indeed

the current step consists of reducing the instance G of gapCSP to an instance

t

d

G of gapCSP such that and the following holds

The vertex set of G is identical to the vertex set of G and each tedge

long path in G is replaced by a corresp onding edge in G which is thus a

t

d regular graph

The constraints in are the natural ones viewing each element of as a

lab eling of the distance t neighborho o d of a vertex see Figure

and checking that two such lab elings are consistent as well as satisfy That

is the following two types of constraints are introduced

consistency If there is a path of length at most t in G going from vertex

u to vertex w and passing through vertex v then the constraint

asso ciated with the G edge b etween vertices u and w mandates the

equality of the entries corresp onding to vertex v in the lab eling of

vertices u and w

satisfying If the G edge v v is on a path of length at most t starting

at u then the constraint asso ciated with the G edge that corre

sp onds to this path enforces the constraint that is asso ciated with

v v

t

Clearly jG j d jG j O jG j b ecause d is a constant and t will b e set

to a constant Indeed the relatively mo derate increase in the size of the graph

corresp onds to the low randomnesscomplexity of selecting a random walk of length

t in G

A dregular graph is a graph in which each vertex is incident to exactly d edges Lo osely

sp eaking an expander graph has the prop erty that each mo derately balanced cut ie partition

of its vertex set has relatively many edges crossing it An equivalent denition also used in the

actual analysis is that the second eigenvalue of the corresp onding adjacency matrix has absolute

value that is b ounded away from d For further details see xE

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

8 9 10 3 20 11 16 4 17 21 12 1 18 5 7 22

13 2 19 14 uv w

6 23 15

u w 89 10 21 v v 18 19

3 4 5 u 7 20 19

6 7 w 23

The alphab et as a lab eling of the distance t neighborho o ds

when rep etitions are omitted In this case d but the selflo ops

are not shown and so the eective degree is three The twosided

arrow indicates one of the edges in G that will contribute to the edge

constraint b etween u and w in G

Figure The amplifying reduction in the second pro of of the PCP Theorem

Turning to the analysis of this step we note that vltG implies

vltG The interesting fact is that the fraction of violated constraints

p p

t that is vltG t vltG c increases by a factor of min

Here we merely provide a rough intuition and refer the interested reader to We

may fo cus on any lab eling to the vertices of G that is consistent with some

lab eling of G b ecause relatively few inconsistencies among the values assigned

to a vertex by the lab eling of other vertices can b e ignored while relatively

many such inconsistencies yield violation of the equality constraints of many

edges in G Intuitively relying on the hypothesis that G is an expander it follows

that the set of violated edgeconstraints of with resp ect to the aforementioned

lab eling causes many more edgeconstraints of to b e violated b ecause each

edgeconstraint of is enforced by many edgeconstraints of The p oint is

that any set F of edges of G is likely to appear on a min t jF jjG j

fraction of the edges of G ie tpaths of G Note that the claim would have

CHAPTER PROBABILISTIC PROOF SYSTEMS

b een obvious if G were a complete graph but it also holds for an expander

p

t gained in the second step makes up for the constant factor The factor of

lost in the rst step as well as the constant factor to b e lost in the last step

Furthermore for a suitable choice of the constant t the aforementioned gain yields

an overall constant factor amplication of vlt However so far we obtained an

t

d

instance of gapCSP rather than an instance of gapCSP where The pur

p ose of the last step is to reduce the latter instance to an instance of gapCSP This

is done by viewing the instance of gapCSP as a weak PCP system analogously

to Exercise and comp osing it with an innerverier using the pro of comp osi

tion paradigm outlined in x We stress that the innerverier used here needs

t

only handle instances of constant size ie having description length O d log jj

and so the verier presented in x will do The resulting PCPsystem uses

def

t

randomness r log jG j O d log jj and a constant number of binary queries

and has rejection probability vltG which is indep endent of the choice of

O

the constant t As in Exercise for f g we can easily obtain an in

stance of gapCSP that has a vltG fraction of violated constraints Fur

thermore the size of the resulting instance which is used as the output G of

r

the threestep reduction is O O jG j where the equality uses the fact that

p

d and t are constants Recalling that vltG min t vltG c

and vltG vltG this completes the outline of the pro of of the

entire lemma

Reection In contrast to the pro of presented in x which combines two

remarkable constructs by using a simple comp osition metho d the current pro of

of the PCP Theorem is based on developing a p owerful combining metho d that

improves the quality of the main system to which it is applied This new metho d

captured by the Amplication Lemma Lemma do es not merely obtain the

b est of the combined systems but rather obtains a b etter system than the one given

However the qualityamplication oered by Lemma is rather mo derate and

thus many applications are required in order to derive the desired result Taking

the opp osite p ersp ective one may say that remarkable results are obtained by a

gradual pro cess of many mo derate amplication steps

PCP and Approximation

The characterization of NP in terms of probabilistically checkable pro ofs plays

a central role in the study of the complexity of natural approximation problems

cf Section To demonstrate this relationship we rst note that any PCP

system V gives rise to an approximation problem that consists of estimating the

maximum acceptance probability for a given input that is on input x the task

is approximating the probability that V accepts x when given oracle access to

the b est p ossible ie we wish to approximate max fPr V x g Thus

if S PCP r q then deciding membership in S is reducible to approximating

We mention that due to a technical diculty it is easier to establish the claimed b ound of

p

t vlt G rather than t vlt G

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

r q

the maximum among exp quantities corresp onding to all eective oracles

r

where each quantity can b e evaluated in time p oly For the validity of this

reduction an approximation up to a constant factor of wil l do

Note that the foregoing approximation problem is parameterized by a PCP ver

ier V and its instances are given their value with resp ect to this verier ie the

instance x has value max fPr V x g This p er se do es not yield a natural

approximation problem In order to link PCP systems with natural approxima

tion problems we take a closer lo ok at the approximation problem asso ciated with

PCP r q

For simplicity we fo cus on the case of nonadaptive PCP systems ie all the

queries are determined b eforehand based on the input and the internal coin tosses

r jxj

of the verier Fixing an input x for such a system we consider the Bo olean

formulae that represent the decision of the verier on each of the p ossible outcomes

of its coin tosses after insp ecting the corresp onding bits in the pro of oracle That is

r jxj

each of these formulae dep ends on q jxj Bo olean variables that represent the

values of the corresp onding bits in the pro of oracle Thus if x is a yesinstance then

r jxj

there exists a truth assignment to these variables that satises all formulae

whereas if x is a noinstance then there exists no truth assignment that satises

r jxj

more than formulae Furthermore in the case that r n O log n given

x we can construct the corresp onding sequence of formulae in p olynomialtime

Hence the PCP Theorem ie Theorem yields NPhardness results regarding

the approximation of the number of simultaneously satisable Boolean formulae of

constant size This motivates the following denition

Denition gap problems for SAT and generalizedSAT For constants q

q

N and the promise problem gapGSAT refers to instances that are each a

sequence of q variable Boolean formulae ie each formula dep ends on at most

q variables The yesinstances are sequences that are simultaneously satisable

whereas the noinstances are sequences for which no Boolean assignment satises

more than a fraction of the formulae in the sequence The promise problem

q

gapSAT is dened analogously except that in this case each instance is a sequence

of disjunctive clause ie each formula in each sequence consists of a single dis

junctive clause

q

Indeed each instance of gapSAT is naturally viewed as q CNF formulae and we

consider an assignment that satises as many clauses of the input CNF as p ossible

O

is NPcomplete which As hinted NP PCP log O implies that gapGSAT

in turn implies that for some constant the problem gapSAT is NPcomplete

The converses hold to o All these claims are stated and proved next

Theorem equivalent formulations of the PCP Theorem The fol lowing

three conditions are equivalent

The PCP Theorem there exists a constant q such that NP PCP log q

q

There exists a constant q such that gapGSAT is NP hard

CHAPTER PROBABILISTIC PROOF SYSTEMS

There exists a constant such that gapSAT is NP hard

The p oint of Theorem is not its mere validity which follows from the valid

ity of each of the three items but rather the fact that its pro of is quite simple

Note that Items and make no reference to PCP Thus their easy to estab

lish equivalence to Item manifests that the hardness of approximating natural

optimization problems lies at the heart of the PCP Theorem In general proba

bilistically checkable pro of systems for NP yield strong inapproximability results

for various classical optimization problems cf Exercise and Section

Pro of We rst show that the PCP Theorem implies the NPhardness of gapGSAT

We may assume without loss of generality that for some constant q and every

S NP it holds that S PCP O log q via a nonadaptive verier b ecause

q

q adaptive queries can b e emulated by nonadaptive queries We reduce S to

O log jxj

gapGSAT as follows On input x we scan all p ossible sequence of outcomes

of the veriers coin tosses and for each such sequence of outcomes we determine

the queries made by the verier as well as the residual decision predicate where this

predicate determines which sequences of answers lead this verier to accept That

O log jxj

is for each randomoutcome f g we consider the residual predicate

determined by x and that sp ecies which q bit long sequence of oracle answers

makes the verier accept x on coins Indeed this predicate dep ends only on q

variables which represent the values of the q corresp onding oracle answers Thus

we map x to a sequence of p oly jxj formulae each dep ending on q variables

q

obtaining an instance of gapGSAT This mapping can b e computed in p olynomial

time and indeed x S resp x S is mapp ed to a yesinstance resp no

q

instance of gapGSAT

Item implies Item by a standard reduction of GSAT to SAT Sp ecically

q q

which in turn reduces to gapSAT reduces to gapSAT gapGSAT for

q

q

q Note that Item implies Item eg given an instance of gapSAT

consider all p ossible conjunctions of disjunctive clauses in the given instance

We complete the pro of by showing that Item implies Item The same

argument shows that Item implies Item This is done by showing that gapSAT

is in PCP log and using the reduction of NP to gapSAT to derive a

q

corresp onding PCP for each set in NP In fact we show that gapGSAT is in

PCP log q and do so by presenting a very natural PCP system In this

PCP system the pro of oracle is supp osed to b e an satisfying assignment and the

verier selects at random one of the q variable formulae in the input sequence

and checks whether it is satised by the assignment given by the oracle This

amounts to tossing logarithmically many coins and making q queries This verier

always accepts yesinstances when given access to an adequate oracle whereas

each noinstances is rejected with probability at least no matter which oracle is

used To amplify the rejection probability to the desired threshold of we

invoke the foregoing verier times and note that

Gap amplifying reductions a reection Item resp Item of Theo

rem implies that GSAT resp SAT can b e reduce to gapGSAT resp to

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

gapSAT This means that there exist gap amplifying reductions of problems

like SAT to themselves where these reductions map yesinstances to yesinstances

as usual while mapping noinstances to noinstances that are far from b eing

yesinstances That is noinstances are mapp ed to noinstances of a sp ecial type

such that a gap is created b etween the yesinstances and noinstances at the

image of the reduction For example in the case of SAT unsatisable formu

lae are mapp ed to formulae that are not merely unsatisable but rather have no

assignment that satises more than a fraction of the clauses Thus PCP

constructions are essentially gap amplifying reductions

More on PCP itself an overview

We start by discussing variants of the PCP characterization of NP and next turn

to PCPs having expressing p ower b eyond NP Needless to say the latter systems

have sup erlogarithmic randomness complexity

More on the PCP characterization of NP

Interestingly the two complexity measures in the PCPcharacterization of NP

can b e traded o such that at the extremes we get NP PCP log O and

NP PCP poly resp ectively

Prop osition For every S NP there exists a logarithmic function ie

log such that for every integer function k that satises k n n it

k

holds that S PCP k O Recall that PCP log poly NP

Pro of Sketch By Theorem we have S PCP O To show that

k

S PCP k O we consider an emulation of the corresp onding verier in

which we try all p ossibilities for the k nbit long prex of its randomtap e

Following the establishment of Theorem numerous variants of the PCP

Characterization of NP were explored These variants refer to a ner analysis of

various parameters of probabilistically checkable pro of systems for sets in NP

Following is a brief summary of some of these studies

The length of PCPs Recall that the eective length of the oracle in any

PCP log log system is p olynomial in the length of the input Furthermore

in the PCP systems underlying the pro of of Theorem the queries refer only to

a p olynomially long prex of the oracle and so the actual length of these PCPs for

NP is p olynomial Remarkably the length of PCPs for NP can be made nearly

linear in the combined length of the input and the standard NPwitness while

maintaining constant query complexity where by nearlylinear we mean linear up

to a polylogarithmic factor For details see This means that a rel

atively modest amount of redundancy in the pro of oracle suces for supp orting

probabilistic verication via a constant number of prob es

With the exception of works that app eared after we provide no references for the results

quoted here We refer the interested reader to Sec

CHAPTER PROBABILISTIC PROOF SYSTEMS

The number of queries in PCPs Theorem asserts that a constant num

b er of queries suce for PCPs with logarithmic randomness and soundness error

of for NP It is currently known that this constant is at most ve whereas

with three queries one may get arbitrary close to a soundness error of The

obvious tradeo b etween the number of queries and the soundness error gives rise

to the robust notion of amortized querycomplexity dened as the ratio b etween the

number of queries and minus the logarithm to based of the soundness error

For every any set in NP has a PCP system with logarithmic randomness

and amortized querycomplexity cf whereas only sets in P have PCPs

of logarithmic randomness and amortized querycomplexity less than

Freebit complexity The motivation to the notion of free bits came from the

PCPtoMaxClique connection see Exercise and Sec but we b elieve

that this notion is of indep endent interest Intuitively this notion distinguishes

b etween queries for which the acceptable answer is determined by previously ob

tained answers ie the verier compares the answer to a value determined by the

previous answers and queries for which the verier only records the answer for

future usage The latter queries are called free b ecause any answer to them is ac

ceptable For example in the linearity test see x the rst two queries are

free and the third is not ie the test accepts if and only if f x f y f x y

The amortized freebit complexity is dene analogously to the amortized query com

plexity Interestingly NP has PCPs with logarithmic randomness and amortized

freebit complexity less than any positive constant

Adaptive versus nonadaptive veriers Recall that a PCP verier is called

nonadaptive if its queries are determined solely based on its input and the outcome

of its coin tosses A general verier called adaptive may determine its queries also

based on previously received oracle answers Recall that the PCP Characterization

of NP ie Theorem is established using a nonadaptive verier however it

turns out that adaptive veriers are more powerful than nonadaptive ones in terms

of quantitative results Sp ecically for PCP veriers making three queries and

having logarithmic randomness complexity adaptive queries provide for soundness

error at most actually for any for any set in NP whereas

nonadaptive queries provide soundness error or less only for sets in P

Nonbinary queries Our denition of PCP allows only binary queries Cer

tainly nonbinary queries can b e emulated by binary queries but the converse do es

not necessarily hold For this reason parallel rep etition is highly nontrivial

Advanced comment The source of trouble is the adversarial settings implicit in the

soundness condition which means that when several binary queries are packed into one non

binary query the adversary need not resp ect the packing ie it may answer inconsistently on

the same binary query dep ending on the other queries packed with it This trouble b ecomes

acute in the case of PCPs b ecause they do not corresp ond to a full information game Indeed

in contrast parallel rep etition is easy to analyze in the case of interactive pro of systems b ecause

they can b e mo deled as full information games this is obvious in the case of publiccoin systems

but also holds for general interactive pro of systems see Exercise

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

in the PCP setting Still a Parallel Rep etition Theorem that refers to indep en

dent invocations of the same PCP is known but it is not applicable for obtaining

soundness error smaller than a constant while preserving logarithmic randomness

Nevertheless using adequate consistency tests one may construct PCP systems

for NP using logarithmic randomness a constant number of nonbinary queries

and soundness error exponential in the length of the answers Currently this is

known only for sublogarithmic answer lengths

Stronger forms of PCP systems for NP

Although the PCP Theorem is famous mainly for its negative applications to the

study of natural approximation problems see Section and x its p o

tential for direct p ositive applications is fascinating Indeed the vision of sp eeding

up the verication of mundane pro ofs is exciting where these pro ofs may refer to

mundane assertions such as the correctness of a sp ecic computation Enabling

such a sp eedup requires a strengthening of the PCP Theorem such that it man

dates ecient verication time rather than merely low querycomplexity of the

verication task Such a strengthening is p ossible

Theorem Theorem strengthened Every set S in NP has a PCP

system V of logarithmic randomnesscomplexity constant querycomplexity and

quadratic timecomplexity Furthermore NPwitnesses for membership in S can be

transformed in polynomialtime to corresponding prooforacles for V

The furthermore part was already stated in Section as a strengthening of

Theorem Thus the novelty in Theorem is that it provides quadratic

verication time rather than p olynomial verication time where the p olynomial

may dep end arbitrarily on the set S Theorem is proved by noting that that

the CNF formulae that is obtained by reducing S to SAT are highly uniform and

thus the verier V that is outlined in x can b e implemented in quadratic

time Indeed the most timeconsuming op eration required of V is evaluating the

lowdegree extension of C which corresp onds to the input formula at a few

p oints In the context of x evaluating in exp onentialtime suces since

this means time that is p olynomial in jj Theorem follows by showing that

a variant of can b e evaluated in p olynomialtime since this means time that is

p olylogarithmic in jj for details see Exercise

PCPs of Proximity Clearly we cannot exp ect a PCP system or any standard

pro of system for that matter to have sublinear verication time since linear

time is required for merely reading the input Nevertheless we may consider a

relaxation of the verication task regarding pro ofs of membership in a set S In

this relaxation the verier is only required to reject any input that is far from

S regardless of the alleged pro of and as usual accept any input that is in S

when accompanied with an adequate pro of Sp ecically in order to allow sub

linear time verication we provide the verier V with direct access to the bits

of the input which is viewed as an oracle as well as with direct access to the

CHAPTER PROBABILISTIC PROOF SYSTEMS

usual PCP pro oforacle and require that the following two conditions hold with

resp ect to some constant

Completeness For every x S there exists a string such that when given access

x

to the oracles x and machine V always accepts

x

Soundness with resp ect to proximity For every string x that is far from S ie

jxj

for every x f g S it holds that x and x dier on at least jxj bits

and every string when given access to the oracles x and machine V

rejects with probability at least

Machine V is called a PCP of proximity and its queries to b oth oracles are counted

in its querycomplexity Indeed such a PCP of proximity was used in x

and the notion is analogous to a relaxation of decision problems that is reviewed

in Section

We mention that every set in NP has a PCPs of proximity of logarithmic

randomnesscomplexity constant querycomplexity and polylogarithmic timecomplexity

This follows by using ideas as underlying the pro of of Theorem see also Ex

ercise

PCP with sup erlogarithmic randomness

Our fo cus so far was on the imp ortant case where the verier tosses logarithmically

many coins and hence the eective pro of length is p olynomial Here we mention

that the PCP Theorem or rather Theorem scales up

Theorem Theorem Generalized Let t be an integer function such

p olyn

that n tn Then Ntimet PCP O log t O

r n

Recall that PCP r q Ntimet for tn p oly n Thus the Ntime

Hierarchy implies a hierarchy of PCP O classes for randomness complexity

ranging b etween logarithmic and p olynomial functions

Chapter Notes

The following historical notes are quite long and still they fail to prop erly discuss

several imp ortant technical contributions that played an imp ortant role in the de

velopment of the area For further details the reader is referred to Sec

Motivated by the desire to formulate the most general type of pro ofs that

may b e used within cryptographic proto cols Goldwasser Micali and Racko

introduced the notion of an interactive proof system Although the main thrust of

their work was the introduction of a sp ecial type of interactive pro ofs ie ones

that are zeroknowledge the p ossibility that interactive pro of systems may b e more

p owerful from NPpro of systems was p ointed out in Indep endently of

Note that the sketched pro of of Theorem yields verication time that is quadratic in the

length of the input and p olylogarithmic in the length of the NPwitness

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

Babai suggested a dierent formulation of interactive pro ofs which he called

ArthurMerlin Games Syntactically ArthurMerlin Games are a restricted form

of interactive pro of systems yet it was subsequently shown that these restricted

systems are as p owerful as the general ones cf The sp eedup result ie

AMf AMf is due to improving over

The rst evidence to the p ower of interactive pro ofs was given by Goldreich Mi

cali and Wigderson who presented an interactive pro of system for Graph Non

Isomorphism Construction More imp ortantly they demonstrated the gen

erality and wide applicability of zeroknowledge proofs Assuming the existence of

oneway function they showed how to construct zeroknowledge interactive pro ofs

for any set in NP Theorem This result has had a dramatic impact on

the design of cryptographic proto cols cf For further discussion of zero

knowledge and its applications to cryptography see App endix C Theorem

ie ZK IP is due to

Probabilistically checkable pro of PCP systems are related to multiprover in

teractive proof systems a generalization of interactive pro ofs that was suggested

by BenOr Goldwasser Kilian and Wigderson Again the main motivation

came from the zeroknowledge p ersp ective sp ecically presenting multiprover

zeroknowledge pro ofs for NP without relying on intractability assumptions Yet

the complexity theoretic prosp ects of the new class denoted MI P have not b een

ignored

The amazing p ower of interactive pro of systems was demonstrated by using

algebraic metho ds The basic technique was introduced by Lund Fortnow Karlo

and Nisan who applied it to show that the p olynomialtime hierarchy and

P

actually P is in IP Subsequently Shamir used the technique to show

that IP P S P AC E and Babai Fortnow and Lund used it to show that

MI P NEXP Our entire pro of of Theorem follows

The aforementioned multiprover pro of system of Babai Fortnow and Lund

hereafter referred to as the BFL pro of system has b een the starting p oint for fun

damental developments regarding NP The rst development was the discovery

that the BFL pro of system can b e scaleddown from NEXP to NP This im

p ortant discovery was made indep endently by two sets of authors Babai Fortnow

Levin and Szegedy and Feige Goldwasser Lovasz and Safra However

the manner in which the BFL pro of is scaleddown is dierent in the two pap ers

and so are the consequences of the scalingdown

Babai et al start by considering only inputs enco ded using a sp ecial error

correcting co de The enco ding of strings relative to this errorcorrecting co de can

b e computed in p olynomial time They presented an almostlinear time algorithm

that transforms NPwitnesses to inputs in a set S NP into transparent proofs

that can b e veried as vouching for the correctness of the enco ded assertion

in probabilistic polylogarithmic time by a Random Access Machine Babai

et al stress the practical asp ects of transparent pro ofs sp ecically for rapidly

checking transcripts of long computations

In contrast in the pro of system of Feige et al the verier stays

p olynomialtime and only two more rened complexity measures ie the ran

CHAPTER PROBABILISTIC PROOF SYSTEMS

domness and query complexities are reduced to p olylogarithmic This eliminates

the need to assume that the input is in a sp ecial errorcorrecting form and yields

a rened quantitative version of the notion of probabilistically checkable pro of

systems introduced in where the renement is obtained by sp ecifying the

randomness and query complexities see Denition Hence whereas the BFL

pro of system can b e reinterpreted as establishing NEXP PCP poly poly

the work of Feige et al establishes NP PCP f f where f n O log n

log log n In retrosp ect we note that the work of Babai et al implies that

NP PCP log polylog

Interest in the new complexity class b ecame immense since Feige et al

demonstrated its relevance to proving the intractability of approximating some nat

ural combinatorial problems sp ecically for MaxClique When using the PCP

toMaxClique connection established by Feige et al the randomness and query

complexities of the verier in a PCP system for an NPcomplete set relate to

the strength of the negative results obtained for the approximation problems This

fact provided a very strong motivation for trying to reduce these complexities and

obtain a tight characterization of NP in terms of PCP The obvious challenge

was showing that NP equals PCP log log This challenge was met by Arora and

Safra Actually they showed that NP PCP log q where q n olog n

Hence a new challenge arose namely further reducing the query complexity

in particular to a constant while maintaining the logarithmic randomness com

plexity Again additional motivation for this challenge came from the relevance of

such a result to the study of natural approximation problems The new challenge

was met by Arora Lund Motwani Sudan and Szegedy and is captured by

the PCP Characterization Theorem which asserts that NP PCP log O

Indeed the PCP Characterization Theorem is a culmination of a sequence of

impressive works These works are rich in innovative ideas

eg various arithmetizations of SAT as well as various forms of pro of comp osi

tion and employ numerous techniques eg lowdegree tests selfcorrection and

pseudorandomness Our overview of the original pro of of the PCP Theorem in

x is based on The alternative pro of outlined in x

is due to Dinur

We mention some of the ideas and techniques involved in deriving even stronger

variants of the PCP Theorem which are surveyed in x These include

the Parallel Rep etition Theorem the use of the LongCo de and the

application of Fourier analysis in this setting We also highlight the

notions of PCPs of proximity and robustness see

ComputationallySound Pro of Systems Argument systems were dened by

Brassard Chaum and Crepeau with the motivation of providing perfect zero

knowledge arguments rather than zeroknowledge proofs for NP A few years

later Kilian demonstrated their signicance b eyond the domain of zero

knowledge by showing that under some reasonable intractability assumptions ev

Our presentation also b enets from the notions of PCPs of proximity and robustness put

forward in

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

ery set in NP has a computationallysound pro of in which the randomness and

communication complexities are p olylogarithmic Interestingly these argument

systems rely on the fact that NP PCP f f for f n p oly log n We men

tion that Micali suggested a dierent type of computationallysound pro of

systems which he called CSpro ofs

Final comment The current chapter is a revision of Chap In particular

more details are provided here for the main topics whereas numerous secondary

topics discussed in Chap are not mentioned here or are only briey men

tioned here We note that a few of the research directions that were mentioned

in Sec have received considerable attention in the p erio d that elapsed

and improved results are currently known In particular the interested reader is

referred to for a study of the length of PCPs and to for a study

of their amortized query complexity Likewise a few op en problems mentioned

in Sec have b een resolved sp ecically the interested reader is referred

to for breakthrough results regarding zeroknowledge

Exercises

Exercise parallel errorreduction for interactive pro of systems Prove

that the error probability in the soundness condition can b e reduced by parallel

rep etitions of the pro of system A pro of app ears in Ap dx C

Guideline As a warmup consider the sp ecial case of publiccoin interactive pro of sys

tems Next generalize the analysis to arbitrary interactive pro of systems by considering

as a mental exp eriment a p owerful verier that emulates the original verier while

b ehaving as in the publiccoin mo del

Exercise Prove that if S is Karpreducible to a set in IP then S IP

Prove that if S is Co okreducible to a set S such that b oth S and f g n S are

in IP then S IP

Exercise Complete the details of the pro of that coNP IP ie the rst

part of the pro of of Theorem In particular supp ose that the proto col for

unsatisability is applied to a CNF formula with n variables and m clauses Then

what is the length of the messages sent by the two parties What is the soundness

error

Exercise Present an interactive pro of system for unsatisability such that on

input a CNF formula having n variables the parties exchange nO log n messages

Guideline Mo dify the rst part of the pro of of Theorem by stripping O log n

summations in each round

We comment that interactive pro ofs are unlikely to have such low complexities see

CHAPTER PROBABILISTIC PROOF SYSTEMS

Exercise an interactive pro of system for P Using the main part of

the pro of of Theorem present a pro of system for the counting set of Eq

Guideline Use a slightly dierent arithmetization of CNF formulae Sp ecically instead

of replacing the clause x y z by the term x y z replace it by the term

x y z The p oint is that this arithmetization maps Bo olean assignments that

satisfy the CNF formula to assignments that when substituted in the corresp onding

arithmetic expression yield the value rather than yielding a somewhat arbitrary p ositive

integer

Exercise Show that QBF can b e reduced to a sp ecial form of noncanonical

QBF in which no variable app ears b oth to the left and to the right of more than

one universal quantier

Guideline Consider a pro cess which pro ceeds from left to right of refreshing vari

ables after each universal quantier Let x x y x x b e a quantierfree

s s st

b o olean formula and let Q Q b e an arbitrary sequence of quantiers Then we

s st

replace the quantied subformula

y Q x Q x x x y x x

s s st st s s st

by the subformula

s

y x x x x Q x Q x x x x y x

s st i s s st st

s i i s

Note that the variables x x do not app ear to the right of the quantier Q in

s s

the replaced formula and that the length of the replaced formula grows by an additive

term of O s This pro cess of refreshing variables is applied from left to right on the

entire sequence of universal quantiers except the inner one for which this refreshing is

useless

Exercise Prove that if two integers in M are dierent then they must b e

dierent mo dulo most of the primes in the interval L where L p oly log M

Prove the same for the interval L L

Guideline Let a b M and supp ose that P P is an enumeration of all the

t

primes that satisfy a b mo d P Using the Chinese Reminder Theorem prove that

i

Q

def

t

Q P M b ecause otherwise a b follows by combining a b mo d Q with

i

i

the hypothesis a b M It follows that t log M Using a lowerbound on the

density of prime numbers the claim follows

See App endix G

For example

z z z z z z z z z z z z

is rst replaced by

z z z z z z z z z z z z z z z

and next written as z z z z z z z z z z z z z z z is replaced by

z z z z z z z z z z

z z z z z z z z z z

i i i

Thus in the resulting formula no variable app ears b oth to the left and to the right of more than

a single universal quantier

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

Exercise on interactive pro ofs with twosided error following

Let IP f denote the class of sets having a twosided error interactive pro of system

in which a total of f jxj messages are exchanged on common input x Sp ecically

supp ose that a suitable prover may cause every yesinstance to b e accepted with

probability at least rather than while no cheating prover can cause a

noinstance to b e accepted with probability greater than rather than

Similarly let AM denote the publiccoin version of IP

Establish IP f AM f by noting that the pro of of Theorem F

which establishes IP f AMf extends to the twosided error setting

Prove that AM f AMf by extending the ideas underlying the

pro of of Theorem which actually establishes that BPP AM where

BPP AM

Using the Round Sp eedup Theorem ie Theorem F conclude that for every

function f N N n fg it holds that IP f AMf IP f

Guideline for Part Fixing an optimal prover strategy for the given twosided

error publiccoin interactive pro of consider the set of verier coins that make the verier

accept any xed yesinstance and apply the ideas underlying the transformation of BPP

to MA AM For further details see

Exercise In continuation to Exercise show that IP f IP f for every

function f N N including f

Guideline Focus on establishing IP IP which is identical to Part of Exer

cise Note that the relevant classes dened in Exercise coincide with IP and

IP that is MA IP and MA IP

Exercise Prove that every P S P AC E complete set S has an interactive pro of

system in which the designated prover can b e implemented by a probabilistic

p olynomialtime oracle machine that is given oracle access to S

Guideline Use Theorem and Prop osition

Exercise checkers following A probabilistic p olynomialtime or

acle machine C is called a checker for the decision problem if the following two

conditions hold

f

For every x it holds that Pr C x where as usual C x denotes

the output of A on input x when given oracle access to f

For every f f g f g and every x such that f x x it holds

f

that Pr C x

Note that nothing is required in the case that f x x but f Prove that

if both S fx x g and S fx x g have interactive proof systems

in which the designated prover can be implemented by a probabilistic polynomial

time oracle machine that is given oracle access to then has a checker Using

Exercise conclude that any P S P AC E complete problem has a checker

CHAPTER PROBABILISTIC PROOF SYSTEMS

def

Guideline On input x and oracle access to f the checker rst obtains f x The

claim x is then checked by combining the verier of S with the probabilistic

p olynomialtime oracle machine that describ es the designated prover while referring its

queries to the oracle f

Exercise weakly optimal deciders for checkable problems following

Prove that if a decision problem has a checker as dened in Exercise then

there exists a probabilistic algorithm A that satises the following two conditions

A solves the decision problem ie for every x it holds that Pr Ax

x

For every probabilistic algorithm A that solves the decision problem

there exists a p olynomial p such that for every x it holds that t x

A

z denotes the number x g where t z resp t pjxj max ft

A A A

jx jpjxj

of steps taken by A resp A on input z

Note that compared to Theorem the claim of optimality is weaker but on the

other hand it applies to decision problems rather than to candid search problems

Guideline Use the ideas of the pro of of Theorem noting that the correctness

of the answers provided by the various candidate algorithms can b e veried by using

the checker That is A invokes copies of the checker while using dierent candidate

algorithms as oracles in the various copies

Exercise on the role of soundness error in zeroknowledge pro ofs

Prove that if S has a zeroknowledge interactive pro of system with p erfect sound

ness ie the soundness error equals zero then S BPP

Guideline Let M b e an arbitrary algorithm that simulates the view of the honest

verier Consider the algorithm that on input x accepts x if and only if M x represents

a valid view of the verier in an accepting interaction ie an interaction that leads the

verier to accept the common input x Use the simulation condition to analyze the case

x S and the p erfect soundness hypothesis to analyze the case x S

Exercise on the role of interaction in zeroknowledge pro ofs Prove

that if S has a zeroknowledge interactive pro of system with a unidirectional com

munication then S BPP

Guideline Let M b e an arbitrary algorithm that simulates the view of the honest

verier and let M x denote the part of this view that consists of the prover message

Consider the algorithm that on input x obtains m M x and emulates the veriers

decision on input x and message m Note that this algorithm ignores the part of M x that

represents the veriers internal coin tosses and uses fresh veriers coins when deciding

on x m

Exercise on the eective length of PCP oracles Supp ose that V is

a PCP verier of querycomplexity q and randomnesscomplexity r Show that

for every xed x the number of p ossible lo cations in the pro of oracle that are

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

examined by V on input x when considering all p ossible internal coin tosses of V

q jxjr jxj

and all p ossible answers it may receive is upp erb ounded by Show

r jxj

that if V is nonadaptive then the upp erb ound can b e improved to q jxj

Guideline In the nonadaptive case all q queries are determined by V s internal coin

tosses

Exercise on the eective randomness of PCPs Supp ose that a set S

has a PCP of querycomplexity q that utilizes pro of oracles of length Show

that for every constant the set S has a nonuniform PCP of query

complexity q soundness error and randomness complexity r such that

r n log n n O By a nonuniform PCP we mean one in which the

verier is a probabilistic p olynomialtime oracle machine that is given direct access

to the bits of a nonuniform p oly n nbit long advice

Guideline Consider a PCP verier V as in the hypothesis and denote its randomness

complexity by r We construct a nonuniform verier V that on input of length n

V

r n

V

obtains as advice a set R f g of cardinality O n n and emulates V

n

on a uniformly selected element of R Show that for a random R of the said size the

n n

verier V satises the claims of the exercise

jxj

Extra hint Fixing any input x S and any oracle f g upp erb ound the probability

that a random set R of the said size is bad where R is bad if V accept x with probability

n n

when selecting its coins in R and using the oracle

n

Exercise on the complexity of sets having certain PCPs Supp ose that

a set S has a PCP of querycomplexity q and randomnesscomplexity r Show that

S can b e decided by a nondeterministic machine that on input of length n makes

r n

at most q n truly nondeterministic steps ie choosing b etween dierent

r n

alternatives and halts within a total number of p oly n steps Conclude

r

r q r

that S Ntime p oly Dtime p oly

r jxj

Guideline For each input x S and each p ossible value f g of the veriers

randomtap e we consider a sequence of q jxj bit values that represent a sequence of

r jxj

oracle answers that make the verier accept Indeed for xed x and f g

each setting of the q jxj oracle answers determine the computation of the corresp onding

verier including the queries it makes

Exercise The FGLSSreduction For any S PCP r q consider

the following mapping of instances for S to instances of the Independent Set

problem The instance x is mapp ed to a graph G V E where V

x x x x

r jxjq jxj

f g consists of pairs such that the PCP verier accepts the input

r jxj

x when using coins f g and receiving the answers to

q jxj

the oracle queries determined by x r and the previous answers Note that V con

x

tains only accepting views of the verier The set E consists of edges that con

x

nect vertices that represents mutually inconsistent views of the said verier that

is the vertex v is connected to the vertex v

q jxj

q jxj

x x x

if there exists i and i such that and q v q v where q v resp

i

i i

i i

See x for denition of nondeterministic machines

CHAPTER PROBABILISTIC PROOF SYSTEMS

x

q v denotes the ith resp i th query of the verier on input x when us

i

ing coins resp and receiving the answers resp

i

i

r jxj

In particular for every f g and if V then

x

f g E

x

Prove that the mapping x G can b e computed in time that is p olynomial

x

r jxjq jxj

in jxj

r jxjf jxj

Note that the number of vertices in G is upp erb ounded by

x

where f q is the freebit complexity of the PCP verier

Prove that for every x the size of the maximum indep endent set in G is at

x

r jxj

most

r jxj

Prove that if x S then G has an indep endent set of size

x

Prove that if x S then the size of the maximum indep endent set in G is

x

r jxj

at most

In general denoting the PCP verier by V prove that the size of the maximum

r jxj

indep endent set in G is exactly max fPr V x g Note the similarity

x

to the pro of of Prop osition

Show that the PCP Theorem implies that the size of the maximum independent set

resp clique in a graph is NPhard to approximate to within any constant factor

Guideline Note that an indep endent set in G corresp onds to a set of coins R and a

x

partial oracle such that V accepts x when using coins in R and accessing any oracle

that is consistent with The FGLSSreduction creates a gap of a factor of b etween

yes and noinstances of S having a standard PCP Larger factors can b e obtained by

considering a PCP that results from rep eating the original PCP for a constant number of

times The result for Clique follows by considering the complement graph

Exercise Using the ideas of Exercise prove that for any tn olog n

it holds that NP PCP t t implies that P NP

Guideline We only use the fact that the FGLSSreduction maps instances of S

PCP t t to instances of the Clique problem and ignore the fact that we actually get a

stronger reduction to a gapClique problem Furthermore when applies to problems

in NP PCP t t the FGLSSreduction runs in p olynomialtime The key observation

is that the FGLSSreduction maps instances of the Clique problem which is in NP

olog n

PCP olog olog to shorter instances of the same problem b ecause n

Thus iteratively applying the FGLSSreduction we can reduce instances of Clique to

instances of constant size This yields a reduction of Clique to a nite set and NP P

follows by the NP completeness of Clique

Exercise a simple but partial analysis of the BLR Linearity Test

For Ab elian groups G and H consider functions from G to H For such a generic

function f consider the linearity or rather homomorphism test that selects uni

formly r s G and checks that f r f s f r s Let f denote the distance

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

of f from the set of homomorphisms of G to H that is f is the minimum

taken over all homomorphisms h G H of Pr f x hx Using the fol

xG

lowing guidelines prove that the probability that the test rejects f denoted f

is at least f f

Supp ose that h is the homomorphism closest to f ie f Pr f x

xG

hx Prove that f Pr f x f y f x y is lowerbounded

xy G

by Pr f x hx f y hy f x y hx y

xy

Hint consider three out of four disjoint cases regarding f x hx f y hy and

f x y hx y that are p ossible when f x f y f x y where these three cases

refer to the disagreement of h and f on exactly one out of the three relevant p oints

Prove that Pr f x hx f y hy f x y hx y f f

xy

Hint lowerbound the said probability by Pr f x hxPr f x hxf y

xy xy

hy Pr f x hx f x y hx y

xy

Note that the lowerbound f f f increases with f only in the

case that f Furthermore the lowerbound is useless in the case that

f Thus an alternative lowerbound is needed in case f approaches

or is larger than it see Exercise

Exercise a b etter analysis of the BLR Linearity Test cf In con

tinuation to Exercise use the following guidelines in order to prove that

f min f Sp ecically fo cusing on the case that f show

that f is f close to some homomorphism and thus f f

def

Dene the vote of y regarding the value of f at x as x f xy f y and

y

def

dene x as the corresp onding plurality vote ie x argmax fjfy

v H

G x v gjg

y

Prove that for every x G it holds that Pr x x f

y y

Extra guideline Fixing x call a pair y y go o d if f y f y y f y

and f x y f y y f x y Prove that for any x a random pair y y

is go o d with probability at least f On the other hand for a go o d y y

x Show that the graph in which edges corresp ond to x it holds that

y y

go o d pairs must have a connected comp onent of size at least f jGj Note

that x is identical for all vertices y in this connected comp onent which in turn

y

contains a ma jority of all y s in G

Prove that is a homomorphism that is prove that for every x y G it

holds that x y x y

Extra guideline Prove that x y x y holds by considering the

def

somewhat ctitious expression p Pr x y x y and showing

xy r G

that p and hence x y x y is false Prove that p by

xy xy

showing that

x f x r f r

y f r f r y

p Pr

xy r

x y f x r f r y

CHAPTER PROBABILISTIC PROOF SYSTEMS

and using Item and some variable substitutions for upp erb ounding by f

the probability of each of the three events in Eq

Prove that f is f close to

Extra guideline Denoting B fx G Pr f x x g prove that

y G y

f jB jjGj Note that if x G n B then f x x

We comment that b etter b ounds on the b ehavior of f as a function of f are

known

Exercise testing matrix identity Let M b e a nonzero mbyn matrix

over GFp Prove that Pr r M s p where r resp s is a

rs

random mary resp nary vector

n

Guideline Prove that if v then Pr v s p and that if M has rank then

s

n

Pr r M p

r

Exercise SAT and CSP with two variables Show that SAT is reducible

fg

to gapCSP for m m where gapCSP is as in Denition Further

more show that the size of the resulting gapCSP instance is linear in the length of

the input formula

Guideline Given an instance of SAT consider the graph in which vertices corresp ond

to clauses of edges corresp ond to pairs of clauses that share a variable and the con

straints represent the natural consistency condition regarding partial assignments that

satisfy the clauses See a similar construction in Exercise

Exercise CSP with two Bo olean variables In contrast to Exercise

fg

prove that for every p ositive function N the problem gapCSP is

solvable in p olynomialtime

fg

Guideline Reduce gapCSP to SAT

Exercise Show that for any xed nite and constant c the problem

gapCSP is in PCP log O

c

Guideline Consider an oracle that for some satisfying assignment for the CSPinstance

G provides a trivial enco ding of the assignment that is for a satisfying assignment

th

V the oracle resp onds to the query v i with the i bit in the binary representation

of v Consider a verier that uniformly selects an edge u v of G and checks the

constraint when applied to the values u and v obtained from the oracle This

uv

verier makes log jj queries and reject each noinstance with probability at least c

Exercise For any constant and d show that gapCSP can b e reduced

to itself such that the instance at the target of the reduction is a dregular expander

and the fraction of violated constraints is preserved up to a constant factor That

is the instance G is reduced to G such that G is a dregular expander

graph and vltG vltG Furthermore make sure that jG j

O jGj and that each vertex in G has at least d selflo ops

PROBABILISTICALLY CHECKABLE PROOF SYSTEMS

Guideline First replace each vertex of degree d by a regular expander of size

d and connect each of the original d edges to a dierent vertex of this expander thus

obtaining a graph of maximum degree Maintain the constraints asso ciated with the

original edges and asso ciate the equality constraint ie if and only if

to each new edge residing in any of the added expanders Next augment the resulting

N vertex graph by the edges of a regular expander of size N while asso ciating with

these edges the trivially satised constraint ie for all Finally

add at least d selflo ops to each vertex using again trivially satised constraints so

to obtain a dregular graph Prove that this sequence of mo dications may only decrease

the fraction of violated constraints and that the decrease is only by a constant factor

The latter assertion relies on the equality constraints asso ciated with the small expanders

used in the rst step

Exercise freebit complexity zero Note that only sets in coRP have

PCPs of query complexity zero Furthermore Exercise implies that only sets

in P have PCP systems of logarithmic randomness and query complexity zero

Show that only sets in P have PCP systems of logarithmic randomness and

freebit complexity zero

Hint Consider an application of the FGLSSreduction to a set having a PCP of freebit

complexity zero

In contrast show that Graph NonIsomorphism has a PCP system of freebit

complexity zero and linear randomnesscomplexity

Exercise freebit complexity one In continuation to Exercise prove

that only sets in P have PCP systems of logarithmic randomness and freebit com

plexity one

Guideline Consider an application of the FGLSSreduction to a set having a PCP of

freebit complexity one and randomnesscomplexity r Note that the question of whether

r

the resulting graph has an indep endent set of size can b e expressed as a CNF formula

r

of size p oly and see Exercise

Exercise Proving Theorem Using the following guidelines pro

vide a pro of of Theorem Let S NP and consider the CNF formulae

that are obtained by the standard reduction of S to SAT ie the one provided

by the pro ofs of Theorems and Decouple the resulting CNF formulae

into pairs of formulae such that represents the hardwiring of the in

x x

put x and represents the computation itself Referring to the mapping of CNF

formulae to lowdegree extensions presented in x show that the lowdegree

extension that corresp ond to can b e evaluated in p olynomialtime ie p oly

nomial in the length of the input to which is O log jj Conclude that the

lowdegree extension that corresp onds to can b e evaluated in time jxj Al

x

ternatively note that it suces to show that the assignmentoracle A considered

in x satises and is consistent with x and is a lowdegree p olynomial

CHAPTER PROBABILISTIC PROOF SYSTEMS

Guideline Note that the circuit constructed in the pro of of Theorem is highly

uniform In particular the relation b etween wires and gates in this circuit can b e repre

sented by constantdepth circuits of unbounded fanin and p olynomialsize ie size that

is p olynomial in the length of the indices of wires and gates

Chapter

Relaxing the Requirements

The philosophers have only interpreted the world in

various ways the p oint is to change it

Karl Marx Theses on Feuerbach

In light of the apparent infeasibility of solving numerous useful computational prob

lems it is natural to ask whether these problems can b e relaxed such that the

relaxation is b oth useful and allows for feasible solving pro cedures We stress two

asp ects ab out the foregoing question on one hand the relaxation should b e suf

ciently go o d for the intended applications but on the other hand it should b e

signicantly dierent from the original formulation of the problem so to escap e the

infeasibility of the latter We note that whether a relaxation is adequate for an

intended application dep ends on the application and thus much of the material

in this chapter is less robust or generic than the treatment of the nonrelaxed

computational problems

Summary We consider two types of relaxations The rst type of

relaxation refers to the computational problems themselves that is for

each problem instance we extend the set of admissible solutions In

the context of search problems this means settling for solutions that

have a value that is suciently close to the value of the optimal

solution with resp ect to some value function Needless to say the

sp ecic meaning of suciently close is part of the denition of the

relaxed problem In the context of decision problems this means that

for some instances b oth answers are considered valid sp ecically we

shall consider promise problems in which the noinstances are far

from the yesinstances in some adequate sense which is part of the

denition of the relaxed problem

The second type of relaxation deviates from the requirement that the

solver provides an adequate answer on each valid instance Instead

the b ehavior of the solver is analyzed with resp ect to a predetermined

CHAPTER RELAXING THE REQUIREMENTS

input distribution or a class of such distributions and bad b ehavior

may o ccur with negligible probability where the probability is taken

over this input distribution That is we replace worstcase analysis by

averagecase or rather typicalcase analysis Needless to say a ma jor

comp onent in this approach is limiting the class of distributions in a way

that on one hand allows for various types of natural distributions and

on the other hand prevents the collapse of the corresp onding notion of

averagecase hardness to the standard notion of worstcase hardness

Organization The rst type of relaxation is treated in Section where we

consider approximations of search or rather optimization problems as well as

approximatedecision problems aka prop erty testing see Section and Sec

tion resp ectively The second type of relaxation known as averagetypical

case complexity is treated in Section The treatment of these two types is

quite dierent Section provides a short and highlevel introduction to various

research areas fo cusing on the main notions and illustrating them by reviewing

some results while providing no pro ofs In contrast Section provides a basic

treatment of a theory of averagetypicalcase complexity fo cusing on some basic

results and providing a rather detailed exp osition of the corresp onding pro ofs

Approximation

The notion of approximation is a very natural one and has arisen also in other

disciplines Approximation is most commonly used in references to quantities eg

the length of one meter is approximately forty inches but it is also used when

referring to qualities eg an approximately correct account of a historical event

In the context of computation the notion of approximation mo dies computational

tasks such as search and decision problems In fact we have already encountered

it as a mo dier of counting problems see Section

Two ma jor questions regarding approximation are what is a go o d approx

imation and can it b e found easier than nding an exact solution The answer

to the rst question seems intimately related to the sp ecic computational task

at hand and to its role in the wider context ie the higher level application a

go o d approximation is one that suces for the intended application Indeed the

imp ortance of certain approximation problems is much more sub jective than the

imp ortance of the corresp onding optimization problems This fact seems to stand

in the way of attempts at providing a comprehensive theory of natural approxi

mation problems eg general classes of natural approximation problems that are

shown to b e computationally equivalent

Turning to the second question we note that in numerous cases natural approx

imation problems seem to b e signicantly easier than the corresp onding original

exact problems On the other hand in numerous other cases natural approx

imation problems are computationally equivalent to the original problems We

shall exemplify b oth cases by reviewing some sp ecic results but will not provide

APPROXIMATION

a general systematic classication b ecause such a classication is not known

We shall distinguish b etween approximation problems that are of a search

type and problems that have a clear decisional avor In the rst case we shall

refer to a function that assigns values to p ossible solutions of a search problem

whereas in the second case we shall refer to the distance b etween instances of a

decision problem We note that sometimes the same computational problem

may b e cast in b oth ways but for most natural approximation problems one of the

two frameworks is more app ealing than the other The common theme underlying

b oth frameworks is that in each of them we extend the set of admissible solutions

In the case of search problems we augment the set of optimal solutions by allowing

also almostoptimal solutions In the case of decision problems we extend the set

of solutions by allowing an arbitrary answer solution to some instances which

may b e viewed as a promise problem that disallows these instances In this case we

fo cus on promise problems in which the yes and noinstances are far apart and

the instances that violate the promise are closed to yesinstances

Teaching note Most of the results presented in this section refer to sp ecic computa

tional problems and with one exception are presented without a pro of In view of the

complexity of the corresp onding pro ofs and the merely illustrative role of these results

in the context of complexity theory we recommend doing the same in class

Search or Optimization

As noted in Section many search problems involve a set of p otential solutions

p er each problem instance such that dierent solutions are assigned dierent val

ues resp costs by some value resp cost function In such a case one is

interested in nding a solution of maximum value resp minimum cost A corre

sp onding approximation problem may refer to nding a solution of approximately

maximum value resp approximately minimum cost where the sp ecication of

the desired level of approximation is part of the problems denition Let us elab

orate

For concreteness we fo cus on the case of a value that we wish to maximize

For greater expressibility or actually for greater exibility we allow the value

of the solution to dep end also on the instance itself Thus for a p olynomially

b ounded binary relation R and a value function f f g f g R we

consider the problem of nding solutions with resp ect to R that maximize the

In contrast systematic classications of restricted classes of approximation problems are

known For example see for a classication of approximate versions of Constraint Satis

faction Problems

In some sense this distinction is analogous to the distinction b etween the two aforementioned

uses of the word approximation

This convention is only a matter of convenience without loss of generality we can express

the same optimization problem using a value function that only dep ends on the solution by

augmenting each solution with the corresp onding instance ie a solution y to an instance x can

b e enco ded as a pair x y and the resulting set of valid solutions for x will consist of pairs of the

form x Hence the foregoing convention merely allows avoiding this cumbersome enco ding

of solutions

CHAPTER RELAXING THE REQUIREMENTS

value of f That is given x such that R x the task is nding y R x such

that f x y v where v is the maximum value of f x y over all y R x

x x

Typically R is in PC and f is p olynomialtime computable Indeed without loss

jxj

of generality we may assume that for every x it holds that R x f g for

some p olynomial see Exercise Thus the optimization problem is recast

as the following search problem given x nd y such that f x y v where

x

v max ff x y g

jxj

x

y fg

We shall fo cus on relative approximation problems where for some gap function

g f g fr R r g the maximization task is nding y such that f x y

v g x Indeed in some cases the approximation factor is stated as a function of

x

the length of the input ie g x g jxj for some g N fr R r g but

often the approximation factor is stated in terms of some more rened parameter

of the input eg as a function of the number of vertices in a graph Typically g

is p olynomialtime computable

Denition g factor approximation Let f f g f g R

N N and g f g fr R r g

Maximization version The g factor approximation of maximizing f wrt is the

jxj

search problem R such that R x fy f g f x y v g xg

x

where v max ff x y g

jxj

x

y fg

Minimization version The g factor approximation of minimizing f wrt is the

jxj

search problem R such that R x fy f g f x y g x c g

x

where c min ff x y g

jxj

x

y fg

We note that for numerous NPcomplete optimization problems p olynomialtime

algorithms provide meaningful approximations A few examples will b e mentioned

in x In contrast for numerous other NPcomplete optimization problems

natural approximation problems are computationally equivalent to the corresp ond

ing optimization problem A few examples will b e mentioned in x where

we also introduce the notion of a gap problem which is a promise problem of

the decision type intended to capture the diculty of the approximate search

problem

A few p ositive examples

Let us start with a trivial example Considering a problem such as nding the

maximum clique in a graph we note that nding a linear factor approximation is

trivial ie given a graph G V E we may output any vertex in V as a jV j

factor approximation of the maximum clique in G A famous nontrivial example

is presented next

Prop osition factor two approximation to minimum Vertex Cover There

exists a polynomialtime approximation algorithm that given a graph G V E

However in this case and in contrast to Footnote the value function f must dep end b oth

on the instance and on the solution ie f x y may no b e oblivious of x

APPROXIMATION

outputs a vertex cover that is at most twice as large as the minimum vertex cover

of G

We warn that an approximation algorithm for minimum Vertex Cover do es not

yield such an algorithm for the complementary search problem of maximum Independent

Set This phenomenon stands in contrast to the case of optimization where an

optimal solution for one search problem eg minimum Vertex Cover yields an

optimal solution for the complementary search problem maximum Independent

Set

Pro of Sketch The main observation is a connection b etween the set of maximal

matchings and the set of vertex covers in a graph Let M b e any maximal matching

in the graph G V E that is M E is a matching but augmenting it by any

single edge yields a set that is not a matching Then on one hand the set of all

vertices participating in M is a vertex cover of G and on the other hand each

vertex cover of G must contain at least one vertex of each edge of M Thus we can

nd the desired vertex cover by nding a maximal matching which in turn can b e

found by a greedy algorithm

Another example An instance of the traveling salesman problem TSP consists

of a symmetric matrix of distances b etween pairs of p oints and the task is nding

a shortest tour that passes through all p oints In general no reasonable approx

imation is feasible for this problem see Exercise but here we consider two

sp ecial cases in which the distances satisfy some natural constraints and pretty

go o d approximations are feasible

Theorem approximations to sp ecial cases of TSP Polynomialtime algo

rithms exist for the fol lowing two computational problems

Providing a factor approximation for the special case of TSP in which the

distances satisfy the triangle inequality

For every providing a factor approximation for the special case

of Euclidean TSP ie for some constant k eg k the p oints reside

in a k dimensional Euclidean space and the distances refer to the standard

Euclidean norm

A weaker version of Part is given in Exercise A detailed survey of Part

is provided in We note the dierence exemplied by the two items of Theo

rem Whereas Part provides a p olynomialtime approximation for a sp ecic

constant factor Part provides such an algorithm for any constant factor Such a

result is called a polynomialtime approximation scheme abbreviated PTAS

A few negative examples

Let us start again with a trivial example Considering a problem such as nding

the maximum clique in a graph we note that given a graph G V E nding

CHAPTER RELAXING THE REQUIREMENTS

a jV j factor approximation of the maximum clique in G is as hard as

nding a maximum clique in G Indeed this result is not really meaningful

In contrast building on the PCP Theorem Theorem one may prove that

o

nding a jV j factor approximation of the maximum clique in a general graph

G V E is as hard as nding a maximum clique in a general graph This follows

from the fact that the approximation problem is NPhard cf Theorem

The statement of such inapproximability results is made stronger by referring

to a promise problem that consists of distinguishing instances of suciently far

apart values Such promise problems are called gap problems and are typically

stated with resp ect to two b ounding functions g g f g R which replace

the gap function g of Denition Typically g and g are p olynomialtime

computable

Denition gap problem for approximation of f Let f be as in Deni

tion and g g f g R

problem of maximizing f consists Maximization version For g g the gap

g g

of distinguishing between fx v g xg and fx v g xg where

x x

v max ff x y g

jxj

x

yfg

problem of minimizing f consists Minimization version For g g the gap

g g

of distinguishing between fx c g xg and fx c g xg where

x x

c min ff x y g

jxj

x

yfg

problem of maximizing the size of a clique in a graph For example the gap

g g

consists of distinguishing b etween graphs G that have a clique of size g G and

graphs G that have no clique of size g G In this case we typically let g G b e a

i

function of the number of vertices in G V E that is g G g jV j Indeed

i

i

letting G denote the size of the largest clique in the graph G we let gapClique

Ls

denote the gap problem of distinguishing b etween fG V E G LjV jg

and fG V E G sjV jg where L s Using this terminology we restate

o

and strengthen the aforementioned jV j factor inapproximability result of

the maximum clique problem

o o

Theorem For some LN N and sN N it holds that gapClique

Ls

is NPhard

The pro of of Theorem is based on a ma jor renement of Theorem that

refers to a PCP system of amortized freebit complexity that tends to zero cf

x A weaker result which follows from Theorem itself is presented in

Exercise

As we shall show next results of the type of Theorem imply the hardness

of a corresp onding approximation problem that is the hardness of deciding a gap

problem implies the hardness of a search problem that refers to an analogous factor

of approximation

APPROXIMATION

Prop osition Let f g g be as in Denition and suppose that these

problem of maximiz functions are polynomialtime computable Then the gap

g g

ing f resp minimizing f is reducible to the g g factor resp g g factor

approximation of maximizing f resp minimizing f

Note that a reduction in the opp osite direction do es not necessarily exist even in

the case that the underlying optimization problem is selfreducible in some natural

sense Indeed this is another dierence b etween the current context of approx

imation and the context of optimization problems where the search problem is

reducible to a related decision problem

Pro of Sketch We fo cus on the maximization version On input x we solve the

problem by making the query x obtaining the answer y and ruling that gap

g g

x has value at least g x if and only if f x y g x Recall that we need to

analyze this reduction only on inputs that satisfy the promise Thus if v g x

x

then the oracle must return a solution y that satises f x y v g xg x

x

which implies that f x y g x On the other hand if v g x then f x y

x

v g x holds for any p ossible solution y

x

problem of mini Additional examples Let us consider gapVC the gap

g g

sL

s L

mizing the vertex cover of a graph where s and L are constants and g G s jV j

s

resp g G L jV j for any graph G V E Then Prop osition implies

L

via Prop osition that for every constant s the problem gapVC is solvable

ss

in p olynomialtime In contrast suciently narrowing the gap b etween the two

thresholds yields an inapproximability result In particular

s eg Theorem For some constants s and L such that L

s and L the problem gapVC is NPhard

sL

The pro of of Theorem is based on a complicated renement of Theorem

Again a weaker result follows from Theorem itself see Exercise

As noted renements of the PCP Theorem Theorem play a key role in

establishing inapproximability results such as Theorems and In that

resp ect it is adequate to recall that Theorem establishes the equivalence of

the PCP Theorem itself and the NPhardness of a gap problem concerning the

maximization of the number of clauses that are satises in a given CNF for

mula Sp ecically gapSAT was dened in Denition as the gap problem

consisting of distinguishing b etween satisable CNF formulae and CNF formu

lae for which each truth assignment violates at least an fraction of the clauses

Although Theorem do es not sp ecify the quantitative relation that underlies

its qualitative assertion when rened and combined with the b est known PCP

construction it do es yield the b est p ossible b ound

Theorem For every v the problem gapSAT is NPhard

v

On the other hand gapSAT is solvable in p olynomialtime

CHAPTER RELAXING THE REQUIREMENTS

Sharp thresholds The aforementioned opp osite results regarding gapSAT ex

v

emplify a sharp threshold on the factor of approximation that can b e obtained

by an ecient algorithm Another app ealing example refers to the following maxi

mization problem in which the instances are systems of linear equations over GF

and the task is nding an assignment that satises as many equations as p ossible

Note that by merely selecting an assignment at random we exp ect to satisfy half

of the equations Also note that it is easy to determine whether there exists an

assignment that satises all equations Let gapLin denote the problem of dis

Ls

tinguishing b etween systems in which one can satisfy at least an L fraction of

the equations and systems in which one cannot satisfy an s fraction or more

of the equations Then as just noted gapLin is trivial for every L

L

and gapLin is feasible for every s In contrast moving b oth thresholds

s

slightly away from the corresp onding extremes yields an NPhard gap problem

Theorem For every constant the problem gapLin is NPhard

The pro of of Theorem is based on a ma jor renement of Theorem In fact

the corresp onding PCP system for NP is merely a reformulation of Theorem

the verier makes three queries and tests a linear condition regarding the answers

while using a logarithmic number of coin tosses This verier accepts any yes

instance with probability at least when given oracle access to a suitable

pro of and rejects any noinstance with probability at least regardless

of the oracle b eing accessed A weaker result which follows from Theorem

itself is presented in Exercise

Gap lo cation Theorems and illustrate two opp osite situations with

resp ect to the lo cation of the gap for which the corresp onding promise prob

lem is hard Recall that b oth gapSAT and gapLin are formulated with resp ect

to two thresholds where each threshold b ounds the fraction of lo cal conditions

ie clauses or equations that are satisable in the case of yes and noinstances

resp ectively In the case of gapSAT the high threshold referring to yesinstances

was set to and thus only the low threshold referring to noinstances remained

a free parameter Nevertheless a hardness result was established for gapSAT and

furthermore this was achieved for an optimal value of the low threshold cf the

foregoing discussion of sharp thresholds In contrast in the case of gapLin set

ting the high threshold to makes the gap problem eciently solvable Thus

the hardness of gapLin was established at a dierent lo cation of the high thresh

old Sp ecically hardness for an optimal value of the ratio of thresholds was

established when setting the high threshold to for any

A nal comment All the aforementioned inapproximability results refer to ap

proximation resp gap problems that are relaxations of optimization problems

in NP ie the optimization problem is computationally equivalent to a decision

problem in NP see Section In these cases the NPhardness of the approx

imation resp gap problem implies that the corresp onding optimization problem

is reducible to the approximation resp gap problem In other words in these

APPROXIMATION

cases nothing is gained by relaxing the original optimization problem b ecause the

relaxed version remains just as hard

Decision or Prop erty Testing

A natural notion of relaxation for decision problems arises when considering the

distance b etween instances where a natural notion of distance is the Hamming

distance ie the fraction of bits on which two strings disagree Lo osely sp eaking

this relaxation called property testing refers to distinguishing inputs that reside

in a predetermined set S from inputs that are relatively far from any input that

resides in the set Two natural types of promise problems emerge with resp ect to

any predetermined set S and the Hamming distance b etween strings

Relaxed decision wrt a xed relative distance Fixing a distance parameter

we consider the problem of distinguishing inputs in S from inputs in S

where

def

jxj

S fx z S f g x z jxjg

and x x z z jfi x z gj denotes the number of bits on

m m i i

which x x x and z z z disagree Thus here we consider a

m m

promise problem that is a restriction or a sp ecial case of the problem of

deciding membership in S

Relaxed decision wrt a variable distance Here the instances are pairs x

where x is as in Type and is a relative distance parameter The

yesinstances are pairs x such that x S whereas x is a noinstance

if x S

We shall fo cus on Type formulation which seems to capture the essential question

of whether or not these relaxations lower the complexity of the original decision

problem The study of Type formulation refers to a relatively secondary question

which assumes a p ositive answer to the rst question that is assuming that the

relaxed form is easier than the original form we ask how is the complexity of the

problem aected by making the distance parameter smaller which means making

the relaxed problem tighter and ultimately equivalent to the original problem

We note that for numerous NPcomplete problems there exist natural Type

relaxations that are solvable in p olynomialtime Actually these algorithms run

in sublinear time sp ecically in p olylogarithmic time when given direct access

to the input A few examples will b e presented in x but as indicated in

x this is not a generic phenomenon Before turning to these examples we

discuss several imp ortant denitional issues

Denitional issues

Prop erty testing is concerned not only with solving relaxed versions of NPhard

problems but rather with solving these problems as well as problems in P in

sublinear time Needless to say such results assume a mo del of computation in

CHAPTER RELAXING THE REQUIREMENTS

which algorithms have direct access to bits in the representation of the input see

Denition

Denition a direct access mo del conventions An algorithm with direct

access to its input is given its main input on a special input device that is accessed

as an oracle see x In addition the algorithm is given the length of the

input and possibly other parameters on a secondary input device The complexity of

such an algorithm is stated in terms of the length of its main input

Indeed the description in x refers to such a mo del but there the main input

is viewed as an oracle and the secondary input is viewed as the input In the

current mo del p olylogarithmic time means time that is p olylogarithmic in the

length of the main input which means time that is p olynomial in the length of the

binary representation of the length of the main input Thus p olylogarithmic time

yields a robust notion of extremely ecient computations As we shall see such

computations suce for solving various prop erty testing problems

Denition prop erty testing for S For any xed the promise

problem of distinguishing S from S is called property testing for S with resp ect

to

Recall that we say that a randomized algorithm solves a promise problem if it

accepts every yesinstance resp rejects every noinstance with probability at

least Thus a randomized prop erty testing for S accepts every input in S

resp rejects every input in S with probability at least

The question of representation The sp ecic representation of the input is of

ma jor concern in the current context This is due to the eect of the represen

tation on the distance measure and to the dependence of direct access machines

on the specic representation of the input Let us elab orate on b oth asp ects

Recall that we dened the distance b etween ob jects in terms of the Hamming

distance b etween their representations Clearly in such a case the choice of

representation is crucial and dierent representations may yield dierent dis

tance measures Furthermore in this case the distance b etween ob jects is

not preserved under various natural representations that are considered

equivalent in standard studies of computational complexity For example

in previous parts of this b o ok when referring to computational problems

concerning graphs we did not care whether the graph was represented by its

adjacency matrix or by its incidencelist In contrast these two representa

tions induce very dierent distance measures and corresp ondingly dierent

prop erty testing problems see x Likewise the use of padding and

other trivial syntactic conventions b ecomes problematic eg when using a

signicant amount of padding all ob jects are deemed close to one another

and prop erty testing for any set b ecomes trivial

APPROXIMATION

Since our fo cus is on sublinear time algorithms we may not aord trans

forming the input from one natural format to another Thus representations

that are considered equivalent with resp ect to p olynomialtime algorithms

may not b e equivalent with resp ect to sublinear time algorithms that have

a direct access to the representation of the ob ject For example adjacency

queries and incidence queries cannot emulate one another in small time ie

in time that is sublinear in the number of vertices

Both asp ects are further claried by the examples provided in x

The essential role of the promise Recall that for a xed constant

we consider the promise problem of distinguishing S from S The promise

means that all instances that are neither in S nor far from S ie not in S

are ignored which is essential for sublinear algorithms for natural problems This

makes the prop erty testing task p otentially easier than the corresp onding stan

dard decision task cf x To demonstrate the p oint consider the set S

consisting of strings that have a ma jority of s Then deciding membership in

S requires linear time b ecause random nbit long strings with bnc ones cannot

b e distinguished from random nbit long strings with bnc ones by probing

a sublinear number of lo cations even if randomization and error probability are

allowed see Exercise On the other hand the fraction of s in the input can

b e approximated by a randomized p olylogarithmic time algorithm which yields a

prop erty tester for S see Exercise Thus for some sets deciding membership

requires linear time while prop erty testing can b e done in p olylogarithmic time

The essential role of randomization Referring to the foregoing example we

note that randomization is essential for any sublinear time algorithm that distin

guishes this set S from say S Sp ecically a sublinear time deterministic

n

algorithm cannot distinguish from any input that has s in each p osition prob ed

n

by that algorithm on input In general on input x a sublinear time deter

ministic algorithm always reads the same bits of x and thus cannot distinguish x

from any z that agrees with x on these bit lo cations

Note that in b oth cases we are able to prove lowerbounds on the time com

plexity of algorithms This success is due to the fact that these lowerbounds are

actually information theoretic in nature that is these lowerbounds actually refer

to the number of queries p erformed by these algorithms

Two mo dels for testing graph prop erties

In this subsection we consider the complexity of prop erty testing for sets of graphs

that are closed under graph isomorphism such sets are called graph properties In

view of the imp ortance of representation in the context of prop erty testing we

explicitly consider two standard representations of graphs cf App endix G

which indeed yield two dierent mo dels of testing graph prop erties

CHAPTER RELAXING THE REQUIREMENTS

The adjacency matrix representation Here a graph G N E is rep

resented in a somewhat redundant form by an N byN Bo olean matrix

M m such that m if and only if fi j g E

G ij ij

ij N

Bounded incidencelists representation For a xed parameter d a graph

G N E of degree at most d is represented in a somewhat redundant

form by a mapping N d N fg such that u i v if v is

G G

th

the i neighbor of u and u i if v has less than i neighbors

G

We stress that the aforementioned representations determine b oth the notion of

distance b etween graphs and the type of queries p erformed by the algorithm As

we shall see the dierence b etween these two representations yields a big dierence

in the complexity of corresp onding prop erty testing problems

Theorem prop erty testing in the adjacency matrix representation For

any xed and each of the fol lowing sets there exists a polylogarithmic time

randomized algorithm that solves the corresponding property testing problem with

resp ect to

For every xed k the set of k colorable graphs

For every xed the set of graphs having a clique resp indep endent

set of density

For every xed the set of N vertex graphs having a cut with at least

N edges

For every xed the set of N vertex graphs having a bisection with at

most N edges

In contrast for some there exists a graph property in NP for which property

testing with resp ect to requires linear time

The testing algorithms asserted in Theorem use a constant number of

queries where this constant is p olynomial in the constant In contrast exact

decision pro cedures for the corresp onding sets require a linear number of queries

The running time of the aforementioned algorithms hides a constant that is exp o

nential in their query complexity except for the case of colorability where the

hidden constant is p olynomial in Note that such dep endencies seem essen

tial since setting N regains the original nonrelaxed decision problems

which with the exception of colorability are all NPcomplete Turning to the

lowerbound asserted in Theorem we mention that the graph prop erty for

which this b ound is proved is not a natural one As in x the lowerbound

on the time complexity follows from a lowerbound on the query complexity

Theorem exhibits a dichotomy b etween graph prop erties for which prop

erty testing is p ossible by a constant number of queries and graph prop erties for

A cut in a graph G N E is a partition S S of the set of vertices ie S S N

and S S and the edges of the cut are the edges with exactly one endp oint in S A

bisection is a cut of the graph to two parts of equal cardinality

APPROXIMATION

which prop erty testing requires a linear number of queries A combinatorial charac

terization of the graph prop erties for which prop erty testing is p ossible in the ad

jacency matrix representation when using a constant number of queries is known

We note that the constant in this characterization may dep end arbitrarily on and

indeed in some cases it is a function growing faster than a tower of exp onents

For example prop erty testing for the set of trianglefree graphs is p ossible by using

a number of queries that dep ends only on but it is known that this number must

grow faster than any p olynomial in

Turning back to Theorem we note that the results regarding prop erty

testing for the sets corresp onding to maxcut and minbisection yield approximation

algorithms with an additive error term of N For dense graphs ie N vertex

graphs having N edges this yields a constant factor approximation for the

standard approximation problem as in Denition That is for every constant

c we obtain a cfactor approximation of the problem of maximizing the size of a

cut resp minimizing the size of a bisection in dense graphs On the other hand

the result regarding clique yields a so called dualapproximation for maximum

clique that is we approximate the minimum number of missing edges in the densest

induced subgraph of a given size

Indeed Theorem is meaningful only for dense graphs This holds in

general for any graph prop erty in the adjacency matrix representation Also note

that prop erty testing is trivial under the adjacency matrix representation for any

graph prop erty S satisfying S eg the set of connected graphs the set

o

of Hamiltonian graphs etc

We now turn to the b ounded incidencelists representation which is relevant

only for b ounded degree graphs The problems of maxcut minbisection and clique

as in Theorem are trivial under this representation but graph connectivity

b ecomes nontrivial and the complexity of prop erty testing for the set of bipartite

graphs changes dramatically

Theorem prop erty testing in the b ounded incidencelists representation

The fol lowing assertions refer to the representation of graphs by incidencelists of

length d

For any xed d and there exists a polylogarithmic time randomized

algorithm that solves the property testing problem for the set of connected

graphs of degree at most d

For any xed d and there exists a sublinear time randomized algorithm

that solves the property testing problem for the set of bipartite graphs of degree

Describing this fascinating result of Alon et al which refers to the notion of regular

partitions introduced by Szemeredi is b eyond the scop e of the current text

In this mo del as shown next prop erty testing of nondense graphs is trivial Sp ecically

N

xing the distance parameter we call a N vertex graph nondense if it has less than

edges The p oint is that for nondense graphs the prop erty testing problem for any set S is

trivial b ecause we may just accept any nondense N vertex graph if and only if S contains

some nondense N vertex graph Clearly the decision is correct in the case that S do es not

contain nondense graphs However the decision is admissible also in the case that S do es contain

some nondense graph b ecause in this case every nondense graph is close to S ie it is not

in S

CHAPTER RELAXING THE REQUIREMENTS

at most d Specically on input an N vertex graph the algorithm runs for

p

e

N time O

For any xed d and some property testing for the set of N vertex

p

regular bipartite graphs requires N queries

For some xed d and property testing for the set of N vertex colorable

graphs of degree at most d requires N queries

The running time of the algorithms asserted in Theorem hides a constant

that is p olynomial in Providing a characterization of graph prop erties accord

ing to the complexity of the corresp onding tester in the b ounded incidencelists

representation is an interesting op en problem

Decoupling the distance from the representation So far we have conned

our attention to the Hamming distance b etween the representations of graphs

This made the choice of representation even more imp ortant than usual ie more

crucial than is common in complexity theory In contrast it is natural to consider

a notion of distance b etween graphs that is indep endent of their representation

For example the distance b etween G V E and G V E can b e dened

as the minimum of the size of symmetric dierence b etween E and the set of edges

in a graph that is isomorphic to G The corresp onding relative distance may b e

dened as the distance divided by jE j jE j or by maxjE j jE j

Beyond graph prop erties

Prop erty testing has b een applied to a variety of computational problems b eyond

the domain of graph theory In fact this type of computational problems rst

emerged in the algebraic domain where the instances to b e viewed as inputs to

the testing algorithm are functions and the relevant prop erties are sets of algebraic

functions The archetypical example is the set of lowdegree p olynomials that is

mvariate p olynomials of total or individual degree d over some nite eld GF q

where m d and q are parameters that may dep end on the length of the input or

satisfy some relationships eg q d m Note that in this case the input

is the full or explicit description of an mvariate function over GFq which

m

means that it has length q log q Viewing the problem instance as a function

suggests a natural measure of distance ie the fraction of arguments on which the

functions disagree as well as a natural way of accessing the instance ie querying

the function for the value of selected arguments

Note that we have referred to these computational problems under a dierent

terminology in x and in x In particular in x we refereed to

the sp ecial case of linear Bo olean functions ie individual degree and q

whereas in x we used the setting q p oly d and m d log d where d is a

b ound on the total degree

Other domains of computational problems in which prop erty testing was stud

ied include geometry eg clustering problems formal languages eg testing

AVERAGE CASE COMPLEXITY

membership in regular sets co ding theory cf App endix E probability the

ory eg testing equality of distributions and combinatorics eg monotone and

junta functions As discuss at the end of x it is often natural to decou

ple the distance measure from the representation of the ob jects ie the way of

accessing the problem instance This is done by introducing a representation

indep endent notion of distance b etween instances which should b e natural in the

context of the problem at hand

Average Case Complexity

Teaching note We view averagecase complexity as referring to the p erformance on

average or rather typical instances and not as the average p erformance on random

instances This choice is justied in x Thus it may b e more justied to refer to

the following theory by the name typicalcase complexity Still the name averagecase

was retained for historical reasons

Our approach so far including in Section is termed worstcase complex

ity b ecause it refers to the p erformance of p otential algorithms on each legitimate

instance and hence to the p erformance on the worst p ossible instance That is

computational problems were dened as referring to a set of instances and p erfor

mance guarantees were required to hold for each instance in this set In contrast

averagecase complexity allows ignoring a negligible measure of the p ossible in

stances where the identity of the ignored instances is determined by the analysis

of potential solvers and not by the problems statement

A few comments are in place Firstly as just hinted the standard statement

of the worstcase complexity of a computational problem esp ecially one having

a promise may also ignores some instances ie those considered inadmissible

or violating the promise but these instances are determined by the problems

statement In contrast the inputs ignored in averagecase complexity are not

inadmissible in any inherent sense and are certainly not identied as such by the

problems statement It is just that they are viewed as exceptional when claiming

that a sp ecic algorithm solve the problem that is these exceptional instances are

determined by the analysis of that algorithm Needless to say these exceptional

instances ought to b e rare ie o ccur with negligible probability

The last sentence raises a couple of issues Most imp ortantly a distribution

on the set of admissible instances has to b e sp ecied In fact we shall consider a

new type of computational problems each consisting of a standard computational

problem coupled with a probability distribution on instances Consequently the

question of which distributions should b e considered in a theory of averagecase

complexity arises This question and numerous other denitional issues will b e

addressed in x

Before pro ceeding let us sp ell out the rather straightforward motivation to the

study of the averagecase complexity of computational problems It is that in real

life applications one may b e p erfectly happy with an algorithm that solves the

problem fast on almost all instances that arise in the relevant application That is

CHAPTER RELAXING THE REQUIREMENTS

one may b e willing to tolerate error provided that it o ccurs with negligible proba

bility where the probability is taken over the distribution of instances encountered

in the application The study of averagecase complexity is aimed at exploring the

p ossible b enet of such a relaxation distinguishing cases in which a b enet exists

from cases in which it do es not exist A key asp ect in such a study is a go o d

mo deling of the type of distributions of instances that are encountered in natural

algorithmic applications

A preliminary question that arises is whether every natural computational prob

lem be solve eciently when restricting attention to typical instances The conjec

ture that underlies this section is that for a wellmotivated choice of denitions the

answer is negative that is our conjecture is that the distributional version of NP

is not contained in the averagecase or typicalcase version of P This means that

some NP problems are not merely hard in the worstcase but are rather typically

hard ie hard on typical instances drawn from some simple distribution Sp ecif

ically hard instances may occur in natural algorithmic applications and not only

in cryptographic or other adversarial applications that are design on purp ose

to pro duce hard instances

The foregoing conjecture motivates the development of an averagecase analogue

of NPcompleteness which will b e presented in this section Indeed the entire

section may b e viewed as an averagecase analogue of Chapter In particular this

averagecase theory identies distributional problems that are typically hard

provided that distributional problems that are typically hard exist at all If one

b elieves the foregoing conjecture then for such complete distributional problems

one should not seek algorithms that solve these problems eciently on typical

instances

Organization A ma jor part of our exp osition is devoted to the denitional issues

that arise when developing a general theory of averagecase complexity These

issues are discussed in x In x we prove the existence of distributional

problems that are NPcomplete in the corresp onding averagecase complexity

sense Furthermore we show how to obtain such a distributional version for any

natural NPcomplete decision problem In x we extend the treatment to

randomized algorithms Additional ramications are presented in Section

The basic theory

In this section we provide a basic treatment of the theory of averagecase com

plexity while p ostp oning imp ortant ramications to Section The basic

treatment consists of the preferred denitional choices for the main concepts as

We highlight two dierences b etween the current context of natural algorithmic applications

and the context of cryptography Firstly in the current context and when referring to problems

that are typically hard the simplicity of the underlying input distribution is of great concern

the simpler this distribution the more app ealing the hardness assertion b ecomes This concern

is irrelevant in the context of cryptography On the other hand see discussion at the b eginning

of Section andor at end of x cryptographic applications require the ability to

eciently generate hard instances together with corresponding solutions

AVERAGE CASE COMPLEXITY

well as the identication of complete problems for a natural class of averagecase

computational problems

Denitional issues

The theory of averagecase complexity is more subtle than may app ear at rst

thought In addition to the generic conceptual diculty involved in dening relax

ations diculties arise from the interface b etween standard probabilistic analysis

and the conventions of complexity theory This is most striking in the deni

tion of the class of feasible averagecase computations Referring to the theory of

worstcase complexity as a guideline we shall address the following asp ects of the

analogous theory of averagecase complexity

Setting the general framework We shall consider distributional problems

which are standard computational problems see Section coupled with

distributions on the relevant instances

Identifying the class of feasible distributional problems Seeking an average

case analogue of classes such as P we shall reject the rst denition that

comes to mind ie the naive notion of average p olynomialtime briey

discuss several related alternatives and adopt one of them for the main treat

ment

Identifying the class of interesting distributional problems Seeking an

averagecase analogue of the class NP we shall avoid b oth the extreme

of allowing arbitrary distributions which collapses averagecase hardness to

worstcase hardness and the opp osite extreme of conning the treatment to

a single distribution such as the uniform distribution

Developing an adequate notion of reduction among distributional problems

As in the theory of worstcase complexity this notion should preserve feasible

solveability in the current distributional context

We now turn to the actual treatment of each of the aforementioned asp ects

Step Dening distributional problems Focusing on decision problems

we dene distributional problems as pairs consisting of a decision problem and a

probability ensemble For simplicity here a probability ensemble fX g is a

n

nN

n

sequence of random variables such that X ranges over f g Thus S fX g

n n

nN

is the distributional problem consisting of the problem of deciding membership in

the set S with resp ect to the probability ensemble fX g The treatment of

n

nN

search problem is similar see x We denote the uniform probability ensemble

n

by U fU g that is U is uniform over f g

n n

nN

We mention that even this choice is not evident Sp ecically Levin see discussion

in advocates the use of a single probability distribution dened over the set of all strings

His argument is that this makes the theory less representationdependent At the time we were

convinced of his argument see but currently we feel that the representationdependent

eects discussed in are legitimate Furthermore the alternative formulation of

comes across as unnatural and tends to confuse some readers

CHAPTER RELAXING THE REQUIREMENTS

Step Identifying the class of feasible problems The rst idea that

comes to mind is dening the problem S fX g as feasible on the average

n

nN

if there exists an algorithm A that solves S such that the average running time

of A on X is b ounded by a p olynomial in n ie there exists a p olynomial p

n

such that Et X pn where t x denotes the runningtime of A on input

A n A

x The problem with this denition is that it very sensitive to the mo del of

computation and is not closed under algorithmic comp osition Both deciencies

are a consequence of the fact that t may b e p olynomial on the average with

A

jx j

resp ect to fX g but t may fail to b e so eg consider t x x if

n A

nN

A

x x and t x x jx x j otherwise coupled with the uniform distribution

A

n

over f g We conclude that the average runningtime of algorithms is not a

robust notion We also doubt the naive app eal of this notion and view the typical

running time of algorithms as dened next as a more natural notion Thus we

shall consider an algorithm as feasible if its runningtime is typically p olynomial

We say that A is typically p olynomialtime on X fX g if there exists a

n

nN

p olynomial p such that the probability that A runs more that pn steps on X

n

is negligible ie for every p olynomial q and all suciently large n it holds that

Pr t X pn q n The question is what is required in the untypical

A n

cases and two p ossible denitions follow

The simpler option is saying that S fX g is typically feasible if there

n

nN

exists an algorithm A that solves S such that A is typically p olynomialtime

on X fX g This eectively requires A to correctly solve S on each

n

nN

instance which is more than was required in the motivational discussion

Indeed if the underlying motivation is ignoring rare cases then we should

ignore them altogether rather than ignoring them in a partial manner ie

only ignore their aect on the runningtime

The alternative which ts the motivational discussion is saying that S X

is typically feasible if there exists an algorithm A such that A typically

solves S on X in p olynomialtime that is there exists a p olynomial p such

that the probability that on input X algorithm A either errs or runs more

n

that pn steps is negligible This formulation totally ignores the untypical

instances Indeed in this case we may assume without loss of generality

that A always runs in p olynomialtime see Exercise but we shall not

do so here in order to facilitate viewing the rst option as a sp ecial case of

the current option

We stress that b oth alternatives actually dene typical feasibility and not average

case feasibility To illustrate the dierence b etween the two options consider the

distributional problem of deciding whether a uniformly selected nvertex graph

An alternative choice taken by Levin see discussion in is considering as feasible

wrt X fX g any algorithm that runs in time that is p olynomial in a function that is

n

nN

linear on the average wrt X that is requiring that there exists a p olynomial p and a function

f g N such that tx px for every x and EX O n This denition is

n

robust ie it do es not suer from the aforementioned deciencies and is arguably as natural

as the naive denition ie Et X p oly n

n A

AVERAGE CASE COMPLEXITY

is colorable Intuitively this problem is typically trivial with resp ect to the

uniform distribution b ecause the algorithm may always say no and b e wrong

with exp onentially vanishing probability Indeed this trivial algorithm is admissi

ble by the second approach but not by the rst approach In light of the foregoing

discussions we adopt the second approach

Denition the class tp c P We say that A typically solves S fX g

n

nN

in p olynomialtime if there exists a polynomial p such that the probability that on

input X algorithm A either errs or runs more that pn steps is negligible We

n

denote by tp cP the class of distributional problems that are typically solvable in

polynomialtime

Clearly for every S P and every probability ensemble X it holds that S X

tp c P However tp c P contains also distributional problems S X with S P

see Exercises and The big question which underlies the theory of

averagecase complexity is whether all natural distributional versions of NP are

in tp cP Thus we turn to identify such versions

Step Identifying the class of interesting problems Seeking to identify

reasonable distributional versions of NP we note that two extreme choices should

b e avoided On one hand we must limit the class of admissible distributions so to

prevent the collapse of averagecase hardness to worstcase hardness by a selection

of a pathological distribution that resides on the worst case instances On the

other hand we should allow for various types of natural distributions rather than

conning attention merely to the uniform distribution Recall that our aim is

addressing all p ossible input distributions that may o ccur in applications and thus

there is no justication for conning attention to the uniform distribution Still

arguably the distributions o ccuring in applications are relatively simple and so

we seek to identify a class of simple distributions One such notion of simple

distributions underlies the following denition while a more lib eral notion will b e

presented in x

Denition the class dist NP We say that a probability ensemble X

fX g is simple if there exists a polynomial time algorithm that on any input

n

nN

x f g outputs Pr X x where the inequality refers to the standard lexico

jxj

graphic order of strings We denote by dist NP the class of distributional problems

consisting of decision problems in NP coupled with simple probability ensembles

In contrast testing whether a given graph is colorable seems typically hard for other dis

tributions see either Theorem or Exercise Needless to say in the latter distributions

b oth yesinstances and noinstances app ear with noticeable probability

Recall that a function N N is negligible if for every p ositive p olynomial q and all

suciently large n it holds that n q n We say that A errs on x if Ax diers from the

indicator value of the predicate x S

Conning attention to the uniform distribution seems misguided by the naive b elief according

to which this distribution is the only one relevant to applications In contrast we b elieve that

for most natural applications the uniform distribution over instances is not relevant at all

CHAPTER RELAXING THE REQUIREMENTS

Note that the uniform probability ensemble is simple but so are many other sim

ple probability ensembles Actually it makes sense to relax the denition such

that the algorithm is only required to output an approximation of Pr X x say

jxj

jxj

to within a factor of We note that Denition interprets simplicity

in computational terms sp ecically as the feasibility of answering very basic ques

tions regarding the probability distribution ie determining the probability mass

assigned to a single nbit long string and even to an interval of such strings This

simplicity condition is closely related to b eing p olynomialtime sampleable via a

monotone mapping see Exercise

Teaching note The following two paragraphs attempt to address some doubts re

garding Denition One may p ostp one such discussions to a later stage

We admit that the identication of simple distributions as the class of inter

esting distribution is signicantly more questionable than any other identication

advocated in this b o ok Nevertheless we b elieve that we were fully justied in re

jecting b oth the aforementioned extremes ie of either allowing all distributions

or allowing only the uniform distribution Yet the reader may wonder whether

or not we have struck the right balance b etween generality and simplicity in

the intuitive sense One sp ecic concern is that we might have restricted the class

of distributions to o much We briey address this concern next

A more intuitive and very robust class of distributions which seems to contain

all distributions that may o ccur in applications is the class of p olynomialtime

sampleable probability ensembles treated in x Fortunately the combi

nation of the results presented in x and x seems to retrosp ectively

endorse the choice underlying Denition Sp ecically we note that enlarging

the class of distributions weakens the conjecture that the corresp onding class of

distributional NP problems contains infeasible problems On the other hand the

conclusion that a sp ecic distributional problem is not feasible b ecomes more ap

p ealing when the problem b elongs to a smaller class that corresp onds to a restricted

denition of admissible distributions Now the combined results of x and

x assert that a conjecture that refers to the larger class of p olynomialtime

sampleable ensembles implies a conclusion that refers to a very simple probability

ensemble which resides in the smaller class Thus the current setting in which

b oth the conjecture and the conclusion refer to simple probability ensembles may

b e viewed as just an intermediate step

Indeed the big question in the current context is whether distNP is contained

in tp c P A p ositive answer esp ecially if extended to sampleable ensembles would

deem the PvsNP Question to b e of little practical signicant However our daily

exp erience as well as much research eort indicate that some NP problems are

not merely hard in the worstcase but rather typically hard This leads to the

conjecture that distNP is not contained in tp cP

Needless to say the latter conjecture implies P NP and thus we should

not exp ect to see a pro of of it In particular we should not exp ect to see a pro of

that some sp ecic problem in distNP is not in tp cP What we may hop e to see

is distNP complete problems that is problems in distNP that are not in tp c P

AVERAGE CASE COMPLEXITY

unless the entire class distNP is contained in tp cP An adequate notion of a

reduction is used towards formulating this p ossibility

Step Dening reductions among distributional problems Intuitively

such reductions must preserve averagecase feasibility Thus in addition to the

standard conditions ie that the reduction b e eciently computable and yield a

correct result we require that the reduction resp ects the probability distribu

tion of the corresp onding distributional problems Sp ecically the reduction should

not map very likely instances of the rst starting problem to rare instances of

the second target problem Otherwise having a typically p olynomialtime al

gorithm for the second distributional problem do es not necessarily yield such an

algorithm for the rst distributional problem Following is the adequate analogue

of a Co ok reduction ie general p olynomialtime reduction where the analogue

of a Karpreduction manytoone reduction can b e easily derived as a sp ecial case

Teaching note One may prefer presenting in class only the sp ecial case of manyto

one reductions which suces for Theorem See Footnote

Denition reductions among distributional problems We say that the

oracle machine M reduces the distributional problem S X to the distributional

problem T Y if the fol lowing three conditions hold

Eciency The machine M runs in polynomialtime

T

Validity For every x f g it holds that M x if an only if x S

T

where M x denotes the output of the oracle machine M on input x and

access to an oracle for T

Domination The probability that on input X and oracle access to T

n

machine M makes the query y is upperbounded by p oly jy j Pr Y y

jy j

That is there exists a polynomial p such that for every y f g and every

n N it holds that

Pr QX y pjy j Pr Y y

n

jy j

where Qx denotes the set of queries made by M on input x and oracle access

to T

In addition we require that the reduction does not make too short queries

that is there exists a polynomial p such that if y Qx then p jy j jxj

In fact one may relax the requirement and only require that M is typically p olynomialtime

with resp ect to X The validity condition may also b e relaxed similarly

Let us sp ell out the meaning of Eq in the sp ecial case of manytoone reductions

T

ie M x if and only if f x T where f is a p olynomialtime computable function

in this case Pr QX y is replaced by Pr f X y That is Eq simplies to

n n

Pr f X y pjy j Pr Y y Indeed this condition holds vacuously for any y that is not

n

jy j

in the image of f

CHAPTER RELAXING THE REQUIREMENTS

The lhs of Eq refers to the probability that on input distributed as X

n

the reduction makes the query y This probability is required not to exceed the

probability that y o ccurs in the distribution Y by more than a p olynomial factor

jy j

in jy j In this case we say that the lhs of Eq is dominated by Pr Y y

jy j

Indeed the domination condition is the only asp ect of Denition that ex

tends b eyond the worstcase treatment of reductions and refers to the distributional

setting The domination condition do es not insist that the distribution induced by

QX equals Y but rather allows some slackness that in turn is b ounded so to

guarantee preservation of typical feasibility see Exercise

We note that the reducibility arguments extensively used in Chapters and

see discussion in Section are actually reductions in the spirit of Deni

tion except that they refer to dierent types of computational tasks

Complete problems

Recall that our conjecture is that distNP is not contained in tp cP which in turn

strengthens the conjecture P NP making infeasibility a typical phenomenon

rather than a worstcase one Having no hop e of proving that dist NP is not

contained in tp c P we turn to the study of complete problems with resp ect to that

conjecture Sp ecically we say that a distributional problem S X is distNP

complete if S X dist NP and every S X distNP is reducible to S X

under Denition

Recall that it is quite easy to prove the mere existence of NPcomplete problems

and that many natural problems are NPcomplete In contrast in the current con

text establishing completeness results is quite hard This should not b e surprising

in light of the restricted type of reductions allowed in the current context The re

striction captured by the domination condition requires that typical instances

of one problem should not b e mapp ed to untypical instances of the other prob

lem However it is fair to say that standard Karpreductions used in establishing

NPcompleteness results map typical instances of one problem to somewhat

bizarre instances of the second problem Thus the current subsection may b e

viewed as a study of reductions that do not commit this sin

Theorem dist NP completeness distNP contains a distributional prob

lem T Y such that each distributional problem in distNP is reducible p er Deni

tion to T Y Furthermore the reductions are via manytoone mappings

Pro of We start by introducing such a distributional problem which is a

natural distributional version of the decision problem S used in the pro of of

u

We stress that the notion of domination is incomparable to the notion of statistical resp

computational indistinguishability On one hand domination is a lo cal requirement ie it

compares the two distribution on a p ointbypoint basis whereas indistinguishability is a global

requirement which allows rare exceptions On the other hand domination do es not require

approximately equal values but rather a ratio that is b ounded in one direction Indeed domina

tion is not symmetric We comment that a more relaxed notion of domination that allows rare

violations as in Footnote suces for the preservation of typical feasibility

The latter assertion is somewhat controversial While it seems totally justied with resp ect

to the pro of of Theorem opinions regarding the pro of of Theorem may dier

AVERAGE CASE COMPLEXITY

t

Theorem Recall that S contains the instance hM x i if there exists

u

i

y f g such that machine M accepts the input pair x y within t steps

it

We couple S with the quasiuniform probability ensemble U that assigns to

u

t jM jjxj

the instance hM x i a probability mass prop ortional to Sp ecically

t

for every hM x i it holds that

jM jjxj

t

Pr U hM x i

n

n

def def

t

where n jhM x ij jM j jxj t Note that under a suitable natural

enco ding the ensemble U is indeed simple

The reader can easily verify that the generic reduction used when reducing

any set in NP to S see the pro of of Theorem fails to reduce distNP

u

to S U Sp ecically in some cases see next paragraph these reductions do

u

not satisfy the domination condition Indeed the diculty is that we have to

reduce all distNP problems ie pairs consisting of decision problems and simple

distributions to one single distributional problem ie S U In contrast

u

considering the distributions induced by the aforementioned reductions we end

up with many distributional versions of S and furthermore the corresp onding

u

distributions are very dierent and are not necessarily dominated by a single

distribution

Let us take a closer lo ok at the aforementioned generic reduction of S to S

u

when applied to an arbitrary S X distNP This reduction maps an instance

p jxj

S

x to a triple M x where M is a machine verifying membership in

S S

S while using adequate NPwitnesses and p is an adequate p olynomial The

S

problem is that x may have relatively large probability mass ie it may b e that

jxj p jxj

S

Pr X x while M x has uniform probability mass ie

S

jxj

p jxj jxj

S

hM x i has probability mass smaller than in U This violates the

S

domination condition see Exercise and thus an alternative reduction is

required

The key to the alternative reduction is an eciently computable enco ding of

strings taken from an arbitrary simple distribution by strings that have a similar

probability mass under the uniform distribution This means that the enco ding

should shrink strings that have relatively large probability mass under the origi

nal distribution Sp ecically this enco ding will map x taken from the ensemble

fX g to a co deword x of length that is upp erb ounded by the logarithm of

n

nN

jx j

Pr X x ensuring that Pr X x O Accordingly the reduction

jxj jxj

p jxj

will map x to a triple M x where jx j O log Pr X x

SX

jxj

and M is an algorithm that given x and x rst veries that x is a prop er

SX

enco ding of x and next applies the standard verication ie M of the problem

S

S Such a reduction will b e shown to satisfy all three conditions ie eciency

t k

For example we may enco de hM x i where M f g and x

k

n

t t

Pr U hM x i equals f g by the string Then

k k

n

jM j jM j jM jjxj jxj

i jfM f g M M gj jfx f g x xgj

jM jjxjt

where i is the ranking of fk k g among all subsets of k t k t

CHAPTER RELAXING THE REQUIREMENTS

validity and domination Thus instead of forcing the structure of the original

distribution X on the target distribution U the reduction will incorp orate the

structure of X in the reduced instance A key ingredient in making this p ossible is

the fact that X is simple as p er Denition

With the foregoing motivation in mind we now turn to the actual pro of that

is proving that any S X distNP is reducible to S U The following

u

technical lemma is the basis of the reduction In this lemma as well as in the

sequel it will b e convenient to consider the accumulative distribution function

def

of the probability ensemble X That is we consider x Pr X x and

jxj

note that f g is p olynomialtime computable b ecause X satises

Denition

Co ding Lemma Let f g b e a p olynomialtime computable function

n

that is monotonically nondecreasing over f g for every n ie x x

jx j n n

for any x x f g For x f g n f g let x denote the string

preceding x in the lexicographic order of nbit long strings Then there exist an

enco ding function C that satises the following three conditions

Compression For every x it holds that jC xj minfjxj log x g

def def

n n

where x x x if x fg and otherwise

Ecient Enco ding The function C is computable in p olynomialtime

n

Unique Deco ding For every n N when restricted to f g the function

C is onetoone ie if C x C x and jxj jx j then x x

jxj

Pro of The function C is dened as follows If x then C x x

jxj

ie in this case x serves as its own enco ding Otherwise ie x

then C x z where z is chosen such that jz j log x and the mapping

of nbit strings to their enco ding is onetoone Lo osely sp eaking z is selected to

equal the shortest binary expansion of a number in the interval x x x

Bearing in mind that this interval has length x and that the dierent intervals

are disjoint we obtain the desired enco ding Details follows

jxj

x and detail the way that z is selected We fo cus on the case that

jxj

for the enco ding C x z If x and x then we let z b e the

longest common prex of the binary expansions of x and x for example

if and then C z with z

Thus in this case z is in the interval x x ie x z x

jxj

For x we let z b e the longest common prex of the binary expansions of

and x and again z is in the relevant interval ie x Finally for x such

that x and x we let z b e the longest common prex of the binary

jxj

expansions of x and and again z is in x x b ecause

n

The lemma actually refers to f g for any xed value of n but the eciency condition

is stated more easily when allowing n to vary and using the standard asymptotic analysis of

algorithms Actually the lemma is somewhat easier to state and establish for p olynomial

time computable functions that are monotonically nondecreasing over f g rather than over

n

f g See further discussion in Exercise

AVERAGE CASE COMPLEXITY

jxj jxj

x and x x imply that x x

jxj

Note that if x x then x

We now verify that the foregoing C satises the conditions of the lemma We

jxj

start with the compression condition Clearly if x then jC xj

jxj

jxj log x On the other hand supp ose that x and

jxj

let us fo cus on the subcase that x and x Let z z z b e

the longest common prex of the binary expansions of x and x Then

x z u and x z v where u v f g We infer that

p oly jxj

X X X

i jz j i i

A

z x x x z

i i

i i

i

and jz j log x jxj follows Thus jC xj minjxj log x

holds in b oth cases Clearly C can b e computed in p olynomialtime by computing

x and x Finally note that C satises the unique deco ding condition by

separately considering the two aforementioned cases ie C x x and C x

z Sp ecically in the second case ie C x z use the fact that x

z x

In order to obtain an enco ding that is onetoone when applied to strings of

dierent lengths we augment C in the obvious manner that is we consider

def

x C x x jxj C x which may b e implemented as C C

xj O log jxj where is the binary expansion of jxj Note that jC

jC xj and that C is onetoone over f g

The machine asso ciated with S X Let b e the accumulative probability func

tion asso ciated with the probability ensemble X and M b e the p olynomialtime

S

machine that veries membership in S while using adequate NPwitnesses ie

p olyjxj

x S if and only if there exists y f g such that M x y Using

the enco ding function C we introduce an algorithm M with the intension of

S

reducing the distributional problem S X to S U such that all instances of

u

S are mapp ed to triples in which the rst element equals M Machine M

S S

is given an alleged enco ding under C of an instance to S along with an alleged

pro of that the corresp onding instance is in S and veries these claims in the ob

vious manner That is on input x and hx y i machine M rst veries that

S

x C x and next veriers that x S by running M x y Thus M veries

S S

membership in the set S fC x x S g while using pro ofs of the form hx y i

such that M x y for the instance C x

S

pjxj

The reduction We maps an instance x of S to the triple M C x

S

def

where pn p n p n such that p is a p olynomial representing the running

S C S

time of M and p is a p olynomial representing the runningtime of the enco ding

S C

algorithm

Note that jy j p oly jxj but jxj p oly jC xj do es not necessarily hold and so S is not

necessarily in NP As we shall see the latter p oint is immaterial

CHAPTER RELAXING THE REQUIREMENTS

Analyzing the reduction Our goal is proving that the foregoing mapping constitutes

a reduction of S X to S U We verify the corresp onding three requirements

u

of Denition

Using the fact that C is p olynomialtime computable and noting that p

is a p olynomial it follows that the foregoing mapping can b e computed in

p olynomialtime

Recall that on input x hx y i machine M accepts if and only if x

S

C x and M accepts x y within p jxj steps Using the fact that C x

S S

uniquely determines x it follows that x S if and only if C x S

which in turn holds if and only if there exists a string y such that M

S

accepts C x hx y i in at most pjxj steps Thus x S if and only if

pjxj

M C x S and the validity condition follows

S u

In order to verify the domination condition we rst note that the foregoing

mapping is onetoone b ecause the transformation x C x is oneto

one Next we note that it suces to consider instances of S that have

u

a preimage under the foregoing mapping since instances with no preimage

trivially satisfy the domination condition Each of these instances ie each

image of this mapping is a triple with the rst element equal to M and

S

the second element b eing an enco ding under C By the denition of U for

pjxj n

every such image hM C x i f g it holds that

S

n

pjxj jM jjC xj

S

Pr U hM C x i

S

n

jC xjO log jxj

c n

jM j

S

where c is a constant dep ending only on S and ie on the

distributional problem S X Thus for some p ositive p olynomial q we

have

pjxj jC xj

Pr U hM C x i q n

S

n

By virtue of the compression condition of the Co ding Lemma we have

jC xj minjxjlog x

It follows that

jC xj

Pr X x

jxj

pjxj

Recalling that x is the only preimage that is mapp ed to hM C x i

S

and combining Eq we establish the domination condition

The theorem follows

Reections The pro of of Theorem highlights the fact that the reduction

used in the pro of of Theorem do es not introduce much structure in the reduced

instances ie do es not reduce the original problem to a highly structured sp ecial

AVERAGE CASE COMPLEXITY

case of the target problem Put in other words unlike more advanced worstcase

reductions this reduction do es not map random ie uniformly distributed

instances to highly structured instances which o ccur with negligible probability

under the uniform distribution Thus the reduction used in the pro of of The

orem suces for reducing any distributional problem in distNP to a distri

butional problem consisting of S coupled with some simple probability ensemble

u

see Exercise

However Theorem states more than the latter assertion That is it states

that any distributional problem in distNP is reducible to the same distributional

version of S Indeed the eort involved in proving Theorem was due to

u

the need for mapping instances taken from any simple probability ensemble which

may not b e the uniform ensemble to instances distributed in a manner that is

dominated by a single probability ensemble ie the quasiuniform ensemble U

Once we have established the existence of one distNP complete problem we

may establish the distNP completeness of other problems in distNP by reduc

ing some dist NP complete problem to them and relying on the transitivity of

reductions see Exercise Thus the diculties encountered in the pro of of

Theorem are no longer relevant Unfortunately a seemingly more severe dif

culty arises almost all known reductions in the theory of NPcompleteness work

by introducing much structure in the reduced instances ie they actually reduce

to highly structured sp ecial cases Furthermore this structure is to o complex in

the sense that the distribution of reduced instances do es not seem simple in the

sense of Denition Actually as demonstrated next the problem is not

the existence of a structure in the reduced instances but rather the complexity of

this structure In particular if the aforementioned reduction is monotone and

length regular then the distribution of the reduced instances is simple enough

ie is simple in the sense of Denition

Prop osition sucient condition for distNP completeness Suppose that

f is a Karpreduction of the set S to the set T such that for every x x f g

the fol lowing two conditions hold

f is monotone If x x then f x f x where the inequalities refer

to the standard lexicographic order of strings

f is lengthregular jx j jx j if and only if jf x j jf x j

Then if there exists an ensemble X such that S X is distNP complete then there

exists an ensemble Y such that T Y is distNP complete

Pro of Sketch Note that the monotonicity of f implies that f is onetoone

and that for every x it holds that f x x Furthermore as shown next f

is p olynomialtime invertible Intuitively the fact that f is b oth monotone and

Note that this cannot b e said of most known Karpreductions which do map random instances

to highly structured ones Furthermore the same structure creating prop erty holds for the

reductions obtained by Exercise

In particular if jz j jz j then z z Recall that for jz j jz j it holds that z z if

and only if there exists w u u f g such that z w u and z w u

CHAPTER RELAXING THE REQUIREMENTS

p olynomialtime computable implies that a preimage can b e found by a binary

search Sp ecically given y f x we search for x by iteratively halving the

interval of p otential solutions which is initialized to y since x f x Note

that if this search is invoked on a string y that is not in the image of f then it

terminates while detecting this fact

Relying on the fact that f is onetoone and lengthregular we dene the

probability ensemble Y fY g such that for every x it holds that Pr Y

n n

jf xj

m

f x Pr X x Sp ecically letting m jf j and noting that is

jxj

onetoone and monotonically nondecreasing we dene

Pr X x if x f y

jxj

m m

if m st y f g n ff x x f g g

Pr Y y

jy j

jy j

otherwise ie if jy j fm m N g

Clearly S X is reducible to T Y via the Karpreduction f which due to

our construction of Y also satises the domination condition Thus using the

hypothesis that dist NP is reducible to S X and the transitivity of reductions see

Exercise it follows that every problem in distNP is reducible to T Y The

key observation to b e established next is that Y is a simple probability ensemble

and it follows that T Y is in dist NP

Lo osely sp eaking the simplicity of Y follows by combining the simplicity of

X and the prop erties of f ie the fact that f is monotone lengthregular and

p olynomialtime invertible The monotonicity and lengthregularity of f implies

m

that Pr Y f x Pr X x More generally for any y f g it holds

jf xj jxj

that Pr Y y Pr X x where x is the lexicographicly largest string such

m

m

that f x y and indeed if jxj m then Pr Y y Pr X x Note

m

m

that this x can b e found in p olynomialtime by the inverting algorithm sketched in

the rst paragraph of the pro of Thus we may compute Pr Y y by nding the

jy j

adequate x and computing Pr X x Using the hypothesis that X is simple it

jxj

follows that Y is simple and the prop osition follows

On the existence of adequate Karpreductions Prop osition implies

that a sucient condition for the distNP completeness of a distributional version

of a NPcomplete set T is the existence of an adequate Karpreduction from the

set S to the set T that is this Karpreduction should b e monotone and length

u

regular While the lengthregularity condition seems easy to imp ose by using

adequate padding the monotonicity condition seems more problematic Fortu

nately it turns out that the monotonicity condition can also b e imp osed by using

adequate padding or rather an adequate marking see Exercises and

We highlight the fact that the existence of an adequate padding or marking is

a prop erty of the set T itself In Exercise we review a metho d for mo difying

any Karpreduction to a monotonically markable set T into a Karpreduction to

Having Y b e uniform in this case is a rather arbitrary choice which is merely aimed at

n

guaranteeing a simple distribution on nbit strings also in this case

We also note that the case in which jy j is not in the image of can b e easily detected and

taken care o accordingly

AVERAGE CASE COMPLEXITY

T that is monotone and lengthregular In Exercise we provide evidence to

the thesis that all natural NPcomplete sets are monotonically markable Combin

ing all these facts we conclude that any natural NPcomplete decision problem can

be coupled with a simple probability ensemble such that the resulting distributional

problem is distNP complete As a concrete illustration of this thesis we state the

corresp onding formal result for the twentyone NPcomplete problems treated in

Karps pap er on NPcompleteness

Theorem a mo dest version of a general thesis For each of the twenty

one NPcomplete problems treated in there exists a simple probability ensemble

such that the combined distributional problem is distNP complete

The said list of problems includes SAT Clique and Colorability

Probabilistic versions

The denitions in x can b e extended so that to account also for randomized

computations For example extending Denition we have

Denition the class tp c BPP For a probabilistic algorithm A a Boolean

function f and a timebound function t N N we say that the string x is tbad for

A with resp ect to f if with probability exceeding on input x either Ax f x

or A runs more that tjxj steps We say that A typically solves S fX g in

n

nN

probabilistic p olynomialtime if there exists a polynomial p such that the probability

that X is pbad for A with respect to the characteristic function of S is negligible

n

We denote by tp c BPP the class of distributional problems that are typically solvable

in probabilistic polynomialtime

The denition of reductions can b e similarly extended This means that in Deni

T

tion b oth M x and Qx mentioned in Items and resp ectively are

random variables rather than xed ob jects Furthermore validity is required to

hold for every input only with probability where the probability space refers

only to the internal coin tosses of the reduction Randomized reductions are closed

under comp osition and preserve typical feasibility see Exercise

Randomized reductions allow the presentation of a dist NP complete problem

that refers to the p erfectly uniform ensemble Recall that Theorem estab

lishes the distNP completeness of S U where U is a quasiuniform ensemble

u

n

t jM jjxj t

ie Pr U hM x i where n jhM x ij We rst

n

fhM x z i U where S note that S U can b e randomly reduced to S

u

u u

n

jz j jM jjxjjz j

hM x i S g and Pr U hM x z i for every hM x z i

u

n

n t

f g The randomized reduction consists of mapping hM x i to hM x z i

t

where z is uniformly selected in f g Recalling that U fU g denotes the

n

nN

uniform probability ensemble ie U is uniformly distributed on strings of length

n

n and using a suitable enco ding we get

Prop osition There exists S NP such that every S X distNP is

randomly reducible to S U

CHAPTER RELAXING THE REQUIREMENTS

Pro of Sketch By the forgoing discussion every S X distNP is randomly

reducible to S U where the reduction go es through S U Thus we fo cus

u

u

on reducing S U to S U where S NP is dened as follows The string

u u u

bin jujbin jv juvw is in S if and only if hu v w i S and dlog juv w je

u u

where bin i denotes the bit long binary enco ding of the integer i ie

the enco ding is padded with zeros to a total length of The reduction maps

hM x z i to the string bin jxjbin jM jMxz where dlog jM j jxj jz je

Noting that this reduction satises all conditions of Denition the prop osi

tion follows

Ramications

In our opinion the most problematic asp ect of the theory describ ed in Section

is the choice to fo cus on simple probability ensembles which in turn restricts dis

tributional versions of NP to the class distNP Denition As indicated

x this restriction raises two opp osite concerns ie that dist NP is either

to o wide or to o narrow Here we address the concern that the class of sim

ple probability ensembles is to o restricted and consequently that the conjecture

distNP tp cBPP is to o strong which would mean that distNP completeness is

a weak evidence for typicalcase hardness An app ealing extension of the class of

simple probability ensembles is presented in x yielding an corresp onding

extension of dist NP and it is shown that if this extension of dist NP is not con

tained in tp cBPP then dist NP itself is not contained in tp c BPP Consequently

distNP complete problems enjoy the b enet of b oth b eing in the more restricted

class ie dist NP and b eing hard as long as some problems in the extended class

is hard

Another extension app ears in x where we extend the treatment from

decision problems to search problems This extension is motivated by the realiza

tion that search problem are actually of greater imp ortance to reallife applications

cf Section and hence a theory motivated by reallife applications must

address such problems as we do next

Prerequisites For the technical development of x we assume familiar

ity with the notion of unique solution and results regarding it as presented in

Section For the technical development of x we assume familiarity

with hashing functions as presented in App endix D In addition the technical

development of x relies on x

Search versus Decision

Indeed as in the case of worstcase complexity search problems are at least as im

p ortant as decision problems Thus an averagecase treatment of search problems

On one hand if the denition of distNP were to o lib eral then membership in distNP would

mean less than one may desire On the other hand if distNP were to o restricted then the

conjecture that distNP contains hard problems would have b een very questionable

AVERAGE CASE COMPLEXITY

is indeed called for We rst present distributional versions of PF and PC cf

Section following the underlying principles of the denitions of tp c P and

distNP

Denition the classes tp c PF and distPC As in Section we con

sider only polynomially bounded search problems that is binary relations R

f g f g such that for some polynomial q it holds that x y R implies

def def

fx R x g jy j q jxj Recall that R x fy x y R g and S

R

A distributional search problem consists of a polynomially bounded search prob

lem coupled with a probability ensemble

The class tp c PF consists of al l distributional search problems that are typ

ically solvable in polynomialtime That is R fX g tp c PF if there

n

nN

exists an algorithm A and a polynomial p such that the probability that on

input X algorithm A either errs or runs more that pn steps is negligible

n

where A errs on x S if Ax R x and errs on x S if Ax

R R

A distributional search problem R X is in distPC if R PC and X is

simple as in Denition

Likewise the class tp cBPPF consists of all distributional search problems that

are typically solvable in probabilistic p olynomialtime cf Denition The

denitions of reductions among distributional problems presented in the context of

decision problem extend to search problems

Fortunately as in the context of worstcase complexity the study of distribu

tional search problems reduces to the study of distributional decision problems

Theorem reducing search to decision distPC tp cBPPF if and only if

distNP tp c BPP Furthermore every problem in distNP is reducible to some

problem in distPC and every problem in distPC is randomly reducible to some

problem in distNP

Pro of Sketch The furthermore part is analogous to the actual contents of the

pro of of Theorem see also Step in the pro of of Theorem Indeed the

reduction of NP to PC presented in the pro of of Theorem extends to the current

context Sp ecically for any S NP we consider a relation R PC such that

S fx R x g and note that for any probability ensemble X the identity

transformation reduces S X to R X

A diculty arises in the opp osite direction Recall that in the pro of of The

orem we reduced the search problem of R PC to deciding membership in

def

S fhx y i y st x y y R g NP The diculty encountered here is

R

that on input x this reduction makes queries of the form hx y i where y is a

prex of some string in R x These queries may induce a distribution that is not

dominated by any simple distribution Thus we seek an alternative reduction

As a warmup let us assume for a moment that R has unique solutions in the

sense of Denition that is for every x it holds that jR xj In this case

CHAPTER RELAXING THE REQUIREMENTS

we may easily reduce the search problem of R PC to deciding membership in

S NP where hx i i S if and only if R x contains a string in which the

R R

th

i bit equals Sp ecically on input x the reduction issues the queries hx i i

where i with p oly jxj and f g which allows for determining the

single string in the set R x f g whenever jR xj The p oint is that this

reduction can b e used to reduce any R X distPC having unique solutions to

X distNP where X equally distributes the probability mass of x under S

R

X to al l the tuples hx i i that is for every i and f g it holds that

Pr X hx i i equals Pr X x

jxj

jhxi ij

Unfortunately in the general case R may not have unique solutions Nev

ertheless applying the main idea that underlies the pro of of Theorem this

diculty can b e overcome We rst note that the foregoing mapping of instances

of the distributional problem R X distPC to instances of S X distNP

R

satises the eciency and domination conditions even in the case that R do es not

have unique solutions What may p ossibly fail in the general case is the validity

condition ie if jR xj then we may fail to recover any element of R x

Recall that the main part of the pro of of Theorem is a randomized reduction

that maps instances of R to triples of the form x m h such that m is uniformly

distributed in and h is uniformly distributed in a family of hashing function

m m

H where p oly jxj and H is as in App endix D Furthermore if R x

m

there exists then with probability over the choices of m and h H

def

m

a unique y R x such that hy Dening R x m h fy R x

m

hy g this yields a randomized reduction of the search problem of R to the

search problem of R such that with noticeable probability the reduction maps

instances that have solutions to instances having a unique solution Furthermore

this reduction can b e used to reduce any R X distPC to R X distPC

where X distributes the probability mass of x under X to al l the triples x m h

m

such that for every m and h H it holds that Pr X x m h

jxmhj

m

equals Pr X x jH j Note that with a suitable enco ding X is indeed

jxj

simple

The theorem follows by combining the two aforementioned reductions That is

we rst apply the randomized reduction of R X to R X and next reduce the

resulting instance to an instance of the corresp onding decision problem S X

R

where X is obtained by mo difying X rather than X The combined randomized

mapping satises the eciency and domination conditions and is valid with notice

able probability The error probability can b e made negligible by straightforward

amplication see Exercise

Simple versus sampleable distributions

Recall that the denition of simple probability ensembles underlying Denition

requires that the accumulating distribution function is p olynomialtime computable

Recall that the probability of an event is said to b e noticeable in a relevant parameter if it is

greater than the recipro cal of some p ositive p olynomial In the context of randomized reductions

the relevant parameter is the length of the input to the reduction

AVERAGE CASE COMPLEXITY

Recall that f g is called the accumulating distribution function of

def

n

X fX g if for every n N and x f g it holds that x Pr X x

n n

nN

where the inequality refers to the standard lexicographic order of nbit strings

As argued in x the requirement that the accumulating distribution func

tion is p olynomialtime computable imp oses severe restrictions on the set of ad

missible ensembles Furthermore it seems that these simple ensembles are indeed

simple in some intuitive sense and that they represent a reasonable alas dis

putable mo del of distributions that may o ccur in practice Still in light of the fear

that this mo del is to o restrictive and consequently that distNP hardness is weak

evidence for typicalcase hardness we seek a maximalistic mo del of distributions

that may o ccur in practice Such a mo del is provided by the notion of p olynomial

time sampleable ensembles underlying Denition Our maximality thesis

is based on the b elief that the real world should b e mo deled as a feasible ran

domized pro cess rather than as an arbitrary pro cess This b elief implies that all

ob jects encountered in the world may b e viewed as samples generated by a feasible

randomized pro cess

Denition sampleable ensembles and the class sampNP We say that a

probability ensemble X fX g is p olynomialtime sampleable if there exists

n

nN

a probabilistic polynomialtime algorithm A such that for every x f g it holds

jxj

that Pr A x Pr X x We denote by sampNP the class of distri

jxj

butional problems consisting of decision problems in NP coupled with sampleable

probability ensembles

We rst note that all simple probability ensembles are indeed sampleable see

Exercise and thus dist NP sampNP On the other hand there exist

sampleable probability ensembles that do not seem simple see Exercise

Extending the scop e of distributional problems from distNP to sampNP fa

cilitates the presentation of complete distributional problems We rst note that

it is easy to prove that every natural NPcomplete problem has a distributional

version in sampNP that is dist NP hard see Exercise Furthermore it is

p ossible to prove that all natural NPcomplete problem have distributional versions

that are sampNP complete In b oth cases natural means that the corresp ond

ing Karpreductions do not shrink the input which is a weaker condition than the

one in Prop osition

Theorem sampNP completeness Suppose that S NP and that every

set in NP is reducible to S by a Karpreduction that does not shrink the input

Then there exists a polynomialtime sampleable ensemble X such that any problem

in sampNP is reducible to S X

The pro of of Theorem is based on the observation that there exists a polynomial

time sampleable ensemble that dominates al l polynomialtime sampleable ensembles

The existence of this ensemble is based on the notion of a universal sampling ma

chine For further details see Exercise

Theorem establishes a rich theory of sampNP completeness but do es not

relate this theory to the previously presented theory of distNP completeness see

CHAPTER RELAXING THE REQUIREMENTS

tpcBPP

sampNP

distNP sampNP-complete [Thm 10.25]

distNP-complete [Thm 10.17 and 10.19]

Figure Two types of averagecase completeness

Figure This is essentially done in the next theorem which asserts that the

existence of typically hard problems in sampNP implies their existence in distNP

Theorem sampNP completeness versus distNP completeness If sampNP

is not contained in tp cBPP then distNP is not contained in tp c BPP

Thus the two typicalcase complexity versions of the PvsNP Question are

equivalent That is if some sampleable distribution versions of NP are not

typically feasible then some simple distribution versions of NP are not typically

feasible In particular if sampNP complete problems are not in tp cBPP then

distNP complete problems are not in tp c BPP

The foregoing assertions would all follow if sampNP were randomly reducible

to distNP ie if every problem in sampNP were reducible under a randomized

version of Denition to some problem in distNP but unfortunately we

do not know whether such reductions exist Yet underlying the pro of of Theo

rem is a more lib eral notion of a reduction among distributional problems

Pro of Sketch We shall prove that if distNP is contained in tp cBPP then the

same holds for sampNP ie sampNP is contained in tp cBPP Relying on

Theorem and Exercise it suces to show that if distPC is contained in

tp c BPPF then the sampleable version of distPC denoted sampPC is contained

in tp c BPPF This will b e shown by showing that under a relaxed notion of a

randomized reduction every problem in sampPC is reduced to some problem in

distPC Lo osely sp eaking this relaxed notion of a randomized reduction only

requires that the validity and domination conditions of Denition when

adapted to randomized reductions hold with resp ect to a noticeable fraction of

the probability space of the reduction We start by formulating this notion when

referring to distributional search problems

We warn that the existence of such a relaxed reduction b etween two sp ecic distributional

problems do es not necessarily imply the existence of a corresp onding standard averagecase

reduction Sp ecically although standard validity can b e guaranteed for problems in PC by

AVERAGE CASE COMPLEXITY

Teaching note The following pro of is quite involved and is b etter left for advanced

reading Its main idea is related to one of the central ideas underlying the currently

known pro of of Theorem This fact as well as numerous other applications of this

idea provide additional motivation for reading the following pro of

Denition A relaxed reduction of the distributional problem R X to the distri

butional problem T Y is a probabilistic p olynomialtime oracle machine M that

mjxj

satises the following conditions with resp ect to a family of sets f f g

x

x f g g where mjxj p oly jxj denotes an upp erb ound on the number of

the internal coin tosses of M on input x

Density of There exists a noticeable function N ie n

x

mjxj

p olyn such that for every x f g it holds that j j jxj

x

Validity with resp ect to For every r the reduction yields a correct an

x x

T T

swer that is M x r R x if R x and M x r otherwise

T

where M x r denotes the execution of M on input x internal coins r and

oracle access to T

Domination with resp ect to There exists a p ositive p olynomial p such that

x

for every y f g and every n N it holds that

Pr Q X y pjy j Pr Y y

n

jy j

where Q x is a random variable dened over the set representing the

x

set of queries made by M on input x coins in and oracle access to T

x

That is Q x is dened by uniformly selecting r and considering the

x

set of queries made by M on input x internal coins r and oracle access to T

In addition as in Denition we also require that the reduction do es

not make to o short queries

The reader may verify that this relaxed notion of a reduction preserves typical

feasibility that is for R PC if there exists a relaxed reduction of R X to

T Y and T Y is in tp cBPPF then R X is in tp c BPPF The key observation

is that the analysis may discard the case that on input x the reduction selects

coins not in Indeed the queries made in that case may b e untypical and the

x

answers received may b e wrong but this is immaterial What matter is that on

input x with noticeable probability the reduction selects coins in and pro duces

x

typical with resp ect to Y queries by virtue of the relaxed domination condition

Such typical queries are answered correctly by the algorithm that typically solves

T Y and if x has a solution then these answers yield a correct solution to x

by virtue of the relaxed validity condition Thus if x has a solution then with

noticeable probability the reduction outputs a correct solution On the other hand

the reduction never outputs a wrong solution even when using coins not in

x

b ecause incorrect solutions are detected by relying on R PC

rep eated invocations of the reduction such a pro cess will not redeem the violation of the standard

domination condition

CHAPTER RELAXING THE REQUIREMENTS

Our goal is presenting for every R X sampPC a relaxed reduction of

R X to a related problem R X dist PC We use the standard notation

X fX g and X fX g

n

nN nN

n

An oversimplied case For starters suppose that X is uniformly distributed on

n

n

some set S f g and that there is a polynomialtime computable and invert

n

n

ible mapping of S to f g where n log jS j Then mapping x to

n n

jxjjxj

x we obtain a reduction of R X to R X where X is uniform

n

nn n nn

over f v v f g g and R v R v or equivalently

jxjjxj

R x R x Note that X is a simple ensemble and R PC

hence R X distPC Also note that the foregoing mapping is indeed a valid

reduction ie it satises the eciency validity and domination conditions Thus

R X is reduced to a problem in distPC and indeed the relaxation was not used

here

A simple but more instructive case Next we drop the assumption that there is

n

a p olynomialtime computable and invertible mapping of S to f g but

n

n

maintain the assumption that X is uniform on some set S f g and as

n n

n

sume that jS j is easily computable from n In this case we may map

n

n

x f g to its image under a suitable randomly chosen hashing function h which

in particular maps nbit strings to nbit strings That is we randomly map x to

n

nn

h hx where h is uniformly selected in a set H of suitable hash func

n

nn

tions see App endix D This calls for redening R such that R h v

corresp onds to the preimages of v under h that are in S Assuming that h is a

n

n nn

mapping of S to f g we may dene R h v R x such that x is

n

the unique string satisfying x S and hx v where the condition x S may

n n

b e veried by providing the internal coins of the sampling procedure that generate

n

x Denoting the sampling pro cedure of X by S and letting S r denote the

n

output of S on input and internal coins r we actually redene R as

nn n n

R h v fhr y i hS r v y R S r g

jxjjxj

We note that hr y i R h hx yields a desired solution y R x

jxj

r x but otherwise all b ets are o since y will b e a solution for if S

jxj

S r x Now although typically h will not b e a mapping of S to

n

n

f g it is the case that for each x S with constant probability over the

n

choice of h it holds that hx has a unique preimage in S under h See the pro of

n

jxjjxj jxj

of Theorem In this case hr y i R h hx implies S r x

which in turn implies y R x We claim that the randomized mapping of

jxj

nn

yields a relaxed x to h hx where h is uniformly selected in H

jxj

n

nn

reduction of R X to R X where X is uniform over H f v

n

n

n

v f g g Needless to say the claim refers to the reduction that on input x

nn

makes the query h hx and returns y if the oracle answer equals hr y i

and y R x

jxj

for The claim is proved by considering the set of choices of h H

x

jxj

which x S is the only preimage of hx under h that resides in S ie

n n

AVERAGE CASE COMPLEXITY

jfx S hx hxgj In this case ie h it holds that hr y i

n x

jxjjxj jxj

R h hx implies that S r x and y R x and the relaxed

validity condition follows The relaxed domination condition follows by noting

jxj jxjjxj

that Pr X x that x is mapp ed to h hx with proba

n

jxj

jxjjxj

j and that x is the only preimage of h hx under the bility jH

jxj

h mapping among x S such that

n x

Before going any further let us highlight the imp ortance of hashing X to n

n

bit strings On one hand this mapping is suciently onetoone and thus with

constant probability the solution provided for the hashed instance ie hx yield

a solution for the original instance ie x This guarantees the validity of the re

duction On the other hand for a typical h the mapping of X to hX covers the

n n

relevant range almost uniformly This guarantees that the reduction satises the

domination condition Note that these two phenomena imp ose conicting require

ments that are b oth met at the correct value of that is the onetoone condition

requires n log jS j whereas an almost uniform cover requires n log jS j

n n

Also note that n log Pr X x for every x in the supp ort of X the

n n

latter quantity will b e in our fo cus in the general case

The general case Finally we get rid of the assumption that X is uniformly dis

n

n

tributed over some subset of f g All that we know is that there exists a prob

n

abilistic p olynomialtime sampling algorithm S such that S is distributed

identically to X In this general case we map instances of R X according to

n

their probability mass such that x is mapp ed to an instance of R that consists of

h hx and additional information where h is a random hash function mapping

nbit long strings to bit long strings such that

x

def

dlog Pr X xe

x

jxj

x

strings in the supp ort of Since in the general case there may b e more than

X we need to augment the reduced instance in order to ensure that it is uniquely

n

asso ciated with x The basic idea is augmenting the mapping of x to h hx with

additional information that restricts X to strings that o ccur with probability at

n

x

Indeed when X is restricted in this way the value of hX uniquely least

n n

determines X

n

n

Let q n denote the randomness complexity of S and S r denote the out

n q n

put of S on input and internal coin tosses r f g Then we randomly

q jxj jxj

x

and h f g map x to h hx h v where h f g f g

q jxj q jxj

x x

is uniformly dis are random hash functions and v f g f g

tributed The instance h v h v of the redened search problem R has solutions

n n

that consists of pairs hr y i such that hS r v h r v and y R S r

As we shall see this augmentation guarantees that with constant probability over

the choice of h h v the solutions to the reduced instance h hx h v corre

sp ond to the solutions to the original instance x

The foregoing description assumes that on input x we can eciently deter

mine which is an assumption that cannot b e justied Instead we select

x

uniformly in f q jxjg and so with noticeable probability we do select the

CHAPTER RELAXING THE REQUIREMENTS

correct value ie Pr q jxj p olyjxj For clarity we make

x

n

n and explicit in the reduced instance Thus we randomly map x f g to

q n

n n

h hx h v f g where f q ng h H h H

n

q n

q n

and v f g are uniformly distributed in the corresp onding sets This

mapping will b e used to reduce R X to R X where R and X fX g

n N

n

are redened yet again Sp ecically we let

n n n

R h v h v fhr y i hS r v h r v y R S r g

and X assigns equal probability to each X for f ng where each

n

n

q n

X is isomorphic to the uniform distribution over H f g H

n

n

q n

q n

f g Note that indeed R X distPC

The foregoing randomized mapping is analyzed by considering the correct choice

for that is on input x we fo cus on the choice Under this conditioning as

x

we shall show with constant probability over the choice of h h and v the instance

n

x

h hx h v x is the only value in the support of X that is mapped to

n

n

and satises fr hS r hx h r v g It follows that for such

n n

x

h hx h v satises S r x h h and v any solution hr y i R

and thus y R x which means that the relaxed validity condition is satised

The relaxed domination condition is satised to o b ecause conditioned on

x

n

x

and for such h h v the probability that X is mapp ed to h hx h v

n

n

x

approximately equals Pr X h hx h v

n

x

We now turn to analyze the probability over the choice of h h and v that the

n

x

h hx h v instance x is the only value in the supp ort of X that is mapp ed to

n

n

and satises fr hS r hx h r v g Firstly we note that

n q n

x

and thus with constant probability over the choice jfr S r xgj

q n

x

n q n

x

there exists r that satises S r x and and v f g of h H

q n

q n

x

h r v Furthermore with constant probability over the choice of h H

q n

q n

x x

and v f g it also holds that there are at most O strings r such

n

fS r h r v g that h r v Fixing such h and v we let S

h v

x

it holds and we note that with constant probability over the choice of h H

n

that is mapp ed to hx under h Thus with that x is the only string in S

h v

constant probability over the choice of h h and v the instance x is the only

n

x

h hx h v and satises value in the supp ort of X that is mapp ed to

n

n

fr hS r hx h r v g The theorem follows

Reection Theorem implies that if sampNP is not contained in tp cBPP

then every distNP complete problem is not in tp c BPP This means that the

hardness of some distributional problems that refer to sampleable distributions im

plies the hardness of some distributional problems that refer to simple distributions

As in other places a suitable enco ding will b e used such that the reduction maps strings of the

same length to strings of the same length ie nbit string are mapp ed to n bit strings for n

n n q n

p oly n For example we may enco de h h hx h v i as hhi hhxi hh i hv i

where each hw i denotes an enco ding of w by a string of length n n q n

AVERAGE CASE COMPLEXITY

Furthermore by Prop osition this implies the hardness of distributional prob

lems that refer to the uniform distribution Thus hardness with resp ect to some

distribution in an utmost wide class which arguably captures all distributions that

may o ccur in practice implies hardness with resp ect to a single simple distribution

which arguably is the simplest one

Relation to oneway functions We note that the existence of oneway func

tions see Section implies the existence of problems in sampPC that are not in

tp c BPPF which in turn implies the existence of such problems in dist PC Sp ecif

ically for a lengthpreserving oneway function f consider the distributional search

problem R ff U g where R ff r r r f g g On the other

f n f

nN

hand it is not known whether the existence of a problem in sampPC n tp cBPPF

implies the existence of oneway functions In particular the existence of a prob

lem R X in sampPC n tp c BPPF represents the feasibility of generating hard

instances for the search problem R whereas the existence of oneway function rep

resents the feasibility of generating instancesolution pairs such that the instances

are hard to solve see Section Indeed the gap refers to whether or not hard

instances can be eciently generated together with corresponding solutions Our

world view is thus depicted in Figure where lower levels indicate seemingly

weaker assumptions

one-way functions exist

distNP is not in tpcBPP (equiv., sampNP is not in tpcBPP)

P is different than NP

Figure Worstcase vs averagecase assumptions

Chapter Notes

In this chapter we presented two dierent approaches to the relaxation of com

putational problems The rst approach refers to the concept of approximation

while the second approach refers to averagecase analysis We demonstrated that

various natural notions of approximation can b e cast within the standard frame

works where the framework of promise problems presented in Section is

the leaststandard framework we used and it suces for casting gap problems and

Note that the distribution f U is uniform in the sp ecial case that f is a p ermutation over

n

n

f g

CHAPTER RELAXING THE REQUIREMENTS

prop erty testing In contrast the study of averagecase complexity requires the

introduction of a new conceptual framework and addressing various denitional

issues

A natural question at this p oint is what have we gained by relaxing the require

ments In the context of approximation the answer is mixed in some natural cases

we gain a lot ie we obtained feasible relaxations of hard problems while in other

natural cases we gain nothing ie even extreme relaxations remain as intractable

as the original versions In the context of averagecase complexity the negative

side seems more prevailing at least in the sense of b eing more systematic In par

ticular assuming the existence of oneway functions every natural NPcomplete

problem has a distributional version that is typicalcase hard where this version

refers to a sampleable ensemble and in fact even to a simple ensemble Fur

thermore in this case some problems in NP have hard distributional versions that

refer to the uniform distribution

Approximation

The following bibliographic comments are quite laconic and neglect mentioning

various imp ortant works including credits for some of the results mentioned in our

text As usual the interested reader is referred to corresp onding surveys

Search or Optimization The interest in approximation algorithms increased

considerably following the demonstration of the NPcompleteness of many nat

ural optimization problems But with some exceptions most notably

the systematic study of the complexity of such problems stalled till the discov

ery of the PCP connection see Section by Feige Goldwasser Lovasz and

Safra Indeed the relatively tight inapproximation results for maxClique

maxSAT and the maximization of linear equations due to Hastad

build on previous work regarding PCP and their connection to approximation cf

eg Sp ecically Theorem is due to while The

orems and are due to The b est known inapproximation result for

minimum Vertex Cover see Theorem is due to but we doubt it is tight

see eg Reductions among approximation problems were dened and

presented in see Exercise which presents a ma jor technique introduced

in For general texts on approximation algorithms and problems as discussed

in Section the interested reader is referred to the surveys collected in

A comp endium of NP optimization problems is available at

Recall that a dierent type of approximation problems which are naturally

asso ciated with search problems refer to approximately counting the number of

solutions These approximation problems were treated in Section in a rather

ad hoc manner We note that a more systematic treatment of approximate counting

problems can b e obtained by using the denitional framework of Section eg

the notions of gap problems p olynomialtime approximation schemes etc

See also

AVERAGE CASE COMPLEXITY

Prop erty testing The study of prop erty testing was initiated by Rubinfeld and

Sudan and reinitiated by Goldreich Goldwasser and Ron While the

fo cus of was on algebraic prop erties such as lowdegree p olynomials the fo cus

of was on graph prop erties and Theorem is taken from The mo del

of b oundeddegree graphs was introduced in and Theorem combines

results from For surveys of the area the interested reader is referred

to

Averagecase complexity

The theory of averagecase complexity was initiated by Levin who in partic

ular proved Theorem In light of the laconic nature of the original text

we refer the interested reader to a survey which provides a more detailed

exp osition of the denitions suggested by Levin as well as a discussion of the con

siderations underlying these suggestions This survey provides also a brief

account of further developments

As noted in x the current text uses a variant of the original denitions

In particular our denition of typicalcase feasibility diers from the original

denition of averagecase feasibility in totally discarding exceptional instances

and in even allowing the algorithm to fail on them and not merely run for an

excessive amount of time The alternative denition was suggested by several

researchers and app ears as a sp ecial case of the general treatment provided in

Turning to x we note that while the existence of distNP complete prob

lems cf Theorem was established in Levins original pap er the ex

istence of distNP complete versions of all natural NPcomplete decision problems

cf Theorem was established more than two decades later in

Section is based on Sp ecically Theorem or rather the

reduction of search to decision is due to and so is the introduction of the class

sampNP A version of Theorem was proven in and our pro of follows

their ideas which in turn are closely related to the ideas underlying the pro of of

Theorem proved in

Recall that we know of the existence of problems in distNP that are hard pro

vided sampNP contains hard problems However these distributional problems do

not seem very natural ie they either refer to somewhat generic decision problems

such as S or to somewhat contrived probability ensembles cf Theorem

u

The presentation of distNP complete problems that combine a more natural deci

sion problem like SAT or Clique with a more natural probability ensemble is an

op en problem

Exercises

Exercise general TSP For any adequate function g prove that the fol

lowing approximation problem is NPHard Given a general TSP instance I rep

resented by a symmetric matrix of pairwise distances the task is nding a tour of

length that is at most a factor g I of the minimum Sp ecically show that the

CHAPTER RELAXING THE REQUIREMENTS

result holds with g I expjI j and for instances in which all distances are

p ositive integers

Guideline Use a reduction from Hamiltonian cycle problem Sp ecically reduce the

instance G n E to an nbyn distance matrix D d such that d

ij ij

ij n

expp oly n if fi j g E and d

ij

Exercise TSP with triangle inequalities Provide a p olynomialtime

factor approximation for the sp ecial case of TSP in which the distances satisfy the

triangle inequality

Guideline First note that the length of any tour is lowerbounded by the weight of

a minimum spanning tree in the corresp onding weighted graph Next note that such a

tree yields a tour of length twice the weight of this tree that may visit some p oints

several times The triangle inequality guarantees that the tour do es not b ecome longer

by shortcuts that eliminate multiple visits at the same p oint

Exercise a weak version of Theorem Using Theorem prove

b a

that for some constants a b when setting LN N and sN N

it holds that gapClique is NPhard

Ls

Guideline Starting with Theorem apply the Expander Random Walk Generator

of Prop osition in order to derive a PCP system with logarithmic randomness and

query complexities that accepts noinstances of length n with probability at most n

The claim follows by applying the FGLSSreduction of Exercise while noting that

x is reduced to a graph of size p oly jxj such that the gap b etween yes and noinstances

is at least a factor of jxj

Exercise a weak version of Theorem Using Theorem prove

that for some constants s L the problem gapVC is NPhard

sL

Guideline Note that combining Theorem and Exercise implies that for some

constants b it holds that gapClique is NPhard where LN b N and sN

Ls

b N The claim follows using the relations b etween cliques indep endent sets and

vertex covers

Exercise a weak version of Theorem Using Theorem prove

that for some constants s L the problem gapLin is NPhard

Ls

Guideline Recall that by Theorems and the gap problem gapSAT is NP

Hard Note that the result holds even if we restrict the instances to have exactly three

not necessarily dierent literals in each clause Applying the reduction of Exercise

note that for any assignment a clause that is satised by is mapp ed to seven equations

of which exactly three are violated by whereas a clause that is not satised by is

mapp ed to seven equations that are all violated by

Exercise natural inapproximability without the PCP Theorem In

contrast to the inapproximability results reviewed in x the NPcompleteness

of the following gap problem can b e established rather easily without referring

AVERAGE CASE COMPLEXITY

to the PCP Theorem The instances of this problem are systems of quadratic

equations over GF as in Exercise yesinstances are systems that have a

solution and noinstances are systems for which any assignment violates at least

one third of the equations

Guideline By Exercise when given such a quadratic system it is NPhard to

determine whether or not there exists an assignment that satises all the equations Using

an adequate smallbias generator cf Section present an amplifying reduction cf

Section of the foregoing problem to itself Sp ecically if the input system has m

equations then we use a generator that denes a sample space of p oly m many mbit

strings and consider the corresp onding linear combinations of the input equations Note

that it suces to b ound the bias of the generator by whereas using an biased

generator yields an analogous result with replaced by

Exercise enforcing multiway equalities via expanders The aim of

this exercise is presenting a technique of Papadimitriou and Yannakakis that

is useful for designing reductions among approximation problems Recalling that

gapSAT is NPhard our goal is proving NPhard of the following gap problem

c

denoted gapSAT which is a sp ecial case of gapSAT Sp ecically the instances

are restricted to CNF formulae with each variable app earing in at most c clauses

where c as is a xed constant Note that the standard reduction of SAT to

the corresp onding sp ecial case see pro of of Prop osition do es not preserve an

approximation gap The idea is enforcing equality of the values assigned to the

auxiliary variables ie the copies of each original variable by introducing equality

constraints only for pairs of variables that corresp ond to edges of an expander

graph see App endix E For example we enforce equality among the values of

m i j

z z by adding the clauses z z for every fi j g E where E is the

set of edges of an mvertex expander graph Prove that for some constants c and

c

the corresp onding mapping reduces gapSAT to gapSAT

Guideline Using dregular expanders in the foregoing reduction we map general CNF

formulae to CNF formulae in which each variable app ears in at most d clauses

Note that the number of added clauses is linearly related to the number of original clauses

Clearly if the original formula is satisable then so is the reduced one On the other hand

consider an arbitrary assignment to the reduced formula ie the formula obtained

by mapping For each original variable z if assigns the same value to almost all

copies of z then we consider the corresp onding assignment in Otherwise by virtue of

the added clauses do es not satisfy a constant fraction of the clauses containing a copy

of z

Recall that in this reduction each o ccurrence of each Bo olean variable is replaced by a new

copy of this variable and clauses are added for enforcing the assignment of the same value to all

m

these copies Sp ecically the m o ccurrence of variable z are replaced by the variables z z

i i i i

while adding the clauses z z and z z for i m The problem is

that almost all clauses of the reduced formula may b e satised by an assignment in which half

of the copies of each variable are assigned one value and the rest are assigned an opp osite value

i i m

That is an assignment in which z z z z violates only one of the

auxiliary clauses introduced for enforcing equality among the copies of z Using an alternative

i j

reduction that adds the clauses z z for every i j m will not do either b ecause the

number of added clauses may b e quadratic in the number of original clauses

CHAPTER RELAXING THE REQUIREMENTS

Exercise deciding ma jority requires linear time Prove that deciding

ma jority requires lineartime even in a direct access mo del and when using a ran

domized algorithm that may err with probability at most

Guideline Consider the problem of distinguishing X from Y where X resp Y is

n n n n

uniformly distributed over the set of nbit strings having exactly bn c resp bn c

zeros For any xed set I n denote the pro jection of X resp Y on I by X resp

n n

n

Y Prove that the statistical dierence b etween X and Y is b ounded by O jI jn

n n n

Note that the argument needs to b e extended to the case that the examined lo cations are

selected adaptively

Exercise testing ma jority in p olylogarithmic time Show that test

ing ma jority in the sense of Denition can b e done in p olylogarithmic

time by probing the input at a constant number of randomly selected lo cations

Exercise on the triviality of some testing problems Show that the

following sets are trivially testable in the adjacency matrix representation ie for

every and any such set S there exists a trivial algorithm that distinguishes

S from S

The set of connected graphs

The set of Hamiltonian graphs

The set of Eulerian graphs

Indeed show that in each case S

Guideline for Item Note that in general the fact that the sets S and S are

testable within some complexity do es not imply the same for the set S S

Exercise an equivalent denition of tp c P Prove that S X tp c P

if and only if there exists a p olynomialtime algorithm A such that the probability

that AX errs in determining membership in S is a negligible function in n

n

Exercise tp cP versus P Part Prove that tp cP contains a problem

S X such that S is not even recursive Furthermore use X U

jxj

Guideline Let S f x x S g where S is an arbitrary nonrecursive set

Exercise tp cP versus P Part Prove that there exists a distribu

tional problem S X such that S P and yet there exists an algorithm solving

S correctly on all inputs in time that is typically p olynomial with resp ect to X

Furthermore use X U

Guideline For any timeconstructible function t N N that is sup erp olynomial and

jxj

subexp onential use S f x x S g for any S Dtimet n P

Exercise simple distributions and monotone sampling We say that

a probability ensemble X fX g is p olynomialtime sampleable via a monotone

n

nN

mapping if there exists a p olynomial p and a p olynomialtime computable function

f such that the following two conditions hold

AVERAGE CASE COMPLEXITY

For every n the random variables f U and X are identically distributed

n

pn

pn

For every n and every r r f g it holds that f r f r where

the inequalities refers to the standard lexicographic order of strings

Prove that X is simple if and only if it is p olynomialtime sampleable via a mono

tone mapping

Guideline Supp ose that X is simple and let p b e a p olynomial b ounding the running

time of the algorithm that on input x outputs Pr X x Thus the binary representa

jxj

pn n

tion of Pr X x has length at most pjxj The desired function f f g f g

jxj

is obtained by dening f r x if the number represented by r resides in the interval

Pr X x Pr X x Note that f can b e computed by binary search using the fact

n n

that X is simple Turning to the opp osite direction we note that any eciently com

pn n

putable and monotone mapping f f g f g can b e eciently inverted by a

binary search Furthermore similar metho ds allow for eciently determining the interval

of pnbit long strings that are mapp ed to any given nbit long string

Exercise reductions preserve typical p olynomialtime solveability

Prove that if the distributional problem S X is reducible to the distributional

problem S X and S X tp cP then S X is in tp cP

Guideline Let B denote the set of exceptional instances for the distributional problem

S X that is B is the set of instances on which the solver in the hypothesis either

errs or exceeds the typical runningtime Prove that Pr QX B is a negligible

n

function in n using b oth Pr y QX pjy j Pr X y and jxj p jy j for every

n

jy j

P

Pr y QX y Qx Sp ecically use the latter condition for inferring that

n

y B

P P

pm Pr y QX which is upp erb ounded by equals

n

mp mn y fy B p jy jng

Pr X B which in turn is negligible in terms of n

m

Exercise reductions preserve errorless solveability In continuation

to Exercise prove that reductions preserve errorless solveability ie solve

ability by algorithms that never err and typically run in p olynomialtime

Exercise transitivity of reductions Prove that reductions among dis

tributional problems as in Denition are transitive

Guideline The p oint is establishing the domination prop erty of the comp osed reduction

The hypothesis that reductions do not make to o short queries is instrumental here

Exercise For any S NP present a simple probability ensemble X such

that the generic reduction used in the pro of of Theorem when applied to

S X violates the domination condition with resp ect to S U

u

n

Guideline Consider X fX g such that X is uniform over f x x

n n

nN

n

f g g

Exercise variants of the Co ding Lemma Prove the following two vari

ants of the Co ding Lemma which is stated in the pro of of Theorem

CHAPTER RELAXING THE REQUIREMENTS

A variant that refers to any eciently computable function f g

that is monotonically nondecreasing over f g ie x x for any

x x f g That is unlike in the pro of of Theorem here it

n n

holds that for every n

As in Part except that in this variant the function is strictly increasing

and the compression condition requires that jC xj log x rather

def

than jC xj minfjxj log xg where x x x

In b oth cases the pro of is less cumbersome than the one presented in the main

text

Exercise Prove that for any problem S X in distNP there exists a simple

probability ensemble Y such that the reduction used in the pro of of Theorem

suces for reducing S X to S Y

u

t

Guideline Consider Y fY g such that Y assigns to the instance hM x i a

n n

nN

def

t

probability mass prop ortional to Pr X x Sp ecically for every hM x i it

x

jxj

def def

n

t jM j t

holds that Pr Y hM x i where n jhM x ij jM j jxj t

n x

t

Alternatively we may set Pr Y hM x i if M M and t p jxj and

n x S S

t

Pr Y hM x i otherwise where M and P are as in the pro of of Theorem

n S S

Exercise monotone markability and monotone reductions In con

tinuation to Exercise we say that a set T is monotonically markable if there

exists a p olynomialtime marking algorithm M such that

For every z f g it holds that M z T if and only if z T

Monotonicity for every jz j jz j and it holds that M z

M z where the inequalities refer to the standard lexicographic order

of strings

Auxiliary length requirements

a If jz j jz j and j j j j then jM z j jM z j

b If jz j jz j and j j j j then jM z j jM z j

c There exists a p olynomial p N N such that for every and every

i t

z f g there exists t p such that jM z j p

i

The rst two requirements imply that jM z j is a function of jz j and jj

which increases with jj The third requirement implies that for every

each string of length at most can b e mapp ed to a string of length p

Note that Condition is repro duced from Exercise whereas Conditions and

are new Prove that if the set S is Karpreducible to the set T and T is monotoni

cally markable then S is Karpreducible to T by a reduction that is monotone and

lengthregular ie the reduction satises the conditions of Prop osition

AVERAGE CASE COMPLEXITY

Guideline Given a Karpreduction f from S to T rst obtain a lengthregular reduction

f from S to T by applying the marking algorithm to f x while using Conditions

and c In particular one can guarantee that if jx j jx j then jf x j jf x j Next

obtain a reduction f that is also monotone eg by letting f x M f x x while

using Conditions and

Exercise monotone markability and markability Prove that if a set

is monotonically markable as p er Exercise then it is markable as p er Ex

ercise

Guideline Let M denote the guaranteed monotonemarking algorithm For starters

assume that M is and dene M z M z hz i Note that the preimage

z can b e found by conducting a binary search for each of the p ossible values of jz j

In the general case we mo dify the construction so that to guarantee that M is

P

nm

Sp ecically let idxn m n i b e the index of n m in an enumeration

i

of all pairs of p ositive integers and p b e as in Condition c Then let M z

n tnm

M z C hz i where tn m n m satises jM j pidxn m

tjz jjj

and C y is a monotone enco ding of y using a tbit long string

t

Exercise some monotonically markable sets Referring to Exercise

verify that each of the twentyone NPcomplete problems treated in in Karps rst

pap er on NPcompleteness is monotonically markable For starters consider

the sets SAT Clique and Colorability

Guideline For SAT consider the following marking algorithm M This algorithm uses two

xed satisable formulae of the same length denoted and such that For

m

any formula and any binary string f g it holds that M

m m

where and use variables that do not app ear in Note that

m

the multiple o ccurrences of can b e easily avoided by using variations of

Exercise randomized reductions Following the outline in x

provide a denition of randomized reductions among distributional problems

In analogy to Exercise prove that randomized reductions preserve fea

sible solveability ie typical solveability in probabilistic p olynomialtime

That is if the distributional problem S X is randomly reducible to the

distributional problem S X and S X tp cBPP then S X is in

tp cBPP

In analogy to Exercise prove that randomized reductions preserve

solveability by probabilistic algorithms that err with probability at most

on each input and typically run in p olynomialtime

Prove that randomized reductions are transitive cf Exercise

Actually Condition combined with the length regularity of f only takes care of mono

tonicity with resp ect to strings of equal length To guarantee monotonicity with resp ect to strings

of dierent length we also use Condition b and jf x j jf x j for jx j jx j

CHAPTER RELAXING THE REQUIREMENTS

Show that the error probability of randomized reductions can b e reduced

while preserving the domination condition

Extend the foregoing to reductions that involve distributional search problems

Exercise simple vs sampleable ensembles Part Prove that any

simple probability ensemble is p olynomialtime sampleable

Guideline See Exercise

Exercise simple vs sampleable ensembles Part Assuming that

P contains functions that are not computable in p olynomialtime prove that

there exists p olynomialtime sampleable ensembles that are not simple

Guideline Consider any R PC and supp ose that p is a p olynomial such that x y R

n

implies jy j pjxj Then consider the sampling algorithm A that on input uniformly

n pn

selects x y f g f g and outputs x if x y R and x otherwise

jxjpjxj jxj

Note that R x Pr A x

Exercise distributional versions of NPC problems Part

Prove that if S is Karpreducible to S by a mapping that do es not shrink the input

u

then there exists a p olynomialtime sampleable ensemble X such that any problem

in distNP is reducible to S X

Guideline Prove that the guaranteed reduction of S to S also reduces S U to

u u

S X for some sampleable probability ensemble X Consider rst the case that the

standard reduction of S to S is length preserving and prove that when applied to a

u

sampleable probability ensemble it induces a sampleable distribution on the instances

of S Note that U is sampleable by Exercise Next extend the treatment to

induces a distribution on the general case where applying the standard reduction to U

n

p olyn

m n

f g rather than a distribution on f g

mn

Exercise distributional versions of NPC problems Part

Prove Theorem ie if S is Karpreducible to S by a mapping that do es

u

not shrink the input then there exists a p olynomialtime sampleable ensemble X

such that any problem in sampNP is reducible to S X

Guideline We establish the claim for S S and the general claim follows by using

u

the reduction of S to S as in Exercise Thus we fo cus on showing that for

u

some suitably chosen sampleable ensemble X any S X sampNP is reducible to

S X Lo osely sp eaking X will b e an adequate convex combination of all sampleable

u

distributions and thus X will neither equal U nor b e simple Sp ecically X fX g

n

nN

is dened such that the sampler for X uniformly selects i n emulates the execution of

n

th n

the i algorithm in lexicographic order on input for n steps and outputs whatever

Needless to say the choice to consider n algorithms in the denition of X is quite arbitrary

n

Any other unbounded function of n that is at most a p olynomial and is computable in p olynomial

th

time will do More generally we may select the i algorithm with p as long as p is a noticeable

i i

function of n Likewise the choice to emulate each algorithm for a cubic number of steps rather

some other xed p olynomial number of steps is quite arbitrary

AVERAGE CASE COMPLEXITY

n

the latter has output or in case the said algorithm has not halted within n steps

Prove that for any S X sampNP such that X is sampleable in cubic time the

standard reduction of S to S reduces S X to S X as p er Denition ie

u u

in particular it satises the domination condition Finally using adequate padding

reduce any S X sampNP to some S X sampNP such that X is sampleable

in cubic time

Exercise search vs decision in the context of sampleable ensembles

Prove that every problem in sampNP is reducible to some problem in sampPC

and every problem in sampPC is randomly reducible to some problem in sampNP

Guideline See pro of of Theorem

Note that applying this reduction to X yields an ensemble that is also sampleable in cubic

time This claim uses the fact that the standard reduction runs in time that is less than cubic

and in fact almost linear in its output and the fact that the output is longer than the input

CHAPTER RELAXING THE REQUIREMENTS

Epilogue

Farewell Hans whether you live or end where you are Your

chances are not go o d The wicked dance in which you are caught

up will last a few more sinful years and we would not wager

much that you will come out whole To b e honest we are not

really b othered ab out leaving the question op en Adventures in

the esh and spirit which enhanced and heightened your ordi

nariness allowed you to survive in the spirit what you probably

will not survive in the esh There were ma jestic moments when

you saw the intimation of a dream of love rising up out of death

and the carnal b o dy Will love someday rise up out of this world

wide festival of death this ugly rutting fever that inames the

rainy evening sky all round

Thomas Mann The Magic Mountain The Thunderbolt

We hop e that this work has succeeded in conveying the fascinating avor of the

concepts results and op en problems that dominate the eld of computational com

plexity We b elieve that the new century will witness even more exciting develop

ments in this eld and urge the reader to try to contribute to them But b efore

bidding go o dbye we wish to express a few more thoughts

As noted in Section so far complexity theory has b een far more success

ful in relating fundamental computational phenomena than in providing denite

answers regarding fundamental questions Consider for example the theory of NP

completeness versus the PversusNP Question or the theory of pseudorandomness

versus establishing the existence of oneway function even under P NP The

failure to resolve questions of the absolute type is the source of common frustra

tion and one often wonders ab out the reasons for this failure

Our feeling is that many of these failures are really due to the diculty of

the questions asked and that one tends to underestimate their hardness b ecause

they are so app ealing and natural Indeed the underlying sentiment is that if

a question is app ealing and natural then answering it should not b e hard We

doubt this sentiment Our own feeling is that the more intuitive a question is

the harder it may b e to answer Our view is that intuitive questions arise from

an encounter with the raw and chaotic reality of life rather than from an articial

construct which is typically endowed with a rich internal structure Indeed natural

CHAPTER RELAXING THE REQUIREMENTS

complexity classes and natural questions regarding computation arise from lo oking

at the reality of computation from the outside and thus lack any internal structure

Sp ecically complexity classes are dened in terms of the external b ehavior of

p otential algorithms ie the resources such algorithms require rather than in

terms of the internal structure of the problem In our opinion this external

nature of the denitions of complexity theoretic questions makes them hard to

resolve

Another hard asp ect regarding the absolute or lowerbound type of ques

tions is the fact that they call for imp ossibility results That is the natural formu

lation of these questions calls for proving the nonexistence of something ie the

nonexistence of ecient pro cedures for solving the problem in question Needless

to say proving the nonexistence of certain ob jects is typically harder than proving

existence of related ob jects indeed see Section Still pro ofs of nonexistence

of certain ob jects are known in various elds and in particular in complexity theory

but such pro ofs tend to either b e trivial see eg Section or are derived by

exhibiting a sophisticated pro cess that transforms the original question to a trivial

one Indeed the latter case is the one that underlies many of the impressive suc

cesses of circuit complexity and all relative results of the highlevel direction have

a similar nature ie of relating one computational question to another Thus

we are not suggesting that the absolute questions of complexity theory cannot

b e resolved but are rather suggesting an intuitive explanation to the diculties of

resolving them

The obvious fact that dicult questions can b e resolved is demonstrated by

several recent results which are mentioned in this b o ok and forced us to mo dify

earlier drafts of it Examples include the logspace graph exploration algorithm

presented in Section and the alternative pro of of the PCP Theorem presented

in x as well as Theorem and the brief mention of the results of