Computational Complexity

A Conceptual Perspective

Oded Goldreich

Department of Computer Science and Applied Mathematics

Weizmann Institute of Science Rehovot Israel

July

Chapter

Randomness and Counting

I owe this almost atro cious variety to an institution which other

republics do not know or which op erates in them in an imp erfect

and secret manner the lottery

Jorge Luis Borges The Lottery In Babylon

So far our approach to computing devices was somewhat conservative we thought

of them as executing a deterministic rule A more lib eral and quite realistic ap

proach which is pursued in this chapter considers computing devices that use a

probabilistic rule This relaxation has an immediate impact on the notion of e

cient computation which is consequently asso ciated with probabilistic p olynomial

time computations rather than with deterministic p olynomialtime ones We

stress that the asso ciation of ecient computation with probabilistic p olynomial

time computation makes sense provided that the failure probability of the latter is

negligible which means that it may b e safely ignored

The quantitative nature of the failure probability of probabilistic algorithm

provides one connection b etween probabilistic algorithms and counting problems

The latter are indeed a new type of computational problems and our fo cus is on

counting eciently recognizable ob jects eg NPwitnesses for a given instance of

set in NP Randomized pro cedures turn out to play an imp ortant role in the

study of such counting problems

Summary Focusing on probabilistic p olynomialtime algorithms we

consider various types of failure of such algorithms giving rise to com

plexity classes such as BPP RP and ZPP The results presented

include BPP P p oly and BPP

We then turn to counting problems sp ecically counting the number

of solutions for an instance of a search problem in PC or equivalently

counting the number of NPwitnesses for an instance of a decision prob

lem in NP We distinguish b etween exact counting and approximate

counting in the sense of relative approximation In particular while

CHAPTER RANDOMNESS AND COUNTING

any problem in PH is reducible to the exact counting class P ap

proximate counting for P is probabilisticly reducible to NP

Additional related topics include the P completeness of various count

ing problems eg counting the number of satisfying assignments to a

given CNF formula and counting the number of p erfect matchings in a

given graph the complexity of searching for unique solutions and the

relation b etween approximate counting and generating almost uni

formly distributed solutions

Prerequisites We assume basic familiarity with elementary probability theory

see Section D In Section we will rely extensively on the formulation pre

sented in Section ie the NP search problem class PC as well as the sets

def def

fX R x g dened for every R PC R x fy x y R g and S

Probabilistic PolynomialTime

Considering algorithms that utilize random choices we extend our notion of ef

cient algorithms from deterministic p olynomialtime algorithms to probabilistic

p olynomialtime algorithms

Rigorous mo dels of probabilistic or randomized algorithms are dened by nat

ural extensions of the basic machine mo del We will exemplify this approach by

describing the mo del of probabilistic Turing machines but we stress that again

the sp ecic choice of the mo del is immaterial as long as it is reasonable A

probabilistic Turing machine is dened exactly as a nondeterministic machine see

the rst item of Denition but the denition of its computation is fundamen

tal ly dierent Sp ecically whereas Denition refers to the question of whether

or not there exists a computation of the machine that started on a sp ecic input

reaches a certain conguration in case of probabilistic Turing machines we refer

to the probability that this event occurs when at each step a choice is selected uni

formly among the relevant possible choices available at this step That is if the

transition function of the machine maps the current statesymbol pair to several

p ossible triples then in the corresp onding probabilistic computation one of these

triples is selected at random with equal probability and the next conguration is

determined accordingly These random choices may b e viewed as the internal coin

tosses of the machine Indeed as in the case of nondeterministic machines we

may assume without loss of generality that the transition function of the machine

maps each statesymbol pair to exactly two p ossible triples see Exercise

We stress the fundamental dierence b etween the ctitious mo del of a non

deterministic machine and the realistic mo del of a probabilistic machine In the case

of a nondeterministic machine we consider the existence of an adequate sequence

of choices leading to a desired outcome and ignore the question of how these

choices are actually made In fact the selection of such a sequence of choices is

merely a mental exp eriment In contrast in the case of a probabilistic machine at

each step a real random choice is made uniformly among a set of predetermined

PROBABILISTIC POLYNOMIALTIME

p ossibilities and we consider the probability of reaching a desired outcome

In view of the foregoing we consider the output distribution of such a proba

bilistic machine on xed inputs that is for a probabilistic machine M and string

x f g we denote by M x the output distribution of M when invoked on

input x where the probability is taken uniformly over the machines internal coin

tosses Needless to say we will consider the probability that M x is a correct

answer that is in the case of a search problem resp decision problem we will b e

interested in the probability that M x is a valid solution for the instance x resp

represents the correct decision regarding x

The foregoing description views the internal coin tosses of the machine as taking

place onthey that is these coin tosses are p erformed online by the machine

itself An alternative mo del is one in which the sequence of coin tosses is provided

by an external device on a sp ecial random input tap e In such a case we view

these coin tosses as p erformed oline Sp ecically we denote by M x r the

uniquely dened output of the residual deterministic machine M when given the

primary input x and random input r Indeed M is a deterministic machine that

takes two inputs the rst representing the actual input and the second representing

def

the random input but we consider the random variable M x M x U

jxj

where jxj denotes the number of coin tosses exp ected by M x

These two p ersp ectives on probabilistic algorithms are clearly related Clearly

the aforementioned residual deterministic machine M yields the online machine

M that on input x selects at random a string r of adequate length and invokes

M x r On the other hand the computation of any online machine M is captured

by the residual machine M that emulates the actions of M x based on an auxiliary

input r obtained by M and representing a p ossible outcome of the internal coin

tosses of M Indeed there is no harm in supplying more coin tosses than are

actually used by M and so the length of the aforementioned auxiliary input may

b e set to equal the time complexity of M For sake of clarity and future reference

we state the following denition

Denition online and oline formulations of probabilistic p olynomialtime

We say that M is a online probabilistic p olynomialtime machine if there exists

a polynomial p such that when invoked on any input x f g machine M

always halts within at most pjxj steps regardless of the outcome of its

internal coin tosses In such a case M x is a random variable

We say that M is a oline probabilistic p olynomialtime machine if there exists

pjxj

a polynomial p such that for every x f g and r f g when

invoked on the primary input x and the randominput sequence r machine M

halts within at most pjxj steps In such a case we wil l consider the random

variable M x U

pjxj

Clearly the online and oline formulations are equivalent ie given a online

probabilistic p olynomialtime machine we can derive a functionally equivalent o

line probabilistic p olynomialtime machine and vice versa Thus in the sequel

we will freely use whichever is more convenient

CHAPTER RANDOMNESS AND COUNTING

Failure probability A ma jor asp ect of randomized algorithms probabilistic

machines is that they may fail see Exercise That is with some sp ecied

failure probability these algorithms may fail to pro duce the desired output

We discuss two asp ects of this failure its type and its magnitude

The type of failure is a qualitative notion One asp ect of this type is whether

in case of failure the algorithm pro duces a wrong answer or merely an indica

tion that it failed to nd a correct answer Another asp ect is whether failure

may o ccur on all instances or merely on certain types of instances Let us

clarify these asp ects by considering three natural types of failure giving rise

to three dierent types of algorithms

a The most lib eral notion of failure is the one of twosided error This

term originates from the setting of decision problems where it means

that in case of failure the algorithm may err in b oth directions ie

it may rule that a yesinstance is a noinstance and vice versa In the

case of search problems twosided error means that when failing the

algorithm may output a wrong answer on any input Furthermore the

algorithm may falsely rule that the input has no solution and it may

also output a wrong solution b oth in case the input has a solution and

in case it has no solution

b An intermediate notion of failure is the one of onesided error Again the

term originates from the setting of decision problems where it means

that the algorithm may err only in one direction ie either on yes

instances or on noinstances Indeed there are two natural cases de

p ending on whether the algorithm errs on yesinstances but not on no

instances or the other way around Analogous cases o ccur also in the

setting of search problems In one case the algorithm never outputs

a wrong solution but may falsely rule that the input has no solution

In the other case the indication that an input has no solution is never

wrong but the algorithm may output a wrong solution

c The most conservative notion of failure is the one of zerosided error In

this case the algorithms failure amounts to indicating its failure to nd

an answer by outputting a sp ecial dont know symbol We stress that

in this case the algorithm never provides a wrong answer

Indeed the forgoing discussion ignores the probability of failure which is the

sub ject of the next item

The magnitude of failure is a quantitative notion It refer to the probability

that the algorithm fails where the type of failure is xed eg as in the

forgoing discussion

When actually using a randomized algorithm we typically wish its failure

probability to b e negligible which intuitively means that the failure event is

so rare that it can b e ignored in practice Formally we say that a quantity is

negligible if as a function of the relevant parameter eg the input length

this quantity vanishes faster than the recipro cal of any p ositive p olynomial

PROBABILISTIC POLYNOMIALTIME

For ease of presentation we sometimes consider alternative upp erb ounds

on the probability of failure These b ounds are selected in a way that al

lows and in fact facilitates error reduction ie converting a probabilistic

p olynomialtime algorithm that satises such an upp erb ound into one in

which the failure probability is negligible For example in case of twosided

error we need to b e able to distinguish the correct answer from wrong an

swers by sampling and in the other types of failure hitting a correct answer

suces

In the following three subsections we will discuss complexity classes corresp onding

to the aforementioned three types of failure For sake of simplicity the failure

probability itself will b e set to a constant that allows error reduction

Randomized reductions Before turning to the more detailed discussion we

note that randomized reductions play an imp ortant role in complexity theory Such

reductions can b e dened analogously to the standard Co okReductions resp

Karpreductions and again a discussion of the type and magnitude of the failure

probability is in place For clarity we sp ellout the twosided error versions

In analogy to Denition we say that a problem is probabilistic p olynomial

time reducible to a problem if there exists a probabilistic p olynomialtime

oracle machine M such that for every function f that solves and for every

x with probability at least jxj the output M x is a correct solution

to the instance x where is a negligible function

In analogy to Denition we say that a decision problem S is reducible

to a decision problem S via a randomized Karpreduction if there exists a

probabilistic p olynomialtime algorithm A such that for every x it holds that

is the characteristic Ax x jxj where resp Pr

S S S S

function of S resp S and is a negligible function

These reductions preserve ecient solvability and are transitive see Exercise

Twosided error The complexity class BPP

In this section we consider the most lib eral notion of probabilistic p olynomialtime

algorithms that is still meaningful We allow the algorithm to err on each input

but require the error probability to b e negligible The latter requirement guarantees

the usefulness of such algorithms b ecause in reality we may ignore the negligible

error probability

Before fo cusing on the decision problem setting let us say a few words on the

search problem setting see Denition Following the previous paragraph we

say that a probabilistic p olynomialtime algorithm A solves the search problem

def

of the relation R if for every x S ie R x fy x y R g it holds

that Pr Ax R x jxj and for every x S it holds that Pr Ax

jxj where is a negligible function Note that we did not require that

when invoked on input x that has a solution ie R x the algorithm always

CHAPTER RANDOMNESS AND COUNTING

outputs the same solution Indeed a stronger requirement is that for every such x

there exists y R x such that Pr Ax y jxj The latter version and

quantitative relaxations of it allow for errorreduction see Exercise

Turning to decision problems we consider probabilistic p olynomialtime algo

rithms that err with negligible probability That is we say that a probabilistic

p olynomialtime algorithm A decides membership in S if for every x it holds

that Pr Ax x jxj where is the characteristic function of S

S S

ie x if x S and x otherwise and is a negligible function

S S

The class of decision problems that are solvable by probabilistic p olynomialtime

algorithms is denoted BPP standing for Boundederror Probabilistic Polynomial

time Actually the standard denition refers to machines that err with probability

at most

Denition the class BPP A decision problem S is in BPP if there exists

a probabilistic polynomialtime algorithm A such that for every x S it holds that

Pr Ax and for every x S it holds that Pr Ax

The choice of the constant is immaterial and any other constant greater than

will do and yields the very same class Similarly the complementary constant

can b e replaced by various negligible functions while preserving the class

Both facts are sp ecial cases of the robustness of the class which is established

using the pro cess of error reduction

Error reduction or condence amplication For N let

BPP denote the class of decision problems that can b e solved in probabilistic

p olynomialtime with error probability upp erb ounded by that is S BPP if

there exists a probabilistic p olynomialtime algorithm A such that for every x it

holds that Pr Ax x jxj By denition BPP BPP However a

wide range of other classes also equal BPP In particular we mention two extreme

cases

For every p ositive p olynomial p and n pn the class BPP

equals BPP That is any error that is noticeably b ounded away from

ie error p olyn can b e reduced to an error of

For every p ositive p olynomial p and n the class BPP equals

BPP That is an error of can b e further reduced to an exp onentially

vanishing error

Both facts are proved by invoking the weaker algorithm ie the one having a

larger error probability b ound for an adequate number of times and ruling by

ma jority We stress that invoking a randomized machine several times means that

the random choices made in the various invocations are indep endent of one another

The success probability of such a pro cess is analyzed by applying an adequate Law

of Large Numbers see Exercise

PROBABILISTIC POLYNOMIALTIME

On the p ower of randomization

A natural question arises Did we gain anything in extending the denition of

ecient computation to include also probabilistic polynomialtime ones

This phrasing seems to o generic We certainly gained the ability to toss coins

and generate various distributions More concretely randomized algorithms are

essential in many settings see eg Chapter Section and App endix C

and seem essential in others see eg Sections What we mean to

ask here is whether al lowing randomization increases the power of polynomialtime

algorithms also in the restricted context of solving decision and search problems

The question is whether BPP extends b eyond P where clearly P BPP

It is commonly conjectured that the answer is negative Sp ecically under some

reasonable assumptions it holds that BPP P see Part of Theorem We

note however that a p olynomial slowdown o ccurs in the pro of of the latter result

that is randomized algorithms that run in time t are emulated by deterministic

algorithms that run in time p oly t Furthermore for some concrete problems

most notably primality testing cf x the known probabilistic p olynomial

time algorithm is signicantly faster and conceptually simpler than the known

deterministic p olynomialtime algorithm Thus we b elieve that even in the con

text of decision problems the notion of probabilistic p olynomialtime algorithms is

advantageous We note that the fundamental nature of BPP will hold even in the

rather unlikely case that it turns out that it oers no computational advantage

ie even if every problem that can b e decided in probabilistic p olynomialtime

can b e decided by a deterministic algorithm of essentially the same complexity

BPP is in the PolynomialTime Hierarchy While it may b e that BPP P

it is not known whether or not BPP is contained in NP The source of trouble

is the twosided error probability of BPP which is incompatible with the absolute

rejection of noinstances required in the denition of NP see Exercise In

view of this ignorance it is interesting to note that BPP resides in the second

level of the PolynomialTime Hierarchy ie BPP This is a corollary of

Theorem

Trivial derandomization A straightforward way of eliminating randomness

from an algorithm is trying all p ossible outcomes of its internal coin tosses collect

ing the relevant statistics and deciding accordingly This yields BPP P S P AC E

EXP which is considered the trivial derandomization of BPP In Section we

will consider various nontrivial derandomizations of BPP which are known under

various intractability assumptions The interested reader who may b e puzzled by

the connection b etween derandomization and computational diculty is referred

to Chapter

Such a result would address a fundamental question regarding the p ower of randomness By

analogy Theorem establishing that IP P S P AC E do es not diminish the imp ortance of any

of these classes

CHAPTER RANDOMNESS AND COUNTING

Nonuniform derandomization In many settings and sp ecically in the con

text of solving search and decision problems the p ower of randomization is su

p erseded by the p ower of nonuniform advice Intuitively the nonuniform advice

may sp ecify a sequence of coin tosses that is go o d for all primary inputs of a

sp ecic length In the context of solving search and decision problems such an

advice must b e go o d for each of these inputs and thus its existence is guaran

teed only if the error probability is low enough so as to supp ort a union b ound

The latter condition can b e guaranteed by errorreduction and thus we get the

following result

Theorem BPP is strictly contained in P p oly

Pro of Recall that P p oly contains undecidable problems Theorem which

are certainly not in BPP Thus we fo cus on showing that BPP P p oly By

the discussion regarding errorreduction for every S BPP there exists a de

terministic p olynomialtime algorithm A and a p olynomial p such that for every

jxj

x it holds that Pr Ax U x Using a union b ound it follows

pjxj

that Pr x f g st Ax r x Thus for every n N

p(n)

r fg

pn n

there exists a string r f g such that for every x f g it holds that

Ax r x Using such a sequence of r s as advice we obtain the desired

n S n

nonuniform machine establishing S P p oly

Digest The pro of of Theorem combines errorreduction with a simple ap

plication of the Probabilistic Metho d cf where the latter refers to proving

the existence of an ob ject by analyzing the probability that a random ob ject is

adequate In this case we sought an nonuniform advice and proved it existence

by analyzing the probability that a random advice is go o d The latter event was

analyzed by identifying the space of advice with the set of p ossible sequences of

internal coin tosses of a randomized algorithm

A probabilistic p olynomialtime primality test

Teaching note Although primality has b een recently shown to b e in P we b elieve

that the following example provides a nice illustration to the p ower of randomized

algorithms

We present a simple probabilistic p olynomialtime algorithm for deciding whether

or not a given number is a prime The only Number Theoretic facts that we use

are

Fact For every prime p each quadratic residue mo d p has exactly two square

ro ots mo d p and they sumup to p

In other contexts see eg Chapters and it suces to have an advice that is go o d on

the average where the average is taken over all relevant primary inputs

That is for every r f p g the equation x r mo d p has two solutions mo dulo p

ie r and p r

PROBABILISTIC POLYNOMIALTIME

Fact For every o dd and nonintegerpower comp osite number N each quadratic

residue mo d N has at least four square ro ots mo d N

Our algorithm uses as a blackbox an algorithm denoted sqrt that given a prime

p and a quadratic residue mo d p denoted s returns the smallest among the two

mo dular square ro ots of s There is no guarantee as to what the output is in the

case that the input is not of the aforementioned form and in particular in the case

that p is not a prime Thus we actually present a probabilistic p olynomialtime

reduction of testing primality to extracting square ro ots mo dulo a prime which is

a search problem with a promise see Section

Construction the reduction On input a natural number N do

If N is either even or an integerpower then reject

Uniformly select r f N g and set s r mo d N

Let r sqrts N If r r mo d N then accept else reject

Indeed in the case that N is comp osite the reduction invokes sqrt on an illegiti

mate input ie it makes a query that violates the promise of the problem at the

target of the reduction In such a case there is not guarantee as to what sqrt an

swers but actually a bluntly wrong answer only plays in our favor In general we

will show that if N is comp osite then the reduction rejects with probability at least

regardless of how sqrt answers We mention that there exists a probabilistic

p olynomialtime algorithm for implementing sqrt see Exercise

Prop osition Construction constitutes a probabilistic polynomialtime re

duction of testing primality to extracting square roots module a prime Further

more if the input is a prime then the reduction always accepts and otherwise it

rejects with probability at least

We stress that Prop osition refers to the reduction itself that is sqrt is viewed

as a p erfect oracle that for every prime P and quadratic residue s mo d P

returns r s such that r s mo d P Combining Prop osition with a

probabilistic p olynomialtime algorithm that computes sqrt with negligible error

probability we obtain that testing primality is in BPP

Pro of By Fact on input a prime number N Construction always accepts

b ecause in this case for every r f N g it holds that sqrtr mo d N N

fr N r g On the other hand supp ose that N is an o dd comp osite that is not

an integerpower Then by Fact each quadratic residue s has at least four

square ro ots and each of these square ro ots is equally likely to b e chosen at Step

in other words s yields no information regarding which of its mo dular square

ro ots was selected in Step Thus for every such s the probability that either

This can b e checked by scanning all p ossible p owers e f log N g and approximately

solving the equation x N for each value of e ie nding the smallest integer i such that

i N Such a solution can b e found by binary search

CHAPTER RANDOMNESS AND COUNTING

sqrts N or N sqrts N equal the ro ot chosen in Step is at most It

follows that on input a comp osite number the reduction rejects with probability

at least

Reection Construction illustrates an interesting asp ect of randomized al

gorithms or rather reductions that is the ability to hide information from a sub

routine Sp ecically Construction generates a problem instance N s without

disclosing any additional information Furthermore a correct solution to this in

stance is likely to help the reduction that is a correct answer to the instance N s

provides probabilistic evidence regarding whether N is a prime where the proba

bility space refers to the missing information regarding how s was generated

Comment Testing primality is actually in P however the deterministic al

gorithm demonstrating this fact is more complex and its analysis is even more

complicated

Onesided error The complexity classes RP and coRP

In this section we consider notions of probabilistic p olynomialtime algorithms

having onesided error The notion of onesided error refers to a natural partition of

the set of instances that is yesinstances versus noinstances in the case of decision

problems and instances having solution versus instances having no solution in the

case of search problems We fo cus on decision problems and comment that an

analogous treatment can b e provided for search problems see the second paragraph

of Section

Denition the class RP A decision problem S is in RP if there exists a

probabilistic polynomialtime algorithm A such that for every x S it holds that

Pr Ax and for every x S it holds that Pr Ax

The choice of the constant is immaterial and any other constant greater than

zero will do and yields the very same class Similarly this constant can b e

replaced by jxj for various negligible functions while preserving the class

Both facts are sp ecial cases of the robustness of the class see Exercise

Observe that RP NP see Exercise and that RP BPP by the

aforementioned errorreduction Dening coRP ff g n S S RP g note

that coRP corresp onds to the opp osite direction of onesided error probability

That is a decision problem S is in coRP if there exists a probabilistic polynomial

time algorithm A such that for every x S it holds that Pr Ax and for

every x S it holds that Pr Ax

The initials RP stands for Random Polynomialtime which fails to convey the restricted type

of error allowed in this class The only nice feature of this notation is that it is reminiscent of NP

thus reecting the fact that RP is a randomized p olynomialtime class that is contained in NP

PROBABILISTIC POLYNOMIALTIME

Relating BPP to RP

A natural question regarding probabilistic p olynomialtime algorithms refers to the

relation b etween twosided and onesided error probability For example is BPP

contained in RP Lo osely sp eaking we show that BPP is reducible to coRP

by onesided error randomized Karpreductions where the actual statement refers

to the promise problem versions of b oth classes briey dened in the following

paragraph Note that BPP is trivially reducible to coRP by twosided error

randomized Karpreductions whereas a deterministic reduction of BPP to coRP

would imply BPP coRP RP see Exercise

First we refer the reader to the general discussion of promise problems in

Section Analogously to Denition we say that the promise problem

S S is in the promise problem extension of BPP if there exists a

yes no

probabilistic polynomialtime algorithm A such that for every x S it holds that

yes

Pr Ax and for every x S it holds that Pr Ax Similarly

is in coRP if for every x S it holds that Pr Ax and for every

yes

x S it holds that Pr Ax Probabilistic reductions among promise

problems are dened by adapting the conventions of Section sp ecically

queries that violate the promise at the target of the reduction may b e answered

arbitrarily

Theorem Any problem in BPP is reducible by a onesided error randomized

Karpreduction to coRP where coRP and possibly also BPP denotes the cor

responding class of promise problems Specically the reduction always maps a

noinstance to a noinstance

It follows that BPP is reducible by a onesided error randomized Co okreduction to

RP Thus using the conventions of Section and referring to classes of promise

RP RP BPP

problems we may write BPP RP In fact since RP BPP BPP

we have BPP RP Theorem may b e paraphrased as saying that the

combination of the onesided error probability of the reduction and the onesided

error probability of coRP can account for the twosided error probability of BPP

We warn that this statement is not a triviality like and in particular

we do not know whether it holds for classes of standard decision problems rather

than for the classes of promise problems considered in Theorem

Pro of Recall that we can easily reduce the error probability of BPPalgorithms

and derive probabilistic p olynomialtime algorithms of exp onentially vanishing er

ror probability But this do es not eliminate the error even on one side alto

gether In general there seems to b e no hop e to eliminate the error unless we

either do something earthshaking or change the setting as done when allowing a

onesided error randomized reduction to a problem in coRP The latter setting can

b e viewed as a twomove randomized game ie a random move by the reduction

followed by a random move by the decision pro cedure of coRP and it enables

applying dierent quantiers to the two moves ie allowing error in one direction

in the rst quantier and error in the other direction in the second quantier

In the next paragraph which is inessential to the actual pro of we illustrate the

p otential p ower of this setting

CHAPTER RANDOMNESS AND COUNTING

Teaching note The following illustration represents an alternative way of proving

Theorem This way seems conceptual simpler but it requires a starting p oint or

rather an assumption that is much harder to establish where b oth comparisons are

with resp ect to the actual pro of of Theorem which follows the illustration

An illustration Supp ose that for some set S BPP there exists a p olynomial p and

an oline BPPalgorithm A such that for every x it holds that Pr A x r

2p (jxj)

r fg

p jxj

x that is the algorithm uses p jxj bits of randomness and

p jxj

has error probability smaller than Note that such an algorithm cannot

b e obtained by standard errorreduction see Exercise Anyhow such a small

error probability allows a partition of the string r such that one part accounts

for the entire error probability on yesinstances while the other part accounts for

the error probability on noinstances Sp ecically for every x S it holds that

p jxj

Pr r f g A x r r whereas for every x S

p (jxj)

r fg

p jxj

and every r f g it holds that Pr A x r r

p (jxj)

r fg

Thus the error on yesinstances is pushed to the selection of r whereas the

error on noinstances is pushed to the selection of r This yields a onesided error

randomized Karpreduction that maps x to x r where r is uniformly selected

p jxj

in f g such that deciding S is reduced to the coRP problem regarding

pairs x r that is decided by the online randomized algorithm A dened

def

by A x r A x r U For details see Exercise The actual pro of

p jxj

which avoids the aforementioned hypothesis follows

The actual starting p oint Consider any BPPproblem with a characteristic function

which in case of a promise problem is a partial function dened only over the

promise By standard errorreduction there exists a probabilistic p olynomialtime

algorithm A such that for every x on which is dened it holds that Pr Ax

x jxj where is a negligible function Lo oking at the corresp onding

oline algorithm A and denoting by p the p olynomial that b ounds the running

time of A we have

Pr A x r x jxj

p(jxj)

r fg

pjxj

for all suciently long xs on which is dened We show a randomized onesided

error Karpreduction of to a promise problem in coRP

The main idea As in the illustrating paragraph the basic idea is pushing the

error probability on yesinstances of to the reduction while pushing the er

ror probability on noinstances to the coRPproblem Focusing on the case that

x this is achieved by augmenting the input x with a random sequence of

mo diers that act on the randominput of algorithm A such that for a go o d

pjxj

choice of mo diers it holds that for every r f g there exists a mo dier in

this sequence that when applied to r yields r that satises A x r Indeed

not all sequences of mo diers are go o d but a random sequence will b e go o d with

high probability and bad sequences will b e accounted for in the error probability

of the reduction On the other hand using only mo diers that are p ermutations

PROBABILISTIC POLYNOMIALTIME

guarantees that the error probability on noinstances only increase by a factor that

equals the number of mo diers we use and this error probability will b e accounted

for by the error probability of the coRPproblem Details follow

The aforementioned mo diers are implemented by shifts of the set of all strings

by xed osets Thus we augment the input x with a random sequence of shifts

pjxj

denoted s s f g such that for a go o d choice of s s it holds

m m

pjxj

that for every r f g there exists an i m such that A x r s We

will show that for any yesinstance x and a suitable choice of m with very high

def

probability a random sequence of shifts is go o d Thus for A hx s s i r

A x r s it holds that with very high probability over the choice of

s s a yesinstance x is mapp ed to an augmented input hx s s i that

m m

is accepted by A with probability On the other hand the acceptance probabil

ity of augmented noinstances for any choice of shifts only increases by a factor of

m In further detailing the foregoing idea we start by explicitly stating the simple

randomized mapping to b e used as a randomized Karpreduction and next dene

the target promise problem

The randomized mapping On input x f g we set m pjxj uniformly select

s s f g and output the pair x s where s s s Note that

m m

this mapping denoted M is easily computable by a probabilistic p olynomialtime

algorithm

The promise problem We dene the following promise problem denoted

s such that jsj pjxj having instances of the form x

yes no

The yesinstances are pairs x s where s s s and m pjxj such

that for every r f g there exists an i satisfying A x r s

The noinstances are pairs x s where again s s s and m pjxj

such that for at least half of the p ossible r f g for every i it holds that

A x r s

To see that is indeed a coRP promise problem we consider the following random

ized algorithm On input x s s where m pjxj js j js j the

m m

algorithm uniformly selects r f g and accepts if and only if A x r s

for some i f mg Indeed yesinstances of are accepted with probability

whereas noinstances of are rejected with probability at least

Analyzing the reduction We claim that the randomized mapping M reduces to

with onesided error Sp ecically we will prove two claims

Claim If x is a yesinstance ie x then Pr M x

yes

Claim If x is a noinstance ie x then Pr M x

We start with Claim which is easier to establish Recall that M x x s s

where s s are uniformly and indep endently distributed in f g We note

that by Eq and x for every p ossible choice of s s f g

and every i f mg the fraction of r s that satisfy A x r s is at most

Thus for every p ossible choice of s s f g for at least half of the

CHAPTER RANDOMNESS AND COUNTING

p ossible r f g there exists an i such that A x r s holds Hence the

reduction M always maps the noinstance x ie x to a noinstance of

ie an element of

Turning to Claim which refers to x we will show shortly that in

this case with very high probability the reduction M maps x to a yesinstance of

We upp erb ound the probability that the reduction fails in case x as

follows

r f g st i A x r s Pr M x Pr

i yes s s

1 m

i A x r s Pr

i s s

1 m

r fg

Y X

A x r s Pr

i s

r fg

where the last inequality is due to Eq It follows that if x then

Pr M x Thus the randomized mapping M reduces to with

yes

onesided error on yesinstances Recalling that coRP the theorem follows

BPP is in PH The traditional presentation of the ideas underlying the pro of of

Theorem uses them for showing that BPP is in the Polynomialtime Hierarchy

where b oth classes refer to standard decision problems Sp ecically to prove that

BPP see Denition dene the p olynomialtime computable predicate

def

A x s r and observe that s r x

x s r x s r

s r x s r x

where Eq is equivalent to s r x s r Note that Claim in the pro of

of Theorem establishes that most sequences s satisfy r x s r whereas

s Similarly Claim Eq only requires the existence of at least one such

establishes that for every s most choices of r violate x s r whereas Eq

only requires that for every s there exists at least one such r We comment that

the same pro of idea yields a variety of similar statements eg BPP MA where

MA is a randomized version of NP dened in Section

Sp ecically the class MA is dened by allowing the verication algorithm V in Denition

p oly jxj

to b e probabilistic and err on noinstances that is for every x S there exists y f g

such that Pr V x y whereas for every x S and every y it holds that Pr V x y

We note that MA can b e viewed as a hybrid of the two aforementioned pairs of

conditions sp ecically each problem in MA satisfy the conjunction of Eq and Claim

Other randomized versions of NP ie variants of MA are considered in Exercise

PROBABILISTIC POLYNOMIALTIME

Zerosided error The complexity class ZPP

We now consider probabilistic p olynomialtime algorithms that never err but may

fail to provide an answer Focusing on decision problems the corresp onding class is

denoted ZPP standing for Zeroerror Probabilistic Polynomialtime The stan

dard denition of ZPP is in terms of machines that output indicating fail

ure with probability at most That is S ZPP if there exists a proba

bilistic polynomialtime algorithm A such that for every x f g it holds that

Pr Ax f x g and Pr Ax x where x if x S

S S S

and x otherwise Again the choice of the constant ie is immate

rial and errorreduction can b e p erformed showing that algorithms that yield a

meaningful answer with noticeable probability can b e amplied to algorithms that

fail with negligible probability see Exercise

Theorem ZPP RP coRP

Pro of Sketch The fact that ZPP RP as well as ZPP coRP follows by a

trivial transformation of the ZPPalgorithm that is replacing the failure indicator

by a no verdict resp yes verdict Note that the choice of what to say in

case the ZPPalgorithm fails is determined by the type of error that we are allowed

In order to prove that RP coRP ZPP we combine the two algorithm

guaranteed for a set in RP coRP The p oint is that we can trust the RP

algorithm resp coNPalgorithm in the case that it says yes resp no but

not in the case that it says no resp yes Thus we invoke b oth algorithms

and output a denite answer only if we obtain an answer that we can trust which

happ en with high probability Otherwise we output

Exp ected p olynomialtime In some sources ZPP is dened in terms of ran

domized algorithms that run in exp ected p olynomialtime and always output the

correct answer This denition is equivalent to the one we used see Exercise

Randomized LogSpace

In this section we discuss probabilistic p olynomialtime algorithms that are further

restricted such that they are allowed to use only a logarithmic amount of space

Denitional issues

When dening spaceb ounded randomized algorithms we face a problem analogous

to the one discussed in the context of nondeterministic spaceb ounded computation

see Section Sp ecically the online and the oline versions formulated in

Denition are no longer equivalent unless we restrict the oline machine to

access its randominput tap e in a unidirectional manner The issue is that in the

context of spaceb ounded computation and unlike in the case that we only care

ab out timeb ounds the outcome of the internal coin tosses in the online mo del

CHAPTER RANDOMNESS AND COUNTING

cannot b e recorded for free Bearing in mind that in the current context we wish

to mo del real algorithms rather than present a ctitious mo del that captures a

fundamental phenomena as in Section it is clear that using the online version

is the natural choice

An additional issue that arises is the need to explicitly b ound the runningtime

of spaceb ounded randomized algorithms Recall that without loss of generality

the number of steps taken by a spaceb ounded nondeterministic machine is at

most exp onential in its space complexity b ecause the shortest path b etween two

congurations in the directed graph of p ossible congurations is upp erb ounded

by its size which in turn is exp onential in the spaceb ound This reasoning fails in

the case of randomized algorithms b ecause the shortest path b etween two cong

urations do es not b ound the exp ected number of random steps required for going

from the rst conguration to the second one In fact as we shall shortly see

failing to upp erb ound the running time of logspace randomized algorithms seems

to allow them to o much p ower that is such unrestricted logspace randomized

algorithms can emulate nondeterministic logspace computations in exp onential

time The emulation consists of rep eatedly invoking the NLmachine while using

random choices in the role of the nondeterministic moves If the input is a yes

instance then in each attempt with probability at least we hit an accepting

tstep nondeterministic computation where t is p olynomial in the input length

Thus the randomized machine accepts such a yesinstance after an exp ected num

b er of trials To allow for the rejection of noinstances rather than lo oping

innitely in vain we wish to implement a counter that counts till or so and

reject the input if this number of trials have failed We need to implement such a

counter within space O log t rather than t which is easy In fact it suces to

have a randomized counter that with high probability counts to approximately

The implementation of such a counter is left to Exercise and using it

we may obtain a randomized algorithm that halts with high probability on every

input always rejects a noinstance and accepts each yesinstance with probability

at least

In light of the foregoing discussion when dening randomized logspace algo

rithms we explicitly require that the algorithms halt in p olynomialtime Mo dulo

this convention the class RL resp BPL relates to NL analogously to the relation

of RP resp BPP to NP Sp ecicially the probabilistic acceptance condition of

RL resp BPL is as in the case of RP resp BPP

Denition the classes RL and BPL We say that a randomized logspace

algorithm is admissible if it always halts in a polynomial number of steps

A decision problem S is in RL if there exists an admissible online random

ized logspace algorithm A such that for every x S it holds that Pr Ax

and for every x S it holds that Pr Ax

A decision problem S is in BPL if there exists an admissible online random

ized logspace algorithm A such that for every x S it holds that Pr Ax

and for every x S it holds that Pr Ax

PROBABILISTIC POLYNOMIALTIME

Clearly RL NL P and BPL P Note that the classes RL and BPL remain

unchanged even if we allow the algorithms to run for expected p olynomialtime and

have nonhalting computations Such algorithms can b e easily transformed into

admissible algorithms by truncating long computations while using a standard

counter which can b e implemented in logarithmicspace Also note that error

reduction is applicable in the current setting while essentially preserving b oth the

time and space b ounds

The accidental tourist sees it all

An app ealing example of a randomized logspace algorithm is presented next It

refers to the problem of deciding undirected connectivity and demonstrated that

this problem is in RL Recall that in Section we proved that this problem is

actually in L but the algorithm and its analysis were more complicated Recall

that Directed Connectivity is complete for NL under logspace reductions For

sake of simplicity we consider the following version of undirected connectivity

which is equivalent under logspace reductions to the version in which one needs

to determine whether or not the input undirected graph is connected In the

current version the input consists of a triple G s t where G is an undirected

graph s t are two vertices in G and the task is to determine whether or not s and

t are connected in G

Construction On input G s t the randomized algorithm starts a p oly jGj

long random walk at vertex s and accepts the triplet if and only if the walk passed

through the vertex t By a random walk we mean that at each step the algorithm

selects uniformly one of the neighbors of the current vertex and moves to it

Observe that the algorithm can b e implemented in logarithmic space b ecause we

only need to store the current vertex as well as the number of steps taken so far

Obviously if s and t are not connected in G then the algorithm always rejects

G s t Prop osition implies that undirected connectivity is indeed in RL

Prop osition If s and t are connected in G V E then a random walk of

length O jV j jE j starting at s passes through t with probability at least

In other words a random walk starting at s visits all vertices of the connected

comp onent of s ie it sees all that there is to see

Pro of Sketch We will actually show that if G is connected then with probability

at least a random walk starting at s visits all the vertices of G For any pair of

vertices u v let X b e a random variable representing the number of steps taken

in a random walk starting at u until v is rst encountered The reader may verify

that for every edge fu v g E it holds that EX jE j see Exercise Next

we let coverG denote the exp ected number of steps in a random walk starting at s

and ending when the last of the vertices of V is encountered Our goal is to upp er

b ound coverG Towards this end we consider an arbitrary directed cyclictour

CHAPTER RANDOMNESS AND COUNTING

C that visits all vertices in G and note that

EX jC j jE j coverG

uv C

In particular selecting C as a traversal of some spanning tree of G we conclude

that coverG jV j jE j Thus with probability at least a random walk

of length jV j jE j starting at s visits all vertices of G

Counting

We now turn to a new type of computational problems which vastly generalize

decision problems of the NPtype We refer to counting problems and more sp ecif

ically to counting ob jects that can b e eciently recognized The search and decision

versions of NP provide suitable denitions of eciently recognized ob jects which

in turn yield corresp onding counting problems

For each search problem having eciently checkable solutions ie a relation

R f g f g in PC see Denition we consider the problem of

counting the number of solutions for a given instance That is on input x

we are required to output jfy x y R gj

For each decision problem S in NP and each corresp onding verication

pro cedure V as in Denition we consider the problem of counting the

number of NPwitnesses for a given instance That is on input x we are

required to output jfy V x y gj

We shall consider these types of counting problems as well as relaxations of

these counting problems that refer to approximating the said quantities see Sec

tions and resp ectively Other related topics include problems with

unique solutions see Section and uniform generation of solutions see

Section Interestingly randomized pro cedures will play an imp ortant role in

the results regarding the aforementioned types of problems

Exact Counting

In continuation to the foregoing discussion we dene the class of problems con

cerned with counting eciently recognized ob jects Recall that PC denotes the

class of search problems having p olynomially long solutions that are eciently

checkable see Denition

Denition counting eciently recognized ob jects P The class P

consists of al l functions that count solutions to a search problem in PC That is

f f g N is in P if there exists R PC such that for every x it holds

that f x jR xj where R x fy x y R g In this case we say that f is the

counting problem asso ciated with R and denote the latter by R ie R f

COUNTING

Every decision problem in NP is Co okreducible to P b ecause every such prob

lem can b e cast as deciding membership in S fx jR xj g for some R PC

see Section It also holds that BPP is Co okreducible to P see Exer

cise The class P is sometimes dened in terms of decision problems as is

implicit in the following prop osition

Prop osition a decisional version of P For any f P deciding mem

def

fx N f x N g is computationally equivalent to computing f bership in S

Actually the claim holds for any function f f g N for which there exists a

pjxj

p olynomial p such that for every x f g it holds that f x

Pro of Since the relation R vouching for f P ie f x jR xj is p oly

nomially b ounded there exists a p olynomial p such that for every x it holds that

pjxj

f x Deciding membership in S is easily reduced to computing f ie

we accept the input x N if and only if f x N Computing f is reducible to

deciding S by using a binary search see Exercise This relies on the fact that

on input x and oracle access to S we can determine whether or not f x N by

pjxj

making the query x N Note that we know a priori that f x

The counting class P is also related to the problem of enumerating all p ossible

solutions to a given instance see Exercise

On the p ower of P

As indicated NP BPP is easily reducible to P Furthermore as stated in

Theorem the entire PolynomialTime Hierarchy as dened in Section is

Co okreducible to P ie PH P On the other hand any problem in P

is solvable in p olynomial space and so P P S P AC E

Theorem Every set in PH is Cookreducible to P

We do not present a pro of of Theorem here b ecause the known pro ofs are

rather technical Furthermore one main idea underlying these pro ofs app ears in

a more clear form in the pro of of Theorem Nevertheless in Section F we

present a pro of of a related result which implies that PH is reducible to P via

randomized Karpreductions

Completeness in P

The denition of P completeness is analogous to the denition of NP completeness

That is a counting problem f is P complete if f P and every problem in P

is Cookreducible to f

We claim that the counting problems asso ciated with the NPcomplete problems

presented in Section are all P complete We warn that this fact is not

due to the mere NPcompleteness of these problems but rather to an additional

prop erty of the reductions establishing their NPcompleteness Sp ecically the

Karpreductions that were used or variants of them have the extra prop erty of

preserving the number of NPwitnesses as captured by the following denition

CHAPTER RANDOMNESS AND COUNTING

Denition parsimonious reductions Let R R PC and let g be a Karp

reduction of S fx R x g to S fx R x g where R x fy

R R

x y R g and R x fy x y R g We say that g is parsimonious with

resp ect to R and R if for every x it holds that jR xj jR g xj In such a case

we say that g is a parsimonious reduction of R to R

We stress that the condition of b eing parsimonious refers to the two underlying

The requirement relations R and R and not merely to the sets S and S

R R

that g is a Karpreduction is partially redundant b ecause if g is p olynomialtime

computable and for every x it holds that jR xj jR g xj then g constitutes a

Sp ecically jR xj jR g xj implies that jR xj Karpreduction of S to S

R R

The reader may ie x S if and only if jR g xj ie g x S

R R

easily verify that the Karpreduction underlying the pro of of Theorem as well

as many of the reductions used in Section are parsimonious see Exercise

Theorem Let R PC and suppose that every search problem in PC is par

simoniously reducible to R Then the counting problem associated with R is P

complete

Pro of Clearly the counting problem asso ciated with R denoted R is in P

To show that every f P is reducible to f we consider the relation R PC

that is counted by f that is R f Then by the hypothesis there exists

a parsimonious reduction g of R to R This reduction also reduces R to R

sp ecically R x R g x for every x

Corollaries As an immediate corollary of Theorem we get that counting

the number of satisfying assignments to a given CNF formula is P complete

Similar statement hold for all the other NPcomplete problems mentioned in Sec

tion and in fact for all NPcomplete problems listed in These corollaries

follow from the fact that all known reductions among natural NPcomplete prob

lems are either parsimonious or can b e easily mo died to b e so

We conclude that many counting problems asso ciated with NPcomplete search

problems are P complete It turns out that also counting problems asso ciated

with eciently solvable search problems may b e P complete

Theorem There exist P complete counting problems that are associated

with eciently solvable search problems That is there exists R PF see Deni

tion such that R is P complete

Pro of Consider the relation R consisting of pairs such that is a DNF

dnf

formula and is an assignment satisfying it Note that the search problem of R

dnf

is easy to solve eg by picking an arbitrary truth assignment that satises the

rst term in the input formula To see that R is P complete consider the

dnf

following reduction from R which is P complete by Theorem Given

SAT

a CNF formula transform into a DNF formula by applying deMorgans

Law and return R where n denotes the number of variables in

dnf

resp

COUNTING

Reections We note that Theorem is not established by a parsimonious

reduction and refer the reader to more artical P complete problems presented

in Exercise This fact should not come as a surprise b ecause a parsimonious

fx y st x y R g is reducible reduction of R to R implies that S

to S fx y st x y R g where in our case S is NPComplete while

R R

S P since R PF Nevertheless the pro of of Theorem is related to

the hardness of some underlying decision problem ie the problem of deciding

whether a given DNF formula is a tautology ie whether R But

dnf

do es there exist a P complete problem that is not based on some underlying

NPcomplete decision problem Amazingly enough the answer is p ositive

Theorem Counting the number of perfect matchings in a bipartite graph is

P complete

Equivalently see Exercise the problem of computing the p ermanent of ma

trices with entries is P complete Recall that the p ermanent of an nbyn

matrix M m denoted permM equals the sum over all p ermutations

m Theorem is proven by comp osing the of n of the pro ducts

i i

following two manytoone reductions asserted in Prop ositions and

resp ectively and using the fact that R is P complete see Theorem

SAT

and Exercise Needless to say the resulting reduction is not parsimonious

Prop osition The counting problem of SAT ie R is reducible to

SAT

computing the permanent of integer matrices Furthermore there exists an even

integer c and a nite set of integers I such that on input a CNF formula the

reduction produces an integer matrix M with entries in I such that permM

c R where m denotes the number of clauses in

SAT

The original pro of of Prop osition uses c and I f g It

can b e shown see Exercise which relies on Theorem that for every

integer n that is relatively prime to c computing the p ermanent mo dulo n

is NPhard under randomized reductions Thus using the case of c this

means that computing the p ermanent mo dulo n is NPhard for any o dd n In

contrast computing the p ermanent mo dulo which is equivalent to computing

the determinant mo dulo is easy ie can b e done in p olynomialtime and even

in NC Thus assuming NP BPP Prop osition cannot hold for an o dd c

b ecause by Exercise it would follow that computing the p ermanent mo dulo

is NPHard We also note that assuming P NP Prop osition cannot

p ossibly hold for a set I containing only nonnegative integers see Exercise

Prop osition Computing the permanent of integer matrices is reducible to

computing the permanent of matrices Furthermore the reduction maps any

integer matrix A into a matrix A such that the permanent of A can be easily

computed from A and the permanent of A

See Section G for basic terminology regarding graphs

CHAPTER RANDOMNESS AND COUNTING

Teaching note We do not recommend presenting the pro ofs of Prop ositions

and in class The highlevel structure of the pro of of Prop osition has the

avor of some sophisticated reductions among NPproblems but the crucial p oint is the

existence of adequate gadgets We do not know of a highlevel argument establishing

the existence of such gadgets nor of any intuition as to why such gadgets exist Instead

the existence of such gadgets is proved by a design that is b oth highly nontrivial and ad

hoc in nature Thus the pro of of Prop osition b oils down to a complicated design

problem that is solved in a way that has little p edagogical value In contrast the pro of

of Prop osition uses two simple ideas that can b e useful in other settings With

suitable hints this pro of can b e used as a go o d exercise

Pro of of Prop osition We will use the corresp ondence b etween the

p ermanent of a matrix A and the sum of the weights of the cycle covers of the

weighted directed graph represented by the matrix A A cycle cover of a graph

is a collection of simple vertexdisjoint directed cycles that covers all the graphs

vertices and its weight is the pro duct of the weights of the corresp onding edges

The SWCC of a weighted directed graph is the sum of the weights of all its cycle

covers

Given a CNF formula we construct a directed weighted graph G such that

the SWCC of G equals equals c R where c is a universal constant and

SAT

m denotes the number of clauses in We may assume without loss of generality

that each clause of has exactly three literals which are not necessarily distinct x

+x +x

+x -x

Figure Tracks connecting gadgets for the reduction to cycle cover

We start with a highlevel description of the construction that refers to clause

gadgets each containing some internal vertices and internal weighted edges which

are unspecied at this point In addition each gadget has three pairs of designated

vertices one pair p er each literal app earing in the clause where one vertex in the

Indeed the conjecture that such gadgets exist can only b e attributed to ingenuity

Here a simple cycle is a strongly connected directed graph in which each vertex has a single

incoming resp outgoing edge In particular selflo ops are allowed

COUNTING

pair is designated as an entry vertex and the other as an exit vertex The graph

G consists of m such gadgets one p er each clause of and n auxiliary vertices

one p er each variable of as well as some additional directed edges each having

weight Sp ecically for each variable we introduce two tracks one p er each of

the p ossible literals of this variable The track asso ciated with a literal consists of

directed edges each having weight that form a simple cycle passing through

the corresp onding auxiliary vertex as well as through the designated vertices that

corresp ond to the o ccurrences of this literal in the various clauses Sp ecically for

each such o ccurrence the track enters the corresp onding clause gadget at the entry

vertex corresp onding to this literal and exits at the corresp onding exitvertex If

a literal do es not app ear in then the corresp onding track is a selflo op on the

corresp onding variable See Figure showing the two tracks of a variable x that

o ccurs p ositively in three clauses and negatively in one clause The entryvertices

resp exitvertices are drawn on the top resp b ottom part of each gadget

On the left is a gadget with the track edges adjacent to it as in the

real construction On the right is a gadget and four out of the nine

external edges two of which are nice used in the analysis

Figure External edges for the analysis of the clause gadget

For the purp ose of stating the desired prop erties of the clause gadget we aug

ment the gadget by nine external edges of weight one p er each pair of not

necessarily matching entry and exit vertices such that the edge go es from the

exitvertex to the entryvertex see Figure We stress that this is an auxiliary

construction that diers from and yet is related to the use of gadgets in the forego

ing construction of G The three edges that link the designated pairs of vertices

that corresp ond to the three literals are called nice We say that a collection of

edges C eg a collection of cycles uses the external edges S if the intersection of

C with the set of the nine external edges equals S We p ostulate the following

three prop erties of the clause gadget

The sum of the weights of all cycle covers of the gadget that do not use any

external edge ie use the empty set of external edges equals zero

CHAPTER RANDOMNESS AND COUNTING

Let V S denote the set of vertices incident to S and say that S is nice if it

is nonempty and the vertices in V S can b e p erfectly matched using nice

edges Then there exists a constant c indeed the one p ostulated in the

prop ositions claim such that for any nice set S the sum of the weights of

all cycle covers that use the external edges S equals c

For any nonnice set S of external edges the sum of the weights of all cycle

covers that use the external edges S equals zero

Note that the foregoing three cases exhaust all the p ossible ones and that the set

of external edges used by a cycle cover must b e a matching ie these edges are

vertex disjoint Using the foregoing conditions it can b e shown that each satisfying

assignment of contributes exactly c to the SWCC of G see Exercise It

follows that the SWCC of G equals c R

SAT

Having established the validity of the abstract reduction we turn to the imple

mentation of the clause gadget The rst implementation is a Deus ex Machina

with a corresp onding adjacency matrix depicted in Figure Its validity for the

value c can b e veried by computing the p ermanent of the corresp onding

submatrices see analogous analysis in Exercise

A more structured implementation of the clause gadget is depicted in Figure

which refers to a hexagon b ox to b e implemented later The b ox contains several

vertices and weighted edges but only two of these vertices called terminals are

connected to the outside and are shown in Figure The clause gadget consists

of ve copies of this b ox where three copies are designated for the three literals

of the clause and are marked LB LB and LB as well as additional vertices

and edges shown in Figure In particular the clause gadget contains the six

aforementioned designated vertices ie a pair of entry and exit vertices p er each

literal two additional vertices shown at the two extremes of the gure and some

edges all having weight Each designated vertex has a selflo op and is incident

to a single additional edge that is outgoing resp incoming in case the vertex

is an entryvertex resp exitvertex of the gadget The two terminals of each

b ox that is asso ciated with some literal are connected to the corresp onding pair

of designated vertices eg the outgoing edge of entry is incident at the right

terminal of the b ox LB Note that the ve b oxes reside on a directed path going

from left to right and the only edges going in the opp osite direction are those

drawn b elow this path

In continuation to the foregoing we wish to state the desired prop erties of the

b ox Again we do so by considering the augmentation of the b ox by external edges

of weight incident at the sp ecied vertices In this case see Figure we

have a pair of antiparallel edges connecting the two terminals of the b ox as well as

two selflo ops one on each terminal We p ostulate the following three prop erties

of the b ox

Clearly any nonempty set of nice edges is a nice set Thus a singleton set is nice if and only

if the corresp onding edge is nice On the other hand any set S of three vertexdisjoint external

edges is nice b ecause V S has a p erfect matching using all three nice edges Thus the notion

of nice sets is nontrivial only for sets of two edges Such a set S is nice if and only if V S

consists of two pairs of corresp onding designated vertices

COUNTING

The gadget uses eight vertices where the rst six are the designated

entry and exit vertices The entryvertex resp exitvertex asso ci

ated with the i literal is numbered i resp i The corresp onding

adjacency matrix follows

B C

Note that the edge can b e contracted but the resulting

vertex graph will not b e consistent with our inessentially stringent

denition of a gadget by which the six designated vertices should b e

distinct

Figure A Deus ex Machina clause gadget for the reduction to cycle cover

The sum of the weights of all cycle covers of the b ox that do not use any

external edge equals zero

There exists a constant b in our case b such that for each of the two

antiparallel edges the sum of the weights of all cycle covers that use this

edge equals b

For any nonempty set S of the selflo ops the sum of the weights of all

cycle covers of the b ox that use S equals zero

Note that the foregoing three cases exhaust all the p ossible ones It can b e shown

that the conditions regarding the b ox imply that the construction presented in

Figure satises the conditions that were p ostulated for the clause gadget see

Exercise Sp ecically we have c b As for b ox itself a smaller Deus ex

Machina is provided by the following by adjacency matrix

C B

where the two terminals corresp ond to the rst and the fourth vertices Its va

lidity for the value b can b e veried by computing the p ermanent of the

corresp onding submatrices see Exercise

CHAPTER RANDOMNESS AND COUNTING

entry1 entry2 entry3

LB1 LB2 LB3

exit1 exit2 exit3

Figure A structured clause gadget for the reduction to cycle cover

On the left is a b ox with p otential edges adjacent to it as in the

gadget construction On the right is a b ox and the four external

edges used in the analysis

Figure External edges for the analysis of the b ox

Pro of of Prop osition The pro of pro ceeds in two steps In the rst

step we show that computing the p ermanent of integer matrices is reducible to

computing the p ermanent of nonnegative matrices This reduction pro ceeds as

follows For an nbyn integer matrix A a let kAk max ja j

ij ij ij

ij n

and Q n kAk We note that given A the value Q can b e computed

A A

in p olynomialtime and in particular log Q n log kAk Given the matrix A

the reduction constructs the nonnegative matrix A a mo d Q ie

ij A

ij n

the entries of A are in f Q g queries the oracle for the p ermanent of

def

A and outputs v permA mo d Q if v Q and Q v otherwise

A A A

The key observation is that

permA permA mo d Q while jpermAj n kAk Q

A A

COUNTING

Thus permA mo d Q which is in f Q g determines permA We

A A

note that permA is likely to b e much larger than Q jpermAj it is merely

that permA and permA are equivalent mo dulo Q

In the second step we show that computing the p ermanent of nonnegative ma

trices is reducible to computing the p ermanent of matrices In this reduction

we view the computation of the p ermanent as the computation of the sum of the

weights of the cycle covers SWCC of the corresp onding weighted directed graph

see pro of of Prop osition Thus we reduce the computation of the SWCC of

directed graphs with nonnegative weights to the computation of the SWCC of un

weighted directed graphs with no parallel edges which corresp ond to matrices

The reduction is via lo cal replacements that preserve the value of the SWCC These

lo cal replacements combine the following two lo cal replacements which preserve

the SWCC

Replacing an edge of weight w w by a path of length t ie t

internal no des with the corresp onding weights w w and selflo ops with

weight on all internal no des

Note that a cyclecover that uses the original edge corresp onds to a cycle

cover that uses the entire path whereas a cyclecover that do es not use the

original edge corresp onds to a cyclecover that uses all the selflo ops

w by t parallel edge paths such that Replacing an edge of weight w

the rst edge on the i path has weight w the second edge has weight

and the intermediate no de has a selflo op with weight Paths of length

two are used b ecause parallel edges are not allowed

Note that a cyclecover that uses the original edge corresp onds to a collection

of cyclecovers that use one out of the t paths and the selflo ops of all other

intermediate no des whereas a cyclecover that do es not use the original edge

corresp onds to a cyclecover that uses all the selflo ops

In particular writing the p ositive integer w having binary expansion

jw j

we may apply the additive replacement for the sum over fi as

g next the pro duct replacement for each and nally the additive

replacement for Applying this pro cess to the matrix A obtained in the

rst step we eciently obtain a matrix A with entries such that permA

permA In particular the dimension of A is p olynomial in the length of the

binary representation of A which in turn is p olynomial in the length of the binary

representation of A Combining the two reductions steps the prop osition follows

Approximate Counting

Having seen that exact counting for relations in PC seems even harder than

solving the corresp onding search problems we turn to relaxations of the counting

problem Before fo cusing on relative approximation we briey consider approxi

mation with large additive deviation

CHAPTER RANDOMNESS AND COUNTING

Let us consider the counting problem asso ciated with an arbitrary R PC

Without loss of generality we assume that all solutions to nbit instances have

the same length n where indeed is a p olynomial We rst note that while

it may b e hard to compute R given x it is easy to approximate R x up to

jxj

an additive error of by randomly samplying p otential solutions for

x Indeed such an approximation is very rough but it is not trivial and in fact

we do not know how to obatin it deterministically In general we can eciently

pro duce at random an estimate of R x that with high probability deviates

from the correct value by at most an additive term that is related to the absolute

jxj

upp erb ound on the number of solutions ie

Prop osition approximation with large additive deviation Let R PC

n n

and be a polynomial such that R f g f g Then for every

polynomial p there exists a probabilistic polynomialtime algorithm A such that for

every x f g and it holds that

jxj

Pr jAx R xj pjxj

As usual is presented to A in binary and hence the running time of Ax is

upperbounded by p oly jxj log

Pro of Sketch On input x and algorithm A sets t pjxj log selects

uniformly y y and outputs jfi x y R gjt

t i

Discussion Prop osition is meaningful in the case that R x pjxj

jxj

holds for some xs But otherwise a trivial approximation ie outputting

the constant value zero meets the b ound of Eq In contrast to this no

tion of additive approximation a relative factor approximation is typically more

meaningful Sp ecically we will b e interested in approximating R x upto a

constant factor or some other reasonable factor In x we consider a natu

ral P complete problem for which such a relative approximation can b e obtained

in probabilistic p olynomialtime We do not exp ect this to happ en for every count

ing problem in P b ecause a relative approximation allows for distinguishing

instances having no solution from instances that do have solutions ie deciding

membership in S is reducible to a relative approximation of R Thus rela

tive approximation for all P is at least as hard as deciding all problems in NP

but in x we show that the former is not harder than the latter that is

relative approximation for any problem in P can b e obtained by a randomized

Co okreduction to NP Before turning to these results let us state the underlying

denition and actually strengthen it by requiring approximation to within a factor

of for

We refrain from formally dening an F factor approximation for an arbitrary F although

we shall refer to this notion in several informal discussions There are several ways of dening the

aforementioned term and they are all equivalent when applied to our informal discussions For

example an F factor approximation of R may mean that with high probability the output Ax

satises RxF jxj Ax F jxj Rx Alternatively we may require that Rx

Ax F jxj Rx or alternatively that RxF jxj Ax Rx

COUNTING

Denition approximation with relative deviation Let f f g N

and N A randomized process is called an approximator of f

if for every x it holds that

Pr jx f xj jxj f x jxj

We say that f is eciently approximable or just approximable if

there exists a probabilistic polynomialtime algorithm A that constitute an

approximator of f

The error probability of the latter algorithm A which has error probability

can b e reduced to by O log rep etitions see Exercise Typically the

running time of A will b e p olynomial in and is called the deviation parameter

Relative approximation for R

dnf

In this subsection we present a natural P complete problem for which constant

factor approximation can b e found in probabilistic p olynomialtime Stronger re

sults regarding unnatural problems app ear in Exercise

Consider the relation R consisting of pairs such that is a DNF

dnf

formula and is an assignment satisfying it Recall that the search problem

of R is easy to solve and that the pro of of Theorem establishes that

dnf

R is P complete via a nonparsimonious reduction Still there exists a

dnf

probabilistic p olynomialtime algorithm that provides a constant factor approxi

mation of R We warn that the fact that R is P complete via a non

dnf dnf

parsimonious reduction means that the constant factor approximation for R

dnf

do es not seem to imply a similar approximation for all problems in P In fact we

should not exp ect each problem in P to have a probabilistic p olynomialtime

constantfactor approximation algorithm b ecause this would imply NP BPP

since a constant factor approximation allows for distinguishing the case in which

the instance has no solution from the case in which the instance has a solution

The following algorithm is actually a deterministic reduction of the task of

approximating R to an additive deviation approximation of the

dnf

C where type provided in Prop osition Consider a DNF formula

each C f g f g is a conjunction Actually we will deal with the more

general problem in which we are implicitly given m subsets S S f g

and wish to approximate j S j In our case each S is the set of assignments

i i

satisfying the conjunction C In general we make two computational assumptions

regarding these sets letting ecient mean implementable in time p olynomial in

n m

Given i m one can eciently determine jS j

i h

S s Given i m and J m one can eciently approximate Pr

j sS

j J

up to an additive deviation of p olyn m

CHAPTER RANDOMNESS AND COUNTING

These assumptions are satised in our setting where S C see Exer

S j is that cise Now the key observation towards approximating j

m m m

X X

S jS j Pr s S S n S

j i sS j i i

j i i j i i i

and that the probabilities in Eq can b e approximated by the second assump

tion This leads to the following algorithm where denotes the desired deviation

parameter ie we wish to obtain j S j

Construction Let m For i to m do

Using the rst assumption compute jS j

def

s Pr Using the second assumption obtain pe p where p

sS i i i

def

pe jS j S Set a

i i j i

j i

Output the sum of the a s

N Let N p jS j We are interested in the quality of the approximation to

i i i i

S P

j S j provided by a Using a p jS j N jS j for all is we

i i i i i i i

i i

P S P P P P

N and S j m jS j m j jS j Using N a have

i i i i i i

P P

N Thus we obtain the following result see a m we get

i i

Exercise

Prop osition For every positive polynomial p the counting problem of R

dnf

is eciently papproximable

Using the reduction presented in the pro of of Theorem we conclude that the

number of unsatisfying assignments to a given CNF formula is eciently p

approximable We warn however that the number of satisfying assignments to

such a formula is not eciently approximable This concurs with the general

phenomenon by which relative approximation may be possible for one quantity but

not for the complementary quantity Needless to say such a phenomenon do es not

o ccur in the context of additivedeviation approximation

Relative approximation for P

Recall that we cannot exp ect to eciently approximate every P problem where

throughout the rest of this section approximation is used as a shorthand for rel

ative approximation as in Denition Sp ecically eciently approximating

R yields an ecient algorithm for deciding membership in S fx R x g

Thus at b est we can hop e that approximating R is not harder than deciding S

ie that approximating R is reducible in p olynomialtime to S This is indeed

the case for every NPcomplete problem ie if S is NPcomplete More gener

ally we show that approximating any problem in P is reducible in probabilistic

p olynomialtime to NP

COUNTING

Theorem For every R PC and positive polynomial p there exists a prob

abilistic polynomialtime oracle machine that when given oracle access to NP

constitutes a p approximator of R where is a negligible function eg

Recall that it suces to provide a p approximator of R for any constant

b ecause error reduction is applicable in this context see Exercise

Also it suces to provide a approximator for every problem in P see

Exercise

Pro of Given x we show how to approximate jR xj to within some constant

factor The desired papproximation can b e obtained as in Exercise

We may also assume that R x by starting with the query is x in S

and halting with output if the answer is negative Without loss of generality

we assume that R x f g where p oly jxj We fo cus on nding some

i i

i f g such that jR xj

We pro ceed in iterations For i we nd out whether or not

i i

jR xj If the answer is p ositive then we halt with output and otherwise

we pro ceed to the next iteration Indeed if we were able to obtain correct answers

i i i

to these queries then the output would satisfy jR xj

Needless to say the key issue is how to check whether jR xj The main

idea is to use a random sieve on the set R x such that each element passes the

i i

sieve with probability Thus we exp ect jR xj elements of R x to pass

the sieve Assuming that the number of elements in R x that pass the random

i i

sieve is indeed bjR xj c it holds that jR xj if and only if some element of

R x passes the sieve Assuming that the sieve can b e implemented eciently the

question of whether or not some element in R x passed the sieve is of an NP

type and thus can b e referred to our NPoracle Combining b oth assumptions

we may implement the foregoing pro cess by pro ceeding to the next iteration as

long as some element of R x passes the sieve Furthermore this implementation

will provide a reasonably go o d approximation even if the number of elements in

R x that pass the random sieve is only approximately equal to jR xj In fact

the level of approximation that this implementation provides is closely related to

the level of approximation that is provided by the random sieve Details follow

Implementing a random sieve The random sieve is implemented by using a family

of hashing functions see Section D Sp ecically in the i iteration we use a

i i

family H such that each h H has a p oly bit long description and maps bit

long strings to ibit long strings Furthermore the family is accompanied with

an ecient evaluation algorithm ie mapping adequate pairs h x to hx and

satises for every S f g

i i

Pr jfy S hy gj jS j

jS j

see Lemma D The random sieve wil l let y pass if and only if hy Indeed

this random sieve is not as p erfect as we assumed in the foregoing discussion but

Eq suggests that in some sense this sieve is go o d enough

CHAPTER RANDOMNESS AND COUNTING

Implementing the queries Recall that for some x i and h H we need to de

termine whether fy R x hy g This type of question can b e cast as

membership in the set

def

fx i h y st x y R hy g S

Using the hypotheses that R PC and that the family of hashing functions has an

ecient evaluation algorithm it follows that S is in NP

The actual procedure On input x S and oracle access to S we pro ceed in

R RH

iterations starting with i and halting at i if not b efore where denotes

the length of the p otential solutions for x In the i iteration where i we

uniformly select h H and query the oracle on whether or not x i h S

If the answer is negative then we halt with output and otherwise we pro ceed to

the next iteration using i i Needless to say if we reach the last iteration

ie i then we just halt with output

Indeed we have ignored the case that x S which can b e easily handled by

a minor mo dication of the foregoing pro cedure Sp ecically on input x we rst

query S on x and halt with output if the answer is negative Otherwise we

pro ceed as in the foregoing pro cedure

The analysis We upp erb ound separately the probability that the pro cedure out

puts a value that is to o small and the probability that it outputs a value that is

to o big In light of the foregoing discussion we may assume that jR xj and

let i blog jR xj c

The probability that the pro cedure halts in a specic iteration i i equals

Pr jfy R x hy gj which in turn is upp erb ounded by

jR xj using Eq with Thus the probability that the pro

cedure halts before iteration i is upp erb ounded by jR xj

which in turn is less than b ecause i log jR xj Thus with prob

ability at least the output is at least jR xj b ecause i

log jR xj

The probability that the pro cedure does not halt in iteration i i equals

Pr i jfy R x hy gj which in turn is upp erb ounded by

where jR xj using Eq with

Thus the probability that the pro cedure do es not halt by iteration i is

upp erb ounded by b ecause i log jR xj Thus with

probability at least the output is at most jR xj b ecause

i log jR xj

Thus with probability at least the foregoing pro cedure outputs

a value v such that v jR xj v Reducing the deviation by using the ideas

presented in Exercise and reducing the error probability as in Exercise

the theorem follows

A b etter b ound can b e obtained by using the hypothesis that for every y when h is uniformly

i i

the value of hy is uniformly distributed in f g In this case Pr jfy selected in H

i i i

Rx hy gj is upp erb ounded by E jfy Rx hy gj jRxj

COUNTING

Perspective The key observation underlying the pro of Theorem is that

while even with the help of an NPoracle we cannot directly test whether the

number of solutions is greater than a given number we can test with the help

of an NPoracle whether the number of solutions that survive a random sieve

is greater than zero If fact we can also test whether the number of solutions

that survive a random sieve is greater than a small number where small means

p olynomial in the length of the input see Exercise That is the complexity

of this test is linear in the size of the threshold and not in the length of its binary

description Indeed in many settings it is more advantageous to use a threshold

that is p olynomial in some eciency parameter rather than using the threshold

zero examples app ear in x and in

Searching for unique solutions

A natural computational problem regarding search problems which arises when

discussing the number of solutions is the problem of distinguishing instances having

a single solution from instances having no solution or nding the unique solution

whenever such exists We mention that instances having a single solution facilitate

numerous arguments see for example Exercise and x Formally

searching for and deciding the existence of unique solutions are dened within the

framework of promise problems see Section

Denition search and decision problems for unique solution instances

The set of instances having unique solutions with respect to the binary relation R

def def

fx jR xj g where R x fy x y R g As usual we is dened as US

def

f g n S fx jR xj g S denote S fx jR xj g and

R R R

The problem of nding unique solutions for R is dened as the search problem

R with promise US S see Denition

R R

In continuation to Denition the candid searching for unique solutions

for R is dened as the search problem R with promise US

The problem of deciding unique solution for R is dened as the promise problem

S see Denition US

R R

Interestingly in many natural cases the promise do es not make any of these prob

lems any easier than the original problem That is for all known NPcomplete

problems the original problem is reducible in probabilistic p olynomialtime to the

corresp onding unique instances problem

Theorem Let R PC and suppose that every search problem in PC is par

simoniously reducible to R Then solving the search problem of R resp deciding

membership in S is reducible in probabilistic polynomialtime to nding unique

S Furthermore there solutions for R resp to the promise problem US

R R

exists a probabilistic polynomialtime computable mapping M such that for ev

S it holds that M x S whereas for every x S it holds that ery x

R R R

Pr M x US p olyjxj R

CHAPTER RANDOMNESS AND COUNTING

We highlight the hypothesis that R is PC complete via parsimonious reductions is

crucial to Theorem see Exercise The large but b oundedaway from

error probability of the randomized Karpreduction M can b e reduced by rep e

titions yielding a randomized Co okreduction with exp onentially vanishing error

probability Note that the resulting reduction may make many queries that violate

the promise and still yields the correct answer with high probability by relying

on queries that satisfy the promise Sp ecically in the case of search problems we

avoid wrong solutions by checking each solution obtained while in the case of deci

S it holds that M x S sion problems we rely on the fact that for every x

R R

Pro of As in the pro of of Theorem the idea is to apply a random sieve on

R x this time with the hop e that a single element survives Sp ecically if we let

each element passes the sieve with probability approximately jR xj then with

constant probability a single element survives and we shall obtain an instance with

a unique solution Sieving will b e p erformed by a random function selected in an

adequate hashing family see Section D A couple of questions arise

How do we get an approximation to jR xj Note that we need such an

approximation in order to determine the adequate hashing family Indeed

we may just invoke Theorem but this will not yield a manytoone

reduction Instead we just select m f p olyjxjg uniformly and note

that if jR xj then Pr m dlog jR xj e p olyjxj Next we

randomly map x to x m h where h is uniformly selected in an adequate

hashing family

How does the question of whether a single element of R x pass the random

sieve translate to an instance of the uniquesolution problem for R Recall

that in the pro of of Theorem the nonemptiness of the set of element of

R x that pass the sieve dened by h was determined by checking mem

b ership of x m h in S NP dened in Eq Furthermore the

number of NPwitnesses for x m h S equals the number of elements

of R x that pass the sieve Using the parsimonious reduction of S to S

RH R

which is guaranteed by the theorems hypothesis we obtained the desired

instance

Note that in case R x the aforementioned mapping always generates a no

instance of S and thus of S Details follow

RH R

Implementation ie the mapping M As in the pro of of Theorem we as

sume without loss of generality that R x f g where p oly jxj We

m m

start by uniformly selecting m f g and h H where H is a family

of eciently computable and pairwiseindep endent hashing functions see Deni

tion D mapping bit long strings to mbit long strings Thus we obtain an

instance x m h of S NP such that the set of valid solutions for x m h

equals fy R x hy g Using the parsimonious reduction g of S to S

RH R

we map x m h to g x m h and it holds that jfy R x hy gj equals

jR g x m hj To summarize on input x the randomized mapping M outputs the

COUNTING

def

are uniformly instance M x g x m h where m f g and h H

selected

S it holds that Pr M x S Assuming The analysis Note that for any x

R R

that x S with probability exactly it holds that m m where

R x

def

dlog jR xj e In this case for a uniformly selected h H we lower m

b ound the probability that fy R x hy g is a singleton Using the

InclusionExclusion Principle we have

gj jfy R x hy Pr

m m

x x

m m

x x

gj jfy R x hy gj Pr jfy R x hy Pr

hH hH

X X

m m

x x

m m

x x

hy hy Pr hy Pr

hH hH

y y Rx y Rx

1 2

jR xj

m m

x x

jR xj

where the last equality is due to the pairwise indep endence prop erty Using

m m

x x

jR xj it follows that Eq is lowerbounded by

Thus Pr M x US and the theorem follows

Comment Theorem is sometimes stated as referring to the unique solution

problem of SAT In this case and when using a sp ecic family of pairwise indep en

dent hashing functions the use of the parsimonious reduction can b e avoided For

details see Exercise

Uniform generation of solutions

We now turn to a new type of computational problems which may b e viewed as

a straining of search problems We refer to the task of generating a uniformly

distributed solution for a given instance rather than merely nding an adequate

solution Needless to say by denition algorithms solving this uniform gener

ation task must b e randomized Focusing on relations in PC we consider two

versions of the problem which dier by the level of approximation provided for the

desired uniform distribution

Denition uniform generation Let R PC and S fx jR xj g

and let be a probabilistic process

We say that solves the uniform generation problem of R if on input x S

the process outputs either an element of R x or a special symbol denoted

such that Pr x R x and for every y R x it holds that

Pr x y j x R x jR xj

Note that a probabilistic algorithm running in strict p olynomialtime is not able to output a

p erfectly uniform distribution on sets of certain sizes Sp ecically referring to the standard mo del

that allows only for uniformly selected binary values such algorithms cannot output a p erfectly

uniform distribution on sets having cardinality that is not a p ower of two

CHAPTER RANDOMNESS AND COUNTING

For N we say that solves the approximate uniform

generation problem of R if on input x S the distribution x is jxj

close to the uniform distribution on R x

In both cases without loss of generality we may require that if x S then

Pr x More generally we may require that never outputs a string

not in R x

Note that the error probability of uniform generation as in Item can b e made

exp onentially vanishing in jxj by employing errorreduction In contrast we are

not aware of any general way of reducing the deviation of an approximate uniform

generation pro cedure as in Item

In x we show that for many search problems approximate uniform gener

ation is computationally equivalent to approximate counting In x we present

a direct approach for solving the uniform generation problem of any search problem

in PC by using an oracle to NP

Relation to approximate counting

We show that for many natural search problems in PC the approximate counting

problem asso ciated with R is computationally equivalent to approximate uniform

generation with resp ect to R Sp ecically we refer to search problems R PC

def

such that R x y fy x y y R g is strongly parsimoniously reducible to

R where a strongly parsimonious reduction of R to R is a parsimonious reduction g

that is coupled with an eciently computable mapping of pairs g x y R to

pairs x hx y R ie h is eciently computable and hx is a mapping

of R g x to R x Note that for many natural search problems R eg the

search problem of SAT and Perfect Matching the corresp onding R is strongly

parsimoniously reducible to R

Recalling that b oth types of approximation problems are parameterized by the

level of precision we obtain the following quantitative form of the aforementioned

equivalence

Theorem Let R PC and let be a polynomial such that for every x y R

it holds that jy j jxj Suppose that R is strongly parsimoniously reducible to

def

R where R x y fy x y y R g

From approximate counting to approximate uniform generation Let n

n and let N be a function satisfying n expp oly n

Then approximate uniform generation for R is reducible in proba

bilistic polynomialtime to approximating R

From approximate uniform generation to approximate counting For every

noticeable N ie n p olyn for every n the problem of

See Section D

We note that in some cases the deviation of an approximate uniform generation pro cedure

can b e reduced See discussion following Theorem

COUNTING

approximating R is reducible in probabilistic polynomialtime to

approximate uniform generation problem of R where n nn

In fact Part holds also in case R is just parsimoniously reducible to R

Note that the quality of the approximate uniform generation asserted in Part

ie is indep endent of the quality of the approximate counting pro cedure ie

to which the former is reduced provided that the approximate counter p erforms

b etter than some threshold On the other hand the quality of the approximate

counting asserted in Part ie do es dep end on the quality of the approximate

uniform generation ie but cannot reach b eyond a certain b ound ie no

ticeable relative devaition Recall that for problems that are NPcomplete under

parsimonious reductions the quality of approximate counting pro cedures can b e

improved see Exercise However Theorem is most useful when applied

to problems that are not NPcomplete b ecause for problems that are NPcomplete

b oth approximate counting and uniform generation are randomly reducible to the

corresp onding search problem see Exercise

Pro of Throughout the pro of we assume for simplicity and in fact without loss

jxj

of generality that R x and R x f g

Towards Part let us rst reduce the uniform generation problem of R to R

rather than to approximating R On input x S we generate a uniformly

distributed y R x by randomly generating its bits one after the other We

pro ceed in iterations entering the i iteration with an i bit long string

def

y such that R x y fy x y y R g is not empty With probability

jR x y jjR x y j we set the i bit to equal and otherwise we set it to equal

We obtain b oth jR x y j and jR x y j by using a parsimonious reduction g of

R fx y y x y y R g PC to R That is we obtain jR x y j

by querying for the value of jR g x y j Ignoring integrality issues all this works

p erfectly ie we generate an nbit string uniformly distributed in R x as long

as we have oracle access to R But we only have oracle access to an approximation

of R and thus a careful mo dication is in place

Let us denote the approximation oracle by A Firstly by adequate error reduc

tion we may assume that for every x it holds that Pr Ax n R x

jxj where n nn In the rest of the analysis we ignore the prob

ability that the estimate deviates from the aforementioned interval and note that

this rare event is the only source of the p ossible deviation of the output distribution

from the uniform distribution on R x Let us assume for a moment that A is

deterministic and that for every x and y it holds that

Ag x y Ag x y Ag x y

We also assume that the approximation is correct at the trivial level where one

jxj

may just check whether or not x y is in R that is for every y f g it

The p ossible deviation is due to the fact that this rare event may o ccur with dierent prob

ability in the dierent invocations of algorithm A

CHAPTER RANDOMNESS AND COUNTING

holds that

Ag x y if x y R and Ag x y otherwise

We mo dify the i iteration of the foregoing pro cedure such that when entering

with the i bit long prex y we set the i bit to f g with probability

Ag x y Ag x y and halt with output with the residual probability

ie Ag x y Ag x y Ag x y Ag x y Indeed Eq

guarantees that the latter instruction is sound since the two main probabilities

sumup to at most If we completed the last ie jxj iteration then we

output the jxjbit long string that was generated Thus as long as Eq

holds but regardless of other asp ects of the quality of the approximation every

y R x is output with probability

jxj

Ag x

Ag x Ag x

jxj

Ag x Ag x Ag x

jxj

which by Eq equals Ag x Thus the pro cedure outputs each ele

ment of R x with equal probability and never outputs a non value that is out

side R x It follows that the quality of approximation only eects the probability

that the pro cedure outputs a non value which in turn equals jR xjAg x

The key p oint is that as long as Eq holds the sp ecic approximate values

obtained by the pro cedure are immaterial with the exception of Ag x all

these values cancel out

We now turn to enforcing Eq and Eq We may enforce Eq

by p erforming the straightforward check of whether or not x y R rather

than invoking Ag x y As for Eq we enforce it articially by using

def

jxjjy j

A x y jxj Ag x y instead of Ag x y Recalling

that Ag x y jxy j jR x y j we have

jxjjy j

A x y jxj jxj jR x y j

jxjjy j

A x y jxj jxj jR x y j

and the claim follows using jxj jxj jxj Note that the

foregoing mo dication only decreases the probability of outputting a non value

jxj

by a factor of jxj where the inequality is due to the setting of

ie n n Finally we refer to our assumption that A is deterministic

This assumption was only used in order to identify the value of Ag x y obtained

and used in the jy j iteration with the value of Ag x y obtained and used

in the jy j iteration but the same eect can b e obtained by just reusing the

former value in the jy j iteration rather than reinvoking A in order to obtain

it Part follows

Towards Part let use rst reduce the task of approximating R to the

task of exact uniform generation for R On input x S the reduction uses

Alternatively we note that since A is a approximator for it must hold that

R z implies Az Also since if R z then Az which

may b e rounded to

COUNTING

the tree of p ossible prexes of elements of R x in a somewhat dierent manner

Again we pro ceed in iterations entering the i iteration with an i bit long

def

string y such that R x y fy x y y R g is not empty At the i

iteration we estimate the bigger among the two fractions jR x y jjR x y j

and jR x y jjR x y j by uniformly sampling the uniform distribution over

R x y That is taking p oly jxj jxj uniformly distributed samples in R x y

we obtain with overwhelmingly high probability an approximation of these frac

tions up to an additive deviation of at most jxj This means that we obtain

a relative approximation upto a factor of jxj for the fraction or fractions

that is resp are bigger than Indeed we may not b e able to obtain such

a go o d relative approximation of the other fraction in case it is very small but

this do es not matter It also do es not matter that we cannot tell which is the

bigger fraction among the two it only matter that we use an approximation that

indicates a quantity that is say bigger than We pro ceed to the next iteration

by augmenting y using the bit that corresp onds to such a quantity Sp ecically

supp ose that we obtained the approximations a y jR x y jjR x y j and

a y jR x y jjR x y j Then we extend y by the bit if a y a y

and extend y by the bit otherwise Finally when we reach y such

jxj

that x y R we output

a a a

jxj

2 1

(jxj)

As in Part actions regarding R in this case uniform generation in R are con

ducted via the parsimonious reduction g to R That is whenever we need to sample

uniformly in the set R x y we sample the set R g x y and recover the corre

sp onding element of R x y by using the mapping guaranteed by the hypothesis

that g is strongly parsimonious Finally note that the deviation from uniform

distribution ie the fact that we can only approximately sample R merely in

tro duces such a deviation in each of our approximations to the relevant fractions

ie to a fraction bigger than Sp ecically on input x using an oracle that

provides a approximate uniform generation for R with overwhelmingly

high probability the output as dened in Eq is in

jxj

jR x j

jxj

jR x j

where the error probability is due to the unlikely case that in one of the iterations

our approximations deviates from the correct value by more than an additive de

jxj

viation term of n Noting that Eq equals jxj jR xj

jxj

and using jxj jxj Part follows and so do es the theorem

A direct pro cedure for uniform generation

We conclude the current chapter by presenting a direct pro cedure for solving the

uniform generation problem of any R PC This pro cedure uses an oracle to

CHAPTER RANDOMNESS AND COUNTING

NP which is unavoidable b ecause solving the uniform generation problem implies

solving the corresp onding search problem One advantage of this pro cedure over

the reduction presented in x is that it solves the uniform generation problem

rather than the approximate uniform generation problem

We are going to use hashing again but this time we use a family of hashing

functions having a stronger uniformity prop erty see Section D Sp ecically

we will use a family of wise indep endent hashing functions mapping bit strings

to mbit strings where b ounds the length of solutions in R and rely on the fact

that such a family satises Lemma D Intuitively such functions partition f g

into cells and Lemma D asserts that these partitions uniformly shatter all

suciently large sets That is for every set S f g of size the partition

induced by almost every function is such that each cell contains approximately

m m

jS j elements of S In particular if jS j then each cell contains

elements of S

Lo osely sp eaking the following pro cedure for uniform generation rst selects

a random hashing function and tests whether it uniformly shatters the target set

S If this condition holds then the pro cedure selects a cell at random and retrieve

the elements of S residing in the chosen cell Finally the pro cedure outputs each

retrieves element in S with a xed probability which is indep endent of the actual

number of elements of S that reside in the chosen cell This guarantees that each

element e S is output with the same probability regardless of the number of

elements of S that resides with e in the same cell

In the following construction we assume that on input x we also obtain a go o d

approximation to the size of R x This assumption can b e enforced by using

an approximate counting pro cedure as a prepro cessing stage Alternatively the

ideas presented in the following construction yield such an approximate counting

pro cedure

fm m g Construction uniform generation On input x and m

x x

def

blog jR xj c and R x f g the oracle machine proceeds as where m

fol lows

Selecting a partition that uniformly shatters R x The machine sets m

max m log and selects uniformly h H Such a function denes

a partition of f g into cells and the hope is that each cell contains

approximately the same number of elements of R x Next the machine

checks that this is indeed the case or rather than no cell contains more that

elements of R x This is done by checking whether or not x h

is in the set S dened as fol lows

def

S fx h v st jfy x y R h y v gj tg

fx h v y y st x h v y y g

t t

For sake of uniformity we allow also the case of m which is rather articial In this

case all hashing functions in H map f g to the empty string which is viewed as and thus

dene a trivial partition of f g ie into a single cell

COUNTING

where x h v y y holds if and only if y y y and for every

t t

NP j t it holds that x y R h y v Note that S

j j

If the answer is positive ie there exists a cell that contains more that

elements of R x then the machine halts with output Otherwise

the machine continues with this choice of h In this case no cell contains

more that elements of R x ie for every v f g it holds that

jfy x y R hy v gj We stress that this is an absolute

guarantee that fol lows from x h S

Selecting a cell and determining the number of elements of R x that are

contained in it The machine selects uniformly v f g and determines

def

jfy x y R hy v gj by making queries to the fol lowing NPset s

def

S fx h v y y st x h v y y g

t t

and Specically for i it checks whether x h v is in S

sets s to be the largest value of i for which the answer is positive

Obtaining all the elements of R x that are contained in the selected cell

and outputting one of them at random Using s the procedure reconstructs

def

fy x y R hy v g by making queries to the fol lowing the set S

NPset

def

fx h v j y y st x h v y y j g S

t t

where x h v y y j holds if and only if x h v y y

t t

holds and the j bit of y y equals Specically for j s and

t v

j j in order to j we make the query x h v

Final ly having recovered S the procedure determine the j bit of y

v j

outputs each y S with probability and outputs otherwise ie

with probability s

Focusing on the case that m log we note that m m log

x x

log jR xj In this case by Lemma D with overwhelmingly high prob

ability each set fy x y R hy v g has cardinality jR xj

Using m log jR xj resp m log jR xj it follows that

x x

m m

jR xj resp jR xj Thus Step can b e easily adapted

to yield an approximate counting pro cedure for R see Exercise However

our aim is to establish the following fact

Prop osition Construction solves the uniform generation problem of R

Pro of By Lemma D and the setting of m with overwhelmingly high probabil

m m

ity a uniformly selected h H partitions R x into cells each containing at

most elements The key observation stated in Step is that if the pro cedure

do es not halt in Step then it is indeed the case that h induces such a partition

CHAPTER RANDOMNESS AND COUNTING

The fact that these cells may contain a dierent number of elements is immaterial

b ecause each element is output with the same probability ie What

matters is that the average number of elements in the cells is suciently large b e

cause this average number determines the probability that the pro cedure outputs

an element of R x rather than Sp ecically the latter probability equals the

aforementioned average number which equals jR xj divided by Using

m max log jR xj log we have jR xj minjR xj

which means that the pro cedure outputs some element of R x with probability at

least min jR xj

Technical comments We can easily improve the p erformance of Construc

tion by dealing separately with the case m In such a case Step

can b e simplied and improved by uniformly selecting and outputting an element

of S which equals R x Under this mo dication the pro cedure outputs some

element of R x with probability at least In any case recall that the proba

bility that a uniform generation pro cedure outputs can b e deceased by rep eated

invocations

Chapter Notes

One key asp ect of randomized pro cedures is their success probability which is ob

viously a quantitative notion This asp ect provides a clear connection b etween

probabilistic p olynomialtime algorithms considered in Section and the count

ing problems considered in Section see also Exercise More app ealing

connections b etween randomized pro cedures and counting problems eg the ap

plication of randomization in approximate counting are presented in Section

These connections justify the presentation of these two topics in the same chapter

Randomized algorithms

Making p eople take an unconventional step requires comp elling reasons and indeed

the study of randomized algorithms was motivated by a few comp elling examples

Ironically the app eal of the two most famous examples discussed next has b een

diminished due to subsequent nding but the fundamental questions that emerged

remain fascinating regardless of the status of these and other app ealing examples

see x

The rst example primality testing For more than two decades primality

testing was the archetypical example of the usefulness of randomization in the con

text of ecient algorithms The celebrated algorithms of Solovay and Strassen

and of Rabin prop osed in the late s established that deciding primality

is in coRP ie these tests always recognize correctly prime numbers but they

may err on comp osite inputs The approach of Construction which only es

tablishes that deciding primality is in BPP is commonly attributed to M Blum

In the late s Adleman and Huang proved that deciding primality is in RP

COUNTING

and thus in ZPP In the early s Agrawal Kayal and Saxena showed

that deciding primality is actually in P One should note however that strong

evidence to the fact that deciding primality is in P was actually available from

the start we refer to Millers deterministic algorithm which relies on the

Extended Riemann Hyp othesis

The second example undirected connectivity Another celebrated example

to the p ower of randomization sp ecically in the context of logspace computa

tions was provided by testing undirected connectivity The randomwalk algorithm

presented in Construction is due to Aleliunas Karp Lipton Lovasz and Rack

o Recall that a deterministic logspace algorithm was found twentyve years

later see Section or

Other randomized algorithms Although randomized algorithms are more

abundant in the context of approximation problems let alone in other compu

tational settings cf x quite a few such algorithms are known also in the

context of search and decision problems We mention the algorithms for nding

p erfect matchings and minimum cuts in graphs see eg Ap dx B or

Sec and note the prominent role of randomization in computational

number theory see eg or Chap For a general textb o ok on ran

domized algorithms we refer the interested reader to

On the general study of BPP Turning to the general study of BPP we note

that our presentation of Theorem follows the pro of idea of Lautemann A

dierent pro of technique which yields a weaker result but found more applications

see eg Theorem and was presented indep endently by Sipser

On the role of promise problems In addition to their use in the formulation of

Theorem promise problems allow for establishing time hierarchy theorems as

in x for randomized computation see Exercise We mention that such

results are not known for the corresp onding classes of standard decision problems

The technical diculty is that we do not know how to enumerate probabilistic

machines that utilize a nontrivial probabilistic decision rule

On the feasibility of randomized computation Dierent p ersp ectives on

this question are oered by Chapter and Section D Sp ecically as advocated

in Chapter generating uniformly distributed bit sequences is not really necessary

for implementing randomized algorithms it suces to generate sequences that lo ok

as if they are uniformly distributed In many cases this leads to reducing the

number of coin tosses in such implementations and at times even to a full but

nontrivial derandomization see Sections and A less radical approach is

presented in Section D which deals with the task of extracting almost uniformly

distributed bit sequences from sources of weak randomness Needless to say these

two approaches are complimentary and can b e combined

CHAPTER RANDOMNESS AND COUNTING

Counting problems

The counting class P was introduced by Valiant who proved that computing

the p ermanent of matrices is P complete ie Theorem Interestingly

like in the case of Co oks introduction of NPcompleteness Valiants motivation

was determining the complexity of a sp ecic problem ie the p ermanent

Our presentation of Theorem is based b oth on Valiants pap er and on

subsequent studies most notably Sp ecically the highlevel structure of the

reduction presented in Prop osition as well as the structured design of the

clause gadget is taken from whereas the Deus Ex Machina gadget presented

in Figure is based on The pro of of Prop osition is also based on

with some variants Turning back to the design of clause gadgets we regret not

b eing able to cite andor use a systematic study of this design problem

As noted in the main text we decided not to present a pro of of Todas The

orem which asserts that every set in PH is Co okreducible to P ie

Theorem A pro of of a related result app ears in Section F implying that

PH is reducible to P via probabilistic p olynomialtime reductions Alternative

pro ofs can b e found in

Approximate counting and related problems The approximation pro ce

dure for P is due to Sto ckmeyer following an idea of Sipser Our

exp osition however follows further developments in the area The randomized

reduction of NP to problems of unique solutions was discovered by Valiant and

Vazirani Again our exp osition is a bit dierent

The connection b etween approximate counting and uniform generation pre

sented in x was discovered by Jerrum Valiant and Vazirani and

turned out to b e very useful in the design of algorithms eg in the Markov Chain

approach see Sec The direct pro cedure for uniform generation

presented in x is taken from

In continuation to x which is based on we refer the interested reader

to which presents a probabilistic p olynomialtime algorithm for approximat

ing the p ermanent of nonnegative matrices This fascinating algorithm is based

on the fact that knowing approximately certain parameters of a nonnegative

matrix M allows to approximate the same parameters for a matrix M provided

that M and M are suciently similar Sp ecically M and M may dier only

on a single entry and the ratio of the corresp onding values must b e suciently

close to one Needless to say the actual observation is not generic but rather

refers to sp ecic parameters of the matrix which include its p ermanent Thus

given a matrix M for which we need to approximate the p ermanent we consider a

sequence of matrices M M M such that M is the all s matrix for which

it is easy to evaluate the said parameters and each M is obtained from M by

i i

reducing some adequate entry by a factor suciently close to one This pro cess of

p olynomially many gradual changes allows to transform the dummy matrix M

into a matrix M that is very close to M and hence has a p ermanent that is very

close to the p ermanent of M Thus approximately obtaining the parameters of

M allows to approximate the p ermanent of M t

COUNTING

Finally we note that Section provides a treatment of a dierent type

of approximation problems Sp ecically when given an instance x for a search

problem R rather than seeking an approximation of the number of solutions ie

R x one seeks an approximation of the value of the b est solution ie b est

y R x where the value of a solution is dened by an auxiliary function

Exercises

Exercise Show that if a search resp decision problem can b e solved by a

probabilistic p olynomialtime algorithm having zero failure probability then the

problem can b e solve by a deterministic p olynomialtime algorithm

Hint replace the internal coin tosses by a xed outcome that is easy to generate deterministically

eg the allzero sequence

Exercise randomized reductions In continuation to the denitions pre

sented at the b eginning of Section prove the following

If a problem is probabilistic p olynomialtime reducible to a problem that

is solvable in probabilistic p olynomialtime then is solvable in probabilistic

p olynomialtime where by solving we mean solving correctly except with

negligible probability

Warning Recall that in the case that is a search problem we required

that on input x the solver provides a correct solution with probability at least

jxj but we did not require that it always returns the same solution

Hint without loss of generality the reduction do es not make the same query twice

Prove that probabilistic p olynomialtime reductions are transitive

Prove that randomized Karpreductions are transitive and that they yield a

sp ecial case of probabilistic p olynomialtime reductions

Dene onesided error and zerosided error randomized Karp and Co ok reduc

tions and consider the foregoing items when applied to them Note that the

implications for the case of onesided error are somewhat subtle

Exercise on the denition of probabilistically solving a search problem

In continuation to the discussion at the b eginning of Section supp ose that

for some probabilistic p olynomialtime algorithm A and a p ositive p olynomial p

def

fz R z g there exists y R x the following holds for every x S

such that Pr Ax y pjxj whereas for every x S it holds that

Pr Ax pjxj

Show that there exists a probabilistic p olynomialtime algorithm that solves

the search problem of R with negligible error probability

Hint See Exercise for a related pro cedure

CHAPTER RANDOMNESS AND COUNTING

Reect on the need to require that one correct solution o ccurs with probabil

ity greater than pjxj Sp ecically what can we do if it is only guar

anteed that for every x S it holds that Pr Ax R x pjxj

and for every x S it holds that Pr Ax pjxj

Note that R is not necessarily in PC Indeed in the case that R PC we can

eliminate the error probability for every x S and p erform errorreduction as in

Exercise errorreduction for BPP For N let BPP denote

the class of decision problems that can b e solved in probabilistic p olynomialtime

with error probability upp erb ounded by Prove the following two claims

For every p ositive p olynomial p and n pn the class BPP

equals BPP

For every p ositive p olynomial p and n the class BPP equals

BPP

Formulate a corresp onding version for the setting of search problem Sp ecically

for every input that has a solution consider the probability that a sp ecic solution

is output

Guideline Given an algorithm A for the syntactically weaker class consider an algo

rithm A that on input x invokes A on x for tjxj times and rules by ma jority For Part

set tn O pn and apply Chebyshevs Inequality For Part set tn O pn and

apply the Cherno Bound

Exercise errorreduction for RP For N we dene the class

of decision problem RP such that it contains S if there exists a probabilistic

p olynomialtime algorithm A such that for every x S it holds that Pr Ax

jxj and for every x S it holds that Pr Ax Prove the following

two claims

For every p ositive p olynomial p the class RP equals RP

For every p ositive p olynomial p the class RP equals RP where n

Hint The onesided error allows using an orrule rather than a ma jorityrule for the

decision

Exercise errorreduction for ZPP For N we dene the class

of decision problem ZPP such that it contains S if there exists a probabilistic

p olynomialtime algorithm A such that for every x it holds that Pr Ax x

jxj and Pr Ax f x g where x if x S and x

S S S

otherwise Prove the following two claims

For every p ositive p olynomial p the class ZPP equals ZPP

COUNTING

For every p ositive p olynomial p the class ZPP equals ZPP where n

Exercise an alternative denition of ZPP We say that the decision prob

lem S is solvable in exp ected probabilistic p olynomialtime if there exists a random

ized algorithm A and a p olynomial p such that for every x f g it holds that

Pr Ax x and the exp ected number of steps taken by Ax is at most

pjxj Prove that S ZPP if and only if S is solvable in exp ected probabilistic

p olynomialtime

Guideline Rep eatedly invoking a ZPP algorithm until it yields an output other than

results in an exp ected probabilistic p olynomialtime solver On the other hand truncating

runs of an exp ected probabilistic p olynomialtime algorithm once they exceed twice the

exp ected number of steps and outputting on such runs we obtain a ZPP algorithm

Exercise Let BPP and coRP b e classes of promise problems as in Theo

rem

Prove that every problem in BPP is reducible to the set fg P by a two

sided error randomized Karpreduction

Hint Such a reduction may eectively decide membership in any set in BPP

Prove that if a set S is Karpreducible to RP resp coRP via a deterministic

reduction then S RP resp S coRP

Exercise randomnessecient errorreductions Note that standard error

reduction as in Exercise yields error probability at the cost of increasing the

randomness complexity by a factor of O log Using the randomnessecient

errorreductions outlined in xD show that error probability can b e obtained

at the cost of increasing the randomness complexity by a constant factor and an

additive term of log Note that this allows satisfying the hypothesis made

in the illustrative paragraph of the pro of of Theorem

Exercise In continuation to the illustrative paragraph in the pro of of Theo

rem consider the promise problem such that fx r

yes no yes

jr j

jr j p jxj r f g A x r r g and fx r x S g Recall

p jxj

that for every x it holds that Pr A x r x

2p (jxj)

r fg

p jxj

Show that mapping x to x r where r is uniformly distributed in f g

constitutes a onesided error randomized Karpreduction of S to

Show that is in the promise problem class coRP

Exercise Prove that for every S NP there exists a probabilistic p olynomial

time algorithm A such that for every x S it holds that Pr Ax and for

every x S it holds that Pr Ax That is A has error probability at

most expp oly jxj on yesinstances but never errs on noinstances Thus

NP may b e ctitiously viewed as having a huge onesided error probability

CHAPTER RANDOMNESS AND COUNTING

Exercise randomized versions of NP In continuation to Footnote

consider the following two variants of MA which we consider the main randomized

version of NP

S MA if there exists a probabilistic p olynomialtime algorithm V such

p olyjxj

that for every x S there exists y f g such that Pr V x y

whereas for every x S and every y it holds that Pr V x y

S MA if there exists a probabilistic p olynomialtime algorithm V such

p olyjxj

that for every x S there exists y f g such that Pr V x y

whereas for every x S and every y it holds that Pr V x y

Prove that MA NP whereas MA MA

Guideline For the rst part note that a sequence of internal coin tosses that makes

V accept x y can b e incorp orated into y itself yielding a standard NPwitness For

the second part apply the ideas underlying the pro of of Theorem and note that an

adequate sequence shifts to b e used by the verier can b e incorp orated in the single

message sent by the prover

Exercise time hierarchy theorems for promise problem versions of BPtime

Fixing a mo del of computation let BPtimet denote the class of promise prob

lems that are solvable by a randomized algorithm of time complexity t that has

a twosided error probability at most The common denition refers only to

decision problems Formulate and prove results analogous to Theorem and

Corollary

Guideline Analogously to the pro of of Theorem we construct a Bo olean function

f by asso ciating with each admissible machine M an input x and making sure that

Pr f x M x where M x denotes the emulation of M x susp ended after

t jxj steps The key p oint is that f is a partial function corresp onding to a promise

problem that is dened only for machines called admissible that have twosided error

at most on every input This restriction allows for a randomized computation of f

with twosided error probability at most on each input on which f is dened

Exercise extracting square ro ots mo dulo a prime Using the follow

ing guidelines present a probabilistic p olynomialtime algorithm that on input a

prime P and a quadratic residue s mo d P returns r such that r s mo d P

Prove that if P mo d then s mo d P is a square ro ot of the

quadratic residue s mo d P

Note that the pro cedure suggested in Item relies on the ability to nd an

odd integer e such that s mo d P and once such e is found we may

output s mo d P In Item we used e P which is o dd since

P mo d

Show that it suces to nd an odd integer e together with a residue t and

e e e e

an even integer e such that s t mo d P b ecause s s t

e e

s t

COUNTING

Given a prime P mo d a quadratic residue s and a quadratic non

residue t equiv t mo d P show that e and e as in Item

can b e eciently found

Prove that for a prime P with probability a uniformly chosen t

f P g satises t mo d P

Note that randomization is used only in the last item which in turn is used only

for P mo d

Exercise smallspace randomized stepcounter A stepcounter is an

algorithm that runs for a number of steps that is sp ecied in its input Actually

such an algorithm may run for a somewhat larger number of steps but halt after

issuing a number of signals as sp ecied in its input where these signals are dened

as entering and leaving a designated state of the algorithm A stepcounter may

b e run in parallel to another pro cedure in order to susp end the execution after a

desired number of steps of the other pro cedure has elapsed We note that there

exists a simple deterministic machine that on input n halts after issuing n signals

while using O log n space and O n time The goal of this exercise is

presenting a randomized stepcounter that allows for many more signals while

using the same amount of space Sp ecically present a randomized algorithm

that on input n uses O log n space and O time and halts after issuing

an exp ected number of signals Furthermore prove that with probability at

least this stepcounter halts after issuing a number of signals that is

nk nk

b etween and

Guideline Rep eat the following exp eriment till reaching success Each trial consists of

uniformly selecting n bits ie tossing n unbiased coins and is deemed successful if all

bits turn out to equal the value ie all outcomes equal head Note that such a trial

can b e implemented by using space O log n mainly for implementing a standard

counter for determining the number of bits Thus each trial is successful with probability

n n

and the exp ected number of trials is

Exercise analysis of random walks on arbitrary undirected graphs

In order to complete the pro of of Prop osition prove that if fu v g is an edge

of the graph G V E then EX jE j Recall that for a xed graph X

uv uv

is a random variable representing the number of steps taken in a random walk that

starts at the vertex u until the vertex v is rst encountered

Guideline Let Z n b e a random variable counting the number of minimal paths

from u to v that app ear along a random walk of length n where the walk starts at the

stationary vertex distribution which is welldened assuming the graph is not bipartite

j i

mo d P Assuming and note that s Write P j

i i

j j

mo d P show how t that for some i i and j it holds that s

i1 i

j j

t mo d P Extra hint to nd i i and j such that s

i1 i 1 i

j j j

mo d P Thus starting with s t mo d P and t

i i we reach i at which p oint we have what we need

CHAPTER RANDOMNESS AND COUNTING

which in turn may b e enforced by adding a selflo op On one hand EX X

uv v u

lim nEZ n due to the memoryless prop erty of the walk On the other hand

n uv

def

letting i if the edge fu v g was traversed from v to u in the i step of such

v u

def

i Z n and a random walk and i otherwise we have

v u uv v u

E i jE j b ecause in each step each directed edge app ears on the walk with

v u

equal probability It follows that EX jE j

Exercise the class PP BPP and its relation to P In contrast to

BPP which refers to useful probabilistic p olynomialtime algorithms the class PP

do es not capture such algorithms but is rather closely related to P A decision

problem S is in PP if there exists a probabilistic p olynomialtime algorithm A such

that for every x it holds that x S if and only if Pr Ax Note that

BPP PP Prove that PP is Co okreducible to P and vise versa

Guideline For S PP by virtue of the algorithm A consider the relation R such that

pjxj

x r R if and only if A accepts the input x when using the randominput r f g

pjxj

where p is a suitable p olynomial Thus x S if and only if jR xj which

in turn can de determined by querying the counting function of R To reduce f P

to PP consider the relation R PC that is counted by f ie f x jR xj and the

decision problem S as dened in Prop osition Let p b e the p olynomial sp ecifying the

length of solutions for R ie x y R implies jy j pjxj and consider the algorithm

A that on input x N pro ceeds as follows With probability it uniformly selects

pjxj

y f g and accepts if and only if x y R and otherwise ie in the other

p(jxj)

Prove that x N S if and only if case it accepts with probability

p(jxj)

Pr A x

Exercise articial P complete problems Show that there exists a re

lation R PC such that R is P complete and S f g

Guideline For any P complete problem R dene R fx y x y R g

jxj

fx x f g g

Exercise enumeration problems For any binary relation R dene the

enumeration problem of R as a function f f g N f g fg such that

f x i equals the i element in jR xj if jR xj i and f x i otherwise

R R

The ab ove denition refers to the standard lexicographic order on strings but any

other ecient order of strings will do

Prove that for any p olynomially b ounded R computing R is reducible to

computing f

Prove that for any R PC computing f is reducible to some problem in

An order of strings is a and onto mapping from the natural numbers to the set of all

strings Such order is called ecient if b oth and its inverse are eciently computable The

standard lexicographic order satises i y if the compact binary expansion of i equals y

that is etc

COUNTING

Guideline Consider the binary relation R fhx bi y x y R y bg

and show that f is reducible to R Extra hint Note that f x i y if and only

if jR hx y ij i and for every y y it holds that jR hx y i j i

Exercise computing the p ermanent of integer matrices Prove that

computing the p ermanent of matrices with entries is computationally equiva

lent to computing the number of p erfect matchings in bipartite graphs

Hint Given a bipartite graph G X Y E consider the matrix M representing the edges

th th

b etween X and Y ie the i j entry in M is if the i vertex of X is connected to the j

entry of Y and note that only p erfect matchings in G contribute to the p ermanent of M

Exercise computing the p ermanent mo dulo Combining Prop osition

and Theorem prove that for every integer n that is relatively prime to

c computing the p ermanent mo dulo n is NPhard under randomized reductions

Since Prop osition holds for c hardness holds for every o dd integer n

Guideline Apply the reduction of Prop osition to the promise problem of deciding

whether a CNF formula has a unique satisable assignment or is unsatisable Use the

fact that n do es not divide any p ower of c

Exercise negative values in Prop osition Assuming P NP prove

that Prop osition cannot hold for a set I containing only nonnegative integers

Note that the claim holds even if the set I is not nite and even if I is the set of

all nonnegative integers

Guideline A reduction as in Prop osition yields a Karpreduction of SAT to deciding

whether the p ermanent of a matrix with entries in I is nonzero Note that the p ermanent

of a nonnegative matrix is nonzero if and only if the corresp onding bipartite graph has

a p erfect matching

Exercise highlevel analysis of the p ermanent reduction Establish the

correctness of the highlevel reduction presented in the pro of of Prop osition

That is show that if the clause gadget satises the three conditions p ostulated in

the said pro of then each satisfying assignment of contributes exactly c to the

SWCC of G whereas unsatisfying assignments have no contribution

Guideline Cluster the cycle covers of G according to the set of track edges that they

use ie the edges of the cycle cover that b elong to the various tracks Note the

corresp ondence b etween these edges and the external edges used in the denition of the

gadgets prop erties Using the p ostulated conditions regarding the clause gadget prove

that for each such set T of track edges if the sum of the weights of all cycle covers that

use the track edges T is nonzero then the following hold

The intersection of T with the set of track edges incident at each sp ecic clause

gadget is nonempty Furthermore if this set contains an incoming edge resp

Actually a sucient condition is that n do es not divide any p ower of c Thus referring to

c hardness holds for every integer n that is not a p ower of On the other hand for

any xed n the p ermanent mo dulo n can b e computed in p olynomialtime Thm

CHAPTER RANDOMNESS AND COUNTING

outgoing edge of some entryvertex resp exitvertex then it also contains an

outgoing edge resp incoming edge of the corresp onding exitvertex resp entry

vertex

If T contains an edge that b elongs to some track then it contains all edges of this

track It follows that for each variable x the set T contains the edges of a single

track asso ciated with x

The tracks picked by T corresp ond to a single truth assignment to the variables of

and this assignment satises b ecause for each clause T contains an external

edge that corresp onds to a literal that satises this clause

It follows that each satisfying assignment of contributes exactly c to the SWCC of

Exercise analysis of the implementation of the clause gadget Establish

the correctness of the implementation of the clause gadget presented in the pro of of

Prop osition That is show that if the b ox satisfy the three conditions p ostu

lated in the said pro of then the clause gadget of Figure satises the conditions

p ostulated for it

Guideline Cluster the cycle covers of a gadget according to the set of nonb ox edges that

they use where nonb ox edges are the edges shown in Figure Using the p ostulated

conditions regarding the b ox prove that for each set S of nonb ox edges if the sum of

the weights of all cycle covers that use the nonb ox edges S is nonzero then the following

hold

The intersection of S with the set of edges incident at each b ox must contain

two nonselo op edges one incident at each of the b oxs terminals Needless to

say one edge is incoming and the other outgoing Referring to the six edges that

connects one of the six designated vertices of the gadget with the corresp onding

b ox terminals as connectives note that if S contains a connective incident at the

terminal of some b ox then it must also contain the connective incident at the other

terminal In such a case we say that this b ox is picked by S

Each of the three literaldesignated b oxes that is not picked by S is traversed

from left to right ie the cycle cover contains an incoming edge of the left terminal

and an outgoing edge of the right terminal Thus the set S must contain a

connective b ecause otherwise no directed cycle may cover the leftmost vertex shown

in Figure That is S must pick some b ox

The set S is fully determined by the nonempty set of b oxes that it picks

The p ostulated prop erties of the clause gadget follow with c b

Exercise analysis of the design of a b ox for the clause gadget Prove

that the by matrix presented in Eq satises the prop erties p ostulated for

the b ox used in the second part of the pro of of Prop osition In particular

Show a corresp ondence b etween the conditions required of the b ox and con

ditions regarding the value of the p ermanent of certain submatrices of the

adjacency matrix of the graph

Hint For example show that the rst condition corresp ond to requiring that the value

of the p ermanent of the entire matrix equals zero The second condition refers to sub

matrices obtained by omitting either the rst row and fourth column or the fourth row

and rst column

COUNTING

Verify that the matrix in Eq satises the aforementioned conditions

regarding the value of the p ermanent of certain submatrices

Prove that no by matrix and thus also no by matrix can satisfy the afore

mentioned conditions

Exercise error reduction for approximate counting Show that the er

ror probability in Denition can b e reduced from or even

p olyjxj to expp oly jxj

Guideline Invoke the weaker pro cedure for an adequate number of times and take the

median value among the values obtained in these invocations

Exercise strong approximation for some P complete problems Show

that there exists P complete problems alb eit unnatural ones for which an

approximation can b e found by a deterministic p olynomialtime algorithm Fur

thermore the runningtime dep ends p olynomially on

Guideline Combine any P complete problem referring to some R PC with a trivial

counting problem eg such as the counting problem asso ciated with R fx y

x y f g g Show that without loss of generality that x y R implies jxj jy j

jxj

and that R x Prove that the counting problem of R fx y x y

R g fx y x y R g is P complete Present a deterministic algorithm that on

input x and outputs an approximation of R x in time p oly jxj

Exercise relative approximation for DNF satisfaction Referring to

the text of x prove the following claims

Both assumptions regarding the general setting hold in case S C

where C denotes the set of truth assignments that satisfy the conjunc

tion C

Guideline In establishing the second assumption note that it reduces to the

conjunction of the following two assumptions

a Given i one can eciently generate a uniformly distributed element of S

Actually generating a distribution that is almost uniform over S suces

b Given i and x one can eciently determine whether x S

Prove Prop osition relating to details such as the error probability in an

implementation of Construction

Note that Construction do es not require exact computation of jS j An

alyze the output distribution in the case that we can only approximate jS j

upto a factor of

Exercise reducing the relative deviation in approximate counting

Prove that for any R PC and every p olynomial p and constant there

exists R PC such that p approximation for R is reducible to

approximation for R

CHAPTER RANDOMNESS AND COUNTING

Guideline For tn pn let R fx y y i x y R g Note

tjxj

tjxj tjxj

that jR xj jR xj and thus if a jR xj then a

tjxj

jR xj

Furthermore for any F n expp oly n prove that there exists R PC such

that p approximation for R is reducible to approximating R to within

a factor of F with error probability

Hint Same as the main part using tn pn log F n

Exercise deviation reduction in approximate counting cont In con

tinuation to Exercise prove that if R is NPcomplete via parsimonious reduc

tions then for every p ositive p olynomial p and constant the problem of

p approximation for R is reducible to approximation for R

Hint Comp ose the reduction to the problem of approximation for R provided in

Exercise with the parsimonious reduction of R to R

Prove that for every function F such that F n expn we can also reduce

the aforementioned problems to the problem of approximating R to within a

factor of F with error probability

Guideline Using R as in Exercise we encounter a technical diculty The issue is

that the comp osition of the amplifying reduction of R to R with the parsimonious

reduction of R to R may increase the length of the instance Indeed the length of the

new instance is p olynomial in the length of the original instance but this p olynomial may

dep end on R which in turn dep ends on F Thus we cannot use F n expn

but F n expn is ne

Exercise Referring to the pro cedure in the pro of Theorem show how to

use an NPoracle in order to determine whether the number of solutions that pass

a random sieve is greater than t You are allowed queries of length p olynomial in

the length of x h and in the size of t

def

fx i h y y st x h y y g where x h y y Hint Consider the set S

t t t

holds if and only if the y are dierent and for every j it holds that x y R hy

j j j

Exercise parsimonious reductions and Theorem Demonstrate the

imp ortance of parsimonious reductions in Theorem by proving the following

There exists a search problem R PC such that every problem in PC is

reducible to R by a nonparsimonious reduction and still the the promise

S is decidable in p olynomialtime problem US

R R

Guideline Consider the following articial witness relation R for SAT in which

R if f g and satises Note that the standard witness relation

of SAT is reducible to R but this reduction is not parsimonious Also note that

S is trivial US and thus US

R R R

There exists a search problem R PC such that R is P complete and

still the the promise problem US S is decidable in p olynomialtime

R R

Guideline Just use the relation suggested in the guideline to Part An al

ternative pro of relies on Theorem and on the fact that it is easy to decide

COUNTING

S when R is the corresp onding p erfect matching relation by computing US

R R

the determinant

Exercise Prove that SAT is randomly reducible to deciding unique solution

for SAT without using the fact that SAT is NPcomplete via parsimonious reductions

Guideline Follow the pro of of Theorem while using the family of pairwise inde

p endent hashing functions provided in Construction D or in Eq Note that

in this case the condition R h can b e directly enco ded as a CNF

SAT

def

formula That is consider the formula such that z z hz and note

h h

that hz can b e written as the conjunction of i clauses where each clause is a CNF

that is logically equivalent to the parity of some of the bits of z where the identity of

these bits is determined by h

Exercise an alternative pro cedure for approximate counting Adapt

Step of Construction so to obtain an approximate counting pro cedure for

Guideline For m the pro cedure invokes Step of Construction until a

negative answer is obtained and outputs for the current value of m For jR xj

this yields a constant factor approximation of jR xj In fact we can obtain a

b etter estimate by making additional queries at iteration m ie queries of the form

x h for i The case jR xj can b e treated by using Step of

Construction in which case we obtain an exact count

Exercise Let R b e an arbitrary PC complete search problem Show that

approximate counting and uniform generation for R can b e randomly reduced to

deciding membership in S where by approximate counting we mean a p

approximation for any p olynomial p

Guideline Note that Construction yields such pro cedures see also Exercise

except that they make oracle calls to some other set in NP Using the NPcompleteness

of S we are done R

CHAPTER RANDOMNESS AND COUNTING

Chapter

Relaxing the Requirements

The philosophers have only interpreted the world in

various ways the p oint is to change it

Karl Marx Theses on Feuerbach

In light of the apparent infeasibility of solving numerous natural computational

problems it is natural to ask whether these problems can b e relaxed in a way that

is b oth useful for applications and allows for feasible solving pro cedures We stress

two asp ects ab out the foregoing question on one hand the relaxation should b e

suciently go o d for the intended applications but on the other hand it should

b e signicantly dierent from the original formulation of the problem so to escap e

the infeasibility of the latter We note that whether a relaxation is adequate for

an intended application dep ends on the application and thus much of the material

in this chapter is less robust or generic than the treatment of the nonrelaxed

computational problems

Summary We consider two types of relaxations The rst type of

relaxation refers to the computational problems themselves that is for

each problem instance we extend the set of admissible solutions In

the context of search problems this means settling for solutions that

have a value that is suciently close to the value of the optimal

solution with resp ect to some value function Needless to say the

sp ecic meaning of suciently close is part of the denition of the

relaxed problem In the context of decision problems this means that

for some instances b oth answers are considered valid put dierently

we consider promise problems in which the noinstances are far from

the yesinstances in some adequate sense which is part of the denition

of the relaxed problem

The second type of relaxation deviates from the requirement that the

solver provides an adequate answer on each valid instance Instead

the b ehavior of the solver is analyzed with resp ect to a predetermined

CHAPTER RELAXING THE REQUIREMENTS

input distribution or a class of such distributions and bad b ehavior

may o ccur with negligible probability where the probability is taken

over this input distribution That is we replace worstcase analysis by

averagecase or rather typicalcase analysis Needless to say a ma jor

comp onent in this approach is limiting the class of distributions in a way

that on one hand allows for various types of natural distributions and

on the other hand prevents the collapse of the corresp onding notion of

averagecase complexity to the standard worstcase complexity

Approximation

The notion of approximation is a natural one and has arisen also in other disci

plines Approximation is most commonly used in references to quantities eg the

length of one meter is approximately forty inches but it is also used when refer

ring to qualities eg an approximately correct account of a historical event In

the context of computation the notion of approximation mo dies computational

tasks such as search and decision problems In fact we have already encountered

it as a mo dier of counting problems see Section

Two ma jor questions regarding approximation are what is a go o d approx

imation and can it b e found easier than nding an exact solution The answer

to the rst question seems intimately related to the sp ecic computational task

at hand and to its role in the wider context ie the higher level application a

go o d approximation is one that suces for the intended application Indeed the

imp ortance of certain approximation problems is much more sub jective than the

imp ortance of the corresp onding optimization problems This fact seems to stand

in the way of attempts at providing a comprehensive theory of natural approxi

mation problems eg general classes of natural approximation problems that are

shown to b e computationally equivalent

Turning to the second question we note that in numerous cases natural approx

imation problems seem to b e signicantly easier than the corresp onding original

exact problems On the other hand in numerous other cases natural approxi

mation problems are computationally equivalent to the original problems We shall

exemplify b oth cases by reviewing some sp ecic results but regret not b eing able

to provide any systematic classication

Mimicking the two standard uses of the word approximation we shall distinguish

b etween approximation problems that are of the search type and problems that

are have a clear decisional avor In the rst case we shall refer to a function that

assigns values to p ossible solutions of a search problem whereas in the second

case we shall refer to distances b etween instances of a decision problem Needless

to say at times the same computational problem may b e cast in b oth ways but for

most natural approximation problems one of the two frameworks is more app ealing

than the other

The common theme is that in b oth cases we extend the set of admissible so

lutions In the case of search problems we extend the set of optimal solutions by

including also almostoptimal solutions In the case of decision problems we extend

APPROXIMATION

the set of solutions by allowing an arbitrary answer solution to some instances

which may b e viewed as a promise problem that disallows these instances In this

case we fo cus on promise problems in which the yes and noinstances are far apart

and the instances that violate the promise are closed to yesinstances

Teaching note Most of the results presented in this section refer to sp ecic computa

tional problems and with one exception are presented without a pro of In view of the

complexity of the corresp onding pro ofs and the merely illustrative role of these results

in the context of complexity theory we recommend doing the same in class

Search or Optimization

As noted in Section many search problems involve a set of p otential solutions

p er each problem instance such that dierent solutions are assigned dierent val

ues resp costs by some value resp cost function In such a case one is

interested in nding a solution of maximum value resp minimum cost A corre

sp onding approximation problem may refer to nding a solution of approximately

maximum value resp approximately minimum cost where the sp ecication of

the desired level of approximation is part of the problems denition Let us elab

orate

For concreteness we fo cus on the case of a value that we wish to maximize For

greater exibility we allow the value of the solution to dep end also on the instance

itself Thus for a p olynomially b ounded binary relation R and a value function

f f g f g R we consider the problem of nding solutions with resp ect

to R that maximize the value of f That is given x such that R x the

task is nding y R x such that f x y v where v is the maximum value

x x

of f x y over all y R x Typically R is in PC and f is p olynomialtime

computable Indeed without loss of generality we may assume that for every x

jxj

it holds that R x f g for some p olynomial see Exercise Thus

the optimization problem is recast as the following search problem given x nd

y such that f x y v where v max ff x y gg

(jxj)

x x

y fg

We shall fo cus on relative approximation problems where for some gap function

g f g fr R r g the maximization task is nding y such that f x y

v g x Indeed in some cases the approximation factor is stated as a function of

the length of the input ie g x g jxj for some g N fr R r g but

often the approximation factor is stated in terms of some more rened parameter

of the input eg as a function of the number of vertices in a graph Typically g

is p olynomialtime computable

Denition g factor approximation Let f f g f g R

N N and g f g fr R r g

In this case we may assume without loss of generality that the function f dep ends only on

the solution This can b e obtained by redening the relation R such that each solution y Rx

consists of a pair of the form x y Needless to say this mo dication cannot b e applied along

with getting rid of R as in Exercise

CHAPTER RELAXING THE REQUIREMENTS

Maximization version The g factor approximation of maximizing f wrt is the

jxj

search problem R such that R x fy f g f x y v g xg

where v max ff x y g

(jxj)

y fg

Minimization version The g factor approximation of minimizing f wrt is the

jxj

search problem R such that R x fy f g f x y g x c g

where c min ff x y g

(jxj)

y fg

We note that for numerous NPcomplete optimization problems p olynomialtime

algorithms provide meaningful approximations A few examples will b e mentioned

in x In contrast for numerous other NPcomplete optimization problems

natural approximation problems are computationally equivalent to the corresp ond

ing optimization problem A few examples will b e mentioned in x where

we also introduce the notion of a gap problem which is a promise problem of

the decision type intended to capture the diculty of the approximate search

problem

A few p ositive examples

Let us start with a trivial example Considering a problem such as nding the

maximum clique in a graph we note that nding a linear factor approximation is

trivial ie given a graph G V E we may output any vertex in V as a jV j

factor approximation of the maximum clique in G A famous nontrivial example

is presented next

Prop osition factor two approximation to minimum Vertex Cover There

exists a polynomialtime approximation algorithm that given a graph G V E

outputs a vertex cover that is at most twice as large as the minimum vertex cover

of G

We warn that an approximation algorithm for minimum Vertex Cover do es not

yield such an algorithm for the complementary problem of maximum Independent

Set This phenomenon stands in contrast to the case of optimization where an

optimal solution for one problem eg minimum Vertex Cover yields an optimal

solution for the complementary problem maximum Independent Set

Pro of Sketch The main observation is a connection b etween the set of maximal

matchings and the set of vertex covers in a graph Let M b e any maximal matching

in the graph G V E that is M E is a matching but augmenting it by any

single edge yields a set that is not a matching Then on one hand the set of all

vertices participating in M is a vertex cover of G and on the other hand each

vertex cover of G must contain at least one vertex of each edge of M Thus we can

nd the desired vertex cover by nding a maximal matching which in turn can b e

found by a greedy algorithm

APPROXIMATION

Another example An instance of the traveling salesman problem TSP consists

of a symmetric matrix of distances b etween pairs of p oints and the task is nding

a shortest tour that passes through all p oints In general no reasonable approx

imation is feasible for this problem see Exercise but here we consider two

sp ecial cases in which the distances satises some natural constraints and pretty

go o d approximations are feasible

Theorem approximations to sp ecial cases of TSP Polynomialtime algo

rithms exists for the fol lowing two computational problems

Providing a factor approximation for the special case of TSP in which the

distances satisfy the triangle inequality

For every providing a factor approximation for the special case

of Euclidean TSP ie for some constant k eg k the p oints reside

in a k dimensional Euclidean space and the distances refer to the standard

Euclidean norm

A weaker version of Part is given in Exercise A detailed survey of Part

is provided in We note the dierence examplied by the two items of Theo

rem Whereas Part provides a p olynomialtime approximation for a sp ecic

constant factor Part provides such an algorithm for any constant factor Such a

result is called a polynomialtime approximation scheme abbrev PTAS

A few negative examples

Let us start again with a trivial example Considering a problem such as nding

the maximum clique in a graph we note that given a graph G V E nding

a jV j factor approximation of the maximum clique in G is as hard as

nding a maximum clique in G Indeed this result is not really meaningful

In contrast building on the PCP Theorem Theorem one may prove that

nding a jV j factor approximation of the maximum clique in G is as hard as

nding a maximum clique in G This follows from the fact that the approximation

problem is NPhard cf Theorem

The statement of inapproximability results is made stronger by referring to a

promise problem that consists of distinguishing instances of suciently far apart

values Such promise problems are called gap problems and are typically stated

with resp ect to two b ounding functions g g f g R which replace the gap

function g of Denition Typically g and g are p olynomialtime computable

Denition gap problem for approximation of f Let f be as in Deni

tion and g g f g R

problem of maximizing f consists Maximization version For g g the gap

g g

1 2

of distinguishing between fx v g xg and fx v g xg where

x x

v max ff x y g

(jxj)

yfg

CHAPTER RELAXING THE REQUIREMENTS

problem of minimizing f consists Minimization version For g g the gap

g g

1 2

of distinguishing between fx c g xg and fx c g xg where

x x

c min ff x y g

(jxj)

yfg

problem of maximizing the size of a clique in a graph For example the gap

g g

1 2

consists of distinguishing b etween graphs G that have a clique of size g G and

graphs G that have no clique of size g G In this case we typically let g G b e a

function of the number of vertices in G V E that is g G g jV j Indeed

letting G denote the size of the largest clique in the graph G we let gapClique

denote the gap problem of distinguishing b etween fG V E G LjV jg

and fG V E G sjV jg where L s Using this terminology we

restate and strengthen the aforementioned jV j factor inapproximation of

the maximum clique problem

o o

Theorem For some LN N and sN N it holds that gapClique

is NPhard

The pro of of Theorem is based on a ma jor renement of Theorem that

refers to a PCP system of amortized freebit complexity that tends to zero cf

x A weaker result which follows from Theorem itself is presented in

Exercise

As we shall show next results of the type of Theorem imply the hardness

of a corresp onding approximation problem that is the hardness of deciding a gap

problem implies the hardness of a search problem that refers to an analogous factor

of approximation

Prop osition Let f g g be as in Denition and suppose that these

problem of maximiz functions are polynomialtime computable Then the gap

g g

1 2

ing f resp minimizing f is reducible to the g g factor resp g g factor

approximation of maximizing f resp minimizing f

Note that a reduction in the opp osite direction do es not necessarily exist even in

the case that the underlying optimization problem is selfreducible in some natural

sense Indeed this is another dierence b etween the current context of approx

imation and the context of optimization problems where the search problem is

reducible to a related decision problem

Pro of Sketch We fo cus on the maximization version On input x we solve the

problem by making the query x obtaining the answer y and ruling that gap

g g

1 2

x has value exceeding g x if and only if f x y g x Recall that we need to

analyze this reduction only on inputs that satisfy the promise Thus if v g x

then the oracle must return a solution y that satises f x y v g xg x

which implies that f x y g x On the other hand if v g x then f x y

v g x holds for any p ossible solution y

APPROXIMATION

problem of mini Additional examples Let us consider gapVC the gap

g g

s L

mizing the vertex cover of a graph where s and L are constants and g G s jV j

resp g G L jV j for any graph G V E Then Prop osition implies

via Prop osition that for every constant s the problem gapVC is solvable

in p olynomialtime In contrast suciently narrowing the gap b etween the two

thresholds yields an inapproximability result In particular

Theorem For some constants s L eg s and L

will do the problem gapVC is NPhard

The pro of of Theorem is based on a complicated renement of Theorem

Again a weaker result follows from Theorem itself see Exercise

As noted renements of the PCP Theorem Theorem play a key role in

establishing inapproximability results such as Theorems and In that

resp ect it is adequate to recall that Theorem establishes the equivalence of

the PCP Theorem itself and the NPhardness of a gap problem concerning the

maximization of the number of clauses that are satises in a given CNF for

mula Sp ecically gapSAT was dened in Denition as the gap problem

consisting of distinguishing b etween satisable CNF formulae and CNF formu

lae for which each truth assignment violates at least an fraction of the clauses

Although Theorem do es not sp ecify the quantitative relation that underlies

its qualitative assertion when rened and combined with the b est known PCP

construction it do es yield the b est p ossible b ound

Theorem For every v the problem gapSAT is NPhard

On the other hand gapSAT is solvable in p olynomialtime

Sharp threshold The aforementioned conicting results regarding gapSAT

exemplify a sharp threshold on the factor of approximation that can b e obtained

by an ecient algorithm Another app ealing example refers to the following maxi

mization problem in which the instances are systems of linear equations over GF

and the task is nding an assignment that satises as many equations as p ossible

Note that by merely selecting an assignment at random we exp ect to satisfy half

of the equations Also note that it is easy to determine whether there exists an

assignment that satises all equations Let gapLin denote the problem of dis

tinguishing b etween systems in which one can satisfy at least an L fraction of the

equations and systems in which one cannot satisfy an s fraction or more of the

equations Then as just noted gapLin is trivial and gapLin is feasible

L s

for every s In contrast moving b oth thresholds slightly away from the

corresp onding extremes yields an NPhard gap problem

Theorem For every constant the problem gapLin is NPhard

The pro of of Theorem is based on a ma jor renement of Theorem In fact

the corresp onding PCP system for NP is merely a reformulation of Theorem

the verier makes three queries and tests a linear condition regarding the answers

CHAPTER RELAXING THE REQUIREMENTS

while using a logarithmic number of coin tosses This verier accepts any yes

instance with probability at least when given oracle access to a suitable

pro of and rejects any noinstance with probability at least regardless

of the oracle b eing accessed A weaker result which follows from Theorem

itself is presented in Exercise

Gap lo cation Theorems and illustrate two opp osite situations with

resp ect to the lo cation of the gap for which the corresp onding promise problem

is hard Recall that b oth gapSAT and gapLin are formulated with resp ect to two

thresholds where each threshold b ounds the fraction of lo cal conditions ie

clauses or equations that are satisable in the case of yes and noinstances re

sp ectively In the case of gapSAT the high threshold referring to yesinstances was

set to and thus only the low threshold referring to noinstances remained a free

parameter Nevertheless a hardness result was established for gapSAT and further

more this was achieved for an optimal value of the low threshold cf the foregoing

discussion of sharp threshold In contrast in the case of gapLin setting the high

threshold to makes the gap problem eciently solvable Thus the hardness of

gapLin was established at a dierent lo cation of the high threshold Sp ecically

hardness for an optimal value of the ratio of thresholds was established when

setting the high threshold to for any

A nal comment All the aforementioned inapproximability results refer to ap

proximation resp gap problems that are relaxations of optimization problems

in NP ie the optimization problem is computational equivalent to a decision

problem in NP see Section In these cases the NPhardness of the approx

imation resp gap problem implies that the corresp onding optimization problem

is reducible to the approximation resp gap problem In other words in these

cases nothing is gained by relaxing the original optimization problem b ecause the

relaxed version remains just as hard

Decision or Prop erty Testing

A natural notion of relaxation for decision problems arises when considering the

distance b etween instances where a natural notion of distance is the Hamming

distance ie the fraction of bits on which two strings disagree Lo osely sp eaking

this relaxation called property testing refers to distinguishing inputs that reside

in a predetermined set S from inputs that are relatively far from any input that

resides in the set Two natural types of promise problems emerge with resp ect to

any predetermined set S and the Hamming distance b etween strings

Relaxed decision wrt a xed distance Fixing a distance parameter we

consider the problem of distinguishing inputs in S from inputs in S

where

def

jxj

S fx z S f g x z jxjg

and x x z z jfi x z gj denotes the number of bits on

m m i i

which x x x and z z z disagree Thus here we consider a

m m

APPROXIMATION

promise problem that is a restriction or a sp ecial case of the problem of

deciding membership in S

Relaxed decision wrt a variable distance Here the instances are pairs x

where x is as in Type and is a distance parameter The yes

instances are pairs x such that x S whereas x is a noinstance if

x S

We shall fo cus on Type formulation which seems to capture the essential question

of whether or not these relaxations lower the complexity of the original decision

problem The study of Type formulation refers to a relatively secondary question

which assumes a p ositive answer to the rst question that is assuming that the

relaxed form is easier than the original form we ask how is the complexity of the

problem aected by making the distance parameter smaller which means making

the relaxed problem tighter and ultimately equivalent to the original problem

We note that for numerous NPcomplete problems there exist natural Type

relaxations that are solvable in p olynomialtime Actually these algorithms run in

sublinear time sp ecically p olylogarithmic time when given direct access to the

input A few examples will b e presented in x As indicated in x

this is not a generic phenomenon But b efore turning to these results we discuss

several imp ortant denitional issues

Denitional issues

Prop erty testing is concerned not only with solving relaxed versions of NPhard

problems but rather solving these problems as well as problems in P in sub

linear time Needless to say such results assume a mo del of computation in which

algorithms have direct access to bits in the representation of the input see De

nition

Denition a direct access mo del conventions An algorithm with direct

access to its input is given its main input on a special input device that is accessed

as an oracle see x In addition the algorithm is given the length of the

input and possibly other parameters on an secondary input device The complexity

of such an algorithm is stated in terms of the length of its main input

Indeed the description in x refers to such a mo del but there the main input

is viewed as an oracle and the secondary input is viewed as the input

Denition prop erty testing for S For any xed the promise

problem of distinguishing S from S is called property testing for S with resp ect

Recall that we say that a randomized algorithm solves a promise problem if it

accepts every yesinstance resp rejects every noinstance with probability at

least Thus a randomized prop erty testing for S accepts every input in S

resp rejects every input in S with probability at least

CHAPTER RELAXING THE REQUIREMENTS

The question of representation The sp ecic representation of the input is of

ma jor concern in the current context This is due to the eect of the represen

tation on the distance measure and to the dependence of direct access machines

on the specic representation of the input Let us elab orate on b oth asp ects

Recall that we dened the distance b etween ob jects in terms of the Hamming

distance b etween their representations Clearly in such a case the choice of

representation is crucial and dierent representations may yield dierent dis

tance measures Furthermore in this case the distance b etween ob jects is

not preserved under various natural representations that are considered

equivalent in standard studies of computational complexity For example

in previous parts of this b o ok when referring to computational problems con

cerning graphs we did not care whether the graphs were represented by their

adjacency matrix or by their incidencelists In contrast these two represen

tations induce very dierent distance measures and corresp ondingly dierent

prop erty testing problems see x Likewise the use of padding and

other trivial syntactic conventions b ecomes problematic eg when using a

signicant amount of padding all ob jects are deemed close to one another

and prop erty testing for any set b ecomes trivial

Since our fo cus is on sublinear time algorithms we may not aord trans

forming the input from one natural format to another Thus representations

that are considered equivalent with resp ect to p olynomialtime algorithms

may not b e equivalent with resp ect to sublinear time algorithms that have

a direct access to the representation of the ob ject For example adjacency

queries and incidence queries cannot emulate one another in small time ie

in time that is sublinear in the number of vertices

Both asp ects are further claried by the examples provided in x

The essential role of the promise Recall that for a xed constant

we consider the promise problem of distinguishing S from S The promise

means that all instances that are neither in S nor far from S ie not in S

are ignored which is essential for sublinear algorithms for natural problems This

makes the prop erty testing task p otentially easier than the corresp onding stan

dard decision task cf x To demonstrate the p oint consider the set S

consisting of strings that have a ma jority of s Then deciding membership in

S requires linear time b ecause random nbit long strings with bnc ones cannot

b e distinguished from random nbit long strings with bnc ones by probing

a sublinear number of lo cations even if randomization and error probability are

allowed see Exercise On the other hand the fraction of s in the input can

b e approximated by a randomized p olylogarithmic time algorithm which yields a

prop erty tester for S see Exercise Thus for some sets deciding membership

requires linear time while prop erty testing can b e done in p olylogarithmic time

The essential role of randomization Referring to the foregoing example we

note that randomization is essential for any sublinear time algorithm that distin

APPROXIMATION

guishes this set S from say S Sp ecically a sublinear time deterministic

algorithm cannot distinguish from any input that has s in each p osition prob ed

by that algorithm on input In general on input x a sublinear time deter

ministic algorithm always reads the same bits of x and thus cannot distinguish x

from any z that agrees with x on these bit lo cations

Note that in b oth cases we are able to prove lowerbounds on the time com

plexity of algorithms This success is due to the fact that these lowerbounds are

actually information theoretic in nature that is these lowerbounds actually refer

to the number of queries p erformed by these algorithms

Two mo dels for testing graph prop erties

In this subsection we consider the complexity of prop erty testing for sets of graphs

that are closed under graph isomorphism such sets are called graph properties In

view of the imp ortance of representation in the context of prop erty testing we

consider two standard representations of graphs cf App endix G which indeed

yield two dierent mo dels of testing graph prop erties

The adjacency matrix representation Here a graph G N E is rep

resented in a somewhat redundant form by an N byN Bo olean matrix

M m such that m if and only if fi j g E

G ij ij

ij N

Bounded incidencelists representation For a xed parameter d a graph

G N E of degree at most d is represented in a somewhat redundant

form by a mapping N d N fg such that u i v if v is

G G

the i neighbor of u and u i if v has less than i neighbors

We stress that the aforementioned representations determine b oth the notion of

distance b etween graphs and the type of queries p erformed by the algorithm As

we shall see the dierence b etween these two representations yields a big dierence

in the complexity of corresp onding prop erty testing problems

Theorem prop erty testing in the adjacency matrix representation For

any xed and each of the fol lowing sets there exists a polylogarithmic time

randomized algorithm that solves the corresponding property testing problem with

resp ect to

For every xed k the set of k colorable graphs

For every xed the set of graphs having a clique resp indep endent

set of density

For every xed the set of N vertex graphs having a cut with at least

N edges

A cut in a graph G N E is a partition S N n S of the set of vertices and the edges

of the cut are the edges with exactly one endp oint in S A bisection is a cut of the graph to two

parts of equal cardinality

CHAPTER RELAXING THE REQUIREMENTS

For every xed the set of N vertex graphs having a bisection with at

most N edges

In contrast for some there exists a graph property in NP for which property

testing with resp ect to requires linear time

The testing algorithms use a constant number of queries where this constant is

p olynomial in the constant We highlight the fact that exact decision pro ce

dure for the corresp onding sets require a linear number of queries The running

time of the aforementioned algorithms hides a constant that is exp onential in their

query complexity except for the case of colorability where the hidden constant

is p olynomial in Note that such dep endencies seem essential since setting

N regains the original nonrelaxed decision problems which with the

exception of colorability are all NPcomplete Turning to the lowerbound we

note that the graph prop erty for which this b ound is proved is not a natural one

Again the lowerbound on the time complexity follows from a lowerbound on the

query complexity

Theorem exhibits a dichotomy b etween graph prop erties for which prop

erty testing is p ossible by a constant number of queries and graph prop erties for

which prop erty testing requires a linear number of queries A combinatorial charac

terization of the graph prop erties for which prop erty testing is p ossible in the adja

cency matrix representation when using a constant number of queries is known

We note that the constant in this characterization may dep end arbitrarily on

and indeed in some cases it is a function growing faster than a tower of

exp onents

Turning back to Theorem we note that the results regarding prop erty

testing for the sets corresp onding to maxcut and minbisection yield approximation

algorithms with an additive error term of N For dense graphs ie N vertex

graphs having N edges this yields a constant factor approximation for the

standard approximation problem as in Denition That is for every constant

c we obtain a cfactor approximation of the problem of maximizing the size of a

cut resp minimizing the size of a bisection in dense graphs On the other hand

the result regarding clique yields a so called dualapproximation for maximum

clique that is we approximate the minimum number of missing edges in the densest

induced graph of a given size

Indeed Theorem is meaningful only for dense graphs The same holds in

general for the adjacency matrix representation Also note that prop erty testing

is trivial under the adjacency matrix representation for any graph prop erty S

satisfying S eg the set of connected graphs the set of Hamiltonian

graphs etc

Describing this fascinating result of Alon et al which refers to the notion of regular

partitions introduced by Szemeredi is b eyond the scop e of the current text

edges may b e accepted if In this mo del all N vertex graphs having less than

and only if there exists such a nondense graph in the predetermined set This trivial decision

regarding nondense graphs is correct b ecause if the set S contains an N vertex graph with less

N N

edges edges then S contains no N vertex graph having less than than

APPROXIMATION

We now turn to the b ounded incidencelists representation which is relevant

only for b ounded degree graphs The problems of maxcut minbisection and clique

as in Theorem are trivial under this representation but graph connectivity

b ecomes nontrivial and the complexity of prop erty testing for the set of bipartite

graphs changes dramatically

Theorem prop erty testing in the b ounded incidencelists representation

The fol lowing assertions refer to the representation of graphs by incidencelists of

length d

For any xed d and there exists a polylogarithmic time randomized

algorithm that solves the property testing problem for the set of connected

graphs of degree at most d

For any xed d and there exists a sublinear randomized algorithm that

solves the property testing problem for the set of bipartite graphs of degree at

most d Specically on input an N vertex graph the algorithm runs for

N time O

For any xed d and some property testing for the set of N vertex

regular bipartite graphs requires N queries

For some xed d and property testing for the set of N vertex colorable

graphs requires N queries

The running time of the algorithms hides a constant that is p olynomial in

Providing a characterization of graph prop erties according to the complexity of the

corresp onding tester in the b ounded incidencelists representation is an interest

ing op en problem

Decoupling the distance from the representation So far we have conned

our attention to the Hamming distance b etween the representations of graphs

This made the choice of representation even more imp ortant than usual ie more

crucial than is common in complexity theory In contrast it is natural to consider

a notion of distance b etween graphs that is indep endent of their representation

For example the distance b etween G V E and G V E can b e dened

as the minimum of the size of symmetric dierence b etween E and the set of edges

in a graph that is isomorphic to G The corresp onding relative distance may b e

dened as the distance divided by jE j jE j or by maxjE j jE j

Beyond graph prop erties

Prop erty testing has b een applied to a variety of computational problems b eyond

the domain of graph theory In fact this area rst emerged in the algebraic domain

where the instances to b e viewed as inputs to the testing algorithm are functions

and the relevant prop erties are sets of algebraic functions The archetypical exam

ple is the set of lowdegree p olynomials that is mvariate p olynomials of total or

individual degree d over some nite eld GF q where m d and q are parameters

CHAPTER RELAXING THE REQUIREMENTS

that may dep end on the length of the input or satisfy some relationships eg

q d m Note that in this case the input is the description of a mvariate

function over GFq which means that it has length q log q Viewing the prob

lem instance as a function suggests a natural measure of distance ie the fraction

of arguments on which the functions disagree as well as a natural way of accessing

the instance ie querying the function for the value of selected arguments

Note that we have referred to these computational problems under a dierent

terminology in x and in x In particular in x we refereed to

the sp ecial case of linear Bo olean functions ie individual degree and q

whereas in x we used the setting q p oly d and m d log d where d is a

b ound on the total degree

Other domains of computational problems in which prop erty testing was stud

ied include geometry eg clustering problems formal languages eg testing

membership in regular sets co ding theory cf App endix E probability the

ory eg testing equality of distributions and combinatorics eg monotone and

junta functions As discuss at the end of x it is often natural to decou

ple the distance measure from the representation of the ob jects ie the way of

accessing the problem instance This is done by introducing a representation

indep endent notion of distance b etween instances which should b e natural in the

context of the problem at hand

Average Case Complexity

Teaching note We view averagecase complexity as referring to the p erformance on

average or typical instances and not as the average p erformance on random instances

This choice is justied in x Thus the current theory may b e termed typicalcase

complexity The term averagecase is retained for historical reasons

Our approach so far including in Section is termed worstcase complex

ity b ecause it refers to the p erformance of p otential algorithms on each legitimate

instance and hence to the p erformance on the worst p ossible instance That is

computational problems were dened as referring to a set of instances and p erfor

mance guarantees were required to hold for each instance in this set In contrast

averagecase complexity allows ignoring a negligible measure of the p ossible in

stances where the identity of the ignored instances is determined by the analysis

of potential solvers and not by the problems statement

A few comments are in place Firstly as just hinted the standard statement

of the worstcase complexity of a computational problem esp ecially one having

a promise may also ignores some instances ie those considered inadmissible

or violating the promise but these instances are determined by the problems

statement In contrast the inputs ignored in averagecase complexity are not

inadmissible in any inherent sense and are certainly not identied as such by

the problems statement It is just that they are viewed as exceptional when

claiming that a sp ecic algorithm solve the problem furthermore these exceptional

AVERAGE CASE COMPLEXITY

instances are determined by the analysis of that algorithm Needless to say these

exceptional instances ought to b e rare ie o ccur with negligible probability

The last sentence raises a couple of issues Firstly a distribution on the set

of admissible instances has to b e sp ecied In fact we shall consider a new type

of computational problems each consisting of a standard computational problem

coupled with a probability distribution on instances Consequently the question of

which distributions should b e considered arises This question and numerous other

denitional issues will b e addressed in x

Before pro ceeding let us sp ell out the rather straightforward motivation to the

study of the averagecase complexity of computational problems It is that in

reallife applications one may b e p erfectly happy with an algorithm that solves

the problem fast on almost all instances that arise in the application That is one

may b e willing to tolerate error provided that it o ccurs with negligible probability

where the probability is taken over the distribution of instances encountered in the

application We stress that a key asp ect in this approach is a go o d mo deling of

the type of distributions of instances that are encountered in natural algorithmic

applications

At this p oint a natural question arises can natural computational problems be

solve eciently when restricting attention to typical instances The b ottomline

of this section is that for a wellmotivated choice of denitions our conjecture is

that the distributional version of NP is not contained in the averagecase or

typicalcase version of P This means that some NP problems are not merely hard

in the worstcase but rather typically hard ie hard on typical instances drawn

from some simple distribution Sp ecically hard instances may o ccur in natural

algorithmic applications and not only in cryptographic or other adversarial

applications that are design on purp ose to pro duce hard instances This conjec

ture motivates the development of an averagecase analogue of NPcompleteness

which will b e presented in this section Indeed the entire section may b e viewed

as an averagecase analogue of Chapter

Organization A ma jor part of our exp osition is devoted to the denitional is

sues that arise when developing a general theory of averagecase complexity These

issues are discussed in x In x we prove the existence of a distri

butional problem that is NPcomplete in the averagecase complexity sense In

x we extend the treatment to randomized algorithms Additional ramica

tions are presented in Section

We highlight two dierences b etween the current context of natural algorithmic applications

and the context of cryptography Firstly in the current context and when referring to problems

that are typically hard the simplicity of the underlying input distribution is of great concern

the simpler this distribution the more app ealing the hardness assertion b ecomes This concern

is irrelevant in the context of cryptography On the other hand see discussion at the b eginning

of Section andor at end of x cryptographic applications require the ability to

eciently generate hard instances together with corresponding solutions

CHAPTER RELAXING THE REQUIREMENTS

The basic theory

In this section we provide a basic treatment of the theory of averagecase com

plexity while p ostp oning imp ortant ramications to Section The basic

treatment consists of the preferred denitional choices for the main concepts as

well as the identication of a complete problem for a natural class of averagecase

computational problems

Denitional issues

The theory of averagecase complexity is more subtle than may app ear in rst

thought In addition to the generic diculty involved in dening relaxations dif

culties arise from the interface b etween standard probabilistic analysis and the

conventions of complexity theory This is most striking in the denition of the

class of feasible averagecase computations Referring to the theory of worstcase

complexity as a guideline we shall address the following asp ects of the analogous

theory of averagecase complexity

Setting the general framework We shall consider distributional problems

which are standard computational problems see Section coupled with

distributions on the relevant instances

Identifying the class of feasible distributional problems Seeking an average

case analogue of classes such as P we shall reject the rst denition that

comes to mind ie the naive notion of average p olynomialtime briey

discuss several related alternatives and adopt one of them for the main treat

ment

Identifying the class of interesting distributional problems Seeking an

averagecase analogue of the class NP we shall avoid b oth the extreme of

allowing arbitrary distributions which collapses averagecase complexity to

worstcase complexity and the opp osite extreme of conning the treatment

to a single distribution such as the uniform distribution

Developing an adequate notion of reduction among distributional problems

As in the theory of worstcase complexity this notion should preserve feasible

solveability in the current distributional context

We now turn to the actual treatment of each of the aforementioned asp ects

Step Dening distributional problems Focusing on decision problems

we dene distributional problems as pairs consisting of a decision problem and a

probability ensemble For simplicity here a probability ensemble fX g is a

We mention that even this choice is not evident Sp ecically Levin see discussion

in advocates the use of a single probability distribution dened over the set of all strings

His argument is that this makes the theory less representationdependent At the time we were

convinced of his argument see but currently we feel that the representationdependent

eects discussed in are legitimate Furthermore the alternative formulation of comes

across as unnatural and tends to confuse some readers

AVERAGE CASE COMPLEXITY

sequence of random variables such that X ranges over f g Thus S fX g

n n

is the distributional problem consisting of the problem of deciding membership in

the set S with resp ect to the probability ensemble fX g The treatment of

search problem is similar see x We denote the uniform probability ensemble

by U fU g that is U is uniform over f g

n n

Step Identifying the class of feasible problems The rst idea that

comes to mind is dening the problem S fX g as feasible on the average

if there exists an algorithm A that solves S such that the average running time

of A on X is b ounded by a p olynomial in n ie there exists a p olynomial p

such that Et X pn where t x denotes the runningtime of A on input

A n A

x The problem with this denition is that it very sensitive to the mo del of

computation and is not closed under algorithmic comp osition Both deciencies

are a consequence of the fact that t may b e p olynomial on the average with

jx j

resp ect to fX g but t may fail to b e so eg consider t x x if

n A

x x and t x x jx x j otherwise coupled with the uniform distribution

over f g We conclude that the average runningtime of algorithms is not a

robust notion We also doubt the naive app eal of this notion and view the typical

running time of algorithms as dened next as a more natural notion Thus we

shall consider an algorithm as feasible if its runningtime is typically p olynomial

We say that A is typically p olynomialtime on X fX g if there exists a

p olynomial p such that the probability that A runs more that pn steps on X

is negligible ie for every p olynomial q and all suciently large n it holds that

Pr t X pn q n The question is what is required in the untypical

A n

cases and two p ossible denitions follow

The simpler option is saying that S fX g is typically feasible if there

exists an algorithm A that solves S such that A is typically p olynomialtime

on X fX g This eectively requires A to correctly solve S on each

instance which is more than was required in the motivational discussion

Indeed if the underlying reasoning is ignoring rare cases then we should

ignore them altogether rather than ignoring them in a partial manner ie

only ignore their aect on the runningtime

The alternative which ts the motivational discussion is saying that S X

is typically feasible if there exists an algorithm A such that A typically

solves S on X in p olynomialtime that is there exists a p olynomial p such

that the probability that on input X algorithm A either errs or runs more

that pn steps is negligible This formulation totally ignores the untypical

instances Indeed in this case we may assume without loss of generality

that A always runs in p olynomialtime see Exercise but we shall not

An alternative choice taken by Levin see discussion in is considering as feasible

wrt X fX g any algorithm that runs in time that is p olynomial in a function that is

linear on the average wrt X that is requiring that there exists a p olynomial p and a function

f g N such that tx px and EX O n This denition is robust ie it

do es not suer from the aforementioned deciencies and is arguably as natural as the naive

denition ie Et X p oly n

n A

CHAPTER RELAXING THE REQUIREMENTS

do so here in order to facilitate viewing the rst option as a sp ecial case of

the current option

We note that b oth alternatives actually dene typical feasibility and not average

case feasibility To illustrate the dierence b etween the two options consider the

distributional problem of deciding whether a uniformly selected nvertex graph

contains a Hamiltonian path Intuitively this problem is typically trivial with

resp ect to the uniform distribution b ecause the algorithm may always say yes and

b e wrong with exp onentially vanishing probability Indeed this trivial algorithm

is admissible by the second approach but not by the rst approach In light of the

foregoing we adopt the second approach

Denition the class tp c P We say that A typically solves S fX g

in p olynomialtime if there exists a polynomial p such that the probability that on

input X algorithm A either errs or runs more that pn steps is negligible We

denote by tp cP the class of distributional problems that are typically solvable in

polynomialtime

Clearly for every S P and every probability ensemble X it holds that S X

tp c P However tp c P contains also distributional problems S X with S P

see Exercises and The big question which underlies the theory of

averagecase complexity is whether natural distributional versions of NP are in

tp c P Thus we turn to identify such versions

Step Identifying the class of interesting problems Seeking to identify

reasonable distributional versions of NP we note that two extreme choices should

b e avoided On one hand we must limit the class of admissible distributions so

to prevent the collapse of averagecase complexity to worstcase complexity by a

selection of a pathological distribution that resides on the worst case instances

On the other hand we should allow for various types of natural distributions rather

than conning attention merely to the uniform distribution which seems misguided

by the naive b elief by which this distribution is the only one relevant to applica

tions Recall that our aim is addressing all p ossible input distributions that may

o ccur in applications and thus there is no justication for conning attention to

the uniform distribution Still arguably the distributions o ccuring in applications

are relatively simple and so we seek to identify a class of simple distributions

One such notion of simple distributions underlies the following denition while

a more lib eral notion will b e presented in x

Denition the class dist NP We say that a probability ensemble X

fX g is simple if there exists a polynomial time algorithm that on any input

In contrast testing whether a given graph contains a Hamiltonian path seems typically

hard for other distributions see Exercise Needless to say in the latter distributions b oth

yesinstances and noinstances app ear with noticeable probability

Recall that a function N N is negligible if for every p ositive p olynomial q and all

suciently large n it holds that n q n We say that A errs on x if Ax diers from the

indicator value of the predicate x S

AVERAGE CASE COMPLEXITY

x f g outputs Pr X x where the inequality refers to the standard lexico

jxj

graphic order of strings We denote by dist NP the class of distributional problems

consisting of decision problems in NP coupled with simple probability ensembles

Note that the uniform probability ensemble is simple but so are many other sim

ple probability ensembles Actually it makes sense to relax the denition such

that the algorithm is only required to output an approximation of Pr X x say

jxj

to within a factor of We note that Denition interprets simplicity

in computational terms sp ecically as the feasibility of answering very basic ques

tions regarding the probability distribution ie determining the probability mass

assigned to a single nbit long string and even to an interval of such strings This

simplicity condition is closely related to b eing p olynomialtime sampleable via a

monotone mapping see Exercise In x we shall consider the more

intuitive and robust class of all p olynomialtime sampleable probability ensembles

and show that it contains all simple ensembles We b elieve that the combina

tion of the results presented in x and x retrosp ectively endorses the

choice underlying Denition We articulate this p oint next

We note that enlarging the class of distributions weakens the conjecture that

the corresp onding class of distributional NP problems contains infeasible prob

lems On the other hand the conclusion that a sp ecic distributional problem is

not feasible b ecomes stronger when the problem b elongs to a smaller class that

corresp onds to a restricted denition of admissible distributions The combined

results of x and x assert that a conjecture that refers to the larger

class of p olynomialtime sampleable ensembles implies a conclusion that refers to

a very simple probability ensemble which resides in the smaller class Thus

the current setting in which b oth the conjecture and the conclusion refer to simple

probability ensembles may b e viewed as just an intermediate step

Indeed the big question in the current context is whether distNP is contained

in tp c P A p ositive answer esp ecially if extended to sampleable ensembles would

deem the PvsNP Question of little practical signicant However our daily ex

p erience as well as much research eort indicate that some NP problems are not

merely hard in the worstcase but rather typically hard This supp orts the

conjecture that distNP is not contained in tp cP

Needless to say the latter conjecture implies P NP and thus we should

not exp ect to see a pro of of it What we may hop e to see is distNP complete

problems that is problems in distNP that are not in tp c P unless the entire class

distNP is contained in tp cP An adequate notion of a reduction is used towards

formulating this p ossibility which in turn is captured by the notion of distNP

complete problems

Step Dening reductions among distributional problems Intuitively

such reductions must preserve averagecase feasibility Thus in addition to the

standard conditions ie that the reduction b e eciently computable and yield a

correct result we require that the reduction resp ects the probability distribu

tion of the corresp onding distributional problems Sp ecically the reduction should

not map very likely instances of the rst starting problem to rare instances of

CHAPTER RELAXING THE REQUIREMENTS

the second target problem Otherwise having a typically p olynomialtime al

gorithm for the second distributional problem do es not necessarily yield such an

algorithm for the rst distributional problem Following is the adequate analogue

of a Co ok reduction ie general p olynomialtime reduction where the analogue

of a Karpreduction manytoone reduction can b e easily derived as a sp ecial case

Teaching note One may prefer presenting in class only the sp ecial case of manyto

one reductions which suces for Theorem See Footnote

Denition reductions among distributional problems We say that the

oracle machine M reduces the distributional problem S X to the distributional

problem T Y if the fol lowing three conditions hold

Eciency The machine M runs in polynomialtime

Validity For every x f g it holds that M x if an only if x S

where M x denotes the output of the oracle machine M on input x and

access to an oracle for T

Domination The probability that on input X and oracle access to T

machine M makes the query y is upperbounded by p oly jy j Pr Y y

jy j

That is there exists a polynomial p such that for every y f g and every

n N it holds that

Pr QX y pjy j Pr Y y

jy j

where Qx denotes the set of queries made by M on input x and oracle access

to T

In addition we require that the reduction does not make too short queries

that is there exists a polynomial p such that if y Qx then p jy j jxj

The lhs of Eq refers to the probability that on input distributed as X

the reduction makes the query y This probability is required not to exceed the

probability that y o ccurs in the distribution Y by more than a p olynomial factor

jy j

in jy j In this case we say that the lhs of Eq is dominated by Pr Y y

jy j

Indeed the domination condition is the only asp ect of Denition that ex

tends b eyond the worstcase treatment of reductions and refers to the distributional

setting The domination condition do es not insist that the distribution induced by

In fact one may relax the requirement and only require that M is typically p olynomialtime

with resp ect to X The validity condition may also b e relaxed similarly

Let us sp ell out the meaning of Eq in the sp ecial case of manytoone reductions ie

M x if and only if f x T where f is a p olynomialtime computable function in this

case Pr QX y is replaced by Pr f X y Assuming that f is onetoone Eq

n n

simplies to Pr X f y pjy j Pr Y y for any y in the image of f Indeed

jy j

jf y j

nothing is required for y not in the image of f

AVERAGE CASE COMPLEXITY

QX equals Y but rather allows some slackness that in turn is b ounded so to

guarantee preservation of typical feasibility see Exercise

We note that the reducibility arguments extensively used in Chapters and

see discussion in Section are actually reductions in the spirit of Deni

tion except that they refer to dierent types of computational tasks

Complete problems

Recall that our conjecture is that distNP is not contained in tp cP which in turn

strengthens the conjecture P NP making infeasibility a typical phenomenon

rather than a worstcase one Having no hop e of proving that dist NP is not

contained in tp c P we turn to the study of complete problems with resp ect to that

conjecture Sp ecically we say that a distributional problem S X is distNP

complete if S X dist NP and every S X distNP is reducible to S X

under Denition

Recall that it is quite easy to prove the mere existence of NPcomplete problems

and many natural problems are NPcomplete In contrast in the current context

establishing completeness results is quite hard This should not b e surprising in

light of the restricted type of reductions allowed in the current context The restric

tion captured by the domination condition requires that typical instances of

one problem should not b e mapp ed to untypical instances of the other problem

However it is fair to say that standard Karpreductions used in establishing NP

completeness results map typical instances of one problem to quite bizarre

instances of the second problem Thus the current subsection may b e viewed as a

study of reductions that do not commit this sin

Theorem dist NP completeness distNP contains a distributional prob

lem T Y such that each distributional problem in distNP is reducible p er Deni

tion to T Y Furthermore the reduction is deterministic and manytoone

Pro of We start by introducing such a problem which is a natural distributional

version of the decision problem S used in the pro of of Theorem Recall

t i

that S contains the instance hM x i if there exists y f g such that M

u it

accepts the input pair x y within t steps We couple S with the quasiuniform

probability ensemble U that assigns to the instance hM x i a probability mass

jM jjxj t

prop ortional to Sp ecically for every hM x i it holds that

jM jjxj

Pr U hM x i

We stress that the notion of domination is incomparable to the notion of statistical resp

computational indistinguishability On one hand domination is a lo cal requirement ie it

compares the two distribution on a p ointbypoint basis whereas indistinguishability is a global

requirement which allows rare exceptions On the other hand domination do es not require

approximately equal values but rather a ratio that is b ounded in one direction Indeed domina

tion is not symmetric We comment that a more relaxed notion of domination that allows rare

violations as in Footnote suces for the preservation of typical feasibility

CHAPTER RELAXING THE REQUIREMENTS

def def

where n jhM x ij jM j jxj t Note that under a suitable enco ding the

ensemble U is indeed simple

The reader can easily verify that the generic reduction used when reducing

any set in NP to S see the pro of of Theorem fails to reduce distNP

to S U Sp ecically in some cases see next paragraph these reductions do

not satisfy the domination condition Indeed the diculty is that we have to

reduce all distNP problems ie pairs consisting of decision problems and simple

distributions to one single distributional problem ie S U Applying the

aforementioned reductions we end up with many distributional versions of S

and furthermore the corresp onding distributions are very dierent and are not

necessarily dominated by a single distribution

Let us take a closer lo ok at the aforementioned generic reduction when applied

to an arbitrary S X distNP This reduction maps an instance x to a triple

p jxj

M x where M is a machine verifying membership in S while using

S S

adequate NPwitnesses and p is an adequate p olynomial The problem is that x

jxj

may have relatively large probability mass ie it may b e that Pr X x

jxj

p jxj p jxj

S S

while M x has uniform probability mass ie hM x i has

S S

jxj

probability mass smaller than in U This violates the domination condition

see Exercise and thus an alternative reduction is required

The key to the alternative reduction is an eciently computable enco ding of

strings taken from an arbitrary simple distribution by strings that have a similar

probability mass under the uniform distribution This means that the enco ding

should shrink strings that have relatively large probability mass under the origi

nal distribution Sp ecically this enco ding will map x taken from the ensemble

fX g to a co deword x of length that is upp erb ounded by the logarithm of

jx j

Pr X x ensuring that Pr X x O Accordingly the reduction

jxj jxj

p jxj

will map x to a triple M x where jx j O log Pr X x

jxj

and M is an algorithm that given x and x rst veries that x is a prop er

enco ding of x and next applies the standard verication ie M of the problem

S Such a reduction will b e shown to satisfy all three conditions ie eciency

validity and domination Thus instead of forcing the structure of the original

distribution X on the target distribution U the reduction will incorp orate the

structure of X in the reduced instance A key ingredient in making this p ossible is

the fact that X is simple as p er Denition

With the foregoing motivation in mind we now turn to the actual pro of that

is proving that any S X distNP is reducible to S U The following

technical lemma is the basis of the reduction In this lemma as well as in the

sequel it will b e convenient to consider the accumulative distribution function

def

of the probability ensemble X That is we consider x Pr X x and

jxj

note that f g is p olynomialtime computable b ecause X satises

t k

For example we may enco de hM x i where M f g and x

t t

Pr U hM x i equals f g by the string Then

k k

jM j jM j jM jjxj jxj

i jfM f g M M gj jfx f g x xgj

jM jjxjt

where i is the ranking of fk k g among all subsets of k t k t

AVERAGE CASE COMPLEXITY

Denition

Co ding Lemma Let f g b e a p olynomialtime computable function

that is monotonically nondecreasing over f g for every n ie x x

jx j n n

for any x x f g For x f g n f g let x denote the string

preceding x in the lexicographic order of nbit long strings Then there exist an

enco ding function C that satises the following three conditions

Compression For every x it holds that jC xj minfjxj log x g

def def

n n

where x x x if x fg and otherwise

Ecient Enco ding The function C is computable in p olynomialtime

Unique Deco ding For every n N when restricted to f g the function

C is onetoone ie if C x C x and jxj jx j then x x

jxj

Pro of The function C is dened as follows If x then C x x

jxj

ie in this case x serves as its own enco ding Otherwise ie x

then C x z where z is chosen such that jz j log x and the mapping

of nbit strings to their enco ding is onetoone Lo osely sp eaking z is selected to

equal the shortest binary expansion of a number in the interval x x x

Bearing in mind that this interval has length x and that the dierent intervals

are disjoint we obtain the desired enco ding Details follows

jxj

We fo cus on the case that x and detail the way that z is selected

jxj

for the enco ding C x z If x and x then we let z b e the

longest common prex of the binary expansions of x and x for example

if and then C z with z

Thus in this case z is in the interval x x ie x z x

jxj

For x we let z b e the longest common prex of the binary expansions of

and x and again z is in the relevant interval ie x Finally for x such

that x and x we let z b e the longest common prex of the binary

jxj

expansions of x and and again z is in x x b ecause

jxj jxj

x and x x imply that x x

jxj

Note that if x x then x

We now verify that the foregoing C satises the conditions of the lemma We

jxj

start with the compression condition Clearly if x then jC xj

jxj

jxj log x On the other hand supp ose that x and

jxj

let us fo cus on the subcase that x and x Let z z z b e

the longest common prex of the binary expansions of x and x Then

x z u and x z v where u v f g We infer that

p oly jxj

X X X

i jz j i i

z x x x z

i i

The lemma actually refers to f g for any xed value of n but the eciency condition

is stated more easily when allowing n to vary and using the standard asymptotic analysis of

algorithms Actually the lemma is somewhat easier to state and establish for p olynomial

time computable functions that are monotonically nondecreasing over f g rather than over

f g See further discussion in Exercise

CHAPTER RELAXING THE REQUIREMENTS

and jz j log x jxj follows Thus jC xj minjxj log x

holds in b oth cases Clearly C can b e computed in p olynomialtime by computing

x and x Finally note that C satises the unique deco ding condition by

separately considering the two aforementioned cases ie C x x and C x

z Sp ecically in the second case ie C x z use the fact that x

z x

To obtain an enco ding that is onetoone when applied to strings of dierent

def

lengths we augment C in the obvious manner that is we consider C x

jxj C x which may b e implemented as C x C x where

is the binary expansion of jxj Note that jC xj O log jxj jC xj

is onetoone and that C

The machine asso ciated with S X Let b e the accumulative probability func

tion asso ciated with the probability ensemble X and M b e the p olynomialtime

machine that veries membership in S while using adequate NPwitnesses ie

p olyjxj

x S if and only if there exists y f g such that M x y Using

the enco ding function C we introduce an algorithm M with the intension of

reducing the distributional problem S X to S U such that all instances of

S are mapp ed to triples in which the rst element equals M Machine M

S S

is given an alleged enco ding under C of an instance to S along with an alleged

pro of that the corresp onding instance is in S and veries these claims in the ob

vious manner That is on input x and hx y i machine M rst veries that

x C x and next veriers that x S by running M x y Thus M veries

S S

x x S g while using pro ofs of the form hx y i membership in the set S fC

such that M x y for the instance C x

pjxj

x The reduction We maps an instance x of S to the triple M C

def

where pn p n p n such that p is a p olynomial representing the running

S C S

time of M and p is a p olynomial representing the runningtime of the enco ding

S C

algorithm

Analyzing the reduction Our goal is proving that the foregoing mapping constitutes

a reduction of S X to S U We verify the corresp onding three requirements

of Denition

Using the fact that C is p olynomialtime computable and noting that p

is a p olynomial it follows that the foregoing mapping can b e computed in

p olynomialtime

Recall that on input x hx y i machine M accepts if and only if x

C x and M accepts x y within p jxj steps Using the fact that C x

S S

uniquely determines x it follows that x S if and only if there exists a string

y of length at most pjxj such that M accepts C x hx y i in at most

Note that jy j p oly jxj but jxj p oly jC xj do es not necessarily hold and so S is not

necessarily in NP As we shall see the latter p oint is immaterial

AVERAGE CASE COMPLEXITY

pjxj

pjxj steps Thus x S if and only if M C x S and the

S u

validity condition follows

In order to verify the domination condition we rst note that the foregoing

mapping is onetoone b ecause the transformation x C x is oneto

one Next we note that it suces to consider instances of S that have

a preimage under the foregoing mapping since instances with no preimage

trivially satisfy the domination condition Each of these instances ie each

image of this mapping is a triple with the rst element equal to M and

the second element b eing an enco ding under C By the denition of U for

pjxj n

every such image hM C x i f g it holds that

pjxj jM jjC xj

Pr U hM C x i

jC xjO log jxj

c n

jM j

where c is a constant dep ending only on S and ie on the

distributional problem S X Thus for some p ositive p olynomial q we

have

pjxj jC xj

Pr U hM C x i q n

By virtue of the compression condition of the Co ding Lemma we have

jC xj minjxjlog x

It follows that

jC xj

Pr X x

jxj

pjxj

x i Recalling that x is the only preimage that is mapp ed to hM C

and combining Eq we establish the domination condition

The theorem follows

Reections The pro of of Theorem demonstrates the fact that the re

duction used in the pro of of Theorem do es not introduce much structure

in the reduced instances ie do es not reduce the original problem to a highly

structured sp ecial case of the target problem Put in other words unlike more

advanced worstcase reductions this reduction do es not map random ie uni

formly distributed instances to highly structured instances which o ccur with neg

ligible probability under the uniform distribution Thus the reduction used in the

pro of of Theorem suces for reducing any distributional problem in distNP

to a distributional problem consisting of S coupled with some simple probability

ensemble see Exercise

However Theorem states more than the latter assertion That is it states

that any distributional problem in distNP is reducible to the same distributional

Note that this cannot b e said of most known Karpreductions which do map random instances

to highly structured ones Furthermore the same structure creating prop erty holds for the

reductions obtained by Exercise

CHAPTER RELAXING THE REQUIREMENTS

version of S Indeed the eort involved in proving Theorem was due to

the need for mapping instances taken from any simple probability ensemble which

may not b e the uniform ensemble to instances distributed in a manner that is

dominated by a single probability ensemble ie the quasiuniform ensemble U

Once we have established the existence of one distNP complete problem we

may establish the distNP completeness of other problems in distNP by reduc

ing some dist NP complete problem to them and relying on the transitivity of

reductions see Exercise Thus the diculties encountered in the pro of

of Theorem are no longer relevant Unfortunately a seemingly more severe

diculty arises almost all know reductions in the theory of NPcompleteness work

by introducing much structure in the reduced instances ie they actually reduce

to highly structured sp ecial cases Furthermore this structure is to o complex in

the sense that the distribution of reduced instances do es not seem simple in the

sense of Denition Designing reductions that avoid the introduction of such

structure has turned out to b e quite dicult still several such reductions are cited

Probabilistic versions

The denitions in x can b e extended so that to account also for randomized

computations For example extending Denition we have

Denition the class tp c BPP For a probabilistic algorithm A a Boolean

function f and a timebound function t N N we say that the string x is tbad for

A with resp ect to f if with probability exceeding on input x either Ax f x

or A runs more that tjxj steps We say that A typically solves S fX g in

probabilistic p olynomialtime if there exists a polynomial p such that the probability

that X is pbad for A with respect to the characteristic function of S is negligible

We denote by tp c BPP the class of distributional problems that are typically solvable

in probabilistic polynomialtime

The denition of reductions can b e similarly extended This means that in Deni

tion b oth M x and Qx mentioned in Items and resp ectively are

random variables rather than xed ob jects Furthermore validity is required to

hold for every input only with probability where the probability space refers

only to the internal coin tosses of the reduction Randomized reductions are closed

under comp osition and preserve typical feasibility see Exercise

Randomized reductions allow the presentation of a dist NP complete problem

that refers to the p erfectly uniform ensemble Recall that Theorem estab

lishes the distNP completeness of S U where U is a quasiuniform ensemble

t jM jjxj t

ie Pr U hM x i where n jhM x ij We rst

note that S U can b e randomly reduced to S U where S fhM x z i

u u

jz j jM jjxjjz j

hM x i S g and Pr U hM x z i for every hM x z i

n t

f g The randomized reduction consists of mapping hM x i to hM x z i

where z is uniformly selected in f g Recalling that U fU g denotes the

uniform probability ensemble ie U is uniformly distributed on strings of length

n and using a suitable enco ding we get

AVERAGE CASE COMPLEXITY

Prop osition There exists S NP such that every S X distNP is

randomly reducible to S U

Pro of Sketch By the forgoing discussion every S X distNP is randomly

reducible to S U where the reduction go es through S U Thus we fo cus

on reducing S U to S U where S NP is dened as follows The string

u u u

bin jujbin jv juvw is in S if and only if hu v w i S and dlog juv w je

u u

where bin i denotes the bit long binary enco ding of the integer i ie

the enco ding is padded with zeros to a total length of The reduction maps

hM x z i to the string bin jxjbin jM jMxz where dlog jM j jxj jz je

Noting that this reduction satises all conditions of Denition the prop osi

tion follows

Ramications

In our opinion the most problematic asp ect of the theory describ ed in Section

is the denition of simple probability ensembles which in turn restricts the def

inition of distNP Denition This restriction strengthens the conjecture

that distNP is not contained in tp c BPP which means that it weakens conditional

results that are based on this conjecture An app ealing extension of the class

distNP is presented in x where it is shown that if the extended class is

not contained in tp c BPP then dist NP itself is not contained in tp c BPP Thus

distNP complete problems enjoy the b enet of b oth b eing in the more restricted

class ie dist NP and b eing hard as long as some problems in the extended class

is hard

Another extension app ears in x where we extend the treatment from

decision problems to search problems This extension is motivated by the realiza

tion that search problem are actually of greater imp ortance to reallife applications

cf Section and hence a theory motivated by reallife applications must

address such problems as we do next

Prerequisites For the technical development of x we assume familiar

ity with the notion of unique solution and results regarding it as presented in

Section For the technical development of x we assume familiarity

with hashing functions as presented in App endix D

Search versus Decision

Indeed as in the case of worstcase complexity search problems are at least as im

p ortant as decision problems Thus an averagecase treatment of search problems

is indeed called for We rst present distributional versions of PF and PC cf

Section following the underlying principles of the denitions of tp c P and

distNP

Denition the classes tp c PF and distPC As in Section we con

sider only polynomially bounded search problems that is binary relations R

CHAPTER RELAXING THE REQUIREMENTS

f g f g such that for some polynomial q it holds that x y R implies

def def

fx R x g jy j q jxj Recall that R x fy x y R g and S

A distributional search problem consists of a polynomially bounded search prob

lem coupled with a probability ensemble

The class tp c PF consists of al l distributional search problems that are typ

ically solvable in polynomialtime That is R fX g tp c PF if there

exists an algorithm A and a polynomial p such that the probability that on

input X algorithm A either errs or runs more that pn steps is negligible

where A errs on x S if Ax R x and errs on x S if Ax

R R

A distributional search problem R X is in distPC if R PC and X is

simple as in Denition

Likewise the class tp cBPPF consists of all distributional search problems that

are typically solvable in probabilistic p olynomialtime cf Denition The

denitions of reductions among distributional problems presented in the context of

decision problem extend to search problems

Fortunately as in the context of worstcase complexity the study of distribu

tional search problems reduces to the study of distributional decision problems

Theorem reducing search to decision distPC tp cBPPF if and only if

distNP tp c BPP Furthermore every problem in distNP is reducible to some

problem in distPC and every problem in distPC is randomly reducible to some

problem in distNP

Pro of Sketch The furthermore part is analogous to the actual contents of the

pro of of Theorem see also Step in the pro of of Theorem Indeed the

reduction of NP to PC presented in the pro of of Theorem extends to the current

context Sp ecically for any S NP we consider a relation R PC such that

S fx R x g and note that for any probability ensemble X the identity

transformation reduces S X to R X

A diculty arises in the opp osite direction Recall that in the pro of of The

orem we reduced the search problem of R PC to deciding membership in

def

S fhx y i y st x y y R g NP The diculty encountered here is

that on input x this reduction makes queries of the form hx y i where y is a

prex of some string in R x These queries may induce a distribution that is not

dominated by any simple distribution Thus we seek an alternative reduction

As a warmup let us assume for a moment that R has unique solutions in the

sense of Denition that is for every x it holds that jR xj In this case

we may easily reduce the search problem of R PC to deciding membership in

S NP where hx i i S if and only if R x contains a string in which the

R R

i bit equals Sp ecically on input x the reduction issues the queries hx i i

where i with p oly jxj and f g which allows for determining the

single string in the set R x f g whenever jR xj The p oint is that this

reduction can b e used to reduce any R X distPC having unique solutions to

AVERAGE CASE COMPLEXITY

S X distNP where X equally distributes the probability mass of x under

X to al l the tuples hx i i that is for every i and f g it holds that

Pr X hx i i equals Pr X x

jxj

jhxi ij

Unfortunately in the general case R may not have unique solutions Nev

ertheless applying the main idea that underlies the pro of of Theorem this

diculty can b e overcome We rst note that the foregoing mapping of instances

X distNP of the distributional problem R X distPC to instances of S

satises the eciency and domination conditions even in the case that R do es not

have unique solutions What may p ossibly fail in the general case is the validity

condition ie if jR xj then we may fail to recover any element of R x

Recall that the main part of the pro of of Theorem is a randomized reduction

that maps instances of R to triples of the form x m h such that m is uniformly

distributed in and h is uniformly distributed in a family of hashing function

m m

is as in App endix D Furthermore if R x where p oly jxj and H H

then with probability over the choices of m and h H there exists a

def

m m

unique y R x such that hy Dening R x m h fy R hy g

this yields a randomized reduction of the search problem of R to the search problem

of R such that with noticeable probability the reduction maps instances that have

solutions to instances having a unique solution Furthermore this reduction can b e

used to reduce any R X distPC to R X distPC where X distributes the

probability mass of x under X to al l the triples x m h such that for every m

m m

x m h equals Pr X x jH j and h H it holds that Pr X

jxj

jxmhj

Note that with a suitable enco ding X is indeed simple

The theorem follows by combining the two aforementioned reductions That is

we rst apply the randomized reduction of R X to R X and next reduce the

resulting instance to an instance of the corresp onding decision problem S X

where X is obtained by mo difying X rather than X The combined randomized

mapping satises the eciency and domination conditions and is valid with notice

able probability The error probability can b e made negligible by straightforward

amplication see Exercise

Simple versus sampleable distributions

Recall that the denition of simple probability ensembles underlying Denition

requires that the accumulating distribution function is p olynomialtime computable

Recall that f g is called the accumulating distribution function of

def

X fX g if for every n N and x f g it holds that x Pr X x

n n

where the inequality refers to the standard lexicographic order of nbit strings

As argued in x the requirement that the accumulating distribution func

tion is p olynomialtime computable imp oses severe restrictions on the set of ad

missible ensembles Furthermore it seems that these simple ensembles are indeed

Recall that the probability of an event is said to b e noticeable in a relevant parameter if it is

greater than the recipro cal of some p ositive p olynomial In the context of randomized reductions

the relevant parameter is the length of the input to the reduction

CHAPTER RELAXING THE REQUIREMENTS

simple in some intuitive sense and hence represent a minimalistic mo del of distri

butions that may o ccur in practice Seeking a maximalistic mo del of distributions

that o ccur in practice we consider the notion of p olynomialtime sampleable en

sembles underlying Denition We b elieve that the class of such ensembles

contains all distributions that may o ccur in practice b ecause we b elieve that the

real world should b e mo deled as a feasible rather than an arbitrary randomized

pro cess

Denition sampleable ensembles and the class sampNP We say that a

probability ensemble X fX g is p olynomialtime sampleable if there exists

a probabilistic polynomialtime algorithm A such that for every x f g it holds

jxj

that Pr A x Pr X x We denote by sampNP the class of distri

jxj

butional problems consisting of decision problems in NP coupled with sampleable

probability ensembles

We rst note that all simple probability ensembles are indeed sampleable see

Exercise and thus dist NP sampNP On the other hand it seems that

there are sampleable probability ensembles that are not simple see Exercise

In fact extending the scop e of distributional problems from dist NP to sampNP

allows proving that every NPcomplete problem has a distributional version in

sampNP that is distNP hard see Exercise Furthermore it is p ossible to

prove that all natural NPcomplete problem have distributional versions that are

sampNP complete

Theorem sampNP completeness Suppose that S NP and that every

set in NP is reducible to S by a Karpreduction that does not shrink the input

Then there exists a polynomialtime sampleable ensemble X such that any problem

in sampNP is reducible to S X

The pro of of Theorem is based on the observation that there exists a polynomial

time sampleable ensemble that dominates al l polynomialtime sampleable ensembles

The existence of this ensemble is based on the notion of a universal sampling

machine For further details see Exercise Recall that when proving The

orem we did not establish an analogous result for simple ensembles but

rather capitalized on the universal nature of S

Theorem establishes a rich theory of sampNP completeness but do es not

relate this theory to the previously presented theory of distNP completeness see

Figure This is done in the next theorem which asserts that the existence of

typically hard problems in sampNP implies their existence in dist NP

Theorem sampNP completeness versus distNP completeness If sampNP

is not contained in tp cBPP then distNP is not contained in tp c BPP

Thus the two typicalcase complexity versions of the PvsNP Question are

equivalent That is if some sampleable distribution versions of NP are not

typically feasible then some simple distribution versions of NP are not typically

AVERAGE CASE COMPLEXITY

tpcBPP

sampNP

distNP sampNP-complete [Thm 10.23]

distNP-complete [Thm 10.17]

Figure Two types of averagecase completeness

feasible In particular if sampNP complete problems are not in tp cBPP then

distNP complete problems are not in tp c BPP

The foregoing assertions would all follow if sampNP were randomly reducible

to distNP ie if every problem in sampNP were reducible under a randomized

version of Denition to some problem in distNP but unfortunately we

do not know whether such reductions exist Yet underlying the pro of of Theo

rem is a more lib eral notion of a reduction among distributional problem

Pro of Sketch We shall prove that if distNP is contained in tp cBPP then the

same holds for sampNP ie sampNP is contained in tp c BPP Actually we

shall show that if distPC is contained in tp c BPPF then the sampleable version of

distPC denoted sampPC is contained in tp cBPPF and refer to Exercise

Sp ecically we shall show that under a relaxed notion of a randomized reduction

every problem in sampPC is reduced to some problem in distPC Lo osely sp eaking

this relaxed notion of a randomized reduction only requires that the validity and

domination conditions of Denition when adapted to randomized reduc

tions hold with resp ect to a noticeable fraction of the probability space of the

reduction We start by formulating this notion when referring to distributional

search problems

Teaching note The following pro of is quite involved and is b etter left for advanced

reading Its main idea is related in one of the central ideas underlying the currently

known pro of of Theorem This fact as well as numerous other applications of this

idea provide a go o d motivation for getting familiar with this idea

We warn that the existence of such a relaxed reduction b etween two sp ecic distributional

problems do es not necessarily imply the existence of a corresp onding standard averagecase

reduction Sp ecically although standard validity can b e guaranteed for problems in PC by

rep eated invocations of the reduction such a pro cess will not redeem the violation of the standard

domination condition

CHAPTER RELAXING THE REQUIREMENTS

Denition A relaxed reduction of the distributional problem R X to the distri

butional problem T Y is a probabilistic p olynomialtime oracle machine M that

satises the following conditions

Notation For every x f g we denote by mjxj p oly jxj the number of

internal coin tosses of M on input x and denote by M x r the execution

of M on input x internal coins r f g and oracle access to T

Validity For some noticeable function N ie n p olyn it

mjxj

holds that for every x f g there exists a set f g of size at

mjxj

least jxj such that for every r the reduction yields a correct

T T

answer ie M x r R x if R x and M x r otherwise

Domination There exists a p ositive p olynomial p such that for every y f g

and every n N it holds that

Pr Q X y pjy j Pr Y y

jy j

where Q x is a random variable dened over the set of the validity

condition representing the set of queries made by M on input x and oracle

access to T That is Q x is dened by uniformly selecting r and

considering the set of queries made by M on input x internal coins r and

oracle access to T In addition as in Denition we also require that

the reduction do es not make to o short queries

The reader may verify that this relaxed notion of a reduction preserves typical

feasibility that is for R PC if there exists a relaxed reduction of R X to

T Y and T Y is in tp cBPPF then R X is in tp c BPPF The key observation

is that the analysis may discard the case that on input x the reduction selects

coins not in Indeed the queries made in that case may b e untypical and the

answers received may b e wrong but this is immaterial What matter is that on

input x with noticeable probability the reduction selects coins in and pro duces

typical with resp ect to Y queries by virtue of the relaxed domination condition

Such typical queries are answered correctly by the algorithm that typically solves

T Y and if x has a solution then these answers yield a correct solution to x

by virtue of the relaxed validity condition Thus if x has a solution then with

noticeable probability the reduction outputs a correct solution On the other hand

the reduction never outputs a wrong solution even when using coins not in

b ecause incorrect solutions are detected by relying on R PC

Our goal is presenting for every R X sampPC a relaxed reduction of

R X to a related problem R X distPC where as usual X fX g

and X fX g

An oversimplied case For starters suppose that X is uniformly distributed on

some set S f g and that there is a polynomialtime computable and invert

ible mapping of S to f g where n log jS j Then mapping x to

n n

jxjjxj

x we obtain a reduction of R X to R X where X is uniform

nn n nn

over f v v f g g and R v R v or equivalently

AVERAGE CASE COMPLEXITY

jxjjxj

R x R x Note that X is a simple ensemble and R PC

hence R X distPC Also note that the foregoing mapping is indeed a valid

reduction ie it satises the eciency validity and domination conditions Thus

R X is reduced to a problem in distPC and indeed the relaxation was not used

here

A simple but more instructive case Next we drop the assumption that there is

a p olynomialtime computable and invertible mapping of S to f g but

maintain the assumption that X is uniform on some set S f g and as

n n

sume that jS j is easily computable from n In this case we may map

x f g to its image under a suitable randomly chosen hashing function h which

in particular maps nbit strings to nbit strings That is we randomly map x to

h hx where h is uniformly selected in a set H of suitable hash func

tions see App endix D This calls for redening R such that R h v

corresp onds to the preimages of v under h that are in S Assuming that h is a

n nn

mapping of S to f g we may dene R h v R x where x is

the unique string satisfying x S and hx v where the condition x S may

n n

b e veried by providing the internal coins of the sampling procedure that generate

x Denoting the sampling pro cedure of X by S and letting S r denote the

output of S on input and internal coins r we actually redene R as

nn n n

R h v fhr y i hS r v y R S r g

jxjjxj jxj

We note that hr y i R h hx yields a solution y R x if S r

jxj

x but otherwise all b ets are o as y will b e a solution for S r x Now

although typically h will not b e a mapping of S to f g it is the case that

for each x S with constant probability over the choice of h it holds that hx

has a unique preimage in S under h See the pro of of Theorem In this

jxjjxj jxj

case hr y i R h hx implies S r x which in turn implies

y R x We claim that the randomized mapping of x to h hx where

jxj

yields a relaxed reduction of R X to R X h is uniformly selected in H

jxj

nn n

where X is uniform over H f v v f g g Needless to say the

claim refers to the reduction that makes the query h hx and returns y

if the oracle answer equals hr y i and y R x

jxj

for The claim is proved by considering the set of choices of h H

jxj

which x S is the only preimage of hx under h that resides in S ie

n n

jfx S hx hxgj In this case ie h it holds that hr y i

n x

jxjjxj jxj

R h hx implies that S r x and y R x and the relaxed

validity condition follows The relaxed domination condition follows by noting

jxj jxjjxj

that Pr X x that x is mapp ed to h hx with proba

jxj

jxjjxj

j and that x is the only preimage of h hx under the bility jH

jxj

h mapping among x S such that

n x

Before going any further let us highlight the imp ortance of hashing X to n

bit strings On one hand this mapping is suciently onetoone and thus with

constant probability the solution provided for the hashed instance ie hx yield

a solution for the original instance ie x This guarantees the validity of the re

CHAPTER RELAXING THE REQUIREMENTS

duction On the other hand for a typical h the mapping of X to hX covers the

n n

relevant range almost uniformly This guarantees that the reduction satises the

domination condition Note that these two phenomena imp ose conicting require

ments that are b oth met at the correct value of that is the onetoone condition

requires n log jS j whereas an almost uniform cover requires n log jS j

n n

Also note that n log Pr X x for every x in the supp ort of X the

n n

latter quantity will b e in our fo cus in the general case

The general case Finally get rid of the assumption that X is uniformly distributed

over some subset of f g All that we know is that there exists a probabilistic

p olynomialtime sampling algorithm S such that S is distributed identi

cally to X In this general case we map instances of R X according to their

probability mass such that x is mapp ed to an instance of R that consists of

h hx and additional information where h is a random hash function mapping

nbit long strings to bit long strings such that

def

dlog Pr X xe

jxj

strings in the supp ort of Since in the general case there may b e more than

X we need to augment the reduced instance in order to ensure that it is uniquely

asso ciated with x The basic idea is augmenting the mapping of x to h hx with

additional information that restricts X to strings that o ccur with probability at

Indeed when X is restricted in this way the value of hX uniquely least

n n

determines X

Let q n denote the randomness complexity of S and S r denote the out

n q n

put of S on input and internal coin tosses r f g Then we randomly

q jxj jxj

and h f g map x to h hx h v where h f g f g

q jxj q jxj

x x

is uniformly dis are random hash functions and v f g f g

tributed The instance h v h v of the redened search problem R has solutions

n n

that consists of pairs hr y i such that hS r v h r v and y R S r

As we shall see this augmentation guarantees that with constant probability over

the choice of h h v the solutions to the reduced instance h hx h v corre

sp ond to the solutions to the original instance x

The foregoing description assumes that on input x we can determine

which is an assumption that cannot b e justied Instead we select uniformly

in f q jxjg and so with noticeable probability we do select the correct

value ie Pr q jxj p olyjxj For clarity we make n

and explicit in the reduced instance Thus we randomly map x f g to

q n

n n

h hx h v f g where f q ng h H h H

q n

and v f g are uniformly distributed in the corresp onding sets This

mapping will b e used to reduce R X to R X where R and X fX g

n N

As in other places a suitable enco ding will b e used such that the reduction maps strings of the

same length to strings of the same length ie nbit string are mapp ed to n bit strings for n

n n q n

p oly n For example we may enco de h h hx h v i as hhi hhxi hh i hv i

where each hw i denotes an enco ding of w by a string of length n n q n

AVERAGE CASE COMPLEXITY

are redened yet again Sp ecically we let

n n n

R h v h v fhr y i hS r v h r v y R S r g

and X assigns equal probability to each X for f ng where each

q n

X is isomorphic to the uniform distribution over H f g H

q n

f g Note that indeed R X distPC

The aforementioned randomized mapping is analyzed by considering the correct

choice for that is on input x we fo cus on the choice Under this

conditioning as we shall show with constant probability over the choice of h h

and v the instance x is the only value in the support of X that is mapped to

n n

h hx h v and satises fr hS r hx h r v g It

h hx h v follows that for such h h and v any solution hr y i R

satises S r x and thus y R x which means that the relaxed validity

condition is satised The relaxed domination condition is satised to o b ecause

conditioned on and for such h h v the probability that X is mapp ed to

x n

n n

x x

h hx h v approximately equals Pr X h hx h v

We now turn to analyze the probability over the choice of h h and v that the

h hx h v instance x is the only value in the supp ort of X that is mapp ed to

and satises fr hS r hx h r v g Firstly we note that

n q n

and thus with constant probability over the choice jfr S r xgj

q n

n q n

there exists r that satises S r x and v f g of h H

q n

and h r v Next we note that with constant probability over the choice of

x x

it holds that x is the only string having probability mass at least h H

under X that is mapp ed to hx under h Finally we prove that with constant

q n

probability over the choice of h H and even when con and h H

q n

ditioning on the previous items the mapping r hS r h r maps the

q n n

g to f g in an almost manner Sp eci set fr Pr X S r

cally with constant probability no other r is mapp ed to the aforementioned pair

hx v Thus the claim follows and so do es the theorem

Reection Theorem implies that if sampNP is not contained in tp cBPP

then every distNP complete problem is not in tp c BPP This means that the

hardness of some distributional problems that refer to sampleable distributions im

plies the hardness of some distributional problems that refer to simple distributions

Furthermore by Prop osition this implies the hardness of distributional prob

lems that refer to the uniform distribution Thus hardness with resp ect to some

distribution in an utmost wide class which arguably captures all distributions that

may o ccur in practice implies hardness with resp ect to a single simple distribution

which arguably is the simplest one

Relation to oneway functions We note that the existence of oneway func

tions see Section implies the existence of problems in sampPC that are not in

tp c BPPF which in turn implies the existence of such problems in dist PC Sp ecif

ically for a lengthpreserving oneway function f consider the distributional search

CHAPTER RELAXING THE REQUIREMENTS

problem R ff U g where R ff r r r f g g On the other

f n f

hand it is not known whether the existence of a problem in sampPC n tp cBPPF

implies the existence of oneway functions In particular the existence of a prob

lem R X in sampPC n tp c BPPF represents the feasibility of generating hard

instances for the search problem R whereas the existence of oneway function rep

resents the feasibility of generating instancesolution pairs such that the instances

are hard to solve see Section Indeed the gap refers to whether or not hard

instances can be eciently generated together with corresponding solutions Our

world view is thus depicted in Figure where lower levels indicate seemingly

weaker assumptions

one-way functions exist

distNP is not in tpcBPP (equiv., sampNP is not in tpcBPP)

P is different than NP

Figure Worstcase vs averagecase assumptions

Chapter Notes

In this chapter we presented two dierent approaches to the relaxation of com

putational problems The rst approach refers to the concept of approximation

while the second approach refers to averagecase analysis We demonstrated that

various natural notions of approximation can b e cast within the standard frame

works where the framework of promise problems presented in Section is the

most nonstandard framework we used and it suces for casting gap problems and

prop erty testing In contrast the study of averagecase complexity requires the

introduction of a new conceptual framework and addressing of various denitional

issues

A natural question at this p oint is what have we gained by relaxing the re

quirements In the context of approximation the answer is mixed in some natural

cases we gain a lot ie we obtained feasible relaxations of hard problems while

in other natural cases we gain nothing ie even extreme relaxations remain as

intractable as the original version In the context of averagecase complexity the

negative side seems more prevailing at least in the sense of b eing more system

atic In particular assuming the existence of oneway functions every natural

Note that the distribution f U is uniform in the sp ecial case that f is a p ermutation over

f g

AVERAGE CASE COMPLEXITY

NPcomplete problem has a distributional version that is hard where this version

refers to a sampleable ensemble Furthermore in this case some problems in NP

have hard distributional versions that refer to the uniform distribution

Another dierence b etween the two approaches is that the theory of approxima

tion seems to lack a comprehensive structure whereas the theory of averagecase

complexity seems to have a to o rigid structure which seems to foil attempts to

present more app ealing distNP complete problems

Approximation

The following bibliographic comments are quite laconic and neglect mentioning

various imp ortant works including credits for some of the results mentioned in our

text As usual the interested reader is referred to corresp onding surveys

Search or Optimization The interest in approximation algorithms increased

considerably following the demonstration of the NPcompleteness of many nat

ural optimization problems But with some exceptions most notably

the systematic study of the complexity of such problems stalled till the discov

ery of the PCP connection see Section by Feige Goldwasser Lovasz and

Safra Indeed the relatively tight inapproximation results for maxClique

maxSAT and the maximization of linear equations due to Hastad

build on previous work regarding PCP and their connection to approximation cf

eg Sp ecically Theorem is due to while Theo

rems and are due to The b est known inapproximation result for

minimum Vertex Cover see Theorem is due to but we doubt it is tight

see eg Reductions among approximation problems were dened and

presented in see Exercise which presents a ma jor technique introduced

in For general texts on approximation algorithms and problems as discussed

in Section the interested reader is referred to the surveys collected in

A comp endium of NP optimization problems is available at

Recall that a dierent type of approximation problems which are naturally as

so ciated with search problems were treated in Section We note that an anal

ogous denitional framework eg gap problems p olynomialtime approximation

schemes etc is applicable also to the approximate counting problems considered

in Section

Prop erty testing The study of prop erty testing was initiated by Rubinfeld and

Sudan and reinitiated by Goldreich Goldwasser and Ron While the

fo cus of was on algebraic prop erties such as lowdegree p olynomials the fo cus

of was on graph prop erties and Theorem is taken from The mo del

of b oundeddegree graphs was introduced in and Theorem combines

results from For surveys of the area the interested reader is referred

CHAPTER RELAXING THE REQUIREMENTS

Averagecase complexity

The theory of averagecase complexity was initiated by Levin who in partic

ular proved Theorem In light of the laconic nature of the original text

we refer the interested reader to a survey which provides a more detailed

exp osition of the denitions suggested by Levin as well as a discussion of the con

siderations underlying these suggestions This survey provides also a brief

account of further developments

As noted in x the current text uses a variant of the original denitions

In particular our denition of typicalcase feasibility diers from the original

denition of averagecase feasibility in totally discarding exceptional instances

and in even allowing the algorithm to fail on them and not merely run for an

excessive amount of time The alternative denition was suggested by several

researchers and app ears as a sp ecial case of the general treatment provided in

Section is based on Sp ecically Theorem or rather the

reduction of search to decision is due to and so is the introduction of the class

sampNP A version of Theorem was proven in and our pro of follows

their ideas which in turn are closely related to the ideas underlying the pro of of

Theorem proved in

Recall that we know of the existence of problems in distNP that are hard pro

vided sampNP contains hard problems However these problems refer to some

what generic decision problems such as S The presentation of distNP complete

problems that combine a more natural decision problem like SAT or Clique with

a simple probability ensemble is an op en problem

Exercises

Exercise general TSP For any function g prove that the following ap

proximation problem is NPHard Given a general TSP instance I represented

by a symmetric matrix of pairwise distances the task is nding a tour of length

that is at most a factor g I of the minimum Show that the result holds with

g I expp oly jI j and for instances in which all distances are p ositive

Guideline By reduction from Hamiltonian path Sp ecically reduce the instance G

n E to an nbyn distance matrix D d such that d expp oly n if

ij ij

ij n

fi j g E and d

Exercise TSP with triangle inequalities Provide a p olynomialtime

factor approximation for the sp ecial case of TSP in which the distances satisfy the

triangle inequality

Guideline First note that the length of any tour is lowerbounded by the weight of

a minimum spanning tree in the corresp onding weighted graph Next note that such a

tree yields a tour of length twice the weight of this tree that may visit some p oints

several times The triangle inequality guarantees that the tour do es not b ecome longer

by shortcuts that eliminate multiple visits at the same p oint

AVERAGE CASE COMPLEXITY

Exercise a weak version of Theorem Using Theorem prove

b a

that for some constants a b when setting LN N and sN N

it holds that gapClique is NPhard

Guideline Starting with Theorem apply the Expander Random Walk Generator

of Prop osition in order to derive a PCP system with logarithmic randomness and

query complexities that accepts noinstances of length n with probability at most n

The claim follows by applying the FGLSSreduction of Exercise while noting that

x is reduced to a graph of size p oly jxj such that the gap b etween yes and noinstances

is at least a factor of jxj

Exercise a weak version of Theorem Using Theorem prove

that for some constants s L the problem gapVC is NPhard

Guideline Note that combining Theorem and Exercise implies that for some

constants b it holds that gapClique is NPhard where LN b N and sN

b N The claim follows using the relations b etween cliques indep endent sets and

vertex covers

Exercise a weak version of Theorem Using Theorem prove

that for some constants s L the problem gapLin is NPhard

Guideline Recall that by Theorems and the gap problem gapSAT is NP

Hard Note that the result holds even if we restrict the instances to have exactly three

not necessarily dierent literals in each clause Applying the reduction of Exercise

note that for any assignment a clause that is satised by is mapp ed to seven equations

of which exactly three are violated by whereas a clause that is not satised by is

mapp ed to seven equations that are all violated by

Exercise natural inapproximability without the PCP Theorem In

contrast to the inapproximability results reviewed in x the NPcompleteness

of the following gap problem can b e established rather easily without referring

to the PCP Theorem The instances of this problem are systems of quadratic

equations over GF as in Exercise yesinstances are systems that have a

solution and noinstances are systems for which any assignment violates at least

one third of the equations

Guideline By Exercise when given such a quadratic system it is NPhard to

determine whether or not there exists an assignment that satises all the equations Using

an adequate smallbias generator cf Section present an amplifying reduction cf

Section of the foregoing problem to itself Sp ecically if the input system has m

equations then we use a generator that denes a sample space of p oly m many mbit

strings and consider the corresp onding linear combinations of the input equations Note

that it suces to b ound the bias of the generator by whereas using an biased

generator yields an analogous result with replaced by

Exercise enforcing multiway equalities via expanders The aim of

this exercise is presenting a ma jor technique of Papadimitriou and Yannakakis

CHAPTER RELAXING THE REQUIREMENTS

which is useful for designing reductions among approximation problems Recall

ing that gapSAT is NPhard our goal is proving NPhard of the following gap

Sp ecically the which is a sp ecial case of gapSAT problem denoted gapSAT

instances are restricted to CNF formulae with each variable app earing in at most c

clauses where c as is a xed constant Note that the standard reduction of SAT

to the corresp onding sp ecial case see pro of of Prop osition do es not preserve

an approximation gap The idea is enforcing equality of the values assigned to the

auxiliary variables ie the copies of each original variable by introducing equal

ity constraints only for pairs of variables that corresp ond to edges of an expander

graph see App endix E For example we enforce equality among the values of

m i j

z z by adding the clauses z z for every fi j g E where E is the

set of edges of am mvertex expander graph Prove that for some constants c and

the corresp onding mapping reduces gapSAT to gapSAT

Guideline Using dregular expanders we map CNF to instances in which each variable

app ears in at most d clauses Note that the number of added clauses is linearly related

to the number of original clauses Clearly if the original formula is satisable then so is

the reduced one On the other hand consider an arbitrary assignment to the reduced

formula ie the formula obtained by mapping For each original variable z if

assigns the same value to almost all copies of z then we consider the corresp onding

assignment in Otherwise by virtue of the added clauses do es not satisfy a constant

fraction of the clauses containing a copy of z

Exercise deciding ma jority requires linear time Prove that deciding

ma jority requires lineartime even in a direct access mo del and when using a ran

domized algorithm that may err with probability at most

Guideline Consider the problem of distinguishing X from Y where X resp Y is

n n n n

uniformly distributed over the set of nbit strings having exactly bn c resp bn c

ones For any xed set I n denote the pro jection of X resp Y on I by X resp

n n

Y Prove that the statistical dierence b etween X and Y is b ounded by O jI jn

n n n

Note that the argument needs to b e extended to the case that the examined lo cations are

selected adaptively

Exercise testing ma jority in p olylogarithmic time Show that test

ing ma jority with resp ect to can b e done in p olylogarithmic time by probing

the input at a constant number of randomly selected lo cations

Recall that in this reduction each o ccurrence of each Bo olean variable is replaced by a new

copy of this variable and clauses are added for enforcing the assignment of the same value to all

these copies Sp ecically the m o ccurrence of variable z are replaced by the variables z z

i i i i

while adding the clauses z z and z z for i m The problem is

that almost all clauses of the reduced formula may b e satised by an assignment in which half

of the copies of each variable are assigned one value and the rest are assigned an opp osite value

i i m

That is an assignment in which z z z z violates only one of the

auxiliary clauses introduced for enforcing equality among the copies of z Using an alternative

i j

reduction that adds the clauses z z for every i j m will not do either b ecause the

number of added clauses may b e quadratic in the number of original clauses

AVERAGE CASE COMPLEXITY

Exercise testing Eulerian graphs in the adjacency matrix representation

Show that in this mo del the set of Eulerian graphs can b e tested in p olylogarithmic

time

Guideline Focus on testing the set of graphs in which each vertex has an even degree

Note that in general the fact that the sets S and S are testable within some complexity

do es not imply the same for the set S S

Exercise an equivalent denition of tp c P Prove that S X tp c P

if and only if there exists a p olynomialtime algorithm A such that the probability

that AX errs in determining membership in S is a negligible function in n

Exercise tp cP versus P Part Prove that tp cP contains a problem

S X such that S is not even recursive Furthermore use X U

jxj

Guideline Let S f x x S g where S is an arbitrary nonrecursive set

Exercise tp cP versus P Part Prove that there exists a distribu

tional problem S X such that S P and yet there exists an algorithm solving

S correctly on all inputs in time that is typically p olynomial with resp ect to X

Furthermore use X U

Guideline For any timeconstructible function t N N that is sup erp olynomial and

jxj

subexp onential use S f x x S g for any S Dtimet n P

Exercise simple distributions and monotone sampling We say that

a probability ensemble X fX g is p olynomialtime sampleable via a monotone

mapping if there exists a p olynomial p and a p olynomialtime computable function

f such that the following two conditions hold

For every n the random variables f U and X are identically distributed

For every n and every r r f g it holds that f r f r where

the inequalities refers to the standard lexicographic order of strings

Prove that X is simple if and only if it is p olynomialtime sampleable via a mono

tone mapping

Guideline Supp ose that X is simple and let p b e a p olynomial b ounding the running

time of the algorithm that on input x outputs Pr X x Consider a mapping denoted

jxj

n n

of to f g such that r is mapp ed to x f g if and only if r Pr X

pn n

x Pr X x The desired function f f g f g can b e obtained from by

considering the binary representation of the numbers in and recalling that the binary

representation of Pr X x has length at most pjxj Note that f can b e computed by

jxj

binary search using the fact that X is simple Turning to the opp osite direction we note

pn n

that any eciently computable and monotone mapping f f g f g can b e

eciently inverted by a binary search Furthermore similar metho ds allow for eciently

determining the interval of pnbit long strings that are mapp ed to any given nbit long string

CHAPTER RELAXING THE REQUIREMENTS

Exercise reductions preserve typical p olynomialtime solveability

Prove that if the distributional problem S X is reducible to the distributional

problem S X and S X tp cP then S X is in tp cP

Guideline Let B denote the set of exceptional instances for the distributional problem

S X that is B is the set of instances on which the solver in the hypothesis either

errs or exceeds the typical runningtime Prove that Pr QX B is a negligible

function in n using b oth Pr y QX pjy j Pr X y and jxj p jy j for every

jy j

Pr y QX y Qx Sp ecically use the latter condition for inferring that

y B

Pr y QX which guarantees that a negligible function in equals

y fy B p jy jng

jy j for any y QX is negligible in n

Exercise reductions preserve errorless solveability In continuation

to Exercise prove that reductions preserve errorless solveability ie solve

ability by algorithms that never err and typically run in p olynomialtime

Exercise transitivity of reductions Prove that reductions among dis

tributional problems as in Denition are transitive

Guideline The p oint is establishing the domination prop erty of the comp osed reduction

The hypothesis that reductions do not make to o short queries is instrumental here

Exercise For any S NP present a simple probability ensemble X such

that the generic reduction used in the pro of of Theorem when applied to

S X violates the domination condition with resp ect to S U

Guideline Consider X fX g such that X is uniform over f x x

n n

f g g

Exercise variants of the Co ding Lemma Prove the following two vari

ants of the Co ding Lemma which is stated in the pro of of Theorem

A variant that refers to any eciently computable function f g

that is monotonically nondecreasing over f g ie x x for any

x x f g That is unlike in the pro of of Theorem here it

n n

holds that for every n

As in Part except that in this variant the function is strictly increasing

and the compression condition requires that jC xj log x rather

def

than jC xj minfjxj log xg where x x x

In b oth cases the pro of is less cumbersome than the one presented in the main

text

Exercise Prove that for any problem S X in distNP there exists a simple

probability ensemble Y such that the reduction used in the pro of of Theorem

suces for reducing S X to S Y

Guideline Consider Y fY g such that Y assigns to the instance hM x i a

n n

def

probability mass prop ortional to Pr X x Sp ecically for every hM x i it

jxj

AVERAGE CASE COMPLEXITY

def def

t jM j t

holds that Pr Y hM x i where n jhM x ij jM j jxj t

n x

Alternatively we may set Pr Y hM x i if M M and t p jxj and

n x S S

Pr Y hM x i otherwise where M and P are as in the pro of of Theorem

n S S

Exercise randomized reductions Following the outline in x

provide a denition of randomized reductions among distributional problems

In analogy to Exercise prove that randomized reductions preserve fea

sible solveability ie typical solveability in probabilistic p olynomialtime

That is if the distributional problem S X is randomly reducible to the

distributional problem S X and S X tp cBPP then S X is in

tp cBPP

In analogy to Exercise prove that randomized reductions preserve

solveability by probabilistic algorithms that err with probability at most

on each input and typically run in p olynomialtime

Prove that randomized reductions are transitive cf Exercise

Show that the error probability of randomized reductions can b e reduced

while preserving the domination condition

Extend the foregoing to reductions that involve distributional search problems

Exercise simple vs sampleable ensembles Part Prove that any

simple probability ensemble is p olynomialtime sampleable

Guideline See Exercise

Exercise simple vs sampleable ensembles Part Assuming that

P contains functions that are not computable in p olynomialtime prove that

there exists p olynomialtime sampleable ensembles that are not simple

Guideline Consider any R PC and supp ose that p is a p olynomial such that x y R

implies jy j pjxj Then consider the sampling algorithm A that on input uniformly

n pn

selects x y f g f g and outputs x if x y R and x otherwise

pjxj jxj

Note that R x Pr A x

Exercise distributional versions of NPC problems Part

Prove that for any NPcomplete problem S there exists a p olynomialtime sam

pleable ensemble X such that any problem in dist NP is reducible to S X We

actually assume that the manytoone reductions establishing the NPcompleteness

of S do not shrink the length of the input

Guideline Prove that the guaranteed reduction of S to S also reduces S U to

u u

S X for some sampleable probability ensemble X Consider rst the case that the

standard reduction of S to S is length preserving and prove that when applied to a

sampleable probability ensemble it induces a sampleable distribution on the instances

is sampleable by Exercise Next extend the treatment to of S Note that U

the general case where applying the standard reduction to U induces a distribution on

p olyn

m n

f g rather than a distribution on f g

CHAPTER RELAXING THE REQUIREMENTS

Exercise distributional versions of NPC problems Part

Prove Theorem ie for any NPcomplete problem S there exists a p olynomial

time sampleable ensemble X such that any problem in sampNP is reducible to

S X As in Exercise we actually assume that the manytoone reductions

establishing the NPcompleteness of S do not shrink the length of the input

Guideline We establish the claim for S and the general claim follows by using the

reduction of S to S as in Exercise Thus we fo cus on showing that for some

suitably chosen sampleable ensemble X any S X sampNP is reducible to S X

Lo osely sp eaking X will b e an adequate convex combination of all sampleable distribu

tions and thus X will not equal U or U Sp ecically X fX g is dened such

that X uniformly selects i n emulates the execution of the i algorithm in lexi

cographic order on input for n steps and outputs whatever the latter has output

or in case the said algorithm has not halted within n steps Prove that for any

S X sampNP such that X is sampleable in cubic time the standard reduction

of S to S reduces S X to S X as p er Denition ie in particular

u u

it satises the domination condition Finally using adequate padding reduce any

S X sampNP to some S X sampNP such that X is sampleable in cubic

time

Exercise search vs decision in the context of sampleable ensembles

Prove that every problem in sampNP is reducible to some problem in sampPC

and every problem in sampPC is randomly reducible to some problem in sampNP

Guideline See pro of of Theorem

Needless to say the choice to consider n algorithms in the denition of X is quite arbitrary

Any other unbounded function of n that is at most a p olynomial and is computable in p olynomial

time will do More generally we may select the i algorithm with p as long as p is a noticeable

i i

function of n Likewise the choice to emulate each algorithm for a cubic number of steps rather

some other xed p olynomial number of steps is quite arbitrary

Note that applying this reduction to X yields an ensembles that is also sampleable in cubic

time This claim uses the fact that the standard reduction runs in time that is less than cubic

and in fact almost linear in its output and the fact that the output is longer than the input

CHAPTER RELAXING THE REQUIREMENTS

App endix D

Probabilistic Preliminaries

and Advanced Topics in

Randomization

What is this Chicken Quesadilla and Seafo o d Salad

Fine but in the same plate This is disgusting

Johan Hastad at Grendels Cambridge

Summary This app endix lumps together some preliminaries regard

ing probability theory and some advanced topics related to the role

and use of randomness in computation Needless to say each of these

app ears in a separate section

The probabilistic preliminaries include our conventions regarding ran

dom variables which are used throughout the b o ok Also included are

overviews of three useful inequalities Markov Inequality Chebyshevs

Inequality and Cherno Bound

The advanced topics include hashing sampling and randomness ex

traction For hashing we describ e constructions of pairwise and twise

indep endent hashing functions and variants of the Leftover Hashing

Lemma which are used a few times in the main text We then review

the complexity of sampling that is the number of samples and the

randomness complexity involved in estimating the average value of an

arbitrary function dened over a huge domain Finally we provide an

overview on the question of extracting almost p erfect randomness from

sources of weak or defected randomness

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

D Probabilistic preliminaries

Probability plays a central role in complexity theory see for example Chapters

We assume that the reader is familiar with the basic notions of probability

theory In this section we merely present the probabilistic notations that are used

throughout the b o ok and three useful probabilistic inequalities

D Notational Conventions

Throughout the entire b o ok we will refer only to discrete probability distributions

Sp ecically the underlying probability space will consist of the set of all strings

of a certain length taken with uniform probability distribution That is the

sample space is the set of all bit long strings and each such string is assigned

probability measure Traditionally random variables are dened as functions

from the sample space to the reals Abusing the traditional terminology we use the

term random variable also when referring to functions mapping the sample space

into the set of binary strings We often do not sp ecify the probability space but

rather talk directly ab out random variables For example we may say that X is a

random variable assigned values in the set of all strings such that Pr X

and Pr X Such a random variable may b e dened over the sample

space f g so that X and X X X One

imp ortant case of a random variable is the output of a randomized pro cess eg a

probabilistic p olynomialtime algorithm as in Section

All our probabilistic statements refer to functions of random variables that

are dened b eforehand Typically we may write Pr f X where X is a random

variable dened b eforehand and f is a function An imp ortant convention is that

al l occurrences of the same symbol in a probabilistic statement refer to the same

unique random variable Hence if B is a Bo olean expression dep ending on

two variables and X is a random variable then Pr B X X denotes the probability

that B x x holds when x is chosen with probability Pr X x For example for

every random variable X we have Pr X X We stress that if we wish to

discuss the probability that B x y holds when x and y are chosen indep endently

with identical probability distribution then we will dene two indep endent random

variables each with the same probability distribution Hence if X and Y are

two indep endent random variables then Pr B X Y denotes the probability that

B x y holds when the pair x y is chosen with probability Pr X x Pr Y y

For example for every two indep endent random variables X and Y we have

Pr X Y only if b oth X and Y are trivial ie assign the entire probability

mass to a single string

Throughout the entire b o ok U denotes a random variable uniformly dis

tributed over the set of strings of length n Namely Pr U equals if

f g and equals otherwise We will often refer to the distribution of U

as the uniform distribution neglecting to qualify that it is uniform over f g In

addition we will o ccasionally use random variables arbitrarily distributed over

n n

f g or f g for some function N N Such random variables are typi

cally denoted by X Y Z etc We stress that in some cases X is distributed

n n n n

D PROBABILISTIC PRELIMINARIES

n n

over f g whereas in other cases it is distributed over f g for some func

tion which is typically a p olynomial We will often talk ab out probability

ensembles which are innite sequence of random variables fX g such that

each X ranges over strings of length b ounded by a p olynomial in n

Statistical dierence The statistical distance aka variation distance b etween

the random variables X and Y is dened as

jPr X v Pr Y v j maxfPr X S Pr Y S g D

We say that X is close resp far to Y if the statistical distance b etween them

is at most resp at least

D Three Inequalities

The following probabilistic inequalities are very useful These inequalities refer to

random variables that are assigned real values and provide upp erb ounds on the

probability that the random variable deviates from its exp ectation

Markov Inequality The most basic inequality is Markov Inequality that applies

to any random variable with b ounded maximum or minimum value For simplicity

it is stated for random variables that are lowerbounded by zero and reads as

follows Let X be a nonnegative random variable and v be a nonnegative real

number Then

Pr X v D

Equivalently Pr X r EX The pro of amounts to the following sequence

EX Pr X x x

X X

Pr X x Pr X x v

Pr X v v

Chebyshevs Inequality Using Markovs inequality one gets a p otentially

stronger b ound on the deviation of a random variable from its exp ectation This

b ound called Chebyshevs inequality is useful when having additional informa

tion concerning the random variable sp ecically a go o d upp er b ound on its vari

def

ance For a random variable X of nite exp ectation we denote by Var X

EX EX the variance of X and observe that Var X EX EX

Chebyshevs inequality then reads as follows Let X be a random variable and

Then

Var X

D Pr jX EX j

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

def

Pro of We dene a random variable Y X EX and apply Markov in

equality We get

Pr jX EX j Pr X EX

EX EX

and the claim follows

Corollary Pairwise Indep endent Sampling Chebyshevs inequality is particu

larly useful in the analysis of the error probability of approximation via rep eated

sampling It suces to assume that the samples are picked in a pairwise indep en

dent manner where X X X are pairwise indep endent if for every i j and

every it holds that Pr X X Pr X Pr X The corol

i j i j

lary reads as follows Let X X X be pairwise independent random variables

with identical expectation denoted and identical variance denoted Then

for every it holds that

D Pr

n n

def

Pro of Dene the random variables X EX Note that the X X s are

i i i i

pairwise indep endent and each has zero exp ectation Applying Chebyshevs in

and using the linearity of the exp ectation equality to the random variable

op erator we get

n n

Var

h i

E X

Now again using the linearity of exp ectation

n n

h i

X X X

E X X X X E E

i i j

i i

i j n

By the pairwise indep endence of the X s we get EX X EX EX and

i i j i j

X we get using E

n X E

The corollary follows

D PROBABILISTIC PRELIMINARIES

Cherno Bound When using pairwise indep endent sample p oints the error

probability in the approximation is decreasing linearly with the number of sample

p oints see Eq D When using totally indep endent sample p oints the error

probability in the approximation can b e shown to decrease exp onentially with

the number of sample p oints The random variables X X X are said to

X b e totally indep endent if for every sequence a a a it holds that Pr

i n

a Pr X a Probability b ounds supp orting the foregoing statement are

i i i

given next The rst b ound commonly referred to as Cherno Bound concerns

random variables ie random variables that are assigned as values either or

and asserts the following Let p and X X X be independent random

variables such that Pr X p for each i Then for every p p we

have

2p(1p)

e D Pr p e

P P

n n

X X pn n and Pr pn Pro of Sketch We upp erb ound Pr

i i

def

X EX we apply Markov Inequality X n is b ounded similarly Letting

i i i

i=1

to the random variable e where is determined to optimize the

expressions that we derive hint p p will do Thus Pr X

n is upp erb ounded by

i=1

n X

Ee e

where the equality is due to the indep endence of the random variables To simplify

the rest of the pro of we establish a suboptimal b ound as follows Using a Taylor

x x

expansion of e eg e x x for x and observing that E X we

E X which equals p p Thus Pr X pn n get Ee

n n

is upp erb ounded by e p p expn p pn which

is optimized at p p yielding exp n Needless to say this

metho d can b e applied in more general settings eg for X rather than

X f g

A more general b ound which refers to indep endent copies of a general b ounded

random variable is given next and is commonly referred to as Ho efding Inequality

Let X X X be n independent random variables with identical probability dis

tribution each ranging over the real interval a b and let denote the expected

value of each of these variables Then for every

(ba)

D e Pr

Ho efding Inequality is useful in estimating the average value of a function dened

over a large set of values esp ecially when the desired error probability needs to

A more general form requires the X s to b e indep endent but not necessarily identical and

def

EX See Ap dx A uses

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

b e negligible ie decrease faster than any p olynomial in the relevant parameter

Such an estimate can b e obtained provided that we can eciently sample the set

and have a b ound on the p ossible values of the function

Pairwise indep endent versus totally indep endent sampling Referring to

Eq D consider for simplicity the case that a b In this case

n independent samples give an approximation that deviates by from the exp ect

value ie with probability denoted that is exp onentially decreasing with

n Such an approximation is called an approximation and can b e achieved

using n O log sample p oints Thus the number of sample p oints

is p olynomially related to and logarithmically related to In contrast

by Eq D an approximation by n pairwise independent samples calls for

setting n O We stress that in both cases the number of samples is

polynomially related to the desired accuracy of the estimation ie The only

advantage of total ly independent samples over pairwise independent ones is in the

dependency of the number of samples on the error probability ie

D Hashing

Hashing is extensively used in complexity theory The typical application is map

ping arbitrary unstructured sets almost uniformly to a structured set of ad

equate size Sp ecically hashing is supp osed to map an arbitrary subset of

n m

f g to f g in an almost uniform manner

m m

For a xed set S of cardinality a mapping f S f g do es

exist but it is not necessarily an ecient one eg it may require knowing the

n m m

entire set S Clearly no single function f f g f g can map each

n m

subset of f g to f g in a manner or even approximately so However

n m m

a random function f f g f g has the prop erty that for every

n m

subset S f g with overwhelmingly high probability f maps S to f g

such that no p oint in the range has to o many f preimages in S The problem

is that a truly random function is unlikely to have a succinct representation let

alone an ecient evaluation algorithm We thus seek families of functions that

have a similar prop erty but do have a succinct representation as well as an ecient

evaluation algorithm

D Denitions

Motivated by the foregoing discussion we consider families of functions fH g

Such that the following prop erties hold

For every S f g with high probability a function h selected uniformly

m m

in H maps S to f g in an almost uniform manner For example we

may require that for any jS j and each p oint y with high probability

over the choice of h it holds that jfx S hx y gj p oly n

D HASHING

The functions in H have succinct representation For example we may

m nm

require that H f g for some p olynomial

The functions in H can b e eciently evaluated That is there exists a

p olynomialtime algorithm that on input a representation of a function h

m n

in H and a string x f g returns hx In some cases we make even

more stringent requirements regarding the algorithm eg that it runs in

linear space

Condition was left vague on purp ose At the very least we require that the

exp ected size of fx S hx y g equals jS j We shall see in Section D

that dierent interpretations of Condition are satised by dierent families of

hashing functions We fo cus on twise indep endent hashing functions dened next

of func Denition D twise indep endent hashing functions A family H

tions from nbit strings to mbit strings is called twise indep endent if for every t

n m

distinct domain elements x x f g and every y y f g it holds

t t

that

t tm

Pr hx y

hH i i

That is a uniformly chosen h H maps every t domain elements to the range in

a totally uniform manner Note that for t it follows that the probability that

a random h H maps two distinct domain elements to the same image equals

Such families of functions are called universal cf but we will fo cus

on the stronger condition of twise indep endence

D Constructions

The following constructions are merely a reinterpretation of the constructions

presented in x Alternatively one may view the constructions presented

in x as a reinterpretation of the following two constructions

Construction D twise indep endent hashing For t m n N such that m

n consider the fol lowing family of hashing functions mapping nbit strings to m

bit strings Each tsequence s s s s f g describes a function

n m

f g f g such that h x equals the mbit prex of the binary repre h

s s

j n

sentation of s x where the arithmetic is that of GF the nite eld of

elements

Prop osition implies that Construction D constitutes a family of twise inde

p endent hash functions Typically we will use either t or t n To make

the construction totally explicit we need an explicit representation of GF

see details following Prop osition An alternative construction for the case

of t may b e obtained analogously to the pairwise indep endent generator of

Prop osition Recall that a Toeplitz matrix is a matrix with all diagonals b eing

homogeneous that is T t is a Toeplitz matrix if t t for all i j

ij ij ij

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

Construction D Alternative pairwise indep endent hashing For m n con

sider the family of hashing functions in which each nbym Toeplitz matrix T and

n m

an mdimensional vector b describes a function h f g f g such that

T b

h x T x b

T b

Prop osition implies that Construction D constitutes a family of pairwise

indep endent hash functions Note that a nbym Toeplitz matrix can b e sp ecied

by n m bits yielding a description length of n m bits An alternative

construction analogous to Eq and requiring m n m bits of representation

uses arbitrary nbym matrices rather than Toeplitz matrices

D The Leftover Hash Lemma

We now turn to the almost uniform cover condition ie Condition mentioned

in Section D One concrete interpretation of this condition is given by the

following lemma and another is implied by it see Theorem D

Lemma D Let m n be integers H be a family of pairwise independent hash

n m

functions and S f g Then for every y f g and every for al l

but at most an it holds that fraction of h H

jS j

D jfx S hx y gj

By pairwise indep endence or rather even by wise indep endence the exp ected

size of fx S hx y g is jS j where the exp ectation is taken uniformly over

all h H The lemma upp er b ounds the fraction of hs that deviate from the

exp ected b ehavior ie for which jh y S j jS j Needless to say

the b ound is meaningful only in case jS j or alternatively for Setting

jS j and fo cusing on the case that jS j we infer that for al l but at

m m

most an fraction of h H it holds that jfx S hx y gj jS j

Thus each range element has approximately the right number of hpreimages in

the set S under almost all h H

n m

Pro of Fixing an arbitrary set S f g and an arbitrary y f g we

estimate the probability that a uniformly selected h H violates Eq D We

dene random variables over the aforementioned probability space such that

h equal if hx y and otherwise The exp ected value of

x x x

def

is jS j and we are interested in the probability that this sum

deviates from the exp ectation Applying Chebyshevs Inequality we get

jS j by the pairwise indep endence of the s and the b ecause Var

x x

fact that E The lemma follows x

D HASHING

A generalization called mixing The pro of of Lemma D can b e easily

extended to show that for every set T f g and every for al l but

at most an fraction of h H it holds that jfx S hx T gj

jT jjS j

jT j jS j Hint redene h if hx T and

x x

otherwise This assertion is meaningfull provided that jT j jS j and in

the case that m n it is called a mixing property

An extremely useful corollary The aforementioned generalization of Lemma D

asserts that most functions b ehave well with resp ect to any xed sets of preimages

n m

S f g and images T f g A seemingly stronger statement which is

nontrivially implied by Lemma D itself is that for all adequate sets S most

m m

map S to f g in an almost uniform manner This is a functions h H

consequence of the following theorem

m n

Theorem D aka Leftover Hash Lemma Let H and S f g be as in

jS j Consider random variable X and H that Lemma D and dene

are uniformly distributed on S and H respectively Then the statistical distance

between H H X and H U is at most

yields a strong extractor Using the terminology of Section D we say that H

with parameters to b e sp elled out there

def

Pro of Let V denote the set of pairs h y that violate Eq D and V

m m

V it holds that f g n V Then for every h y H

Pr H H X h y Pr H h Pr hX y

Pr H U h y

On the other hand by Lemma D which asserts Pr H y V for every

y f g and the setting of we have Pr H U V It follows that

V Pr H H X V Pr H H X

Pr H U V

Using all these upp erb ounds we upp erb ounded the statistical dierence b etween

H H X and H U denoted by separating the contribution of V and V

Sp ecically we have

jPr H H X h y Pr H U h y j

m m

hy H fg

jPr H H X h y Pr H U h y j

hy V

Pr H H X h y Pr H U h y

hy V

That is for X and as in Theorem D and any for all but at most an fraction of

the functions h H it holds that hX is close to U

m n

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

and the claim follows

An alternative pro of of Theorem D Dene the collision probability of a

random variable Z denote cpZ as the probability that two indep endent samples

def

Pr Z z Theorem D of Z yield the same result Alternatively cpZ

follows by combining the following two facts

A general fact If Z N and cpZ N then Z is close to the

uniform distribution on N

We prove the contrapositive Assuming that the statistical distance b etween

Z and the uniform distribution on N equals we show that cpZ

def

N This is done by dening L fz Pr Z z N g and lower

b ounding cpZ by using the fact that the collision probability is minimized

on uniform distributions Sp ecically considering the uniform distributions

on L and N n L resp ectively we have

Pr Z N n L Pr Z L

N jLj D cpZ jLj

jLj N jLj

Using Pr Z L where jLjN the rhs of Eq D equals

m m m

The collision probability of H H X is at most jS jjH j

Furthermore this holds even if H is only universal

The pro of is by a straightforward calculation Sp ecically note that cpH H X

m m

Pr H x jH j E cphX whereas E cphX jS j

hH hH

x x S

n n

1 2

H x The sum equals jS j jS j jS j and so cpH H X

m m

j jS j jH

Note that it follows that H H X is jS jclose to H U which is a

stronger b ound than the one provided in Theorem D

Stronger uniformity via higher indep endence Recall that Lemma D as

serts that for each p oint in the range of the hash function with high probability

over the choice of the hash function this xed point has approximately the exp ected

number of preimages in S A stronger condition asserts that with high probability

over the choice of the hash function every point in its range has approximately

the exp ected number of preimages in S Such a guarantee can b e obtained when

using nwise indep endent hashing functions

Lemma D Let m n be integers H be a family of nwise independent hash

functions and S f g Then for every for al l but at most an

m m n m

n jS j fraction of the functions h H Eq D holds for every

y f g

D SAMPLING

Indeed the lemma should b e used with jS jn In particular using m

log jS j log n guarantees that with high probability each range elements has

m m

jS j preimages in S Under this setting of parameters jS j n

which is p oly n whenever p olyn Needless to say this guarantee is stronger

than the conclusion of Theorem D

Pro of The pro of follows the fo otsteps of the pro of of Lemma D taking advan

tage of the fact that here the random variables ie the s are nwise indep en

dent For t n this allows using the socalled t moment analysis which

generalizes the second moment analysis of pairwise indep endent samplying pre

sented in Section D As in the pro of of Lemma D we x any S and y and

jS j dene h if and only if hx y Letting E

x x x

E we start with Markov inequality and

x x

t t

Q P

x x S i

1 2t

t m t

jS j

Using twise indep endence we note that only the terms in Eq D that do not

vanish are those in which each variable app ears with multiplicity This mean that

only terms having less than t distinct variables contribute to Eq D Now for

jS j

every j t we have less than t tj jS j terms with j distinct

m j

variables and each such term contributes less than to the sum Thus

Eq D is upp erb ounded by

m j m

t jS j tt t

m t m t

jS j j jS j jS j

where the rst inequality assumes jS j n since the claim hold vacuously other

wise This upp erb ounds the probability that a random h H violates Eq D

with reprect to a xed y Using a union b ound on all y f g the lemma fol

lows

D Sampling

In many settings rep eated sampling is used to estimate the average of a huge set of

values Namely given a value function f g R one wishes to approximate

def

x without having to insp ect the value of at each p oint of the

xfg

domain The obvious thing to do is sampling the domain at random and obtaining

an approximation to by taking the average of the values of on the sample p oints

It turns out that certain pseudorandom sequences of sample p oints may serve

almost as well as truly random sequences of sample p oints and thus the current

problem is indeed related to Section

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

D Formal Setting

It is essential to have the range of b e b ounded or else no reasonable approx

imation is p ossible For simplicity we adopt the convention of having b e

the range of and the problem for other predetermined ranges can b e treated

analogously Our notion of approximation dep ends on two parameters accuracy

denoted and error probability denoted We wish to have an algorithm that

with probability at least gets within of the correct value This leads to the

following denition

Denition D sampler A sampler is a randomized algorithm that on input

parameters n length accuracy and error and oracle access to any function

f g outputs with probability at least a value that is at most

def

away from x Namely

xfg

Pr jsampler n j

where the probability is taken over the internal coin tosses of the sampler also

called its random seed

A nonadaptive sampler is a sampler that consists of two deterministic algorithms

a sample generating algorithm G and a evaluation algorithm V On input n

and a random seed of adequate length algorithm G generates a sequence of queries

denoted s s f g Algorithm V is given the corresponding values ie

s s and outputs an estimate to

We are interested in the complexity of sampling quantied as a function of the

parameters n and Sp ecically we will consider three complexity measures

The sample complexity ie the number of oracle queries made by the sampler the

randomness complexity ie the length of the random seed used by the sampler

and the computational complexity ie the runningtime of the sampler We say

that a sampler is ecient if its runningtime is p olynomial in the total length of

its queries ie p olynomial in b oth its sample complexity and in n We will fo cus

on ecient samplers Furthermore we will fo cus on ecient samplers that have

optimal upto a constant factor sample complexity and will wish the randomness

complexity to b e as low as p ossible

D Known Results

We note that all the following p ositive results refer to nonadaptive samplers

whereas the lower b ound hold also for general samplers For more details on these

results see Sec and the references therein

The naive sampler The straightforward metho d or the naive sampler consists

of uniformly and independently selecting suciently many sample p oints queries

and outputting the average value of the function on these p oints Using Cherno

log

sample p oints suce As indicated next the naive Bound it follows that O

D SAMPLING

sampler is optimal upto a constant factor in its sample complexity but is quite

wasteful in randomness

log

It is known that samples are needed in any sampler and that that

samplers that make sn queries require randomness at least n log

log sn O These lower b ounds are tight as demonstrated by non

explicit and inecient samplers These facts guide our quest for improvements

which is aimed at nding more randomnessecient ways of eciently generating

sample sequences that can b e used in conjunction with an appropriate evaluation

algorithm V We stress that V need not necessarily take the average of the values

of the sampled p oints

The pairwiseindep endent sampler Using a pairwiseindep endence genera

tor cf x for generating sample p oints along with the natural evaluation

algorithm which outputs the average of the values of these p oints we can ob

tain a great saving in the randomness complexity In particular using a seed of

length n we can generate O pairwiseindep endent sample p oints which

by Eq D suce for getting accuracy with error Thus this Pairwise

Indep endent sampler uses n random bits rather than the log n

coins used by the naive sampler Furthermore for constant the Pairwise

Indep endent Sampler is optimal upto a constant factor in b oth its sample and

randomness complexities However for small ie o this sampler is

wasteful in sample complexity

The MedianofAverages sampler A new idea is required for going fur

ther and a relevant to ol random walks on expander graphs see Sections

and E is needed to o Sp ecically we combine the PairwiseIndependent Sam

pler with the Expander Random Walk Generator see Prop osition to obtain

a new sampler The new sampler uses a tlong random walk on an expander with

def

vertex set f g for generating a sequence of t O log related seeds for

t invocations of the PairwiseIndependent Sampler where each of these invoca

tions uses the corresp onding n bits to generate a sequence of O samples in

f g Furthermore each of these invocations returns a value that with prob

ability at least is close to Theorem see also Exercise is used

to show that with probability at least expt most of these t invo

cations return an close approximation Hence the median among these t values

is an approximation to the correct value The resulting sampler called the

log

and randomness MedianofAverages Sampler has sample complexity O

complexity n O log which is optimal upto a constant factor in b oth

complexities

Further improvements The randomness complexity of the MedianofAverages

Sampler can b e improved from n O log to n O log while main

log

This is done by replacing taining its optimal sample complexity of O

the Pairwise Indep endent Sampler by a sampler that picks a random vertex in a

suitable expander and samples all its neighbors

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

Averaging Samplers Averaging aka Oblivious samplers are nonadaptive

samplers in which the evaluation algorithm is the natural one that is it merely

outputs the average of the values of the sampled p oints Indeed the Pairwise

Indep endent Sampler is an averaging sampler whereas the MedianofAverages

Sampler is not Interestingly averaging samplers have applications for which ordi

nary nonadaptive samplers do not suce Averaging samplers are closely related

to randomness extractors dened and discussed in Section D

An o dd p ersp ective Recall that a nonadaptive sampler consists of a sample

generator G and an evaluator V such that for every f g it holds that

Pr jV s s j

s s GU

1 m

Thus we may view G as a pseudorandom generator that is sub jected to a distin

guishability test that is determined by a xed algorithm V and an arbitrary function

f g where we assume that Pr jV U U j

n n

What is a bit o dd here is that except for the case of averaging samplers the

distinguishability test contains a central comp onent ie the evaluator V that is

p otentially custommade to help the generator G pass the test

D Randomness Extractors

Extracting almostp erfect randomness from sources of weak ie defected ran

domness is crucial for the actual use of randomized algorithms pro cedures and

proto cols The latter are analyzed assuming that they are given access to a p erfect

random source while in reality one typically has access only to sources of weak

ie highly imp erfect randomness Randomness extractors are ecient pro ce

dures that p ossibly with the help of little extra randomness enhance the quality

of random sources converting any source of weak randomness to an almost p erfect

one In addition randomness extractors are related to several other fundamental

problems to b e further discussed later

One key parameter which was avoided in the foregoing discussion is the class of

weak random sources from which we need to extract almost p erfect randomness It

is preferable to make as little assumptions as p ossible regarding the weak random

source In other words we wish to consider a wide class of such sources and

require that the randomness extractor often referred to as the extractor works

well for any source in this class A general class of such sources is dened in

xD but rst we wish to mention that even for very restricted classes of sources

no deterministic extractor can work To overcome this imp ossibility result two

approaches are used

Another asp ect in which samplers dier from the various pseudorandom generators discussed

in Chapter is in the aim to minimize rather than maximize the number of blo cks denoted

here by m in the output sequence However also in case of samplers the aim is to maximize the

blo cklength denoted here by n

For example consider the class of sources that output nbit strings such that no string

o ccurs with probability greater than ie twice its probability weight under the uniform distribution

D RANDOMNESS EXTRACTORS

Seeded extractors The rst approach consists of considering randomized ex

tractors that use a relatively small amount of randomness in addition to

the weak random source That is these extractors obtain two inputs a

short truly random seed and a relatively long sequence generated by an arbi

trary source that b elongs to the sp ecied class of sources This suggestion is

motivated in two dierent ways

The application may actually have access to an almostp erfect random

source but bits from this source are much more exp ensive than bits

from the weak ie lowquality random source Thus it makes sense

to obtain few highquality bits from the almostp erfect source and use

them to purify the cheap bits obtained from the weak lowquality

source

In some applications eg when using randomized algorithms it may

b e p ossible to scan over all p ossible values of the seed and run the algo

rithm using the corresp onding extracted randomness That is we obtain

a sample r from the weak random source and invoke the algorithm on

extracts r for every p ossible seed s ruling by ma jority This al

ternative is typically not applicable to cryptographic andor distributed

settings

Few indep endent sources The second approach consists of considering deter

ministic extractors that obtain samples from a few say two independent

sources of weak randomness Such extractors are applicable in any setting

including in cryptography provided that the application has access to the

required number of indep endent weak random sources

In this section we fo cus on the rst type of extractors ie the seeded extractors

This choice is motivated b oth by the relatively more mature state of the research

in that direction and the closer connection b etween this direction and other topics

in complexity

D Denitions and various p ersp ectives

We rst present a denition that corresp onds to the foregoing motivational discus

sion and later discuss its relation to other topics in complexity

D The Main Denition

A very wide class of weak random sources corresp onds to sources for which no

sp ecic output is to o probable cf That is the class is parameterized by a

probability b ound and consists of all sources X such that for every x it holds

that Pr X x In such a case we say that X has minentropy at least

log Indeed we represent sources as random variables and assume that

Pr X x log Pr X x Recall that the entropy of a random variable X is dened as

Indeed the minentropy of X equals min flog Pr X xg and is always upp erb ounded by

its entropy

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

they are distributed over strings of a xed length denoted n An n k source is a

source that is distributed over f g and has minentropy at least k

An interesting sp ecial case of n k sources is that of sources that are uniform

over a subset of strings Such sources are called n k at A simple but useful

observation is that each n k source is a convex combination of n k at sources

Denition D extractor for n k sources

d n m

An algorithm Ext f g f g f g is called an extractor with error

for the class C if for every source X in C it holds that ExtU X is close

to U If C is the class of n k sources then Ext is called a k extractor

An algorithm Ext is called a strong extractor with error for C if for every

source X in C it holds that U ExtU X is close to U U A strong

d d d m

k extractor is dened analogously

Using the decomp osition of n k sources to n k at sources it follows that

Ext is a k extractor if and only if it is an extractor with error for the class

of n k at sources A similar claim holds for strong extractors Thus much of

the technical analysis is conducted with resp ect to the class of n k at sources

For example it is easy to see that for d log n O there exists a k

d n k

extractor Ext f g f g f g The pro of is by the Probabilistic

Metho d and uses a union b ound on the set of all n k at sources

We seek however explicit extractors that is extractors that are implementable

by p olynomialtime algorithms We note that the evaluation algorithm of any fam

ily of pairwise indep endent hash functions mapping nbit strings to mbit strings

k m

constitutes a strong k extractor for see the alternative pro of of

Theorem D However these extractors necessarily use a long seed ie d m

must hold and in fact d n m holds in Construction D In Section D

we survey constructions of ecient k extractors that obtain logarithmic seed

length ie d O log n But b efore doing so we provide a few alternative

p ersp ectives on extractors

An imp ortant note on logarithmic seed length The case of logarithmic

seed length is of particular imp ortance for a variety of reasons Firstly when

emulating a randomized algorithm using a defected random source as in Item of

the motivational discussion of seeded extractors the overhead is exp onential in the

length of the seed Thus the emulation of a generic probabilistic p olynomialtime

algorithm can b e done in p olynomial time only if the seed length is logarithmic

Similarly the applications discussed in xD and xD are feasible only if

the seed length is logarithmic Lastly we note that logarithmic seed length is an

absolute lowerbound for k extractors whenever n k k and m and

d n k

The probability that a random function Ext f g f g f g is not an extractor

exp which is with error for a xed n k at source is upp erb ounded by

smaller than

D RANDOMNESS EXTRACTORS

D Extractors as averaging samplers

There is a close relationship b etween extractors and averaging samplers which

are mentioned towards the end of Section D We rst show that any averaging

n m t

sampler gives rise to an extractor Let G f g f g b e the sample gen

erating algorithm of an averaging sampler having accuracy and error probability

That is G uses n bits of randomness and generates t sample p oints in f g

such that for every f f g with probability at least the average of

def

f where f Ef U Dene the f values of these p oints is in the interval

n m th

Ext t f g f g such that Exti r is the i sample generated by

Gr We shall prove that Ext is a k extractor for k n log

Supp ose towards the contradiction that there exists a n k at source X such

that for some S f g it is the case that Pr ExtU X S Pr U S

d m

where d log t and t f g Dene

n m

B fx f g Pr ExtU x S jS j g

k n

Then jB j Dening f z if z S and f z otherwise we

def

f Ef U jS j But for every r B the f average of the sample have

Gr is greater than f in contradiction to the hypothesis that the sampler has

error probability with resp ect to accuracy

We now turn to show that extractors give rise to averaging samplers Let Ext

d n m

f g f g f g b e a k extractor Consider the sample generation

n m

algorithm G f g f g dene by Gr Exts r d We prove

sfg

that it corresp onds to an averaging sampler with accuracy and error probability

Supp ose towards the contradiction that there exists a function f f g

n k n

such that for strings r f g the average f value of the

def

f Ef U by more than Supp ose without loss sample Gr deviates from

of generality that for at least half of these r s the average is greater than f

and let B denote the set of these r s Then for X that is uniformly distributed on

B and is thus a n k source we have

Ef ExtU X Ef U

d m

which using jf z j for every z contradicts the hypothesis that ExtU X is

close to U

D Extractors as randomnessecient errorreductions

As may b e clear from the foregoing discussion extractors yield randomnessecient

metho ds for errorreduction Indeed errorreduction is a special case of the sam

pling problem obtained by considering Bo olean functions Sp ecically for a two

jxj

sided error decision pro cedure A consider the function f f g f g

such that f r if Ax r and f r otherwise Assuming that

x x

the probability that A is correct is at least say error reduc

tion amounts to providing a sampler with accuracy and any desired error prob

ability for the Bo olean function f In particular any k extractor x

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

d n jxj

Ext f g f g f g with k n log will do provided

d d

is feasible eg p oly jxj where represents the randomness com

plexity of the original algorithm A The question of interest here is how do es n

which represents the randomness complexity of the corresp onding sampler grow

as a function of jxj and

n jxj

Errorreduction using the extractor Ext p oly jxj f g f g

error probability randomness complexity

original algorithm jxj

resulting algorithm may dep end on jxj n function of jxj and

Jumping ahead see Part of Theorem D we note that for every one

p olyjxj

can obtain n O jxj log for any Note that for

O jxj

this b ound on the randomnesscomplexity of errorreduction is b etter

than the b ound of n jxj O log that is provided for the reduction of

onesided error by the Expander Random Walk Generator of Section alb eit

the number of samples here is larger ie p oly jxj rather than O log

Mentioning the reduction of onesided error probability brings us to a cor

resp onding relaxation of the notion of an extractor which is called a disp erser

Lo osely sp eaking a k disp erser is only required to hit with p ositive probabil

ity any set of density greater than in its image rather than pro duce a distribution

that is close to uniform

d n m

Denition D disp ersers An algorithm Dsp f g f g f g is

called a k disp erser if for every n k source X the support of Dsp U X covers

m m

at least points Alternatively for every set S f g of size greater

than it holds that Pr DspU X S

Disp ersers can b e used for the reduction of onesided error analogously to the

use of extractors for the reduction of twosided error Sp ecically regarding the

aforementioned function f and assuming that either Pr f U or

x x

jxj

d n jxj

f U we may use any k disp erser Dsp f g f g f g

jxj

in attempt to nd a p oint z such that f z Indeed if Pr f U

x x

jxj

d k

then jfz s f g f Dsps z gj and thus the onesided error can

b e reduced from to while using n random bits

D Other p ersp ectives

Extractors and disp ersers have an app ealing interpretation in terms of bipartite

d n

graphs Starting with disp ersers we view a disp erser Dsp f g f g

m n m

f g as a bipartite graph G f g f g E such that E fx Dsps x

n d k

x f g s f g g This graph has the prop erty that any subset of ver

tices on the left ie in f g has a neighborho o d that contains at least a

fraction of the vertices of the right which is remarkable in the typical case where

d is small eg d O log n and n k m whereas m k or at least

m k Furthermore if Dsp is eciently computable then this bipartite graph

D RANDOMNESS EXTRACTORS

is strongly constructible in the sense that given a vertex on the left one can e

d n m

ciently nd all its neighbors An extractor Ext f g f g f g yields

an analogous graph with a even stronger prop erty the neighborho o d multiset of

any subset of vertices on the left covers the vertices on the right in an almost

uniform manner

An o dd p ersp ective In addition to viewing extractors as averaging samplers

which in turn may b e viewed within the scop e of the pseudorandomness paradigm

we mention here an even more o dd p ersp ective Sp ecically randomness extractors

may b e viewed as randomized by the seed algorithms designed on purp ose such

that to b e fo oled by any weak random source but not by an even worse source

d n m

Consider a k extractor Ext f g f g f g for say

m k log n and d O log n and a p otential test T parameterized

by a set S f g such that Pr T x Pr ExtU x S ie on

S d

n d

input x f g the test uniformly selects s f g and outputs if and

only if Exts x S Then for every n k source X the test T do es not

distinguish X from U ie Pr T X Pr T U b ecause ExtU X

n S S n d

is close to ExtU U since each is close to U On the other hand for

d n m

every n k d at source Y there exists a set S such that T distinguish

Y from U with gap eg for S that equals the supp ort of ExtU Y it

n d

holds that Pr T Y and Pr T U jS j

S S n

Furthermore this class of tests detects as defected with probability any source

that has entropy b elow k d Thus this weird class of tests views each n k

source as pseudorandom while detecting sources of lower entropy eg entropy

lower than k d as nonpseudorandom Indeed this p ersp ective stretches the

pseudorandomness paradigm quite far

D Constructions

Recall that we seek explicit constructions of extractors that is functions Ext

d n m

f g f g f g that can b e computed in p olynomialtime The ques

tion of course is of parameters that is having k extractors with m as large

as p ossible and d as small as p ossible We rst note that typically m k

d log O and d log n k O must hold regard

less of explicitness The aforementioned b ounds are in fact tight that is there

exists nonexplicit k extractors with m k d log O and

d log n k O The obvious goal is to meet these b ounds via explicit

constructions

For any such source Y the distribution Z Ext U Y has entropy at most k m

and thus is far from U and far from Ext U U The lowerbound on the statistical

m n

distance of Z to U can b e proven by the contrapositive if Z is close to U then its entropy

m m

is at least m eg by using Fanos inequality see Thm

That is for and m d

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

D Some known results

Despite tremendous progress on this problem and o ccasional claims regarding

optimal explicit constructions the ultimate goal was not reached yet However

we are pretty close In particular we have the following

Theorem D explicit constructions of extractors Explicit k extractors of

d n m

the form Ext f g f g f g exist in the fol lowing cases

For any constants with d O log n and m k

For any constants with d log n and m k p olylog n

For any expk log k with d O log n and m k log k

Part is due to and the other two parts are due to where these

works build on previous ones which are not cited here We note that for sake

of simplicity we did not quote the b est p ossible b ounds Furthermore we did not

mention additional incomparable results which are relevant for dierent ranges of

parameters In general it seems that the last word has not b een said yet indeed

the current results are close to optimal but this cannot b e said ab out the way that

they are achieved In view of the foregoing we refrain from trying to provide an

overview of the pro of of Theorem D and review instead a conceptual insight

that op ened the do or to much of the recent developments in the area

D The pseudorandomness connection

We conclude this section with an overview of a fruitful connection b etween extrac

tors and certain pseudorandom generators The connection discovered by Tre

visan is surprising in the sense that it go es in a nonstandard direction it

transforms certain pseudorandom generators into extractors As argued throughout

this b o ok most conspicuously at the end of Section computational ob jects

are typically more complex than the corresp onding information theoretical ob jects

Thus if pseudorandom generators and extractors are at all related which was not

susp ected b efore then this relation should not b e exp ected to help in the con

struction of extractors which seem an information theoretic ob ject Nevertheless

the discovery of this relation did yield a breakthrough in the study of extractors

But b efore describing the connection let us wonder for a moment Just lo oking

at the syntax we note that pseudorandom generators have a single input ie the

seed while extractors have two inputs ie the nbit long source and the dbit

long seed But taking a second lo ok at the NisanWigderson Generator ie the

combination of Construction with an amplication of worstcase to average

case hardness we note that this construction can b e viewed as taking two inputs

a dbit long seed and a hard predicate on d bit long strings where d d

We note that once the connection b ecame b etter understo o d inuence started going in the

right direction from extractors to pseudorandom generators

Indeed to t the current context we have mo died some notations In Construction the

length of the seed is denoted by k and the length of the input for the predicate is denoted by m

D RANDOMNESS EXTRACTORS

Now an app ealing idea is to use the nbit long source as a truthtable description

of a worsecase hard predicate which indeed means setting n The key

observation is that even if the source is only weakly random we expect it to represent

a predicate that is hard on the worstcase

Recall that the aforementioned construction is supp osed to yield a pseudoran

dom generator whenever it starts with a hard predicate In the current context

where there are no computational restrictions pseudorandomness is supp osed to

hold against any computationally unbounded distinguisher and thus here pseudo

randomness means b eing statistically close to the uniform distribution on strings

of the adequate length denoted Intuitively this makes sense only if the ob

served sequence is shorter that the amount of randomness in the source and seed

which is indeed the case ie k d where k denotes the minentropy of the

source Hence there is hop e to obtain a go o d extractor this way

To turn the hop e into a reality we need a pro of which is sketched next Lo ok

ing again at the NisanWigderson Generator we note that the pro of of indistin

guishability of this generator provides a blackbox pro cedure for computing the un

derlying predicate when given oracle access to any p otential distinguisher Sp ecif

ically in the pro ofs of Theorems and which holds for any

this blackbox pro cedure was implemented by a relatively smal l circuit which de

p ends on the underlying predicate Hence this pro cedure contains relatively little

information regarding the underlying predicate on top of the observed bit long

output of the extractorgenerator Sp ecically for some xed p olynomial p the

amount of information enco ded in the pro cedure and thus available to it is upp er

def

b ound by b p while the pro cedure is supp ose to compute the underlying

predicate correctly on each input That is this amount of information is supp osed

to fully determine the underlying predicate which in turn is identical to the nbit

long source Thus if the source has minentropy exceeding b then it cannot b e

fully determine using only b bits of information It follows that the foregoing con

struction constitutes a b O extractor outputting b bits where

the constant is the one used in the pro of of Theorem and the argument

holds provided that b n Note that this extractor uses a seed of length

d O d O log n The argument can b e extended to obtain k p oly k

extractors that output k bits using a seed of length d O log n provided that

k n

We note that the foregoing description has only referred to two abstract prop

erties of the NisanWigderson Generator the fact that this generator uses

any worstcase hard predicate as a blackbox and the fact that its analysis

uses any distinguisher as a blackbox In particular we viewed the amplication

of worstcase hardness to inapproximability p erformed in Theorem as part

of the construction of the pseudorandom generator An alternative presentation

which is more selfcontained replaces the amplication step of Theorem by a

direct argument in the current information theoretic context and plugs the result

ing predicate directly into Construction The advantages of this alternative

include using a simpler amplication since amplication is simpler in the informa

d d

Recalling that n the restriction implies n

APPENDIX D PROBABILISTIC PRELIMINARIES AND ADVANCED TOPICS IN RANDOMIZATION

tion theoretic setting than in the computational setting and deriving transparent

construction and analysis which mirror Construction and Theorem re

sp ectively

The alternative presentation The foregoing analysis transforms a generic dis

tinguisher into a pro cedure that computes the underlying predicate correctly on

each input which fully determines this predicate Hence an upp erb ound on the

information available to this pro cedure yields an upp erb ound on the number of

p ossible outcomes of the source that are bad for the extractor In the alternative

presentation we transforms a generic distinguisher into a pro cedure that approx

imates the underlying predicate that is the pro cedure yields a function that is

relatively close to the underlying predicate If the p otential underlying predicates

are far apart then this directly yields the desired b ound on the number of bad

outcomes that corresp ond such predicates Thus the idea is to enco de the nbit

long source by an error correcting co de of length n p oly n and relative dis

tance n and use the resulting co deword as a truthtable of a predicate

for Construction Such co des coupled with ecient enco ding algorithms

do exist see Section E and the b enet in using them is that each n bit long

string determined by the information available to the aforementioned approxima

tion pro cedure may b e nclose to at most O n co dewords which

corresp ond to p otential predicates That is the resulting extractor converts the

n d

nbit input x into a codeword x f g viewed as a predicate over f g

where d log n and evaluates this predicate at the projections of the dbit

long seed where these projections are determined by the corresponding set system

ie the long sequence of d subsets of d The analysis mirrors the pro of of

Theorem and yields a b ound of O n on the number of bad outcomes

for the source where O upp erb ounds the information available to the approx

imation pro cedure and O n upp erb ounds the number of sourceoutcomes that

when enco ded are each nclose to the approximation pro cedure

D Recommended reading

The interested reader is referred to a survey of Shaltiel This survey con

tains a comprehensive introduction to the area including an overview of the ideas

that underly the various constructions In particular the survey describ es the ap

proaches used b efore the discovery of the pseudorandomness connection the con

nection itself and the constructions that arise from it and the third generation

of constructions that followed

Bibliography

S Aaronson Complexity Zo o A continueously up dated website at

Zoo httpqwikicaltecheduwikiComplexity

LM Adleman and M Huang Primality Testing and Abelian Varieties Over

Finite Fields SpringerVerlag Lecture Notes in Computer Science Vol

Preliminary version in th STOC

M Agrawal N Kayal and N Saxena PRIMES is in P Annals of Mathe

matics Vol pages

M Ajtai J Komlos E Szemeredi Deterministic Simulation in LogSpace

In th ACM Symposium on the Theory of Computing pages

R Aleliunas RM Karp RJ Lipton L Lovasz and C Racko Random

walks universal traversal sequences and the complexity of maze problems In

th IEEE Symposium on Foundations of Computer Science pages

N Alon L Babai and A Itai A fast and Simple Randomized Algorithm

for the Maximal Indep endent Set Problem J of Algorithms Vol pages

N Alon and R Boppana The monotone circuit complexity of Bo olean func

tions Combinatorica Vol pages

N Alon E Fischer I Newman and A Shapira A Combinatorial Charac

terization of the Testable Graph Prop erties Its All Ab out Regularity In

th ACM Symposium on the Theory of Computing to app ear

N Alon O Goldreich J Hastad R Peralta Simple Constructions of Almost

k wise Indep endent Random Variables Journal of Random structures and

Algorithms Vol No pages

N Alon and JH Sp encer The Probabilistic Method John Wiley Sons

Inc

R Armoni On the derandomization of spaceb ounded computations In

the pro ceedings of Random SpringerVerlag Lecture Notes in Computer

Science Vol pages

BIBLIOGRAPHY

S Arora Approximation schemes for NPhard geometric optimization prob

lems A survey Math Programming Vol pages July

S Arora C Lund R Motwani M Sudan and M Szegedy Pro of Verication

and Intractability of Approximation Problems Journal of the ACM Vol

pages Preliminary version in rd FOCS

S Arora and S Safra Probabilistic Checkable Pro ofs A New Characteriza

tion of NP Journal of the ACM Vol pages Preliminary

version in rd FOCS

H Attiya and J Welch Distributed Computing Fundamentals Simulations

and Advanced Topics McGrawHill

L Babai Trading Group Theory for Randomness In th ACM Symposium

on the Theory of Computing pages

L Babai L Fortnow and C Lund NonDeterministic Exp onential Time has

TwoProver Interactive Proto cols Computational Complexity Vol No

pages Preliminary version in st FOCS

L Babai L Fortnow L Levin and M Szegedy Checking Computations in

Polylogarithmic Time In rd ACM Symposium on the Theory of Computing

pages

L Babai L Fortnow N Nisan and A Wigderson BPP has Sub exp onen

tial Time Simulations unless EXPTIME has Publishable Pro ofs Complexity

Theory Vol pages

L Babai and S Moran ArthurMerlin Games A Randomized Pro of System

and a Hierarchy of Complexity Classes Journal of Computer and System

Science Vol pp

E Bach and J Shallit Algorithmic Number Theory Volume I Ecient

Algorithms MIT Press

B Barak NonBlackBox Techniques in Crypptography PhD Thesis Weiz

mann Institute of Science

W Baur and V Strassen The Complexity of Partial Derivatives Theor

Comput Sci pages

P Beame and T Pitassi Prop ositional Pro of Complexity Past Present and

Future In Bul letin of the European Association for Theoretical Computer

Science Vol June pp

A Beimel Y Ishai E Kushilevitz and JF Raymond Breaking the

O n barrier for informationtheoretic private information retrieval

In rd IEEE Symposium on Foundations of Computer Science pages

BIBLIOGRAPHY

M Bellare O Goldreich and E Petrank Uniform Generation of NP

witnesses using an NPoracle Information and Computation Vol pages

M Bellare O Goldreich and M Sudan Free Bits PCPs and Non

Approximability Towards Tight Results SIAM Journal on Computing

Vol No pages Extended abstract in th FOCS

S BenDavid B Chor O Goldreich and M Luby On the Theory of Average

Case Complexity Journal of Computer and System Science Vol pages

A BenDor and S Halevi In nd Israel Symp on Theory of Computing and

Systems IEEE Computer So ciety Press pages

M BenOr O Goldreich S Goldwasser J Hastad J Kilian S Micali

and P Rogaway Everything Provable is Probable in ZeroKnowledge In

Crypto SpringerVerlag Lecture Notes in Computer Science Vol

pages

M BenOr S Goldwasser J Kilian and A Wigderson MultiProver Inter

active Pro ofs How to Remove Intractability In th ACM Symposium on

the Theory of Computing pages

M BenOr S Goldwasser and A Wigderson Completeness Theorems for

NonCryptographic FaultTolerant Distributed Computation In th ACM

Symposium on the Theory of Computing pages

E BenSasson O Goldreich P Harsha M Sudan and S Vadhan Robust

PCPs of proximity Shorter PCPs and Applications to Co ding In th ACM

Symposium on the Theory of Computing pages Full version in

ECCC TR

E BenSasson and M Sudan Simple PCPs with Polylog Rate and Query

Complexity ECCC TR

L Berman and J Hartmanis On isomorphisms and density of NP and other

complete sets SIAM Journal on Computing Vol pages

Extended abstract in th STOC

M Blum A MachineIndependent Theory of the Complexity of Recursive

Functions Journal of the ACM Vol pages

M Blum and S Micali How to Generate Cryptographically Strong Sequences

of PseudoRandom Bits SIAM Journal on Computing Vol pages

Preliminary version in rd FOCS

M Blum M Luby and R Rubinfeld SelfTestingCorrecting with Appli

cations to Numerical Problems Journal of Computer and System Science

Vol No pages

BIBLIOGRAPHY

A Bogdanov K Obata and L Trevisan A lower b ound for testing

colorability in b oundeddegree graphs In rd IEEE Symposium on Foun

dations of Computer Science pages

A Bogdanov and L Trevisan On worstcase to averagecase reductions for

NP problems In Proc th IEEE Symposium on Foundations of Computer

Science pages

A Bogdanov and L Trevisan Averagecase complexity a survey In prepa

ration

R Boppana J Hastad and S Zachos Do es CoNP Have Short Interactive

Pro ofs Information Processing Letters May pages

R Boppana and M Sipser The complexity of nite functions In Handbook

of Theoretical Computer Science Volume A Algorithms and Complexity

J van Leeuwen editor MIT PressElsevier pages

A Boro din Computational Complexity and the Existence of Complexity

Gaps Journal of the ACM Vol pages

A Boro din On Relating Time and Space to Size and Depth SIAM Journal

on Computing Vol pages

G Brassard D Chaum and C Crepeau Minimum Disclosure Pro ofs of

Knowledge Journal of Computer and System Science Vol No pages

Preliminary version by Brassard and Crepeau in th FOCS

L Carter and M Wegman Universal Hash Functions Journal of Computer

and System Science Vol pages

GJ Chaitin On the Length of Programs for Computing Finite Binary Se

quences Journal of the ACM Vol pages

AK Chandra DC Kozen and LJ Sto ckmeyer Alternation Journal of the

ACM Vol pages

D Chaum C Crepeau and I Damgard Multiparty unconditionally Secure

Proto cols In th ACM Symposium on the Theory of Computing pages

B Chor and O Goldreich On the Power of TwoPoint Based Sampling

Jour of Complexity Vol pages Preliminary version dates

B Chor and O Goldreich Unbiased Bits from Sources of Weak Randomness

and Probabilistic Communication Complexity SIAM Journal on Computing

Vol No pages

BIBLIOGRAPHY

A Church An Unsolvable Problem of Elementary Number Theory Amer

J of Math Vol pages

A Cobham The Intristic Computational Diculty of Functions In Proc

Iternational Congress for Logic Methodology and Philosophy of Science

pages

SA Co ok The Complexity of Theorem Proving Pro cedures In rd ACM

Symposium on the Theory of Computing pages

SA Co ok A overview of Computational Complexity Turing Award Lecture

CACM Vol pages

SA Co ok A Taxonomy of Problems with Fast Parallel Algorithms Infor

mation and Control Vol pages

SA Co ok and RA Reckhow Stephen A Co ok Rob ert A Reckhow The

Relative Eciency of Prop ositional Pro of Systems J of Symbolic Logic

Vol pages

D Copp ersmith and S Winograd Matrix multiplication via arithmetic pro

gressions Journal of Symbolic Computation pages

TM Cover and GA Thomas Elements of Information Theory John Wiley

Sons Inc NewYork

P Crescenzi and V Kann A comp endium of NP Optimization problems

Available at httpwwwnadakthseviggowwwcompendium

W Die and ME Hellman New Directions in Cryptography IEEE Trans

on Info Theory IT Nov pages

I Dinur The PCP Theorem by Gap Amplication ECCC TR

I Dinur and O Reingold Assignmenttesters Towards a combinatorial pro of

of the PCPTheorem In th IEEE Symposium on Foundations of Computer

Science pages

I Dinur and S Safra The imp ortance of b eing biased In th ACM Sym

posium on the Theory of Computing pages

J Edmonds Paths Trees and Flowers Canad J Math Vol pages

S Even Graph Algorithms Computer Science Press

S Even AL Selman and Y Yacobi The Complexity of Promise Problems

with Applications to PublicKey Cryptography Information and Control

Vol pages

BIBLIOGRAPHY

U Feige S Goldwasser L Lovasz and S Safra On the Complexity of

Approximating the Maximum Size of a Clique Unpublished manuscript

U Feige S Goldwasser L Lovasz S Safra and M Szegedy Approximating

Clique is almost NPcomplete Journal of the ACM Vol pages

Preliminary version in nd FOCS

U Feige D Lapidot and A Shamir Multiple NonInteractive Zero

Knowledge Pro ofs Under General Assumptions SIAM Journal on Com

puting Vol pages

U Feige and A Shamir Witness Indistinguishability and Witness Hiding

Proto cols In nd ACM Symposium on the Theory of Computing pages

E Fischer The art of uninformed decisions A primer to prop erty test

ing Bul letin of the European Association for Theoretical Computer Science

Vol pages

GD Forney Concatenated Codes MIT Press Cambridge MA

L Fortnow R Lipton D van Melkebeek and A Viglas Timespace lower

b ounds for satisability Journal of the ACM Vol pages

November

L Fortnow J Romp el and M Sipser On the p ower of multiprover interac

tive proto cols In rd IEEE Symp on Structure in Complexity Theory pages

See errata in th IEEE Symp on Structure in Complexity

Theory pages

S Fortune A Note on Sparse Complete Sets SIAM Journal on Computing

Vol pages

M F urer O Goldreich Y Mansour M Sipser and S Zachos On Complete

ness and Soundness in Interactive Pro of Systems Advances in Computing

Research a research annual Vol Randomness and Computation S Mi

cali ed pages

ML Furst JB Saxe and M Sipser Parity Circuits and the Polynomial

Time Hierarchy Mathematical Systems Theory Vol pages

Preliminary version in nd FOCS

O Gab er and Z Galil Explicit Constructions of Linear Size Sup erconcen

trators Journal of Computer and System Science Vol pages

MR Garey and DS Johnson Computers and Intractability A Guide to the

Theory of NPCompleteness WH Freeman and Company New York

BIBLIOGRAPHY

D Gillman A cherno b ound for random walks on expander graphs In

th IEEE Symposium on Foundations of Computer Science pages

O Goldreich Foundation of Cryptography Class Notes Computer Science

Dept Technion Israel Spring Sup erseded by

O Goldreich A Note on Computational Indistinguishability Information

Processing Letters Vol pages May

O Goldreich Notes on Levins Theory of AverageCase Complexity ECCC

TR Dec

O Goldreich Modern Cryptography Probabilistic Proofs and Pseudorandom

ness Algorithms and Combinatorics series Vol Springer

O Goldreich Foundation of Cryptography Basic Tools Cambridge Univer

sity Press

O Goldreich Foundation of Cryptography Basic Applications Cambridge

University Press

O Goldreich Short Lo cally Testable Co des and Pro ofs Survey ECCC

O Goldreich On Promise Problems a survey in memory of Shimon Even

ECCC TR

O Goldreich S Goldwasser and S Micali How to Construct Random

Functions Journal of the ACM Vol No pages

O Goldreich S Goldwasser and A Nussb oim On the Implementation of

Huge Random Ob jects In th IEEE Symposium on Foundations of Com

puter Science pages

O Goldreich S Goldwasser and D Ron Prop erty testing and its connection

to learning and approximation Journal of the ACM pages July

O Goldreich and H Krawczyk On the Comp osition of ZeroKnowledge

Pro of Systems SIAM Journal on Computing Vol No February

pages Preliminary version in th ICALP

O Goldreich and LA Levin Hardcore Predicates for any OneWay Func

tion In st ACM Symposium on the Theory of Computing pages

O Goldreich S Micali and A Wigderson Pro ofs that Yield Nothing but

their Validity or All Languages in NP Have ZeroKnowledge Pro of Systems

Journal of the ACM Vol No pages Preliminary version

in th FOCS

BIBLIOGRAPHY

O Goldreich S Micali and A Wigderson How to Play any Mental Game

A Completeness Theorem for Proto cols with Honest Ma jority In th ACM

Symposium on the Theory of Computing pages

O Goldreich N Nisan and A Wigderson On Yaos XORLemma ECCC

O Goldreich and D Ron Prop erty testing in b ounded degree graphs Algo

rithmica pages

O Goldreich and D Ron A sublinear bipartite tester for b ounded degree

graphs Combinatorica Vol pages

O Goldreich R Rubinfeld and M Sudan Learning p olynomials with queries

the highly noisy case SIAM J Discrete Math Vol pages

O Goldreich S Vadhan and A Wigderson On interactive pro ofs with a

laconic provers Computational Complexity Vol pages

O Goldreich and A Wigderson Computational Complexity In The Prince

ton Companion to Mathematics to app ear

S Goldwasser and S Micali Probabilistic Encryption Journal of Computer

and System Science Vol No pages Preliminary version

in th STOC

S Goldwasser S Micali and C Racko The Knowledge Complexity of

Interactive Pro of Systems SIAM Journal on Computing Vol pages

Preliminary version in th STOC Earlier versions date to

S Goldwasser S Micali and RL Rivest A Digital Signature Scheme Secure

Against Adaptive ChosenMessage Attacks SIAM Journal on Computing

April pages

S Goldwasser and M Sipser Private Coins versus Public Coins in Interactive

Pro of Systems Advances in Computing Research a research annual Vol

Randomness and Computation S Micali ed pages Extended

abstract in th STOC

SW Golomb Shift Register Sequences HoldenDay Aegean Park

Press revised edition

J Hartmanis and RE Stearns On the Computational Complexity of of

Algorithms Transactions of the AMS Vol pages

J Hastad Almost Optimal Lower Bounds for Small Depth Circuits Ad

vances in Computing Research a research annual Vol Randomness and

Computation S Micali ed pages Extended abstract in

th STOC pages

BIBLIOGRAPHY

J Hastad Clique is hard to approximate within n Acta Mathematica

Vol pages Preliminary versions in th STOC

and th FOCS

J Hastad Getting optimal inapproximability results In th ACM Sympo

sium on the Theory of Computing pages

J Hastad R Impagliazzo LA Levin and M Luby A Pseudorandom Gen

erator from any Oneway Function SIAM Journal on Computing Volume

Number pages Preliminary versions by Impagliazzo

et al in st STOC and Hastad in nd STOC

J Hastad and S Khot Query ecient PCPs with p efect completeness In

nd IEEE Symposium on Foundations of Computer Science pages

A Healy S Vadhan and E Viola Using nondeterminism to amplify hardness

In th ACM Symposium on the Theory of Computing pages

JE Hop croft and JD Ullman Introduction to Automata Theory Languages

and Computation AddisonWesley

D Ho chbaum ed Approximation Algorithms for NPHard Problems PWS

N Immerman Nondeterministic Space is Closed Under Complementation

SIAM Journal on Computing Vol pages

R Impagliazzo Hardcore Distributions for Somewhat Hard Problems In

th IEEE Symposium on Foundations of Computer Science pages

R Impagliazzo and LA Levin No Better Ways to Generate Hard NP In

stances than Picking Uniformly at Random In st IEEE Symposium on

Foundations of Computer Science pages

R Impagliazzo and A Wigderson PBPP if E requires exp onential circuits

Derandomizing the XOR Lemma In th ACM Symposium on the Theory

of Computing pages

R Impagliazzo and A Wigderson Randomness vs Time Derandomization

under a Uniform Assumption Journal of Computer and System Science

Vol pages

R Impagliazzo and M Yung Direct ZeroKnowledge Computations In

Crypto SpringerVerlag Lecture Notes in Computer Science Vol

pages

M Jerrum A Sinclair and E Vigo da A PolynomialTime Approximation

Algorithm for the Permanent of a Matrix with NonNegative Entries Journal

of the ACM Vol pages

BIBLIOGRAPHY

M Jerrum L Valiant and VV Vazirani Random Generation of Combina

torial Structures from a Uniform Distribution Theoretical Computer Science

Vol pages

N Kahale Eigenvalues and Expansion of Regular Graphs Journal of the

ACM Vol pages September

R Kannan H Venkateswaran V Vinay and AC Yao A Circuitbased

Pro of of Todas Theorem Information and Computation Vol pages

RM Karp Reducibility among Combinatorial Problems In Complexity

of Computer Computations RE Miller and JW Thatcher eds Plenum

Press pages

RM Karp and RJ Lipton Some connections b etween nonuniform and uni

form complexity classes In th ACM Symposium on the Theory of Com

puting pages

RM Karp and M Luby MonteCarlo algorithms for enumeration and re

liability problems In th IEEE Symposium on Foundations of Computer

Science pages

RM Karp and V Ramachandran Parallel Algorithms for SharedMemory

Machines In Handbook of Theoretical Computer Science Vol A Algorithms

and Complexity

M Karchmer and A Wigderson Monotone Circuits for Connectivity Require

Sup erlogarithmic Depth SIAM J Discrete Math Vol pages

Preliminary version in th STOC

MJ Kearns and UV Vazirani An introduction to Computational Learning

Theory MIT Press

S Khot and O Regev Vertex Cover Might b e Hard to Approximate to

within In th IEEE Conference on Computational Complexity pages

VM Khrap chenko A metho d of determining lower b ounds for the com

plexity of Pischemes In Matematicheskie Zametki pages

in Russian English translation in Mathematical Notes of the Academy of

Sciences of the USSR pages

J Kilian A Note on Ecient ZeroKnowledge Pro ofs and Arguments In

th ACM Symposium on the Theory of Computing pages

DE Knuth The Art of Computer Programming Vol Seminumerical

Algorithms AddisonWesley Publishing Company Inc rst edition

and second edition

BIBLIOGRAPHY

A Kolmogorov Three Approaches to the Concept of The Amount Of In

formation Probl of Inform Transm Vol

E Kushilevitz and N Nisan Communication Complexity Cambridge Uni

versity Press

RE Ladner On the Structure of Polynomial Time Reducibility Journal of

the ACM Vol pages

C Lautemann BPP and the Polynomial Hierarchy Information Processing

Letters pages

FT Leighton Introduction to Parallel Algorithms and Architectures Arrays

Trees Hypercubes Morgan Kaufmann Publishers San Mateo CA

LA Levin Universal Search Problems Problemy Peredaci Informacii

pages Translated in problems of Information Transmission

pages

LA Levin Randomness Conservation Inequalities Information and Inde

p endence in Mathematical Theories Information and Control Vol pages

LA Levin Average Case Complete Problems SIAM Journal on Computing

Vol pages

LA Levin Fundamentals of Computing SIGACT News Education Forum

sp ecial th issue Vol pages

M Li and P Vitanyi An Introduction to Kolmogorov Complexity and its

Applications Springer Verlag August

CJ Lu O Reingold S Vadhan and A Wigderson Extractors optimal up

to constant factors In th ACM Symposium on the Theory of Computing

pages

A Lub otzky R Phillips and P Sarnak Ramanujan Graphs Combinatorica

Vol pages

M Luby and A Wigderson Pairwise Indep endence and Derandomization

TR International Computer Science Institute ICSI Berkeley

ISSN

C Lund L Fortnow H Karlo and N Nisan Algebraic Metho ds for In

teractive Pro of Systems Journal of the ACM Vol No pages

Preliminary version in st FOCS

F MacWilliams and N Sloane The theory of errorcorrecting codes North

Holland

BIBLIOGRAPHY

GA Margulis Explicit Construction of Concentrators In Russian Prob

Per Infor Vol pages English translation in Problems of

Infor Trans pages

S Micali Computationally Sound Pro ofs SIAM Journal on Computing

Vol pages Preliminary version in th FOCS

GL Miller Riemanns Hyp othesis and Tests for Primality Journal of Com

puter and System Science Vol pages

PB Miltersen and NV Vino dchandran Derandomizing ArthurMerlin

Games using Hitting Sets Journal of Computational Complexity to app ear

Preliminary version in th FOCS

R Motwani and P Raghavan Randomized Algorithms Cambridge University

Press

M Naor Bit Commitment using Pseudorandom Generators Journal of

Cryptology Vol pages

J Naor and M Naor Smallbias Probability Spaces Ecient Constructions

and Applications SIAM Journal on Computing Vol pages

M Naor and M Yung Universal OneWay Hash Functions and their Crypto

graphic Application In st ACM Symposium on the Theory of Computing

pages

N Nisan Pseudorandom bits for constant depth circuits Combinatorica

Vol pages

N Nisan Pseudorandom Generators for Space Bounded Computation Com

binatorica Vol pages

N Nisan RL SC Journal of Computational Complexity Vol pages

N Nisan and A Wigderson Hardness vs Randomness Journal of Computer

and System Science Vol No pages

N Nisan and D Zuckerman Randomness is Linear in Space Journal of

Computer and System Science Vol pages

CH Papadimitriou Computational Complexity Addison Wesley

CH Papadimitriou and M Yannakakis Optimization Approximation and

Complexity Classes In th ACM Symposium on the Theory of Computing

pages

N Pipp enger and MJ Fischer Relations among complexity measures Jour

nal of the ACM Vol pages

BIBLIOGRAPHY

E Post A Variant of a Recursively Unsolvable Problem Bul l AMS Vol

pages

MO Rabin Digitalized Signatures In Foundations of Secure Computation

RA DeMillo et al eds Academic Press

MO Rabin Digitalized Signatures and Public Key Functions as Intractable

as Factoring MITLCSTR

MO Rabin Probabilistic Algorithm for Testing Primality Journal of Num

ber Theory Vol pages

R Raz A Parallel Rep etition Theorem SIAM Journal on Computing

Vol pages Extended abstract in th STOC

R Raz and A Wigderson Monotone Circuits for Matching Require Linear

Depth Journal of the ACM Vol pages Preliminary

version in nd STOC

A Razb orov Lower b ounds for the monotone complexity of some Bo olean

functions In Doklady Akademii Nauk SSSR Vol No pages

English translation in Soviet Math Doklady pages

A Razb orov Lower b ounds on the size of b oundeddepth networks over a

complete basis with logical addition In Matematicheskie Zametki Vol

No pages English translation in Mathematical Notes of the

Academy of Sci of the USSR Vol pages

AR Razb orov and S Rudich Natural Pro ofs Journal of Computer and

System Science Vol pages

O Reingold Undirected STConnectivity in LogSpace In th ACM Sym

posium on the Theory of Computing pages

O Reingold S Vadhan and A Wigderson Entropy Waves the ZigZag

Graph Pro duct and New ConstantDegree Expanders and Extractors An

nals of Mathematics Vol pages Preliminary version

in st FOCS pages

HG Rice Classes of Recursively Enumerable Sets and their Decision Prob

lems Trans AMS Vol pages

RL Rivest A Shamir and LM Adleman A Metho d for Obtaining Digital

Signatures and Public Key Cryptosystems CACM Vol Feb pages

D Ron Prop erty testing In Handbook on Randomization Volume II

pages Editors S Ra jasekaran PM Pardalos JH Reif

and JDP Rolim

BIBLIOGRAPHY

R Rubinfeld and M Sudan Robust characterization of p olynomials with

applications to program testing SIAM Journal on Computing Vol

pages

M Saks and S Zhou RSPACES DSPACES In th IEEE Sym

posium on Foundations of Computer Science pages

WJ Savitch Relationships b etween nondeterministic and deterministic tap e

complexities JCSS Vol pages

A Selman On the structure of NP Notices Amer Math Soc Vol

page

R Shaltiel Recent Developments in Explicit Constructions of Extractors In

Current Trends in Theoretical Computer Science The Chal lenge of the New

Century Vol Algorithms and Complexity World scietic Editors

G Paun G Rozenberg and A Salomaa Preliminary version in Bul letin of

the EATCS pages

R Shaltiel and C Umans Simple Extractors for All MinEntropies and a

New PseudoRandom Generator In nd IEEE Symposium on Foundations

of Computer Science pages

CE Shannon A Symbolic Analysis of Relay and Switching Circuits Trans

American Institute of Electrical Engineers Vol pages

CE Shannon A mathematical theory of communication Bel l Sys Tech

Jour Vol pages

CE Shannon Communication Theory of Secrecy Systems Bel l Sys Tech

Jour Vol pages

A Shamir IP PSPACE Journal of the ACM Vol No pages

Preliminary version in st FOCS

A Shpilka Lower Bounds for Matrix Pro duct SIAM Journal on Computing

pages

M Sipser A Complexity Theoretic Approach to Randomness In th ACM

Symposium on the Theory of Computing pages

M Sipser Introduction to the Theory of Computation PWS Publishing

Company

R Smolensky Algebraic Metho ds in the Theory of Lower Bounds for Bo olean

Circuit Complexity In th ACM Symposium on the Theory of Computing

pages

RJ Solomono A Formal Theory of Inductive Inference Information and

Control Vol pages

BIBLIOGRAPHY

R Solovay and V Strassen A Fast MonteCarlo Test for Primality SIAM

Journal on Computing Vol pages Addendum in SIAM Jour

nal on Computing Vol page

DA Spielman Advanced Complexity Theory Lectures and

Notes by D Lewin and S Vadhan March Available

from httpwwwcsyaleeduhomesspielmanAdvComplexity as

lectps and lectps

LJ Sto ckmeyer The PolynomialTime Hierarchy Theoretical Computer

Science Vol pages

L Sto ckmeyer The Complexity of Approximate Counting In th ACM

Symposium on the Theory of Computing pages

V Strassen Algebraic Complexity Theory In Handbook of Theoretical Com

puter Science Volume A Algorithms and Complexity J van Leeuwen edi

tor MIT PressElsevier pages

M Sudan Deco ding of Reed Solomon co des b eyond the errorcorrection

b ound Journal of Complexity Vol pages

M Sudan Algorithmic introduction to co ding theory Lecture notes Avail

able from httptheorycsailmitedumadhuFT

M Sudan L Trevisan and S Vadhan Pseudorandom generators without

the XOR Lemma Journal of Computer and System Science Vol No

pages

R Szelep csenyi A Metho d of Forced Enumeration for Nondeterministic Au

tomata Acta Informatica Vol pages

S Toda PP is as hard as the p olynomialtime hierarchy SIAM Journal on

Computing Vol pages

BA Trakhtenbrot A Survey of Russian Approaches to Perebor Brute Force

Search Algorithms Annals of the History of Computing Vol pages

L Trevisan Constructions of NearOptimal Extractors Using Pseudo

Random Generators In st ACM Symposium on the Theory of Computing

pages

V Trifonov An O log n log log n Space Algorithm for Undirected st

Connectivity In th ACM Symposium on the Theory of Computing pages

CE Turing On Computable Numbers with an Application to the Entschei

dungsproblem Proc Londom Mathematical Soceity Ser Vol pages

A Correction ibid Vol pages

BIBLIOGRAPHY

C Umans Pseudorandom generators for all hardness Journal of Computer

and System Science Vol pages

S Vadhan A Study of Statistical ZeroKnowledge Pro ofs PhD

Thesis Department of Mathematics MIT Available from

httpwwweecsharvardedusalilpapersphdthesisabshtml

S Vadhan An Unconditional Study of Computational Zero Knowledge In

th IEEE Symposium on Foundations of Computer Science pages

LG Valiant The Complexity of Computing the Permanent Theoretical

Computer Science Vol pages

LG Valiant A theory of the learnable CACM Vol pages

LG Valiant and VV Vazirani NP Is as Easy as Detecting Unique Solutions

Theoretical Computer Science Vol pages

J von Neumann First Draft of a Rep ort on the EDVAC Contract No

WORD Mo ore School of Electrical Engineering Univ of Penn

Philadelphia Reprinted in part in Origins of Digital Computers Selected

Papers SpringerVerlag Berlin Heidelb erg pages

J von Neumann Zur Theorie der Gesellschaftsspiele Mathematische An

nalen pages

I Wegener The Complexity of Boolean Functions WileyTeubner

I Wegener Branching Programs and Binary Decision Diagrams Theory and

Applications SIAM Monographs on Discrete Mathematics and Applications

A Wigderson The amazing p ower of pairwise indep endence In th ACM

Symposium on the Theory of Computing pages

AC Yao Theory and Application of Trapdo or Functions In rd IEEE

Symposium on Foundations of Computer Science pages

AC Yao Separating the PolynomialTime Hierarchy by Oracles In th

IEEE Symposium on Foundations of Computer Science pages

AC Yao How to Generate and Exchange Secrets In th IEEE Symposium

on Foundations of Computer Science pages

D Zuckerman Simulating BPP Using a General Weak Random Source

Algorithmica Vol pages

D Zuckerman RandomnessOptimal Oblivious Sampling Journal of Ran

dom Structures and Algorithms Vol Nr December pages