Computational Complexity

Home , Computational resource, Oded Goldreich

Avi Wigderson Oded Goldreich

School of Mathematics Department of Computer Science

Institute for Advanced Study Weizmann Institute of Science

Princeton NJ USA Rehovot Israel

aviiasedu odedgoldreichweizmann aci l

Octob er

Abstract

The strive for eciency is ancient and universal as time is always short for humans Com

putational Complexity is a mathematical study of the what can b e achieved when time and

other resources are scarce

In this brief article we will introduce quite a few notions Formal mo dels of computation

and measures of eciency the P vs NP problem and NPcompleteness circuit complexity and

pro of complexity randomized computation and pseudorandomness probabilistic pro of systems

cryptography and more A glossary of complexity classes is included in an app endix We highly

recommend the given bibliography and the references therein for more information

Contents

Introduction

Preliminaries

Computability and Algorithms

Ecient Computability and the class P

The P versus NP Question

Ecient Verication and the class NP

The Big Conjecture

NP versus coNP

Reducibility and NPCompleteness

Lower Bounds

Bo olean Circuit Complexity

Basic Results and Questions

Monotone Circuits

BoundedDepth Circuits

Formula Size

Why Is It Hard to Prove Lower Bounds

Arithmetic Circuits

Univariate Polynomials

Multivariate Polynomials

Pro of Complexity

Logical Pro of Systems

Algebraic Pro of Systems

Geometric Pro of Systems

Randomized Computation

Counting at Random

Probabilistic Pro of Systems

Interactive Pro of Systems

ZeroKnowledge Pro of Systems

Probabilistically Checkable Pro of systems

Weak Random Sources

The Bright Side of Hardness

Pseudorandomness

Hardness versus Randomness

Pseudorandom Functions

Cryptography

The Tip of an Iceb erg

Relaxing the Requirements

AverageCase Complexity

Approximation

Other Complexity Measures

Other Notions of Computation

Concluding Remarks

Bibliography

App endix Glossary of Complexity Classes

A Algorithmbased classes

A Circuitbased classes

Introduction

Computational Complexity or Complexity Theory is a central subeld of the theoretical founda

tions of Computer Science It is concerned with the study of the intrinsic complexity of computa

tional tasks This study tends to aim at generality it fo cuses on natural computational resources

and considers the eect of limiting these resources on the class of problems that can b e solved

It also tends to asymptotics studying this complexity as the size of data grows Another related

subeld represented in this volume deals with the design and analysis of algorithms for specic

classes of computational problems that arise in a variety of areas of mathematics science and

engineering

The halfcentury history of Complexity Theory has witnessed two main research eorts or

directions The rst direction is aimed towards actually establishing concrete lower b ounds on

the complexity of problems via an analysis of the evolution of the pro cess of computation Thus

in a sense the heart of this direction is a lowlevel analysis of computation Most research in

circuit complexity and in pro of complexity falls within this category In contrast a second research

eort is aimed at exploring the connections among computational problems and notions without

b eing able to provide absolute statements This eort may b e viewed as a highlevel study of

computation The theory of NPcompleteness the study of probabilistic pro of systems as well as

pseudorandomness and cryptography all falls within this category

Preliminaries

This exp osition considers only nite ob jects enco ded by nite binary sequences called strings For

a natural number n we denote by f g the set of all binary sequences of length n hereafter

referred to as nbit strings The set of all strings is denoted f g that is f g f g

jxj

For x f g we denote by jxj the length of x ie x f g At times we asso ciate

f g f g with f g Natural numbers will b e enco ded by their binary expansion

Computability and Algorithms

We are all familiar with computers and the ability of computer programs to manipulate data

But how do es one capture al l computational pro cesses Before b eing formal we oer a lo ose

description capturing many articial as well as natural pro cesses and invite the reader to compare

it with physical theories

A computation is a pro cess that mo dies an environment via rep eated applications of a prede

termined rule The key restriction is that this rule is simple in each application it dep ends and

aects only a small p ortion of the environment called the active zone We contrast the apriori

bounded size of the active zone and of the mo dication rule with the apriori unbounded size of

the entire environment We note that although each application of the rule has a very limited

eect the eect of many applications of the rule may b e very complex The computation rule

esp ecially when designed to eect a desired computation is often referred to as an algorithm

Such pro cesses naturally compute functions and their complexity is naturally captured by the

number of steps they apply Let us elab orate

We are interested in the transformation of the environment eected by the computational pro

cess Typically the initial environment to which the computation is applied enco des an input string

and the end environment ie at termination of the computation enco des an output string We

We assume that when invoked on any nite initial environment the computation halts after a nite number of

consider the mapping from inputs to outputs induced by the computation that is for each p ossible

input x we consider the output y obtained at the end of a computation initiated with input x and

say that the computation maps input x to output y We also consider the number of steps ie

applications of the rule taken by the computation for each input The latter function is called

the time complexity of the computational pro cess While time complexity is dened p er input one

often considers it p er input length taking the maximum over all inputs of the same length

To dene computation and computation time rigorously one needs to sp ecify some mo del of

computation that is provide a concrete denition of environments and a class of rules that may

b e applied to them Such a mo del corresp onds to an abstraction of a real computer b e it a PC

mainframe or network of computers One simple abstract mo del that is commonly used is that

of Turing machines see eg Thus sp ecic algorithms and their complexity are typically

formalized by corresp onding Turing machines We stress however that most results in the Theory of

Computation hold regardless of the sp ecic computational mo del used as long as it is reasonable

ie satises the aforementioned simplicity condition

The ab ove discussion has implicitly referred to computations and Turing machines as a means of

computing functions Sp ecically a Turing machine M computes the function f f g f g

dened by f x y if when invoked on input x machine M halts with output y For example

we may refer to the computation of the integer multiplication function which given an enco ding of

two integers returns the enco ding of their pro duct However computations can also b e viewed as a

means of solving problems or making decisions which are captured resp ectively by relations

and sets

Search problems are captured by binary relations R f g f g with the semantics that

y is called a valid solution for problem instance x if and only if x y R Machine M solves the

search problem R if x f x R whenever a solution for x exists that is given an instance x

that has a valid solution machine M nds some valid solution for x For example we may refer

to a machine that given a system of p olynomial equations returns a valid solution

Decision problems are captured by sets S f g with the semantics that S is the set of

yesinstances of the problem We say that M solves the decision problem S if it holds that

f x if and only if x S that is given an instance x machine M determines whether or

not x S For example we may refer to a machine that given a natural number determines

whether or not it is prime At times it will b e convenient to view decision problems as Bo olean

functions dened on the set of all strings ie S f g f g rather than as sets of strings

ie S f g

In the rest of this exp osition we asso ciate the machine M with the function f computed by

it that is we write M x instead of f x

Ecient Computability and the class P

So far we have mathematically dened all tasks that can b e computed in principle and the time

such computations take Now we turn to dene what can b e computed eciently and then discuss

the choices made in this denition

We call an algorithm ecient if it terminates within time that is polynomial in the length of

its inputs Understanding the class of problems called P b elow that have such algorithms is the

ma jor goal of Computational Complexity Theory

Denition the complexity class P A decision problem S f g is solvable in p olynomial

steps

time if there exists a deterministic polynomialtime Turing machine M such that M x if and

only if x S The class of decision problems that are solvable in polynomial time is denoted P

The asymptotic analysis of runningtime ie considering runningtime as a function of the

input length turned out to b e crucial for revealing structure in the theory of ecient computation

The choice of polynomial time may seem arbitrary and theories could b e developed with other

choices but again proved itself right Polynomials are viewed as the canonical class of slowly

growing functions that enjoy closure prop erties relevant to computation Sp ecically the class is

closed under addition multiplication and comp osition The growth rate of p olynomials allows us

to consider as ecient essentially all problems for which practical computer programs exist and

the closure prop erties of p olynomials guarantee robustness of the notion of ecient computation

Finally while n time algorithms are called ecient here despite their blatant impracticality

one rarely discovers even an n time algorithm for a natural problem and when this happ ens

improvements to n or n time which b order on the practical typically follow

It is imp ortant to contrast P to the class EXP of all problems solvable in time exponential in

the length of their inputs Exp onential running time is considered blatantly inecient and if the

problem has no faster algorithm then it is deemed intractable It is known via a basic technique

called diagonalization that P EXP furthermore some problems in EXP do require exp onential

time We note that almost all problems and classes considered in this pap er will b e in EXP via

trivial brute force algorithms and the main question will b e if much faster algorithms can b e

devised for them

Note that so far we restricted computation to b e a deterministic pro cess In Section we pursue

the imp ortant relaxation of allowing randomness coin tosses in computation and its impact on

eciency and other computational notions

The P versus NP Question

In a nutshell the P versus NP question is whether creativity can be automated This applies to all

tasks for which a successful completion can b e easily recognized A particular sp ecial case which

is in fact quite general and has natural app eal to Mathematicians is the task of determining if a

mathematical statement is true Here successful completion is a pro of so the P versus NP Question

can b e informally stated as whether verifying pro ofs which we view as a mechanical pro cess is

or is not much easier than nding a pro of which we view as creative In general the class NP

captures all problems for which an adequate solution when given can b e eciently veried while

the class P captures all problems that can b e solved eciently without such external help We

now turn to formally dene these notions

Ecient Verication and the class NP

The fundamental class NP of decision problems consists of the class of sets S for which there exist

short pro ofs that x S of membership that which can b e eciently veried These two ingredients

are captured by two prop erties of an auxiliary binary relation R P in which all y for which

x y R have p olynomial in jxj length and such a pro of y exists i x S thus certifying

or witnessing or proving this fact

The acronym NP stands for Nondeterministic Polynomialtime where a nondeterministic machine is a ctitious

computing device used in an alternative denition of the class NP The nondeterministic moves of such a machine

corresp ond to guessing a pro of in Denition

Denition the complexity class NP A binary relation R f g f g is called p oly

nomially b ounded if there exists a polynomial p such that jy j pjxj holds for every x y R A

decision problem S is in NP if there exists a polynomially bounded binary relation R such that

R is in P and x S if and only if there exists y such that x y R Such a y is called a proof

S S

or witness of memb ership of x S

We note that trivially NP EXP since we can go over all p ossible y s in exp onential time

Can this trivial algorithm b e improved Since P is the class of sets for which membership can b e

eciently decided without b eing given a pro of it follows that P N P Thus the P versus NP

Question can b e cast as follows does the existence of an ecient verication procedure for proofs

of membership in a certain set imply the ability to eciently decide membership in the same set

Op en Problem Is NP equal to P

Natural search problems arise from every p olynomially b ounded relation R P namely given

x nd any y for which x y R if such a solution exists Note that the p olynomial b ound on

the length of y guarantees that the search problem is not trivially intractable as would b e the case

if all solutions had length that is sup erp olynomial in the length of the instance Furthermore

R P implies that the search problem is natural in the sense that one can eciently recognize

the validity of a solution to a problem instance One often views NP as the class of all such search

problems that is the class of search problems referring to relations R P that are p olynomially

b ounded The search analog of the P versus NP question is whether the ecient verication of

candidate solutions necessarily entails that valid solutions are easy to nd Indeed the search and

decision versions of the P versus NP question are equivalent

The Big Conjecture

It is widely b elieved that P NP Settling this conjecture is certainly the most imp ortant op en

problem in Computer Science and among the most signicant in Mathematics The P NP

Conjecture is supp orted by our strong intuition developed over centuries in a variety of human

activities that nding solutions is far harder than verifying their adequacy Further empirical

evidence in favor of the conjecture is given by the fact that literally thousands of NP problems in a

wide variety of mathematical and scientic disciplines are not known to b e solvable in p olynomial

time in spite of extensive research attempts aimed at providing ecient pro cedures for solving

them One famous example is the Integer Factorization problem given a natural number nd its

prime factorization

The section on Circuit Complexity Section is devoted to attempts to prove this conjecture

discussing some partial results and limits of the techniques used so far

NP versus coNP

Assuming that P NP it is unclear whether the existence of an ecient verication pro cedure for

pro ofs of membership in a set implies the existence of an ecient verication pro cedure for pro ofs

of nonmembership in that set Let coNP denote the class of sets that are complements of sets in

def

NP ie co NP ff g n S S N P g

Op en Problem Is NP equal to co NP

It is widely b elieved that co NP NP Indeed this implies P NP Here again intuition

from Mathematics is extremely relevant to verify that a set of logical constraints is mutually

inconsistent that a family of p olynomial equations have no common ro ot that a set of regions

in space has empty intersection seems far harder to prove than their complements exhibiting the

consistent valuation ro ot p oint resp Indeed only when rare extra mathematical structure is

available do we have duality theorems or complete systems of invariants implying computational

equivalence of the set and its complement The section on Pro of Complexity Section deals

further with this conjecture and attempts to resolve it

Reducibility and NPCompleteness

In this section we attempt to identify the hardest problems in NP For this we shall dene a

natural partial order on decision problems called p olynomialtime reducibility and dene maximal

elements in NP under this order to b e complete We note that reductions and completeness are

key concepts in Complexity Theory

A general notion of polynomialtime reducibility among computational problems is obtained

by considering a p olynomialtime machine for solving one problem eg computing a function f

that may issue queries to another problem eg to a function g Thus if the latter problem can

b e solved eciently then so can the former One restricted notion of a reduction which refers to

decision problems requires the reduction machine to issue a single query and output the answer it

obtains In this case a simpler formulation follows

Denition Polynomialtime Reducibility A set S is p olynomialtime reducible to the set

T if there exist polynomialtime computable function h such that x S if and only if hx T

Denition NPCompleteness A decision problem S is NPcomplete if S is in NP and every

decision problem in NP is polynomialtime reducible to S

Thus NPcomplete decision problems are universal in the sense that providing a p olynomial

time pro cedure for solving any of them will immediately imply p olynomialtime pro cedures for

solving all other problems in NP and in particular al l NPcomplete decision problems Further

more in a sense each of these NPcomplete problems eciently enco des all the other problems

and in fact all NP search problems For example the Integer Factorization problem can b e e

ciently enco ded in any NPcomplete problem which may have nothing to do with integers Thus

at rst glance it seems very surprising that NPcomplete problems exist at all

Theorem There exist NPcomplete decision problems Furthermore the fol lowing decision prob

lems are NPcomplete

SAT Given a propositional formula decide whether or not it is satisable

Coloring Given a planar map decide whether or not it is colorable

Such a machine is called an oracle machine and in the ab ove case we say that it computes the function f by

issuing queries to the oracle function g such that for query q the answer is g q

The problem remains NPcomplete even when instances are restricted to b e in Conjunctive Normal Form CNF

and even when each clause has exactly three literals These formulae are said to b e in CNF form and the set of

satisable CNF formulae is denoted SAT

Recall that the celebrated color Theorem asserts that colors always suce In contrast to the NPcompleteness

of deciding colorability it is easy to decide colorability of arbitrary graphs and in particular of planar maps

Subset Sum Given a sequence of integers a a and b decide whether or not there exists a set

a b I such that

The decision problems mentioned ab ove are but three examples among literally thousands of natural

NPcomplete problems from a wide variety of mathematical and scientic disciplines Hundreds

of such problems are listed in

Assuming that P NP no NPcomplete problem has a p olynomialtime decision pro cedure

Consequently the corresp onding NP search problems cannot b e solved in p olynomial time Thus

pro ofs of NPcompleteness are often taken as evidence to the intrinsic diculty of a problem

Positive applications of NPcompleteness are also known in some cases a claim regarding

all NPsets is proved by establishing the claim only for some NPcomplete set and noting that

p olynomialtime reductions preserve the claimed prop erty Famous examples include the existence

of ZeroKnowledge pro ofs established rst for coloring see Section and the PCP Theorem

established rst for SAT see Section

We note that almost every natural problem in NP ever considered turns out to b e either NP

complete or in P Curiously only a handful of natural problems including Integer Factorization

and Graph Isomorphism are not known to b elong to either of these classes and indeed there is

strong evidence they dont

Lower Bounds

In this section we survey some basic attempts at proving lower b ounds on the complexity of natural

computational problems In the rst part Circuit Complexity we describ e lower b ounds for the

size of circuits that solve natural computational problems This can b e viewed as a program whose

longterm goal is proving that P NP In the second part Pro of Complexity we describ e lower

b ounds on the length of prop ositional pro ofs of natural tautologies This can b e viewed as a

program whose longterm goal is proving that NP co NP Both mo dels refer to the nite mo del

of directed acyclic graphs DAGs which we dene next

A DAG GV E consists of a nite set of vertices V and a set of ordered pairs called directed

edges E V V in which there are no directed cycles The vertices with no incoming edges are

called the inputs of the DAG G and the vertices with no outgoing edges are called the outputs We

will restrict ourselves to DAGs in which the number of incoming edges to every vertex is at most

If the number of outgoing edges from every no de is at most the DAG is called a tree Finally

we assume that every vertex can b e reached from some input via a directed path The size of a

DAG will b e its number of edges

To make a DAG into a computational device or a pro of each noninput vertex will b e marked

by a rule converting values in its predecessors to values at that vertex It is easy to see that the

vertices of every DAG can b e linearly ordered such that predecessors of every vertex if any app ear

b efore it in the ordering Thus if the input vertices are lab eled with some values we can lab el the

remaining vertices in that order one at a time till all vertices and in particular all outputs are

lab eled

For computation the noninput vertices will b e marked by functions called gates which make

the DAG a circuit If we lab el the input vertices by sp ecic values from some domain the outputs

will b e determined by them and the circuit will naturally dene a function from input values to

output values

For pro ofs the noninput vertices will b e marked by sound deduction or inference rules which

make the DAG a proof If we lab el the inputs by formulae that are axioms in a given pro of system

the output again will b e determined by them and will yield the tautology proved by this pro of

We note that b oth settings t the paradigm of simplicity shared by computational mo dels

discussed in the previous section the rules are simple by denition they are applied to at most

previous values The main dierence is that this mo del is nite each DAG can compute only

functionspro ofs with a xed input length To allow all input lengths one must consider families

of DAGs one for each thus signicantly extending the p ower of the computation mo del b eyond

that of the notion of algorithm dened earlier However as we are interested in lower b ounds

here this is legitimate and one can hop e that the niteness of the mo del will p otentially allow for

combinatorial techniques to analyze its p ower and limitations Furthermore these mo dels allow for

the introduction and study of meaningful restricted classes of computations

We use the following asymptotic notation For f g N N by f O g resp f g we

mean that there exists a constant c such that f n c g n resp f n c g n for all

n N

Bo olean Circuit Complexity

In Bo olean circuits all inputs outputs and values at intermediate no des of the DAG are bits The

set of allowed gates is naturally taken to b e a complete basis one that allows the circuit to compute

al l Bo olean functions The sp ecic choice of a complete basis hardly eects the study of circuit

complexity A typical choice is the set f g of resp ectively conjunction disjunction each on

bits and negation on bit

Denition Denote by S f the size of the smal lest Boolean circuit computing f

We will b e interested in sequences of functions ff g where f is a function on n input bits and

n n

will study the complexity S f asymptotically as a function of n With some abuse of notation

def

for f x f x we let S f denote the integer function that assigns to n the value S f

jxj

We note that dierent circuits in particular having a dierent number of inputs are used for

each f Still there may b e a simple description of this sequence of circuits say an algorithm that

on input n pro duces a circuit computing f In case such an algorithm exists and works in time

p olynomial in the size of its output we say that the corresp onding sequence of circuits is uniform

Note that if f has a uniform sequence of p olynomialsize circuits then f P On the other hand it

can b e shown that any f P has a uniform sequence of p olynomialsize circuits Consequently

a sup erp olynomial circuit lowerbound on any function in NP would imply that P NP

But Denition makes no reference to uniformity and indeed the sequence of smallest circuits

computing ff g may b e highly nonuniform Indeed nonuniformity makes the circuit mo del

stronger than Turing machines or equivalently than the mo del of uniform circuits there exist

functions f that cannot b e computed by Turing machines regardless of their running time but

do have linearsize circuits So isnt proving circuit lower b ounds a much harder task than we need

to resolve the P vs NP question

The answer is that there is a strong sentiment that the extra p ower provided by nonuniformity

is irrelevant to the P vs NP question that is it is conjectured that NPcomplete sets do not

have p olynomialsize circuits This conjecture is supp orted by the fact that its failure will yield

an unexp ected collapse in the complexity of standard computations Furthermore the hop e is

that abstracting away the supp osedly irrelevant uniformity condition will allow for combinatorial

techniques to analyze the p ower and limitations of p olynomialsize circuits wrt NPsets This

hop e has materialized in the study of restricted classes of circuits see Sections and

We also mention that Bo olean circuits are a natural computational mo del corresp onding to

hardware complexity and so their study is of indep endent interest Moreover some of the

techniques for analyzing Bo olean functions found applications elsewhere eg in computational

learning theory combinatorics and game theory

Basic Results and Questions

We have already mentioned several basic facts ab out Bo olean circuits in particular the fact that

they can eciently simulate Turing Machines The next basic fact is that most Boolean functions

require exponential size circuits which is due to the gap b etween the number of functions and the

number of small circuits

So hard functions for circuits and hence for Turing machines ab ound However the hardness

ab ove is proved via a counting argument and thus supplies no way of putting a nger on one

hard function Using more conventional language we cannot prove such hardness for any explicit

function f eg for an NPcomplete function like SAT or even for functions in EXP The situation

is even worse no nontrivial lowerbound is known for any explicit function Note that for any

function f on n bits which dep ends on all its inputs we trivially must have S f n just to

read the inputs The main op en problem of circuit complexity is b eating this trivial b ound

Op en Problem Find an explicit Boolean function f or even a lengthpreserving function f

for which S f is not O n

A particularly basic sp ecial case of this problem is the question whether addition is easier to p er

n n n n n n

form than multiplication Let ADD f g f g f g and MULT f g f g f g

denote resp ectively the addition and multiplication functions on a pair of integers presented in

binary For addition we have an optimal upp er b ound that is S ADD O n For multiplication

the standard elementary school quadratictime algorithm can b e greatly improved via Discrete

Fourier Transforms to slightly sup erlinear yielding S MULT O n log n Now the question

is whether or not there exist linearsize circuits for multiplication ie is S MULT O n

Unable to prove any nontrivial lower b ound we now turn to restricted mo dels There has b een

some remarkable successes in developing techniques for proving strong lower b ounds for natural

restricted classes of circuits We describ e the most imp ortant ones

General Bo olean circuits as describ ed ab ove can compute every function and can do it at least

as eciently as general uniform algorithms Restricted circuits may b e only able to compute a

sub class of all functions eg monotone functions The restriction makes sense when either the

related classes of functions or the computations represented by the restricted circuits are natu

ral from a programming or a mathematical viewp oint The mo dels discussed b elow satisfy this

condition

Monotone Circuits

An extremely natural restriction comes by forbidding negation from the set of gates namely allowing

only f g The resulting circuits are called monotone circuits and it is easy to see that they can

compute every function f f g f g that is monotone with resp ect to the standard partial

order on nbit strings x y i for every bit p osition i we have x y

i i

It is as easy to see that most monotone functions require exp onential size monotone circuits

Still proving a sup erp olynomial lower b ound on an explicit monotone function was op en for over

years till the invention of the socalled approximation method

Let CLIQUE b e the function that given a graph on n vertices by its adjacency matrix outputs

p p

n namely all pairs of vertices in some n i it contains a complete subgraph of size say

subset are connected by edges This function is clearly monotone Moreover it is known to b e

NPcomplete

Theorem There are no polynomialsize monotone circuits for CLIQUE

We note that similar lowerbounds are known for functions in P

BoundedDepth Circuits

The next restriction is structural we allow all gates but limit the depth of the circuit The depth

of a DAG is simply the length of the longest directed path in it So in a sense depth captures the

parallel time to compute the function if a circuit has depth d then the function can b e evaluated

by enough pro cessors in d phases where in each phase many gates are evaluated at once Parallel

time is another imp ortant computational resource

We will restrict d to b e a constant which still is interesting not only as parallel time but

also due to the relation of this mo del to expressibility in rst order logic as well as to complexity

classes ab ove NP called the Polynomialtime Hierarchy see section I I I In the current setting of

constantdepth circuits we allow unbounded fanin ie gates and gates taking any number

of incoming edges as otherwise each output bit can dep end only on a constant number of input

bits

Let PAR for parity denote the sum mo dulo two of the input bits and MAJ for ma jority b e

i there are more s than s among the input bits The invention of the random restriction

method led to the following basic result

Theorem For al l constant d PAR and MAJ have no polynomial size circuit of depth d

Interestingly MAJ remains hard for constantdepth p olynomialsize circuits even if the circuits

are also allowed unbounded fanin PARgates this result is based on yet another pro of technique

approximation by polynomials However the converse do es not hold and the class of constant

depth p olynomialsize circuits with MAJgates seems quite p owerful

Formula Size

The nal restriction is again structural we require the DAG to b e a tree Intuitively this forbids

the computation from reusing a previously computed partial result and if it is needed again it has

to b e recomputed The resulting circuits are simply formulae which are natural not only for their

prevalent mathematical use but also since their size can b e related to the memory requirements of

a Turing machine Here we go back to the standard basis of negation and bit input gates

One of the oldest results on Circuit Complexity is that PAR and MAJ are nontrivial in this mo del

The pro of follows a simple combinatorial or information theoretic argument

Theorem Boolean formuale for nbit PAR and MAJ require size n size

This should b e contrasted with the linearsize circuits that exist for b oth functions We comment

that S PAR O n is trivial but S MAJ O n is not

Can we give sup erp olynomial lower b ounds on formula size One of the cleanest metho ds sug

gested is the communication complexity method which we demonstrate informally with an example

Consider two players the rst having a prime number x and the second having a composite

number y Clearly any two such numbers must dier on at least one bit p osition in their

binary expansion ie there exists an i st x y and it is the goal of the parties to nd such

i i

an i To that end the party exchange messages according to a predetermined proto col and the

question is what is the communication complexity in terms of total number of bits exchanged on

the worstcase input pair of the b est such proto col Proving a sup erlogarithmic lowerbound will

establish the widely b elieved conjecture that testing primality has no p olynomial size formulae

Note that a lower b ound of purely information theoretic nature no computational restriction were

paced on the parties implies a computational one

Why Is It Hard to Prove Lower Bounds

The failure to obtain nontrivial lower b ounds for general circuit in a span of years raises

the question of whether there is a fundamental reason for this failure The same may b e asked

ab out any long standing mathematical problem eg the Riemann Hyp othesis and the typical

vague answer would b e that probably the current to ols and ideas which may well have b een

successful at attacking related easier problems do not suce Complexity Theory can make this

vague statement into a theorem Thus we have a formal excuse for our failure so far we can

classify a general set of ideas and to ols which are resp onsible for virtually all restricted lower

b ounds known yet must necessarily fail for proving general ones This introspective result suggests

a framework called Natural Proofs which encapsulates all known lower b ound techniques It shows

that natural pro ofs of general circuit lower b ounds for explicit functions surprisingly imply

ecient algorithms of a type conjectured not to exist eg for integer factoring

One interpretation of the aforementioned result is an indep endence result of general circuit

lower b ounds from a certain natural fragment of Peano Arithmetic This may hint that the P

vs NP problem may b e indep endent from PA or even Set Theory although few b elieve the latter

to b e the case

Arithmetic Circuits

We now leave the Bo olean rind and discuss circuits over general elds Fix any eld F The gates

of the DAG will now b e the standard and op erations in the eld This requires two immediate

clarications First to allow using constants of the eld one adds a sp ecial input vertex whose

value is the constant of the eld Moreover multiplication by any eld element eg is

free Second one may wonder ab out division However we will b e mainly interested in computing

p olynomials and for computing p olynomials over innite elds division can b e eciently emulated

by the other op erations

Now the inputs of the DAG will hold elements of the eld F and hence so will all computed

n m

values at vertices Thus an arithmetic circuit computes a p olynomial map p F F and every

such p olynomial map is computed by some circuit We denote by S p the size of a smallest circuit

computing p when no subscript is given F Q the eld of rational numbers As usual well b e

interested in sequences of p olynomials one for every input size and will study size asymptotically

It is easy to see that over any xed nite eld arithmetic circuits can simulate Bo olean circuits

on Bo olean inputs with only constant factor loss in size Thus the study of arithmetic circuits

fo cuses more on innite elds where lower b ounds may b e easier to obtain

This result as the aforementioned one rely on the existence of oneway functions see Section

As in the Bo olean case the existence of hard functions is easy to establish via dimension

considerations rather than counting argument and we will b e interested in explicit families of

p olynomials However the notion of explicitness is more delicate here eg allowing p olynomials

with algebraically indep endent co ecients would yield strong lower b ounds which are of no interest

whatso ever Very roughly sp eaking p olynomials are called explicit if the mapping from monomials

to a nite description of their co ecients has an ecient program

An imp ortant parameter which is absent in the Bo olean mo del is the degree of the p olynomials

computed It is obvious for example that a degree d p olynomial even in one variable ie n

requires size at least log d We briey consider the univariate case in which d is the only measure of

input size which already contains striking and imp ortant problems Then we move to the general

multivariate case in which as usual n the number of inputs will b e the main parameter

Univariate Polynomials

How tight is the log d lower b ounds for the size of an arithmetic circuit computing a degree d

p olynomial A simple dimension argument shows that for most degree d p olynomials p S p

d However we know of no explicit one

Op en Problem Find an explicit polynomial p of degree d such that S p is not O log d

Two concrete examples are illuminating Let px x and q x x x x d

Clearly S p log d via rep eated squaring so the trivial lower b ound is tight On the other

hand it is a ma jor op en problem to determine S q and the conjecture is that S q log d

To realize the imp ortance of this question we state the following fact If S q log d then

Integer Factorization can be done in polynomialtime

Multivariate Polynomials

We are now back to p olynomials with n variables To make n our only input size parameter it is

convenient to restrict ourselves to p olynomials whose total degree is at most n

Once again almost every p olynomial p in n variables requires size S p expn via a

dimension argument and we seek explicit p olynomial families that are hard Unlike in the

Bo olean world here there are slightly nontrivial lower b ounds via elementary to ols from algebraic

geometry

n n n

n log n x x Theorem S x

The same techniques extend to prove a similar lowerbound for other natural p olynomials such

as the symmetric p olynomials and the determinant Establishing a stronger lowerbound for any

explicit p olynomial is a ma jor op en problem Another is obtaining a sup erlinear lower b ound for

a p olynomial map of constant even total degree Outstanding candidates for the latter are the

linear maps computing the Discrete Fourier Transform over the Complex numbers or the Walsh

transform over the Rationals for b oth O n log n algorithms are known but no sup erlinear lower

b ounds

We now fo cus on sp ecic p olynomials of central imp ortance The most natural and well studied

candidate for the last op en problem is the matrix multiplication function MM let A B b e two

m m matrices of variables over F and dene MMA B to b e the n m entries of the matrix

A B Thus MM is a set of n explicit bilinear forms over the n input variables It is known that

S MM n On the other hand the obvious m n algorithm can b e improved

Theorem For every eld F S MM O n

So what is the complexity of MM even if one counts only multiplication gates Is it linear or

almostlinear or is it the case that S MM n for some This is indeed a famous op en

problem

We next consider the determinant and p ermanent p olynomials DET and PER resp over the

n m variables representing an m m matrix While DET plays a ma jor role in classical mathemat

ics PER is somewhat esoteric though it app ears in Statistical Mechanics and Quantum Mechanics

In the context of complexity theory b oth p olynomials are of great imp ortance b ecause they capture

natural complexity classes DET has relatively low complexity and is closely related to the class of

p olynomials having p olynomialsized arithmetic formulae whereas PER seems to have high com

plexity and it is complete for the counting class P which contains NP Thus it is conjectured

that PER is not p olynomialtime reducible to DET A sp ecic type of reduction that makes sense in

this algebraic context is by pro jection

Denition Let X and Y be two disjoint nite sets of variables Let p F X and q F Y be

two polynomials We say that there is a projection from p to q over F denoted p q if there exists

a function h X Y F such that px q hx

Clearly if p q then S p S q Let DET and PER denote these functions restricted to

F F m m

mbym matrices It is known that PER DET but to yield a p olynomialtime reduction one

would need a pro jection of PER to DET It is conjectured that no such pro jection exists

p oly m

Op en Problem Is PER DET

O (1)

Pro of Complexity

The concept of proof is what distinguishes the study of Mathematics from all other elds of human

inquiry Mathematicians have gathered millennia of exp erience to attribute such adjectives to pro ofs

as insightful original deep and most notably dicult Can one quantify mathematically the

diculty of proving various theorems This is exactly the task undertaken in Pro of Complexity It

seeks to classify theorems according to the diculty of proving them much like Circuit Complexity

seeks to classify functions according to the diculty of computing them In pro ofs just like

in computation there will b e a number of mo dels called proof systems capturing the p ower of

reasoning allowed to the prover

We will consider only prop ositional pro of systems and so our theorems will b e tautologies We

will see so on why the complexity of proving tautologies is highly nontrivial and amply motivated

The formal denition of a pro of system sp ells out what we take for granted the eciency of

the verication pro cedure

Denition A propositional proof system is a polynomialtime Turing machine M with the

property that T is a tautology if and only if there exist a pro of such that M T

Here eciency of the verication pro cedure refers to its runningtime measured in terms of the total length of

the al leged theorem and proof In contrast in Sections and we consider the runningtime as a function of the

length of the al leged theorem

In agreement with standard formalisms see b elow the pro of is seen as coming b efore the theorem

Note that the denition guarantees completeness and soundness as well as verication eciency

of the pro of system It judiciously ignores the size of the pro of of the tautology T which is

a measure of how complex it is to prove T in the system M For each tautology T let s T

denote the size of the shortest pro of of T in M ie the length of the shortest string such that M

accepts T Abusing notation we let s n denotes the maximum s T over all tautologies

M M

T of length n

The following simple observation provides a basic connection of this concept with computational

complexity and the ma jor question of Section

Theorem There exists a proof system M such that s is polynomial if and only if NP coNP

It is natural to start attacking this formidable problem by considering rst simple and thus

weaker pro of systems and then move on to more and more complex ones Moreover natural

pro of systems capturing basic restricted types and primitives of reasoning as well as natural

tautologies suggest themselves as ob jects for this study In the rest of this section we fo cus on such

restricted pro of systems

Dierent branches of Mathematics such as logic algebra and geometry provide dierent such

systems often implicitly A typical system would have a set of axioms and a set of deduction

rules A pro of would pro ceed to derive the desired tautology in a sequence of steps each pro ducing

a formula often called a line of the pro of which is either an axiom or follows from previous

formulae via one of the deduction rules Clearly a Turing machine can easily verify the validity

of such a pro of

This p erfectly t our DAG mo del The inputs will b e lab eled by the axioms the internal

vertices by deduction rules which in turn infer a formula for that vertex from the formulae at

the vertices p ointing to it

There is an equivalent and somewhat more convenient view of simple pro of systems namely

as simple refutation systems First recalling that SAT is NPcomplete see Footnote note

that every negation of a tautology can b e written as a conjunction of clauses with each clause

b eing a disjunction of only literals variables or their negation Now if we take these clauses as

axioms and derive using the rules of the system a contradiction eg the negation of an axiom

or b etter yet the empty clause then we have proved the tautology since we have proved that

its negation yields a contradiction We will use the refutation viewp oint throughout and often

exchange tautology and its negation contradiction

So we turn to study the pro of length s T of tautologies T in pro of systems The rst

observation revealing a ma jor dierence b etween pro of complexity and circuit complexity is that

the trivial counting argument fails The reason is that while the number of functions on n bits is

there are at most tautologies of this length Thus in pro of complexity even the existence

of a hard tautology not necessarily explicit would b e of interest As we shall see however most

known lower b ounds in restricted pro of systems apply to very natural tautologies

The rest of this section is divided to three parts on logical algebraic and geometric pro of

systems We will briey describ e imp ortant representatives and basic results in each

Logical Pro of Systems

The pro of systems in this section will all have lines that are Bo olean formulae and the dierences

will b e in the structural limits imp osed on these formulae

General pro of systems as in Denition can also b e adapted to this formalism by considering a deduction rule

that corresp onds to a single step of the machine M However the deduction rules considered b elow are even simpler

and more imp ortantly they are natural

The most basic pro of system called Frege system puts no restriction on the formulae manipu

lated by the pro of It has one derivation rule called the cut rule A C B C A B adding

any other sound rule like modus ponens has little eect on the length of pro ofs in this system

Frege systems are basic in the sense that they in several variants are the most common in Logic

and in that p olynomial length pro ofs in these systems naturally corresp onds to p olynomialtime

reasoning ab out feasible ob jects

The ma jor op en problem in pro of complexity is to nd any tautology as usual we mean a

family of tautologies that has no p olynomialsize pro of in the Frege system

As lower b ounds for Frege are hard we turn to subsystems of Frege which are interesting and

natural The most widely studied system is Resolution whose imp ortance stems from its use by

most prop ositional as well as rst order automated theorem provers The formulae allowed in

Resolution refutations are simply clauses disjunctions and so the derivation cut rule simplies to

the resolution rule A x B x A B for clauses A B and variable x

An example of a tautology that is easy for Frege and hard for Resolution is the pigeonhole

expressing the fact that there is no onetoone mapping of m pigeons to n m principle PHP

holes

n O n n

Theorem s PHP n but s PHP

Frege

Resolution

n n

Algebraic Pro of Systems

Just as a natural contradiction in the Bo olean setting is an unsatisable collection of clauses a

natural contradiction in the algebraic setting is a system of p olynomials without a common ro ot

Moreover CNF formulae can b e easily converted to a system of p olynomials one p er clause over

x which ensure Bo olean values any eld One often adds the p olynomials x

A natural pro of system related to Hilb erts Nullstellensatz and to computations of Grobner

bases in symbolic algebra programs is Polynomial Calculus abbreviated PC The lines in this

system are p olynomials represented explicitly by all co ecients and it has two deduction rules

For any two p olynomials g h the rule g h g h and for any p olynomial g and variable x the

rule g x x g Strong size lower b ounds obtained from degree lower b ounds are known for this

i i

system For example enco ding the pigeonhole principle as a contradicting set of constant degree

p olynomials we have

n m

over every eld Theorem For every n and every m n s PHP

Geometric Pro of Systems

Yet another natural way to represent contradictions is a by a set of regions in space that have

empty intersection Again we care mainly ab out discrete say Bo olean domains and a wide

source of interesting contradictions are Integer Programs from Combinatorial Optimization Here

the constraints are ane linear inequalities with integer co ecients so the regions are subsets

of the Bo olean cub e carved out by halfspaces The most basic system is called Cutting Planes

CP Its lines are linear inequalities with integer co ecients Its deduction rules are the obvious

addition of inequalities and the less obvious dividing the co ecients by a constant and rounding

taking advantage of the integrality of the solution space

is easy in this system exp onential lower b ounds are known for other tautologies While PHP

We mention that they are obtained from the monotone circuit lower b ounds of Section

Randomized Computation

As hinted in Section until now we restricted computations to rep eatedly executing a deter

ministic rule A more lib eral approach pursued in this section considers computing devices that

use a probabilistic or randomized rule We still fo cus on p olynomialtime computations but

these are probabilistic ie can toss coins Sp ecically we allow probabilistic rules that choose

uniformly among two outcomes We comment that probabilistic computations are b elieved to take

place in reallife algorithms that are employed in a variety of applications eg random sampling

MonteCarlo simulations etc

Rigorous mo dels of probabilistic machines are dened by natural extensions of the basic mo del

yielding probabilistic Turing machines For a probabilistic machine M and string x f g we

denote by M x the distribution of the output of M when invoked on input x where the probability

is taken over the machines random moves Considering decision problems we want this distribution

to yield the correct answer with high probability for every input This leads to the denition of

BPP for Bounded error Probabilistic Polynomial time

Denition BPP A Boolean function f is in BPP if there exists a probabilistic polynomial

time machine M such that for every x f g PrM x f x

The error b ound is arbitrary for any k p oly jxj the error can b e reduced to by invoking

the program O k times and taking a ma jority vote of the answers We stress that the random

moves in the dierent invocations are indep endent

Again it is trivial that BPP EXP via enumerating all p ossible outcomes of coin tosses

and taking a ma jority vote The relation of BPP to NP is not known but it is known that if

NP P then also BPP P Finally nonuniformity can replace randomness every function in

BPP has p olynomialsize circuits But the fundamental question is whether or not randomization

adds computing p ower over determinism for decision problems

Op en Problem Does P BPP

While quite a few problems are known to b e in BPP but not known to b e in P there is

overwhelming evidence that the answer to the question ab ove is p ositive namely randomization

do es not add extra p ower in the context of decision problems we elab orate a bit in Section

Counting at Random

One imp ortant question regarding NP search problems is that of determining how many solutions

a sp ecic instance has This captures a host of interesting problems from various disciplines eg

counting the number of solutions to a system of multivariate p olynomials counting the number of

p erfect matchings of a graph computing the volume of a p olytop e given by linear inequalities in

high dimension computing various parameters of physical systems etc

In most of these problems even approximate counting would suce Clearly approximate

counting allows one to determine whether a solution exists at all For example counting the

number of satisfying assignments for a given prop ositional formula even approximately allows

one to determine whether the formula is satisable Interestingly the converse is also true

The sense in which these applications actually utilize random moves is a dierent question The p oint is that

one analyzes these computations as though they are taking random moves

A central example is Identity Testing given an arithmetic circuit over Q decide if it computes the identically

zero p olynomial

Theorem There exists a probabilistic polynomialtime oracle machine that on input a for

is within a factor mula and oracle access to SAT outputs an integer that with probability at least

of of the number of satisfying assignments for

We comment that an analogous statement holds for any NPcomplete problem

The approximation factor can b e reduced to j j for any xed constant c However it

is b elieved that an exact count cannot b e obtained via a probabilistic p olynomialtime oracle with

oracle access to SAT We mention that computing the aforementioned quantity or computing the

number of solutions to any NPsearch problem is p olynomialtime reducible to computing the

p ermanent of p ositive integer matrices

For some of the problems mentioned ab ove approximate counting can b e done without the SAT

oracle There are p olynomialtime probabilistic algorithms for approximating the p ermanent of

p ositive matrices approximating the volume of p olytop es and more These follow a connection

of approximate counting to the related problem of uniform generation of solutions and the con

struction and analysis of adequate Markov chains for solving the related sampling problems see

Chap

Probabilistic Pro of Systems

The glory attributed to the creativity involved in nding pro ofs makes us forget that it is the less

gloried pro cess of verication that denes pro of systems

The notion of a verication pro cedure presupp oses the notion of computation and furthermore

the notion of ecient computation b ecause verication unlike coming up with pro ofs is supp osed

to b e easy It will b e convenient here to view a pro of system for a set S eg of satisable

formulae as a game b etween an allp owerful prover and an ecient verier Both receive an input

x and the prover attempts to convince the verier that x S Completeness dictates that the

prover succeeds for every x S and soundness dictates that any prover fails for every x S

When taking the most natural choice of eciency requirement namely restricting the verier

to b e a deterministic p olynomialtime machine we get back the denition of the class NP slightly

rephrased a set S is in NP if and only if membership in S can be veried by a deterministic

polynomialtime machine when given an al leged proof of polynomial length ie p olynomial in jxj

Now we relax the eciency requirement and let the verier b e a probabilistic p olynomialtime

machine allowing it to rule by statistical evidence and hence to err with low probability which

is explicitly b ounded and can b e reduced via rep etitions This relaxation is not suggested as a

substitute to the notion Mathematical truth however as we shall b elow it turns out to yield

enormous advance in computer science

Interactive Pro of Systems

When the verier is deterministic we can always assume that the prover simply sends it a single

message the purp orted pro of and based on this message the verier decides whether to accept

or reject the common input x as a member of the target set S

When the verier is probabilistic interaction may add p ower We thus consider a randomized

interaction b etween the parties which may b e viewed as an interrogation by a p ersistent student

12 0 0

See Footnote Here up on issuing any query the machine is told whether or not is satisable

We stress that this reduction do es not preserve the quality of an approximation

This may explain the historical fact that notions of computation were rst rigorously formulated in the context

of logic

asking the teacher tough questions in order to b e convinced of correctness Since the verier

ought to b e ecient ie run in time p olynomial in jxj this interaction is b ounded to have at

most these many rounds The class IP for Interactive Pro ofs contains all sets S for which there

is a verier that accepts every x S with probability after interacting with an adequate prover

but rejects any x S with probability at least no matter what strategy is employed by the

prover

A ma jor result asserts that interactive pro ofs exists for every set in P S P AC E ie having a

decision pro cedure that uses a p olynomial amount of memory but p ossibly working in exp onential

time

Theorem IP P S P AC E

While it is not known if NP P S P AC E it is widely b elieved to b e the case and so it seems

that interactive pro ofs are more p owerful than standard noninteractive and deterministic pro ofs

ie NPpro ofs In particular since co NP P S P AC E Theorem implies that there are such

interactive pro ofs for every set in co NP eg the set of prop ositional tautologies whereas some

coNPsets are b elieved not to have NPpro ofs

ZeroKnowledge Pro of Systems

Here the thrust is not to prove more theorems but rather to have pro ofs with additional prop erties

Randomized and interactive verication pro cedures as in Section allow the meaningful in

tro duction of zeroknowledge proofs which are pro ofs that yield nothing b eyond their own validity

Such pro ofs seem counterintuitive and undesirable for educational purp oses but they are very

useful in cryptography

For example a zeroknowledge proof that a certain prop ositional formula is satisable do es

not reveal a satisfying assignment to the formula nor any partial information regarding such an

assignment eg whether the rst variable can assume the value true In general whatever the

verier can eciently compute after interacting with a zeroknowledge prover can b e eciently

reconstructed from the assertion itself without interacting with anyone

Clearly any set in BPP has a zeroknowledge pro of in which the prover says nothing and the

verier decides by itself What is surprising is that zeroknowledge pro ofs seem to exist also for

sets that are not in BPP In particular

Theorem Assuming the existence of oneway functions see Section every set in NP has

a zeroknowledge proof system

Probabilistically Checkable Pro of systems

Let us return to the noninteractive mo de in which the verier receives a alleged written pro of

But now we restrict its access to the pro of so as to read only a small part of it which may b e

randomly selected An excellent analogy is to imagine a referee trying to decide the correctness of

a long pro of by sampling a few lines of the pro of It seems hop eless to detect a single bug unless

the entire pro of is read but this intuition is valid only for the natural way of writing down

pro ofs and fails when robust formats of pro ofs are used and one is willing to settle for statistical

evidence

Interestingly it turns out that asking tough questions is not b etter than asking random questions

Such robust pro of systems are called PCPs for Probabilistically Checkable Pro ofs Lo osely

sp eaking a pcp system for a set S consists of a probabilistic p olynomialtime verier having access

to an oracle that represents a pro of in redundant form where as in case of NPpro ofs the length

of the pro of is p olynomial in the length of the input The verier accesses only a constant number

of the oracle bits and accepts every x S with probability when given access to an adequate

oracle but rejects any x S with probability at least no matter to which oracle it is given

access

Theorem The PCP Theorem Each set in NP has a pcp system Furthermore there

exists a polynomialtime procedure for converting any NPproof to the corresponding pcporacle

Indeed the pro of of the PCP Theorem suggests a new way of writing robust pro ofs in which any

bug must spread all over One imp ortant application of the PCP Theorem and its variants is

the connection to the complexity of combinatorial approximation For example it is NPcomplete

to decide if for a given linear system of equations over GF the fraction of mutually satisable

equations is greater than or smaller than

Weak Random Sources

We now return to the question of how to obtain the assumed randomness for all the probabilistic

computations discussed in this section Although randomness seems to b e present in the world eg

the p erceived randomness in the weather Geiger counters Zener dio des real coin ips etc it do es

not seem to b e in the p erfect form of unbiased and indep endent coin tosses as p ostulated ab ove

Thus to actually use randomized pro cedures we need to convert weak sources of randomness

into almost p erfect ones Very general mathematical mo dels capturing such weak sources have

b een prop osed Algorithms converting the randomness in them into a distribution that in close

to uniform namely unbiased indep endent stream of bits are called randomness extractors and

near optimal ones have b een constructed This large b o dy of work is surveyed eg in We

mention that this question turned out to b e related to certain types of pseudorandom generators

cf Section as well as to combinatorics and co ding theory

The Bright Side of Hardness

The Big Conjecture according to which P NP means that there are computational problems of

great interest that are inherently intractable This is bad news but there is a bright side to the

matter computational hardness alas in a stronger form than known to follow from P NP has

many fascinating conceptual consequences as well as imp ortant practical applications Sp ecically

in accordance with our intuition we shall assume that not all ecient pro cesses can b e eciently

reversed or inverted Furthermore we shall assume that hardness to invert is a typical rather

than pathological phenomenon for some eciently computable functions That is we assume that

oneway functions exist

Denition OneWay Functions A function f f g f g is called oneway if the

fol lowing two conditions hold

easy to compute the function f is computable in polynomial time

The analogy to error correcting co des is indeed in place and the cross fertilization b etween these two areas has

b een very signicant

hard to invert for every probabilistic polynomialtime machine M every positive polynomial

p and al l suciently large n

h i

M f x f f x Pr

where x is uniformly distributed in f g

For example the widely b elieved conjecture according to which integer factorization is intractable

for a noticeable fraction of the instances implies the existence of oneway functions On the other

hand if P NP then no oneway functions exist One important open problem is whether P NP

implies the existence of oneway functions

Below we discuss the connection b etween computational diculty in the form of oneway

functions on the one hand and two imp ortant computational theories on the other hand the

theory of Pseudorandomness and the theory of Cryptography

One fundamental concept which is pivotal to b oth these theories is the concept of computational

indistinguishability Lo osely sp eaking two ob jects are said to b e computationally indistinguishable

if no ecient pro cedure can tell them apart Here ob jects will b e probability distributions on

nite binary sequences We actually consider probability ensembles where an ensemble is a family

of distributions each on strings of dierent length eg the uniform ensemble is the family fU g

where U is the uniform distribution on all nbit strings

Denition Computational Indistinguishability The probability ensembles fP g and

fQ g are called computationally indistinguishable if for every probabilistic polynomialtime ma

chine M every positive polynomial p and al l suciently large n

n n

jPrM P PrM Q j

n n

Computational indistinguishability is a strict coarsening of statistical indistinguishability We

fo cus on the nontrivial cases of pairs of ensembles that are computationally indistinguishable

although they are statistically very dierent It is easy to show that such pairs do exist but we

further fo cus on pairs of such ensembles that are eciently samplable Interestingly such pairs

exists if and only if oneway functions exist

Pseudorandomness

We call an ensemble pseudorandom if it is computationally indistinguishable from the random

ie uniform ensemble A pseudorandom generator is an ecient deterministic pro cedure that

stretches short random strings into longer pseudorandom strings

Denition Pseudorandom Generators A deterministic polynomialtime machine G is

called a pseudorandom generator if there exists a monotonically increasing function N N such

that the probability ensembles fU g and fGU g are computationally indistinguishable

nN nN

The function is called the stretch measure of the generator and the nbit input of the generator

is called its seed

The ensemble fP g is eciently samplable if there exists a probabilistic p olynomialtime machine M such

n2N

that M and P are identically distributed for every n

18 m

Recall that U denotes the uniform distribution over f g Thus GU is dened as the output of G on a

m n

uniformly selected nbit input string

That is pseudorandom generators yield a particularly interesting case of computational indistin

guishability the distribution GU which is eciently samplable using only n truly random coins

and so has entropy n is computationally indistinguishable from the uniform distribution over

nbit long strings having entropy n n The ma jor question which we turn to deal with is

of course do pseudorandom generators exist

Hardness versus Randomness

By its very denition the notion of a pseudorandom generator is connected to computational

diculty ie the computational diculty of determining that the generators output is not truly

random It turns out that the connection b etween the two notions is even stronger

Theorem Pseudorandom generators exist if and only if oneway functions exist Furthermore

if pseudorandom generators exist then they exist for any stretch measure that is a polynomial

Theorem converts computational diculty hardness into pseudorandomness and vice versa

Furthermore its pro of links computational indistinguishability to computational unpredictability

hinting that computational diculty of predicting an information theoretically determined event

is linked to randomness or to the app earance of b eing random

Pseudorandom generators allow for a drastic reduction in the amount of true randomness used

in any ecient randomized pro cedure Rather than using indep endent coin tosses such pro cedures

can use the output of a pseudorandom generator which in turn can b e generated deterministically

based on many fewer coin tosses used to select the generators seed The eect of this replacement

on the b ehavior of such pro cedures will b e negligible In algorithmic applications where it is

p ossible to invoke the same pro cedure many times and rule by a ma jority vote one can derive

deterministic pro cedures by trying all p ossible seeds In particular using a seemingly stronger

notion of pseudorandom generators which work in time exp onential in their seeds and pro duce

sequences that lo ok random to tests of a xed p olynomialtime complexity allows to convert

any probabilistic p olynomialtime algorithm into a deterministic one implying that BPP P

Such pseudorandom generators exist under plausible conjectures regarding computational diculty

which seem far weaker than the existence of oneway functions Thus for example

Theorem If for some constant S SAT then BPP P Moreover SAT can be

O n

replaced by any problem computable in time

Pseudorandom Functions

Pseudorandom generators allow for the ecient generation of long pseudorandom sequences from

short random seeds Pseudorandom functions are even more p owerful they allow for ecient

direct access to a huge pseudorandom sequence which is infeasible to scan bitbybit That is

pseudorandom functions are eciently computable ensembles of functions that are indistinguish

able from truly random functions by any ecient pro cedure that can obtain the function values at

arguments of its choice We refrain from presenting a precise denition but do mention a central

result pseudorandom functions can be constructed given any pseudorandom generator We also

mention that pseudorandom functions have many applications most notably in cryptography

Cryptography

Cryptography has existed for millennia However in the past it was fo cused on one basic problem

that of providing secret communications By contrast the mo dern computational theory of cryp

tography is interested in al l tasks involving several communicating agents in which the following

often conicting desires are crucial privacy namely the protection of secrecy and resilience

namely the ability to withstand malicious b ehavior of participants Perhaps the b est example to

illustrate these diculties is playing a game of Poker over the telephone ie the new age players

cannot rely on physical implements such as cards dealt from a deck that is visible by all play

ers In general cryptography is concerned with the construction of schemes that maintain any

desired functionality under malicious attempts aimed at making these schemes deviate from their

prescrib ed functionality

As with pseudorandomness there are two key assumptions underlying the new theory First

that all parties including the adversary are computationally limited they are mo deled as prob

abilistic p olynomialtime machines and hence computationally indistinguishable distributions are

equivalent as far as these parties are concerned Second that a certain type of computationally

hard problem exists namely oneway functions and in some cases stronger versions called trapdoor

permutations which in turn are implied by the hardness of integer factorization In fact all the

results mentioned b elow hold if trap do or p ermutations exist and cannot hold if oneway functions

do not exist

Starting with the traditional problem of providing secret communication over insecure channels

we note that pseudorandom functions which can b e constructed based on any oneway function

provide a satisfactory solution for this problem The communicating parties sharing a pseudoran

dom function may exchange information in secrecy by masking it with the values of the function

evaluated at adequately selected arguments which may b e agreedup on a priori or transmitted in

the clear That is the parties use a pseudorandom function as a secret key in predetermined

encryption and decryption pro cedures Still the communicating parties have to agree on this key

b eforehand or transmit this key through an auxiliary secret channel

The need for a priori agreement on a secret key is removed when using publickey encryption

schemes in which the key used for encryption can b e made public while only the dierent key used

for decryption is kept secret In particular in such schemes it is infeasible to recover the decryption

key from the encyptionkey although such random pairs of keys can b e generated eciently Secure

publickey encryption schemes ie providing for secret communication without any prior secret

agreement can b e constructed based on trap do or p ermutations

A general framework for casting cryptographic problems consists of sp ecifying a random pro cess

which maps m inputs to m outputs The inputs to the pro cess are to b e thought of as lo cal inputs of

m parties and the m outputs are their corresp onding lo cal outputs The random pro cess describ es

the desired functionality That is if the m parties were to trust each other or trust some outside

party then they could each send their lo cal input to the trusted party who would compute the

outcome of the pro cess and send each party the corresp onding output Lo osely sp eaking a secure

implementation of such a functionality is an mparty proto col in which the impact of malicious

parties is eectively restricted to application of the prescrib ed functionality to inputs chosen by the

corresp onding parties One ma jor result in this area is the following

Theorem Assuming the existence of trapdoor permutations any eciently computed function

ality can be securely implemented

The Tip of an Iceb erg

Even within the topics discussed ab ove many imp ortant notions and results have not b een dis

cussed for space reasons Furthermore other imp ortant topics and even wide areas have not b een

mentioned at all Here we briey discuss some of these topics and areas

Relaxing the Requirements

The P vs NP Question as well as most of the discussion so far fo cuses on a simplied view of the

goals of ecient computations Sp ecically we have insisted on ecient pro cedures that always

give the exact answer In practice one may b e content with ecient pro cedures that typically

give an approximate answer Indeed b oth terms in quotation marks require clarication

AverageCase Complexity

One may consider pro cedures that answer correctly on a large fraction of the instances But this

assumes that all instances are equally interesting in practice which is typically not the case On

the other hand demanding success under all input distributions gives back worstcase complexity

A very app ealing theory of averagecase complexity cf demands success only for the family of

all input distributions that can b e eciently sampled

Approximation

What do we mean by an approximation to a computational problem There are many p ossible an

swers and their signicance dep ends on the sp ecics of the application For optimization problems

the answer is obvious wed like to get close to the optimum see For search problems we

may b e satised with a solution that is close in some metric to b eing valid For decision problems

ie determining set membership we may ask how close the input is under some relevant distance

measure to an instance in the set cf

Other Complexity Measures

Until now we have fo cused on the running time of pro cedures which is arguably the most imp or

tant complexity measure However other complexity measures such as the amount of workspace

consumed during the computation are also imp ortant cf Another imp ortant issue is the

extent to which a computation can b e p erformed in parallel that is sp eedingup the computation

by splitting the work among several computing devices which are viewed as comp onents of the

same parallel machine and they are provided with direct access to the same memory mo dule In

addition to the parallel time a fundamentally imp ortant complexity measure in such a case is the

number of parallel computing devices used cf

Other Notions of Computation

Following are a few of the computational mo dels we did not discuss Mo dels of distributed comput

ing refer to distant computing devices each given a lo cal input which may b e viewed as a part of

a global input In typical studies one wishes to minimize the amount of communication b etween

these devices and certainly avoid the communication of the entire input In addition to measures

of communication complexity a central issue is asynchrony cf We note that the communica

tion complexity of twoargument and manyargument functions is studied as a measure of their

complexity cf but in these studies communication prop ortional to the length of the input

is not ruled out but rather app ears frequently While b eing information theoretic in nature

this mo del has many connections to complexity theory Altogether dierent types of computational

problems are investigated in the context of computational learning theory cf and the study

of online cf Finally Quantum Computation investigates the p ossibility of using quantum

mechanics to sp eed up computation cf

Concluding Remarks

We hop e that this ultrabrief survey conveys the fascinating avor of the concepts results and op en

problems that dominate the eld of computational complexity One imp ortant feature of the eld

we did not do justice to is the remarkable web of often surprising connections b etween dierent

subareas and its impact on progress For further details on the material discussed in Sections

the reader is referred to standard textb o oks such as For further details on the material

discussed in Sections and the reader is referred to and resp ectively For

further details on the material discussed in Sections and the reader is referred to and also

to for further details on Section

References

H Attiya and J Welch Distributed Computing Fundamentals Simulations and Advanced Topics

McGrawHill

A Boro din and R ElYaniv Online Computation and Comp etitive Analysis Cambridge Uni

versity Press

P Beame and T Pitassi Propositional Proof Complexity Past Present and Future in

Bulletin of the EATCS Vol June

R Boppana and M Sipser The complexity of nite functions in

MR Garey and DS Johnson Computers and Intractability A Guide to the Theory of NP

Completeness WH Freeman and Company

O Goldreich Notes on Levins Theory of AverageCase Complexity In ECCC TR

O Goldreich Mo dern Cryptography Probabilistic Pro ofs and Pseudorandomness Algorithms

and Combinatorics series Vol Springer

O Goldreich Foundation of Cryptography in two volumes Basic Tools and Basic Applications

Cambridge University Press and

D Ho chbaum ed Approximation Algorithms for NPHard Problems PWS

RM Karp and V Ramachandran Parallel Algorithms for SharedMemory Machines in

MJ Kearns and UV Vazirani An intro duction to Computational Learning Theory MIT Press

A Kitaev A Shen M Vyalyi Classical and Quantum Computation AMS

E Kushilevitz and N Nisan Communication Complexity Cambridge University Press

J van Leeuwen ed Handb o ok of Theoretical Computer Science Vol A Algorithms and Com

plexity MIT PressElsevier

D Ron Property Testing A Tutorial in Handb o ok on Randomized Computing Volume I I

Kluwer Academic Publishers

R Shaltiel Recent Developments in Explicit Constructions of Extractors in Bulletin of the

EATCS Vol

M Sipser Intro duction to the Theory of Computation PWS

V Strassen Algebraic Complexity Theory in

App endix Glossary of Complexity Classes

Complexity classes are sets of computational problems where each class contains problems that

can b e solved with sp ecic computational resources Examples of such classes eg P and NP

are presented in the essay Computational Complexity Sec IV and the reader is referred there

for further discussion of the notions of computation and complexity

To dene a complexity class one sp ecies a mo del of computation a complexity measure like

time or space and a b ound on it The prevailing mo del of computation is that of Turing machines

which in turn capture the notion of uniform algorithms Another imp ortant mo del is the one of

nonuniform circuits The term uniformity refers to whether the algorithm is the same one for

every input length or whether a dierent algorithm or rather a circuit is considered for each

input length Recall from Sec IV that complexity is always measured as a function of the input

length

We fo cus on natural complexity classes obtained by considering natural complexity measures

and b ounds which contain natural computational problems Furthermore almost all of these

classes can b e characterized by natural problems which capture every problem in the class

Such problems are called complete for the class which means that they are in the class and every

problem in the class can b e easily reduced to them where easily means that the reduction takes

less resources than what each of the problems seems to require individually We stress the fact that

complete problem not only exist but rather are natural and make no reference to computational

mo dels or resources Ecient algorithm for a complete problem implies an algorithm of similar

eciency for al l problems in the class

A Algorithmbased classes

The two main complexity measures considered in the context of uniform algorithms are the num

b er of steps taken by the algorithm ie its time complexity and the amount of memory or

workspace consumed by the computation ie its space complexity In our Sec IV essay we de

ne the timecomplexity classes P and NP cf Sec co NP cf Sec and BPP cf Sec

In addition we mention a couple of other classes asso ciated with probabilistic p olynomialtime

The set S is in RP if there exists a probabilistic p olynomialtime machine M such that x S

while x S implies PrM x Also co RP ff g n S implies PrM x

S RP g The latter class contains the problem of deciding whether a given arithmetic circuit

over Q computes the identically zero p olynomial

The decision problem S f g f g is in ZPP if there exists a probabilistic p olynomial

time machine M such that for every x it holds that M x fS x g and PrM x S x

where s a sp ecial failure symbol Equivalently ZPP is the class of all sets which have

a probabilistic algorithm which always returns the correct andswer and runs in expected

p olynomial time

Clearly ZPP RP co RP RP NP BPP

When dening spacecomplexity classes one counts only the space consumed by the actual

computation and not the space o ccupied by the input and output This is formalized by p ostulating

that the input is read from a readonly device resp the output is written on a writeonly device

Four imp ortant classes of decision problems are

The class L consists of problems solvable in logarithmic space That is a set S is in L if there

exists a standard ie deterministic algorithm of logarithmic spacecomplexity for deciding

membership in S This class contains some simple computational problems eg matrix

multiplication and arguably captures the most spaceecient computations

The class RL consists of problems solvable by a randomized algorithm of logarithmic space

complexity This class contains the problem of deciding whether a given undirected graph is

connected This problem is not known to b e in L

The class NL is the nondeterministic analogue of L and is traditionally dened in terms

of nondeterministic machines of logarithmic spacecomplexity Alternatively analogously to

the denition of NP a set S is in NL if there exists a p olynomially b ounded binary relation

R L such that x S if and only if there exists y such that x y R The class NL

S S

contains the problem of deciding whether there exists a directed path b etween two given

vertexes in a given directed graph In fact the latter problem is complete for the class under

def

logarithmicspace reductions Interestingly co NL ff g n S S N Lg equals NL

The class P S P AC E consists of decision problems solvable in p olynomial space This class

contains very dicult problems including the computation of winning strategies for any

ecient party games as discussed b elow

Clearly L RL NL P and NP P S P AC E

Turning back to timecomplexity we mention the classes E and EXP corresp onding to problems

O n p oly n

that can b e solved by a deterministic algorithm in time and resp ectively for nbit

long inputs Clearly P S P AC E EXP

Two classes related to the class NP are the counting class P and the Polynomialtime

hierarchy Functions in P count the number of solutions to an NPtype search problem eg

compute the number of satisfying assignments of a given formula Formally a function f is in P

if there exists an NPtype relation R such that f x jfy x y R gj Clearly P problems

are solvable in p olynomial space Surprisingly the p ermanent of p ositive integer matrices is P

complete ie it is in P and any function in P is p olynomialtime reducible to it

The Polynomialtime hierarchy PH consists of sets S such that there exists a constant k and

a k ary p olynomially b ounded relation R P such that x S if and only if y y y y

such that x y y y y y R Indeed NP corresp onds to the sp ecial case where k

Interestingly PH is p olynomialtime reducible to P

Odeds Note

Sets in the Polynomialtime hierarchy and in the class P S P AC E capture the complexity of

I prefer to

nding winning strategies in certain ecient party game In such games the two players compute

omit the

their next move from any given p osition in p olynomial time in terms of the initial p osition and

paragraph

a winning p osition can b e recognized in p olynomialtime For example a set S as ab ove can b e

viewed via a k move game in which starting from a given p osition x the rst party takes the rst

move y the second resp onds with y etc and the winner is determined by whether or not the

transcript x y y of the game is in R That is x S if starting at the initial p osition x the

rst party has a winning strategy in the k move game determined by R Thus sets in PH resp

P S P AC E corresp onds to games with a constant number of resp p olynomailly many moves

A Circuitbased classes

See Sec IV for discussion of circuits as computing devices The two main complexity measures

considered in the context of nonuniform circuits are the number of gates or wires in the circuit

ie the circuits size and the length of the longest directed path from an input to an output ie

the circuits depth

The main motivation for the introduction of complexity classes based on circuits is the devel

opment of lowerbounds For example the class of problems solvable by p olynomialsize circuits

denoted P p oly is a sup erset of P b ecause it clearly contains P as well as any subset of fg

whereas there exists such sets that represents decision problems that are not solvable ie by any

uniform algorithm Thus showing that NP is not contained in P p oly would imply P NP

For further discussion see Sec IV

The class AC discussed in our Sec IV article cf Sec consists of sets recognized by

constantdepth p olynomialsize circuits of unbounded fanin The analogue class that allows also

unbounded fanin ma joritygates or equivalently thresholdgates is denoted T C For any non

negative integer k the class of sets recognized by p olynomialsize circuits of bounded fanin resp

unbounded fanin having depth O log n where n is the input length is denoted NC resp

def

k k k k k

AC Clearly NC AC NC and NC NC

k N

We mention that the class NC NL is the habitat of most natural computational problems

of Linear Algebra solving a linear system of equations as well as computing the rank inverse and

determinant of a matrix The class NC contains all symmetric functions regular languages as

well as word problems for nite groups and monoids The class AC contains all prop erties of nite

ob jects expressible by rstorder logic