On Teaching the Basics of Complexity Theory



In Memory of Shimon Even

Oded Goldreich

Department of Computer Science and Applied Mathematics

Weizmann Institute of Science Israel

Email odedgoldreichweizmann aci l

August

Abstract

We outline a conceptual framework for teaching the basic notions and results of complexity

theory Our fo cus is on using denitions and on organizing the presentation in a way that

reects the fundamental nature of the material We do not attempt to provide a selfcontained

presentation of the material itself but rather outline our suggestions regarding how this material

should b e presented in class In addition we express our opinions on numerous related issues

We fo cus on the PvsNP Question the general notion of a reduction and the theory of

NPcompleteness In particular we suggest presenting the PvsNP Question b oth in terms

of search problems and in terms of decision problems where NP is viewed as a class of pro of

systems As for the theory of NPcompleteness we suggest highlighting the mere existence of

NPcomplete sets



This essay was written towards publication in a forthcoming b o ok in memory of Shimon Even

Contents

Introduction

Teaching and current student p erception of Complexity Theory

The source of trouble and eliminating it

Concrete suggestions

A parenthetical comment on computability versus complexity

Organization

P versus NP

The search version nding versus checking

The decision version proving versus verifying

Equivalence of the two formulations

Reductions and Selfreducibility

The general notion of a reduction

Selfreducibility of search problems

NPcompleteness

Denitions

The existence of NPcomplete problems

CSAT SAT and other NPcomplete problems

NP sets that are neither in P nor NPcomplete

Three additional topics

The class coNP and NPcompleteness

Optimal search algorithms for NPrelations

Promise Problems

A brief overview of Complexity Theory

Historical Notes

Introduction

This is a highly opinionated essay that advocates a conceptoriented approach towards teaching

technical material such as the basics of complexity theory In addition to making various sug

gestions I express my opinion on a variety of related issues I do hop e that this essay will stir

discussion and maybe even aect the way some courses are b eing taught

Teaching and current student p erception of Complexity Theory

Shimon Even had a passion for go o d teaching and so writing this essay in his memory seems most

appropriate In my opinion go o d teaching is an art and needless to say Shimon was one of its

top masters It is hard if at all p ossible to cultivate artistic talents but there are certain basic

principles that underly each art form and these can b e discussed

One central asp ect of go o d teaching is putting things in the right p ersp ective that is a p er

sp ective that claries the motivation for the various denitions and results Nothing should b e

easier when it comes to complexity theory It is easy to provide a go o d p ersp ective on the basic

notions and results of complexity theory b ecause these are of fundamental nature and of great in

tuitive app eal Unfortunately often this is not the way this material is taught The annoying and

quite amazing consequences are students that have only a vague understanding of the conceptual

meaning of these fundamental notions and results

The source of trouble and eliminating it

In my opinion it all b oils down to taking the time to explicitly discuss the conceptual meaning

of denitions and results After all the most imp ortant asp ects of a scientic discovery are the

intuitive question that it addresses the reason that it addresses this question the way it phrases the

question the approach that underlies its answer and the ideas that are embedded in the answer

All these have to b e reected in the way the discovery is presented In particular one should use

the right denitions ie those that reect b etter the fundamental nature of the notion b eing

dened and pro ceed in the conceptually right order Two concrete examples follow

Typically NP is dened as the class of languages recognized by nondeterministic p olynomial

time machines Even bright students may have a hard time guring out by themselves why one

should care ab out such a class On the other hand when dening NP as the class of assertions

that have easily veriable pro ofs each student is likely to understand its fundamental nature

Furthermore the message b ecomes even more clear when discussing the search version analogue

Similarly one typically takes the students throughout the detailed pro of of Co oks Theorem

b efore communicating to them the striking message ie that universal problems exist at all

let alone that many natural problems like SAT are universal Furthermore in some cases this

message is not communicated explicitly at all

Concrete suggestions

The rest of this essay provides concrete suggestions for teaching the basics of complexity theory

where by the basics I mean the PvsNP Question and the theory of NPcompleteness This material

is typically taught as part of an undergraduate course on computability and complexity theory and

However exceptions do exists There are teachers and even textb o oks that deviate from the standard practice

b eing bashed here

my suggestions are targeted primarily at computer scientists teaching such a course However I

b elieve that my suggestions are valid regardless of the context in which this material is b eing taught

I assume that the basic material itself is wellknown to the reader Thus my focus is not on

the material itself but rather on how it should be presented in class The two most imp ortant

suggestions were already mentioned ab ove

The teacher should communicate the fundamental nature of the PvsNP Question while

referring to denitions that clearly reect this nature In particular I suggest explicitly

presenting the implication of the PvsNP Question on the complexity of search problems in

addition to presenting the implication to decision problems

The teacher should communicate the striking signicance of the mere existence of NP

complete problems let alone natural ones b efore exhausting the students with complicated

reductions

Additional suggestions include providing a general p ersp ective on the concept of a reduction es

tablishing tight relations b etween the complexity of search and decision problems decoupling the

pro of of NPhardness of SAT by using CircuitSAT as an intermediate problem and mentioning

some additional topics eg NPsets that are neither in P nor NPcomplete rather than a host of

NPcompleteness results

I advocate a mo delindep endent presentation of the questions and results of complexity theory

I claim that most questions and results in complexity theory like all results of computability the

ory hold for any reasonable mo del of computation and can b e presented with minimal reference

to the sp ecics of the mo del In fact in most cases the sp ecic mo del of computation is irrele

vant Typically the presentation needs to refer to the sp ecics of the mo del of computation only

when enco ding the relation b etween consecutive instantaneous congurations of computation see

Section However such an enco ding is p ossible for any reasonable mo del of computation and

this fact should b e stressed

It is also imp ortant to start a course or series of lectures by providing a wide p ersp ective on

its sub ject matter which in this case is complexity theory I would say that complexity theory is a

central eld of Theoretical Computer Science concerned with the study of the intrinsic complexity

of computational tasks where this study tend to aim at generality The eld fo cuses on natural

computational resources most notably time and the eect of limiting these resources on the class

of problems that can b e solved Put in other words complexity theory aims at understanding the

nature of ecient computation I suggest reiterating the wider goals of complexity theory at the

end of the course or series of lectures and illustrating them at that p oint by sketching a few of

the active research directions and the results obtained in them My own suggestion for such a brief

overview is presented in Section

Finally until we reach the day in which every student can b e assumed to have understo o d

the meaning of the PvsNP Question and of NPcompleteness I suggest not to assume such an

understanding when teaching an advanced complexity theory course Instead I suggest starting

such a course with a fast discussion of the PvsNP Question and NPcompleteness making sure

The sp ecics of the reasonable mo del are irrelevant for all questions and results mentioned in this essay except

for Theorem where the mo del is imp ortant only for the exact b ound on the slowdown of the optimal algorithm

Similarly the sp ecics of the mo del eect the exact quantitative form of hierarchy theorems but not their mere

existence Finally in contrary to some b eliefs the sp ecics of the mo del are irrelevant also for most results regarding

space complexity provided that reasonable accounting of workspace is applied

that the students understand the conceptual meaning of these basics Needless to say the rest of

the course should also clarify the conceptual meaning of the material b eing taught

A parenthetical comment on computability versus complexity

This essay refers to the current situation in many schools where the basics of complexity theory

are taught within a course in which material entitled computability plays at least an equal role

The essay is conned to the complexity part of such a course and takes the computability part

for granted

Let me seize the opp ortunity and express my opinion on this combined course on computability

and complexity theory In my opinion complexity theory should play the main role in this course

whereas the basic concepts and results of computability theory should b e regarded as an imp ortant

preliminary material That is I view computability theory as setting the stage for the study of

the complexity of the computational tasks that can b e automated at all Thus the computability

asp ects of such a course should b e conned to establishing that the intuitive notion of an algorithm

can b e rigorously dened and to emphasizing the uncomputability of most functions and of some

natural functions eg the Halting predicate This includes introducing the idea of a universal

algorithm but do es not included extensive programming with Turing machines or extensive study

of complexityfree Turing reductions Needless to say I opp ose the teaching of nite automata

let alone contextfree grammars within such a course

Expanding up on the opinions expressed in the last paragraph is b eyond the scop e of the current

essay On the other hand the rest of this essay is indep endent of the foregoing remarks That is

it refers to the basic material of complexity theory regardless of the question within which course

this material is taught and what role do es it play in such a course

Organization

Section contains a presentation of the PvsNP Question b oth in terms of search problems and

in terms of decision problems Section contains a general treatment of reductions as well as a

subsection on selfreducibility of search problems Section contains a presentation of the basic

denitions and results of the theory of NPcompleteness as well as a mention of the existence of

NPsets that are neither in P nor NPcomplete Section mentions three additional topics that

are typically not taught in a basic course on computability and complexity theory These topics

include the conjectured nonexistence of coNPsets that are NPcomplete the existence of optimal

search algorithms for NPrelations and the notion of promise problems

As a general rule the more standard the material is the less detail we provide ab out is actual

technical contents Our fo cus is on the conceptual contents of the material and technical details

are given merely for illustration We stress again that this essay is not supp osed to serve as a

textb o ok but rather as a conceptual framework

The essay is augmented by a brief overview of complexity theory Unlike the rest of this essay

which assumes familiarity with the material this overview Section is supp osed to b e accessible

to the novice or an outsider and may b e used accordingly One p ossible use is as a base

for introductory comments on complexity theory to b e made either at the b eginning of a graduate

course on the topic or at the end of the currently prevailing undergraduate course on computability

and complexity theory

In fact this essay is based on my notes for three lectures covering the basic material which were given in a

graduate course on complexity theory see

P versus NP

Most students have heard of P and NP b efore but we susp ect that many have not obtained a go o d

explanation of what the PvsNP Question actually represents This unfortunate situation is due to

using the standard technical denition of NP which refers to nondeterministic p olynomialtime

rather than using somehat more cumbersome denitions that clearly capture the fundamental

nature of NP Below we take the alternative approach In fact we present two fundamental

formulations of the PvsNP Question one in terms of search problems and the other in terms of

decision problems

Ecient computation The teacher should discuss the asso ciation of ecient computation with

p olynomialtime algorithms stressing that this asso ciation merely provides a convenient way of

addressing fundamental issues In particular p olynomials are merely a closed set of mo derately

growing functions where closure means closure under addition multiplication and functional

comp osition These closure prop erties guarantee the closure of the class of ecient algorithms under

natural algorithmic comp osition op erations such as sequential execution and subroutine calls The

sp ecics of the mo del of computation are also immaterial as long as the mo del is reasonable

this strengthening of the ChurchTuring Thesis is called the CobhamEdmonds Thesis

The search version nding versus checking

In the eyes of nonexp erts search problems are more natural than decision problems typically

p eople seeks solutions more than they stop to wonder whether or not solutions exist Thus we

recommend starting with a formulation of the PvsNP Question in terms of search problems

Admittingly the cost is more cumbersome formulations presented in Figure but it is more

than worthwhile Furthermore the equivalence to the decision problem formulation gives rise to

conceptually app ealing exercises

We fo cus on p olynomiallyb ounded relations where a relation R f g f g is p olynomially

b ounded if there exists a p olynomial p such that for every x y R it holds that jy j pjxj For

such a relation it makes sense to ask whether given an instance x one can eciently nd a

solution y such that x y R The p olynomial b ound on the length of the solution ie y

guarantees that the intrinsic complexity of outputting a solution may not b e due to the length or

mere typing of the required solution

The class P as a natural class of search problems With each p olynomiallyb ounded relation

R we asso ciate the following search problem given x nd y such that x y R or state that

no such y exists The class P corresponds to the class of search problems that are solvable

in p olynomialtime that is a relation R or rather the search problem of R is p olynomialtime

solvable if there exists a p olynomialtime algorithm that given x nd y such that x y R or state

that no such y exists

Indeed we claim that these fundamental issues are actually indep endent of the aforementioned asso ciation

For example the question of whether nding a solution is harder than verifying its validity makes sense under any

reasonable notion of hardness Similarly the claim that factoring or any other NP problem is easily reducible

to SAT holds for many reasonable notions of easy to compute mappings

We leave it to the teacher whether to actually dene P resp NP as a class of search problems or to reserve

this notion for the relevant class of decision problems and merely talk ab out a corresp ondence b etween the search

and decision problem classes Our own preference is to introduce dierent notations for the search problem classes

see Figure

The class NP as another natural class of search problems A p olynomiallyb ounded rela

tion R is called an NPrelation if given an alleged instancesolution pair one can eciently check

whether or not the pair is valid that is there exists a p olynomialtime algorithm that given x and

y determines whether or not x y R The class NP corresponds to the class of search problems

for NPrelations and contains a host of natural search problems It is reasonable to fo cus on

search problems for NPrelations b ecause the ability to eciently recognize a valid solution seems

to b e a natural prerequisite for a discussion regarding the complexity of nding such solutions

Indeed one can introduce unnatural nonNPrelations for which the search problem is solvable

in p olynomialtime still the restriction to NPrelations is very natural

The P versus NP question in terms of search problems Is it the case that the search

problem of any NPrelation can be solved in polynomialtime In other words if it is easy to check

whether or not a given solution for a given instance is correct then is it also easy to nd a solution

to a given instance

If P NP in terms of search problems then this would mean that whenever solutions to

given instances can b e eciently veried for correctness it is also the case that these solutions can

b e eciently found when given only the instance This would mean that all reasonable search

problems ie all NPrelations are easy to solve Needless to say such a situation would contradict

the intuitive feeling and daily exp erience that some reasonable search problems are hard to solve

On the other hand if P NP then there exist reasonable search problems ie some NPrelations

that are hard to solve This conforms with our daily exp erience by which some reasonable problems

are easy to solve whereas others are hard to solve

Recall that search problems refer to binary relations For such a relation R the corresp onding search

problem is given x to nd y such that x y R or assert that no such y exists We suggest dening

two classes of search problems

PF standing for PolyFind denotes the class of search problems that are solvable in

p olynomialtime That is R PF if there exists a p olynomial time algorithm that given

x nds y such that x y R or assert that no such y exists

PC standing for PolyCheck denotes the class of search problems that corresp ond to

p olynomiallyb ounded binary relations that are checkable in p olynomialtime That is

R PC if the following two conditions hold

For some p olynomial p if x y R then jy j pjxj

There exists a p olynomialtime algorithm that given x y determines whether or not

x y R

In terms of search problems the PvsNP Question consists of asking whether or not PC is contained

in PF The conjectured inequality P NP implies that PC n PF

Figure PvsNP in terms of search problems notational suggestions

The decision version proving versus verifying

We suggest starting by asserting the natural stature of decision problems b eyond their role in the

study of search problems After all some p eople do care ab out the truth and so determining

whether a given ob ject has some claimed prop erty is an app ealing problem The PvsNP Question

refers to the complexity of answering such questions for a wide and natural class of prop erties

asso ciated with the class NP The latter class refers to prop erties that have ecient pro of systems

allowing for the verication of the claim that a given ob ject has a predetermined prop erty ie is

a member of a predetermined set

For an NPrelation R we denote the set of instances having a solution by L that is L fx

R R

y x y R g Such a set is called an NPset and NP denotes the class of all NPsets Intuitively

an NPset is a set of valid statements ie statements of membership of a given x in L that

R

can b e eciently veried when given adequate pro ofs ie a corresp onding NPwitness y such that

x y R This leads to viewing NPsets as pro of systems

NPpro of systems Pro of systems are dened in terms of their verication pro cedures Here

we fo cus on the natural class of ecient verication pro cedures where eciency is represented by

p olynomialtime computations We should either require that the time is p olynomial in terms of

the statement or conne ourselves to short pro ofs that is pro ofs of length that is b ounded by a

p olynomial in the length of the statement NPrelations corresp ond to pro of systems with ecient

verication pro cedures Sp ecically the NPrelation R corresp onds to the pro of system with a

verication pro cedure that checks whether or not the alleged statementproof pair is in R This

pro of system satises the natural completeness and soundness conditions every true statement ie

x L has a valid pro of ie an NPwitness y such that x y R whereas false statements

R

ie x L have no valid pro ofs ie x y R for all y s

R

Recall that decision problems refer to membership in sets We suggest dening two classes of decision

problems which indeed coincide with the standard denitions of P and NP

P denotes the class of decision problems that are solvable in p olynomialtime That is S P

if there exists a p olynomial time algorithm that given x determines whether or not x S

NP denotes the class of decision problems that have NPpro of systems The latter are dened

in terms of a deterministic p olynomialtime verication algorithm That is S NP if there

exists a p olynomial p and a p olynomialtime algorithm V that satisfy the following completeness

and soundness conditions

Completeness if x S then there exists y of length at most pjxj such that V x y

Such a string y is called an NPwitness for x S

Soundness if x S then for every y it holds that V x y

Indeed the p oint is dening NP as a class of sets of assertions having ecient verication

pro cedures

In terms of decision problems the PvsNP Question consists of asking whether or not NP is contained

in P Since P NP the question is phrased as whether or not NP equals P

Figure PvsNP in terms of decision problems notational suggestions

The P versus NP question in terms of decision problems Is it the case that NPproofs

are useless That is is it the case that for every eciently veriable pro of system one can easily

determine the validity of assertions without b eing given suitable pro ofs If that were the case

then pro ofs would b e meaningless b ecause they would have no fundamental advantage over directly

determining the validity of the assertion Denoting by P the class of sets that can b e decided

eciently ie by a p olynomialtime algorithm the conjecture P NP asserts that pro ofs are

useful there exists NPsets that cannot b e decided by a p olynomialtime algorithm and so for these

sets obtaining a pro of of membership for some instances is useful b ecause we cannot eciently

determine membership by ourselves

Equivalence of the two formulations

We strongly recommend proving that the two formulations of the PvsNP Questions are equivalent

That is the search problem of every NPrelation is solvable in p olynomial time if and only if

membership in any NPset can b e decided in p olynomial time see Figure This justies the

fo cus on the latter simpler formulation

Referring the notations of Figures and we prove that PC PF if and only if NP P

Supp ose that the inclusion holds for the search version ie PC PF Let L b e an arbitrary

NPset and R b e the corresp onding witness relation Then R is a NPrelation and by the

L L

hypothesis its search problem is solvable in p olynomial time ie R PC PF This yields

L

a p olynomialtime decision pro cedure for L ie given x try to nd y such that x y R

L

and output yes i such a y was found Thus NP P follows

Supp ose that NP P as classes of sets and let R b e an arbitrary NPrelation Then the set

def

0 00 0 00 0 00 0 00

fx y y st x y y R g where y y denotes the concatenation of y and y is in S

R

NP and hence in P This yields a p olynomialtime algorithm for solving the search problem of

R by extending a prex of a p otential solution bit by bit while using the decision pro cedure

to determine whether or not the current prex is valid Thus PC PF follows

Figure A pro of that PC PF if and only if NP P

We also suggest mentioning that NP is sometimes dened as the class of sets that can b e

decided by a ctitious device called a nondeterministic p olynomialtime machine and that this

is the source of the notation NP The reason that this class of ctitious devices is imp ortant is

b ecause it captures indirectly the denition of NPpro of systems We suggest proving that indeed

the denition of NP in terms of nondeterministic p olynomialtime machine equals our denition

of NP in terms of the class of sets having NPpro of systems

Reductions and Selfreducibility

We assume that many students have heard of reductions but again we fear that most of them

have obtained a conceptually p o or view of their nature We b elieve that this is due to exp ositions

that start with a technical denition of manytoone p olynomialtime reductions ie Karp

reductions rather than with a motivational discussion Below we take an the alternative approach

presenting rst the general notion of p olynomialtime reductions among computational problems

and viewing the notion of a Karpreduction as an imp ortant sp ecial case that suces and is more

convenient in many cases

The general notion of a reduction

Reductions are pro cedures that use functionally sp ecied subroutines That is the functionality

of the subroutine is sp ecied but its op eration remains unsp ecied and its runningtime is counted

at unit cost Analogously to algorithms which are mo deled by Turing machines reductions can b e

mo deled as oracle Turing machines A reduction solves one computational problem which may b e

either a search or decision problem by using oracle or subroutine calls to another computational

problem which again may b e either a search or decision problem We fo cus on ecient ie

p olynomialtime reductions which are often called Co ok reductions

The key prop erty of reductions is that they translate ecient pro cedures for the subroutine into

ecient pro cedures for the invoking machine That is if one problem is Co okreducible to another

problem and the latter is p olynomialtime solvable then so is the former

The most p opular case is of reducing decision problems to decision problems but we will also

consider reducing search problems to search problems or reducing search problems to decision

problems Indeed a go o d exercise is showing that the search problem of any NPrelation can b e

reduced to deciding membership in some NPset which is the actual contents of the second item

of Figure

A Karpreduction is a sp ecial case of a reduction from a decision problem to a decision problem

Sp ecically for decision problems L and L we say that L is Karpreducible to L if there is a

reduction of L to L that operates as fol lows On input x an instance for L the reduction computes

x makes query x to the oracle L ie invokes the subroutine for L on input x and answers

whatever the latter returns This Karpreduction is often represented by the p olynomialtime

computable mapping of x to x that is a p olynomialtime computable f is called a Karpreduction

of L to L if for every x it holds that x L i f x L

Indeed a Karpreduction is a syntactically restricted notion of a reduction This restricted case

suces for many cases eg most imp ortantly for the theory of NPcompleteness when developed

for decision problems but not in case we want to reduce a search problem to a decision problem

Furthermore whereas each decision problem is reducible to its complement some decision problems

are not Karpreducible to their complement eg the trivial decision problem Likewise each

decision problem in P is reducible to any computational problem by a reduction that do es not use

the subroutine at all whereas such a trivial reduction is disallowed by the syntax of Karpreductions

Nevertheless a p opular exercise of dubious nature is to show that any decision problem in P is

Karpreducible to any nontrivial decision problem

We comment that Karpreductions may and should b e augmented in order to handle re

ductions of search problems to search problems Such an augmented Karpreduction of the search

problem of R to the search problem of R op erates as follows On input x an instance for R the

reduction computes x makes query x to the oracle R ie invokes the subroutine for searching

R on input x obtaining y such that x y R and uses y to compute a solution y to x

ie x y R Thus such a reduction can b e represented by two p olynomialtime computable

mappings f and g such that x g x y R for any y that solves f x ie y that satises

f x y R Indeed in general unlike in the case of decision problems the reduction cannot

just return y as an answer to x

We say that two problems are computationally equivalent if they are reducible to one another

This means that the two problems are essentially equally hard or equally easy

We call a decision problem trivial if it refers to either the empty set or the set of all strings

Selfreducibility of search problems

We suggest introducing the notion of selfreducibility for several reasons Most imp ortantly it

further justies the fo cus on decision problems see discussion following Prop osition In addition

it illustrates the general notion of a reduction and asserts its relevance b eyond the theory of NP

completeness

The search problem of R is called selfreducible if it can b e reduced to the decision problem

of L fx y x y R g rather than to the set S as in Figure Note that the decision

R R

problem of L is always reducible to the search problem for R eg invoke the search subroutine

R

and answer yes if and only if it returns some string rather than the no solution symbol

We will see that all NPrelations that corresp ond to NPcomplete sets are selfreducible mostly

via natural reductions We start with SAT the set of satisable Bo olean formulae in CNF

Let R b e the set of pairs such that is a satisfying assignment to the formulae Note

S AT

that R is an NPrelation ie it is p olynomiallyb ounded and easy to decide by evaluating a

S AT

Bo olean expression

Prop osition R is selfreducible The search problem of R is reducible to S AT

S AT S AT

Thus the search problem of R is computationally equivalent to deciding membership in S AT

S AT

Hence in studying the complexity of S AT we also address the complexity of the search problem of

R This justies the relevance of decision problems to search problems in a stronger sense than

S AT

established in Section The study of decision problems determines not only the complexity of

the class of NPsearch problems but rather determines the complexity of each individual search

problem that is selfreducible

Pro of Given a formula we use a subroutine for SAT in order to nd a satisfying assignment

to in case such an assignment exists First we query SAT on itself and return no solution

if the answer we get is false Otherwise we let initiated to the empty string denote a prex

of a satisfying assignment of We pro ceed in iterations where in each iteration we extend by

one bit This is done as follows First we derive a formula denoted by setting the rst j j

variables of according to the values Next we query SAT on which means that we ask

whether or not is a prex of a satisfying assignment of If the answer is p ositive then we set

else we set b ecause if is a prex of a satisfying assignment of and is not a

prex of a satisfying assignment of then must b e a prex of a satisfying assignment of

A key p oint is that each formula which contains Bo olean variables as well as constants can

b e simplied to contain no constants in order to t the canonical denition of SAT which disallows

Bo olean constants That is after replacing some variables by constants we should simplify clauses

according to the straightforward b o olean rules eg a false literal can b e omitted from a clause

and a true literal app earing in a clause allows omitting the entire clause

Advanced comment A reduction analogous to the one used in the pro of of Prop osition can b e

presented also for other NPsearch problems and not only for NPcomplete ones Consider for

Our usage of this term diers from the traditional one Traditionally a decision problem is called selfreducible

if it is Co okreducible to itself via a reduction that on input x only makes queries that are smaller than x according

to some appropriate measure on the size of strings Under some natural restrictions ie the reduction takes the

disjunction of the oracle answers such reductions yield reductions of search to decision as discussed in the main

text

We assume that the students have heard of NPcompleteness If this assumption do es not hold for your class

then the presentation of the following material should b e p ostp oned to Section or to an even later stage

example the problem Graph Colorability and prexes of a coloring of the input graph Note

however that in this case the pro cess of getting rid of constants representing partial solutions

is more involved In general if you dont see a natural selfreducibility pro cess for some NP

complete relation you should know that a selfreduction pro cess do es exist alas it maybe not b e

a natural one

Theorem The search problem of the NPrelation of any NPcomplete set is selfreducible

Pro of Let R b e an NPrelation of the NPcomplete set L In order to reduce the search problem

R

of R to deciding L we comp ose the three reductions mentioned next

R

The search problem of R is reducible to the search problem of R by the NPcompleteness

S AT

of the latter

The search problem of R is reducible to S AT by Prop osition

S AT

The decision problem S AT is reducible to the decision problem L by the NPcompleteness

R

of the latter

The theorem follows

NPcompleteness

Some or most students have heard of NPcompleteness b efore but we susp ect that many have

missed imp ortant conceptual p oints Sp ecically we stress that the mere existence of NPcomplete

sets regardless of whether this is SAT or some other set is amazing

Denitions

The standard denition is that a set is NPcomplete if it is in NP and every set in NP is reducible

to it via a Karpreduction Indeed there is no reason to insist on Karpreductions rather than

using arbitrary reductions except that the restricted notion suces for all p ositive results and is

easier to work with

We will also refer to the search version of NPcompleteness We say that a binary relation is

NPcomplete if it is an NPrelation and every NPrelation is reducible to it

We stress that the mere fact that we have dened something ie NPcompleteness do es not

mean that this thing exists ie that there exist ob jects that satisfy the denition It is indeed

remarkable that NPcomplete problems do exist Such problems are universal in the sense that

solving them allows solving any other reasonable problem

The existence of NPcomplete problems

We suggest not to confuse the mere existence of NPcomplete problems which is remarkable by

itself with the even more remarkable existence of natural NPcomplete problems We b elieve

that the following pro of facilitates the delivery of this message as well as fo cusing on the essence

of NPcompleteness rather than on more complicated technical details

Details can left as an exercise to the student You may hint that a partial coloring can b e hardwired into the

graph by augmenting the graph with adequate gadgets that force equality or inequality b etween the colors of two

vertices of our choice

Theorem There exist NPcomplete relations and sets

Pro of The pro of as well as any other NPcompleteness pro of is based on the observation that

some NPrelations resp NPsets are rich enough to enco de all NPrelations resp NPsets

This is most obvious for the generic NPrelation denoted R and dened b elow which is used

U

to derive the simplest pro of of the current theorem

t

The relation R consists of pairs hM x i y such that M is a description of a deterministic

U

Turing machine that accepts the pair x y within t steps where jy j t Instead of requiring

that jy j t one may require that M is canonical in the sense that it reads its entire input b efore

def

x y x y R g is an halting It is easy to see that R is an NPrelation and thus L f

U U U

t

NPset Indeed R is recognizable by a universal Turing machine which on input hM x i y

U

emulates t steps of the computation of M on x y and U indeed stands for universal machine

Thus the pro of extends to any reasonable mo del of computation which has adequate universal

machines

We now turn to showing that any NPrelation is reducible to R As a warmup let us rst show

U

that any NPset is Karpreducible to L Let R b e an NPrelation and L fx y x y R g

U R

b e the corresp onding NPset Let p b e a p olynomial b ounding the length of solutions in R ie

R

jy j p jxj for every x y R let M b e a p olynomialtime machine deciding membership

R R

of alleged x y pairs in R and let t b e a p olynomial b ounding its runningtime Then the

R

t jxjp jy j

R R

Karpreduction maps an instance x for L to the instance hM x i

R

Note that this mapping can b e computed in p olynomialtime and that x L if and only if

t jxjp jy j

R R

hM x i L

R U

To reduce the search problem of R to the search problem of R we use essentially the same

U

t jxjp jy j

R R

reduction On input an instance x for R we make the query hM x i to the

R

search problem of R and return whatever the latter returns Note that if x L then the

U R

answer will b e no solution whereas for every x and y it holds that x y R if and only if

t jxjp jy j

R R

hM x i y R

R U

t

Advanced comment Note that the role of in the denition of R is to make R an NP

U U

def

relation In contrast consider the relation R fhM xi y M xy g which corresp onds

H

to the halting problem Indeed the search problem of any relation an in particular of any NP

relation is Karpreducible to the search problem of R but the latter is not solvable at all ie

H

x hM xi outputs y such that there exists no algorithm that halts on every input and on input

x y R i such a y exists

H

CSAT SAT and other NPcomplete problems

Once the mere existence of NPcomplete problems has b een established we suggest informing the

students of the fact that many natural problems are NPcomplete and demonstrating this fact with

a few examples Indeed SAT is a go o d rst example b oth b ecause the reduction to it is instructive

and b ecause it is a convenient starting p oint to further reductions As a second example we suggest

various variants of the Set Cover problem Additional reductions may b e deferred to homework

assignments and presenting them in class seems inadequate in the context of a course on complexity

theory

We suggest establishing the NPcompleteness of SAT by a reduction from the circuit satisfaction

problem CSAT after establishing the NPcompleteness of the latter Doing so allows decoupling

two imp ortant issues in the pro of of the NPcompleteness of SAT the emulation of Turing

machines by circuits and the enco ding of circuits by formulae with auxiliary variables Following

is a rough outline which fo cuses on the decision version

CSAT Dene Bo olean circuits as directed acyclic graphs with internal vertices called gates

lab eled by Bo olean op erations of arity either or and external vertices called terminals that

are asso ciated with either inputs or outputs When setting the inputs of such a circuit all internal

no des are assigned values in the natural way and this yields a value to the outputs called an

evaluation of the circuit on the given input Dene the satisability problem of such circuits as

determining for a given circuit whether there exists a setting to its inputs that makes its rst

output evaluate to Prove the NPcompleteness of the circuit satisfaction problem CSAT

by reducing any NPset to it where the set is represented by the machine that recognizes the

corresp onding NPrelation The reduction b oils down to enco ding p ossible computations of a

Turing machine by a corresp onding layered circuit where each layer represents an instantaneous

conguration of the machine and the relation b etween consecutive congurations is captured by

uniform lo cal gadgets in the circuit For further details see Figure The pro of extends to

any other reasonable mo del of ecient computation

Following are some additional comments on the pro of of the NPcompleteness of CSAT These com

ments refer to the highlevel structure of the reduction and do not provide a full lowlevel description

of it

For a machine M as in the pro of of Theorem we will represent the computation of M on input

R R

x y where x is the input to the reduction and y is undetermined by a circuit C that takes such

x

a string y as input Thus C y if and only if M accepts x y and so C is satisable if and

x R x

only if x L The reduction maps x to a circuit C as follows

R x

th st th

The circuit C consists of layers such that the i layers of wires connecting the i and i

x

th

layers of vertices represents the instantaneous conguration of M x y just b efore the i step In

R

st

particular the gates of the i layer are designed to guaranteed that the instantaneous conguration

th

of M x y just b efore the i step is transformed to the instantaneous conguration of M x y

R R

st

just b efore the i step Only the rst layer of C dep ends on x itself which is hardwired into

x

the circuit The rest of the construction dep ends only on jxj and M

R

Figure Enco ding computations of a Turing machine in a Bo olean circuit

The ab ove reduction is called generic b ecause it explicitly refers to any generic NPset

However the common practice is to establish NPcompleteness by a reduction from some NP

complete set ie a set already shown to b e NPcomplete This practice is based on the fact that

if an NPcomplete problem is reducible to some problem in NP then is NPcomplete The

pro of of this fact b oils down to asserting the transitivity of reductions

SAT Dene Bo olean formulae which may b e viewed as Bo olean circuits with a tree structure

Prove the NPcompleteness of the formula satisfaction problem SAT even when the formula is

given in a nice form ie CNF The pro of is by a reduction from CSAT which in turn b oils

down to introducing auxiliary variables in order to cut the computation of a deep circuit into a

conjunction of related computations of shallow ie depth circuits which may b e presented as

CNF formulae The aforementioned auxiliary variables hold the p ossible values of the internal

wires of the circuit

SAT Note that the formulae resulting from the latter reduction are in conjunctive normal form

CNF with each clause referring to three variables ie two corresp onding to the input wires of

a no degate and one to its output wire Thus the ab ove reduction actually establishes the NP

completeness of SAT ie SAT restricted to CNF formula with up to three variables p er clause

Alternatively reduce SAT for CNF formula to SAT ie satisability of CNF formula by

replacing long clauses with conjunctions of threevariable clauses using auxiliary variables

In order to establish the NPcompleteness of the search version of the aforementioned problems

we need to present a p olynomialtime mapping of solutions for the target problem eg SAT to

solutions for the origin problem eg CSAT Note that such a mapping is typically explicit in the

argument establishing the validity of the Karpreduction

Set Cover and other problems If time p ermits one may want to present another class of

NPcomplete problems and our choice is of Set Cover There is a simple reduction from SAT to Set

Cover with the sets corresp onding to the sets of clauses that are satised when assigning a sp ecic

Bo olean variable a sp ecic Bo olean value When applied to a restricted version of SAT in which

each variable app ears in at most three clauses the same reduction implies the NPcompleteness of

a version of Set Cover in which each set contains at most three elements Indeed one should rst

establish the NPcompleteness of the aforementioned restricted version of SAT Using the restricted

version of Set Cover one may establish the NPcompleteness of Exact Cover even when restricted

to element sets The latter problem is a convenient starting p oint for further reductions

NP sets that are neither in P nor NPcomplete

Many to say the least other NPsets have b een shown to b e NPcomplete A very partial list in

cludes Graph Colorability Subset Sum Exact Set Cover and the Traveling Salesman Problem

Hundreds of other natural problems can b e found in Things reach a situation in which some

computer scientists seem to exp ect any NPset to b e either NPcomplete or in P This naive view

is wrong

Theorem Assuming NP P there exist NPsets that are neither NPcomplete nor in P

We mention that some natural problems eg factoring are conjectured to b e neither solvable in

p olynomialtime nor NPhard where a problem is NPhard if any NPset is reducible to solving

See discussion following Theorem We recommend to either state Theorem without a pro of

or merely provide the pro of idea which is sketched next

Pro of idea The pro of is by mo difying a set in NP n P such that to fail all p ossible reductions to

this set and all p ossible p olynomialtime decision pro cedures for this set Sp ecically we start

with some L NP n P and derive L L which is also in NP n P by making each reduction say

of L to L fail by dropping nitely many elements from L until the reduction fails whereas all

p ossible p olynomialtime fail to decide L which dier from L only on a nite number of inputs

We use the fact that any reduction of some set in NP n P to a nite set ie a nite subset of L

must fail and this failure is due to a nite set of queries whereas any ecient decision pro cedure

for L or L mo died on nitely many inputs must fail on some nite p ortion of all p ossible inputs

of L The pro cess of mo difying L into L pro ceeds in iterations alternatively failing a p otential

reduction by dropping suciently many strings from the rest of L and failing a p otential decision

pro cedure by including suciently many strings from the rest of L This can b e done eciently

b ecause it is inessential to determine the optimal p oints of alternation where suciently many

strings were dropp ed resp included to fail a p otential reduction resp decision pro cedure

Thus L is the intersection of L and some set in P which implies that L NP n P

Three additional topics

The following topics are typically not mentioned in a basic course on complexity Still p ending on

time constraints we suggest covering them at some minimal level

The class coNP and NPcompleteness

By prep ending the name of a complexity class of decision problems with the prex co we mean

the class of complement sets that is

def

co C ff g n L L C g

Sp ecically co NP ff g n L L N P g is the class of sets that are complements of NP

sets That is if R is an NPrelation and L fx y x y R g is the asso ciated NPset then

R

f g n L fx y x y R g is the corresp onding coNPset

R

It is widely b elieved that NP is not closed under complementation ie NP co NP Indeed

this conjecture implies P NP b ecause P is closed under complementation The conjecture

NP co NP means that some coNPsets eg the complements of NPcomplete sets do not have

NPpro of systems that is there is no NPpro of system for proving that a given formula is not

satisable

If indeed NP co NP then some NPsets cannot b e Karpreducible to any coNPset How

ever each NPset is reducible to some coNPset by a straightforward Co okreduction that just ips

the answer and so the nonexistence of such Karpreduction do es not seem to represent anything

really fundamental In contrast we mention that NP coNP implies that some NPsets cannot

b e reduced to sets in the intersection NP coNP even under general ie Co ok reductions

Sp ecically

Theorem If NP co NP contains an NPhard set then NP co NP

Recall that a set is NPhard if every NPset is reducible to it p ossibly via a general reduction

Since NP co NP is conjectured to b e a prop er sup erset of P it follows using the conjecture

NP co NP that there are NPsets that are neither in P nor NPhard sp ecically the sets in

NP co NP n P are neither in P nor NPhard Notable candidates are sets related to the integer

factorization problem eg the set of pairs N s such that s has a square ro ot mo dulo N that is

a quadratic residue mo dulo N and the least signicant bit of s equals

Pro of Supp ose that L NP co NP is NPhard Given any L co NP we will show that

L NP We will merely use the fact that L reduces to L which is in NP co NP Such a

def

f g n L via a general reduction whereas L L NP reduction exists b ecause L is reducible

and thus is reducible to L which is NPhard

Sp ecically we claim that sets in NP n coNP cannot b e Karpreducible to sets in coNP In fact we prove

that only sets in co NP are Karpreducible to sets in coNP Equivalently let us prove that only sets in NP are

0

Karpreducible to sets in NP where the equivalence follows by noting that a reduction of L to L is also a reduction

  0 0

of f g n L to f g n L Indeed supp ose that L Karpreduces to L NP Then L NP by virtue of the

0 0 0

NPrelation fx y f x y R g where R is the witness relation of L

To show that L NP we will present an NPrelation R that characterizes L ie L

fx y x y R g The relation R consists of pairs of the form x z w z w

t t t

where on input x the reduction of L to L accepts after making the queries z z obtaining the

t

corresp onding answers and for every i it holds that if then w is an NPwitness for

t i i

z L whereas if then w is an NPwitness for z f g n L

i i i i

def

L f g n L are NPsets and refer to the We stress that we use the fact that b oth L and

corresp onding NPwitnesses Note that R is indeed an NPrelation The length of solutions is

b ounded by the runningtime of the reduction and the corresp onding NPwitnesses Membership

in R is decided by checking that the sequence of z s matches a p ossible queryanswer sequence

i i

in an accepting execution of the reduction ignoring the correctness of the answers and that all

answers ie s are correct The latter condition is easily veried by use of the corresp onding

i

NPwitnesses

Optimal search algorithms for NPrelations

The title of this section sounds very promising but our guess is that the students will b e less excited

once they see the pro of We claim the existence of an optimal search algorithm for any NPrelation

Furthermore we will explicitly present such an algorithm and prove that it is optimal without

knowing its running time

Theorem For every NPrelation R there exists an algorithm A that satises the fol lowing

A correctly solves the search problem of R

There exists a polynomial p such that for every algorithm A that correctly solves the search

0

x problem of R and for every x L fz y z y R g it holds that t x O t

R A

A

0

denotes the number of steps taken by A resp A on input x pjxj where t resp t

A

A

We stress that the hidden constant in the Onotation dep ends only on A but in the following

pro of the dep endence is exp onential in the length of the description of algorithm A and it is not

known whether a b etter dep endence can b e achieved Optimality holds in a p ointwise manner

ie for every input and the additive p olynomial term ie pjxj is insignicant in case the

NPproblem is not solvable in p olynomialtime On the other hand the optimality of algorithm A

refers only to inputs that have a solution ie x L Interestingly we establish the optimality

R

of A without knowing what its optimal runningtime is Furthermore the PsvNP Question boils

down to determining the running time of a single explicitly presented algorithm ie the optimal

algorithm A Finally we note that the theorem as stated refers only to mo dels of computation

that have machines that can emulate a given number of steps of other machines with a constant

overhead We mention that in most natural mo dels the overhead of such emulation is at most

e

0

x pjxj p olylogarithmic in the number of steps in which case it holds that t x O t

A

A

Pro of sketch Fixing R we let M b e a p olynomialtime algorithm that decides membership in

R and let p b e a p olynomial b ounding the runningtime of M We present the following algorithm

A that merely runs all p ossible search algorithms in parallel and checks the results provided by

each of them using M halting whenever it obtains a correct solution

Since there are innitely many p ossible algorithms we should clarify what we mean by running

them all in parallel What we mean is to run them at dierent rates such that the innite

That is we need to verify that on input x after obtaining the answers to the rst i queries the

i

th

i query made by the reduction equals z

i

th

sum of rates converges to or any other constant Sp ecically we will run the i p ossible

algorithm at rate i Note that a straightforward implementation of this idea may create

a signicant overhead involved in switching frequently from the computation of one machine to

th

another Instead we present an alternative implementation that pro ceeds in iterations In the j

j j th

iteration for i we emulate i steps of the i machine Each of these emulations

is conducted in one chunk and thus the overhead of switching b etween the various emulations is

insignicant in comparison to the total number of steps b eing emulated We stress that in case

some of these emulations halts with output y algorithm A invokes M on input x y and output

y if and only if M x y Furthermore the verication of a solution provided by a candidate

algorithm is also emulated at the exp ense of its stepcount Put in other words we augment each

algorithm with a canonical pro cedure ie M that checks the validity of the solution oered by

the algorithm

In order to guarantee that A also halts on x L we let it run an exhaustive search for a

R

solution in parallel to all searches and halt with output in case this exhaustive search fails

Clearly whenever Ax outputs y ie y it must hold that x y R To show the

optimality of A we consider an arbitrary algorithm A that solves the search problem of R Our

aim is to show that A is not much slower than A Intuitively this is the case b ecause the overhead

of A results from emulating other algorithms in addition to A but the total number of emulation

steps wasted due to these algorithms is inversely prop ortional to the rate of algorithm A which

in turn is exp onentially related to the length of the description of A The punchline is that since

A is xed the length of its description is a constant

Promise Problems

Promise problems are a natural generalization of decision problems and search problems can

b e generalized in a similar manner In fact in many cases promise problems provide the more

natural formulation of a decision problem Formally promise problems refer to a threeway partition

of the set of all strings into yesinstances noinstances and instances that violate the promise A

p otential decider is only required to distinguish yesinstances from noinstances and is allowed

arbitrary b ehavior on inputs that violate the promise Standard decision problems are obtained as

a sp ecial case by p ostulating that all inputs are allowed ie the promise is trivial

In contrary to the common p erception promise problems are no osho ot for abnormal situa

tions but are rather the norm Indeed the standard and natural presentation of natural decision

problems is actually in terms of promise problems although the presentation rarely refers explic

itly to the terminology of promise problems Consider a standard entry in or any similar

comp endium reading something like given a planar graph determine whether or not A more

formal statement will refer to strings that represent planar graphs Either way the natural formu

lation actually refers to a promise problem where the promise in this case is that the input is a

planar graph

We comment that the discrepancy b etween the intuitive promise problem formulation and the

standard formulation in terms of decision problems can b e easily bridged in the case that there

exists an ecient algorithm for determining membership in the promise set ie the set of in

stances that satisfy the promise In this case the promise problem is computationally equivalent to

deciding membership in the set of yesinstances However in case the promise set is not tractable

the terminology of promise problems is unavoidable Examples include the notion of unique solu

tions the formulation of gap problems that capture various approximation tasks and complete

problems for various probabilistic complexity classes For a recent survey on promise problems and

their applications the interested reader is referred to

A brief overview of Complexity Theory

The following text was originally written as a brief overview of complexity theory intended for

the novice It can also b e used as a basis for communicating the essence of complexity theory to

the outside ie to scientists in other disciplines and even to the general interested public Thus

unlike the rest of this essay which is intended for the teacher this section is intended for the student

or for other outsiders that the teacher may wish to address The text starts with an overview

of the PvsNP Question and the theory of NPcompleteness rep eating themes that were expressed

in the previous sections Still in light of the dierent p otential uses of this text I preferred not to

eliminate this part of the overview

Complexity Theory is concerned with the study of the intrinsic complexity of computational

tasks Its nal goals include the determination of the complexity of any welldened task Addi

tional nal goals include obtaining an understanding of the relations b etween various computa

tional phenomena eg relating one fact regarding computational complexity to another Indeed

we may say that the former type of goals is concerned with absolute answers regarding sp ecic com

putational phenomena whereas the latter type is concerned with questions regarding the relation

b etween computational phenomena

Interestingly the current success of Complexity Theory in coping with the latter type of goals

has b een more signicant In fact the failure to resolve questions of the absolute type led to the

ourishing of metho ds for coping with questions of the relative type Putting aside for a moment

the frustration caused by the failure we must admit that there is something fascinating in the

success in some sense establishing relations b etween phenomena is more revealing than making

statements ab out each phenomenon Indeed the rst example that comes to mind is the theory

of NPcompleteness Let us consider this theory for a moment from the p ersp ective of these two

types of goals

Complexity Theory has failed to determine the intrinsic complexity of tasks such as nding

a satisfying assignment to a given satisable prop ositional formula or nding a coloring of a

given colorable graph But it has established that these two seemingly dierent computational

tasks are in some sense the same or more precisely are computationally equivalent We nd

this success amazing and exciting and hop e that the reader shares our feeling The same feeling

of wonder and excitement is generated by many of the other discoveries of Complexity Theory

Indeed the reader is invited to join a fast tour of some of the other questions and answers that

make up the eld of Complexity Theory

We will indeed start with the P versus NP Question Our daily exp erience is that it is harder

to solve a problem than it is to check the correctness of a solution eg think of either a puzzle

or a research problem Is this exp erience merely a coincidence or do es it represent a fundamental

fact of life or a prop erty of the world Could you imagine a world in which solving any problem

is not signicantly harder than checking a solution to it Would the term solving a problem not

lose its meaning in such a hypothetical and imp ossible in our opinion world The denial of the

plausibility of such a hypothetical world in which solving is not harder than checking is what

P dierent than NP actually means where P represents tasks that are eciently solvable and

NP represents tasks for which solutions can b e eciently checked

The mathematically or theoretically inclined reader may also consider the task of proving

theorems versus the task of verifying the validity of pro ofs Indeed nding pro ofs is a sp ecial

type of the aforementioned task of solving a problem and verifying the validity of pro ofs is

a corresp onding case of checking correctness Again P dierent than NP means that there

are theorems that are harder to prove than to b e convinced of correctness when presented with a

pro of This means that the notion of a pro of is meaningful ie that pro ofs do help when trying

to b e convinced of the correctness of assertions Here NP represents sets of assertions that can b e

eciently veried with the help of adequate pro ofs and P represents sets of assertions that can b e

eciently veried from scratch ie without pro ofs

In light of the foregoing discussion it is clear that the PversusNP Question is a fundamental

scientic question of farreaching consequences The fact that this question seems b eyond our cur

rent reach led to the development of the theory of NPcompleteness Lo osely sp eaking this theory

identies a set of computational problems that are as hard as NP That is the fate of the Pversus

NP Question lies with each of these problems if any of these problems is easy to solve then so

are all problems in NP Thus showing that a problem is NPcomplete provides evidence to its in

tractability assuming of course P dierent than NP Indeed demonstrating NPcompleteness

of computational tasks is a central to ol in indicating hardness of natural computational problems

and it has b een used extensively b oth in computer science and in other disciplines NPcompleteness

indicates not only the conjectured intractability of a problem but rather also its richness in the

sense that the problem is rich enough to enco de any other problem in NP The use of the term

enco ding is justied by the exact meaning of NPcompleteness which in turn is based on estab

lishing relations b etween dierent computational problems without referring to their absolute

complexity

The foregoing discussion of the PversusNP Question also hints to the importance of repre

sentation a phenomenon that is central to complexity theory In general complexity theory is

concerned with problems the solutions of which are implicit in the problems statement That is

the problem contains all necessary information and one merely needs to pro cess this information

in order to supply the answer Thus complexity theory is concerned with manipulation of in

formation and its transformation from one representation in which the information is given to

another representation which is the one desired Indeed a solution to a computational problem

is merely a dierent representation of the information given that is a representation in which the

answer is explicit rather than implicit For example the answer to the question of whether or not

a given Bo olean formula is satisable is implicit in the formula itself but the task is to make the

answer explicit Thus complexity theory claries a central issue regarding representation that is

the distinction b etween what is explicit and what is implicit in a representation Furthermore it

even suggests a quantication of the level of nonexplicitness

In general complexity theory provides new viewp oints on various phenomena that were consid

ered also by past thinkers Examples include the aforementioned concepts of pro ofs and represen

tation as well as concepts like randomness knowledge interaction secrecy and learning We next

discuss some of these concepts and the p ersp ective oered by complexity theory

The concept of randomness has puzzled thinkers for ages Their p ersp ective can b e describ ed

as ontological they asked what is randomness and wondered whether it exist at all or is the

world deterministic The p ersp ective of complexity theory is b ehavioristic it is based on dening

ob jects as equivalent if they cannot b e told apart by any ecient pro cedure That is a coin toss is

dened to b e random even if one b elieves that the universe is deterministic if it is infeasible

to predict the coins outcome Likewise a string or a distribution of strings is random if it

In contrast in other disciplines solving a problem may require gathering information that is not available in

the problems statement This information may either b e available from auxiliary past records or b e obtained by

conducting new exp eriments

is infeasible to distinguish it from the uniform distribution regardless of whether or not one can

generate the latter Interestingly randomness or rather pseudorandomness dened this way is

eciently expandable that is under a reasonable complexity assumption to b e discussed next

short pseudorandom strings can b e deterministically expanded into long pseudorandom strings

Indeed it turns out that randomness is intimately related to intractability Firstly note that the

very denition of pseudorandomness refers to intractability ie the infeasibility of distinguishing

a pseudorandomness ob ject from a uniformly distributed ob ject Secondly as hinted ab ove a

complexity assumption that refers to the existence of functions that are easy to evaluate but

hard to invert called oneway functions imply the existence of deterministic programs called

pseudorandom generators that stretch short random seeds into long pseudorandom sequences In

fact it turns out that the existence of pseudorandom generators is equivalent to the existence of

oneway functions

Complexity Theory oers its own p ersp ective on the concept of know ledge and distinguishes

it from information It views knowledge as the result of a hard computation Thus whatever

can b e eciently done by anyone is not considered knowledge In particular the result of an easy

computation applied to publicly available information is not considered knowledge In contrast

the value of a hard to compute function applied to publicly available information is knowledge

and if someb o dy provides you with such a value then it has provided you with knowledge This

discussion is related to the notion of zeroknowledge interactions which are interactions in which no

knowledge is gained Such interactions may still b e useful b ecause they may assert the correctness

of sp ecic knowledge that was provided b eforehand

The foregoing paragraph has explicitly referred to interaction It has p ointed one p ossible

motivation for interaction gaining knowledge It turns out that interaction may help in a variety

of other contexts For example it may b e easier to verify an assertion when allowed to interact with

a prover rather than when reading a pro of Put dierently interaction with some teacher may b e

more b enecial than reading any b o ok We comment that the added p ower of such interactive proofs

is ro oted in their b eing randomized ie the verication pro cedure is randomized b ecause if the

veriers questions can b e determined b eforehand then the prover may just provide the transcript

of the interaction as a traditional written pro of

Another concept related to knowledge is that of secrecy knowledge is something that one party

has while another party do es not have and cannot feasibly obtain by itself thus in some sense

knowledge is a secret In general complexity theory is related to Cryptography where the latter

is broadly dened as the study of systems that are easy to use but hard to abuse Typically such

systems involve secrets randomness and interaction as well as a complexity gap b etween the ease

of prop er usage and the infeasibility of causing the system to deviate from its prescrib ed b ehavior

Thus much of Cryptography is based on complexity theoretic assumptions and its results are

typically transformations of relatively simple computational primitives eg oneway functions

into more complex cryptographic applications eg a secure encryption scheme

We have already mentioned the context of learning when referring to learning from a teacher

versus learning from a b o ok Recall that complexity theory provides evidence to the advantage of

the former This is in the context of gaining knowledge ab out publicly available information In

contrast computational learning theory is concerned with learning ob jects that are only partially

available to the learner ie learning a function based on its value at a few random lo cations or

even at lo cations chosen by the learner Complexity Theory sheds light on the intrinsic limitations

of learning in this sense

Complexity Theory deals with a variety of computational tasks We have already mentioned two

fundamental types of tasks searching for solutions or nding solutions and making decisions

eg regarding the validity of assertion We have also hinted that in some cases these two types

of tasks can b e related Now we consider two additional types of tasks counting the number of

solutions and generating random solutions Clearly b oth the latter tasks are at least as hard as

nding arbitrary solutions to the corresp onding problem but it turns out that for some natural

problems they are not signicantly harder Sp ecically under some natural conditions on the

problem approximately counting the number of solutions and generating an approximately random

solution is not signicantly harder than nding an arbitrary solution

Having mentioned the notion of approximation we mention that the study of the complexity

of nding approximate solutions has also received a lot of attention One type of approximation

problems refers to an ob jective function dened on the set of p otential solutions Rather than

nding a solution that attains the optimal value the approximation task consists of nding a

solution that obtains an almost optimal value where the notion of almost optimal may b e

understo o d in dierent ways giving rise to dierent levels of approximation Interestingly in many

cases even a very relaxed level of approximation is as dicult to achieve as the original exact

search problem ie nding an approximate solution is as hard as nding an optimal solution

Surprisingly these hardness of approximation results are related to the study of probabilistically

checkable proofs which are pro ofs that allow for ultrafast probabilistic verication Amazingly

every pro of can b e eciently transformed into one that allows for probabilistic verication based on

probing a constant number of bits in the alleged pro of Turning back to approximation problems

we note that in other cases a reasonable level of approximation is easier to achieve than solving the

original exact search problem

Approximation is a natural relaxation of various computational problems Another natural

relaxation is the study of averagecase complexity where the average is taken over some simple

distributions representing a mo del of the problems instances that may o ccur in practice We

stress that although it was not stated explicitly the entire discussion so far has referred to worst

case analysis of algorithms We mention that worstcase complexity is a more robust notion than

averagecase complexity For starters one avoids the controversial question of what are the instances

that are imp ortant in practice and corresp ondingly the selection of the class of distributions for

which averagecase analysis is to b e conducted Nevertheless a relatively robust theory of average

case complexity has b een suggested alb eit it is far less developed than the theory of worstcase

complexity

In view of the central role of randomness in complexity theory as evident say in the study of

pseudorandomness probabilistic pro of systems and cryptography one may wonder as to whether

the randomness needed for the various applications can b e obtained in reallife One sp ecic ques

tion which received a lot of attention is the p ossibility of purifying randomness or extracting

go o d randomness from bad sources That is can we use defected sources of randomness in order

to implement almost p erfect sources of randomness The answer dep ends of course on the mo del

of such defected sources This study turned out to b e related to complexity theory where the most

tight connection is b etween some type of randomness extractors and some type of pseudorandom

generators

So far we have fo cused on the time complexity of computational tasks while relying on the

natural asso ciation of eciency with time However time is not the only resource one should

care ab out Another imp ortant resource is space the amount of temp orary memory consumed

by the computation The study of space complexity has uncovered several fascinating phenomena

which seem to indicate a fundamental dierence b etween space complexity and time complexity For

example in the context of space complexity verifying pro ofs of validity of assertions of any sp ecic

type has the same complexity as verifying pro ofs of invalidity for the same type of assertions

In case the reader feels dizzy it is no wonder We to ok an ultrafast airtour of some mountain

tops and dizziness is to b e exp ected Needless to say a go o d graduate course in complexity theory

should consist of climbing some of these mountains by fo ot step by step and stopping to lo ok

around and reect

Absolute Results aka LowerBounds As stated upfront absolute results are not known

for many of the big questions of complexity theory most notably the PversusNP Question

However several highly nontrivial absolute results have b een proved For example it was shown

that using negation can sp eedup the computation of monotone functions which do not require

negation for their mere computation In addition many promising techniques were introduced

and employed with the aim of providing a lowlevel analysis of the progress of computation

However the fo cus of this overview was on the connections among various computational problems

and notions which may b e viewed as a highlevel study of computation

Historical Notes

Many sources provide historical accounts of the developments that led to the formulation of the

P vs NP Problem and the development of the theory of NPcompleteness see eg We thus

refrain from attempting to provide such an account

One technical p oint that we mention is that the three founding pap ers of the theory of NP

completeness ie refer to the three dierent terms of reductions used ab ove Sp ecically

Co ok used the general notion of p olynomialtime reduction often referred to as Co okreductions

The notion of Karpreductions originates from Karps pap er whereas its augmentation to search

problems originates from Levins pap er It is worth noting that unlike Co ok and Karps works

which treat decision problems Levins work is stated in terms of search problems

The existence of NPsets that are neither in P nor NPcomplete ie Theorem was proven by

Ladner Theorem was proven by Selman and the existence of optimal search algorithms for

NPrelations ie Theorem was proven by Levin Interestingly the latter result was proved

in the same pap er in which Levin presented the discovery of NPcompleteness indep endently of

Co ok and Karp Promise problems were explicitly introduced by Even Selman and Yacobi

Acknowledgments

I am grateful to Arny Rosenberg Alan Selman Salil Vadhan and an anonymous referee for their

useful comments and suggestions

References

SA Co ok The Complexity of Theorem Proving Pro cedures In rd STOC pages

S Even AL Selman and Y Yacobi The Complexity of Promise Problems with Applications

to PublicKey Cryptography Inform and Control Vol pages

MR Garey and DS Johnson Computers and Intractability A Guide to the Theory of NP

Completeness WH Freeman and Company New York

O Goldreich Introduction to Complexity Theory notes for a onesemester course Weiz

mann Institute of Science Spring Available from httpwwwwisdomweizmann

acilodedcchtml

O Goldreich On Promise Problems In memory of Shimon Even ECCC TR

January

RM Karp Reducibility among Combinatorial Problems In Complexity of Computer Com

putations RE Miller and JW Thatcher eds Plenum Press pages

RE Ladner On the Structure of Polynomial Time Reducibility Jour of the ACM

pages

LA Levin Universal Search Problems Problemy Peredaci Informacii pages

Translated in problems of Information Transmission pages

A Selman On the structure of NP Notices Amer Math Soc Vol page