P NP and NP Completeness
The Basics of Computational Complexity
Oded Goldreich
Department of Computer Science and Applied Mathematics
Weizmann Institute of Science Rehovot Israel
June
I
to Dana
c
Copyright by Oded Goldreich
Permission to make copies of part or all of this work for p ersonal or classro om use is
granted without fee provided that copies are not made or distributed for pro t or com
mercial advantage and that new copies b ear this notice and the full citation on the rst
page Abstracting with credit is p ermitted II
Preface
The strive for e ciency is ancient and universal as time and other resources are
always in shortage Thus the question of which tasks can b e p erformed e ciently
is central to the human exp erience
A key step towards the systematic study of the aforementioned question is a
rigorous de nition of the notion of a task and of pro cedures for solving tasks These
de nitions were provided by computability theory which emerged in the s
This theory fo cuses on computational tasks and considers automated pro cedures
i e computing devices and algorithms that may solve such tasks
In fo cusing attention on computational tasks and algorithms computability
theory has set the stage for the study of the computational resources like time that
are required by such algorithms When this study fo cuses on the resources that are
necessary for any algorithm that solves a particular task or a task of a particular
type the study b ecomes part of the theory of Computational Complexity also
known as Complexity Theory
Complexity Theory is a central eld of the theoretical foundations of Computer
Science It is concerned with the study of the intrinsic complexity of computational
tasks That is a typical Complexity theoretic study refers to the computational
resources required to solve a computational task or a class of such tasks rather
than referring to a sp eci c algorithm or an algorithmic schema Actually research
in Complexity Theory tends to start with and focus on the computational resources
themselves and addresses the e ect of limiting these resources on the class of
tasks that can b e solved Thus Computational Complexity is the general study
of the what can b e achieved within limited time and or other limited natural
computational resources
The most famous question of complexity theory is the P vs NP Question and
the current b o ok is fo cused on it The P vs NP Question can b e phrased as asking
whether or not nding solutions is harder than checking the correctness of solu
tions An alternative formulation asks whether or not discovering pro ofs is harder
than verifying their correctness that is is proving harder than verifying The fun
In contrast when the fo cus is on the design and analysis of sp eci c algorithms rather than
on the intrinsic complexity of the task the study b ecomes part of a related sub eld that may
b e called Algorithmic Design and Analysis Furthermore Algorithmic Design and Analysis tends
to b e sub divided according to the domain of mathematics science and engineering in which the
computational tasks arise In contrast Complexity Theory typically maintains a unity of the
study of tasks solvable within certain resources regardless of the origins of these tasks III
IV
damental nature of this question is evident in each of these formulations which
are in fact equivalent It is widely b elieved that the answer to these equivalent for
mulations is that nding resp proving is harder than checking resp verifying
that is it is b elieved that P is di erent from NP
At present when faced with a seemingly hard problem in NP we can only
hop e to prove that it is not in P assuming that NP is di erent from P This is
where the theory of NP completeness which is based on the notion of an e cient
reduction comes into the picture In general one computational problem is e
ciently reducible to another problem if it is p ossible to e ciently solve the former
when provided with an e cient algorithm for solving the latter A problem in
NP is NP complete if any problem in NP is e ciently reducible to it Amazingly
enough NP complete problems exist and furthermore hundreds of natural compu
tational problems arising in many di erent areas of mathematics and science are
NP complete
The main fo cus of the current b o ok is on the P vs NP Question and the theory
of NP completeness Additional topics that are covered include the treatment
of the general notion of an e cient reduction b etween computational problems
which provides a tighter relation b etween the aforementioned search and decision
problems The b o ok also provides adequate preliminaries regarding computational
problems and computational mo dels
Relation to a di erent b o ok of the author The current b o ok is a revision of
Chapter and Section of the author s b o ok Computational Complexity A Con
ceptual Perspective The revision was aimed at making the b o ok more friendly
to the novice In particular several pro ofs were further detailed and numerous
exercises were added
Web site for notices regarding this b o ok We intend to maintain a web site
listing corrections of various types The lo cation of the site is
http www wisdom weizmann ac il oded bc book html
Overview
This b o ok starts by providing the relevant background on computability theory
which is the setting in which complexity theoretic questions are b eing studied
Most imp ortantly this preliminary chapter i e Chapter provides a treatment
of central notions such as search and decision problems algorithms that solve
such problems and their complexity Sp ecial attention is given to the notion of a
universal algorithm
The main part of this b o ok i e Chapters is fo cused on the P vs NP
Question and on the theory of NP completeness Additional topics covered in this
part include the general notion of an e cient reduction with a sp ecial emphasis
on self reducibility the existence of problems in NP that are neither NP complete
nor in P the class coNP optimal search algorithms and promise problems A brief
overview of this main part follows
Lo osely sp eaking the P vs NP Question refers to search problems for which the
correctness of solutions can b e e ciently checked i e there is an e cient algorithm
that given a solution to a given instance determines whether or not the solution
is correct Such search problems corresp ond to the class NP and the question is
whether or not all these search problems can b e solved e ciently i e is there an
e cient algorithm that given an instance nds a correct solution Thus the P vs
NP Question can b e phrased as asking whether or not nding solutions is harder
than checking the correctness of solutions
An alternative formulation in terms of decision problems refers to assertions
that have e ciently veri able pro ofs of relatively short length Such sets of
assertions corresp ond to the class NP and the question is whether or not pro ofs
for such assertions can b e found e ciently i e is there an e cient algorithm that
given an assertion determines its validity and or nds a pro of for its validity
Thus the P vs NP Question can b e phrased as asking whether or not discovering
proofs is harder than verifying their correctness that is is proving harder than
verifying or are pro ofs valuable at all
Indeed it is widely b elieved that the answer to the two equivalent formulations
is that nding resp discovering is harder than checking resp verifying that
is that P is di erent than NP The fact that this natural conjecture is unsettled
seems to b e one of the big sources of frustration of complexity theory The author s
opinion however is that this feeling of frustration is out of place In any case at
present when faced with a seemingly hard problem in NP we cannot exp ect to V
VI
prove that the problem is not in P unconditionally The b est we can exp ect is a
conditional pro of that the said problem is not in P based on the assumption that
NP is di erent from P The contrapositive is proving that if the said problem is in
P then so is any problem in NP i e NP equals P This is where the theory of
NP completeness comes into the picture
The theory of NP completeness is based on the notion of an e cient reduction
which is a relation b etween computational problems Lo osely sp eaking one com
putational problem is e ciently reducible to another problem if it is p ossible to
e ciently solve the former when provided with an e cient algorithm for solving
the latter Thus the rst problem is not harder to solve than the second one A
problem in NP is NP complete if any problem in NP is e ciently reducible to
it Thus the fate of the entire class NP with resp ect to inclusion in P rests with
each individual NP complete problem In particular showing that a problem is
NP complete implies that this problem is not in P unless NP equals P Amazingly
enough NP complete problems exist and furthermore hundreds of natural compu
tational problems arising in many di erent areas of mathematics and science are
NP complete
The foregoing paragraphs refer to material that is covered in Chapters
Sp eci cally Chapter is devoted to the P vs NP Question p er se Chapter is
devoted to the notion of an e cient reduction and Chapter is devoted to the
theory of NP completeness We mention that that NP complete problems are not
the only seemingly hard problems in NP that is if P is di erent than NP then NP
contains problems that are neither NP complete nor in P see Section
Additional related topics are discussed in Chapter In particular in Sec
tion it is shown that the P vs NP Question is not ab out inventing sophisticated
algorithms or ruling out their existence but rather b oils down to the analysis of
a single known algorithm that is we will present an optimal search algorithm for
any problem in NP while having not clue ab out its time complexity
The b o ok also includes a brief overview of complexity theory see Epilogue and
a laconic review of some p opular computational problems see App endix
To the Teacher
According to a common opinion the most imp ortant asp ect of a scienti c work
is the technical result that it achieves whereas explanations and motivations are
merely redundancy introduced for the sake of error correction and or comfort It
is further b elieved that like in a work of art the interpretation of the work should
b e left with the reader
The author strongly disagrees with the aforementioned opinions and argues
that there is a fundamental di erence b etween art and science and that this dif
ference refers exactly to the meaning of a piece of work Science is concerned with
meaning and not with form and in its quest for truth and or understanding sci
ence follows philosophy and not art The author holds the opinion that the most
imp ortant asp ects of a scienti c work are the intuitive question that it addresses
the reason that it addresses this question the way it phrases the question the ap
proach that underlies its answer and the ideas that are embedded in the answer
Following this view it is imp ortant to communicate these asp ects of the work
The foregoing issues are even more acute when it comes to complexity theory
rstly b ecause conceptual considerations seems to play an even more central role
in complexity theory than in other scienti c elds Secondly and even more
imp ortantly complexity theory is extremely rich in conceptual content Thus
communicating this content is of primary imp ortance and failing to do so misses
the most imp ortant asp ects of complexity theory
Unfortunately the conceptual content of complexity theory is rarely communi
cated explicitly in b o oks and or surveys of the area The annoying and quite
amazing consequences are students that have only a vague understanding of the
meaning and general relevance of the fundamental notions and results that they
were taught The author s view is that these consequences are easy to avoid by tak
ing the time to explicitly discuss the meaning of de nitions and results A closely
related issue is using the right de nitions i e those that re ect b etter the fun
damental nature of the notion b eing de ned and emphasizing the conceptually
right results The current b o ok is written accordingly Two concrete and central
examples follow
We avoid non deterministic machines as much as p ossible As argued in several
places e g Section we b elieve that these ctitious machines have a negative
e ect b oth from a conceptual and technical p oint of view The conceptual damage
caused by using non deterministic machines is that it is unclear why one should VI I
VI I I
care ab out what such machines can do Needless to say the reason to care is clear
when noting that these ctitious machines o er a convenient but rather slothful
way of phrasing fundamental issues The technical damage caused by using non
deterministic machines is that they tend to confuse the students Furthermore they
do not o er the b est way to handle more advanced issues e g counting classes
In contrast to using a ctitious mo del as a pivot we de ne NP in terms of
pro of systems such that the fundamental nature of this class and the P vs NP
Question are apparent We also push to the front a formulation of the P vs NP
Question in terms of search problems We b elieve that this formulation may app eal
to non exp erts even more than the formulation of the P vs NP Question in terms
of decision problems The aforementioned formulation refers to classes of search
problems that are analogous to the decision problem classes P and NP Sp eci cally
we consider the classes PF and PC see De nitions and where PF consists
of search problems that are e ciently solvable and PC consists of search problems
having e ciently checkable solutions
To summarize we suggest presenting the P vs NP Question b oth in terms of
search problems and in terms of decision problems Furthermore when presenting
the decision problem version we suggest introducing NP by explicitly referring to
the terminology of pro of systems rather than using the more standard formulation
which is based on non deterministic machines
Finally we highlight a central recommendation regarding the presentation of
the theory of NP completeness We b elieve that from a conceptual p oint of view
the mere existence of NP complete problems is an amazing fact We thus suggest
emphasizing and discussing this fact In particular we recommend rst proving
the mere existence of NP complete problems and only later establishing the fact
that certain natural problems such as SAT are NP complete
Organization In Chapter we present the basic framework of computational
complexity which serves as a stage for the rest of the b o ok In particular we
formalize the notions of search and decision problems see Section algorithms
solving them see Section and their time complexity see Sec In
Chapter we present the two formulations of the P vs NP Question The general
notion of a reduction is presented in Chapter where we highlight its applicability
outside the domain of NP completeness Chapter is devoted to the theory of
NP completeness whereas Chapter treats three relatively advanced topics i e
the framework of promise problems the existence of optimal search algorithms
for NP and the class coNP The b o ok ends with an Epilogue which provides a
brief overview of complexity theory and an App endix that reviews some p opular
computational problems which are used as examples in the main text
Teaching note This b o ok contains many teaching notes which are typeset as the
current one
Contents
Preface III
Overview V
To the Teacher VI I
Computational Tasks and Mo dels
Representation
Computational Tasks
Search Problems
Decision Problems
Promise Problems an advanced comment
Uniform Mo dels Algorithms
Overview and General Principles
A Concrete Mo del Turing Machines
The actual mo del
The Church Turing Thesis
Uncomputable Functions
On the existence of uncomputable functions
The Halting Problem
Turing reductions
A few more undecidability results
Universal Algorithms
Time and Space Complexity
Oracle Machines
Restricted Mo dels
Non Uniform Mo dels Circuits and Advice
Bo olean Circuits
Machines That Take Advice
Restricted Mo dels
Complexity Classes
Exercises IX
X CONTENTS
The P versus NP Question
The Search Version Finding Versus Checking
The Class P as a Natural Class of Search Problems
The Class NP as Another Natural Class of Search Problems
The P Versus NP Question in Terms of Search Problems
The Decision Version Proving Versus Verifying
The Class P as a Natural Class of Decision Problems
The Class NP and NP Pro of Systems
The P Versus NP Question in Terms of Decision Problems
Equivalence of the two Formulations
Technical Comments Regarding NP
The Traditional De nition of NP
In Supp ort of P Being Di erent from NP
Philosophical Meditations
Exercises
Polynomial time Reductions
The General Notion of a Reduction
The Actual Formulation
Sp ecial Cases
Terminology and a Brief Discussion
Reducing Optimization Problems to Search Problems
Self Reducibility of Search Problems
Examples
Self Reducibility of NP Complete Problems
Digest and General Perspective
Exercises
NP Completeness
De nitions
The Existence of NP Complete Problems
Some Natural NP Complete Problems
Circuit and Formula Satis ability CSAT and SAT
The NP Completeness of CSAT
The NP Completeness of SAT
Combinatorics and Graph Theory
NP sets that are Neither in P nor NP Complete
Re ections on Complete Problems
Exercises
Three relatively advanced topics
Promise Problems
De nitions
Search problems with a promise
Decision problems with a promise
Reducibility among promise problems
CONTENTS XI
Applications
The Standard Convention of Avoiding Promise Problems
Optimal search algorithms for NP
The class coNP and its intersection with NP
Exercises
Notes
Epilogue A Brief Overview of Complexity Theory
A Some Computational Problems
A Graphs
A Bo olean Formulae
Bibliography
XI I CONTENTS
List of Figures
A single step by a Turing machine
A circuit computing f x x x x x x x x x
Recursive construction of parity circuits and formulae
Consecutive computation steps of a Turing machine
The idea underlying the reduction of CSAT to SAT
The reduction to G C the clause gadget and its sub gadget
The reduction to G C connecting the gadgets
The non generic reductions presented in Section
The world view under P coNP NP NP XI I I
Chapter
Computational Tasks and
Mo dels
This chapter provides the necessary preliminaries for the rest of the b o ok that is
we discuss the notion of a computational task and present computational mo dels
for describing metho ds for solving such tasks
We start by introducing the general framework for our discussion of computa
tional tasks or problems This framework refers to the representation of instances
as binary sequences see Section and fo cuses on two types of tasks searching
for solutions and making decisions see Section
Once computational tasks are de ned we turn to metho ds for solving such
tasks which are describ ed in terms of some model of computation The description
of such mo dels is the main contents of this chapter Sp eci cally we consider two
types of mo dels of computation uniform mo dels and non uniform mo dels see
Sections and resp ectively The uniform models correspond to the intuitive
notion of an algorithm and will provide the stage for the rest of the b o ok which
fo cuses on e cient algorithms In contrast non uniform mo dels e g Bo olean
circuits facilitate a closer lo ok at the way a computation progresses and will b e
only used sp oradically in this b o ok
Additional comments ab out the contents of this chapter Sections
corresp onds to the contents of a traditional Computability course except that
our presentation emphasizes some asp ects and deemphasizes others In particu
lar the presentation highlights the notion of a universal machine see Sec
justi es the asso ciation of e cient computation with p olynomial time algorithm
Sec and provides a de nition of oracle machines Sec This mate
rial with the exception of Kolmogorov Complexity is taken for granted in the rest
of the current b o ok In contrast Section presents basic preliminaries regard
ing non uniform mo dels of computation i e various types of Bo olean circuits
and these are only used lightly in the rest of the b o ok We also call the reader s
attention to the discussion of generic complexity classes in Section Thus
CHAPTER COMPUTATIONAL TASKS AND MODELS
whereas Sections and are absolute prerequisites for the rest of this
b o ok Section is not
Teaching note The author b elieves that there is no real need for a semester long
course in Computability i e a course that fo cuses on what can b e computed rather
than on what can b e computed e ciently Instead undergraduates should take a
course in Computational Complexity which should contain the computability asp ects
that serve as a basis for the study of e cient computation i e the rest of this course
Sp eci cally the former asp ects should o ccupy at most one third of the course and the
fo cus should b e on basic complexity issues captured by P NP and NP completeness
which may b e augmented by a selection of some more advanced material Indeed such
a course can b e based on the current b o ok p ossibly augmented by a selection of some
topics from say
Representation
In mathematics and related sciences it is customary to discuss ob jects without
sp ecifying their representation This is not p ossible in the theory of computation
where the representation of ob jects plays a central role In a sense a computation
merely transforms one representation of an ob ject to another representation of the
same ob ject In particular a computation designed to solve some problem merely
transforms the problem instance to its solution where the latter can b e though
of as a p ossibly partial representation of the instance Indeed the answer to
any fully sp eci ed question is implicit in the question itself and computation is
employed to make this answer explicit
Computational tasks refers to ob jects that are represented in some canonical
way where such canonical representation provides an explicit and full but
not overly redundant description of the corresp onding ob ject We will consider
only nite ob jects like numbers sets graphs and functions and keep distinguish
ing these types of ob jects although actually they are all equivalent While the
representation of numbers sets and functions is quite straightforward we refer the
reader to App endix A for a discussion of the representation of graphs
In order to facilitate a study of metho ds for solving computational tasks the
latter are de ned with resp ect to in nitely many p ossible instances each b eing a
nite ob ject Indeed the comparison of di erent metho ds seems to require the
consideration of in nitely many p ossible instances otherwise the choice of the lan
guage in which the metho ds are describ ed may totally dominated and even distort
the discussion cf e g the discussion of Kolmogorov Complexity in Sec
Strings We consider nite ob jects each represented by a nite binary sequence
n
called a string For a natural number n we denote by f g the set of all strings
of length n hereafter referred to as n bit long strings The set of all strings is
n
denoted f g that is f g f g For x f g we denote by jxj
n N
jxj th
the length of x i e x f g and often denote by x the i bit of x i e i
COMPUTATIONAL TASKS
x x x x For x y f g we denote by xy the string resulting from
jxj
concatenation of the strings x and y
At times we asso ciate f g f g with f g the reader should merely
consider an adequate enco ding e g the pair x x y y f g f g
m n
may b e enco ded by the string x x x x y y f g Likewise we
m m n
may represent sequences of strings of xed or varying length as single strings
When we wish to emphasize that such a sequence or some other ob ject is to b e
considered as a single ob ject we use the notation h i e g the pair x y is enco ded
as the string hx y i
Numbers Unless stated di erently natural numbers will b e enco ded by their
n
binary expansion that is the string b b b f g enco des the number
n
P
n
i
b where typically we assume that this representation has no leading
i
i
zeros i e b Rational numbers will b e represented as pairs of natural
n
numbers In the rare cases in which one considers real numbers as part of the
input to a computational problem one actually mean rational approximations of
these real numbers
Sp ecial symbols We denote the empty string by i e f g and j j
and the empty set by It will b e convenient to use some sp ecial symbols that are
not in f g One such symbol is which typically denotes an indication e g
pro duced by some algorithm that something is wrong
Computational Tasks
Two fundamental types of computational tasks are the so called search problems
and decision problems In b oth cases the key notions are the problem s instances
and the problem s sp eci cation
Search Problems
A search problem consists of a sp eci cation of a set of valid solutions p ossibly an
empty one for each p ossible instance That is given an instance one is required
to nd a corresp onding solution or to determine that no such solution exists
For example consider the problem in which one is given a system of equations
and is asked to nd a valid solution Needless to say much of computer science
is concerned with solving various search problems e g nding shortest paths in
a graph sorting a list of numbers nding an o ccurrence of a given pattern in a
given string etc Furthermore search problems corresp ond to the daily notion
of solving a problem e g nding one s way b etween two lo cations and thus a
discussion of the p ossibility and complexity of solving search problems corresp onds
to the natural concerns of most p eople
In the following de nition of solving search problems the p otential solver is a
function which may b e thought of as a solving strategy and the sets of p ossible
CHAPTER COMPUTATIONAL TASKS AND MODELS
solutions asso ciated with each of the various instances are packed into a single
binary relation
def
De nition solving a search problem Let R f g f g and R x
fy x y R g denote the set of solutions for the instance x A function f
f g f g f g solves the search problem of R if for every x the fol lowing
holds if R x then f x R x and otherwise f x
Indeed R f x y f g f g y R x g and the solver f is required to
nd a solution i e given x output y R x whenever one exists i e the set
R x is not empty It is also required that the solver f never outputs a wrong
solution i e if R x then f x R x and if R x then f x which
in turn means that f indicates whether x has any solution
A sp ecial case of interest is the case of search problems having a unique solution
for each p ossible instance that is the case that jR x j for every x In this
case R is essentially a total function and solving the search problem of R means
computing or evaluating the function R or rather the function R de ned by
def
R x y if and only if R x fy g Popular examples include sorting a sequence
of numbers multiplying integers nding the prime factorization of a comp osite
number etc
Decision Problems
A decision problem consists of a sp eci cation of a subset of the p ossible instances
Given an instance one is required to determine whether the instance is in the
sp eci ed set e g the set of prime numbers the set of connected graphs or the
set of sorted sequences For example consider the problem where one is given a
natural number and is asked to determine whether or not the number is a prime
One imp ortant case which corresp onds to the aforementioned search problems is
the case of the set of instances having a solution w r t some xed search problem
that is for any binary relation R f g f g we consider the set fx R x
g Indeed b eing able to determine whether or not a solution exists is a prerequisite
to b eing able to solve the corresp onding search problem as p er De nition
In general decision problems refer to the natural task of making binary decision
a task that is not uncommon in daily life e g determining whether a tra c light
is red In any case in the following de nition of solving decision problems the
p otential solver is again a function that is in this case the solver is a Bo olean
function which is supp osed to indicate membership in a predetermined set
De nition solving a decision problem Let S f g A function f
f g f g solves the decision problem of S or decides memb ership in S if for
every x it holds that f x if and only if x S
We often identify the decision problem of S with S itself and identify S with its
characteristic function i e with the function f g f g de ned such
S
that x if and only if x S Note that if f solves the search problem of R S
UNIFORM MODELS ALGORITHMS
def
then the Bo olean function f f g f g de ned by f x if and only if
f x solves the decision problem of fx R x g
Re ection Most p eople would consider search problems to b e more natural than
decision problems typically p eople seeks solutions more often than they stop to
wonder whether or not solutions exist De nitely search problems are not less
imp ortant than decision problems it is merely that their study tends to require
more cumbersome formulations This is the main reason that most exp ositions
choose to fo cus on decision problems The current b o ok attempts to devote at
least a signi cant amount of attention also to search problems
Promise Problems an advanced comment
Many natural search and decision problems are captured more naturally by the
terminology of promise problems in which the domain of p ossible instances is a
subset of f g rather than f g itself In particular note that the natural
formulation of many search and decision problems refers to instances of a certain
type e g a system of equations a pair of numbers a graph whereas the natural
representation of these ob jects uses only a strict subset of f g For the time
b eing we ignore this issue but we shall re visit it in Section Here we just
note that in typical cases the issue can b e ignored by p ostulating that every
string represents some legitimate ob ject e g each string that is not used in the
natural representation of these ob jects is p ostulated as a representation of some
xed ob ject
Uniform Mo dels Algorithms
We nally reach the heart of the current chapter which is the de nition of uniform
mo dels of computation Before presenting such mo dels let us brie y motivate the
need for their formal de nitions Indeed we are all familiar with computers and
with the ability of computer programs to manipulate data But this familiarity
is ro oted in p ositive exp erience that is we have some exp erience regarding some
things that computers can do In contrast complexity theory is fo cused at what
computers cannot do or rather with drawing the line b etween what can b e done
and what cannot b e done Drawing such a line requires a precise formulation of
al l p ossible computational pro cesses that is we should have a clear de nition
of al l p ossible computational pro cesses rather than some familiarity with some
computational pro cesses
Overview and General Principles
Before b eing formal let we o er a general and abstract description of the notion
of computation This description applies b oth to arti cial pro cesses taking place
in computers and to pro cesses that are aimed at mo deling the evolution of the
natural reality b e it physical biological or even so cial
CHAPTER COMPUTATIONAL TASKS AND MODELS
A computation is a pro cess that mo di es an environment via rep eated applica
tions of a predetermined rule The key restriction is that this rule is simple in each
application it dep ends and a ects only a small p ortion of the environment called
the active zone We contrast the a priori bounded size of the active zone and of
the mo di cation rule with the a priori unbounded size of the entire environment
We note that although each application of the rule has a very limited e ect the
e ect of many applications of the rule may b e very complex Put in other words a
computation may mo dify the relevant environment in a very complex way although
it is merely a pro cess of rep eatedly applying a simple rule
As hinted the notion of computation can b e used to mo del the mechanical
asp ects of the natural reality that is the rules that determine the evolution of
the reality rather than the sp eci c state of the reality at a sp eci c time In this
case the starting p oint of the study is the actual evolution pro cess that takes place
in the natural reality and the goal of the study is nding the computation rule
that underlies this natural pro cess In a sense the goal of science at large can b e
phrased as nding simple rules that govern various asp ects of reality or rather
one s abstraction of these asp ects of reality
Our fo cus however is on arti cial computation rules designed by humans in
order to achieve sp eci c desired e ects on a corresp onding arti cial environment
Thus our starting p oint is a desired functionality and our aim is to design compu
tation rules that e ect it Such a computation rule is referred to as an algorithm
Lo osely sp eaking an algorithm corresp onds to a computer program written in a
high level abstract programming language Let us elab orate
We are interested in the transformation of the environment as e ected by the
computational pro cess or the algorithm Throughout almost all of this b o ok we
will assume that when invoked on any nite initial environment the computation
halts after a nite number of steps Typically the initial environment to which
the computation is applied enco des an input string and the end environment i e
at termination of the computation enco des an output string We consider the
mapping from inputs to outputs induced by the computation that is for each
p ossible input x we consider the output y obtained at the end of a computation
initiated with input x and say that the computation maps input x to output y
Thus a computation rule or an algorithm determines a function computed by
it this function is exactly the aforementioned mapping of inputs to outputs
In the rest of this b o ok i e outside the current chapter we will also consider
the number of steps i e applications of the rule taken by the computation on
each p ossible input The latter function is called the time complexity of the com
putational pro cess or algorithm While time complexity is de ned p er input we
will often considers it p er input length taking the maximum over all inputs of the
same length
In order to de ne computation and computation time rigorously one needs
to sp ecify some mo del of computation that is provide a concrete de nition of
environments and a class of rules that may b e applied to them Such a mo del cor
resp onds to an abstraction of a real computer b e it a PC mainframe or network
of computers One simple abstract mo del that is commonly used is that of Tur
UNIFORM MODELS ALGORITHMS
ing machines see Sec Thus sp eci c algorithms are typically formalized
by corresp onding Turing machines and their time complexity is represented by
the time complexity of the corresp onding Turing machines We stress however
that almost all results in the Theory of Computation hold regardless of the sp eci c
computational mo del used as long as it is reasonable i e satis es the aforemen
tioned simplicity condition and can p erform some apparently simple computations
What is b eing computed The foregoing discussion has implicitly referred
to algorithms i e computational pro cesses as means of computing functions
Sp eci cally an algorithm A computes the function f f g f g de ned by
A
f x y if when invoked on input x algorithm A halts with output y However
A
algorithms can also serve as means of solving search problems or making de
cisions as in De nitions and Sp eci cally we will say that algorithm A
solves the search problem of R resp decides membership in S if f solves the
A
search problem of R resp decides membership in S In the rest of this exp osition
we asso ciate the algorithm A with the function f computed by it that is we write
A
A x instead of f x For sake of future reference we summarize the foregoing
A
discussion in a de nition
De nition algorithms as problem solvers We denote by A x the output
of algorithm A on input x Algorithm A solves the search problem R resp the
decision problem S if A viewed as a function solves R resp S
Organization of the rest of Section In Sec we provide a rough
description of the mo del of Turing machines This is done merely for sake of pro
viding a concrete mo del that supp orts the study of computation and its complexity
whereas the material in this b o ok will not dep end on the sp eci cs of this mo del In
Sec and Sec we discuss two fundamental prop erties of any reasonable
mo del of computation the existence of uncomputable functions and the existence
of universal computations The time and space complexity of computation is
de ned in Sec We also discuss oracle machines and restricted mo dels of
computation in Sec and Sec resp ectively
A Concrete Mo del Turing Machines
The mo del of Turing machines o er a relatively simple formulation of the notion
of an algorithm The fact that the mo del is very simple complicates the design of
machines that solve problems of interest but makes the analysis of such machines
simpler Since the fo cus of complexity theory is on the analysis of machines and not
on their design the trade o o ers by this mo del is suitable for our purp oses We
stress again that the mo del is merely used as a concrete formulation of the intuitive
notion of an algorithm whereas we actually care ab out the intuitive notion and
not ab out its formulation In particular all results mentioned in this b o ok hold for
any other reasonable formulation of the notion of an algorithm
CHAPTER COMPUTATIONAL TASKS AND MODELS
The mo del of Turing machines is not meant to provide an accurate or tight
mo del of real life computers but rather to capture their inherent limitations and
abilities i e a computational task can b e solved by a real life computer if and only
if it can b e solved by a Turing machine In comparison to real life computers the
mo del of Turing machines is extremely over simpli ed and abstract away many
issues that are of great concern to computer practice However these issues are
irrelevant to the higher level questions addressed by complexity theory Indeed as
usual go o d practice requires more re ned understanding than the one provided by
a go o d theory but one should rst provide the latter
Historically the mo del of Turing machines was invented b efore mo dern com
puters were even built and was meant to provide a concrete mo del of computation
and a de nition of computable functions Indeed this concrete mo del clari ed
fundamental prop erties of computable functions and plays a key role in de ning
the complexity of computable functions
The mo del of Turing machines was envisioned as an abstraction of the pro cess
of an algebraic computation carried out by a human using a sheet of pap er In
such a pro cess at each time the human lo oks at some lo cation on the pap er and
dep ending on what he she sees and what he she has in mind which is little
he she mo di es the contents of this lo cation and shifts his her lo ok to an adjacent
lo cation
The actual mo del
Following is a high level description of the mo del of Turing machines the interested
reader is referred to standard textb o oks e g for further details Recall
that we need to sp ecify the set of p ossible environments the set of machines or
computation rules and the e ect of applying such a rule on an environment
The environment The main comp onent in the environment of a Turing ma
chine is an in nite sequence of cells each capable of holding a single symbol i e
member of a nite set f g This sequence is envisioned as starting at a
left most cell and extending in nitely to the right cf Figure In addition
the environment contains the current lo cation of the machine on this sequence and
the internal state of the machine which is a member of a nite set Q The afore
mentioned sequence of cells is called the tap e and its contents combined with the
machine s lo cation and its internal state is called the instantaneous con guration of
the machine
The machine itself i e the computation rule The main comp onent in
the Turing machine itself is a nite rule i e a nite function called the transition
function which is de ned over the set of all p ossible symbol state pairs Sp eci cally
the transition function is a mapping from Q to Q f g where
In contrast the abstract de nition of recursive functions yields a class of computable
functions without referring to any mo del of computation but rather based on the intuition that
any such mo del should supp ort recursive functional comp osition
UNIFORM MODELS ALGORITHMS
b
5 32 2 2 3 1 0 ------
b
5 32 2 2 3 3 0 ------
Figure A single step by a Turing machine
f g corresp ond to a movement instruction which is either left or right
or stay resp ectively In addition the machine s description sp eci es an initial
state and a halting state and the computation of the machine halts when the
machine enters its halting state Envisioning the tap e as in Figure we use the
convention by which if the machine tries to move left of the end of the tap e then
it is considered to have halted
We stress that in contrast to the nite description of the machine the tap e has
an a priori unbounded length and is considered for simplicity as b eing in nite
A single application of the computation rule A single computation step of
such a Turing machine dep ends on its current lo cation on the tap e on the contents
of the corresp onding cell and on the internal state of the machine Based on the
latter two elements the transition function determines a new symbol state pair as
well as a movement instruction i e left or right or stay The machine
mo di es the contents of the said cell and its internal state accordingly and moves
as directed That is supp ose that the machine is in state q and resides in a cell
containing the symbol and supp ose that the transition function maps q to
q D Then the machine mo di es the contents of the said cell to mo di es
its internal state to q and moves one cell in direction D Figure shows a
single step of a Turing machine that when in state b and seeing a binary symbol
replaces with the symbol maintains its internal state and moves one
p osition to the right
Formally we de ne the successive con guration function which maps each in
stantaneous con guration to the one resulting by letting the machine take a single
step This function mo di es its argument in a very minor manner as describ ed
in the foregoing paragraph that is the contents of at most one cell i e at which
the machine currently resides is changed and in addition the internal state of the
machine and its lo cation may change to o
Figure corresp onds to a machine that when in the initial state i e a replaces the
symbol by mo di es its internal state to b and moves one p osition to the right Indeed
marking the leftmost cell in order to allow for recognizing it in the future is a common
practice in the design of Turing machines
CHAPTER COMPUTATIONAL TASKS AND MODELS
Initial and nal environments The initial environment or con guration of
a Turing machine consists of the machine residing in the rst i e left most cell
and b eing in its initial state Typically one also mandates that in the initial con
guration a pre x of the tap e s cells hold bit values which concatenated together
are considered the input and the rest of the tap e s cells hold a sp ecial symbol
which in Figure is denoted by Once the machine halts the output is de
ned as the contents of the cells that are to the left of its lo cation at termination
time Thus each machine de nes a function mapping inputs to outputs called
the function computed by the machine
Multi tap e Turing machines We comment that in most exp ositions one
refers to the lo cation of the head of the machine on the tap e rather than to
the lo cation of the machine on the tap e The standard terminology is more
intuitive when extending the basic mo del which refers to a single tap e to a mo del
that supp orts a constant number of tap es In the corresp onding mo del of so called
multi tap e machines the machine maintains a single head on each such tap e and
each step of the machine dep ends and e ects the cells that are at the machine s
head lo cation on each tap e The input is given on one designated tap e and the
output is required to app ear on some other designated tap e As we shall see in
Section the extension of the mo del to multi tape Turing machines is crucial
to the de nition of space complexity A less fundamental advantage of the mo del
of multi tape Turing machines is that it facilitates the design of machines that
compute functions of interest
Teaching note We strongly recommend avoiding the standard practice of teaching
the student to program with Turing machines These exercises seem very painful and
p ointless Instead one should prove that the Turing machine mo del is exactly as p ow
erful as a mo del that is closer to a real life computer see the following sanity check
that is a function can b e computed by a Turing machine if and only if it is computable
by a machine of the latter mo del For starters one may prove that a function can b e
computed by a single tap e Turing machine if and only if it is computable by a multi tape
e g two tape Turing machine
The Church Turing Thesis
The entire p oint of the mo del of Turing machines is its simplicity That is in
comparison to more realistic mo dels of computation it is simpler to formu
late the mo del of Turing machines and to analyze machines in this mo del The
Church Turing Thesis asserts that nothing is lost by considering the Turing ma
chine mo del A function can be computed by some Turing machine if and only if
it can be computed by some machine of any other reasonable and general model
of computation
By an alternative convention the machine halts while residing in the left most cell and the
output is de ned as the maximal pre x of the tap e contents that contains only bit values
UNIFORM MODELS ALGORITHMS
This is a thesis rather than a theorem b ecause it refers to an intuitive notion
i e the notion of a reasonable and general model of computation that is left unde
ned on purp ose The mo del should b e reasonable in the sense that it should allow
only computation rules that are simple in some intuitive sense For example
we should b e able to envision a mechanical implementation of these computation
rules On the other hand the mo del should allow to compute simple functions
that are de nitely computable according to our intuition At the very least the
mo del should allow to emulate Turing machines i e compute the function that
given a description of a Turing machine and an instantaneous con guration returns
the successive con guration
A philosophical comment The fact that a thesis is used to link an intuitive
concept to a formal de nition is common practice in any science or more broadly
in any attempt to reason rigorously ab out intuitive concepts Any attempt to
rigorously de ne an intuitive concept yields a formal de nition that necessarily
di ers from the original intuition and the question of corresp ondence b etween these
two ob jects arises This question can never b e rigorously treated b ecause one of
the ob jects that it relates to is unde ned That is the question of corresp ondence
b etween the intuition and the de nition always transcends a rigorous treatment
i e it always b elongs to the domain of the intuition
A sanity check Turing machines can emulate an abstract RAM To gain
con dence in the Church Turing Thesis one may attempt to de ne an abstract
Random Access Machine RAM and verify that it can b e emulated by a Turing
machine An abstract RAM consists of an in nite number of memory cells each
capable of holding an integer a nite number of similar registers one designated
as program counter and a program consisting of instructions selected from a nite
set The set of p ossible instructions includes the following instructions
reset r where r is an index of a register results in setting the value of
register r to zero
inc r where r is an index of a register results in incrementing the content
of register r Similarly dec r causes a decrement
load r r where r and r are indices of registers results in loading to
register r the contents of the memory lo cation m where m is the current
contents of register r
store r r stores the contents of register r in the memory analogously
to load
cond goto r where r is an index of a register and do es not exceed the
program length results in setting the program counter to if the content
of register r is non negative
The program counter is incremented after the execution of each instruction and
the next instruction to b e executed by the machine is the one to which the program
counter p oints and the machine halts if the program counter exceeds the program s
CHAPTER COMPUTATIONAL TASKS AND MODELS
length The input to the machine may b e de ned as the contents of the rst n
memory cells where n is placed in a sp ecial input register
We note that the abstract RAM mo del as de ned ab ove is as p owerful as
the Turing machine mo del see the following details However in order to make
the RAM mo del closer to real life computers we may augment it with additional
instructions that are available on real life computers like the instruction add r r
resp mult r r that results in adding resp multiplying the contents of reg
isters r and r and placing the result in register r We suggest proving that
this abstract RAM can be emulated by a Turing machine see Exercise We
emphasize this direction of the equivalence of the two mo dels b ecause the RAM
mo del is introduced in order to convince the reader that Turing machines are not
to o weak as a mo del of general computation The fact that they are not to o
strong seems self evident Thus it seems p ointless to prove that the RAM mo del
can emulate Turing machines Still note that this is indeed the case by using
the RAM s memory cells to store the contents of the cells of the Turing machine s
tap e and holding its head lo cation in a sp ecial register
Re ections Observe that the abstract RAM mo del is signi cantly more cum
b ersome than the Turing machine mo del Furthermore seeking a sound choice
of the instruction set i e the instructions to b e allowed in the mo del creates
a vicious cycle b ecause the sound guideline for such a choice should have b een
allowing only instructions that corresp ond to simple op erations whereas the lat
ter corresp ond to easily computable functions This vicious cycle was avoided in
the foregoing paragraph by trusting the reader to include only instructions that are
available in some real life computer We comment that this empirical considera
tion is justi able in the current context b ecause our current goal is merely linking
the Turing machine mo del with the reader s exp erience of real life computers
Uncomputable Functions
Strictly sp eaking the current subsection is not necessary for the rest of this b o ok
but we feel that it provides a useful p ersp ective
On the existence of uncomputable functions
In contrast to what every layman would think we know that not all functions are
computable Indeed an imp ortant message to b e communicated to the world is
that not every wel l de ned task can be solved by applying a reasonable automated
pro cedure i e a pro cedure that has a simple description that can b e applied to
any instance of the problem at hand Furthermore not only is it the case that
there exist uncomputable functions but it is rather the case that most functions
are uncomputable In fact only relatively few functions are computable
Theorem on the scarcity of computable functions The set of computable
functions is countable whereas the set of al l functions from strings to string has
cardinality
UNIFORM MODELS ALGORITHMS
We stress that the theorem holds for any reasonable mo del of computation In
fact it only relies on the p ostulate that each machine in the mo del has a nite
description i e can b e describ ed by a string
Pro of Since each computable function is computable by a machine that has
a nite description there is a mapping of computable functions to strings
whereas the set of all strings is in corresp ondence to the natural numbers On
the other hand there is a corresp ondence b etween the set of Bo olean functions
i e functions from strings to a single bit and the set of real number in
This corresp ondence asso ciates each real r to the function f N f g
th
such that f i is the i bit in the in nite binary expansion of r
The Halting Problem
In contrast to the discussion in Sec at this p oint we consider also machines
that may not halt on some inputs The functions computed by such machines are
partial functions that are de ned only on inputs on which the machine halts Again
we rely on the p ostulate that each machine in the mo del has a nite description
and denote the description of machine M by hM i f g The halting function
def
h f g f g f g is de ned such that h hM i x if and only if M
halts on input x The following result go es b eyond Theorem by p ointing to an
explicit function of natural interest that is not computable
Theorem undecidability of the halting problem The halting function is not
computable
The term undecidability means that the corresp onding decision problem cannot b e
solved by an algorithm That is Theorem asserts that the decision problem
asso ciated with the set h f hM i x h hM i x g is not solvable by
an algorithm i e there exists no algorithm that given a pair hM i x decides
whether or not M halts on input x Actually the following pro of shows that there
exists no algorithm that given hM i decides whether or not M halts on input hM i
Pro of We will show that even the restriction of h to its diagonal i e the func
def
tion d hM i h hM i hM i is not computable Note that the value of d hM i
refers to the question of what happ ens when we feed M with its own description
which is indeed a nasty but legitimate thing to do We will actually do some
thing worse towards the contradiction we will consider the value of d when
evaluated at a machine that is related to a hypothetical machine that supp osedly
computes d
We start by considering a related function d and showing that this function
is uncomputable The function d is de ned on purp ose so to foil any attempt to
compute it that is for every machine M the value d hM i is de ned to di er
from M hM i Sp eci cally the function d f g f g is de ned such
def
that d hM i if and only if M halts on input hM i with output That is
d hM i if either M do es not halt on input hM i or its output do es not equal
CHAPTER COMPUTATIONAL TASKS AND MODELS
the value Now supp ose towards the contradiction that d is computable by
some machine denoted M Note that machine M is supp osed to halt on every
0 0
d d
i But by de nition of d it holds that halts on input hM input and so M
0 0
d d
d hM i if and only if M halts on input hM i with output i e if and
0 0 0
d d d
only if M hM i Thus M hM i d hM i in contradiction to the
0 0 0 0 0
d d d d d
computes d hypothesis that M
0
d
We next prove that d is uncomputable and thus h is uncomputable b ecause
d z h z z for every z To prove that d is uncomputable we show that if d
is computable then so is d which we already know not to b e the case Indeed
supp ose towards the contradiction that A is an algorithm for computing d i e
A hM i d hM i for every machine M Then we construct an algorithm for
computing d which given hM i invokes A on hM i where M is de ned to
op erate as follows
On input x machine M emulates M on input x
If M halts on input x with output then M halts
If M halts on input x with an output di erent from then M enters an
in nite lo op and thus do es not halt
Otherwise i e M do es not halt on input x then machine M do es not
halt b ecause it just stays stuck in Step forever
Note that the mapping from hM i to hM i is easily computable by augmenting
M with instructions to test its output and enter an in nite lo op if necessary and
that d hM i d hM i b ecause M halts on x if and only if M halts on x with
output We thus derived an algorithm for computing d i e transform the input
hM i into hM i and output A hM i which contradicts the already established
fact by which d is uncomputable
Turing reductions
The core of the second part of the pro of of Theorem is an algorithm that
solves one problem i e computes d by using as a subroutine an algorithm that
solves another problem i e computes d or h In fact the rst algorithm is
actually an algorithmic scheme that refers to a functionally sp eci ed subroutine
rather than to an actual implementation of such a subroutine which may not
exist Such an algorithmic scheme is called a Turing reduction see formulation in
Sec Hence we have Turing reduced the computation of d to the computa
tion of d which in turn Turing reduces to h The natural p ositive meaning of
a Turing reduction of f to f is that when given an algorithm for computing f we
obtain an algorithm for computing f In contrast the pro of of Theorem uses
the unnatural negative counter positive if as we know there exists no al
gorithm for computing f d then there exists no algorithm for computing f d
which is what we wanted to prove Jumping ahead we mention that resource
b ounded Turing reductions e g p olynomial time reductions play a central role
in complexity theory itself and again they are used mostly in a negative way
We will de ne such reductions and extensively use them in subsequent chapters
UNIFORM MODELS ALGORITHMS
A few more undecidability results
We brie y review a few app ealing results regarding undecidable problems
Rice s Theorem The undecidability of the halting problem or rather the fact
that the function d is uncomputable is a sp ecial case of a more general phe
nomenon Every non trivial decision problem regarding the function computed by
a given Turing machine has no algorithmic solution We state this fact next clar
ifying the de nition of the aforementioned class of problems Again we refer to
Turing machines that may not halt on all inputs
Theorem Rice s Theorem Let F be any non trivial subset of the set of al l
computable partial functions and let S be the set of strings that describe machines
F
that compute functions in F Then deciding membership in S cannot be solved by
F
an algorithm
Theorem can b e proved by a Turing reduction from d We do not provide a pro of
b ecause this is to o remote from the main sub ject matter of the b o ok Still the
interested reader is referred to Exercise We stress that Theorems and
hold for any reasonable mo del of computation referring b oth to the p otential
solvers and to the machines the description of which is given as input to these
solvers Thus Theorem means that no algorithm can determine any non
trivial property of the function computed by a given computer program written in
any programming language For example no algorithm can determine whether or
not a given computer program halts on each possible input The relevance of this
assertion to the pro ject of program veri cation is obvious
The Post Corresp ondence Problem We mention that undecidability arises
also outside of the domain of questions regarding computing devices given as
input Sp eci cally we consider the Post Correspondence Problem in which the input
consists of two sequences of non empty strings and and
k k
the question is whether or not there exists a sequence of indices i i f k g
We stress that the length of this sequence is not such that
i i i i
1 1
a priori bounded
Theorem The Post Correspondence Problem is undecidable
Again the omitted pro of is by a Turing reduction from d or h and the interested
reader is referred to Exercise
The set S is called a non trivial subset of U if b oth S and U n S are non empty Clearly if F
is a trivial set of computable functions then the corresp onding decision problem can b e solved by
a trivial algorithm that outputs the corresp onding constant bit
In contrast the existence of an adequate sequence of a sp eci ed length can b e determined in
time that is exp onential in this length
CHAPTER COMPUTATIONAL TASKS AND MODELS
Universal Algorithms
So far we have used the p ostulate that in any reasonable mo del of computation
each machine or computation rule has a nite description Furthermore we
also used the fact that such mo del should allow for the easy mo di cation of such
descriptions such that the resulting machine computes an easily related function
see the pro of of Theorem Here we go one step further and p ostulate that the
description of machines in this mo del is e ective in the following natural sense
there exists an algorithm that given a description of a machine resp computation
rule and a corresp onding environment determines the environment that results
from p erforming a single step of this machine on this environment resp the e ect
of a single application of the computation rule This algorithm can in turn b e
implemented in the said mo del of computation assuming this mo del is general see
the Church Turing Thesis Successive applications of this algorithm leads to the
notion of a universal machine which for concreteness is formulated next in terms
of Turing machines
De nition universal machines A universal Turing machine is a Turing ma
chine that on input a description of a machine M and an input x returns the value
of M x if M halts on x and otherwise does not halt
That is a universal Turing machine computes the partial function u on pairs
hM i x such that M halts on input x in which case it holds that u hM i x
M x That is u hM i x M x if M halts on input x and u is unde ned on
hM i x otherwise We note that if M halts on all p ossible inputs then u hM i x
is de ned for every x
We stress that the mere fact that we have de ned something i e a universal
Turing machine do es not mean that it exists Yet as hinted in the foregoing dis
cussion and obvious to anyone who has written a computer program and thought
ab out what he she was doing universal Turing machines do exist
Theorem There exists a universal Turing machine
Theorem asserts that the partial function u is computable In contrast it can
b e shown that any extension of u to a total function is uncomputable That is for
any total function u that agrees with the partial function u on all the inputs on
which the latter is de ned it holds that u is uncomputable see Exercise
Pro of Given a pair hM i x we just emulate the computation of machine M
on input x This emulation is straightforward b ecause by the e ectiveness of the
description of M we can iteratively determine the next instantaneous con guration
of the computation of M on input x If the said computation halts then we will
obtain its output and can output it and so on input hM i x our algorithm
returns M x Otherwise we turn out emulating an in nite computation which
means that our algorithm do es not halt on input hM i x Thus the foregoing
emulation pro cedure constitutes a universal machine i e yields an algorithm for
computing u
UNIFORM MODELS ALGORITHMS
As hinted already the existence of universal machines is the fundamental fact
underlying the paradigm of general purp ose computers Indeed a sp eci c Turing
machine or algorithm is a device that solves a sp eci c problem A priori solving
each problem would have required building a new physical device that allows for
this problem to b e solved in the physical world rather than as a thought exp eri
ment The existence of a universal machine asserts that it is enough to build one
physical device that is a general purp ose computer Any sp eci c problem can
then b e solved by writing a corresp onding program to b e executed or emulated
by the general purp ose computer Thus universal machines corresp ond to general
purp ose computers and provide the basis for separating hardware from software
Furthermore the existence of universal machines says that software can b e viewed
as part of the input
In addition to their practical imp ortance the existence of universal machines
and their variants has imp ortant consequences in the theories of computability
and computational complexity To demonstrate the p oint we note that Theo
rem implies that many questions ab out the b ehavior of a xed universal ma
chine on certain input types are undecidable For example it follows that for
some xed machines i e universal ones there is no algorithm that determines
whether or not the xed machine halts on a given input Revisiting the pro of of
Theorem see Exercise it follows that the Post Corresp ondence Problem
remains undecidable even if the input sequences are restricted to have a sp eci c
length i e k is xed A more imp ortant application of universal machines to the
theory of computability follows
A detour Kolmogorov Complexity The existence of universal machines
which may b e viewed as universal languages for writing e ective and succinct
descriptions of ob jects plays a central role in Kolmogorov Complexity Lo osely
sp eaking the latter theory is concerned with the length of e ective descriptions
of ob jects and views the minimum such length as the inherent complexity of the
ob ject that is simple ob jects or phenomena are those having short description
resp short explanation whereas complex ob jects have no short description
Needless to say these e ective descriptions have to refer to some xed language
i e to a xed machine that given a succinct description of an ob ject pro duces
its explicit description Fixing any machine M a string x is called a description
of s with resp ect to M if M x s The complexity of s with resp ect to M de
noted K s is the length of the shortest description of s with resp ect to M
M
Certainly we want to x M such that every string has a description with resp ect
to M and furthermore such that this description is not signi cantly longer than
the description with resp ect to a di erent machine M The following theorem
make it natural to use a universal machine as the p oint of reference i e as the
aforementioned M
Theorem complexity w r t a universal machine Let U be a universal ma
chine Then for every machine M there exists a constant c such that K s
U
0
s c for every string s K M
CHAPTER COMPUTATIONAL TASKS AND MODELS
The theorem follows by setting c O jhM ij and observing that if x is a de
scription of s with resp ect to M then hM i x is a description of s with resp ect
to U Here it is imp ortant to use an adequate enco ding of pairs of strings e g
the pair is enco ded by the string Fix
k k k
ing any universal machine U we de ne the Kolmogorov Complexity of a string s as
def
K s K s The reader may easily verify the following facts
U
K s jsj O for every s
Hint apply Theorem to a machine that computes the identity map
ping
There exist in nitely many strings s such that K s jsj
n
Hint consider s Alternatively consider any machine M such that
jM x j jxj for every x
Some strings of length n have complexity at least n Furthermore for every
n and i
n n i
jfs f g K s n igj
Hint di erent strings must have di erent descriptions with resp ect to U
It can b e shown that the function K is uncomputable see Exercise The
pro of is related to the paradox captured by the following description of a nat
ural number the smallest natural number that can not be described by
an English sentence of up to a thousand letters The paradox amounts
to observing that if the foregoing number is well de ned then we reach contradic
tion by noting that the foregoing sentence uses less than one thousand letters
Needless to say the foregoing sentence presupp oses that any English sentence is
a legitimate description in some adequate sense e g in the sense captured by
Kolmogorov Complexity Sp eci cally the foregoing sentence presupp oses that we
can determine the Kolmogorov Complexity of each natural number and thus that
we can e ectively pro duce the smallest number that has Kolmogorov Complexity
exceeding some threshold by relying on the fact that natural numbers have arbi
trary large Kolmogorov Complexity Indeed the paradox suggests a pro of to the
fact that the latter task cannot b e p erformed that is there exists no algorithm
that given t produces the lexicographically rst string s such that K s t b e
cause if such an algorithm A would have existed then K s O jhAi j log t in
contradiction to the de nition of s
Time and Space Complexity
Fixing a mo del of computation e g Turing machines and focusing on algorithms
that halt on each input we consider the number of steps i e applications of
the computation rule taken by the algorithm on each p ossible input The lat
ter function is called the time complexity of the algorithm or machine that is
t f g N is called the time complexity of algorithm A if for every x on
A
input x algorithm A halts after exactly t x steps A
UNIFORM MODELS ALGORITHMS
We will b e mostly interested in the dep endence of the time complexity on the
input length when taking the maximum over all inputs of the relevant length
That is for t as in the foregoing paragraph we will consider T N N de ned
A A
def
n
by T n max ft x g Abusing terminology we sometimes refer to T
A A A
x f g
as the time complexity of A
The time complexity of a problem As stated in the preface typically com
plexity theory is not concerned with the time complexity of a sp eci c algorithm
It is rather concerned with the time complexity of a problem assuming that this
problem is solvable at all by some algorithm Intuitively the time complexity of
such a problem is de ned as the time complexity of the fastest algorithm that solves
this problem assuming that the latter term is well de ned Actually we shall b e
interested in upp er and lower bounds on the time complexity of algorithms that
solve the problem Thus when we say that a certain problem has complexity T
we actually mean that has complexity at most T Likewise when we say that
requires time T we actually mean that has time complexity at least T
Recall that the foregoing discussion refers to some xed mo del of computa
tion Indeed the complexity of a problem may dep end on the sp eci c mo del
of computation in which algorithms that solve are implemented The following
Cobham Edmonds Thesis asserts that the variation in the time complexity is not
to o big and in particular is irrelevant to much of the current fo cus of complexity
theory e g for the P vs NP Question
The Cobham Edmonds Thesis As just stated the time complexity of a prob
lem may dep end on the mo del of computation For example deciding membership
in the set fxx x f g g can b e done in linear time on a two tape Turing
machine but requires quadratic time on a single tap e Turing machine see Exer
cise On the other hand any problem that has time complexity t in the mo del
of multi tape Turing machines has complexity O t in the mo del of single tap e
Turing machines The Cobham Edmonds Thesis asserts that the time complexities
in any two reasonable and general mo dels of computation are p olynomially re
lated That is a problem has time complexity t in some reasonable and general
model of computation if and only if it has time complexity p oly t in the model of
single tap e Turing machines
Indeed the Cobham Edmonds Thesis strengthens the Church Turing Thesis
It asserts not only that the class of solvable problems is invariant as far as rea
sonable and general mo dels of computation are concerned but also that the time
complexity of the solvable problems in such mo dels is p olynomially related
E cient algorithms As hinted in the foregoing discussions much of complexity
theory is concerned with e cient algorithms The latter are de ned as p olynomial
Advanced comment We note that the naive assumption that a fastest algorithm for
solving a problem exists is not always justi ed see Sec On the other hand the
assumption is essentially justi ed in some imp ortant cases see e g Theorem But even in
these cases the said algorithm is fastest or optimal only up to a constant factor
CHAPTER COMPUTATIONAL TASKS AND MODELS
time algorithms i e algorithms that have time complexity that is upp er b ounded
by a p olynomial in the length of the input By the Cobham Edmonds Thesis the
de nition of this class is invariant under the choice of a reasonable and general
mo del of computation The asso ciation of e cient algorithms with p olynomial time
computation is grounded in the following two considerations
Philosophical consideration Intuitively e cient algorithms are those that
can b e implemented within a number of steps that is a mo derately growing
function of the input length To allow for reading the entire input at least
linear time should b e allowed On the other hand apparently slow algorithms
and in particular exhaustive search algorithms which take exp onential
time must b e avoided Furthermore a go o d de nition of the class of e cient
algorithms should b e closed under natural comp osition of algorithms as well
as b e robust with resp ect to reasonable mo dels of computation and with
resp ect to simple changes in the enco ding of problems instances
Cho osing p olynomials as the set of time b ounds for e cient algorithms sat
isfy all the foregoing requirements p olynomials constitute a closed set of
mo derately growing functions where closure means closure under addition
multiplication and functional comp osition These closure prop erties guaran
tee the closure of the class of e cient algorithm under natural comp osition
of algorithms as well as its robustness with resp ect to any reasonable and
general mo del of computation Furthermore p olynomial time algorithms
can conduct computations that are apparently simple although not neces
sarily trivial and on the other hand they do not include algorithms that are
apparently ine cient like exhaustive search
Empirical consideration It is clear that algorithms that are considered e
cient in practice have running time that is b ounded by a small p olynomial
at least on the inputs that o ccur in practice The question is whether any
p olynomial time algorithm can b e considered e cient in an intuitive sense
The b elief which is supp orted by past exp erience is that every natural prob
lem that can b e solved in p olynomial time also has a reasonably e cient
algorithm
We stress that the asso ciation of e cient algorithms with p olynomial time compu
tation is not essential to most of the notions results and questions of complexity
theory Any other class of algorithms that supp orts the aforementioned closure
prop erties and allows to conduct some simple computations but not overly com
plex ones gives rise to a similar theory alb eit the formulation of such a theory
may b e more complicated Sp eci cally all results and questions treated in this
b o ok are concerned with the relation among the complexities of di erent computa
tional tasks rather than with providing absolute assertions ab out the complexity
of some computational tasks These relations can b e stated explicitly by stating
how any upp er b ound on the time complexity of one task gets translated to an
upp er b ound on the time complexity of another task Such cumbersome state
For example the NP completeness of SAT cf Theorem implies that any algorithm
solving SAT in time T yields an algorithm that factors comp osite numbers in time T such that
UNIFORM MODELS ALGORITHMS
ments will maintain the contents of the standard statements they will merely b e
much more complicated Thus we follow the tradition of fo cusing on p olynomial
time computations while stressing that this fo cus is b oth natural and provides the
simplest way of addressing the fundamental issues underlying the nature of e cient
computation
Universal machines revisited The notion of time complexity gives rise to a
time b ounded version of the universal function u presented in Sec Sp eci
def
cally we de ne u hM i x t y if on input x machine M halts within t steps and
def
outputs the string y and u hM i x t if on input x machine M makes more
than t steps Unlike u the function u is a total function Furthermore unlike any
extension of u to a total function the function u is computable Moreover u is
computable by a machine U that on input X hM i x t halts after p oly jX j
steps Indeed machine U is a variant of a universal machine i e on input X ma
chine U merely emulates M for t steps rather than emulating M till it halts and
p otentially inde nitely Note that the number of steps taken by U dep ends on
the sp eci c mo del of computation and that some overhead is unavoidable b ecause
emulating each step of M requires reading the relevant p ortion of the description
of M
Space complexity Another natural measure of the complexity of an algo
rithm or a task is the amount of memory consumed by the computation We
refer to the memory used for storing some intermediate results of the computation
Since computations that utilize memory that is sub linear in their input length are
of natural interest it is imp ortant to use a mo del in which one can di erentiate
memory used for computation from memory used for storing the initial input or
the nal output In the context of Turing machines this is done by considering
multi tape Turing machines such that the input is presented on a sp ecial read only
tap e called the input tap e the output is written on a sp ecial write only tap e
called the output tap e and intermediate results are stored on a work tape Thus
the input and output tap es cannot b e used for storing intermediate results The
space complexity of such a machine M is de ned as a function s such that s x
M M
is the number of cells of the work tape that are scanned by M on input x As in
def
n
the case of time complexity we will usually refer to S n max fs x g
A A
x f g
Oracle Machines
The notion of Turing reductions which was discussed in Sec is captured by
the following de nition of so called oracle machines Lo osely sp eaking an oracle
machine is a machine that is augmented such that it may p ose questions to the
T n p oly n T p oly n More generally if the correctness of solutions for n bit
instances of some search problem R can b e veri ed in time t n then the hypothesis regarding
SAT implies that solutions for n bit instances of R can b e found in time T such that T n
t n T O t n
CHAPTER COMPUTATIONAL TASKS AND MODELS
outside We consider the case in which these questions called queries are answered
consistently by some function f f g f g called the oracle That is if
the machine makes a query q then the answer it obtains is f q In such a case we
say that the oracle machine is given access to the oracle f For an oracle machine
f
M a string x and a function f we denote by M x the output of M on input
x when given access to the oracle f Re examining the second part of the pro of
of Theorem observe that we have actually describ ed an oracle machine that
computes d when given access to the oracle d
The notion of an oracle machine extends the notion of a standard computing
device machine and thus a rigorous formulation of the former extends a formal
mo del of the latter Sp eci cally extending the mo del of Turing machines we derive
the following mo del of oracle Turing machines
De nition using an oracle
An oracle machine is a Turing machine with a special additional tape called
the oracle tap e and two special states called oracle invo cation and oracle sp oke
The computation of the oracle machine M on input x and access to the oracle
f f g f g is de ned based on the successive con guration function
For con gurations with state di erent from oracle invocation the next con g
uration is de ned as usual Let be a con guration in which the machine s
state is oracle invocation and suppose that the actual contents of the oracle
tape is q i e q is the contents of the maximal pre x of the tape that holds
bit values Then the con guration fol lowing is identical to except that
the state is oracle sp oke and the actual contents of the oracle tape is f q
The string q is called M s query and f q is called the oracle s reply
The output of the oracle machine M on input x when given oracle access to
f
f is denote M x
We stress that the running time of an oracle machine is the number of steps made
during its own computation and that the oracle s reply on each query is obtained
in a single step
Restricted Mo dels
We mention that restricted mo dels of computation are often mentioned in the
context of a course on computability but they will play no role in the current b o ok
One such mo del is the mo del of nite automata which in some variant coincides
with Turing machines that have space complexity zero equiv constant
In our opinion the most imp ortant motivation for the study of these restricted
mo dels of computation is that they provide simple mo dels for some natural or
arti cial phenomena This motivation however seems only remotely related to
This ts the de nition of the actual initial contents of a tape of a Turing machine cf
Sec A common convention is that the oracle can b e invoked only when the machine s
head resides at the left most cell of the oracle tap e
NON UNIFORM MODELS CIRCUITS AND ADVICE
the study of the complexity of various computational tasks which calls for the con
sideration of general mo dels of computation and the evaluation of the complexity
of computation with resp ect to such mo dels
Teaching note Indeed we reject the common coupling of computability theory with
the theory of automata and formal languages Although the historical links b etween
these two theories at least in the West can not b e denied this fact cannot justify
coupling two fundamentally di erent theories esp ecially when such a coupling promotes
a wrong p ersp ective on computability theory Thus in our opinion the study of any
of the lower levels of Chomsky s Hierarchy Chap should b e decoupled from the
study of computability theory let alone the study of complexity theory
Non Uniform Mo dels Circuits and Advice
In the current b o ok we only use non uniform mo dels of computation as a source of
some natural computational problems cf Sec We mention however that
these mo dels are typically considered for other purp oses see a brief discussion
b elow
By a non uniform mo del of computation we mean a mo del in which for each
p ossible input length a di erent computing device is considered while there is
no uniformity requirement relating devices that corresp ond to di erent input
lengths Furthermore this collection of devices is in nite by nature and in absence
of a uniformity requirement this collection may not even have a nite description
Nevertheless each device in the collection has a nite description In fact the
relationship b etween the size of the device resp the length of its description and
the length of the input that it handles will b e of ma jor concern
Non uniform mo dels of computation are considered either towards the devel
opment of lower bound techniques or as providing simpli ed upp er b ounds on the
ability of e cient algorithms In b oth cases the uniformity condition is eliminated
in the interest of simplicity and with the hop e and b elief that nothing substantial
is lost as far as the issues at hand are concerned In the context of developing lower
b ound the hop e is that the niteness of all parameters i e the input length and
the device s description will allow for the application of combinatorial techniques
to analyze the limitations of certain settings of parameters
We will fo cus on two related mo dels of non uniform computing devices Bo olean
circuits Sec and machines that take advice Sec The former mo del
is more adequate for the study of the evolution of computation i e development of
lower bound techniques whereas the latter is more adequate for mo deling purp oses
e g limiting the ability of e cient algorithms
Advanced comment The second case refers mainly to e cient algorithms that are given
a pair of inputs of p olynomially related length such that these algorithms are analyzed with
resp ect to xing one input arbitrarily and varying the other input typically at random Typical
examples include the context of de randomization cf Sec and the setting of zero
knowledge cf Sec
CHAPTER COMPUTATIONAL TASKS AND MODELS
Teaching note In the context of this b o ok non uniform mo dels of computation will
only b e used for giving rise to natural computational problems e g the satis ability
of Bo olean Circuits cf Sec and Formulae cf Sec This use provides a
concrete motivation to the study of the current section furthermore we b elieve that
some familiarity with the non uniform mo dels is b ene cial p er se
Bo olean Circuits
The most p opular mo del of non uniform computation is the one of Bo olean circuits
Historically this mo del was introduced for the purp ose of describing the logic
op eration of real life electronic circuits Ironically nowadays this mo del provides
the stage for some of the most practically removed studies in complexity theory
which aim at developing metho ds that may eventually lead to an understanding
of the inherent limitations of e cient algorithms
A Bo olean circuit is a directed acyclic graph with labels on the vertices to b e
discussed shortly For sake of simplicity we disallow isolated vertices i e vertices
with no incoming or outgoing edges and thus the graph s vertices are of three
types sources sinks and internal vertices
Internal vertices are vertices having incoming and outgoing edges i e they
have in degree and out degree at least In the context of Bo olean cir
cuits internal vertices are called gates Each gate is lab eled by a Bo olean
op eration where the op erations that are typically considered are and
corresp onding to and or and neg In addition we require that gates la
b eled have in degree The in degree of gates and gates may b e any
number greater than zero and the same holds for the outdegree of any gate
The graph sources i e vertices with no incoming edges are called input
terminals Each input terminal is lab eled by a natural number which is to
b e thought of the index of an input variable For sake of de ning formulae
see Sec we allow di erent input terminals to b e lab eled by the same
number
The graph sinks i e vertices with no outgoing edges are called output ter
minals and we require that they have in degree Each output terminal is
lab eled by a natural number such that if the circuit has m output terminals
then they are lab eled m That is we disallow di erent output ter
minals to b e lab eled by the same number and insist that the lab els of the
output terminals are consecutive numbers Indeed the lab els of the output
terminals will corresp ond to the indices of lo cations in the circuit s output
For sake of simplicity we also mandate that the lab els of the input terminals are
consecutive numbers
See App endix A
This is not needed in case of general circuits b ecause we can just feed outgoing edges of the
same input terminal to many gates Note however that this is not allowed in case of formulae
NON UNIFORM MODELS CIRCUITS AND ADVICE
1 2 4 3
neg neg neg
and and and
and 0 or or
1 2
Figure A circuit computing f x x x x x x x x x
A Bo olean circuit with n di erent input lab els and m output terminals induces
n m
and indeed computes a function from f g to f g de ned as follows For
n
any xed string x f g we iteratively de ne the value of vertices in the circuit
such that the input terminals are assigned the corresp onding bits in x x x
n
and the values of other vertices are determined in the natural manner That is
th
An input terminal with lab el i f ng is assigned the i bit of x i e
the value x
i
If the children of a gate of in degree d that is lab eled have values v v v
d
d
then the gate is assigned the value v The value of a gate lab eled or
i
i
is determined analogously
Indeed the hypothesis that the circuit is acyclic implies that the following
natural pro cess of determining values for the circuit s vertices is well de ned
As long as the value of some vertex is undetermined there exists a vertex
such that its value is undetermined but the values of all its children are
determined Thus the pro cess can make progress and terminates when the
values of all vertices including the output terminals are determined
The value of the circuit on input x i e the output computed by the circuit on
input x is y y y where y is the value assigned by the foregoing pro cess
m i
where all non sinks are required to have out degree exactly
This convention slightly complicates the construction of circuits that ignore some of the input
values Sp eci cally we use arti cial gadgets that have incoming edges from the corresp onding
input terminals and compute an adequate constant To avoid having this constant as an output
terminal we feed it into an auxiliary gate such that the value of the latter is determined by the
other incoming edge e g a constant fed into an gate See example of dealing with x in
Figure
CHAPTER COMPUTATIONAL TASKS AND MODELS
to the output terminal lab eled i We note that there exists a polynomial time
algorithm that given a circuit C and a corresponding input x outputs the value of
C on input x This algorithm determines the values of the circuit s vertices going
from the circuit s input terminals to its output terminals
We say that a family of circuits C computes a function f f g f g
n
n N
if for every n the circuit C computes the restriction of f to strings of length n In
n
other words for every x f g it must hold that C x f x
jxj
Bounded and unbounded fan in One is often interested in circuits in which
each gate has at most two incoming edges In this case the types of two argument
Bo olean op erations that we allow is immaterial as long as we consider a full
basis of such op erations i e a set of op erations that can implement any other
two argument Bo olean op eration Such circuits are called circuits of b ounded fan
in In contrast other studies are concerned with circuits of unb ounded fan in where
each gate may have an arbitrary number of incoming edges Needless to say in
the case of circuits of unbounded fan in the choice of allowed Bo olean op erations
is imp ortant and one fo cuses on op erations that are uniform across the number
of op erants e g and
Circuit size as a complexity measure The size of a circuit is the number of
its edges When considering a family of circuits C that computes a function
n
n N
f f g f g we are interested in the size of C as a function of n
n
Sp eci cally we say that this family has size complexity s N N if for every n the
size of C is s n The circuit complexity of a function f denoted s is the in mum
n f
of the size complexity of all families of circuits that compute f Alternatively for
each n we may consider the size of the smallest circuit that computes the restriction
of f to n bit strings denoted f and set s n accordingly We stress that non
n f
uniformity is implicit in this de nition b ecause no conditions are made regarding
the relation b etween the various circuits used to compute the function on di erent
input lengths
On the circuit complexity of functions We highlight some simple facts re
garding the circuit complexity of functions These facts are in clear corresp ondence
to facts regarding Kolmogorov Complexity mentioned in Sec and establishing
them is left as an exercise see Exercise
Most imp ortantly any Bo olean function can b e computed by some family
of circuits and thus the circuit complexity of any function is well de ned
Furthermore each function has at most exp onential circuit complexity
Some functions have p olynomial circuit complexity In particular any func
tion that has time complexity t i e is computed by an algorithm of time
Advanced comment We also note that in contrast to Footnote the circuit mo del and
the circuit size complexity measure supp ort the notion of an optimal computing device each
function f has a unique size complexity s and not merely upp er and lower bounds on its
f complexity
NON UNIFORM MODELS CIRCUITS AND ADVICE
complexity t has circuit complexity p oly t Furthermore the corresp ond
ing circuit family is uniform in a natural sense to b e discussed in the next
paragraph
Almost all Bo olean functions have exp onential circuit complexity Sp eci
n
cally the number of functions mapping f g to f g that can b e computed
s
by some circuit of size s is smaller than s
Note that the rst fact implies that families of circuits can compute functions that
are uncomputable by algorithms Furthermore this phenomenon o ccurs also when
restricting attention to families of p olynomial size circuits See further discussion
in Sec
Uniform families A family of p olynomial size circuits C is called uniform
n
n N
if given n one can construct the circuit C in p oly n time Note that if a function
n
is computable by a uniform family of polynomial size circuits then it is computable
by a polynomial time algorithm This algorithm rst constructs the adequate cir
cuit which can b e done in p olynomial time by the uniformity hypothesis and
then evaluate this circuit on the given input which can b e done in time that is
p olynomial in the size of the circuit
Note that limitations on the computing p ower of arbitrary families of p olynomial
size circuits certainly hold for uniform families of p olynomial size circuits which
in turn yield limitations on the computing p ower of p olynomial time algorithms
Thus lower bounds on the circuit complexity of functions yield analogous lower
b ounds on their time complexity Furthermore as is often the case in mathematics
and science disp osing of an auxiliary condition that is not well understoo d i e
uniformity may turn out fruitful Indeed this has o ccured in the study of classes
of restricted circuits which is reviewed in Sec
Machines That Take Advice
General non uniform circuit families and uniform circuit families are two extremes
with resp ect to the amounts of non uniformity in the computing device Intu
itively in the former non uniformity is only b ounded by the size of the device
whereas in the latter the amounts of non uniformity is zero Here we consider a
mo del that allows to decouple the size of the computing device from the amount
of non uniformity which may range from zero to the device s size Sp eci cally we
consider algorithms that take a non uniform advice that dep ends only on the
input length The amount of non uniformity will b e de ned to equal the length of
the corresp onding advice as a function of the input length
De nition taking advice We say that algorithm A computes the function
f using advice of length N N if there exists an in nite sequence a such
n
n N
that
For every x f g it holds that A a x f x
jxj
For every n N it holds that ja j n n
CHAPTER COMPUTATIONAL TASKS AND MODELS
The sequence a is called the advice sequence
n
n N
Note that any function having circuit complexity s can b e computed using advice
of length O s log s where the log factor is due to the fact that a graph with v
vertices and e edges can b e describ ed by a string of length e log v Note that the
mo del of machines that use advice allows for some sharp er b ounds than the ones
stated in Sec every function can b e computed using advice of length such
n
that n and some uncomputable functions can b e computed using advice
of length
Theorem the p ower of advice There exist functions that can be computed
using one bit advice but cannot be computed without advice
Pro of Starting with any uncomputable Bo olean function f N f g consider
the function f de ned as f x f jxj Note that f is Turing reducible to f e g
on input n make any n bit query to f and return the answer Thus f cannot b e
computed without advice On the other hand f can b e easily computed by using
the advice sequence a such that a f n that is the algorithm merely
n n
n N
outputs the advice bit and indeed a f jxj f x for every x f g
jxj
Restricted Mo dels
The mo del of Bo olean circuits cf Sec allows for the introduction of many
natural sub classes of computing devices Following is a laconic review of a few
of these sub classes For further detail regarding the study of these sub classes
the interested reader is referred to Since we shal l refer to various types of
Boolean formulae in the rest of this book we suggest not to skip the fol lowing two
paragraphs
Bo olean formulae In general Bo olean circuits the non sink vertices are al
lowed arbitrary out degree This means that the same intermediate value can b e
re used without b eing re computed and while increasing the size complexity by
only one unit Such free re usage of intermediate values is disallowed in Bo olean
formulae which are formally de ned as Bo olean circuits in which all non sink ver
tices have out degree This means that the underlying graph of a Bo olean formula
is a tree see xA and it can b e written as a Bo olean expression over Bo olean
variables by traversing this tree and registering the vertices lab els in the order tra
versed Indeed we have allowed di erent input terminals to b e assigned the same
lab el in order to allow formulae in which the same variable o ccurs multiple times
As in case of general circuits one is interested in the size of these restricted circuits
i e the size of families of formulae computing various functions We mention that
quadratic lower b ounds are known for the formula size of simple functions e g
parity whereas these functions have linear circuit complexity This discrepancy
is depicted in Figure
Indeed this Turing reduction is not e cient i e it runs in exp onential time in jnj log n
but this is immaterial in the current context
NON UNIFORM MODELS CIRCUITS AND ADVICE
PARITY PARITY PARITY PARITY PARITY PARITY of x .... x of x ...x of x .... x of x ...x of x .... x of x ...x 1 n n+1 2n 1 n n+1 2n 1 n n+1 2n
neg neg neg neg
and and and and
or
or
Figure Recursive construction of parity circuits and formulae
Formulae in CNF and DNF A restricted type of Bo olean formulae consists
of formulae that are in conjunctive normal form CNF Such a formula consists of
a conjunction of clauses where each clause is a disjunction of literals each b eing
either a variable or its negation That is such formulae are represented by layered
circuits of unbounded fan in in which the rst layer consists of neg gates that
compute the negation of input variables the second layer consist of or gates that
compute the logical or of subsets of inputs and negated inputs and the third layer
consists of a single and gate that computes the logical and of the values computed
in the second layer Note that each Bo olean function can b e computed by a family
of CNF formulae of exp onential size see Exercise and that the size of CNF
formulae may b e exp onentially larger than the size of ordinary formulae computing
the same function e g parity For a constant k a formula is said to b e in k
CNF if its CNF has disjunctions of size at most k An analogous restricted type
of Bo olean formulae refers to formulae that are in disjunctive normal form DNF
Such a formula consists of a disjunction of a conjunctions of literals and when each
conjunction has at most k literals we say that the formula is in k DNF
Constant depth circuits Circuits have a natural structure i e their struc
ture as graphs One natural parameter regarding this structure is the depth of a
circuit which is de ned as the longest directed path from any source to any sink Of
sp ecial interest are constant depth circuits of unbounded fan in We mention that
sub exp onential lower b ounds are known for the size of such circuits that compute
a simple function e g parity
Monotone circuits The circuit mo del also allows for the consideration of mono
tone computing devices a monotone circuit is one having only monotone gates
e g gates computing and but no negation gates i e gates Needless
to say monotone circuits can only compute monotone functions where a function
n
f f g f g is called monotone if for any x y it holds that f x f y
where x x y y if and only if for every bit p osition i it holds that
n n
x y One natural question is whether as far as monotone functions are con
i i
CHAPTER COMPUTATIONAL TASKS AND MODELS
cerned there is a substantial loss in using only monotone circuits The answer is
yes there exist monotone functions that have p olynomial circuit complexity but
require sub exp onential size monotone circuits
Complexity Classes
Complexity classes are sets of computational problems Typically such classes are
de ned by xing three parameters
A type of computational problems see Section Indeed most classes refer
to decision problems but classes of search problems promise problems and
other types of problems are also considered
A model of computation which may b e either uniform see Section or
non uniform see Section
A complexity measure and a limiting function or a set of functions which
put together limit the class of computations of the previous item that is
we refer to the class of computations that have complexity not exceeding the
sp eci ed function or set of functions For example in Sec we men
tioned time complexity and space complexity which apply to any uniform
mo del of computation We also mentioned p olynomial time computations
which are computations in which the time complexity as a function do es
not exceed some p olynomial i e is a member of the set of p olynomial func
tions
The most common complexity classes refer to decision problems and are sometimes
de ned as classes of sets rather than classes of the corresp onding decision problems
That is one often says that a set S f g is in the class C rather than saying
that the problem of deciding membership in S is in the class C Likewise one talks
of classes of relations rather than classes of the corresp onding search problems i e
saying that R f g f g is in the class C means that the search problem of
R is in the class C
Exercises
Exercise Prove that any function that can b e computed by a Turing machine
can b e computed by a machine that never moves left of the end of the tap e
Guideline Mo dify the original machine by using a marking of the leftmost cell of the
tap e Needless to say this marking corresp onds to an extension of the tap e s symbols
Exercise single tap e versus multi tape Turing machines Prove that
a function can b e computed by a single tap e Turing machine if and only if it is
computable by a multi tape e g two tape Turing machine
Guideline The emulation of the multi tape Turing machine on a single tap e machine is
th
based on storing all the original tap es on a single tap e such that the i cell of the single
COMPLEXITY CLASSES
th th
tap e records the contents of the i cell of each of the original tap es In addition the i
cell of the single tap e records an indication as to which of the original heads reside in
th
the i cell of the corresp onding original tap e To emulate a single step of the original
machine the new machine scans its tap e nds all original head lo cations and retrieves
the corresp onding cell contents Based on this information the emulating machine e ects
the corresp onding step according to the original transition function by mo difying its
single tap e s contents in an analogous manner
Exercise computing the sum of natural numbers Prove that a Tur
ing machine can add natural numbers that is outline a multi tape Turing ma
chine that on input a pair of integers in binary representation outputs their sum
Sp eci cally show that the straightforward addition algorithm can b e implemented
in linear time by a multi tape Turing machine
Guideline A straightforward implementation of addition on a two tape Turing machine
th
starts by copying the two input integers to the second tap e such that the i least
signi cant bits of b oth integers resides in the same cell
Exercise Turing machines vs abstract RAM Prove that abstract RAMs
can b e emulated by a Turing machine
Guideline Note that during the emulation of the abstract RAM we only need to
hold the input the contents of all registers and the contents of the memory cells that
were accessed during the computation Thus at each time the Turning machine s tap e
contains a list of the RAM s memory cells that were accessed so far as well as their current
contents When we emulate a RAM instruction we rst check whether the relevant RAM
cell app ears on this list and augment the list by a corresp onding entry or mo dify this
entry as needed
Exercise Rice s Theorem Theorem Let F and S b e as in The
F
orem Present a Turing reduction of d to S
F
Guideline Let f denote the function that is unde ned on all inputs Assume without
loss of generality that f F let f denote an arbitrary function in F and let M b e
an arbitrary x machine that computes f Then the reduction maps an input hM i for
d into an input hM i for S such that on input x machine M op erates as follows
F
First machine M emulates M on input hM i
If M halts in Step then M emulates M x and outputs whatever it do es
Note that the mapping from hM i to hM i is easily computable by augmenting M with
the xed machine M If h hM i then machine M reaches Step and thus M x
f x which in turn implies hM i S On the other hand if h hM i then machine
F
M remains stuck in Step and thus M do es not halt on any x which in turn implies
hM i S b ecause M computes f
F
Exercise Post Corresp ondence Problem Theorem Present a Turing
reduction of h to the Post Corresp ondence Problem denoted PCP Furthermore use
a reduction that maps an instance hM i x of h to a pair of sequences
k k
CHAPTER COMPUTATIONAL TASKS AND MODELS
such that only and dep end on x whereas k as well as the other strings dep end
only on M
Guideline Consider a a mo di ed version of the Post Corresp ondence Problem denoted
MPCP in which the rst index in the solution sequence must equal i e i Reduce h
to MPCP and next reduce MPCP to PCP The main reduction i e of h to MPCP maps hM i x
to such that a solution sequence i e i i s t
i i
k k
1
yields a full description of the computation of M on input x i e the sequence
i
of all instantaneous con gurations in this computation Sp eci cally will describ e the
initial con guration of M on input x whereas will b e essentially empty except for a
delimiter denoted which is also used at the b eginning and at the end of Assuming
that the set of tap e symbols and the set of states of M are disjoint i e Q
con gurations will b e describ ed as sequences over their union i e sequences over Q
where Q Other pairs include
i i
For every tap e symbol we shall have for some i We shall also
i i
have for some i Such pairs re ect the preservation of the tap e s
i i
contents whenever the head lo cation is not present at the current cell
For every non halting state q and every transition regarding q we shall have a pair
re ecting this transition For example if the transition function maps q to
q then we have q and q for some i For left movement
i i
i e if the transition function maps q to q we have q and
i
q Assuming that blank symbols i e are only written to the left of
i
other black symbols and when moving left if the transition function maps q
to q then we have q and q rather than q
i i i
Assuming that the machine halts in state p only when it resides in the leftmost cell
and after writing blanks in all cells we have p and for some i
i i
for every t Note that in a solution sequence i i such that
i i i
1
such that the latter contains exactly is a pre x of it holds that
i i i i
t t
1 1
con guration less than the former The relations b etween the pairs guarantee
i i
that these pre ces are pre ces of the sequence of all instantaneous con gurations in the
computation of M on input x and a solution can b e completed only if this computation
halts For details see Sec or Sec
Exercise total functions extending the universal function Let u b e
the function computed by any universal machine for a xed reasonable mo del
of computation Prove that any extension of u to a total function i e any total
function u that agrees with the partial function u on all the inputs on which the
latter is de ned is uncomputable
Guideline The claim is easy to prove for the sp ecial case of the total function u that
extends u such that the sp ecial symbol is assigned to inputs on which u is unde ned
def def
i e u hM i x if u is not de ned on hM i x and u hM i x u hM i x otherwise
In this case h hM i x if and only if u hM i x and so the halting function h is
Turing reducible to u In the general case we may adapt the pro of of Theorem by
using the fact that for any machine M that halts on every input it holds that u hM i x
u hM i x for every x and in particular for x hM i
COMPLEXITY CLASSES
Exercise uncomputability of Kolmogorov Complexity Prove that Kol
mogorov Complexity function denoted K is uncomputable
Guideline Consider for every integer t the string s that is de ned as the lexicograph
t
def
fK s min ically rst string of Kolmogorov Complexity exceeding t i e s
t
s f g
tg Note that s is well de ned and has length at most t Assuming that K is computable
t
we reach a contradiction by noting that s has description length O log t b ecause
t
it may b e describ ed by combining a xed machine that computes K with the integer t
Exercise single tap e vs two tape Turing machines revisited Prove that
deciding membership in the set fxx x f g g requires quadratic time on
a single tap e Turing machine Note that this decision problem can b e solved in
linear time on a two tape Turing machine
Guideline Proving the former fact is quite non trivial One pro of is by a reduction
from a communication complexity problem Sec Intuitively a single tape Turing
machine that decides membership in the aforementioned set can b e viewed as a channel of
communication b etween the two parts of the input Sp eci cally fo cusing our attention on
n n n
inputs of the form y z for y z f g note that each time that the machine passes
from the one part to the other part it carries O bits of information in its internal
state while making at least n steps The pro of is completed by invoking the linear lower
b ound on the communication complexity of the two argument identity function i e
id y z if y z and id y z otherwise cf Chap
Exercise on the circuit complexity of functions Prove the following
facts
The circuit complexity of any Bo olean function is at most exp onential
n n
Guideline f f g f g can b e computed by a circuit of size O n
n
that implements a lo ok up table
Some functions have p olynomial circuit complexity In particular any func
tion that has time complexity t i e is computed by an algorithm of time
complexity t has circuit complexity p oly t Furthermore the corresp ond
ing circuit family is uniform
Guideline Consider a Turing machine that computes the function and consider
its computation on a generic n bit long input The corresp onding computation can
b e emulated by a circuit that consists of t n layers such that each layer represents
an instantaneous con guration of the machine and the relation b etween consecutive
con gurations is captured by uniform lo cal gadgets in the circuit For further
details see the pro of of Theorem which presents a similar emulation
Almost all Bo olean functions have exp onential circuit complexity Sp eci
n
cally the number of functions mapping f g to f g that can b e computed
s
by some circuit of size s is smaller than s
Guideline Show that without loss of generality we may consider circuits of
b ounded fan in The number of such circuits having v vertices and s edges is at
s
v
most v
CHAPTER COMPUTATIONAL TASKS AND MODELS
Exercise the class P p oly We denote by P the class of decision prob
lems that can b e solved in p olynomial time with advice of length and by P p oly
the union of P p taken over all p olynomials p Prove that a decision problem is in
P p oly if and only if it has p olynomial circuit size complexity
Guideline Supp ose that a problem can b e solved by a p olynomial time algorithm
A using the p olynomially b ounded advice sequence a We obtain a family of
n
n N
p olynomial size circuits that solves the same problem by observing that the computa
tion of A a x can b e emulated by a circuit of p oly jxj size which incorporates a
jxj jxj
and is given x as input That is we construct a circuit C such that C x A a x
n n n
n
holds for every x f g analogously to the way C is constructed in the pro of of
x
Theorem where it holds that C y M x y for every y of adequate length On
x R
the other hand given a family of p olynomial size circuits we obtain a p olynomial time
advice taking machine that emulates this family when using advice that provide the de
scription of the relevant circuits Indeed we use the fact that a circuit of size s can b e
describ ed by a string of length O s log s
Exercise Prove that every Bo olean function can b e computed by a family
of DNF resp CNF formulae of exp onential size
n n
Guideline For any a f g consider the function f g f g such that
a
x if x a and x otherwise Note that any function can b e computed by
a a a
n
a single conjunction of n literals and that any Bo olean function f f g f g can
W
A corresp onding CNF formula can b e obtained by applying b e written as
a
a f a
de Morgan s Law to the DNF obtained for f
Chapter
The P versus NP Question
Our daily exp erience is that it is harder to solve problems than it is to check the
correctness of solutions to these problems Is this exp erience merely a coincidence
or do es it represent a fundamental fact of life or a prop erty of the world This is
the essence of the P versus NP Question where P represents search problems that
are e ciently solvable and NP represents search problems for which solutions can
be e ciently checked
Another natural question captured by the P versus NP Question is whether
proving theorems is harder that verifying the validity of these pro ofs In other
words the question is whether deciding membership in a set is harder than b eing
convinced of this membership by an adequate pro of In this case P represents
decision problems that are e ciently solvable whereas NP represents sets that have
e ciently checkable proofs of membership
These two formulations of the P versus NP Question are rigorously presented
and discussed in Sections and resp ectively The equivalence of these for
mulations is shown in Section and the common b elief that P is di erent from
NP is further discussed in Section We start by recalling the notion of e cient
computation
Teaching note Most students have heard of P and NP b efore but we susp ect that
many of them have not obtained a go o d explanation of what the P vs NP Question
actually represents This unfortunate situation is due to using the standard techni
cal de nition of NP which refers to the ctitious and confusing device called a non
deterministic p olynomial time machine Instead we advocate the use of slightly more
cumbersome de nitions sketched in the foregoing paragraphs and elab orated in Sec
tions and which clearly capture the fundamental nature of NP
The notion of e cient computation Recall that we asso ciate e cient com
putation with p olynomial time algorithms This asso ciation is justi ed by the
fact that p olynomials are mo derately growing functions and the set of p olynomials
Advanced comment In this b o ok we consider deterministic p olynomial time algorithms
as the basic mo del of e cient computation A more lib eral view which includes also probabilistic
CHAPTER THE P VERSUS NP QUESTION
is closed under op erations that corresp ond to natural comp osition of algorithms
Furthermore the class of p olynomial time algorithms is indep endent of the sp eci c
mo del of computation as long as the latter is reasonable cf the Cobham
Edmonds Thesis Both issues are discussed in Sec
Advanced note on the representation of problem instances As noted in
Sec many natural search and decision problems are captured more natu
rally by the terminology of promise problems cf Section where the domain of
p ossible instances is a subset of f g rather than f g itself For example com
putational problems in graph theory presume some simple enco ding of graphs as
strings but this enco ding is typically not onto i e not all strings enco de graphs
and thus not all strings are legitimate instances However in these cases the set
of legitimate instances e g enco dings of graphs is e ciently recognizable i e
membership in it can b e decided in p olynomial time Thus arti cially extending
the set of instances to the set of all p ossible strings and allowing trivial solutions
for the corresp onding dummy instances do es not change the complexity of the
original problem We further discuss this issue in Section
The Search Version Finding Versus Check
ing
Teaching note Complexity theorists are so accustomed to fo cus on decision prob
lems that they seem to forget that search problems are at least as natural as decision
problems Furthermore to many non exp erts search problems may seem even more
natural than decision problems Typically p eople seek solutions more often than they
pause to wonder whether or not solutions exist Thus we recommend starting with
a formulation of the P vs NP Question in terms of search problems Admittingly the
cost is more cumbersome formulations but it is more than worthwhile
Teaching note In order to re ect the imp ortance of the search version as well as
allow less cumbersome formulations we chose to introduce notations for the two search
classes corresp onding to P and NP these classes are denoted PF and PC standing
for Polynomial time Find and Polynomial time Check resp ectively The teacher may
prefer using notations and terms that are more evocative of P and NP such as P search
and NP search and actually we also do so in some motivational discussions Still in
our opinion in the long run the students and the eld may b e served b etter by using
standard lo oking notations
Much of computer science is concerned with solving various search problems as in
De nition Examples of such problems include nding a solution to a system of
linear or p olynomial equations nding a prime factor of a given integer nding a
p olynomial time algorithms see or Chap We stress that the most imp ortant
facts and questions that are addressed in the current b o ok hold also with resp ect to probabilistic
p olynomial time algorithms
THE SEARCH VERSION FINDING VERSUS CHECKING
spanning tree in a graph nding a short traveling salesman tour in a metric space
and nding a scheduling of jobs to machines such that various constraints are
satis ed Furthermore search problems corresp ond to the daily notion of solving
problems and are thus of natural general interest In the current section we will
consider the question of which search problems can be solved e ciently
One type of search problems that cannot b e solved e ciently consists of search
problems for which the solutions are to o long in terms of the problem s instances
In such a case merely typing the solution amounts to an activity that is deemed
ine cient Thus we fo cus our attention on search problems that are not in this
class That is we consider only search problems in which the length of the solution
is b ounded by a p olynomial in the length of the instance Recalling that search
problems are asso ciated with binary relations see De nition we fo cus our
attention on p olynomially b ounded relations
De nition p olynomially b ounded relations We say that R f g
f g is p olynomially b ounded if there exists a polynomial p such that for every
x y R it holds that jy j p jxj
Recall that x y R means that y is a solution to the problem instance x where
R represents the problem itself For example in the case of nding a prime factor
of a given integer we refer to a relation R such that x y R if the integer y is a
prime factor of the integer x Likewise in the case of nding a spanning tree in a
given graph we refer to a relation R such that x y R if y is a spanning tree of
the graph x
For a p olynomially b ounded relation R it makes sense to ask whether or not
given a problem instance x one can e ciently nd an adequate solution y i e
nd y such that x y R The p olynomial b ound on the length of the solution
i e y guarantees that a negative answer is not merely due to the length of the
required solution
The Class P as a Natural Class of Search Problems
Recall that we are interested in the class of search problems that can b e solved ef
ciently that is problems for which solutions whenever they exist can b e found
e ciently Restricting our attention to p olynomially b ounded relations we iden
tify the corresp onding fundamental class of search problem or binary relation
denoted PF standing for Polynomial time Find The relationship b etween
PF and the standard de nition of P will b e discussed in Sections and
The following de nition refers to the formulation of solving search problems pro
vided in De nition
De nition e ciently solvable search problems
The search problem of a polynomially bounded relation R f g f g
is e ciently solvable if there exists a polynomial time algorithm A such that
def
for every x it holds that A x R x fy x y R g if and only if R x
is not empty Furthermore if R x then A x indicating that x has
no solution
CHAPTER THE P VERSUS NP QUESTION
We denote by PF the class of search problems that are e ciently solvable
and corresp ond to p olynomially b ounded relations That is R PF if
R is polynomially bounded and there exists a polynomial time algorithm that
given x nds y such that x y R or asserts that no such y exists
Note that R x denotes the set of valid solutions for the problem instance x Thus
the solver A is required to nd a valid solution i e satisfy A x R x whenever
such a solution exists i e R x is not empty On the other hand if the instance
x has no solution i e R x then clearly A x R x The extra condition
also made in De nition requires that in this case A x Thus algorithm
A always outputs a correct answer which is a valid solution in the case that such
a solution exists and otherwise provides an indication that no solution exists
We have de ned a fundamental class of problems and we do know of many
natural problems in this class e g solving linear equations over the rationals
nding a p erfect matching in a graph etc However we must admit that we
do not have a go o d understanding regarding the actual contents of this class i e
we are unable to characterize many natural problems with resp ect to membership
in this class This situation is quite common in complexity theory and seems to
b e a consequence of the fact that complexity classes are de ned in terms of the
external b ehavior of p otential algorithms rather than in terms of the internal
structure of the problem Turning back to PF we note that while it contains
many natural search problems there are also many natural search problems that
are not known to b e in PF A natural class containing a host of such problems is
presented next
The Class NP as Another Natural Class of Search
Problems
Natural search problems have the prop erty that valid solutions for them can b e
e ciently recognized That is given an instance x of the problem R and a candidate
solution y one can e ciently determine whether or not y is a valid solution for x
with resp ect to the problem R i e whether or not y R x The class of all such
search problems is a natural class per se b ecause it is not clear why one should care
ab out a solution unless one can recognize a valid solution once given Furthermore
this class is a natural domain of candidates for PF b ecause the ability to e ciently
recognize a valid solution seems to b e a natural alb eit not absolute prerequisite
for a discussion regarding the complexity of nding such solutions
We restrict our attention again to p olynomially b ounded relations and consider
the class of relations for which membership of pairs in the relation can b e decided
e ciently We stress that we consider deciding membership of given pairs of the
form x y in a xed relation R and not deciding membership of x in the set
def
fx R x g The relationship b etween the following de nition and the S
R
standard de nition of NP will b e discussed in Sections and
Additional examples include sorting integers nding shortest paths in graphs nding patterns
in strings and a variety of other tasks that are typically the fo cus of various courses on algorithms
THE SEARCH VERSION FINDING VERSUS CHECKING
De nition search problems with e ciently checkable solutions
The search problem of a polynomially bounded relation R f g f g
has e ciently checkable solutions if there exists a polynomial time algorithm A
such that for every x and y it holds that A x y if and only if x y R
We denote by PC standing for Polynomial time Check the class of search
problems that correspond to polynomially bounded binary relations that have
e ciently checkable solutions That is R PC if the fol lowing two conditions
hold
For some polynomial p if x y R then jy j p jxj
There exists a polynomial time algorithm that given x y determines
whether or not x y R
Note that the algorithm p ostulated in Item must also handle inputs of the form
x y such that jy j p jxj Such inputs which are evidently not in R by Item
are easy to handle by merely determining jxj jy j and p jxj Thus the crux of
Item is typically in the case that the input x y satis es jy j p jxj
The class PC contains thousands of natural problems e g nding a traveling
salesman tour of length that do es not exceed a given threshold nding the prime
factorization of a given comp osite nding a truth assignment that satis es a given
Bo olean formula etc In each of these natural problems the correctness of solu
tions can b e checked e ciently e g given a traveling salesman tour it is easy to
compute its length and check whether or not it exceeds the given threshold
The class PC is the natural domain for the study of which problems are in PF
b ecause the ability to e ciently recognize a valid solution is a natural prerequisite
for a discussion regarding the complexity of nding such solutions We warn how
ever that PF contains unnatural problems that are not in PC see Exercise
The P Versus NP Question in Terms of Search Prob
lems
Is it the case that every search problem in PC is in PF That is if one can
e ciently check the correctness of solutions with resp ect to some p olynomially
b ounded relation R then is it necessarily the case that the search problem of R
can b e solved e ciently In other words if it is easy to check whether or not a
given solution for a given instance is correct then is it also easy to nd a solution
to a given instance
If PC PF then this would mean that whenever solutions to given instances
can b e e ciently checked for correctness it is also the case that such solutions
can b e e ciently found when given only the instance This would mean that all
In the traveling salesman problem TSP the instance is a matrix of distances b etween cities
and a threshold and the task is to nd a tour that passes all cities and covers a total distance
that do es not exceed the threshold
CHAPTER THE P VERSUS NP QUESTION
reasonable search problems i e all problems in PC are easy to solve Needless to
say such a situation would contradict the intuitive feeling and the daily exp erience
that some reasonable search problems are hard to solve Furthermore in such a
case the notion of solving a problem would lose its meaning b ecause nding a
solution will not b e signi cantly more di cult than checking its validity
On the other hand if PC n P F then there exist reasonable search problems
i e some problems in PC that are hard to solve This conforms with our basic
intuition by which some reasonable problems are easy to solve whereas others are
hard to solve Furthermore it recon rms the intuitive gap b etween the notions of
solving and checking asserting that in some cases solving is signi cantly harder
than checking
As an illustration to the foregoing paragraph consider various puzzles e g
Jigsaw puzzles mazes crossword puzzles Sudoku puzzles etc In each of these
puzzles checking the correctness of a solution is very easy whereas nd a solution
is sometimes extremely hard
The Decision Version Proving Versus Veri
fying
As we shall see in the sequel the study of search problems e g the PC vs PF
Question can b e reduced to the study of decision problems Since the latter
problems have a less cumbersome terminology complexity theory tends to fo cus
on them and maintains its relevance to the study of search problems via the afore
mentioned reduction Thus the study of decision problems provides a convenient
way for studying search problems For example the study of the complexity of de
ciding the satis ability of Bo olean formulae provides a convenient way for studying
the complexity of nding satisfying assignments for such formulae
We wish to stress however that decision problems are interesting and natural
per se i e b eyond their role in the study of search problems After all some
p eople do care ab out the truth and so determining whether certain claims are true
is a natural computational problem Sp eci cally determining whether a given ob
ject e g a Bo olean formula has some predetermined prop erty e g is satis able
constitutes an app ealing computational problem The P vs NP Question refers to
the complexity of solving such problems for a wide and natural class of prop erties
asso ciated with the class NP The latter class refers to prop erties that have e
cient pro of systems allowing for the veri cation of the claim that a given ob ject
has a predetermined prop erty i e is a member of a predetermined set Jumping
ahead we mention that the P vs NP Question refers to the question of whether
prop erties that have e cient pro of systems can also b e decided e ciently without
pro ofs Let us clarify all these notions
Prop erties of ob jects are mo deled as subsets of the set of all p ossible ob jects i e
a prop erty is asso ciated with the set of ob jects having this prop erty For example
the prop erty of b eing a prime is asso ciated with the set of prime numbers and
the prop erty of b eing connected resp having a Hamiltonian path is asso ciated
THE DECISION VERSION PROVING VERSUS VERIFYING
with the set of connected resp Hamiltonian graphs Thus we fo cus on deciding
membership in sets as in De nition The standard formulation of the P vs
NP Question refers to the questionable equality of two natural classes of decision
problems denoted P and NP and de ned in Sec and Sec resp ectively
The Class P as a Natural Class of Decision Problems
Needless to say we are interested in the class of decision problems that are e ciently
solvable This class is traditionally denoted P standing for Polynomial time The
following de nition refers to the formulation of solving decision problems provided
in De nition
De nition e ciently solvable decision problems
A decision problem S f g is e ciently solvable if there exists a polyno
mial time algorithm A such that for every x it holds that A x if and
only if x S
We denote by P the class of decision problems that are e ciently solvable
As in De nition we have de ned a fundamental class of problems which con
tains many natural problems e g determining whether or not a given graph is
connected but we do not have a go o d understanding regarding its actual contents
i e we are unable to characterize many natural problems with resp ect to mem
b ership in this class In fact there are many natural decision problems that are
not known to reside in P and a natural class containing a host of such problems
is presented next This class of decision problems is denoted NP for reasons that
will b ecome evident in Section
The Class NP and NP Pro of Systems
We view NP as the class of decision problems that have e ciently veri able pro of
systems Lo osely sp eaking we say that a set S has a pro of system if instances
in S have valid pro ofs of membership i e pro ofs accepted as valid by the sys
tem whereas instances not in S have no valid pro ofs Indeed pro ofs are de ned
as strings that when accompanying the instance are accepted by the e cient
veri cation pro cedure We say that V is a veri cation pro cedure for membership
in S if it satis es the following two conditions
Completeness True assertions have valid pro ofs i e pro ofs accepted as valid
by V Bearing in mind that assertions refer to membership in S this means
that for every x S there exists a string y such that V x y i e V
accepts y as a valid pro of for the membership of x in S
Soundness False assertions have no valid pro ofs That is for every x S
and every string y it holds that V x y which means that V rejects y as
a pro of for the membership of x in S
CHAPTER THE P VERSUS NP QUESTION
We note that the soundness condition captures the security of the veri cation
pro cedure that is its ability not to b e fo oled by anything into pro claiming a
wrong assertion The completeness condition captures the viability of the veri
cation pro cedure that is its ability to b e convinced of any valid assertion when
presented with an adequate pro of We stress that in general pro of systems
are de ned in terms of their veri cation pro cedures which must satisfy adequate
completeness and soundness conditions Our fo cus here is on e cient veri cation
pro cedures that utilize relatively short pro ofs i e pro ofs that are of length that
is p olynomially b ounded by the length of the corresp onding assertion
Let us consider a couple of examples b efore turning to the actual de nition
Starting with the set of Hamiltonian graphs we note that this set has a veri cation
pro cedure that given a pair G accepts if and only if is a Hamiltonian path in
the graph G In this case serves as a pro of that G is Hamiltonian Note that such
pro ofs are relatively short i e the path is actually shorter than the description
of the graph and are easy to verify Needless to say this pro of system satis es
the aforementioned completeness and soundness conditions Turning to the case
of satis able Bo olean formulae given a formula and a truth assignment the
veri cation pro cedure instantiates according to and accepts if and only if
simplifying the resulting Bo olean expression yields the value true In this case
serves as a pro of that is satis able and the alleged pro ofs are indeed relatively
short and easy to verify
De nition e ciently veri able pro of systems
A decision problem S f g has an e ciently veri able proof system if
there exists a polynomial p and a polynomial time veri cation algorithm V
such that the fol lowing two conditions hold
Completeness For every x S there exists y of length at most p jxj
such that V x y
Such a string y is called an NP witness for x S
Soundness For every x S and every y it holds that V x y
Thus x S if and only if there exists y of length at most p jxj such that
V x y
In such a case we say that S has an NP proof system and refer to V as its
veri cation procedure or as the pro of system itself
Advanced comment In continuation of Footnote we note that in this chapter we consider
deterministic p olynomial time veri cation pro cedures and consequently the completeness and
soundness conditions that we state here are error less In contrast we mention that various types
of probabilistic p olynomial time veri cation pro cedures as well as probabilistic completeness
and soundness conditions are also of interest see e g Chap A common theme that
underlies b oth treatments is that e cient veri cation is interpreted as meaning veri cation by a
pro cess that runs in time that is p olynomial in the length of the assertion In the current chapter
we use the equivalent formulation that considers the running time as a function of the total length
of the assertion and the pro of but require that the latter has length that is p olynomially b ounded
by the length of the assertion
THE DECISION VERSION PROVING VERSUS VERIFYING
We denote by NP the class of decision problems that have e ciently veri able
proof systems
We note that the term NP witness is commonly used In some cases V or the
set of pairs accepted by V is called a witness relation of S We stress that the same
set S may have many di erent NP pro of systems see Exercise and that in
some cases the di erence is not arti cial see Exercise
Using De nition it is typically easy to show that natural decision prob
lems are in NP All that is needed is designing adequate NP pro ofs of member
ship which is typically quite straightforward and natural b ecause natural decision
problems are typically phrased as asking ab out the existence of a structure or
ob ject that can b e easily veri ed as valid For example SAT is de ned as the set
of satis able Bo olean formulae which means asking ab out the existence of satis
fying assignments Indeed we can e ciently check whether a given assignment
satis es a given formula which means that we have a veri cation pro cedure for
an NP pro of system for SAT
Note that for any search problem R in PC the set of instances that have a so
def
fx R x g is in NP Sp eci cally lution with resp ect to R i e the set S
R
def
for any R PC consider the veri cation pro cedure V such that V x y if and
only if x y R and note that the latter condition can b e decided in p oly jxj
time Thus any search problem in PC can be viewed as a problem of searching
for e ciently veri able proofs i e NP witnesses for membership in the set of
instances having solutions On the other hand any NP pro of system gives rise to
a natural search problem in PC that is the problem of searching for a valid pro of
i e an NP witness for the given instance Sp eci cally the veri cation pro cedure
V yields the search problem that corresp onds to R f x y V x y g Thus
S NP if and only if there exists R PC such that S fx R x g
The last paragraph suggests another easy way of showing that natural decision
problems are in NP just thinking of the corresp onding natural search problem
The p oint is that natural decision problems in NP are phrased as referring to
whether a solution exists for the corresp onding natural search problem For exam
ple in the case of SAT the question is whether there exists a satisfying assignment
to given Bo olean formula and the corresp onding search problem is nding such
an assignment In all these cases it is easy to check the correctness of solutions
that is the corresp onding search problem is in PC which implies that the decision
problem is in NP
Observe that P NP holds A veri cation pro cedure for claims of member
ship in a set S P may just ignore the alleged NP witness and run the decision
pro cedure that is guaranteed by the hypothesis S P that is V x y A x
where A is the aforementioned decision pro cedure Indeed the latter veri cation
pro cedure is quite an abuse of the term b ecause it makes no use of the pro of
however it is a legitimate one As we shall shortly see the P vs NP Question refers
In most cases this is done without explicitly de ning V which is understo o d from the context
and or by common practice In many texts V is not called a pro of system nor a veri cation
pro cedure of such a system although this term is most adequate
CHAPTER THE P VERSUS NP QUESTION
to the question of whether such pro of oblivious veri cation pro cedures can b e used
for every set that has some e ciently veri able pro of system Indeed given that
P NP the P vs NP Question is whether NP P
The P Versus NP Question in Terms of Decision Prob
lems
Is it the case that NP proofs are useless That is is it the case that for every ef
ciently veri able pro of system one can easily determine the validity of assertions
without lo oking at the pro of If that were the case then pro ofs would b e meaning
less b ecause they would o er no fundamental advantage over directly determining
the validity of the assertion The conjecture P NP asserts that pro ofs are useful
there exists sets in NP that cannot b e decided by a p olynomial time algorithm
and so for these sets obtaining a pro of of membership for some instances is useful
b ecause we cannot e ciently determine membership by ourselves
In the foregoing paragraph we viewed P NP as asserting the advantage of
obtaining pro ofs over deciding the truth by ourselves That is P NP asserts that
in some cases verifying is easier than deciding A slightly di erent p ersp ective
is that P NP asserts that nding pro ofs is harder than verifying their validity
This is the case b ecause for any set S that has an NP pro of system the ability to
e ciently nd pro ofs of membership with resp ect to this system i e nding an
NP witness of membership in S for any given x S yields the ability to decide
membership in S Thus for S NP n P it must b e harder to nd pro ofs of
membership in S than to verify the validity of such pro ofs which can b e done in
p olynomial time
Equivalence of the two Formulations
As hinted several times the two formulations of the P vs NP Questions are equiva
lent That is every search problem having e ciently checkable solutions is solvable
in p olynomial time i e PC PF if and only if membership in any set that has
an NP pro of system can b e decided in p olynomial time i e NP P Recalling
that P NP whereas PF is not contained in PC Exercise we prove the
following
Theorem PC PF if and only if P NP
Pro of Supp ose on the one hand that the inclusion holds for the search version
i e PC PF We will show that this implies the existence of an e cient algo
rithm for nding NP witnesses for any set in NP which in turn implies that this
set is in P Sp eci cally let S b e an arbitrary set in NP and V b e the corresp ond
ing veri cation pro cedure i e satisfying the conditions in De nition Then
def
R f x y V x y g is a p olynomially b ounded relation in PC and by the
hypothesis its search problem is solvable in p olynomial time i e R PC PF
Denoting by A the p olynomial time algorithm solving the search problem of R we
TECHNICAL COMMENTS REGARDING NP
decide membership in S in the obvious way That is on input x we output if
and only if A x where the latter event holds if and only if A x R x
which in turn o ccurs if and only if R x equiv x S Thus NP P and
NP P follows
Supp ose on the other hand that NP P We will show that this implies
an e cient algorithm for determining whether a given string y is a pre x of some
solution to a given instance x of a search problem in PC which in turn yields an
e cient algorithm for nding solutions Sp eci cally let R b e an arbitrary search
def
problem in PC Then the set S fhx y i y s t x y y R g is in NP
R
is in P by the hypothesis NP P This yields b ecause R PC and hence S
R
a p olynomial time algorithm for solving the search problem of R by extending
a pre x of a p otential solution bit by bit while using the decision pro cedure to
determine whether or not the current pre x is valid That is on input x we rst
check whether or not hx i S and output indicating R x in case
R
hx i S Otherwise hx i S and we set y Next we pro ceed in
R R
iterations maintaining the invariant that hx y i S In each iteration we set
R
y y if hx y i S and y y if hx y i S If none of these conditions
R R
hold which happ ens after at most p olynomially many iterations then the current
y satis es x y R Thus for an arbitrary R PC we obtain that R PF
and PC PF follows
Re ection The rst part of the pro of of Theorem asso ciates with each set
S in NP a natural relation R in PC Sp eci cally R consists of all pairs x y
such that y is an NP witness for membership of x in S Thus the search problem
of R consists of nding such an NP witness when given x as input Indeed R
is called the witness relation of S and solving the search problem of R allows to
decide membership in S Thus R PC PF implies S P In the second part
of the pro of we asso ciate with each R PC a set S in NP but S is more
R R
def
fx y s t x y R g which gives rise to R as its expressive than the set S
R
witness relation Sp eci cally S consists of strings that enco de pairs x y such
R
that y is a pre x of some string in R x fy x y R g The key observation
is that deciding membership in S allows to solve the search problem of R that
R
is S P implies R PF
R
Conclusion Theorem justi es the traditional fo cus on the decision version
of the P vs NP Question Indeed given that b oth formulations of the question are
equivalent we may just study the less cumbersome one
Technical Comments Regarding NP
Recall that when de ning PC resp NP we have explicitly con ned our atten
tion to search problems of p olynomially b ounded relations resp NP witnesses of
p olynomial length An alternative formulation may allow a binary relation R to
b e in PC resp S NP if membership of x y in R can b e decided in time that
CHAPTER THE P VERSUS NP QUESTION
is p olynomial in the length of x resp the veri cation of a candidate NP witness
y for membership of x in S is required to b e p erformed in p oly jxj time This
alternative formulation do es not upp er b ound the length of the solutions resp
NP witnesses but such an upp er b ound follows in the sense only a p oly jxj bit
long pre x of the solution resp NP witness can b e insp ected in order to deter
mine its validity Indeed such a pre x is as go o d as the full length solution resp
NP witness itself Thus the alternative formulation is essentially equivalent to
the original one
We shall often assume that the length of solutions for any search problem in PC
resp NP witnesses for a set in NP is determined rather than upp er b ounded
by the length of the instance That is for any R PC resp veri cation pro cedure
V for a set in NP we shall assume that x y R resp V x y implies
jy j p jxj rather than jy j p jxj for some xed p olynomial p This assumption
can b e justi ed by trivial mo di cation of R resp V see Exercise
We comment that every problem in PC resp NP can b e solved in exp onential
time i e time exp p oly jxj for input x This can b e done by an exhaustive
search among all p ossible candidate solutions resp all p ossible candidate NP
witnesses Thus NP EXP where EXP denote the class of decision problems
that can b e solved in exp onential time i e time exp p oly jxj for input x
The Traditional De nition of NP
Unfortunately De nition is not the commonly used de nition of NP Instead
traditionally NP is de ned as the class of sets that can b e decided by a cti
tious device called a non deterministic p olynomial time machine which explains
the source of the notation NP The reason that this class of ctitious devices is in
teresting is due to the fact that it captures indirectly the de nition of NP pro ofs
Since the reader may come across the traditional de nition of NP when studying
di erent works the author feels obliged to provide the traditional de nition as well
as a pro of of its equivalence to De nition
De nition non deterministic p olynomial time Turing machines
A non deterministic Turing machine is de ne as in Sec except that the
transition function maps symbol state pairs to subsets of triples rather than
to a single triple in Q f g Accordingly the con guration
fol lowing a speci c instantaneous con guration may be one of several possi
bilities each determine by a di erent possible triple Thus the computations
of a non deterministic machine on a xed input may result in di erent outputs
In the context of decision problems one typically considers the question of
whether or not there exists a computation that starting with a xed input
halts with output We say that the non deterministic machine M accept x if
there exists a computation of M on input x that halts with output The set
accepted by a non deterministic machine is the set of inputs that are accepted
by the machine
THE TRADITIONAL DEFINITION OF NP
A non deterministic p olynomial time Turing machine is de ned as one that
makes a number of steps that is polynomial in the length of the input Tra
ditional ly NP is de ned as the class of sets that are each accepted by some
non deterministic polynomial time Turing machine
We stress that De nition refers to a ctitious mo del of computation Sp ecif
ically De nition makes no reference to the number or fraction of p ossible
computations of the machine on a sp eci c input that yield a sp eci c output
De nition only refers to whether or not computations leading to a certain out
put exist for a sp eci c input The question of what do es the mere existence of
such p ossible computations mean in terms of real life is not addressed b ecause
the mo del of a non deterministic machine is not meant to provide a reasonable
mo del of a real life computer The mo del is meant to capture something com
pletely di erent i e it is meant to provide an elegant de nition of the class NP
while relying on the fact that De nition is equivalent to De nition
Teaching note Whether or not De nition is elegant is a matter of taste For sure
many students nd De nition quite confusing p ossibly b ecause they assume that it
represents some natural mo del of computation and consequently they allow themselves
to b e fo oled by their intuition regarding such mo dels Needless to say the students
intuition regarding computation is irrelevant when applied to a ctitious mo del
Note that unlike other de nitions in this b o ok De nition makes explicit
reference to a sp eci c mo del of computation Still a similar extension can b e
applied to other mo dels of computation by considering adequate non deterministic
computation rules Also note that without loss of generality we may assume that
the transition function maps each p ossible symbol state pair to exactly two triples
see Exercise
Theorem De nition is equivalent to De nition That is a set S has
an NP proof system if and only if there exists a non deterministic polynomial time
machine that accepts S
Pro of Supp ose on one hand that the set S has an NP pro of system and let us
denote the corresp onding veri cation pro cedure by V Let p b e a p olynomial that
determines the length of NP witnesses with resp ect to V i e V x y implies
jy j p jxj Consider the following non deterministic p olynomial time machine
denoted M On input x machine M pro ceeds as follows
Makes m p jxj non deterministic steps pro ducing non deterministically
m
a string y f g
Emulates V x y and outputs whatever it do es
Advanced comment In contrast the de nition of a probabilistic machine refers to this
number or equivalently to the probability that the machine pro duces a sp eci c output when the
probability is essentially taken uniformly over all p ossible computations Thus a probabilistic
machine refers to a natural mo del of computation that can b e realized provided we can equip the
machine with a source of randomness For details see Sec
CHAPTER THE P VERSUS NP QUESTION
We stress that these non deterministic steps may result in pro ducing any m bit
p jxj
string y Recall that x S if and only if there exists y f g such that
V x y It follows that x S if and only if there exists a computation of M on
input x that halts with output and thus x S if and only if M accepts x This
implies that the set accepted by M equals S and since M is a non deterministic
p olynomial time machine it follows that S is in NP according to De nition
Supp ose on the other hand that there exists a non deterministic p olynomial
time machine M that accepts the set S and let p b e a p olynomial upp er b ounding
the time complexity of M Consider a deterministic machine M that on input
x y where y has length m p jxj emulates a computation of M on input x
while using the bits of y to determine the non deterministic steps of M That is
th th th
the i step of M on input x is determined by the i bit of y such that the i step
th
of M follows the rst p ossibility in the transition function if and only if i bit of
y equals Note that x S if and only if there exists y of length p jxj such that
M x y Thus M gives rise to an NP pro of system for S and so S NP
according to De nition
In Supp ort of P Being Di erent from NP
Intuition and concepts constitute the elements of all our knowl
edge so that neither concepts without an intuition in some way
corresp onding to them nor intuition without concepts can yield
knowledge
Immanuel Kant
Kant sp eaks of the imp ortance of both philosophical considerations referred to
as concepts and empirical considerations referred to as intuition to science
referred to as sound knowledge
It is widely b elieved that P is di erent than NP that is that PC contains
search problems that are not e ciently solvable and that there are NP pro of sys
tems for sets that cannot b e decided e ciently This b elief is supp orted by b oth
philosophical and empirical considerations
Philosophical considerations Both formulations of the P vs NP Question re
fer to natural questions ab out which we have strong conceptions The notion of
solving a search problem seems to presume that at least in some cases or in gen
eral nding a solution is signi cantly harder than checking whether a presented
solution is correct This translates to P C n P F Likewise the notion of a pro of
seems to presume that at least in some cases or in general the pro of is useful in
determining the validity of the assertion that is that verifying the validity of an
assertion may b e made signi cantly easier when provided with a pro of This trans
lates to P NP which also implies that it is signi cantly harder to nd pro ofs
than to verify their correctness which again coincides with the daily exp erience of
researchers and students
PHILOSOPHICAL MEDITATIONS
Empirical considerations The class NP or rather PC contains thousands
of di erent problems for which no e cient solving pro cedure is known Many of
these problems have arisen in vastly di erent disciplines and were the sub ject of
extensive research of numerous di erent communities of scientists and engineers
These essentially indep endent studies have all failed to provide e cient algorithms
for solving these problems a failure that is extremely hard to attribute to sheer
coincidence or a stroke of bad luck
The common b elief or conjecture that P NP is indeed very app ealing and
intuitive The fact that this natural conjecture is unsettled seems to b e one of
the sources of frustration of complexity theory The author s opinion however is
that this feeling of frustration is not in place In contrast the fact that complexity
theory evolves around natural and simply stated questions that are so di cult to
resolve makes its study very exciting
Throughout the rest of this b o ok we will adopt the conjecture that P is di erent
from NP In few places we will explicitly use this conjecture whereas in other places
we will present results that are interesting if and only if P NP e g the entire
theory of NP completeness b ecomes uninteresting if P NP
Philosophical Meditations
Who ever do es not value preo ccupation with thoughts can skip this chapter
Rob ert Musil The Man without Qualities Chap
The inherent limitations of our scienti c knowledge were articulated by Kant who
argued that our knowledge cannot transcend our way of understanding The ways
of understanding are predetermined they precede any knowledge acquisition and
are the precondition to such acquisition In a sense Wittgenstein re ned the
analysis arguing that knowledge must b e formulated in a language and the latter
must b e sub ject to a sound mechanism of assigning meaning Thus the inherent
limitations of any p ossible meaning assigning mechanism imp ose limitations on
what can b e meaningfully said
Both philosophers sp oke of the relation b etween the world and our thoughts
They to ok for granted or rather assumed that in the domain of well formulated
thoughts e g logic every valid conclusion can b e e ectively reached i e every
valid assertion can b e e ectively proved Indeed this naive assumption was refuted
by G odel In a similar vain Turing s work asserts that there exist wel l de ned
problems that cannot be solved by wel l de ned methods
The latter assertion transcends the philosophical considerations of the rst para
graph It asserts that the limitations of our ability are not only due to the gap
b etween the world as is and our mo del of it In contrast the foregoing asser
tion refers to inherent limitations on any rational pro cess even when this pro cess
is applied to well formulated information and is aimed at a well formulated goal
Indeed in contrast to naive presumptions not every well formulated problem can
b e e ectively solved
CHAPTER THE P VERSUS NP QUESTION
The P NP conjecture go es even b eyond the foregoing assertion It limits the
domain of the discussion to fair problems that is to problems for which valid
solutions can b e e ciently recognized as such Indeed there is something feigned in
problems for which one cannot e ciently recognize valid solutions Avoiding such
feigned and or unfair problems P NP means that even with this limitation
there exist problems that are inherently unsolvable in the sense that they cannot
b e solved e ciently That is in contrast to naive presumptions not every problem
that refers to e ciently recognizable solutions can be solved e ciently In fact the
gap b etween the complexity of recognizing solutions and the complexity of nding
them vouches for the meaningfulness of the notion of a problem
Exercises
Exercise PF contains problems that are not in PC Show that PF con
tains some unnatural problems that are not in PC
Guideline Consider the relation R f x x f g g f x x S g where S is
some undecidable set Note that R is the disjoint union of two binary relations denoted
R and R where R is in PF whereas R is not in PC Furthermore for every x it holds
that R x
Exercise Show that the following search problems are in PC
Finding a traveling salesman tour of length that do es not exceed a given
threshold when also given a matrix of distances b etween cities
Finding the prime factorization of a given comp osite
Solving a given system of quadratic equations over a nite eld
Finding a truth assignment that satis es a given Bo olean formula
For Item use the fact that primality can b e tested in p olynomial time
Exercise Show that any S NP has many di erent NP pro of systems i e
veri cation pro cedures V V such that V x y do es not imply V x y
i j
for i j
Guideline For V and p b e as in De nition de ne V x y if jy j p jxj i and
i
there exists a pre x y of y such that V x y
Exercise Relying on the fact that primality is decidable in p olynomial time
and assuming that there is no p olynomial time factorization algorithm present two
natural but fundamentally di erent NP pro of systems for the set of comp osite
numbers
Guideline Consider the following veri cation pro cedures V and V for the set of com
p osite numbers Let V n y if and only if y n and n is not a prime and
V n m if and only if m is a non trivial divisor of n Show that valid pro ofs with
resp ect to V are easy to nd whereas valid pro ofs with resp ect to V are hard to nd
PHILOSOPHICAL MEDITATIONS
Exercise Show that for every R PC there exists R PC and a p olynomial
p jxj
p such that for every x it holds that R x f g and R PF if and only
if R PF Formulate and prove a similar fact for NP pro of systems
Guideline Note that for every R PC there exists a p olynomial p such that for every
def
p jxj jy j
x y R it holds that jy j p jxj De ne R such that R x fy
x y R g and prove that R PF if and only if R PF
Exercise In continuation of Exercise show that for every set S NP and
every su ciently large p olynomial p there exists an NP pro of system V such that
all NP witnesses to x S are of length p jxj i e if V x y then jy j p jxj
Guideline Starting with an NP pro of system V for S and a p olynomial p such that
V x y implies jy j p jxj for every p olynomial p p de ne V such that
0
p jxj jy j
V x y if V x y and V x y otherwise
Exercise In continuation of Exercise show that for every set S NP
and every bijection N N such that b oth and are upp er b ounded by
p olynomials there exists set S NP such that S P if and only if S P
and there exists an NP pro of system V such that all NP witnesses to x S
are of length jxj
0
def
p jxj jxj
Guideline For an adequate bijective p olynomial p consider S fx g and
0
p jxj jxj
the NP pro of system V such that V x y V x y and V x y if
jx j fp n n N g Now use Exercise
Exercise Show that for every S NP there exists an NP pro of system V
def
fy V x y g are disjoint such that the witness sets W
x
Guideline Starting with an NP pro of system V for S consider V such that V x y
if y hx y i and V x y and V x y otherwise
Exercise Regarding De nition show that if S is accepted by some non
deterministic machine of time complexity t then it is accepted by a non deterministic
machine of time complexity O t that has a transition function that maps each p os
sible symbol state pair to exactly two triples
Guideline First note that a k way non deterministic choice can b e emulated by log k
non deterministic binary choices Indeed this requires duplicating the set of states of
the machine Also note that one can introduce ctitious non deterministic choices by
duplicating the set of states of the machine
CHAPTER THE P VERSUS NP QUESTION
Chapter
Polynomial time Reductions
We present a general notion of p olynomial time reductions among computational
problems and view the notion of a Karp reduction as an imp ortant sp ecial case
that su ces and is more convenient in many cases Reductions play a key role
in the theory of NP completeness which is the topic of Chapter In the current
chapter we stress the fundamental nature of the notion of a reduction per se and
highlight two sp eci c applications i e reducing search and optimization problems
to decision problems Furthermore in these applications it will b e imp ortant to
use the general notion of a reduction i e Co ok reduction rather than Karp
reduction
Teaching note We assume that many students have heard of reductions but we fear
that most have obtained a conceptually p o or view of their fundamental nature In
particular we fear that reductions are identi ed with the theory of NP completeness
while reductions have numerous other imp ortant applications that have little to do with
NP completeness or completeness with resp ect to some other class Furthermore we
b elieve that it is imp ortant to show that natural search and optimization problems can
b e reduced to decision problems
The General Notion of a Reduction
Reductions are pro cedures that use functionally sp eci ed subroutines That is
the functionality of the subroutine is sp eci ed but its op eration remains unsp eci ed
and its running time is counted at unit cost Analogously to algorithms which
are mo deled by Turing machines reductions can b e mo deled as oracle Turing
machines A reduction solves one computational problem which may b e either
a search or a decision problem by using oracle or subroutine calls to another
computational problem which again may b e either a search or a decision problem
CHAPTER POLYNOMIAL TIME REDUCTIONS
The Actual Formulation
The notion of a general algorithmic reduction was discussed in Sec and
Sec These reductions called Turing reductions cf Sec and mo d
eled by oracle machines cf Sec made no reference to the time complexity
of the main algorithm i e the oracle machine Here we fo cus on e cient i e
p olynomial time reductions which are often called Cook reductions That is we
consider oracle machines as in De nition that run in time that is p olynomial
in the length of their input We stress that the running time of an oracle machine
is the number of steps made during its own computation and that the oracle s
reply on each query is obtained in a single step
The key prop erty of e cient reductions is that they allow for the transformation
of e cient implementations of the subroutine into e cient implementations of the
task reduced to it That is as we shall see if one problem is Co ok reducible to
another problem and the latter is p olynomial time solvable then so is the former
The most p opular case is that of reducing decision problems to decision prob
lems but we will also consider reducing search problems to search problems and
reducing search problems to decision problems Note that when reducing to a de
cision problem the oracle is determined as the unique valid solver of the decision
problem since the function f f g f g solves the decision problem of
membership in S if for every x it holds that f x if x S and f x oth
erwise In contrast when reducing to a search problem the oracle is not uniquely
determined b ecause there may b e many di erent valid solvers since the function
f f g f g f g solves the search problem of R if for every x it holds
def
that f x R x fy x y R g if R x and f x otherwise We
capture b oth cases in the following de nition
De nition Co ok reduction A problem is Co ok reducible to a problem
if there exists a polynomial time oracle machine M such that for every function f
f f
that solves it holds that M solves where M x denotes the output of M on
input x when given oracle access to f
Note that resp may b e either a search problem or a decision problem or
even a yet unde ned type of a problem At this p oint the reader should verify
that if is Co ok reducible to and is solvable in p olynomial time then so is
see Exercise which also asserts other prop erties of Co ok reductions
Observe that the second part of the pro of of Theorem is actually a Co ok
reduction of the search problem of any R in PC to a decision problem regarding a
related set S fhx y i y s t x y y R g which in NP Thus that pro of
R
establishes the following result
Theorem Every search problem in PC is Cook reducible to some decision
problem in NP
We shall see a tighter relation b etween search and decision problems in Section
that is in some cases R will b e reduced to S fx y s t x y R g rather
R
than to S
R
Indeed the solver is unique only if for every x it holds that jR x j
THE GENERAL NOTION OF A REDUCTION
Sp ecial Cases
A Karp reduction is a sp ecial case of a reduction from a decision problem to a
decision problem Sp eci cally for decision problems S and S we say that S is
Karp reducible to S if there is a reduction of S to S that operates as fol lows On
input x an instance for S the reduction computes x makes query x to the oracle
S i e invokes the subroutine for S on input x and answers whatever the latter
returns This reduction is often represented by the p olynomial time computable
mapping of x to x that is the standard de nition of a Karp reduction is actually
as follows
De nition Karp reduction A polynomial time computable function f is
called a Karp reduction of S to S if for every x it holds that x S if and only if
f x S
Thus syntactically sp eaking a Karp reduction is not a Co ok reduction but it
trivially gives rise to one i e on input x the oracle machine makes query f x
and returns the oracle answer Being slightly inaccurate but essentially correct
we shall say that Karp reductions are sp ecial cases of Co ok reductions
Needless to say Karp reductions constitute a very restricted case of Co ok
reductions Still this restricted case su ces for many applications e g most
imp ortantly for the theory of NP completeness when developed for decision prob
lems but not for reducing a search problem to a decision problem Furthermore
whereas each decision problem is Co ok reducible to its complement some decision
problems are not Karp reducible to their complement see Exercises and
We comment that Karp reductions may and should b e augmented in order
to handle reductions of search problems to search problems Such an augmented
Karp reduction of the search problem of R to the search problem of R op erates
as follows On input x an instance for R the reduction computes x makes
query x to the oracle R i e invokes the subroutine for searching R on input
x obtaining y such that x y R and uses y to compute a solution y to x
i e y R x Thus such a reduction can b e represented by two p olynomial
time computable mappings f and g such that x g x y R for any y that is
a solution of f x i e for y that satis es f x y R Indeed in general
unlike in the case of decision problems the reduction cannot just return y as an
answer to x This augmentation is called a Levin reduction and analogously to
the case of a Karp reduction it is often identi ed with the two aforementioned
p olynomial time computable mappings themselves i e the mappings of x to x
and the mappings of x y to y
De nition Levin reduction A pair of polynomial time computable func
tions f and g is called a Levin reduction of R to R if f is a Karp reduction of
0
fx y s t x y R g and for every S fx y s t x y R g to S
R R
x S and y R f x it holds that x g x y R where R x fy
R
x y R g
Indeed the function f preserves the existence of solutions that is for any x it
holds that R x if and only if R f x As for the second function i e g
CHAPTER POLYNOMIAL TIME REDUCTIONS
it maps any solution y for the reduced instance f x to a solution for the original
instance x where this mapping may also dep end on x We mention that it is
natural to consider also a third function that maps solutions for R to solutions for
R see Exercise
Terminology and a Brief Discussion
In the sequel whenever we neglect to mention the type of a reduction we refer to a
Co ok reduction Two additional terms which are often used in advanced studies
are presented next
We say that two problems are computationally equivalent if they are reducible
to one another This means that the two problems are essentially as hard or
as easy Note that computationally equivalent problems need not reside in
the same complexity class
For example as we shall see in Section there exist many natural R PC
such that the search problem of R and the decision problem of S fx
R
y s t x y R g are computationally equivalent although even syntacti
cally the two problems do not b elong to the same class i e R PC whereas
S NP Also each decision problem is computationally equivalent to its
R
complement although the two problems may not b elong to the same class
see e g Section
We say that a class of problems C is reducible to a problem if every
problem in C is reducible to We say that the class C is reducible to the
class C if for every C there exists C such that is reducible to
For example Theorem asserts that PC is reducible to NP
The fact that we allow Co ok reductions is essential to various imp ortant connec
tions b etween decision problems and other computational problems For example
as will b e shown in Section a natural class of optimization problems is reducible
to NP Also recall that PC is reducible to NP cf Theorem Furthermore
as will b e shown in Section many natural search problems in PC are reducible
to a corresp onding natural decision problem in NP rather than merely to some
problem in NP In all these results the reductions in use are and must b e
Co ok reductions
Reducing Optimization Problems to Search
Problems
Many search problems refer to a set of p otential solutions asso ciated with each
problem instance such that di erent solutions are assigned di erent values resp
costs For example in the context of nding a clique in a given graph the size
of the clique may b e considered the value of the solution Likewise in the context
of nding a partition of a given graph the number of edges with b oth endp oints
REDUCING OPTIMIZATION PROBLEMS TO SEARCH PROBLEMS
in the same side of the partition may b e considered the cost of the solution In
such cases one may b e interested in nding a solution that has value exceeding
some threshold resp cost b elow some threshold Alternatively one may seek a
solution of maximum value resp minimum cost
For simplicity let us fo cus on the case of a value that we wish to maximize
Still the two di erent aforementioned ob jectives i e exceeding a threshold and
optimization give rise to two di erent auxiliary search problems related to the
same relation R Sp eci cally for a binary relation R and a value function f
f g f g R we consider two search problems
Exceeding a threshold Given a pair x v the task is to nd y R x such
that f x y v where R x fy x y R g That is we are actually
referring to the search problem of the relation
def
f hx v i y x y R f x y v g R
f
where hx v i denotes a string that enco des the pair x v
Maximization Given x the task is to nd y R x such that f x y v
x
where v is the maximum value of f x y over all y R x That is we are
x
actually referring to the search problem of the relation
def
R f x y R f x y max ff x y gg
f
0
y R x
Examples of value functions include the size of a clique in a graph the amount of
ow in a network with link capacities etc The task may b e to nd a clique of
size exceeding a given threshold in a given graph or to nd a maximum size clique
in a given graph Note that in these examples the base search problem i e
the relation R is quite easy to solve and the di culty arises from the auxiliary
Indeed one may condition on the value of a solution presented in R and R
f
f
p oly jxj
trivialize R i e let R x f g for every x and imp ose all necessary
structure by the function f see Exercise
We con ne ourselves to the case that f is p olynomial time computable which
in particular means that f x y can b e represented by a rational number of length
p olynomial in jxj jy j We will show next that in this case the two aforementioned
search problems i e of R and R are computationally equivalent
f
f
Theorem For any polynomial time computable f f g f g R and
a polynomially bounded binary relation R let R and R be as in Eq and
f
f
Eq respectively Then the search problems of R and R are computationally
f
f
equivalent
Note that for R PC and polynomial time computable f it holds that R PC
f
Combining Theorems and it follows that in this case both R and R are
f
f
reducible to NP We note however that even in this case it do es not necessarily
hold that R PC See further discussion following the pro of f
CHAPTER POLYNOMIAL TIME REDUCTIONS
Pro of The search problem of R is reduced to the search problem of R by
f
f
nding an optimal solution for the given instance and comparing its value to the
given threshold value That is we construct an oracle machine that solves R by
f
Sp eci cally on input x v the machine issues the making a single query to R
f
query x to a solver for R obtaining the optimal solution y or an indication
f
that R x computes f x y and returns y if f x y v Otherwise i e
either y or f x y v the machine returns an indication that R x v
f
Turning to the opp osite direction we reduce the search problem of R to the
f
search problem of R by rst nding the optimal value v max ff x y g
f x
y R x
by binary search on its p ossible values and next nding a solution of value v
x
In b oth steps we use oracle calls to R For simplicity we assume that f assigns
f
positive integer values and let p oly jxj b e such that f x y for every
y R x Then on input x we rst nd v maxff x y y R x g by making
x
oracle calls of the form hx v i The p oint is that v v if any only if R hx v i
x f
which in turn is indicated by the oracle answer to the query hx v i Making
queries we determine v see Exercise Note that in case R x all answers
x
will indicate that R hx v i which we treat as if v Finally we make the
f x
query x v and halt returning the oracle s answer which is y R x such that
x
f x y v if v and an indication that R x otherwise
x x
Pro of s digest Note that the rst direction uses the hypothesis that f is p olynomial
time computable whereas the opp osite direction only used the fact that the optimal
value lies in a nite space of exp onential size that can b e e ciently searched
While the rst direction can b e proved using a Levin reduction this seems imp os
sible for the opp osite direction in general
On the complexity of R and R We fo cus on the natural case in which
f
f
R PC and f is p olynomial time computable In this case Theorem asserts
that R and R are computationally equivalent A closer lo ok reveals however
f
f
that R PC always holds whereas R PC do es not necessarily hold That
f
f
is the problem of nding a solution for a given instance that exceeds a given
threshold is in the class PC whereas the problem of nding an optimal solution
is not necessarily in the class PC For example the problem of nding a clique
of a given size K in a given graph G is in PC whereas the problem of nding a
maximum size clique in a given graph G is not known and is quite unlikely to
b e in PC although it is Co ok reducible to PC Indeed the class of problems that
are reducible to PC is a natural and interesting class Needless to say for every
R PC and p olynomial time computable f the former class contains R
f
Self Reducibility of Search Problems
The results to b e presented in this section further justify the fo cus on decision
problems Lo osely sp eaking these results show that for many natural relations R
See Exercise
SELF REDUCIBILITY OF SEARCH PROBLEMS
the question of whether or not the search problem of R is e ciently solvable i e
is in PF is equivalent to the question of whether or not the decision problem
implicit in R i e S fx y s t x y R g is e ciently solvable i e
R
is in P In fact we will show that these two computational problems i e R
and S are computationally equivalent Note that the decision problem of S
R R
is easily reducible to the search problem of R and so our fo cus is on the other
direction That is we are interested in relations R for which the search problem
of R is reducible to the decision problem of S In such a case we say that R is
R
self reducible
Teaching note Our usage of the term self reducibility di ers from the traditional
one Traditionally a decision problem is called downwards self reducible if it is Co ok
reducible to itself via a reduction that on input x only makes queries that are smaller
than x according to some appropriate measure on the size of instances Under some
natural restrictions i e the reduction takes the disjunction of the oracle answers such
reductions yield reductions of search to decision as discussed in the main text For
further details see Exercise
De nition the decision implicit in a search and self reducibility The de
cision problem implicit the search problem of R is deciding membership in the set
S fx R x g where R x fy x y R g The search problem of R is
R
called self reducible if it can be reduced to the decision problem of S
R
Note that the search problem of R and the problem of deciding membership in
S refer to the same instances The search problem requires nding an adequate
R
solution i e given x nd y R x whereas the decision problem refers to the
question of whether such solutions exist i e given x determine whether or not
R x is non empty Thus S is really the decision problem implicit in R
R
b ecause it is a decision problem that one implicitly solves when solving the search
problem of R Indeed for any R the decision problem of S is easily reducible to
R
the search problem for R see Exercise It follows that if a search problem R
is self reducible then it is computationally equivalent to the decision problem S
R
Note that the general notion of a reduction i e Co ok reduction seems inher
ent to the notion of self reducibility This is the case not only due to syntactic
considerations but rather due to the following inherent reason An oracle to any
decision problem returns a single bit p er invocation while the intractability of a
search problem in PC must b e due to lacking more than a single bit of information
see Exercise
We shall see that self reducibility is a prop erty of many natural search problems
including all NP complete search problems This justi es the relevance of decision
problems to search problems in a stronger sense than established in Section
Recall that in Section we showed that the fate of the search problem class PC
w r t PF is determined by the fate of the decision problem class NP w r t P
Here we show that for many natural search problems in PC i e self reducible
ones the fate of such a problem R w r t PF is determined by the fate of the
decision problem S w r t P where S is the decision problem implicit in R
R R
Recall that R PC implies S NP R
CHAPTER POLYNOMIAL TIME REDUCTIONS
Examples
We now present a few search problems that are self reducible We start with SAT
see Section A the set of satis able Bo olean formulae in CNF and consider
the search problem in which given a formula one should provide a truth assignment
that satis es it The corresp onding relation is denoted R that is R
SAT SAT
if is a satisfying assignment to the formula The decision problem implicit in
R is indeed SAT Note that R is in PC i e it is p olynomially b ounded
SAT SAT
and membership of in R is easy to decide by evaluating a Bo olean
SAT
expression
Prop osition R is self reducible The search problem of R is reducible
SAT SAT
to SAT
Thus the search problem of R is computationally equivalent to deciding mem
SAT
b ership in SAT Hence in studying the complexity of SAT we also address the
complexity of the search problem of R
SAT
Pro of We present an oracle machine that solves the search problem of R by
SAT
making oracle calls to SAT Given a formula we nd a satisfying assignment to
in case such an assignment exists as follows First we query SAT on itself and
return an indication that there is no solution if the oracle answer is indicating
SAT Otherwise we let initiated to the empty string denote a pre x of a
satisfying assignment of We pro ceed in iterations where in each iteration we
extend by one bit as long as do es not set all variables of This is done as
follows First we derive a formula denoted by setting the rst j j variables
of according to the values We then query SAT on which means that we
ask whether or not is a pre x of a satisfying assignment of If the answer
is p ositive then we set else we set This pro cedure relies on the
fact that if is a pre x of a satisfying assignment of and is not a pre x of a
satisfying assignment of then must b e a pre x of a satisfying assignment of
We wish to highlight a key p oint that has b een blurred in the foregoing de
scription Recall that the formula is obtained by replacing some variables by
constants which means that per se contains Bo olean variables as well as Bo olean
constants However the standard de nition of SAT disallows Bo olean constants in
its instances Nevertheless can b e simpli ed such that the resulting formula
contains no Bo olean constants This simpli cation is p erformed according to the
straightforward Bo olean rules That is the constant false can b e omitted from
any clause but if a clause contains only o ccurrences of the constant false then
the entire formula simpli es to false Likewise if the constant true app ears in
a clause then the entire clause can b e omitted and if all clauses are omitted then
the entire formula simpli es to true Needless to say if the simpli cation pro cess
yields a Bo olean constant then we may skip the query and otherwise we just use
the simpli ed form of as our query
While the problem seems rather technical in the current setting since it merely amounts
to whether or not the de nition of SAT allows Bo olean constants in its instances th analogous
problem is far from b eing so technical in other cases see Exercises and
SELF REDUCIBILITY OF SEARCH PROBLEMS
Other examples Reductions analogous to the one used in the pro of of Prop o
sition can b e presented also for other search problems and not only for NP
complete ones Two such examples are searching for a coloring of a given graph
and searching for an isomorphism b etween a given pair of graphs where the rst
problem is known to b e NP complete and the second problem is b elieved not to
b e NP complete In b oth cases the reduction of the search problem to the cor
resp onding decision problem consists of iteratively extending a pre x of a valid
solution by making suitable queries in order to decide which extension to use
Note however that in these two cases the pro cess of getting rid of constants rep
resenting partial solutions is more involved Sp eci cally in the case of Graph
Colorability resp Graph Isomorphism we need to enforce a partial coloring of
a given graph resp a partial isomorphism b etween a given pair of graphs see
Exercises and resp ectively
Re ection The pro of of Prop osition as well as the pro ofs of similar results
consists of two observations
For every relation R in PC it holds that the search problem of R is reducible
to the decision problem of S fhx y i y s t x y y R g Such a
R
reduction is explicit in the pro of of Theorem and is implicit in the pro of
of Prop osition
For sp eci c R PC e g S deciding membership in S is reducible to
SAT
R
deciding membership in S fx y s t x y R g This is where the
R
sp eci c structure of SAT was used allowing for a direct and natural transfor
mation of instances of S to instances of S
R
R
We comment that if S is NP complete then S which is always in NP is
R
R
reducible to S by the mere fact that S is NP complete this comment is
R R
elab orated in the following Sec
For an arbitrary R PC deciding membership in S is not necessarily reducible to
R
deciding membership in S Furthermore deciding membership in S is not nec
R
R
essarily reducible to the search problem of R See Exercises and
In general self reducibility is a prop erty of the search problem and not of the
decision problem implicit in it Furthermore under plausible assumptions e g
the intractability of factoring there exists relations R R PC having the same
implicit decision problem i e fx R x g fx R x g such that R is
self reducible but R is not see Exercise However for many natural decision
problems this phenomenon do es not arise that is for many natural NP decision
problems S any NP witness relation associated with S i e R PC such that
fx R x g S is self reducible Indeed see the advanced Sec
CHAPTER POLYNOMIAL TIME REDUCTIONS
Self Reducibility of NP Complete Problems
Teaching note In this advanced section we assume that the students have heard of
NP completeness Actually we only need the students to know the de nition of NP
completeness i e a set S is NP complete if S NP and every set in NP is reducible
to S Yet the teacher may prefer p ostp oning the presentation of the following advanced
discussion to Section or even to a later stage
Recall that in general self reducibility is a prop erty of the search problem R and
not of the decision problem implicit in it i e S fx R x g In contrast
R
in the sp ecial case of NP complete problems self reducibility holds for any witness
relation asso ciated with the NP complete decision problem That is al l search
problems that refer to nding NP witnesses for any NP complete decision problem
are self reducible
Theorem For every R in PC such that S is NP complete the search problem
R
of R is reducible to deciding membership in S
R
In many cases as in the pro of of Prop osition the reduction of the search
problem to the corresp onding decision problem is quite natural The following
pro of presents a generic reduction which may b e unnatural in some cases
Pro of In order to reduce the search problem of R to deciding S we comp ose
R
the following two reductions
A reduction of the search problem of R to deciding membership in S
R
fhx y i y s t x y y R g
As stated in Sec in the paragraph titled re ection such a reduction
is implicit in the pro of of Prop osition as well as b eing explicit in the pro of
of Theorem
A reduction of S to S
R
R
This reduction exists by the hypothesis that S is NP complete and the
R
fact that S NP Note that we do not assume that this reduction is a
R
Karp reduction and furthermore it may b e an unnatural reduction
The theorem follows
Digest and General Perspective
Recall that we presented p olynomial time reductions as e cient algorithms that
use functionally sp eci ed subroutines That is an e cient reduction of problem
to problem is an e cient algorithm that solves while making subroutine calls
to any pro cedure that solves This presentation ts the natural p ositive
application of such a reduction that is combining such a reduction with an e cient
implementation of the subroutine solving we obtain an e cient algorithm for
solving We note that the existence of a p olynomial time reduction of to
DIGEST AND GENERAL PERSPECTIVE
actually means more than the latter implication For example also an ine cient
algorithm for solving yields something for that is if is solvable in time t
then is solvable in time t such that t n p oly n t p oly n for example if
log n log p oly n O log n
2 2
t n n then t n p oly n n Thus the existence of a
p olynomial time reduction of to yields an upp er b ound on the time complexity
of in terms of the time complexity of
We note that tighter relations b etween the complexity of and can b e
established whenever the reduction satis es additional prop erties For example
supp ose that is p olynomial time reducible to by a reduction that makes queries
of linear length i e on input x each query has length O jxj Then if is
solvable in time t then is solvable in time t such that t n p oly n t O n
p
p p
O n O log n
O n n
for example if t n then t n We further
note that b ounding other complexity measures of the reduction e g its space
complexity allows to relate the corresp onding complexities of the problems
In contrast to the foregoing p ositive applications of p olynomial time reduc
tions the theory of NP completeness presented in Chapter is famous for its
negative application of such reductions Let us elab orate The fact that is
p olynomial time reducible to means that if solving is feasible then solving
is feasible The direct p ositive application starts with the hypothesis that is
feasibly solvable and infers that so is In contrast the negative application
uses the counter positive it starts with the hypothesis that solving is infeasible
and infers that the same holds for
Exercises
Exercise Verify the following prop erties of Co ok reductions
Co ok reductions preserve e cient solvability If is Co ok reducible to
and is solvable in p olynomial time then so is
Co ok reductions are transitive i e if is Co ok reducible to and is
Co ok reducible to then is Co ok reducible to
Co ok reductions generalize e cient decision pro cedures If is solvable in
p olynomial time then it is Co ok reducible to any problem
In continuation of the last item show that a problem is solvable in p olynomial
time if and only if it is Co ok reducible to a trivial problem e g deciding member
ship in the empty set
Exercise Show that Karp reductions and Levin reductions are transitive
Exercise Show that some decision problems are not Karp reducible to their
complement e g the empty set is not Karp reducible to f g
A p opular exercise of dubious nature is showing that any decision problem in P
is Karp reducible to any non trivial decision problem where the decision problem
regarding a set S is called non trivial if S and S f g It follows that
every non trivial set in P is Karp reducible to its complement
CHAPTER POLYNOMIAL TIME REDUCTIONS
Exercise reducing search problems to optimization problems For ev
ery p olynomially b ounded relation R resp R PC present a function f resp
a p olynomial time computable function f such that the search problem of R is
computationally equivalent to the search problem in which given x v one has to
p oly jxj
nd a y f g such that f x y v
Hint use a Bo olean function
Exercise binary search Show that using binary queries of the form is
z v it is p ossible to determine the value of an integer z that is a priori known
to reside in the interval
Guideline Consider a pro cess that iteratively halves the interval in which z is known
to reside in
Exercise Prove that for any R the decision problem of S is easily reducible
R
to the search problem for R and that if R is in PC then S is in NP
R
Guideline Consider a reduction that invokes the search oracle and answer if and only
if the oracle returns some string rather than the no solution symbol
Exercise Prove that if R PC n P F is self reducible then the relevant Co ok
reduction makes more than a logarithmic number of queries to S More generally
R
prove that if R PC n PF is Co ok reducible to any decision problem then this
reduction makes more than a logarithmic number of queries
Guideline Note that the oracle answers can b e emulated by trying all p ossibilities and
that the correctness of the output of the oracle machine can b e e ciently tested
Exercise Show that the standard search problem of Graph Colorability is
self reducible where this search problem consists of nding a coloring for a given
input graph
Guideline Iteratively extend the current pre x of a coloring of the graph by making
adequate oracle calls to the decision problem of Graph Colorability Sp eci cally enco de
t
the question of whether or not f g is a pre x of a coloring of the graph
t
G as a query regarding the colorability of an auxiliary graph G Note that we merely
need to check whether G has a coloring in which the equalities and inequalities induced
by the pre x of the coloring hold This can b e done by adequate gadgets e g
t
inequality is enforced by an edge b etween the corresp onding vertices whereas equality is
enforced by an adequate subgraph that includes the relevant vertices as well as auxiliary
vertices
Exercise Show that the standard search problem of Graph Isomorphism is
self reducible where this search problem consists of nding an isomorphism b e
tween a given pair of graphs
Guideline Iteratively extend the current pre x of an isomorphism b etween the two
N vertex graphs by making adequate oracle calls to the decision problem of Graph Iso
t
morphism Sp eci cally enco de the question of whether or not N is a
t
For Part of Exercise equality is b etter enforced by combining the two vertices
DIGEST AND GENERAL PERSPECTIVE
pre x of an isomorphism b etween G N E and G N E as a query regard
ing isomorphism b etween two auxiliary graphs G and G This can b e done by attaching
adequate gadgets to pairs of vertices that we wish to b e mapp ed to one another by the
th
isomorphism For example we may connect each of the vertices in the i pair to an
auxiliary star consisting of N i vertices
Exercise downwards self reducibility We say that a set S is down
wards self reducible if there exists a Co ok reduction of S to itself that only makes
queries that are each shorter than the reduction s input i e if on input x the
reduction makes the query q then jq j jxj
Show that SAT is downwards self reducible with resp ect to a natural enco ding
of CNF formulae Note that this enco ding should have the prop erty that
instantiating a variable in a formula results in a shorter formula
A harder exercise consists of showing that Graph Colorability is downwards
self reducible with resp ect to some reasonable enco ding of graphs Note that
this enco ding has to b e selected carefully if it is to work for a pro cess anal
ogous to the one used in Exercise
Supp ose that S is downwards self reducible by a reduction that outputs the
disjunction of the oracle answers Note that this is the case for SAT Show
that in this case S is characterized by a witness relation R PC i e
S fx R x g that is self reducible i e the search problem of R is
Co ok reducible to S Needless to say it follows that S NP
O
Guideline De ne R such that x hx x i is in R if x S f g and
t t
for every i f t g on input x the self reduction makes a set of queries
i
that contains x Prove that if x S then a sequence x hx x i R exists
i t
by forward induction whereas x hx x i R implies x S by backward
t
induction from x S Finally prove that R PC by noting that t jx j
t
Note that the notion of downwards self reducibility may b e generalized in some
natural ways For example we may say that S is downwards self reducible also
in case it is computationally equivalent via Karp reductions to some set that is
downwards self reducible in the foregoing strict sense Note that Part still
holds
Exercise NP problems that are not self reducible
Assuming that P NP coNP show that there exists a search problem
that is in PC but is not self reducible
Guideline Given S NP co NP n P present relations R R PC such
that S fx R x g fx R x g Then consider the relation
R f x y x y R g f x y x y R g and prove that R PF but
S f g
R
Note that on some instances the reduction may make no queries at all This prevent a
p ossible non viability of the de nition due to very short instances
CHAPTER POLYNOMIAL TIME REDUCTIONS
Prove that if a search problem R is not self reducible then R PF and
the set S fhx y i y s t x y y R g is not Co ok reducible to
R
S fx y s t x y R g
R
Exercise extending any pre x of any solution versus PC and PF
Assuming that P NP present a search problem R in P C P F such that deciding
S is not reducible to the search problem of R
R
Guideline Consider the relation R f x x x f g g f x y x y R g
where R is an arbitrary relation in PC n PF and prove that R PF but S P
R
Exercise In continuation of Exercise present a natural search problem
R in PC such that if factoring integers is intractable then the search problem R
and so also S is not reducible to S
R
R
Guideline Consider the relation R such that N Q R if the integer Q is a non trivial
divisor of the integer N Use the fact that the set of prime numbers is in P
Exercise In continuation of Exercises and show that under suitable
assumptions there exists relations R R PC having the same implicit decision
problem i e fx R x g fx R x g such that R is self reducible
but R is not
Exercise Provide an alternative pro of of Theorem without referring to
the set S fhx y i y s t x y y R g Hint Use Theorem
R
Guideline Theorem implies that R is Co ok reducible to some decision problem in
NP which is reducible to S due to the NP completeness of S
R R
Chapter
NP Completeness
In light of the di culty of settling the P vs NP Question when faced with a hard
problem H in NP we cannot exp ect to prove that H is not in P unconditionally
The b est we can exp ect is a conditional pro of that H is not in P based on the
assumption that NP is di erent from P The contrapositive is proving that if H is
in P then so is any problem in NP i e NP equals P One p ossible way of proving
such an assertion is showing that any problem in NP is p olynomial time reducible
to H This is the essence of the theory of NP completeness
Teaching note Some students heard of NP completeness b efore but we susp ect that
many have missed imp ortant conceptual p oints Sp eci cally we fear that they missed
the p oint that the mere existence of NP complete problems is amazing let alone that
these problems include natural ones such as SAT We b elieve that this situation is a
consequence of presenting the detailed pro of of Co ok s Theorem as the very rst thing
right after de ning NP completeness In contrast we suggest starting with a pro of that
Bounded Halting is NP complete
De nitions
The standard de nition of NP completeness refers to decision problems Below
we will also present a de nition of NP complete or rather PC complete search
problems In b oth cases NP completeness of a problem combines two conditions
is in the class i e b eing in NP or PC dep ending on whether is a
decision or a search problem
Each problem in the class is reducible to This condition is called NP
hardness
Although a p erfectly go o d de nition of NP hardness could have allowed arbitrary
Co ok reductions it turns out that Karp reductions resp Levin reductions su ce
for establishing the NP hardness of all natural NP complete decision resp search
CHAPTER NP COMPLETENESS
problems Consequently NP completeness is usually de ned using this restricted
notion of a p olynomial time reduction
De nition NP completeness of decision problems restricted notion A set
S is NP complete if it is in NP and every set in NP is Karp reducible to S
A set is NP hard if every set in NP is Karp reducible to it Indeed there is no
reason to insist on Karp reductions rather than using arbitrary Co ok reductions
except that the restricted notion su ces for all known demonstrations of NP
completeness and is easier to work with An analogous de nition applies to search
problems
De nition NP completeness of search problems restricted notion A binary
relation R is PC complete if it is in PC and every relation in PC is Levin reducible
to R
In the sequel we will sometimes abuse the terminology and refer to search problems
as NP complete rather than PC complete Likewise we will say that a search
problem is NP hard rather than PC hard if every relation in PC is Levin reducible
to it
We stress that the mere fact that we have de ned a prop erty i e NP completeness
do es not mean that there exist ob jects that satisfy this prop erty It is indeed re
markable that NP complete problems do exist Such problems are universal in
the sense that solving them allows to solve any other reasonable problem i e
problems in NP
The Existence of NP Complete Problems
We suggest not to confuse the mere existence of NP complete problems which
is remarkable by itself with the even more remarkable existence of natural NP
complete problems The following pro of delivers the rst message as well as fo cuses
on the essence of NP completeness rather than on more complicated technical
details The essence of NP completeness is that a single computational problem
may e ectively enco de a wide class of seemingly unrelated problems
Theorem There exist NP complete relations and sets
Pro of The pro of as well as any other NP completeness pro ofs is based on the
observation that some decision problems in NP resp search problems in PC are
rich enough to enco de all decision problems in NP resp all search problems
in PC This fact is most obvious for the generic decision and search problems
denoted S and R and de ned next which are used to derive the simplest pro of
u u
of the current theorem
We consider the following relation R and the decision problem S implicit in
u u
x y s t x y R g Both problems refer to the same type of R i e S f
u u u
t
instances which in turn have the form x hM x i where M is a description of a
THE EXISTENCE OF NP COMPLETE PROBLEMS
standard deterministic Turing machine x is a string and t is a natural number
The number t is given in unary rather than in binary in order to guarantee
that b ounds of the form p oly t are p olynomial rather than exp onential in the
instance s length This implies that various complexity measures e g time and
length that can b e b ounded by a p olynomial in t yield b ounds that are p olynomial
t
in the length of the instance i e jhM x ij O jM j jxj t
t
De nition The relation R consists of pairs hM x i y such that M accepts the
u
def
f input pair x y within t steps where jy j t The corresp onding set S x
u
t
x y R g consists of triples hM x i such that machine M accepts y s t
u
some input of the form x within t steps
It is easy to see that R is in PC and that S is in NP Indeed R is
u u u
t
recognizable by a universal Turing machine which on input hM x i y emulates
t steps of the computation of M on x y Note that this emulation can b e
t
conducted in p oly jM j jxj t p oly j hM x i y j steps and recall that
R is p olynomially b ounded by its very de nition The fact that S NP
u u
follows similarly We comment that u indeed stands for universal i e universal
machine and the pro of extends to any reasonable mo del of computation which
has adequate universal machines
We now turn to show that R and S are NP hard in the adequate sense i e
u u
R is PC hard and S is NP hard We rst show that any set in NP is Karp
u u
reducible to S Let S b e a set in NP and let us denote its witness relation by
u
R that is R is in PC and x S if and only if there exists y such that x y R
Let p b e a p olynomial b ounding the length of solutions in R i e jy j p jxj
R R
for every x y R let M b e a p olynomial time machine deciding membership
R
of alleged x y pairs in R and let t b e a p olynomial b ounding its running
R
time Then the desired Karp reduction maps an instance x for S to the instance
t jxj p jxj
R R
hM x i for S that is
R u
def
t jxj p jxj
R R
x f x hM x i
R
Note that this mapping can b e computed in p olynomial time and that x S if
t jxj p jxj
R R
and only if f x hM x i S Details follow
R u
First note that the mapping f do es dep end of course on S and so it may
dep end on the xed ob jects M p and T which dep end on S Thus computing
R R R
f on input x calls for printing the xed string M copying x and printing a number
R
of s that is a xed p olynomial in the length of x Hence f is p olynomial time
computable Second recall that x S if and only if there exists y such that
jy j p jxj and x y R Since M accepts x y R within t jxj jy j
R R R
steps it follows that x S if and only if there exists y such that jy j p jxj and
R
M accepts x y within t jxj jy j steps It follows that x S if and only if
R R
f x S
u
Instead of requiring that jy j t one may require that M is canonical in the sense that it
reads its entire input b efore halting
Alternatively S NP follows from R PC b ecause for every R PC it holds that
u u
S fx y s t x y Rg is in NP R
CHAPTER NP COMPLETENESS
We now turn to the search version For reducing the search problem of any
R PC to the search problem of R we use essentially the same reduction On
u
t jxj p jxj
R R
input an instance x for R we make the query hM x i to the
R
search problem of R and return whatever the latter returns Note that if x S
u
then the answer will b e no solution whereas for every x and y it holds that
t jxj p jxj
R R
x y R if and only if hM x i y R Thus a Levin reduction
R u
of R to R consists of the pair of functions f g where f is the foregoing Karp
u
reduction and g x y y Note that indeed for every f x y R it holds that
u
x g x y x y R
t
Advanced comment Note that the role of in the de nition of R is to allow
u
placing R in PC In contrast consider the relation R that consists of pairs
u
u
hM x ti y such that M accepts hx y i within t steps Indeed the di erence is
that in R the time b ound t app ears in unary notation whereas in R it app ears
u
u
is PC hard it is not in PC b ecause membership in binary Then although R
u
in R cannot b e decided in p olynomial time see x Going even
u
further we note that omitting t altogether from the problem instance yields a
def
search problem that is not solvable at all That is consider the relation R
H
f hM xi y M x y g which is related to the halting problem Indeed the
search problem of any relation an in particular of any relation in PC is Karp
reducible to the search problem of R but the latter is not solvable at all i e
H
there exists no algorithm that halts on every input and on input x hM xi outputs
y such that x y R if and only such a y exists
H
Bounded Halting and Non Halting
We note that the problem shown to b e NP complete in the pro of of Theorem
is related to the following two problems called Bounded Halting and Bounded
Non Halting Fixing any programming language the instance to each of these
problems consists of a program and a time b ound t presented in unary The
decision version of Bounded Halting resp Bounded Non Halting consists of
determining whether or not there exists an input of length at most t on which
the program halts in t steps resp do es not halt in t steps whereas the search
problem consists of nding such an input
The decision version of Bounded Non Halting refers to a fundamental compu
tational problem in the area of program veri cation sp eci cally the problem of
determining whether a given program halts within a given time bound on al l inputs
of a given length We have mentioned Bounded Halting b ecause it is often re
ferred to in the literature but we b elieve that Bounded Non Halting is much more
The length parameter need not equal the time b ound Indeed a more general version of the
problem refers to two b ounds and t and to whether the given program halts within t steps on
each p ossible bit input It is easy to prove that the problem remains NP complete also in the
case that the instances are restricted to have parameters and t such that t p for any xed
p olynomial p e g p n n rather than p n n as used in the main text
SOME NATURAL NP COMPLETE PROBLEMS
relevant to the pro ject of program veri cation b ecause one seeks programs that
halt on all inputs rather than programs that halt on some input
It is easy to prove that b oth problems are NP complete see Exercise
t
Note that the two decision problems are not complementary i e M may b e
a yes instance of b oth decision problems
Re ection The fact that Bounded Non Halting is probably intractable i e is
intractable provided that P NP is even more relevant to the pro ject of program
veri cation than the fact that the Halting Problem is undecidable The reason
b eing that the latter problem as well as other related undecidable problems refers
to arbitrarily long computations whereas the former problem refers to an explicitly
b ounded number of computational steps Sp eci cally Bounded Non Halting is
concerned with the existence of an input that causes the program to violate a certain
condition i e halting within a given time bound
In light of the foregoing discussion the common practice of bashing Bounded
Non Halting as an unnatural problem seems very o dd at an age in which com
puter programs plays such a central role Nevertheless we will use the term
natural in this traditionally and o dd sense in the next title which actually refers
to natural computational problems that seem unrelated to computation
Some Natural NP Complete Problems
Having established the mere existence of NP complete problems we now turn to
prove the existence of NP complete problems that do not explicitly refer to com
putation in the problem s de nition We stress that thousands of such problems
are known and a list of several hundreds can b e found in
We will prove that deciding the satis ability of prop ositional formulae is NP
complete i e Co ok s Theorem and also present some combinatorial problems
that are NP complete This presentation is aimed at providing a small sample
of natural NP completeness results as well as some to ols towards proving NP
completeness of new problems of interest We start by making a comment regarding
the latter issue
The reduction presented in the pro of of Theorem is called generic b ecause
it explicitly refers to any generic NP problem That is we actually presented
a scheme for the design of reductions from any desired NP problem to the single
problem proved to b e NP complete Indeed in doing so we have followed the def
inition of NP completeness However once we know some NP complete problems
a di erent route is op en to us We may establish the NP completeness of a new
t
Indeed M can not b e a no instance of b oth decision problems but this do es not make
the problems complementary In fact the two decision problems yield a three way partition of
t t
the instances M pairs M such that for every input x of length at most t the
t
computation of M x halts within t steps pairs M for which such halting o ccurs on some
t
inputs but not on al l inputs and pairs M such that there exists no input of length at
most t on which M halts in t steps Note that instances of type are exactly the no instances
of Bounded Non Halting whereas instances of type are exactly the no instances of Bounded
Halting
CHAPTER NP COMPLETENESS
problem by reducing a known NP complete problem to the new problem This
alternative route is indeed a common practice and it is based on the following
simple prop osition
Prop osition If an NP complete problem is reducible to some problem in
NP then is NP complete Furthermore reducibility via Karp reductions resp
Levin reductions is preserved
Pro of The pro of b oils down to asserting the transitivity of reductions Sp ecif
ically the NP hardness of means that every problem in NP is reducible to
which in turn is reducible to Thus by transitivity of reduction see Exer
cise every problem in NP is reducible to which means that is NP hard
and the prop osition follows
Circuit and Formula Satis ability CSAT and SAT
We consider two related computational problems CSAT and SAT which refer in
the decision version to the satis ability of Bo olean circuits and formulae resp ec
tively We refer the reader to the de nition of Bo olean circuits formulae and CNF
formulae that app ear in Sec
Teaching note We suggest establishing the NP completeness of SAT by a reduction
from the circuit satisfaction problem CSAT after establishing the NP completeness
of the latter Doing so allows to decouple two imp ortant parts of the pro of of the NP
completeness of SAT the emulation of Turing machines by circuits and the emulation
of circuits by formulae with auxiliary variables
The NP Completeness of CSAT
Recall that Bo olean circuits are directed acyclic graphs with internal vertices called
gates lab eled by Bo olean op erations of arity either or and external vertices
called terminals that are asso ciated with either inputs or outputs When setting
the inputs of such a circuit all internal no des are assigned values in the natural
way and this yields a value to the output s called an evaluation of the circuit on
the given input The evaluation of circuit C on input z is denoted C z We fo cus
on circuits with a single output and let CSAT denote the set of satis able Bo olean
circuits that is a circuit C is in CSAT if there exists an input z such that C z
We also consider the related relation R f C z C z g
CSAT
Theorem NP completeness of CSAT The set resp relation CSAT resp
R is NP complete resp PC complete
CSAT
Pro of It is easy to see that CSAT NP resp R PC Thus we turn to
CSAT
showing that these problems are NP hard We will fo cus on the decision version
but also discuss the search version
SOME NATURAL NP COMPLETE PROBLEMS
We will present again but for the last time in this b o ok a generic reduction
this time of any NP problem to CSAT The reduction is based on the observa
tion mentioned in Sec see also Exercise that the computation of
p olynomial time algorithms can b e emulated by p olynomial size circuits In the
current context we wish to emulate the computation of a xed machine M on
input x y where x is xed and y varies but jy j p oly jxj and the total num
b er of steps of M x y is p olynomial in jxj jy j Thus x will b e hard wired
into the circuit whereas y will serve as the input to the circuit The circuit itself
denoted C will consists of layers such that each layer will represent an in
x
stantaneous con guration of the machine M and the relation b etween consecutive
con gurations in a computation of this machine will b e captured by uniform
lo cal gadgets in the circuit The number of layers will dep end on x and on the
p olynomial that upp er b ounds the running time of M and an additional gadget
will b e used to detect whether the last con guration is accepting Thus only the
rst layer of the circuit C which will represent an initial con guration with in
x
put pre xed by x will dep end on x The punch line is that determining whether
p oly jxj
for a given x there exists a y f g such that M x y in a given
number of steps will b e reduced to whether there exists a y such that C y
x
Performing this reduction for any machine M that corresp onds to any R PC
R
as in the pro of of Theorem we establish the fact that CSAT is NP complete
Details follow
Recall that we wish to reduce an arbitrary set S NP to CSAT Let R p
R
M and t b e as in the pro of of Theorem i e R is the witness relation of S
R R
whereas p b ounds the length of the NP witnesses M is the machine deciding
R R
membership in R and t is its p olynomial time b ound Without loss of generality
R
and for simplicity supp ose that M is a one tap e Turing machine We will
R
construct a Karp reduction that maps an instance x for S to a circuit denoted
def
f x C such that C y if and only if M accepts the input x y within
x x R
t jxj p jxj steps Thus it will follow that x S if and only if there exists
R R
p jxj
R
y f g such that C y i e if and only if C CSAT The circuit
x x
C will dep end on x as well as on M p and t We stress that M p and t
x R R R R R R
are xed whereas x varies and is thus explicit in our notation
Before describing the circuit C let us consider a p ossible computation of M
x R
on input x y where x is xed and y represents a generic string of length p jxj
R
Such a computation pro ceeds for at most t t jxj p jxj steps and corre
R R
sp onds to a sequence of at most t instantaneous con gurations each of length
t Each such con guration can b e enco ded by t pairs of symbols where the rst
symbol in each pair indicates the contents of a cell and the second symbol indicates
either a state of the machine or the fact that the machine is not lo cated in this
cell Thus each pair is a member of Q f g where is the nite work
alphab et of M Q is its nite set of internal states and is an indication that
R
the machine is not present at a cell The initial con guration includes hx y i as
input and the decision of M x y can b e read from the leftmost cell of the last R
CHAPTER NP COMPLETENESS
con guration With the exception of the rst row the values of the entries in each
row are determined by the entries of the row just ab ove it where this determination
re ects the transition function of M Furthermore the value of each entry in the
R
said array is determined by the values of up to three entries that reside in the row
ab ove it see Exercise Thus the aforementioned computation is represented
by a t t array where each entry enco des one out of a constant number of
p ossibilities which in turn can b e enco ded by a constant length bit string See
Figure
(1,a) (1,-) (0,-)(y1 ,-) (y2 ,-) (-,-) (-,-) (-,-) (-,-) (-,-) initial configuration (with input 110 y1 y2 ) (3,-) (1,b) (0,-)(y1 ,-) (y2 ,-) (-,-) (-,-) (-,-) (-,-) (-,-)
(3,-) (1,-) (0,b) (y1 ,-) (y2 ,-) (-,-) (-,-) (-,-) (-,-) (-,-)
(3,-) (1,c) (0,-)
(3,c) (1,-) (0,-)
(1,-) (1,f) (0,-)
last configuration
Figure An array representing ten consecutive computation steps on input
y y Blank characters as well as the indication that the machine is not present
in the cell are marked by a hyphen The three arrows represent the determina
tion of an entry by the three entries that reside ab ove it The machine underlying
this example accepts the input if and only if the input contains a zero
The circuit C has a structure that corresp onds to the aforementioned array
x
Each entry in the array is represented by a constant number of gates such that when
C is evaluated at y these gates will b e assigned values that enco de the contents
x
of the said entry in the computation of M x y In particular the entries of
R
the rst row of the array are enco ded by hard wiring the reduction s input i e
x and feeding the circuit s input i e y to the adequate input terminals That
is the circuit has p jxj real input terminals corresp onding to y and the
R
hard wiring of constants to the other O t p jxj gates that represent the rst
R
row is done by simple gadgets as in Figure Indeed the additional hard wiring
in the rst row corresp onds to the other xed elements of the initial con guration
We refer to the output convention presented in Sec by which the output is written in
the leftmost cells and the machine halts at the cell to its right
SOME NATURAL NP COMPLETE PROBLEMS
i e the blank symbols and the enco ding of the initial state and of the initial
lo cation cf Figure The entries of subsequent rows will b e enco ded or
rather computed at evaluation time by using constant size circuits that determine
the value of an entry based on the three relevant entries in the row ab ove it Recall
that each entry is enco ded by a constant number of gates and thus these constant
size circuits merely compute the constant size function describ ed in Exercise
In addition the circuit C has a few extra gates that check the values of the
x
entries of the last row in order to determine whether or not it enco des an accepting
con guration Note that the circuit C can b e constructed in p olynomial time
x
from the string x b ecause we just need to enco de x in an appropriate manner as
well as generate a highly uniform grid like circuit of size O t jxj p jxj
R R
Although the foregoing construction of C capitalizes on various sp eci c details
x
of the one tap e Turing machine mo del it can b e easily adapted to other natural
mo dels of e cient computation by showing that in such mo dels the transformation
from one con guration to the subsequent one can b e emulated by a p olynomial
time constructible circuit Alternatively we recall the Cobham Edmonds Thesis
asserting that any problem that is solvable in p olynomial time on some reason
able mo del can b e solved in p olynomial time by a one tap e Turing machine
Turning back to the circuit C we observe that indeed C y if and only
x x
if M accepts the input x y while making at most t t jxj p jxj steps
R R R
Recalling that S fx y s t jy j p jxj x y R g and that M decides
R R
membership in R in time t we infer that x S if and only if f x C CSAT
R x
Furthermore x y R if and only if f x y R It follows that f is a
CSAT
def
Karp reduction of S to CSAT and for g x y y it holds that f g is a Levin
reduction of R to R The theorem follows
CSAT
The NP Completeness of SAT
Recall that Bo olean formulae are sp ecial types of Bo olean circuits i e circuits
having a tree structure We further restrict our attention to formulae given in
conjunctive normal form CNF We denote by SAT the set of satis able CNF
formulae i e a CNF formula is in SAT if there exists an truth assignment such
that We also consider the related relation R f g
SAT
Theorem NP completeness of SAT The set resp relation SAT resp
R is NP complete resp PC complete
SAT
In continuation of Footnote we note that it su ces to check the values of the two leftmost
entries of the last row We assumed here that the circuit propagates a halting con guration to
the last row Alternatively we may check for the existence of an accepting halting con guration
in the entire array since this condition is quite simple
Advanced comment A more e cient construction which generate almost linear sized
e
circuits i e circuits of size O t jxj p jxj is known see
R R
Advanced comment Indeed presenting such circuits is very easy in the case of all natural
mo dels e g the RAM mo del where each bit in the next con guration can b e expressed by a
simple Bo olean formula in the bits of the previous con guration
For an alternative de nition see Section A
CHAPTER NP COMPLETENESS
Pro of Since the set of p ossible instances of SAT is a subset of the set of instances
of CSAT it is clear that SAT NP resp R PC To prove that SAT is NP
SAT
hard we reduce CSAT to SAT and use Prop osition The reduction b oils down
to introducing auxiliary variables in order to cut the computation of an arbitrary
deep circuit into a conjunction of related computations of shallow circuits
i e depth circuits of unbounded fan in which in turn may b e presented as a
CNF formula The aforementioned auxiliary variables hold the possible values of
the internal gates of the original circuit and the clauses of the CNF formula enforce
the consistency of these values with the corresp onding gate op eration For example
if gate and gate feed into gate which is a gate then the corresp onding
i j k
auxiliary variables g g g should satisfy the Bo olean condition g g g
i j k k i j
which can b e written as a CNF with four clauses Details follow
1 2 3 1 2 3 3 g1 g1 g2 g2 g3 g3 g4 g4 gate1 and
or and neg and gate2 or eq eq eq eq
gate3 and and
gate4 neg
Figure Using auxiliary variables i e the g s to cut a depth circuit into
i
a CNF The dashed regions will b e replaced by equivalent CNF formulae The
dashed cycle representing an unbounded fan in and gate is the conjunction of all
constant size circuits which enforce the functionalities of the original gates and
the variable that represents the gate that feed the output terminal in the original
circuit
We start by Karp reducing CSAT to SAT Given a Bo olean circuit C with
n input terminals and m gates we rst construct m constant size formulae on
n m variables where the rst n variables corresp ond to the input terminals of
th
the circuit and the other m variables corresp ond to its gates The i formula will
th
dep end on the variable that corresp ond to the i gate and the variables that
corresp ond to the vertices that feed into this gate i e vertices in case of gate
or gate and a single vertex in case of a gate where these vertices may b e either
input terminals or other gates This constant size formula will b e satis ed by
a truth assignment if and only if this assignment matches the gate s functionality
i e feeding this gate with the corresp onding values result in the corresp onding
output value Note that these constant size formulae can b e written as constant
size CNF formulae in fact as CNF formulae Taking the conjunction of these
Recall that any Bo olean function can b e written as a CNF formula having size that is ex
p onential in the length of its input cf Exercise which in this case is a constant i e
SOME NATURAL NP COMPLETE PROBLEMS
m formulae and the variable asso ciated with the gate that feeds into the output
terminal we obtain a formula in CNF see Figure where n and m
Note that can b e constructed in p olynomial time from the circuit C that is
the mapping of C to f C is p olynomial time computable We claim that C
is in CSAT if and only if is in SAT
Supp ose that for some string s it holds that C s Then assigning to
th th
the i auxiliary variable the value that is assigned to the i gate of C when
evaluated on s we obtain together with s a truth assignment that satis es
This is the case b ecause such an assignment satis es all m constant size
CNFs as well as the variable asso ciated with the output of C
On the other hand if satis es then the rst n bits in corresp ond to an
input on which C evaluates to This is the case b ecause the m constant size
CNFs guarantee that the variables of are assigned values that corresp ond
to the evaluation of C on the rst n bits of while the fact that the variable
asso ciated with the output of C has value true guarantees that this evaluation
of C yields the value
Note that the latter mapping of to its n bit pre x is the second mapping
required by the de nition of a Levin reduction
Thus we have established that f is a Karp reduction of CSAT to SAT and that
augmenting f with the aforementioned second mapping yields a Levin reduction
of R to R
CSAT SAT
Comment The fact that the second mapping required by the de nition of a
Levin reduction is explicit in the pro of of the validity of the corresp onding Karp
reduction is a fairly common phenomenon Actually see Exercise typical pre
sentations of Karp reductions provide two auxiliary p olynomial time computable
mappings in addition to the main mapping of instances from one problem e g
CSAT to instances of another problem e g SAT The rst auxiliary mapping
is of solutions for the preimage instance e g of CSAT to solutions for the image
instance of the reduction e g of SAT whereas the second mapping go es the other
way around For example the pro of of the validity of the Karp reduction of CSAT to
SAT denoted f sp eci ed two additional mappings h and g such that C s R
CSAT
implies f C h C s R and f C R implies C g C R
SAT SAT CSAT
Sp eci cally in the pro of of Theorem we used h C s s a a where a
m i
th
is the value assigned to the i gate in the evaluation of C s and g C b eing
the n bit pre x of Note that only the main mapping i e f and the second
auxiliary mapping i e g are required in the de nition of a Levin reduction
SAT Note that the formulae resulting from the Karp reduction presented in
the pro of of Theorem are in conjunctive normal form CNF with each clause
either or Indeed note that the Bo olean functions that we refer to here dep ends on
Bo olean variables since they indicate whether or not the corresp onding values resp ect the gate s functionality
CHAPTER NP COMPLETENESS
referring to at most three variables Thus the foregoing reduction actually estab
lishes the NP completeness of SAT i e SAT restricted to CNF formula with up
to three variables p er clause Alternatively one may Karp reduce SAT i e satis
ability of CNF formula to SAT i e satis ability of CNF formula by replacing
long clauses with conjunctions of three variable clauses using auxiliary variables
see Exercise Either way we get the following result where the furthermore
part is proved by an additional reduction
Prop osition SAT is NP complete Furthermore the problem remains NP
complete also if we restrict the instances such that each variable appears in at most
three clauses
Pro of The furthermore part is proved by a reduction from SAT We just replace
each o ccurrence of each Bo olean variable by a new copy of this variable and add
clauses to enforce that all these copies are assigned the same value Sp eci cally if
variable z o ccurs t times in the original CNF formula then we introduce t new
t th
variables i e its copies denoted z z and replace the i o ccurrence of
i i i
z in by z In addition we add the clauses z z for i t where t
is understo o d as Thus each variable app ears at most three times in the new
i i i i
formula Note that the clause z z is logically equivalent to z z
and thus the conjunction of the aforementioned t clauses is logically equivalent to
t
z z z The validity of the reduction follows
Related problems Note that instances of SAT can b e viewed as systems of
Bo olean conditions over Bo olean variables Such systems can b e emulated by vari
ous types of systems of arithmetic conditions implying the NP hardness of solving
the latter types of systems Examples include systems of integer linear inequalities
see Exercise and systems of quadratic equalities see Exercise
Combinatorics and Graph Theory
Teaching note The purp ose of this section is to exp ose the students to a sample
of NP completeness results and pro of techniques i e the design of reductions among
computational problems
We present just a few of the many app ealing combinatorial problems that are known
to b e NP complete Throughout this section we fo cus on the decision versions of
the various problems and adopt a more informal style Sp eci cally we will present
a typical decision problem as a problem of deciding whether a given instance which
b elongs to a set of relevant instances is a yes instance or a no instance rather
than referring to deciding membership of arbitrary strings in a set of yes instances
For further discussion of this style and its rigorous formulation see Section We
will also neglect showing that these decision problems are in NP indeed for natural
problems in NP showing membership in NP is typically straightforward
SOME NATURAL NP COMPLETE PROBLEMS
Set Cover We start with the set cover problem in which an instance consists of
a collection of nite sets S S and an integer K and the question for decision
m
S
m
is whether or not there exist at most K sets that cover S i e indices
i
i
S S
m K
S i i such that S
i K i
j
i j
Prop osition Set Cover is NP complete
Pro of Sketch We sketch a reduction of SAT to Set Cover For a CNF formula
with m clauses and n variables we consider the sets S S S S
t n t
f n f
f mg such that S resp S is the set of the indices of the clauses of
i t
i f
th th
that are satis ed by setting the i variable to true resp false That is if the i
th
variable app ears unnegated resp negated in the j clause then j S resp
i t
j S Indeed S S equals the set of clauses containing an o ccurrence of
i t
i f i f
th
the i variable and the union of all these n sets equals f mg Now on input
def
the reduction outputs the Set Cover instance f S S n where
n
S S fm ig and S S fm ig for i n
i i
i t
i f
Note that f is computable in p olynomial time and that if is satis ed by
i ng covers f m ng since then the collection fS
n i
i
n m i ng covers m i ng covers m and fS fS
i i
i i
fm m ng Thus S AT implies that f is a yes instance of Set
Cover On the other hand each cover of fm m ng f m ng must
include either S or S for each i since these are the only sets that cover the
i i
element m i Thus a cover of f m ng using n of the S s must contain for
j
every i either S or S but not b oth Setting accordingly i e if and
i i i i
i ng covers f mg only if S is in the cover implies that fS
i i
i
which in turn implies that satis es Thus if f is a yes instance of
n
Set Cover then SAT
Exact Cover and XC The exact cover problem is similar to the set cover prob
lem except that here the sets that are used in the cover are not allowed to intersect
That is each element in the universe should b e covered by exactly one set in the
cover Restricting the set of instances to sequences of sets each having exactly three
elements we get the restricted problem called Exact Cover XC where it is
unnecessary to sp ecify the number of sets to b e used in the cover The problem XC
is rather technical but it is quite useful for demonstrating the NP completeness of
other problems by reducing XC to them see for example Exercise
Prop osition Exact Cover is NP complete
Indeed it follows that the Exact Cover in which sets of arbitrary size are allowed
is NP complete This follows b oth for the case that the number of sets in the desired
cover is unsp eci ed and for the various cases in which this number is b ounded i e
upp er b ounded or lower bounded or b oth
Clearly in case of Set Cover the two formulations i e asking for exactly K sets or at most
K sets are computationally equivalent
CHAPTER NP COMPLETENESS
Pro of Sketch The reduction is obtained by comp osing three reductions We rst
reduce a restricted case of SAT to a restricted version of Set Cover denoted SC
in which each set has at most three elements and an instance consists as in the
general case of a sequence of nite sets as well as an integer K Sp eci cally
we refer to SAT instances that are restricted such that each variable app ears in
at most three clauses and recall that this restricted problem is NP complete see
Prop osition Actually we further reduce this restricted version of SAT to a
more restricted version denoted r SAT in which each literal app ears in at most
two clauses see Exercise Now we reduce r SAT to SC by using the very
same reduction presented in the pro of of Prop osition while observing that the
size of each set in the reduced instance is at most three i e one more than the
number of o ccurrences of the corresp onding literal
Next we reduce SC to the following restricted case of Exact Cover denoted
XC in which each set has at most three elements an instance consists of a sequence
of nite sets as well as an integer K and the question is whether there exists an
exact cover with at most K sets The reduction maps an instance S S K
m
of SC to the instance C K such that C is a collection of all subsets of each of the
sets S S Since each S has size at most we introduce at most non empty
m i
subsets p er each such set and the reduction can b e computed in p olynomial time
The reader may easily verify the validity of this reduction see Exercise
Finally we reduce XC to XC Consider an instance S S K of XC
m
S
m
and supp ose that S n If n K then this is de nitely a no instance
i
i
which can b e mapp ed to a dummy no instance of XC and so we assume that
def
x K n Note that x represents the excess covering ability of an
exact cover having K sets each having three elements Thus we augment the set
system with x new elements denoted n K and replace each S such that
i
jS j by a sub collection of sets that cover S as well as arbitrary elements
i i
from fn K g That is in case jS j the set S is replaced by the sub
i i
collection S fn g S f K g whereas a singleton S is replaced by the sets
i i i
S fj j g for every j j in fn K g In addition we add all p ossible
i
subsets of fn K g This completes the description of the third reduction
the validity of which is left as an exercise see Exercise
Vertex Cover Indep endent Set and Clique Turning to graph theoretic
problems see Section A we start with the Vertex Cover problem which is
a sp ecial case of the Set Cover problem The instances consists of pairs G K
where G V E is a simple graph and K is an integer and the problem is whether
or not there exists a set of at most K vertices that is incident to all graph edges
i e each edge in G has at least one endp oint in this set Indeed this instance
of Vertex Cover can b e viewed as an instance of Set Cover by considering the
collection of sets S where S denotes the set of edges incident at vertex v
v v V v
i e S fe E v eg Thus the NP hardness of Set Cover follows from the
v
NP hardness of Vertex Cover but this implication is unhelpful for us here we
already know that Set Cover is NP hard and we wish to prove that Vertex Cover
is NP hard We also note that the Vertex Cover problem is computationally
SOME NATURAL NP COMPLETE PROBLEMS
equivalent to the Independent Set and Clique problems see Exercise and
thus it su ces to establish the NP hardness of one of these problems
Prop osition The problems Vertex Cover Independent Set and Clique
are NP complete
Teaching note The following reduction is not the standard one see Exercise
but is rather adapted from the FGLSS reduction see This is done in anticipation
of the use of the FGLSS reduction in the context of the study of the complexity of
approximation cf e g or Sec Furthermore although the following
reduction creates a larger graph the author nds it more clear than the standard
reduction
Pro of Sketch We show a reduction from SAT to Independent Set On input
a CNF formula with m clauses and n variables we construct a graph with m
vertices denoted G The vertices are group ed in m cliques each corresp onding
to one of the clauses and containing vertices that corresp ond to the truth as
signments to the variables in the clause that satisfy the clause In addition to
the internal edges of these m cliques we add an edge b etween each pair of vertices
that corresp ond to partial assignments that are mutual ly inconsistent That is
th
if a sp eci c satisfying assignment to the variables of the i clause is inconsis
th
tent with some satisfying assignment to the variables of the j clause then we
connect the corresp onding vertices by an edge In particular no edges are placed
b etween cliques that represent clauses that share no common variable Note that
the internal edges of the m cliques may b e viewed as a sp ecial case of the edges
connecting mutually inconsistent partial assignments To summarize on input
the reduction outputs the pair G m where G is the aforementioned graph and
m is the number of clauses in
We stress that each clique of the graph G contains only vertices that corresp ond
to partial assignments that satisfy the corresp onding clause that is the single
partial assignments that do es not satisfy this clause is not represented as a vertex in
G Recall that the edges placed among vertices represent partial assignments that
are not mutually consistent Thus valid truth assignments to the entire formula
corresp ond to indep endent sets in G and the size of the latter represents the
number of clauses that are satis ed by the assignment These observations underlie
the validity of the reduction which is argued next
Note that if is satis able by a truth assignment then there are no edges
b etween the m vertices that corresp ond to the partial satisfying assignments for
individual clauses derived from This assertion holds b ecause any truth assign
ment to yields an indep endent set that contains a single vertex from each clique
that corresp onds to a clause that is satis ed by such that this vertex corresp onds
to the partial assignment to this clause s variables derived from Thus if is a
satisfying assignment then the aforementioned indep endent set contains a vertex
from each of the m cliques It follows that SAT implies that G has an inde
p endent set of size m On the other hand any indep endent set of size m in G
must contain exactly one vertex in each of the m cliques and thus induces a truth
CHAPTER NP COMPLETENESS
assignment that satis es This assertion follows by combining the following two
facts
Each indep endent set in G induces a consistent truth assignment to
b ecause the partial assignments selected in the various cliques must b e
consistent
Any indep endent set that contains a vertex from a sp eci c clique induces a
truth assignment that satis es the corresp onding clause
Thus if G has an indep endent set of size m then SAT
Graph Colorability G C In this problem the instances are graphs and the
question is whether or not the graph can b e colored using three colors such that
neighboring vertices are not assigned the same color
Prop osition Graph Colorability is NP complete
Pro of Sketch We reduce SAT to G C by mapping a CNF formula to the graph
G that consists of two sp ecial designated vertices a gadget p er each variable
of a gadget p er each clause of and edges connecting some of these comp onents
as follows
The two designated vertices are called ground and false and are connected
by an edge that ensures that they must b e given di erent colors in any
coloring of G We will refer to the color assigned to the vertex ground resp
false by the name ground resp false The third color will b e called
true
The gadget asso ciated with variable x is a pair of vertices asso ciated with
the two literals of x i e x and x These vertices are connected by an
edge and each of them is also connected to the vertex ground Thus in any
coloring of G one of the vertices asso ciated with the variable is colored
true and the other is colored false
2 x T1 1 M 3 y T2
T3
Figure The clause gadget and its sub gadget In a generic coloring of the
sub gadget it must hold that if x y then x y Thus if the three terminals
of the gadget are assigned the same color then M is also assigned the color
SOME NATURAL NP COMPLETE PROBLEMS
The gadget asso ciated with a clause C is depicted in Figure It contains a
master vertex denoted M and three terminal vertices denoted T T and
T The master vertex is connected by edges to the vertices ground and
false and thus in any coloring of G the master vertex must b e colored
true The gadget has the prop erty that it is p ossible to color the terminals
with any combination of the colors true and false except for coloring all
terminals with false That is in any coloring of G if no terminal of a
clause gadget is colored ground then at least one of these terminals is colored
true
The terminals of the gadget asso ciated with clause C will b e identi ed with
the vertices of variable gadgets that are asso ciated with the corresp onding
literals app earing in C This means that each clause gadget shares its ter
minals with the corresp onding variable gadgets and that the various clause
gadgets are not vertex disjoint but may rather share some terminals i e
those asso ciated with literals that app ear in several clauses See Figure
The aforementioned asso ciation forces each terminal to b e colored either true
or false in any coloring of G By the foregoing discussion it follows that
in any coloring of G at least one terminal of each clause gadget must b e
colored true
GROUND FALSE the two designated verices
variable gadgets
clause gadgets
Figure A single clause gadget and the relevant variables gadgets
Verifying the validity of the reduction is left as an exercise see Exercise
Digest The reductions presented in the current section are depicted in Fig
ure where b old arrows indicate reductions presented explicitly in the pro ofs of
Alternatively we may use disjoint gadgets and connect each terminal with the corresp ond
ing literal in the corresp onding vertex gadget Such a connection i e an auxiliary gadget
should force the two end p oints to have the same color in any coloring of the graph
CHAPTER NP COMPLETENESS
the various prop ositions indicated by their index Note that r SAT and SC are
only mentioned inside the pro of of Prop osition
(4.9) (4.9) r3SAT 3SC 3XC
(4.9)
4.6 4.7 4.8 CSAT SAT 3SAT SC
4.10 VC IS 4.11 Clique
G3C
Figure The non generic reductions presented in Section
NP sets that are Neither in P nor NP Complete
As stated in Section thousands of problems have b een shown to b e NP complete
cf Ap dx which contains a list of more than three hundreds main entries
Things reached a situation in which p eople seem to exp ect any NP set to b e either
NP complete or in P This naive view is wrong Assuming NP P there exist
sets in NP that are neither NP complete nor in P where here NP hardness al lows
also Cook reductions
Theorem Assuming NP P there exist a set T in NP n P such that some
sets in NP are not Cook reducible to T
Theorem asserts that if NP P then NP is partitioned into three non empty
classes the class P the class of problems to which NP is Co ok reducible and the
rest denote NPI We already know that the rst two classes are not empty
and Theorem establishes the non emptiness of NPI under the condition that
NP P which is actually a necessary condition b ecause if NP P then every
set in NP is Co ok reducible to any other set in NP
The following pro of of Theorem presents an unnatural decision problem
in NPI We mention that some natural decision problems e g some that are
computationally equivalent to factoring are conjectured to b e in NPI We also
mention that if NP coNP where coNP ff g n S S N P g then
def
NP coNP P NPI holds as a corollary to Theorem Thus if
NP coNP then n P is a natural subset of NPI and the non emptiness
of NPI follows provided that P Recall that Theorem establishes the
non emptiness of NPI under the seemingly weaker assumption that NP P
NP SETS THAT ARE NEITHER IN P NOR NP COMPLETE
Teaching note We recommend either stating Theorem without a pro of or merely
presenting the pro of idea
Pro of Sketch The basic idea is mo difying an arbitrary set in NP n P so as to
fail all p ossible reductions from NP to the mo di ed set as well as all p ossible
p olynomial time decision pro cedures for the mo di ed set Sp eci cally starting
with S NP n P we derive S S such that on one hand there is no p olynomial
time reduction of S to S while on the other hand S NP n P The pro cess of
mo difying S into S pro ceeds in iterations alternatively failing a p otential reduction
by dropping su ciently many strings from the rest of S and failing a p otential
decision pro cedure by including su ciently many strings from the rest of S
Sp eci cally each p otential reduction of S to S can b e failed by dropping nitely
many elements from the current S whereas each p otential decision pro cedure can
b e failed by keeping nitely many elements of the current S These two assertions
are based on the following two corresp onding facts
Any p olynomial time reduction of any set not in P to any nite set e g
a nite subset of S must fail b ecause only sets in P are Co ok reducible to
a nite set Thus for any nite set F and any p otential reduction i e
a p olynomial time oracle machine there exists an input x on which this
reduction to F fails
We stress that the aforementioned reduction fails while the only queries that
are answered p ositively are those residing in F Furthermore the aforemen
tioned failure is due to a nite set of queries i e the set of all queries made
by the reduction when invoked on an input that is smaller or equal to x
Thus for every nite set F S S any reduction of S to S can b e
failed by dropping a nite number of elements from S and without dropping
elements of F
For every nite set F any p olynomial time decision pro cedure for S n F
must fail b ecause S is Co ok reducible to S n F Thus for any p otential
decision pro cedure i e a p olynomial time algorithm there exists an input
x on which this pro cedure fails
We stress that this failure is due to a nite pre x of S n F i e the set
fz S n F z xg Thus for every nite set F any p olynomial time
decision pro cedure for S n F can b e failed by keeping a nite subset of S n F
As stated the pro cess of mo difying S into S pro ceeds in iterations alternatively
failing a p otential reduction by dropping nitely many strings from the rest of S
and failing a p otential decision pro cedure by including nitely many strings from
the rest of S This can b e done e ciently b ecause it is inessential to determine the
rst possible points of alternation in which su ciently many strings were dropp ed
resp included to fail the next p otential reduction resp decision pro cedure It
su ces to guarantee that adequate p oints of alternation alb eit highly non optimal
ones can b e e ciently determined Thus S is the intersection of S and some set
in P which implies that S NP Following are some comments regarding the
implementation of the foregoing idea
CHAPTER NP COMPLETENESS
The rst issue is that the foregoing plan calls for an e ective enumeration of
all p olynomial time oracle machines resp p olynomial time algorithms However
none of these sets can b e enumerated by an algorithm Instead we enumerate
all corresp onding machines along with all p ossible p olynomials and for each pair
M p we consider executions of machine M with time b ound sp eci ed by the
p olynomial p That is we use the machine M obtained from the pair M p by
p
susp ending the execution of M on input x after p jxj steps We stress that we do
not know whether machine M runs in p olynomial time but the computations of
any p olynomial time machine is covered by some pair M p
Next let us clarify the pro cess in which reductions and decision pro cedures are
ruled out We present a construction of a lter set F in P such that the nal set
S will equal S F Recall that we need to select F such that each p olynomial
time reduction of S to S F fails and each p olynomial time pro cedure for deciding
S F fails The key observation is that for every nite F each p olynomial time
reduction of S to S F F fails whereas for every nite F each p olynomial
time pro cedure for deciding S F n F fails Furthermore each of these failures
o ccur on some input and such an input can b e determined by nite p ortions of
S and F Thus we alternate b etween failing p ossible reductions and decision
pro cedures on some inputs while not trying to determine the optimal p oints
of alternation but rather determining p oints of alternation in an e cient manner
which in turn allows for e ciently deciding membership in F Sp eci cally we
let F fx f jxj mo d g where f N f g N will b e de ned such that
i each of the rst f n machines is failed by some input of length at most n
and ii the value f n can b e computed in p oly n time
The value of f n is de ned by the following pro cess that p erforms exactly
n computation steps where cubic time is a rather arbitrary choice The pro cess
st
pro ceeds in an a priori unknown number of iterations where in the i iteration
st
we try to nd an input on which the i mo di ed machine fails Sp eci cally
st
in the i iteration we scan all inputs in lexicographic order until we nd an
st
input on which the i mo di ed machine fails where this machine is an oracle
machine if i is o dd and a standard machine otherwise If we detect a failure of
st
the i machine then we increment i and pro ceed to the next iteration When
we reach the allowed number of steps i e n steps we halt outputting the current
value of i i e the current i is output as the value of f n Needless to say this
st
description is heavily based on determining whether or not the i machine fails
on sp eci c inputs Intuitively these inputs will b e much shorter than n and so
p erforming these decisions in time n or so is not out of the question see next
paragraph
st
In order to determine whether or not a failure of the i machine o ccurs
on a particular input x we need to emulate the computation of this machine on
input x as well as determine whether x is in the relevant set which is either S or
S S F Recall that if i is even then we need to fail a standard machine
which attempts to decide S and otherwise we need to fail an oracle machine
which attempts to reduce S to S Thus for even i we need to determine
whether x is in S S F whereas for o dd i we need to determine whether
REFLECTIONS ON COMPLETE PROBLEMS
x is in S as well as whether some other strings which app ear as queries are in
S Deciding membership in S NP can b e done in exp onential time by using
the exhaustive search algorithm that tries all p ossible NP witnesses Indeed this
means that when computing f n we may only complete the treatment of inputs
that are of logarithmic in n length but anyhow in n steps we can not hop e to
reach in our lexicographic scanning strings of length greater than log n As for
deciding membership in F this requires ability to compute f on adequate integers
That is we may need to compute the value of f n for various integers n but as
noted n will b e much smaller than n since n p oly jxj p oly log n Thus
the value of f n is just computed recursively while counting the recursive steps
in our total number of steps The p oint is that when considering an input x
we may need the values of f only on f p jxj g where p is the p olynomial
i i
st
b ounding the running time of the i mo di ed machine and obtaining such
a value takes at most p jxj steps We conclude that the number of steps
i
st
p erformed towards determining whether or not a failure of the i machine
o ccurs on the input x is upp er b ounded by an exp onential function of jxj
As hinted in the foregoing paragraph the pro cedure will complete n steps
much b efore examining inputs of length greater than log n but this do es not
matter What matters is that f is unbounded see Exercise Furthermore
by construction f n is computed in p oly n time
Comment The pro of of Theorem actually establishes that for every S P
there exists S P such that S is Karp reducible to S but S is not Cook reducible
to S Thus if P NP then there exists an in nite sequence of sets S S
in NP n P such that S is Karp reducible to S but S is not Co ok reducible
i i i
to S That is there exists an in nite hierarchy of problems alb eit unnatural
i
ones all in NP such that each problem is easier than the previous ones in the
sense that it can b e reduced to the previous problems while these problems cannot
b e reduced to it
Re ections on Complete Problems
This b o ok will p erhaps only b e understo o d by those who have
themselves already thought the thoughts which are expressed in
it or similar thoughts It is therefore not a text b o ok Its ob ject
would b e attained if it a orded pleasure to one who read it with
understanding
Ludwig Wittgenstein Tractatus Logico Philosophicus
Indeed this section should b e viewed as an invitation to meditate together on
questions of the type what enables the existence of complete problems Accordingly
We do not b other to present a more e cient implementation of this pro cess That is we may
a ord to recompute f n every time we need it rather than store it for later use
The said Karp reduction of S to S maps x to itself if x F and otherwise maps x to a
xed no instance of S
CHAPTER NP COMPLETENESS
the style is intentionally naive and imprecise this entire section may b e viewed as
an op en ended exercise asking the reader to consider substantiations of the vague
text
Teaching note This section exercise may b e unsuitable for most undergraduate stu
dents We de nitely do not intend it for presentation in class
We know that NP complete problems exist The question we ask here is what
asp ects in our mo deling of problems enables the existence of complete problems
We should of course b ear in mind that completeness refers to a class of problems
the complete problem should enco de each problem in the class and b e itself in
the class Since the rst asp ect hereafter referred to as enco dability of a class is
amazing enough at least to a layman we start by asking what enables it We
identify two fundamental paradigms regarding the mo deling of problems that
seem essential to the enco dability of any in nite class of problems
Each problem refer to an in nite set of p ossible instances
The sp eci cation of each problem uses a nite description e g an algorithm
that enumerates all the p ossible solutions for any given instance
These two paradigms seem somewhat con icting yet put together they suggest the
de nition of a universal problem Sp eci cally this problem refers to instances of
the form D x where D is a description of a problem and x is an instance to that
problem and a solution to the instance D x is a solution to x with resp ect to
the problem describ ed by D Intuitively this universal problem can enco de any
other problem provided that problems are mo deled in a way that conforms with
the foregoing paradigms solving the universal problem allows solving any other
problem
Note that the foregoing universal problem is actually complete with resp ect
to the class of all problems but it is not complete with resp ect to any class that
contains only algorithmically solvable problems b ecause this universal problem
is not solvable Turning our attention to classes of solvable problems we seek ver
sions of the universal problem that are complete for these classes One archetypical
di culty that arises is that given a description D as part of the instance to the
universal problem we cannot tell whether or not D is a description of a problem
in a predetermined class C b ecause this decision problem is unsolvable This fact
is relevant b ecause if the universal problem requires solving instances that refer
to a problem not in C then intuitively it cannot b e itself in C
This seems the most naive notion of a description of a problem An alternative notion of
a description refers to an algorithm that recognizes all valid instance solution pairs as in the
de nition of NP However at this p oint we allow also non e ective descriptions as giving rise
to the Halting Problem
Recall however that the universal problem is not algorithmically solvable Thus b oth
parts of the implication are false i e this problem is not solvable and needless to say there
exists unsolvable problems Indeed the notion of a problem is rather vague at this stage it
certainly extends b eyond the set of all solvable problems
Here we ignore the p ossibility of using promise problems which do enable avoiding such
instances without requiring anybo dy to recognize them Indeed using promise problems resolves
this di culty but the issues discussed following the next paragraph remain valid
REFLECTIONS ON COMPLETE PROBLEMS
Before turning to the resolution of the foregoing di culty we note that the
aforementioned mo deling paradigms are pivotal to the theory of computation at
large In particular so far we made no reference to any complexity consideration
Indeed a complexity consideration is the key to resolving the foregoing di culty
The idea is mo difying any description D into a description D such that D is
always in C and D agrees with D in the case that D is in C i e in this case
they describ ed exactly the same problem We stress that in the case that D is
not in C the corresp onding problem D may b e arbitrary as long as it is in C
Such a mo di cation is p ossible with resp ect to many complexity theoretic classes
We consider two di erent types of classes where in b oth cases the class is de ned
in terms of the time complexity of algorithms that do something related to the
problem e g recognize valid solutions as in the de nition of NP
Classes de ned by a single time bound function t e g t n n In this
case any algorithm D is mo di ed to the algorithm D that on input x
emulates up to t jxj steps of the execution of D x The mo di ed version
of the universal problem treats the instance D x as D x This version
can enco de any problem in the said class C
But will this version of the universal problem b e itself in C The answer
dep ends b oth on the e ciency of emulation in the corresp onding computa
tional mo del and on the growth rate of t For example for triple exp onential
t the answer will b e de nitely yes b ecause t jxj steps can b e emulated in
p oly t jxj time in any reasonable mo del while t j D x j t jxj
p oly t jxj On the other hand in most reasonable mo dels the emulation
of t jxj steps requires t jxj time while for any p olynomial t it holds that
t n O t n
Classes de ned by a family of in nitely many functions of di erent growth
rate e g p olynomials We can of course select a function t that grows
faster than any function in the family and pro ceed as in the prior case but
then the resulting universal problem will de nitely not b e in the class
Note that in the current case a complete problem will indeed b e striking b e
cause in particular it will b e asso ciated with one function t that grows more
mo derately than some other functions in the family e g a xed p olynomial
grows more mo derately than other p olynomials Seemingly this means that
the algorithm describing the universal machine should b e faster than some
algorithms that describ e some other problems in the class This impression
presumes that the instances of b oth problems are approximately of the same
length and so we intensionally violate this presumption by arti cially increas
ing the length of the description of the instances to the universal problem For
example if D is asso ciated with the time b ound t then the instance D x
D
1
2
t t jxj
D
0
to the universal problem is presented as say D x where in
the case of NP we used t n n
We b elieve that the last item explains the existence of NP complete problems But
what about the NP completeness of SAT
CHAPTER NP COMPLETENESS
We rst note that the NP hardness of CSAT is an immediate consequence of
the fact that Bo olean circuits can emulate algorithms This fundamental fact is
ro oted in the notion of an algorithm which p ostulates the simplicity of a single
computational step and holds for any reasonable mo del of computation Thus for
every D and x the problem of nding a string y such that D x y is enco ded
as nding a string y such that C y where C is a Bo olean circuit that
D x D x
is easily derived from D x In contrast to the fundamental fact underlying the
NP hardness of CSAT the NP hardness of SAT relies on a clever trick that allows
to enco de instances of CSAT as instances of SAT
As stated the NP completeness of SAT is proved by enco ding instances of
CSAT as instances of SAT Similarly the NP completeness of other new problems
is proved by enco ding instances of problems that are already known to b e NP
complete Typically these enco dings op erate in a lo cal manner mapping small
comp onents of the original instance to lo cal gadgets in the pro duced instance
Indeed these problem sp eci c gadgets are the core of the enco ding phenomenon
Presented with such a gadget it is typically easy to verify that it works Thus
one cannot be surprised by most of these gadgets but the fact that they exist for
thousands of natural problem is de nitely amazing
Exercises
Exercise Prove that Bounded Halting and Bounded Non Halting are NP
complete where the problems are de ned as follows The instance consists of a pair
t
M where M is a Turing machine and t is an integer The decision version of
Bounded Halting resp Bounded Non Halting consists of determining whether
or not there exists an input of length at most t on which M halts resp do es not
halt in t steps whereas the search problem consists of nding such an input
Guideline Either mo dify the pro of of Theorem or present a reduction of say the
search problem of R to the search problem of Bounded Non Halting Indeed the
u
exercise is more straightforward in the case of Bounded Halting
Exercise In the pro of of Theorem we claimed that the value of each entry
in the array of con gurations of a machine M is determined by the values of the
three entries that reside in the row ab ove it as in Figure Present a function
f where Q f g that substantiates this claim
M
Guideline For example for every it holds that f
M
More interestingly if the transition function of M maps q to p
then for every Q it holds that f q p and
M
f q
M
Exercise Present and analyze a reduction of SAT to SAT
The fact that CSAT is in NP is a consequence of the fact that the circuit evaluation problem
is solvable in p olynomial time
REFLECTIONS ON COMPLETE PROBLEMS
th
Guideline For a clause C consider auxiliary variables such that the i variable indicates
whether one of the rst i literals is satis ed and replace C by a CNF that uses the
t
original variables of C as well as the auxiliary variables For example the clause x
i
i
is replaced by the conjunction of CNFs that are logically equivalent to the formulae
y x x y y x for i t and y We comment that this is
i i i t
not the standard reduction but we nd it conceptually more app ealing The standard
t
reduction replaces the clause x by the conjunction of the CNFs x x z
i
i
z x z for i t and z
i i i t
Exercise e cient solvability of SAT In contrast to the NP completeness
of SAT prove that SAT i e the satis ability of CNF formulae is in P
Guideline Consider the following forcing process for CNF formulae If the formula
contains a singleton clause i e a clause having a single literal then the corresp onding
variable is assigned the only value that satis es the clause and the formula is simpli ed
accordingly p ossibly yielding a constant formula which is either true or false The
pro cess is rep eated until the formula is either a constant or contains only non singleton
clauses Note that a formula is satis able if and only if the formula obtained from
by the forcing pro cess is satis able Now consider the following algorithm for solving the
search problem asso ciated with SAT
Cho ose an arbitrary variable in For each f g denote by the formula
obtained from by assigning this variable the value and applying the forcing
pro cess to the resulting formula
Note that is either a Bo olean constant or a CNF formula which is a conjunction
of some clauses of
If for some f g the formula equals the constant true then we halt with
a satisfying assignment for the original formula
If b oth assignments yield the constant false i e for every f g the formula
equals false then we halt asserting that the original formula is unsatis able
Otherwise i e for each f g the formula is a non constant CNF
formula we select f g arbitrarily set and goto Step
Proving the correctness of this algorithm b oils down to observing that the arbitrary choice
made in Step is immaterial Indeed this observation relies on the fact that we refer to
CNF formulae which implies that the forcing pro cess either yields a constant or a CNF
formula which is a conjunction of some clauses of the original
Exercise Integer Linear Programming Prove that the following prob
lem is NP hard An instance of the problem is a systems of linear inequalities
say with integer constants and the problem is to determine whether the system
has an integer solution A typical instance of this decision problem follows
x y z
x z
Proving that the problem is in NP requires showing that if a system of linear inequalities
has an integer solution then it has an integer solution in which all numbers are of length that is
p olynomial in the length of the description of the system Such a pro of is b eyond the scop e of
the current textb o ok
CHAPTER NP COMPLETENESS
x
x
Guideline Reduce from SAT Sp eci cally consider an arithmetization of the input CNF
by replacing with addition and x by x Thus each clause gives rise to an inequality
e g the clause x y is replaced by the inequality x y which simpli es
to x y Enforce a solution by introducing inequalities of the form x and
x for every variable x
Exercise Maximum Satis ability of Linear Systems over GF Prove
that the following problem is NP complete An instance of the problem consists of
a systems of linear equations over GF and an integer k and the problem is to
determine whether there exists an assignment that satis es at least k equations
Note that the problem of determining whether there exists an assignment that
satis es all the equations is in P
Guideline Reduce from SAT using the following arithmetization Replace each clause
t
that contains t literals by linear GF equations that corresp ond to the
di erent non empty subsets of these literals and assert that their sum mo dulo equals
one for example the clause x y is replaced by the equations x y x and
y Identifying ffalse true g with f g prove that if the original clause is satis ed
t
v then exactly of the corresp onding equations are satis ed by a Bo olean assignment
by v whereas if the original clause is unsatis ed by v then none of the corresp onding
equations is satis ed by v
Exercise Satis ability of Quadratic Systems over GF Prove that the
following problem is NP complete An instance of the problem consists of a system
of quadratic equations over GF and the problem is to determine whether there
exists an assignment that satis es all the equations Note that the result holds also
for systems of quadratic equations over the reals by adding conditions that force
values in f g
Guideline Start by showing that the corresp onding problem for cubic equations is NP
complete by a reduction from SAT that maps the clause x y z to the equation
x y z Reduce the problem for cubic equations to the problem for
quadratic equations by introducing auxiliary variables that is given an instance with
variables x x introduce the auxiliary variables x s and add equations of the form
n i j
x x x
i j i j
Exercise restricted versions of SAT Prove that the following restricted
version of SAT denoted r SAT is NP complete An instance of the problem con
sists of a CNF formula such that each literal app ears in at most two clauses and
the problem is to determine whether this formula is satis able
Guideline Recall that Prop osition establishes the NP completeness of a version of
SAT in which the instances are restricted such that each variable app ears in at most three
clauses So it su ces to reduce this restricted problem to r SAT This reduction is based
REFLECTIONS ON COMPLETE PROBLEMS
on the fact that if all three o ccurrences of a variable are of the same type i e they are
all negated or all non negated then this variable can b e assigned a value that satis es all
clauses in which it app ears and so the variable and the clauses in which it app ear can b e
omitted from the instance Thus the desired reduction consists of applying the foregoing
simpli cation to all relevant variables Alternatively a closer lo ok at the reduction used in
the pro of of Prop osition reveals the fact that this reduction maps any CNF formula
to a CNF formula in which each literal app ears in at most two clauses
Exercise Verify the validity of the three reductions presented in the pro of of
Prop osition that is we refer to the reduction of r SAT to SC the reduction
of SC to XC and the reduction of XC to XC
Exercise Clique and Indep endent Set An instance of the Independent
Set problem consists of a pair G K where G is a graph and K is an integer
and the question is whether or not the graph G contains an indep endent set i e
a set with no edges b etween its members of size at least K The Clique prob
lem is analogous Prove that b oth problems are computationally equivalent via
Karp reductions to the Vertex Cover problem
Exercise an alternative pro of of Prop osition Consider the fol
lowing sketch of a reduction of SAT to Independent Set On input a CNF
formula with m clauses and n variables we construct a graph G consisting of m
triangles corresp onding to the three literals in the m clauses augmented with
th th
edges that link con icting literals That is if x app ears as the i literal of the j
th th
clause then we draw an edge literal of the j clause and x app ears as the i
th th th th
b etween the i vertex of the j triangle and the i vertex of the j triangle
Prove that SAT if and only if G has an indep endent set of size m
Exercise Verify the validity of the reduction presented in the pro of of Prop o
sition
Exercise Subset Sum Prove that the following problem is NP complete
The instance consists of a list of n integers denoted a a b and the question
n
is whether or not a subsets of the a s sums up to b i e exists I n such that
i
P
a b Establish the NP completeness of this problem called subset sum
i
i I
by reduction from XC
Guideline Given an instance S S of XC where without loss of generality
m
S S k consider the following instance of subset sum that consists of a list of
m
P P
k
j j
m for every i m m integers such that b m and a
i
j S j
i
Some intuition may b e gained by writing all integers in base m
Exercise additional prop erties of standard reductions In continua
tion of the discussion in the main text consider the following augmented form of
Levin reductions Such a reduction of R to R consists of three p olynomial time
0
and the following mappings f h g such that f is a Karp reduction of S to S
R R
two conditions hold
For every x y R it holds that f x h x y R
CHAPTER NP COMPLETENESS
For every f x y R it holds that x g x y R
We note that this de nition is actually the one used by Levin in except that
he restricted h and g to only dep end on their second argument
Prove that such a reduction implies b oth a Karp reduction and a Levin Reduction
and show that all reductions presented in this chapter satisfy this augmented re
quirement Furthermore prove that in all these cases the main mapping i e f
is and p olynomial time invertible
Exercise parsimonious reductions Let R R PC and let f b e a Karp
0
fx R x g We say that f is parsi reduction of S fx R x g to S
R R
monious with resp ect to R and R if for every x it holds that jR x j jR f x j
For each of the reductions presented in this chapter checked whether or not it
is parsimonious For the reductions that are not parsimonious nd alternative
reductions that are parsimonious cf Sec
Exercise on p olynomial time invertible reductions following
We say that a set S is markable if there exists a p olynomial time marking algo
rithm M such that
For every x f g it holds that
a M x S if and only if x S
b jM x j jxj
There exists a p olynomial time de marking algorithm D such that for every
x f g it holds that D M x
Note that all natural NP sets e g those considered in this chapter are markable
e g for SAT one may mark a formula by augmenting it with additional satis
able clauses that use sp ecially designated auxiliary variables Prove that if S is
Karp reducible to S and S is markable then S is Karp reducible to S by a length
increasing one to one and polynomial time invertible mapping Infer that for
any natural NP complete problem S any set in NP is Karp reducible to S by a
length increasing one to one and p olynomial time invertible mapping
Guideline Let f b e a Karp reduction of S to S and let M b e the guaranteed marking
algorithm Consider the reduction that maps x to M f x x
Exercise on the isomorphism of NP complete sets following
Supp ose that S and T are Karp reducible to one another by length increasing one
to one and p olynomial time invertible mappings denoted f and g resp ectively
Using the following guidelines prove that S and T are e ectively isomorphic
that is present a p olynomial time computable and invertible one to one mapping
def
such that T S f x x S g
When given a string that is not in the image of the mapping the inverting algorithm returns
a sp ecial symbol
REFLECTIONS ON COMPLETE PROBLEMS
def def
Let F ff x x f g g and G fg x x f g g Using the
length preserving condition of f resp g prove that F resp G is a prop er
subset of f g Prove that for every y f g there exists a unique triple
j x i f g f g f g N that satis es one of the following two
conditions
def
i
a j x G f g n G and y g f x
def
i
b j x F f g n F and y g f g x
i i
In b oth cases h z z h z h h z and g f z g f z Hint
consider the maximal sequence of inverse op erations g f g that
can b e applied to y and note that each inverse shrinks the current string
def def
i i
f g f g x x G i g and U F i g f g f x x Let U
Prove that U U is a partition of f g Using the fact that f and g are
length increasing and p olynomial time invertible present a p olynomial time
pro cedure for deciding membership in the set U
i
Prove the same for the sets V f f g x x F i g and V
i
f f g f x x G i g
def def
Note that U G and de ne x f x if x U and x g x
otherwise
a Prove that is a Karp reduction of S to T
b Note that maps U to f U ff x x U g V and U to
g U fg x x U g V Prove that is one to one and onto
Observe that x f x if x f U and x g x otherwise
Prove that is a Karp reduction of T to S Infer that S T
Using Exercise infer that all natural NP complete sets are isomorphic
Exercise Referring to the pro of of Theorem prove that the function f
is unbounded i e for every i there exists an n such that n steps of the pro cess
st
de ned in the pro of allow for failing the i machine
Guideline Note that f is monotonically non decreasing b ecause more steps allow to fail
at least as many machines Assume towards the contradiction that f is b ounded Let
i sup ff n g and n b e the smallest integer such that f n i If i is o dd then the
n N
set F determined by f is co nite b ecause F fx f jxj mo d g fx jxj n g
st
In this case the i machine tries to decide S F which di ers from S on nitely many
strings and must fail on some x Derive a contradiction by showing that the number of
steps taken till reaching and considering this x is at most exp p oly jxj which is smaller
than n for some su ciently large n A similar argument applies to the case that i is
even where we use the fact that F fx jxj n g is nite and so the relevant reduction
of S to S F must fail on some input x
CHAPTER NP COMPLETENESS
Chapter
Three relatively advanced
topics
In this chapter we discuss three relatively advanced topics The rst topic which
was eluded to in previous chapters is the notion of promise problems Section
Next we present an optimal algorithm for solving candid NP search problems
Section Finally in Section we brie y discuss the class denoted coNP
of sets that are complements of sets in NP
Teaching note The foregoing topics are typically not mentioned in a basic course on
complexity Still p ending on time constraints we suggest discussing them at least at a
high level
Promise Problems
Promise problems are a natural generalization of search and decision problems
where one explicitly considers a set of legitimate instances rather than consider
ing any string as a legitimate instance As noted previously this generalization
provides a more adequate formulation of natural computational problems and in
deed this formulation is used in all informal discussions For example in Sec
we presented such problems using phrases like given a graph and an integer or
given a collection of sets In other words we assumed that the input instance
has a certain format or rather we promised the solver that this is the case
Indeed we claimed that in these cases the assumption can b e removed without af
fecting the complexity of the problem but we avoided providing a formal treatment
of this issue which is done next
Teaching note The notion of promise problems was originally introduced in the
context of decision problems and is typically used only in that context However we
b elieve that promise problems are as natural in the context of search problems
CHAPTER THREE RELATIVELY ADVANCED TOPICS
De nitions
Promise problems are de ned by sp ecifying a set of admissible instances Candidate
solvers of these problems are only required to handle these admissible instances
Intuitively the designer of an algorithm solving such a problem is promised that
the algorithm will never encounter an inadmissible instance and so the designer
need not care ab out how the algorithm p erforms on inadmissible inputs
Search problems with a promise
In the context of search problems a promise problem is a relaxation in which one
is only required to nd solutions to instances in a predetermined set called the
promise The requirement regarding e cient checkability of solutions is adapted in
an analogous manner
De nition search problems with a promise A search problem with a promise
consists of a binary relation R f g f g and a promise set P Such a
problem is also referred to as the search problem R with promise P
The search problem R with promise P is solved by algorithm A if for every
x P it holds that x A x R if x S fx R x g and A x
R
otherwise where R x fy x y R g
def
n
The time complexity of A on inputs in P is de ned as T n max ft x g
A
AjP x P f g
n
where t x is the running time of A x and T n if P f g
A
AjP
The search problem R with promise P is in the promise problem extension of
PF if there exists a polynomial time algorithm that solves this problem
The search problem R with promise P is in the promise problem extension of
PC if there exists a polynomial T and an algorithm A such that for every
x P and y f g algorithm A makes at most T jxj steps and it holds
that A x y if and only if x y R
We stress that nothing is required of the solver in the case that the input violates
the promise i e x P in particular in such a case the algorithm may halt with
a wrong output Indeed the standard formulations of PF and PC are obtained
by considering the trivial promise P f g In addition to the foregoing
motivation for promise problems we mention one natural class of search problems
with a promise These are search problem in which the promise is that the instance
def
fx has a solution i e in terms of the foregoing notation P S where S
R R
y s t x y R g We refer to such search problems by the name candid search
problems
In this case it do es not matter whether the time complexity of A is de ned on inputs in P
or on all p ossible strings Supp ose that A has p olynomial time complexity T on inputs in P
then we can mo dify A to halt on any input x after at most T jxj steps This mo di cation may
only e ects the output of A on inputs not in P which is OK by us The mo di cation can b e
implemented in p olynomial time by computing t T jxj and emulating the execution of A x
for t steps A similar comment applies to the de nition of PC P and NP
Here we refer to the formulation of PC outlined in Section
PROMISE PROBLEMS
De nition candid search problems An algorithm A solves the candid search
problem of the binary relation R if for every x S i e for every x y R it
R
holds that x A x R The time complexity of such an algorithm is de ned as
def
n
n max ft x g where t x is the running time of A x and T
A A
x P f g AjS
R
n
n if S f g T
R
AjS
R
Note that nothing is required when x S In particular algorithm A may ei
R
ther output a wrong solution although no solutions exist or run for more than
jxj steps The rst case can b e essentially eliminated whenever R PC T
AjS
R
Furthermore for R PC if we know the time complexity of algorithm A e g
n in p oly n time then we may mo dify A into an algo if we can compute T
AjS
R
rithm A that solves the general search problem of R i e halts with a correct
0
n p oly n However we do not n T output on each input in time T
A
AjS
R
necessarily know the running time of an algorithm that we consider Furthermore
as we shall see in Section the naive assumption by which we always know the
running time of an algorithm that we design is not valid either
Decision problems with a promise
In the context of decision problems a promise problem is a relaxation in which one
is only required to determine the status of instances that b elong to a predetermined
set called the promise The requirement of e cient veri cation is adapted in an
analogous manner In view of the standard usage of the term we refer to deci
sion problems with a promise by the name promise problems Formally promise
problems refer to a three way partition of the set of all strings into yes instances
no instances and instances that violate the promise Standard decision problems
are obtained as a sp ecial case by insisting that all inputs are allowed i e the
promise is trivial
De nition promise problems A promise problem consists of a pair of non
intersecting sets of strings denoted S S and S S is called the promise
yes no yes no
The promise problem S S is solved by algorithm A if for every x S
yes no yes
it holds that A x and for every x S it holds that A x The
no
promise problem is in the promise problem extension of P if there exists a
polynomial time algorithm that solves it
The promise problem S S is in the promise problem extension of NP if
yes no
there exists a polynomial p and a polynomial time algorithm V such that the
fol lowing two conditions hold
Completeness For every x S there exists y of length at most p jxj
yes
such that V x y
Soundness For every x S and every y it holds that V x y
no
We stress that for algorithms of p olynomial time complexity it do es not matter
whether the time complexity is de ned only on inputs that satisfy the promise or
on all strings see Footnote Thus the extended classes P and NP like PF
and PC are invariant under this choice
CHAPTER THREE RELATIVELY ADVANCED TOPICS
Reducibility among promise problems
The notion of a Co ok reduction extend naturally to promise problems when p os
tulating that a query that violates the promise of the problem at the target of the
reduction may b e answered arbitrarily That is the oracle machine should solve
the original problem no matter how queries that violate the promise are answered
The latter requirement is consistent with the conceptual meaning of reductions and
promise problems Recall that reductions captures pro cedures that make subrou
tine calls to an arbitrary pro cedure that solves the reduced problem But in the
case of promise problems such a solver may b ehave arbitrarily on instances that
violate the promise We stress that the main prop erty of a reduction is preserved
see Exercise if the promise problem is Cook reducible to a promise problem
that is solvable in polynomial time then is solvable in polynomial time
Caveat The extension of a complexity class to promise problems do es not nec
essarily inherit the structural prop erties of the standard class For example in
contrast to Theorem there exists promise problems in NP coNP such that
every set in NP can b e Co ok reduced to them see Exercise Needless to say
NP coNP do es not seem to follow from Exercise See further discussion at
the end of Sec
Applications
The following discussion refers b oth to the decision and search versions of promise
problems Recall that promise problems o er the most direct way of formulating
natural computational problems Indeed this is a ma jor application of the notion
of promise problems although this application usually go es unnoticed Formally
the presentation of natural computational problems refers usually implicitly to
some natural format and the notion of a promise problem allows us to discard
inputs that do not adhere to this format and fo cus on those that do adhere to
it For example when referring to computational problems regarding graphs the
promise mandates that the input is a graph or rather a standard representation
of some graph
Restricting a computational problem In addition to the foregoing applica
tion of promise problems we mention their use in formulating the natural notion of
a restriction of a computational problem to a subset of the instances Sp eci cally
such a restriction means that the promise set of the restricted problem is a subset
of the promise set of the unrestricted problem
De nition restriction of computational problems
It follows that Karp reductions among promise problems are not allowed to make queries
that violate the promise Sp eci cally we say that the promise problem is Karp
yes no
reducible to the promise problem if there exists a p olynomial time mapping f
yes no
such that for every x resp x it holds that f x resp f x
yes no
yes no
This practice is supp orted by the fact that the said format is easily recognizable see
Sec
PROMISE PROBLEMS
For any P P and binary relation R we say that the search problem R
with promise P is a restriction of the search problem R with promise P
We say that the promise problem S S is a restriction of the promise
yes no
problem S S if both S S and S S hold
yes no yes no
yes no
For example when we say that SAT is a restriction of SAT we refer to the fact
that the set of allowed instances is now restricted to CNF formulae rather than to
arbitrary CNF formulae In b oth cases the computational problem is to determine
satis ability or to nd a satisfying assignment but the set of instances i e the
promise set is further restricted in the case of SAT The fact that a restricted
problem is never harder than the original problem is captured by the fact that the
restricted problem is Karp reducible to the original one via the identity mapping
Other uses and some reservations In addition to the two aforementioned
generic uses of the notion of a promise problem we mention that this notion
provides adequate formulations for a variety of sp eci c computational complex
ity notions and results Examples include the notion of unique solutions see
Sec and the formulation of gap problems as capturing various approxi
mation tasks see Sec In all these cases promise problems allow to
discuss natural computational problems and make statements ab out their inherent
complexity Thus the complexity of promise problems and classes of such prob
lems addresses natural questions and concerns Consequently demonstrating the
intractability of a promise problem that b elongs to some class e g saying that
some promise problem in NP cannot b e solved by a p olynomial time algorithm
carries the same conceptual message as demonstrating the intractability of a stan
dard problem in the corresp onding class In contrast as indicated in the caveat
of Sec structural prop erties of promise problems may not hold for the
corresp onding classes of standard problems e g see Exercise Indeed we do
distinguish here b etween the inherent or absolute prop erties such as intractability
and structural or relative prop erties such as reducibility
The Standard Convention of Avoiding Promise Prob
lems
Recall that although promise problems provide a go o d framework for presenting
natural computational problems we managed to avoid this framework in previous
chapters This was done by relying on the fact that for all the natural problems
considered in the previous chapters it is easy to decide whether or not a given
instance satis es the promise For example given a formula it is easy to decide
whether or not it is in CNF or CNF Actually the issue arises already when
talking ab out formulae What we are actually given is a string that is supp osed to
enco de a formula under some predetermined enco ding scheme and so the promise
which is easy to decide for natural enco ding schemes is that the input string is a
valid enco ding of some formula In any case if the promise is e ciently recognizable
CHAPTER THREE RELATIVELY ADVANCED TOPICS
i e membership in it can b e decided in p olynomial time then we may avoid
mentioning the promise by using one of the following two nasty conventions
Fictitious extending the set of instances to the set of al l possible strings and
allowing trivial solutions for the corresp onding dummy instances For ex
ample in the case of a search problem we may either de ne all instance
that violate the promise to have no solution or de ne them to have a trivial
solution e g b e a solution for themselves that is for a search problem R
with promise P we may consider the standard search problem of R where
R is mo di ed such that R x for every x P or say R x fxg for
every x P In the case of a promise decision problem S S we
yes no
may consider the problem of deciding membership in S which means that
yes
instances that violate the promise are considered as no instances
Considering every string as a valid encoding of some object that satis es the
promise That is xing any string x that satis es the promise we consider
every string that violates the promise as if it were x In the case of a search
problem R with promise P this means considering the standard search
problem of R where R is mo di ed such that R x R x for every x P
Similarly in the case of a promise decision problem S S we consider
yes no
the problem of deciding membership in S provided x S and otherwise
yes no
we consider the problem of deciding membership in f g n S
no
We stress that in the case that the promise is e ciently recognizable the aforemen
tioned conventions or mo di cations do not e ect the complexity of the relevant
search or decision problem That is rather than considering the original promise
problem we consider a search or decision problem without a promise that is
computational equivalent to the original one Thus in some sense we loss noth
ing by studying the latter problem rather than the original one i e the original
promise problem However to get to this situation we need the notion of a promise
problem which allows a formulation of the original natural problem
Indeed even in the case that the original natural promise problem and the
problem without a promise that was derived from it are computationally equiv
alent it is useful to have a formulation that allows to distinguish b etween them
as we do distinguish b etween the di erent NP complete problems although they
are all computationally equivalent This conceptual concern b ecomes of crucial
imp ortance in the case to b e discussed next that the promise referred to in the
promise problem is not e ciently recognizable
The p oint is that the foregoing transformations of promise problems into com
putationally equivalent standard decision and search problems do not necessar
ily preserve the complexity of the problem in the case that the promise is not
e ciently recognizable In this case the terminology of promise problems is un
avoidable Consider for example the problem of deciding whether a Hamiltonian
graph is colorable On the face of it such a problem may have fundamentally
di erent complexity than the problem of deciding whether a given graph is b oth
Hamiltonian and colorable
OPTIMAL SEARCH ALGORITHMS FOR NP
In spite of the foregoing issues we adopt the convention of fo cusing on standard
decision and search problems That is by default all computational problems and
complexity classes discussed in other sections of this b o ok refer to standard decision
and search problems and the only exception in which we refer to promise problems
outside the current section is explicitly stated as such see Section
Optimal search algorithms for NP
Actually this section refers to solving the candid search problem of any relation in
PC Recall that PC is the class of search problems that allow for e cient checking
of the correctness of candidate solutions see De nition and that the candid
search problem is a search problem in which the solver is promised that the given
instance has a solution see De nition
We claim the existence of an optimal algorithm for solving the candid search
problem of any relation in PC Furthermore we will explicitly present such an
algorithm and prove that it is optimal in a very strong sense for any algorithm
solving the candid search problem of R PC our algorithm solves the same
problem in time that is at most a constant factor slower ignoring a xed additive
p olynomial term which may b e disregarded in the case that the problem is not
solvable in p olynomial time Needless to say we do not know the time complexity
of the aforementioned optimal algorithm indeed if we knew it then we would
have resolved the P vs NP Question In fact the P vs NP Question b oils down
to determining the time complexity of a single explicitly presented algorithm i e
the optimal algorithm claimed in Theorem
Theorem For every binary relation R PC there exists an algorithm A that
satis es the fol lowing
Algorithm A solves the candid search problem of R
There exists a polynomial p such that for every algorithm A that solves the
0
x p jxj for any candid search problem of R it holds that t x O t
A A
0
x denotes the number of steps taken by A x S where t x resp t
R A A
resp A on input x
Interestingly we establish the optimality of A without knowing what its optimal
running time is Furthermore the optimality claim is p oint wise i e it refers to
any input rather than global i e referring to the worst case time complexity
as a function of the input length
We stress that the hidden constant in the O notation dep ends only on A
but in the following pro of this dep endence is exp onential in the length of the
description of algorithm A and it is not known whether a b etter dep endence can
b e achieved Indeed this dep endence as well as the idea underlying it constitute
one negative asp ect of this otherwise amazing result Another negative asp ect is
that the optimality of algorithm A refers only to inputs that have a solution i e
inputs in S Finally we note that the theorem as stated refers only to mo dels of R
CHAPTER THREE RELATIVELY ADVANCED TOPICS
computation that have machines that can emulate a given number of steps of other
machines with a constant overhead We mention that in most natural mo dels the
overhead of such emulation is at most p oly logarithmic in the number of steps in
e
0
x p jxj which case it holds that t x O t
A A
Pro of Sketch Fixing R we let M b e a p olynomial time algorithm that decides
membership in R and let p b e a p olynomial b ounding the running time of M
as a function of the length of the rst element in the input pair Using M we
present an algorithm A that solves the candid search problem of R as follows On
input x algorithm A emulates in parallel the executions of all p ossible search
algorithms on input x checks the result provided by each of them using M
and halts whenever it recognizes a correct solution Indeed most of the emulated
algorithms are totally irrelevant to the search but using M we can screen the bad
solutions o ered by them and output a go o d solution once obtained
Since there are in nitely many p ossible algorithms it may not b e clear what
we mean by the expression emulating all p ossible algorithms in parallel What
we mean is emulating them at di erent rates such that the in nite sum of these
th
rates converges to or to any other constant Sp eci cally we will emulate the i
p ossible algorithm at rate i which means emulating a single step of this
algorithm p er i emulation steps p erformed for all algorithms Note that
a straightforward implementation of this idea may create a signi cant overhead
involved in switching frequently from the emulation of one machine to the emulation
of another Instead we present an alternative implementation that pro ceeds in
iterations
th j j
In the j iteration for i algorithm A emulates i steps
th
of the i machine where the machines are ordered according to the lexicographic
order of their descriptions Each of these emulations is conducted in one chunk
and thus the overhead of switching b etween the various emulations is insigni cant
in comparison to the total number of steps b eing emulated In the case that
some of these emulations on input x halts with output y algorithm A invokes
M on input x y and output y if and only if M x y Furthermore the
veri cation of a solution provided by a candidate algorithm is also emulated at the
exp ense of its step count Put in other words we augment each algorithm with
a canonical pro cedure i e M that checks the validity of the solution o ered by
the algorithm
By its construction whenever A x outputs a string y i e y it must hold
that x y R To show the optimality of A we consider an arbitrary algorithm
A that solves the candid search problem of R Our aim is to show that A is
not much slower than A Intuitively this is the case b ecause the overhead of A
results from emulating other algorithms in addition to A but the total number
of emulation steps wasted due to these algorithms is inversely prop ortional to
the rate of algorithm A which in turn is exp onentially related to the length of
the description of A The punch line is that since A is xed the length of its
description is a constant Details follow
For every x let us denote by t x the number of steps taken by A on in
put x where t x also accounts for the running time of M x that is t x
THE CLASS CONP AND ITS INTERSECTION WITH NP
0 0
x is the number of steps taken by A x itself Then x p jxj where t t
A A
th
the emulation of t x steps of A on input x is covered by the j iteration of A
0
j jA j
provided that t x where jA j denotes the length of the description
of A Indeed we use the fact that the algorithms are emulated in lexicographic
0
jA j
order and note that there are at most algorithms that precede A in
0
x lexicographic order Thus on input x algorithm A halts after at most j
A
0 0
x p jxj after emulating a total x jA j log t iterations where j
A A
number of steps that is at most
j 2
j x
0
A
j
X X
0
def
j x jA j
0
A
0
t x x p jxj t
A
i
j i
j 2
P P P
where the inequality uses
2
i i i
i i i i i
P
j x
0
j j x
A
0
A
and The question of how much time is required for emulat
j
ing these many steps dep ends on the sp eci c mo del of computation In many mo d
els of computation e g two tape Turing machine emulation is p ossible within
p oly logarithmic overhead i e t steps of an arbitrary machine can b e emulated
e
by O t steps of the emulating machine and in some mo dels this emulation can
even b e p erformed with constant overhead The theorem follows
Comment By construction the foregoing algorithm A do es not halt on input
x S This can b e easily recti ed by letting A emulate a straightforward ex
R
haustive search for a solution and halt with output if and only if this exhaustive
search indicates that there is no solution to the current input This extra emulation
can b e p erformed in parallel to all other emulations e g at a rate of one step for
the extra emulation p er each step of everything else
The class coNP and its intersection with NP
By prep ending the name of a complexity class of decision problems with the pre x
co we mean the class of complement sets that is
def
coC ff g n S S C g
Sp eci cally coNP ff g n S S N P g is the class of sets that are comple
ments of sets in NP
Recalling that sets in NP are characterized by their witness relations such that
x S if and only if there exists an adequate NP witness it follows that their
complement sets consists of all instances for which there are no NP witness i e
x f g n S if there is no NP witness for x b eing in S For example SAT NP
implies that the set of unsatis able CNF formulae is in coNP Likewise the set
of graphs that are not colorable is in coNP Jumping ahead we mention that
it is widely b elieved that these sets are not in NP
CHAPTER THREE RELATIVELY ADVANCED TOPICS
Another p ersp ective on coNP is obtained by considering the search problems
in PC Recall that for such R PC the set of instances having a solution i e
S fx y s t x y R g is in NP It follows that the set of instances having
R
no solution i e f g n S fx y x y R g is in coNP
R
It is widely b elieved that NP coNP which means that NP is not closed
under complementation Indeed this conjecture implies P NP b ecause P is
closed under complementation The conjecture NP coNP means that some
sets in coNP do not have NP pro of systems b ecause NP is the class of sets having
NP pro of systems As we will show next under this conjecture the complements
of NP complete sets do not have NP pro of systems for example there exists no
NP pro of system for proving that a given CNF formula is not satis able We rst
establish this fact for NP completeness in the standard sense i e under Karp
reductions as in De nition
Prop osition Suppose that NP coNP and let S NP such that every set
def
in NP is Karp reducible to S Then S f g n S is not in NP
Pro of Sketch We rst observe that the fact that every set in NP is Karp
reducible to S implies that every set in coNP is Karp reducible to S We next
claim that if S is in NP then every set that is Karp reducible to S is also in NP
S we conclude that S NP implies coNP NP Applying the claim to S
which in turn implies NP coNP see Exercise in contradiction to the main
hypothesis
We now turn to prove the foregoing claim that is we prove that if S has an NP
proof system and S is Karp reducible to S then S has an NP proof system Let
V b e the veri cation pro cedure asso ciated with S and let f b e a Karp reduction
of S to S Then we de ne the veri cation pro cedure V for membership in S
by V x y V f x y That is any NP witness that f x S serves as an
NP witness for x S and these are the only NP witnesses for x S This may
not b e a natural pro of system for S but it is de nitely an NP pro of system
for S
Assuming that NP coNP Prop osition implies that sets in NP coNP
cannot b e NP complete with resp ect to Karp reductions In light of other limita
tions of Karp reductions see e g Exercise one may wonder whether or not
the exclusion of NP complete sets from the class NP coNP is due to the use
of a restricted notion of reductions i e Karp reductions The following theorem
asserts that this is not the case some sets in NP cannot be reduced to sets in the
intersection NP coNP even under general reductions i e Co ok reductions
Theorem If every set in NP can be Cook reduced to some set in NP coNP
then NP coNP
In particular assuming NP coNP no set in NP coNP can b e NP complete
even when NP completeness is de ned with resp ect to Co ok reductions Since
NP coNP is conjectured to b e a prop er sup erset of P it follows assuming
NP coNP that there are decision problems in NP that are neither in P nor
THE CLASS CONP AND ITS INTERSECTION WITH NP
NP hard i e sp eci cally the decision problems in NP coNP n P We stress
that Theorem refers to standard decision problems and not to promise problems
see Section and Exercise
Pro of Analogously to the pro of of Prop osition the current pro of b oils down
to proving that if S is Cook reducible to a set in N P coNP then S N P coNP
Using this claim the theorem s hypothesis implies that NP NP coNP which
in turn implies NP coNP and NP coNP see Exercise
Fixing any S and S NP coNP such that S is Co ok reducible to S we
prove that S NP and the pro of that S coNP is similar Let us denote
by M the oracle machine reducing S to S That is on input x machine M
makes queries and decides whether or not to accept x and its decision is correct
provided that all queries are answered according to S To show that S NP
we will present an NP pro of system for S This pro of system denoted V accepts
an alleged instance witness pair of the form x h z w z w i if the
t t t
following two conditions hold
On input x machine M accepts after making the queries z z and ob
t
taining the corresp onding answers
t
That is V check that on input x after obtaining the answers to
i
th
the rst i queries the i query made by M equals z In addition V
i
checks that on input x and after receiving the answers machine M
t
halts with output indicating acceptance
Note that V do es not have oracle access to S The pro cedure V rather
th
emulates the computation of M x by answering for each i the i query of
M x by using the bit provided to V as part of its input The correctness
i
of these answers will b e veri ed by V separately i e see the next item
For every i it holds that if then w is an NP witness for z S
i i i
whereas if then w is an NP witness for z f g n S
i i i
Thus if this condition holds then it is the case that each indicates the
i
correct status of z with resp ect to S i e if and only if z S
i i i
def
f g n S have S We stress that we have used the fact that b oth S and
NP pro of systems and we have referred to the corresp onding NP witnesses
Note that V is indeed an NP pro of system for S Firstly the length of the
corresp onding witnesses is b ounded by the running time of the reduction and the
length of the NP witnesses supplied for the various queries Next note that V
runs in p olynomial time i e verifying the rst condition requires an emulation of
the p olynomial time execution of M on input x when using the s to emulate the
i
oracle whereas verifying the second condition is done by invoking the relevant NP
pro of systems Finally observe that x S if and only if there exists a sequence
def
y z w z w such that V x y In particular V x y
t t t
def
Alternatively we show that S co NP by applying the following argument to S f g n S
S is Co ok reducible to S via S or alternatively that S is Co ok reducible to and noting that
f g n S NP co NP
CHAPTER THREE RELATIVELY ADVANCED TOPICS
holds only if y contains a valid sequence of queries and answers as made in a
computation of M on input x and oracle access to S and M accepts based on
that sequence
The world view a digest Recall that on top of the P NP conjecture we
mentioned two other conjectures which clearly imply P NP
The conjecture that NP coNP equivalently NP coNP NP
This conjecture is equivalent to the conjecture that CNF formulae have no
short pro ofs of unsatis ability i e the set f g n SAT has no NP pro of
system
The conjecture that NP coNP P
Notable candidates for the class NP coNP P include decision problems
that are computationally equivalent to the integer factorization problem i e
the search problem in PC in which given a comp osite number the task is
to nd its prime factors
Combining these conjectures we get the world view depicted in Figure which
also shows the class of coNP complete sets de ned next
NP NPC P
coNPC coNP
Figure The world view under P coNP NP NP
De nition A set S is called coNP hard if every set in coNP is Karp reducible
to S A set is called coNP complete if it is both in coNP and coNP hard
Indeed insisting on Karp reductions is essential for a distinction b etween NP
hardness and coNP hardness In contrast recall that the class of problems that
are Co ok reducible to NP resp to coNP contains NP coNP This class
NP
commonly denoted P is further discussed in Exercise
THE CLASS CONP AND ITS INTERSECTION WITH NP
Exercises
Exercise Co ok reductions preserve e cient solvability of promise problems
Prove that if the promise problem is Co ok reducible to a promise problem that
is solvable in p olynomial time then is solvable in p olynomial time Note that
the solver may not halt on inputs that violate the promise
Guideline Use the fact that any p olynomial time algorithm that solves any promise
problem can b e mo di ed such that it halts on all inputs
Exercise NP complete promise problems in coNP following Consider
the promise problem xSAT having instances that are pairs of CNF formulae The
yes instances consists of pairs such that is satis able and is unsatis
able whereas the no instances consists of pairs such that is unsatis able and
is satis able
Show that xSAT is in the intersection of the promise problem classes that
are analogous to NP and coNP
Prove that any promise problem in NP is Co ok reducible to xSAT In de
signing the reduction recall that queries that violate the promise may b e
answered arbitrarily
Guideline Note that the promise problem version of NP is reducible to SAT
and show a reduction of SAT to xSAT Sp eci cally show that the search problem
asso ciated with SAT is Co ok reducible to xSAT by adapting the ideas of the pro of
of Prop osition That is supp ose that we know or assume that is a pre x of
a satisfying assignment to and we wish to extend by one bit Then for each
by setting the rst j j variables f g we construct a formula denoted
and of according to the values We query the oracle ab out the pair
extend accordingly i e we extend by the value if and only if the answer is
p ositive Note that if b oth and are satis able then it do es not matter which
bit we use in the extension whereas if exactly one formula is satis able then the
oracle answer is reliable
Pinp oint the source of failure of the pro of of Theorem when applied to
the reduction provided in the previous item
Exercise For any class C prove that C coC if and only if C coC
Exercise Prove that a set S is Karp reducible to some set in NP if and only
if S is in NP
Guideline For the non trivial direction see the pro of of Prop osition
Exercise Recall that the empty set is not Karp reducible to f g whereas
any set is Co ok reducible to its complement Thus our fo cus here is on the Karp
reducibility of non trivial sets to their complements where a set is non trivial if it
is neither empty nor contains all strings Furthermore since any non trivial set in
P is Karp reducible to its complement see Exercise we assume that P NP
and fo cus on sets in NP n P
CHAPTER THREE RELATIVELY ADVANCED TOPICS
Prove that NP coNP implies that some sets in N P n P are Karp reducible
to their complements
Prove that NP coNP implies that some sets in NP n P are not Karp
reducible to their complements
Guideline Use NP complete sets in b oth parts and Exercise in the second part
Exercise TAUT is coNP complete Prove that the following problem de
noted TAUT is coNP complete even when the formulae are restricted to DNF
An instance of the problem consists of a DNF formula and the problem is to de
termine whether this formula is a tautology i e a formula that evaluates to true
under every p ossible truth assignment
SAT i e the complement of SAT using the fact that is Guideline Reduce from
unsatis able if and only if is a tautology
NP NP
Exercise the class P Recall that P denotes the class of problems
that are Co ok reducible to NP Prove the following simple facts
For every class C the class of problems that are Co ok reducible to C equals
NP
the class of problems that are Co ok reducible to coC In particular P
equals the class of problems that are Co ok reducible to coNP
NP NP NP
The class P is closed under complementation i e P coP
NP
Note that each of the foregoing items implies that P contains NP coNP
Exercise Assuming that NP coNP prove that the problem of nding the
maximum clique resp indep endent set in a given graph is not in PC Prove the
same for the following problems
Finding a minimum vertex cover in a given graph
Finding an assignment that satis es the maximum number of equations in a
given system of linear equations over GF cf Exercise
Guideline Note that the set of pairs G K such that the graph G contains no clique
of size K is co NP complete
Exercise the class P p oly revisited In continuation of Exercise prove
that P p oly equals the class of sets that are Co ok reducible to a sparse set where
a set S is called sparse if there exists a p olynomial p such that for every n it holds
n
that jS f g j p n
Guideline For any set in P p oly enco de the advice sequence a as a sparse set
n
n N
n th
f i n N i ja jg where is the i bit of a For the opp osite direction
n i n n i n
note that the emulation of a Co ok reduction to a set S on input x only requires knowledge
S
p oly jxj
i
of S f g
i
THE CLASS CONP AND ITS INTERSECTION WITH NP
Exercise In continuation of Exercise we consider the class of sets that
are Karp reducible to a sparse set It can b e proved that this class contains SAT if
and only if P NP see Here we only consider the sp ecial case in which the
sparse set is contained in a p olynomial time decidable set that is itself sparse e g
the latter set may b e f g in which case the former set may b e an arbitrary unary
set Actually the aim of this exercise is establishing the following seemingly
stronger claim
If SAT is Karp reducible to a set S G such that G P and G n S is
sparse then SAT P
Using the hypothesis we outline a p olynomial time pro cedure for solving the search
problem of SAT and leave the task of providing the details as an exercise The
pro cedure lo oking for a satisfying assignment conducts a DFS on the tree of
all p ossible partial truth assignment to the input formula while truncating the
search at no des that corresp ond to partial truth assignments that were already
demonstrated to b e useless i e corresp ond to a partial truth assignment that
cannot b e completed to a satisfying assignment
Guideline The key observation is that each internal no de which yields a formula derived
from the initial formulae by instantiating the corresp onding partial truth assignment is
mapp ed by the Karp reduction either to a string not in G in which case we conclude
that the sub tree contains no satisfying assignments and backtrack from this no de or to
a string in G In the latter case unless we already know that this string is not in S
we start a scan of the sub tree rooted at this node However once we backtrack from this
internal no de we know that the corresp onding member of G is not in S and we will never
scan again a sub tree ro oted at a no de that is mapp ed to this string which was detected
to b e in G n S Also note that once we reach a leaf we can check by ourselves whether or
not it corresp onds to a satisfying assignment to the initial formula When analyzing the
forgoing pro cedure prove that on input an n variable formulae the number of times we
S
p oly j j
i
start to scan a sub tree is at most n j f g G n S j
i
For an n variable formulae the leaves of the tree corresp ond to all p ossible n bit long strings
and an internal no de corresp onding to is the parent of the no des corresp onding to and
CHAPTER THREE RELATIVELY ADVANCED TOPICS
Notes
The following brief account decouples the development of the theory of computation
which was the fo cus of Chapter from the emergence of the P vs NP Question
and the theory of NP completeness studied in Chapters
On computation and e cient computation
The interested reader may nd numerous historical accounts of the developments
that led to the emergence of the theory of computation The following brief account
is di erent from most of these historical accounts in that its p ersp ective is the one
of the current research in computer science
The theory of uniform computational devices emerged in the work of Turing
This work put forward a natural mo del of computation based on concrete machines
indeed Turing machines which has b een instrumental for subsequent studies In
particular this mo del provides a convenient stage for the introduction of natural
complexity measures referring to computational tasks
The notion of a Turing machine was put forward by Turing with the explicit in
tention of providing a general formulation of the notion of computability The
original motivation was providing a formalization of Hilb ert s challenge p osed in
and known as Hilb ert s Tenth Problem which called for designing a metho d
for determining the solvability of Diophantic equations Indeed this challenge re
ferred to a sp eci c decision problem later called the Entscheidungsproblem Ger
man for the Decision Problem but Hilb ert did not provide a formulation of the
notion of a metho d for solving a decision problem We mention that in
the Entscheidungsproblem was proved to b e undecidable see
In addition to introducing the Turing machine mo del and arguing that it cor
resp onds to the intuitive notion of computability Turing s pap er introduces
universal machines and contains pro ofs of undecidability e g of the Halting Prob
lem Rice s Theorem Theorem is proven in and the undecidability of
the Post Corresp ondence Problem Theorem is proven in
The Church Turing Thesis is attributed to the works of Church and Tur
ing In b oth works this thesis is invoked for claiming that the fact that some
problem cannot b e solved in a sp eci c mo del of computation implies that this prob
lem cannot b e solved in any reasonable mo del of computation The RAM mo del
is attributed to von Neumann s rep ort
CHAPTER THREE RELATIVELY ADVANCED TOPICS
The asso ciation of e cient computation with p olynomial time algorithms is
attributed to the pap ers of Cobham and Edmonds It is interesting to note
that Cobham s starting p oint was his desire to present a philosophically sound
concept of e cient algorithms whereas Edmonds s starting p oint was his desire to
articulate why certain algorithms are go o d in practice
The theory of non uniform computational devices emerged in the work of Shan
non which introduced and initiated the study of Bo olean circuits The formu
lation of machines that take advice as well as the equivalence to the circuit mo del
originates in
On NP and NP Completeness
Many sources provide historical accounts of the developments that led to the formu
lation of the P vs NP Problem and to the discovery of the theory of NP completeness
see e g Sec and Still we feel that we should not refrain from of
fering our own impressions which are based on the texts of the original papers
Nowadays the theory of NP completeness is commonly attributed to Co ok
Karp and Levin It seems that Co ok s starting p oint was his interest in
theorem proving pro cedures for prop ositional calculus P Trying to provide
evidence to the di culty of deciding whether or not a given formula is a tautology
he identi ed NP as a class containing many apparently di cult problems cf
e g P and showed that any problem in NP is reducible to deciding
membership in the set of DNF tautologies In particular Co ok emphasized the
imp ortance of the concept of p olynomial time reductions and the complexity class
NP b oth explicitly de ned for the rst time in his pap er He also showed that
CLIQUE is computationally equivalent to SAT and envisioned a class of problems of
the same nature
Karp s pap er can b e viewed as ful lling Co ok s prophecy Stimulated by
Co ok s work Karp demonstrated that a large number of classic di cult computa
tional problems arising in elds such as mathematical programming graph theory
combinatorics computational logic and switching theory are NP complete and
thus equivalent P Sp eci cally his list of twenty one NP complete prob
lems includes Integer Linear Programming Hamilton Circuit Chromatic Number
Exact Set Cover Steiner Tree Knapsack Job Scheduling and Max Cut Interest
ingly Karp de ned NP in terms of veri cation pro cedures i e De nition
p ointed to its relation to backtrack search of p olynomial b ounded depth
P and viewed NP as the residence of a wide range of imp ortant computa
tional problems which are not in P
Indep endently of these developments while b eing in the USSR Levin proved the
existence of universal search problems where universality meant NP completeness
The starting p oint of Levin s work was his interest in the perebor conjecture
asserting the inherent need for brute force in some search problems that have e
ciently checkable solutions i e problems in PC Levin emphasized the implication
of p olynomial time reductions on the relation b etween the time complexity of the
related problems for any growth rate of the time complexity asserted the NP
completeness of six classical search problems and claimed that the underlying
THE CLASS CONP AND ITS INTERSECTION WITH NP
metho d provides a mean for readily obtaining similar results for many other
imp ortant search problems
It is interesting to note that although the works of Co ok Karp and
Levin were received with di erent levels of enthusiasm none of the contempo
raries realized the depth of the discovery and the di culty of the question posed i e
the P vs NP Question This fact is evident in every account from the early s
and may explain the frustration of the corresp onding generation of researchers
which exp ected the P vs NP Question to b e resolved in their life time if not in a
matter of years Needless to say the author s opinion is that there was absolutely
no justi cation for these exp ectations and that one should have actually exp ected
quite the opp osite
We mention that the three founding pap ers of the theory of NP completeness
i e Co ok Karp and Levin use the three di erent types of reductions
used in this b o ok Sp eci cally Co ok uses the general notion of p olynomial time
reduction often referred to as Co ok reductions De nition The notion
of Karp reductions De nition originates from Karp s pap er whereas its
augmentation to search problems i e De nition originates from Levin s pa
p er It is worth stressing that Levin s work is stated in terms of search prob
lems unlike Co ok and Karp s works which treat decision problems
The reductions presented in Sec are not necessarily the original ones
Most notably the reduction establishing the NP hardness of the Indep endent Set
problem i e Prop osition is adapted from In contrast the reductions
presented in Sec are merely a re interpretation of the original reduction as
presented in The equivalence of the two de nitions of NP i e Theorem
was proved in
The existence of NP sets that are neither in P nor NP complete i e Theo
rem was proven by Ladner Theorem was proven by Selman
and the existence of optimal search algorithms for NP relations i e Theorem
was proven by Levin Interestingly the latter result was proved in the same
pap er in which Levin presented the discovery of NP completeness indep endently
of Co ok and Karp Promise problems were explicitly introduced by Even Selman
and Yacobi see for a survey of their numerous applications
We mention that the standard reductions used to establish natural NP completeness
results have several additional prop erties or can b e mo di ed to have such prop erties
These prop erties include an e cient transformation of solutions in the direction of
the reduction see Exercise the preservation of the number of solutions see
Exercise b eing computable by a log space algorithm and b eing invertible
in p olynomial time see Exercise We also mention the fact that all known
NP complete sets are e ectively isomorphic see Exercise
CHAPTER THREE RELATIVELY ADVANCED TOPICS
Epilogue A Brief Overview
of Complexity Theory
Out of the tough came forth sweetness
Judges
The following brief overview is intended to give a avor of the questions addressed
by Complexity Theory This overview is quite vague and is merely meant as a
teaser towards further study cf e g
Complexity Theory is concerned with the study of the intrinsic complexity of
computational tasks Its nal goals include the determination of the complex
ity of any well de ned task Additional goals include obtaining an understanding
of the relations b etween various computational phenomena e g relating one fact
regarding computational complexity to another Indeed we may say that the
former type of goals is concerned with absolute answers regarding sp eci c compu
tational phenomena whereas the latter type is concerned with questions regarding
the relation b etween computational phenomena
Interestingly so far Complexity Theory has b een more successful in coping with
goals of the latter relative type In fact the failure to resolve questions of the
absolute type led to the ourishing of metho ds for coping with questions of the
relative type Musing for a moment let us say that in general the di culty
of obtaining absolute answers may naturally lead to seeking conditional answers
which may in turn reveal interesting relations b etween phenomena Furthermore
the lack of absolute understanding of individual phenomena seems to facilitate the
development of metho ds for relating di erent phenomena Anyhow this is what
happ ened in Complexity Theory
Putting aside for a moment the frustration caused by the failure of obtaining
absolute answers we must admit that there is something fascinating in the success
to relate di erent phenomena in some sense relations b etween phenomena are
more revealing than absolute statements ab out individual phenomena Indeed the
rst example that comes to mind is the theory of NP completeness Let us consider
this theory for a moment from the p ersp ective of these two types of goals
The quote is commonly interpreted as meaning that b ene t arose out of misfortune
CHAPTER THREE RELATIVELY ADVANCED TOPICS
Complexity theory has failed to determine the intrinsic complexity of tasks such
as nding a satisfying assignment to a given satis able prop ositional formula
or nding a coloring of a given colorable graph But it has succeeded in
establishing that these two seemingly di erent computational tasks are in some
sense the same or more precisely are computationally equivalent We nd this
success amazing and exciting and hop e that the reader shares these feelings The
same feeling of wonder and excitement is generated by many of the other discoveries
of Complexity theory Indeed the reader is invited to join a fast tour of some of
the other questions and answers that make up the eld of Complexity theory
We will indeed start with the P versus NP Question and indeed brie y review
the contents of Chapter Our daily exp erience is that it is harder to solve a
problem than it is to check the correctness of a solution e g think of either a puzzle
or a research problem Is this exp erience merely a coincidence or do es it represent
a fundamental fact of life i e a prop erty of the world Could you imagine a
world in which solving any problem is not signi cantly harder than checking a
solution to it Would the term solving a problem not lose its meaning in such a
hypothetical and imp ossible in our opinion world The denial of the plausibility
of such a hypothetical world in which solving is not harder than checking
is what P di erent from NP actually means where P represents tasks that are
e ciently solvable and NP represents tasks for which solutions can b e e ciently
checked
The mathematically or theoretically inclined reader may also consider the
task of proving theorems versus the task of verifying the validity of pro ofs Indeed
nding pro ofs is a sp ecial type of the aforementioned task of solving a problem
and verifying the validity of pro ofs is a corresp onding case of checking correctness
Again P di erent from NP means that there are theorems that are harder to
prove than to b e convinced of their correctness when presented with a pro of This
means that the notion of a pro of is meaningful that is pro ofs do help when
seeking to b e convinced of the correctness of assertions Here NP represents sets
of assertions that can b e e ciently veri ed with the help of adequate pro ofs and
P represents sets of assertions that can b e e ciently veri ed from scratch i e
without pro ofs
In light of the foregoing discussion it is clear that the P versus NP Question is
a fundamental scienti c question of far reaching consequences The fact that this
question seems b eyond our current reach led to the development of the theory of
NP completeness Lo osely sp eaking this theory presented in Chapter identi es
a set of computational problems that are as hard as NP That is the fate of the
P versus NP Question lies with each of these problems if any of these problems is
easy to solve then so are all problems in NP Thus showing that a problem is NP
complete provides evidence to its intractability assuming of course P di erent
than NP Indeed demonstrating the NP completeness of computational tasks is
a central to ol in indicating hardness of natural computational problems and it
has b een used extensively b oth in computer science and in other disciplines We
note that NP completeness indicates not only the conjectured intractability of a
problem but rather also its richness in the sense that the problem is rich enough
THE CLASS CONP AND ITS INTERSECTION WITH NP
to enco de any other problem in NP The use of the term enco ding is justi
ed by the exact meaning of NP completeness which in turn establishes relations
b etween di erent computational problems without referring to their absolute
complexity
The foregoing discussion of NP completeness hints to the importance of repre
sentation since it referred to di erent problems that enco de one another Indeed
the imp ortance of representation is a central asp ect of complexity theory In gen
eral complexity theory is concerned with problems for which the solutions are
implicit in the problem s statement or rather in the instance That is the prob
lem or rather its instance contains all necessary information and one merely
needs to pro cess this information in order to supply the answer Thus complex
ity theory is concerned with manipulation of information and its transformation
from one representation in which the information is given to another representa
tion which is the one desired Indeed a solution to a computational problem is
merely a di erent representation of the information given that is a representation
in which the answer is explicit rather than implicit For example the answer to
the question of whether or not a given Bo olean formula is satis able is implicit in
the formula itself but the task is to make the answer explicit Thus complex
ity theory clari es a central issue regarding representation that is the distinction
b etween what is explicit and what is implicit in a representation Furthermore it
even suggests a quanti cation of the level of non explicitness
In general complexity theory provides new viewp oints on various phenomena
that were considered also by past thinkers Examples include the aforementioned
concepts of solutions pro ofs and representation as well as concepts like random
ness knowledge interaction secrecy and learning We next discuss the latter
concepts and the p ersp ective o ered by complexity theory
The concept of randomness has puzzled thinkers for ages Their p ersp ective
can b e describ ed as ontological they asked what is randomness and wondered
whether it exist at all or is the world deterministic The p ersp ective of complexity
theory is b ehavioristic it is based on de ning ob jects as equivalent if they cannot
b e told apart by any e cient pro cedure That is a coin toss is de ned to b e
random even if one b elieves that the universe is deterministic if it is infeasible
to predict the coin s outcome Likewise a string or a distribution of strings is
random if it is infeasible to distinguish it from the uniform distribution regard
less of whether or not one can generate the latter Interestingly randomness or
rather pseudorandomness de ned this way is e ciently expandable that is under
a reasonable complexity assumption to b e discussed next short pseudorandom
strings can b e deterministically expanded into long pseudorandom strings Indeed
it turns out that randomness is intimately related to intractability Firstly note
that the very de nition of pseudorandomness refers to intractability i e the infea
sibility of distinguishing a pseudorandomness ob ject from a uniformly distributed
ob ject Secondly as stated a complexity assumption which refers to the exis
In contrast in other disciplines solving a problem may require gathering information that is
not available in the problem s statement This information may either b e available from auxiliary
past records or b e obtained by conducting new exp eriments
CHAPTER THREE RELATIVELY ADVANCED TOPICS
tence of functions that are easy to evaluate but hard to invert called one way
functions implies the existence of deterministic programs called pseudorandom
generators that stretch short random seeds into long pseudorandom sequences In
fact it turns out that the existence of pseudorandom generators is equivalent to
the existence of one way functions
Complexity theory o ers its own p ersp ective on the concept of know ledge and
distinguishes it from information Sp eci cally complexity theory views knowledge
as the result of a hard computation Thus whatever can b e e ciently done by any
one is not considered knowledge In particular the result of an easy computation
applied to publicly available information is not considered knowledge In contrast
the value of a hard to compute function applied to publicly available information
is knowledge and if someb o dy provides you with such a value then it has provided
you with knowledge This discussion is related to the notion of zero knowledge
interactions which are interactions in which no knowledge is gained Such interac
tions may still b e useful b ecause they may convince a party of the correctness of
sp eci c data that was provided b eforehand For example a zero knowledge inter
active pro of may convince a party that a given graph is colorable without yielding
any coloring
The foregoing paragraph has explicitly referred to interaction viewing it as a
vehicle for gaining knowledge and or gaining con dence Let us highlight the latter
application by noting that it may b e easier to verify an assertion when allowed to
interact with a prover rather than when reading a pro of Put di erently interaction
with a go o d teacher may b e more b ene cial than reading any b o ok We comment
that the added p ower of such interactive proofs is ro oted in their b eing randomized
i e the veri cation pro cedure is randomized b ecause if the veri er s questions
can b e determined b eforehand then the prover may just provide the transcript of
the interaction as a traditional written pro of
Another concept related to knowledge is that of secrecy knowledge is something
that one party may have while another party do es not have and cannot feasibly
obtain by itself thus in some sense knowledge is a secret In general complexity
theory is related to Cryptography where the latter is broadly de ned as the study
of systems that are easy to use but hard to abuse Typically such systems involve
secrets randomness and interaction as well as a complexity gap b etween the ease
of prop er usage and the infeasibility of causing the system to deviate from its pre
scrib ed b ehavior Thus much of Cryptography is based on complexity theoretic
assumptions and its results are typically transformations of relatively simple com
putational primitives e g one way functions into more complex cryptographic
applications e g secure encryption schemes
We have already mentioned the concept of learning when referring to learning
from a teacher versus learning from a b o ok Recall that complexity theory provides
evidence to the advantage of the former This is in the context of gaining knowledge
ab out publicly available information In contrast computational learning theory
is concerned with learning ob jects that are only partially available to the learner
i e reconstructing a function based on its value at a few random lo cations or even
at lo cations chosen by the learner Complexity theory sheds light on the intrinsic
THE CLASS CONP AND ITS INTERSECTION WITH NP
limitations of learning in this sense
Complexity theory deals with a variety of computational tasks We have already
mentioned two fundamental types of tasks searching for solutions or rather nd
ing solutions and making decisions e g regarding the validity of assertions We
have also hinted that in some cases these two types of tasks can b e related Now
we consider two additional types of tasks counting the number of solutions and
generating random solutions Clearly b oth the latter tasks are at least as hard as
nding arbitrary solutions to the corresp onding problem but it turns out that for
some natural problems they are not signi cantly harder Sp eci cally under some
natural conditions on the problem approximately counting the number of solutions
and generating an approximately random solution is not signi cantly harder than
nding an arbitrary solution
Having mentioned the notion of approximation we note that the study of the
complexity of nding approximate solutions is also of natural imp ortance One
type of approximation problems refers to an ob jective function de ned on the set
of p otential solutions Rather than nding a solution that attains the optimal
value the approximation task consists of nding a solution that attains an al
most optimal value where the notion of almost optimal may b e understo o d
in di erent ways giving rise to di erent levels of approximation Interestingly in
many cases even a very relaxed level of approximation is as di cult to obtain as
solving the original exact search problem i e nding an approximate solution is
as hard as nding an optimal solution Surprisingly these hardness of approxima
tion results are related to the study of probabilistically checkable proofs which are
pro ofs that allow for ultra fast probabilistic veri cation Amazingly every pro of
can b e e ciently transformed into one that allows for probabilistic veri cation
based on probing a constant number of bits in the alleged pro of Turning back
to approximation problems we mention that in other cases a reasonable level of
approximation is easier to achieve than solving the original exact search problem
Approximation is a natural relaxation of various computational problems An
other natural relaxation is the study of average case complexity where the aver
age is taken over some simple distributions representing a mo del of the prob
lem s instances that may o ccur in practice We stress that although it was not
stated explicitly the entire discussion so far has referred to worst case analysis
of algorithms We mention that worst case complexity is a more robust notion
than average case complexity For starters one avoids the controversial question
of what are the instances that are imp ortant in practice and corresp ondingly
the selection of the class of distributions for which average case analysis is to b e
conducted Nevertheless a relatively robust theory of average case complexity has
b een suggested alb eit it is less developed than the theory of worst case complexity
In view of the central role of randomness in complexity theory as evident say
in the study of pseudorandomness probabilistic pro of systems and cryptography
one may wonder as to whether the randomness needed for the various applications
can b e obtained in real life One sp eci c question which received a lot of atten
tion is the p ossibility of purifying randomness or extracting go o d randomness
from bad sources That is can we use defected sources of randomness in or
CHAPTER THREE RELATIVELY ADVANCED TOPICS
der to implement almost p erfect sources of randomness The answer dep ends of
course on the mo del of such defected sources This study turned out to b e related
to complexity theory where the most tight connection is b etween some type of
randomness extractors and some type of pseudorandom generators
So far we have fo cused on the time complexity of computational tasks while
relying on the natural asso ciation of e ciency with time However time is not
the only resource one should care ab out Another imp ortant resource is space
the amount of temp orary memory consumed by the computation The study
of space complexity has uncovered several fascinating phenomena which seem to
indicate a fundamental di erence b etween space complexity and time complexity
For example in the context of space complexity verifying pro ofs of validity of
assertions of any sp eci c type has the same complexity as verifying pro ofs of
invalidity for the same type of assertions
In case the reader feels dizzy it is no wonder We to ok an ultra fast air tour of
some mountain tops and dizziness is to b e exp ected For a totally di erent touring
exp erience we refer the interested reader to our b o ok which o ers climbing
some of these mountains by fo ot and stopping often for appreciation of the view
and re ection
Absolute Results a k a Lower Bounds As stated in the b eginning of
this epilogue absolute results are not known for many of the big questions of
complexity theory most notably the P versus NP Question However several
highly non trivial absolute results have b een proved For example it was shown
that using negation can sp eed up the computation of monotone functions which
do not require negation for their mere computation In addition many promising
techniques were introduced and employed with the aim of providing a low level
analysis of the progress of computation However as stated up front the fo cus of
this epilogue was elsewhere
App endix A
Some Computational
Problems
Although we view sp eci c natural computational problems as secondary to nat
ural complexity classes we do use the former for clari cation and illustration of
the latter This app endix provides de nitions of such computational problems
group ed according to the type of ob jects to which they refer i e graphs and
Bo olean formula
We start by addressing the central issue of the representation of the various
ob jects that are referred to in the aforementioned computational problems The
general principle is that elements of all sets are compactly represented as binary
strings without much redundancy For example the elements of a nite set S
e g the set of vertices in a graph or the set of variables app earing in a Bo olean
formula will b e represented as binary strings of length log jS j
A Graphs
Graph theory has long b ecome recognized as one of the more
useful mathematical sub jects for the computer science student to
master The approach which is natural in computer science is the
algorithmic one our interest is not so much in existence pro ofs or
enumeration techniques as it is in nding e cient algorithms for
solving relevant problems or alternatively showing evidence that
no such algorithms exist Although algorithmic graph theory was
started by Euler if not earlier its development in the last ten
years has b een dramatic and revolutionary
Shimon Even Graph Algorithms
A simple graph G V E consists of a nite set of vertices V and a nite set of
edges E where each edge is an unordered pair of vertices that is E ffu v g
APPENDIX A SOME COMPUTATIONAL PROBLEMS
u v V u v g This formalism do es not allow self lo ops and parallel edges which
are allowed in general i e non simple graphs where E is a multi set that may
contain in addition to two element subsets of V also singletons i e self lo ops
The vertex u is called an end p oint of the edge fu v g and the edge fu v g is said
to b e incident at v In such a case we say that u and v are adjacent in the graph
and that u is a neighb or of v The degree of a vertex in G is de ned as the number
of edges that are incident at this vertex
We will consider various sub structures of graphs the simplest one b eing paths
A path in a graph G V E is a sequence of vertices v v such that for every
def
i f g it holds that v and v are adjacent in G Such a path is said
i i
to have length A simple path is a path in which each vertex app ears at most
once which implies that the longest p ossible simple path in G has length jV j
The graph is called connected if there exists a path b etween each pair of vertices
in it
A cycle is a path in which the last vertex equals the rst one i e v v
The cycle v v is called simple if and jfv v gj i e if v v
i j
then i j mo d and the cycle u v u is not considered simple A graph is
called acyclic or a forest if it has no simple cycles and if it is also connected then
it is called a tree Note that G V E is a tree if and only if it is connected and
jE j jV j and that there is a unique simple path b etween each pair of vertices
in a tree
A subgraph of the graph G V E is any graph G V E satisfying V V
and E E Note that a simple cycle in G is a connected subgraph of G in which
each vertex has degree exactly two An induced subgraph of the graph G V E
is any subgraph G V E that contain all edges of E that are contained in V
In such a case we say that G is the subgraph induced by V
Directed graphs We will also consider simple directed graphs a k a digraphs
where edges are ordered pairs of vertices In this case the set of edges is a subset
of V V n f v v v V g and the edges u v and v u are called anti parallel
General i e non simple directed graphs are de ned analogously The edge u v
is viewed as going from u to v and thus is called an outgoing edge of u resp
incoming edge of v The out degree resp in degree of a vertex is the number of
its outgoing edges resp incoming edges Directed paths and the related ob jects
are de ned analogously for example v v is a directed path if for every i
it holds that v v is a directed edge which is directed from v to v It is
i i i i
common to consider also a pair of anti parallel edges as a simple directed cycle
A directed acyclic graph DAG is a digraph that has no directed cycles Every
DAG has at least one vertex having out degree resp in degree zero called a sink
resp a source A simple directed acyclic graph G V E is called an inward
resp outward directed tree if jE j jV j and there exists a unique vertex
called the ro ot having out degree resp in degree zero Note that each vertex
in an inward resp outward directed tree can reach the ro ot resp is reachable
from the ro ot by a unique directed path
Note that in any DAG there is a directed path from each vertex v to some sink resp from
A GRAPHS
Representation Graphs are commonly represented by their adjacency matrix
and or their incidence lists The adjacency matrix of a simple graph G V E is a
jV j by jV j Bo olean matrix in which the i j th entry equals if and only if i and
j are adjacent in G The incidence list representation of G consists of jV j sequences
th
such that the i sequence is an ordered list of the set of edges incident at vertex i
Computational problems Simple computational problems regarding graphs
include determining whether a given graph is connected and or acyclic and nd
ing shortest paths in a given graph Another simple problem is determining whether
a given graph is bipartite where a graph G V E is bipartite or colorable if
there exists a coloring of its vertices that do es not assign neighboring vertices the
same color All these problems are easily solvable by BFS
Moving to more complicated tasks that are still solvable in p olynomial time we
mention the problem of nding a p erfect matching or a maximum matching in a
given graph where a matching is a subgraph in which all vertices have degree a
p erfect matching is a matching that contains all the graph s vertices and a maximum
matching is a matching of maximum cardinality among all matching of the said
graph
Turning to seemingly hard problems we mention that the problem of deter
mining whether a given graph is colorable i e G C is NP complete A few
additional NP complete problems follow
A Hamiltonian path resp Hamiltonian cycle in the graph G V E is a
simple path resp cycle that passes through all the vertices of G Such a
path resp cycle has length jV j resp jV j The problem is to determine
whether a given graph contains a Hamiltonian path resp cycle
An indep endent set resp clique of the graph G V E is a set of vertices
V V such that the subgraph induced by V contains no edges resp
contains all p ossible edges The problem is to determine whether a given
graph has an indep endent set resp a clique of a given size
A vertex cover of the graph G V E is a set of vertices V V such that
each edge in E has at least one end p oint in V Note that V is a vertex
cover of G if and only if V n V is an indep endent set of V
A natural computational problem which is b elieved to b e neither in P nor NP
complete is the graph isomorphism problem The input consists of two graphs
G V E and G V E and the question is whether there exist a and
onto mapping V V such that fu v g is in E if and only if f u v g is in
E Such a mapping is called an isomorphism
some source to each vertex v In an inward resp outward directed tree this sink resp source
must b e unique The condition jE j jV j enforces the uniqueness of these paths b ecause
combined with the reachability condition it implies that the underlying graph obtained by
disregarding the orientation of the edges is a tree
APPENDIX A SOME COMPUTATIONAL PROBLEMS
A Bo olean Formulae
In Sec Bo olean formulae are de ned as a sp ecial case of Bo olean circuits
cf Sec Here we take the more traditional approach and de ne Bo olean
formulae as structured sequences over an alphab et consisting of variable names and
various connectives It is most convenient to de ne Bo olean formulae recursively
as follows
A variable is a Bo olean formula
If are Bo olean formulae and is a t ary Bo olean op eration then
t
is a Bo olean formula
t
Typically we consider three Bo olean op erations the unary op eration of negation
denoted neg or and the b ounded or unbounded conjunction and disjunction
denoted and resp ectively Furthermore the convention is to shorthand
t
by and to write or instead of and similarly
i t t
i
for
Two imp ortant sp ecial cases of Bo olean formulae are CNF and DNF formulae
A CNF formula is a conjunction of disjunctions of variables and or their negation
t
t
i
where each is either that is is a CNF if each has the form
i j i j i i
i
j
a variable or a negation of a variable and is called a literal If for every i it holds
that t then we say that the formula is a CNF Similarly DNF formulae are
i
de ned as disjunctions of conjunctions of literals
The value of a Bo olean formula under a truth assignment to its variables is
t
de ned recursively along its structure For example has the value true
i
i
under an assignment if and only if every has the value true under We say
i
that a formula is satis able if there exists a truth assignment to its variables
such that the value of under is true
The set of satis able CNF resp CNF formulae is denoted SAT resp SAT
and the problem of deciding membership in it is NP complete The set of tau
tologies i e formula that have the value true under any assignment is coNP
complete even when restricted to DNF formulae
Bibliography
S Arora and B Barak Complexity Theory A Modern Approach Cambridge
University Press to app ear
L Berman and J Hartmanis On isomorphisms and density of NP and other
complete sets SIAM Journal on Computing Vol pages
A Church An Unsolvable Problem of Elementary Number Theory Amer
J of Math Vol pages
A Cobham The Intristic Computational Di culty of Functions In Proc
Iternational Congress for Logic Methodology and Philosophy of Science
pages
S A Co ok The Complexity of Theorem Proving Pro cedures In rd ACM
Symposium on the Theory of Computing pages
J Edmonds Paths Trees and Flowers Canad J Math Vol pages
S Even Graph Algorithms Computer Science Press
S Even A L Selman and Y Yacobi The Complexity of Promise Problems
with Applications to Public Key Cryptography Information and Control
Vol pages
U Feige S Goldwasser L Lov asz S Safra and M Szegedy Approximating
Clique is almost NP complete Journal of the ACM Vol pages
Preliminary version in nd FOCS
S Fortune A Note on Sparse Complete Sets SIAM Journal on Computing
Vol pages
M R Garey and D S Johnson Computers and Intractability A Guide to the
Theory of NP Completeness W H Freeman and Company New York
O Goldreich On Promise Problems a survey in memory of Shimon Even
ECCC TR
BIBLIOGRAPHY
O Goldreich Computational Complexity A Conceptual Perspective Cam
bridge University Press
D Ho chbaum ed Approximation Algorithms for NP Hard Problems PWS
J E Hop croft and J D Ullman Introduction to Automata Theory Languages
and Computation Addison Wesley
R M Karp Reducibility among Combinatorial Problems In Complexity
of Computer Computations R E Miller and J W Thatcher eds Plenum
Press pages
R M Karp and R J Lipton Some connections b etween nonuniform and uni
form complexity classes In th ACM Symposium on the Theory of Com
puting pages
E Kushilevitz and N Nisan Communication Complexity Cambridge Uni
versity Press
R E Ladner On the Structure of Polynomial Time Reducibility Journal of
the ACM Vol pages
L A Levin Universal Search Problems Problemy Peredaci Informacii
pages in Russian English translation in Problems of Infor
mation Transmission pages
M Li and P Vitanyi An Introduction to Kolmogorov Complexity and its
Applications Springer Verlag August
Y Matiyasevich Hilbert s Tenth Problem MIT Press
R Motwani and P Raghavan Randomized Algorithms Cambridge University
Press
N Pipp enger and M J Fischer Relations among complexity measures Jour
nal of the ACM Vol pages
E Post A Variant of a Recursively Unsolvable Problem Bul l AMS Vol
pages
H G Rice Classes of Recursively Enumerable Sets and their Decision Prob
lems Trans AMS Vol pages
A Selman On the structure of NP Notices Amer Math Soc Vol
page
C E Shannon A Symbolic Analysis of Relay and Switching Circuits Trans
American Institute of Electrical Engineers Vol pages
M Sipser Introduction to the Theory of Computation PWS Publishing
Company
BIBLIOGRAPHY
B A Trakhtenbrot A Survey of Russian Approaches to Perebor Brute Force
Search Algorithms Annals of the History of Computing Vol pages
C E Turing On Computable Numbers with an Application to the Entschei
dungsproblem Proc Londom Mathematical Soceity Ser Vol pages
A Correction ibid Vol pages
J von Neumann First Draft of a Rep ort on the EDVAC Contract No
W ORD Mo ore School of Electrical Engineering Univ of Penn
Philadelphia Reprinted in part in Origins of Digital Computers Selected
Papers Springer Verlag Berlin Heidelb erg pages
Index
coNP Author Index
Church A
EXP
Cobham A
NP
Co ok S A
as pro of system
Edmonds J
as search problem
Even S
Optimal search
Karp R M
traditional de nition
Ladner R E
NPC see NP Completeness
Levin L A
NPI
Selman A L
P
Shannon C E
as search problem
Turing A M
PC
Yacobi Y
Algorithms see Computability the
PF
ory
Computability theory
Computational problems
Bo olean Circuits
SAT
b ounded fan in
XC
constant depth
Bipartiteness
depth
Bounded Halting
Monotone
Bounded Non Halting
size
Clique
unbounded fan in
CSAT
uniform
Entscheidungsproblem
Bo olean Formulae
Exact Set Cover
Factoring Integers
clauses
Graph Colorability
CNF
Graph Colorability
DNF
Graph Isomorphism
literals
Halting Problem
Hamiltonian path
Church Turing Thesis
Indep endent Set
Circuits see Bo olean Circuits
Perfect Matching
CNF see Bo olean Formulae
Primality Testing
Cobham Edmonds Thesis
Complexity classes SAT
INDEX
Polynomial time Reductions Set Cover
Undirected Connectivity
Self reducibility Vertex Cover
Turing reductions Computational Tasks and Mo dels
Rice s Theorem
Constant depth circuits see Bo olean
Search problems
Circuits
versus decision
Co ok reductions see Reduction
Decision problems
Self reducibility see Reduction
Space Complexity
Finite automata
Formulae see Bo olean Formulae
Time complexity
Turing machines
Graph theory
multi tape
non deterministic
Halting Problem see Computational
single tap e
problems
with advice
Hilb ert s Tenth Problem
Turing reductions see Reductions
Karp reductions see Reduction
Uncomputable functions
Kolmogorov Complexity
Undecidability
Universal algorithms
Levin reductions see Reduction
Universal machines
Monotone circuits see Bo olean Cir
cuits
NP Completeness
Optimal search for NP
Oracle machines
P versus NP Question
Polynomial time Reductions see Re
duction
Post Corresp ondence Problem
Promise Problems
Reductions
Co ok Reductions
Downwards self reducibility
Karp Reductions
Levin Reductions
parsimonious