Languages That Capture Complexity Classes

y

Neil Immerman

Intro duction

Wepresent in this pap er a series of languages adequate for expressing exactly

those prop erties checkable in a series of computational complexity classes For

example weshow that a prop erty of graphs resp ectively groups binary strings

etc is in p olynomial time if and only if it is expressible in the rst order

language of graphs resp ectively groups binary strings etc together with a

least xed p oint op erator As another example a prop erty is in logspace if and

only if it is expressible in rst order logic together with a deterministic transitive

closure op erator

The ro ots of our approach to complexity theory go back to when Fagin

showed that the NP prop erties are exactly those expressible in second order

existential sentences It follows that second order logic expresses exactly those

prop erties which are in the p olynomial time hierarchy Weshow that adding

suitable transitive closure op erators to second order logic results in languages

capturing p olynomial space and exp onential time resp ectively

The existence of such natural languages for each imp ortant complexity class

sheds a new light on complexitytheory These languages rearm the imp or

tance of the complexity classes as much more than machine dep endent issues

Furthermore a whole new approach is suggested Upp er b ounds algorithms

can b e pro duced by expressing the prop ertyofinterest in one of our languages

Lower b ounds maybedemonstratedbyshowing that such expression is imp os

sible

For example from the ab ovewe knowthatP NP if and only if every

second order prop erty is already expressible using rst order logic plus least

xed p oint Similarly nondeterministic logspace is dierent from P just if there

is some sentence using the xed p oint op erator which cannot b e expressed with

a single application of transitive closure

SIAM J of Computing A preliminary version of this pap er app eared

as

y

Research supp orted by an NSF p ostdo ctoral fellowship Current address Computer

Science Dept UMass Amherst MA

In previous work Im Imb weshowed that the complexity of a prop

erty is related to the number of variables and quantiers in a uniform sequence

of sentences where each expresses the prop erty for structures of

n

size n Our present formulation is more pleasing b ecause it considers single

sentences in more p owerful languages

The rst order expressible prop erties at rst seemed to o weak to corresp ond

to any natural complexity class However we found that a prop erty is expressible

by a sequence of rst order sentences where each has a b ounded

n

numb er of quantiers if and only if this prop erty is recognized byasimilar

sequence of p olynomial size b o olean circuits of b ounded depth It follows that

the results of Furst Saxe and Sipser FSS and Sipser Si translate

precisely into a pro of that certain prop erties are not expressible in anyrst

order language

In this pap er wealsointro duce a reduction b etween problems that is new to

complexitytheory First order translations as the name implies are xed rst

order sentences which translate one kind of structure into another This is a very

natural way to get a reduction and at the same time it is very restrictive It

seems plausible to prove that such reductions do not exist b etween certain prob

lems We present problems which are complete for logspace nondeterministic

logspace p olynomial time etc via rst order translations

This pap er is organized as follows Section intro duces the complexity

classes we will b e considering Section discusses rst order logic Sections

and intro duce the languages under consideration Section considers the

relationship b etween rst order logic and p olynomial size b ounded depth cir

cuits The present lack of knowledge concerning the separation of our various

languages is discussed

Complexity Classes and Complete Problems

In this section we dene those complexity classess whichwe will capture with

languages in the following sections We list complete problems for some of the

classes In later sections wewillshowhow to express these complete problems in

the appropriate languages and we will also show that the problems are complete

via rst order translations More information ab out complexity classes maybe

found in AHU

Consider the following well known sequence of containments

L NL L P NP P

Here L is deterministic logspace and NL is nondeterministic logspace L

S S

L is the logspace hierarchyP P is the p olynomial time

k k

k  k 

hierarchy Most knowledgable p eople susp ect that all of the classes in the ab ove

containment are distinct but it is not known that they are not all equal

We b egin our list of complete problems with the graph accessibility problem

GAP fGj apathinG from v to v g

 n

Theorem Sa GAP is logspacecomplete for NL

We will see later that GAP is complete for NL in a much stronger sense

The GAP problem maybeweakened to a deterministic logspace problem by

only considering those graphs whichhave at most one edge leaving anyvertex

GAP fGjG has outdegree and a path in G from v to v g

 n

Theorem HIM GAP is oneway logspacecomplete for L

A problem whichliesbetween GAP and GAP in complexityis

UGAP fGjG undirected and a path in G from v to v g

 n

Let BPL b ounded probability logspace b e the set of problems S suchthat

there exists a logspace coinipping machine M and if w S then ProbM

accepts w while if w S then ProbM accepts w It follows

from the next theorem that UGAP is in BPL Thus UGAP is probably easier

than GAP

Theorem AKLL If r is a random walk of length jE jjV j in

an undirectedconnectedgraph G then the probability that r includes al l vertices

in G is greater than or equal to one half

Lewis and Papadimitriou LP dene symmetric machines to b e nondeter

ministic turing machines whose next move relation on instantaneous descriptions

is symmetric That is if a symmetric machine can move from conguration A

to conguration B then it is also allowed to move from B to A Let SymL b e

the class of problems accepted by symmetric logspace machines

Theorem LP UGAP is logspacecomplete for SymL

John Reif Re extended the notion of symmetric machines to allow al

ternation Essentially an alternating symmetric machine has a symmetric next

move relation except where it alternates b etween existential and universal states

S

Let SymL SymL b e the symmetric logspace hierarchy Reif

k

k 

showed that several interesting prop erties including planarity for graphs of

b ounded valence are in the symmetric logspace hierarchy It follows that they

are also in BPL

O 

Reif also showed that BPL is contained in O log ntimeandn pro cessors

on a probabilistic hardware mo dication machine HMM A

a b

c

Figure An alternating graph with two universal no des a c

Theorem Re

O 

SymL BP L ProbHMMTIMElog nPROCn

One may also consider harder versions of the GAP problem Let an alternat

ing graph G V E A b e a directed graph whose vertices are lab elled universal

or existential A V is the set of universal vertices Alternating graphs have

a dierent notion of accessibility Let APATHxy b e the smallest relation on

vertices of G such that

APATHxx

If x is existential and for some edge hx z i APATHzy holds then APATHxy

If x is universal there is at least one edge leaving x and for all edges hx z i

APATHzy holds then APATHxy

See Figure where APATHab holds but APATHcb do es not Let

AGAP fGjAP AT H v v g

G  n

It is not hard to see that AGAP is the alternating version of GAPandthus

is complete for ASPACElogn Recalling that this class is equal to P CKS

wehave

Theorem Im AGAPislogspacecomplete for P

Note that the AGAP problem is easily seen to b e equivalent to the monotone circuit value

problem shown to b e complete for P in Go

First Order Logic

In this section weintro duce the necessary notions from logic The reader is

refered to En for more background material

A nite structure with vocabulary hR R c c i is a tuple S

k r

hf n gR R c c i consisting of a universe U f n g

k r

and relations R R on U corresp onding to the relation symbols R R of

k

k

and constants c c from U corresp onding to the constantsymbols c c from

r

r

For example if hE i consists of a single binary relation symb ol then



a structure G hfn gEi with vo cabulary is a graph on n vertices



Similarly if hM i consists of a single monadic relation symbol then a

structure S hfn gMi with vo cabulary is a binary string of length n

If is a vo cabulary let

ST RUC T fGjG is a structure with vo cabulary g

We will think of a problem as a set of structures of some vo cabulary Of

course it suces to only consider problems on binary strings but it is more

interesting to b e able to talk ab out other vo cabularies eg graph problems as

well

Let the relation sxy b e the successor relation on the naturals ie sx y

holds i x y Throughout this pap er we will assume that s is a logical

relation symb ol denoting the successor relation We will also assume that al l

structures under consideration have at least two elements Furthermore the

constantsymb ols and m will always refer to the rst and last elements of the

universe resp ectively

Wenow dene the rst order language L to b e the set of formulas built

up from the relation and constantsymbols of and the logical relation symb ols

and constant symbols sm using logical connectives variables

x y z and quantiers

If L let MOD b e the set of nite mo dels of

MOD fG ST RUC T jG j g

Let FO b e the set of all rst order expressible problems

FO fS j L S MODg

The following result is well known Fa AU Im but a new pro of of

the strictness of the containmentfollows from Corollary

Theorem FO is strictly containedinL

As we will see the availability of a successor or ordering relation seems crucial to simulate

computation The last two assumptions are nonstandard and unnecessaryHowever they are

convenient b ecause they make some pro ofs neater Otherwise wewould have to mo dify many

formulas to the less app ealing form xy x y xy x y xym

First Order Logic Plus Transitive Closure

In view of Theorem we wish to strengthen rst order logic so that wemay

express prop erties of complexity at least logspace Let x x y y beany

k k

formula It represents a binary relation on ktuples We add to our language the

op erator TC where TC denotes the reexive transitive closure of the relation

Let FO TC b e the set of prop erties expressible using rst order logic

plus the op erator TC Let FO pos TC b e the set of prop erties expressible

using only p ositive applications of TC ie not within any negation signs It is

easy to see for example that GAP FO pos TC

GAP TCE x y m

Theorem NL FO pos TC

Pro of It is easy to see that NL contains FO pos TC If Ax y canbe

checked in NL then so can TCAx y simply guess an A path Going the

other waywe are given a nondeterministic logspace turing machine M accepting

a set S of structures of a certain vo cabulary Wemust pro duce a sentence

in FO TCsuchthat

S fG ST RUC T jG j g

The rst idea needed to write is that an instantaneous description ID

ofMisofsizeO log n and can b e co ded with nitely manyvariables ranging

from to n

For example supp ose that M accepts a graph problem ie the vo cabulary

i consists of a single binary relation symbol Supp ose also that hE

Musesk log n bits of work tap e for problems of size n Input to M con

sists of n bits the adjacency matrix for E An ID for M is a k tuple

hq r r w h w h i Here q co des Ms state and variables r r co de the

k k

input head p osition Note that the input head is lo oking at a or according

as E r r holds or do es not hold in the input structure Finally w w co de

k

the k log n bits of Ms work tap e One h is equal to j where the work head

i

th

is p ointing to the j bit of w the rest of the h s are n

i i

th

The second idea is that using TC we can compute the j bit of w Let

i

ON w j mean that jlog n and bit j of w is on Starting with the successor

relation s we can use TC to express addition and then use TC againtotellifa

certain bit in a variable is on

Lemma The fol lowing predicates areexpressible in FO pos TC

PLUSx y z x y z

ON w j

Pro of Using s we can saythatthereisanedgefromhx y i to hu v i if u x

and v y

x y u v su x sy v

Using transitive closure we then get

PLUSx y z TCx y z

Now dene as follows

w j w j z PLUSw w z w z sz w sj j

Let ODDz abbreviate xy PLUSx x y sy z It follows that

ON w j z ODDz TC w j z

Once we can tell what the work head is lo oking at we can write the predicate



NEXT ID ID meaning that ID follows from ID in one moveofM Note

a b b a

that the successor relation is used not only to co de n bits into a single variable

but also to say that the read head moves one space to the left or right Any

input structure is given to a turing machine in some order and it maysearch

the structure in that order

Using one more p ositive application of TC we can express PATHID ID

a b

meaning that there is a computation of M starting at ID and leading to ID

a b

Let PATHID ID where ID and ID are Ms initial and nal IDs

i f i f

resp ectivelyThen FOpos TC and an input structure G of the correct

vo cabulary satises if and only if M accepts GThisproves Theorem

The sentence PATHID ID in the ab ove pro of has an interesting

i f

form It is written with several p ositive applications of TCbutweshowinthe

next theorem that these may b e merged into one Thus for each problem C in

NL there is a kary rst order formula such that a structure G is a member

of C i G satises TC m This suggests that GAP is complete for NL

via an extremely weak kind of reduction We call these new reductions rst

order translations and we discuss them right after weshowthat can indeed

b e written in this simple form

Theorem Let FO pos TC Then is equivalent to a formula of

the form

TC u u v v m

k k

where is rst order and resp m istheconstant symbol resp m

repeated k times



This is a standard constructions See Fa Im Ima Imb

Pro of By induction on the complexityof There are ve cases

The formula is atomic or the negation of an atomic formula In this

case let u v be variables not o ccuring in Then TCu v m

TC x yq r We wish to replaceq r with m Put

s t x s t y s t x s t m y q

s t m s t m x y

s t m x r s t m y m

The variables t s split the path in three stages st Setx toq

and go to next stage st m Takea step and stay in this stage

When you reachr go to next stage st mm Setx m and stop

Thus as desired

TCs t x s t y m

xT C u v x m Here the notation means that the transitive

closure is taken over the relation u v and the variable x o ccurs free in

Put

ux v x u x v

u v x x x u m v m x m

Notice that allows a guess of x on the rst step and thus

TCux v x m

xT C u v x m In this case wesimulate the x bysearching

through all xs in order using s Im not sure whether the result holds if

s is not available Put

ux v x u m u v x x x u m v sx x

Thus TC ux v x m

TC TC x yu v m m In this case the formula has

free variablesx y u v The inner transitive closure is onx y treating the

other variables as parameters The outer transitive closure is onu vWe

combine these two TCs into a single transitive closure on dened as

follows

x u v y u v x y u v u

x m u v u u v v x yu v

x m y u v

We claim that TC x u v y u v m This holds b ecause a

path consists exactly of a series of u v paths from tom with u v

xed At the end of anysuchpathweknow that TC x y u v m

holds The path maynow appropriately step from m u vto v w

ie reach v and b egin trying to movefromvtow

Note that the cases of disjunction and conjunction follow easily from cases

and resp ectively

Denition Let and bevocabularies where hR R i and R is

r i

an a ary relation symbol Let k beaconstant An interpretation ofL

i

in L is a sequenceofr formulas

U x x x x x i r

k i k a k

i

from L such that the sentence x x U x is valid

k

Suchaninterpretation translates any structure G ST RUC T toa

structure G ST RUC T dened as follows

k

G j U x Universe G hx x i G

k

G

i G j x x R hx x

i a a

i i

i

See En for a discussion of interpretations b etween theories See also LG

for a pro of that SAT is NP complete via elementary reductions essentially

the same thing as our rst order translations

Denition Given two problems S ST RUC T and T ST RUC T

a rst order translation of S to T is an interpretation of L in L such

that

G S G T

As our rst example weprove

Corollary GAP is complete for NL via rst order translations

Pro of Let S ST RUC T beany problem in NL By theorems and

there is a constant k and a formula x x y y L suchthat

k k

given AST RUC T wehave

AS Aj TC m

iLet hU i be an interpretation of L in L where Let hE

U x x x x y x y Then is a rst order translation of S to

GAP

The rst order translation is an extremely weak kind of reduction and it is

clearly appropriate for comparing the p ower of logical languages It follows from

Theorem that rst order translations are a uniform version of constant depth

reductions cf CSV It would seem that pro jection reductions SV are

even weaker However an examination of the pro of of Theorem reveals that

not only havewe removed all but one o ccurrence of TC but wehave removed

all the quantiers as well In fact wehaveproved the following stronger version

of Theorem

Theorem Let FO pos TC Then is equivalent to a formula of

the form TC m where is a quantier free formula in disjunctive normal

form

I

where

i i

i

Each is a conjunction of the logical atomic relations s and their

i

negations

Each is atomic or negated atomic

i

For i j and are mutual ly exclusive

i j

Call a rst order translation hU i a projection translation if

j

eachofitsformulas is in the form of in the ab ove theorem It follows that

Corollary GAP is complete for NL via projection translations

Note that pro jection translations are a uniform version of the pro jection

reductions of SV

We close this section with a discussion of the complexity class FO TC

We mistakenly claimed in Im that the equality L FO TC could

b e derived simply by closing b oth sides of the equation of Theorem under

negation Unfortunately this is wrong and wenow susp ect that the two classes

are distinct Wecanprove the following

Theorem

L FO TC

k

k 

Pro of Let M b e a L turing machine and let A b e an input structure to

k

M Let the predicates EP AT H x yresp AP AT H x y mean thatx

M M

andy are IDs of M such that there is a computation path of M eachofwhose

IDs is existential resp universal except fory whichisuniversal or nal resp

existential or nal It is immediate from Theorem that EPATH and APATH

are expressible in FO pos TC Let ID and ID b e Ms initial and nal

i f

IDs Then M accepts A if and only if A satises the following sentence

ID ID ID Q ID E P AT H ID ID

 k k i

AP AT H ID ID E P AT H ID ID ID ID

 k f

Note that the ab ove pro of requires no nesting of TCs and TCs whichis

whywe susp ect that L FO TC

Other Transitive Closure Op erators

We next employ other op erators not quite as strong as TC to capture weaker

complexity classes For example let ST C b e the symmetric transitive closure

op erator Thus if x y is a rst order relation on ktuples then ST C is

the symmetric transitive closure of

ST C TCx y y x

The following theorem whose pro of is similar to the pro of of Theorem

shows that ST C captures the p ower of symmetric log space

Theorem

SymL FO pos ST C

U GAP is complete via rst order translations for SymL

We can also add a deterministic version of transitive closure whichwe call

x y let DT C Given a rst order relation

x y x y z x z y z

d

x y is true just if y is the unique descendentof xNow dene That is

d

DT C TC

d

Note that FO pos DT C is closed under negation Thus

Theorem

L FO DT C

GAP is complete for L via rst order translations

Toprove Theorem resp wemust check that the pro ofs of Theorem

Lemma and Theorem go through when we replace TC by STC

resp DTC Notice that the o ccurrences of TC in the pro of of Lemma are

of the form TC s t where for every tuplex there is at most oney suchthat

x y and furthermore there is noy suchthat t y It follows that

TC s t DT C s t ST C s t

Thus Lemma still holds with STC or DTC replacing TC as needed See

LP for a more detailed discussion of when nondeterministic TC compu

tations or deterministic DTC computations may b e replaced by symmetric

STC computations

To nish the pro of of part of Theorems and wefollow the pro of of

Theorem We can express NEXT ID ID usingON and no additional

a b

uses of TC Finally to express PATH from NEXT one additional use of the

STC for Theorem or DTC for Theorem suces

To prove part of Theorems and we observe that Theorem holds

when TC is replace by STC or DTC First consider STC The reader should

check that the theorem and pro of remain true if we replace TC everywhere by

STC A typical example is case Here xS T C u v x m Welet

u x v x u m u v x x x u m v sx x

and nd that

ST C ux v x m m

Toprove this last equivalence supp ose that holds Then for all x there is a

symmetric xpathp from tomItfollows that

x

p hm i p hmm mi p m

 m

is a path from tom m Converselyany minimal path from tomm

must include the edges

hm i hm i hmm mi

in that order Furthermore the part of the path b etween any consecutive pair of

the ab ove edges will have constant second comp onent But this just says that

for all x ST C u v x mm ie holds

In the DTC case wemust mo dify the construction of the pro of of The

orem so that a deterministic path is not transformed into a nondeter

ministic path The most interesting case is the existential quantier

xD T C u v x m Here instead of letting the path nder guess the cor

rect x we force the path to try all xs and go tom when a correct one is found

k

We use the fact that there is a path in an n vertex graph if and only if there is

k

such a path of length at most n Let ar ity z ar ity w ar ity u In

i i

the following z is a counter used to cut o a cycling path and w is used to nd

the edge leaving u if one exists We will abuse notation and write sz z

to mean thatz is the successor ofz in the lexicographical ordering induced

by the successor relation sLet

u z w x v z w x u v z w m x m

u m sx x z m v z w

u m sx x z m w m v z w u w x

u m x x sz z w v w u w x

u m x x z z m sw w v u u w x

Then

DT C m

This completes the pro ofs of Theorems and

In spite of an over optimistic claim in Im we are not sure whether

SymL FO ST C butwe susp ect that equalitydoesnot hold We

can prove the following

Theorem

SymL FO ST C BP L

Pro of The rst inclusion follows as in the pro of of Theorem The second

inclusion follows from Theorem and standard techniques see for example

Re Once weknow that the predicate x y is in BPL we can test if

x y holds with exp onentially low probablityoferror Then we can takea

random walk and use Theorem to see that ST C x y is also testable

in BPL

The p enultimate op erator we add in this section is least xed p oint LF P

Given a rst order op erator on relations

x Q z Q z M x z R R

k k

wesay that is monotone if R R implies R R For a monotone

dene

LF P minfRjRRg

It is well known that LF P exists and is computable in p olynomial time

in the size of the structure involved

Example The least xedpoint operator is a way of formalizing inductive

denitions of new relations Recal l the AGAP property discussedinSection

Consider the fol lowing monotone operator

h

Rx y x y z E x z Rz y

i

Ax z E x z Rz y

It is easy to see that

LF P AP AT H

and in fact

Theorem ImaVa P FO LF P

Instead of using LFP weintro duce a variantofitthatismoreinkeeping with

DTCSTC and TC Supp ose that x y and x are given formulas where

ar ity x ar ity y k For a given structure A these formulas dene an

alternating graph G whose universe consists of ktuples from A whose edge

relation is the set of pairs of ktuples for which holds in A and whose universal

no des are those ktuples satisfying We dene the alternating transitive closure

operator ATC to b e the op erator whose value on the pair x yxis

AP AT H We can dene ATC more precisely in terms of LFP as follows

G

Refering to example let

h

Rx y x y z x z Rz y

i

x z x z Rz y

Denition AT C LF P

We conclude this section with

Theorem

P FO ATC

AGAP is complete for P via rst order translations

Pro of This follows as in the pro ofs of Theorems and recalling

that ASPACElog n P CKS The construction of NEXT ID ID in

a b

the pro of of Theorem works without change noting that ATC is at least as

powerful as TC Writing the formula x meaning that ID x is in a universal

state is trivial The ASPACElog n machine M accepts a structure A i Aj

AT C NEXT ID ID

i f

Toprove part we just note that Theorem remains true with ATC

substituted for TC

Second Order Logic

In second order logic wehave rst order logic plus the abilitytoquantify over

relations on the universe The following theorem of Fagin was our original

motivation for this line of research

nd

Theorem Fa NP O existential

Fagins theorem says that a prop erty is recognizable in NP i it is expressible

by a second order existential formula Note that we no longer need sas

a logical symb ol b ecause in second order logic we can say There exists a

binary relation which is a total ordering on the universeClosing b oth sides of

Theorem under negation gives us that a problem is in the p olynomial time

hierarchy i it is expresible in second order logic

nd

Theorem St P O

k

Fagins original result used kary relations to enco de the O n bits of an

k

entire NT IMEn computation Thus he showed

k nd

NT IMEn O existential arityk NP

Lynch Ly p oints out that in the presence of addition as a new logical

relation on the universe the second order existential sentences can guess merely

k

the n moves and piece together the whole computation in arityk Thus he

shows

k nd

Theorem Ly NT IMEn O existential with PLUS arity k

Corollary

k nd

TIMEn O with PLUS arity k

c

c

k

Pro of Let S TIMEn It follows that

c

S A w w Q w T A w

c c

k k

where w w are b ounded in length by jAj and T A w isanNT IMEjAj

c

k

prop erty if c is o dd and a CONT IM EjAj prop erty if c is even It follows

from Theorem that T is expressible as a second order existential arityk

formula R R if c is o dd or if c is even R R Finally we

c c c c

k

can replace each w of length at most n by a relation R on A of aritykThus

i i

S A Aj R R Q R

c c

Conversely let b e a second order aritykformula

R R Q R

c c

where is rst order

In particular x Q x M R x where M is a constant size quanti

a a

k

er free formula It follows that we can test in TIMEn whether an input

c a

A satises This is done by guessing the Rs and xs in the appropriate existen

tial or universal state It remains to check the truth of a constant size quantier

free sentence This can easily b e done in time linear in jhAR R ij

c

Note that in the ab ove results the relation PLUSneed only b e added when

k is or otherwise it is denable

As in the previous section we can add closure op erators to second order logic

in order to express prop erties which seem computationally more dicult than

the p olynomial time hierarchyIfR S isasentence expressing a binary su

R and S thenTC ST C DT C p er relation on ktuples of relations

express the transitive closure symmetric transitive closure deterministic tran

sitive closure resp ectivelyof It is not hard to show

Theorem For k

k nd

DS P ACE n O arity k DT C

k nd

Sym S P AC E n O arity k pos ST C

k nd

NSPACEn O arity k pos TC

k

pro of A kary relation R over an n elementuniverse consists of n bits It is an

k

easy exercise to co de an O n space instantaneous description with a set of kary

relations X X and to write the rst order sentence X X Y Y

c M c c

meaning that ID Y follows from X in one move of turing machine M One apli

cation of the appropriate transitive closure op erator expresses an entire compu

tation

The following follows immediately

nd nd nd

Corollary PSPACE O TC O ST C O DT C

In similar fashion we can add a second order ATC op erator Recalling that

S

k

k n

AS P AC E n DT I M E c we discover

c

Theorem For k

k

n nd

DT I M E c O arity k AT C

c

Lower Bounds

Alower b ound byFurst Saxe and Sipser has interesting consequences for us

See FSS and also Aj and Ya The PARITY problem is to determine

whether the n inputs to a b o olean circuit include an even number which are on

Theorem FSS PARITY cannot becomputedbyasequenceofpoly

nomial size bounded depth circuits

Theorem interests us greatly b ecause of the following relation b etween

b ounded depth p olynomial size circuits and rst order sentences

Theorem Given a problem S and an integer d the fol lowing areequiv

alent

S is recognizedbyasequence of depth d polynomial size circuits with

bounded fanin at the bottom level

a

There exists a new logical relation R N and a rst order formula

inwhich R occurs such that expresses S The formula contains d

alternating blocks of quantiers

Pro of Wemay assume that the similaritytyp e of S is hM i

consisting of a single monadic predicate Thus to a rst order formula an input

of size n is a structure A hf n gM iTo a b o olean circuit the same

A

input is the string of n b o olean variables a a where a i M

n i A

Supp ose that holds and that the circuits C C accepting S have

k

depth d and size at most n and that their b ottom level has fanin at most

k For deniteness assume that the C s top gate is an and gate and that

i

their b ottom gate is an or gate The other cases are analagous Wemayalso

assume by rep eating p ortions of the circuit if necessary that the fanin at

k k

all levels of C b esides the b ottom is n Wemay lab el eachofthen edges

n

leaving a gate by a k digit integer in nary z z z In this wayeachor

k

gate v at the b ottom level has a lab el z z z z z Eachsuchgate

k
dk

in C is a disjunction c c of literals a ora Dene the relation Q

n k i i

as follows Qm z z z yj holds i j resp jm and literal a

dk y

resp a o ccurs in the gate with lab el z z z in C I know that this

y
dk n

is a mouthful but the p oint is that the logical relation Q co des all the circuits

C C Dene the sentence

Qm z y M y z z z z z z y

k

k d dk

Rm z y m M y

It follows that for any structure A Aj if and only if AS

What remains to b e shown is that we can remove the quantier y so that our

formula has d alternating blo cks of quantiers as required Dene the relation

R as follows For all m z if only y y satisfy Qm z y Qm z y m

k c

then cho ose w w distinct from these y s note that wemay assume that

c i

kn Let

i c j m Rm z Qm z hm z w ji

i

and put

y y y y y y y y Rm z y Rm z y m

k
 k k

Rm z y Rm z y m Rm z y M y

k k

Rm z y m M y Rm z y m M y

k k

Finally let

z z z z z z

k

k d dk

Then is as required

The converse is simpler Assume Then the sentence must

b e in the following form

z z z z Q z z R M z

k

k d d dk

where is quantier free By replacing eachuniversal resp existential blo ck

of quantiers by anandresp or gate we can rewrite as a circuit of

k

depth d and fanin n in which each b ottom gate whose lab el is the d k tuplec

is assigned the value R M c The formula R M c for a xed assignment

ofc is a b o olean combination of formulas of the form M c ie a constant

ij

size b o olean combination of literals indep endantofn Eachsuch formula can

b e written in disjunctive normal form if Q is and conjunctive normal form if

d

its The resulting circuit is p olynomial size and depth d with the b ottom

level of b ounded fanin as required

Note that the role of the new logical relation R in the ab ove pro of is to

translate a nonuniform sequence of circuits into a single rst order formula A

strong lower b ound on the expressivepower of rst order logic follows easily

from Theorems and

k

Corollary For any k and any logical relation R on N

PARITY FO R

Sipser also proved the following hierarchy result for b ounded depth circuits

Theorem Si For al l d thereisaproblem S acceptedbypolyno

d

mial size depth d circuits but not by polynomial size depth d circuits which

have bounded fanin at the bottom level

The problem S used by Sipser can b e expressed by the rst order formula

d

x x Q x Lx x

d d d

over structures with a single relation L of arityd A corollary of the ab ove

theorem is a very strong hierarchy result for rst order logic

k

Corollary For al l d for al l k and for any logical relation R on N

FOd alternations FO R d alternations

Conclusions

Wehaveshown that the imp ortant complexity classes C have corresp onding

languages L such that C consists of exactly those prop erties expressible in

L Thus questions of complexity can all b e translated to expressibility issues

Separating complexity classes is equivalenttoshowing that certain of the ab ove

op erators are more p owerful than others Ecient algorithms may b e obtained

simply by describing a problem in one of our languages Wehave also demon

strated a basic connection b etween b o olean circuits and rst order logic

Op en questions and work to b e done include the following

More precise b ounds on the arityofformulas needed to express prop erties

of a given complexity are needed

Wehaveintro duced rst order translations as a new kind of reduction

Many op en questions arise It seems p ossible to prove that rst order

translations b etween certain problems cannot exist Be careful however

showing that there is no rst order translation from SAT to GAP would

provethatNP L

More knowledge is needed concerning the increase in expressibility gained

TC

by alternating applications of op erators and negation Let be rst

k

order logic in which k alternations of TC and negation are allowed starting

TC

with TC Thus for example FOposTC NLWe conjecture

the following

TC TC

a

ST C ST C

b

From a pro of of any of the prop er containments in a or b it would

follow that L P Thus wewould b e satised with the more mo dest

hierarchy results without s as a logical symb ol

TC TC

c wo s wo s

ST C ST C

d wo s wo s

Wehave shown that

SymL FO ST C BP L

It would b e fascinating to know whether or not either of the ab ovecon

tainments is prop er

Of course everyone would like to see a pro of that some second order prop

erty is not expressible in rst order plus least xed p oint This would

imply that P NP

Finallywe hop e that attractiveversions of the ab ove languages will b e de

velop ed for actual use as programming andor database query languages

Acknowledgements My ideas for this pap er have b een claried during many

helpful discussions with colleagues Thanks in particular to Sam Buss Steve

Lucas John Reif Adi Shamir and Mike Sipser

References

AV Aho JE Hop croft and JD Ullman The Design and Analysis of

Computer Algorithms Addison Wesley

AV Aho and JD Ullman Universality of Data Retrieval Languages

Sixth Symp on Principles of Programming Languages

M Ajtai Formulae on Finite Structures Annals of Pure and Applied

Logic

Aleliunas Karp Lipton Lovaz and Racko Random Walks Universal

Traversal Sequences and the Complexity of Maze Problems th IEEE

FOCS Symp

Ashok Chandra and David Harel Structure and Complexity of Relational

Queries st Symp on Foundations of Computer Science

Also app eared in JCSS

Ashok Chandra Larry Sto ckmeyer Dexter Kozen Alternation JACM

No

Ashok Chandra Larry Sto ckmeyer and Uzi Vishkin Constant Depth Re

ducibility SIAM J of Comp No

H Enderton A Mathematical Introduction to Logic Academic Press

Ron Fagin Generalized FirstOrder Sp ectra and PolynomialTime Recog

nizable Setsin Complexity of Computation ed R Karp SIAMAMS

Proc

M Furst JB Saxe and M Sipser Parity Circuits and the Polynomial

Time Hierarchy nd IEEE FOCS Symp

L Goldschlager The Monotone and Planar Circuit Value Problems are

Log Space Complete for P SIGACT News No

Etienne Grandjean The Sp ectra of FirstOrder Sentences and Computa

tional Complexity SIAMJofComp No

Juris Hartmanis Neil Immerman and Stephen Mahaney OneWayLog

Tap e Reductions th IEEE FOCS Symposium

Neil Immerman Number of Quantiers is Better than Number of Tap e

Cells JCSS No June

Neil Immerman Upp er and Lower Bounds for First Order Expressibility

JCSS No

Neil Immerman Relational Queries Computable in Polynomial Time

th ACM STOC Symp

Neil Immerman Languages Which Capture Complexity Classes th

ACM STOC Symp

Harry Lewis and Christos H Papadimitriou Symmetric Space Bounded

Computation Theoret Comp Sci bf No

L Lovasz and PGacs Some Remarks on Generalized Sp ectra Zeitchr

f math Logik und Grund lagen d Math Bd

James Lynch Complexity Classes and Theories of Finite Mo dels Math

Sys Theory

John Reif Symmetric Complementation JACM No April

W Savitch Maze Recognizing Automata and Nondeterministic Tap e

Complexity JCSS

Michael Sipser Borel Sets and Circuit Complexity th Symp on Theory

of Computation

Larry Sto ckmeyer The PolynomialTime Hierarchy Theoretical Comp

Sci

S Skyum and LG Valiant A Complexity Theory Based on Bo olean

Algebra JACM No April

Larry Sto ckmeyer and Uzi Vishkin Simulation of Parallel Random Access

Machines by Circuits SIAM J of Comp No

G Turan On the Denability of Prop erties of Finite Graphs

LG Valiant Reducibility By Algebraic Pro jections LEnseignement

mathematique T XXVI I I

M Vardi Complexity of Relational Query Languages th Symposium

on Theory of Computation

Andrew ChiChih YaoSeparating the PolynomialTime HierarchybyOr

acles th IEEE Symp on Foundations of Comp Sci