This is page

Printer Opaque this

Time Hardware and Uniformity

David Mix Barrington

Neil Immerman

ABSTRACT We describ e three orthogonal complexity measures parallel

time amount of hardware and degree of nonuniformity which together

parametrize most complexity classes Weshow that the descriptive com

plexity framework neatly captures these measures using three parameters

quantier depth number of variables and typ e of numeric predicates

sp ectively A fairly simple picture arises in which the basic questions in

complexity theory solved and unsolved can b e understo o d as ques

tions ab out tradeos among these three dimensions

Intro duction

An initial presentation of complexity theory usually makes the implicit as

sumption that problems and hence complexity classes are linearly ordered

by diculty In the Chomsky Hierarchyeach new typ e of automaton

can decide more languages and the Time Hierarchy Theorem tells us that

adding more time allowsaTuring machine to decide more languages In

deed the word complexity is often used eg in the study of algorithms

to mean worstcase running time under which problems

are linearly ordered

Those of us who study structural complexityknow that the situation is

actually more complicated For one thing if wewant to mo del parallel com

putation we need to distinguish b etween algorithms whichtakethesame

amountofwork ie sequential time wecarehowmany pro cesses are

op erating in parallel and howmuch parallel time is taken These two dimen

sions of complexity are identiable in the usual mo dels b o olean circuits

width and depth PRAMs or other explicit parallel machines number

of pro cessors and parallel time alternating Turing machines space and

alternations or even deterministic Turing machines space and reversals

Hongs b o ok H gives an interesting general treatment of similarity b e

tween mo dels and duality b etween these two complexity measures

Figure gives a twodimensional layout of some wellknown and less

wellknown complexity classes To b e precise ab out our axes wehave

chosen unb ounded fanin circuit depth as our measure of parallel time

and circuit width as our measure of numb er of pro cesses This assumes

that the circuits have b een arranged into levels and that eachedgeinto

David Mix Barrington Neil Immerman

O

n

PH PSPACE EXPTIME

O

log n

q AC q NC q P

O

n AC AC NC P PSPACE

O

log n SC q NC PSPACE

O log n LOGSPACE q NC PSPACE

O NC q NC PSPACE

O O

O O log n n

O O log n log n n

width d e p t h

Figure Two Dimensions of Complexity Classes

a gate comes from either an input or a gate on the immediately previous

level Thus the depth is the numb er of levels and the width is dened as

the size of the largest level

The columns of our chart represent b ounds on depth and the rows

b ounds on width Blanks indicate classes where b oth are b ounded b elow

p olynomiallyandthus the resulting circuits cannot access the entire input

The named classes are fairly standard except for the use of a prex q to

indicate a change from a p olynomial to a quasip olynomial size b ound B

Thus q NC is the class of languages decidable by circuit families of p olylog

depth and quasip olynomial size it is a robust class which o ccurs several

times on the chart Finally question marks denote classes whichhaveno

distinctive names known to the authors and ab out whichweknow nothing

other than the obvious containment relations with their neighbors

Of course merely sp ecifying combinatorial b ounds on the circuits in a

family do es not fully sp ecify a For example any unary

language even an uncomputable one has a circuit family of O size

and depth which decides it In the circuit context we usually sp eak of

restricting circuit families byauniformity condition wesay that the

circuit must b e computable or that questions ab out it must b e answerable

by resourceb ounded computation It is equally sensible to sp eak of non

uniformity as a resource more of whichallows a circuit family to decide

more languages This resource forms the third axis of our parametrization

of complexity classes It exists in other mo dels as well given to

Turing machines KL or precomputation in parallel machines A

Ateach p oint on our twodimensional chart wehave a range of com

plexity classes obtained byvarying the uniformity condition For example

if b oth size and depth are p olynomially b ounded the chart indicates the

class P of languages decided by p olynomialtime Turing machines This

claim is true if the circuits are Puniform computable by a p olytime

Turing machine or if they are DLOGTIME uniform direct connection

language decidable by a randomaccess Turing machine in time O log n

see BIS or any uniformity condition in b etween Howeverifweallow

ourselves more than p olynomial time to compute the circuit wemaybe

Time Hardware and Uniformity

able to decide more languages If we allow ourselves enough extra time

we can denitely do so For example if we allow more than exp onential

time we can decide the unary version of the universal language for Turing

machines with some sup erp olynomial time b ound On the other hand one

can imagine uniformity conditions so restrictive that a general simulation

of a Turing machine is imp ossible

In general as wepassupward adding more nonuniformity along the

third axis we pass through three regions one with to o little nonuniformity

where the basic constructions relating the circuit mo del to other mo d

els cannot b e carried out a second robust region where a wide range of

denitions give the same class and a third region where additional non

uniformity gives steadily larger classes The distinction can b e quite imp or

tant as we see in the case of the class NC Asshown in BIS we can dene

avery restrictive uniformity notion under whichNC b ecomes the class of

regular languages If our nonuniformity resource is b etween DLOGTIME

and NC itself we get a robust class equal to ALOGTIME And if weallow

p olynomial time to build our circuits we can then do integer division and

related problems BCH which as far as weknow we couldnt do b efore

This may b e an example of where nonuniformity can replace one of the

other two resources In some cases we know of limits on the p otential p ower

of nonuniformity to do so at least sub ject to complexitytheoretic assump

tions For example Karp and Lipton KL haveshown that no amountof

nonuniformity can allowPtosimulate all of uniform NP unless the

p olynomial hierarchy collapses to the second level It would b e interesting

to have a parallel result for P and NC and recentwork of Ogihara Cai

and Sivakumar O CS has made progress toward this One would liketo

derive unlikely complexitytheoretic consequences from for example the

hyp othesis that nonuniform NC contains uniform P or equivalently that

there is a sparse set complete for P under NC Turing reductions Cai and

Sivakumar show that if there is a sparse set complete for P under logspace

uniform NC manyone reductions then P is equal to logspace uniform

NC Dvan Melkeb eek vM has subsequently shown a similar result for

sparse sets complete under truthtable reductions

In general our techniques for proving lower b ounds on

are combinatorial and algebraic and apply to the total ly nonuniform ver

sions of the circuit classes A notable exception is the result by Allender and

Gore AG that the integer p ermanent function is not in DPOLYLOGTIME

uniform q ACC though for all weknowitmightbeinLOGSPACEuniform

ACC

There is a certain amountofoversimplication in thinking of eachof

our three parameters as a single axis For one thing our nonuniformity

resource is dened in terms of the complexity of languages which talk

ab out the circuits so this dimension may b e as nonlinear as the whole

picture However the uniformity conditions we normally consider happ en

to b e linearly ordered More imp ortantly the parallel time axis is dened

David Mix Barrington Neil Immerman

in terms of particular primitive op erations on the data In the circuit mo del

a single gate computes an AND or OR in a parallel machine the most

powerful op erations are the concurrent read and concurrent write and the

alternations of a Turing machine are also dened in terms of AND and OR

But there is nothing sacred ab out AND and OR in each mo del wecan

consider other op erations such as MAJORITY more p owerful than AND

and OR or mo dular counting orthogonal to them In the circuit mo del

these op erations are emb o died as new kinds of gates in Turing machines

as new acceptance conditions as in the classes P or PP and in parallel

machines as new global op erations such as the scan op eration on the

Connection Machine

Here we consider these three dimensions and the variety of op erations in

the framework of descriptive complexity where we measure the complexity

of a language by the syntactic resources in a particular logical formalism

needed to express the prop ertyofmembershipinitIIIb We

review this framework in Section b elow It has b een known for some

time that two parameters in descriptive complexitynumber of variables and

quantier depth corresp ond exactly to space and parallel time in either the

circuit or PRAM mo dels Ib More recently along with Straubing BIS

B wehave extended the framework to deal with the third dimension

and with more general op erations in the context of rstorder formulas

or constantdepth p olysize circuits There varying uniformity conditions

corresp ond to new atomic predicates and any asso ciative op eration with an

identity can b e mo deled by a new typ e of quantier

With this approach one can deal with a uniform family of circuits in

terms of a single logical formula which denes the entire family It is then

p ossible to sp eak of logically uniform circuits and the traditional unifor

mity conditions which happ en to coincide with logical uniformitysuchas

DLOGTIME uniformity for constantdepth circuits are thus b etter moti

vated Furthermore pro ofs using logical uniformity are arguably simpler

two examples of this are the uniformityAG B of the upp er b ounds on

the p ower of ACC Y BT GKR and the relationship b etween threshold

n

circuits and algebraic circuits over GF ReBFSFVB

In this pap er weshow that the descriptive complexity framework can

deal with all three dimensions and with general op erations in virtually

any p ossible combination Sp ecically

In Section we review the descriptive complexity framework and

dene quantiers for arbitrary op erations

In Section weprove that the relationship of Ibbetween descrip

tive complexity circuits and PRAMs holds in the presence of these

arbitrary op erations adding the new quantiers to the logical for

malism corresp onds exactly to adding a new typ e of gate to the cir cuits or a new global op eration to the PRAMs If b oth circuit depth

Time Hardware and Uniformity

and width are p olynomially b ounded the circuits are DLOGTIME

uniform

In Section we show that adding limited secondorder variables to the

formalism corresp onds to increasing the size b ound on the b o olean

circuits or the pro cessor b ound on the PRAMs This extends the

treatment of quasip olynomialsize circuit classes in B

In Section weshow that adding new atomic predicates to the formal

ism corresp onds to allowing the b o olean circuits to b e less uniform

This is true b oth in the p olynomialsize and in largersize domains

In Section we attempt to extend the logical framework to describ e

circuit widths of less than p olynomial or equivalentlyvariables to

taling to olog n bits The main idea is to allowonlyvariables which

range over subp olynomialsize sets but there are complications b e

cause the basic logical language assumes for example that it is p os

sible to refer to every p osition in the input which normally entails a

variable ranging from to n Though it is not entirely satisfactory

wedodevelop enough machinery to bring the constantwidth char

acterizations of NC B and PSPACE CF into the framework

Background Descriptive Complexity

In this section wewillgiveanoverview of the basic denitions of descriptive

complexity theoryFor more detailed presentations of this same material

see I or I We will closely follow the development in IL esp ecially

for the notions of reductions and op erators

Any complexity theory starts with a formally dened set of problems a

mo del of computation and a set of resource b ounds Our central notion will

b e to replace deciding a problem by describing it Our mo del of computa

tion is to write down a logical formula which is true of an input exactly

when the decision problem has a p ositive answer Our resource measures

will then b e prop erties of the logical formula suchasthenumb er and typ e

of quantiers variables and atomic predicates it contains

Our general is to take some input and return a yes

oranoanswer We need to have the input in a form which our formulas can

talk ab out so we co de all our inputs as nite logical structuresFor example

supp ose the input is a binary string w w w of length n Our atomic

n

statements ab out this string will b e to name the bit in p osition iwhich

we will do by a predicate M i Following the traditions of mathematical

logic we co de the string as a structure A with universe f ng the

w

input p ositions and one unary relation M on this universe Thus A

w

A

w

hf ngM i where





A

w



w i M i

David Mix Barrington Neil Immerman

On the other hand our problem might b e a set of directed graphs rather

than a set of strings In this case our structures would haveauniverse

consisting of the vertices and a binary relation E on this universe suchthat

E i j is true if and only if there is an edge from i to j Or our input might

be an entire relational database with several dierent relations dened on

a xed universe What we need to know b efore we can describ e a problem

is exactly which relation symb ols and constant symbols whichmaybe

thought of as ary function symb ols are available in our language

In general a vocabulary

a a

t

c c i hR

s

t

is any tuple of input relation symb ols and constant symbols For example

the vo cabulary of strings is hM i consisting of a single monadic re

s

lation symbolandthevo cabulary of graphs hE sti consists of the

g

binary edge predicate and two constant symb ols

To dene a structure A over vo cabulary we need to provide a nite

A A

universe a table for each relation R and a value for each constant c We

i i

will use the notation jAj to denote the universe of AandjjAjj its cardinality

In this pap er all our universes will b e ordered and wewillthus assume they

they consist of the rst n p ositiveintegers

jAj f ng where n jjAjj

Formally a structure A of vo cabulary is a tuple

A A A A

A hf ngR R c c i

t s

A a A

i

Here R is a subset of jAj for each i t and c is an elementof jAj for

i j

each j s

For any dene STRUC to b e the set of all nite structures of vo

cabulary Dene a complexity theoretic problem to b e any subset of

STRUC for some For example a problem over binary strings is a

subset of STRUC and a graph problem is a subset of STRUC

s g

Next we need to build a system of logical formulas to talk ab out

a problem with vo cabulary Along with the symb ols of we will al

ways have a xed set of numeric relation symb ols and constantsymb ols

SUC BIT min max Wethenallow the b o olean connectives

weintro duce variables x y z ranging over jAj and allow rstorder

quantication of these variables by the usual quantiers and

Here refers to the usual ordering on the universe f ng SUC is the

successor relation and min and max refer to the rst and last elements in the

th

total ordering BIT i j holds i the j bit in the binary expansion of i isaone

These relations are called numeric b ecause for example BITi j and i j

describ e the numeric values of i and j and do not refer to any input predicates

Time Hardware and Uniformity

This denition gives us a descriptive complexityclassFO consisting

of all problems subsets of STRUC which can b e dened using the

rstorder language L When the vo cabulary is understo o d well call

this class just FO It turns out that this class is a familiar one from circuit

complexity and parallel complexity it equals the logtime uniform version

of the circuit class AC and also the problems solvable in constanttimeby

a natural parallel machine a version of a PRAM with p olynomially many

pro cessors BIS

All of our later descriptive complexity classes maybeviewed as augmen

tations of this class FO As a rst example consider adding a new quantier

to and We dene the ma jorityquantier M so that Mxx holds if

more than half the elements of the universe satisfy x With this new to ol

in our rstorder language we can dene a larger set of problems called

FOM This class is also familiar as it is equal to the logtime uniform

version of ThC the problems solvable by constantdepth p olynomialsize

threshold circuits BIS After building up some more machinerywewill

describ e a general metho d to pro duce new op erators of this kind

FirstOrder Reductions

In this section we dene rstorder reductions whichprovide the most

natural way to reduce one problem to another in the descriptive setting

They are exactly manyone reductions which are denable by rstorder

formulas At the end of the section we will also dene some imp ortant

sub classes of rstorder reductions

Supp ose that S STRUC andT STRUC are anytwo problems

a

a

r

c c i In order to reduce S to T wemust Let hR R

s

r

provide a wayforany structure A to b e mapp ed to a new structure I A

STRUC Before giving the denition weprovide an example

Example Let graph reachabilityREACH denote the following prob

lem given a graph G and vertices s t determine if there is a path from

s to t in G Let REACH b e the restriction of the REACH problem to

u

undirected graphs Let REACH b e the restriction of the REACH prob

d

lem in whichweonlyallow deterministic paths ie if the edge u v is

on the path then this must b e the unique edge leaving u Notice that the

REACH problem is reducible to the REACH problem as follows Given

d u

a directed graph GletG b e the undirected graph that results from G by

the following steps

Remove all edges out of t

Remove all multiple edges leaving anyvertex

Make each remaining edge undirected

David Mix Barrington Neil Immerman

Observe that there is a deterministic path in G from s to t i there is a

path from s to t in G

The following rstorder formula accomplishes these three steps and

du

is thus a rstorder reduction from REACH to REACH More precisely

d u

the rstorder reduction is the expression I true st whose

du xy du

meaning is Make the new edge relation fx y j g and map s to s

du

and t to t

x y E x y x t z E x z z y

x y x y y x

du

The reason for the formula true is that wewant to put all elements of

the input structure into the output structure







jI Aj g jAj Aj true jAj

In this example the universes of the domain and range of the reduction

were equal and hence had the same size The general situation is a bit

more complicated b ecause wewant to allow the p ossibility of reducing one

problem to another which has a p olynomially larger universe Here wexan

integer k and let jI Aj b e a rstorder denable set of k tuples of elements

a

i

of jAjEach relation R of has a formula whose free variables are

i

i

k k

whichmaybeviewed as a free variables each x x x x

i

a a

i i

of whichisak tuple Similarlyeachconstantsymbol c of has asso ciated

j

k

with it a k tuple t t of constants from We recapitulate by giving

j j

the formal denition

Denition FirstOrder Reductions Let and be be twovocab

a

a

r

ularies with hR R c c iLetS STRUC and T

s

r

STRUC betwo problems Let k b e a p ositiveinteger Let

t t i I k h

s r

x x

d

b e a tuple consisting of an r tuple of formulas and an stuple of k tuples

of constants all from L Here d max a

i i

Then I induces a mapping I from STRUC toSTRUC as follows

Let A STRUC beany structure of vo cabulary and let n jjAjj

Then the structure I A is dened as follows

I A I A

I A I A

I AhjI AjR R c c i

r s

The universe is given by





k



Aj g g jI Aj g g jAj

k k

Time Hardware and Uniformity

That is the universe is the set of k tuples of A satisfying The ordering

I A

is given of jI Aj is the lexicographic ordering inherited from AEach c

j

I A

k

is determined bythek tuple of constants t t The relation R

j j

i

by the formula fori r

i





I A

k k k



R u u u u Aj u u

i

a a a

i

i i i

If the structure A interprets some variablesu then these may app ear

freely in the the s and t s of I and the denition of I A still makes

i j

sense This will b e imp ortant in Denition where we dene op erators

in terms of rstorder reductions

Supp ose that I is a manyone reduction from S to T ie for all A in

STRUC

AS I A T

Then wesay that I is a k ary rstorder reduction of S to T Furthermore if

the s are quantierfree and do not include BIT then I is a quantierfree

i

reduction

Valiant V dened a very lowlevel nonuniform reduction called a pro

jection A pro jection is a manyone reduction f f g f g such

that each bit of f w dep ends on at most one bit of w It can b e thought

of as a reduction computed by a circuit of depth zero dep ending on the

details of the denitions We next dene rstorder pro jections a syntactic

restriction of rstorder interpretations

Denition FirstOrder Pro jections Let I be a k ary rstorder re

duction from S to T as in Denition Let I h t t i

r s

Supp ose further that the s all satisfy the following projection condition

i

i s s

where the s are mutually exclusiveformulas in which no input relations

j

o ccur and each is a literal ie an atomic formula P x x or its

j j j

a

negation

k k

i holds in I A u In this case the predicate R hu u i hu

i

a a

i i

if u is true or if u is true for some j t and the corresp onding

j

literal u holds in AThus each bit in the binary representation of I A

j

is determined by at most one bit in the binary representation of AWesay

that I is a rstorder projection

Finally dene a quantierfreeprojection to b e a rstorder pro jection

that is also a quantierfree reduction Write S T S T to mean

fop qfp

that S is reducible to T via a rstorder pro jection resp ectively a quantier

free pro jection

Lo oking back at Example we see that I is a quantierfree reduc

du

tion but it is not a pro jection This is b ecause the formula x y looks du

David Mix Barrington Neil Immerman

at more than one bit of its input it dep ends on E x y and E y x and in

fact it may dep end on all E x z andE y zasz varies over all vertices

In fact there is a qfp that reduces REACH to REACH butwewillnot

d u

construct it here

General Operators

Firstorder reductions giveusamechanism for forming an op erator out

of any problem The op erator whichwe will also call acts rather

like a quantier and in fact this construction extends the generalized

quantiers of BIS A very similar construction is used in MP and in

KV where the op erator we call would b e denoted Q

Denition IL Op erator Form of a Problem Let and be vo

cabularies and let STRUC beany problem Let I be any rstorder

reduction with I STRUC STRUC Then I isawellformed

formula in the language FO over vo cabulary with the semantics

AjI I A

For example consider the case where and so is set of binary

s

strings A k ary reduction I then consists of a pair of formulas h i

k

from L with free variables x x I is a sentence that holds of

a structure ASTRUC exactly if the structure I AisinThis

k

structure I A is the string of length at most jAj consisting of the truth





k k



Aj a values of as the variables x x range over a jAj

k

The op erator is a generalized quantier binding the variables x x

Wehave already seen three sp ecial cases of this phenomenon cho osing

to b e the AND OR or MAJORITY languages gives the andM

quantiers resp ectively The generalized quantiers in BIS corresp onding

to languages over larger alphab ets t into this framework with having a

unary predicate for each letter of the language

We can rep eat this pro cess applying the op erator to a formula in which

already app ears We dene the complexity class FO to b e the set of

problems expressible by rstorder formulas in this extended language We

t

say that a problem S is rstorder Turing reducible to S i

S FO

If the problem is complete for C FO via a lowerlevel reduction

such as fop or qfp this means that we do not need the full p ower of arbitrary

applications of in FO In other words a normal form theorem for

FO applies

Denition IL Wesay that the language FO has the fop Normal

Form Property i every formula FO is equivalent to a formula

where

I

Time Hardware and Uniformity

where I is a rstorder pro jection If I is a quantierfree pro jection then

wesaythatFO has the qfp Normal Form Property

As an example recall the problem REACH from Example Consider

the following quantierfree pro jection I

c

I truehmin min i hmax max i

c

x x y y

x y E x y x x SUCx y y min

The formula describ es a graph on pairs of vertices There is an path

from hmin min i to hmax max i i every vertex in the graph is reachable

from minFor undirected graphs this is equivalent to connectivity

CONNECTIVITY REACHI

c

It is shown in I and I that the language FOTC is equal to NL

and has the qfp normal form prop ertyItthus follows that every problem

in NL is expressible in the form of Equation with I replaced by other

c

qfps

The role of arbitrary problems as generalized quantiers is nowsumma

rized by

Fact IL Let beaproblem and C acomplexity class that is closed

under rstorder Turing reductions Then

t

is complete for C if and only if C FO

fo

is complete for C if and only if C FO and FO has

fop

the fop normal form property

is complete for C if and only if C FO and FO has

qfp

the qfp normal form property

First Uniformity Theorem

An apparent limitation of rstorder logic as a means of describing problems

is that a single formula has only a xed numb er of quantiers and can

thus a priori only represent a language whichisdecidedby circuits of

constant depth Adding op erators for new functions expands the expressible

problems considerably but in order to deal with circuits with arbitrary

depth b ounds we need the notion of iterated quantier blocks Rather than

a single formula wenowhave a family of formulas varying with the input

size nbutinavery predictable wayFor any function tn we let our

David Mix Barrington Neil Immerman

formula consist of tn syntactic copies of a blo ck of quantiers followed by

some base formula It is imp ortant to note that when we rep eat a quantier

blo ck the variables are not changed Thus an FOtn formula uses only a

b ounded numb er of distinct variables For more detail see I

Denition IAsetC of structures of vo cabulary is a member

of FOtn i there exist quantierfree formulas M i k from

i

L and a quantier blo ck

 

QB Q v M Q v M

k k k

tn

suchthatifwe let QB M forn then for all structures

n

A of vo cabulary with jjAjj n A C Aj

n

This covers only ordinary quantiers wenow generalize Denition

to get the class FOtn by allowing some of the quantiers Q to b e

i

applications of

Recall that the op erator should b e used in the form I where I is

a rstorder reduction Denition

I h t t i

x x r s

d

We will write such an o ccurrence of in the quantier blo ckas

r

b x x t t

d s

r

where we use a tuple of b o olean variables b to co de a value b etween and

r

r This expression thus binds the b o olean variables b and the individual

variables x x The meaning is given by

d

r r r r

b x x t t h b b b r t t i

d s s

r

where b i denotes the substitution of the number i for the b o olean

r

variables b in

Denition FOtnAsetC of structures of vo cabulary is a

member of FOtn i there exist generalized quantiers G G G

k

and quantier free formula M from L and a quantier blo ck

 

G G QB

k

tn

suchthatifwe let QB M forn then for all structures

n

A of vo cabulary with jAj n

AC Aj

n

Here each generalized quantier G is either a limited quantier

i

G Q v M

i i i i

as in Denition or an application of

r

G b x x t t

i d s

Time Hardware and Uniformity

Similarlywe dene INDtn to b e the set of problems denable

by rstorder inductive denitions with op erators with inductive depth

at most tn Extending the argument in I this complexity class is

equivalent to the restriction of FOLFP in which all applications of

LFP have depth of nesting at most tn A typical example as we will see

is that the language FOM log n consisting of quantierblo cks including

the ma jority quantier iterated log n times is equal to the class ThC of

problems accepted by threshold circuits of depth O log n

Our main result in this section generalizing basic results in BIS and IL

is that the known relationships b etween rstorder descriptive complexity

classes and other standard parallel complexity classes are not aected by

the intro duction of the op erators This result will b e the basis of all our

discussions of uniformity in the remainder of the pap er First however we

must sp ecify the exact denitions of our various parallel mo dels

Following Ib wecho ose as our parallel machine mo del the CRCW

PRAM with p olynomially muchhardware The complexity class CRAMtn

is the set of problems solvable bysucha CRCWPRAM in parallel time

tn To generalize wemay allow the CRAM to have sp ecial hardware that

can execute the op erator in constant time We call the problems solvable

in tn parallel time by this augmented machine the class CRAMtn

The circuit complexity class ACtn is the set of problems solvable

by depthtn circuits with AND OR and gates of unb ounded fanin

Note that since need not b e a symmetric function the string or structure

dening the circuit must somehow sp ecify the order of input to the gates

The uniformity condition then will constrain the diculty of computing

the predicate INg h i meaning that gate h is input number i of gate

g This predicate is the natural generalization of the direct connection

language of Ru BIS

A circuit is a directed acyclic graph The leaves of the circuit are the

input no des Every other vertex is a gate The edges of the circuit indicate

connections b etween no des The edge a bwould indicate that the output

of gate a is an input to gate b

Dene the vo cabulary of circuits hE IN L G G G G ri

c

t

where E x y is a directed edge relation meaning that the output of no de

x is an input to no de y Lx means that the leaf x takes input value

G x G y G z and G w mean that the no des x y z and w are

t

and or not and gates resp ectively The constant r refers to

the ro ot no de or output of the circuit

In b elow rstorder uniform means there is a rstorder reduction

th n

I STRUC STRUC The n circuit is given by C I where

s c n

n

STRUC is the string consisting of n zeros In the predicates

s

of C must all b e computable in time DT I M E log n where n jjAjj is

n

the size of the input

In SV the nonuniform versions of CRAMtn and ACtn were

shown to b e equal Since then many other connections have b een estab

David Mix Barrington Neil Immerman

lished and they remain true in the uniform setting

Theorem For constructible and polynomial ly bounded tn and any

problem operator the fol lowing classes areequal

FO tn

INDtn

Firstorder uniform ACtn

DLOGTIME uniform ACtn

CRAMtn

Pro of

This is very similar to the pro of of the same fact without

Ib The pro of that INDtn contains CRAMtn is least dif

ferent we can inductively describ e the entire conguration of the CRAM

at some time t in terms of its conguration at time t In addition to

all the other cases when the CRAM uses the op eration this is simu

lated by the inductive denition using the same op eration Similarlythat

CRAMtn contains FOtn requires the same change as the

CRAM simulates the formula the new op erator can b e p erformed by

the CRAM when it is invoked in the formula Finally weshow by induc

tion on the structure of the inductive denition that FOtn contains

INDtn The idea is that we can rewrite any p ositive rstorder in

ductive denition into a quantier blo ck as in Denition Again the

pro of from Ib go es through with the change that o ccurrences of in

the inductive denition are copied into the quantier blo ck

This is immediate since DLOGTIME is contained in FOBIS

Here we are given the rstorder uniform ACtn circuits

We pro duce an FO inductive denition of the value of each gate in the

circuit The depth of the induction will b e equal to the depth of the circuit

We will dene by induction in FO the predicate VALUEg b whose

intuitive meaning is that the value of the gate g is the b o olean bThe

inductive denition for VALUE will b e a disjunction over the p ossible kinds

of gates leaf Thus the denition of VALUE is given as follows



VALUEg b DEFINEDg G g Lg b

L

G g C g b G g D g b



G g N g b G g T g b

t

Here G g meaning that g is a leaf is an abbreviation for xE x g

L

DEFINEDg meaning that g is ready to b e dened is an abbreviation for

xcE x g VALUEx c

Time Hardware and Uniformity

The predicate C g says that all of g s inputs are true D g says that

some of g s inputs are true and N g says that its input is false

C g hE h g VALUEh

D g hE h g VALUEh

N g hE h g hE h g VALUEh

The most interesting case is T g b In this case the inputs to g co de

a set of relations that are a valid input to the problem Consider the

simplest case in which this problem is co ded as a single relation R In this

case wehave

T g I where I h i

x

x hINg h x

x hINg h x VALUEh

Note that the existence of the predicate IN giving the numb ering of

the inputs to each gate is crucial here In BIS the weaker condition was

used that the ordering of the inputs was given Then the assumption that

all op erators were monoidal was used the appropriate formula was con

structed byentering b enign identityelements b etween the valid inputs

Since we are dealing with arbitrary op erators wehavenochoice but to

insist on the existence of the numb ering Note also that if FO contains

the class ThC then we can express the numb ering of the inputs if we are

given the ordering of the inputs

Here we are given the quantierblo ckwhich when iterated

tn times expresses the problem in question for structures of size nWe

showhow in DLOGTIME to recognize the direct connection language of

the equivalentAC circuits The idea is that each gate in the circuit

corresp onds to a quantier or b o olean connective or o ccurrence of in

the quantier blo ck indexed bythevalues of all the b ounded number of

variables and the time All that we need from DLOGTIME is the p ower

to compute the successor of a log nbit numb er and to nd a particular

numb er in a b ounded size tuple of log nbit numb ers

There were two reasons for the requirementthat tn b e p olynomially

b ounded in Theorem The rst is that the denition of IND requires

monotone inductive denitions which automatically close in at most p oly

nomially many steps This can b e alleviated bychanging to an iteration

op erator ITER which do es not require monotonicity This is equivalent

to replacing the least xed p oint op erator LFP by a partial xed p oint

op erator PFP The second reason concerns the class DLOGTIME uni

form ACtn For times tn greater than p olynomial we need more

David Mix Barrington Neil Immerman

than DTIMElog n exactly to read and copyvariables of length more than

log nThus to talk ab out uniformity for sup erp olynomialsize circuits we

interpret DLOGTIME uniform to mean DTIMElog n tn Similarly

rstorder uniform means that the circuits are rstorder recognizable This

automatically implies that the rstorder variables needed to describ e such

a circuit would b e of size logn tn bits With these p oints mo di

ed as ab ove Theorem remains true when the restriction that tnbe

p olynomially b ounded is removed

Variables That Are Longer Than log n Bits

We know from Ib Ithatthenumber v of log nbit variables in a

v

formula corresp onds approximately to the amount of hardware n in a

circuit or CRAMThus a constantnumber of log n bit variables gives us the

usual b ound of p olynomial hardware A constantnumber of secondorder

variables ie variables with p olynomially many bits gives exp onential

hardware In the following result the corresp ondence is not p erfect simply

b ecause the CRAM mo del with priority write is a dierent concurrent

write mo del of parallelism from the FOtn mo del InterestinglyinFact

using the more robust measure of DSPACE the b ound is tight

Fact Ib Let CRAMtn PROC pn bethecomplexity class

CRAMtn restricted to machines using at most O pn processors Let

INDtn VARv n bethecomplexity class INDtn restricted to induc

tive denitions using at most v n distinct variables Assume for simplicity

p

that both tn and the maximum size of a register wordare o nandthat

is a natural number Then

CRAMtn PROC n

INDtn VAR

CRAMtn PROC n

Fact Ib The PH is equal to the set of prop

erties checkable by a CRAM using exponential ly many processors and con

stant time



k

n

PH CRAM PROC

k

Fact I For k

k

DSPACEn VARk

In this section weshowhow to dene the number of variables in the

FOtn mo del in suchawaythateven when the number of variables is

more than a constant the simple uniform quantier blo ck structure is

Time Hardware and Uniformity

preserved Then we use this new denition to generalize Theorem to

two of our three complexity dimensions

Denition For v n a FOtn VARv n formula will havetwo

sorts of variables the domain variables xyz ranging over the uni

verse f ng plus extended variables X Y Z eachofv nlogn

bits

The extended variables maybequantied just like domain variables

However the extended variables do not o ccur as arguments to any input

relations Their only role is as arguments in the BIT predicate That is we

th

may assert BIT x Y meaning that the x bit of Y is a one Note that

this only makes sense for extended variables with at most n bits In this

pap er we only consider p olynomially b ounded v n for whichwe can use a

tuplex of domain variables One could consider even larger v n byusing

intermediate size variables b etween x and Y Dene FOtn VARv n

with v n to b e the extension to FOtn that we get by including a

b ounded number of v nlognbit extended variables

Furthermore extended variables can b e used in the natural way to dene

reductions that increase the size of a structure by more than a p olynomial

Thus wehave a denition of FOtn VARv n for any v n

As an example illustrating the ab ove denition the next prop osition

says that secondorder logic is rstorder logic with p olynomially many

variables

Prop osition For any problem and any function tn

O

SOtn FOtn VARn

Wecannow state our TwoDimensional Resources Theorem

Theorem For constructible v n constructible tn and any

problem operator the fol lowing classes areequal

FO tn VARO v n

ITERtn VARO v n

O v n log n

FO uniform ACtn WIDTH

O O v nlogn

DTIMEv n logtnunifACtn WIDTH

O v nlogn

CRAMtn HARD

The pro of of Theorem is quite similar to the pro of of Theorem

Wesimulate the computations as b efore and just check that the width

hardwarevariable resources needed are appropriate

David Mix Barrington Neil Immerman

Uniformity The Third Dimension

It has b een understo o d for a while that nonuniformity corresp onds in the

descriptive setting to the addition of numeric predicates Recall that a

numeric predicate is a predicate suchas or BIT that dep ends only on

the numeric values of its arguments not on any of the input predicates

Fact I Aproblem S is in Nonuniform AC i for some numeric

predicate N S is expressible in the language FON

It seems after a fair amountofinvestigation and soul searching Ib

BIS L that the right lowest level of uniformity corresp onds to the nu

meric predicates BIT This is equivalent to the set Once we

have a little bit of computation such as a ma jorityquantier or the de

terministic transitive closure op erator all that is needed is and BIT is

sup eruous BIS I

The following theorem gives two examples of a very general phenomenon

The idea of capturing p olynomialtime uniformity via the unary form of an

EXPTIME complete problem is from A

Theorem Theorem remains true in both the nonuniform and the

polynomialtime uniform settings Moreprecisely the classes mentionedin

that theorem remain equal in the fol lowing cases

When an arbitrary numeric predicate is addedtorstorder logic and

an arbitrary polynomial length advice string is given to the CRAM

and to the circuits

Let E be a numeric predicate that codes the unary version of an EXP

TIME complete problem Add E to the rstorder languages FO and

IND add a table for E to the CRAM and change DLOGTIME

uniform to polynomialtime uniform

One feature of uniformity that we nd amazing is that very lowlevel

uniformity seems to suce Similarly natural complete problems tend to

remain complete via very lowlevel reductions such as fops Whythisistrue

is not completely clear In part the answer is the existence of universal

Turing machines and thus universal complete problems However we feel

that there is more to it than that Here is a typical example

Observation The fol lowing classes areallequal

O

FOn

DLOGTIME uniform polynomialsize circuits

LOGSPACE uniform polynomialsize circuits

polynomialtime uniform polynomialsize circuits

P

Time Hardware and Uniformity

Variables that are Shorter than log n Bits

One would exp ect circuits of constant width rather than depth to b e very

weak but the following twointeresting characterizations tell us that this

is not so

Theorem BBIS NC DLOGTIME or NC uniform is ex

actly the set of languages recognized by DLOGTIME uniform

O

families of O width and n depth

Theorem CF PSPACE is exactly the set of languages recognized

O

n

by NC uniform boolean circuit families of O width and depth

Pro of sketch It is clear that such circuits can b e simulated in PSPACE

It remains to use these circuits to simulate a PSPACE Turing machine for

which it suces to solve the reachability problem on the exp onentialsize

conguration graph By the standard Savitch construction we get a circuit

of p olynomial depth and exp onential size and by standard tricks wecan

turn this into a b o olean formula with fanin twowhichisvery uniform

In particular we can take the p olynomiallength gate numb er and in

FO recover the twoTuring machine congurations whichgave risetothe

gate

We then apply the construction of B to get an exp onentiallength

branching program of constant width easily convertible into the desired

circuit of constant width Essentially as explained in BIS a gate number

in the constantwidth circuit enco des b oth a leaf no de of the p olydepth

circuit and hence a pair of congurations of the original Turing machine

and an indication of which element of the group S is to b e computed

bythislevel Determining the latter means passing over the entire gate

numb er or from the ro ot to the leaf of the p olydepth circuit p erforming

an op eration in S at every step whichisanNC complete problem

Our goal in this section is to expand the descriptive complexity frame

work to encompass results such as Theorems and That is wewant

to extend the previous notions in this pap er to the situation where wehave

fewer than one log nbit variable or equivalently circuit width less than

p olynomial

By analogy with Section wewould like to dene a class FOtn

VARv n for v n o whichwould b e equivalent to uniform circuits

O v nlogn

of width and depth tn The diculty in doing so is that

the basic rstorder variables of our formalism havelogn bits so that

quantifying over one of them would app ear to exceed the width b ound

Wehave to prohibit explicit use of suchvariables while retaining them in

order to talk ab out the others

It is absolutely necessary that our formulas like the b ottleneckma

chines of CF have access to a readonly clo ck Thus we will allowthe

David Mix Barrington Neil Immerman

formulas within the quantier blo cktohave access to a variable twhich

will indicate which iteration of the quantier blo ckwe are currently in As

O

n

t will always b e only p olynomially many bits so we long as tn

may access its individual bits by using the BIT predicate and a vector of

ordinary variables But we need to restrict ordinary quantiers to maintain

the width b ound Consider the following

Denition Fewer than one variable For v n o and tn

O

n

anFOtn VARv n formula will have three sorts of variables

ordinary ones x y z ranging over the universe f ng limited ones

a b c of v nlogn bits each and a variable t whose value is an integer

th

equal to in the base formula M and equal to j in the j quantier blo ck

to the left of M Both ordinary and extended variables may b e quantied

while t is syntactically like a constant Only ordinary variables may b e used

to access the input but all may b e used in the BIT predicate Finallythe

quantier blo ck B may not have an ordinary variable x that o ccurs freely

in B

The purp ose of the nal restriction is to force the circuits obtained from

these formulas as in the pro of of Theorem to p erio dically have levels

O v n log n

with only gates Wewould prefer to have a denition where

al l the levels were so b ounded but this denition pro duces a circuit class

which is equivalent allowing us to prove

O

Theorem FOn VAR log n DLOGTIME uniform NC

Pro of

For every t tn b ecause of the sp ecial condition weknowthat

t r

B M has free variables totaling r O bits Dene f ttobethe bit

string dened by the truth value of this formula for all p ossible values of the

r bits The string f t is a rstorder function of f t t and the input

predicates Since FO NC this means that eachbitoff t is computable

from these values via a b ounded width p olynomialsize circuit Wecanjust

string all of these tn b oundedwidth circuits together one after the other

to get the b oundedwidth p olynomialsize circuit for the original problem

By Theorem weknow that there are DLOGTIMEuniform

constantwidth p olynomialdepth circuit families for any language in NC

r

Let b e the width and tn b e the depth of the circuits for the problem in

r

question The values of the gates at level t of the circuit are determined

in a DLOGTIMEdenable wayfromt and the values at level t and

from the input Recall that DLOGTIME is contained in FOBISThus

we can write this relationship in a rstorder formula b b G

r t

r

Here G b b is a relation that co des the state of the gates at

t

r

time t Using a standard syntactic trick Corollary in I we

Time Hardware and Uniformity

can form a quantier blo ck B so that

G b b B G b b

t r t r

Here B mayquantify some ordinary variables in order to lo ok at the bits

of the input that t tells it to but no such ordinary variables will b e left

free Thus B is the quantier blo ckthatwe are lo oking for and wehave

that

t

G b b B G

t r

as desired

Similarlyweshow

O

n

Theorem FO VAR log n PSPACE

Pro of

O

n

This is immediate from previous results b ecause FO VAR log n

O

n

FO PSPACE

We pro ceed much as in the similar case ab ove using the result of

th

Theorem except that our nite function simulating the eect of the t

level of gates is nowNC computable rather than DLOGTIME computable

from t and the input However we can now use Theorem ab ovetosim

O

ulate the necessary NC predicate via a FOn VAR log nformula

Wenow just iterate this formula exp onentially many times getting the

desired result

Conclusions

Wehavepresented a very general threedimensional view of complexity

The dimensions are parallel time amount of hardware and amountof

precomputation and corresp ond closely to quantierdepth number of vari

ables and complexityofnumeric predicates resp ectively The tradeos b e

tween quantierdepth number of variables and the complexityofnumeric

predicates are to say the least worthyofmuch future investigation

Acknow ledgments Both authors gratefully acknowledge the supp ort of the

NSF Computer and Computation Theory program through grants CCR

Barrington and CCR CCR Immerman Thanks

also to ChiJen Lu Ken Regan and ZhiLi Zhang for valuable discussions

David Mix Barrington Neil Immerman

References

A E Allender Puniform circuit complexity JACM

AG E Allender and V Gore A uniform circuit lower b ound for the

p ermanent SIAM J Comput

B D Barrington Boundedwidth p olynomialsize branching pro

grams recognize exactly those languages in NC J Comp Syst

Sci

B DAM Barrington Extensions of an idea of McNaughton Math

Systems Theory

B DAM Barrington Quasip olynomial size circuit classes th

Structure in Complexity Theory Conference

BIS D Barrington N Immerman H Straubing On uniformity within

NC JCSS

BCH P Beame S Co ok HJ Ho over Log depth circuits for division

and related problems SIAM J Comput

BT R Beigel and J Tarui On ACC Computational Complexity

BFS J Boyar G S Frandsen and C Sturtivant An arithmetic mo del of

computation equivalent to threshold circuits Theoretical Computer

Science

CF JinYi Cai and MerrickFurst PSPACE survives threebit b ottle

necks International Journal on Foundations of Computer Science

CS JinYi Cai and D Sivakumar The resolution of a Hartmanis conjec

ture th Annual Symposium on Foundations of Computer Science

FVB G S Frandsen M Valence and D A M Barrington Some results

on uniform arithmetic circuit complexity Math Systems Theory

GKR F Green J Kobler K Regan T SchwentickandJToran The

power of the middle bit of a P function J Comp Syst Scito

app ear

H Hong JiaWei Computation Computability Similarity and Duality

New York John Wiley Sons

Time Hardware and Uniformity

I N Immerman Relational Queries Computable in Polynomial

Time Information and Control

I N Immerman Languages that capture complexity classes SIAM

J Comput

I Neil Immerman Nondeterministic Space is Closed Under Comple

mentation SIAM J Comput

I N Immerman Descriptive and computational complexity Com

putational Complexity Theory ed J Hartmanis Proc Symp in Ap

plied Math American Mathematical So ciety

Ib N Immerman Expressibility and parallel complexity SIAM J

Comput

k

I N Immerman DSPACEn VARk th StructureinCom

plexity Theory Conf

IL N Immerman S Landau The complexity of iterated multiplica

tion Information and Computation

KL R M Karp and R J Lipton Turing machines that take advice

Enseign Math

KV Phokion Kolaitis and JoukoVaananen Generalized Quantiers and

Pebble Games on Finite structures Annals of Pure and Applied

Logic

L S Lindell A purely logical characterization of circuit uniformity

th Structure in Complexity Theory Conf

MP JA Makowsky and YB Pnueli Computable quantiers and logics

over nite structures to app ear in Quantiers Generalizations Ex

tensions and Variants of Elementary Logic Kluwer Academic Pub

lishers

O Mitsunori Ogihara Sparse Phard sets yield spaceecientalgo

rithms th Annual Symposium on Foundations of Computer Sci

ence

Re J Reif On threshold circuits and p olynomial computation nd

Structure in Complexity Theory Conf

Ru L Ruzzo On uniform circuit complexity J Comp Sys Sci

SV Larry Sto ckmeyer and Uzi Vishkin Simulation of Parallel Random

Access Machines by Circuits SIAM J of Comp

David Mix Barrington Neil Immerman

V LG Valiant Reducibilityby algebraic pro jections LEnseigne

ment mathematique

vM D van Melkeb eek Reducing P to a sparse set using a constantnum

b er of queries collapses P to L Eleventh Annual IEEE Conference

on Computational Complexity formerly known as STRUCTURES

to app ear

Y A C C Yao On ACC and threshold circuits Proc st Ann

IEEE Symp Foundations of Computer Science