k

DSPACEn   VARk 

Neil Immerman

Computer Science Dept

University of Massachusetts

Amherst MA

immermancsumassedu

draft of a revised version of I

Section proves our main result Section includes Abstract

p otential applications of this result and suggests some

In this pap er weprove that the set of prop erties check

attacks on the dicult questions in complexity theory

k

able byaTuring machine in DSPACEn is exactly

from the descriptive p oint of view

equal to the set of prop erties describable by a uniform

sequence of rstorder sentences using at most k

Background and Denitions

distinct variables We provethatthisisalsoequalto

the set of prop erties describable using an iterative def

FirstOrder Logic

inition for a nite set of relations of arity k Thisisa

nement of the theorem PSPACE VARO I

a a

k

A vocabulary hR c c i is a tuple of

r

k

We suggest some directions for exploiting this result to

a

i

relation symb ols and constantsymbols R is a re

i

derive tradeos b etween the number of variables and

lation symb ol of arity a In the sequel we will of

i

the quantierdepth in desciptive complexity This has

ten omit the sup erscripts to improve readability A

applications to parallel complexity

nite structure with vo cabulary is a tuple A

A A A A

hf n gR R c c i consisting of a uni

r

k

A A A

verse U f n g and relations R R of

k

Intro duction

A

arities a a on U corresp onding to the relation

k

a

a

A A A

k

symbols R R of and constants c c from U

r

k

In Descriptive Complexity one analyzes the complexity

corresp onding to the constantsymb ols c c from

r

of a language in terms of the complexity of describing

We write jAj to denote n the cardinalityoftheuni

the language It is known that the quantierdepth and

verse of A Let STRUC denote the set of nite

number of variables needed to express the memb ership

structures of vo cabulary

prop erty of a language is closely related to the parallel

For example if consists of a single binary rela

g

time and amountofhardware needed to check whether

tion symbol E standing for edge then a structure

an input is in the language For a long time the basic

G hfn gEi with vo cabulary is a graph on

g

question of complexity namely what are the trade

nvertices Similarly if consists of a single unary re

s

os b etween time and hardware has remained quite

lation symbol M then a structure S hfn gMi

op en Wehave b een attempting to understand this

with vo cabulary is a binary string of length n

s

question in terms of the tradeo b etween number of

Let the symbol denote the usual ordering on the

variables and quantierdepth In this pap er wetighten

natural numbers We will include as a logical rela

the known relationship b etween number of variables

tion in our rstorder languages This seems necessary

and deterministic space

in order to simulate machines whose inputs are struc

This pap er is organized as follows Section gives

tures given in some order For convenience wealso

the relevantbackground denitions and references

include the constantsymbols and m referring to the

rst and last elements of the structure resp ectivelyand Research supp orted in part by NSF grant CCR

R x y x y z E x z Rz y the logical relation sx y true when y is the immedi

The formula formalizes an inductive denition of ate successor of x in the ordering For technical

E whichmay b e more suggestively written as follows reasons we also include the logical relation BIT where

BITx y holds i the xth bit in the binary expansion

E x y x y z E x z E z y

of y is a one

For any structure A with vo cabulary induces

g

Wenow dene the rstorder language tobe

a map from binary relations on the universe of A to

the set of formulas built from the relation and con

binary relations on the universe of A

stantsymbols of and the logical relation symb ols and

constant symbols sBIT m using logical con Aj R a b R ha bi

A

nectives variables x y z and quantiers

Such a map is called monotonic if for all R S

A sentence is a formula with no free variables

Every sentence L is either true or false in any

R S R S

A A

structure A STRUC We write Aj to mean

that A satises Let MOD denote the set of all

Note that since R app ears only p ositively in in Equa

r

mo dels of

tion is monotonic Let denote iterated r

A A

A

times With dened as in Equation A any graph

MOD A STRUC Aj

and r observe that

Here either is previously sp ecied or it is the vocab

r

ha bijAj distancea b r

A

ulary of all nonlogical symb ols o ccurring in

We will think of a problem as a set of structures of n

Thus in particular if n jAjthen E

A

some vo cabulary It suces to only consider prob

the least xed p ointof ie the minimal relation T

A

lems on binary strings but it is more interesting to b e

such that T T This is a general situation as

A

able to talk ab out other vo cabularies eg graph prob

wenowshow in the nite version of the KnasterTarski

lems as well For deniteness we will x a scheme

Theorem

for co ding an input structure as a binary string If

A A A A

i is a structure c c R A hf n gR

Theorem CH M Let R be a new re

r

k

of vo cabulary thenA will b e enco ded as a binary

lation symbol of arity k andlet R x x be

k

a a

k

string binAoflengthI nn n r dlog ne

an Rpositive rstorder formula ie R occurs only

consisting of one bit for each a tuple p otentially in the

within an even number of negation signs Then for

i

relation R anddlog ne bits to name each constant c

any nite structure Atheleast xedpoint of ex

i j

A

r

Thus we reserve n to indicate the size of the universe

ists and is equal to where r is minimal so that

A

r

r

of the input structure I n the length of binA is

A

A

p olynomially related to n and in the case where con

sists of a single unary relation ie inputs are binary

Pro of Since is Rp ositive is monotonic Thus

A

k

strings I nn

suchanr exists and is less than or equal to n where

Dene the FO to b e the set of all

n jAj This is true b ecause each application of

A

rstorder expressible problems FO is a uniform ver

st

b efore the r adds some new k tuple to the relation

sion of the circuit class AC BIS and it is equal to

k

and there are only n p ossible k tuples

the set of problems acceptable in constanttimeona

r

is a xed p ointof Let T be any Thus

A

A

p olynomial size concurrent parallel random access ma

i

other xed p oint and let R for i r It

i

A

chine I

is easy to see by induction on i that R T Of course

i

R T Assuming R T the monotonicityof

i

Inductive Denitions

implies that R T ie R T Thus

A A i A i

r

is the least xed p oint as claimed

A

A useful way to increase the p ower of rstorder logic

k

without jumping all the way up to second order logic Theorem tells us that any R p ositiveformula

k

is to add the p ower to dene new relations by induc R x x determines a k ary least xed p oint

k

tion For example consider the vo cabulary hE i of relation We will write LFP k to denote this

g

R x x

k

graphs We can dene the reexive transitive closure least xed p oint The least xed p oint op erator LFP

E of E as follows Let R b e a binary relation variable thus formalizes the denition of new relations byin

and consider the formula duction

Pro of This is a straightforward induction on the Denition Dene FO LFP the language

complexityof Wewillshow that the lemma holds of rstorder inductive denitions by adding a least

with M in the following restricted form xed p oint op erator LFP to rstorder logic If

s

k k

R x x isanR p ositiveformula in FO

k

M x z x z x z

LFP then LFP k is a formula in FO

s i i k i

 k R x x

k

LFP denoting the least xed p ointof

The only interesting case is the inductivestepwhen

and

Immerman and Vardi indep endently characterized

the complexityofFO LFP as follows

Q y N Q y N

t t t

Theorem I V FO LFP P

x x N Rx x

k t k

The numb er of iterations until an inductive denition

Q z M Q z M

s s s

closes is called its depth Inductive depth turns out to

x x M Rx x

k s k

b e linearly related to parallel time cf Theorem

k

where wemay assume that the y s and z s are disjoint

Denition Let R x x beanRp ositive

k

Let

formula where R is a relation symbol of arity k and

let A b e a structure of size n Dene the depth of in

QB Q y N Q y N

t t t

A in symbols j j to b e the minimum r suchthat

A

QB Q z M Q z M

s s s

r r

Aj

Let ux denote the formula with variables

k

As wesaw in the pro of of Theorem j jn

A

u u substituted for x x and dene the

k k

Dene the depth of as a function of n equal to the

quantierfree formulas

maximum depth of in A for any structure A of size

n

S b N ux b M ux

t s

jjn maxfj jjn jAjg

A

T u x u x

k k

In this case wehave that

Denition Let INDf n b e the sublanguage of

bb QB QB uS xT Rx x

FO LFP in whichwe only include least xed p oints

k

of rstorder formulas for which jj is O f n

Note that the ab ove requantication of the x s

Iterating FirstOrder Formulas

i

means that these variables may o ccur freely in

Theorem shows that the least xed p oint op erator

M M but they are b ound in M and

s s

amounts to a p olynomial iteration op erator This is

Rx x Note that the same variables maynow

k

even more apparent when we put the inductive deni

b e requantied Let us write QB to denote the quan

tions into the following simple normal form Recall the

tier blo ckQ z M Q z M x x M

s s s k s

notation xM meaning xM andxM

Thus in particular for any structure Aandany r N

meaning xM

r r

Aj QB false

A

Lemma M I Let be an Rpositive

rstorder formula Then can be written in the fol

r

Here QB means QB rep eated r times literally

lowing form

It follows immediately that if t jjn and A is any

R x x

k

structure of size n then

Q z M Q z M x x M Rx x

s s s k s k

t

Aj LFP QB false

where the M s are quantierfree formulas in which R

i

does not occur

Wenow dene FOtn to b e the set of prop erties

In the logical literature where structures are usually innite

dened byquantier blo cks iterated tn times this is called the closureordinal cf M

Example IWeshowhow to transfer a log n Denition AsetC STRUC is a member

depth inductive denition of the transitiveclosureofa of FOtn i there exist quantier free formulas M

i

graph to an equivalentFO log n denition i k fromL a tuple of constantsc anda

quantier blo ck

Let E b e the edge predicate for a graph G with n

vertices We can inductively dene E the reexive

QB Q x M Q x M

k k k

transitive closure of Gasfollows

tn

suchthatifwe let QB M c for n

n

E x y x y E x y z E x z E z y

then for all ASTRUC with jAj n

Let P x y mean that there is a path of length at

n

A C Aj

n

most n from x to y Then we can rewrite the ab ove

denition of E as

P x y x y E x y z P x z P z y

n

n n

In the pro of of Lemma weintro duced quantied

This can b e rewritten

b o olean variables into the quantier blo ck to replace

logical ands and ors In Theorem wewillbe

P x y zM z P x z P z y

n

n n

carefully counting the numb er of domain variables The

following lemma shows that the numb er of domain vari

where M x y E x y Note that there is no

ables need not b e increased to take care of conjunctions

free o ccurrence of the variable z after the z quantier

and disjunctions

Thus in this case zM is equivalenttoM

Next

Lemma Suppose that we have two quantier

blocks with identical quantiers in identical order ig

P x y zM z uv M P u v

n

n

noring any boolean quantiers

where M u x v z u z v y Now

QB Q v M Q v M

s s s

QB Q v N Q v N

s s s

P x y zM z uv M xy M P x y

n

n

Then the conjunction and disjunction of these quanti

where M x u y v Thus

er blocks may be written in the same form

dlog ne

P x y QB P x y

n

Pro of The conjunction for example can b e written

where QB zM z uv M xy M Note

with an extra unversally quantied b o olean variable

that

QB QB bQ v R Q v R

s s x

P x y QB false

It follows that where

dlog ne

R b M b N

P x y QB false

i i i

n

and thus E FOlog n as claimed

Inductive depth and rstorder iterations are inti

mately connected with parallel complexity Dene

Unb ounded Iterations

CRAMtn to b e the set of problems accepted by

concurrent parallel random access machines in parallel

As wesaw in Theorem inductive denitions must

time O tn using p olynomially much hardware Then

close in at most p olynomially many steps b ecause of

their monotonicityNowwe generalize inductive de

Theorem I For al l polynomial ly bounded

nitions to iterative denitions in which the requirement

and constructible tn

of monotonicity is removed



CRAMtn INDtn FOtn

This is equivalent to the addition of a while op erator V

Namely the contents of cell p at time t is a function Denition Dene ITERtn arity k tobethe

of the contents of cells p pp at time t set of prop erties denable by iterating tn times the

simultaneous rstorder denitions of a set of c relations

We will write a logical formula C x b meaning that

t

of arity k for some constant c Of course after tn

after step t of M s computation the cell at p ositionx is

k

cn

b Herex x x is a k tuple of variables ranging

k

iterations these relations will either reach a xed

over the set f n g and b is a tuple of b o olean

point or b e in a cycle Thus dene

variables co ding an elementof

k

The following is an iterative denition of C

t

cn

ITER arity k ITERarity k

C x b

t

c

W

C x a C x a C x a

t t t

ha a a ib



Similarywe dene arbitrary iterations of quantier

Here the disjunction is over the nite set of quadru

blo cks

ples a a a b such that the rst three symbols

Denition Dene FOtn VARk tobe

lead to the fourth symbol in one moveof M Notethat

the set of prop erties denable in the form

this set of quadruples is exactly a representation of M s

state table

tn

QB M

It is trivial to write C with k domain variables

Furthermore M accepts its input i it eventually

for some quantier blo ck QB containing domain vari

reaches its accept state Let co de the appropriate

ables from the set fx x g and some b o olean

k

accept symbol Thus M accepts its input i eventually

variables b b The formula M contains only b o ol

c

C holds

t

ean variables As in the denition of ITERarity k the

The lemma will b e proved once weshow the follow

truth assignmentofallthevariables will cycle or sta

k

ing

cn

iterations Thus dene balize after at most tn

Claim There is a quantier block QB containing

k

cn

k domain variables such that Equation may be

VARk FO VARk

rewritten as

c

C x b QB C x b

t t

The pro of of Claim is purely symb ol manipula

The Pro ofs

tion We rst write quantier blo cks QB and QB

whose job it is to replacex byx andx resp ec

In this section weprove our main theorem

tively ie for anyformula wehave

Theorem For k

x QB x

k

x QB x

DSPACEn VARk ITERarity k

The pro of is accomplished via three lemmas proving

the following containments

These quantier blo cks can b e written with k do

main variables The idea is to add one tox by replacing

k k

DSPACEn VARk ITERarity k DSPACEn

x with its successor or if x mby replacing x by

k k k

andx by its successor or etc Just to give a taste

k

k

Lemma DSPACEn VARk

of this kind of construction here is QB for the case

k For convenience the pair of b o olean variables

k

Pro of Let M be a DSPACEn Turing machine M s



k

For details on the co ding of inputs see for example I

work space consists of n tap e cells eachofwhich holds

Note that we are assuming that the whole input is on the work

a symb ol from some nite alphab et

tap e at time Thus for example if we are considering a graph



The contents of M s tap e at time t is a deter

problem with an n bit adjacencty matrix then wemust have

k  ministic lo cal transformation of the contents at time t

k

c is thoughtofasaninteger in the range to The This last inclusion is obvious b ecause O n bitssuf

quantier blo ck b egins by guessing the variable number ce to record the current meaning of the b ounded num

or that will b e incremented The symbol s b er of relations of arity k Each bit of each relation in

denotes the successor relation the next iteration may then b e computed byevaluat

h

ing a xed rst order formula This can b e done in

k

For k QB cc f g

DSPACElog nandthus certainly in DSPACEn

yc sx y c x m y

x x y

Conclusions and Directions

yc y x c sx y

c x m y x x y

i

The fundamental challenge in computational complex

yc y x c sx yx x y

ity theory is to understand the tradeo b etween parallel

time and hardware We had previously shown an exact

Thus wehaveQB QB and triviallyQBOb

relationship b etween quantierdepth and parallel time

serve that the desired QB of Claim is a p ositive

on a CRAM Theorem Nowwehaveshown an

b o olean combination of these three quantier blo cks

exact relationship b etween number of variables and de

It follows from Lemma that QB exists and has

terministic space Further work is needed to determine

k domain variables as desired This completes the

a meaningful nearly exact relationship b etween simul

pro of of the Claim and thus of Lemma

taneous descriptive measures and simultaneous parallel

time and hardware This is a tricky problem b ecause it

dep ends on the interconnection patterns and the con

Lemma VARk ITERarity k

ventions for concurrent reads and writes See I for

further discussion

Pro of Here wehave a quantier blo ck of the form

One instance of the ab ove tradeo problem seems

particularly worthyofstudy Consider the following

QB Q x M B Q x M B

i r i r r

r

very dierentcharacterizations of PSPACE

where each i f k g theM are quantier

j i

Theorem I

free and the B are blo cks of b o olean quantiers over

i

the b o olean variables fb b gWe can convert the

c

O 

n

PSPACE FO SOn

iteration of QB into an iterative denition of relations

as follows Let R b e a set of k ary relation symbols

st

Thus the descriptivepower of a b ounded number of

fcg

for s r andt f g Thus t sp ecies

rstorder variables ie O log n bits and exp onential

an assignment to all the b o olean variables Intuitively

quantier depth is equal to the descriptivepower of a

the iterative denition for the R s is given as follows

st O

b ounded numb er of secondorder variables ie n

bits and p olynomial quantier depth Wewould like

R x xc x

st i k

s

to know if there is anything in b etween For example

what quantier depth is necessary and sucientwhen



Q x M B R x xd x

s i s s st i k

s s

log n rstorder variables are available

Here one is added to s mo dulo r Note that the no

Tomake this problem more concrete consider the

tation Rx xb x means that the variable

i k

following problem Dene a k local graph to b e a graph

x is omited In the ab ove formula the variable to b e

n

i

on vertex set f g such that for eachvertex u there is

quantied next is safely omitted This is the reason

th

auniquenextvertex v and the i bit of v is determined

that arity k suces

by bits i k i k i k of u Note that a k lo cal

The ab oveformula is a bit misleading in that wemust

graph can b e presented as a table of size O n

write out the b o olean quantier blo ck B For example

s

Dene the lo cal graph accessibility problem LGAP



the formula b R x would b e expanded to

j st

to b e the set of lo cal graphs such that there is a path in

the graph from the vertex with all zeros to the vertex

R x R x

stjb stjb

j j

with all ones

The following prop ostion is clear

Prop osition LGAP is complete for DSPACEn

k

Lemma ITERarity k DSPACEn via simultaneously logspace and linear time reductions

I N Immerman Upp er and Lower Bounds for Furthermore as in Theorem the LGAP prob

n

First Order Expressibility JCSS No lem can b e expressed in FO VAR and also as

a secondorder sentence with three unary relation vari

ables ie n bits and quantier depth O n

I N Immerman Relational Queries Com

Nowwehopetochallenge manypeopletowork hard

putable in Polynomial Time Information and

on the following

Control A preliminary ver

sion of this pap er app eared in th ACM

Problem What is the tradeo between number of

STOC Symp

variables and quantierdepth for describing LGAP

I N Immerman Languages That Capture

Complexity Classes SIAM J Comput

The main to ol currently available for studying Prob

No

lem is the sort of communication complexity game

I N Immerman Expressibility and Paral lel

intro duced in KW A similar game called the

Complexity SIAM J of Comput

separability game is describ ed in I These games

as stated only consider circuit depth and quantier

k

I N Immerman DSPACEn VARk

depth resp ectivelyHowever one can add a notion of

Sixth IEEE Structure in Complexity Theory

number of variable bits k by forcing the players to

Symp July

have only k bits of active memory b etween rounds or

k

KW M Karchmer and A Wigderson Mono

equivalentlytohave at most dierent piles to split

tone circuits for connectivity require sup er

all the dierent structures into

logarithmic depth SIAM J Discrete Math

Finallyitmay b e p ossible to learn more ab out the

relationship b etween DSPACEn and NSPACEnby

using our main theorem

M Y Moschovakis Elementary Induction on Ab

stract Structures North Holland

Acknowledgments Thanks to SushantPatnaik

V M Vardi Complexity of Relational Query

for asking me the question whose answer is the main

Languages th ACM Symposium on Theory

theorem Thanks to Susan Landau for some crucial

of Computation

suggestions concerning this pap er Thanks to Jose An

tonio MedinaPeralta for a careful reading of a previous

version of this pap er

References

BIS D Mix Barrington N Immerman H Straub

ing On Uniformity Within NC JCSS

No

CFI J Cai M Furer N Immerman An Opti

mal Lower Bound on the Number of Variables

for Graph Identication th IEEE FOCS

Symp

CH A Chandra and D Harel Structure and

Complexity of Relational Queries JCSS

F R Fagin FiniteMo del Theory a Personal

Persp ective Third Intl Conf Database The

ory

G Y Gurevich Logic and the Challenge of

Computer Science in Current Trends in The

oretical Computer Science ed Egon Borger

Computer Science Press

I N Immerman Number of Quantiers is Bet

terthanNumber of Tap e Cells JCSS

No June