Solving Sparse, Symmetric, Diagonally-Dominant Linear Systems in Time

¢¡¤£¦¥¨§ © ¥ Daniel A. Spielman Shang-Hua Teng Department of Mathematics Department of Computer Science Massachusetts Institute of Technology Boston University and

Akamai Technologies Inc. Abstract quires ST)U7V- space. In contrast, if is symmetric and has

non-zero entries, then one can use the Conjugate Gradi- )N8-

ent method, as a direct method, to solve for OJN' in We present a linear-system solver that, given an -by-

time and )U- space! Until Vaidya’s revolutionary intro- symmetric positive semi-deﬁnite, diagonally dominant ma-

duction of combinatorial preconditioners [24], this was the

trix with non-zero entries and an -vector , pro-

best complexity bound for the solution of general PSDDD duces a vector within relative distance of the solution

systems.

!#"%$'&()+*,!-/.!0213546- . to in time , where is the

log of the ratio of the largest to smallest non-zero entry of The two most popular families of methods for solving lin-

87 9 :<;>= . If the graph of has genus or does not have a ear systems are the direct methods and the iterative meth-

minor, then the exponent of can be improved to the min- ods. Direct methods, such as Gaussian elimination, perform

¥>?A@CB ¥I?B -

imum of and EDF*CG¨-H . The key contribution of arithmetic operations that produce treating the entries of

our work is an extension of Vaidya’s techniques for con- and symbolically. As discussed in Section 1.4, direct structing and analyzing combinatorial preconditioners. methods can be used to quickly compute if the matrix has special topological structure. Iterative methods, which are discussed in Section 1.5, compute successively better approximations to . The Cheby-

1. Introduction shev and Conjugate Gradient methods take time propor- Y[Z \-]"^$¨&[_Y[Z(`-/*C -

tional to XW to produce approxima-

Y#Z[`- Sparse linear systems are ubiquitous in scientiﬁc comput- tions to with relative error , where is the ratio of ing and optimization. In this work, we develop fast algo- the largest to the smallest non-zero eigenvalue of . These

rithms for solving some of the best-behaved linear systems: algorithms are improved by preconditioning—essentially

abJL' a

those speciﬁed by symmetric, diagonally dominant matri- solving abJL! for a preconditioner that is

Y EKc/ad-

ces with positive diagonals. We call such matrices PSDDD carefully chosen so that Z is small and so that it is a as they are positive semi-deﬁnite and diagonally dominant. easy to solve linear systems in a . These systems in may Such systems arise in the solution of certain elliptic differ- be solved using direct methods, or by again applying itera- ential equations via the ﬁnite element method, the model- tive methods. ing of resistive networks, and in the solution of certain net- Vaidya [24] discovered that for PSDDD matrices one

work optimization problems [23, 19, 15, 25, 26]. could use combinatorial techniques to construct matrices

While one is often taught to solve a linear system that provably satisfy both criteria. In his seminal work,

MJN

by computing KJL and then multiplying by , this ap- Vaidya shows that when corresponds to a subgraph of

Y Mc/aO- proach is quite inefﬁcient for sparse linear systems—the the graph of , one can bound Z by bounding

best known bound on the time required to compute OJL the dilation and congestion of the best embedding of the

7¤ /PRQ JN

- is ) [9] and the representation of typically re- graph of into the graph of . By using preconditioners derived by adding a few edges to maximum spanning

Partially supported by NSF grant CCR-0112487. spiel- trees, Vaidya’s algorithm ﬁnds -approximate solutions to

[email protected] PSDDD linear systems of maximum valence e in time Partially supported by NSF grant CCR-9972532. [email protected]

P¡ L"%$'&[_Y[Z[`-/*C -/- 5e¨U-/! . When these systems have spe- result to general planar PSDDD linear systems. cial structure, such as having a sparsity graph of bounded Due to space limitations, some proofs of have been omitted, genus or avoiding certain minors, he obtains even faster al- but will appear in the on-line and full versions of the paper.

gorithms. For example, his algorithm solves planar linear

/e¨U- "^$¨&(_Y `-/*C -/- systems in time Z . This paper follows the outline established by Vaidya: our contributions

1.1. Background and Notation

Y Mc/ad- are improvements in the techniques for bounding Z , a construction of better preconditioners, a construction that

depends upon average degree rather than maximum degree,

,.-+/,10 A symmetric matrix is semi-positive deﬁnite if and an analysis of the recursive application of our algo- 2 for all vectors , . This is equivalent to having all eigenval- rithm. ues of non-negative. 2 As Vaidya’s paper was never published , and his manuscript In most of the paper, we will focus on Laplacian matri-

lacked many proofs, the task of formally working out his re- ces: symmetric matrices with non-negative diagonals and

sults fell to others. Much of its content appears in the the- 5

465U/798 non-positive off-diagonals such that for all 3 , . sis of his student, Anil Joshi [16]. Gremban, Miller and However, our results will apply to the more general fam- Zagha[13, 14] explain parts of Vaidya’s paper as well as ex- ily of positive semideﬁnite, diagonally dominant (PSDDD)

tend Vaidya’s techniques. Among other results, they found

;7<8 7=:>0

matrices, where a matrix is diagonally dominant if :

ways of constructing preconditioners by adding vertices to 5

5? : @7<8 : for all 3 . We remark that a symmetric matrix is the graphs and using separator trees. PSDDD if and only if it is diagonally dominant and all of Much of the theory behind the application of Vaidya’s tech- its diagonals are non-negative.

niques to matrices with non-positive off-diagonals is devel- In this paper, we will restrict our attention to the solution

oped in [3]. The machinery needed to apply Vaidya’s tech-

of linear systems of the form where is a PS-

niques directly to matrices with positive off-diagonal ele- OJN DDD matrix. When is non-singular, that is when

ments is developed in [5]. The present work builds upon OJN' exists, there exists a unique solution , to the lin- an algebraic extension of the tools used to prove bounds

ear system. When is singular and symmetric, for every

on Y(Z[Mc/aO- by Boman and Hendrickson [6]. Boman and

BA \- A \-

Span there exists a unique Span such

Hendrickson [7] have pointed out that by applying one of

that . If is the Laplacian of a connected graph,

their bounds on support to the tree constructed by Alon,

¢ then the null space of is spanned by . Karp, Peleg, and West [1] for the -server problem, one ob-

There are two natural ways to formulate the problem of ﬁnd-

a Y EKcRaO-

tains a spanning tree preconditioner with Z

ing an approximate solution to a system . A vector

§ ¨¡© § ¨¡© § ¨¡© '- ¦¥

¤£

. They thereby obtain a solver for PS-

D¤ GDIH #D'JD

has relative residual error if . We

DDD systems that produces -approximate solutions in time

say that a solution is an -approximate solution if it is at

!1354¨"^$¨&[EY(Z[\-R*,!- 8! . In their manuscript, they asked

relative distance at most from the actual solution—that is,

whether one could possibly augment this tree to obtain a

DIH KD D if D . One can relate these two notions of

better preconditioner. We answer this question in the af-

approximation by observing that relative distance of to

R7L"%$'& )U-/- ﬁrmative. An algorithm running in time the solution and the relative residual error differ by a multi-

has also recently been obtained by Maggs, et. al. [18].

Y `-

plicative factor of at most Z . We will focus our atten-

! - The present paper is the ﬁrst to push past the ) bar- tion on the problem of ﬁnding -approximate solutions.

rier. It is interesting to observe that this is exactly the point

Y `- The ratio Z is the ﬁnite condition number of . The

at which one obtains sub-cubic time algorithms for solving L

!MD DH/,#DU*ND=,#D norm of a matrix, D , is the maximum of ,

dense PSDDD linear systems. 7

and equals the largest eigenvalue of if is symmetric.

;QPSR

`- DHMD

Reif [21] proved that by applying Vaidya’s techniques re- For non-symmetric matrices, O and are typi-

cursively, one can solve bounded-degree planar positive cally different. We let : denote the number of non-zero

UWVYX \- U[Z\LE\-

deﬁnite diagonally dominant linear systems to relative ac- entries in , and and denote the small-

) H1364 "%$'&(EY+`-/*,!-/- curacy in time . We extend this est and largest non-zero elements of in absolute value, respectively.

1 For the reader unaccustomed to condition numbers, we note that for The condition number plays a prominent role in the analy- an PSDDD matrix in which each entry is speciﬁed using bits of

sis of iterative linear system solvers. When is PSD, it is

precision, !#"¦"%$'&()%*+" .

Y \-]"^$¨&[ *,!- 2 Vaidya founded the company Computational Applications and Sys- known that, after Z iterations, the Cheby- tem Integration (http://www.casicorp.com) to market his linear sys- shev iterative method and the Conjugate Gradient method tem solvers.

produce solutions with relative residual error at most .

To obtain an -approximate solution, one need merely run from the observation in [3] that if , where sat-

Y#Z] McRa -(HAY(Z EKc/ad- "^$¨&[EY(Z[\-/- times as many iterations. If has non-zero isﬁes the above condition, then .

entries, each of these iterations takes time )8- . When ap- So, it sufﬁces to ﬁnd a preconditioner after subtracting off

plying the preconditioned versions of these algorithms to the maximal diagonal matrix that maintains positive diago-

JL JN

a solve systems of the form a , the number nal dominance.

of iterations required by these algorithms to produce an -

We then use an idea of Gremban [13] for handling posi-

Y EKcRaO- "%$'&(EY E\-R*,!- Z accurate solution is bounded by Z

tive off-diagonal entries. If is a symmetric matrix such

where

/798 7Q0 5 : @7<8 :

that for all 3 , , then Gremban decomposes

? ?

-Ua -U

into , where is the diagonal of ,

U[Z\ c Y[Z[Mc/ad- U[Z\

¢¤£ ¥¦¢¨§ ¢¤£ ¥¦¢¨§

? © ? ©

a -

- is the matrix containing all negative off-diagonal entires of

, and contains all the positive off-diagonals. Gremban

a _\- ad- for symmetric and with Span Span . How-

then considers the linear system -

ever, each iteration of these methods takes time plus

the time required to solve linear systems in a . In our initial

E E %$

algorithm, we will use direct methods to solve these sys-

#" " "

tems, and so will not have to worry about approximate so-

&$[ ET

lutions. For the recursive application of our algorithms, we and observes that its solution will have and that

will use our algorithm again to solve these systems, and so will be the solution to . Thus, by making this trans- *+ will have to determine how well we need to approximate the formation, we can convert any ')( linear system into solution. For this reason, we will analyze the Chebyshev it- one with non-negative off diagonals. One can understand eration instead of the Conjugate Gradient, as it is easier to this transformation as making two copies of every vertex analyze the impact of approximation in the Chebyshev iter- in the graph, and two copies of every edge. The edges cor- ations. However, we expect that similar results could be ob- responding to negative off-diagonals connect nodes in the tained for the preconditioned Conjugate Gradient. For more same copy of the graph, while the others cross copies. To

information on these methods, we refer the reader to [12] or capture the resulting family of graphs, we deﬁne a weighted

[8]. graph to be a Gremban cover if it has £ vertices and

? ?

c H <3 c -NA- <3 2c U-NA

for 3 , if and only if

? ?

1.2. Laplacians and Weighted Graphs

M93 c - O<3 2c U-

, and ,

? ?

c H <3 c U- A. 93 2c -NA

for 3 , if and only if

? ?

All weighted graphs in this paper have positive weights.

M93 c U- M<3 2c - , and , and

There is a natural isomorphism between weighted ? ,

<3 c3 U- the graph contains no edge of the form .

graphs and Laplacian matrices: given a weighted graph

cbc\-

, we can form the Laplacian matrix in

When necessary, we will explain how to modify our argu-

@798 M<3 cF- <3 cF- A

which for , and with diago-

ments to handle Laplacians that are Gremban covers. C nals determined by the condition . Conversely, a weighted graph is naturally associated to each Lapla- Finally, if is the Laplacian of an unconnected graph, then cian matrix. Each vertex of the graph corresponds to both a the blocks corresponding to the connected components may row and column of the matrix, and we will often abuse no- be solved independently. tation by identifying this row/column pair with the associated vertex.

1.4. Direct Methods

We note that if and are weighted graphs on the same 7

vertex set with disjoint sets of edges, then the Laplacian of The standard direct method for solving symmetric linear

the union of and is the sum of their Laplacians. 7 systems is Cholesky factorization. Those unfamiliar with Cholesky factorization should think of it as Gaussian elim- 1.3. Reductions ination in which one simultaneously eliminates on rows and columns so as to preserve symmetry. Given a permu-

tation matrix ' , Cholesky factorization produces a lower-

In most of this paper we just consider Laplacian matrices

/0/(- ' ' - triangular matrix / such that . Because one of connected graphs. This simpliﬁcation is enabled by two can use forward and back substitution to multiply vectors

reductions.

>JL / J - by / and in time proportional to the number of non-

First, we note that it sufﬁces to construct preconditioners for zero entries in / , one can use the Cholesky factorization of

@798 7 465 : @7<8 : 3 : / : - matrices satisfying , for all . This follows to solve the system in time . ©¨

Each pivot in the factorization comes from the diagonal of form

, and one should understand the permutation as provid-

a /§¦ / c ¦

where c

ing the order in which these pivots are chosen. Many heuris- "

tics exist for producing permutations ' for which the num-

/ is lower triangular, and the left column and right columns

ber of non-zeros in / is small. If the graph of is a tree, in the above representations correspond to the eliminated

then a permutation ' that orders the vertices of from the

/ : H £,

and remaining vertices respectively. Moreover, :

£,

leaves up will result in an / with at most non-zero , and this Cholesky factorization may be performed in time

entries. In this work, we will use results concerning matri- ?

: : - ) . ces whose sparsity graphs resemble trees with a few addi-

tional edges and whose graphs have small separators, which The following Lemma may be proved by induction. / we now review. Lemma 1.3. Let a be a Laplacian matrix and let and

be the matrices arising from the partial Cholesky fac-

a cbc\-

If is the Laplacian matrix of a weighted graph ,

torization of a according to the trim order. Let be the and one eliminates a vertex of degree , then the remain-

ing matrix has the form set of eliminated vertices, and let be the set of remain-

(cR.H-

ing vertices. For each pair of vertices in joined

by a simple path containing only vertices of , let 8

1 4

2 c

be the Laplacian of the graph containing just one edge be-

¥ ¥

* 7E- . * 7

tween and of weight 7 , where the are the

where is the Laplacian of the graph in which and its weights on the path between and . Then,

attached edge have been removed. Similarly, if a vertex

of degree £ is eliminated, then the remaining matrix is the

the matrix is the sum of the Laplacian of the induced

Laplacian of the graph in which the vertex and its ad- P

graph on and the sum all the Laplacians 8 ,

1 4

jacent edges have been removed, and an edge with weight

E.H- C / C C C

¥ ¥ ¥

? , where is the all-ones vector, and is

* * -

* is added between the two neighbors of

the characteristic vector of , and

, where and are the weights of the edges connect-

¤- /0/ ing to its neighbors. the matrix is the sum of the Laplacian of the

trim

¢¡¤£ graph containing all edges deleted by and the

Given a graph with edge set , where the edges ¥

¡ diagonal matrix that is for vertices in and else-

in form a tree, we will perform a partial Cholesky factor-

where, minus the sum of all the Laplacians 8 . 4

ization of in which we successively eliminate all the de- 1

U[Z)\NE -(H

gree 1 and 2 vertices that are not endpoint of edges in . We Corollary 1.4. For a and as in Lemma 1.3,

;UWZ)\#EaO- UWVYXUE -(0 UTV XUEaO-/*, introduce the algorithm trim to deﬁne the order in which and . the vertices should be eliminated, and we call the trim order the order in which trim deletes vertices. Other topological structures may be exploited to produce

elimination orderings that result in sparse / . In particu-

c c - Algorithm: trim lar, Lipton, Rose and Tarjan [17] prove that if the sparsity

graph is planar, then one can ﬁnd such an / with at most

U- U ¡R7 - 1. While contains a vertex of degree one that is not an )T"%$'& non-zero entries in time . In general, endpoint of an edge in , remove that vertex and its Lipton, Rose and Tarjan prove that if a graph can be dis-

adjacent edge. sected by a family of small separators, then / can be made

sparse. The precise deﬁnition and theorem follow.

2. While contains a vertex of degree two that is not an

endpoint of an edge in , remove that vertex and its Deﬁnition 1.5. A subset of vertices ¦ of a graph

c M- )U- : ¦[:.H U-

adjacent edges, and add an edge between its two neigh- with vertices is an -separator if ,

E ¦

bors. and the vertices of can be partitioned into two sets

and such that there are no edges from to , and

Proposition 1.1. The output of trim is a graph with at ©

: : c : :+H £,+*

@ .

¥@: : :

most vertices and : edges.

E-

¡ Deﬁnition 1.6. Let be a positive function. A graph

c - c -

Remark 1.2. If and are Gremban covers,

c M- E-

with vertices has a family of -separators if for

then we can implement trim so that the output graph is $

TH every , every subgraph with vertices has a

also a Gremban cover. Moreover, the genus and maximum

- -separator. size clique minor of the output graph do not increase.

Theorem 1.7 (Nested Dissection: Lipton-Rose-Tarjan).

After performing partial Cholesky factorization of the ver- Let be an by symmetric PSD matrix, be a

UU- 9,(- tices in the trim order, one obtains a factorization of the constant, and be a positive function of . Let

UU- ,¡ E\- _- a a

. If has a family of -separator, then the into . We deﬁne a weighted embedding of into to be

Nested Dissection Algorithm of Lipton, Rose and Tarjan a function that maps each edge of into a weighted

+)U-6£ [-5 ¥¤ / / - a

can, in time, factor into simple path in linking the endpoints of . Formally,

! #" $

UU- £ -57L"%$'& £¤

so that / has at most non-zeros. is a weighted embedding if for all

2)(

BA % IA a!& ' Fc N-§

, is a simple path connect-

*T+ ,-

To apply this theorem, we note that many families of ing from one endpoint of to the other. We let path

graphs are known to have families of small separa- denote this set of edges in this path in a . For , we

' ,- 4

* &,

tors. Gilbert, Hutchinson, and Tarjan [10] show that all Z

8 Z

deﬁne wd and the weighted -

path

1/. 4

11. 4

graphs of vertices with genus bounded by have a fam- *

A a N-

congestion of an edge under to be wc

¨§ ¦ U-

ily of -separators, and Plotkin, Rao and Smith [20] *

4 ' ,-2 + Fc N- &, Z wd

path -

. 11. 4 © show that any graph that excludes : as minor has a fam-

Our analysis of our preconditioners relies on the following

T"%$'& L- ily of -separators. extension of the support graph theory.

Theorem 2.1 (Support Theorem). Let be the Laplacian 1.5. Iterative Methods

matrix of a weighted graph and a be the Laplacian ma-

trix of a subgraph ( of . Let be a weighted embedding

Iterative methods such as Chebyshev iteration and Conju- of into ( . Then

gate Gradient solve systems such as by succes-

Y(Z[KcRaO-(H U[Z\ N-

sively multiplying vectors by the matrix , and then taking wc

Z3,54 linear combinations of vectors that have been produced so far. The preconditioned versions of these iterative methods To understand this statement, the reader should ﬁrst con-

take as input another matrix a , called the preconditioner,

. + ¨c N- sider the case in which all the weights , Z and

and also perform the operation of solving linear systems in .

Y EKcRaO- are 1. In this case, the Support Theorem says that Z a . In this paper, we will restrict our attention to the precon-

is at most the maximum over edges of the sum of the ditioned Chebyshev method as it is easier to understand the

lengths of the paths through . This improves upon the up- effect of imprecision in the solution of the systems in a on

per bound on Y#Z EKcRaO- stated by Vaidya and proved in Bern the method’s output. In the non-recursive version of our al- et. al. of the maximum congestion times the maximum di- gorithms, we will exploit the standard analysis of Cheby- lation, and it improves upon the bound proved by Boman

shev iteration (see [8]), adapted to our situation: and Hendrickson which was the sum of the dilations. This a

Theorem 1.8 (Preconditioned Chebyshev). Let and statement also extends the previous theories by using frac-

be Laplacian matrices, let be a vector, and let satisfy tions of edges in to route edges in . That said, our proof

. At each iteration, the preconditioned Chebyshev of the Support Theorem owes a lot to the machinery devel-

method multiplies one vector by , solves one linear system oped by Boman and Hendrickson and our is analogous to 6

in a , and performs a constant number of vector additions. their matrix .

At the th iteration, the algorithm maintains a solution a We ﬁrst recall the deﬁnition of the support of in , de-

satisfying 2EKcRaO-

noted 7 :

(

FE §

J S 1 4

DC] - D H Y[Z[`- Y(Z EaO- D D

72Mc/ad- UWV X8%:9;=<¡> 0?9 c@>6aBA

Gremban proved that one can use support to characterize

In the non-recursive versions of our algorithms, we will pre- [Z O :

compute the Cholesky factorization of the preconditioners

E\- ad- a , and use these to solve the linear systems encountered Lemma 2.2. If Null Null , then

by preconditioned Chebyshev method. In the recursive ver- §

O Mc/ad- 72EKcRaO-C72Ea cR\-

sions, we will perform a partial Cholesky factorization of Z

/ c c/ / -

a , into a matrix of the form , construct Vaidya observed

a preconditioner for , and again use the preconditioned

Chebyshev method to solve - the systems in . Lemma 2.3. If ( is a subgraph of the weighted graph ,

a (

is the Laplacian of and is the Laplacian of , then

The essence of support theory is the realization that one Lemma of Bern et. al. and the Rank-One Support Lemma

(Z(Mc/ad- can bound O by constructing an embedding of of Boman-Hendrickson:

? ?¡ ¢ £ 6?

Lemma 2.4 (Splitting Lemma). Let and if is the Gremban cover of such a graph, then the

? ?¤ £ ¢ V?

a a a

and let . Then, same bound holds and we can ensure that is a Grem-

ban cover as well.

72Mc/aO- H U[Z\ 72@7/cRa@7_- 7

Proof. Everything except the statement concerning Grem-

For an edge and a weighted embedding of into ban covers follows immediately from Theorem 2.1 and

a , we let denote the Laplacian of the graph containing Lemmas 3.7, 3.8, and 3.13.

a only the weighted edge and denote the Laplacian of

. In the case that is Gremban cover, we apply the algorithm

A *T!-

the graph containing the edges path with weights precondition to the graph that it covers, but keeping all

Z3 ' Fc N-

. We have: weights positive. We then set and to be both images of

A Lemma 2.5 (Weighted Dilation). For an edge , each edge output by the algorithm. Thus, the size of the set

is at most twice what it would otherwise be.

2 c/a - + ,-

7 wd . . For our purposes, the critical difference between these two graphs is that a cycle in the covered graph corresponds in Proof. Follows from Boman and Hendrickson’s Rank-One the Gremban cover to either two disjoint cycles or a double- Support Lemma. traversal of that cycle. Altering the arguments to compen- sate for this change increases the bound of Lemma 3.10 by Proof of Theorem 2.1. Lemma 2.5 implies at most a factor of © , and the bound of Lemma 3.13 by at

most D .

¥'§

72 c + ,- a -

. .

The spanning tree ¡ is built using an algorithm of Alon,

We then have Karp, Peleg, and West [1]. The edges in the set are con-

¥ structed by using other information generated by this algo-

* *

2McU[Z\ N- aO-QH 72Mc N- \Z -

7 wc wc

Z&,&

&,&

Z rithm. In particular, the AKPW algorithm builds its span- ¥

ning tree by ﬁrst building a spanning forest, then building

2Mc ¨ ,- a - 7 wd

. a spanning forest over that forest, and so on. Our algorithm

¥ ,

. works by decomposing the trees in these forests, and then

U[Z\ 72 c ¨ - a -

H wd

, .

. adding a representative edge between each set of vertices in

¥ c H the decomposed trees. where the second-to-last inequality follows from the Split- Throughout this section, we assume without loss of gener- ting Lemma. ality that the maximum weight of an edge is 1.

3. The Preconditioner 3.1. The Alon-Karp-Peleg-West Tree

In this section, we construct and analyze our preconditioner. We build our preconditioners by adding edges to the span-

ning trees constructed by Alon, Karp, Peleg and West [1].

c bc \-

Theorem 3.1. Let be a Laplacian and In this subsection, we review their algorithm, state the prop-

its corresponding weighted graph. Let have vertices erties we require of the trees it produces, and introduce the

>QHA

and edges. For any positive integer , the algorithm notation we need to deﬁne and analyze our preconditioner.

"%$'& -

precondition, described below, runs in

, ¡ The AKPW algorithm is run with the parameters

time and outputs a spanning tree of and a set

©¨

§ ¨¡©>

¥ § ¨© § ¨© § ¨¡©

£ §

and R , and the parameters

§ ¨¡©

of edges such that

, § "^$¨&2

D and are used in its analysis.

¡ £

(1) if a is the Laplacian corresponding to , then We assume, without loss of generality, that the maximum

; ¦

¥ § ¨© § ¨© § ¨¡© '-

Mc/aO- H £ 7 weight edge in has weight 1. The AKPW algorithm be-

Z , and ¢

gins by partitioning the edge set by weight as follows:

: H >67N"%$'& +* "%$'&2"%$'&

(2) : .

7 7

¥ ¥ §

N7 A * O' ,-QH *

: ©

Moreover, if has genus or has no minor, then

A ' ,-

For each edge , let class be the index such that

: H > U"%$'& U"^$¨& +* "%$'&2"%$'& U- A

(2’) : , class .

1/. 4

5 5

The AKPW algorithm iteratively applies a modiﬁcation of to be the tree of containing vertex .

5 5

an algorithm of Awerbuch [2], which we call cluster,

c -(A 7

7 ,

whose relevant properties are summarized in the following

5 5 5 5

! ! !

£ E

lemma.

7 7 7

7 , and .

Lemma 3.2 (Colored Awerbuch). There exists an algo- 5

We observe that is comprised of edges from

rithm with template 5

§¤§ §

c c

, and that each edge in has both end-

§ §¤§

¢¡¤£¦¥¨§ © ¤

c,+c c c -Hc

points in the same tree of .

§ §¤§

Alon, et. al. prove:

c O- , c c

where is a graph, is a number,

are disjoint subsets of , and is a spanning forest of , Lemma 3.3 (AKPW Lemma 5.4). The algo-

such that

rithm AKPW terminates. Moreover, for every 3 ,

" " " "

1 JL64

* ,'H : N7=:5*, J H

" " " "

7 7

"%$'&

(1) each forest of has depth at most , ,

H 3 H @7

(2) for each , the number of edges in class be- 5

E $#

,¤ : : 3 H §

We remark that , so for , 7 . The tween vertices in the same tree of is at most , times

following lemma follows from the proof of Lemma 5.5 of @7

the number of edges in class between vertices in

0 : :

distinct trees of , and [1] and the observation that .

¡£¦¥¨§ ©

' +

: /7¡: - Lemma 3.4. For each simple path in and for each

(3) runs in time 7 .

'&% (':+H UWV X+ J +Cc - , : .

The other part of the AKPW algorithm is a subroutine with

template

¡© ¡ ©

c -Hc 3.2. Tree Decomposition

that takes as input a graph and a spanning forest of

, and outputs the multigraph $ obtained by contracting Our preconditioner will construct the edge set by decom- the vertices of each tree in to a single vertex. This con- posing the trees in the forests produced by the AKPW al-

traction removes all resulting self-loops (which result from gorithm, and adding edges between the resulting sub-trees. edges between vertices in the same tree), but keeps an im- In this section, we deﬁne the properties the decomposition

age of each edge between distinct trees of . The classes, must satisfy and describe the decomposition algorithm. !

weights, and names of the edges are preserved, so that each

Deﬁnition 3.5. For a tree and a set of edges between

edge in can be mapped back to a unique pre-image in . the vertices of , we deﬁne an -decomposition of to be

*) c 7L- )

We can now state the AKPW algorithm: a pair where is a collection of subsets of the ver-

tices of and 7 is a map from into sets or pairs of sets

) -

Algorithm: AKPW in satisfying

1 4

A+)

1. Set and . 1. for each set , the graph induced by on is

5 connected, 4

2. While 1 has more than one vertex

5 5

§¤§ §

5 5

1 4Hc,Uc c c -

(a) Set cluster . 2. for each edge in there is exactly one set

J=+

5 5 5

¡© ¡ © ¡

containing that edge, and

+54 1 4Hc -

(b) Set 1

?A¥

A : 72+ - :

¡ £ ¡

72+ - points of lie in ; otherwise, one endpoint of

3. Set 5

2' ,- lies in one set in 7 , and the other endpoint lies in

The tree output by the AKPW algorithm is the union of the the other. 5

pre-images of the edges in forests ¡ . Our preconditioner

A ) We note that there can be sets containing just one

will include these edges, and another set of edges con-

structed using the forests 5 . vertex of . ! For a weighted set of edges ! and an -decomposition

To facilitate the description and analysis of our algorithm, !

*) c7L- A,)

, we deﬁne the -weight of a set by

we deﬁne

4 £ 5

.-O -0/2143 M' ,-

- ,6

, .

. 11. 4

to be the forest on formed from the union of the

¦ ¦

£ ¢

¡ £ £ ¡

JN - 4 M+ ,-

/7143 -

pre-images of edges in , We also deﬁne , .

Our preconditioner will use an algorithm for computing 3.3. Constructing the Preconditioner

A )

small -decompositions in which each set with

: : has bounded -weight. We can now describe our algorithm for constructing the pre- Lemma 3.6 (Tree Decomposition). There exists an algo- conditioner. We will defer a discussion of how to efﬁciently

rithm with template implement the algorithm to Lemma 3.8.

¡ The algorithm will make use of the parameter

¡£¢¥¤§

*) c7L- c c§¦#-

! !

, JN H §

? if

1 4

/2143

: : : : -

that runs in time and outputs an -

£ J¨¤JL

, otherwise

*) c7L-

decomposition satisfying

c - - Precondition

¥ Algorithm:

A+) : : -QH¨¦

1. for all such that , - , and

¦ ¦

) :+H ¥ -/*©¦

2. : .

1. Run . Set to the number of iter-

§¤§ §

¡ ¡

c c

ations taken by AKPW, and record and

! !

§¤§ §

,c .

* -

Proof. We let denote the set of vertices in the sub-

tree rooted at , and for a set of vertices , let

2. For to

! ! !

(

A & % O - /7143 * -/-

%¥ . We then deﬁne .

(

§¤§ §

% c c

Let denote the root of the tree. Our algorithm will pro- (a) let be the set of trees in .

O ,- ¥ ceed as if it were computing via a depth-ﬁrst traver-

(b) for 3 to

sal of the tree, except that whenever it encounters a subtree

! !

N*)£

of weight more than ¦ , it will place nodes from that sub- i. let be the subset of edges in with end-

tree into a set in ) and remove them from the tree. There points in

( §¤§

are three different cases which determine how the nodes are §

% c c ' c7L-

ii. Set to

5 B

placed into the set and how 7 is constructed.

75c c : :/*=> 1 45-

decompose

H H

If, when processing a node , the algorithm has traversed a iii. for each , let 8 be the max-

(

§ §¤§ ?

%2 c c O -

subset of the children of , such that imum weight edge in between and

¢ ¢ '?

O - 0¦N*)£

, then a set is created, all the nodes , and add to .

(

%7 - 7

in 7 are placed in , and those nodes in

£ Lemma 3.7. Let be the set of edges produced by Pre-

*)7E-

7 are deleted from the tree. If a node is encoun-

condition. Then,

N*)£¤H -5- H¦

tered such that ¦ , then a set is cre-

* -

ated, the nodes in are placed in , and those nodes in

7 7 7

: :+HG § > > "%$'& +* "%$'&2"%$'&

are deleted from the tree. In either case, for each node

(

A - 72' ,- 72' ,- %

we set .

Moreover, if has no : minor, then §

If a node is encountered which is not handled by either of

'> U"^$¨& L"%$'& +* "^$¨&2"^$¨& U-

O* - ¦

the preceeding cases and for which , then two sets

(

* - %2

and are created, and those nodes

- A -

in are deleted from the tree. For each edge , Proof. Let be the total number of sets produced by ap-

(

72' ,- A - %2 -

is added to and for each edge , plying decompose to the trees in . We ﬁrst bound

4 5

72+ -

is added to . . We have

¥ ¥

When the algorithm ﬁnally returns from examining the root, ¦

5 5

1 4

H ¥&> -/* : :

all the remaining nodes are placed in a ﬁnal set, and this

5 5

2' ,- A

set is added to 7 for each edge with endpoints

¥ ¥

¦ ¦

§ B

1 4

- ¥&>/*N: : -

in this set. The algorithm maintains the invariant that when- 7

5 7

ever it returns from examining a node , it has either deleted

d - ¦N* £

, or removed enough vertices below so that .

¦ ¦

*©¦

To see that the algorithm produces at most ¥ sets, we To bound this sum, we set

note that each edge in can contribute its weight to at most 2

if ,

" " #

two sets, and that every time the algorithm forms sets, it ei- !

" "

! "

4 3

&% 7

' if , and

¦N* £

" "

ther forms one set with weight at least or two sets with 7

" "

total weight at least . 7 if

*T' ,- + Fc ,- H : N7=:/*, J

We observe that Lemma 3.3 implies 7 , and path and . Otherwise, we let be the

5 5

? B

+ 7

[0 3 § 1 4 tree in containing the endpoints of and let be the

7 for . As is increasing, we have

72' ,- :

function output by decompose on input . If : ,

¥ ¥ ¥ ¥

' ,-

¦ ¦

5 5

! then we let path be the simple path in connecting

B B

(

1 4 1 4 JN

-(H *

7 7

% c 72' ,-

the endpoints of . Otherwise, we let

? 5 5 5

7 7

¨¡

J and let be the edge added between and . We

¥ ¥

*T' ,- 5

7 then let path be the concatenation of the simple path

1 4 JL

S8 S8

in from to , the edge and the simple path in

7 7

from to .

¥ ¥

5 5

7 7

1 4 J JN

9 H : : * , ¤

7 The two properties that we require of are encapsulated in

5? 7

7 the following lemma.

¦©¨

¥ ¥ ¥

H : N7¡: §

- 0 ? H

' '

Lemma 3.9. For all , 8

1 4

+' § £'-Hc

: : §(c

H and

L L

¥ ? ¥

E.H- 0 465 9L ¨c -QH § -

5 5

' 7

7 For all , .

¥ B ?

5 1 4 J JL

H63 § H ¥ §5> as H6, for . Thus, ,

and, because we add at most one edge between each pair of We now derive the upper bound we need on the maximum

: :+H G § 7>67 these sets, we have . weighted congestion of the embedding .

As observed by Vaidya, a result of Mader [4] implies that 5

' + Lemma 3.10. For each and each simple path in ,

if a graph does not have a complete graph on vertices as

5 §

a minor, then the average degree of every minor of is ?

H § £¨-

U"^$¨& -

MN-C9L ¨c N-6-

. Hence, the number of edges added to at iter- class

Z&,

L"%$'&§

ation is at most , and so

¥ §

Proof.

: :+H U"^$¨& H G § > U"^$¨&

¥ ¥

¥ ¥ ¥

MN-C9L ¨c N-5- M N- 9L 'c -

7 :¡ © class

Finally, a graph of genus does not have a minor. ?

Z&, ' Z&,

1 4

5 ¢

UWVYX J ,c

M N- 9L ¨c - ?

Using the dynamic trees data structure of Sleator and Tar- '

' '

jan [22], we prove: ¥

J + =¤

UTV X c

L ¨c -

Lemma 3.8. If is a graph with vertices and edges, 9

then the output of precondition can be produced in ?

H § £'-

"%$'& - ) time.

where the third-to-last inequality follows from Lemma 3.4,

A '

3.4. Analyzing the Preconditioner the second-to-last inequality follows from , and the

last inequality follows from Lemma 3.9 .

We will use weighted embeddings of edges into paths in Lemma 3.11. For each edge ,

¡ £

to bound the quality of our preconditioners. The 5

?@ §

' ,-(H £§ - M+ ,-

9L ¨c - weights will be determined by a function , which we

now deﬁne to be 5

A + ¢

Proof. Let 7 , let be the forest in contain-

*) c7L-

ing the endpoints of , and let be the output of de-

9L 'c -

1 J J¨=+64¤£

: 72' ,- : ¥§¦©¨ ¨ Otherwise. compose on input . If , the is routed over the

simple path in connecting its endpoints, so we can ap-

A A *T' ,-

For each edge and each edge path , we ply Lemma 3.10 to show

' Fc N- 9L ¨c N-6-

will set class . We will construct so as 5

? §

' ,-(H § £'- M+ -

' ,- N- §

to guarantee class class .

(

2' ,- % c

It remains to deﬁne the paths over which edges are embed- Otherwise, let 7 , and observe that

¡ £

*+c - A *T' ,- ded. For an edge in , if then we set path contains two simple paths in and the edge

. Applying Lemma 3.10 to each of these paths and re- 4. One-Shot Algorithms

8 -FH M S8 -¤0 * calling class , which implies , we obtain

Our ﬁrst algorithm constructs a preconditioner a for

5 5 5

? ? ?T@ §

+ +

¨ ,-(H £ § £'- M' ,- O' ,-QH ¦£§ - M+ ,- wd the matrix , performs a partial Cholesky factoriza-

tion of a by eliminating the vertices in trim order to obtain

/ c c/ / -

a , performs a further Cholesky factor-

/ / -

ization of into , and applies the preconditioned

'A 4

Lemma 3.12. For each and for each

Chebyshev algorithm. In each iteration of the precondi-

? @ §

' ,-(H ¦£§ - : :R*=>

wd tioned Chebyshev algorithm, we solve the linear systems

¦ £

,- Z&, a

path - in by back-substitution through the Cholesky factoriza-

. 11.64

tions.

5 +

Proof. Let be the tree in containing the endpoints

) c 7L-

of , and let be the output of decompose on in-

Theorem 4.1 (One-Shot). Let be an -by- PSDDD ma-

put . There are two cases two consider: can either be

trix with non-zero entries. Using a single application of

an edge of , or can be one of the edges . If is

our preconditioner, one can solve the system to

¢ )

an edge of , let be the set in containing its end-

¡ =!6 ¡ H1354¨"%$'&(EY+`-/*,!-

relative accuracy in time .

8 points. Otherwise, if is one of the edges , let be

Moreover, if if the sparsity graph of does not contain a

: : : J:

the larger of the sets or . If , then 9

minor isomorphic to the complete graph on vertices, or

B ¥ ©

the only edge having in its path is itself, in which case

7 9 *

if it has genus at most , for , then the expo-

¥'§%¥ @ ¥2? B ?£¢ ¥

:§

the lemma is trivial. So, we may assume : . In ei-

£ - -

nent of can be reduced to .

6A *T' ,-

ther case, each edge for which path must have

A;72' ,-

. Thus,

¥ ¥

* *

¨ ,-(H ' ,-

wd wd

¦ ¦

£ £ 5

,- Z3, ,- ,6

path -

. 11. 4 . 11.64 ¥

5 Proof. The time taken by the algorithm is the sum of the

? @

M' ,-N £§ -

H time required to compute the preconditioner, perform the

¦ £ 5

,- ,6

11.64

. partial Cholesky factorization of , pre-process (either

-N £§ -

- performing Cholesky factorization or inverting it), and the

5 5

? @ B

+ 1 4

£§ - : :R*=>

H product of the number of iterations and the time required

? @ §

per iteration. In each case, we will set > for some

H £§ - : :/*>

constant ¥ , and note that the number of iterations will be

¤ ¤

1 J 4¦/7¡ H1354

Lemma 3.13. Let , and be constructed as above. Let , and that the matrix will depend on .

¡ £

( . Then,

If we do not assume that has special topological structure,

7 H1354

then is a matrix on vertices. If we solve sys-

U[Z\ N- £

Z&,54 >

tems in by Cholesky factorization, then it will take time

Q H1354 ¤

to perform the factorization and time

A ( N-

Proof. For any edge , we let class and com-

H1354 ¤

to solve each system. So, the total time

¤ ¤ ¤

? © ¥ ©

pute

13!J 49R7¡ !1354 Q !1 64

¥ *

will be . Setting , ¥

we obtain the ﬁrst result.

* *

N- + ,-2 ' Fc N-

wc wd

, Z3,

;

7 :

path - If the graph has genus or does not have a mi-

. 11. 4

¥ ¥ L

nor, or is the Gremban cover of such a graph, then can

+ ,-29L 'c -

¦£ -

5 ¦

£ apply part of Theorem 3.1. Thus, is a matrix on

Z&, ,-

path ¤

. 11.64

U9¡ H1354

?@ vertices. In the Gremban cover case, the pre-

9L ¨c -N £§ - : :/*>

H conditioner is a Gremban cover, and so the partial Cholesky 5

factorization can ensure that is a Gremban cover as

?A¥ ?@

§ -N¦£ § - : :/*=>!c

H well. As the Gremban cover of a graph has a similar fam-

: :R*=> £ ily of separators to the graph it covers, in either case we can apply the algorithm of Lipton, Rose and Tarjan to ob- where the second-to-last inequality follows from Lemma

tain the Choleksy factorization of . By Theorem 1.7, with

¥ ? B _.H-

3.12, the last inequality follows from Lemma 3.9 , and

)£

* , the time required to perform the factorization

1% R9¡L= R7/4<!1354 ¤ the last equality follows from . will be , and the time required to

solve the system will be Moreover, if the graph of does not contain a minor iso-

9 ¤

¤ morphic to the complete graph on vertices, or has genus

13 J 49R7 1^7R9¡+54< H1354¢¡

) 7R9

at most , or is the Gremban cover of such a graph, then

¥ ?@,B ? ¢ ¥

the exponent of can be reduced to .

1 J 4¦/7¡+ H1364£¡

BT? ¥ ¥

We note that if E\- is planar, then the algorithm take time

£ - H

provided ¥ . We will obtain the desired result

nearly linear in . We believe that we can improve the term

¥ D -/* ¥

by setting . ¥

1 4

Y[Z[`-

¥ to .

1 4

/7 -

5. Recursive Algorithms The following lemma is needed to bound YNZ[ and

7 -

Y[Z[a .

We now show how to apply our algorithm recursively to im- Proposition 5.3 (Fiedler). Suppose is an by Lapla-

prove upon the running time of the algorithm presented in cian matrix. Then the smallest non-zero eigenvalue of is

UE\-R*

Theorem 4.1. at least UWVYX and the largest eigenvalue of is at

;U[Z\L`- Y#Z E\-QH L .U[Z\L`-/*#UWV XU`- most Thus, .

The recursive algorithm is quite straightforward: it ﬁrst con-

structs the top-level preconditioner a for matrix .

The following two lemmas allow us to bound the accuracy

It then eliminates to vertices of a in the trim order to obtain

a;7

of the solutions to systems in in terms of the accuracy of

/ ¦ / -

the partial Cholesky factorization a , where

2 the solutions to the corresponding systems in .

c c/

¦ . When an iteration of the preconditioned

/§¦ / Lemma 5.4. Let a be a Laplacian matrix and let Chebyshev algorithm needs to solve a linear system in a , we use forward- and backward-substitution to solve the sys- be the partial Cholesky factorization obtained by eliminat-

ing vertices of a in the trim order. Then

/ -

tems in / and , but recursively apply our algorithm to

solve the linear system in . ¦

? ¥ ¥

G ) -

Y+ / -(H

We will use a recursion of depth ¤ , a constant to be de-

UWVYX `-

termined later. We let denote the initial matrix.

a 7 7 / 7 ¦(7/ -

We let denote the preconditioner for , 7 be

/§¦ /(-

Lemma 5.5. Let a be a Laplacian matrix and let

+ 7 the partial Cholesky factorization of a in trim order, and

¨ be the partial Cholesky factorization obtained by eliminat-

2 2

7 c c/@7

¦ . To analyze the algorithm, we must deter-

¤A ad-

ing vertices of a in the trim order. For any Span ,

mine the relative error 7 to which we will solve the systems

/ JN

let be the solution to and let satisfy

in 7 . The bound we apply is derived from the following

D H #DGD /(-

D . Let be the solution to . Then

lemma, which we derive from a result of Golub and Over-

a D@H !Y+ / -5Y+ -JD D ton [11]. D Lemma 5.1 (Preconditioned Inexact Chebyshev

Proof of Theorem 5.2. Without loss of generality, we as- a

Method). Let and be Laplacian matrices satisfy-

¥ §¤§ § § §¤§

U[Z\N`- 'c c/ a c cRa

sume . For , , and

72a cR\-W0

ing . Let be the solution to . If, in

§¤§ §

c c / / as deﬁned above, we can apply Proposition 5.3, each iteration of the preconditioned Chebyshev Method, a Corollary 1.4, and Lemma 5.4 to show:

vector ¥ is returned satisfying

? © © ,

¨§ §

U[Z\Na -QH U[Z\L -(H U[Z\L`-

7 7

¦¥ c D D@H QD D c

a where ,

UWVYX+a 7 -(0 UTV XUE/7 -(0 U[Z)\#E\-/*, ¢

JL ,

H Y[Z[ad-C72EKc/ad- CG

where £ , then the -th iter-

Y+a 7_-(H L7 +C

, and

ate, , output by the algorithm will satisfy

Y+ -(H L7 L '9U[Z\L`-/*#UWV X+`-5-

7 , and

J S 1 4

D D/H £ Y(Z E\- Y(Z EaO- D D

Y+ / 7_-(H UWZ)\L\-R*#UWVYX `-

Our main theorem is: In the recursive algorithm we will solve systems in 7 , for

¥ 0

3 , to accuracy

Theorem 5.2 (Recursive). Let be an -by- PSDDD

matrix with non-zero entries. Using the recursive algo- 7

¥ §

UP 7

£'GC

7 rithm, one can solve the system to relative accu-

racy in time By Lemma 5.5 and the above bounds, we then obtain so-

a;7 #E\-

U[Z)\ lutions to the systems in to sufﬁcient accuracy to apply

! H1364 JL 0 1354

"^$¨&[E - "%$'&#) -5-

Lemma 5.1.

UWVYX+\-

7 7

Let be the number of edges of . When constructing the [8] A. M. Bruaset. A Survey of Preconditioned Iteratitve Meth-

7 )¤7 - ¥ preconditioner, we set > , for a to be chosen later. ods. Longman Scientiﬁc and Technical, 1995.

¤ [9] D. Coppersmith and S. Winograd. On the asymptotic com-

1%7 4¡

F7 H

Thus, by Theorem 3.1 and Proposition 1.1, ,

¤ ¤

plexity of matrix multiplication. SIAM Journal on Comput-

@75cRa@7 - 1^7 4 13 J 4< H1364

and Y(Z[ . + ing, 11(3):472–492, Aug. 1982. We now prove by induction that the running time of the al- [10] J. R. Gilbert, J. P. Hutchinson, and R. E. Tarjan. A separa-

gorithm obtained from a depth ¤ recursion is tor theorem for graphs of bounded genus. Journal of Algo-

rithms, 5(3):391–407, Sept. 1984.

¢¤£

U[Z)\#E\-

[11] G. H. Golub and M. Overton. The convergence of inexact

H1354

c ¤ "^$¨&[)

where

+\- UWVYX Chebychev and Richardson iterative methods for solving lin-

ear systems. Numerische Mathematik, 53:571–594, 1988.

¥ ¥

¥ [12] G. H. Golub and C. F. Van Loan. Matrix Computations,

- £ ¥L- £]¦£ ¥N- c

/7143 2nd. Edition. The Johns Hopkins University Press, Balti-

? 7

more, MD, 1989.

¥ ¥§¦

@ ©

E [13] K. Gremban. Combinatorial Preconditioners for Sparse,

-/* £ ¥ /2143

and /7143 . In the limit, approaches

¥ ©+? @

Symmetric, Diagonally Dominant Linear Systems. PhD the-

-/* ¥ ¤ from above. The base case, , follows from sis, Carnegie Mellon University, CMU-CS-96-123, 1996. Theorem 4.1. [14] K. Gremban, G. Miller, and M. Zagha. Performance eval- The preprocessing time is negligible as the partial Cholesky uation of a new parallel preconditioner. In 9th IPPS, pages

65–69, 1995. ¦ factorizations used to produce the 7 take linear time, and

[15] L. A. Hageman and D. M. Young. Applied iterative meth- the full Cholesky factorization is only performed on . ods. Computer Science and Applied Mathematics. Academic Thus, the running time is bounded by the iterations. The in- Press, New York, NY, USA, 1981. duction follows by observing that the iteration time is

[16] A. Joshi. Topics in Optimization and Sparse Linear Systems.

¢¤£

©¨©¨

¨ ?

¡ PhD thesis, UIUC, 1997.

"^$¨& EY+ -5Y+a -/*, - !1354

, which

¢¤£ [17] R. J. Lipton, D. J. Rose, and R. E. Tarjan. Generalized nested

proves the inductive hypothesis because . As ¥

¥§¦ dissection. SIAM Journal on Numerical Analysis, 16(2):346–

¤ , there exists an for which . 358, Apr. 1979.

[18] B. M. Maggs, G. L. Miller, O. Parekh, R. Ravi, and S. L. M.

: ;>= When the graph of does not contain a minor or has

Woo. Solving symmetric diagonally-dominant systems by 7 9 genus at most , we apply a similar analysis. In this case,

preconditioning. 2002.

9¡ H1364

H 7

we have 7 . Otherwise, our proof is sim-

E E

[19] S. F. McCormick. Multigrid Methods, volume 3 of Frontiers

¥ -/* £

ilar, except that we set 7 ,

¥§¦

in Applied Mathematics. SIAM Books, Philadelphia, 1987.

-R* ¥

and obtain 7 , and note that

¥ ¦ ?@,B

¥ [20] S. Plotkin, S. Rao, and W. D. Smith. Shallow excluded mi- H . nors and improved graph decompositions. In 5th SODA, pages 462–470, 1994. References [21] J. Reif. Efﬁcient approximate solution of sparse linear systems. Computers and Mathematics with Applications, 36(9):37–58, 1998. [1] N. Alon, R. M. Karp, D. Peleg, and D. West. A graph- [22] D. D. Sleator and R. E. Tarjan. A data structure for dynamic

theoretic game and its application to the -server problem. SIAM Journal on Computing, 24(1):78–100, Feb. 1995. trees. Journal of Computer and System Sciences, 26(3):362– 391, 1983. [2] B. Awerbuch. Complexity of network synchronization. Jour- [23] G. Strang and G. J. Fix. An Analysis of the Finite Element nal of the ACM, 32(4):804–823, Oct. 1985. Method. Prentice-Hall, Englewood Cliffs, NJ 07632, USA, [3] M. Bern, J. Gilbert, B. Hendrickson, N. Nguyen, and 1973. S. Toledo. Support-graph preconditioners. submitted to [24] P. M. Vaidya. Solving linear equations with symmetric di- SIAM J. Matrix Anal. & Appl. agonally dominant matrices by constructing good precondi- [4] B. Bollobas. Extremal Graph Theory. Academic Press, Lon- tioners. Unpublished manuscript UIUC 1990. A talk based don, 1 edition, 1978. on the manuscript was presented at the IMA Workshop [5] E. Boman, D. Chen, B. Hendrickson, and S. Toledo. on Graph Theory and Sparse Matrix Computation, October Maximum-weight-basis preconditioners. to appear in Nu- 1991, Minneapolis., 1990. merical Linear Algebra and Applications. [25] R. S. Varga. Matrix Iterative Analysis. Prentice-Hall, En- [6] E. Boman and B. Hendrickson. Support theory for precondi- glewood Cliffs, New Jersey, 1962. Second Edition, Springer, tioning. submitted to SIAM J. Matrix Anal. & Appl (Revised Berlin, 2000. 10/02). [26] D. M. Young. Iterative solution of large linear systems. Aca- [7] E. Boman and B. Hendrickson. On spanning tree precondi- demic Press, New York, NY, USA, 1971. tioners. Manuscript, Sandia National Lab., 2001.