15-853: Algorithms in the R eal W orld Lecture 6 , September 30, 2002

Graph S eparators Part II

L ect ur er : Prof. Guy Blelloch S cr ibe: Flavio L er da

1. Separator theorems At the end of last class we intr oduced the concept of a s eparator theorem: a s epar at or theorem proves that it is possible to obtain a good s epar ator for a cer t ai n class of graphs .

A good s epar ator was defined as having a cut of size s maller than a fi x ed cons tant times a funct i on of the s iz e of the gr aph, and s uch that the s iz e of the bigger of the two s ub-graphs generated by the cut is smaller than a fi x ed fraction of the s iz e of the or iginal graph.

I t is important to notice that it does not make s ens e to have a s epar at or theorem for a s i ngl e gr aph because it is always possible to find a bi g enough bound s uch that any s eparator is good enough. What we need i s a cl ass of graphs where the s iz e of the gr aphs can vary with a parameter n.

We defined a cl ass of graphs so that each sub-graph of a graph that belongs to the class also belongs to that class. T his is useful for developing the theor y, but for some appl i cati on this is not always the cas e: however, in many cas es , the r es ults are s till applicable in practice.

For instance, if we cons ider the r outing gr aph of the I nternet, it has a good s epar ator, but there ar e s ome s ub- graphs that are highly connected and they do not have a

Page 1 of 14 Algorithms in the R eal World – Graph S eparators II good s epar ator . The algor ithms based on graph separators s till work well in practice in cases like this , as long as the highly connected s ub-graphs are s mall enough.

We will start refreshing s ome of the definitions .

Def init ion 1.1: A class of graphs is a s et S of graphs that is closed under the s ub-graph relation.

We can define a ver t ex -s eparator theorem as follows:

Def init ion 1.2: A class of graphs S s atis fies a f (n) - -s eparator theorem if there ar e cons tants α <1 and β > 0 s uch that for every graph G = (V , E) in the cl ass S there ex is ts a cut set C ⊆ V which partitions the gr aph G in two s ub-graphs A and B s uch that C ≤ β f ( G ) , A ≤ α G and B ≤ α G .

And anal ogous l y an edge-s eparator theorem:

Def init ion 1.3: A class of graphs S s atis fies a f (n) -edge- s eparator theorem if there ar e cons tants α <1 and β > 0 s uch that for every graph G = (V , E) in the clas s S there exis ts a cut set C ⊆ E which partitions the gr aph G in two s ub-graphs A and B s uch that C ≤ β f ( G ) , A ≤ α G and B ≤ α G .

As we s howed i n the pr evious class, every good edge- s eparator can be tur ned i nto a good vertex-s eparator. T herefore if a cl ass of graphs has an f (n) -edge-s eparator theorem, it also has an f (n) -vertex-s eparator theorem.

However the other way around i s not always true. For ins tance, planar graphs (as we will see below) have a f (n) -vertex-s eparator theorem: however, if we cons ider

Page 2 of 14 Algorithms in the R eal World – Graph S eparators II the planar graph with n vertices obtained connecting one vertex to all the other s (to for m a s t ar ) , it has a good vertex-s eparator (the cut which contains only the center of the s tar is a ver t ex -s eparator), but it does not have any good edge s epar ator : in fact, in order to divide the ver tices in half it is necessary to have a cut that contains n 2 edges .

Figure 1 . 1 – Planar graph with a good vertex separator but not a good edge s eparator.

We will show that planar graphs satisfy a n -vertex- s eparator theorem. Also a par t i cul ar class of d - dimens ional mes hes satisfies a n(d −1) d -vertex-s eparator of which planar graphs represent the cas e d = 2 .

T heor em 1.4: Any graph from a cl ass with a n1−ε - s eparator theorem with ε > 0 has O(n) edges .

T his means that if a gr aph is from a cl ass that has a l es s than linear separator theorem the aver age degr ee of the graph is constant.

T he pr oof is left as an exercise.

2. Separator Trees

A s epar ator is the tr ee induced by recursively finding s eparators until you are left with single ver tices . The r oot

Page 3 of 14 Algorithms in the R eal World – Graph S eparators II of the tr ee contains the or iginal graph; each node of the tree is either a l eaf if the node contains only a ver tex , or it has two childr en: the children contain the par titions of the graph in the par ent node defined by a s epar at or (either edge- or vertex-s eparator).

G

C A B

Figure 2 . 1 – A s epar ator tree.

S ometimes the ver tices in the cut are carried to both children, sometimes only to one of them, sometimes to neither.

A s epar ator tree is fully balanced i f the two children are equal sized ( wi thi n one ver tex difference).

T heor em 2.1: For a cl as s of graphs S s atis fying an (α, β ) n1−ε -edge-s eparator theorem, we can generate a per fect l y balanced s epar ator tree with separator size C ≤ kβ f ( G ) .

Figure 2 . 1 – Unbalanced and bal anced s epar ator tree.

Page 4 of 14 Algorithms in the R eal World – Graph S eparators II

P r oof: T he s epar ator tree obtained by the (α, β ) n1−ε -edge- s eparator has a linear order (n ) of leafs, which correspond to the ver tices of the gr aph. First find a pat h in the s eparator tree fr om the r oot to the middl e leaf ( n 2 ). Cons ider cutting the tr ee s o that all the nodes on the left of the middle node ar e on one s ide and the r es t are on the other side. It is possible to s epar ate thes e two halves by cutting all edges in the nodes on the s elected path: the maximum number of edges cut in the nodes , starting at the top, is: βn1−ε + β (αn)1−ε + β (α 2n)1−ε +... = βn1−ε (1+α +α 2 +...)1−ε T he ter m (1+α +α 2 + ...)1−ε is constant, assuming that α and ε are cons tants and s i nce ther e ar e at most n levels in the tree. So we can define: 1−ε 2 1−ε  1  k = (1+α +α + ...) ≤   1+α  and we have: C ≤ kβn1−ε and s i nce G = n and f (n) = n1−ε : C ≤ kβ f ( G ) W

What this theorem means is that if the s iz e of the cut is s ub-linear to the s iz e of the s iz e of the gr aph, given a (α, β ) f (n) -s eparator theorem we can convert it into a (1 2,kβ ) f (n) -s eparator theorem with k > 1 (and ther efor e allowing a bi gger cut size).

3. We ar e going to for mulate now a s epar at or theorem for planar graphs. First we need to define the clas s of planar graphs formally.

Def init ion 3.1: T he s et of planar graphs is the s et of graphs that can be embedded i n a pl ane or in a s pher e s o that no two edges cross.

Page 5 of 14 Algorithms in the R eal World – Graph S eparators II

I t is easy to s ee that this is a cl ass of graphs since if a graph can be embedded i n a pl ane ( or a s pher e) without any edges crossing, any sub-graph of this graph can be embedded i n the s ame plane ( or sphere) without any edges crossing as well. Therefore the s et of planar graphs is closed under the s ub-graph relation and for ms a cl ass of graphs .

Als o note that if a gr aph is planar, an embeddi ng in a plane ( or in a s pher e) can be found i n linear time.

T heor em 3.2 (Planar Separators – L ipt on-T ar j an, 1977): T he class of planar graphs obeys a (2 3,4) n - vertex-s eparator theorem. P r oof: T he pr oof of this theorem is going to be cons tr uctive, in which we s how that there ex is ts such a s epar ator by giving an algorithm constructing i t.

T he algor ithm, which is linear-time, is described i n the following and a ps eudo-code for it is given in Figure 3 . 1 .

Firs t of all we need to find an embeddi ng of the planar graph in the plane: then we per for m a br eadt h first search s tarting fr om a r andom vertex. The s tar ting ver tex can be any vertex, even if in practice it is better to choos e a peripheral vertex, since this improves the per for mance of the algor ithm.

I f the number of levels that are necessary to vis it the whole gr aph is less than n (Figure 3 . 1 ) , then we call a s ub-procedure called CU T S H ALLOW, which gives a s eparator which satisfies the cons tr aints – we will describe this procedure later on.

T his represents the cas e wher e the gr aph is not very deep.

Page 6 of 14 Algorithms in the R eal World – Graph S eparators II

I f there ar e mor e than n levels , we will look for the level j in the B F S which contains the n 2 th vertex. If this level contains less than n vertices (Figure 3 . 2 ) , then the vertices in the level j are a s epar at or for the gr aph becaus e they the s pace in two halves , one befor e level j and one after : since level j contains the n 2 vertex, there ar e at most n 2 −1 vertices in the levels before ( and after ) level j . These two s ets will be s ets A and B in which the gr aph is partitioned whos e car dinality is at most n 2 −1. This would be enough to s atis fy the bounds imposed on the s iz e of the s ub-graphs .

Figure 3 . 1 – A S hallow gr aph.

T his represents the cas e wher e the gr aph is not very thick, where at each level of the gr aph there ar e not many vertices .

Figure 3 . 2 – A thi n graph.

What we s till have to tak e car e of are thos e gr aphs that are too deep to fall in the fir s t case but have s ome levels

Page 7 of 14 Algorithms in the R eal World – Graph S eparators II around the middle of the gr aph that contain too many vertices to be us ed to par tition the gr aph (Figure 3 . 3 ) .

I f there ar e mor e than n vertices in level j we will look for levels i and k , such that i < j < k , k − i < n , and ther e are less than n vertices in levels i and k .

i j k Figure 3 . 3 – Levels i, j, and k .

By a s i mpl e counting ar gument it is possible to s how that s uch level must exist. If there wer e no s uch i and k , then for every value of i and k we would have that there ar e more than n vertices in both levels, which means that for each level l s uch that i < l < k there ar e at least n vertices on each level. Since i and k can be at most n levels away, there mus t be n levels with more than n vertices in each of them, which means that there ar e mor e than n n = n vertices in the gr aph. But the gr aph has exactly n vertices so that’s a cont r adi ct i on and l evel s i and k which satisfy the given properties must exis t.

I f the number of vertices before level i is less than n 3 , then we can use level i as a s epar at or (the s ame applies if the number of vertices after level k is less than n 3 ).

Otherwis e we can extract the gr aph between levels i and k and s ubs ti tute level i with a s i ngl e r oot vertex: calling CUT S HALLOW on this graph will return a cut of this sub- graph. We can piece together the two par ts of the gr aph obtained wi th CUT S HALLOW with the two par ts obtained

Page 8 of 14 Algorithms in the R eal World – Graph S eparators II removing levels i through j to for m a 2 3−1 3 partition of the gr aph, adding the bigger of the left and r i ght components to the bigger of the two par titions .

Let R be the number of vertices in the levels before i ; let S be the number of vertices in the levels after k . Then the number of nodes between level i and k is n − R − S : the procedure CUT S HAL L OW computes a par t i t i oni ng wi th is in the wor s t-cas e 1 3 − 2 3 . If we assume, without loss of generality, that R < S we have that: 2 A = R + (n − R − S ) 3 1 B = S + (n − R − S ) 3 s ince we assumed R < S and fr om before R < n 3 and S < n 3 , we have: 2 2n 2R 2S 2n R − 2S A = R + (n − R − S ) = R + −−=+ 3 3 3 3 3 3 and: R − 2S < 0 becaus e R < S , so: 2n R − 2S 2n A < + < 3 3 3 s imilarly for B : 1 n R S n 2S − R B = S + (n − R − S) = S + −−=+ 3 3 3 3 3 3 but: n 2 2S − R 2S 2n n < < 3 = < 3 3 3 9 3 becaus e R < S and S < n 3 , so: n n 2n B < + = 3 3 3 W

SEPARATOR(G) Find an embedding of G in the plane. Perform BFS.

Page 9 of 14 Algorithms in the R eal World – Graph S eparators II

If number of levels < n Then C = CUTSHALLOW(G). Done. Fi.

Find level j containing n 2 vertex.

If Lj < n Then

C = Lj . Done. Fi.

Find i and j such that: i < j < k ; k − i < n ;

Li < n ;

Lk < n . If the number of vertices before level i is greater than n 3 Then

C = Li . Done. Fi.

If the number of vertices after level k is greater than n 3 Then

C = Lk . Done. Fi.

Extract the part of the graph between levels i and k. Replace level i with a single root vertex. Call CUTSHALLOW. Add the bigger of the left and right components to the smaller of the two partitions and vice versa.

Done. Figure 3 . 4 – Lipton-T arj an algorithm.

T he s ubroutine CUT S HALLOW assumes that the DF S of the graph has d levels and r etur ns a 2 3−1 3 -vertex-s eparator of size at most 2d +1.

Page 10 of 14 Algorithms in the R eal World – Graph S eparators II

Here’s the intuition behind the CUT S HALLOW procedure: let’s consider the B F S tree: this tree is embedded i n the planar graph, and s i nce the gr aph is planar, it is embedded in a pl ane ( or sphere).

We can therefore find a pat h from the r oot to one of the vertices including at most one ver tex from each level: therefore this path will have at most d +1 vertices , including the r oot. Taking another such path we s plit the tree ( and the gr aph) in two and we have a cut of size at mos t 2d +1, because they share the r oot.

I t is interes ting to notice that a ver s i on of this theorem for s lightly loos er bounds, a n log n -vertex-s eparator theorem, dates back to Ungar in 1951.

4. Kernighan-Lin Heuristic Even if the planar graph separator theorem by Lipton and T arj an propos es an algorithm to find s uch a s epar at or , this is not the currently most used al gor i thm for this purpose.

Kernighan and L i n proposed a heur i s t i c t o impr ove a gi ven edge s epar ator based on “hill climbing” which is most used in practice. The initial partitioning can be obtained us i ng a different algorithm (for example the CUT S HALLOW pres ented befor e) , or by randomly partitioning the gr aph.

T his algorithm was developed i n the contex t of circuit des ign for solving the pr oblem of routing connection between boards : a s y s t em would contai n different boards that need to be inter connected. If we can find an edge s eparator (with a s mall amount of crossing edges ) we can reduce the number of interconnections that are necessary among the boar ds .

T his algorithm allows for weighted edges , which means that each edge is assigned a wei ght and the pr oblem we want to s olve is to find an edge s epar ator with a s mal l

Page 11 of 14 Algorithms in the R eal World – Graph S eparators II weight, where the weight of the s epar ator is the s um of the weight of the edges that belong to the cut.

T he algorithm assumes we have an initial cut, with two equal sized par ti ti ons A and B . We want then to s wap a s ubs et X of A with a s ubs et Y of B , where X and Y have the s ame s iz e: the hope is that this switch will reduce the cut size.

Finding the optimal subset would s ol ve the optimal s eparator program itself, therefore it must be NP-hard.

I ns tead, the K er nighan and L i n propose a heur i s t i c wh i ch s waps a s i ngl e pair of vertices at a t i me, swappi ng that pair that at the time mos t decreases the cut size, or less increas es it.

T he idea i s to impr ove the cut size when possible with a greedy scheme. When this is not impossible, instead of s toppi ng, the algor ithm allows a s wap to incr eas e the cut s ize: this is useful when we r eached a l ocal minimum, s ince it might take us out of it being able to r each a better res ult later on.

Here ar e s ome definitions that we will use dur ing the explanation of the algor ithm:

w(u,v) the weight of the edge between u and v . C(A, B) the weighted cut between A and B . I(v) the s um of the weights of the edges incident in v which stay within the s ame partition. E(v) the s um of the weights of the edges incident in v that go to the other partition. G(v) = E(v) − I(v)

Page 12 of 14 Algorithms in the R eal World – Graph S eparators II

the s ingle ver tex gain, the r eduction of the cut size achieved by putting v on the other side. G(u,v) = G(u) + G(v)− 2w(u,v) the pair wise gain, the r eduction of the cut size achieve by swappi ng u and v .

A B A B

u u

Figure 4 . 1 – I (u)= 1, E(u)=3, G(u)=2 on a gr aph with unit weights.

Given a gr aph G = (V , E) and a par t i t i oni ng (A, B) of its vertices in two equal size s ubs ets , the K er nighan-Lin algorithm puts every pair (u,v) where u ∈ A and v ∈ B in a priority queue Q bas ed on the pr ior ity G(u,v) , the pair wise gain.

T hen it repeats V 2 times the following s teps : it extract from Q the pair (u,v) with the highes t priority, removes from Q every pair involving either u or v ; updates the priorities of the neighbor s of u and v , updating their pos ition in the queue if necessary: if w is a nei ghbor of u , its priority is increased by 2w(u, w) if w is on the s ame s ide as u before changing the par titions , decreases it by the s ame quantity otherwise; swaps the ver tices u and v .

At the end, the par tition obtained corresponds to the original one wher e the s ubs ets A and B have been s wapped. We s elect the bes t intermediate cut Ck . This requires to be able to r econs tr uct the bes t cut, either by undoing s ome of the s waps , redo the s equence of swaps

Page 13 of 14 Algorithms in the R eal World – Graph S eparators II that led to s uch a cut , or saving the bes t cut during processing.

We can estimate the r unning time as O(n2 ) for each iteration of the loop, so the total running time would be O(n3 ) .

Fiducci a-Mattheys es is a var i at i on of Kernighan-Lin where we cons ider individual vertices instead of pairs.

Firs t we build two queues Qa and Qb which contain the vertices in A and B res pectively, based on the pr ior ity G(u) .

T hen at each step, that we r epeat again V 2 times , we s elect the two ver tices at the top of the two queues , and we s wap them. We don’t need to r emove any other vertices , but we s till need to update the pr ior ities for the neighbors of the two chos en vertices.

Again, at the end, we have to s elect the bes t cut among the inter mediate ones .

T his version of the algor ithm has a r unning ti me that is O(V + E ) us ing appr opr i ate data s t r uct ur es : remember that each queue now has a s i z e that is linear in the number of vertices, while befor e it was quadratic.

Page 14 of 14