Expander Graphs and Their Applications

Expander Graphs and their Applications

Lecture notes for a course by Nati Linial and Avi Wigderson The Hebrew University, Israel.

January 1, 2003

Contents

1 The Magical Mystery Tour 7 1.1SomeProblems...... 7 1.1.1 Hardnessresultsforlineartransformation...... 7 1.1.2 ErrorCorrectingCodes...... 8 1.1.3 De-randomizing Algorithms ...... 9

1.2MagicalGraphs...... 10

´Òµ 1.2.1 A Super Concentrator with Ç edges...... 12 1.2.2 ErrorCorrectingCodes...... 12 1.2.3 De-randomizing Random Algorithms ...... 13 1.3Conclusions...... 14

2 Graph Expansion & Eigenvalues 15 2.1Deﬁnitions...... 15 2.2ExamplesofExpanderGraphs...... 16 2.3TheSpectrumofaGraph...... 16 2.4TheExpanderMixingLemmaandApplications...... 17 2.4.1 DeterministicErrorAmpliﬁcationforBPP...... 18 2.5HowBigCantheSpectralGapbe?...... 19

3 Random Walks on Expander Graphs 21 3.1Preliminaries...... 21 3.2 A random walk on an expander is rapidly mixing...... 21 3.3 A random walk on an expander yields a sequence of “good samples” ...... 23 3.3.1 Application: amplifying the success probability of random algorithms ...... 24 3.4Usingexpandersforhardnessofapproximation...... 25 3.4.1 ProofofTheorem3.11...... 26

4 A Geometric View of Expander Graphs 29 4.1TheClassicalIsoperimetricProblem...... 29 4.2GraphIsoperimetricproblems...... 29 4.2.1 Thediscretecube...... 30 4.3TheconstructionofMargulis,Gabber-Galil...... 31 4.4 The Cheeger constant, Cheeger inequality ...... 31 4.5Expansionandthespectralgap...... 33 4.5.1 TheRayleighquotient...... 33 4.5.2 Themaintheorem...... 33

5 Expander graphs have a large spectral gap 35

5.1 Comments about the previous lecture ...... 35

´Gµ 5.2 An upper bound on h ...... 35

5.3 The Inﬁnite d-RegularTree...... 37

3 4 CONTENTS

6 Upper Bound on Spectral Gap 39

6.1Remindertopreviouslecture...... 39

6.2 Lower bound on ½ ...... 39

7 The Margulis construction 43

8 The Zig Zag Product 47 8.1 Introduction ...... 47 8.2TheConstruction...... 48 8.3EntropyAnalysis...... 49

9 Metric Embedding 51 9.1BasicDeﬁnitions...... 51

9.2FindingtheMinimalDistortion...... 51 Ð

9.3 Embeddings in ¾ ...... 53 9.3.1 Embeddingthecube...... 53 9.3.2 Embeddingexpandergraphs...... 53

10 Error Correcting Codes 55 10.1 Deﬁnition of Error Correcting Codes ...... 55 10.1.1Motivation...... 55 10.2 Asymptotic bounds ...... 56 10.2.1 A lower bound: Gilbert Varshamov ...... 56 10.2.2 An Upper Bound - The Balls Bound ...... 57 10.3LinearCodes...... 57 10.4UsingExpanderstogenerateErrorCorrectingCodes...... 57 10.4.1 Deﬁning Codes using Bipartite Graphs ...... 57 10.4.2CodesUsingLeftSideExpanders...... 58

11 Lossless Conductors and Expanders 61 11.1Min-entropy...... 61 11.2 Conductors and lossless expanders ...... 63 11.2.1 Conductors ...... 63 11.2.2Losslessexpanders...... 64 11.3TheConstruction...... 64 11.3.1 The zigzag product for conductors ...... 65 11.3.2 A speciﬁc example of a lossless conductor ...... 66

12 Cayley graph expanders 69

13 On eigenvalues of random graphs and the abundance of Ramanujan graphs 73 13.1 The eigenvalue distribution of random matrices and regular graphs ...... 73 13.2 Random lifts and general Ramanujan graphs ...... 74

14 Some Eigenvalue Theorems 77 List of Figures

1.1LeslieG.Valiant...... 8 1.2 Clude Shannon...... 9 1.3MichaelRabin...... 10 1.4AconstructionofSuperConcentratorsusingMagicalGraphs...... 13 1.5Aconstructionofanerrorcorrectingcode...... 14

4.1SteinerSymmetrization...... 30

4.2 ...... 30

Å ;Å ¾ 4.3 C divides M into ½ ...... 32

7.1Thediamond...... 45

10.1 Illustrating Upper and Lower Bounds on rate vs. the relative distance ...... 56 10.2AVariablesandConstraintsGraph...... 58

11.1 Zigzag product of bipartite graphs ...... 65

11.2 Entropy ﬂow in a lossless conductor ...... 66

F Ó C

12.1 A Cayley graph of ¿ ...... 70 ¾

5 6 LIST OF FIGURES Chapter 1

The Magical Mystery Tour

Notes taken by Ran Gilad-Bachrach

Summary: Since the introduction of Expander Graphs during the 1970’s they turn to be a signiﬁcant tool both in theory and practice. They have been used in solving problems in communication and construction of error correcting codes as well as a tool for proving results in number theory and computational complexity. In this course we will explore Expander Graphs, both the properties and the use of such graphs will be studied. The goal of this lecture is to sample the wide range of applications for expander graphs. This should serve as a motivation for the rest of the course.

1.1 Some Problems

To begin our tour we will look at three questions from three very different domains. Note that in these problems the connection to graph theory, and especially to expander graphs is not clear.

1.1.1 Hardness results for linear transformation

= ÆÈ È 6= ÆÈ Maybe the most important open problem in mathematics these days is the famous È (or ) problem. Although it has been studied for decades now, almost no signiﬁcant progress has been made. One of the reasons lies in the fact that we have very few problems that are known to be hard. During the 1970’th, Leslie Valiant [Val76]

addressed this problem. He deﬁned the following simple problem:

A F A Ò ¢ Ò

Problem 1. Let F be a ﬁnite ﬁeld. Let be a linear transformation over ,i.e. is an matrix. We would

7! AÜ like to build a circuit which computes the transformation Ü . Each gate of this circuit computes addition or multiplication. How many gates do we need in this network?

Assume for instance that the transformation A represent the Fourier Transform. Cooley and Tukey [CW65] pre-

´Ò ÐÓg Òµ

sented the Fast Fourier Transform (FFT) which computes the transformation using Ç gates. However there

´Òµ

is no matching lower bound so it might be possible to do the computation using only Ç gates. The implications of

´Òµ a Very Fast Fourier Transform, i.e an Ç algorithm for computing the transform are hard to over estimate.

By counting the number of circuits and comparing to the number of linear transformations it could be veriﬁed that

Ò = ÐÓg Ò

the average size of such circuit is Ç gates, however we don’t know of any transformation which needs

´Òµ

more then Ç gates.

´Òµ Valiant [Val76] tried to present transformations for which the number of gates needed is greater then Ç .He

suggested that super regular transformation have this property: A Deﬁnition 1.1 (Super Regular Matrix). AmatrixA is Super Regular if any rectangular sub-matrix of has full rank.

7 8 CHAPTER 1. THE MAGICAL MYSTERY TOUR

Figure 1.1: Leslie G. Valiant

The main observation of Valiant was that if we look at the graph layout of a circuit which computes a super regular

matrix then this graph is a Super Concentrator:

Ò Á

Deﬁnition 1.2 (Super Concentrator). AgraphG is a Super Concentrator if it has input vertexes denoted by and

Ç k Ë Á Ì Ç k jË j = j Ì j = k

Ò output vertexes denoted by such that for every and every and of size (i.e. )there

G Ë Ì

exists k paths in from to which are vertexes disjoint. Ò

Valiant conjectured that any Super Concentrator graph must have edges and hence any circuit which computes Ò

a super regular matrix must have gates. However, Valiant himself disproved the conjecture and presented super

´Òµ concentrators with Ç edges, and as you might have guessed this is where expanders come into the picture. For the moment we will skip to a totally different problem.

1.1.2 Error Correcting Codes

One of the most fundamental problems in communication is noise. Assume that Alice has a message of k bits which she would like to deliver to Bob over some communication channel. The problem is that the channel might interfere in the way and thus the message that Bob receives might be different then the one that Alice sent. During the 1940’s Clude Elwood Shannon has developed the theory of communication which is called Information Theory. In his innovative paper “A Mathematical Theory of Communication” [Sha48] the problem of communication over noisy channel is one of the problems he addressed. Let us ﬁrst deﬁne the problem1:

Problem 2 (communication over noisy channel). Alice and Bob can communicate over a noisy channel that might k

change a proportion Ô of the bits sent through it. How can Alice send Bob a message of bits?

Shannon presented an answer to the above given question. He suggested building a dictionary (or code) C

Ò k

¼; ½g jC j =¾ k C Ò

f such that .Every -bits message is encoded by a code word in and transmitted. Bob receives k

bits and ﬁnds the closest code word in C in terms of hamming distance and determines the -bits associated with it. If

¾ÔÒ k the minimal distance between two words in C is greater then it is guaranteed that the -bits that Bob will ﬁnd is exactly the bits Alice encoded.

Therefore the problem of communicating over noisy channel is reduced to the problem of ﬁnding a good dictionary

C j C (code). A good dictionary is one that is both big (i.e. j is big) and at the same time the length of the words in is small. This is seen from the next deﬁnition:

1The problem as described here is a simpliﬁcation of the original problem presented by Shannon. 1.1. SOME PROBLEMS 9

Figure 1.2: Clude Shannon.

f¼; ½g Deﬁnition 1.3 (the rate and distance of a dictionary). Let C be a dictionary. The rate of the dictionary

is deﬁned as

ÐÓg jC j

Ê = Ò

while the distance of the code is

ÑiÒ d ´c ;c µ

c 6=c ¾C À ½ ¾

½ ¾

Æ =

Ò d

where À is the hamming distance.

As we saw before the distance of a dictionary governs it’s ability to overcome noisy channels while the rate counts

the efﬁciency of the code. At this point we can reﬁne the problem just state:

fC g

Problem 3 (reﬁned communication problem). Is it possible to design a series of dictionaries k such that

k =½

jC j =¾ Æ > ¼ Ê > ¼

¼ ¼ k , the distance of each dictionary is greater than and the rate of each code is greater than .

We will see that a solution to this problem can be found using expander graphs. However, we will now present yet another problem.

1.1.3 De-randomizing Algorithms

k k

Rabin [Rab80] presented in 1980 an algorithm for checking primality. Given an integer Ü of bits and a set of

f ´Ü; Ö µ Ü f ´Ü; Ö µ=½ Ü

random bits Ö the algorithm computes a function such that if is primal , on the other hand if is

´Ü; Ö µ=½ ½=4 not primal f with probability smaller then . Applying this algorithm over and over again can reduce the

error to be arbitrary small. However this process involves the use of more and more random bits.

f¼; ½g

The primality test is a special case of a Random Polynomial algorithm (RP). Let Ä be a language. An

¾f¼; ½g Ä

algorithm which decides on Ü weather it is in or not is Random Polynomial, if it runs in polynomial time

k µ Ü ¾ Ä ½=4

and using poly ´ random bits and gives an answer which is always one if and has probability smaller than

Ü ¾= Ä

to give ½ if .

f¼; ½g

Problem 4 (Saving Random Bits). Assume that that Ä has a random polynomial algorithm. How many =d random bits are needed in order to give an answer with probability of mistake smaller then ½ ? 10 CHAPTER 1. THE MAGICAL MYSTERY TOUR

Figure 1.3: Michael Rabin

1.2 Magical Graphs

In the previous section we presented three problems which seems to be unrelated. We will now present a new object

called Magical Graph which will enable us to ﬁnd a solution for all these problems.

Ò Ä

Deﬁnition 1.4 (Magical Graph). Let G be a two sided graph with vertexes on each side. Let be the vertexes on

Ä d Ê G ´d; Òµ the left side and Ê the vertexes on the right. Assume that any vertex in has neighbors in . We say that is

magical graph if it has the following two properties

Ò d

Ë Ä jË j µj ´Ë µjjË j

1. For any such that =

¿d 4

Ò Ò Ò

Ë Ä jË j =µj ´Ë µjjË j ·

2. For any such that <

¿d ¾ ¿d

Ë µ Ë G where ´ is the set of neighbors of in .

We will now turn to explore some properties of magical graphs.

8 Ò ´d; Òµ

Lemma 1.5. For each d and sufﬁciently large there exists a magical graph.

¾ Ä d Ê Proof. Construct a random graph as follows: for each vertex Ú choose randomly vertexes in and connect

them with Ú . We claim that with high probability the graph generated by this process is a magical graph.

Ò d

Ä × = jË j Ì Ê Ø = jÌ j < jË j

Let Ë be such that .Let be such that .

¿d 4

X Ë Ì

Let Ë;Ì be an indicator random variable for the event that all the edges from go to . It is clear that if

X =¼

Ë;Ì then the ﬁrst property in the deﬁnition of magical graphs hold.

×d

Ø X

The probability of the event Ë;Ì is and therefore using a union bound

¾ ¿

X X

4 5

[X =½] X > ¼ Ë;Ì

Pr Ë;Ì Pr

Ë;Ì Ë;Ì

×d

Ë;Ì

Ò=¿

Ò Ò

½¾

¿d ½¾

½ Ò ½ ½

·ÒÀ · ÐÓg ÒÀ

µ ´ µ ´

¿d ½¾ ¿ ½¾

Ò ¾

¾ Ò=8

Ò ¾

1.2. MAGICAL GRAPHS 11

¾ Ò=8

Ò ¾

When Ò is sufﬁciently large is smaller then 0.25 and therefore the probability that requirement 1 in the

:75 deﬁnition of magical graph will holds is greater than ¼ .

We use the same technique to bound the probability that requirement 2 in the deﬁnition of magic graph hold. For

Ò Ò Ò

Ë Ä < jË j Ì Ê j Ì j < jË j · Y

every such that and such that let Ë;Ì be an indicator random variable

¿d ¾ ¿d

Ë Ì Y =¼ for the event that all the edges from go to . As in the previous case, if Ë;Ì then the second property in the

deﬁnition of magical graphs hold.

×d

Ø Y

The probability of the event Ë;Ì is and therefore using a union bound

¾ ¿

X X

4 5

Y > ¼ [Y =½] Ë;Ì

Pr Pr Ë;Ì

Ë;Ì Ë;Ì

×d

Ë;Ì

Ò Ò

¾ ¿d

Ò Ò

¾ ¾

Òd ½ ½ ½

· ¾ÒÀ ÐÓg ·

µ µ ´ ´

¾ ¾ ¾ ¿d

Ò ¾

´ µ

Ò ¾

´ µ

Ò Ò ¾ ¼:¾5

For sufﬁciently large and hence the second property of magical graph holds with probability

:75 ¼ at least.

Finally if we chose a sufﬁciently large Ò we get that the two requirements of magical graphs hold with probability

:5 ´Ò; dµ greater than ¼ . Therefore not only that there exist an magic graph but there are many of those.

Before introducing the solutions to the above mentioned problems, we will present a small variation on magical

Ò Ê

graphs. We will delete ¾ vertexes from , i.e. the right side of the graph such that the main properties of the graph

will remain:

G B Ê jB j Ú ¾ Ä

Lemma 1.6. Let be a magical graph then there exists such that ¾ and for each vertex there

is at most one neighbor in B .

B Ä = Ä

Proof. We will present an algorithm which constructs the set . We begin by holding the two sets of vertexes ¼

Ê = Ê B

and ¼ and we reset to be the empty set.

Ú ¾ Ê ¾d Ú B Ä

i·½

At each iteration we choose i with degree at most .Weadd to and then construct such that

Ä = Ä Ò ´Ú µ Ê = Ê Ò ´ ´Ú µµ Ú Ä

·½ i i·½ i i

i and , i.e. we delete all the neighbors of from and delete all the second-

Ú Ê Ú ¾ Ê i

degree neighbors of from i . We keep the process running as long as there is a vertex with degree at most d

¾ .

Ù ¾ Ä B From the way we constructed the set B it is clear that any has at most one neighbor in . We would like to

count how many iterations can we do with the above algorithm, this will give us a lower bound on the size of B .

jÄ jÒ ¾di Ú B ¾d

At each step we have that i since the vertex which we add to has degree at most .Alsowe

jÊ j Ò d´Ò jÄ j µ Ä d i

have that i since each vertex in has degree . Since the number of vertexes in the graph

Ä [ Ê d jÄ j Ê

i i i

induced on i is at most , the average degree of the vertexes in is at most

d jÄ j d jÄ j

i i

jÊ j Ò d´Ò jÄ jµ

i i

Ò jÄ j ½ Ê ¾d i

and hence as long as i the average degree of the vertexes in is at most and therefore there

¾d ½ d

exist a vertex with degree at most ¾ .

Ò ½

Ò jÄ jÒ ¾di i jÄ j ½

¾ i

Since i we have that as long as that as required. Therefore we can

4d ¾d ½

Ò B

build a set of size at ¾ .

12 CHAPTER 1. THE MAGICAL MYSTERY TOUR Ò

We will call modiﬁed magical graph a magical graph that a set of size ¾ of vertexes were deleted from the right

4d side as described in lemma 1.6. Since magical graphs exists as we saw in lemma 1.5 and can be modiﬁed as we saw in lemma 1.6 we now turn to

use this construction to solve the problems presented in the ﬁrst section of this lecture.

´Òµ 1.2.1 A Super Concentrator with Ç edges As we saw in section 1.1.1 Valiant’s conjecture was that a super concentrator must have many edges. This sounds

reasonable from the deﬁnition of super concentrators (see deﬁnition 1.2). However we will see that using magical

´Òµ

graphs it is possible to build such graphs with only Ç edges.

G Ò G Ò

Let be a modiﬁed magical graph such that there are vertexes on the left side of but only ¾ vertexes

on the right side as we saw in lemma 1.6.

Ë Ä jË j j ´Ë µj j Ë j

By the construction of G we have that for each such that has . Hence by Hall’s

Ë j Ä Ë ´Ë µ

theorem [Die97, Theorem 2.1.2] for any j in there is a perfect matching from to . We will use these ¾

facts to build a super concentrator. Ò

The construction can be presented recursively. Let ¼ be the minimal size of modiﬁed magical graph. If we are Ò

required to build a super concentrator with less then ¼ vertexes we just return the full two sided graph. The full two

Ë ´Òµ

sided graph is a concentrator with Ò edges (we will use notation for the number of edges).

Ò> Ò G

Assume we would like to build a super concentrator with ¼ vertexes. Let be a modiﬁed magical graph

Ò Ò

Ò Ò C Ò ¾

with vertexes on the left side and ¾ vertexes on the right. Let be a super concentrator with input

4d 4d

and output edges. Such a concentrator exists according to our induction assumption.

C Ò

Using G and we will construct a new concentrator with inputs and outputs. The inputs of the new concentrator

G C G

will be the left side of G. We connect the right hand side of to the inputs of . We will place another copy of G

on the outputs of C , this is illustrated in ﬁgure 1.4(a). Finally we add direct edges between the two copies of , each G vertex on the left side of G we placed in the input is connected to the matching vertex on the left side of we placed in the output as illustrated in ﬁgure 1.4(b).

We would like to show that the graph constructed is indeed concentrator and count the number of edges in this

Ì jË j =

graph. Let Ë be a set of vertexes from the input of the new graph and be vertexes on the output such that

k Ò=¾ G j ´Ë µj jË j Ì j = k

j .If then due to the properties of the modiﬁed magical graph we know that and

´Ì µjjÌ j Ë ´Ë µ

j . Using Halls marriage theorem it is possible to construct a perfect matching between and and

´Ì µ C Ë ´Ë µ Ì ´Ì µ

on the other side between Ì and .Since is a super concentrator, the matches of in and of in

Ë Ì

can be connected by k disjoint paths and hence and can be connected by disjoint paths.

Ì jË j = jÌ j = k>Ò=¾ k Ò=¾ Ë If the two sets Ë and are big, i.e. then there must exists at least vertexes in that

are matched to vertexes in Ì by direct edges. These edges form paths and hence we can exclude these vertexes from

Ì Ò=¾ Ë and their matches from . After doing so we are left with groups of size at most which we already know how to treat.

After we proved that the graph we constructed is a super concentrator we turn to count the number of edges. Let

´Òµ Ò Ë ´Òµ=jC j ·¾jGj

Ë be the number of edges in the graph with inputs. From the construction we know that .We

¡¡

jGj = Òd j C j = Ë Ò ½ Ë ´Òµ

also know that and ¾ . Hence we obtain a recursive formula for :

¡¡

Ò>Ò Ë ´ Òµ ¾Òd · Ë Ò ½ ¾

1. for ¼ we have that .

Ò Ò Ë ´ Òµ Ò

2. for ¼ we have that .

Solving this recursive formula we get

Ë ´ Òµ cÒ

c = Ò ·8d

such that ¼ .

´Òµ Therefore, using magical graphs it is possible to construct super concentrators with Ç edges.

1.2.2 Error Correcting Codes

We now turn to present a solution to Shannon’s problem of correcting errors over communication channels. Again let

µ G Ò´½

be a modiﬁed magical graph, i.e. a magical graph such that it’s right side consist of less then ¾ vertexes. d 1.2. MAGICAL GRAPHS 13

Figure 1.4: A construction of Super Concentrators using Magical Graphs LR RL LR RL

G C G G C G

(a) ﬁrst stage (b) second stage

Ä jË j Ú ¾ Ë Ù ¾ Ê Ú Ù

Let Ë be such that then there exists and such that is the only neighbor of in the

¿d

¿d d

Ù ¾ ´Ë µ Ò ´Ë ÒfÚ gµ j ´Ë µj´ ½µ jË j > jË j Ù

group Ë ,i.e. . This follows since . To prove the existence of

4 ¾

Ú E Ë ´Ë µ d jË j jE j¾ j ´Ë µj j ´Ë µj Ë j

and , consider the set of edges between and .Then and so j which is ¾

a contradiction.

f¼; ½g ¾

We use this construction to build a code C of size such that the hamming distance between any two

Ò ½ k ½

distinct code words is at least so the code has distance and the rate is = .

¿d ¿d Ò ¿d

Ò´½ µ

G f¼; ½g f¼; ½g

Let G be a modiﬁed magical graph. We will view the graph as a function from to by

G ´ Üµ = ¨ Ü

¾ ´Ùµ

assigning a parity function to each vertex in the right side, i.e. Ú .

C = fÜ ¾f¼; ½g jG´Üµ=¼g C I.e. a word Ü is in the code if the parity assigned to each vertex on the right side is zero. Figure 1.5 demonstrates

the code.

Ò Ò=4d

µ C f¼; ½g Ò´½ jC j¾ C

is a linear sub space of deﬁned by ¾ linear equations and hence .Since is a

linear code (i.e. a linear sub-space) the minimal distance between two code words in C is the minimal weight of a

Ü ¾f¼; ½g Ü Ä Ë

non-zero code word in C .Let . We can look at as an indicating function of vertexes in .Let be the

jË j < Ù ¾ Ê

set of vertexes to which Ü assigns the value 1. If then there exists a unique neighbor, i.e there exists

¿d

¾ Ë Ú Ù Ë Ù

and Ú such that is the only neighbor of in . Hence the parity function associated with will assign the value

Ü Ü ¾= C C Ò=¿d ½ to and hence . Therefore the minimal distance between two code words in is at least . Therefore we presented a way to construct “good” dictionaries (codes) using magical graphs.

1.2.3 De-randomizing Random Algorithms

The last problem we presented was that of random algorithms. Let LANG be a language such that there exists an

k Ö k f ´Ü; Ö µ

algorithm such that when it receives Ü of size and a string of random bits, it calculates a function such

¾ f ´Ü; Ö µ=½ Ü ¾= f ´Ü; Ö µ=½ ½=½¾ that if Ü LANG then but if LANG then with probability at most over the choice

of Ö .

Ò =¾ Ë Ä

Let G be a magical graph over vertexes. By the deﬁnition of such graphs we know that for every

Ò Ò

Ë j j ´Ë µj Ü ¾= B B =

such that j we have that . Now assume that LANG. Let be the “bad” set i.e.

¿d ½¾

Ò Ó

jB j ¾f¼; ½g j f ´Ü; Ö µ=½

Ö . We know that .

½¾

Ä G d k

For each string of k random bits we assign a vertex in . The graph then direct us to string of bits which are

Ê Ö ;:::;Ö f ´Ü; ¡µ d associated to vertexes in , we will call these strings ½ . Our algorithm will apply the function with 14 CHAPTER 1. THE MAGICAL MYSTERY TOUR

Figure 1.5: A construction of an error correcting code

Ö ;:::;Ö ½ Ü ¾ f ´Ü; Ö µ=¼ j

d j

½ . If all the values we receive were then we will predict that LANG. If for some then ¾

we know for sure that Ü=LANG.

Ü ¾= Ö ;:::;Ö ¾ B ´Ö µ B Ë Ä d

Our algorithm will fail only if LANG and ½ ,i.e. .Let be the set of vertexes for

Ò Ò

Ë = fÚ ¾ Ä j ´Ú µ B g jB j jË j

which all their neighbors are in B ,so .Since we have that and therefore

½¾ ¿d ½

our algorithm will fail with probability at most, using only k random bits.

¿d

1.3 Conclusions

In the first section of this lecture we presented three problems of different nature. All these problem had no direct connection to graph theory. However we saw that by constructing magical graphs we could find a solution to these problems. During our discussion we explored some of the features of magical graphs. What other magic can these graphs do? what properties do they have. Can we construct these graphs efficiently? All these questions are the topic of this course. We will explore both the theory and applications of magical graphs. And one last word, the magical graphs we used in this lecture are a special case of the exciting family of Expanders. Chapter 2

Graph Expansion & Eigenvalues

Notes taken by Danny Harnik

Summary: After deﬁning families of expander graphs and giving some examples of such families, we discuss some algebraic properties of graphs. Mainly, we discuss the connection between the expansion property of a graph to the eigenvalues of the graph’s adjacency matrix. We also see an application of expander graphs for error ampliﬁcation with a small amount of random bits.

2.1 Deﬁnitions

We begin with some notes and notations: d

¯ Throughout this lecture (and course) we discuss -regular graphs (graphs in which all vertices have the same

G =´Î; Eµ jÎ j = Ò degree d). denote a graph by and . We allow self loops and multiple edges in the graph.

¯ Unlike the previous lecture, we discuss general graphs and not only bipartite graphs.

Ë; Ì Î Ë Ì E ´Ë; Ì µ= f´Ù; Ú µjÙ ¾ Ë; Ú ¾ Ì; ´Ù; Ú µ ¾ E g ¯ For denote the set of all edges between and by .

Deﬁnition 2.1.

@Ë @Ë = E ´Ë; Ë µ 1. The Edge Boundary of a set Ë , denoted ,is . This is actually the set of outgoing edges from

Ë .

h´Gµ

2. The Expansion Parameter of G, denoted ,isdeﬁnedas:

j@Ëj

h´Gµ= ÑiÒ

g fË jjË j jË j ¾

We note that there are other notions of expansion that can be studied. The most popular is counting the number of

neighboring vertices of any small set Ë , rather than the number of outgoing edges.

Deﬁnition 2.2. Family Of Expander Graphs

fG g i ¾ Æ

A family of Expander graphs i where is a collection of graphs with the following properties:

¯ G d Ò d fÒ g

i i

The graph i is a -regular graph of size ( is the same constant for the whole family). is a monotone

Ò Ò ·½

growing series that doesn’t grow too fast (e.g. i ).

¯ i h´G µ ¯>¼

For all , i .

When discussing a family of expander graphs one should also consider the time required to construct such a graph. There are two natural versions for the requirement on the constructibility of graphs:

15 16 CHAPTER 2. GRAPH EXPANSION & EIGENVALUES

Deﬁnition 2.3. i

1. A family of expander graphs is called Mildly Explicit if there is a Polynomial-Time algorithm that given ½ G

creates i .

i; Ú ; k µ

2. A family of expander graphs is called Very Explicit if there is a Polynomial-Time algorithm that given ´

i ¾ Æ;Ú ¾ Î k ¾f½; ¡¡¡ ;dg k Ú G

(where and ) computes the th neighbor of vertex in the graph i .

The second deﬁnition is useful for very large graphs, where one cannot construct the whole graph, but rather works locally on a small part of the graph.

2.2 Examples of Expander Graphs

G Î = Z ¢ Z

Ñ Ñ Ñ

1. This family of graphs Ñ lies on a grid: . =4

Thedegreeis d and the edges are described as follows:

Ü; Ý µ ´Ü · Ý; Ýµ; ´Ü Ý; Ýµ; ´Ü; Ý · Üµ ´Ü; Ü Ý µ Ñ Vertex ´ has edges to and (all operations are done modulo ).

Margulis ( 7¿) showed that this is an expander family.

¯ ¯

Gaber & Galil ( 8¼) showed that this is an -expander family ( for a speciﬁc ).

Ô Ô Î = Z d =¿ Ü Ô

2. This family has graphs of size (for all prime ). Here Ô and . Each vertex is connected to its

·½;Ü ½ Ü neighbors and its inverse (i.e. Ü and ).

This was shown to be an ¯-expander family by Lubotsky, Philips and Sarnak (88).

2.3 The Spectrum of a Graph

A´Gµ Ò ¢ Ò ´Ù; Ú µ

The Adjacency Matrix of a graph G, denoted ,isan matrix that for each contains the number of

Ù Ú d A´Gµ d

edges in G between vertex and vertex . Since the graph is -regular, the sum of each row and column in is .

A´Gµ Ú ; ¡¡¡ ;Ú

Ò ½

By deﬁnition the matrix is symmetric and therefore has an orthonormal base ¼ , with eigenvalues

; ; ¡¡¡ ; i AÚ = Ú

½ Ò ½ i i i

¼ such that for all we have . Without loss of generality we assume the eigenvalues are

¡¡¡ A´Gµ G

½ Ò ½ sorted in descending order ¼ . The eigenvalues of are called the Spectrum of the graph . The spectrum of a graph contains a lot of information regarding the graph. Here are some examples of observations

that demonstrate this connection between the spectrum of a d-regular graph and its properties:

¯ = d

¯ > ½

The graph is connected iff ¼

¯ =

Ò ½ The graph is bipartite iff ¼

In the rest of the lecture we will discuss the connection between the expansion of a graph and its spectrum. In particular, the graphs second eigenvalue is related to the expansion parameter of the graph.

Theorem 2.4.

h´Gµ ¾d´d µ

½ ¾

This Theorem is due to Cheeger & Buser in the continuous case, and to Tamner, Alon & Milman in the discrete

case.

The theorem actually proves that ½ , also known as the Spectral Gap, can give a good estimate on the

´Gµ > ¯

expansion of a graph. Moreover, the graph is an expander ( h ) if and only if the spectral gap is bounded

d > ¯

( ½ ). We do not prove this theorem at this stage (will be proved later in the course). Instead we show a Lemma that allows us to ﬁnd connections between the expansion property and the second eigenvalue. 2.4. THE EXPANDER MIXING LEMMA AND APPLICATIONS 17

2.4 The Expander Mixing Lemma and Applications

= ÑaÜ´j j; j jµ

Ò ½ Denote ½ .

Since the eigenvalues are already sorted, this means that is larger than the absolute value of all eigenvalues (except

= d

¼ ).

Lemma 2.5. Expander Mixing Lemma for all Ë; Ì :

djË jjÌ j

j jjE ´Ë; Ì µj jË jjÌ j Ò

This lemma can be viewed as relating the second eigenvalue to the question of how "random" the graph is. The

djË jjÌ j Ì

left hand side compares the expected number of edges between Ë and in a random graph ( ) and the actual

E ´Ë; Ì µj number of edges between the two sets (j ). This difference is small when is small. So a small (or large

spectral graph) means a graph with allot of "randomness".

Ë Ì Ú ¾ Ë

Ì Ë

Proof. Denote by Ë and the characteristic vectors of and ( is a vector with ones for all and zeros in

=¦ « Ú =¦ ¬ Ú

i i i Ì j j j

all other places). Let Ë and be their representation as linear combinations of the orthonormal

Ú ; ¡¡¡ ;Ú Ú = ½= Ò

Ò ½ ¼

base ¼ ,where .Wehave:

jE ´Ë; Ì µj = A

Ë Ì

= ´¦ « Ú µA´¦ ¬ Ú µ

i i i j j j Ú

and since the i ’s are eigenvectors and orthonormal:

jE ´Ë; Ì µj = ´¦ « Ú µ´¦ ¬ AÚ µ

i i i j j j

= ´¦ « Ú µ´¦ ¬ Ú µ

i i i j j j j

= ¦ « ¬

i i i i

jË j jÌ j

Ô Ô Ô

« = h ; i = ¬ =

Ë ¼

Since ¼ and :

Ò Ò Ò

jË jjÌ j

Ò ½

jE ´Ë; Ì µj = ·¦ « ¬

¼ i i i

i=½

jË jjÌ j

Ò ½

·¦ « ¬ = d

i i i

i=½ Ò

Due to the triangle inequality and the deﬁnition of :

jË jjÌ j

Ò ½

« ¬ j j = j¦ jjE ´Ë; Ì µj d

i i i

i=½

Ò ½

¦ j « ¬ j

i i i

i=½

Ò ½

¦ j« ¬ j

i i

i=½

And by the Cauchy-Schwartz inequality:

jË jjÌ j

j k«k k¬ k jjE ´Ë; Ì µj d

¾ ¾

= k k k k

Ë ¾ Ì ¾

= jË jjÌ j

Following is an example of an application of the lemma above: 18 CHAPTER 2. GRAPH EXPANSION & EIGENVALUES

2.4.1 Deterministic Error Ampliﬁcation for BPP

f : f¼; ½g ! f¼; ½g

In this example we are given a function in BÈ È . This means a function and a probabilistic

f Ö ¾f¼; ½g

polynomial time algorithm A that approximates in the sense that for random we have:

ÈÖ [A´Ü; Ö µ 6= f ´Üµ]

Ö 4

Our goal is to reduce the probability of error. This can be achieved by simply repeating A with different random

Ø A c c

coins Ö , and taking a majority vote. Using calls to one can reduce the error to for some constant .However

¡ Ñ this requires a large number of coin tosses (Ø in this case). The question is: can we make the error smaller with a

small number of random coins?

B Ñ B d G

We introduce an algorithm that uses just random coins: uses a -regular expander graph ¾ (of size

Æ =¾ Ú ¾ G A

). The algorithm chooses a random vertex ¾ and takes a majority vote on the output of on each of

Ú ´Ú µ Ú G

the neighbors of (denote by the set of the neighbors of in ¾ ).

B ´Ü; Ú µ=Å aj ÓÖ iØÝ A´Ü; Ùµ

Ù¾ ´Ú µ

Claim 2.6.

ÈÖ [B ´Ü; Ú µ 6= f ´Üµ] 4´ µ

B Ì A

Proof. Let Ë be the set of vertices that algorithm makes errors on, and be the set of vertices that algorithm

= fÚ jB ´Ü; Ú µ 6= f ´Üµg Ì = fÙjA´Ü; Ùµ 6= f ´Üµg Ú ¾ Ë

makes errors on (Ë and ). By deﬁnition, every has at least

d Æ

jE ´Ë; Ì µj jË j jÌ j

neighbors in Ì .So: On the other hand, due to the Expander Mixing Lemma and since :

¾ 4

djË jjÌ j

jE ´Ë; Ì µj · jË jjÌ j

Æ djË j

jË j ·

4 4

Combined together we get:

djË j Æ

jË j

4 4

And ﬁnally:

jË j

ÈÖ[B ´Ü; Ú µ 6= f ´Üµ] =

4´ µ d

The above claim shows that error ampliﬁcation can be done, but gives a rather poor ampliﬁcation rate. There are a

few ways to improve this:

¯ G A A

One can take instead of ¾ , with adjacency matrix , the graph with matrix . In this new graph (containing

k k

an edge for any path of length k in the original graph) we have degree , but also second eigenvalue giving

¾k µ

a reduced error of ´ .

½ ½

¯ It is worth noting that in a good expander we can achieve giving an error of about . So to get an error

d ½

smaller than a given ¯ one should use graphs of degree . ¯

¯ We will see in the next lecture constructions with better ampliﬁcation using random walks on expander graphs. 2.5. HOW BIG CAN THE SPECTRAL GAP BE? 19

2.5 How Big Can the Spectral Gap be?

We conclude with the question of how small can be.

As an example we can check the most connected graph - The Clique.

Ò Ã d = Ò ½

The -clique Ò is the graph were every vertex is connected to all its neighbors and has degree .The

Á Â

spectrum of the Clique can be easily calculated by viewing the Clique’s matrix as Â were is the all ones matrix

Á Ã [Ò ½; ½; ½; ¡¡¡ ; ½] =½

and is the identity matrix. The spectrum of Ò is . hence .

Ò d But this is for d and we are interested in the behavior of when is much smaller (usually a constant). This case is discussed in the following Theorem due to Alon-Boppana:

Theorem 2.7. for every d-regular graph:

¾ d ½ Ó ´½µ Ò

We will not prove this theorem here, but instead show a weaker statement:

Claim 2.8. for every d-regular graph:

d´½ Ó ´½µµ Ò

Proof.

note: In this discussion we don’t allow multiple edges (the Adjacency matrix contains only zeros and ones).

G A A

Given a d-regular graph with adjacency matrix , we look at the trace of (trace is the sum of the values in

d ´A µ = d the diagonal) . On one hand, since the matrix is symmetric, and the sum of each row/column is we have ii

for all i:

Ì Ö ace´A µ=Ò ¡ d:

On the other hand:

¾ ¾

Ì Ö ace´A µ =

¾ ¾

d ·´Ò ½µ

together we get

Ò d

Ò ½

and:

d´½ Ó ´½µµ Ò 20 CHAPTER 2. GRAPH EXPANSION & EIGENVALUES Chapter 3

Random Walks on Expander Graphs

Notes taken by Boaz Barak and Udi Wieder

Summary: In this lecture we consider random walks on expander graphs. We will see that the Ø vertices Ø on a length Ø random walk on a expander graph “look like” (in some respects) random independently

chosen vertices. This occurs even though sampling a length Ø walk on a (constant-degree) expander

requires a significantly smaller number of random bits than sampling Ø random vertices. We will use these properties for two applications. The first application is a randomness-efficient error reduction procedure for randomized algorithms. The second application is proving a strong hardness-of-approximation result for the maximum clique problem.

3.1 Preliminaries

´Ò; d; «µ G Ò ´Gµ;::: ; ´Gµ

Ò ½

graphs. For a graph on vertices we denote by ¼ the eigenvalues of the adjacency

G ´Gµ ´Gµ ::: ´Gµ G

½ Ò ½

matrix of ,where ¼ (recall that all the eigenvalues are real numbers since is

Ò ´Ò; dµ

undirected and so the adjacency matrix is symmetric). We say that a graph G on vertices is an -graph if it is

d ´Gµ=d «<½ G ´Ò; d; «µ G ´Ò; dµ

-regular. In this case ¼ . For a number , we say that is an -graph if is an -graph

ÑaÜ´j ´Gµj; j ´Gµjµ «d

Ò ½

and ½ .

~Ù;~Ú ¾ Ê ~Ù ~Ú h~Ù; ~Ú i ~Ù ~Ú i

Vectors and norms. For two vectors ,wedeﬁnethedot product of and , denoted to be i .

i=½

~Ù ¾ Ê Ð Ð Ð ~Ù

¾ ½

For a vector we deﬁne the ½ , and norms of as:

def

= j~Ù j k~Ùk

i=½

½=¾

def

k~Ùk = h~Ù; ~Ùi = ~Ù

¾ i

i=½

def

j~Ù j k~Ùk = ÑaÜ

½iÒ

Ô~ ¾ Ê ½ i Ò ~Ô ¼

Probability vectors. We say that a vector is a probability vector if for every , i and

Ô~ = ½ ~Ù ~Ù =

i . We denote by the probability vector that corresponds to the uniform distribution. That is,

i=½

;:::;½µ

´½ . Ò

3.2 A random walk on an expander is rapidly mixing.

In this section we show that a random walk on the vertices of an expander mixes rapidly towards the stationary

´Ò; d; «µ A distribution. Let G be an expander, and let be it’s adjacency matrix.

22 CHAPTER 3. RANDOM WALKS ON EXPANDER GRAPHS

G ´X ;X ;:::µ ½

Deﬁnition 3.1. A random walk over the vertices of is a stochastic process deﬁning a series of vertices ¼

X G X

i·½

in which ¼ is a vertex of chosen by some initial distribution and is chosen uniformly at random from the X neighbors of i .

A random walk is in fact a Markov chain where the set of states of the chain is the set of vertices of the graph.

A A

Deﬁnition 3.2. The normalized adjacency matrix of G is deﬁned to be and is denoted by . d

The following facts are easy to verify:

^ ½

1. A is double stochastic; i.e. every column and every row sums up to .

^ ^ ^ ^ ^

;:::; =½ ÑaÜfj j; j jg = « A

Ò ¼ ½ Ò

2. Denote by ¼ the eigenvalues of ,then and .

^ G

A can be viewed as the transition matrix of the Markov chain deﬁned by the random walk over the vertices of .

G Ô~ Y X

In other words let X be a random vertex in with probability vector .Let be a uniformly chosen neighbor of .

Y Ô

We claim that the probability vector of is given by A~. To see this write the Bayesian equation:

ÈÖ[Y = i] = ÈÖ[Y = ijX = j ] ¡ ÈÖ[X = j ]

= A Ô

ij j

=´A~Ôµ

^ Ø

A similar argument shows that A is the transition matrix of the Markov chain deﬁned by random walks of length ;

´A µ i j Ø

i.e. ij is the probability a random walk starting at reached in exactly steps.

~Ù = ~Ù Clearly A therefore the following theorem holds:

Theorem 3.3. The stationary distribution of the random walk on G is the uniform distribution.

The main result of this section is the following:

Ø Ø

A Ô~ ~Ùk Ò ¡ « Ô~

Theorem 3.4. k for any distribution vector . ½

In other words theorem 3.4 states that it doesn’t matter what the initial distribution of the random walk is (it might ½ be concentrated in one vertex), if «< we need to take only a logarithmic number of steps to get a distribution which

is close to the uniform up to a polynomial factor.

~ ~ ~

´Ú^ ; Ú^ ;:::;Ú^ µ A

½ Ò

Proof. Since is symmetric, it has an orthonormal base ¼ .

Ô ~Ù ~¯ = ~Ô ~Ù

Decompose ~ into the sum of the uniform distribution and an error vector . The sum of the coordinates

Ô ½ ~Ù ~¯ ¼

of ~ is ,andthesameistruefor therefore we have that the sum of the coordinates of is . Thismeansthat

~ ~

h~Ù; ~¯i =¼ ~¯ ´Ú^ ;:::;Ú^ µ Ò

.Inotherwords is spanned by ½ .Sowehave:

^ ^

A~Ô ~Ù = A ´~Ù · ~¯µ ~Ù

^ ^

= A~Ù · A~¯ ~Ù

= ~Ù · A~¯ ~Ù

= A~¯

Therefore we have:

^ ^

kA~Ô ~Ùk =kA~¯ k

¾ ¾

«k~¯ k

«kÔ~ k

¾ ¯ where in the ﬁrst inequality we used the fact that ~ is spanned by an orthonormal set of vectors for which the largest

eigenvalue is «. We deduce that

Ø Ø

kA Ô~ ~Ùk « ¾

and therefore that

Ø Ø

kA Ô~ ~Ùk Ò ¡ « ½ 3.3. A RANDOM WALK ON AN EXPANDER YIELDS A SEQUENCE OF “GOOD SAMPLES” 23

3.3 A random walk on an expander yields a sequence of “good samples”

Ò; d; «µ G

Consider the following problem: In an ´ -graph there is a large set of good vertices (satisfying some condi-

jB j

B Î = ¬ Ü ;:::;Ü Ð

tion) and we wish to ﬁnd one of them. Let be the set of bad vertices, and assume that .Let ½

Ð Î ÈÖ[8iÜ ¾ B ] ¬ Ð ÐÓg Ò

be vertices chosen uniformly at random from ,then i . This approach uses random Ð

bits. We will show that by choosing one vertex randomly, and then performing a random walk in G of length ,the Ð probability that the random walk is conﬁned to B is exponentially small in .

First some intuition. Recall the expander mixing lemma, proven in the previous lecture:

´Ò; d; «µ Ë; Ì Î ´Gµ

Theorem 3.5 (Expander Mixing Lemma). Let G be an -graph, then for every it holds that

¬ ¬

djË jjÌ j

¬ ¬

jË jjÌ j «dÒ E ´Ë; Ì µ «d

¬ ¬ Ò

Previously the expander mixing lemma was interpreted as saying that the number of edges between any two sets

of vertices is not far from the expected for sets of those sizes. Now divide the inequality by dÒ to receive the following

inequality:

¬ ¬

jË jjÌ j E ´Ë; Ì µ

¬ ¬

(3.1)

¬ ¬

Ò dÒ

i; j µ G i ¾ Ë j ¾ Ì

Consider the following test: select uniformly at random ´ two vertices of . Check whether and .

jË jjÌ j

The term ¾ can be interpreted as the probability that this test succeeds.

i; j µ G i ¾ Ë j ¾ Ì

Now consider a different test: select uniformly at random ´ an edge in . Check whether and .

´Ë;Ì µ

The term E can be interpreted as the probability that this test succeeds. Note that the size of the probability

dÒ

¾ Òd

domain of the ﬁrst test is Ò while the size of the probability domain of the second test is only , yet the difference ½ between success probabilities is only a small constant «. In other words a random walk of length can be viewed as

discrepancy sets over sets of two vertices. Next we will show that random walks of length Ø are in fact discrepancy

sets for sets of Ø vertices.

jB j

G ´Ò; d; «µ B Î ¬ = X ¾ Î Ê

Let be an -graph, and with density . Choose ¼ uniformly at random and let

jÎ j

X ;:::;X G X ´B; Øµ B

Ø ¼

¼ be a random walk on starting at . Denote by the event that the random walk is conﬁned to ;

8iX ¾ B

i.e. that i .

[´B; Øµ] ´¬ · «µ

Theorem 3.6. ÈÖ

È = È B

Let B be a projection on the space of vectors supported in ,i.e.

½ = j ¾ B

if i

È = ij

¼ otherwise

ÈÚ Ú If Ú is a distribution vector, then is the residual distribution vector of the distribution conditioned on being in

the set B . We need two lemmas:

[´B; Øµ] = k´È Aµ È~Ùk

Lemma 3.7. ÈÖ

~Ú B Proof. The action of È over a probability vector is to nullify all the coordinated outside the set , this transforms

a probability vector into the residual probability vector of the same distribution, but conditioned on being in B . Thus

È~Ù B Ù

is the residual probability of the uniform distribution conditioned to be in . AÈ ~ is the residual distribution

AÈ ~Ù B

after a random step has been taken. È is the residual probability conditioned on the random step remaining in .

È Aµ È~Ù Ø

Repeating this we see that ´ is the residual probability vector of the random initial point and all steps being

B ÈÖ[´B; Ì µ]

in B . Since we don’t care where in we end up, we need to sum the coordinates of this vector, hence is

´È Aµ È~Ùk

indeed given by k . ½

Lemma 3.8. For any non negative vector Ú :

kÈ AÈ~Ú k ´¬ · «µ ¡k~Ú k

¾ ¾

24 CHAPTER 3. RANDOM WALKS ON EXPANDER GRAPHS ^

Proof. The idea of the proof is that A shrinks all components of a vector except the uniform distribution component,

whereas È shrinks the uniform component without increasing anything else. Together they reduce all parts of the

vector.

È~Ú È~Ú =´È~Ú µ ·´È~Ú µ ´È~Ú µ È~Ú ~Ù h´È~Ú µ ; ´È~Ú µ i =¼

? ?

k k Decompose into k where is the projection of on and .By

the triangle inequality we know that

^ ^ ^

kÈ AÈ ~Ú k kÈ A´È~Ú µ k · kÈ A´È~Ú µ k

¾ ¾ ¾

È ´È~Ú µ ~Ù ½ A

First we look how affects k .Since is an eigenvector of with eigenvalue we know:

kÈ A´È~Ú µ k = kÈ ´È~Ú µ k = ¬ ¡k´È~Ú µ k

k k k

¾ ¾ ¾

´È~Ú µ ´a;a;:::;aµ a

where the second equality is true since k is of the form where is some scalar.

kÚ k k´È~Ú µ k hÚ; ~Ùi k´È~Ú µ k k

If we ﬁx then k is maximized when is maximized. In other words is maximized when

¾ ¾ ¾

Ú ~Ù

~ is some scalar multiple of . Therefore

kÈ A´È~Ú µ k ¬ k~Ú k :

¾ ¾

´È~Ú µ È ¼

Next we look at the effect on ? . ’s effect is to multiply some coordinates by without changing the others,

A È ´È~Ú µ ~Ù so can only shrink a vector. Since ? is perpendicular to , it is spanned by the remaining eigenvectors of ,

all with eigenvalues at most «. This implies that

^ ^

kÈ A´È~Ú µ k kA´È~Ú µ k «k´È~Ú µ k

? ? ?

¾ ¾ ¾

k´È~Ú µ k k~Ú k kÈ A´È~Ú µ k «k~Ú k ?

We note that ? and conclude that . Adding the two parts together we

¾ ¾ ¾ ¾

conclude that:

kÈ AÈ~Ú k ´¬ · «µ ¡k~Ú k

¾ ¾

Now we use the lemma to prove theorem 3.6:

Proof. (theorem 3.6)

Ø Ø

^ ^

k´È Aµ È~Ùk Ò ¡k´È Aµ È~Ùk

½ ¾

Ò ¡k´È AÈ µ ~Ùk =

Ò ¡ ´¬ · «µ k~Ùk

=´¬ · «µ

3.3.1 Application: amplifying the success probability of random algorithms

ÊÈ A Ü ¾ Ä

Let Ä be some language in and assume that is a randomized algorithm that decides whether with a

Ñ ¬ ´Ò; d; «µ

one sided error. Assume that A tosses coins and has an error probability of . Build an -graph such that

= f¼; ½g A Ü B

Î ; i.e. the vertex set of the graph is the probability domain of ’s coin tosses. Fix some input and let

´Üµ A

be all the coin tosses for which A is wrong. Now let be the following algorithm:

Ú ¾ Î

1. pick a vertex ¼ uniformly and at random.

Ø ´Ú ;Ú ;:::;Ú µ

½ Ø

2. perform a random walk of length resulting with the set of vertices ¼ .

A´Ü; Ú µ

3. return i

i=¼

A direct implication of theorem 3.6 yields that

¼ Ø

ÈÖ[A ]=ÈÖ[8iÚ ¾ B ] ´¬ · «µ

fails i

· Ø ÐÓg d = Ñ · Ç ´Øµ The error probability is reduced exponentially while the number of random bits used is only Ñ . Next we will show that the same trick can amplify the success probability of a two-sided error algorithm. In order to show this we need to restate theorem 3.6 in a stronger version.

3.4. USING EXPANDERS FOR HARDNESS OF APPROXIMATION. 25

jB j

B ;B ;:::;B Î = ¬ ´B; Øµ

½ Ø i

Theorem 3.9. Let ¼ be subsets of such that . Deﬁne to be the event that a random

´X ;X ;:::;X µ 8i X ¾ B

½ Ø i i

walk ¼ has the property that . It holds that

Ø ½

ÈÖ[´B; Øµ] ´ ¬ ¬ · «µ

i i·½

i=¼

¬ ¬ Ø Note that one should be able to strengthen that by a factor of ¼ , but our simple argument used in the proof

of Lemma 3.8 seems to lose that. È

The proof of theorem 3.9 is indeed similar to the proof of theorem 3.6. Let i be the projection matrix correspond- B

ingtotheset i . Lemma 3.7 should be restated such that

ÈÖ[´B; Øµ] = k ´È AµÈ ~Ùk

i ¼

i=½

The analouge of Lemma 3.8 is

kÈ AÈ ~Ú k ´ ¬ ¬ · «µkÚ k

i·½ i i i·½

;

and therefore theorem 3.9 follows.

BÈ È A Ü ¾ Ä

Now let Ä be a language in and assume that is a randomized algorithms that decides whether with a

A Ñ ´Ò; d; «µ

two sided error probability of ¬ . As before assume that tosses coins and build an -graph such that

½¼

= f¼; ½g A Ü B

Î ; i.e. the vertex set of the graph is the probability domain of ’s coin tosses. Fix some input and let

´Üµ A

be all the coin tosses for which A is wrong. Now let be the following algorithm:

Ú ¾ Î

1. pick a vertex ¼ uniformly and at random.

Ø ´Ú ;Ú ;:::;Ú µ

½ Ø

2. perform a random walk of length resulting with the set of vertices ¼ .

Ñaj ÓÖ iØÝ fA´Ü; Ú µg

3. return i

A Ú B Ã [Ø] jÃ j i ¾ Ã

fails iff a majority of the i ’s are in . Fix a set of indices such that . For each let

B = B

i . We deduce from Theorem 3.6 that

jÃ j

ÈÖ[8i ¾ ÃÚ ¾ B ] ´¬ · «µ ´¬ · «µ i

(note that sometimes the walk makes more that one transition before testing for membership in B ). By assuming that

« · ¬

« is small enough such that and applying the union bound we deduce that:

Ø Ø

¾ ¾

½ ½

¼ Ø Ø

ÈÖ[A ] ¾ ¡ ´¬ · «µ = ¾ ¡

fails

8 ¾

· Ç ´Øµ We achieve an exponential reduction in the error probability using only Ñ random bits. The following table sums up the parameters of the techniques presented for error reduction:

Method Error Probability No. of random bits

½ Ñ

random algorithm A

½¼

A ¾ Ø ¡ Ñ

Ø independent repetitions of

½ ½

Ò; Ø; µ Ñ

Sampling a point and it’s neighbors in an ´ -graph.

µ ´Ò; d; ¾ Ñ · Ç ´Øµ

A random walk of length Ø on an -graph 4¼

3.4 Using expanders for hardness of approximation.

In this section we show another application for random walks on expanders. We will show that we can use such

walks in order to establish a hardness of approximation result for an ÆÈ optimization problem - the maximum clique problem.

26 CHAPTER 3. RANDOM WALKS ON EXPANDER GRAPHS

! ´Gµ G ! ´Gµ

For a graph G,wedeﬁne to be the clique number of .Thatis, is the size of the maximum set

Î ´Gµ Ë

Ë such that all vertices in are neighbors of each other. Computing exactly the clique number of a graph is ÈCÈ ÆÈ-hard. Using the Theorem, it is possible to prove that it is even hard to approximate the clique number to

within a constant factor. That is, we have the following theorem:

Theorem 3.10. There exists a number ¼ , such that it is -hard to distinguish between the following cases:

In this section we will show that even obtaining a very rough approximation for ! is -hard. That is, we

! ´Gµ Ò ¯>¼ will show that it is ÆÈ-hard to approximate even within a factor of for some . That is, we will prove

the following theorem:

¼ A

Theorem 3.11. There exists a number ¯> , such that if there is a polynomial-time algorithm such that for every Ò

graph G with vertices

Note: The results of this section have been superseded by a result of Håstad that it is ÆÈ-hard to approximate

½ «

´Gµ Ò «>¼ ! even within a factor for every . However, our approach will involve simpler analysis (and of course, expander graphs).

3.4.1 Proof of Theorem 3.11.

To illustrate the main ideas behind the proof of Theorem 3.11, we will prove a weaker version of this theorem. In the

ÊÈ ÆÈ = È

weak version we will prove under the same assumption the weaker conclusion that ÆÈ (instead of ). ¼

Lemma 3.12 (Theorem 3.11, weak version). There exists a number ¯> , such that if there is a polynomial-time

G Ò

algorithm A such that for every graph with vertices

After we prove Lemma3.12, we will use the ideas of the proof, along with random walks on expanders to obtain Theorem 3.11, which can be looked at as a derandomized version of Lemma 3.12.

Proof of Lemma 3.12

(Since this proof will be superseded by the proof of Theorem 3.11, we allow ourselves some slackness.)

We will let ¯ be some constant, whose value will be determined later. Suppose that there exists a polynomial-time

algorithm A that distinguishes between the two cases of Theorem 3.11. We will show that there exists a probabilistic

ÆÈ ÊÈ polynomial-time algorithm B to distinguish between the two cases of Theorem 3.10, thus showing that .

Our algorithm B will work as follows:

BÈÈ ÆÈ

It may seem as if we only show that ÆÈ but it is not hard to show (using the self-reducibility of -complete problems) that if

BÈÈ ÆÈ ÊÈ ÆÈ then . 3.4. USING EXPANDERS FOR HARDNESS OF APPROXIMATION. 27

Algorithm B

G Ò

¯ Input: A graph on vertices.

À Ø G Ø =

1. Construct the graph À ,where is the following -th power of ,for

Òµ Î ´À µ Î Ø Î

¢´ÐÓg :Thevertexset is the set of all -tuples in . The edge set

E ´À µ h´Ú ;:::;Ú µ; ´Ù ;:::;Ù µi E ´À µ

Ø ½ Ø

is deﬁned as follows: ½ is an edge in iff

fÚ ;:::;Ú g[ fÙ ;:::;Ù g G

Ø ½ Ø

the set ½ is a clique in .

¼ Ø ¢´½µ

À Ñ = ¢´a µ = Ò 2. Let À be the graph obtained from by sampling ( )

vertices from À at random and taking their induced graph.

¼ ¯

A´À µ >Ò ¼ ¯

3. Return ½ if and otherwise. ( will be determined later.)

Ç ´ÐÓg Òµ À

It may seem like Algorithm B runs in time Ò instead of polynomial-time because the size of the graph

Ç ´ÐÓg Òµ À will be Ò . However the construction of in Step 1 can be done implicitly (that is, we don’t to write out the

full graph À ) and so Algorithm B can be implemented in probabilistic polynomial-time.

We have the following claim:

Proof. Clearly if Ë is a clique in then the set is a clique. Therefore .

Ø ¼ ¼

´À µ ! ´Gµ Ë À Ë

On the other hand we claim that ! . Indeed, if is a clique in then the union of all tuples in is

¼ Ø

jË j >k k

a clique in G.If then it must be that this union contains more than elements.

jË j

Ë Î ´À µ À Ë

For every clique , the expected fraction of vertices in that are in is Ø . With high probability we

jÎ j

jË j

Ë Î ´À µ Ë À ¢´ µ

will have that for every clique , the fraction of vertices in chosen to be in is Ø . Therefore we

jÎ j

! ´À µ ! ´Gµ

! ´À µ=¢´ ¡ Ñµ=¢´ ¡ Ñµ Ø

have that with high probability we will have that Ø . We see that:

jÎ j Ò

¼ Ø

´Gµ aÒ ! ´À µ a Ñ = ¢´½µ

1. If ! then .

ÐÓg ½:½

¼ Ø ¾¯

´Gµ ½:½aÒ ! ´À µ ½:½ ¢´½µ Ñ ¯

2. If ! then (for ).

¾ ÐÓg a

! ´Gµ > ½:½aÒ ¼ ! ´Gµ

The Actual Proof Now that we have proved Lemma 3.12, we will now use the ideas of this proof, along with random walk on expander

graphs, to prove Theorem 3.11. We will again let ¯ be some constant, whose value will be determined later, and assume

that there exists a polynomial-time algorithm A that distinguishes between the two cases of Theorem 3.11. We will

¼ B

use A this time to show that there exists a deterministic polynomial-time algorithm to distinguish between the two

= È

cases of Theorem 3.10, thus showing that ÆÈ .

¼ B

The only difference between Algorithm B and Algorithm , described in Section 3.4.1, is that in Step 2, Al-

¼ ¼ À

gorithm B will use a derandomized sampling to construct the graph . The sampling will work in the following

Ò; d; «µ G Î ´G µ=Î ´Gµ Ø

way. We will construct a ´ -expander such that . We will then choose the set of -tuples to be

Ø Ø G Ñ

sampled in À as the set of all -tuples that represent a length walk in the graph . We again let denote the number

¼ Ø ½

Ñ = Òd d G Ø = ¢´ÐÓg Òµ d

of vertices in À , note that (where is the degree of ). If and is constant then this value

Ø G is polynomial in Ò. What we have already seen is that a random length walk in does sometimes behave similarly

to a random Ø-tuple. We need to show that this holds also in this context.

We start with the following claim:

´Gµ aÒ Ë Î ´Gµ G Ø

Claim 3.13. Suppose that ! . For every clique in , the probability that a length random walk

Ø Ø

Ë Ë ´a · «µ in G doesn’t leave (i.e., is contained in )isatmost . Proof. This is a direct application of Theorem 3.6. As a corollary we obtain the following:

28 CHAPTER 3. RANDOM WALKS ON EXPANDER GRAPHS

¼ Ø

´Gµ aÒ ! ´À µ ´a · «µ Ñ

Corollary 3.14. If ! then

¼ ¼

Î ´À µ À Í G

Proof. AsetÍ is a clique in if and only if all tuples in are part of the same clique in .Sincewe

jÍ j

´Gµ aÒ ´a · «µ

assume that ! , this means that can be at most . Ñ

For the other direction, we need to prove the following claim:

´Gµ ½:½aÒ Ë Î ´Gµ G jË j½:½aÒ

Claim 3.15. Suppose that ! .Let be a maximum sized clique in (i.e., ). The

Ø Ø

G Ë Ë ´½:½a ¾«µ probability that a length Ø random walk in doesn’t leave (i.e., is contained in )isatleast .

Proof. This is an application of the following theorem that is analogous to Theorem 3.6:

6«

Theorem 3.16. In the notation of Theorem 3.6, suppose that ¬> . Then,

ÈÖ[´B; Øµ] ´¬ ¾«µ

Theorem 3.16 provides a lower bound on the probability that a random walk does not leave a speciﬁed set of

¬ Ø fraction ¬ . It shows that this probability is not much smaller than than (which is what happens if we choose independent random vertices). The proofs of Theorem 3.6, 3.16 can be found in the paper “Derandomized Graph Products” by Alon, Feige, Wigderson and Zuckerman.2 We remark that an analogous theorem to Theorem 3.9 also holds (i.e., a lower bound on the probability to stay in changing sets).

We now have the following corollary:

¼ Ø

´Gµ ½:½aÒ ! ´À µ ´½:½a «µ Ñ

Corollary 3.17. If ! then .

«<½ a «

Using both corollaries we see that if we choose « small enough such that and (we can take

c c

the graph G for some constant to ensure this) then we get that

¼ Ø

´Gµ aÒ ! ´À µ ¬ Ñ ¬<½

1. If ! then for some constant .

¼ Ø

´Gµ ½:½aÒ ! ´À µ Ñ >¬

2. If ! then for some constant .

Ø ¯ ¾¯ ¼

=¬µ = Ò = Ñ ¯ ;¯ A Since ´ for some constants we see that we can use to distinguish between the two cases.

2 Available from Avi Wigderson’s homepage on http://www.math.ias.edu/ avi/PUBLICATIONS/. Chapter 4

A Geometric View of Expander Graphs

Notes taken by Eran Ofek and Erez Waisbard

Summary: In the previous lectures we dealt with expander graphs in the combinatorial aspect (algo- rithmic and complexity applications), the algebraic aspect (spectral gap) and probabilistic aspect (rapidly mixing random walks). In this lecture we start dealing with the geometric/differential aspect of expander graphs. We introduce the construction of Margulis for expander graphs which is in fact a continuous graph with an expansion property. We show an analogy between expansion in graphs and the Cheeger constant which is deﬁned for Riemannian surfaces. We also show the connection between the expansion constant and the spectral gap.

4.1 The Classical Isoperimetric Problem

A very natural (and ancient) question in geometry is the following: Of all simple closed curves in the plane of a given length, which curve encloses the greatest area?

The solution to this question is obviously a circle. Although this fact was already known to the Greeks, they could not prove it. The ﬁrst proof for this fact (which can be considered rigorous) is the proof of Jacob Steiner (1800’s). This

proof uses a method called Steiner Symmetrization. We will brieﬂy sketch the idea of this method. Let Ã be a closed

¾ £ ¾

Ê Ã Ð Ã Ê

plane curve, let Ð bealinein .Thesymmetrization of with respect to which we denote by is a region in

Ð Ã Ã

which is symmetric about Ð such that any line perpendicular to intersects iff it intersects , and the intersections

£ Ã

have the same length; furthermore the intersections of lines perpendicular to Ð with are connected. It can shown

£ Ã

(via calculus) that Ã has the same area as and it’s boundary length does not increase with respect to K. In fact, if Ã Ð is not parallel to a line of symmetry of then symmetrization decreases boundary length.

To gain some intuition for the correctness of this statement, we will show it for the special case in which Ã is a

polygon which is composed of parallel trapezoids as demonstrated in ﬁgure 4.1.

£ Ã In this case Ã is accepted from if we transform each trapezoid into a symmetric trapezoid (a trapezoid of equal sides length) with the same bases and height as illustrated in ﬁgure 4.1. The area of the new trapezoid remains the

same. Furthermore, the sum of the side lengths can only decrease (this fact can be easily veriﬁed). It follows that the

£ Ã boundary length of Ã is less or equal to that of . We remark that the area and boundary length of any closed curve is accepted by considering the limit of the area and boundary length of shapes of this special form (see ﬁgure 4.2). This gives intuition for the correctness of the claim in general case.

4.2 Graph Isoperimetric problems

In the spirit of the last section, one can deﬁne an analogous problem in graphs (rather than in the Euclidean space). The graph is analogous to the plane, closed curves are analogous to subsets of vertices, the "area" of a subset of vertices is

29 30 CHAPTER 4. A GEOMETRIC VIEW OF EXPANDER GRAPHS

Symmetry line l

Figure 4.1: Steiner Symmetrization

Figure 4.2:

it’s cardinality and the "boundary length" of a subset is the number of edges which go out from it (or the vertices of

these edges which are outside the set). More speciﬁcally we deﬁne the following isoperimetric problems: k

Deﬁnition 4.1. The edge isoperimetric problem:givenagraphG and a number ﬁnd

fjE ´Ë; ¨ ´G; k µ = ÑiÒ Ë µj : jË j = k g

Ë Î k

Deﬁnition 4.2. The vertex isoperimetric problem:givenagraphG and a number ﬁnd

¨ ´G; k µ = ÑiÒ fj ´Ë µ Ò Ë j : jË j = k g

Ë Î

4.2.1 The discrete cube

Let us consider ﬁrst a well known graph for which the isoperimetric problem is partially solved. The discrete cube G

graph d is formally deﬁned as:

Î ´G µ=f¼; ½g

E ´G µ=f´Ú ;Ú µ: Ú ;Ú ¾f¼; ½g ; Ú ;Ú ½g

d ½ ¾ ½ ¾ ¾ the Hamming distance between ½ is

An equivalent deﬁnition for the d-dimensional cube graph is by recursion:

4.3. THE CONSTRUCTION OF MARGULIS, GABBER-GALIL 31

¯ G Ã ¾

½ equals (i.e. two vertices connected by an edge).

¯ G G i i

·½ d d is accepted by taking two copies of and connecting vertex in the ﬁrst copy to vertex in the second

copy (for all i).

Ð Ð

k =¾ ¨ ´G ;kµ ¾ Ð d

It is known that if then E is achieved by a set of vertices which induces an -dimensional cube.

Ð Ð

¨ ´G ; ¾ µ=¾ ´Ò Ð µ d

In this case E .

¡ ¡ ¡

d d d

k = ¨ ´G ;kµ Ë Ö · · ::: · d

For the vertex expansion Î is achieved by any set which is a ball of radius

¼ ½ Ö Ú

around some vertex ¼ :

Ë = fÚ : Ú ¾f¼; ½g ; Ú; Ú Ö g

The Hamming distance between ¼

4.3 The construction of Margulis, Gabber-Galil

In this section we describe one of the ﬁrst explicit construction of an expander graph. In contrast to the expanders we

encountered so far in this course, the construction they give is over a continuous set.

´¼; ½µ Á ¢ Á We denote by Á the interval . The set of vertices is all the points in the continuous cube . Two linear

transformations deﬁne the edges:

´Ü; Ý µ ! ´Ü · Ý; Ýµ

Ì mod 1

´Ü; Ý µ ! ´Ü; Ü · Ý µ

Ë mod 1

½ ½

Ü; Ý µ Ì ´Ü; Ý µ;Ë´Ü; Ý µ;Ì ´Ü; Ý µ;Ë ´Ü; Ý µ 4 The neighbors of a point ´ are the points: . Thus the graph is -regular. The expansion property of this graph is described by the following theorem:

Theorem 4.3. (Margulis,Gabber-Galil)

¼ A Á ¢ Á ´Aµ

There exists some ¯> such that for any measurable set with ( denotes the Lebegue measure) ¾

the following holds:

´ ´Aµ [ Aµ ´½ · ¯µ´Aµ;

½ ½

Aµ=Ë Áµ [ Ì Áµ [ Ë Áµ [ Ì Áµ A where ´ is the set of all points which are neighbors of points in .

It s worthwile to mention here the following conjecture:

´Aµ

Conjecture 4.4. (Linial) For any measurable subset A, such that ,

Á [ Ë Áµ [ Ì Áµµ Áµ;

;aµ ´a; ¼µ ´a; aµ ´¼; aµ ´ a; ¼µ ´ a; aµ

with equality achieved by the hexagon whose vertices are ´¼ , , , , , for some ¼ small a> .

4.4 The Cheeger constant, Cheeger inequality

In this section we introduce the Cheeger constant. Loosely speaking, this constant represents the "expansion" of a

curve.

´Òµ

Ò Å

Deﬁnition 4.5. Let Å be an -dimensional Riemann surface. The Cheeger constant of is deﬁned to be:

surface which divides

Å Å ; :::Å Ø into ½ 32 CHAPTER 4. A GEOMETRIC VIEW OF EXPANDER GRAPHS

M 2 C

M 1

Å ;Å ¾

Figure 4.3: C divides M into ½

´C µ C ´Å µ Å

½ Ò i i where Ò is the area of and is the volume of . An intuitive demonstration of the deﬁnition is illustrated in ﬁgure 4.3

The analogy between the Cheeger constant and expansion is as follows:

× G

Å , G

C is analogous to a cut in ,

ÑiÒ ´Å µ × ÑiÒfjË j; jË jg

Ò i

i .

Å ´Òµ f : Å ´Òµ ! Ê ¡´f µ=

Given an Ò-dimensional Riemann surface , and a function , then its Laplacian is

´f µµ

div´grad . The Laplacian is a linear operator, and its eigenvalues are all the numbers , for which there is a

: Å ´Òµ ! Ê ¡f = f function f satisfying . All its eigenvalues are non-negative, and its lowest eigenvalue is zero,

corresponding to the constant function.

Theorem 4.6. Let Å be a Riemann surface as described before, denote by the lowest positive eigenvalue of the

Laplacian of Å ,then .

=´Î; Eµ

We will now explain the discrete analogs for the gradient and divergence operators in graphs. Let G be

Å Î ¢ E G

an undirected graph. Select an orientation for the edges of G.Let be the adjacency matrix of where the

Å ½ ´ ½µ e Ú ¼

entry Ú;e equals if the edge enters (leaves) and otherwise.

: Î ! Ê G f Î

The gradient: Let f be a function on the vertices of . can be thought of as a row vector with

7! fÅ f E f

entries. The gradient operator is f . The gradient of is a vector with entries which tells us how does

e Ù Ú ´fÅµ = f f

Ú Ù

change along the edges of the graphs. I.e., if is the edge from to ,then e .

: E ! Ê G g E

The divergence: Let g be a function on the edges of . is a column vector with entries. The

g 7! Åg g Î Åg = g ´eµ

divergence operator is . The divergence of is a dimensional vector with Ú eentersv

´eµ

eleavesv g . Ò

The Laplacian: If we go through with the analogy between real functions in Ê and functions on the vertices of

Ø Ø

7! ÅÅ f f : Î ! Ê Ä = ÅÅ a graph, then the discrete analog of the Laplacian will be: f (for ). The matrix is

called the Laplacian of G. A simple calculation shows that L is the following symmetric matrix:

½; ´i; j µ ¾ E

Ä =

i;j

; i = j

deg(i)

G d A

In the case that is a -regular graph (with G it’s corresponding adjacency matrix):

¯ Ä = d ¡ Á A

G .

¯ Ä [¼; ·¾d] A [ d; ·d]

The spectrum of is in (since the spectrum of G is in ).

4.5. EXPANSION AND THE SPECTRAL GAP 33

¯ ´A µ=d ´Äµ=¼ ´A µ=d ´Äµ

G Ò i G Ò i·½

½ corresponds to and in general .

¯ G ´A µ ´A µ Ä

G ¾ G The spectral gap of ( ½ ) equals to the lowest positive eigenvalue of .

Notice the similarity between Theorem 4.6 and the upper bound on h given by Theorem 4.9.

4.5 Expansion and the spectral gap

In this section we show the that a graph has high expansion (high Cheeger constant) iff it has a large spectral gap.

4.5.1 The Rayleigh quotient A For a real symmetric matrix A one can obtain the eigenvalues of using a special quotient known as the Rayleigh

quotient.

A :::

¾ Ò

Theorem 4.7. Let be a real symmetric matrix and let ½ be its corresponding eigenvalues. Then:

Ø Ø Ø

ÜAÜ ÜAÜ ÜAÜ

= ÑaÜ ; = ÑaÜ ::: = ÑaÜ

½ ¾ Ò

jjÜjj jjÜjj jjÜjj

jjÜjj=½ jjÜjj=½;Ü?Ü jjÜjj=½;Ü?Ü ;:::;Ü?Ü

½ ½ Ò ½

Ü i where i is an eigenvector corresponding to .

4.5.2 The main theorem

Before stating the theorem, we formally deﬁne the expansion constant:

=´Î; Eµ

Deﬁnition 4.8. The expansion constant of a graph G ,is

jE ´Ë; Ë µj

h´Gµ=ÑiÒ

jÎ j

Ë Î;jË j

jË j

=´Î; Eµ Theorem 4.9. Let G be a ﬁnite, connected, k-regular graph without loops. Let be the second eigenvalue

of G.Then

h´Gµ ¾k ´k µ ¾

Proof. In this lecture we only prove the lower bound on h, showing that a large gap implies high expansion. In order

fAf

k ¾h´Gµ f ? ½ k ¾h´Gµ Ë Î f

to prove that , we will give a vector for which .For we deﬁne to

jjf jj

be the following weighted cut function:

f = jË j½ jË j½

Ë Ë

using the Rayleigh quotient we get:

fAf

jjf jj

We will evaluate the rhs. Starting with the denominator we get that:

¾ ¾ ¾

jjf jj = jË j jË j · jË j jË j = jË jjË j´jË j · jË jµ=ÒjË jjË j

moving on to the numerator we get:

Ø ¾ ¾

=¾jE ´Ë µjjË j ·¾jE ´Ë µjjË j ¾jË jjË jjE ´Ë; Ë µj fAf (4.1)

Since G is a k-regular graph

k jË j =¾jE ´Ë µj · jE ´Ë; Ë µj

and

k jË j =¾jE ´Ë µj · jE ´Ë; Ë µj

34 CHAPTER 4. A GEOMETRIC VIEW OF EXPANDER GRAPHS

jE ´Ë µj ¾jE ´Ë µj

substituting ¾ and in (4.1) yields:

Ø ¾

fAf = Òk jË jjË j Ò jE ´Ë; Ë µj

We now plug it in and get

Ø ¾

fAf Òk jË jjË j Ò jE ´Ë; Ë µj Ë µj ÒjE ´Ë;

= = k

jjf jj

ÒjË jjË j jË jjË j

Fix Ë to be a set for which

jE ´Ë; Ë µj

h´Gµ=

jË j

it follows that:

Òh´Gµ

k k ¾h´Gµ

jË j

Ë j

(since j ). ¾ Chapter 5

Expander graphs have a large spectral gap

Notes taken by Yael Vinner

´Gµ

Summary: In the previous lecture we started proving upper and lower bounds on h ,deﬁnedas

e´Ë;Ë µ

h´Gµ = ÑiÒ µ h´Gµ ´

Ë j

j . We proved a lower bound of . In this lecture we will prove

jË j ¾

´Gµ ¾d´d µ d

an upper bound of h . We will also discuss the -regular inﬁnite tree, which is the

Ô Ô

¾ d ½; ¾ d ½] optimal expander, and show that its spectrum is [ .

5.1 Comments about the previous lecture

G Å Ò = jÎ ´Gµj

¢Ñ

Given a graph , we choose an arbitrary orientation for its edges, and deﬁne the matrix Ò where

Ñ = jE ´Gµj Å Ù =½:::Ò e =½:::Ñ

and . The entries Ùe for and , are deﬁned as follows:

½ e =´Ù ! Ú µ

½ e =´Ú ! Ùµ

Å =

Ùe

¼ ÓØheÖ Û i×e

Ä = Å ¡ Å f : Î ! Ê

The Laplacian of G is deﬁned as the matrix . Then for any ,wehave

Ì Ì Ì ¾

fÄf = fÅÅ f ==k fÅ k

k¡ k Ð ´fÅµ = f ´Üµ f ´Ý µ

=´Ü!Ý µ

where is the norm. Furthermore, e , and therefore we can write

¾ ¾

´f ´Üµ f ´Ý µµ k fÅ k =

´Ü;Ý µ¾E

Another comment, is about the variational description of the eigenvalues of a matrix, which is

ÜAÜ

= ÑaÜ

Ü?Ü ;:::;Ü

k Ü k

k ½

This can be re-written in the following way:

ÜAÜ

= ÑiÒ ÑaÜ

Ü¾F

k Ü k F ;diÑ´F µ=Ò k ·½

´Gµ

5.2 An upper bound on h

e´Ë;Ë µ

´ µ G h´Gµ = ÑiÒ

Ë j

Theorem 5.1. For any connected graph , deﬁne j .Then

jË j

h´Gµ ¾d´d µ

35 36 CHAPTER 5. EXPANDER GRAPHS HAVE A LARGE SPECTRAL GAP

Proof.

: Î ! Ê

Deﬁnition 5.2. Given a function f ,deﬁne

¾ ¾

B = jf ´Üµ f ´Ý µj

´Ü;Ý µ¾E

¬ < ¬ < ¡¡¡ < ¬ f Î Ä

½ Ö i

Deﬁnition 5.3. Let ¼ be the different values achieved by over . Then we deﬁne for

=½;:::;Ö

i as follows:

Ä = fÜ ¾ Î jf ´Üµ ¬ g

i i

Ä Ä ¡¡¡ Ä

½ Ö This deﬁnition leads to ¼ .

We will use the three following claims.

C ¾ ¾

e´Ä ;Ä µ´¬ ¬ µ B = i

Claim 5.4. f .

i i i ½

i=½

´Ü; Ý µ ¾ E f ´Üµ=¬ >¬ = f ´Ý µ ´Ü; Ý µ B

Õ f

Proof. For ,if Ô , ’s contribution to is

¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾

´¬ ¬ µ=´¬ ¬ µ·´¬ ¬ µ·¡¡¡ ·´¬ ¬ µ

Ô Õ Ô Ô ½ Ô ½ Ô ¾ Õ ·½ Õ

¾ ¾

E ´¬ ¬ µ ´Ü; Ý µ f ´Üµ = ¬ > ¬ = f ´Ý µ Õ

When we sum over , appears once for every edge such that Ô and

i i ½

¾ ¾

Ô i>Õ ´¬ ¬ µ ´Ü; Ý µ Ü ¾ Ä Ý ¾= Ä i

.Inotherwords, appears exactly once for every edge such that i and ,and

i i ½

µ e´Ä ;Ä

in total i times, which gives us the desired result.

B ¾d¡kfÅ k¡kf k

Claim 5.5. f .

Proof. Using the Cauchy-Schwartz inequality, we have

X X

¾ ¾

B = jf ´Üµ f ´Ý µj = jf ´Üµ·f ´Ý µj¡jf ´Üµ f ´Ý µj

E E

× ×

X X

¾ ¾

´f ´Üµ·f ´Ý µµ ´f ´Üµ f ´Ý µµ

E E

We know that ×

´f ´Üµ f ´Ý µµ =k fÅ k E

and evaluating the second term in the product gives us

× × ×

X X X

¾ ¾ ¾ ¾

´f ´Üµ·f ´Ý µµ ¾ ´f ´Üµ·f ´Ý µµ = ¾d f ´Üµ= ¾d¡kf k

E E Î

from which we can conclude the inequality in the claim.

¼ jË ÙÔÔ´f µj Ë ÙÔÔ´f µ Î f ´Üµ 6=¼

Claim 5.6. If f and ,where is the subset of where ,then

B h´Gµ k f k

f ¬ = ¼

Proof. Since equals zero on more than half of the coordinates, and is positive on the rest, ¼ , and for every

e´Ä ;Ä µ

i ½ jÄ j h´Gµ

i . Therefore . Plugging this inequality into claim 5.4 yields

¾ jÄ j

Ö Ö

X X

C ¾ ¾ ¾ ¾

B = e´Ä ;Ä µ´¬ ¬ µ h´Gµ jÄ j´¬ ¬ µ=

f i i

i i i ½ i i ½

i=½ i=½

Ö Ö

X X

¾ ¾ ¾

¬ ¡jfÜjf ´Üµ=¬ gj = h´Gµ k f k ¬ ´jÄ j jÄ jµ=h´Gµ = h´Gµ

i i i·½

i i

i=½ i=½

5.3. THE INFINITE D -REGULAR TREE 37

A d Ä = dÁ A g Ä i

For each eigenvalue i of , is an eigenvalue of the Laplacian, .Let be an eigenvector of

d A f = g g

(and A) with eigenvalue ,where is the second largest eigenvalue of .Deﬁne , i.e. equal to where

jË ÙÔÔ´f µj g

g is positive, and zero elsewhere. Without loss of generality, , since otherwise we would look at

· ·

= Ë ÙÔÔ´f µ Ü ¾ Î

which is also an eigenvector with the same eigenvalue. Deﬁne Î . Then for every we can write

X X

´Äf µ´Üµ=df ´Üµ a f ´Ý µ=dg ´Üµ a g ´Ý µ

ÜÝ ÜÝ

Ý ¾Î

dg ´Üµ a g ´Ý µ=´Äg µ´Üµ=´d µg ´Üµ

ÜÝ

Ý ¾Î

´Üµ=¼ Ü=¾ Î

Since f for any , we can write

X X

¾ Ì ¾

fÅ k = fÄf = f ´Üµ ¡ ´Äf µ´Üµ ´d µ g ´Üµ=

k (5.1)

Ü¾Î

¾ ¾

=´d µ f ´Üµ=´d µ k f k

Ü¾Î

From claim 5.5 we have

B ¾d¡kfÅ k¡kf k f

and from claim 5.6 we have

B h´Gµ k f k f

If we combine these results we get

h´Gµ k f k ¾d¡kfÅ k¡ kf k

If we square this equation and combine with (5.1) we get

¾ ¾ ¾ ¾

h ´Gµ k f k ¾d¡kfÅ k ¾d´d µ k f k

and therefore

h ´Gµ ¾d´d µ

5.3 The Inﬁnite d-Regular Tree

A d Ì

Let us look at the infinite adjacency matrix Ì of the infinite -regular tree . The infinite vectors we work with are

Ð ´Î ´Ì µµ

those in ¾ :

Ð ´Î ´Ì µµ = fÜ : Î ´Ì µ ! Êj Ü < ½g

Ú ¾Î ´Ì µ

Á µ Á Á µ Á Á µ

Ì Ì

Deﬁne Ì to be the set of all ’s such that is non invertible. This is the set of all ’s such that

Ù ¾ Ð ´A Á µÙ =¼ ´A Á µ Ð

Ì Ì ¾ is not one to one, i.e. there is a vector ¾ such that ,or is not onto .

Theorem 5.7.

Ô Ô

´A µ=[ ¾ d ½; ¾ d ½]

¾ Î ´Ì µ

Given Ú (the root of the tree),

¾ ´Aµ ´µ Æ ¾= Ê aÒg e´Á Aµ

Ú Æ

where Ú is deﬁned as follows:

½ Ù = Ú

Æ ´Ùµ=

¼ Ù 6= Ú

The direction ´ is easy. The other direction requires a proof which will not be given here.

38 CHAPTER 5. EXPANDER GRAPHS HAVE A LARGE SPECTRAL GAP

Æ ¾ Ê aÒg e´Á A µ Ì

We wish to ﬁnd out for which values of , Ú .

f ¾ Ð

We are trying to ﬁnd a function ¾ such that

Æ =´Á Aµf

Ú (5.2)

Ù; Û Ú f ´Ùµ=f ´Û µ

Without loss of generality, f is spherical, meaning if are the same distance from then .Thisis

f g

true since if g is a solution to (5.2) then so is which is the spherical symmetrization of (for all vertices with a given

Ú f g f ´Ùµ d ´Ù; Ú µ

distance from , will be the average of on these vertices). Therefore, depends only on the distance Ì .

Ü ;Ü ;::: Ù d ´Ù; Ú µ=Ö f ´Ùµ=Ü

½ Ì Ö

We need to deﬁne a sequence of numbers ¼ such that all vertices with will have .

fÜ g f

Substituting the sequence i for in (5.2), we get the following recursion:

i=¼

Ü = dÜ ·½

¼ ½

Ü = Ü ·´d ½µÜ

i i ½ i·½

i i

; i Ü = A · B

¾ i

We will try to ﬁnd two numbers ½ , such that for every , . To ﬁnd these numbers, we need to

½ ¾

solve the equation

= ½·´d ½µ

The solutions to this equation are

´ ¦ = 4´d ½µµ

½;¾

¾´d ½µ

< 4´d ½µ ¾ ´Aµ j j = j j = f Ð

;¾ ½ ¾ ¾

If ( )then ½ are complex and . In this case, is not in :

d ½

i i

i ¾

¾ ¾

jÜ j = ¢´´d ½µ iÜ ¢´´d ½µ µ k f k ¢´´´d ½µ µ µ µ i

i , and for every appears times in , each time contributing ¢´½µ

to the sum. This means that each level i in the tree contributes to the sum, which results in the sum being inﬁnite.

> 4´d ½µ ;

¾ ½

On the other hand, if , then one of the roots ½ ,say ,islessthan in absolute value, in

d ½

=¼ i f

which case we can choose B , so the contribution of the ’th level of the tree to ’s norm will be exponentially

i f ¾ Ð A = dA ·½ 6= d

small in , and therefore ¾ . Also, there is a solution to ,since :

= ´ 4´d ½µµ

¾´d ½µ

¾´d ½µ 4´d ½µ

=½ ½

d ¾ This equality cannot hold for d> , since the r.h.s. is negative. Chapter 6

Upper Bound on Spectral Gap

Notes taken by Yishai Beeri

Summary: This lecture presents a lower bound for ½ , the second-largest eigenvalue of a -regular graph:

µ ¾ ´d ½µ´½ ¡ ¾

½ , which is related to the graph’s diameter . As the diameter grows, so does

the lower bound for ½ . This can also be viewed as an upper bound on the graph’s spectral gap.

6.1 Reminder to previous lecture A

In the previous lecture we discussed the inﬁnite d-regular tree and the spectrum of its (inﬁnite) adjacency matrix .

´Aµ

We deﬁned the spectrum as follows:

´Aµ:=fj´Á Aµ i× ÒÓØ iÒÚ eÖ ØibÐ e g

And we showed that for the d-regular inﬁnite tree:

Ô Ô

´Aµ=[ ¾ d ½ ; ¾ d ½]

k d

In this lecture we will use the ﬁnite ( -tall) -regular tree to prove a lower bound on ½ for general graphs.

6.2 Lower bound on ½

We show a lower bound on ½ for general -regular graphs:

d G Ò ¡

Theorem 6.1. There exists a constant c s.t. for any -regular graph of size and diameter :

¾ ´d ½µ´½ µ

Where ½ is the second largest eigenvalue of .

Notes:

D iaÑ´Gµ > ª´ÐÓg Òµ

¯ It is easy to show that in the above graph , and hence it follows that for

d ½ ¾

any ﬁxed d> :

¾ ´d ½µ´½ Ç ´ µµ:

ÐÓg Ò

1The original statement of the lower bound is due to N. Alon and R. Boppana, and appears in A. Nilli On the second eigenvalue of a graph Discrete Math., 91(2):207-210, 1991 2This stronger statement and proof is taken from J. Friedman Some geometric aspects of graphs and their eigenfunctions Duke Math. J. 69(3):487-525, 1993.

40 CHAPTER 6. UPPER BOUND ON SPECTRAL GAP

Ò = d ·½ G Ò G Ò ½; ½;:::; ½

¯ In the case that ( is the -clique) the eigenvalues of are { }, since

G A · Á = Â Â Ò; ¼;:::;¼ if A is the adjacency matrix for ,then and ’s eigenvalues are, naturally, { }.

While at ﬁrst this may seem to be a counter-example to our theorem, note that the theorem deals

Ò; ¡

with the case where d is ﬁxed and are going to inﬁnity.

ÜAÜ

g ¯ = ÑaÜ f

¾ ½

Since ~ we expect a proof for the above bound might use a speciﬁc "test

Ü?½

jjÜjj

È Ì

fAf

f f ´Üµ=¼ ¾ d ½´½

function" (=eigenvector), e.g. ﬁnd a vector s.t. and ¾

Ü¾Î ´Gµ

jjf jj

c µ

¾ .

d´×; Øµ = ¡ f

Proof Sketch: Taking two nodes ×; Ø with , we build a spherical function that will be positive for the

= b c ½ × k Ø

nodes within a distance of k from , and negative on the nodes that are within a distance from .The

g d k

values of f will be derived from the spherical function with maximal eigenvalue for the -regular tree of height ,

Ø k f Af f

treating × and as roots of (separate) -tall trees. We will show that for the positive part of we have ,and

fAf

Ø Ø ¾

f Af f fAf f f = jjf jj

likewise for the negative part of we have ,sothat ,giving ¾ . Finally,

jjf jj

f ´Üµ=¼

the positive and negative parts of f will be normalized to ensure , letting apply as a lower bound for

½ .

= b c ½ ×; Ø ¾ G d´×; Øµ=¡ ¼ i k

Proof. Set k . Select two nodes with distance .Forall deﬁne:

Ë := fÚ jd´×; Ú µ=ig i

and

Ì := fÚ jd´Ø; Ú µ=ig i

and in addition deﬁne

[

É := Î ´Gµ Ò ´ Ë [ Ì µ

i i

¼ik

Note:This is simply a breadth-ﬁrst-search dividing the graph into layers according to the distance from

Ø É ½

× and . represents the "middle ground" with at least layer of nodes. There are, of course, no edges

Ë Ì i

between any i and any .

Ì k A

Denote to be the ﬁnite tree of height ,andmark Ì to be its adjacency matrix.

:[¼;:::;k ·½] ! Ê Ì

Claim 6.2. There is a single spherical function g on that satisﬁes:

g ´k ·½µ=¼

g k ·½ A

(we extend ’s domain in include even though it is not part of Ì ), and

A g = g

g A A Ì ( is an eigenfunction of Ì with eigenvalue ), where is the maximal eigenvalue of .

This claim can be proved using the same technique as shown in the previous lecture dealing with the inﬁnite d-tree.

We will not prove this claim here, but will show a speciﬁc function g that displays these properties.

: Î ´Gµ ! Ê

Deﬁnition 6.3. Deﬁne f as follows:

c g ´iµ 9i; Ú ¾ Ë

½ i

c g ´iµ 9i; Ú ¾ Ì

f ´Ú µ=

¾ i

¼ ÓØheÖ Û i×e

c ;c ¼ ¾

where ½ .

f F :[¼; :::; k ] ! Ê f ´Ú µ=F ´iµ ´µ Ú ¾ Ë

Since is spherical, it will be convenient to deﬁne with i . We will

next show that f gives us the desired properties:

6.2. LOWER BOUND ON ½ 41

Lemma 6.4. If g above is non-negative and monotonically non-increasing, then:

Ú ¾ Ë => ´Af µ´Ú µ f ´Ú µ i

and

Ú ¾ Ì => ´Af µ´Ú µ f ´Ú µ

Ú ¾ Ë i G d Ú ½ Ô d i ½ ¼ Õ d Ô ½

Proof. Let i for some . is -regular, so has neighbors in level , another

d Ô Õ i ·½

neighbors in the same level i, and the remaining neighbors in level , giving:

Áf µ´Ú µ = ÔF í ½µ · ÕF íµ·´d Ô Õ µF í ·½µ

= Ôc g í ½µ · Õc g íµ·´d Ô Õ µc g í ·½µ

½ ½ ½

g A k d Ù i

but for and the matrix Ì (of the -tall -tree), we know that for a node of level :

g íµ=Á g µíµ=g í ½µ·´d ½µg í ·½µ

½ g since each node has exactly one neighbor in the previous level, and d neighbors in the next level. As is non-

negative and non-increasing we get:

Áf µ´Ú µ = c [Ôg í ½µ · Õgíµ·´d Ô Õ µg í · ½µ]

c [g ´i ½µ·´d ½µg ´i · ½µ]

= c Á g µíµ=c g íµ

½ Ì ½

= f ´Ú µ

Ú ¾ Ì F c ¾

The same argument is used for i , with the approriate redeﬁnition of the function , and using .

fAf

f ? ½

Corollary 6.5. ¾ and .

jjf jj

´Af µ´Ú µjjf ´Ú µj Ú ¾ Î ´Gµ Ò É Ú ¾ É f ´Ú µ=¼

Proof. The previous lemma gives j for .For , note that ,inwhich

´Af µ´Ú µjjf ´Ú µj

case j is trivial. From this follows:

fAf = = f ´Ú µ´Af µ´Ú µ

Ú ¾Î ´Gµ

X X X

= f ´Ú µÁf µ´Ú µ· f ´Ú µÁf µ´Ú µ· f ´Ú µÁf µ´Ú µ

9i;Ú ¾Ë 9i;Ú ¾Ì Ú ¾É

i i

X X

f ´Ú µf ´Ú µ f ´Ú µf ´Ú µ·

9i;Ú ¾Ì 9i;Ú ¾Ë

i i

= f f

Ú ¾ Ì f ´Ú µ ¼ Ú ¾ Éf´Ú µ=¼

since for i and for .Fromthisweget:

fAf

Ì Ì

fAf f f =>

jjf jj

f ´Ú µ=¼ f ? ½ c c ¾

Next, we show that ,(namely: ). This is easily achieved by selecting ½ and such that

X X

f ´Ú µ f ´Ú µ=

Ú ¾Ì Ú ¾Ë

i i

42 CHAPTER 6. UPPER BOUND ON SPECTRAL GAP

g ´k ·

Definition 6.6. Finally, we will find the spherical function g that is non negative, non-increasing, that satisfies

½µ = ¼, and that has the maximal eigenvalue . We’ll also show that this yields the desired bound:

i=¾

g ´iµ:=´d ½µ ×iÒ´ ´k ·½ iµµ

where .

¾k ·¾

g g ´k ·½µ= ¼ A g = g

It is easy to check that is non-negative, non increasing, and that . We now show that Ì

=¾ d ½ cÓ×´ µ j

with .Forall :

´A g µ´j µ = ½g ´j ½µ · ´d ½µg ´j ·½µ

´j ½µ=¾ ´j ·½µ=¾

= ´d ½µ ×iÒ´ ´k ·½ ´j ½µµµ · ´d ½µ´d ½µ ×iÒ´ ´k ·½ ´j · ½µµµ

´ j ·½µ=¾ ´ j ·½µ=¾

= ´d ½µ ×iÒ´ ´k ·¾ j µµ · ´d ½µ ×iÒ´ ´k j µ

j=¾

= d ½´d ½µ ´×iÒ´ ´k ·¾ j µµ · ×iÒ´ ´k j µµµ

« ¬ «·¬

µ cÓ×´ µ «µ · ×iÒ´¬ µ = ¾ ×iÒ´ j

And since ×iÒ´ ,wegetforall :

¾ ¾

j=¾

×iÒ´ ´k ·½ j µµ ´A g µ´j µ = ¾ d ½ cÓ×´ µ´d ½µ

= ¾ d ½ cÓ×´ µg ´j µ

cÓ×´ µ ´½ µ

To arrive at our bound, we need to show that ¾ . Recall:

= =~

¾k ·¾ ¡

c ½ = b cÓ×´Üµ

since k , and use the Taylor expansion for the fucntion :

cÓ×´Üµ=½ · Ó´Ü µ ¾

to get:

cÓ×´ µ ½ =´½ µ

¾ ¾

¾¡ ¡

with c . ¾ Chapter 7

The Margulis construction

Notes taken by Statter Dudu

8 Z ¢ Z Ò Summary: We deﬁne an explicit family of -regular graphs on the torus Ò , and prove that this is

a family of expander graphs.

Construction 7.1. Let

½ ¾ ½ ¼ ½ ¼

Ì = ;Ì = ;e = ;e = ;

½ ¾ ½ ¾

¼ ½ ¾ ½ ¼ ½

8 Î = Z ¢ Z Ú =´Ü; Ý µ Ò and deﬁne the following -regular graph G=(V,E) on the vertex set Ò . Each vertex is adjacent

to the four vertices

Ì Ú; Ì Ú; Ì Ú · e ;Ì Ú · e ;

½ ¾ ½ ½ ¾ ¾ Ú where all the calculations are performed mod Ò. The other four neighbours of are obtained by the four inverse

transformations. (Note that this is an 8-regular undirected graph, that may have multiple edges and self loops.)

´Gµ 5 ¾ < 8

Theorem 7.2. ¾

G d Ò

We have already seen that if is a -regular graph of size and eigenvalues (of its adjacency matrix) ½

::: = d h´Gµ ´d µ=¾

Ò ½ ¾

¾ ,then , then its Cheeger constant satisﬁes . Therefore, a lower bound on the gap

d Ò between ¾ and that is independent of implies that this is a family of expanding graphs. Margulis proved a similar result in 1973 but couldn’t give an explicit lower bound on the gap. Galil and Gabber (1981) used continuous harmonic analysis to derive a lowerbound on the gap. Boppana simpliﬁed the proof, and Jimbo, Marouka (1985) improved it furthermore using discrete Fourier transform.

As we have seen before, by the Rayleigh quotient Theorem,

ÑaÜfj j; j jg = ÑaÜfj j :=¼; jjf jj =½g

¾ Ò ½

= ¾ ÑaÜfj f ´iµf ´j µj :=¼; jjf jj =½g;

´i;j µ¾E

Ú = ½= Ò f : Z ! C ½

where ½ is the eigenvector corresponding to , and the maxima are taken over all . It follows,

f : Z ! C ´Üµ=¼

that it is sufﬁcient to prove that for every complex function satisfying f ,

X X

5 ¾

j f ´Üµf ´Ý µj jf ´Üµj :

´Ü;Ý µ¾E

By the deﬁnition of our graph, this is equivalent to:

f ´#µ=¼ ! C : Z

Theorem 7.3. For any f satisfying , the following inequality holds:

X X

¾ 5

jf ´Üµj j f ´#µ f ´Ì #µ·f ´Ì # · e µ·f ´Ì #µ·f ´Ì # · e µ j

½ ½ ¾ ¾ ¾

½ (7.1)

#¾Z Ò

44 CHAPTER 7. THE MARGULIS CONSTRUCTION

´Üµ=¼

Discrete Fourier transform It is hard to use the condition f so we will move to a new space using Ü

the discrete Fourier transform.

: G ! C G C

A character of an abelian group G is a homomorphism , mapping addition in to multiplication in .

k k k

Z : Z ! C b ¾ Z ´aµ=! ! Ò b

It can be seens, that the characters of are: b for ,where . Here, is the th

Ò Ò Ò

¾ i=Ò k

! = e a; b ¾ Z = a b i

root of the unity ( ), and for any their inner product is i .

j =½

f : Z ! C f = f ´bµ b

Since the characters b are an orthonormal basis, we can express any as ,where

X X

½ ½

f ´bµ ¡ ´aµ= f ´bµ ¡ ! : f ´aµ==

b b

k=¾ k=¾

Ò Ò

b b

k k

f : Z ! C : Z ! C

is called the Discrete Fourier Transform - (DFT) of f .

Ò Ò

To prove Theorem 7.3, we express the condition of equation (7.1) using the Fourier coefﬁcients of f . The condition

f ´#µ=¼ ´¼; ¼µ=¼

is equivalent to f . Using identities 2,4 in appendix 1, one can easily prove that Theorem 7.3 #

reduces to:

: Z Ò ´¼; ¼µ ! C

Theorem 7.4. Any function F must satisfy

X X

¾ 5

½ ½

¾ # #

¾ ½

jF ´#µj : µ]j #µ´½ · ! µ·F ´Ì #µ´½ · ! j F ´#µ ¡ [F ´Ì

½ ¾

¾ ¾

Ò´¼;¼µ #¾Z Ò´¼;¼µ #=´# ;# µ¾Z

½ ¾

Ò Ò

= jF j G G : Z ´¼; ¼µ ! Ê

Let G .Then is a real function . Moving the absolute value inside the summation, Ò

and using the equality

j½·! j =¾j cÓ×´ µj; Ò

it follows that it is sufﬁcient to prove:

: Z ´¼; ¼µ ! Ê

Theorem 7.5. Any function G , must satisfy

X X

5 ¾ # #

¾ ½

½ ½

j · G´Ì j] G ´#µ: #µj cÓ× G´#µ ¡ [G´Ì #µj cÓ×

½ ¾

Ò Ò 4 #

We would like to convert the LHS into a sum of squares that will match the RHS. To do this we use the following

inequality, valid for any real a; b;

¾ ¾

¾ab a · b :

¾ ¾ ¾

µ ! Ê :´Z Ü; Ý ¾ Z

Let be a function satisfying for all

Ò Ò

´Ü; Ý µ ¡ ´Ý; Üµ=½;

(7.2)

# =´# ;# µ ¾ Z Ò ´¼; ¼µ ¾

and in addition, that for all ½

# ¾ # 5

½ ¾

½ ½

j cÓ× j¡[ ´#; Ì #µ· ´#; Ì j¡[ ´#; Ì #µ· ´#; Ì : #µ] · j cÓ× #µ] ½

¾ (7.3)

¾ ½

Ò Ò ¾

¾ Z Ò ´¼; ¼µ

Since for any Ô; Õ

¾ ¾

¾G´ÔµG´Õ µ ´Ô; Õ µG ´Ôµ· ´Õ; ÔµG ´Õ µ;

it follows that

½ ½ ½

¾ ¾

#µ] #; #µG ´Ì ¾ ¡ ÄÀ Ë j cÓ× #µG ´#µ· ´Ì j¡ [ ´#; Ì

¾ ¾ ¾

½ ½ ½

¾ ¾

#µ]: #; #µG ´Ì #µG ´#µ· ´Ì · j cÓ× j¡ [ ´#; Ì

½ ½ ½ Ò

Ì # Ì #

¾ ¾ ½

Since ½ doesn’t change ,and doesn’t change , it follows that:

# #

½ ¾

½ ½

¾ ¾

¾ ¡ ÄÀ Ë G ´#µ ¡jcÓ× j¡[ ´#; Ì #µ· ´#; Ì #µ] · G ´#µ ¡jcÓ× j¡[ ´#; Ì #µ· ´#; Ì #µ]

¾ ½

Ò Ò

¾ 5

G ´#µ:

¾ #

Therefore, if we just ﬁnd a function satisfying the requirements (7.2) and (7.3), Theorem 7.2 would follow, and

¾ Z

we are done. To define , we first define a partial order on .So,let

Ü Ü ¼ ;

a´Üµ=

: Ò Ü Ü

if Ò

¼ ¼

a´Üµ Ò a´Ü ÑÓ d Òµ = a´Üµ ´# ;# µ > ´# ;# µ ¾

(Notice that is invariant under mod ,i.e. .) Then we say that ½ if

½ ¾

¼ ¼

a´# µ a´# µ a´# µ a´# µ ¾

½ and and at least one of the inqualities is strong. That is the distance to the X axis and/or

½ ¾ to the Y axis is bigger.

The deﬁnition of is:

¼ ¼

; ´# ;# µ > ´# ;# µ « ¾

if ½

½ ¾

¼ ¼

´´# ;# µ; ´# ;# µµ =

µ ;# ; ´# ;# µ < ´#

½ ¾

½ ¾ if

¾ ½

: ½

otherwise:

« = # ¾

This definition of obviously satisfies (7.2). We will show that for , also (7.3) is satisfied for any

¾ ¾

Z # =´# ;# µ ¾ Z Ò ´¼; ¼µ Ò ´¼; ¼µ ¾

. We deﬁne the diamond to be the set of all ½ , satisfying

Ò Ò

a´# µ·a´# µ :

½ ¾ ¾

θ 2

(n/2,0)

(-n/2,0) (n/2,0) θ 1

(0,-n/2)

Figure 7.1: The diamond

We analyze the case where # is outside or inside the diamond separately.

¯ Outside the diamond

(7.3) will follow from the fact that the cosines are small. So, it is sufﬁcient to prove that

# ¾ # 5

½ ¾

¾« ¡ ´j cÓ× j · j cÓ× jµ ;

Ò Ò ¾

or that

# #

½ ¾

cÓ× j · j cÓ× j ¾:

j (7.4)

Ò Ò

46 CHAPTER 7. THE MARGULIS CONSTRUCTION

cÓ× #

We can assume w.l.o.g. that we are in the ﬁrst quadrant. Since is decreasing, for any given value of ½ ,

# = Ò=¾ # cÓ× ½

the LHS of (7.4) is maximized, when ¾ . Therefore, using the convexity of we get that

# µ ´

# # #

½ ¾ ½

cÓ× ¾: · cÓ× cÓ× · cÓ× ¾ cÓ× =

Ò Ò Ò Ò 4 .

¯ Inside the diamond

In this case, we just bound the cosines by 1, and would like to prove that

5 ¾

½ ½

´#; Ì #µ· ´#; Ì #µ· ´#; Ì #µ· ´#; Ì #µ : ¾

½ (7.5)

½ ¾

# a´# µ·a´# µ < ¾

It is not difﬁcult to verify that for every satisfying ½ , one of the following two cases must ¾

hold:

½ ½

Ì # Ì # Ì # Ì # ># <# ¾

1. Three of the four points ½ , , and are and one is .

½ ¾

½ ½

Ì # Ì # Ì # Ì # ># # ¾

2. Two of the four points ½ , , and are and two are incomparable with .

½ ¾

¿ ¾

« =5=4 « ·¾

In the ﬁrst case, the LHS of (7.5) is · , while in the second case it is . Substituting ,this

« «

:65 5 ¾=¾= ¿:5¿ ::: gives a upper bound of ¿ or the LHS of (7.3). This is not as good as , but it does prove that the graphs are a family of good expanders.

Appendix

Using the same deﬁnitions as before the following properties of the discrete Fourier transform can by easily concluded

from the deﬁnitions:

´aµ=¼¸ f ´¼µ=¼

1. f a

2. The dot product of two functions remains invariant under the transform i.e for any f; g

X X

f áµgbáµ: f áµg áµ=

a a

= g

3. Parseval’s identity: A special case of 2, where f

X X

¾ ¾

jf ´aµj = jf ´aµj :

a a

k ¢ k Z b ¾ Z g ´Üµ=f ´AÜ · bµ

4. The shift property : If A is a non-singular matrix with entries over Ò, and Ò

then

½ Ø

f ´´A µ Ý µ: bg ´Ý µ=!

5. The inverse formula :

Notes taken by Eyal Bigman Summary: In this lecture we shall introduce a new kind of graph product called the Zig Zag Product.We shall analyze the expansion properties of the zig zag product of expanding graphs and use them to create an explicit recursive construction of a family of good expanders. Furthermore we will connect the entropy of a random walk on the graph and its expansion.

8.1 Introduction

=<Î;E > d jÎ j = Ò

We begin with some standard deﬁnitions: Let G be a -regular graph ( ). The adjacency matrix

A´Gµ=´a µ G Ò ¢ Ò a = k ij

ij of the graph is a symmetric non-negative integer matrix such that iff there are

i;j =½

G = A i j d

k edges between vertices and . It follows from regularity that the sum of every row is . The matrix is

Ô ¾<

the normalized adjacency matrix, it is the transition matrix of a random walk on G. Thus if is a probability

GÔ Ø ·½

distribution on the vertices at time Ø then is the distribution at time .

´Ò; d; «µ «

G is a -expander if is an upper bound on the second eigenvalue of the normalized adjacency matrix

« =½ ´½ «µd=¾ h´Gµ h´Gµ ½

¾ ( ). It follows from the spectral gap theorem that where is the expansion of

G « ¼

G. Thus is a better expander when is close to .

¾ ¼ ¼

= ´Î; E µ ´Ù; Û µ ¾ E G

The square G is a graph on the same vertices and iff there is a path of length 2 in

¾ ¾ ¾

Û A G A G G

from Ù to .If is the adjacency matrix of then is the adjacency matrix of . It is easy to see that is a

The zig zag product will be an unsymmetric binary function, that given an Ñ-regular graph of size and a -regular

d ÑÒ

graph of size Ñ, it yields a -regular graph of size . After we deﬁne the zig zag product we will prove:

´Ò; Ñ; «µ À ´Ñ; d; ¬ µ G À

Theorem 8.1 (The Zig Zag Theorem). Let G be a -expander and be a -expander then z

¾ ¾

ÒÑ; d ;f´«; ¬ µµ f ´«; ¬ µ=« · ¬ · ¬ is a ´ -expander where .

We will present a proof that uses only elementary linear algebra. With some more algebra this bound is improved

´« · ¬ µ=« · ¬ «; ¬ < ½ f ´« · ¬ µ < ½ in [RVW02] to f ,andif , it can be shown .

Before we proceed with the deﬁnition let us show how it can be used for an explicit construction of a family of

µ À ´d ;d;

expanders with constant degree. For d constant let be a -expander, there is a probabilistic proof that such 4

an expander exists and since d is constant we can ﬁnd such a graph by an exhaustive search in constant time, there are

also efﬁcient constructions of such graphs - see [RVW02]. Deﬁne recursively:

G = À

À =´G µ

G z

Ò·½ Ò

4Ò ¾

G ´d ;d ; Ò µ

Proposition 8.2. Ò is a -expander for all

=½ Ò G

Proof. By induction. The case Ò follows from the deﬁnition. If we assume the proposition for then is a

½ ½

4Ò 4 4´Ò·½µ ¾

µ µ ´d ;d ; G ´d ;d ; ·½

-expander and from theorem 8.1 it follows that Ò is a graph

4 ¾

47 48 CHAPTER 8. THE ZIG ZAG PRODUCT

8.2 The Construction

½ Ñ

´Ò; Ñ; «µ À ´Ñ; d; ¬ µ Ú ¾ Î ´Gµ e ;:::e

Let G be a -expander and be a -expander. For every vertex let be the

Ú Ú

½;:::;Ñ [Ñ]

edges connected to the vertex. Also, we regard the vertices of À as the integers , denoted by . To obtain

À Ú Ñ ´Ú; ½µ;:::´Ú; Ñµ

G z , we replace every vertex with a cloud of vertices one for every edge connected. The G

vertices within a cloud are connected by “miniedges” so that every cloud forms a mini copy of À . The edges of are

augmented on both sides by these miniedges to form the edges of G z . More formally:

¼ ¼

À =<Î´Gµ ¢ [Ñ];E > ´´Ú; iµ; ´Ù; j µµ ¾ E k; Ð ¾ [Ñ]

Deﬁnition 8.3. G z ,where iff there are some such that

k Ð

i; k µ; ´Ð; j µ ¾ Î ´E µ e = e

´ and .

Ú Ù À

It is essential of course for the sake of well deﬁnedness that the degree of G equals the size of .

À ÑÒ d

Proof of the Zig Zag theorem. It is easy to see that G z is a graph of size and degree . The expansion constant

of G z is a bound on the second eigenvalue of the transition matrix of the random walk on the graph. Each step of

the random walk on an edge of G z can be regarded as a random step on a miniedge within a cloud, a deterministic

Ú; iµ

step on an edge connecting two clouds and another random step within a cloud. Thus, the transition from ´ to

Ù; j µ ´Ú; iµ ´Ú; kµ k ¾ [Ñ] ´Ú; kµ ´Ù; Ð µ

´ consists of a random step from to for some , a deterministic step from to and

Ù; Ð µ ´Ù; j µ

another random step from ´ to .

The transition matrix of the random walk on G z will therefore be the product of the transition matrices of these

~ ~ ~ ~ ^

G À = À G À = À ª Á z À

three steps, i.e. ,where Ò is the transition matrix of a random step in each cloud, and

k Ð

e = e

½ if

Ú Ù

G =

´Ú;kµ;´Ù;Ðµ

¼ otherwise

~ ~

À k; k G k ½

is a matrix of transpositions. It is easy to see that k .

À ½ G z

is a regular graph thus the constant vector ÑÒ is an eigenvector, it follows from Rayleigh’s theorem that

\ \

jf G Àf j jf G Àf j

z z ¾

= ÑaÜ f ? ½ « · ¬ · ¬

¾ f ?½ ÑÒ ¾

¾ . Therefore it sufﬁces to show for every .

ÑÒ

kf k kf k

k ? k ?

~ ~

f f = f · Àf f

We can write any vector as ,where is a vector that is constant within each cloud, and Àf

k k

½ À Àf = f

sums up to zero within each cloud. Since Ñ is an eigenvector of with eigenvalue one, we get that .

f ? ½

Therefore, for every ÑÒ :

~ ~ ~

f G Àf j = jf À GÀf j

j z

k k k ? ? ?

~ ~ ~ ~ ~ ~ ~ ~ ~

= jf À GÀf j ·¾jf À GÀf j · jf À GÀf j

k k k ? ? ?

~ ~ ~ ~ ~ ~

= jf Gf j ·¾jf GÀf j · jf À G Àf j

k k k ?

f ? ½ k Gf k « k f k f

Now, since ÑÒ , we know that . Also, since sums up to zero in each cloud, we have

? ?

Àf k ¬ k f k

k . Therefore:

k k k ¾

jf Gf j« k f k

k ? k ? k ?

~ ~ ~ ~

jf GÀf jkG k¡kf k¡ k Àf k ¬ k f k¡kf k

? ? ? ¾ ¾ ? ¾

~ ~ ~ ~ ~

jf À GÀf jkG k¡kÀf k ¬ k f k

The inequalities follow from Cauchy Schwartz and the deﬁnition of the operator norm.

k ¾ k ? ¾ ? ¾

f G Àf > j« k f k ·¾¬ k f k¡ kf k ·¬ k f k

j z

¾ k ? ¾ ¾

« k f k ·¾¬ k f k¡ k f k ·¬ k f k

¾ k ¾ ? ¾ k ? ¾ ¾ ¾

= « k f k ·¬ ¡ ´k f k · k f k ´k f k kf kµ µ·¬ k f k

¾ ¾ k ? ¾

=´« · ¬ · ¬ µ¡kf k ¬ ¡ ´k f k k f kµ

¾ ¾

´« · ¬ · ¬ µ¡kf k 8.3. ENTROPY ANALYSIS 49

8.3 Entropy Analysis

Ô À ´Ôµ= Ô ÐÓg´Ô µ i

There are several different deﬁnitions of entropy for distribution , the classical deﬁnitions i ,

i=½

À À ´Ôµ= ÐÓg´k Ô k µ À ´Ôµ= ÐÓg´k Ô k µ A

¾ ¾ ½ ½

the ¾ entropy and the min-entropy . For any transition matrix

« Ô = ½ · f

with expansion constant and distribution Ò

½ ½

¾ ¾ ¾ ¾ ¾ ¾ ¾ ¾

½ · Af k k ½ k · k Af k ´´½ µ·« µ k Ô k k AÔ k =k

Ò Ò

kf k

= ½

where . Hence

kÔk

¾ ¾ ¾ ¾ ¾

À ´AÔµ=À ´Ôµ · ÐÓg´´½ µ·« µ=À ´Ôµ · ÐÓg´½ ´½ « µ µ=À ´Ôµ ¡

¾ ¾ ¾ ¾ E

¾ ¾ ¾ ¾ ¾ ¾

¼ < ½ « ½ ¼ ½ ´½ « µ < ½ ¡ = ÐÓg´´½ µ·« µ > ¼ >¼

implies hence E as long as .This ¡

shows that the entropy increases by at least E as long as there is a positive nonuniform component. It follows that

« À À À

¾ ½ for better expanders ( smaller) the ¾ entropy grows faster. It can be shown that the and the are correlated,

therefore the same increase is true for the min-entropy. ¼ We will show next that the classical entropy also grows when > but we shall make very different considera- tions. It is currently unknown how fast the entropy grows and how the growth rate relates to the expansion constant. We note that for all the deﬁnitions of entropy, the increase of entropy for nonuniform distributions is essentially the

second law of thermodynamics.

Y À ´X; Y µ=

For random variables X and with some joint distribution we have the standard entropy equation

´X µ·À ´Y jX µ X Y X

À thus the entropy of the joint distribution of and is the sum of the entropy of and the

jX Î ¢ [Ñ]

conditioned entropy of Y . In the case of the zig zag graph the set of vertices is , the random variables we Y

will analyze will be the projections X and to the ﬁrst and second coordinates. We will see that a random step on the

À ´Y jX µ zig zag graph will increase both the entropy of X and the conditioned entropy and thus increase the entropy of the joint distribution. As we mentioned before a random step consists of a random step within a cloud (zig) a deterministic step between

clouds and another random step within a cloud (zag). Since the zig and the zag steps are within a cloud, they only

jX affect the second coordinate. They are random steps therefore they increase the conditioned entropy of Y as long

as that is less than maximal. For a distribution which is uniform on every cloud the zig and zag steps have no effect. Y We can think of X and as basins of entropy, taking either a zig or a zag step is like pouring entropy from an external source into the basin. How much entropy can be poured in? Well that depends on the capacity of the random variable (maximum entropy) and the amount of entropy present in the basin, i.e. how far is the distribution on every cloud from uniform. What about the deterministic step between the clouds? Well this step is deterministic and therefore it does not

change the total amount of entropy in the system, but that does not mean it cannot change the division of the entropy Y

between X and .

½ Ô = e ª X Y Ñ

Let us think of the extreme case Ú where the entropy of is zero and the entropy of is maximal,

Ñ Ô

i.e. the distribution is concentrated uniformly on the vertices of the cloud Ú (obviously remains unchanged by the

´Y jX µ = ÐÓg´Ñµ À ´X; Y µ=À ´X µ· À ´Y jX µ = ¼ · ÐÓg ´Ñµ zig step). In this case À and . After the deterministic

step there is equal probability to be in any one of Ú ’s neighboring clouds, but within these clouds the distribution ¼

is concentrated on the vertex that is actually connected to a vertex in Ú . The entropy in these clouds is .The Ú

entropy of clouds not connected to Ú remains zero, and the entropy in is zero. Thus we see that the entropy of

ÐÓg´Ñµ Y

X goes from zero to and the entropy of plummets to zero. We still have the same entropy in the system

´X; Y µ=À ´X µ·À ´Y jX µ = ÐÓg´Ñµ·¼ X Y Y

À , but the division between and changes, all the entropy of is passed

X Y

to X . In the general case the same thing happens, entropy is exchanged between and . Since deterministic steps X

are reversible, not necessarily is the entropy transfered from Y to , or from a variable with high entropy to a variable

with low entropy. Nevertheless, it can be shown that if the entropy of Y is maximal and the joint distribution is

not uniform then the deterministic step reduces the entropy of Y . Thus we see that the deterministic step does not X

change the entropy in the systems but pours entropy from one vessel to another, and in the case that Y is full and is X not, some entropy is poured from Y to .

We can see now the effect of a random step, ﬁrst entropy is poured into Y , if it is not full then some entropy Y is added to the joint distribution. Next entropy is exchanged between X and , necessarily making room for more

50 CHAPTER 8. THE ZIG ZAG PRODUCT

X Y entropy in Y if it is full and is not. Finally yet more entropy is poured into and again if it is not full already then the entropy of the joint distribution is increased. Thus we see that the total amount of entropy in the system necessarily increases unless the two basins are full

already. Why do we have to pour in entropy from an external source twice? Because in cases like Ô above, whatever

entropy was poured while Y was full will be wasted and the entropy of the whole system will remain the same. Only

the second step will make room in Y for more entropy, thus if we avoid the zag step the entropy in the system will

not increase. On the other hand, since the deterministic step is reversible it could also be the case that Y will be ﬁlled

from X and thus the entropy poured in the zag step will be wasted. It can be shown (or follows immediatly form Y reversability) that in such a case Y is not full from the start. Thus if we don’t pour entropy on at the beginig it may not be possible later. Therefore both the zig and the zag steps are essential in order to assure that the entropy will actually increase. Chapter 9

Metric Embedding

Notes taken by Tamir Hazan Ò

Summary: We can embed any metric space into Ê , but with some distortion of the distances. We show that the graph metric of expander graphs is the hardest metrics to embed, in the sense that of all ﬁnite metric spaces on a given number of points, expanders require the largest distortion.

9.1 Basic Deﬁnitions

X; dµ

´ is a metric space if

d : X ¢ X ! Ê

¯ .

d´Ü; Ý µ=¼ Ü = Ý

¯ if and only if .

d´Ü; Ý µ=d´Ý; Üµ

¯ .

d´Ü; Ý µ d´Ü; Þ µ·d´Þ; Ýµ

¯ .

´X; dµ Ð Ð ¾

In this lecture we will examine how to approximate a ﬁnite metric by the metric space ¾ . is the metric

¾ Ò Ò ¾

´Ý Þ µ ´Ê ; k¡ kµ Ý; Þ ¾ Ê jjÝ Þ jj = i

space such that for every , i .

i=½

Ò Ò

´X; dµ ´Ê ;Ð µ f : X ! Ê

Given the metric spaces and ¾ and a transformation we deﬁne:

kf ´Ü µ f ´Ü µk

½ ¾

¯ eÜÔaÒ×iÓÒ´f µ= ÑaÜ

Ü ;Ü ¾X

½ ¾

d´Ü ;Ü µ

½ ¾

d´Ü ;Ü µ

½ ¾

¯ cÓÒØÖ acØiÓÒ´f µ=ÑaÜ

Ü ;Ü ¾X

½ ¾

kf ´Ü µ f ´Ü µk

½ ¾

¯ di×ØÓÖ ØiÓÒ´f µ=eÜÔaÒ×iÓÒ´f µ ¡ cÓÒØÖ acØiÓÒ´f µ

f½; ¾; ¿; 4g;dµ

It is clear that there are metric spaces that need to be embedded with distortion. E.g the metric ´ with

´½; 4µ = d´¾; 4µ = d´¿; 4µ=½ d´i; j µ=¾ f½; ¾; 4g f½; ¿; 4g f¾; ¿; 4g

d ,and otherwise, since , , must be on the same Ò line in Ê .

9.2 Finding the Minimal Distortion

´X; dµ

In this section we will present some of the properties of embeddings in Ð . Given a metric space we denote its

C ´X; dµ

minimal distortion by ¾ .

X; dµ Ç ´ÐÓg Òµ

Theorem 9.1. Bourgain (1985) Any n-point metric space ´ can be embedded into dimensional Eu-

´ÐÓg Òµ

clidean space with Ç distortion.

C ´X; dµ

Theorem 9.2. Linial, London, and Rabinovich (1995) There is a polynomial time algorithm which computes ¾ .

52 CHAPTER 9. METRIC EMBEDDING

´X; dµ jX j = Ò f : X ! Ð

Proof. The proof is based on semi deﬁnite programming. Let be a metric space with .Let ¾ .

´f µ=½ di×ØÓÖ ØiÓÒ´f µ

Since we can assume that without loss of generality that cÓÒØÖ acØiÓÒ ,then if and only if

i< j Ò

for every ½ :

¾ ¾ ¾ ¾

´£µ d´Ü ;Ü µ kf ´Ü µ f ´Ü µk d´Ü ;Ü µ

i j i j i j

ÈËD Z Ü ZÜ ¼

We say that a matrix Z is positive semi deﬁnite (denoted )if is symmetric and for every

¾ Ê

Ü .

Ù = f ´Ü µ Í Z = Í ¡ Í Z ¾ ÈËD i

Let i be the rows of the matrix .Let . It is clear that since it is symmetric, and

Ò Ì Ì Ì Ì Ì Ì Ì ¾

¾ Ê Ü ZÜ = Ü Í ¡ Í Ü =´Í Üµ ¡ ´Í Üµ=kÍ Ük ¼

for every Ü , .

¾ ÈËD Z = Í ¡ Í Í Z

But also the converse is true, if Z then for some matrix . To see that, note that is

= A£A A £ £

symmetric, and therefore diagonizable. Thus Z for some matrix and a diagonal matrix .Let be the

Ô Ô Ô Ô Ô

Ì Ì

´ £µ = ´£µ Z = A £ £ A =´A £µ ¡ ´A £µ ii

diagonal matrix which ii .Then .

Ù = f ´Ü µ ´£µ Z ¾ ÈËD i

Therefore instead of finding i which satisfies we can find a that satisfies

¾ ¾ ¾

´££µ d´Ü ;Ü µ Þ · Þ ¾Þ d´Ü ;Ü µ ;

i j ii jj ij i j

¾ Ì

kÙ Ù k = Þ · Þ ¾Þ Z = Í ¡ Í

j ii jj ij

since i for .

C ´X; dµ Z

Thus we conclude that ¾ if and only if there is a positive semi deﬁnite matrix such that for all

i; j (**) holds. This is a linear programming problem (more precisely a convex programming problem) which can be solved in polynomial time by the ellipsoid algorithm.

The algorithm above constructs a primal problem and solves it by the ellipsoid algorithm. Looking at the dual

C ´X; dµ

problem gives us an interesting characterization of ¾ . When we transform a primal problem to its dual we take

¾ ÈËD

a non negative combination of its constraints. But how do we look at the constraint Z ?

Z ¾ ÈËD É ¾ ÈËD Õ Þ ¼ ij

Lemma 9.3. if and only if for all , ij .

i;j

É É = Õ ¡ Õ ÈËD

i j

Proof. Let be a matrix such that ij . Previously we showed that such a matrix is . Therefore

Õ ZÕ = Õ Þ ¼ Z ¾ ÈËD ij

ij implying that .

i;j

É ¾ ÈËD É = Õ ¡ Õ Õ ; :::; Õ

i j ½ Ò

It can be easily seen that any of rank 1 must have the form ij for some values .

Z ÈËD É ¾ ÈËD Õ Þ ¼ ij

Thus, if is then for every symmetric matrix of rank 1, ij . The lemma follows

i;j ÈËD

from the fact that any ÈËD matrix is a non negative linear combination of rank 1 matrices. To see this, note

¾ ÈËD A£A £ È

that any È can be written as ,where is a diagonal matrix of the same rank as . Therefore

Ö aÒk ´È µ

A£ A È =

ii .

i=½

Our primal problem is:

¯ Õ Þ ¼ É ¾ ÈËD ij

ij for all .

¯ d´Ü ;Ü µ Þ · Þ ¾Þ i; j

j ii jj ij

i for all .

¾ ¾

¯ Þ · Þ ¾Þ d´Ü ;Ü µ i; j

jj ij i j

ii for all .

C ´X; dµ

We proceed by deriving an explicit formula for ¾ from the dual problem:

Theorem 9.4. Ú

Ô d´Ü ;Ü µ

ij i j

Ô >¼

C ´X; dµ= ÑaÜ

~ ~

Ô d´Ü ;Ü µ

È ¾ÈËD;È¡½=¼

ij i j

Ô <¼

Proof. In the following proof we shall assume that ¾ . Therefore when we inspect a non negative combi-

nation of the constraints of the primal problem (the dual problem) we must get a contradiction.

É ¾ ÈËD <Õ;Þ>= Õ Þ ¼ ij

Let us look at the constraints of the ﬁrst type (for all , ij ). Since the collection

a < Õ; Þ > ÈËD < Ô; Þ >

of matrices is convex, a non negative combination k is equal to for some matrix

Ô ¾ ÈËD Ô Þ ¼ È ¾ ÈËD ij

. Thus the combination of the ﬁrst type constraints gives us ij for some .

> ¼

A contradiction can be reached if a combination of all the constraints result in ¼ . Unfortunately so far we have

Ô Þ ¼ È ¾ ÈËD Þ i 6= j

ij ij

ij for some . So to eliminate the for , we take the following linear combination of ij the rest of the constraints: Ä

9.3. EMBEDDINGS IN ¾ 53

¯ Ô =¼ Þ kÐ

If kÐ then we multiply the constraints involving by zero.

¯ Ô > ¼ Ô =¾ d´Ü ;Ü µ Þ · Þ ¾Þ

kÐ k Ð kk ÐÐ kÐ

If kÐ then we multiply by the constraint

¾ ¾

Ô < ¼ Ô =¾ Þ · Þ ¾Þ d´Ü ;Ü µ ¯

kÐ kk ÐÐ kÐ k Ð

If kÐ then we multiply by the constraint

Þ È

To eliminate ii , we have to choose such that

Ô · Ô =¼:

ii ij

j 6=i

Therefore the combination of all the constraints gives

X X

¾ ¾ ¾

Ô d´Ü ;Ü µ ¼: ´£µ Ô d´Ü ;Ü µ ·

ij i j ij i j

Ô <¼ Ô >¼

ij ij

£µ

We get our contradiction if ´ is violated. Thus we conclude the theorem. Ð

9.3 Embeddings in ¾

9.3.1 Embedding the cube

Ö Ö

f¼; ½g Ð Ö id : f¼; ½g !

Given a cube we can easily ﬁnd an embedding in ¾ with distortion . Given the embedding

id´Üµ=Ü cÓÒØÖ acØiÓÒ´idµ= Ö eÜÔaÒ×iÓÒ´idµ= ½

Ê such that , we get that and . Using our main theorem we can

Ö Ö

¢ ¾ È

show that this is the best embedding of the cube. Let deﬁne the ¾ matrix :

È ´i; j µ= ½ d´i; j µ=½

¯ if .

È ´i; j µ= Ö ½ i = j

¯ if .

È ´i; j µ= ½ d´i; j µ=Ö

¯ if .

È ´i; j µ= ¼

¯ otherwise.

½ = ¼ È ¾ ÈËD È

It is easy to check that È ,andthat (the later holds, since has the same eigenvectors as the

È È

¾ Ö ¾ Ö ¾

C ´X; dµ Ö Ô d´Ü ;Ü µ =¾ ¡ Ö Ô d´Ü ;Ü µ =¾ ¡ Ö

¾ ij i j i j

cube). Since ij ,and , we get that .

Ô <¼ Ô >¼

ij ij

9.3.2 Embedding expander graphs

G =´Î; Eµ jÎ j = Ò k ¯

Let be a k-regular graph, , with ¾ . As before it is simple to see that an expander can

Ç ´ÐÓg Òµ Ð Ê

be embedded with distortion in ¾ . Indeed, take the expander and put it as a simplex in . Since every

eÜÔaÒ×iÓÒ =½ cÓÒØÖ acØiÓÒ = diaÑ´Gµ G

two nodes of the simplex have distance ½ we get that ,and .Since is an

´Gµ=Ç ´ÐÓg Òµ

expander then diaÑ . As before this result is tight.

=´Î; E µ G À

Lemma 9.5. Let À be a graph with the same vertex set as . 2 vertices are adjacent in if their distance

ÐÓg Ò ¾ À B Ò=¾

in G is at least .Then has a matching of edges.

k Ö Ö = ÐÓg Ò ¾

Proof. G is k-regular graph thus every vertex has at most vertices at distance from it. If then

¾ À Ò=¾ there are at most Ò= nodes at this distance. Since all the vertices of have degree then it has a matching of the desired size. This follows from Dirac’s sufﬁcient condition for a Hamiltonian circuit. (Modern Graph Theory B.

Bollobas p. 106-107).

G =´Î; Eµ jÎ j = Ò k ¯ C ´Gµ = ª´ÐÓg Òµ ¾ Theorem 9.6. Let be a k-regular graph, , with ¾ .Then .

54 CHAPTER 9. METRIC EMBEDDING

À B

Proof. Let B be the adjacency matrix of the matching we found in . For simplicity we assume that is a complete

~ ~

´B Á µ À È = kÁ A · À È ½= ¼ È ¾ ÈËD Ü?½

matching in .Let G in . It is easy to see that . since for every ,

Ì Ì Ì

Ü ÈÜ = Ü ´kÁ A µÜ · Ü ´¯=¾µ´B Á µÜ

Ì ¾

G Ü ´kÁ A µÜ ¯jjÜjj

G since is expander.

X X

Ì ¾ ¾ ¾ ¾ ¾

Ü ´B Á µÜ = ´¾Ü Ü Ü Ü µ ¾ ´Ü · Ü µ=¾jjÜjj

i j

i j i j

´i;j µ edgeiÒB

C ´Gµ

To get the lower bound on ¾ we note that

¾ ¾

d´i; j µ Ô ¡ Ò´ÐÓg Ò ¾µ

Ô >¼

ÐÓg Ò ¾

since the distances of the entries in B are at least .

d´i; j µ Ô = kÒ

Ô <¼

C ´Gµ = ª´ÐÓg Òµ

Thus ¾ . Chapter 10

Error Correcting Codes

Notes taken by Elon Portugaly

¼; ½g Summary: An error correcting code is set of words in f . Its distance is the minimal Hamming distance between two codewords. Therefore, if we transmit a codeword through a noisy channel that ﬂips some of the bits, then we can correct the errors, as long as the number of bits ﬂipped is bounded by half the distance. There is a trade-off between the size of the code and the number of errors it can correct. There are lower bounds (Gilbert Varshamov) and upper bounds (The Balls Bound, MRRW) on the size

and correction capabilities of codes.

¼; ½g A linear code is an error correcting code that is also linear subspace of f . Expanders can be used to build error correcting codes, that have large size and distance. These codes are also efﬁciently decodable.

10.1 Deﬁnition of Error Correcting Codes

f¼; ½g

Deﬁnition 10.1. A Code is a set C .

C d i×Ø´C µ ÑiÒ d ´Ü; Ý µ d ´Ü; Ý µ À

Deﬁnition 10.2. The distance of is d À ,where is the hamming distance

Ü6=Ý

Ü;Ý ¾C

Ý Ü Ý

between Ü and (the number of coordinates and differ on).

ÐÓg jC j

Ö aØe´C µ

Deﬁnition 10.3. Therateof C is r .

C j d When deﬁning a code, we desire both j and to be as large as possible.

10.1.1 Motivation The setting we look at, is as follows:

We would like to send information through a noisy channel, that may ﬂip some of the bits we send. We code our

f¼; ½g information using a set of words C that we transmit through the channel, and assume that the number of bits the channel may ﬂip in the transmitted word is bounded from above.

In this setting, it is clear that we would like the number of codewords available to be as large as possible, therefore

C j we want j to be large. We also would like to be able to reconstruct the codeword that was sent from the corrupted

codeword received. Therefore, we would like that no two codewords could appear the same after they have been

d>¾k corrupted by the noise. If the number of bits the channel can ﬂip is limited by k ,and , the last requirement is

fulﬁlled. When d is larger, we can deal with noisier channels.

C j C ÐÓg jC j Ò

j strings can be encoded using . Therefore, we can transmit information bits, using channel bits.

jC j

This achieves channel utilization of ÐÓg , which is the rate of the code. Ò Note: A Code refers to the set of codewords and not to the process of encoding/decoding. However, in any practical aplication, we would like the code to be efﬁciently encodable and decodable.

55 56 CHAPTER 10. ERROR CORRECTING CODES

10.2 Asymptotic bounds

10.2.1 A lower bound: Gilbert Varshamov

This bound shows that good codes can be built.

d Theorem 10.4. One can build a length Ò code with distance and size . volume of a radius d hamming ball

Proof. Following is an exponential time greedy algorithm that builds such a code:

Ë = f¼; ½g ;C = ;

¯ Initiate . Ë

¯ Repeat until is empty:

¾ Ë

– Pick any point Ü , and add it in the code.

d Ü

– Remove all the points in Ë that are within distance from .

¡ ¡

È È

d d

Ò Ò

f¼; ½g

Analysis: The volume of a hamming ball of radius d in is . Therefore, at most points

i=¼ i=¼

i i

Ë ¾

are removed from Ë in each iteration, and since the number of points in at the beginning is , the number of

Ò Ò

¾ ¾

C Ò

iterations and thus the size of at the end of the process must be at least d .

volume of a radius d ball

´ µ

i=¼

¡ ¡

Ò Ò

d Ò

ÒÀ ´Æ µ

Æ d À ¾

Deﬁne .For we have ,where is the binary entropy function

i=¼

Ò ¾ i ÆÒ

À ´Üµ= Ü ÐÓg Ü ´½ Üµ ÐÓg ´½ Üµ:

Therefore

ÖÒ Ò´½ À ´Æ µµ

¾ ¾ ;

ÆÒ

Ö ½ À ´Æ µ implying that for large Ò, the rate of the above code satisﬁes .

balls bound

GV MRRW

1/2 1 δ

Figure 10.1: Illustrating Upper and Lower Bounds on rate vs. the relative distance 10.3. LINEAR CODES 57

10.2.2 An Upper Bound - The Balls Bound

Ò d C jC j

Theorem 10.5. For any length , distance code we have d .

volume of a ball of radius

½ À ´Æ=¾µ

This theorem implies the asymptotic bound: Ö .

C d=¾ C

Proof. Given a distance d code , a draw ball of radius around each of the points of . If any two balls intersect, d then the distance between them is smaller than d, which contradicts the deﬁnition of . Therefore, the balls are disjoint, and their number is limited by the overall volume divided by the volume of each ball.

A much stronger upper bound was shown by McEliece, Rodemich, Rumsey and Welsh 77 (MRRW). The relations between the three bounds is illustrated in Figure 10.1.

10.3 Linear Codes

f¼; ½g

A linear code is a code that is a linear subspace of the Ò-dimensional space . (In other words, it is closed under Ò coordinate-wise addition mod ¾.) Such codes have a polynomial size (in ) description. (For instance by specifying a basis.) The Gilbert Varshamov algorithm can be modiﬁed to generate linear codes, and thus the Gilbert Varshamov bound

applies to linear codes. Obviously, any general upper bound applies to linear codes as well. ¼

Observation 10.6. Since any linear code C must include the codeword, then for linear codes,

´C µ= ÑiÒ eig hØ´Üµ;

d i×Ø w

¼6=Ü¾C

´Üµ Ü where weig hØ is the number of non-zero coordinates of .

Note: Although, for a linear code, the encoding can be done in polynomial time, the decoding is in general Ü

NP-hard. I.e., given a linear code C (using some reasonable representation), and a vector , the problem Ü of ﬁnding the element of C that is closest to is an NP-hard problem.

10.4 Using Expanders to generate Error Correcting Codes

10.4.1 Deﬁning Codes using Bipartite Graphs Ñ

Consider a bipartite graph with Ò vertices on the left and vertices on the right side. We call the vertices on left ½ variables, and the vertices on the right constraints. Each variable may assume the value of ¼ or , and we say that a

constraint is satisﬁed if the sum of all the variables adjacent to it is zero mod ¾.

Example 10.7. The graph in Figure 10.2 illustrates such a variables and constraints graph. For this graph, constraint

4 Ü¿·Ü6·Ü9·Ü½4 = ¼ Ý ½¼ Ü½¾ · Ü½4=¼ Ý is satisﬁed iff while constraint is satisﬁed iff . Note that the same

variable may appear in more than one constraint.

¾f¼; ½g Ü

To deﬁne the code we refer to the variables as the coordinates of a vector Ü . A vector is in the code iff

´Gµ G

it satisfies all the constraints defined by the graph. We denote by C , the code defined by . The code then, is the

Ò Ñ

Ò jC j¾ Ö

set of all solutions of Ñ linear equations on variables. Therefore, or . Ò 58 CHAPTER 10. ERROR CORRECTING CODES

variables constraints

x1 x2 y1 x3 y2 x4 y3 x5 y4 x6 y5 x7 y6 x8 y7 x9 y8 x10 y9 x11 y10 x12 y11 x13 y12 x14 x15

Figure 10.2: A Variables and Constraints Graph

10.4.2 Codes Using Left Side Expanders

G =´Î ; Î ;Eµ jÎ j = Ò jÎ j = Ñ

Ê Ä Ê

Theorem 10.8. Sipser and Spielman (95): Let Ä be a bipartite graph of size , ,

jË j k Ë Î «Ò j ´Ë µj >

that is -regular on the left. Assume furthermore, that for any Ä of size at most , .Then

´C ´Gµµ >«Ò d i×Ø .

Proof.

Ë Î «Ò Ý ¾ Î Ê

Lemma 10.9. Every set Ä of size at most satisﬁes the Unique Neighbor Property. I.e, there exists

´Ý µ \ Ë j =½

such that j

Ë Î «Ò Ë Ý ¾

Proof: Consider a set Ä of size .IftheUnique Neighbor Property does not hold for , then each

Ë µ Ë Ë ¾ ´Ë µ > ¾ jË j = k jË j

´ has at least two neighbors in . Therefore the number of edges leaving is at least , ¾

which contradicts the left regularity of G.

´C µ «Ò

Assume for the purpose of contradiction that di×Ø . Then there exists a nonzero codeword whose weight

Û X Ú

is at most «Ò (Observation 10.6). Let be such a codeword, and be its support (the set of coordinates where

Ü =½ Ý ¾ ´X µ X

Ú ), and let be a vertex guaranteed by the Unique Neighbor Property for . The constraint deﬁned by

Ý Û Ý is that the sum of all the variables that are neighbors of is even, but in the assignment deﬁned by , only one of

those variables is assigned the value 1. Therefore, the constraint cannot be satisﬁed, and Û cannot be a codeword. A contradiction.

Theorem 10.10. Efﬁcient Decoding, Sipser and Speilman (95): In the above conditions, if the expansion of sets of

¿ «

k Ò > ! !

size at most «Ò is , and if the distance of the input word from a codeword is at most , then the following

4 ¾

decoding algorithm will return ! in a linear number of iterations:

While there exists a variable such that most of its neighboring constraints are not satisﬁed, ﬂip it.

¼ ¼

A ! A = fÚ : ! 6= ! g A

Proof. Let be the set of errors in ,i.e. Ú .If is empty, we are done. Otherwise, assume

Aj«Ò that j . (We need the assumption to guaranty the expansion, and we will prove later that this assumption holds

throughout the running of the algorithm.)

Aµ Ë Í

Partition ´ to satisﬁed neighbors and unsatisﬁed neighbors . Then:

Í j · jË j = j ´Aµj > k jAj:

j (10.1) 4

10.4. USING EXPANDERS TO GENERATE ERROR CORRECTING CODES 59

´Aµ=Í [ Ë jÍ j Í ¾jË j

Now, count the edges between A and . There are at least edges leaving , and at least edges

Ë A

leaving Ë (every vertex in must have at least two neighbors in ). Therefore,

jÍ j ·¾jË j k jAj:

Combining this with (10.1) we get that

k jAj jÍ j¾jË j > ¾´ k jAj jÍ jµ; 4

and therefore,

Í j > k jAj:

j (10.2)

jAj jAj A A

So more than k neighbors of the members of are unsatisﬁed. Therefore there is a variable in that has

½ k

> unsatisﬁed neighbors. That means that as long as there are errors, there is a variable that most of its neighbors

Í j

are unsatisﬁed. Since by deﬁnition, j decreases with every step, we deduce that:

«Ò Í Corollary 10.11. If the distance from ! does not exceed throughout the run of the algorithm, then will reach

the empty set, and the algorithm will halt with the codeword ! .

« «

Ò Ò A «Ò jA j jÍ jj ´A µjk

¼ ¼

To show that is always , note that in the beginning, ¼ , and therefore .

¾ ¾

Í j

Therefore, since j is decreasing, throughout the running of the algorithm

Í jk Ò:

j (10.3)

jA j ¦½ jAj «Ò i

We know that at any step i changes by . Therefore, if at any time exceeds , there must be a time when

Ò jA j = «Ò «Ò jÍ j >k

i (we can assume that is an integer). Then by (10.2), , which is a contradiction to (10.3). ¾ 60 CHAPTER 10. ERROR CORRECTING CODES Chapter 11

Lossless Conductors and Expanders

Notes taken by Ariel Elbaz, Yuval Filmus and Michal Igell

Summary: In this lecture we deﬁne conductors, a generalization of expanders. Extractors, dispersers and condensers are all types of conductors. We use the new structures to explicitly construct lossless expanders.

11.1 Min-entropy

Min-entropy measures the rarity of the most probable event. If all events occur at probability at most ¾ , then the

X Ë

min-entropy is at most k , and vice versa. For a random variable (or a distribution) over some ﬁnite set ,let

ËÙÔÔ ´X µ=fÜ ¾ Ë :ÈÖ[X = Ü] > ¼g:

Deﬁnition 11.1. The min-entropy of a distribution X

Ó Ò Ó Ò

= ÑaÜ ÐÓg ÈÖ[X = Ü] À ´X µ = ÑiÒ ÐÓg

Ü Ü

ÈÖ[X = Ü]

¾ ËÙÔÔ´X µ

where minimum and maximum are taken over Ü . ¾ Note: throughout this work, ÐÓg is always taken to base .

Deﬁnition 11.2. The Rényi entropy of a distribution X is

h i

´ÈÖ[X = Ü]µ = ÐÓg E ÈÖ[X = Ü] : À ´X µ= ÐÓg

¾ Ü

Lemma 11.3. The min-entropy and Rényi entropy of a distribution X obey the following inequality:

À ´X µ À ´X µ ¾À ´X µ

½ ¾ ½ Ë The left inequality is tight iff X is uniformly distributed on some set .

Proof. Since ÐÓg is a monotone increasing function, we have

¢ £

À ´X µ= ÐÓg E ÈÖ[X = Ü]

ÐÓg ÑaÜ ÈÖ[X = Ü]=À ´X µ

Ü¾ËÙÔÔ´X µ

62 CHAPTER 11. LOSSLESS CONDUCTORS AND EXPANDERS

ËÙÔÔ´X µ

and equality holds iff X is uniformly distributed over .

Ü = aÖgÑaÜ ÈÖ[X = Ü]

On the other hand, let Å .Wehave

À ´X µ= ÐÓg ´ÈÖ[X = Ü]µ

¾ ¾

= ÐÓg ´ÈÖ[X = Ü]µ · ´ÈÖ[X = Ü ]µ

Ü6=Ü

ÐÓg ´ÈÖ[X = Ü ]µ

= ¾ ÐÓg ÈÖ[X = Ü ] =¾À ´X µ

Å ½

A distribution which is uniformly distributed on some set Ë is called a ﬂat distribution, and both its min-entropy

jË j Ë and its Rényi entropy are ÐÓg . These are maximal among distributions with support . Flat distributions combine to make the most general distributions:

Lemma 11.4. Every distribution over a finite set is a convex combination of a finite number of flat distributions. In

È È

Ô =½ X X = Ô Í ´Ë µ Ë ËÙÔÔ ´X µ Ô ¼

i i i i

other words, if is a distribution then i ,where , and .

jËÙÔÔ ´X µj ËÙÔÔ ´X µ = fÜg X = Í ´Üµ Ü =

Proof. The proof goes by induction on .If then . Otherwise, let Ñ

aÖgÑiÒ ÈÖ[X = Ü] Ô =ÈÖ[X = Ü ] X = Ô Í ´ËÙÔÔ ´X µµ · ´½ Ô µY ËÙÔÔ ´Y µ ´

Ñ Ñ Ñ

,andlet Ñ .Wehave ,where

ËÙÔÔ´X µ Ü ¾= ËÙÔÔ ´Y µ

since Ñ .

Y X À ´Y µ·

If we combine a distribution with a ﬂat distribution , the joint distribution has min-entropy equal to ½

À ´X µ

½ :

X À ´X µ= c Y À ´X; Y µ ½

Lemma 11.5. Suppose is a ﬂat distribution, with ½ , and let be another distribution. Then

k Ü ¾ ËÙÔÔ´X µ À ´Y jX = Üµ k c

iff for every , we have ½ .

À ´X; Y µ k À ´Y jX = Üµ k c Ü ¾ ËÙÔÔ´X µ ÈÖ[Y = Ý jX = ½

Proof. Suppose ﬁrst that ½ .If for some ,then

c k c k c k

] ¾ Ý ¾ ËÙÔÔ´Y µ X ÈÖ[X = Ü; Y = Ý ] ¾ ¾ =¾

Ü for some .Since is ﬂat, , contradicting the promise

À ´X; Y µ k

½ .

À ´Y jX = Üµ k c Ü ¾ ËÙÔÔ´X µ Ý ¾ ËÙÔÔ ´Y µ

Next, suppose that ½ for all . Then for all ,wehave

c k c k c k

ÈÖ[Y = Ý jX = Ü] ¾ X ÈÖ[Y = Ý ] ¾ ¾ =¾ À ´Y µ k

.Since is ﬂat, . Therefore, ½ .

We complete this section with a technical lemma, showing how to divide a joint distribution according to condi-

tional min-entropy.

Y Ë ¯ j ÈÖ[X ¾ È ] ÈÖ[Y ¾ È ]j¯

Deﬁnition 11.6. Two distributions X and over the same set are said to be -close if

Ë X Y ¯ j ÈÖ[X = ×] ÈÖ[Y = ×]j¾¯

for every subset È . Alternatively, and are -close if .Weleavethe ×

reader to show that the two deﬁnitions are equivalent.

X X ¯> ¼ a Y Y

¾ ½ ¾

Lemma 11.7. Let ½ and be two distributions. Given and , there exist distributions and such that

¯ ´X ;X µ ´Y ;Y µ ¯

¾ ½ ¾

The joint distributions ½ and are -close;

^ ^

¯ Ý ;Y µ Ý ; Y µ Ý ; Y µ

¾ ½ ¾ ½ ¾

The joint distribution ½ is a convex combination of two other joint distributions and ,

À ´X ;X µ ÐÓg

½ ¾

both having min-entropy at least ½ ;

^ ^ ^

¯ Ü ¾ ËÙÔÔ ´Y µ À ´Y jY = Üµ a

½ ¾ ½

For all ½ we have ;

¯ Ü ¾ ËÙÔÔ ´Y µ À ´Y jY = Üµ a

½ ¾ ½ For all ½ we have ;

11.2. CONDUCTORS AND LOSSLESS EXPANDERS 63

^ ^

´Y ; Y µ ´Y ; Y µ ËÙÔÔ ´X µ À ´X jX = Üµ

¾ ½ ¾ ½ ½ ¾ ½

Proof. We begin by constructing ½ and . We split according to :

X = fÜ : À ´X jX = Üµ ag;

½ ¾ ½

X = fÜ : À ´X jX = Üµ

½ ¾ ½

Now we can deﬁne

^ ^ ^

ÈÖ[Y = Ý ; Y = Ý ]=ÈÖ[X = Ü ;X = Ü jÜ ¾ X ];

½ ½ ¾ ¾ ½ ½ ¾ ¾ ½

ÈÖ[Y = Ý ; Y = Ý ]=ÈÖ[X = Ü ;X = Ü jÜ ¾ X ]:

½ ½ ¾ ¾ ½ ½ ¾ ¾ ½

^ ^

Y X Y X ½

In other words, ½ gets only values in ,and is restricted to .

^ ^ ^

Ô =ÈÖ[X ¾ X ] ´Y ; Y µ ½=Ô

½ ¾

If ½ , then the probability of each event in is multiplied by , and the probability of each

^ ^

Ý ; Y µ ½=´½ Ôµ ¯ Ô ½ ¯ Ý ; Y µ Ý ; Y µ

¾ ½ ¾ ½ ¾

event in ½ is multiplied by . Therefore, if then the min-entropy of and

^ ^

ÐÓg ½=¯ ´X ;X µ=ÔÝ ; Y µ·´½ ÔµÝ ; Y µ Ý ;Y µ=´X ;X µ

¾ ½ ¾ ½ ¾ ½ ¾ ½ ¾

is reduced by at most .Since ½ we can take .

Ô < ¯ Ý ; Y µ Ý ;Y µ = Ý ; Y µ

¾ ½ ¾ ½ ¾

If, for example , ½ still has high enough min-entropy, and so we take .This

¯ ´X ;X µ ¾

distribution is -close to ½ :

¬ ¬

ÈÖ[X = Ü ;X = Ü ] ÈÖ[Y = Ü ; Y = Ü ] ·

½ ½ ¾ ¾ ½ ½ ¾ ¾

Ü ¾X;Ü

½ ¾

¬ ¬

ÈÖ[X = Ü ;X = Ü ] ÈÖ[Y = Ü ; Y = Ü ] =

½ ½ ¾ ¾ ½ ½ ¾ ¾

Ü ¾X;Ü

½ ¾

Ô · ½ ´½ Ôµ=¾Ô< ¾¯:

½ Ô

11.2 Conductors and lossless expanders

11.2.1 Conductors Loosely speaking, a conductor is a function that transfers entropy from its inputs to its output. In other words, if the

input distributions have high min-entropy, then the output distribution will have high min-entropy.

Ò d Ñ

E : f¼; ½g ¢f¼; ½g ! f¼; ½g ´k ;a;¯µ

Deﬁnition 11.8 (conductors). A function is a ÑaÜ -conductor if for any

X f¼; ½g À ´X µ=k k E ´X; Í µ ¯ Y

ÑaÜ d

distribution on satisfying ½ , the distribution is -close to a distribution

· a whose min-entropy is at least k .

Remark: In this text we identify a distribution with a random variable sampled from it.

E X k k Í d

In other words, gets two inputs: a distribution with min-entropy ÑaÜ , and the uniform distribution

¯ k · a (with min-entropy d). The output is -close to a distribution with min-entropy at least . The following deﬁnitions (deﬁnitions 3.4 to 3.7 from [CRSW02]) are several special cases of conductors, which

we will later use.

Ò d Ñ

: f¼; ½g ¢f¼; ½g !f¼; ½g ´¯; aµ

Deﬁnition 11.9 (extracting conductors). A function E is an extracting con-

¼ k Ñ a k X f¼; ½g E ´X; Í µ ´k · a; ¯µ

ductor if for any ,andany -source over , the distribution d is a -source.

Ò d Ñ

: f¼; ½g ¢f¼; ½g !f¼; ½g ´¯; aµ ´Ñ a; ¯µ

Note that if E is an extracting conductor, it is also an extractor.

Ò d Ñ

E : f¼; ½g ¢f¼; ½g !f¼; ½g ´k ;¯µ

Deﬁnition 11.10 (lossless conductors). A function is an ÑaÜ lossless conductor

¼ k k k X f¼; ½g E ´X; Í µ ´k · d; ¯µ d if for any ÑaÜ ,andany -source over , the distribution is a -source.

The next two deﬁnitions require that the preﬁx of the output is an extracting conductor:

64 CHAPTER 11. LOSSLESS CONDUCTORS AND EXPANDERS

Ò d Ñ b

E; C i : f¼; ½g ¢f¼; ½g ! f¼; ½g ¢f¼; ½g

Deﬁnition 11.11 (buffer conductors). A pair of functions h is an

´k ;a;¯µ E ´¯; aµ hE; C i ´k ;¯µ ÑaÜ

ÑaÜ buffer conductor if is an extracting conductor, and is an -lossless conductor.

Ò d Ñ b

E; C i : f¼; ½g ¢f¼; ½g !f¼; ½g ¢f¼; ½g

Deﬁnition 11.12 (permutation conductors). A pair of functions h ,

· d = Ñ · b ´¯; aµ E ´¯; aµ hE; C i

where Ò is an permutation conductor if is an extracting conductor, and is a

Ò·d

¼; ½g permutation over f .

11.2.2 Lossless expanders

Having deﬁned conductors, we deﬁne lossless expanders and reveal the connection.

d ´k ;¯µ k k ÑaÜ

Deﬁnition 11.13. A -regular bipartite graph is a ÑaÜ -lossless expander if every set of vertices on the

¯µd ¡ k left side has at least ´½ neighbors.

That is, a lossless expander has almost the maximal expansion possible for a d-regular graph, for small enough

subsets. An alternative view is that for every subset on the left side, most neighbors will be unique neighbors, i.e.

k Ñ=d

neighboring a single vertex of the set. Naturally, ÑaÜ should be somewhat smaller than for this to be possible,

where Ñ is the number of vertices on the right side.

Expanders, condensers and dispersers (which we don’t deﬁne here) can be viewed as special cases of conductors.

d Ò

¾ ¾

For example, a lossless conductor E can be viewed as a -regular bipartite graph with vertices on the left side and

Ñ k k k ·d

ÑaÜ

¾ ¾ ´½ ¯µ¾

¾ vertices on the right side, where each set of vertices has at least neighbors on the right

ÑaÜ

;¯µ

side. In other words, we get a ´¾ -lossless expander.

´Å=Dµ

We can explicitly construct constant-degree lossless expanders which losslessly expand sets of size Ç ,

Ñ d

=¾ D =¾

where Å is the number of right vertices, and is the left degree.

> ¼ D = ´Æ =¯Å µ

Theorem 11.14. For any ¯ , there is an explicit family of -regular bipartite graphs which are

Ç ´¯Å =D µ;¯µ Æ Å < Æ

´ -lossless expanders, where is the number of vertices on the left side, is the number of

= Ç ´Å µ c Æ = Ç ´Å µ vertices on the right side, Æ and is a constant. Note that since , the degree itself is bounded

by a constant depending only on ¯.

¯Å

; ÐÓg D; ¯µ

We will prove the theorem by constructing an explicit family of ´ÐÓg -conductors. D

11.3 The Construction

The required lossless conductors will be constructed using a zigzag product. Let us recall the zigzag product for

expanders:

d ×

Deﬁnition 11.15 (The zigzag product of two regular bipartite graphs). Let À be a -regular bipartite graph with

× Ò

vertices on each side, and let G be an -regular bipartite graph with vertices on each side.

À d ×Ò

The zigzag product G z is a -regular bipartite graph with vertices on each side, which we may conceive

À G ´Ü; Ý µ ¾ [Ò] ¢ [×] ´Ü; Ý µ

as Ò copies of , one per each vertex of . Pick a left vertex . The edges emanating from are

d] ¢ [d] ´a; bµ

labeled using labels from [ . The edge labeled is determined as follows: a

1. Take a left to right step in the local copy of À (use to choose an edge). À

2. Take a left to right step on G, that is between copies of . b

3. Take a left to right step in the new local copy of À (use to choose an edge).

¼ ¼ ¼

Ü; Ý µ Ü ¾ G Ý Ü

We expand on the second step. Suppose after the ﬁrst step we are at ´ .Let be the -th neighbor of ,

¼ ¼ ¼

Þ Ü ´Ü; Ý µ ´Ü ;Þµ and let Ü be the -th neighbor of . Then the second step takes us from to . 11.3. THE CONSTRUCTION 65

GHZ G GHZ H

Figure 11.1: Zigzag product of bipartite graphs

11.3.1 The zigzag product for conductors We now deﬁne the zigzag product for conductors, and show that composing conductors with carefully selected param-

eters, we get constant degree lossless expanders, as in theorem 11.14.

À G À

Recall that the zigzag theorem, proved in a previous lecture, shows that G z is an expander if both and

À À G À

are such. Moreover, the degree of G z is related to , while its size and expansion are related to both and .

G À µ = deg ´À µ G À À

Unfortunately, while deg´ z , the expansion of z is the minimum between the expansion of

G À deg ´G À µ and the expansion of G. That is, the expansion of z is at most the square root of z (this can be seen

by considering a set consisting of a single copy of À ). This expansion is too low for lossless expanders, which require expansion almost as big as the degree.

Recall also that in the proof of the zigzag theorem, of the two random steps (steps 1 and 3 in deﬁnition 11.15),

¾ d only one is "used" and contributes to the output entropy. That is, out of d choices, only are surely increasing the entropy. Here we try to avoid this loss of entropy by buffering the random choices made at each step, and then using a lossless conductor, together with some fresh truly random bits, to condense the leftovers of entropy. The name "conductor" suggests an analogy to electricity or water conductors. Another analogy to water is to think

of the lossless conductor construction as putting a bucket beneath each object, so that when we pour randomness k

(water) into it, the leftovers (unused randomness, beyond the ÑaÜ bound), are stored for later use. In the zigzag product for conductors we make use of several objects. We remark that they can be explicitly constructed using lemmas from [CRSW02].

Let us then assume that we have in our hands the following objects:

Ò d Ñ b

½ ½ ½ ½

hE ;C i : f¼; ½g ¢f¼; ½g !f¼; ½g ¢f¼; ½g ½ 1. ½ , a permutation conductor that can be taken from lemma

4.4 in [CRSW02];

b d d Ò

¾ ½ ¾ ¾

¢f¼; ½g !f¼; ½g ¢f¼; ½g hE ;C i : f¼; ½g ¾

2. ¾ , a buffer conductor;

Ñ d b ·b

¿ ¿ ½ ¾

!f¼; ½g ¢f¼; ½g E : f¼; ½g

3. ¿ , a lossless conductor.

E E ¿ Both ¾ and can be taken from lemma 4.13 in [CRSW02].

We describe the zigzag product for conductors.

Ò Ò d

½ ¾ ¾

Ò = Ò · Ò d = d · d Ñ = Ñ · Ñ Ü ¾ f¼; ½g Ü ¾ f¼; ½g Ö ¾ f¼; ½g

¾ ¾ ¿ ½ ¿ ½ ¾ ¾

Set ½ , and .For , , and

Ö ¾f¼; ½g

¿ ,deﬁne

Ò d Ñ

E : f¼; ½g ¢f¼; ½g !f¼; ½g

66 CHAPTER 11. LOSSLESS CONDUCTORS AND EXPANDERS

def

E ´Ü Ü ;Ö Ö µ = Ý Ý

¾ ¾ ¿ ½ ¿

by ½ ,where

¯ ´Ý ;Þ µ=hE ;C i´Ü ;Ö µ

¾ ¾ ¾ ¾ ¾ ¾

¯ ´Ý ;Þ µ=hE ;C i´Ü ;Ý µ

½ ½ ½ ½ ½ ¾

¯ Ý = E ´Þ Þ ;Ö µ

¿ ¿ ½ ¾ ¿

Þ Þ hE ;C i hE ;C i

½ ½ ½ ¾ ¾ and ¾ are buffers of and .

Figure 2 shows an example of this construction.

¾ ´Ý ;Þ µ hE ;C i ´Ü ;Ö µ E

i i i i i ¿

Our notation in ﬁgure is that i are the output of on the inputs (except for which has

hE ;C i hE ;C i Ö = Ý E

½ ¾ ¾ ½ ¾ ¿

only one output). As ½ gets it’s seed from , we get that , and since gets it’s input from

hE ;C i hE ;C i Ü = Þ Þ

½ ¾ ¾ ¿ ½ ¾

½ and , we get that .

À À

Recall that the zigzag product for bipartite graphs, G z ,uses twice. In the new construction, the ﬁrst use is

E Ü Ý E

¾ ¾ ½

replaced with ¾ . This ensures that when has high min-entropy, is close to uniform, and is a good seed for .

À E E ¿

The second use of is replaced with ¿ , which is a lossless conductor. The role of is to transfer entropy lost in

E E ¾ ½ and to the output.

n d X1 X2 R2 R3 (n-20a) (20a) (a) (a)

Y2 Z2 (14a) (21a)

E 1

Z1 (14a)

E 3

Y1 Y3 (n-20a) (17a)

n-3a

Figure 11.2: Entropy ﬂow in a lossless conductor

11.3.2 A speciﬁc example of a lossless conductor

It will be constructive for the sake of explanation to look at a concrete example of the construction.

µ = ½¼¼¼ ÐÓg´ d =¾a

Set a and .Thenwehave

Ò ¾¼a ½4a Ò ¾¼a ½4a

¯ hE ;C i : f¼; ½g ¢f¼; ½g !f¼; ½g ¢f¼; ½g ´Ò ¿¼a; 6a; ¯µ ½ ½ ,an permutation conductor;

11.3. THE CONSTRUCTION 67

¾¼a a ½4a ¾½a

¯ hE ;C i : f¼; ½g ¢f¼; ½g !f¼; ½g ¢f¼; ½g ´½4a; ¼;¯µ ¾

¾ ,a -buffer conductor;

¿5a a ½7a

¯ E : f¼; ½g ¢f¼; ½g !f¼; ½g ´½5a; a; ¯µ

¿ ,a lossless conductor.

The result is

Ò ¾a Ò ¿a

E : f¼; ½g ¢f¼; ½g !f¼; ½g

Ò ¿¼a; ¾a; 4¯µ

which is a ´ -lossless conductor.

´Ü Ü ;Ö Ö µ Ý Ý

¾ ¾ ¿ ½ ¿

Let us try and follow the entropy ﬂow from the input ½ to the output .

k = À ´X ;X µ k k ·¾a

½ ¾ Let ½ , then we show that if we start with min-entropy , we end up with min-entropy of .

Two main ideas for this example are emphasized:

hE ;C i; hE ;C i hE ;C i hE ;C i

½ ¾ ¾ ½ ½ ¾ ¾ 1. The entropy is conserved by ½ , because is a permutation conductor, and is

a buffer conductor. Therefore, we get

k · a = À ´X ;X ;Ê µ=À ´X ½;Y ;Z µ=À ´Y ;Z ;Z µ

½ ½ ¾ ¾ ½ ¾ ¾ ½ ½ ½ ¾

E E Y À ´Y ;Z ;Z µ=

¾ ½ ½ ½ ½ ¾

2. We want to verify that enough entropy is transferred, using ½ and ,to . We know that

k · a À ´Y µ k ½4a À ´Z ;Z jY µ ½5a E

½ ½ ½ ¾ ½ ¿

, and if we prove that ½ ,then . In that case ,whichisa

´½5a; a; ¯µ a Ê Y Z ;Z

¿ ½ ¾

-conductor will conduct bits of entropy from ¿ to . That is, all the entropy of will be Y

transferred to the output ¿ , without any entropy loss, as we want.

À ´Y µ k ½4a ½ To prove that ½ , we look at the two cases, which lemma 11.7 essentially shows that are sufﬁcient to

prove the general case.

Ü ¾ ËÙÔÔ ´X µ À ´X jX = Ü µ ½4a

½ ½ ¾ ½ ½

Case 1 For all ½ ,wehave .

À ´Y jX = Ü µ=½4a Ü ¾ ËÙÔÔ´X µ Y hE ;C i

¾ ½ ½ ½ ½ ¾ ½ ½

In this case, ½ ,forany . Therefore can be used as a seed for

Ü ¾ ËÙÔÔ ´X µ À ´X µ k ¾¼a E 6a

½ ½ ½ ½

for any ½ . We know that , and therefore conducts bits of entropy from

Y À ´Y µ k ½4a

½ ½

the seed into ½ , and we get that .

Ü ¾ ËÙÔÔ ´X µ À ´X jX = Ü µ ½4a

½ ½ ¾ ½ ½

Case 2 For all ½ ,wehave .

À ´X ;X¾µ = k À ´X µ k ½4a E

½ ½ ½ ¾

Since ½ , it follows that . Therefore, since is a lossless extrac-

À ´Y jX = Ü µ À ´X jX = Ü µ Ü ¾ ËÙÔÔ´X µ À ´X ;Y µ

¾ ½ ½ ½ ¾ ½ ½ ½ ½ ½ ½ ¾

tor, ½ for any . It follows that

À ´X ;X µ=k hE ;C i À ´Y ;Z µ k À ´Y µ

½ ½ ¾ ½ ½ ½ ½ ½ ½

.Since ½ is a permutation, also , and again we get that

½4a

k .

Ý ¾ ËÙÔÔ´Y µ ½

To complete the example, for any ½ ,

À ´Z ;Z jY = Ý µ À ´Y ;Z ;Z µ À ´Y µ ´k · aµ ´k ½4aµ= ½5a:

½ ½ ¾ ½ ½ ½ ½ ½ ¾ ½ ½

E a Ê Y À ´Y jY = Ý µ

¿ ¿ ½ ¿ ½ ½

Therefore, the lossless extractor ¿ transfers bits of entropy from to , and we get that

À ´Y ;Y µ k ·¾a À ´Z ;Z jY = Ý µ·a

½ ½ ¿ ½ ¾ ½ ½ ½ . This implies that, , as needed. 68 CHAPTER 11. LOSSLESS CONDUCTORS AND EXPANDERS Chapter 12

Cayley graph expanders

Notes taken by Eyal Rozenman

Summary: We describe ideas leading to an elementary construction of Cayley graphs which are ex-

panders with relatively small degree.

Ë À h ¾ À h = × ¡ × :::¡ ×

¾ k

A set of elements in a group is a generating set if every element of can be written as ½

× ¾ Ë

with i .

´À; Ë µ À Ë

Deﬁnition 12.1. The Cayley graph C of a group and a generating set is a graph whose vertices are the

´g; hµ g ¡ × = h × ¾ Ë

elements of À ,andwhere is an edge if for some .

× ¾ Ë × ¾ Ë ´g; hµ

This generally deﬁnes a directed graph. If the set of generators Ë is symmetric -i.e. iff ,then

h; g µ jË j is an edge iff ´ is, and we have an undirected graph. It is a regular graph of degree .

Example 12.2.

¯ C = Z=dZ Ë = f·½; ½g d

The additive cyclic group d with generators is the cycle on vertices.

¯ ´F µ

The additive group of the vector space ¾ over the ﬁeld with two elements is generated by the standard basis

e =´½; ¼;:::;¼µ;e ;:::;e d

¾ d vectors ½ . The Cayley graph is the discrete cube - a -regular graph. Consider the following construction, resulting in the graph depicted in ﬁgure 12.1. The degree of the discrete cube

is equal to the number of vertices in the d-cycle, so we can form a zigzag product of the two. Let’s look at the simpler

´F µ d C d

replacement product. In this product we replace every vertex of ¾ by a cloud of vertices representing .On

C d

each cloud we preserve the edges of the original d . We also connect each vertex in the cloud to one of the neighbors

F ´Ú; hµ ´Ú · e ;hµ

of the cloud in . For example, let’s connect vertex to h . Like the zigzag construction, this product ¾ is an expander if the original two graphs are expanders.

We started with two Cayley graphs and created a third graph by a graph-theoretic construction. Is the resulting

C F

graph also a Cayley graph of some group? The answer is yes. It’s the semidirect product of d and . To deﬁne this ¾

product we shall need (alas) some more deﬁnitions:

A : B ! AÙØ´Aµ

Deﬁnition 12.3. An action of a group B on a group is a group homomorphism .Inotherwords,

b ¾ B A =

b ¡b b b

each element corresponds to an automorphism b of , and we demand that .

½ ¾ ½ ¾

A A Ó B

Deﬁnition 12.4. Suppose a group B acts on a group .Thesemidirect product is a group whose elements are

a; bµ a ¾ A b ¾ B

pairs ´ where and .Wedeﬁne

á ;b µ ¡ á ;b µ=á ¡ á µ;b ¡ b µ:

½ ½ ¾ ¾ ½ b ¾ ½ ¾ ½

Example 12.5.

¯ A ¢ B

The direct product of two groups is a special case of a semidirect product where b is the identity

A b ¾ B á ;b µ ¡ á ;b µ=á ¡ a ;b ¡ b µ

½ ¾ ¾ ½ ¾ ½ ¾ automorphism of for all . In this case ½ .

70 CHAPTER 12. CAYLEY GRAPH EXPANDERS

F Ó C

Figure 12.1: A Cayley graph of ¿

d d

¯ C F h ¾ C F d

We have an action of d on : the element cyclically permutes the coordinates of by h places. We

¾ ¾

F Ó C

can thus deﬁne d . ¾

It turns out that under certain conditions there is a close relation between the semidirect product of groups and the

A a ¾ A

replacement product of their Cayley graphs. Suppose a group B acts on a group .Theorbit of an element

B f ´aµjb ¾ B g Ú ¾ F C d

under the action of is the set b . For example, the orbit of under the action of is the set of ¾

all cyclic shifts of Ú

A; B Ë ;Ë jB j = jË j

B A

Claim 12.6. Suppose we have two groups with sets of generators A , such that . Further, suppose

B A Ë Ü ¾ Ë Ë := ´½;Ë µ [f´Ü; ½µg

A B

that acts on in such a way that A is the orbit of one of the elements .Then

A Ó B C ´A Ó B; Ë µ C ´A; Ë µ C ´B; Ë µ B

generates , and is a replacement product of A and .

Ë A Ó B ´Ë ; ½µ [ ´½;Ë µ B

Proof. To see that indeed generates notice that the elements A generate it. Now observe that

´½;bµ ¡ ´Ü; ½µ ¡ ´½;b µ=´ ´Üµ; ½µ ´Ë ; ½µ ´Ü; ½µ ´½;Ë µ [f´Ü; ½µg

A B

b , so we can indeed generate all of starting with ,so

C ´A Ó B; Ë µ B C ´B; Ë µ

is generating. Look at . It consists of clouds of the elements of , with the graph of B on them,

á; bµ ¡ ´½;× µ=á; b ¡ × µ á; bµ ¡ ´Ü; ½µ = á ¡ ´Üµ;bµ

b b

since b . Between clouds we have edges like . So the cloud of

a a C ´A; Ë µ the element is indeed connected by one edge to each of the clouds of the neighbors of in the graph A ,and this is a replacement product.

Exercise 12.7. Under the assumptions of the claim, we can also describe the zigzag product of two Cayley graphs as

Ó B

a Cayley graph on A . Which generating set do we need?

¿ ¿

F C F Ó C ¿

Example 12.8. Look at and ¿ with the generators used above. In the Cayley graph of with generators

¾ ¾

=´½; ¼; ¼µ ¾ F

as in the claim, the neighbors of the cloud of Ú are:

´Ú; ¼µ ¡ ´e ; ¼µ = ´Ú · e ; ¼µ

¼ ¼

´Ú; ½µ ¡ ´e ; ¼µ = ´Ú · e ; ½µ

¼ ½

´Ú; ¾µ ¡ ´e ; ¼µ = ´Ú · e ; ¾µ

¼ ¾

And this is indeed a replacement product.

Recall that a replacement product of two expander graphs is again an expander. So we can try to make an expander d

graph which is also a Cayley graph using the semidirect product. As a starter, we look for generators for F that make ¾ 71

it an expander. Since this is an abelian group, then for every set of generators the eigenvectors of the graph are the

<Ü;Ý >

f ´Üµ=´ ½µ e ;:::e

½ d

Fourier basis - which are vectors of the type Ý . For the standard basis the eigenvalues are

X X

f ´e µ= ´ ½µ = d ¾jÝ j

Ý i

i i

jÝ j Ý = d; = d ¾ ¾

Where is the Hamming weight of the binary vector .So ½ . The second normalized eigenvalue

d ¾µ=d

is ´ and obviously, this is not a good expander. How do we ﬁnd another set of generators which does give an

Ë = fÚ ;Ú ;:::;Ú g jË j¢d A

¾ Ö

expander? Suppose we have a set ½ of generators. Write an matrix whose rows are the

Ý 6=¼

elements of Ë . To be an expander, we need to satisfy, for every

X X

<Ú ;Ý >

f ´Ú µ= ´ ½µ c ¡jË j

Ý i

i i

< ½ d AÜ Ü 6= ¼ for some constant c , independent of . This means that the vectors (with ) don’t have too much

difference between the numbers of 0’s and 1’s. In particular, each word must have a sufﬁcient number of 1’s. In short

d ¢ d - a good expander in this case gives an error correcting code. Using this intuition, one would try a random ¾

matrix A, which we know gives a good code, and indeed

d ¢ d ¾d ¡ Æ

Claim 12.9. For a random ¾ binary matrix, almost surely for all nonzero (for ¼ some constant Æ> ). So we have an expander. With two "small" ﬂaws: (a) It’s not explicit, and (b) The degree is too large. To get rid

of ﬂaw (b) we want not just an arbitrary set of generating vectors. We want the generators to be one orbit under the C

action of Ò , for example. Luckily, this also turns out to work

¾ F A Ù Ú

Claim 12.10. Pick two random vectors Ù; Ú . Consider the matrix generated by the orbits of and under the

C ¾d ¢ d Ù Ú A ¾d ¡ Æ

action of d , that is, the matrix of the cyclic shifts of and .Then (a.s.) satisﬁes

Æ>¼

for all nonzero Ü (for some constant ). C

So now, by using a semidirect product with d we get an expander. The only difference is that we used two orbits,

instead of one as in claim 12.6.

Ë k B

Exercise 12.11. Claim 12.6 still holds when A is a union of orbits under the action of . The Cayley graph of the

jË j · k semidirect will have B generators.

We can now use this idea to give a counterexample to

G Ë Ò

Conjecture 12.12. If a group sequence Ò is an expander with one set of generators of bounded size then it is Í

also an expander with any other set of generators Ò of bounded size.

ËÄ ´F µ ËÄ ´Ôµ Ô ·½

Ô ¾

Recall that we have two matrices that (with their inverses) make ¾ an expander. acts on the

Ô·½

F C F Ó ËÄ ´Ôµ

Ô d

elements of the projective plane over , so just as we did with we can form the semidirect product ¾ .

Ô·½

ËÄ F

It also turns out that there are two orbits of ¾ that make an expander (random orbits will do). So the semidirect

¾ Ô

product is an expander with 4 generators, for every . On the other hand, we have another set of generators for which

Ô·½ Ô·½

F F Ó ËÄ ´Ôµ

is not an expander - the standard basis. This gives a set of 3 generators for ¾ which is not an

¾ ¾ expander (recall that the replacement product is never a better expander than its components).

Can we iterate the semidirect product construction to get a sequence of Cayley expander graphs the same way we

G F [G] F Ô did with the zigzag product? For a group let Ô be the group ring over . we would like to make the additive

group an expander using a constant number of orbits under the action of G. If we could do that in general, we could

G = F Ô G [G ] Ó G

i·½ Ô i ½ i

deﬁne a sequence iteratively by i . This is indeed possible with a proper choice of and :

´Ò=¾µ ´Ò=¾µ

´C ´G ;Ë µµ ½=¾ Ë ÐÓg jG j ÐÓg

Ò Ò Ò Ò

Theorem 12.13. ¾ and where is the iterated logarithm. G

This is the (almost) best construction of this type we can hope to get using this construction, since the group Ò is

[G ] Ò G F

Ò ½ ½ Ô

a solvable group with solvability index at most (as Ò is a normal subgroup with abelian quotient ).

Ò ½

´Òµ

½=¾ ÐÓg jG j Ò

In this case it is known that any generating set which gives ¾ has cardinality at least

F [G]

The property we need to make Ô an expander with a constant number of orbits is

72 CHAPTER 12. CAYLEY GRAPH EXPANDERS

d ;d ;:::;d G jfi : d <Ögj

¾ Ø i

Theorem 12.14. Let ½ be the dimensions of the irreducible representations of .if

Ö c F [G]

for all integer and some constant , then there is a constant number of orbits of Ô that make it an expander.

F [G] G

Miraculously, Ô inherits this property if is a monomial group, which means all its irreducible represen-

G F [G] tations are induced from one-dimensional representations of subsets of .Furthermore, Ô is also monomial! Even better, we can ﬁnd the generating orbits explicitly. This gives a sequence of explicit Cayley graphs which are expanders with an "almost constant" number of generators. Chapter 13

On eigenvalues of random graphs and the abundance of Ramanujan graphs

Notes taken by Danny Gutfreund

Summary: In this lecture we will be looking at eigenvalue distributions of random graphs and matrices. Our starting point is the question: Is it true that almost every graph is Ramanujan? First, we consider generalizations of this question that look at the distribution of eigenvalues in general (and not only the second eigenvalue). We state “Wigner’s semicircle law”, and give a partial proof to a version of this law (By McKay) for regular graphs. In the second part of the lecture, we define lifts of graphs and extend the definition of Ramanujan graphs to general (non-regular) graphs. We then state some conjectures and results regarding the abundance of Ramanujan graphs (under this new definition).

13.1 The eigenvalue distribution of random matrices and regular graphs

Open problem 13.1. Is it true that almost every d-regular graph is Ramanujan? More formally, is it true that,

ÐiÑ ÈÖ´ ´G µ ¾ d ½µ=½

¾ Ò

Ò!½

G d Ò

Where Ò is a random -regular graph of size .

Friedman gave a positive answer upto an ¯ additive factor, ¼

Theorem 13.2. For every ¯> ,

ÐiÑ ÈÖ´ ´G µ ¾ d ½·¯µ=½

¾ Ò

Ò!½

G d Ò

Where Ò is a random -regular graph of size . Extending this question to random symmetric matrices, and the distribution of all the eigenvalues, we have the

following theorem by Wigner, known as “Wigner’s semicircle law”.

A Ò ¢ Ò Ê a i 6= j ij

Theorem 13.3. Let Ò be a symmetric matrix over ,where ( ) are sampled independently from a

F a G

distribution , and ii from a distribution .

´A µ ¡¡¡ ´A µ A

Ò Ò Ò Ò Let ½ be the eigenvalues of .

Deﬁne the empiric distribution,

Ï ´Üµ= jfi : ´A µ Ügj

Ò i Ò Ò

Let,

Ï ´Üµ= ÐiÑ Ï ´¾Ü Òµ

Ò!½

74CHAPTER 13. ON EIGENVALUESOF RANDOM GRAPHS AND THE ABUNDANCE OF RAMANUJAN GRAPHS

= ÚaÖ´F µ=ÚaÖ´Gµ

where .

Ê Ê

k k

k; jÜj dF ; jÜj dG < ½ Ï ´Üµ ½ Ü jÜj ½

If it holds that 8 ,then is continuous with density if , and 0 otherwise.

Going back to eigenvalues of adjacency matrices of d-regular graphs, we have the following version of Wigner’s

semicircle law, proven by McKay.

G d k ¿ C ´G µ = Ó´jG jµ

k Ò Ò

Theorem 13.4. Let Ò be an inﬁnite sequence of -regular graphs, such that, for all , ,

C ´G µ k G

Ò Ò where k is the number of length cycles in .

Deﬁne,

jfi : ´G µ Ügj F ´G ;Üµ=

i Ò Ò

jG j Ò

Then for every Ü,

d 4´d ½µ Þ

F ´Üµ= ÐiÑ F ´G ;Üµ= dÞ

¾ ¾

Ò!½

¾ ´d Þ µ

d ½ ¾ The idea of the proof is that under the assumption that there are very few short cycles, the neighborhood of almost every node looks like a tree. So we can count the number of cycle-free paths from a node to itself. We state this

formally in the following lemma.

d Ú Ö

Lemma 13.5. Let G be a -regular graph, and let be a node such that there are no cycles in its -neighborhood.

Ú Ö Ö =¾×

Then the number of paths of length Ö that start and end in is 0 if is odd, and if it is,

¾× j j

j × j

´Ö µ= d ´d ½µ

× ¾× j

j =½

Ú Ö

Proof. Clearly, if the Ö -neighborhood of does not contain any cycles, then every length path that starts and ends in Ö

Ú is cycle-free, and hence must be even.

Ö =¾× Ö Ú ¼ = Æ ;Æ ¡¡¡Æ =¼ Æ

½ Ö i

Let . Every length path that starts and ends in , deﬁnes a sequence, ¼ .Where

Ú i i Æ ¼ jÆ Æ j =½

i i ½

is the distance from after we did steps on the path. Clearly, for every , i ,and . We would

j Æ ¡¡¡Æ Ö like to count the number of such sequences in which exactly out of ¼ are 0. This is a simple generalization of

Catalan numbers and the answer is,

¾× j j

× ¾× j

j Ú d

We know that a path that deﬁnes such a sequence visits Ú exactly times. Each time it leaves it has exactly

d Ú Æ Æ =½

·½ i

possibilities for the next step. This gives the term . In steps that go away from ,i.e.when i ,wehave

d different nodes that we can continue too (we cannot backtrack to the previous node). and for steps that go toward

× j

Ú we have no choice because we have to go back on the same edge that we used before. There are steps of each

× j

d ½µ

type, altogether we have ´ possibilities for such steps.

C ´G µ=Ó´jG jµ k ¿ Ö Ö

Ò Ò

If k (for every ), then for every constant , almost every node has cycle-free -neighborhood.

È ´G µ Ö G

Ò Ò Let Ö be the number of simple paths of length from a node to itself in . Then from Lemma 13.5 we conclude

that,

È ´G µ

Ö Ò

ÐiÑ = ´Ö µ

Ò!½

F ´Üµ Ö dF = ´Ö µ

Therefore, the function must satisfy Ü ,forevery . In order to ﬁnish the proof of the theorem

´Üµ we need some inverse transformation, that calculates F out of its moments. This is achieved using the Chebyshev polynomials, but we do not include the details in this lecture note.

13.2 Random lifts and general Ramanujan graphs

We start by deﬁning the notion of coverings and lifts.

13.2. RANDOM LIFTS AND GENERAL RAMANUJAN GRAPHS 75

À f : Î ´À µ ! Î ´Gµ

Deﬁnition 13.6. Let G and be two graphs. We say that a function is a covering, if for every

Ú ¾ Î ´À µ f Ú ´Ú µ ´f ´Ú µµ G

, the restriction of to the set of neighbors of , À , is one to one and onto .

G À G

If a covering function from À to exists, we say that lifts .

À À À

Example 13.7. The zig-zag product of G and (where is the smaller of the two) lifts .

G Ò

Remark 13.8. If G is connected then every covering function on has a covering number , such that for every

½ ½

¾ Î ´Gµ jf ´Ú µj = Ò e ¾ E ´Gµ jf ´eµj = Ò

Ú , , and for every , .

G Ú ¾ Î ´Gµ f ´Ú µ Ú

Deﬁnition 13.9. Let f be a covering function on . For a node , we say that is the ﬁber of .

G Ä ´Gµ G Ò

Let be a connected graph. We denote by Ò the set of graphs that are lifts of with covering number .

Ä ´Gµ À ¾ Ä ´Gµ Î ´À µ=Î ´Gµ ¢ [Ò] Ò

We would like to characterize the members of Ò .If then . That is, for every

¾ Î ´Gµ ´Ú; ½µ; ¡¡¡ ; ´Ú; Òµ Î ´À µ Ú À

Ú , the nodes in are the ﬁber of . Next we deﬁne the edges of , for every edge

´Ú; Ùµ ¾ E ´Gµ ¾ Ë À ´´Ú; iµ; ´Ù; ´iµµµ

we take some permutation Ò and deﬁne the following edges in , (for every

i ¾ [Ò] Ä ´Gµ

). Thus every choice of permutations deﬁnes a member in Ò . This also gives us a way to sample random

elements from this set. G

What can we say about the eigenvalues of lifts of G? We know that the eigenvalues of are also the eigenvalues

: Î ´Gµ ! Ê G À G

of its lifts. To see this, let h be an eigenfunction of with eigenvalue ,andlet cover with the

: Î ´À µ ! Î ´Gµ h Æ f À À G map f ,then is an eigenfunction of with the same eigenvalue. Thus if lifts , we can

talk about its old eigenvalues, i.e. those that were inherited from G, and its new eigenvalues, which are the rest of the

eigenvalues. G

Definition 13.10. The universal covering space of a graph G is a graph that lifts all the lifts of . d Example 13.11. The infinite d-regular tree is the universal covering space of every -regular graph. Grienberg and Lubotzky gave the following definition which extends the definition of Ramanujan graphs to general

(non-regular) graphs.

G G

Deﬁnition 13.12. We say that a graph is Ramanujan if the absolute value of every eigenvalue of except ¼ is at 1 most the spectral radius of the universal covering space of G.

Conjecture 13.13. For every graphs G, if we lift it high enough then almost surely we will get a Ramanujan graph.

Lubotzky and Nagnibeda falsified this conjecture by constructing an infinite tree Ì , that covers infinite number of

ﬁnite graphs such that none of them is Ramanujan. £

Friedman showed that the construction of Lubotzky and Nagnibeda, in fact, constructs a single graph G that

is covered by all the graphs that Ì covers and has large second eigenvalue (i.e. larger than the spectral radius of

£ G Ì ). Since all the other graphs inherit the eigenvalues of , none of them can be Ramanujan. He then rephrased

Conjecture 13.13 as follows,

À À Conjecture 13.14. For every graph G, if we lift it high enough to a graph , then the new eigenvalues of will

almost surely be at most the spectral radius of the universal covering of À .

To support his conjecture, Friedman proved the following theorem.

Theorem 13.15. Let be a graph with a largest eigenvalue ¼ , and let be the spectral radius of its universal

covering space. Then in almost every (high enough) lift of G, every new eigenvalue satisﬁes,

· Ó´½µ ¼

This is a generalization of the following result by Broder and Shamir.

d Ç ´d

Theorem 13.16. For almost every -regular graph, the second eigenvalue is at most µ.

d = d = d

To see that Theorem 13.15 generalizes 13.16, note that for -regular graphs, ¼ and .

´A µ Ì

We refer the reader to lecture 5. There we deﬁned the spectrum Ì of the adjacency matrix of an inﬁnite tree . The spectral radius is the

´A µ

maximal (absolute) value in Ì . 76CHAPTER 13. ON EIGENVALUESOF RANDOM GRAPHS AND THE ABUNDANCE OF RAMANUJAN GRAPHS Chapter 14

Some Eigenvalue Theorems

Notes taken by Yonatan Bilu

Summary: The last lecture in the course surveys several bounds on eigenvalues of symmetric matrices,

and in particular those of graphs. Throughout this summary, the eigenvalues of a graph refer to the

:::

¾ Ò eigenvalues of its adjacency matrix, and are denoted by ½ . The class ended in a picnic, where juicy watermelon slices were served on Harry Potter plates, along side

Harry Potter napkins, to students wearing silly Harry Potter paper hats.

¯ c = c´¯; d G d Ò

Theorem 14.1. For every d and there is a ), such that if is a -regular graph on vertices, then the

d ½ ¯ cÒ number of its eigenvalues with absolute value greater than ¾ is at least . Proof. See “Elementary Number Theory, Group Theory and Ramanujan Graphs”, by G. Davidoff, P. Sarnak and A.

Vallete. The proof makes use of the notorious Cebyshev˘ Polynomials.

: È Ê

Theorem 14.2. (Füredi & Komlo˘s ’81) Let ½ be a random distribution on with expectation and variance ,

È Ê

and ¾ a random distribution on with expectation and variance as well. Assume further that both distributions

A Ò ¢ Ò È

are bounded. Let be a real, symmetric matrix, with off-diagonal entries chosen i.i.d. according to ½ , and

È ½ Ò diagonal entries chosen i.i.d. according to ¾ . Then with probability tending to as tends to inﬁnity the following

holds:

ÑaÜ j j < ¾Ò · Ç ´Ò ÐÓg Òµ

¾ i

1. i

¾ 1

Æ ´´Ò ½µ · · ; ¾ µ

2. ½ Ò Proof. The proof is derived by looking at moments of increasing order. An alternative approach, by Kahn and Sze-

meredi, relies on the Rayleigh quotient to understand the behavior of random matrices.

µ : d = Ç ´d

Theorem 14.3. (Broder & Shamir ’87) For almost all -regular graphs, ¾ .

G ¾d Ò d ; :::; d

Proof. Let be a random -regular graph on vertices, generated by choosing random permutations, ½ ,

´Ú; ´Ú µµ Ú ¾ [Ò] i ¾ [d] È

and deﬁning an edge i for every and .Let be the transition matrix of the Markov chain

¾d :::

¾ Ò

deﬁned by the graph, i.e. it’s adjacency matrix divided by .Let ½ be its eigenvalues, and

k k ¾k ¾k Ò

= ÑaÜfj j; j jg =½ k f È ØÖ ´È µ ½ g

Ò ½

¾ .Since ,andforany , are the eigenvalues of ,wehave ,

i i=½

and therefore:

½ ½

¾k ¾k

E ´ µ ´E ´ µµ ´E ´ØÖ ´È µµ ½µ ; (14.1)

where the inequality on the left follows from Jansen’s inequality.

G Ú Ë

Let Ú beavertexof . A path in the graph that starts at is uniquely deﬁned be a word over the alphabet

½ ½

f ; ; :::; ; g d

½ (these permutations label the edges of the path). Let us generate the graph by going along a

d ¾

1 ¾

´; µ Æ refers to the Normal Distribution with expectation and variance .

78 CHAPTER 14. SOME EIGENVALUE THEOREMS

´Ùµ

random path, and choosing i uniformly among the currently allowed values. In other words, say we are currently

i ¾ [d] ¯ ¾f ½; ½g ´Ùµ

at a vertex Ù. We choose uniformly at random and .If is already deﬁned, we move to that

´Ùµ

vertex. We call this a “forced move”. Otherwise, we choose uniformly at random among the values not yet

i ¯

taken by . We call this a “free move”.

´Ë µ Ë Ë

Deﬁne Öed to be the reduction of ,thatis,whatremainsof if we repeatedly remove all two consecutive

;

letters in it of the form i .

¾k

´È µ Ë Since we want to bound ØÖ , we would like to bound the probability that induces a path that starts and ends

in Ú . We divide this event into three:

´Ë µ 1. Öed is the identity.

2. The path deﬁned by Ë has exactly one (simple) loop.

3. The path deﬁned by Ë has at least two loops.

We bound each of these in the following three lemmas:

¾k

Lemma 14.4. Let Ë be a word of length generated as above, then:

ÈÖ[Öed´Ë µ=;] ´ µ

d d

Proof. The idea is to count closed paths in the inﬁnite ¾ -regular tree, using Catalan numbers.

¼ ¼

¾k Ë = Öed´Ë µ × = jË j

Lemma 14.5. Let Ë be a word of length generated as above, and let . Denote , and assume ½

×> , then:

½ ×

[Ë ] · Ç ´ µ

ÈÖ start at 1, has exactly one loop, and ends in 1

Ò Ò ×

Proof. Assume w.l.o.g. that Ë is a loop (otherwise, we argue for the loop, which is even shorter than ), and denote

½=Ú ; :::; Ú =½ Ú

× × ½

the vertices it visits ¼ . Since all these vertices are distinct, the choice made at is free. Therefore,

½ ½ ×

= · Ç ´ µ ½

the probability that at this point vertex is chosen is ¾

Ò × Ò Ò

¾k

Lemma 14.6. Let Ë be a word of length generated as above,then:

[Ë ] Ç ´ µ

ÈÖ has two loops

¾ Ò

Proof. For Ë to have two loops there have to be two “free choices” where we choose a vertex that was already visited k

before. The probability of choosing a vertex that was already chosen before, is at most ¾ . The probability of this

happening at two speciﬁc steps, is at most ¾ . By the union bound, the probability that it happens at some two steps

Ç ´ µ

is ¾ . Ò

Note that Lemma 14.5 is not exactly what we need, since it deals with a reduced word. Broder and Shamir also

Ë j =¾k jÖed´Ë µj = ×

bound the probability that when j , . With this, they show that the probability of the second case

½ k ¾

µ · Ç ´

is bounded by k . This, together with the other two bounds yields:

Òd

4 k

k ½ k ¾ ¾

· µ µ · · Ç ´ [Ë ] ´

ÈÖ starts at v and returns to v (14.2)

k ¾

d Ò Òd Ò

¾ k

k ´ µ = k ´¾

Finally, we need to choose the that minimizes the RHS, so we roughly need ¾ ,or

d Ò

Ó´½µµ ÐÓg Ò µ ´½ · Ó´½µµ ´µ ´

. Putting this back in 14.1, we get E . The proof is ﬁnished by showing that

d=¾ d

is concentrated around its mean by using martingales.

¾d ½·¯ ¯> ¼

Note: Currently, the best result is by Joel Friedman, who showed ¾ ,forall . 79

Note: The random model of Broder & Shamir, that of choosing d random matchings, does not induce a ¾

uniform distribution on d-regular graphs. However, this model is contiguous to the uniform model, that is, any graph property that occurs in one of them w.h.p., occurs w.h.p. in the other as well. For details see chapter 9 in “Random Graphs” by S. Janson, T. Luczak and A. Rucinski.´ Another source of reference is Nick Wormald’s survey, “Models of random regular graphs”, which appears in “Surveys in Combinatorics”, 1999, J.D. Lamb and D.A. Preece, eds. 80 CHAPTER 14. SOME EIGENVALUE THEOREMS Bibliography

[CRSW02] M. Capalbo, O. Reingold, S. Vadhan and A. Wigderson, Randomness Conductors and Constant-Degree Lossless Expanders, Extended Abstract, STOC 2002. Randomness Conductors and Constant-Degree Expan- sion Beyond the Degree/2 Barrier. [CW65] J. W. Cooley and Tukey J. W. An algorithm for the machine calculation of complex fourier series. Mathe- matics of Computation, 90(19):297–301, 1965. [Die97] R. Diestel. Graph Theory. Springer-Verlag, 1997. [KMS94] D. Karger, R. Motwani and M. Sudan, Approximate graph coloring by semideﬁnite programming Proc. 35th IEEE Symposium on Foundations of Computer Science, pp. 2–13 (1994) [Rab80] M. O. Rabin. Probabilistic algorithm for testing primality. Journal of Number Theory, 12(128-138), 1980. [RVW02] O. Reingold, S. Vadhan, and A. Wigderson. Entropy waves, the zig-zag graph product, and new constant- degree expanders. Annals of Mathematics, 155(1):157–187, 2002. [Sha48] C. E. Shannon. A mathematical theory of communication. Bell System Technical Journal, 27, July and October 1948. [Val76] L. G. Valiant. Graph-theoretic properties in computational complexity. JCSS, 13(3):278–285, 1976. [Vla02] Vlad Ciubotariu, The Perfect Graph Theorem, University of Waterloo student project, (April 2002) from http://www.student.math.uwaterloo.ca/ cs762/Projects/index.php