Lecture 2: Spectra of Graphs 1 Definitions

Home , Regular graph, Spectral graph theory

Spectral Graph Theory and Applications WS 2011/2012

Lecture 2: Spectra of Graphs

Lecturer: Thomas Sauerwald & He Sun

Our goal is to use the properties of the adjacency/Laplacian matrix of graphs to ﬁrst under- stand the structure of the graph and, based on these insights, to design eﬃcient algorithms. The study of algebraic properties of graphs is called algebraic graph theory. One of the most useful algebraic properties of graphs are the eigenvalues (and eigenvectors) of the adjacency/Laplacian matrix.

1 Deﬁnitions

Deﬁnition 2.1. Let G = (V,E) be an undirected graph with vertex set [n] := {1, . . . , n}. The adjacency matrix of G is an n by n matrix A given by ( 1 if i and j are adjacent Ai,j = 0 otherwise

If G is a multi-graph, then Ai,j is the number of edges between vertex i and vertex j.

• The sum of elements in every row/column equals the degree of the corresponding vertex.

• If G is undirected, then A is symmetric.

Example. The adjacency matrix of a triangle is

 0 1 1   1 0 1  1 1 0

Deﬁnition 2.2. We deﬁne the Laplacian matrix L of graph G as follows:

 deg(i) if i = j  Li,j = −1 if i and j are adjacent  0 otherwise where deg(i) is the degree of vertex i.

Let D be an n by n diagonal matrix with deg(1),..., deg(n) as diagonal elements. We can rewrite L as L = D − A. In particular, if G is d-regular, then L = d · I − A.

Given a matrix A, a vector x 6= 0 is deﬁned to be an eigenvector of A if and only if there is a λ ∈ C such that Ax = λx. In this case, λ is called an eigenvalue of A.

1 Deﬁnition 2.3 (graph spectrum). Let A be the adjacency matrix of an undirected graph G with n vertices. Then A has n real eigenvalues, denoted by λ1 ≥ · · · ≥ λn. These eigenvalues associated with their multiplicities compose the spectrum of G.

Here are some basic facts about the graph spectrum.

Lemma 2.4. Let G be any undirected simple graph with n vertices. Then Pn 1. i=1 λi = 0. Pn 2 Pn 2. i=1 λi = i=1 deg(i).

3. If λ1 = ··· = λn, then E[G] = ∅.

4. degavg ≤ λ1 ≤ degmax. p 5. degmax ≤ λ1 ≤ degmax. Proof. We only prove the ﬁrst three items. (1) Since G does not have self-loops, all the diagonal elements of A are zero. By the deﬁnition Pn Pn of trace, we have ı=1 λi = tr(A) = i=1 Ai,i = 0. Pn 2 2 Pn 2 2 (2) By the properties of matrix trace, we have ı=1 λi = tr A = i=1 Ai,i. Since Ai,i is the degree of vertex i, tr(A2) equals the sum of every vertex’s degree in G. Pn (3) Combing i=1 λi = 0 with λ1 = ··· = λn, we have λi = 0 for every vertex i. By item (2), we have deg(i) = 0 for any vertex i. Therefore E[G] = ∅.

k For a graph G with adjacency matrix A and integer k ≥ 1, Au,v is the number of walks of length k from u to v .

Pn p 1/p Let kxkp := ( i=1 |xi| ) . Then for any 1 ≤ p ≤ q < ∞, it holds that

1/p−1/q kxkq ≤ kxkp ≤ n · kxkq.

Lemma 2.5. For any graph G with m edges, the number of cycles of length k in G is bounded by O mk/2.

Proof. Let A be the adjacency matrix of G with eigenvalues λ1, . . . , λn. Thus the number of k Pn k Ck (cycles of length k) in G is bounded by tr A /(2k) = i=1 λi /(2k). For any k ≥ 3, it holds that n !1/k n !1/k n !1/2 X k X k X 2 1/2 λi ≤ |λi| ≤ |λi| = (2 · m) . i=1 i=1 i=1 k k/2 k/2 Hence tr A ≤ (2m) and the number of Ck is at most O m . Example. Some examples for diﬀerent spectra of graphs:

• For the complete graph Kn, the eigenvalues are n − 1 with multiplicity 1 and −1 with multiplicity n − 1. √ √ • For the complete bipartite graph Km,n, the eigenvalues are + mn, − mn and 0 with multiplicity m + n − 2.

• For the cycle Cn, the spectrum is 2 cos(2πj/n)(j = 0, 1, . . . , n − 1). Two assumptions that we make throughout the course are as follows:

2 1. We only consider undirected graphs. Note that if G is not undirected, then A and L is not symmetric any more and the eigenvalues of A and L could be complex numbers.

A matrix A = (ai,j)n×n is called a Hermitian matrix if ai,j = aj,i for any element ai,j. Hermitian matrices always have real eigenvalues.

2. Unless mentioned otherwise, we consider regular graphs.

Lemma 2.6. Consider any undirected graph G with adjacency matrix A.

1. If G is d-regular, then λ1 = d and |λi| ≤ d for i = 2, . . . , n.

2. G is connected iﬀ λ2 < d, i.e., the eigenvalue d has multiplicity 1. Moreover, the number of connected components of G equals the multiplicity of eigenvalue d.

3. If G is connected, then G is bipartite iﬀ λn = −d. Lemma 2.7. All eigenvalues of L are non-negative.

Proof. Follows from Lemma 2.6 and the deﬁnition of L = D − A.

For studying regular graphs, it is convenient to work with the normalized adjacency matrix M of graph G. For any d-regular graph with adjacency matrix A, deﬁne 1 M := · A. d

Throughout this course, we use λ1 ≥ · · · ≥ λn to denote the eigenvalues of matrix M of graph G. For regular graphs, λ1 = 1 and we mainly consider the second largest eigenvalue in absolute value. The formal deﬁnition is as follows.

Deﬁnition 2.8 (spectral expansion). The spectral expansion of graph G is deﬁned by λ := max {|λ2|, |λn|}, i.e. λ = max kAxk kxk=1,x⊥u

Courant-Fischer Formula. Let B be an n by n symmetric matrix with eigenvalues λ1 ≤ · · · ≤ λn and corresponding eigenvectors v1, . . . , vn. Then

T T x Bx λ1 = min x Bx = min , kxk=1 x6=0 xT · x T T x Bx λ2 = min x Bx = min , kxk=1 x6=0 xT · x x⊥v1 x⊥v1 T T x Bx λn = max x Bx = max . kxk=1 x6=0 xTx

It is well known that λ relates to various graph properties. In particular, we shall see that there is a close connection between λ and the expansion of the graph.

Lemma 2.9. s n − d λ ≥ . d(n − 1)

3 2 Pn 2 2 Proof. Follows from tr M = n/d = i=1 λi ≤ 1 + (n − 1)λ .

Theorem 2.10. [Alo86]√ Any inﬁnite family of d-regular graphs {Gn}n∈N has spectral expansion (as n → ∞) at least 2 d − 1/d − o(1).

Definition√ 2.11 (Ramanujan graphs). A family of d-regular graphs with spectral expansion at most 2 d − 1/d is called Ramanujan graphs. Although Friedman [Fri91] showed that random d-regular graphs are close to being Ramanu- jan in the sense that λ satisfies √ λ ≤ 2 d − 1/d + 2 log(d)/d + o(1), constructing families of Ramanujan graphs with arbitrary degree is one of the biggest open problems in this area. So far, we only know the construction of Ramanujan graphs with certain degrees and these constructions are based on deep algebraic knowledge. See [LPS88] for example. Another quite important problem is to find a combinatorial construction of Ramanujan graphs. At the end of this section, we list some more interesting facts on eigenvalues of graphs: 1. If graphs G and H are isomorphic, then there is a permutation matrix P such that P · A(G) · PT = A(H) and hence the matrices A(G) and A(H) are similar. 2. There are nonisomorphic graphs with the same spectrum. See Figure 1.

Figure 1: An example for two graphs which are not isomorphic but have the same spectrum. Their common graph spectrum is 2, 0, 0, 0, −2.

2 Combinatorial Expansion of Graphs

For any d-regular graph G = (V,E), let Γ(v) be the set of neighbors of v, i.e., Γ(v) = {u | (u, v) ∈ E }. 0 For any subset S ⊆ V , let Γ(S) = ∪v∈SΓ(v) and Γ (S) = Γ(S) ∪ S. Moreoever, for any set S ⊆ V we define ∂S := E(S, S). Definition 2.12 (vertex expansion). A graph G with n vertices is said to have vertex expansion (K,A) if |Γ(S)| min ≥ A. S : |S|≤K |S| If K = n/2, then for simplicity we call G an A-expander. Informally expanders are graphs with the property that every subset (under some constraint on their size) has many neighbors outside the set. Moreover, we can use different ways to study expanders: (1) Combinatorically, expanders are highly connected graphs, and to disconnect a large part of the graph, one has to remove many edges; (2) Geometrically, every vertex set has a relatively very large boundary; (3) From the Probabilistic view, expanders are graphs whose behavior is “like” random graphs. (4) Algebraically, expanders are the real-symmetric matrix whose first positive eigenvalue of the Laplace operator is bounded away from zero.

4 Figure 2: Comparison of the vertex expansion and the edge expansion of a set of vertices of size 5.

Deﬁnition 2.13 (edge expansion). The edge expansion of a graph G = (V,E) is deﬁned by

|∂S| h(G) := min . S : |S|≤|V |/2 |S|

To explain edge expansion, let us see two examples. (1) If G is not connected, we choose one connected component as S so that |E(S, S)| = 0. Therefore h(G) = 0. (2) If G is a complete graph Kn, then |E(S, S)| = |S| · (n − |S|) and h(G) = dn/2e.

Deﬁnition 2.14 (expanders). Let d ∈ N. A sequence of d-regular graphs {Gi}i∈N of size increasing with i is a family of expanders if there is a constant > 0 such that h(Gi) ≥ for all i.

Usually, when speaking of an expander Gi, we actually mean a family of graphs {Gi}i∈N, where each graph in {Gi}i∈N is d-regular and its expansion is lower bounded by > 0. Observation 2.15. Any expander graph is a connected graph.

3 Spectral Expansion vs. Combinatorial Expansion

The next result shows that small spectral expansion implies large vertex expansion.

Theorem 2.16 (spectral expansion ⇒ vertex expansion). If G has spectral expansion λ, then 1 for all 0 < α < 1, G has vertex expansion αn, (1−α)λ2+α . Before showing the proof, we introduce some notations at first. For any probability distribution π, the support of π is defined by support(π) = {x : πx > 0}. Definition 2.17. Given a probability distribution π, the collision probability of π is defined to P 2 be the probability that two independent samples from π are equal, i.e. CP(π) = x πx. Lemma 2.18. Let u = (1/n, . . . , 1/n) be the uniform distribution. Then for every probability distribution π ∈ [0, 1]n, we have

1. CP(π) = ||π||2 = ||π − u||2 + 1/n. 2. CP(π) ≥ 1/|support(π)|.

Proof. (1) We write π as π = u + (π − u) where u⊥(π − u). By Pythagorean theorem

CP(π) = ||π||2 = ||π − u||2 + ||u||2 = ||π − u||2 + 1/n.

(2) By Cauchy-Schwarz inequality, we get

 2 X X 2 1 =  πx ≤ |support(π)| · πx x∈support(π) x

5 and hence X 1 CP(π) = π2 ≥ . x |support(π)| x

Let us turn to the proof of Theorem 2.16.

Proof. Let |S| ≤ αn. Choose a probability distribution π that is uniform on S and 0 on the S, i.e. 1 1 1 π = , ,..., , 0,..., 0 . |S| |S| |S|

Note that M is a real symmetric matrix, then M has n orthonormal eigenvectors v1, . . . , vn, then Pn we can decompose π as i=1 πi where πi is a constant multiplicity of vi. Then CP(π) = 1/|S| and by Lemma 2.18 (2), 1 1 CP(Mπ) ≥ = . |support(Mπ)| |Γ(S)|

On the other hand, by item (1) of Lemma 2.18 we have 1 CP(Mπ) − = kMπ − uk2 n 2 = kMu + Mπ2 + ··· + Mπn − uk 2 = kλ2π2 + ··· + λnπnk 1 1 1 ≤ λ2kπ − uk2 = λ2 CP(π) − = λ2 − . n |S| n

Hence 1 1 1 1 1 − ≤ CP(Mπ) − ≤ λ2 − , |Γ(S)| n n |S| n and 1 |S| |S| |Γ(S)| ≥ = = 2 1 1 1 2 |S| |S| λ2 + (1 − λ2) · |S|/n λ |S| − n + n λ 1 − n + n |S| |S| ≥ = . λ2 + (1 − λ2)α α + (1 − α)λ2

Theorem 2.19 (vertex expansion ⇒ spectral expansion). Let G be a d-regular graph. For every δ > 0 and d > 0, there exists γ > 0 such that if G is a d-regular (1 + δ)-expander according to Deﬁnition 2.12, then it G has spectral expansion (1 − γ). Speciﬁcally, we can take γ = Ω(δ2/d).

When talking about expanders, we often mean a family of d-regular graphs satisfying one of the following two equivalent properties:

• Every graph in the family has spectral expansion λ. • Every graph in the family is a (1 + δ)-expander for some constant δ.

6 References

[Alo86] N. Alon. Eigenvalues and expanders. Combinatorica, 6(2):83–96, 1986.

[Fri91] Joel Friedman. On the second eigenvalue and random walks in random d-regular graphs. Combinatorica, 11:331–362, 1991.

[LPS88] A. Lubotzky, R. Phillips, and P. Sarnak. Ramanujan graphs. Combinatorica, 8:261– 277, 1988.