Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Graphs and Spectral Methods in Complex Networks stuﬀ Francesco Caravelli Walks, spectra and the UCL resolvent

Perron- Frobenius theory

Markov YRNCS Workshop, Lucca, September 26th Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Motivation for a spectral approach

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the • Everybody knows what an eigenvalue is; resolvent • It is fast to calculate eigenvalues; Perron- Frobenius • It is in general a very elegant approach; theory

Markov • Even if you don’t want, soon or later you’ll hit an eigenvalue problem. Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli What we are not be talking about. Motivations

Matrices, • The spectrum of random matrices. Graphs and stuff • The spectrum of complex matrices. Walks, spectra and the • The spectrum of random matrices. resolvent Simple reason: you find these things anywhere, books, lectures. Perron- Frobenius I thought you might well just go and search for these stuff theory

Markov yourselves. Chains and Mixing times

First paggage So: what are we talking about? time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli What we are not be talking about.

Motivations • The spectrum of random matrices. Matrices, Graphs and • The spectrum of complex matrices. stuﬀ • The spectrum of random matrices. Walks, spectra and the resolvent Simple reason: you ﬁnd these things anywhere, books, lectures.

Perron- I thought you might well just go and search for these stuﬀ Frobenius theory yourselves.

Markov Chains and Mixing times So: what are we talking about?

First paggage Basically, stuﬀ you don’t ﬁnd so explicitly written in books. Or, time in general, not in one book. Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Propaganda...

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ Based on the upcoming review.. Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Let us start from the simplest eigenvalue problem (let’s assume Francesco A is ): Caravelli

Motivations A~x = λ~x Matrices, Graphs and which states that A can be decomposed in vectors: stuﬀ

Walks, spectra X k k and the Aij = λk (ρ ) ⊗ (ρ ) resolvent i j k Perron- Frobenius theory where ~ρ k is an eigenvector of A, corresponding to the Markov Chains and eigenvalue λk , and Mixing times First paggage X k k time Aij = λk (ξi ) ⊗ (ρj ) Resolvent and k aggregation if A is not symmetric, and ξ and ρ are left and right Pseudospectra

Applications eigenvectors.

Resources Spectral Methods in Complex Networks For a graph, there are several adjacency matrices which can be Francesco introduced. We deﬁne Caravelli

Motivations A˜(β) = A(1 − β) + βI Matrices, Graphs and stuﬀ where A is the standard unweighted adjacency matrix, while I Walks, spectra is the identity matrix. and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time

Resolvent and aggregation Pseudospectra For β = 0. Applications

Resources Spectral Methods in Complex Networks There is a theorem which links the adjacency matrix of the line Francesco Caravelli graph of the graph G,L(G), to Bij :

Motivations T Matrices, A(L(G)) = B B − 2Il Graphs and stuﬀ where Il is the identity matrix of size l. The laplacian can be Walks, spectra and the written in terms of the incidence matrix: resolvent Perron- L = BBT = D − A Frobenius theory Markov The incidence matrix is important in many ways. In fact, Chains and Mixing times whenever a link is added to the system, we can write: First paggage time T T LG+{e} = BG BG + zz Resolvent and aggregation

Pseudospectra where z is a vector associated to the new link.

Applications

Resources Spectral Methods in Complex Networks Tools: Gerschgorin circles Francesco There are a series of tools which can be used in an eigenvalue Caravelli problem. The ﬁrst, is the Gerschgorin theorem. Let A be a Motivations P matrix, and ri = j6=i aij . We introduce the i−th Gershgorin Matrices, Graphs and domain Di = {|z − aii | ≤ ri | z ∈ C}. Then the eigenvalue stuﬀ spectrum lies in the complex plane, within the union of all the Walks, spectra and the Gershgorin domains. The domains bound the position of the resolvent eigenvalues according to the so-called diagonal dominance (the Perron- Frobenius bigger are the diagonal terms, the smaller are the Gerschgorin theory domains). Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Tools: Isospectrality

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra Important tool: studying isospectral matrices. If M is a matrix, and the −1 resolvent then PMP is isospectral to M for any P with determinant

Perron- non-zero. Frobenius theory

Markov One can see this from the fact that this transformation makes Chains and Mixing times invariant both the trace operator and the determinant.

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Tools: Normality (defectiveness)

Francesco Caravelli

Motivations A matrix is normal if it can be diagonalized in a complex Matrices, Graphs and eigenbasis, i.e. stuﬀ Walks, spectra ∗ and the A = UΣU resolvent

Perron- There are several measures of normality, and for instance one Frobenius theory can check how far M,

Markov Chains and ∗ ∗ Mixing times M = A A − AA

First paggage time is far from zero. Resolvent and In general matrices such that A = A† = At∗ enjoy a spectral aggregation theorem. Pseudospectra

Applications

Resources Spectral Methods in Another matrix of interest is the graph laplacian. This is given Complex T Networks by L = B B = A − D, where D is the diagonal matrix with the Francesco degree on the diagonal. An important relation is the one of the Caravelli trace: Motivations Important properties of the laplacian: number of zero Matrices, Graphs and eigenvalues of L is related to the number of disconnected stuﬀ components. In fact, A graph G is connected if and only if 0 is Walks, spectra and the a simple eigenvalue of the Laplacian of G. resolvent For any vector ~x and Laplacian matrix L, on the graph G, we Perron- Frobenius have: theory T X 2 ~x L~x = (xi − xj ) Markov Chains and (vi ,vj )∈E(G) Mixing times

First paggage and for weighted graphs: time Resolvent and T X 2 aggregation ~x L~x = wij (xi − xj )

Pseudospectra (vi ,vj )∈E(G) Applications

Resources where wij is the weight of the link (i, j). Spectral Methods in Complex Networks

Francesco Properties of the laplacian Caravelli X Motivations Trace(L) = µj = 2L Matrices, j Graphs and stuﬀ where L is the number of links. The trace of the square of the Walks, spectra and the laplacian operator, can show the relationship between degrees resolvent

Perron- and number of links: Frobenius theory 2 X 2 X 2 Markov Trace(L ) = µj = dj + 2L Chains and j j Mixing times First paggage The eigenvalues of the laplacian are all positive. In fact, one time can see that ~xt L~x = (B~x)t (B~x), which is always positive or Resolvent and aggregation zero. This shows that L is a semipositive operator. Pseudospectra

Applications

Resources Spectral Methods in Complex One can define two laplacian matrices for directed graphs: Networks Francesco L+ = Din − A (1) Caravelli L− = Dout − At (2) Motivations Matrices, in P out P t Graphs and with Dii = j Aij and Dii = j Aij . One can then define stuff different versions of the laplacian. Walks, spectra and the First, one can define: resolvent Perron- L+ + L− Frobenius L = (3) theory 1 2 Markov Chains and or: Mixing times

First paggage p + − − + time L2 = L L + L L (4) Resolvent and aggregation We’ll focus on eqn. (3) later on. Some people deﬁne the Pseudospectra laplacian for directed graphs using the asymptotic probability Applications distribution (we’ll say something later). Resources Spectral Methods in Complex Networks

Francesco Caravelli Another interesting matrix is the graph transition matrix:

Motivations −1 Matrices, P = D A Graphs and stuff one can prove that P has the same “physics” of L. It is not Walks, spectra and the isospectral to the laplacian, but the difference between the resolvent eigenvalue is the same. Perron- Frobenius theory Proof : Use isospectrality, then Markov − 1 − 1 0 Chains and P ≈ D 2 AD 2 = P = (1 − L˜). where L˜ is the normalized Mixing times − 1 − 1 First paggage laplacian: 1 − D 2 AD 2 . This means that γP = 1 − γL, but time that difference of eigenvalues is (-) the same. Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Walks, spectra and the resolvent

Francesco Caravelli There are a series of theorems wich connect the eigenvalues of Motivations the adjacency matrix, with the connectivity of the graph. Let P Matrices, us introduce di = Aij . Then it can be proven that Graphs and j stuﬀ Q p |det(A)| = j dj . Also the number of walks in a graph is Walks, spectra and the bounded by the graph connectivity. Let us call the number of resolvent complete walks Perron- Frobenius theory X k Nk = (A )ij Markov Chains and ij Mixing times k First paggage where A is the k-th power of the adjacency matrix. Then it time can be proven that: Resolvent and aggregation X k Pseudospectra Nk ≤ dj Applications j=1

Resources Spectral Methods in Complex Networks An important quantity is the generator of walks (again, show P k Francesco this by taking the derivatives wrt z), NG (z) = k Nk z . By Caravelli introducing a vector u~ made of one, this quantity can be Motivations proven to be related to the resolvent of the adjacency matrix: Matrices, Graphs and stuﬀ T 1 NG (z) = u~ u~ Walks, spectra 1 − zA and the resolvent We can introduce the matrix J = u~T ⊗ u~ = {u u }, use several Perron- i j Frobenius matrix identities (Sherman-Morrison), to show that: theory ~T −1~ Markov det(A + zJ) = det(A)det(I + zu A u) = Chains and (1 + zu~T A−1u~)det(A), which implies a very elegant formula for Mixing times

First paggage the generator of walks (replace A → 1 − zA): time Resolvent and 1det(1 + z(J − A)) aggregation N (z) = − 1 G z det(1 − zA) Pseudospectra

Applications

Resources Spectral A solution to many problems is given by the resolvent. Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times We see that an important quantity enters these equations: First paggage time 1 Resolvent and R (z) = aggregation A 1 − zA Pseudospectra In mathematics, the resolvent formalism is a technique for Applications applying concepts from complex analysis to the study of the Resources spectrum of (linear) operators. Spectral Methods in Complex Networks

Francesco Caravelli A couple of important formulas for resolvents, are the so called First and Second resolvent equations. Motivations First resolvent equation: Matrices, Graphs and stuﬀ R(A, z1) − R(A, z2) = (z2 − z1)R(A, z1)R(A, z2) Walks, spectra and the resolvent = (z2 − z1)R(A, z2)R(A, z1). (5)

Perron- Frobenius and second resolve equation, assuming C = B − A, relates the theory resolvents of B and A: Markov Chains and Mixing times R(B, z) = R(A, z) − (I − R(A, z)C)−1 (6) First paggage time

Resolvent and These are usually very useful in calculations. aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco The determinant of the resolvent of L can be then written in Caravelli term of z. In fact:

Motivations −1 T Matrices, det(L − µI ) = det(LG − µI )det(I + (LG − µI ) zz ) Graphs and stuﬀ and now, using the rank one formula Walks, spectra −1 T −1 and the (A + uv T )−1 = A−1 − A uv A , we arrive at: resolvent 1+v T A−1u

Perron- Frobenius T zz T theory det(I + (LG − )) = 1 + z(LG − µI )z Markov µ Chains and Mixing times The incidence matrix is important, because its rank depends First paggage time strongly on the connectivity of the graph. In fact, if the graph Resolvent and is connected, the graph of the incidence matrix is N-1, where N aggregation is the number of nodes. Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli In general, positive deﬁnite matrices are the square of another Motivations matrix: L = MMT (and same for the adjacency matrices Matrices, Graphs and A = RRT ). stuﬀ A possible extension here, would be to make the incidence Walks, spectra and the matrix weighted. Then one possible question would be, what resolvent

Perron- happens to the singular values of this matrix? Frobenius theory The laplacian is connected to the incidence matrix of a graph

Markov as we have seen, but also to the degree matrix and to the Chains and Mixing times adjacency matrix of the line graph. What happens to the

First paggage eigenvalues? time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Let us use the expansion around z = 0: Complex Networks ∞ Francesco X k k Caravelli RA(z) = z A k=0 Motivations

Matrices, From the formula above, it is clear that for large walks, the Graphs and stuﬀ important eigenvalue that gives the behaviour of the generating

Walks, spectra function is the Perron-Frobinus eigenvalue and eigenvector. In and the 1 resolvent fact the resolvent of a semi-positive matrix A, given by 1−zA , Perron- converges if z ≤ λ1. (But what if A does not have a spectral Frobenius theory decomposition?)

Markov Chains and Mixing times We can in fact also introduce the spectral gap, given by the

First paggage highest and second highest eigenvalue. In fact, this quantity is time related to how fast this expansion converges. Resolvent and aggregation One can also related the number of spanning trees, to the µ ···µ Pseudospectra J 2 n determinant of det(L + n2 ) = n . Note that, in fact N has Applications a name: graph complexity, which is the number of spanning Resources trees in a graph. Spectral Methods in Complex Networks Francesco Let λ1 ≥ λ2 ≥ · · · λN be the ordered eigenvalues of A. Then Caravelli the spectral gap is bounded for an unweighted graph as Motivations σ(A) = λ1 − λ2 ≤ N, where N is the size of the graph. It is Matrices, Graphs and possible to bound from below the highest eigenvalue. This is stuﬀ given by Walks, spectra m 1 and the λ1 ≥ max(max(A )jj ) m . resolvent m≥1 j

Perron- Frobenius This means that the spectral gap is somehow related to the theory number of closed walks. The number of closed walks is in fact Markov Chains and given by: Mixing times First paggage X k k X k time Wk = (A )ii = trace(A ) = λi Resolvent and i i aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco In turn, the number of closed walks can be extracted from the Caravelli generator of closed walks for a graph G (A is the adjacency

Motivations matrix of G):

Matrices, Graphs and ∞ ∞ N stuﬀ X X X W (z; j) = (Ak ) zk = λk ρr ρr zk Walks, spectra G jj r j j and the k=0 k=0 r=0 resolvent

Perron- where ρ are the eigenvectors (can you prove this?). If the two Frobenius theory sums commute (which is a condition on the eighest

Markov eigenvalue), then you can write: Chains and Mixing times N r r First paggage X ρj ρj time WG (z; j) = 1 − zλr Resolvent and r=1 aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks What is the Perron-Frobenius Francesco theory? Caravelli

Motivations A positive matrix. Then there is a positive number r (Perron

Matrices, root) which is an eigenvaue of A, and there is an associated Graphs and stuﬀ right eigenvector ~v and left eigenvector w~ . The following limit

Walks, spectra exist: and the resolvent Ak Perron- ~ ~ t Frobenius lim k = v · w theory k→∞ r Markov Before introducing the derivatives of the vector and the root, Chains and Mixing times we introduce the generalized inverse (Moore-Penrose) of a First paggage matrix Q. If Q is such that Qu~ = ~b, then we have that time ~b = Q−1˜u~ + α~v for a certain scalar α. Resolvent and aggregation The Moore-Penrose inverse can be calculated through the Pseudospectra singular value decomposition of Q, by inverting the non-zero Applications singular values and keeping zero those which are zero, and then Resources taking the transpose operation. Note that Q−1˜Q = QQ−1˜ 6= I . Spectral Methods in Complex Networks Francesco One can show that when the spectral gap is finite, and the Caravelli k ∗ Perron root is equal to one, then limk→∞ A = A , with Motivations Matrices, A∗ = ~v · w~ t . Graphs and stuff Walks, spectra Now, for ergodic operators, implying that the graph is indeed and the resolvent strongly connected (from any i you reach any j in finite steps), Perron- you can show that the right eigenvector is equal to ~v = ~1, Frobenius theory while w~ = ~π is the asymptotic distribution. Thus Markov Chains and ∗ Mixing times Aij = πj First paggage time implying that asymptotically the distribution does not depend Resolvent and aggregation on the starting point of your random walk.

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Francesco In general, you can see that, if you write A = P λ w k ⊗ v k , Caravelli k k you have that: Motivations Matrices, s X s k k Graphs and A = λk w ⊗ v stuﬀ k Walks, spectra and the and thus if there is a dominating eigenvalue, you have that resolvent As ≈ λs w k ⊗ v k . Perron- 1 Frobenius theory Theorem: If the graph is strongly connected, the adjacency Markov Chains and matrix has a unique Perron root. Written in matricial terms, Mixing times s First paggage there exist an integer s, such that, ∀ i, j, Aij 6= 0. This means time that from any node you can reach another node in a ﬁnite Resolvent and aggregation number of steps, or that the matrix is ”ergodic”.

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Francesco One can do a similar line of thought using the the Singular Caravelli Value Decomposition approach. Each matrix M has a Motivations decomposition of the form: Matrices, Graphs and † stuﬀ M = LΣR (7) Walks, spectra and the resolvent where L is the matrix of the left eigenvectors, meanwhile R is

Perron- the matrix of the right eigenvectors. Σ contains the singular Frobenius theory values on the diagonal only, which are the square root of the t t Markov eigenvalues of the matrices MM and M M (these are Chains and Mixing times identical.. easy to prove..). First paggage Now, Σ’s are in decreasing order, so that one can show that if time there are singular values zero, the matrix can be ”compressed” Resolvent and aggregation by removing the last k columns of L and R. Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli It is easy to show that the ”ﬁrst order approximation” of R,

Motivations using the SVD, one can use only the singular value.. this is

Matrices, given by: Graphs and † stuﬀ Mjt ≈ M1 jt = Lj1Σ11R1t (8) Walks, spectra and the which is proportional to the matrix coming from the resolvent Perron-Frobenius asymptotic matrix. The proportionality Perron- ∞ † Frobenius constant is easy to be seen, M = rLj1R1t . The theory proportionality constant is thus: Markov Chains and Mixing times † Σ11 † Σ11 ∞ First paggage M1 jt = Lj1Σ11R1t = rLj1R1t = Mjt (9) time r r

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex “Perron-Properties” Networks Deutsch and Neuman proved that the derivative of the Francesco Caravelli Perron-Frobenius eigenvalue, is equal, if sρ = λ1, to :

Motivations ∂λ1 −1˜ Matrices, = (1 − RR )ji (10) Graphs and ∂aij stuﬀ

Walks, spectra −1˜ and the where R is the generalized inverse of R. resolvent 2 Perron- ∂ λ1 −1˜ −1˜ −1˜ −1˜ Frobenius = (1 − RR )li Rjk + (1 − RR )jk Rli (11) theory ∂aij ∂akl Markov Chains and Mixing times with R = λ1I − A.For the derivative of the Perron-Frobenius

First paggage vector, let us assume that A depends on a variable A(t), and time 0 A = ∂t A(t). If ~v1 is the Perron Frobenius vector, such that Resolvent and T aggregation ~z ~v1 = 1, we have the following formula for the derivative of Pseudospectra ~v1: 0 −1˜ 0 0T T −1˜ 0 Applications ~v1 = R A ~v1 − (~z ~v1)~v1 − (~z R A ~v1)~v1 (12) Resources Spectral Methods in Complex Networks Fisher’s inequality Francesco Let me now consider left (˜x) and right (x) eigenvectors for the Caravelli Perron-Frobenius eigenvalues of a matrix. Motivations We have that: Matrices, Graphs and stuﬀ x˜i Aij = λ1x˜j (13) Walks, spectra and the resolvent and

Perron- Aij xj = λ1xi (14) Frobenius theory the important fact is that right and left eigenvectors have the Markov P P i x˜i j Aij Chains and same eigenvalue. If you use (13), then λ1 = P and if Mixing times x˜i P P i i xi j Aji First paggage you use (14) λ1 = P . so in both cases we have that: time i xi Resolvent and aggregation mini Coli ≤ λ1 ≤ maxi Coli (15) Pseudospectra Applications and Resources mini Rowi ≤ θPF ≤ maxi Rowi (16) Spectral Methods in Complex Networks Convergence

Francesco Caravelli It is important to know the solution of the following problem: Motivations given ~x positive, and Matrices, 1 Graphs and o~ = ~x stuﬀ 1 − A Walks, spectra and the when is the vector o~ positive ? This can also be stated as: resolvent When does the series I + A + A2 + ··· converges? We can say Perron- Frobenius that if it converges, it converges to 1/(1 − A). To show this, theory we can use the formula Markov 2 k k+1 Chains and (1 + A + A + ··· + A )(1 − A) = 1 − A . Since A is positive Mixing times Ak is positive. Assuming that (1 − A) is invertible, and noticing First paggage time that (1 + A + A2 + ··· + Ak )(1 − A) ≤ I , we can state that Resolvent and (1 + A + A2 + ··· + Ak ) ≤ (I − A)−1. If the series converges, aggregation ∞ Pseudospectra then A = 0 (in the limit sense), than S(1 − A) = I , which 1 Applications proves that S = 1−A if the series converges. Resources Spectral Methods in Complex Networks Convergence

Francesco Caravelli

Motivations Hawkins-Simon theorem: In order for the operator (I − A) to Matrices, be positively invertible, all the principal minors of A must be Graphs and stuﬀ positive. Proof: lenghty. Walks, spectra Let A be a non-negative operator. Then there is a ρc > 0 such and the resolvent A that ∀ρ > ρc s.c. (I − ρ ) is positively invertible. It is Perron- important to understand that ρ is given by the spectral radius Frobenius c theory of the matrix, or better, the Perron-Frobenius eigenvalue. It Markov can be proved in fact, that for positive matrices the spectral Chains and Mixing times radius is positive, i.e., the greater eigenvalue of the matrix is First paggage positive. Then, it is easy to show that 1 + A + ( A )2 + ··· is time ρ ρ 1 Resolvent and positive: in fact, A is positive, ρ is positive, and thus A if it aggregation 1−( ρ ) Pseudospectra exist is positive, as it is the sum of positive terms (in the limit).

Applications

Resources Spectral Methods in Complex A Networks Moreover, since the norm of ρ is less than one, the series Francesco converges. Caravelli 1 It also shows that, in order for 1−A to exist, its Motivations Perron-Frobenius eigenvalue must be less than one. This is an Matrices, Graphs and important fact. stuﬀ This also implies the following corollary: it is a necessary and Walks, spectra and the suﬃcient condition that all the eigenvalues of A have positive resolvent real part. In fact, suppose µ is an eigenvalue. This means that Perron- Frobenius the norm of theory

Markov Chains and |µI − (I − A)| = |(µ − 1)I + A| = |(1 − µ)I − A| = 0 Mixing times First paggage If (I − A) is positively invertible, then this implies that the real time

Resolvent and part of (1 − µ) is positive, as 1 − µ cannot be greater than the aggregation spectral radius. This implies that the positive real part of µ is Pseudospectra positive (viceversa easy). Applications

Resources Spectral Methods in Complex Networks A very important theorem in graph theorem is the one relating Francesco Caravelli the number of spanning trees with the determinant of the resolvent of the laplacian. In fact: Motivations

Matrices, Graphs and N stuﬀ X k CL(x) = det(L − xI ) = Ck (L)x Walks, spectra and the k=0 resolvent m P and Cm(L) = (−1) γ(Fm), where the sum in Fm is on all Perron- Fm Frobenius the possible ways of identifying two nodes, and γ(G) is the theory number of spanning trees of a graph G. Markov Chains and What does this say? It is basically putting in relation the Mixing times number of spanning trees with all the possible ways of First paggage time identifying nodes. Resolvent and aggregation A similar and equally important relation, connects the resolvent

Pseudospectra of the adjacency matrix, with the removal of a link.

Applications

Resources Spectral Methods in This relation is very important in order to study bottlenecks: Complex Networks d X Francesco det(x1 − A) = det(x1 − A ) Caravelli dx {j} j Motivations

Matrices, where A{j} implies setting to zero a link j, and the sum is over Graphs and stuﬀ all the possible links.

Walks, spectra another important formula is given by: and the resolvent X 1 X λk d Perron- ( )jj = = log(det(x1 − A)) Frobenius x1 − A x − λk dx theory j k

Markov Chains and The important question here is: how does the Perron-Frobenius Mixing times

First paggage eigenvector ~v1 behaves in weighted matrices? One way to see time at this question is by using matrix perturbation of matrices. Resolvent and aggregation The ﬁrst thing to notice, is that the Perron vector of a

Pseudospectra nonnegative matrix A, is also the Perron vector of the resolvent −1 Applications R(A, ρ) = (ρI − A) for ρ greater than the Perron-Frobenius Resources eigenvalue λ1. Spectral Methods in Complex Networks KL divergence and Perron root Francesco The Kullback-Leibler divergence is given by: Caravelli X Motivations D(f ||g) = f (ω)[log f (ω) − log g(ω)] (17) Matrices, ω Graphs and stuﬀ If A is a matrix, and its Perron root is ρ(A) meanwhile pk and Walks, spectra and the qk are left and right Perron eigenvectors, then one can write: resolvent

Perron- Frobenius log ρ(A) ≤ D(w||s) − D(w||A s) (18) theory Markov for any vector s, and for w = p q . This implies that: Chains and k k k Mixing times First paggage log ρ(A) = infs [D(w||s) − D(w||A s)] (19) time

Resolvent and aggregation this is true for any convex function F :

Pseudospectra X sk X 1 (A s)k Applications wk F ( ) = wk F ( ) (20) pk ρ(A) pk Resources k k Spectral Methods in Complex k Networks How fast does A converges in k? Francesco We talked about convergence.. but how fast does it converge? Caravelli Let us introduce Markov chains. Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time One can study the heat kernel: Resolvent and At aggregation ht (A) = e Pseudospectra ~ Applications and in particular the convergence to the stationary state p :

Resources K(t) = |ht (A)u~ − p~| Spectral Methods in Complex Networks

Francesco Caravelli It is possible to write the dynamics in components and show: Motivations X Matrices, K(t) = |h (A)u~ − p~| ≈ e−λk t ρ Graphs and t k stuﬀ k Walks, spectra and the resolvent which shows that the convergence is logarithmic in the spectral

Perron- gap: Frobenius theory

Markov τ ≈ log(|λ1 − λ2|) Chains and Mixing times This shows that the mixing time for Markov chains is First paggage time inﬂuenced strongly by the spectral gap!

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli There is a relation between the eigenvalues of the laplacian and Motivations the one of the Markov chain matrix. This is given by: Matrices, Graphs and stuﬀ (1 − λk )dmin ≤ σk ≤ (1 − λk )dmax Walks, spectra and the resolvent where λk are the eigenvalues of M, and σk the eigenvalues of Perron- the laplacian, and dmin/max are the minimum and maximum Frobenius theory degrees of the graph and where σk is the eigenvalue of the Markov laplacian. Chains and Mixing times Any probability distribution π which satisﬁes the detailed First paggage balance condition π(i)P = π(j)P is stationary, i.e. ~πP = ~π. time ij ij

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral The mixing time of a Markov chain depends on the spectral Methods in Complex gap. In fact, if ~π is the equilibrium distribution, and P is the Networks Markov operator, we have that: Francesco Caravelli ~πP = ~π Motivations P Matrices, If d(i) = j Pij , and π satisfies the balancing condition, then Graphs and we can introduce k defined as: stuff

Walks, spectra π(i) π(j) and the k = = resolvent d(i) d(j)

Perron- Frobenius then one has: theory X X Markov 1 = π(i) = k d(i) = k2m Chains and Mixing times i i 1 First paggage with k = . One then obtains the result that the stationary time 2m distribution is proportional to the degree of the vertices, i.e. Resolvent and aggregation d(i) Pseudospectra π(i) = . 2m Applications

Resources The expected return time is defined as the expected time that it takes for a random walker to come back to i, if it just visited it. Spectral 1 2m Methods in This is given by R(i) = π(i) = d(i) . One can prove that the Complex Networks power of the limiting distribution converges to the a very Francesco specific value. In fact, if P √is derived√ from the normalized Caravelli adjacency matrix, i.e. N = DA D has the same eigenvalues P r r Motivations of P. One can then write N in spectral form, Nij = r λr vi vj . Matrices, p Graphs and The eigenvector diag( d(i)) is positive with eigenvalue 1 in N stuff , as it can checked. If the graph is not bipartite, λn > −1. One Walks, spectra and the can easily check, using the relation between the eigenvectors of resolvent P and those of N, that the eigenvalue one has ~v = √1 w~ , 1 2m Perron- p Frobenius and v1(i) = π(i). Then, one has: theory

Markov √ √ √ √ k k −1 X k t −1 Chains and P = DN D = λ D~v ~vr D Mixing times r r r First paggage time √ √ t −1 Resolvent and and one has ( D~vr ~vr D )ij = δij π(j). As such, aggregation √ √ Pseudospectra k ~ X k t −1 P = Π + λr D~vr ~vr D Applications r=2 Resources . Spectral Methods in Complex Networks Francesco Thus, Caravelli s k X k d(j) Motivations (P ) = π(j) + λ ~v ~v ~v ij r ri rj r d(i) Matrices, r=2 Graphs and stuff If |λr | < 1, then the distribution converges to Π. One can then Walks, spectra and the prove that for a random walk starting at node i, resolvent q d(j) k P(Xk = j) − π(j)| ≤ λ , where λ = max{|λ2|, |λn|}. If Perron- d(i) Frobenius we call d(t) = max ||pt (i, ·) − π ||, then we know that the theory i i t Markov theory of Markov chains, d(t) ≈ Cλ , where λ is the second Chains and t 0 t Mixing times eigenvalue of the Markov operator, with p = p P , for a 0 First paggage vector p . In fact, the mixing time is defined as the time τ, time such that after this d(τ + ) → 0 . The mixing rate is defined Resolvent and aggregation for non bipartite graphs.

Pseudospectra

Applications

Resources Spectral Let’s suppose that we have an absorbing Markov chain, in Methods in P Complex which j Mij < 1 ∀i. The fundamental matrix is given by the Networks inverse of the matrix Francesco Caravelli N = (1 − M)−1 Motivations

Matrices, : why is it fundamental? Because Nij is the probability that a Graphs and stuﬀ bouncing particle will be in state j, if starting in state i. How is

Walks, spectra this related to the matrix: and the resolvent 1 Nγ = Perron- 1 − γM Frobenius theory with γ < 1 and M a non-absorbing Markov chain? Let’s Markov 0 Chains and assume that M is a submatrix of a bigger matrix M , and that Mixing times this is given by: First paggage time ~ 0 (1 − ρ)M ρ1 Resolvent and M = t aggregation ~0 1 Pseudospectra ~ ~ Applications where 1 and 0 are vectors made of 1’s and 0’s respectively, of

Resources the same size as M, n. Now, the n + 1 state can be interpreted as the absorbing state with probability ρ ≤ 1, we have that: Spectral Methods in Complex Networks Francesco ~0t ⊗ ~0 ~0 Caravelli M0∞ → ~0t 1 Motivations

Matrices, which implies that the state is absorbing with probability 1. In Graphs and stuﬀ general, we will have that:

Walks, spectra and the (1 − ρ)nM [1 − (1 − ρ)n]~1 resolvent M0n = ~t Perron- 0 1 Frobenius theory and thus we can interpret, since 0 < 1 − ρ < 1, N as the block Markov γ Chains and part of: Mixing times

First paggage ∞ 1 1 ! time 1 X [∞ − ]~1 “ = ” M0k “ = ” 1−(1−ρ)M ρ Resolvent and 1 − M0 ~0t ∞ aggregation k=1 Pseudospectra

Applications

Resources Spectral We now introduce functions on Markov states, and the Methods in Complex normalized laplacian of a graph. A function on a graph, is Networks something which is deﬁned on the nodes j, f (j), or on the Francesco Caravelli states of the Markov chain. Let me consider now what are

Motivations called harmonic forms. These are given by functions such that:

Matrices, X Graphs and f (i) = Mij f (j) stuff j Walks, spectra and the If M represents a game, and f the payoff, then this says that resolvent the game is fair. We now introduce the Dirichlet form, given by Perron- Frobenius (f , g) =< (I − M)f , g) >π where π is the equilibrium theory distribution of the Markov chain M, f and g are functions on Markov Chains and the states, and: Mixing times X X First paggage (f , g) = [ (I − M)ik f (k)][δij πi ]g(j) time ij k Resolvent and aggregation X X = πi (Iij − Mij )f (j)g(i) Pseudospectra i j Applications X X Resources = πi (f (i) − Mij f (j))g(i) (21) i j Spectral Methods in Complex Networks Francesco As we will see shortly, this can be interpreted as the normalized Caravelli graph laplacian, given by ∆˜ = √1 (D − A) √1 , which is D D Motivations isospectral to D−1(D − A) (and (D − A)D−1). In fact, if the Matrices, 1 Graphs and graph is strongly connected, then D A is a(n) (ergodic) Markov stuff operator. In fact, if P is symmetric, we have: Walks, spectra and the resolvent 1 X (f , g) = πi Mij [f (i) − f (j)][g(i) − g(j)] Perron- 2 Frobenius ij theory X 0 0 Markov = Q(e)f (e)g (e) (22) Chains and Mixing times edges

First paggage time 1 where Q(e) = 2 πi Mij , e = (i, j) is an edge, and Resolvent and 0 aggregation f (e) = f (i) − f (j) (anagolously for g).

Pseudospectra

Applications

Resources Spectral The spectral gap can be charachterized using the Dirichlet Methods in Complex form. Let us assume that the Markov chain is reversible, i.e. Networks there is a distribution πi , such that πi Mij = Mji πj . This implies Francesco Caravelli that detailed balance condition for the distribution exists, i.e. P P P Motivations i πi Mij = i πj Mij = πj i Mij = πj . The theorem states (f ,f ) Matrices, that the spectral gap is given by 1 − γ2 = minf 2 , with Graphs and |f | stuﬀ < f , 1 >π= 0. The proof goes as follows. You can expand the P Walks, spectra function f : f = < f , fj >π fj . < f , 1 >π= 0 implies that and the j resolvent < f , f1 >π= 0. If you expand the Dirichlet form in Perron- eigenfunctions, this means that Frobenius theory Markov (f , f ) = < (1 − M)f , f >π Chains and Mixing times X 2 = | < f , fj >π | (1 − λj ) First paggage time j X Resolvent and ≥ (1 − λ ) | < f , f > |2 aggregation 2 j j=2 Pseudospectra 2 Applications = (1 − λ2)|f | (23) Resources which relates the minimization procedure to the spectral gap. Spectral Methods in Complex Networks

Francesco Caravelli 2 This equality says that (1 − λ2)|f | ≤ (f , f ). We can use this Motivations to connect the spectral gap to paths among the states. For a Matrices, 1 Graphs and certain path γi ,i ,··· ,i , one can prove that (1 − λ2) ≥ , with stuﬀ 1 2 n C C given by: Walks, spectra and the resolvent 1 X Perron- C = maxe∈A{ |γ|π(x)π(y)} (24) Frobenius Q(e) theory e˜∈γ

Markov Chains and The proof is rather easy. You have that Mixing times P f (i1) − f (in) = df (e). First paggage e∈γi1,··· ,in time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Using Cauchy-Schwarz, you obtain Methods in P 2 Complex |f (i1) − f (in)| ≤ |γ| e∈γ |df (e)| . If you multiply by Networks i1,··· ,in (1/2)π π and sum over i and j, you obtain Francesco i j Caravelli P ij (1/2)πi πj |f (i1) − f (in)| ≤ P P 2 Motivations | j(1/2)πi πj γ| |df (e)| Now, the left hand side is i e∈γi1,··· ,in Matrices, P P 2 Graphs and equal to var(f )π = i |f (i) − j πj f (j)| πi . The right hand stuﬀ side is instead bounded by C(f , f ), with Walks, spectra and the resolvent 1 X C = maxe { |γ|π(x)π(y)}, Perron- Q(e) Frobenius x,y theory 1 Markov |γ| being the length of the path; this implies 1−λ ≥ C. For Chains and P 2 Mixing times weighted edges, |γ|w = e w(e), and First paggage time 1 X C = max { |γ| π(x)π(y)}. Resolvent and e w(e)Q(e) w aggregation x,y Pseudospectra

Applications This implies that the spectral gap depends on the congestion

Resources ratio: C is large if there is an edge e such that e is on γxy for many choices of x and y. Spectral Methods in Complex Easy words.Given a collection of edges A, ∂A is the collelction Networks of edges going from inside A to outside. The Cheeger constant Francesco Caravelli can then be written as:

Motivations |∂A| Matrices, C = minA{ : A ∈ V (G), 0 < |A| ≤ |V |/2} (25) Graphs and |A| stuﬀ Example: Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time Figure: Ring of size N = 2K. Resolvent and aggregation

Pseudospectra Take A = {1, ··· , K}, then ∂A = {(K, K + 1), (1, N)}. Thus, Applications |∂A|/|A| = 2/K, and for K → ∞ C goes to zero. Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations As we saw, the spectral gap is indeed related to the relaxation Matrices, time of a Markov chain. If fact, the relaxation time is given by Graphs and 1 stuff t = . The smaller the gap, the more time it takes for the rel 1−γ2 Walks, spectra states to ”mix” (case of bottlenecks, for instance). In fact, we and the resolvent see that trel ≥ C. Perron- Frobenius theory Using this tool you can show that for for riffle shuffling of a Markov Chains and deck of 52 cards, one needs on average ≈ 7.55 ≈ 8 shuffles for Mixing times thermalization. First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations The average first-passage time m(k|i) is defined as the average Matrices, Graphs and number of steps a random walker, starting in node i, will take stuff to enter state k for the first time. More precisely, we define the Walks, spectra and the minimum time until hitting state k as resolvent Tik = min(t ≥ 0|s(t) = k and s(0) = i) for one realization of Perron- Frobenius the stochastic process. The average first-passage time is the theory expectation of this quantity. This is also called hitting time, or Markov Chains and access time. In general m(i|j) 6= m(j|i). Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral We have the relation: Methods in Complex n Networks X Francesco m(k|i) = 1 + pij m(k|j), i 6= j Caravelli j=1,j6=k

Motivations

Matrices, the average commuting time is given by Graphs and n(i, j) = m(i|j) + m(j|i). In particular, the stuﬀ

Walks, spectra Coppersmith-Tetali-Winkler theorem states that and the resolvent m(i|j) + m(j|k) + m(k|i) = m(i|k) + m(k|j) + m(j|i). This

Perron- implies that any graph can be ordered such that Frobenius theory m(i|j) ≤ m(j|i). One can deﬁne the quantity

Markov n(i|j) = m(i|j) + m(j|i). This is a distance, as Chains and Mixing times n(i, j) ≤ n(i|k) + n(k|j). One can prove that the probability

First paggage that a random walk starting at i, visits j before returning to i is time 2m given by d(i)n(i|j) . The recursion relation for the hitting time Resolvent and aggregation can be deﬁned above can be rewritten in matrix form:

Pseudospectra

Applications M + 2mD = J + PM

Resources where J is the matrix made only of one. Spectral One cannot solve for M because 1 − P is singular. One can Methods in −1 Complex then introduce the matrix Z = (1 − P + Π) . One can check Networks that M = J − 2mZD + ΠM. One can thus calculate the access Francesco Caravelli time from the matrix Z: Z − Z Motivations M(i, j) = 2m jj ij Matrices, d(j) Graphs and stuﬀ and so, if one diagonalizes the matrix N introduced before, Walks, spectra and the 2 resolvent X 1 vrj vri vrj M(i, j) = 2m ( − p ) Perron- 1 − λr d(j) Frobenius r=2 d(j)d(i) theory

Markov and so: Chains and Mixing times X 1 vrj vri 2 N(i, j) = 2m (p − p ) First paggage 1 − λr d(j) d(i) time r=2

Resolvent and aggregation and, importantly, one can bound from below and above N(i, j): Pseudospectra 1 1 2m 1 1 Applications m( + ) ≤ N(i, j) ≤ ( + ) d(i) d(j) 1 − λ2 d(i) d(j) Resources where one can see the role of the spectral gap on the bound. Spectral Methods in Complex Networks

Francesco Caravelli We can easily see, that symmetric matrices are always at Motivations equilibrium. We will try to obtain informations about the Matrices, Graphs and matrix M from its eigenvalues. We note for instance that the stuﬀ matrix M˜ is antisymmetric. Being such, Tr(M˜) = 0. Also is Walks, spectra and the the size of M, which we denote with n, is even, then M˜ has resolvent

Perron- only complex eigenvalues, while if n is odd, all the eigenvalues Frobenius ˜ theory of M are complex but one, which is zero. Now, we note that if

Markov the row sum of any matrix Q, for each row, is equal to s, then Chains and Mixing times s is an eigenvalue of Q. We can prove this by applying Q to ~ ~ ~ First paggage the vector made of ones 1. In fact, this implies that Q1 = s1. time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Resolvent and aggregation

Francesco Caravelli Let us assume we have a matrix A of size n2. Let us then Pk Motivations consider also a partitioning of n into k parts, i ri = n. We Pn Matrices, also consider the following sequence, Ni = n − (ri − 1), Graphs and j=1 stuﬀ Ni + ri = Ni−1 + 1 and with N0 = n, Nk = k. We want to Walks, spectra divide the aggregation of A into pieces. and the resolvent First of all, let me introduce the following sequence of Perron- aggregation operators for A: Frobenius theory Markov G = {G , ··· , G } Chains and k 1 Mixing times

First paggage H = {Hk , ··· , H1} time

Resolvent and where Gi is a matrix of size Ni xNi−1 and Hi has size Ni−1xNi . aggregation Now, 0i,j is a matrix of zeros of size ixj, ~1i is a vertical vector Pseudospectra ←− of size i, and 1 j is a horizontal vector of size j, and Ik is the Applications identity matrix of rank k. Resources Spectral Then, we define Gi as: Methods in Complex Networks   Ii−1 0 N←−i i−1 Francesco G = ~ Caravelli i  0i−1,Ni 1 ri 0Ni  0 I Motivations Ni ri Ni   Matrices, I ~0 Graphs and i−1 Ni ,i−1 1 stuff Hi =  0i−1,N ~1r 0r N   i ri i i i  Walks, spectra ~ and the 0i INi resolvent which are the aggregation operators defined in Zhang. Perron- Frobenius In general, we denote with A∗ = GAH. We notice that there theory

Markov are the properties Chains and ∗ ∗ ∗ Mixing times (A + B) = A + B (26) First paggage time If the couple (G, H), takes a matrix of size n to size q + 1, we Resolvent and have that GH = Iq+1, with Iq+1 identity of size (q + 1). Also, aggregation we note that Pseudospectra (A∗)n = ((AU)n)∗ (27) Applications Resources with U = HG. AU = A¯ is a very important matrix in what follows. Spectral Also, we note that Methods in ∗ ∗ Complex A¯ = A (28) Networks

Francesco and, more importantly, we note the identity: Caravelli 1 1 = ( )∗ (29) Motivations ρI − A∗ ρI − A¯ Matrices, Graphs and this proof, comes from expanding the resolvent and applying stuﬀ carefully the aggregation for each term, using the linearity Walks, spectra and the property. Another important operation, is the disaggregation resolvent operation. We observe that, for any operator φ, applied on a Perron- Frobenius matrix Q of size (q + 1)2 with the following properties: theory 2 Markov • φ(Q) has dimensions n Chains and n n Mixing times • (φ(Q)) = φ(Q )

First paggage • φ(Q1 + Q2) = φ(Q1) + φ(Q2) time • (φ(Q))∗ = Q Resolvent and aggregation If these properties are satisfied, then also the following Pseudospectra properties are satisfied: Applications • 1 1 1 ρI −φ(Q) = ρ (I − φ(I )) + φ( ρ1−Q ) Resources • 1 ∗ 1 ( ρI −φ(Q) ) = ρI −Q Spectral A disaggregation operator is given by: Methods in Complex Networks Q∗ = HQG (30) Francesco Caravelli ∗ We can introduce the matrix A∗, which is aggregated and then Motivations disaggregated. This matrix depends on the vector used for the Matrices, Graphs and aggregation. stuff ∗ ∗ −1 ∗ The matrices A and (ρ1 − A ) , can be obtained from A∗ Walks, spectra ∗ −1 and the and (ρ1 − A∗) , respectively, by aggregation. Also, the matrix resolvent (ρ1 − A∗)−1, can be constructed from (ρI − A∗)−1, by the Perron- ∗ Frobenius aggregation operation theory Markov 1 1 1 Chains and = (I − I ) + ( ) (31) Mixing times ∗ ∗ ∗ ∗ ρI − A∗ ρ ρI − A First paggage time In the aggregation problem, the comparison to be made is Resolvent and ∗ aggregation between φ(A ) and A. In fact, the difference Pseudospectra

Applications 1 1 Dρ = ∗ − (32) Resources ρI − φ(A ) ρI − A Spectral Methods in Complex Networks There are two types of possible disaggregation and aggregation Francesco Caravelli operator. The direct and indirect ones: ? 0 0 Motivations • A = H AG transposed aggregation Matrices, • A = G 0AH0 transposed disaggregation Graphs and ? stuff We also define the aggregated matrix Ai as: Walks, spectra and the resolvent Ai = Gi ··· G1AH1 ··· Hi Perron- Frobenius theory and thus: Markov An = GnAHn Chains and Mixing times is the fully aggregated matrix where, step by step, we First paggage time aggregated different set of nodes, assuming that A is ordered in Resolvent and a sequence of nodes r ··· r nodes we want to aggregate. G is aggregation 1 k k

Pseudospectra the product Gk Gk−1 ··· G1, and same for Hk .

Applications

Resources Spectral A straightforward exercise shows the structure of Hk and Gk : Methods in Complex Networks  ←−  1 r1 0 ··· 0 ··· Francesco ←− Caravelli  0 ··· 0 1 ······ 0   r2  Gn =  ←−  Motivations  0 ··· 0 ··· 0 1 r3 ···  Matrices, ············ Graphs and stuﬀ while Hn is the transpose of Gn where each block is normalized Walks, spectra to one. We can assume this sort of normalization if the and the resolvent industry we average on are of the same size, i.e., there is Perron- perfect competition. Frobenius theory We deﬁne U = HnGn. The matrix can be calculated, but this is Markov Chains and calculated from block matrices of 1/ri ’s of size ri . Mixing times Thus, the resolvent of the aggregated matrix A∗ can be written First paggage time as:

Resolvent and aggregation 1 1 = Gn Hn Pseudospectra 1 − An 1 − AU Applications Where now A multiplies U inside the inverse. First of all, let us Resources notice that the eigenvalues and eigenvectors of U are trivial in the following sense. Spectral Methods in Complex Networks The non-zero eigenvalues of U are ones and the relative Francesco eigenvectors are ~1. The other eigenvalues are zero, and the Caravelli eigenvectors of the zero eigenvalues have for ri 1 distributed Motivations with a symmetric distribution between −1 and 1. Let P be the Matrices, Graphs and matrix of the eigenvectors of U. In the P basis, U has the stuﬀ eigenvalues on the diagonal and 0 anywhere else, and the block Walks, spectra and the matrices are zero, and on the ri + 1 entry there is the resolvent eigenvalue ri+1 (assuming r0 = 0). Perron- Frobenius Thus, we can write the Leontief inverse in the basis of U: theory

Markov Chains and Mixing times 1 k 1 −1 k k = G P −1 −1 P H First paggage 1 − A 1 − P APP UP time k 1 −1 k Resolvent and = G P −1 P H (33) aggregation 1 − P APUd Pseudospectra with U being the diagonalized matrix U. Applications d

Resources Spectral Methods in Now, the structure of P is easy, for each block, the first vector Complex 1 Networks is ~1r , while in the inverse the first row is ~1r , while the other 1 ri i Francesco vectors must have the property that they sum to 0 with values Caravelli out of a symmetric distribution on the interval [-1,1]. We can Motivations calculate P−1AP then. For each diagonal block, the first entry Matrices, Graphs and is given by the sum of all the values of the block of size ri on stuff the diagonal. The off diagonal terms instead. When I multiply Walks, spectra and the by Ud , all the non-first block entries are set to zero, and thus resolvent you obtain a matrix of the form: Perron- Frobenius   theory < a >11 < a >12 ··· Markov −1 Chains and P APUd =  < a >21 < a >22 ···  (34) Mixing times ········· First paggage time Where I put only the entries of the block (i neglected the Resolvent and aggregation zeros) and the indices < a >ij means the sum over all the

Pseudospectra elements in the block ij. Now, the role of the matrix Ud is to

Applications set all the entries not in the first entry of the block to precisely Resources a zero value, leaving off only the average. Spectral In the case in which this matrix is not aggregated, U is the Methods in −1 Complex identity matrix. Thus we have to compare P APUd with Networks P−1AP. However, the argument is that, the effect of P, if the Francesco Caravelli values of A are more or less randomly distributed and positive,

Motivations the entries which are set to zero by Ud are ≈ 0 anyway. The

Matrices, reason is that these are averaged over a distribution (the other Graphs and stuﬀ eigenvectors of P) which are distributed over [-1,1] (i tested

Walks, spectra this numerically in Fig. 2). This means that we have: and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Figure: Distribution of values of the vectors with 0 eigenvalues in P. Spectral Methods in Complex Networks

Francesco One finds: Caravelli −1 −1 P AP = P APUd + δA (35) Motivations Thus we have: Matrices, Graphs and stuff 1 1 1 1 − ≈ δA Walks, spectra 1 − P−1APU 1 − P−1AP 1 − P−1AP 1 − P−1AP and the d resolvent (36) Perron- which is a small correction. To show how the two trophic levels Frobenius ∗ ˜ ∗ theory are related, let me define li and li . li are the trophic levels Markov calculated from aggregating and then calculating the trophic Chains and Mixing times levels. l˜i is the aggregated trophic level on the group, i.e. the First paggage sum of all the trophic levels in that group of industries. Let me time call ~1 the vector made of one of ”disaggregated” size, and 1˜ Resolvent and aggregation the vector of size ”aggregated”. Pseudospectra

Applications

Resources n Spectral Now, it turns out that we can write 1˜ = H ~1 and we can write Methods in ∗ n Complex li = G li . Thus we can compare trophic levels against Networks ∗ averaged trophic levels: l − l˜i : Francesco i Caravelli 1 1 1 1 l∗−l˜ = G n Hn1˜−G n ~1 = G n Hn1˜−G n Hn1˜ Motivations i i 1 − AU 1 − A 1 − AU 1 − A Matrices, Graphs and (37) stuﬀ and thus we get back to the equation we described before: Walks, spectra and the 1 1 resolvent ∗ ˜ n n˜ li − li = G ( − )H 1 (38) Perron- 1 − AU 1 − A Frobenius theory and thus: Markov Chains and Mixing times ∗ ˜ 1 1 −1 ˜ li − li = GnP( −1 δA −1 )P Hn1 First paggage 1 − P APUd 1 − P APUd time 1 −1 1 Resolvent and = Gn( P δAP )Hn1˜ aggregation 1 − AU 1 − AU Pseudospectra 1 1 = ( G P δAP−1 H )1˜ (39) Applications 1 − A∗ n n 1 − A∗ Resources ∗ where I’ve used HnGn = I and A = GnAHn. Spectral Methods in Complex Networks

Francesco Caravelli From the previous identity, we can derive an equation for δA:

Motivations −1 P AP(1 − Ud ) = δA (40) Matrices, Graphs and stuﬀ and thus, by multiplying the equation on the right and left by Walks, spectra P−1 and P respectively: and the resolvent −1 −1 Perron- AP(1 − Ud )P = PδAP . (41) Frobenius theory −1 −1 Markov Now we can use the identity P P = I and P UP = Ud , and Chains and Mixing times obtain:

First paggage time A(1 − U) = PδAP−1. (42) Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations r Matrices, Since l˜ = P i l , this means that l∗ = r < l > , Graphs and i k=1 n−Ni +k−1 i i i stuﬀ l˜i with < l >i = r , i.e., the average trophic level over the group, Walks, spectra i and the thus proving the proportionality factor. It turns out that even if resolvent δA is not a small perturbation this is true. In fact, we can Perron- Frobenius repeat the same procedure for each term in the perturbation theory and prove that all the terms are zero. Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Dynamical systems

Francesco Caravelli Bound for continuous DynSys (close to ﬁxed point):

Motivations |x(t)| ≤ |eAt x(0)| ≤ |etA||x(0)| Matrices, Graphs and stuff you see that etA controls perturbations.. it’s important. For Walks, spectra and the instance.. in population ecology.. and in many other fields.. In resolvent general, one is interested in bounding the greatest eigenvalue of Perron- Frobenius this matrix, but for non-normal operators the spectral abscissa theory might not be enough. Markov ∗ Chains and Usually, one uses the reactivity matrix A + A to make the Mixing times bound: First paggage time d tA 1 ∗ Resolvent and |e x(0)| = x(0)(A + A )/2x(0) aggregation dt |x(0)| Pseudospectra Applications and introduced the numerical abscissa ω(A) = maxz Eigs(A) Resources using the spectrum of the hermitean operator above. Spectral To see how the resolvent enters into the matter.. we note that Methods in Complex using Kato’s theorem, every matrix with m distinct eigenvalues Networks can be decomposed as: Francesco Caravelli X A = P λ + D Motivations j j j Matrices, Graphs and stuff where Pj is the spectral projector Walks, spectra and the Z resolvent 1 −1 Pj = (z − A) dz Perron- 2πi Frobenius theory and Markov Z Chains and z − λj −1 Mixing times D = (z − A) dz j 2πi First paggage time is called the nilpotent. Every matrix function can be written Resolvent and 1 R −1 aggregation using the Cauchy integral,f (A) = 2πi f (z)(z − A) dz and Pseudospectra thus −1 X 1 Applications (z − A) = Pj z − λj Resources j Spectral Methods in Complex Networks Pseudospectra

Francesco Caravelli We deﬁne the following set σ(A) for a linear operator: Motivations

Matrices, σ(A) = {z ∈ C : ||(zI − A)~v|| < } (43) Graphs and stuﬀ N Walks, spectra for some v ∈ C with ||v|| = 1. We note that σ1 (A) ⊂ σ2 (A) and the resolvent if 1 ≤ 2. The set σ(A) is called pseudospectrum (we denote Perron- with σ(A) the spectrum). The we note that the resolvent norm Frobenius theory is invariant under unitary matrix transformations: Markov Chains and 1 1 1 Mixing times ∗ | ∗ | = |U U| = | | (44) First paggage zI − U AU zI − A zI − A time

Resolvent and This implies that also aggregation ∗ Pseudospectra σ(A) = σ(U AU) (45) Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Operatively speaking you can find the boundaries such that the Graphs and stuff resolvent is almost infinite: Walks, spectra 1 and the σ (A) = {z ∈ C : ||(zI − A)−1~v|| < } (46) resolvent Perron- Frobenius theory at the eigenvalues, the resolvent will have infinite norm, and

Markov study the boundary for varying . Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Francesco In general, every normal matrix has a unitary transformation Caravelli which diagonalized the matrix. A normal matrix, is a matrix Motivations such that [A, A†] = 0, where A† is the conjugate transpose Matrices, matrix. This implies that both symmetric and anti-symmetric Graphs and stuﬀ matrices can be diagonalized; the former, with real eigenvalues, Walks, spectra and the the latter with imaginary eigenvalues (with eigenvalues coming resolvent in pairs). Perron- Frobenius Let us assume that a matrix is diagonalizable but not theory necessarily normal, and V the set of eigenvectors. The Markov Chains and important thing about normal matrices is that if δ(A) is the Mixing times ball of size around each eigenvalue, then we have First paggage time

Resolvent and σ(A) = σ(A) + δ(A) ∀ > 0 aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks smax (V ) Francesco k(V ) = (47) Caravelli smin(V )

Motivations where smax/min are the set of maximal and minimal singular Matrices, Graphs and values of V . The following theorem is true: stuﬀ Walks, spectra σ(A) + δ (A) ⊂ σ (A) ⊂ σ(A) + δ (A) (48) and the k(V ) resolvent Perron- We can use the following theorem for perturbations of inverses, Frobenius theory to bound the resolvent.

Markov −1 Chains and ||A || Mixing times −1 ||(A + E) || ≤ −1 (49) First paggage 1 − ||E||||A || time −1 Resolvent and where we assume that A is invertible, and ||E||||A || < 1. By aggregation applying this to the resolvent, we see that the norm approaches Pseudospectra ∞ when we get closer to the spectrum. Applications

Resources Spectral Methods in Complex Networks Francesco One important fact regarding perturbations of non-normal Caravelli matrices, is that small perturbations can drastically change the Motivations eigenvalues of a matrix. Let consider the matrix C given by all Matrices, Graphs and ones on the diagonal, -1 on the diagonal above. The stuﬀ determinant of the matrix C is 1, and all the eigenvalues are Walks, spectra and the one. Let us now set the bottom left corner to . One can see resolvent that det(A + B − λI ) = 0 implies (1 − λ)N + = 0. Thus, Perron- N Frobenius λ = 1 − (−) . There are N roots now, and thus the theory degeneracy has been split. The magnitude is the displacement Markov −16 Chains and can be huge! For N=100 and = 10 , the spectrum Mixing times displacement is 0.69, and thus of order 1. This implies that the First paggage time behaviour of transient phenomena can be very diﬀerent if Resolvent and perturbed even slightly. aggregation

Pseudospectra

Applications

Resources tA Spectral Let us consider the matrix e . For the matrices Methods in Complex M1 = [−1 1; 0 − 1] and M2 = [−1 5; 0 − 2]. This matrix Networks represents many physical phenomena, as for instance diﬀusion. Francesco Caravelli Let V be the matrix of eigenvectors of A. We deﬁne −1 Motivations K(V ) = ||V ||||V ||. Let α(A) be the spectral abscissa of A,

Matrices, α(A) = maxi {Re(λi )}. Then, one can prove that: Graphs and tα(A) tA tα(V ) stuﬀ e ≤ ||e || ≤ k(V )e (50) Walks, spectra and the for normal matrices K(V ) = 1. Also, one can show that: resolvent n n n Perron- ρ(A) ≤ ||A || ≤ k(V )ρ(A) (51) Frobenius theory where ρ(A) is the spectral radius of the operator. Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Graphs and k stuﬀ One can prove bunch of theorems in bounding A using the k+1 k ρ(A) Walks, spectra pseudospectral radius, i.e.— |A | ≤ and and the resolvent tA α(A) supt |e | ≥ . α is the spectral abscissa.. it basically is the Perron- Frobenius real part of the maximum eigenvalue of the real axis. theory

Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Transient phenomena dominated by Caravelli ∗ ω(A) = supi σi ([A + A ]/2). This is the behavior: Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli For non-normal matrices, these are important features to Motivations

Matrices, consider, for instance in population dynamics, in which you Graphs and Consider for instance the matrix 64x64 (Toeplix): stuﬀ

Walks, spectra and the 0.5 2 0 0 0 ··· resolvent 0 -0.5 2 0 0 ··· Perron- Frobenius 0 0 0.5 2 0 ··· theory 0 0 0 -0.5 2 ··· Markov Chains and 0 0 0 0 0.5 ··· Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time Note that the pseudospectra extend well beyond the unit disk Resolvent and for small values of epsilon. (Notice that EigTool has drawn the aggregation unit circle as a thin line.) For example, the 10( − 21) Pseudospectra pseudospectrum contains points of modulus 1.05. This implies Applications that norm(Ak ) will grow to be at least 0.05/e−21 = 5e19 for Resources some k before eventually converging to zero. Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Economics: Input-Output Francesco Leontief’ model. Caravelli

Motivations Matrices, In many cases, it is important to know what are the properties Graphs and stuﬀ of the resolvent operator. For instance, this problem is relevant Walks, spectra in economy: if A represents the production function in a and the resolvent Leontief economy, A is a semi-positive operator, and the Perron- resolvent of this operator is related to the total output of an Frobenius theory economy. The statement can be made as follows: if ~y is the Markov input of an economy, at equilibrium its output o~ will be given Chains and Mixing times by: First paggage 1 time o~ = d~ 1 − A Resolvent and aggregation Convergence → the economy produces only surplus Pseudospectra (Hawkins-Simon theorem) Applications

Resources Spectral Methods in Complex Networks

Francesco This equation can be derived from the assumption that the Caravelli internal demand of a production system is at equilibrium with Motivations the external demand: Matrices, Graphs and stuﬀ o~ = Ao~ + d~ Walks, spectra and the where one has made the demand exogenous. One can also resolvent write an equation for prices by assuming equilibrium in a closed Perron- Frobenius system. Prices then have to follow an eigenvalue equation: theory

Markov Chains and p~ = A˜p~ Mixing times First paggage and thus the price vector exist if A˜ has an eigenvalue 1. p~ is time

Resolvent and the Perron-Frobenius vector, and change in production schemes aggregation can change prices as well. Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Ecology

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ Assuming that A is the food dependency matrix of an Walks, spectra and the ecosystem, what does the quantity resolvent Perron- 1 Frobenius ~l = ~1 theory 1 − A Markov Chains and Mixing times represent? These are called trophic levels.

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent Where do you stand in the food pyramid? Perron- Frobenius In ecology, it is important to see what happens to perturbations theory of the diet matrix A, by the introduction of a new species for Markov Chains and instance, or changes in diet due to external factors: Mixing times A → A + δA. Then one is interested in the perturbed trophic First paggage time levels, Resolvent and 1 aggregation ~l0 = [ ]~1 Pseudospectra 1 − (A + δA) Applications 1 1 1 1 = [1 + δA + δA δA + ··· ]~1 Resources 1 − A 1 − A 1 − A 1 − A Spectral Methods in Complex Networks Network “Equilibrium”

Francesco Caravelli We say that a network ﬂow is at equilibrium, if the inﬂow is

Motivations equal to the outflow. This implies that, for a flow matrix M: Matrices, X X Graphs and M = M (52) stuff ij ji j j Walks, spectra and the resolvent What are the conditions on the equilibrium? Can these be Perron- Frobenius connected to the eigenvalues of a matrix? We can for instance theory study the following matrix: Markov Chains and Mixing times M˜ = M − MT (53) First paggage time If node i is at equilibrium, we have: Resolvent and aggregation X ˜ X T X Pseudospectra Mij = (Mij − Mij ) = (Mij − Mji ) = 0 (54) Applications j j j Resources Spectral Methods in Complex Networks

Perron- only complex eigenvalues, while if n is odd, all the eigenvalues Frobenius ˜ theory of M are complex but one, which is zero. Now, we note that if

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli Thus, we prove easily the following lemma: Motivations Matrices, ˜ Graphs and A necessary condition for M being at equilibrium, is that M has stuﬀ at least one eigenvalue 0. Walks, spectra and the resolvent This condition is necessary, but not suﬃcient. In fact, we can Perron- P Frobenius have i i = 0 even if not each single i is zero. Also, if n is theory odd, a zero eigenvalue is always present. If n is even though, Markov ˜ Chains and and M has no zero eigenvalue, we can be sure that the matrix Mixing times is not at equilibrium. In general, we are interested in the First paggage P 2 time quantity i i to be zero. Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex ˜ Networks We can nevertheless study how to manipulate M. If we call P ˜ ˜~ Francesco i = j Mij , we have M1 = ~. We can thus study the following Caravelli quantity: Motivations ρ = Tr[MJ˜ ] (55) Matrices, Graphs and where J is the matrix made only of ones. We ﬁrst note the stuﬀ following fact: ρ = P . This is easy to prove. In fact, we Walks, spectra i i and the have: resolvent MJ˜ = (~,~, ··· ,~), (56) Perron- Frobenius theory ˜ P P ˜ P P P and thus Tr[MJ] = i ( j Mij Jji ) = i ( j δjk k ) = j j . Markov ˜ Chains and However, since all the eigenvalues of MJ are zero but one, it Mixing times P means that λ1 = j j , and that the spectral gap of the matrix First paggage time MJ˜ is somehow a (bad) measure of how far from the Resolvent and equilibrium the matrix M is: if it is nonzero, we can be sure aggregation

Pseudospectra that the matrix is not at equilibrium. If it zero though, we

Applications cannot state that the matrix was at equilibrium.

Resources Spectral Methods in Complex Networks Things can, however, be fixed by considering the square of the ˜ Francesco matrix M: Caravelli R = −M˜ T M˜. (57) Motivations It is easy to show that |~1|2 = ~1T R~1 = P 2. In fact: Matrices, R i i Graphs and stuff −~1T M˜ T M˜~1 = (M˜~1)T · (M˜~1) = ~T · ~. Walks, spectra and the resolvent We can thus now study the following quantity: Perron- Frobenius 2 2 theory ρ2 = Tr[JRJ] = Tr[M˜ J ] (58) Markov Chains and It is easy to show that, analogously to the case before, now Mixing times M˜ 2J2 has a spectral gap given by P 2. We can thus make First paggage i time the following statement: Resolvent and aggregation A necessary and sufficienty condition for M being at ˜ 2 2 Pseudospectra equilibrium, is that the spectral gap of the matrix M J is

Applications equal to zero.

Resources Spectral Methods in Complex Networks Controllability of linear dynamical Francesco systems Caravelli

Motivations

Matrices, Now we want to show the relation between the spectrum of an Graphs and stuﬀ operator and the controllability of the simplest, linear

Walks, spectra dynamical system. and the resolvent First we introduce the concept of controllability. The Kalman’s

Perron- criterion states that, if you have a linear system of the form: Frobenius theory

Markov x˙ = C~x + Du~ (59) Chains and Mixing times then, we say that the system (the pair C,Du~) con be controlled First paggage time if, given a duration t > 0, there exist a piecewice continuous Resolvent and functionu ¯(t) on [0, t] such that the integralx ¯(t) controlled by aggregation

Pseudospectra u¯, satisﬁesx ¯(t) = xt . This means, that you can operate on

Applications u~(t) in order to bring the system where you want.

Resources Spectral Methods in Complex Networks Francesco The system in fact, can be integrated: Caravelli Z t Motivations Ct C(t−t˜) e ~x0 + e Du¯(t˜)dt˜ = xt (60) Matrices, 0 Graphs and stuﬀ

Walks, spectra A necessary and suﬃcient condition for the pair (A,B) to be and the resolvent controllable, is that the rank of the matrix given by: the 2 n−1 Perron- rank([Du~, CDu~, C Du~, ··· , C Du~]) = n. Frobenius theory

Markov Great formula! Let us focus now on a subproblem to give the Chains and Mixing times idea: we consider C to be a symmetric matrix. In Leontief 1 First paggage systems, for instance, D = α I and C = (1 − A), it is a time dynamical “reinvestment” problem. Let us work this out, Resolvent and aggregation assuming A is symmetric.

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Thus, the rank condition becomes: Caravelli

Motivations 1 1 1 2 1 n−1 Matrices, rank([ d~, (1 − A)d~, (1 − A) d~, ··· , (1 − A) d~]) = n Graphs and α α2 α3 αn stuﬀ Walks, spectra which becomes a condition on the determinant of the matrix and the resolvent being non-zero, e.g. Perron- Frobenius theory 1 1 1 2 1 n−1 Markov D = det([ d~, (1−A)d~, (1−A) d~, ··· , (1−A) d~]) 6= 0 Chains and α α2 α3 αn Mixing times First paggage Since A is symmetric, we can expand any vector in the time ~ Resolvent and eigenvectors of both the matrices 1 − A and A, given by uk , aggregation and with eigenvalues 1 − λk and λk respectively. Pseudospectra

Applications

Resources Spectral We notice then that the determinant is invariant under Methods in Complex orthogonal transformations, Networks D = det(OMOT ) = det(OT OM) = det(M), which implies Francesco Caravelli that we can consider the following equivalent problem in the

Motivations basis of u~k :

Matrices, n−1 Graphs and  ρ1 ρ1(1−λ1) ρ1(1−λ1)  stuﬀ α α2 ··· αn n−1 ρ2 ρ2(1−λ2) ρ2(1−λ2) Walks, spectra    α α2 ··· αn  and the D = det   resolvent  ············  n−1 Perron- ρn ρn(1−λn) ρn(1−λn) Frobenius α α2 ··· αn theory

Markov and, by using now the properties of the determinant, we can Chains and write: Mixing times

First paggage n−1 time  (1−λ1) (1−λ1)  1 α ··· αn−1 n n−1 Resolvent and n (1−λ2) (1−λ2) aggregation Y ρ  1 ···  D = ( i )det  α αn−1  n   Pseudospectra α  ············  i=1 n−1 Applications (1−λn) (1−λn) 1 α ··· αn−1 Resources which can be easily recognized as a Vandermonde determinant. Spectral Methods in Complex Networks

Francesco Caravelli Thus:

Motivations n n n n Y ρi Y 1 − λk 1 − λr Y ρi Y λr λk Matrices, D = ( ) ( − ) = ( ) ( − ) Graphs and αn α α αn α α stuﬀ i=1 k6=r i=1 k6=r

Walks, spectra and the Thus, what we learn is the following condition on the demand resolvent vector d~ and on the eigenvalues of the matrix A. In the ﬁrst Perron- Frobenius place, we see that if there is at least one projection of the theory

Markov demand vector on the eigenvectors of the matrix A, then the Chains and Mixing times determinant is zero. Moreover, if the matrix A has degenerate 1 First paggage eigenvalues, than the determinant is zero . time

Resolvent and aggregation

Pseudospectra

Applications Resources 1The case of a non homogeneous matrix D is less trivial. We ﬁrst set B = αi δij . Spectral Methods in Complex Networks Useful formulas: interlacing Francesco A symmetry of a matrix A, is another matrix P of the same Caravelli size, such that: Motivations [P, A] = PA − AP = 0 Matrices, Graphs and The operation [·, ·] is called commutator, and if it is zero, this stuﬀ means that A and P can be diagonalized in the same base. In Walks, spectra and the fact: PA~x = λ~x = AP~x. resolvent Why is the commutator important? This is important in Perron- Frobenius quantum mechanics, but in our setting this might be useful in theory order to interlace the eigenvalues of an operator. Two series λ Markov i Chains and and µj are interlacing if: Mixing times

First paggage µ1 ≤ λ1 ≤ µ2 ≤ λ2 ≤ · · · time Resolvent and Let’s assume that there is an orthogonal operator O such that: aggregation T Pseudospectra A = O BO. With B and A two matrices and bj and aj their

Applications eigenvalues. Then the eigenvalues of B interlace those of A,

Resources and if two eigenvalues are equal, then the two corresponding eigenvectors are the same. Spectral Methods in Complex Networks Also, several bounds on the spectra of a matrix, can be done Francesco Caravelli using the singular values of a matrix. Any matrix M can be decomposed as M = UΣV T , where U and V are orthogonal Motivations operators, and Σ is a diagonal matrix which contains the Matrices, Graphs and singular values in decreasing order stuﬀ σ (M) ≥ σ (M) ≥ · · · σ (M). There are several bounds on the Walks, spectra 1 2 N and the eigenvalues of a matrix. resolvent Perron- Y Y Frobenius |λi (A)| ≤ |σi (A)| theory i i Markov Chains and Mixing times X p X p |λi (A)| ≤ |σi (A)| First paggage time i i Resolvent and k 1/k aggregation lim |σi (A )| = |λi (A)| k→∞ Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Useful formulas

Francesco Notationwise: Deﬁne ∂Xij Xkl = δik δjl . This means that Caravelli ∂X F (X ) = B → ∂Xij F (X ) = Bij . Note that Motivations X −1T = X T −1 = X −T . Matrices, Graphs and −R stuﬀ ∂X det(X ) = det(X )X Walks, spectra −T and the ∂X det(AXB) = det(AXB)X resolvent

Perron- ∂det(Y ) −1 ∂Y Frobenius = det(Y )Trace(Y ) theory ∂x ∂x Markov X ∂det(Y ) Chains and δij det(Y ) = Xjk Mixing times ∂Xik k First paggage time iﬀ X square and invertible: Resolvent and T T −T aggregation ∂X det(X AX ) = 2det(X AX )X Pseudospectra

Applications If X is square and A symmetric:

Resources T T T −1 ∂X det(X AX ) = 2det(X AX )AX (X AX ) Spectral Methods in Complex Networks If X is not square and A not symmetric: Francesco Caravelli T T T −1 T T T −1 ∂X det(X AX ) = det(X AX )(AX (X AX ) +A X (X A X ) ) Motivations Matrices, If is small, Graphs and stuﬀ 1 2 2 2 Walks, spectra det(I +A) = 1+det(A)+Trace(A)+ (Trace(A) −Trace(A )) and the 2 resolvent Perron- Inverses: Frobenius −1 −1 −1 theory ∂x Y = −Y (∂x Y )Y Markov −1 −1 −1 Chains and ∂ X = −X X Mixing times Xij kl ki jl

First paggage −1 −1 −1 T time ∂X det(AX B) = −(X BAX ) Resolvent and 1 aggregation −1 T ∂X Trace((X + A) ) = −( 2 ) Pseudospectra (A + X )

Applications

Resources Spectral Methods in Complex 2 Networks Spectral gap and bottlenecks

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times Can we say that the link (4, 5) is a bottleneck? Can we see it First paggage time from the spectral gap? This graph is called “Dumbbell” graph.

Resolvent and We can study the laplacian for instance. aggregation

Pseudospectra

Applications

Resources 2Based on work of a MSc student I’ve supervised, Sameen Khan. Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ

Walks, spectra and the resolvent

Perron- Frobenius theory

Markov Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources We study the spectral gap as a function of the weight of the link (4,5), observing a linear relationship (which is good). Spectral Methods in Complex Networks

Francesco Caravelli

Motivations

Matrices, There is a relationship between the mixing time of a graph and Graphs and stuﬀ the Cheeger constant. This is:

Walks, spectra and the C resolvent τ ≤ |γ − γ | Perron- 1 2 Frobenius theory and thus when the spectral gap increases, the mixing time Markov decreases as the inverse (is bounded). Chains and Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Macroscopic notions of entropy Francesco Basic deﬁnition of entropy. Let us consider a Markov operator Caravelli Mij on N states, i.e.1 ≤ i, j ≤ N , such that: Motivations Matrices, N Graphs and X stuﬀ Mij = 1. (61) Walks, spectra j=1 and the resolvent

Perron- with 0 ≤ Mij ≤ 1. Entropy gives a measure of how mixing are Frobenius theory some states, i.e. how much one state is related to the other

Markov states j. We can the deﬁne the following quantity: Chains and Mixing times N First paggage 1 X 1S = − M log(M ) (62) time i N ij ij Resolvent and j=1 aggregation

Pseudospectra that for reasons it will be clear soon, we call ﬁrst order 3 Applications entropy . This quantity is indeed deﬁned on the interval 1 Resources 0 ≤ Si ≤ 1. 3We assume that 0 · log(0) = 0. Spectral Methods in Complex Networks

Francesco Caravelli

Motivations It is clear, however, the this definition is purely local, i.e., the Matrices, mixing we define is a local concept, as the first order entropy Graphs and stuff gives a sense of how much mixing there at the second step. Walks, spectra Generalizing this entropy for longer times, i.e. when the M and the resolvent operator is applied several times, is not obvious. In fact, if the Perron- operator is ergodic, we have that: Frobenius theory n ∗ Markov lim M = M . (63) Chains and n→∞ Mixing times

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks In this case, the operator M∗ is indeed trivial, i.e. each row of Francesco Caravelli M∗ is identical, due to the ergodic theorem, and thus eqn. 62

Motivations is non-trivially generalizable. On the inﬁnite time limit, we

Matrices, would have a trivial entropy from the generalization of eqn. 62: Graphs and stuﬀ N Walks, spectra ∗ 1 X X and the Si = lim − ( Mij1 ··· Mjk j ) · resolvent k→∞ N j j1,··· ,jk =1 Perron- Frobenius N theory X · log( Mij1 ··· Mj j ) Markov k Chains and j1,··· ,jk =1 Mixing times N First paggage 1 X ∗ ∗ ∗ ∗ time = − M log(M ) = S ≡ S (64) N ij ij i Resolvent and j=1 aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli This is indeed the result of the ergodic theorem, the ﬁnal state entropy is independent from the initial condition. Motivations

Matrices, However, here we argue that there a deﬁnition of entropy Graphs and stuﬀ which indeed depends on the initial condition, which is the

Walks, spectra entropy on the paths the particle is indeed going through. and the resolvent We deﬁne the following entropy on the paths the particle went

Perron- through after k-steps: Frobenius theory N Markov k 1 X Chains and Si = − Mii1 ··· Mik−1ik log(Mii1 ··· Mik−1ik ) (65) Mixing times Nk i1,··· ,ik =1 First paggage time k Resolvent and this quantity is still deﬁned on the interval 0 ≤ Si ≤ 1. aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Let us now deﬁne the following operation δ and δ2 on the Francesco Caravelli k-entropy: k k k δ Si ≡ Si − Si (66) Motivations 2 k k k Matrices, δ Si ≡ δ Si − δ Si (67) Graphs and stuﬀ And one can prove: Walks, spectra and the resolvent Lemma. ∀i and ∀k, δ k S ≥ 0 and δ2 k S ≤ 0. Perron- i i Frobenius theory

Markov The Lemma above implies the following Theorem: Chains and Mixing times ∗ First paggage Theorem. ∀i, ∃ a unique 0 ≤ Si ≤ 1 s.t. time k ∗ Resolvent and lim Si = Si . aggregation k→∞ Pseudospectra

Applications

Resources Spectral Methods in Complex We now introduce the following notation.We denote with {γ} , Networks k

Francesco a path, a string of states of length k, {i1 ··· ik } and with { i γj } Caravelli an inﬁnite string of states of the form {i ··· j}. We then denote Motivations as M({{ i γj }}k ) the ordered product Matrices, Graphs and Y stuﬀ = Mii1 Mi1i2 ··· Mik−2j

Walks, spectra { i γj }k and the resolvent

Perron- and with M({ i γj }) the inﬁnite product , Frobenius theory Y Markov = Mii1 Mi1i2 ··· Mi∞j . Chains and Mixing times { i γj }

First paggage time We also denote with P ( P ), the sums over all {{ i γj }k } {{ i γj }} Resolvent and aggregation possible paths of length k (inﬁnite) starting in i and ending in j, and P the sum over all possible paths of length k Pseudospectra { i γ}k Applications (inﬁnite) starting at i.

Resources Spectral Methods in Complex Networks

Francesco Caravelli We can then write, compactly: Motivations

Matrices, k X Graphs and Si = − M({ i γ}k ) log(M({ i γ}k )). (68) stuﬀ {{ i γ}k } Walks, spectra and the resolvent It is easy now to see that this can be written in terms of Perron- products: Frobenius theory k − Si Y M({ i γ}k ) Markov e = M({ i γ}k ) (69) Chains and Mixing times {{ i γ}k } First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Exact formula for Macroscopic Francesco Entropy Caravelli

Motivations Matrices, If one uses the 1/k weighting, by the Cesaro mean rule, the Graphs and stuﬀ entropy of each node is equal to each other. In order to Walks, spectra distinguish two diﬀerent nodes, one has to introduce a and the resolvent “forgetting” parameter : Perron- Frobenius k−1 N theory X k S = − M ··· M log(M ··· M ) (70) Markov i ii1 ik−1ik ii1 ik−1ik Chains and N i1,··· ,ik =1 Mixing times

First paggage time We can now write the recursion rule:

Resolvent and aggregation M k+1S~ = k ( k S~ + 1S~) (71) Pseudospectra k−1 Applications

Resources Spectral Methods in Complex Networks

Francesco Caravelli

Motivations And writing down all the terms, recursively, we ﬁnd:

Matrices, Graphs and k stuﬀ X k+1S~ = nMn 1S~ (72) Walks, spectra and the n=0 resolvent Perron- which in turn can be written at ∞ as: Frobenius theory 1 Markov ∞ ~ 1 ~ Chains and S = S (73) Mixing times 1 − M

First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Resources

Francesco Caravelli

Motivations

Matrices, Graphs and • My dropbox folder... stuﬀ • Walks, spectra ..or the Matrix Cookbook (google) and the resolvent • Book “Spectra and Pseudospectra” Perron- • Lectures on mixing times, markov chains and ergodicity Frobenius theory • Articles from Dietzenbacher, Deutsch, Neuman.... Markov Chains and • Books/lecture notes on Graph Spectra, and the book Mixing times

First paggage “Graph spectra of complex networks” time • Great tool for pseudospectrum: eigtool from Oxford Resolvent and aggregation

Pseudospectra

Applications

Resources Spectral Methods in Complex Networks Contact details

Francesco Caravelli

Motivations

Matrices, Graphs and stuﬀ Contact: [email protected] Walks, spectra and the These slides, and the review will be uploaded to my website: resolvent

Perron- Frobenius http://sites.google.com/site/francescocaravelli theory

Markov Chains and Twitter: @CritMemr Mixing times (updates on memristors, criticality and so on) First paggage time

Resolvent and aggregation

Pseudospectra

Applications

Resources