Isospectral Transformations: a New Approach to Analyzing Multidimensional Systems and Networks

Isospectral Graph Reductions Applications of Isospectral Transformations Summary

Isospectral Transformations: A New Approach to Analyzing Multidimensional Systems and Networks

Leonid Bunimovich

Leonid Bunimovich Isospectral Transformations: A New Approach to Analyzing Multidimensional Systems and Networks Isospectral Graph Reductions Networks and Graphs Applications of Isospectral Transformations Graph Reductions Summary Outline

1 Isospectral Graph Reductions Networks and Graphs Graph Reductions

2 Applications of Isospectral Transformations Eigenvalue Approximations Dynamical Network Stability Improved Escape Estimates in Open Systems

3 Summary

What is a network?

Basic Deﬁnition: A network is a collection of elements that interact in some way.

World Wide Web1

1 http://www3.nd.edu/ networks/Image%20Gallery/gallery.htm

High School Friendship Network2

2 http://www-personal.umich.edu/ mejn/networks/school.gif

Neural Network3

3 http://www.livescience.com/40855-brain-connections-no-neuron-is-an-island.html

Question: To what extent can a network be simpliﬁed (reduced) while maintaining one or more of its basic characteristics?

What does it mean to simplify a network? What network characteristic should we preserve?

To answer these questions we ﬁrst need a way of representing a network.

Graph of a Network A network can be represented by a graph G = (V, E, ω) with vertices V and edges E where (i) V represent the network elements; (ii) E the interactions between network elements; and (iii) ω : E → W gives the edge weights of the network edges.

E. Coli Metabolic Network3 3 http://www.kavrakilab.org/bioinformatics/metapath

Deﬁnition

If G = (V, E, ω) where V = {v1,..., vn} and eij is the edge from vi to vj, the weighted adjacency matrix M = M(G) of G is

ω(eij) if eij ∈ E Mij = . 0, otherwise

Question: Is it possible to reduce the number of vertices in a graph while maintaining the eigenvalues, including multiplicities, of its weighted adjacency matrix?

Deﬁnition For a graph G we let the edge weights W be rational functions of the form ω(λ) = p(λ)/q(λ) where (i) p(λ) and q(λ) have no common factors; and (ii) deg(p) 6 deg(q).

λ−2 1 λ

1 2 1 λ G

Deﬁnition If G = (V, E, ω) then det(M − λI) = p(λ)/q(λ) ∈ W. We call

σ(G) = {λ ∈ C : p(λ) = 0} the spectrum and −1 σ (G) = {λ ∈ C : q(λ) = 0} the inverse spectrum of G.

λ−2 1 λ

1 2 1 λ G

λ−2λ2+λ4−λ5 det(M − λI) = λ2 √ 1/3 2/3 1 −1 σ(G) = {(−1) , −(−1) , 2 (−1 ± 5)} and σ (G) = {0, 0}

Deﬁnition For G = (V, E, ω) the nonempty vertex set S ⊆ V is a structural set of G if each nontrivial cycle of G contains a vertex of S. We let st(G) denote the set of all structural sets of G.

v1 v2 G Figure: S = {v1, v2} a structural set of G

Deﬁnition For G = (V, E, ω) the nonempty vertex set S ⊆ V is a structural set of G if each nontrivial cycle of G contains a vertex of S. We let st(G) denote the set of all structural sets of G.

v v v v 3 1 v5 2 6 G Figure: T = {v1, v3, v4, v5} not a structural set of G

Deﬁnition

For G = (V, E) with S = {v1,..., vm} ∈ st(G) let Bij(G, S) be the set of paths or cycles from vi to vj having no interior vertices in S S. Furthermore, let BS(G)= Bij(G, S) denote the 16i,j6m branches of G with respect to S.

v1 v2

G Figure: Each branch is colored either red, brown, green, or blue.

Deﬁnition

Let β ∈ BS(G). If β = v1,..., vm, m > 2 and ωij = ω(eij) then

m−1 i=1 ωi,i+1 Pω(β) = m−1 Qi=2 (λ − ωii)

is the branch product of β. If Qm = 2 then Pω(β) = ω12.

ω22

ω12 ω23 β

ω12ω23 Figure: Pω(β) = . λ−ω22

Deﬁnition

For G = (V, E, ω) with the structural set S = {v1 ..., vm}, let RS(G) = (S, E, µ) where eij ∈ E if Bij(G, S) 6= ∅ and

µ(eij) = Pω(β), 1 6 i, j 6 m. β∈BXij(G,S)

We call RS(G) the isospectral reduction of G over S.

1 1 1 1 1

1 1 v1 v2 1 1 1 1 G

1 λ−1 1 1 λ−1 1 + λ v1 v2 1 λ

RS(G)

Question: What is the relation between the eigenvalues of G and the eigenvalues of RS(G)?

Proposition: (Bunimovich, Webb) For G = (V, E, ω) and S ∈ st(G)

det[M(G) − λI] det[M(RS(G)) − λI] = det[M(G|¯S) − λI]

where S¯ is the complement of S in V.

Theorem: (Bunimovich, Webb) For G = (V, E, ω) and S ∈ st(G)

−1 −1 σ(RS(G)) = [σ(G) ∪ σ (G|¯S)] − [σ(G|¯S) ∪ σ (G)].

Corollary: (Bunimovich, Webb) If G = (V, E, ω) has complex-valued weights and S ∈ st(G) then

σ(RS(G)) = σ(G) − σ(G|¯S).

v1 v2

σ(G) = {2, −1, 1, 1, 0, 0} σ(G|¯S) = {1, 1, 0, 0}

1 λ−1 1 1+ 1 λ−1 v v λ 1 1 2 λ

σ(RS(G)) = σ(G) − σ(G|¯S) = {2, −1}

Since RS(G) is a graph with edge weights in W, it is possible to further reduce this graph, i.e. sequentially reduce G.

Question: To what extent is the structure of a graph preserved under different sequences of reductions?

Let R(G; S1,..., Sm) be the graph G reduced ﬁrst over S1, then S2 and so on up to the vertex set Sm.

Theorem: Commutativity of Reductions (Bunimovich, Webb) Suppose the graph G = (V, E, ω) is reduced over the vertex sets S1,..., Sm and T1,..., Tn. If Sm = Tn then R(G; S1,..., Sm) = R(G; T1,..., Tn).

Theorem: Existence and Uniqueness (Bunimovich, Webb) Let G = (V, E, ω). Then for any nonempty set S ⊆ V there is a sequence of reductions that reduces G to the unique graph RS[G] = (S, E, µ).

Theorem: Spectral Equivalence (Bunimovich, Webb) Let τ be a rule that determines a unique vertex set τ(G) ⊂ V for any graph G = (V, E, ω). Then the relation ∼ G ∼ H if Rτ(G)[G] = Rτ(H)[H]

is an equivalence relation on the set of all graphs.

Example max in-degree Let τ be the rule τ = “in-degree > 2 ”.

v1 v2 v1 v2

GH 4 4 v1 λ v2 4 λ λ ∼ Rτ(G)[G] = Rτ(H)[H]

Question: What is the result if a network is reduced using a number of different rules?

Question: What is the result if a network is reduced using a number of different rules? Rule 1: τ=“vertex degree > 4.”

Rτ(G)[G] (unweighted)

Question: What is the result if a network is reduced using a number of different rules? max centrality Rule 2: µ=“centrality > 2 .”

Rµ(G)[G] (unweighted)

Different rules lead to different core subnetworks.

1 Isospectral Graph Reductions Networks and Graphs Graph Reductions

2 Applications of Isospectral Transformations Eigenvalue Approximations Dynamical Network Stability Improved Escape Estimates in Open Systems

3 Summary

Eigenvalue Approximations Question: Can you bound or approximate the the eigenvalues of a network?

n n×n If A ∈ C let ri(A) = |Aij|, for 1 6 i 6 n. j= , j6=i X1 Theorem: (Gershgorin) Let A be an n × n matrix with complex entries. Then all eigenvalues of A are located in the set

n [ Γ(A) = {z ∈ C : |z − Aii| 6 ri(A)}. i=1

Leonid Bunimovich Isospectral Transformations: A New Approach to Analyzing Multidimensional Systems and Networks Isospectral Graph Reductions Eigenvalue Approximations Applications of Isospectral Transformations Dynamical Network Stability Summary Improved Escape Estimates in Open Systems Example: Gershgorin Theorem Example Consider the graph G with adjacency matrix

" 0 1 0 0 1 # 0 0 1 0 0 5×5 M = 0 0 0 1 1 ∈ C . 1 0 0 0 0 1 1 1 1 0

1 1 1 1 i 1v5 1 • v4 v2 -1 ••2 • 1 1 −i 1 1

v3 G Γ(M)

n For the graph G let r(M)i = j=1,j6=i |Mij| for 1 6 i 6 n. P Deﬁnition Let the Gershgorin-type region of the graph G be the set

n [ ΓW(G) = {λ ∈ C : |λ − Mii| 6 r(M)i}. i=1

Let Ω(G) be the complex values at which M = M(G) is undeﬁned.

Theorem (Bunimovich, Webb)

The eigenvalues σ(G) ⊆ ΓW(G) ∪ Ω(G).

Question: How do isospectral graph reductions effect Gershgorin-type regions?

Theorem: Improved Gershgorin Regions (Bunimovich, Webb) Let G = (V, E, ω) where S is any nonempty subset of V. Then

ΓW(RS[G]) ⊆ ΓW(G).

v1 λ+1 λ+1 v1 λ2 v1 λ2 1 1 1 1 λ+1 λ

1 1 2λ+1 1 1 v2 2λ+1 2λ+1 v5 λ v4 v2 λ2 λ λ2 λ2 1 1 1 1 v3 1 v3 1 λ+1 λ λ2 v3 GR(G; S) R(G; S, T)

ΓW(G) = Γ(G) ΓW(R(G; S)) ΓW(R(G; S, T))

Other Approximation Results

The eigenvalue approximation techniques associated with Brauer, Brualdi, and Varga can likewise be extended to the n×n larger class of matrices W . Moreover, each of these eigenvalue approximations improve under isospectral reduction.

Network Stability

Most real networks are dynamic.

Question: Under what condition(s) will a network exhibit stable dynamics?

Deﬁnition

Local Dynamics: Ti : Xi → Xi, where i ∈ I = {1,..., n}, and (Xi, d) is a complete metric space. Interactions: F : L X → X , I ⊆ I. i j∈Ii j i i Dynamical Network: F ◦ T : X → X where F, T and X are the direct products of all Fi, Ti and Xi respectively.

A dynamical network is the pair (F, X) where F = F ◦ T.

Deﬁnition n×n For the dynamical network (F, X) let SF ∈ R be given by

∂Fi (SF)ij = sup | (x)| for 1 6 i, j 6 n. x∈X ∂xj

We call SF the stability matrix of (F, X).

Theorem (Bunimovich & Webb) The dynamical network (F, X) has a globally attracting ﬁxed point if the spectral radius ρ(SF) < 1.

Leonid Bunimovich Isospectral Transformations: A New Approach to Analyzing Multidimensional Systems and Networks Isospectral Graph Reductions Eigenvalue Approximations Applications of Isospectral Transformations Dynamical Network Stability Summary Improved Escape Estimates in Open Systems Example: Global Stability Example: Let (F, R4) be the dynamical network given by   tanh(x2) + tanh(x4) + c  tanh(x1) + c  F(x) =   , c ∈ R.  tanh(x2) + tanh(x4) + c  tanh(x3) + c

  0 1 0 0 s √  1 0 1 0  1 + 5 SF =   and ρ(SF) = ≈ 1.27  0 0 0 1  2 1 0 1 0

Since ρ(SF) > 1 it is unknown if (F, X) is stable based on our criterium.

To get a better estimate of a dynamical network’s stability we consider its underlying graph structure.

Deﬁnition:

The graph GF = (V, E) of (F, X) is the (unweighted) graph where V = {v1,..., vn} and eij ∈ E if Fi(x) depends on xj.

Deﬁnition:

For GF = (V, E), the subset C ⊆ V is a complete structural set if (i) C ∈ st(GF) containing each loop of GF; and (ii) there is at most one branch from any vi to vj in C.

Our example (F, R4) has the following graph with the complete structural set C = {v1, v3}. v2

v1 v3

v4 GF

Deﬁnition:

Suppose C is a complete structural set of GF = (V, E). For each vi ∈ C and vj ∈/ C, replace the variable xj of Fi by the function Fj. The resulting network is the restriction (RCF, X|C) of (F, X) to C.

v1 v3

v4 GF

F1 = F1(x2, x4) ⇒ RCF1 = F1(F2(x1), F4(x3)) F2(x1) F4(x3)

Example: 4 The restriction (RCF, R |C) over C = {v1, v3} has components tanh tanh(x1) + c + tanh tanh(x3) + c + c RCF(x1, x3) = tanh tanh(x3) + c + tanh tanh(x1) + c + c

11 2 sech2(|c| − ) sech (|c| − 1) sech2(|c| − ) 1 1 1 v1 v3 1 v1 sech2(|c| − ) v3 11 v4 1 q √ 1+ 5 2 ρ(SF) = 2 ≈ 1.27 ρ(SRCF) = 2sech (|c| − 1)

ρ(SRCF) < 1 if |c| > 1.881.

Theorem: (Bunimovich, W.)

Let (RCF, X|C) be a restriction of (F, X). If ρ(SRCF) < 1 then (F, X) has a globally attracting ﬁxed point.

Restrictions allow for improved stability estimates of dynamical networks and, more generally, high-dimensional dynamical systems. In terms of the graph structure of a network, a network restriction is analogous to an isospectral graph reduction.

Escape Estimates in Open Dynamical Systems

Question: Can you estimate how much a system is leaking?

Deﬁnition n Let f : I → I have the Markov partition ξ = {ξi}i=1. If H = ξ` then we let fH : I → I be the open dynamical system deﬁned by

f (x) if x ∈/ H fH(x) = . x otherwise

The set H is the hole through which points in I escape.

Deﬁnition

For fH : I → I we deﬁne Gf = (V, E) to be the graph with (i) vertices V = {v1,..., vn} and; (ii) edges E = {eij : ξj ⊆ f (ξi), i 6= `}.

1.0

0.8 v2 v 0.6 v1 3

0.4

0.2 v4

0.2 0.4 0.6 0.8 1.0 Gf ξ1 = H ξ2 ξ3 ξ4

Figure: Example of an open system fH : I → I and its graph Gf .

Deﬁnition of Survival Probability n n k Let E (fH)= µ {x ∈ I : fH(x) ∈ H, fH(x) ∈/ H, 0 6 k < n} where µ is Lebesgue measure.

n Question: Can we ﬁnd or estimate the quantity E (fH) for some n > 0?

Deﬁnition m×m Deﬁne the matrix AH ∈ R by

0 −1 inf |f (x)| for ξij 6= ∅, i 6= `, x∈ξij (AH)ij = 1 6 i, j 6 m.  0 otherwise 

m×m Similarly, deﬁne AH ∈ R using “ sup ” instead of “ inf ”.

T Let 1 = [1,..., 1] and eH = [0,..., µ(ξ`),..., 0] n n n n Let E (fH) = 1AHeH and E (fH) = 1AHeH.

Theorem (Bunimovich & Webb) n n n For any n > 0 we have E (fH) 6 E (fH) 6 E (fH).

0.6

0.5

0.4

0.3 n E (fH) 0.2

0.1 n E (fH) 1 2 3 4 n−axis n n Figure: The bounds E (fH) and E (fH) are shown for the open system fH : I → I considered in the previous example.

Remark

Using a structural set S ∈ st(Gf ) it is possible to deﬁne a delayed ﬁrst return map RfS : I → I of the open system fH : I → I to the elements of ξ indexed by S

1.0

0.8

0.6

0.4

0.2

0.0 0.2 0.4 0.6 0.8 1.0 ξ1 = H ξ2 ξ3 ξ42ξ41 Figure: The delayed ﬁrst return map RfS : I → I where S = {v1, v3, v4}. Here, RgS is delayed on the set ξ42 shown in red.

Theorem (Bunimovich & Webb)

Let S ∈ st(Gf ). Then for any n > 0

n n n n n E (fH) 6 E (RfS) 6 E (fH) 6 E (RfS) 6 E (fH).

0.015

0.010 n 10 n E (RfS) E (fH) 0.005 5 n n E (fH) E (RfS) 4 6 8 10 12 2 3 4 5 6

Delayed ﬁrst return maps allow for improved escape estimates.

Leonid Bunimovich Isospectral Transformations: A New Approach to Analyzing Multidimensional Systems and Networks Isospectral Graph Reductions Applications of Isospectral Transformations Summary Outline

1 Isospectral Graph Reductions Networks and Graphs Graph Reductions

2 Applications of Isospectral Transformations Eigenvalue Approximations Dynamical Network Stability Improved Escape Estimates in Open Systems

3 Summary

Existence & Uniqueness: If S is ANY subset of a network’s elements, it is possible to isospectrally reduce the network to a smaller network whose elements are the set S. Moreover, this reduction is unique. Commutativity of Reductions: If a network is reduced via a sequence of reductions, the resulting network depends only on the ﬁnal collection of elements over which the network is reduced.

Core Subnetworks: Any characteristic of a network’s nodes or edges, e.g. degree, centrality, can be used to isospectrally reduce a network. The reduced network is, moreover, IRREDUCIBLE with respect to this chosen characteristic. These CORE networks are typically different for different characteristics and represent hidden features of the original network’s structure (topology). Spectral Equivalence: With respect to any of these characteristics, the collection of ALL networks gets partitioned into classes of spectrally equivalent networks.

Flexibility: There is a variety of isospectral transformations that can be used to modify a network. One can, therefore, use the one that is most relevant to the speciﬁc problem/question at hand. Ease of Computation: An isospectral transformation as it can be carried out using any standard software. Applications: So far, all applications of the theory of isospectral transformations have lead to new advances. This includes new results in the areas of eigenvalue estimates, network stability, and estimates of survival probabilities in open dynamical systems.

The theory of isospectral transformations and their applications can be found in:

Thank You

Leonid Bunimovich Isospectral Transformations: A New Approach to Analyzing Multidimensional Systems and Networks