Breakdown of Interdependent Directed Networks Xueming Liu, ∗ † H

i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 1 — #1 i i

Supporting Information: Breakdown of interdependent directed networks Xueming Liu, ∗ † H. Eugene Stanley † and Jianxi Gao ‡ ∗Key Laboratory of Image Information Processing and Intelligent Control, School of Automation, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China,†Center for Polymer Studies and Department of Physics, Boston University, Boston, MA 02215, and ‡Center for Complex Network Research and Department of Physics, Northeastern University, Boston, MA 02115 Submitted to Proceedings of the National Academy of Sciences of the United States of America

I. Notions related to interdependent networks Multiplex networks. The agents (nodes) participate in ev- Both natural and engineered complex systems are not isolated ery layer of the network simultaneously. The connections but interdependent and interconnected. Such diverse infras- among these agents in different layers represent different tructures as water supply systems, transportation networks, relationships [14, 15, 28, 29, 13, 30, 31]. For example, a fuel delivery systems, and power stations are coupled together user of online social networks can subscribe to two or more [1]. To study the interdependence between networks, Buldyrev networks and build social relationships with other users on et al. [2] developed an analytic framework based on the gener- a range of social platforms (e.g., LinkedIn for a network of ating function formalism [3, 4] and discovered that the interde- professional contacts or Facebook for a network of friends) pendence between networks sharply increases system vulner- [11, 14]. ability, because node failures in one network can lead to the Temporal networks. This is a group of networks that repre- failure of dependent nodes in other networks, and this process sents the same networked system during different time pe- can become recursive and cause a failure cascade and system riods [16, 17]. The system dynamically changes with time: collapse. For example, electrical blackouts that affect large re- some nodes are added or deleted and links become active gions are usually the result of cascading failures between the or inactive during different time periods. For example, in a interdependent communication network and power grid [5]. telephone network, links represent sequences of nearly in- The development of interdependent networks triggered much stantaneous contacts [32, 33]. Telephone networks during research on the topic. Parshani et al. [6] studied a model different time periods form temporal networks [17]. similar to a real-world system: two partially interdependent Multilayer networks. These multi-dimensional multiplex networks. Shao et al. [7] developed a theoretical framework networks can be represented by a quadruplet M = for understanding the robustness of interdependent networks (VM ,EM ,V,L) (see Fig. S1), in which M is a multilayer with a varying number of supports and dependency relation- network, V the union of all the nodes of all layers, L a ships. Baxter et al. [8] studied avalanches in interdependent sequence of the sets of elementary layers (each elementary networks. Gao et al. [9] developed a general framework for layer contains multiple networks within which the nodes studying percolation in a network of networks. Focusing on are connected by different interaction type, time, or other interdependent networks and networks of networks, there are chacteristics), VM a set that contains only the node-layer other approaches to describing complex systems that comprise combinations in which a node is present in the correspond- multiple networks, e.g., interconnected networks [10, 11], mul- ing layer, and EM a set of pairs of possible combinations tilayer networks [12, 13], multiplex networks [14, 15], tempo- of nodes and elementary layers [12, 13]. ral networks [16, 17, 7] etc. They approaches differ from each A network of networks. This is a general concept describ- other and are not interchangeable. Here we describe these ap- ing systems composed of multiple networks where the net- proaches and in Fig. S1 provide schematic diagrams for the works may be of different types and may be interdependent different multiple networks. [34], be interconnected [35], may share interactions that are antagonistic (the functioning of one node causes the failure Interdependent networks. The nodes within one network of another node) [36, 37], or be coupled with other complex are connected by connectivity links, and the nodes of differ- relations [38]. A network of networks model has been used ent networks are adjacent to one another via dependency in the study of a wide range of topics, including the per- links [2, 6, 18]. Connectivity links, e.g. the friendships be- colation of a network of interdependent networks [9], the tween individuals in a social network, the business connec- spreading of an epidemic on a network of interconnected tions in a financial network, or the cables between Internet networks [27], and the robustness and restoration of a net- routers [19, 20, 21, 22, 23] enable the nodes to function co- work of ecological networks [35]. operatively as a network. Dependency links represent the functional dependency relations between two elements: if one fails, the other will also fail. For example, the functioning of a router in a computer network depends on a power station in a power grid. If a power station stops functioning, Reserved for Publication Footnotes the dependent router also stop functioning [2, 23, 24, 25]. Interconnected networks. The nodes within one network are connected by connectivity links, and the nodes of different networks are also connected by connectivity links. Alternately, interconnected networks can be regarded as interconnected communities within a single and larger network [10, 11, 26, 27]. For example, the transportation system in a city may contain a network of bus stops and sub- way stations. People may use both when moving from one place to another within the city [11].

www.pnas.org — — PNAS Issue Date Volume Issue Number 1–8

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 2 — #2 i i

II. Generating functions in a directed network III. Analytic framework of cascading failures The generating function of a directed network is built for the The system of interdependent directed networks contains two joint probability distribution of in- and out-degrees [39, 40], directed networks, network A and network B, consisting of NA denoted as and NB nodes respectively. Network A is characterized by the ∞ degree distribution PA(kin, kout), where kin and kout are the X kin kout Φ(x, y) = P (kin, kout)x y . [S1] in- and out- degree of a node respectively, whose generating

kin,kout function is

When computing the size of the giant weakly connected com- ∞ ponent (GWCC), the directness of links can be ignored. The X kin kout ΦA(x, y) = PA(kin, kout)x y , generating function takes the form Φ(w)(x) = Φ(x, x) and its kin,kout (w) (w)0 (w)0 normalized derivative is Φ1 (x) = Φ (x)/Φ (1). The 0 GWCC exists if Φ(w) (1) > 1 and the size of the GWCC W and the generating functions for the branching processes are can be obtained from the relations (w) ∂yΦA(x, y)|y=1 W = 1 − Φ (tc), [S2] ΦA1(x, 1) = , (w) ∂yΦA(1, 1) where tc = Φ1 (tc). The computation of the final GWCC in the interdependent directed networks is the same as the and computation of final connected component in the interdepen- ∂xΦA(x, y)|x=1 dent undirected networks [6, 18], which can be covered by our ΦA1(1, y) = . analytic framework. ∂xΦA(1, 1) The giant strongly connected component (GSCC) is the interception of the giant in-component and the giant out- Analogously, network B is characterized by the degree distri- component [3]. The giant in- and out-component are respec- bution PB (kin, kout), whose generating function is tively characterized by the in- and out-degree distributions, ∞ whose generating functions are Φ(x, 1) and Φ(1, y) respec- X kin kout tively. The normalized derivatives for them are respectively ΦB (x, y) = PB (kin, kout)x y , kin,kout ∂yΦ(x, y)|y=1 ∂xΦ(x, y)|x=1 Φ1(x, 1) = , Φ1(1, y) = . ∂yΦ(1, 1) ∂xΦ(1, 1) and the generating functions for the branching processes are [S3] The size of the giant in- I and out-component O can be de- ∂yΦB (x, y)|y=1 termined in the similar way as the study of the GWCC [39], ΦB1(x, 1) = ∂yΦB (1, 1) which are respectively I = 1 − Φ(xc, 1) and O = 1 − Φ(1, yc), where x = Φ (x , 1) and y = Φ (1, y ). The nontrivial so- c 1 c c 1 c and lutions xc < 1 and yc < 1 respectively mean the probability that the connected component obtained by moving against the ∂xΦB (x, y)|x=1 ΦB1(1, y) = . link directions starting from a randomly chosen link is finite, ∂xΦB (1, 1) and the probability that the connected component obtained by moving along the link directions starting from a randomly Network A and network B are coupled by dependency links, chosen link is finite. with the coupling strength qA and qB ranging from 0 to 1, The probability that a node belongs to the giant in- where qA is the fraction of nodes in network A (A-nodes) that kin kout depend on the nodes in network B (B-nodes), and qB is the component is 1 − xc , and 1 − yc is the probability that a node belongs to the giant out-component. Each node belong- fraction of B-nodes that depend on A-nodes. ing to the GSCC is the node belonging to the giant in- and out-component at the same time. Thus the probability that a A. Dynamic process of cascading failures. We present the solu- kin kout node belongs to the GSCC is (1−xc )(1−yc ). The relative tion for the final GWCC and GSCC size step by step according size of the GSCC in a single network is [39] to the cascading process, as shown in Fig. S2. To generalize

X kin kout the theory of the computation of the final GWCC and the S = P (kin, kout)(1 − xc )(1 − yc ) final GSCC, we define pA(p1) and pB (p2) as the fraction of kin,kout nodes belonging to the giant components after the removal of = 1 − Φ(xc, 1) − Φ(1, yc) + Φ(xc, yc). [S4] a fraction of 1 − p1 nodes from network A and a fraction of It can be inferred that the generating function for computing 1 − p2 nodes from network B respectively. When computing the size of the GSCC is the size of the final GWCC, Φ(s)(x, y) = Φ(x, 1) + Φ(1, y) − Φ(x, y). [S5] p (p ) = W (p ) = 1 − Φ(w)(p x + 1 − p ), The relative size of GSCC can be written as A 1 A 1 A 1 A 1 (s) S = 1 − Φ (xc, yc), [S6] where where x = Φ (x , 1) and y = Φ (1, y ). (w) c 1 c c 1 c x = Φ (p x + 1 − p ), Once a fraction 1 − p of nodes are randomly removed from A A1 1 A 1 a network, the fraction of nodes belonging to the final GSCC, (s) and denoted by p∞ , is given by (s) (w) p∞ = pS(p), [S7] pB (p2) = WB (p2) = 1 − ΦB (p2xB + 1 − p2), where S(p) = 1−Φ(s)(px (p)+1−p, py (p)+1−p) and x (p), c c c where yc(p) respectively satisfy xc(p) = Φ1(pxc(p) + 1 − p, 1) and (w) yc(p) = Φ1(1, pyc(p) + 1 − p). xB = ΦB1 (p2xB + 1 − p2).

2 www.pnas.org — — Footline Author

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 3 — #3 i i

A A B B While where zin, zout, zin and zout are arbitrary complex variables (s) satisfying pA(p1) = SA(p1) = 1 − ΦA (p1xA + 1 − p1, p1yA + 1 − p1) and  1 − zA 1 − zA  in = out = (s)  A A pB (p2) = SB (p2) = 1 − Φ (p2xB + 1 − p2, p2yB + 1 − p2)  1 − ΦA1(zin, 1) 1 − ΦA1(1, zout) B   (s) B B for the computation of the final GSCC size, where  p1(1 − qA(1 − (1 − Φ (zin, zout))p2)), B [S11] x = Φ (p x + 1 − p , 1), y = Φ (1, p y + 1 − p ) 1 − zB 1 − zB A A1 1 A 1 A A1 1 A 1  in = out =  B B and  1 − ΦB1(zin, 1) 1 − ΦB1(1, zout)   p (1 − q (1 − (1 − Φ(s)(zA, zA ))p )). xB = ΦB1(p2xB + 1 − p2, 1), yB = ΦB1(1, p2yB + 1 − p2). 2 B A in out 1 At each stage t of the cascade failures, ψ0 and φ0 are respec- t t For any given parameters qA, qB , p1, p2 and arbitrary degree tively defined as the fraction of nodes remaining in network distributions PA(kin, kout), PB (kin, kout), by solving Eq. (S11), A and network B, and ψt and φt the fractions of nodes be- A A B B we obtain the values of zin, zout, zin and zout, then we sub- longing to the giant components (which remain functional) of stitute them into Eq. (S10) to obtain the sizes of the final networks A and B. After initial attacks on both networks, the GSCC for both networks. Substituting the generating func- remaining fractions of nodes in network A and network B are 0 0 tions of the giant weakly connected component (GWCC) size ψ1 = p1 and φ1 = p2 respectively; The functional parts in net- 0 0 0 0 with the functions of the GSCC size, this analytic framework work A and network B are ψ1 = ψ1pA(ψ1) and φ1 = φ1pB (φ1) can be used to compute the size of the final GWCC, which is respectively. Due to the interdependence between network A equivalent to the giant cluster when regarding the networks as and B, the failure in either network will cause further failure undirected networks. on the other network. At the second stage of the cascading For simplicity and without lose of generality, we set qA = failure, the failed B-nodes will cause additional qA(1−φ1) frac- q = q and p = p = p. When network A and network tion of A-nodes to fail. Thus the remaining fraction of nodes B 1 2 B share the same joint degree distribution P (kin, kout) and in network A is 0 0 N → ∞, ψm = φm and ψm = φm. We introduce two new 0 0 ψ2 = p1[1 − qA(1 − φ1)] = p1[1 − qA(1 − pB (φ1)p2)], variables for simplification,

and the fraction of nodes in the giant component ψ2 = 0 0 0 0 0 0 zin = ψmxA + 1 − ψm zout = ψmyA + 1 − ψm. [S12] ψ2pA(ψ2). Similarly, the remaining fraction of nodes in network B is 0 0 (s) (s) φ2 = p2[1 − qB (1 − pA(ψ2)p1)], Thus the size of the final GSCC ψ∞ in network A and φ∞ in network B are the same as and the fraction of nodes in the giant component φ2 = 0 0 φ2pB (φ2) Accordingly, the sequences of ψn and φn can be (s) (s) (s) (1 − zin)(1 − Φ (zin, zout)) determined step by step. The general form for them is ψ∞ = φ∞ = , [S13] 1 − Φ1(zin, 1) 0 0 0 ψ1 = p1, ψ1 = ψ1pA(ψ1) 0 0 0 φ1 = p2, φ1 = φ1pB (φ1) where zin and zout are arbitrary complex variables satisfying ψ0 = p (1 − q (1 − p (φ0 )p )) ψ = ψ0 p (ψ0 )..., 2 1 A B 1 2 2 2 A 2  1 − z 0 0 0 0 in = p(1 − q(1 − (1 − Φ(s)(z , z )p)), ψn = p1(1 − qA(1 − pB (φn−1)p2)), ψn = ψnpA(ψn)  in out  1 − Φ1(zin, 1) φ0 = p (1 − q (1 − p (ψ0 )p )), φ = φ0 p (φ0 ). [S8] n 2 B A n 1 n n B n  1 − zout (s)  = p(1 − q(1 − (1 − Φ (zin, zout)p)). At the end of the cascade, no further failures happen. The re- 1 − Φ1(1, zout) maining fraction of nodes in network A and network B reach [S14] 0 0 0 0 stable values ψm = ψm+1 and φm = φm+1 respectively. The For any given q, p and P (kin, kout), by solving Eq. (S14), 0 0 two unknowns ψm and φm are determined by we obtain zin and zout, then we substitute zin and zout into Eq. (S10) to obtain the size of GSCC for both networks. From ( ψ0 = p (1 − q (1 − p (φ0 )p )), m 1 A B m 2 Eq. (S14), zout can be treated as a function of zin and vice 0 0 [S9] (s) φm = p2(1 − qB (1 − pA(ψm)p1)). verse. We introduce an analytic function R (zin, q) ≡ 1/p for zin ∈ [0, 1] as the root of Eq. (S14) when it is regarded as a The GSCC sizes of two interdependent ER networks in each quadratic equation of 1/p. Since 1/p cannot be negative, only step of the cascading failure is shown in Fig. S3, which agree one root of Eq. (S14) has a physical meaning, well with the simulation results (symbols). q B. Stationary state. For two interdependent networks with ar- (s) 2 (s) (s) (1 − Φ (zin, zout))(1 − q + (1 − q) + 4qψ∞ ) bitrary degree distributions, we can compute the sizes of the R (zin, q) = . 2(1 − zin) GWCC and the GSCC in network A and those in network B [S15] at each stage of the process of cascade failures using this ana- Accordingly, when compute the size of the final GWCC, lytic framework. When the number of nodes in both networks (s) zin = zout and N → ∞, the size of the final GSCC ψ∞ in network A and (s) φ∞ in network B are (w) 0 0 (w) xA = yA = Φ1 (ψmxA + 1 − ψm) = Φ1 (zin). [S16]  (1 − zA)(1 − Φ(s)(zA, zA ))  ψ(s) = in A in out ,  ∞ A Using Eqs. (S12) and (S16), we find that  1 − ΦA1(z , 1) in [S10] B (s) B B 1 − z  (s) (1 − zin)(1 − ΦB (zin, zout)) 0 in  ψm = . [S17]  φ∞ = B . (w) 1 − ΦB1(zin, 1) 1 − Φ1 (zin)

Footline Author PNAS Issue Date Volume Issue Number 3

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 4 — #4 i i

(s) The size of the ﬁnal GWCC is zc, i.e., F (zc, q) = 0. Thus at the critical coupling strength (s) (w) qc2, the function F (z, q) = 0 has only one solution z = zvc, (w) (w) (1 − zin)[1 − Φ (zin)] ψ∞ = φ∞ = ψm = (w) , [S18] which can be guaranteed if and only if 1 − Φ1 (zin) (s) (s) V (zvc, qc2) = ∂zF (z, q)|z=z ,q=q = 0. where zin satisﬁes vc c2

1 − zin (w) Thus the critical point q can be determined by the following = p[1 − q(1 − [1 − Φ (z )]p)]. [S19] c2 (w) in equations 1 − Φ1 (zin) ( (s) F (zvc, qc2) = 0, To solve Eq. (S19), we introduce an analytic function [S23] (s) (w) 1 V (zvc, qc2) = 0. R (zin, q) ≡ p for zin ∈ [0, 1] as

q (ii) The other critical coupling strength qc1 separates the (w) 2 (w) [1 − Φ1 (zin)] 1 − q + (1 − q) + 4qψ∞ hybrid and the first order phase transitions. In both first or- (w) R (zin, q) = , der and hybrid phase transitions we observe a peak at z = zfc 2(1 − zin) (s) which is the smaller root of F (zfc, q) = 0. But the differ- [S20] ence between these two is that in the region of a first order which can be used to analyze the phase transition of interde- phase transition R(z , q) is the maximum value of the func- pendent undirected networks. fc tion R(s)(z, q) for z ∈ [0, 1], while in the region of hybrid phase (s) (s) transition R (zfc, q) < limz→1 R (z, q). Thus at the critical IV. Critical points in phase transitions coupling strength qc1, the system should satisfy In the main text, the diverse percolation behaviors in inter-  (s) (s) dependent directed ER networks and interdependent SF net- R (zfc, qc1) = lim R (zfc, qc1),  z→1 works with in- and out- degree correlations and without in- [S24] (s) and out- degree correlations indicate that the system shows:  F (zfc, qc1) = 0. (i) a second order phase transition when q < qc2 and the perco- II lation threshold is pc ; (ii) a first order phase transition when V. Percolation on interdependent ER directed networks I q > qc1 and the percolation threshold is pc ; (iii) a hybrid phase The node degrees in a directed Erd˝os-Rényi (ER) network are transition when qc2 < q < qc1, where the system undergoes a Poisson-distributed and have a generating function I sharp jump at pc followed by a second order phase transition II (x+y−2) at pc [41]. Notes that for the interdependent undirected ER Φ(x, y) = e 2 , [S25] networks, there is no hybrid phase transition. In the following paragraphs we present a theoretical analysis of the percolation where < k > is the average degree of the network. Since the II I thresholds pc and pc . in- and out-degree of a node in ER networks are independent, (i) When the system exhibits a second order phase tran- Φ(x, y) is equivalent to Φ(x, x). The generating functions for sition, R(s)(z, q) is a monotonically increasing function of z, computing the size of GSCC in ER networks can be written and the maximum value of R(s)(z, q) is obtained when z → 1, (s) (s) 2 (x−1) (x−1) 2 (x−1) which corresponds to the reciprocal of percolation threshold Φ (x) = 2e − e Φ1 (x) = e . II (s) pc . Moreover, for the hybrid phase, R (z, q) is a non- [S26] monotonic increasing function of z, but the maximum value of Substituting the generating functions of ER networks R(s)(z, q) is also obtained when z → 1, corresponding to the Eq. (S26) into Eq. (S10), the size of the final GSCC in the II interdependent directed ER networks is reciprocal of percolation threshold pc . Thus the percolation

threshold can be written (s) 2 (z−1) p∞ = (1 − z)(1 − e ), [S27] II 1 1 pc = (s) = 0 . [S21] limz→1 R (z, q) Φ1(1, 1)(1 − q) where z satisﬁes

Note that for the case of ER networks with average degree hki, (s) 1 (z−1) II R (z, q) = = (1 − e 2 ) we obtain pc = 2/(hki(1 − q)). p (ii) When the system displays a hybrid or a ﬁrst order q (s) (z−1) phase transition, R (z, q) as a function of z has a peak (1 − q + (1 − q)2 + 4q(1 − z)(1 − e 2 )) (s) × . at zc which is the smaller root of F (zc, q) = 0, where 2(1 − z) (s) (s) F (z, q) = ∂zR (z, q). Accordingly, the percolation thresh- [S28] I old pc is I 1 The derivation of R(s)(z, q) is pc = (s) . [S22] R (zc, q) (1−z) The solutions of the critical coupling strengths q and q dR(z) e 2 − (1 − z) − 1 c1 c2 F (s)(z, q) ≡ = 2 by numerical simulations have been proposed by Zhou et al. (1−z) dz p(1 − z)(e 2 − 1) [18], and next we will show how to analyze the critical coupling (z−1) strengths explicitly. 2q{e 2 [ (1 − z) − 1] + 1} − 2 , [S29] (i) The critical coupling strength qc2 separates the second p(1 − q + a(s))a(s) order and the hybrid phase transitions. In the region of a second order phase transition, R(s)(z, q) monotonically increases where a is (s) as z increases, i.e., F (z, q) ≥ 0 for any z ∈ [0, 1]. In the q (s) (s) (z−1) region of a hybrid phase transition, R (z, q) has a peak at a = (1 − q)2 + 4q(1 − z)(1 − e 2 ). [S30]

4 www.pnas.org — — Footline Author

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 5 — #5 i i

I II The percolation thresholds pc , pc and the critical coupling VI. Interdependent scale-free networks strengths qc1, qc2 for the final GSCC size of interdependent Scale-free (SF) networks approximate real networks such as directed ER networks can be calculated based on the functions protein-protein interaction network [42], Internet [43] and so- (s) (s) (s) (s) R (z, q), F (z, q) and V (z, q) ≡ dF (z, q)/dz. The per- cial networks [44]. Given two interdependent directed SF net- colation threshold works [45, 46], they are characterized by pA(kin, kout) and pB (kin, kout), where the power-law exponents of the in- and II 1 A A pc = , [S31] out-degree distribution are λin and λout respectively in net- (1 − q) B B 2 work A and λin and λout in network B. we apply our framework to study its robustness and compute the critical coupling I II where the size of the final GSCC continuously decreases to 0 strengths qc2, qc1 and the percolation thresholds pc and pc . in second order and hybrid phase transition. Both the cases that the in- and out-degree of a given node Similarly, when computing the size of the final GWCC in in the SF network are independent or correlated are studied interdependent directed ER networks we can ignore the di- in our work. The correlation between the in- and out-degree rectness. The generating function and its derivation can be results in different behavior of the system. written Φ(w)(x) = Φ(w)(x) = e(x−1). [S32] A. The system of uncorrelated SF networks. In a directed SF 1 network in which there is no correlation between the in-degree Substituting Eq. (S32) into Eq. (S18), we get the size of the and out-degree of a given node, the degree distribution is de- final GWCC, fined by the generating function 1−λ (w) PMin 1−λin in kin p = 1 − z, [S33] m [(kin + 1) − kin ]x ∞ Φ(x, y) = in 1−λ 1−λin [(Min + 1) in − m ] where z satisfies in PMout 1−λout 1−λout kout [(kout + 1) − k ]y mout out (z−1) p 2 × 1−λ , [S38] 1 (1 − e )(1 − q + (1 − q) + 4q(1 − z)) [(M + 1)1−λout − m out ] = . out out p 2(1 − z) where min, Min are respectively the minimum and maximum [S34] in-degree, and mout, Mout are the minimum and maximum (w) The derivation of the function R (z, q) is out-degree of the SF network respectively; λin and λout are respectively the power-law exponents of the in- and out- degree 2 (1−z) distributions. For simplicity and without loss of generality, we (w) e − (1 − z) − 1 F (z, q) = 2 assume that the in- and out-degree distributions are the same (1−z) p(1 − z)(e 2 − 1) and the degree distributions are the same in network A and 2mq network B. It follows that m = m = m, M = M = M − . [S35] in out in out p(1 − q + a(w))a(w) and λin = λout = λ. In an uncorrelated SF network, the in- and out-degree of a where a(w) = p(1 − q)2 + 4q(1 − z). given node are independent. Φ(x, y) is equivalent to Φ(x, x), so that the generating functions for computing the size of the The percolation thresholds pI , pII and the critical cou- c c GSCC can be written pling strength qc1, qc2 of the final GWCC can be obtained  M 1−λ 1−λ k (w) (w) (w) 2 P [(k + 1) − k ]x based on the functions R (z, q), F (z, q) and V (z, q) ≡  Φ(s)(x) = m (w)  1−λ 1−λ 2 dF (z, q)/dz. Here qc1 = qc2 = qc, i.e., there is no hybrid  [(M + 1) − m ]  transition in the percolation on the final GWCC of interde-  −(PM [(k + 1)1−λ − k1−λ]xk)2 pendent directed ER networks. The system changes from a − m ,  [(M + 1)1−λ − m1−λ]2 first-order to a second-order phase transition when q deduces.   PM 1−λ 1−λ k At the critical coupling strength qc, both conditions for the  [(k + 1) − k ]kx  Φ (x, 1) = Φ (1, x) = m . first-order and the second-order phase transitions are satis-  1 1 (M + 1)1−λ − m1−λ fied, and we get [S39] (w) By substituting Eq. (S39) into Eq. (S8), we get the GSCC size F (z, qc)|z→1 = 0. [S36] of the interdependent SF networks at each stage of cascading From Eqs. (S35) and (S36), the critical coupling strength is failure. When computing the size of the GWCC, the directness can √ be ignored. Since the in-degree and out-degree of a node are < k > +1 − 1 + 2 < k > q = . [S37] independent, the generating function can be transformed into c < k > (PM [(k + 1)1−λ − k1−λ]xk)2 Φ(w)(x) = m , [S40] When q < qc, the size of the final GWCC of interdependent [(M + 1)1−λ − m1−λ]2 directed ER networks shows a second order phase transition, II 1 The generating function of the branching process is defined as whose percolation threshold is pc = (1−q) . When q ≥ qc, the system shows the first order phase transition and the per- PM [(k + 1)1−λ − k1−λ]xk Φ(w)(x) = m colation threshold 1 [(M + 1)1−λ − m1−λ] PM 1−λ 1−λ k−1 (zc−1) (w) [(k + 1) − k ]kx I 1 (1 − e )(1 − q + a ) m × M . [S41] pc = (w) = , P 1−λ 1−λ R (zc) 2(1 − zc) m [(k + 1) − k ]k Substituting Eqs. (S40) and (S41) into Eqs. (S18) and (w) p 2 where a = (1 − q) + 4q(1 − zc) and zc satisfies (S20) we obtain the size of the final GWCC and the function (w) (w) F (zc, q) = 0. R (z, q) for interdependent uncorrelated SF networks.

Footline Author PNAS Issue Date Volume Issue Number 5

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 6 — #6 i i

B. The system of correlated SF networks. In some real net- C. Analysis of the critical coupling strength qc1. In the main works, the in-degree kin and out-degree kout of a node are text, we find that the critical coupling strength satisfies the correlated [47]. We use a coefficient α to measure the correla- equation tion between the in- and out-degree. The in- and out-degree (s) (s) R (zfc, qc1) = lim R (z, qc1), [S47] are positively correlated when 0 < α < 1 and negatively cor- z→1 related when −1 < α < 0. Besides, there is no correlation (s) (s) between the in- and out-degree when α = 0; The correlation where F (zfc, qc1) = 0. When z → 1, ψ∞ = 0 so we can α between kin and kout is assumed to obey kout ∝ kin. The aver- infer from Eq. S15 that α kin age out-degree for all the nodes with kin is α < k >, where (s) 0 lim R (z, q) = Φ1(1, 1)(1 − q). [S48] < k >=< kin >=< kout > denoting the average in- or out- z→1 degree. Thus we assume P (kout|kin) is a Poisson distribution with a generating function When studying the final GWCC of interdependent SF net-

kα works with or without in-degree and out-degree correla- in (y−1) X kout tions, and when studying the final GSCC of the interde- G(y) = P (kout|kin)y = e in . [S42] pendent SF networks with in-degree and out-degree cor- The generating function for the degree distribution of the correlations and the maximum in-/out-degree M → ∞ and (s) related network is 2 < λ ≤ 3, limz→1 R (z, q) → ∞ for q ∈ [0, 1). Thus, (s) (s) kα limz→1 R (z, q) > R (zfc, q) for q ∈ [0, 1), and qc1 = 1 in (y−1) X kin Φcor(x, y) = P (kin)x e in . [S43] in these three cases. Because in numerical computation the maximum in-/out-degree M cannot reach ∞, qc1 < 1. For simplicity and without loss of generality, we set α = 1 to show the percolation of robustness in the system of correlated SF networks, which can be extended to the general case of VII. International trade networks α. The generating function of the degree in a correlated SF Table S1 shows eight international commodity-specific trade network is networks in the year 2013. The nodes of each represent PM [(k + 1)1−λ − k1−λ](xey−1)k countries, and the directed links represent trading relations. Φ (x, y) = m . [S44] cor (M + 1)1−λ − m1−λ The networks are arranged into interdependent network pairs, yielding 28 interdependent network pairs. When we apply our The computation of the size of the final GSCC in the system analytic framework to these 28 systems to compute the final of correlated SF networks is different from that in a system of GSCC size, we find that there are gaps between the theoreti- uncorrelated networks. The correlation between the in- and cal computed final GSCC size and the simulation results (see out-degree of a given node of the SF network lead to a result Fig. S4a), which are caused by the correlation between the that the equation zin = zout does not hold in the system of two networks [49]. If we ignore this correlation, the size of the uncorrelated SF networks. The generating functions for com- final GSCC computed by the analytic framework agrees with puting the size of the final GSCC of interdependent correlated the simulation (see Fig. S4b), indicating that our analytic networks are framework quantifying robustness in interdependent directed  (s) networks accurately reflects the robustness in real-world sys- Φ (x, y) = Φcor(x, 1) + Φcor(1, y) − Φcor(x, y),  tems.  PM 1−λ 1−λ k  m [(k + 1) − k ]kx  Φ1(x, 1) = , PM 1−λ 1−λ . [S45] m [(k + 1) − k ]k  VIII. Randomized networks  PM 1−λ 1−λ y−1 k  m [(k + 1) − k ]k(e ) In this paper we focus on three types of randomized interde-  Φ1(1, y) =  PM 1−λ 1−λ m [(k + 1) − k ]k pendent directed networks: Substituting Eq. (S45) into Eq. (S10), we get the analytic solutions for the final GSCC size of the interdependent corre- (1) ER randomized (ER-Rand) networks: ER random- lated networks. ized networks are produced by fixing the number of nodes The generating functions for studying the final GWCC are N and links L, and turning the real network into a directed ER network [50].  PM 1−λ 1−λ x−1 k (2) Degree-preserving randomized (Degree-Rand) net- (w) m [(k + 1) − k ](xe )  Φ (x) = Φcor(x, x) = , works: Degree-preserving randomized networks are real-  (M + 1)1−λ − m1−λ . ized by fixing the elements of the in-degree and out-degree PM 1−λ 1−λ x−1 k−1 x−1  (w) m [(k + 1) − k ]k(xe ) e (x + 1) sequences, and randomly rewiring the nodes in the real net-  Φ (x) = .  1 PM 1−λ 1−λ work [51]. m [(k + 1) − k ]k [S46] (3) Dependency randomized networks: Dependency ran- In a way analogous to our procedure for computing the final domized networks are produced by fixing the connections GWCC in a system of ER networks, we determine the critical of each network, and randomly rewiring the dependency I between two real networks. coupling strengths qc1, qc2 and the percolation thresholds pc , II pc .

6 www.pnas.org — — Footline Author

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 7 — #7 i i

1. Rinaldi SM, Peerenboom JP, Kelly TK (2001) Identifying, understanding, and analyz- 27. Saumell-Mendiola A, Serrano MA,´ BoguñáM (2012) Epidemic spreading on intercon- ing critical infrastructure interdependencies. Control Systems, IEEE 21(6):11–25. nected networks. Physical Review E 86(2):026106. 2. Buldyrev SV, Parshani R, Paul G, Stanley HE, Havlin S (2010) Catastrophic cascade 28. Cellai D, López E, Zhou J, Gleeson JP, Bianconi G (2013) Percolation in multiplex of failures in interdependent networks. Nature 464(7291):1025–1028. networks with overlap. Physical Review E 88(5):052811. 3. Newman ME, Strogatz SH, Watts DJ (2001) Random graphs with arbitrary degree 29. Gomez S et al. (2013) Diffusion dynamics on multiplex networks. Physical review distributions and their applications. Physical Review E 64(2):026118. letters 110(2):028701. 4. Shao J, Buldyrev SV, Braunstein LA, Havlin S, Stanley HE (2009) Structure of shells 30. SoláL et al. (2013) Eigenvector centrality of nodes in multiplex networks. Chaos: An in complex networks. Physical Review E 80(3):036105. Interdisciplinary Journal of Nonlinear Science 23(3):033131. 5. Rosato V et al. (2008) Modelling interdependent infrastructures using interacting dy- 31. De Domenico M et al. (2013) Mathematical formulation of multilayer networks. Phys- namical models. International Journal of Critical Infrastructures 4(1-2):63–79. ical Review X 3(4):041022. 6. Parshani R, Buldyrev SV, Havlin S (2010) Interdependent networks: Reducing the 32. Candia J et al. (2008) Uncovering individual and collective human dynamics from mo- coupling strength leads to a change from a first to second order percolation transition. bile phone records. Journal of Physics A: Mathematical and Theoretical 41(22):224015. Physical review letters 105(4):048701. 33. Onnela JP et al. (2007) Structure and tie strengths in mobile communication networks. 7. Shao J, Buldyrev SV, Havlin S, Stanley HE (2011) Cascade of failures in coupled Proceedings of the National Academy of Sciences 104(18):7332–7336. network systems with multiple support-dependence relations. Physical Review E 34. Gao J, Buldyrev SV, Havlin S, Stanley HE (2011) Robustness of a network of networks. 83(3):036116. Physical Review Letters 107(19):195701. 8. Baxter G, Dorogovtsev S, Goltsev A, Mendes J (2012) Avalanche collapse of interde- 35. Aguirre J, Sevilla-Escoboza R, GutiérrezR, Papo D, BuldúJ (2014) Synchroniza- pendent networks. Physical review letters 109(24):248701. tion of interconnected networks: the role of connector nodes. Physical review letters 9. Gao J, Buldyrev SV, Stanley HE, Xu X, Havlin S (2013) Percolation of a general 112(24):248701. network of networks. Physical Review E 88(6):062816. 10. Brummitt CD, DSouza RM, Leicht E (2012) Suppressing cascades of load in interde- 36. Zhao K, Bianconi G (2013) Percolation on interacting, antagonistic networks. Journal pendent networks. Proceedings of the National Academy of Sciences 109(12):E680– of Statistical Mechanics: Theory and Experiment 2013(05):P05005. E689. 37. Aguirre J, Papo D, BuldúJM (2013) Successful strategies for competing networks. 11. De Domenico M, Solé-RibaltaA, Gómez S, Arenas A (2014) Navigability of inter- Nature Physics 9(4):230–234. connected networks under random failures. Proceedings of the National Academy of 38. Pocock MJ, Evans DM, Memmott J (2012) The robustness and restoration of a net- Sciences 111(23):8351–8356. work of ecological networks. Science 335(6071):973–977. 12. KiveläM et al. (2014) Multilayer networks. Journal of Complex Networks 2(3):203– 39. Dorogovtsev SN, Mendes JFF, Samukhin AN (2001) Giant strongly connected com- 271. ponent of directed networks. Physical Review E 64(2):025101. 13. Boccaletti S et al. (2014) The structure and dynamics of multilayer networks. Physics 40. Schwartz N, Cohen R, Ben-Avraham D, BarabásiAL, Havlin S (2002) Percolation in Reports 544(1):1–122. directed scale-free networks. Physical Review E 66(1):015104. 14. Mucha PJ, Richardson T, Macon K, Porter MA, Onnela JP (2010) Community struc- 41. Hu Y, Ksherim B, Cohen R, Havlin S (2011) Percolation in interdependent and inter- ture in time-dependent, multiscale, and multiplex networks. science 328(5980):876– connected networks: Abrupt change from second-to first-order transitions. Physical 878. Review E 84(6):066116. 15. Granell C, GómezS, Arenas A (2013) Dynamical interplay between awareness and 42. Rual JF et al. (2005) Towards a proteome-scale map of the human protein–protein epidemic spreading in multiplex networks. Physical review letters 111(12):128701. interaction network. Nature 437(7062):1173–1178. 16. Pan RK, SaramäkiJ (2011) Path lengths, correlations, and centrality in temporal 43. Barabási AL, et al. (2009) Scale-free networks: a decade and beyond. science networks. Physical Review E 84(1):016105. 325(5939):412. 17. Holme P, SaramäkiJ (2012) Temporal networks. Physics reports 519(3):97–125. 44. Borgatti SP, Mehra A, Brass DJ, Labianca G (2009) Network analysis in the social 18. Zhou D, Gao J, Stanley HE, Havlin S (2013) Percolation of partially interdependent sciences. science 323(5916):892–895. scale-free networks. Physical Review E 87(5):052812. 45. BogunáM, Pastor-Satorras R, Vespignani A (2004) Cut-offs and finite size effects in 19. Watts DJ, Strogatz SH (1998) Collective dynamics of small-worldnetworks. nature scale-free networks. The European Physical Journal B-Condensed Matter and Complex 393(6684):440–442. Systems 38(2):205–209. 20. BarabásiAL, Albert R (1999) Emergence of scaling in random networks. science 46. Gao J, Buldyrev S, Havlin S, Stanley H (2012) Robustness of a network formed by 286(5439):509–512. n interdependent networks with a one-to-one correspondence of dependent nodes. 21. Albert R, BarabásiAL (2002) Statistical mechanics of complex networks. Reviews of Physical Review E 85(6):066134. modern physics 74(1):47. 47. Zamora-López G, ZlatićV, Zhou C, StefanˇcićH,ˇ Kurths J (2008) Reciprocity of 22. Pastor-Satorras R, Vespignani A (2007) Evolution and structure of the Internet: A networks with degree correlations and arbitrary degree sequences. Phys. Rev. E statistical physics approach. (Cambridge University Press), pp. 69–208. 77(1):016106. 23. Parshani R, Buldyrev SV, Havlin S (2011) Critical effect of dependency groups on the 48. Barigozzi M, Fagiolo G, Garlaschelli D (2010) Multinetwork of international trade: A function of networks. Proceedings of the National Academy of Sciences 108(3):1007– commodity-specific analysis. Physical Review E 81(4):046104. 1010. 49. Buldyrev SV, Shere NW, Cwilich GA (2011) Interdependent networks with identical 24. Bashan A, Parshani R, Havlin S (2011) Percolation in networks composed of connec- degrees of mutually dependent nodes. Physical Review E 83(1):016112. tivity and dependency links. Physical Review E 83(5):051127. 50. Liu YY, Slotine JJ, BarabásiAL (2011) Controllability of complex networks. Nature 25. Li W, Bashan A, Buldyrev SV, Stanley HE, Havlin S (2012) Cascading failures in 473(7346):167–173. interdependent lattice networks: The critical role of the length of dependency links. Physical review letters 108(22):228702. 51. Maslov S, Sneppen K (2002) Specificity and stability in topology of protein networks. 26. Dickison M, Havlin S, Stanley HE (2012) Epidemics on interconnected networks. Phys- Science 296(5569):910–913. ical Review E 85(6):066109.

Footline Author PNAS Issue Date Volume Issue Number 7

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 8 — #8 i i

Table S1. The information of the 8 international trade networks

Network ID N L Commodity Codes Commodity Robustness 1 240 5819 3401 Soaps 0.2624 2 240 4056 3303 Perfumes and toilet waters 0.2416 3 240 3399 2402 Cigars 0.2299 4 240 4461 4802 Uncoated paper 0.2171 5 240 1785 0701 Potatoes fresh or chilled 0.1889 6 240 2222 0401 Milk and cream 0.1822 7 240 1368 0702 Tomatoes 0.1530 8 240 1368 0201 Meat and edible meat oﬀal 0.1258

8 www.pnas.org — — Footline Author

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 9 — #9 i i

A Network 1 B Network 1

Network 2 Network 2 C D Social network 1 Time t1

Social network 2 Time t2

Social network 3 Time t3

Social network 4 Time t4

E F

1 2 3 LA

1 2

3 LB 1 3 LY 1 2

Fig. S1. Schematic diagrams of networks of networks. (A) Interdependent networks: network 1 and network 2 are coupled by dependency links (green dashed lines). the nodes within networks are connected by connectivity links (black solid lines). The networks in each layer can be directed or undirected. Here we use the interdependent directed networks as used in our work. (B) Interconnected networks: the nodes within a network and the nodes of different networks are all connected by connectivity links (black solid lines). (C ) Multiplex networks: the nodes of every layer are the same, and the links of each layer represent different interacting relations. Here we take multiplex social networks for example: the nodes of each social network are the same actors, and different layers represent different social platforms, e.g. Fackbook, Twiiter, LinkedIn etc. (D) Temporal networks: different layers of networks represent a same networked system and different layers represent different times. The system dynamically evolves with time. The nodes connected by blue dashed lines represent a same entity at different times. (E) A multilayer networks M = (VM ,EM ,V,L): the union set of all the nodes V = {1, 2, 3}; two elementary sets L1 = {LA, LB} and L2 = {LX, LY }. There are four different layers: (LA, LX), (LA, LY ), (LB, LX) and (LB, LY ), so the set of nodes in their corresponding layers VM = {(1, LA, LX), (2, LA, LX), (1, LA, LY ), (2, LA, LY ), (3, LA, LY ), (1, LB, LX), (2, LB, LX), (3, LB, LX), (1, LB, LY ), (3, LB, LY )} ⊆ V × L1 × L2. Set EM ⊆ VM × VM includes all the links (black dashed lines and black solid lines) in the multilayer network M. (F) Networks of networks have different structures, e.g. a line, a tree, a circle or other structures with or without loops [9, 46]. Each layer may be different networks, for example, lattice (nodes in light green), small-world network (nodes in blue), ER network (nodes in green) and SF network (nodes in rose red). The networks are fully or partially coupled by different interaction links: dependency links (green dashed lines), connectivity links (black solid lines), antagonistic interactions (red dashed lines), the temporal relations (blue dashed lines) and other interactions or the combinations of diverse interactions (black dashed lines).

Footline Author PNAS Issue Date Volume Issue Number 9

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 10 — #10 i i

A Step 0

Initial failure 1 Network A Initial failure 2

Network B Initial failure 3

B Step 1 C Step 2 Network A Network A

Network B Network B

Step 4 D Step 3 E Network A Network A

Network B Network B

Fig. S2. Schematic demonstration of the cascading process. (A) Network A and network B both contain 10 nodes, where qA = 5/10 fraction of A-nodes depend on B-nodes and qB = 4/10 fraction of B-nodes depend on A-nodes (dashed lines). At the initial attacking process p1 = 1/10 A-nodes (red) and p2 = 2/10 B-nodes (red) are removed. (B) The red nodes depend on the removed nodes, so they fail at the ﬁrst step of the cascading failure. (C ) At the second step, the A-nodes and B-nodes (red) which do not belong to the giant strongly connected component (GSCC) stop functioning. (D) The red A-nodes fails for its dependency on a failed B-nodes. (E) No more failures happen. The remaining nodes form the ﬁnal GSCC of the interdependent networks.

10 www.pnas.org — — Footline Author

i i

i i i i “”SI Appendix”” — 2015/12/23 — 9:13 — page 11 — #11 i i

0.5 Network A: =7, p =0.54, q =0.4 A 1 A Network B: =10, p =0.65, q =0.5 0.4 B 2 B φ(s) 0.3

0.2

0.1 ψ(s)

0 0 2 4 6 8 10 12 Step Fig. S3. The dynamic of cascading failures of two interdependent directed ER network. The red solid line represents the analytic result of the GSCC size of network A at each step of the cascading failure, and the blue line is the analytic result of the GSCC size of network B. They agree well with the simulation results (N=10000000) denoted by symbols. In the simulations, we set that hkAi = 7, hkB i = 10, qA = 0.4, qB = 0.5, p1 = 0.54, and p2 = 0.65.

Fig. S4. Affect of degree-degree correlations between networks (A) The size of the final GSCC during the whole attacking process for two interdependent networks with the commodities being cigars and soaps respectively. The solid lines are the analytic results and symbols are the simulation results, which do not agree very well with each other. (B) To eliminate the degree-degree correlation between networks, we randomly connect the nodes between two networks with dependency links. And we perform the cascading failures on the dependency randomized pair of networks, and find that the analytic results and simulation results agree well with each other. Each result is averaged over 600 realizations.

Footline Author PNAS Issue Date Volume Issue Number 11

i i