Arxiv:1609.01214V1 [Physics.Soc-Ph] 5 Sep 2016

Spectral entropies as information-theoretic tools for complex network comparison Manlio De Domenico1 and Jacob Biamonte2, 3 1Departament d'Enginyeria Informàtica i Matemàtiques, Universitat Rovira i Virgili, 43007 Tarragona, Spain 2Quantum Complexity Science Initiative Department of Physics, University of Malta, MSD 2080 Malta 3Institute for Quantum Computing University of Waterloo, Waterloo, N2L 3G1 Ontario, Canada Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we use techniques inspired by quantum statistical mechanics to define an entropy measure for complex networks and to develop a set of information-theoretic tools, based on network spectral properties, such as Rényi q−entropy, generalized Kullback-Leibler and Jensen-Shannon divergences, the latter allowing us to define a natural distance measure between complex networks. First we show that by minimizing the Kullback-Leibler divergence between an observed network and a parametric network model, inference of model parameter(s) by means of maximum-likelihood estimation can be achieved and model selection can be performed appropriate information criteria. Second, we show that the information-theoretic metric quantifies the distance between pairs of networks and we can use it, for instance, to cluster the layers of a multilayer system. By applying this framework to networks corresponding to sites of the human microbiome, we perform hierarchical cluster analysis and recover with high accuracy existing community-based associations. Our results imply that spectral based statistical inference in complex networks results in demonstrably superior performance as well as a conceptual backbone, filling a gap towards a network information theory. I. INTRODUCTION an output state of a system. This then necessitates a method to quantify this information, its time-dependence in terms of storage and transfer of information, ideally Shannon's entropy [1] and information-theoretic de- between levels of a multilayer system, which can fur- rived measures have been successfully applied in a range ther be imperfect due to randomness or noise. A sim- of disciplines, from revealing time scale dependence in ilar challenge was addressed by quantum theorists be- neural coding [2,3] to quantifying quantum informa- ginning decades ago when faced with quantifying infor- tion [4{6] and the complexity of genetic sequences [7,8] or matic properties of quantum states [4{6, 34{37]. In- to unravel the mesoscale organization of interconnected deed, the modern theory of quantum information, as the systems [9{12], to cite just a few emblematic achieve- name suggests, is fundamentally built upon the quan- ments. However, when it comes to complex networks, an tification of physical information, entropic based quan- appropriate definition of entropy has remained elusive| tifications of non-locality, entanglement [34] and the in- applicability being often limited to the probability distri- herent complexity of the quantum model [35]. These bution of some network descriptor (such as the normal- decades of research have placed entropic measures cen- ized distribution of node degrees). tral to the modern theory, leading to quantum generaliza- Complex network theory dates to the discovery of sev- tions of Shannon's classical information theory [4{6, 34{ eral fundamental features [13, 14] and complex networks 37]. However, for a variety of reasons, these results can't have been widely used to model and better understand be applied directly to complex networks, although clas- the organization of complex systems and their dynam- sical information-theoretic tools have been successfully ics [15{20], the controllability of their constituents [21] adopted to solve specific problems [9, 38{46]. arXiv:1609.01214v1 [physics.soc-ph] 5 Sep 2016 and their resilience to structural and dynamical pertur- Here we address a similarly motivated challenge: in- bations [22{27]. A recent and ongoing drive in complex spired by how entropy is calculated in quantum systems, networks is to merge ideas with quantum information we define an interconnectivity-based density matrix to science, including entangled networks for communica- calculate the von Neumann entropy of a network. We tion [28{30], developing a theory of node centrality in analytically prove that our definition satisfies the desired quantum walks on graphs [31] and in the detection of additivity properties similar to quantum thermodynamic community structures formed in quantum systems [32]. entropy|though the proof of this differs tremendously In the same spirit as the Church{Turing{Deutsch prin- from the quantum case|which opens the door to avoid ciple [33], like all physical systems it is possible to view the shortcomings, recently pointed out in [47], imposed complex networks in terms of information processing, by sub-additivity failing as found in past approaches. in which information changes in time from an input to We exploit this entropy to develop a set of information- 2 theoretic tools which apply to complex networks, such i.e. it is equal to the Shannon entropy of the eigenvalues as Rényi q-entropy, generalized Kullback-Leibler and of the density matrix, where by convention 0 log2 0 := 0. Jensen-Shannon divergences, thus providing a backbone In this work we are not limited in having a quantum to an information-theoretic approach to network science. setup where the matrix ρ can be called a \density ma- Our framework, based on spectral properties of networks, trix" in the physical sense, but instead build on this idea allows one to probe contemporary problems faced in to define a matrix from a network that satisfies the same complex network science. For instance, we show that mathematical properties of a density matrix. Kullback-Leibler minimization can be used to infer the First, such a density matrix should be positive defi- parameter(s) of a model aimed to fit an observed net- nite and symmetric, therefore ρ = Z−1e−βH { with H a work. By exploiting results of classical information the- symmetric matrix with non-negative eigenvalues, Z and ory, we are able to introduce the spectral counterpart of β real numbers { is a suitable candidate. Second, the likelihood maximization and use it in practical applica- eigenvalues of ρ must sum to unity, thus imposing the tions to fit network models and perform model selection constraint Z = Tr e−βH. Here, we use the following den- based on appropriate Akaike and Bayesian information sity matrix, defined by criteria, as well as minimum description length. The e−βL strength of our approach relies on the fact that it uses ρ = ;Z = Tr e−βL; (3) the network as a whole, instead of a subset of network's Z descriptors, to attack parameter inference and model se- where L = D−A denotes the combinatorial graph Lapla- lection problems. cian, being A the adjacency matrix of the network and Another important byproduct of the proposed infor- D the diagonal matrix of node degrees. Eq. (3) resembles mation theoretic framework is the possibility to quan- the Gibbs state of a quantum system where the Hamil- tify the distance between complex networks. This prob- tonian equals the Laplacian matrix, i.e. H = L, where β lem is of high interest in recent applications concerning is a scale parameter. multilayer systems, such as time-varying and multiplex It is worth remarking that this functional form appears networks [48{50]. This type of systems consist of nodes in many problems of physical interest. For instance, if β replicated across several networks that exhibit different parameterizes time, setting Z = 1 and H = L, i.e. the types of relationships or connectivity [51]. The possibil- Laplacian matrix, then e−Lt is the evolution matrix gov- ity to compare layers from an information-theoretic point N of view has been recently explored to aggregate them in erning purely diffusive dynamics and its trace P e−λit order to reduce the structure and the complexity of bio- i=1 is N times the average return probability, i.e. the prob- logical, transportation and social multiplex networks [47]. ability that a walker starting at any node i will return To numerically probe our method, we cluster the layer of to its origin at time t. Another case of interest is when an empirical system and compare it against the existing β parameterizes temperature of a quantum system with classification. Hamiltonian H and Z = Tr e−βH, so that ρ would pro- vide the Gibbs state of the system in equilibrium at finite temperature. Although we are interested mainly in non- II. VON NEUMANN ENTROPY OF A negative values of β, it is interesting to show that when COMPLEX NETWORK β = −1, Z = 1 and H = A we recover the well-known communicability matrix introduced by Estrada [52{57] In quantum mechanics, probability distributions are (see Ref. [58] for a thorough review) or a normalized ver- encoded by density matrices. A density matrix ρ is a sion of it, if Z = Tr eA. Hermitian and positive semi-definite matrix, with trace In Ref. [59], a matrix is built in such a way that its equal to unity which is used to represent both mixed and mathematical properties satisfy the requirements of a pure quantum states. A system is in a pure state j i, if density matrix. For an undirected complex network, L and only if the bound Tr ρ2 ≤ 1 is saturated. The density is symmetric and positive semi-definite but its eigenval- matrix admits a spectral decomposition as ues do not sum to unity. However, the eigenvalues of the proposed matrix, defined by ρbgs = L= Tr L, evidently N X do, therefore ρbgs is a mathematically suitable density ρ = λijφiihφij (1) matrix [45, 59] and hence, Eq.

Load more