A Submodular Approach
Total Page:16
File Type:pdf, Size:1020Kb
SPARSEST NETWORK SUPPORT ESTIMATION: A SUBMODULAR APPROACH Mario Coutino, Sundeep Prabhakar Chepuri, Geert Leus Delft University of Technology ABSTRACT recovery of the adjacency and normalized Laplacian matrix, in the In this work, we address the problem of identifying the underlying general case, due to the convex formulation of the recovery problem, network structure of data. Different from other approaches, which solutions that only retrieve the support approximately are obtained, are mainly based on convex relaxations of an integer problem, here i.e., thresholding operations are required. we take a distinct route relying on algebraic properties of a matrix Taking this issue into account, and following arguments closely representation of the network. By describing what we call possible related to the ones used for network topology estimation through ambiguities on the network topology, we proceed to employ sub- spectral templates [10], the main aim of this work is twofold: (i) to modular analysis techniques for retrieving the network support, i.e., provide a proper characterization of a particular set of matrices that network edges. To achieve this we only make use of the network are of high importance within graph signal processing: fixed-support modes derived from the data. Numerical examples showcase the ef- matrices with the same eigenbasis, and (ii) to provide a pure com- fectiveness of the proposed algorithm in recovering the support of binatorial algorithm, based on the submodular machinery that has sparse networks. been proven useful in subset selection problems within signal pro- cessing [18–22], as an alternative to traditional convex relaxations. Index Terms— Graph signal processing, graph learning, sparse graphs, network deconvolution, network topology inference. 1. INTRODUCTION 2. PRELIMINARIES In recent years, large efforts have been made to understand how tra- Consider a network being represented by an undirected graph G = ditional tools from signal processing can be adapted for cases where fV; Eg, which consists of a finite set of nodes V with jVj = N and a the acquired data is not defined over a regular domain but over a net- set of edges (connections) E with jEj = M. If there is a connection work [1]. This surge of interest is due to the fact that such network- in the network between nodes i and j, then (i; j) 2 E. A signal supported signals are able to model complex transport networks [2], or function f : V! defined on the nodes of the network can the activity of the brain [3], epidemic dynamics and gene coexpres- R be collected in a length-N vector x, where the nth element of x sion [4], to name a few. As a result, the field of graph signal process- represents the function value at the nth node in V. Since x resides ing (GSP) has emerged [5, 6]. on the graph (network), we refer to the function x as a graph signal. Assuming the topology of the network which supports a sig- N×N nal directly influences the behaviour of it, GSP develops algorithms Let us introduce a network topology matrix S 2 R , where which exploit the network structure to perform classical signal pro- the (i; j)th entry of S denoted by si;j can only be nonzero if (i; j) 2 cessing tasks such as estimation, detection and filtering [7–9]. As a E. This matrix acts as a possible representation of the network con- result, appropriate knowledge of the relations (edges) among the el- nectivity. Within the field of GSP, this matrix is usually referred to ements (nodes) of the network (graph) is required for any algorithm as graph shift operator [6]. that requires connectivity information. Therefore, in this paper, we For undirected graphs, S is symmetric, and thus it admits the focus on the problem of estimating the underlying network structure following eigenvalue decomposition in which the data is embedded. S = UΛU T (1) Several works have been devoted to find the underlying network where the eigenvectors U and the eigenvalues Λ of S provide a (graph) structure within the data [10–17]. Some of those works fit notion of frequency in the graph setting [5,6]. Specifically, U forms the observed data to particular structural models [12, 13], or find an orthonormal Fourier-like basis for graph signals with the graph a network which renders the observed signal smooth in its topol- frequencies denoted by fλ gN . ogy [17]. In addition, similar to the approach presented here, there n n=1 are works [10] that use the network modes to retrieve a sparse net- For a general network matrix, S, that does not admit an or- work support by means of a convex problem. However, despite the thonormal eigenvalue decomposition, we might consider a decom- fact that those works successfully address the problem of network position through its singular values. Although, the SVD decompo- estimation, each of them making different structural assumptions, sition does not carry a graph frequency interpretation as in the case they do not completely address the characterization of general net- of the eigenvalue decomposition, the theory of network support esti- work matrices, i.e., networks with disconnected components, net- mation presented in this work holds for both decompositions. In this work, we aim to reconstruct the support of the network S works with self loops, etc. Moreover, even though that for sim- Q ple graphs (networks), [10] provides theoretical guarantees for the by means of a set of output data, i.e., fyigi=1, under the assumption that the matrix representation of the network, S, and the covariance This work is part of the ASPIRE project (project 14926 within the STW T matrix of the data, Ry , Efyy g, share the same eigenspace [23, OTP program), which is financed by the Netherlands Organization for Scien- 24]. Therefore, the only side information used for this purpose are tific Research (NWO). Mario Coutino is partially supported by CONACYT. the so called modes of the network. 978-1-5386-4410-2/18/$31.00 ©2018 IEEE 200 DSW 2018 A |A|×N 3. THE AMBIGUOUS CLASS OF NETWORK MATRICES where we have defined T U = ΦA(U ∗ U) 2 R . Alternatively, we can consider another formulation for the con- Following the assumption that Ry and S share the same eigenbasis, dition expressed in (8) in terms of the covariance matrix Ry and its we have the following property: inverse. That is, considering [cf. (2)] −1 RyS = SRy: (2) S = R SR (11) Q Now, let us assume that from the set of available data, fyigi=1, and applying the vectorization operation to (11), we obtain R −1 T both the covariance matrix, y, and its eigenvalue decomposi- [I − (R ⊗ R )]Φ c Φ c vec(S) = 0: T y y A A (12) tion, UΩU , are obtained. Similar to the spectral templates c approach [10], we know that S = UΛU T , for some diagonal where A is the set that indexes the nonzero entries of vec(S). Note N×N that while condition (10) characterizes the matrices in terms of miss- matrix Λ 2 R . Hence, the support identification problem is reduced to finding a matrix Λ which meets the desired properties for ing links and leverages the spectral decomposition of S, the condi- a particular choice of network matrix, e.g., integer-valued matrix, tion (12) characterizes more general kind of matrices, i.e., not eigen- nonnegativity, zero diagonal, etc. That is, decomposable matrices, in terms of connections between nodes. T Using the previous results [cf. (6)-(12)], we can summarize the minimize kSk0 subject to S = UΛU ; (3) Λ2D main result of this section in the following theorem: where D is the set of permissible matrices, i.e., constraint set. Theorem 1. (Nullspace Property) Differently from [10], here we are interested in a generative so- Given an orthonormal basis U, and a sparsity pattern defined by a lution based on algebraic properties of fixed-support sparse matri- A set A, the matrices within the set JU are the matrices of the form ces. This approach is motivated from the fact that for a particular S = UΛU T whose eigenvalues are given by problem instance (nature of the data) there is no clear distinction A between network matrices with the same support sharing the same Λ = diagfBU αg: (13) eigenbasis. That is, given A N×d A Here, the matrix BU 2 R is a basis for the nullspace of T U , T T Si = UΛiU and Sj = UΛj U 6= Si; (4) i.e., A A spanfBU g = nullfT U g; where Λi and Λj are diagonal matrices, and Si and Sj have the d same support, which network matrix should be employed? and α 2 R is the expansion coefficient vector. To tackle this problem, first we have to understand the class of matrices that falls within this ambiguous family, i.e., the family of Proof. The proof directly follows from the expressions in (6)-(10). jointly diagonalizable matrices sharing the same sparsity support. This family of matrices is formally defined as the set A T The result of the theorem provides a certificate of uniqueness J = fS : S = UΛU and [vec(S)] = 0; 8 i 2 Ag; (5) U i of the network matrix in the following sense: the network matrix is A U A where and are the index set defining the zero entries of the considered unique, for a fixed sparsity pattern A, if the set JU has matrix and the eigenbasis, respectively. In the following section, elements given by we characterize this set of matrices to leverage their properties for S = αS0; (14) support identification.