Approximate fast graph Fourier transforms via multi-layer sparse approximations Luc Le Magoarou, Rémi Gribonval, Nicolas Tremblay To cite this version: Luc Le Magoarou, Rémi Gribonval, Nicolas Tremblay. Approximate fast graph Fourier transforms via multi-layer sparse approximations. IEEE transactions on Signal and Information Processing over Networks, IEEE, 2018, 4 (2), pp.407–420. 10.1109/TSIPN.2017.2710619. hal-01416110v3 HAL Id: hal-01416110 https://hal.inria.fr/hal-01416110v3 Submitted on 16 Jun 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Copyright 1 Approximate fast graph Fourier transforms via multi-layer sparse approximations Luc Le Magoarou, Remi´ Gribonval, Fellow, IEEE, Nicolas Tremblay Abstract—The Fast Fourier Transform (FFT) is an algorithm (FFT) [4] allows to apply the Fourier transform in only of paramount importance in signal processing as it allows to apply O(n log n) arithmetic operations. The FFT is a fast linear 2 the Fourier transform in O(n log n) instead of O(n ) arithmetic algorithm [5], which implies that the classical Fourier matrix operations. Graph Signal Processing (GSP) is a recent research domain that generalizes classical signal processing tools, such as can be factorized into sparse factors, as discussed in [6]. Given the Fourier transform, to situations where the signal domain is that classical signal processing is equivalent to graph signal given by any arbitrary graph instead of a regular grid. Today, processing on the ring graph, it is natural to wonder if this there is no method to rapidly apply graph Fourier transforms. kind of factorization can be generalized to other graphs. We propose in this paper a method to obtain approximate We proposed in a previous work a method to obtain com- graph Fourier transforms that can be applied rapidly and stored 1 efficiently. It is based on a greedy approximate diagonalization putationally efficient approximations of matrices, based on of the graph Laplacian matrix, carried out using a modified multi-layer sparse factorizations [6]. The method amounts to version of the famous Jacobi eigenvalues algorithm. The method approximate a matrix of interest A 2 Rp×q with a product of is described and analyzed in detail, and then applied to both sparse matrices, as synthetic and real graphs, showing its potential. A ≈ S ::: S ; (2) Index Terms—Graph signal processing, Fast Fourier Trans- J 1 form, Greedy algorithms, Jacobi eigenvalues algorithm, Sensor where the matrices S ;:::; S are sparse, allowing for cheap networks. 1 J storage and multiplication. We applied this method to the graph Fourier matrix U of various graphs in [7], in order I. INTRODUCTION to get approximate Fast Graph Fourier Transforms (FGFT) RAPHS are powerful mathematical objects used to The method showed good potential. However in the context G model pairwise relationships between elements of a of graph signal processing, this approach suffers from at least set. Graph theory has been extensively developed since the two limitations: eighteenth century, and has found a variety of applications, (L1) It requires a full diagonalization of the graph Laplacian ranging from biology to computer science or linguistics [1]. L before it can be applied. Indeed, the method takes the Recently, methods have been developed to analyze and graph Fourier matrix U as input. Performing this diag- process signals defined over the vertices of a graph [2], [3], onalization costs O(n3) arithmetic operations, which is instead of over a regular grid, as is classically assumed in prohibitive if n is large. discrete signal processing. The starting point of graph signal (L2) It provides non-orthogonal approximate FGFTs. Indeed, processing is to define a Fourier transform, via an analogy the details of the method make it difficult to get sparse with classical signal processing. Depending on the preferred and orthogonal factors S1;:::; SJ . This leads to ap- analogy, there exists several definitions of graph Fourier trans- proximate graph Fourier transforms that are not easily forms [2], [3]. Following [2], we choose in this paper to define invertible, which can be a problem for applications the graph Fourier basis as the eigenvector matrix of the graph where signal reconstruction is needed. Laplacian operator L (whose precise definition is given in We propose in this paper another method, that does not n×n Section II-A). In a graph with n vertices, L 2 R and suffer from these limitations, to obtain approximate FGFTs. L = UΛUT ; (1) In order to overcome (L1), we consider directly the Laplacian matrix L as input. To overcome (L2), we look for factors n×n where U 2 R is an orthogonal matrix whose columns are S1;:::; SJ constrained to be in a set of sparse and orthogonal the graph Fourier modes and Λ 2 Rn×n is a non-negative matrices built with Givens rotations [8] (as explained in diagonal matrix whose diagonal entries correspond to the detail in section III). The proposed method amounts to an squared graph frequencies. approximate diagonalization of the Laplacian matrix L, as The graph Fourier matrix U being non-sparse in general, ^ T T applying it costs O(n2) arithmetic operations. In the classical L ≈ S1 ::: SJ ΛSJ ::: S1 ; (3) signal processing case, the well-known Fast Fourier Transform where the matrices S1;:::; SJ are both sparse and orthogonal. The product U^ = S1 ::: SJ constitutes an efficient Luc Le Magoarou ([email protected]) and Remi´ Gribon- ^ val ([email protected]) are both with Inria, Rennes, France, approximate graph Fourier matrix and Λ is a diagonal PANAMA team. Nicolas Tremblay is with CNRS, GIPSA-lab, Grenoble, France. This work was supported in part by the European Research Council, 1An m × n matrix is “efficient” if it is associated with a linear algorithm PLEASE project (ERC-StG- 2011-277906). involving strictly less than mn scalar multiplications. 2 Exact spectrum of the signal 4 proposed in Section V. Finally, an application to graph signal 3 filtering is presented in Section VI. 2 II. PROBLEM FORMULATION 1 In this section, we give a concrete formulation of the main 0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 problem addressed in this paper. We first set up the notations and conventions used, before presenting the objective in detail. Approximate spectrum of the signal, computed 23 times faster 4 We then discuss the advantages in terms of computational 3 complexity expected from the method and end with a pre- 2 sentation of the related work. 1 0 A. Notations and conventions 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 General notations. Matrices are denoted by bold upper-case Fig. 1. Example of an approximate spectrum computed with our method on letters: A; vectors by bold lower-case letters: a; the ith a real-world sensor graph. Details in Section V. column of a matrix A by: ai; its entry located at the ith row and jth column by: aij. Sets are denoted by calligraphic symbols: A, and we denote by δ the characteristic function matrix whose diagonal entries are approximations of the A of the set A in the optimization sense (δ (x) = 0 if x 2 A, squared graph frequencies. A δA(x) = +1 otherwise). The standard vectorization operator is denoted vec(·). The ` -pseudonorm is denoted k·k (it Contributions. Given a graph Laplacian matrix, the main 0 0 counts the number of non-zero entries), k·k denotes the objective of this paper is to find an approximate graph F Frobenius norm, and k·k the spectral norm. By abuse of Fourier matrix U^ that both i) approximately diagonalizes the 2 notations, kAk = kvec(A)k . Id denotes the identity matrix. Laplacian and ii) is computationally efficient. The proposed 0 0 method is a greedy strategy that amounts to truncating and Graph Laplacian. We consider in this paper undirected slightly modifying the famous Jacobi eigenvalue algorithm weighted graphs, denoted G fV; E; Wg, where V represents [9]. Note that it could in principle be applied to any symmetric , the set of vertices (otherwise called nodes), E ⊂ V × V is matrix (covariance matrix, normalized Laplacian, etc.), but the set of edges, and W is the weighted adjacency matrix of the focus of the present paper is on the graph combinatorial the graph. We denote n jVj the total number of vertices Laplacian. Indeed, while a general symmetric matrix has , and the adjacency matrix W 2 n×n is symmetric and such no reason to be well approximated with a limited number R that w = w is non-zero only if (i; j) 2 E and represents of Givens factors, graph Laplacians are shown empirically ij ji the strength of the connection between nodes i and j. We to have this interesting property for several popular graph define the degree matrix D 2 n×n as a diagonal matrix with families. The proposed method is compared experimentally R 8i, d Pn w , and the combinatorial Laplacian matrix with the direct approximation method of [7] on various graphs, ii , j=1 ij showing that it is much faster while obtaining good results. In L , D − W (we only consider this type of Laplacian matrix fact, we obtain approximate FGFTs of complexity O(n log n) in this paper, and hereafter simply call it the Laplacian).