DOCSLIB.ORG

Counterfactual Explanations for Graph Neural Networks

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Counterfactual Explanations for Graph Neural Networks CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networks Ana Lucic Maartje ter Hoeve Gabriele Tolomei University of Amsterdam University of Amsterdam Sapienza University of Rome Amsterdam, Netherlands Amsterdam, Netherlands Rome, Italy [email protected] [email protected] [email protected] Maarten de Rijke Fabrizio Silvestri University of Amsterdam Sapienza University of Rome Amsterdam, Netherlands Rome, Italy [email protected] [email protected] ABSTRACT that result in an alternative output response (i.e., prediction). If Given the increasing promise of Graph Neural Networks (GNNs) in the modifications recommended are also clearly actionable, this is real-world applications, several methods have been developed for referred to as achieving recourse [12, 28]. explaining their predictions. So far, these methods have primarily To motivate our problem, we consider an ML application for focused on generating subgraphs that are especially relevant for computational biology. Drug discovery is a task that involves gen- a particular prediction. However, such methods do not provide erating new molecules that can be used for medicinal purposes a clear opportunity for recourse: given a prediction, we want to [26, 33]. Given a candidate molecule, a GNN can predict if this understand how the prediction can be changed in order to achieve molecule has a certain property that would make it effective in a more desirable outcome. In this work, we propose a method for treating a particular disease [9, 19, 32]. If the GNN predicts it does generating counterfactual (CF) explanations for GNNs: the minimal not have this desirable property, CF explanations can help identify perturbation to the input (graph) data such that the prediction the minimal change one should make to this molecule, such that it changes. Using only edge deletions, we find that our method can has this desirable property. This could help us not only generate a generate CF explanations for the majority of instances across three new molecule with the desired property, but also understand what widely used datasets for GNN explanations, while removing less structures contribute to a molecule having this property. than 3 edges on average, with at least 94% accuracy. This indicates Although GNNs have shown state-of-the-art results on tasks that our method primarily removes edges that are crucial for the involving graph data [3, 38], existing methods for explaining the original predictions, resulting in minimal CF explanations. predictions of GNNs have primarily focused on generating sub- graphs that are relevant for a particular prediction [1, 4, 14, 16, 20, ACM Reference Format: 22, 30, 34, 36, 37]. However, none of these methods are able to iden- Ana Lucic, Maartje ter Hoeve, Gabriele Tolomei, Maarten de Rijke, and Fab- tify the minimal subgraph automatically since they all require the rizio Silvestri. 2021. CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networks. In DLG-KDD’21: Deep Learning on Graphs, August user to specify in advance the size of the subgraph, (. We show that 14–18, 2021, Online. ACM, New York, NY, USA, 6 pages. https://doi.org/ even if we adapt existing methods to the CF explanation problem, 10.1145/1122445.1122456 and try varying values for (, such methods are not able to produce valid, accurate explanations, and are therefore not well-suited to 1 INTRODUCTION solve the CF explanation problem. To address this gap, we propose CF-GNNExplainer, which generates CF explanations for GNNs. Advances in machine learning (ML) have led to breakthroughs in Similar to other CF methods proposed in the literature [12, 29], several areas of science and engineering, ranging from computer CF-GNNExplainer works by perturbing data at the instance-level. vision, to natural language processing, to conversational assistants. In this work, the instances are nodes in the graph since we focus on Parallel to the increased performance of ML systems, there is an node classification. In particular, our method iteratively removes increasing call for the “understandability” of ML models [7]. Un- edges from the original adjacency matrix based on matrix sparsi- derstanding why an ML model returns a certain output in response fication techniques, keeping track of the perturbations that lead to a given input is important for a variety of reasons such as model to a change in prediction, and returning the perturbation with debugging, aiding decison-making, or fulfilling legal requirements the smallest change w.r.t. the number of edges. We evaluate CF- [5]. Having certified methods for interpreting ML predictions will GNNExplainer on three public datasets for GNN explanations and help enable their use across a variety of applications [18]. measure its effectiveness using four metrics: fidelity, explanation Explainable AI (XAI) refers to the set of techniques “focused on size, sparsity, and accuracy. CF-GNNExplainer is able to generate exposing complex AI models to humans in a systematic and inter- CF examples with at least 94% accuracy, while removing fewer than pretable manner”[21]. A large body of work on XAI has emerged three edges on average. In this work, we (i) formalize the problem in recent years [2, 8]. Counterfactual (CF) explanations are used to of generating CF explanations for GNNs, and (ii) propose a novel explain predictions of individual instances in the form: “If X had method for generating CF explanations for GNNs. been different, Y would not have occurred” [25]. CF explanations are based on CF examples: modified versions of the input sample DLG-KDD’21, August 14–18, 2021, Online Ana Lucic, Maartje ter Hoeve, Gabriele Tolomei, Maarten de Rijke, and Fabrizio Silvestri 2 PROBLEM FORMULATION we want to find % that minimally perturbs 퐴E, and use it to compute ¯ Let 5 (퐴, -;, ) be any GNN, where 퐴 is the adjacency matrix, - is 퐴E = % ⊙퐴E. If an element %8,9 = 0, this results in the deletion of the the feature matrix, and , is the learned weight matrix of 5 (i.e., edge between node 8 and node 9. When % is a matrix of ones, this 퐴 and - are the inputs of 5 , and 5 is parameterized by , ). We indicates that all edges in 퐴E are used in the forward pass. Similar define the subgraph neighbourhood of a node E as a tuple of the to Srinivas et al. [24], we first generate an intermediate, real-valued ˆ nodes and edges relevant for the computation of 5 (E) (i.e., those matrix % with entries in »0, 1¼, apply a sigmoid transformation, in the ℓ-hop neighbourhood of 5 ): GE = (퐴E,-E), where 퐴E is the then threshold the entries to arrive at a binary %: entries above 0.5 subgraph adjacency matrix and -E is the node feature matrix for become 1, while those below 0.5 become 0. In the case of undirected nodes that are at most ℓ hops away from E. We define a node E as a graphs (i.e., those with symmetric adjacency matrices), instead of ˆ tuple of the form E = (퐴E,G), where G is the feature vector for E. generating % directly, we first generate a perturbation vector which In general, a CF example G¯ for an instance G according to a we then use to populate %ˆ in a symmetric manner. trained classifier 5 is found by perturbing the features of G such that 5 (G) =6 5 (G¯) [31]. An optimal CF example G¯∗ is one that minimizes 3.2 Counterfactual Generating Model the distance between the original instance and the CF example, We want our perturbation matrix % to only act on 퐴E, not 퐴˜E, in according to some distance function 3. The resulting optimal CF order to preserve self-loops in the message passing of 5 (i.e., we ∗ ∗ explanation is ∆G = G¯ − G [15]. always want a node representation update to include the node’s For graph data, it may not be enough to simply perturb node representation from the previous layer). Therefore, we first rewrite features, especially since they are not always available. This is why Equation 2 for our illustrative one-layer case to isolate 퐴E: we are interested in generating CF examples by perturbing the h −1/2 −1/2 i graph structure instead. In other words, we want to change the 5 (퐴E,-E;, ) = softmax (퐷E + 퐼) (퐴E + 퐼)(퐷E + 퐼) -E, (3) relationships between instances, rather than change the instances To generate CFs, we propose a new function 6, which is based on 5 , themselves. Therefore, a CF example for graph data has the form but it is parameterized by % instead of by , . We update the degree E¯ = (퐴¯ ,G), where G is the feature vector and 퐴¯ is a perturbed E E matrix 퐷E based on % ⊙ 퐴E, add the identity matrix to account for version of 퐴 . 퐴¯ is obtained by removing some edges from 퐴 , E E E self-loops (as in 퐷˜E in Equation 2), and call this 퐷¯E: such that 5 (E) =6 5 (E¯). h −1/2 −1/2 i Following Wachter et al. [31] and Lucic et al. [15], we generate (퐴E,-E,, ; %) = softmax 퐷¯E (% ⊙ 퐴E + 퐼)퐷¯E -E, . (4) CF examples by minimizing a loss function of the form: In other words, 5 learns the weight matrix while holding the data L = L?A43 (E, E¯ j 5 ,6) + VL38BC (E, E¯), (1) constant, while 6 is optimized to find a perturbation matrix that where E is the original node, 5 is the original model, 6 is the CF is then used to generate new data points (i.e., CF examples) while holding the weight matrix (i.e., model) constant. Another distinc- model that generates E¯, and L?A43 is a prediction loss that encour- ages 5 (E) =6 5 (E¯).
Load more

Recommended publications
  • Immanants of Totally Positive Matrices Are Nonnegative
    IMMANANTS OF TOTALLY POSITIVE MATRICES ARE NONNEGATIVE JOHN R. STEMBRIDGE Introduction Let Mn{k) denote the algebra of n x n matrices over some field k of characteristic zero. For each A>valued function / on the symmetric group Sn, we may define a corresponding matrix function on Mn(k) in which (fljji—• > X\w )&i (i)'"a ( )' (U weSn If/ is an irreducible character of Sn, these functions are known as immanants; if/ is an irreducible character of some subgroup G of Sn (extended trivially to all of Sn by defining /(vv) = 0 for w$G), these are known as generalized matrix functions. Note that the determinant and permanent are obtained by choosing / to be the sign character and trivial character of Sn, respectively. We should point out that it is more traditional to use /(vv) in (1) where we have used /(W1). This change can be undone by transposing the matrix. If/ happens to be a character, then /(w-1) = x(w), so the generalized matrix function we have indexed by / is the complex conjugate of the traditional one. Since the characters of Sn are real (and integral), it follows that there is no difference between our indexing of immanants and the traditional one. It is convenient to associate with each AeMn(k) the following element of the group algebra kSn: [A]:= £ flliW(1)...flBiW(B)-w~\ In terms of this notation, the matrix function defined by (1) can be described more simply as A\—+x\Ai\, provided that we extend / linearly to kSn. Note that if we had used w in place of vv"1 here and in (1), then the restriction of [•] to the group of permutation matrices would have been an anti-homomorphism, rather than a homomorphism.
    [Show full text]
  • Some Classes of Hadamard Matrices with Constant Diagonal
    BULL. AUSTRAL. MATH. SOC. O5BO5. 05B20 VOL. 7 (1972), 233-249. • Some classes of Hadamard matrices with constant diagonal Jennifer Wallis and Albert Leon Whiteman The concepts of circulant and backcircul-ant matrices are generalized to obtain incidence matrices of subsets of finite additive abelian groups. These results are then used to show the existence of skew-Hadamard matrices of order 8(1*f+l) when / is odd and 8/ + 1 is a prime power. This shows the existence of skew-Hadamard matrices of orders 296, 592, 118U, l6kO, 2280, 2368 which were previously unknown. A construction is given for regular symmetric Hadamard matrices with constant diagonal of order M2m+l) when a symmetric conference matrix of order km + 2 exists and there are Szekeres difference sets, X and J , of size m satisfying x € X =» -x £ X , y £ Y ~ -y d X . Suppose V is a finite abelian group with v elements, written in additive notation. A difference set D with parameters (v, k, X) is a subset of V with k elements and such that in the totality of all the possible differences of elements from D each non-zero element of V occurs X times. If V is the set of integers modulo V then D is called a cyclic difference set: these are extensively discussed in Baumert [/]. A circulant matrix B = [b. •) of order v satisfies b. = b . (j'-i+l reduced modulo v ), while B is back-circulant if its elements Received 3 May 1972. The authors wish to thank Dr VI.D. Wallis for helpful discussions and for pointing out the regularity in Theorem 16.
    [Show full text]
  • An Infinite Dimensional Birkhoff's Theorem and LOCC-Convertibility
    An infinite dimensional Birkhoff's Theorem and LOCC-convertibility Daiki Asakura Graduate School of Information Systems, The University of Electro-Communications, Tokyo, Japan August 29, 2016 Preliminary and Notation[1/13] Birkhoff's Theorem (: matrix analysis(math)) & in infinite dimensinaol Hilbert space LOCC-convertibility (: quantum information ) Notation H, K : separable Hilbert spaces. (Unless specified otherwise dim = 1) j i; jϕi 2 H ⊗ K : unit vectors. P1 P1 majorization: for σ = a jx ihx j, ρ = b jy ihy j 2 S(H), P Pn=1 n n n n=1 n n n ≺ () n # ≤ n # 8 2 N σ ρ i=1 ai i=1 bi ; n . def j i ! j i () 9 2 N [ f1g 9 H f gn 9 ϕ n , POVM on Mi i=1 and a set of LOCC def K f gn unitary on Ui i=1 s.t. Xn j ih j ⊗ j ih j ∗ ⊗ ∗ ϕ ϕ = (Mi Ui ) (Mi Ui ); in C1(H): i=1 H jj · jj "in C1(H)" means the convergence in Banach space (C1( ); 1) when n = 1. LOCC-convertibility[2/13] Theorem(Nielsen, 1999)[1][2, S12.5.1] : the case dim H, dim K < 1 j i ! jϕi () TrK j ih j ≺ TrK jϕihϕj LOCC Theorem(Owari et al, 2008)[3] : the case of dim H, dim K = 1 j i ! jϕi =) TrK j ih j ≺ TrK jϕihϕj LOCC TrK j ih j ≺ TrK jϕihϕj =) j i ! jϕi ϵ−LOCC where " ! " means "with (for any small) ϵ error by LOCC". ϵ−LOCC TrK j ih j ≺ TrK jϕihϕj ) j i ! jϕi in infinite dimensional space has LOCC been open.
    [Show full text]
  • Alternating Sign Matrices and Polynomiography
    Alternating Sign Matrices and Polynomiography Bahman Kalantari Department of Computer Science Rutgers University, USA [email protected] Submitted: Apr 10, 2011; Accepted: Oct 15, 2011; Published: Oct 31, 2011 Mathematics Subject Classifications: 00A66, 15B35, 15B51, 30C15 Dedicated to Doron Zeilberger on the occasion of his sixtieth birthday Abstract To each permutation matrix we associate a complex permutation polynomial with roots at lattice points corresponding to the position of the ones. More generally, to an alternating sign matrix (ASM) we associate a complex alternating sign polynomial. On the one hand visualization of these polynomials through polynomiography, in a combinatorial fashion, provides for a rich source of algo- rithmic art-making, interdisciplinary teaching, and even leads to games. On the other hand, this combines a variety of concepts such as symmetry, counting and combinatorics, iteration functions and dynamical systems, giving rise to a source of research topics. More generally, we assign classes of polynomials to matrices in the Birkhoff and ASM polytopes. From the characterization of vertices of these polytopes, and by proving a symmetry-preserving property, we argue that polynomiography of ASMs form building blocks for approximate polynomiography for polynomials corresponding to any given member of these polytopes. To this end we offer an algorithm to express any member of the ASM polytope as a convex of combination of ASMs. In particular, we can give exact or approximate polynomiography for any Latin Square or Sudoku solution. We exhibit some images. Keywords: Alternating Sign Matrices, Polynomial Roots, Newton’s Method, Voronoi Diagram, Doubly Stochastic Matrices, Latin Squares, Linear Programming, Polynomiography 1 Introduction Polynomials are undoubtedly one of the most significant objects in all of mathematics and the sciences, particularly in combinatorics.
    [Show full text]
  • MATLAB Array Manipulation Tips and Tricks
    MATLAB array manipulation tips and tricks Peter J. Acklam E-mail: [email protected] URL: http://home.online.no/~pjacklam 14th August 2002 Abstract This document is intended to be a compilation of tips and tricks mainly related to efficient ways of performing low-level array manipulation in MATLAB. Here, “manipu- lation” means replicating and rotating arrays or parts of arrays, inserting, extracting, permuting and shifting elements, generating combinations and permutations of ele- ments, run-length encoding and decoding, multiplying and dividing arrays and calcu- lating distance matrics and so forth. A few other issues regarding how to write fast MATLAB code are also covered. I'd like to thank the following people (in alphabetical order) for their suggestions, spotting typos and other contributions they have made. Ken Doniger and Dr. Denis Gilbert Copyright © 2000–2002 Peter J. Acklam. All rights reserved. Any material in this document may be reproduced or duplicated for personal or educational use. MATLAB is a trademark of The MathWorks, Inc. (http://www.mathworks.com). TEX is a trademark of the American Mathematical Society (http://www.ams.org). Adobe PostScript and Adobe Acrobat Reader are trademarks of Adobe Systems Incorporated (http://www.adobe.com). The TEX source was written with the GNU Emacs text editor. The GNU Emacs home page is http://www.gnu.org/software/emacs/emacs.html. The TEX source was formatted with AMS-LATEX to produce a DVI (device independent) file. The PS (PostScript) version was created from the DVI file with dvips by Tomas Rokicki. The PDF (Portable Document Format) version was created from the PS file with ps2pdf, a part of Aladdin Ghostscript by Aladdin Enterprises.
    [Show full text]
  • The Pagerank Algorithm Is One Way of Ranking the Nodes in a Graph by Importance
    The PageRank 13 Algorithm Lab Objective: Many real-world systemsthe internet, transportation grids, social media, and so oncan be represented as graphs (networks). The PageRank algorithm is one way of ranking the nodes in a graph by importance. Though it is a relatively simple algorithm, the idea gave birth to the Google search engine in 1998 and has shaped much of the information age since then. In this lab we implement the PageRank algorithm with a few dierent approaches, then use it to rank the nodes of a few dierent networks. The PageRank Model The internet is a collection of webpages, each of which may have a hyperlink to any other page. One possible model for a set of n webpages is a directed graph, where each node represents a page and node j points to node i if page j links to page i. The corresponding adjacency matrix A satises Aij = 1 if node j links to node i and Aij = 0 otherwise. b c abcd a 2 0 0 0 0 3 b 6 1 0 1 0 7 A = 6 7 c 4 1 0 0 1 5 d 1 0 1 0 a d Figure 13.1: A directed unweighted graph with four nodes, together with its adjacency matrix. Note that the column for node b is all zeros, indicating that b is a sinka node that doesn't point to any other node. If n users start on random pages in the network and click on a link every 5 minutes, which page in the network will have the most views after an hour? Which will have the fewest? The goal of the PageRank algorithm is to solve this problem in general, therefore determining how important each webpage is.
    [Show full text]
  • A Riemannian Fletcher--Reeves Conjugate Gradient Method For
    A Riemannian Fletcher-Reeves Conjugate Gradient Method for Title Doubly Stochastic Inverse Eigenvalue Problems Author(s) Yao, T; Bai, Z; Zhao, Z; Ching, WK SIAM Journal on Matrix Analysis and Applications, 2016, v. 37 n. Citation 1, p. 215-234 Issued Date 2016 URL http://hdl.handle.net/10722/229223 SIAM Journal on Matrix Analysis and Applications. Copyright © Society for Industrial and Applied Mathematics.; This work is Rights licensed under a Creative Commons Attribution- NonCommercial-NoDerivatives 4.0 International License. SIAM J. MATRIX ANAL. APPL. c 2016 Society for Industrial and Applied Mathematics Vol. 37, No. 1, pp. 215–234 A RIEMANNIAN FLETCHER–REEVES CONJUGATE GRADIENT METHOD FOR DOUBLY STOCHASTIC INVERSE EIGENVALUE PROBLEMS∗ TENG-TENG YAO†, ZHENG-JIAN BAI‡,ZHIZHAO §, AND WAI-KI CHING¶ Abstract. We consider the inverse eigenvalue problem of reconstructing a doubly stochastic matrix from the given spectrum data. We reformulate this inverse problem as a constrained non- linear least squares problem over several matrix manifolds, which minimizes the distance between isospectral matrices and doubly stochastic matrices. Then a Riemannian Fletcher–Reeves conjugate gradient method is proposed for solving the constrained nonlinear least squares problem, and its global convergence is established. An extra gain is that a new Riemannian isospectral flow method is obtained. Our method is also extended to the case of prescribed entries. Finally, some numerical tests are reported to illustrate the efficiency of the proposed method. Key words. inverse eigenvalue problem, doubly stochastic matrix, Riemannian Fletcher–Reeves conjugate gradient method, Riemannian isospectral flow AMS subject classifications. 65F18, 65F15, 15A18, 65K05, 90C26, 90C48 DOI. 10.1137/15M1023051 1.
    [Show full text]
  • An Alternate Exposition of PCA Ronak Mehta
    An Alternate Exposition of PCA Ronak Mehta 1 Introduction The purpose of this note is to present the principle components analysis (PCA) method alongside two important background topics: the construction of the singular value decomposition (SVD) and the interpretation of the covariance matrix of a random variable. Surprisingly, students have typically not seen these topics before encountering PCA in a statistics or machine learning course. Here, our focus is to understand why the SVD of the data matrix solves the PCA problem, and to interpret each of its components exactly. That being said, this note is not meant to be a self- contained introduction to the topic. Important motivations, such the maximum total variance or minimum reconstruction error optimizations over subspaces are not covered. Instead, this can be considered a supplementary background review to read before jumping into other expositions of PCA. 2 Linear Algebra Review 2.1 Notation d n×d R is the set of real d-dimensional vectors, and R is the set of real n-by-d matrices. A vector d n×d x 2 R , is denoted as a bold lower case symbol, with its j-th element as xj. A matrix X 2 R is denoted as a bold upper case symbol, with the element at row i and column j as Xij, and xi· and x·j denoting the i-th row and j-th column respectively (the center dot might be dropped if it d×d is clear in context). The identity matrix Id 2 R is the matrix of ones on the diagonal and zeros d elsewhere.
    [Show full text]
  • Some Remarks on the Permanents of Circulant (0,1) Matrices
    University of Wollongong Research Online Faculty of Engineering and Information Faculty of Informatics - Papers (Archive) Sciences 1982 Some remarks on the permanents of circulant (0,1) matrices Peter Eades Cheryl E. Praeger Jennifer Seberry University of Wollongong, [email protected] Follow this and additional works at: https://ro.uow.edu.au/infopapers Part of the Physical Sciences and Mathematics Commons Recommended Citation Eades, Peter; Praeger, Cheryl E.; and Seberry, Jennifer: Some remarks on the permanents of circulant (0,1) matrices 1982. https://ro.uow.edu.au/infopapers/1010 Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library: [email protected] Some remarks on the permanents of circulant (0,1) matrices Abstract Some permanents of circulant (0,1) matrices are computed. Three methods are used. First, the permanent of a Kronecker product is computed by directly counting diagonals. Secondly, Lagrange expansion is used to calculate a recurrence for a family of sparse circulants. Finally, a "complement expansion" method is used to calculate a recurrence for a permanent of a circulant with few zero entries. Also, a bound on the number of different permanents of circulant matrices with a given row sum is obtained. Disciplines Physical Sciences and Mathematics Publication Details Eades, P, Praeger, C and Seberry, J, Some remarks on the permanents of circulant (0,1) matrices, Utilitas Mathematica, 23, 1983, 145-159. This journal article is available at Research Online: https://ro.uow.edu.au/infopapers/1010 SOME REHARKS ON THE PERMANENTS OF CIRCULANT (0,1) MATRICES Peter Eades, Cheryl E.
    [Show full text]
  • Signal and System Analysis Fall 2021 Lab#2 – Matrix and Array in MATLAB
    EECS 360: Signal and System Analysis Fall 2021 Lab#2 – Matrix and Array in MATLAB Matrices are the basic elements of the MATLAB environment. A matrix is a two-dimensional array consisting of m rows and n columns. Special cases are column vectors (n = 1) and row vectors (m = 1). In this lab, we will illustrate how to create array and matrices and how to apply different operations on matrices. 1. ENTERING A VECTOR We define a matrix of dimension (n × m) as follows, where n stands for number of rows and m stands for number of columns. 1 2 . m 2 . n Rows ..... ..... n .... m Columns A vector is a special case of a matrix, where either number of rows or number of columns goes to 1. Row Vector: An array of dimension (1 × n) is called a row vector. Column Vector: An array of dimension (m × 1) is called a column vector. The elements of vectors in MATLAB are enclosed by square brackets and are separated by spaces or by commas. For example, in Matlab, you can define a Row Vector as: A = [1 2 3 4] or A = [1,2,3,4]. Column vectors are created in a similar way, however, semicolon (;) must separate the components of a column vector. For example, Column Vector: B= [1; 2; 3; 4] *** A row vector is converted to a column vector using the transpose operator and vice versa. The transpose operation is denoted by an apostrophe or a single quote (’). AB= ' or BA= ' PRACTICE 1: Create one row vector (1 5) and one column vector (5 1) and perform transpose operation to convert them from one type to another (i.e.
    [Show full text]
  • The Greedy Strategy in Optimizing the Perron Eigenvalue
    The greedy strategy for optimizing the Perron eigenvalue ∗ Aleksandar Cvetkovi´c, †Vladimir Yu. Protasov ‡ Abstract We address the problems of minimizing and of maximizing the spectral radius over a compact family of non-negative matrices. Those problems being hard in general can be efficiently solved for some special families. We consider the so-called prod- uct families, where each matrix is composed of rows chosen independently from given sets. A recently introduced greedy method works very fast. However, it is applicable mostly for strictly positive matrices. For sparse matrices, it often diverges and gives a wrong answer. We present the “selective greedy method” that works equally well for all non-negative product families, including sparse ones. For this method, we prove a quadratic rate of convergence and demonstrate its efficiency in numerical examples. The numerical examples are realised for two cases: finite uncertainty sets and poly- hedral uncertainty sets given by systems of linear inequalities. In dimensions up to 2000, the matrices with minimal/maximal spectral radii in product families are found within a few iterations. Applications to dynamical systems and to the graph theory are considered. Keywords: iterative optimization method, non-negative matrix, spectral radius, relax- ation algorithm, cycling, spectrum of a graph, quadratic convergence, dynamical system, stability AMS 2010 subject classification: 15B48, 90C26, 15A42 arXiv:1807.05099v2 [math.OC] 18 May 2020 1. Introduction The problem of minimizing or maximizing the spectral radius of a non-negative matrix over a general constrained set can be very hard. There are no efficient algorithms even for ∗The research is supported by the RSF grant 20-11-20169 †GSSI (L’Aquila, Italy), ‡University of L’Aquila (Italy), Moscow State University (Moscow, Russia), e-mail: [email protected] 1 solving this problem over a compact convex (say, polyhedral) set of positive matrices.
    [Show full text]
  • Ground Metric Learning
    Ground Metric Learning Marco Cuturi [email protected] David Avis [email protected] Graduate School of Informatics Kyoto University Editor: Gert Lanckriet Abstract Optimal transport distances have been used for more than a decade in machine learning to compare histograms of features. They have one parameter: the ground metric, which can be any metric between the features themselves. As is the case for all parameterized dis- tances, optimal transport distances can only prove useful in practice when this parameter is carefully chosen. To date, the only option available to practitioners to set the ground met- ric parameter was to rely on a priori knowledge of the features, which limited considerably the scope of application of optimal transport distances. We propose to lift this limitation and consider instead algorithms that can learn the ground metric using only a training set of labeled histograms. We call this approach ground metric learning. We formulate the problem of learning the ground metric as the minimization of the difference of two convex polyhedral functions over a convex set of metric matrices. We follow the presentation of our algorithms with promising experimental results which show that this approach is useful both for retrieval and binary/multiclass classification tasks. 1. Introduction We consider in this paper the problem of learning a distance for normalized histograms. Normalized histograms – finite-dimensional vectors with nonnegative coordinates whose sum is equal to 1 – arise frequently in natural language processing, computer vision, bioin- formatics and more generally areas involving complex datatypes. Objects of interest in such areas are usually simplified and are represented as a bag of smaller features.
    [Show full text]