Expander Graph and Communication-Efficient Decentralized Optimization
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Interlacing Families I: Bipartite Ramanujan Graphs of All Degrees
Annals of Mathematics 182 (2015), 307{325 Interlacing families I: Bipartite Ramanujan graphs of all degrees By Adam W. Marcus, Daniel A. Spielman, and Nikhil Srivastava Abstract We prove that there exist infinite families of regular bipartite Ramanujan graphs of every degree bigger than 2. We do this by proving a variant of a conjecture of Bilu and Linial about the existence of good 2-lifts of every graph. We also establish the existence of infinite families of \irregular Ramanujan" graphs, whose eigenvalues are bounded by the spectral radius of their universal cover. Such families were conjectured to exist by Linial and others. In particular, we prove the existence of infinite families of (c; d)-biregular bipartite graphs with all nontrivial eigenvalues bounded by p p c − 1 + d − 1 for all c; d ≥ 3. Our proof exploits a new technique for controlling the eigenvalues of certain random matrices, which we call the \method of interlacing polynomials." 1. Introduction Ramanujan graphs have been the focus of substantial study in Theoret- ical Computer Science and Mathematics. They are graphs whose nontrivial adjacency matrix eigenvalues are as small as possible. Previous constructions of Ramanujan graphs have been sporadic, only producing infinite families of Ramanujan graphs of certain special degrees. In this paper, we prove a variant of a conjecture of Bilu and Linial [4] and use it to realize an approach they suggested for constructing bipartite Ramanujan graphs of every degree. Our main technical contribution is a novel existence argument. The con- jecture of Bilu and Linial requires us to prove that every graph has a signed adjacency matrix with all of its eigenvalues in a small range. -
Expander Graphs
4 Expander Graphs Now that we have seen a variety of basic derandomization techniques, we will move on to study the first major “pseudorandom object” in this survey, expander graphs. These are graphs that are “sparse” yet very “well-connected.” 4.1 Measures of Expansion We will typically interpret the properties of expander graphs in an asymptotic sense. That is, there will be an infinite family of graphs Gi, with a growing number of vertices Ni. By “sparse,” we mean that the degree Di of Gi should be very slowly growing as a function of Ni. The “well-connectedness” property has a variety of different interpretations, which we will discuss below. Typically, we will drop the subscripts of i and the fact that we are talking about an infinite family of graphs will be implicit in our theorems. As in Section 2.4.2, we will state many of our definitions for directed multigraphs (which we’ll call digraphs for short), though in the end we will mostly study undirected multigraphs. 80 4.1 Measures of Expansion 81 4.1.1 Vertex Expansion The classic measure of well-connectedness in expanders requires that every “not-too-large” set of vertices has “many” neighbors: Definition 4.1. A digraph G isa(K,A) vertex expander if for all sets S of at most K vertices, the neighborhood N(S) def= {u|∃v ∈ S s.t. (u,v) ∈ E} is of size at least A ·|S|. Ideally, we would like D = O(1), K =Ω(N), where N is the number of vertices, and A as close to D as possible. -
Ramanujan Graphs of Every Degree
Ramanujan Graphs of Every Degree Adam Marcus (Crisply, Yale) Daniel Spielman (Yale) Nikhil Srivastava (MSR India) Expander Graphs Sparse, regular well-connected graphs with many properties of random graphs. Random walks mix quickly. Every set of vertices has many neighbors. Pseudo-random generators. Error-correcting codes. Sparse approximations of complete graphs. Spectral Expanders Let G be a graph and A be its adjacency matrix a 0 1 0 0 1 b 1 0 1 0 1 e 0 1 0 1 0 c 0 0 1 0 1 1 1 0 1 0 d Eigenvalues λ1 λ2 λn “trivial” ≥ ≥ ···≥ If d-regular (every vertex has d edges), λ1 = d Spectral Expanders If bipartite (all edges between two parts/colors) eigenvalues are symmetric about 0 If d-regular and bipartite, λ = d n − “trivial” a b 0 0 0 1 0 1 0 0 0 1 1 0 0 0 0 0 1 1 c d 1 1 0 0 0 0 0 1 1 0 0 0 e f 1 0 1 0 0 0 Spectral Expanders G is a good spectral expander if all non-trivial eigenvalues are small [ ] -d 0 d Bipartite Complete Graph Adjacency matrix has rank 2, so all non-trivial eigenvalues are 0 a b 0 0 0 1 1 1 0 0 0 1 1 1 0 0 0 1 1 1 c d 1 1 1 0 0 0 1 1 1 0 0 0 e f 1 1 1 0 0 0 Spectral Expanders G is a good spectral expander if all non-trivial eigenvalues are small [ ] -d 0 d Challenge: construct infinite families of fixed degree Spectral Expanders G is a good spectral expander if all non-trivial eigenvalues are small [ ] ( 0 ) -d 2pd 1 2pd 1 d − − − Challenge: construct infinite families of fixed degree Alon-Boppana ‘86: Cannot beat 2pd 1 − Ramanujan Graphs: 2pd 1 − G is a Ramanujan Graph if absolute value of non-trivial eigs 2pd 1 − [ -
X-Ramanujan-Graphs.Pdf
X-Ramanujan graphs Sidhanth Mohanty* Ryan O’Donnell† April 4, 2019 Abstract Let X be an infinite graph of bounded degree; e.g., the Cayley graph of a free product of finite groups. If G is a finite graph covered by X, it is said to be X-Ramanujan if its second- largest eigenvalue l2(G) is at most the spectral radius r(X) of X, and more generally k-quasi- X-Ramanujan if lk(G) is at most r(X). In case X is the infinite D-regular tree, this reduces to the well known notion of a finite D-regular graph being Ramanujan. Inspired by the Interlacing Polynomials method of Marcus, Spielman, and Srivastava, we show the existence of infinitely many k-quasi-X-Ramanujan graphs for a variety of infinite X. In particular, X need not be a tree; our analysis is applicable whenever X is what we call an additive product graph. This additive product is a new construction of an infinite graph A1 + ··· + Ac from finite “atom” graphs A1,..., Ac over a common vertex set. It generalizes the notion of the free product graph A1 ∗ · · · ∗ Ac when the atoms Aj are vertex-transitive, and it generalizes the notion of the uni- versal covering tree when the atoms Aj are single-edge graphs. Key to our analysis is a new graph polynomial a(A1,..., Ac; x) that we call the additive characteristic polynomial. It general- izes the well known matching polynomial m(G; x) in case the atoms Aj are the single edges of G, and it generalizes the r-characteristic polynomial introduced in [Rav16, LR18]. -
Arxiv:1711.08757V3 [Cs.CV] 26 Jul 2018 1 Introduction
Deep Expander Networks: Efficient Deep Networks from Graph Theory Ameya Prabhu? Girish Varma? Anoop Namboodiri Center for Visual Information Technology Kohli Center on Intelligent Systems, IIIT Hyderabad, India [email protected], fgirish.varma, [email protected] https://github.com/DrImpossible/Deep-Expander-Networks Abstract. Efficient CNN designs like ResNets and DenseNet were pro- posed to improve accuracy vs efficiency trade-offs. They essentially in- creased the connectivity, allowing efficient information flow across layers. Inspired by these techniques, we propose to model connections between filters of a CNN using graphs which are simultaneously sparse and well connected. Sparsity results in efficiency while well connectedness can pre- serve the expressive power of the CNNs. We use a well-studied class of graphs from theoretical computer science that satisfies these properties known as Expander graphs. Expander graphs are used to model con- nections between filters in CNNs to design networks called X-Nets. We present two guarantees on the connectivity of X-Nets: Each node influ- ences every node in a layer in logarithmic steps, and the number of paths between two sets of nodes is proportional to the product of their sizes. We also propose efficient training and inference algorithms, making it possible to train deeper and wider X-Nets effectively. Expander based models give a 4% improvement in accuracy on Mo- bileNet over grouped convolutions, a popular technique, which has the same sparsity but worse connectivity. X-Nets give better performance trade-offs than the original ResNet and DenseNet-BC architectures. We achieve model sizes comparable to state-of-the-art pruning techniques us- ing our simple architecture design, without any pruning. -
Expander Graphs
Basic Facts about Expander Graphs Oded Goldreich Abstract. In this survey we review basic facts regarding expander graphs that are most relevant to the theory of computation. Keywords: Expander Graphs, Random Walks on Graphs. This text has been revised based on [8, Apdx. E.2]. 1 Introduction Expander graphs found numerous applications in the theory of computation, ranging from the design of sorting networks [1] to the proof that undirected connectivity is decidable in determinstic log-space [15]. In this survey we review basic facts regarding expander graphs that are most relevant to the theory of computation. For a wider perspective, the interested reader is referred to [10]. Loosely speaking, expander graphs are regular graphs of small degree that exhibit various properties of cliques.1 In particular, we refer to properties such as the relative sizes of cuts in the graph (i.e., relative to the number of edges), and the rate at which a random walk converges to the uniform distribution (relative to the logarithm of the graph size to the base of its degree). Some technicalities. Typical presentations of expander graphs refer to one of several variants. For example, in some sources, expanders are presented as bi- partite graphs, whereas in others they are presented as ordinary graphs (and are in fact very far from being bipartite). We shall follow the latter convention. Furthermore, at times we implicitly consider an augmentation of these graphs where self-loops are added to each vertex. For simplicity, we also allow parallel edges. We often talk of expander graphs while we actually mean an infinite collection of graphs such that each graph in this collection satisfies the same property (which is informally attributed to the collection). -
Connections Between Graph Theory and Cryptography
Connections between graph theory and cryptography Connections between graph theory and cryptography Natalia Tokareva G2C2: Graphs and Groups, Cycles and Coverings September, 24–26, 2014. Novosibirsk, Russia Connections between graph theory and cryptography Introduction to cryptography Introduction to cryptography Connections between graph theory and cryptography Introduction to cryptography Terminology Cryptography is the scientific and practical activity associated with developing of cryptographic security facilities of information and also with argumentation of their cryptographic resistance. Plaintext is a secret message. Often it is a sequence of binary bits. Ciphertext is an encrypted message. Encryption is the process of disguising a message in such a way as to hide its substance (the process of transformation plaintext into ciphertext by virtue of cipher). Cipher is a family of invertible mappings from the set of plaintext sequences to the set of ciphertext sequences. Each mapping depends on special parameter a key. Key is removable part of the cipher. Connections between graph theory and cryptography Introduction to cryptography Terminology Deciphering is the process of turning a ciphertext back into the plaintext that realized with known key. Decryption is the process of receiving the plaintext from ciphertext without knowing the key. Connections between graph theory and cryptography Introduction to cryptography Terminology Cryptography is the scientific and practical activity associated with developing of cryptographic security -
Layouts of Expander Graphs
CHICAGO JOURNAL OF THEORETICAL COMPUTER SCIENCE 2016, Article 1, pages 1–21 http://cjtcs.cs.uchicago.edu/ Layouts of Expander Graphs † ‡ Vida Dujmovic´∗ Anastasios Sidiropoulos David R. Wood Received January 20, 2015; Revised November 26, 2015, and on January 3, 2016; Published January 14, 2016 Abstract: Bourgain and Yehudayoff recently constructed O(1)-monotone bipartite ex- panders. By combining this result with a generalisation of the unraveling method of Kannan, we construct 3-monotone bipartite expanders, which is best possible. We then show that the same graphs admit 3-page book embeddings, 2-queue layouts, 4-track layouts, and have simple thickness 2. All these results are best possible. Key words and phrases: expander graph, monotone layout, book embedding, stack layout, queue layout, track layout, thickness 1 Introduction Expanders are classes of highly connected graphs that are of fundamental importance in graph theory, with numerous applications, especially in theoretical computer science [31]. While the literature contains various definitions of expanders, this paper focuses on bipartite expanders. For e (0;1], a bipartite 2 graph G with bipartition V(G) = A B is a bipartite e-expander if A = B and N(S) > (1 + e) S for A [ j j j j j j j j every subset S A with S j j . Here N(S) is the set of vertices adjacent to some vertex in S. An infinite ⊂ j j 6 2 family of bipartite e-expanders, for some fixed e > 0, is called an infinite family of bipartite expanders. There has been much research on constructing and proving the existence of expanders with various desirable properties. -
Zeta Functions of Finite Graphs and Coverings, III
Graph Zeta Functions 1 INTRODUCTION Zeta Functions of Finite Graphs and Coverings, III A. A. Terras Math. Dept., U.C.S.D., La Jolla, CA 92093-0112 E-mail: [email protected] and H. M. Stark Math. Dept., U.C.S.D., La Jolla, CA 92093-0112 Version: March 13, 2006 A graph theoretical analog of Brauer-Siegel theory for zeta functions of number …elds is developed using the theory of Artin L-functions for Galois coverings of graphs from parts I and II. In the process, we discuss possible versions of the Riemann hypothesis for the Ihara zeta function of an irregular graph. 1. INTRODUCTION In our previous two papers [12], [13] we developed the theory of zeta and L-functions of graphs and covering graphs. Here zeta and L-functions are reciprocals of polynomials which means these functions have poles not zeros. Just as number theorists are interested in the locations of the zeros of number theoretic zeta and L-functions, thanks to applications to the distribution of primes, we are interested in knowing the locations of the poles of graph-theoretic zeta and L-functions. We study an analog of Brauer-Siegel theory for the zeta functions of number …elds (see Stark [11] or Lang [6]). As explained below, this is a necessary step in the discussion of the distribution of primes. We will always assume that our graphs X are …nite, connected, rank 1 with no danglers (i.e., degree 1 vertices). Let us recall some of the de…nitions basic to Stark and Terras [12], [13]. -
Expander Graphs and Their Applications
Expander Graphs and their Applications Lecture notes for a course by Nati Linial and Avi Wigderson The Hebrew University, Israel. January 1, 2003 Contents 1 The Magical Mystery Tour 7 1.1SomeProblems.............................................. 7 1.1.1 Hardnessresultsforlineartransformation............................ 7 1.1.2 ErrorCorrectingCodes...................................... 8 1.1.3 De-randomizing Algorithms . .................................. 9 1.2MagicalGraphs.............................................. 10 ´Òµ 1.2.1 A Super Concentrator with Ç edges............................. 12 1.2.2 ErrorCorrectingCodes...................................... 12 1.2.3 De-randomizing Random Algorithms . ........................... 13 1.3Conclusions................................................ 14 2 Graph Expansion & Eigenvalues 15 2.1Definitions................................................. 15 2.2ExamplesofExpanderGraphs...................................... 16 2.3TheSpectrumofaGraph......................................... 16 2.4TheExpanderMixingLemmaandApplications............................. 17 2.4.1 DeterministicErrorAmplificationforBPP........................... 18 2.5HowBigCantheSpectralGapbe?.................................... 19 3 Random Walks on Expander Graphs 21 3.1Preliminaries............................................... 21 3.2 A random walk on an expander is rapidly mixing. ........................... 21 3.3 A random walk on an expander yields a sequence of “good samples” . ............... 23 3.3.1 Application: amplifying -
Large Minors in Expanders
Large Minors in Expanders Julia Chuzhoy∗ Rachit Nimavaty January 26, 2019 Abstract In this paper we study expander graphs and their minors. Specifically, we attempt to answer the following question: what is the largest function f(n; α; d), such that every n-vertex α-expander with maximum vertex degree at most d contains every graph H with at most f(n; α; d) edges and vertices as a minor? Our main result is that there is some universal constant c, such that f(n; α; d) ≥ n α c c log n · d . This bound achieves a tight dependence on n: it is well known that there are bounded- degree n-vertex expanders, that do not contain any grid with Ω(n= log n) vertices and edges as a minor. The best previous result showed that f(n; α; d) ≥ Ω(n= logκ n), where κ depends on both α and d. Additionally, we provide a randomized algorithm, that, given an n-vertex α-expander with n α c maximum vertex degree at most d, and another graph H containing at most c log n · d vertices and edges, with high probability finds a model of H in G, in time poly(n) · (d/α)O(log(d/α)). We also show a simple randomized algorithm with running time poly(n; d/α), that obtains a similar result with slightly weaker dependence on n but a better dependence on d and α, namely: if G is α3n an n-vertex α-expander with maximum vertex degree at most d, and H contains at most c0d5 log2 n edges and vertices, where c0 is an absolute constant, then our algorithm with high probability finds a model of H in G. -
Arxiv:1511.09340V2 [Math.NT] 25 Mar 2017 from Below by Logk−1(N) and It Could Get As Large As a Scalar Multiple of N
DIAMETER OF RAMANUJAN GRAPHS AND RANDOM CAYLEY GRAPHS NASER T SARDARI Abstract. We study the diameter of LPS Ramanujan graphs Xp;q. We show that the diameter of the bipartite Ramanujan graphs is greater than (4=3) logp(n) + O(1) where n is the number of vertices of Xp;q. We also con- struct an infinite family of (p + 1)-regular LPS Ramanujan graphs Xp;m such that the diameter of these graphs is greater than or equal to b(4=3) logp(n)c. On the other hand, for any k-regular Ramanujan graph we show that the distance of only a tiny fraction of all pairs of vertices is greater than (1 + ) logk−1(n). We also have some numerical experiments for LPS Ramanu- jan graphs and random Cayley graphs which suggest that the diameters are asymptotically (4=3) logk−1(n) and logk−1(n), respectively. Contents 1. Introduction 1 1.1. Motivation 1 1.2. Statement of results 3 1.3. Outline of the paper 5 1.4. Acknowledgments 5 2. Lower bound for the diameter of the Ramanujan graphs 6 3. Visiting almost all points after (1 + ) logk−1(n) steps 9 4. Numerical Results 11 References 14 1. Introduction 1.1. Motivation. The diameter of any k-regular graph with n vertices is bounded arXiv:1511.09340v2 [math.NT] 25 Mar 2017 from below by logk−1(n) and it could get as large as a scalar multiple of n. It is known that the diameter of any k-regular Ramanujan graph is bounded from above by 2(1 + ) logk−1(n) [LPS88].