DOCSLIB.ORG

Symmetric Boolean Factor Analysis with Applications to Instahide

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Symmetric Boolean Factor Analysis with Applications to Instahide Symmetric Boolean Factor Analysis with Applications to InstaHide Sitan Chen∗ Zhao Song† Runzhou Tao‡ Ruizhe Zhang§ Abstract In this work we examine the security of InstaHide, a recently proposed scheme for distributed learning [HSLA20]. A number of recent works have given reconstruction attacks for InstaHide in various regimes [CLSZ21, Car20, HST+20] by leveraging an intriguing connection to the follow- ing matrix factorization problem: given the Gram matrix of a collection of m random k-sparse Boolean vectors in 0, 1 r, recover the vectors (up to the trivial symmetries). Equivalently, this can be thought{ of as} a sparse, symmetric variant of the well-studied problem of Boolean factor analysis, or as an average-case version of the classic problem of recovering a k-uniform hypergraph from its line graph. As previous algorithms either required m to be exponentially large in k [CLSZ21] or only applied to k = 2 [Car20, HST+20], they left open the question of whether InstaHide possesses some form of “fine-grained security” against reconstruction attacks for moderately large k. In this work, we answer this in the negative by giving a simple O(mω+1) time algorithm for the above matrix factorization problem. Our algorithm, based on tensor decomposition, only requires m = Ω(r). We complement this result with a quasipolynomial-time algorithm for a worst-case setting of the problem where the collection of k-sparse vectors is chosen arbitrarily. e arXiv:2102.01570v1 [cs.LG] 2 Feb 2021 ∗[email protected]. MIT. This work was supported in part by NSF CAREER Award CCF-1453261, NSF Large CCF-1565235 and Ankur Moitra’s ONR Young Investigator Award. †[email protected]. Princeton University. ‡[email protected]. Columbia University. §[email protected]. The University of Texas at Austin. 1 Introduction We study the following symmetric variant of the well-known nonnegative matrix factorization (NMF) problem [AGKM12, Moi13, RSW16, SWZ17, SWZ19]. Given a symmetric, nonnegative m m × matrix M and rank parameter r, we are interested in the optimization problem: min M WW⊤ (1) W∈S − where is some family of nonnegative m r matrices and denotes some matrix norm. S × k · k When consists of all nonnegative m r matrices, this is the problem of symmetric NMF, S × which is closely related to kernel k-means [DHS05] and has been studied in a variety of contexts like graph clustering, topic modeling, and computer vision [ZS05, YHD+12, YGL+13, CRDH08, KG12, HXZ+11, WLW+11, KDP12]. We consider the setting where consists of matrices with 0, 1 -valued entries, in which case S { } (1) is the symmetric version of the well-studied problem of binary matrix factorization (BMF) [ZLDZ07, CIK17, BBB+19, FGL+19, KPRW19], which is connected to a diverse array of problems like LDPC codes [RPG16], optimizing passive OLED displays [KPRW19], and graph partitioning [CIK17]. For instance, if we equip 0, 1 with the structure of the Boolean semiring1 (in which case { } BMF is sometimes called Boolean factor analysis) and M is the adjacency matrix of undirected graph G, then (1) is equivalent to finding the best possible covering of G by r cliques. Boolean Factor Analysis and Distributed Learning Our motivation for studying Boolean factor analysis comes from an intriguing connection, recently identified by [CLSZ21, CDG+20], to designing attacks on a proposed scheme for private distributed learning called InstaHide [HSLA20]2. At a high level (see Section 5 for details), InstaHide is a procedure for taking a private dataset of size r and generating a synthetic dataset m via the following procedure. Fix a sparsity parameter k, and for every i = 1,...,m: 1) sample a random subset S of 1, . , r of size k, 2) define v to { } be the average of the k private feature vectors indexed by S, 3) add to the synthetic dataset the entrywise absolute value of v. The security of InstaHide has recently been placed under some scrutiny [Jag20, CLSZ21, CDG+20, Car20, HST+20], and a number of works have given attacks for reconstructing private feature vec- tors out of synthetic ones generated by InstaHide. [CLSZ21] gave a provable attack running in poly(m) time when the private dataset is Gaussian but required m to scale exponentially in k,3 while [CDG+20] gave a heuristic attack on real image data for k = 2. These works left open the following question which is the main focus of the present work: Question 1.1. Do there exist efficient reconstruction attacks on InstaHide for arbitrary k? Interestingly, [CLSZ21, CDG+20] both essentially reduce the reconstruction problem to solving the following instantiation of the optimization problem (1): ⊤ m×r min M WW 0 for m,r,k = W 0, 1 : Wj 0 = k j [m] , (2) W∈Sm,r,k k − k S { ∈ { } k k ∀ ∈ } where denotes number of nonzero entries. k · k0 In the reduction, the input M to the optimization problem (2) is the similarity matrix whose (i, j)-th entry is 1 if synthetic feature vectors i and j have at least one constituent private feature 1In the Boolean semiring, addition is given by logical OR, and multiplication is given by logical AND 2In a follow-up work, this was generalized to the text domain [HSC+20]. 3In the special case of k = 2, the bound in [CLSZ21] was later refined by [HST+20]. 1 vector in common, and 0 otherwise (see Definition 5.4). In practice, it is very reasonable to assume one can extract M from the synthetic dataset e.g. by training an appropriate neural network [CDG+20], and when the private dataset is Gaussian, there is a provable algorithm for doing so [CLSZ21]. The key point is that this matrix admits an exact factorization M = WW⊤ over the Boolean semiring, where W 0, 1 m×r is the matrix whose (i, ℓ)-th coordinate is 1 if synthetic feature ∈ { } vector i contains private feature vector ℓ, and is 0 otherwise (see Fact 5.5). In particular, because every synthetic feature vector has k constituent private feature vectors, W . [CLSZ21] ∈ Sm,r,k used a rather involved combinatorial argument to “partially” factorize M, while [CDG+20] gave a heuristic algorithm for completely factorizing M when k = 2 by solving a min-cost max-flow (see Section 1.1 for details). Our first result is to give a polynomial-time algorithm for (2) when M is the similarity matrix for a synthetic dataset generated by InstaHide: Theorem 1.2. Fix any integer 2 k r, failure probability δ (0, 1), and suppose m ≤ ≤ ∈ ≥ Ω(rk log(1/δ)). Let W 0, 1 m×r be generated by the following random process: for every i [m], ∈ { } ∈ the i-th row of W is a uniformly random k-sparse binary vector. Define M , WW⊤ where matrix multiplicatione is over the Boolean semiring. There is an algorithm which runs in O(mω+1) time and, with probability 1 δ over the randomness of W, outputs a matrix W 0, 1 m×r whose − ∈ { } columns are a permutation of those of W.4 c Note that not only is the minimum m for which Theorem 1.2 holds a fixed polynomial in r, k (rather than rΩ(k)), but in fact the dependence on r is near optimal (Remark 1.5)! As a consequence, we show a synthetic dataset generated by InstaHide is vulnerable to efficient reconstruction attacks as soon as its size is comparable to that of the private dataset from which it was generated, answering Question 1.1 in the affirmative: Theorem 1.3 (Informal, see Theorem 5.1). Fix any integer k 2, failure probability δ (0, 1), ≥ ∈ and suppose m Ω(rk log(d/δ)). Given a synthetic dataset of size m generated by InstaHide, ≥ together with its similarity matrix, there is an O(mω+1 + d r m)-time algorithm for approximately · · recovering the magnitudese of the “heavy” coordinates of every feature vector in the private dataset. Here, a coordinate of a feature vector is “heavy” if its magnitude is Ω(k) times the average value of any private image in that coordinate. One can interpret Theorem 1.2 as handling a realizable, average-case version of (2), i.e. it assumes input M admits an exact factorization and furthermore was generated by a random process. One can also ask how to handle worst-case instances of (2) where M can be arbitrary and in particular need not admit an exact factorization, and indeed we complement the results above with a quasipolynomial-time algorithm for (2) in this setting: Theorem 1.4 (Worst-case guarantee, see Theorem C.3 for formal statement). Given a symmetric matrix M Zm×m, rank parameter r m, and accuracy parameter ǫ (0, 1], there is an algorithm ∈ −1 2 ≤ ∈ that runs in mO(ǫ k log r) time and outputs W such that ∈ Sm,r,k ⊤ ⊤ 2 M WW 0 cmin M WW 0 + ǫm , k − k ≤ W∈Sm,r,k k − k where matrix multiplication canc bec over R or over the Boolean semiring. 4ω ≈ 2.373 is the exponent of matrix multiplication. 2 1.1 Related Work Symmetric NMF The most standard setting of NFM is minU V M UV⊤ , where U, V , ≥0 k − k range over all possible m r matrices with nonnegative entries. This has been the subject of a sig- × nificant body of theoretical and applied work, and we refer to the survey [Gil12] for a comprehensive overview of this literature. Symmetric NMF has received comparatively less attention but is nevertheless a popular clus- tering technique [HXZ+11, KDP12, DHS05] where, similar to our application to InstaHide, one takes the input matrix M to be some similarity matrix for a collection of data points and interprets the factorization W as specifying a soft clustering of these points into r groups, where Wi,ℓ is the “probability” that point i lies in cluster ℓ.
Load more

Recommended publications
  • Perron–Frobenius Theory of Nonnegative Matrices
    Buy from AMAZON.com http://www.amazon.com/exec/obidos/ASIN/0898714540 CHAPTER 8 Perron–Frobenius Theory of Nonnegative Matrices 8.1 INTRODUCTION m×n A ∈ is said to be a nonnegative matrix whenever each aij ≥ 0, and this is denoted by writing A ≥ 0. In general, A ≥ B means that each aij ≥ bij. Similarly, A is a positive matrix when each aij > 0, and this is denoted by writing A > 0. More generally, A > B means that each aij >bij. Applications abound with nonnegative and positive matrices. In fact, many of the applications considered in this text involve nonnegative matrices. For example, the connectivity matrix C in Example 3.5.2 (p. 100) is nonnegative. The discrete Laplacian L from Example 7.6.2 (p. 563) leads to a nonnegative matrix because (4I − L) ≥ 0. The matrix eAt that defines the solution of the system of differential equations in the mixing problem of Example 7.9.7 (p. 610) is nonnegative for all t ≥ 0. And the system of difference equations p(k)=Ap(k − 1) resulting from the shell game of Example 7.10.8 (p. 635) has Please report violations to [email protected] a nonnegativeCOPYRIGHTED coefficient matrix A. Since nonnegative matrices are pervasive, it’s natural to investigate their properties, and that’s the purpose of this chapter. A primary issue concerns It is illegal to print, duplicate, or distribute this material the extent to which the properties A > 0 or A ≥ 0 translate to spectral properties—e.g., to what extent does A have positive (or nonnegative) eigen- values and eigenvectors? The topic is called the “Perron–Frobenius theory” because it evolved from the contributions of the German mathematicians Oskar (or Oscar) Perron 89 and 89 Oskar Perron (1880–1975) originally set out to fulfill his father’s wishes to be in the family busi- Buy online from SIAM Copyright c 2000 SIAM http://www.ec-securehost.com/SIAM/ot71.html Buy from AMAZON.com 662 Chapter 8 Perron–Frobenius Theory of Nonnegative Matrices http://www.amazon.com/exec/obidos/ASIN/0898714540 Ferdinand Georg Frobenius.
    [Show full text]
  • 4.6 Matrices Dimensions: Number of Rows and Columns a Is a 2 X 3 Matrix
    Matrices are rectangular arrangements of data that are used to represent information in tabular form. 1 0 4 A = 3 - 6 8 a23 4.6 Matrices Dimensions: number of rows and columns A is a 2 x 3 matrix Elements of a matrix A are denoted by aij. a23 = 8 1 2 Data about many kinds of problems can often be represented by Matrix of coefficients matrix. Solutions to many problems can be obtained by solving e.g Average temperatures in 3 different cities for each month: systems of linear equations. For example, the constraints of a problem are represented by the system of linear equations 23 26 38 47 58 71 78 77 69 55 39 33 x + y = 70 A = 14 21 33 38 44 57 61 59 49 38 25 21 3 cities 24x + 14y = 1180 35 46 54 67 78 86 91 94 89 75 62 51 12 months Jan - Dec 1 1 rd The matrix A = Average temp. in the 3 24 14 city in July, a37, is 91. is the matrix of coefficient for this system of linear equations. 3 4 In a matrix, the arrangement of the entries is significant. Square Matrix is a matrix in which the number of Therefore, for two matrices to be equal they must have rows equals the number of columns. the same dimensions and the same entries in each location. •Main Diagonal: in a n x n square matrix, the elements a , a , a , …, a form the main Example: Let 11 22 33 nn diagonal of the matrix.
    [Show full text]
  • Recent Developments in Boolean Matrix Factorization
    Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20) Survey Track Recent Developments in Boolean Matrix Factorization Pauli Miettinen1∗ and Stefan Neumann2y 1University of Eastern Finland, Kuopio, Finland 2University of Vienna, Vienna, Austria pauli.miettinen@uef.fi, [email protected] Abstract analysts, and van den Broeck and Darwiche [2013] to lifted inference community, to name but a few examples. Even more The goal of Boolean Matrix Factorization (BMF) is recently, Ravanbakhsh et al. [2016] studied the problem in to approximate a given binary matrix as the product the framework of machine learning, increasing the interest of of two low-rank binary factor matrices, where the that community, and Chandran et al. [2016]; Ban et al. [2019]; product of the factor matrices is computed under Fomin et al. [2019] (re-)launched the interest in the problem the Boolean algebra. While the problem is computa- in the theory community. tionally hard, it is also attractive because the binary The various fields in which BMF is used are connected by nature of the factor matrices makes them highly in- the desire to “keep the data binary”. This can be because terpretable. In the last decade, BMF has received a the factorization is used as a pre-processing step, and the considerable amount of attention in the data mining subsequent methods require binary input, or because binary and formal concept analysis communities and, more matrices are more interpretable in the application domain. In recently, the machine learning and the theory com- the latter case, Boolean algebra is often preferred over the field munities also started studying BMF.
    [Show full text]
  • A Complete Bibliography of Publications in Linear Algebra and Its Applications: 2010–2019
    A Complete Bibliography of Publications in Linear Algebra and its Applications: 2010{2019 Nelson H. F. Beebe University of Utah Department of Mathematics, 110 LCB 155 S 1400 E RM 233 Salt Lake City, UT 84112-0090 USA Tel: +1 801 581 5254 FAX: +1 801 581 4148 E-mail: [email protected], [email protected], [email protected] (Internet) WWW URL: http://www.math.utah.edu/~beebe/ 12 March 2021 Version 1.74 Title word cross-reference KY14, Rim12, Rud12, YHH12, vdH14]. 24 [KAAK11]. 2n − 3[BCS10,ˇ Hil13]. 2 × 2 [CGRVC13, CGSCZ10, CM14, DW11, DMS10, JK11, KJK13, MSvW12, Yan14]. (−1; 1) [AAFG12].´ (0; 1) 2 × 2 × 2 [Ber13b]. 3 [BBS12b, NP10, Ghe14a]. (2; 2; 0) [BZWL13, Bre14, CILL12, CKAC14, Fri12, [CI13, PH12]. (A; B) [PP13b]. (α, β) GOvdD14, GX12a, Kal13b, KK14, YHH12]. [HW11, HZM10]. (C; λ; µ)[dMR12].(`; m) p 3n2 − 2 2n3=2 − 3n [MR13]. [DFG10]. (H; m)[BOZ10].(κ, τ) p 3n2 − 2 2n3=23n [MR14a]. 3 × 3 [CSZ10, CR10c]. (λ, 2) [BBS12b]. (m; s; 0) [Dru14, GLZ14, Sev14]. 3 × 3 × 2 [Ber13b]. [GH13b]. (n − 3) [CGO10]. (n − 3; 2; 1) 3 × 3 × 3 [BH13b]. 4 [Ban13a, BDK11, [CCGR13]. (!) [CL12a]. (P; R)[KNS14]. BZ12b, CK13a, FP14, NSW13, Nor14]. 4 × 4 (R; S )[Tre12].−1 [LZG14]. 0 [AKZ13, σ [CJR11]. 5 Ano12-30, CGGS13, DLMZ14, Wu10a]. 1 [BH13b, CHY12, KRH14, Kol13, MW14a]. [Ano12-30, AHL+14, CGGS13, GM14, 5 × 5 [BAD09, DA10, Hil12a, Spe11]. 5 × n Kal13b, LM12, Wu10a]. 1=n [CNPP12]. [CJR11]. 6 1 <t<2 [Seo14]. 2 [AIS14, AM14, AKA13, [DK13c, DK11, DK12a, DK13b, Kar11a].
    [Show full text]
  • COMPUTING RELATIVELY LARGE ALGEBRAIC STRUCTURES by AUTOMATED THEORY EXPLORATION By
    COMPUTING RELATIVELY LARGE ALGEBRAIC STRUCTURES BY AUTOMATED THEORY EXPLORATION by QURATUL-AIN MAHESAR A thesis submitted to The University of Birmingham for the degree of DOCTOR OF PHILOSOPHY School of Computer Science College of Engineering and Physical Sciences The University of Birmingham March 2014 University of Birmingham Research Archive e-theses repository This unpublished thesis/dissertation is copyright of the author and/or third parties. The intellectual property rights of the author or third parties in respect of this work are as defined by The Copyright Designs and Patents Act 1988 or as modified by any successor legislation. Any use made of information contained in this thesis/dissertation must be in accordance with that legislation and must be properly acknowledged. Further distribution or reproduction in any format is prohibited without the permission of the copyright holder. Abstract Automated reasoning technology provides means for inference in a formal context via a multitude of disparate reasoning techniques. Combining different techniques not only increases the effectiveness of single systems but also provides a more powerful approach to solving hard problems. Consequently combined reasoning systems have been successfully employed to solve non-trivial mathematical problems in combinatorially rich domains that are intractable by traditional mathematical means. Nevertheless, the lack of domain specific knowledge often limits the effectiveness of these systems. In this thesis we investigate how the combination of diverse reasoning techniques can be employed to pre-compute additional knowledge to enable mathematical discovery in finite and potentially infinite domains that is otherwise not feasible. In particular, we demonstrate how we can exploit bespoke symbolic computations and automated theorem proving to automatically compute and evolve the structural knowledge of small size finite structures in the algebraic theory of quasigroups.
    [Show full text]
  • View This Volume's Front and Back Matter
    Combinatorics o f Nonnegative Matrice s This page intentionally left blank 10.1090/mmono/213 Translations o f MATHEMATICAL MONOGRAPHS Volume 21 3 Combinatorics o f Nonnegative Matrice s V. N. Sachko v V. E. Tarakano v Translated b y Valentin F . Kolchl n | yj | America n Mathematica l Societ y Providence, Rhod e Islan d EDITORIAL COMMITTE E AMS Subcommitte e Robert D . MacPherso n Grigorii A . Marguli s James D . Stashef f (Chair ) ASL Subcommitte e Steffe n Lemp p (Chair ) IMS Subcommitte e Mar k I . Freidli n (Chair ) B. H . Ca^KOB , B . E . TapaKaHO B KOMBMHATOPMKA HEOTPMUATEJIBHbl X MATPM U Hay^Hoe 143/iaTeJibCTB O TBE[ , MocKBa , 200 0 Translated fro m th e Russia n b y Dr . Valenti n F . Kolchi n 2000 Mathematics Subject Classification. Primar y 05-02 ; Secondary 05C50 , 15-02 , 15A48 , 93-02. Library o f Congress Cataloging-in-Publicatio n Dat a Sachkov, Vladimi r Nikolaevich . [Kombinatorika neotritsatel'nyk h matrits . English ] Combinatorics o f nonnegativ e matrice s / V . N . Sachkov , V . E . Tarakano v ; translate d b y Valentin F . Kolchin . p. cm . — (Translations o f mathematical monographs , ISS N 0065-928 2 ; v. 213) Includes bibliographica l reference s an d index. ISBN 0-8218-2788- X (acid-fre e paper ) 1. Non-negative matrices. 2 . Combinatorial analysis . I . Tarakanov, V . E. (Valerii Evgen'evich ) II. Title . III . Series. QA188.S1913 200 2 512.9'434—dc21 200207439 2 Copying an d reprinting . Individua l reader s o f thi s publication , an d nonprofi t librarie s acting fo r them, ar e permitted t o make fai r us e of the material, suc h a s to copy a chapter fo r use in teachin g o r research .
    [Show full text]
  • Additional Details and Fortification for Chapter 1
    APPENDIX A Additional Details and Fortification for Chapter 1 A.1 Matrix Classes and Special Matrices The matrices can be grouped into several classes based on their operational prop- erties. A short list of various classes of matrices is given in Tables A.1 and A.2. Some of these have already been described earlier, for example, elementary, sym- metric, hermitian, othogonal, unitary, positive definite/semidefinite, negative defi- nite/semidefinite, real, imaginary, and reducible/irreducible. Some of the matrix classes are defined based on the existence of associated matri- ces. For instance, A is a diagonalizable matrix if there exists nonsingular matrices T such that TAT −1 = D results in a diagonal matrix D. Connected with diagonalizable matrices are normal matrices. A matrix B is a normal matrix if BB∗ = B∗B.Normal matrices are guaranteed to be diagonalizable matrices. However, defective matri- ces are not diagonalizable. Once a matrix has been identified to be diagonalizable, then the following fact can be used for easier computation of integral powers of the matrix: A = T −1DT → Ak = T −1DT T −1DT ··· T −1DT = T −1DkT and then take advantage of the fact that ⎛ ⎞ dk 0 ⎜ 1 ⎟ K . D = ⎝ .. ⎠ K 0 dN Another set of related classes of matrices are the idempotent, projection, invo- lutory, nilpotent, and convergent matrices. These classes are based on the results of integral powers. Matrix A is idempotent if A2 = A, and if, in addition, A is hermitian, then A is known as a projection matrix. Projection matrices are used to partition an N-dimensional space into two subspaces that are orthogonal to each other.
    [Show full text]
  • A Absolute Uncertainty Or Error, 414 Absorbing Markov Chains, 700
    Index Bauer–Fike bound, 528 A beads on a string, 559 Bellman, Richard, xii absolute uncertainty or error, 414 Beltrami, Eugenio, 411 absorbing Markov chains, 700 Benzi, Michele, xii absorbing states, 700 Bernoulli, Daniel, 299 addition, properties of, 82 Bessel, Friedrich W., 305 additive identity, 82 Bessel’s inequality, 305 additive inverse, 82 best approximate inverse, 428 adjacency matrix, 100 biased estimator, 446 adjoint, 84, 479 binary representation, 372 adjugate, 479 Birkhoff, Garrett, 625 affine functions, 89 Birkhoff’s theorem, 702 affine space, 436 bit reversal, 372 algebraic group, 402 bit reversing permutation matrix, 381 algebraic multiplicity, 496, 510 block diagonal, 261–263 amplitude, 362 rank of, 137 Anderson-Duffin formula, 441 using real arithmetic, 524 Anderson, Jean, xii block matrices, 111 Anderson, W. N., Jr., 441 determinant of, 475 angle, 295 linear operators, 392 canonical, 455 block matrix multiplication, 111 between complementary spaces, 389, 450 block triangular, 112, 261–263 maximal, 455 determinants, 467 principal, 456 eigenvalues of, 501 between subspaces, 450 Bolzano–Weierstrass theorem, 670 annihilating polynomials, 642 Boolean matrix, 679 aperiodic Markov chain, 694 bordered matrix, 485 Arnoldi’s algorithm, 653 eigenvalues of, 552 Arnoldi, Walter Edwin, 653 branch, 73, 204 associative property Brauer, Alfred, 497 of addition, 82 Bunyakovskii, Victor, 271 of matrix multiplication, 105 of scalar multiplication, 83 C asymptotic rate of convergence, 621 augmented matrix, 7 cancellation law, 97 Autonne, L., 411 canonical angles, 455 canonical form, reducible matrix, 695 B Cantor, Georg, 597 Cauchy, Augustin-Louis, 271 back substitution, 6, 9 determinant formula, 484 backward error analysis, 26 integral formula, 611 backward triangle inequality, 273 Cauchy–Bunyakovskii–Schwarz inequality, 272 band matrix, 157 Cauchy–Goursat theorem, 615 Bartlett, M.
    [Show full text]
  • Matrix Multiplication Based Graph Algorithms
    Matrix Multiplication and Graph Algorithms Uri Zwick Tel Aviv University February 2015 Last updated: June 10, 2015 Short introduction to Fast matrix multiplication Algebraic Matrix Multiplication j i Aa ()ij Bb ()ij = Cc ()ij Can be computed naively in O(n3) time. Matrix multiplication algorithms Complexity Authors n3 — n2.81 Strassen (1969) … n2.38 Coppersmith-Winograd (1990) Conjecture/Open problem: n2+o(1) ??? Matrix multiplication algorithms - Recent developments Complexity Authors n2.376 Coppersmith-Winograd (1990) n2.374 Stothers (2010) n2.3729 Williams (2011) n2.37287 Le Gall (2014) Conjecture/Open problem: n2+o(1) ??? Multiplying 22 matrices 8 multiplications 4 additions Works over any ring! Multiplying nn matrices 8 multiplications 4 additions T(n) = 8 T(n/2) + O(n2) lg8 3 T(n) = O(n )=O(n ) ( lgn = log2n ) “Master method” for recurrences 푛 푇 푛 = 푎 푇 + 푓 푛 , 푎 ≥ 1 , 푏 > 1 푏 푓 푛 = O(푛log푏 푎−휀) 푇 푛 = Θ(푛log푏 푎) 푓 푛 = O(푛log푏 푎) 푇 푛 = Θ(푛log푏 푎 log 푛) 푓 푛 = O(푛log푏 푎+휀) 푛 푇 푛 = Θ(푓(푛)) 푎푓 ≤ 푐푛 , 푐 < 1 푏 [CLRS 3rd Ed., p. 94] Strassen’s 22 algorithm CABAB 11 11 11 12 21 MAABB1( 11 Subtraction! 22 )( 11 22 ) CABAB12 11 12 12 22 MAAB2() 21 22 11 CABAB21 21 11 22 21 MABB3 11() 12 22 CABAB22 21 12 22 22 MABB4 22() 21 11 MAAB5() 11 12 22 CMMMM 11 1 4 5 7 MAABB6 ( 21 11 )( 11 12 ) CMM 12 3 5 MAABB7( 12 22 )( 21 22 ) CMM 21 2 4 7 multiplications CMMMM 22 1 2 3 6 18 additions/subtractions Works over any ring! (Does not assume that multiplication is commutative) Strassen’s nn algorithm View each nn matrix as a 22 matrix whose elements
    [Show full text]
  • Sign Pattern Matrix (Or Sign Pattern, Or Pattern)
    An Introduction to Sign Pattern Matrices and Some Connections Between Boolean and Nonnegative Sign Patterns Frank J. Hall Department of Mathematics and Statistics Georgia State University Atlanta, GA 30303 U.S.A. E-mail: [email protected] 1 1. Introduction The origins of sign pattern matrices are in the 1947 book Foundations of Economic Analysis by the No- bel Economics Prize winner P. Samuelson, who pointed to the need to solve certain problems in economics and other areas based only on the signs of the entries of the matrices. (the exact values of the entries of the matrices may not always be known) The study of sign pattern matrices has become somewhat synonymous with qualitative matrix analysis. Because of the interplay between sign pattern matrices and graph theory, the study of sign patterns is regarded as a part of combinatorial matrix theory. 2 The 1987 dissertation of C. Eschenbach, directed by C.R. Johnson, studied sign patterns that “require” or “al- low” certain properties and summarized the work on sign patterns up to that point. In 1995, Richard Brualdi and Bryan Shader produced a thorough treatment Matrices of Sign-Solvable Linear Systems on sign pattern matrices from the sign-solvability vantage point. Since 1995 there has been a considerable number of pa- pers on sign patterns and some generalized notions such as ray patterns. Handbook of Linear Algebra, 2007, CRC Press, Chap- ter 33 Sign Pattern Matrices (Hall/Li) 3 A matrix whose entries are from the set {+, −, 0} is called a sign pattern matrix (or sign pattern, or pattern).
    [Show full text]
  • C-SALT: Mining Class-Specific Alterations in Boolean Matrix
    C-SALT: Mining Class-Specific ALTerations in Boolean Matrix Factorization Sibylle Hess and Katharina Morik TU Dortmund University, Computer Science 8, Dortmund, Germany fsibylle.hess,[email protected], http://www-ai.cs.uni-dortmund.de/PERSONAL/fmorik,hessg.html Abstract. Given labeled data represented by a binary matrix, we con- sider the task to derive a Boolean matrix factorization which identifies commonalities and specifications among the classes. While existing works focus on rank-one factorizations which are either specific or common to the classes, we derive class-specific alterations from common factoriza- tions as well. Therewith, we broaden the applicability of our new method to datasets whose class-dependencies have a more complex structure. On the basis of synthetic and real-world datasets, we show on the one hand that our method is able to filter structure which corresponds to our model assumption, and on the other hand that our model assumption is justified in real-world application. Our method is parameter-free. Keywords: Boolean matrix factorization, shared subspace learning, non- convex optimization, proximal alternating linearized optimization 1 Introduction When given labeled data, a natural instinct for a data miner is to build a dis- criminative model that predicts the correct class. Yet in this paper we put the focus on the characterization of the data with respect to the label, i.e., finding similarities and differences between chunks of data belonging to miscellaneous classes. Consider a binary matrix where each row is assigned to one class. Such data emerge from fields such as gene expression analysis, e.g., a row reflects the genetic information of a cell, assigned to one tissue type (primary/relapse/no tumor), market basket analysis, e.g., a row indicates purchased items at the as- signed store, or from text analyses, e.g., a row corresponds to a document/article arXiv:1906.09907v1 [cs.LG] 17 Jun 2019 and the class denotes the publishing platform.
    [Show full text]
  • Engineering Boolean Matrix Multiplication for Multiple-Accelerator Shared-Memory Architectures
    Engineering Boolean Matrix Multiplication for Multiple-Accelerator Shared-Memory Architectures MATTI KARPPA, Aalto University, Finland PETTERI KASKI, Aalto University, Finland We study the problem of multiplying two bit matrices with entries either over the Boolean algebra ¹0; 1; _; ^º or over the binary field ¹0; 1; +; ·º. We engineer high-performance open-source algorithm implementations for contemporary multiple-accelerator shared-memory architectures, with the objective of time-and-energy-efficient scaling up to input sizes close to the available shared memory capacity. For example, given two terabinary-bit square matrices as input, our implementations compute the Boolean product in approximately 2100 seconds (1.0 Pbop/s at 3.3 pJ/bop for a total of 2.1 kWh/product) and the binary product in less than 950 seconds (2.4 effective Pbop/s at 1.5 effective pJ/bop for a total of 0.92 kWh/product) on an NVIDIA DGX-1 with power consumption atpeak system power (3.5 kW). Our contributions are (a) for the binary product, we use alternative-basis techniques of Karstadt and Schwartz [SPAA ’17] to design novel alternative-basis variants of Strassen’s recurrence for 2 × 2 block multiplication [Numer. Math. 13 (1969)] that have been optimized for both the number of additions and low working memory, (b) structuring the parallel block recurrences and the memory layout for coalescent and register-localized execution on accelerator hardware, (c) low-level engineering of the innermost block products for the specific target hardware, and (d) structuring the top-level shared-memory implementation to feed the accelerators with data and integrate the results for input and output sizes beyond the aggregate memory capacity of the available accelerators.
    [Show full text]