DOCSLIB.ORG

Arxiv:2006.08391V2 [Cs.LG] 7 Nov 2020 Applications

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Arxiv:2006.08391V2 [Cs.LG] 7 Nov 2020 Applications On Lipschitz Regularization of Convolutional Layers using Toeplitz Matrix Theory Alexandre Araujo, Benjamin Negrevergne, Yann Chevaleyre, Jamal Atif PSL, Universite´ Paris-Dauphine, CNRS, LAMSADE, MILES Team, Paris, France Abstract (2000) used to approximate the maximum singular value of a linear function. Although generic and accurate, this technique This paper tackles the problem of Lipschitz regularization of Convolutional Neural Networks. Lipschitz regularity is now is also computationally expensive, which impedes its usage established as a key property of modern deep learning with in large training settings. implications in training stability, generalization, robustness In this paper we introduce a new upper bound on the largest against adversarial examples, etc. However, computing the singular value of convolution layers that is both tight and easy exact value of the Lipschitz constant of a neural network is to compute. Instead of using the power method to iteratively known to be NP-hard. Recent attempts from the literature in- approximate this value, we rely on Toeplitz matrix theory troduce upper bounds to approximate this constant that are and its links with Fourier analysis. Our work is based on either efficient but loose or accurate but computationally ex- the result (Gray et al. 2006) that an upper bounded on the pensive. In this work, by leveraging the theory of Toeplitz singular value of Toeplitz matrices can be computed from matrices, we introduce a new upper bound for convolutional layers that is both tight and easy to compute. Based on this the inverse Fourier transform of the characteristic sequence result we devise an algorithm to train Lipschitz regularized of these matrices. We first extend this result to doubly-block Convolutional Neural Networks. Toeplitz matrices (i.e., block Toeplitz matrices where each block is Toeplitz) and then to convolutional operators, which can be represented as stacked sequences of doubly-block 1 Introduction Toeplitz matrices. From our analysis immediately follows The last few years have witnessed a growing interest in Lip- an algorithm for bounding the Lipschitz constant of a con- schitz regularization of neural networks, with the aim of volutional layer, and by extension the Lipschitz constant of improving their generalization (Bartlett, Foster, and Telgar- the whole network. We theoretically study the approximation sky 2017), their robustness to adversarial attacks (Tsuzuku, of this algorithm and show experimentally that it is more Sato, and Sugiyama 2018; Farnia, Zhang, and Tse 2019), or efficient and accurate than competing approaches. their generation abilities (e.g for GANs: Miyato et al. 2018; Finally, we illustrate our approach on adversarial robust- Arjovsky, Chintala, and Bottou 2017). Unfortunately com- ness. Recent work has shown that empirical methods such as puting the exact Lipschitz constant of a neural network is adversarial training (AT) offer poor generalization (Schmidt NP-hard (Virmaux and Scaman 2018) and in practice, exist- et al. 2018), and can be improved by applying Lipschitz reg- ing techniques such as Virmaux and Scaman (2018); Fazlyab ularization (Farnia, Zhang, and Tse 2019). To illustrate the et al. (2019) or Latorre, Rolland, and Cevher (2020) are benefit of our new method, we train a large, state-of-the-art difficult to implement for neural networks with more than Wide ResNet architecture with Lipschitz regularization and one or two layers, which hinders their use in deep learning show that it offers a significant improvement over adver- arXiv:2006.08391v2 [cs.LG] 7 Nov 2020 applications. sarial training alone, and over other methods for Lipschitz To overcome this difficulty, most of the work has focused regularization. In summary, we make the three following on computing the Lipschitz constant of individual layers in- contributions: stead. The product of the Lipschitz constant of each layer is an upper-bound for the Lipschitz constant of the entire net- 1. We devise an upper bound on the singular values of the op- work, and it can be used as a surrogate to perform Lipschitz erator matrix of convolutional layers by leveraging Toeplitz regularization. Since most common activation functions (such matrix theory and its links with Fourier analysis. as ReLU) have a Lipschitz constant equal to one, the main bottleneck is to compute the Lipschitz constant of the under- 2. We propose an efficient algorithm to compute this upper lying linear application which is equal to its maximal singular bound which enables its use in the context of Convolutional value. The work in this line of research mainly relies on the Neural Networks. celebrated iterative algorithm by Golub and Van der Vorst 3. We use our method to regularize the Lipschitz constant of Copyright © 2021, Association for the Advancement of Artificial neural networks for adversarial robustness and show that Intelligence (www.aaai.org). All rights reserved. it offers a significant improvement over AT alone. 2 Related Work works are theoretically interesting but lack scalability (i.e., A popular technique for approximating the maximal singular the bound can only be computed on small networks). value of a matrix is the power method (Golub and Van der Finally, in parallel to the development of the results in Vorst 2000), an iterative algorithm which yields a good ap- this paper, we discovered that Yi (2020) have studied the proximation of the maximum singular value when the algo- asymptotic distribution of the singular values of convolutional rithm is able to run for a sufficient number of iterations. layers by using a related approach. However, this author Yoshida and Miyato (2017); Miyato et al. (2018) have does not investigate the robustness applications of Lipschitz used the power method to normalize the spectral norm of regularization. each layer of a neural network, and showed that the resulting models offered improved generalization performance and 3 A Primer on Toeplitz and block Toeplitz generated better examples when they were used in the con- matrices text of GANs. Farnia, Zhang, and Tse 2019 built upon the In order to devise a bound on the Lipschitz constant of a work of Miyato et al. (2018) and proposed a power method convolution layer as used by the Deep Learning community, specific for convolutional layers that leverages the deconvo- we study the properties of doubly-block Toeplitz matrices. lution operation and avoid the computation of the gradient. In this section, we first introduce the necessary background They used it in combination with adversarial training. In on Toeplitz and block Toeplitz matrices, and introduce a new the same vein, Gouk et al. (2018) demonstrated that regular- result on doubly-block Toeplitz matrices. ized neural networks using the power method also offered Toeplitz matrices and block Toeplitz matrices are well- improvements over their non-regularized counterparts. Fur- known types of structured matrices. A Toeplitz matrix (re- thermore, Tsuzuku, Sato, and Sugiyama (2018) have shown spectively a block Toeplitz matrix) is a matrix in which each that a neural network can be more robust to some adversarial scalar (respectively block) is repeated identically along diag- attacks, if the prediction margin of the network (i.e., the dif- onals. ference between the first and the second maximum logit) is n × n A higher than a minimum threshold that depends on the global An Toeplitz matrix is fully determined by a two- fa g nm × nm Lipschitz constant of the network. Building on this observa- sided sequence of scalars: h h2N , whereas an B tion, they use the power method to compute an upper bound block Toeplitz matrix is fully determined by a two-sided fB g N = {−n+1; : : : ; n− on the global Lipschitz constant, and maximize the prediction sequence of blocks h h2N , where 1g B m × m margin during training. Finally, Virmaux and Scaman (2018) and where each block h is an matrix. have used automatic differentiation combined with the power method to compute a tighter bound on the global Lipschitz 0 a0 a1 ··· an−1 1 0 B0 B1 ··· Bn−1 1 : : : : : : : constant of neural networks. Despite a number of interesting B a−1 a0 : : C B B−1 B0 : : C results, using the power method is expensive and results in A = B : : C B = B : : C: B : : : C B : : : C prohibitive training times. @ : a0 a1 A @ : B0 B1 A a ··· a a Other approaches to regularize the Lipschitz constant of −n+1 −1 0 B−n+1 ··· B−1 B0 neural networks have been proposed by Sedghi, Gupta, and Long (2019) and Singla and Feizi (2019). The method Finally, a doubly-block Toeplitz matrix is a block of Sedghi, Gupta, and Long (2019) exploits the properties of Toeplitz matrix in which each block is itself a Toeplitz ma- circulant matrices to approximate the maximal singular value trix. In the remainder, we will use the standard notation of a convolutional layer. Although interesting, this method ( · )i;j2f0;:::;n−1g to construct (block) matrices. For example, results in a loose approximation of the maximal singular A = (aj−i)i;j2f0;:::;n−1g and B = (Bj−i)i;j2f0;:::;n−1g. value of a convolutional layer. Furthermore, the complex- ity of their algorithm is dependent on the convolution input 3.1 Bound on the singular value of Toeplitz and which can be high for large datasets such as ImageNet. More block Toeplitz matrices recently, Singla and Feizi (2019) have successfully bounded A standard tool for manipulating (block) Toeplitz matri- the operator norm of the Jacobian matrix of a convolution ces is the use of Fourier analysis. Let fahgh2N be the se- layer by the Frobenius norm of the reshaped kernel. This quence of coefficients of the Toeplitz matrix A 2 n×n technique has the advantage to be very fast to compute and to R and let fBhgh2N be the sequence of m × m blocks of be independent of the input size but it also results in a loose the block Toeplitz matrix B.
Load more

Recommended publications
  • Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A
    1 Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A. Deri, Member, IEEE, and José M. F. Moura, Fellow, IEEE Abstract—We define and discuss the utility of two equiv- Consider a graph G = G(A) with adjacency matrix alence graph classes over which a spectral projector-based A 2 CN×N with k ≤ N distinct eigenvalues and Jordan graph Fourier transform is equivalent: isomorphic equiv- decomposition A = VJV −1. The associated Jordan alence classes and Jordan equivalence classes. Isomorphic equivalence classes show that the transform is equivalent subspaces of A are Jij, i = 1; : : : k, j = 1; : : : ; gi, up to a permutation on the node labels. Jordan equivalence where gi is the geometric multiplicity of eigenvalue 휆i, classes permit identical transforms over graphs of noniden- or the dimension of the kernel of A − 휆iI. The signal tical topologies and allow a basis-invariant characterization space S can be uniquely decomposed by the Jordan of total variation orderings of the spectral components. subspaces (see [13], [14] and Section II). For a graph Methods to exploit these classes to reduce computation time of the transform as well as limitations are discussed. signal s 2 S, the graph Fourier transform (GFT) of [12] is defined as Index Terms—Jordan decomposition, generalized k gi eigenspaces, directed graphs, graph equivalence classes, M M graph isomorphism, signal processing on graphs, networks F : S! Jij i=1 j=1 s ! (s ;:::; s ;:::; s ;:::; s ) ; (1) b11 b1g1 bk1 bkgk I. INTRODUCTION where sij is the (oblique) projection of s onto the Jordan subspace Jij parallel to SnJij.
    [Show full text]
  • Polynomial Sequences Generated by Linear Recurrences
    Innocent Ndikubwayo Polynomial Sequences Generated by Linear Recurrences: Location and Reality of Zeros Polynomial Sequences Generated by Linear Recurrences: Location and Reality of Zeros Linear Recurrences: Location by Sequences Generated Polynomial Innocent Ndikubwayo ISBN 978-91-7911-462-6 Department of Mathematics Doctoral Thesis in Mathematics at Stockholm University, Sweden 2021 Polynomial Sequences Generated by Linear Recurrences: Location and Reality of Zeros Innocent Ndikubwayo Academic dissertation for the Degree of Doctor of Philosophy in Mathematics at Stockholm University to be publicly defended on Friday 14 May 2021 at 15.00 in sal 14 (Gradängsalen), hus 5, Kräftriket, Roslagsvägen 101 and online via Zoom, public link is available at the department website. Abstract In this thesis, we study the problem of location of the zeros of individual polynomials in sequences of polynomials generated by linear recurrence relations. In paper I, we establish the necessary and sufficient conditions that guarantee hyperbolicity of all the polynomials generated by a three-term recurrence of length 2, whose coefficients are arbitrary real polynomials. These zeros are dense on the real intervals of an explicitly defined real semialgebraic curve. Paper II extends Paper I to three-term recurrences of length greater than 2. We prove that there always exist non- hyperbolic polynomial(s) in the generated sequence. We further show that with at most finitely many known exceptions, all the zeros of all the polynomials generated by the recurrence lie and are dense on an explicitly defined real semialgebraic curve which consists of real intervals and non-real segments. The boundary points of this curve form a subset of zero locus of the discriminant of the characteristic polynomial of the recurrence.
    [Show full text]
  • Explicit Inverse of a Tridiagonal (P, R)–Toeplitz Matrix
    Explicit inverse of a tridiagonal (p; r){Toeplitz matrix A.M. Encinas, M.J. Jim´enez Departament de Matemtiques Universitat Politcnica de Catalunya Abstract Tridiagonal matrices appears in many contexts in pure and applied mathematics, so the study of the inverse of these matrices becomes of specific interest. In recent years the invertibility of nonsingular tridiagonal matrices has been quite investigated in different fields, not only from the theoretical point of view (either in the framework of linear algebra or in the ambit of numerical analysis), but also due to applications, for instance in the study of sound propagation problems or certain quantum oscillators. However, explicit inverses are known only in a few cases, in particular when the tridiagonal matrix has constant diagonals or the coefficients of these diagonals are subjected to some restrictions like the tridiagonal p{Toeplitz matrices [7], such that their three diagonals are formed by p{periodic sequences. The recent formulae for the inversion of tridiagonal p{Toeplitz matrices are based, more o less directly, on the solution of second order linear difference equations, although most of them use a cumbersome formulation, that in fact don not take into account the periodicity of the coefficients. This contribution presents the explicit inverse of a tridiagonal matrix (p; r){Toeplitz, which diagonal coefficients are in a more general class of sequences than periodic ones, that we have called quasi{periodic sequences. A tridiagonal matrix A = (aij) of order n + 2 is called (p; r){Toeplitz if there exists m 2 N0 such that n + 2 = mp and ai+p;j+p = raij; i; j = 0;:::; (m − 1)p: Equivalently, A is a (p; r){Toeplitz matrix iff k ai+kp;j+kp = r aij; i; j = 0; : : : ; p; k = 0; : : : ; m − 1: We have developed a technique that reduces any linear second order difference equation with periodic or quasi-periodic coefficients to a difference equation of the same kind but with constant coefficients [3].
    [Show full text]
  • Toeplitz and Toeplitz-Block-Toeplitz Matrices and Their Correlation with Syzygies of Polynomials
    TOEPLITZ AND TOEPLITZ-BLOCK-TOEPLITZ MATRICES AND THEIR CORRELATION WITH SYZYGIES OF POLYNOMIALS HOUSSAM KHALIL∗, BERNARD MOURRAIN† , AND MICHELLE SCHATZMAN‡ Abstract. In this paper, we re-investigate the resolution of Toeplitz systems T u = g, from a new point of view, by correlating the solution of such problems with syzygies of polynomials or moving lines. We show an explicit connection between the generators of a Toeplitz matrix and the generators of the corresponding module of syzygies. We show that this module is generated by two elements of degree n and the solution of T u = g can be reinterpreted as the remainder of an explicit vector depending on g, by these two generators. This approach extends naturally to multivariate problems and we describe for Toeplitz-block-Toeplitz matrices, the structure of the corresponding generators. Key words. Toeplitz matrix, rational interpolation, syzygie 1. Introduction. Structured matrices appear in various domains, such as scientific computing, signal processing, . They usually express, in a linearize way, a problem which depends on less pa- rameters than the number of entries of the corresponding matrix. An important area of research is devoted to the development of methods for the treatment of such matrices, which depend on the actual parameters involved in these matrices. Among well-known structured matrices, Toeplitz and Hankel structures have been intensively studied [5, 6]. Nearly optimal algorithms are known for the multiplication or the resolution of linear systems, for such structure. Namely, if A is a Toeplitz matrix of size n, multiplying it by a vector or solving a linear system with A requires O˜(n) arithmetic operations (where O˜(n) = O(n logc(n)) for some c > 0) [2, 12].
    [Show full text]
  • A Note on Multilevel Toeplitz Matrices
    Spec. Matrices 2019; 7:114–126 Research Article Open Access Lei Cao and Selcuk Koyuncu* A note on multilevel Toeplitz matrices https://doi.org/110.1515/spma-2019-0011 Received August 7, 2019; accepted September 12, 2019 Abstract: Chien, Liu, Nakazato and Tam proved that all n × n classical Toeplitz matrices (one-level Toeplitz matrices) are unitarily similar to complex symmetric matrices via two types of unitary matrices and the type of the unitary matrices only depends on the parity of n. In this paper we extend their result to multilevel Toeplitz matrices that any multilevel Toeplitz matrix is unitarily similar to a complex symmetric matrix. We provide a method to construct the unitary matrices that uniformly turn any multilevel Toeplitz matrix to a complex symmetric matrix by taking tensor products of these two types of unitary matrices for one-level Toeplitz matrices according to the parity of each level of the multilevel Toeplitz matrices. In addition, we introduce a class of complex symmetric matrices that are unitarily similar to some p-level Toeplitz matrices. Keywords: Multilevel Toeplitz matrix; Unitary similarity; Complex symmetric matrices 1 Introduction Although every complex square matrix is similar to a complex symmetric matrix (see Theorem 4.4.24, [5]), it is known that not every n × n matrix is unitarily similar to a complex symmetric matrix when n ≥ 3 (See [4]). Some characterizations of matrices unitarily equivalent to a complex symmetric matrix (UECSM) were given by [1] and [3]. Very recently, a constructive proof that every Toeplitz matrix is unitarily similar to a complex symmetric matrix was given in [2] in which the unitary matrices turning all n × n Toeplitz matrices to complex symmetric matrices was given explicitly.
    [Show full text]
  • Totally Positive Toeplitz Matrices and Quantum Cohomology of Partial Flag Varieties
    JOURNAL OF THE AMERICAN MATHEMATICAL SOCIETY Volume 16, Number 2, Pages 363{392 S 0894-0347(02)00412-5 Article electronically published on November 29, 2002 TOTALLY POSITIVE TOEPLITZ MATRICES AND QUANTUM COHOMOLOGY OF PARTIAL FLAG VARIETIES KONSTANZE RIETSCH 1. Introduction A matrix is called totally nonnegative if all of its minors are nonnegative. Totally nonnegative infinite Toeplitz matrices were studied first in the 1950's. They are characterized in the following theorem conjectured by Schoenberg and proved by Edrei. Theorem 1.1 ([10]). The Toeplitz matrix ∞×1 1 a1 1 0 1 a2 a1 1 B . .. C B . a2 a1 . C B C A = B .. .. .. C B ad . C B C B .. .. C Bad+1 . a1 1 C B C B . C B . .. .. a a .. C B 2 1 C B . C B .. .. .. .. ..C B C is totally nonnegative@ precisely if its generating function is of theA form, 2 (1 + βit) 1+a1t + a2t + =exp(tα) ; ··· (1 γit) i Y2N − where α R 0 and β1 β2 0,γ1 γ2 0 with βi + γi < . 2 ≥ ≥ ≥···≥ ≥ ≥···≥ 1 This beautiful result has been reproved many times; see [32]P for anP overview. It may be thought of as giving a parameterization of the totally nonnegative Toeplitz matrices by ~ N N ~ (α;(βi)i; (~γi)i) R 0 R 0 R 0 i(βi +~γi) < ; f 2 ≥ × ≥ × ≥ j 1g i X2N where β~i = βi βi+1 andγ ~i = γi γi+1. − − Received by the editors December 10, 2001 and, in revised form, September 14, 2002. 2000 Mathematics Subject Classification. Primary 20G20, 15A48, 14N35, 14N15.
    [Show full text]
  • A Note on Inversion of Toeplitz Matrices$
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Applied Mathematics Letters 20 (2007) 1189–1193 www.elsevier.com/locate/aml A note on inversion of Toeplitz matrices$ Xiao-Guang Lv, Ting-Zhu Huang∗ School of Applied Mathematics, University of Electronic Science and Technology of China, Chengdu, Sichuan, 610054, PR China Received 18 October 2006; accepted 30 October 2006 Abstract It is shown that the invertibility of a Toeplitz matrix can be determined through the solvability of two standard equations. The inverse matrix can be denoted as a sum of products of circulant matrices and upper triangular Toeplitz matrices. The stability of the inversion formula for a Toeplitz matrix is also considered. c 2007 Elsevier Ltd. All rights reserved. Keywords: Toeplitz matrix; Circulant matrix; Inversion; Algorithm 1. Introduction Let T be an n-by-n Toeplitz matrix: a0 a−1 a−2 ··· a1−n ··· a1 a0 a−1 a2−n ··· T = a2 a1 a0 a3−n , . . .. an−1 an−2 an−3 ··· a0 where a−(n−1),..., an−1 are complex numbers. We use the shorthand n T = (ap−q )p,q=1 for a Toeplitz matrix. The inversion of a Toeplitz matrix is usually not a Toeplitz matrix. A very important step is to answer the question of how to reconstruct the inversion of a Toeplitz matrix by a low number of its columns and the entries of the original Toeplitz matrix. It was first observed by Trench [1] and rediscovered by Gohberg and Semencul [2] that T −1 can be reconstructed from its first and last columns provided that the first component of the first column does not vanish.
    [Show full text]
  • TOEPLITZ MATRIX APPROXIMATION by Suliman Al-Homidan
    TOEPLITZ MATRIX APPROXIMATION by Suliman Al-Homidan Department of Mathematics, King Fahd University of Petroleum and Minerals, Dhahran 31261, PO Box 119, Saudi Arabia 1 1. Matrix nearness problems 2. Hybrid Methods for Finding the Nearest Euclidean Distance Matrix 3. Educational Testing Problem 4. Hybrid Methods for Minimizing Least Distance Functions with Semi-Definite Matrix Constraints 5. The Problem 6. l1Sequential Quadratic Programming Method 2 1 Matrix nearness problems Given a matrix F ∈ IRn×n then consider the problem minimize kF − Dk (kF − Dk ≤ |F |) D Such that D has property P P can be any one or mixture of: * Symmetry * Skew-Symmetry * Poitive semi-definiteness * Orthogonality, Unitary * Normality * Rank-deficiency, Singularity 3 * D in a linesr space * D with some fixed columns, rows, subma- trix * Instability * D with a given λ, repeated λ * D is Euclidean Distance Matrix * D is Toeplitz or Hankel 4 2 Hybrid Methods for Finding the Nearest Euclidean Distance Matrix Definition A matrix D ∈ IRn×n is called a Euclidean dis- tance matrix iff there exist n points x1,..., xn in an affine subspace of dimension IRm (m ≤ n − 1) such that 2 dij = kxi − xjk2 ∀i, j. (1) The Euclidean distance problem can now be stated as follows. Given a matrix F ∈ IRn×n, find the Euclidean distance matrix D ∈ IRn×n that minimizes kF − DkF (2) where k.kF denotes the Frobenius norm. see Al-Homidan and Fletcher [1] 5 3 Educational Testing Problem The educational testing problem. can be ex- pressed as maximize eT θ θ ∈ IRn subject to F − diag θ ≥ 0 θi ≥ 0 i = 1, ..., n (3) where e = (1, 1, ..., 1)T .
    [Show full text]
  • Pfaffians of Toeplitz Payoff Matrices
    Pfaffians of Toeplitz payoff matrices Ron Evans Department of Mathematics University of California at San Diego La Jolla, CA 92093-0112 [email protected] Nolan Wallach Department of Mathematics University of California at San Diego La Jolla, CA 92093-0112 [email protected] April 19, 2019 2010 Mathematics Subject Classification. 15A15, 15B05, 15B57, 91A05 Key words and phrases. Toeplitz skew-symmetric matrix, Pfaffian, nullity, matrix game, payoff matrix. Abstract The purpose of this paper is to evaluate the Pfaffians of certain Toeplitz payoff matrices associated with integer choice matrix games. 1 1 Introduction For w 2 R, let M(n; k; w) denote the n × n skew-symmetric Toeplitz matrix whose k superdiagonals in the upper right corner have all entries −w, and whose remaining n − k − 1 superdiagonals have all entries 1. For example, M(6; 3; w) is the matrix 2 0 1 1 −w −w −w 3 6 −1 0 1 1 −w −w 7 6 7 6 −1 −1 0 1 1 −w 7 6 7 : 6 w −1 −1 0 1 1 7 6 7 4 w w −1 −1 0 1 5 w w w −1 −1 0 In earlier work [4], the Pfaffians of (scalar multiples of) M(2n; 2n − 2; w) were evaluated. Our main result (Theorem 2.4) evaluates the Pfaffians of the matrices M(2n; m; w) for all n ≥ 1 with n ≥ m ≥ 0. Up to sign, these Pfaffians turn out to be the polynomials Fm(w) defined in (2.1). We remark that Fm(w) is the difference of two Chebyshev polynomials of the second kind, namely Fm(w) = Um((w + 1)=2) − Um−1((w + 1)=2): A connection between Chebyshev polynomials of the second kind and Pfaf- fians of some skew-symmetric Toeplitz band matrices may be found in [2].
    [Show full text]
  • On Some Properties of Positive Definite Toeplitz Matrices and Their Possible Applications
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector On Some Properties of Positive Definite Toeplitz Matrices and Their Possible Applications Bishwa Nath Mukhexjee Computer Science Unit lndian Statistical Institute Calcutta 700 035, Zndia and Sadhan Samar Maiti Department of Statistics Kalyani University Kalyani, Nadia, lndia Submitted by Miroslav Fiedler ABSTRACT Various properties of a real symmetric Toeplitz matrix Z,,, with elements u+ = u,~_~,, 1 Q j, k c m, are reviewed here. Matrices of this kind often arise in applica- tions in statistics, econometrics, psychometrics, structural engineering, multichannel filtering, reflection seismology, etc., and it is desirable to have techniques which exploit their special structure. Possible applications of the results related to their inverse, determinant, and eigenvalue problem are suggested. 1. INTRODUCTION Most matrix forms that arise in the analysis of stationary stochastic processes, as in time series data, are of the so-called Toeplitz structure. The Toeplitz matrix, T, is important due to its distinctive property that the entries in the matrix depend only on the differences of the indices, and as a result, the elements on its principal diagonal as well as those lying within each of its subdiagonals are identical. In 1910, 0. Toeplitz studied various forms of the matrices ((t,_,)) in relation to Laurent power series and called them Lforms, without assuming that they are symmetric. A sufficiently well-structured theory on T-matrix LINEAR ALGEBRA AND ITS APPLICATIONS 102:211-240 (1988) 211 0 Elsevier Science Publishing Co., Inc., 1988 52 Vanderbilt Ave., New York, NY 10017 00243795/88/$3.50 212 BISHWA NATH MUKHERJEE AND SADHAN SAMAR MAITI also existed in the memoirs of Frobenius [21, 221.
    [Show full text]
  • Structure of Isometry Group of Bilinear Spaces ୋ
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Linear Algebra and its Applications 416 (2006) 414–436 www.elsevier.com/locate/laa Structure of isometry group of bilinear spaces ୋ Dragomir Ž. Ðokovic´ Department of Pure Mathematics, University of Waterloo, Waterloo, Ont., Canada N2L 3G1 Received 5 November 2004; accepted 27 November 2005 Available online 18 January 2006 Submitted by H. Schneider Abstract We describe the structure of the isometry group G of a finite-dimensional bilinear space over an algebra- ically closed field of characteristic not two. If the space has no indecomposable degenerate orthogonal summands of odd dimension, it admits a canonical orthogonal decomposition into primary components and G is isomorphic to the direct product of the isometry groups of the primary components. Each of the latter groups is shown to be isomorphic to the centralizer in some classical group of a nilpotent element in the Lie algebra of that group. In the general case, the description of G is more complicated. We show that G is a semidirect product of a normal unipotent subgroup K with another subgroup which, in its turn, is a direct product of a group of the type described in the previous paragraph and another group H which we can describe explicitly. The group H has a Levi decomposition whose Levi factor is a direct product of several general linear groups of various degrees. We obtain simple formulae for the dimensions of H and K. © 2005 Elsevier Inc. All rights reserved. Keywords: Bilinear space; Isometry group; Asymmetry; Gabriel block; Toeplitz matrix 1.
    [Show full text]
  • ON EUCLIDEAN DISTANCE MATRICES of GRAPHS∗ 1. Introduction. a Matrix D ∈ R N×N Is a Euclidean Distance Matrix (EDM), If Ther
    Electronic Journal of Linear Algebra ISSN 1081-3810 A publication of the International Linear Algebra Society Volume 26, pp. 574-589, August 2013 ELA ON EUCLIDEAN DISTANCE MATRICES OF GRAPHS∗ GASPERˇ JAKLICˇ † AND JOLANDA MODIC‡ Abstract. In this paper, a relation between graph distance matrices and Euclidean distance matrices (EDM) is considered. It is proven that distance matrices of paths and cycles are EDMs. The proofs are constructive and the generating points of studied EDMs are given in a closed form. A generalization to weighted graphs (networks) is tackled. Key words. Graph, Euclidean distance matrix, Distance, Eigenvalue. AMS subject classifications. 15A18, 05C50, 05C12. n n 1. Introduction. A matrix D R × is a Euclidean distance matrix (EDM), ∈ if there exist points x Rr, i =1, 2,...,n, such that d = x x 2. The minimal i ∈ ij k i − j k possible r is called an embedding dimension (see, e.g., [3]). Euclidean distance matri- ces were introduced by Menger in 1928, later they were studied by Schoenberg [13], and other authors. They have many interesting properties, and are used in various applications in linear algebra, graph theory, and bioinformatics. A natural problem is to study configurations of points xi, where only distances between them are known. n n Definition 1.1. A matrix D = (d ) R × is hollow, if d = 0 for all ij ∈ ii i =1, 2,...,n. There are various characterizations of EDMs. n n Lemma 1.2. [8, Lemma 5.3] Let a symmetric hollow matrix D R × have only ∈ one positive eigenvalue λ and the corresponding eigenvector e := [1, 1,..., 1]T Rn.
    [Show full text]