DOCSLIB.ORG

Diffeomorphic Image Registration by Geodesic Shooting

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Diffeomorphic Image Registration by Geodesic Shooting Small Deformation Model Large-Deformation Diffeomorphic Metric Mapping Geodesic Shooting Diffeomorphic Image Registration by Geodesic Shooting John Ashburner Wellcome Trust Centre for Neuroimaging UCL Institute of Neurlogy London, UK. John Ashburner Geodesic Shooting Small Deformation Model Large-Deformation Diffeomorphic Metric Mapping Geodesic Shooting Outline 1 Small Deformation Model Differential Operator Optimisation 2 Large-Deformation Diffeomorphic Metric Mapping Diffeomorphisms & Geodesics LDDMM Itself 3 Geodesic Shooting Generating Deformations from Initial Velocities Optimising the Initial Velocities Initial Momentum Examples John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Small Deformation Approximation µ ° (x−v(x)) f ° (x+v(x)) f ° (x+v(x)) µ ° (x−v(x)) x−v(x) x+v(x) x+v(x) x−v(x) |Jx−v(x)| |Jx+v(x)| |Jx+v(x)| |Jx−v(x)| John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Small Deformation Model Should fit a model to the data (as opposed to fitting the data to the model). The objective function for least-squares fitting is often expressed as: 1 1 E = jjLvjj2 + jjf − µ (Id − v) jj2 2 2σ2 f is our image (data). µ is the template (model). v is a displacemet field. Id is an identity transform. L is a differential operator. John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Linear Elasticity Operator The differential operator used is essentially the same linear elasticity operator used by Christensen (where D indicates computing the Jacobian: Z 2 λ1 T 2 2 2 jjLvjj = jjDv + (Dv) jj + λ2tr(Dv) + λ3jjvjj dx x2Ω 4 This has three terms that penalise different things. 1 Length changes (Dv + (Dv)T ). 2 Volume changes (tr(Dv)). 3 Absolute displacement (v { needed to make the operator invertable). John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Discretisation Images and deformations are really stored discretely, and made continuous by interpolation. Generalised interpolation involves representing fields etc by linear combinations of basis functions. Linear interpolation, which involves very local triangular basis functions (bi (x)), so: X v(x) = wi bi (x) i John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Differential Operator in 1D In 1D, a simple operator could compute the gradients by Gw, where: 0−1 1 0 0 :::1 B 0 −1 1 0 :::C B C B 0 0 −1 1 :::C G = B C B 0 0 0 −1 :::C @ . A . .. Penalty (jjLvjj2) could be the sum of squares of the gradients. Computed by (Gw)T (Gw) ≡ wT GT Gw ≡ wT Aw. p Note that the norm jjLvjj (ie wT Aw) can be thought of as a distance. John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Differential Operator in 2D For more dimensions, we can use Kroneker tensor products. 0 1 g11B ::: g1nB B . .. C G ⊗ B = @ . A gm1B ::: gmnB So, computing horizontal and vertical gradients of a square image (reshaped to a column vector) may be achieved using the following matrix: I ⊗ G G ⊗ I John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Differential Operator for Computing Jacobians (2D) If we stack the x component of the displacement field on top of the y component, then we can compute elements of the Jacobians by: 0I ⊗ G 0 1 BG ⊗ I 0 C B C w @ 0 I ⊗ GA 0 G ⊗ I The divergence of the displacement is simply the trace of the Jacobian, so can be computed using: I ⊗ GG ⊗ I w John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Symmetric and Anti-symmetric Matrices A square matrix (J) can be decomposed into the sum of a symmetric and anti-symmetric (skew symmetric) matrix. The symmetric part is (J + JT )=2. The anti-symmetric part is (J − JT )=2. The symmetric elements of the Jacobian of the displacement can be obtained from: 0 I ⊗ G 0 1 @ 0 G ⊗ I A w 1 1 2 G ⊗ I 2 I ⊗ G Similar large sparse matrices can be generated for 3D data. John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Matrix Exponentials A matrix exponential (\expm" in MATLAB) is given by: 1 X 1 eJ = Jk k! k=0 For matrices very close to zero, eJ ' I + J. The determinant jeJj may be computed from etr(J). If J is anti-symmetric, then eJ is orthogonal (ie a rigid rotation). 0 0 1 01 00 0 −11 00 0 01 J = w1 @−1 0 0A + w2 @0 0 0 A + w3 @0 0 1A 0 0 0 1 0 0 0 −1 0 We do not wish to penalise rotations, so only penalise the symmetric parts of the Jacobians. John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Greens Function Note that the Green's function (K) of the operator may be computed from the (pseudo)inverse of A, although it is often easier to use Fourier methods. If you think of applying a differential operator as a convolution, then applying the Green's function is a deconvolution. John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Gauss-Newton Optimisation An iterative procedure, using both first and second derivatives. 2 −1 (i+1) (i) @ E @E w = w − 2 @w w(i) @w w(i) @E = Aw + g @w @2E = A + H @w2 John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Gradients 2 3 g11 6g217 6 7 6g317 Using trilinear interpolation to 6 7 6 . 7 represent the displacement fields, 6 . 7 6 7 where the density of the control 6gI 17 g = 6 7 points is equal to the pixel 6g117 6 7 spacing. Therefore 6g 7 6 217 #(x ) = [x − w ; x − w ]. 6g 7 i i1 i1 i2 i2 6 317 6 . 7 4 . 5 gI 1 1 g = (f (x ) − µ(#(x ))) ((r µ) ◦ (#(x ))) il σ2 i i l i John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Hessian 2 3 h111 0 0 ::: 0 h121 0 0 ::: 0 6 0 h211 0 ::: 0 0 h221 0 ::: 0 7 6 7 6 0 0 h311 ::: 0 0 0 h321 ::: 0 7 6 7 6 . .. .. 7 6 . 7 6 7 6 0 0 0 ::: hI 11 0 0 0 ::: hI 217 H = 6 7 6h112 0 0 ::: 0 h122 0 0 ::: 0 7 6 7 6 0 h 0 ::: 0 0 h 0 ::: 0 7 6 212 222 7 6 0 0 h ::: 0 0 0 h ::: 0 7 6 312 322 7 6 . .. .. 7 4 . 5 0 0 0 ::: hI 12 0 0 0 ::: hI 22 1 h = ((r µ) ◦ #(x ))((r µ) ◦ #(x )) ilm σ2 l i m i John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting Matrix Solution A A B A multigrid approach used to −1 solve (A + H) (Aw + g). Based A A B C B C B on a relaxation scheme at several resolutions. See Numerical Recipes (2nd edition onwards) for A A B C B C B C B C B more details. E E E E E E E John Ashburner Geodesic Shooting Small Deformation Model Differential Operator Large-Deformation Diffeomorphic Metric Mapping Optimisation Geodesic Shooting At The Solution At the solution, the derivatives of the objective function are zero. @E = Aw + g = 0 @w Therefore w = A−1g We see that the displacements are a smoothed version of a difference image multiplied by the warped gradients of the template. v = K ((f − µ(#))((rµ) ◦ #)) John Ashburner Geodesic Shooting Small Deformation Model Diffeomorphisms & Geodesics Large-Deformation Diffeomorphic Metric Mapping LDDMM Itself Geodesic Shooting Outline 1 Small Deformation Model Differential Operator Optimisation 2 Large-Deformation Diffeomorphic Metric Mapping Diffeomorphisms & Geodesics LDDMM Itself 3 Geodesic Shooting Generating Deformations from Initial Velocities Optimising the Initial Velocities Initial Momentum Examples John Ashburner Geodesic Shooting Small Deformation Model Diffeomorphisms & Geodesics Large-Deformation Diffeomorphic Metric Mapping LDDMM Itself Geodesic Shooting Nonlinearity of Shapes µ f According to David Mumford (Fields Medal, 1974): \Shapes are the ultimate non-linear µ ° θ f ° φ sort of thing" Relative shapes can not be added and subtracted (ie, they are nonlinear). Instead, θ φ deformations should combined by composing them together. Deformations that are smooth and invertable |Jθ| |Jφ| are known as diffeomorphisms, and form a mathematical group. John Ashburner Geodesic Shooting Small Deformation Model Diffeomorphisms & Geodesics Large-Deformation Diffeomorphic Metric Mapping LDDMM Itself Geodesic Shooting Linear Methods for Data on Manifolds We are dealing with non- Euclidean geometry, so conceptualise the curved spaces as manifolds embedded in higher dimensions. John Ashburner Geodesic Shooting Small Deformation Model Diffeomorphisms & Geodesics Large-Deformation Diffeomorphic Metric Mapping LDDMM Itself Geodesic Shooting Diffeomorphism Groups A group is a set, G, together with an operation, ◦, that combines any two elements a and b to form another element, denoted a ◦ b. To qualify as a group, the set and operation must satisfy four requirements known as the group axioms: Closure. For all a, b in G, the result of the operation, a ◦ b, is also in G. Associativity. For all a, b and c in G,(a ◦ b) ◦ c = a ◦ (b ◦ c).
Load more

Recommended publications
  • The General Linear Group
    18.704 Gabe Cunningham 2/18/05 [email protected] The General Linear Group Definition: Let F be a field. Then the general linear group GLn(F ) is the group of invert- ible n × n matrices with entries in F under matrix multiplication. It is easy to see that GLn(F ) is, in fact, a group: matrix multiplication is associative; the identity element is In, the n × n matrix with 1’s along the main diagonal and 0’s everywhere else; and the matrices are invertible by choice. It’s not immediately clear whether GLn(F ) has infinitely many elements when F does. However, such is the case. Let a ∈ F , a 6= 0. −1 Then a · In is an invertible n × n matrix with inverse a · In. In fact, the set of all such × matrices forms a subgroup of GLn(F ) that is isomorphic to F = F \{0}. It is clear that if F is a finite field, then GLn(F ) has only finitely many elements. An interesting question to ask is how many elements it has. Before addressing that question fully, let’s look at some examples. ∼ × Example 1: Let n = 1. Then GLn(Fq) = Fq , which has q − 1 elements. a b Example 2: Let n = 2; let M = ( c d ). Then for M to be invertible, it is necessary and sufficient that ad 6= bc. If a, b, c, and d are all nonzero, then we can fix a, b, and c arbitrarily, and d can be anything but a−1bc. This gives us (q − 1)3(q − 2) matrices.
    [Show full text]
  • Normal Subgroups of the General Linear Groups Over Von Neumann Regular Rings L
    PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 96, Number 2, February 1986 NORMAL SUBGROUPS OF THE GENERAL LINEAR GROUPS OVER VON NEUMANN REGULAR RINGS L. N. VASERSTEIN1 ABSTRACT. Let A be a von Neumann regular ring or, more generally, let A be an associative ring with 1 whose reduction modulo its Jacobson radical is von Neumann regular. We obtain a complete description of all subgroups of GLn A, n > 3, which are normalized by elementary matrices. 1. Introduction. For any associative ring A with 1 and any natural number n, let GLn A be the group of invertible n by n matrices over A and EnA the subgroup generated by all elementary matrices x1'3, where 1 < i / j < n and x E A. In this paper we describe all subgroups of GLn A normalized by EnA for any von Neumann regular A, provided n > 3. Our description is standard (see Bass [1] and Vaserstein [14, 16]): a subgroup H of GL„ A is normalized by EnA if and only if H is of level B for an ideal B of A, i.e. E„(A, B) C H C Gn(A, B). Here Gn(A, B) is the inverse image of the center of GL„(,4/S) (when n > 2, this center consists of scalar invertible matrices over the center of the ring A/B) under the canonical homomorphism GL„ A —►GLn(A/B) and En(A, B) is the normal subgroup of EnA generated by all elementary matrices in Gn(A, B) (when n > 3, the group En(A, B) is generated by matrices of the form (—y)J'lx1'Jy:i''1 with x € B,y £ A,l < i ^ j < n, see [14]).
    [Show full text]
  • Variational Problems on Flows of Diffeomorphisms for Image Matching
    QUARTERLY OF APPLIED MATHEMATICS VOLUME LVI, NUMBER 3 SEPTEMBER 1998, PAGES 587-600 VARIATIONAL PROBLEMS ON FLOWS OF DIFFEOMORPHISMS FOR IMAGE MATCHING By PAUL DUPUIS (LCDS, Division of Applied Mathematics, Brown University, Providence, RI), ULF GRENANDER (Division of Applied Mathematics, Brown University, Providence, Rl), AND MICHAEL I. MILLER (Dept. of Electrical Engineering, Washington University, St. Louis, MO) Abstract. This paper studies a variational formulation of the image matching prob- lem. We consider a scenario in which a canonical representative image T is to be carried via a smooth change of variable into an image that is intended to provide a good fit to the observed data. The images are all defined on an open bounded set GcR3, The changes of variable are determined as solutions of the nonlinear Eulerian transport equation ==v(rj(s;x),s), r)(t;x)=x, (0.1) with the location 77(0;x) in the canonical image carried to the location x in the deformed image. The variational problem then takes the form arg mm ;||2 + [ |Tor?(0;a;) - D(x)\2dx (0.2) JG where ||v|| is an appropriate norm on the velocity field v(-, •), and the second term at- tempts to enforce fidelity to the data. In this paper we derive conditions under which the variational problem described above is well posed. The key issue is the choice of the norm. Conditions are formulated under which the regularity of v(-, ■) imposed by finiteness of the norm guarantees that the associated flow is supported on a space of diffeomorphisms. The problem (0.2) can Received March 15, 1996.
    [Show full text]
  • On Generalizations of Sylow Tower Groups
    Pacific Journal of Mathematics ON GENERALIZATIONS OF SYLOW TOWER GROUPS ABI (ABIADBOLLAH)FATTAHI Vol. 45, No. 2 October 1973 PACIFIC JOURNAL OF MATHEMATICS Vol. 45, No. 2, 1973 ON GENERALIZATIONS OF SYLOW TOWER GROUPS ABIABDOLLAH FATTAHI In this paper two different generalizations of Sylow tower groups are studied. In Chapter I the notion of a fc-tower group is introduced and a bound on the nilpotence length (Fitting height) of an arbitrary finite solvable group is found. In the same chapter a different proof to a theorem of Baer is given; and the list of all minimal-not-Sylow tower groups is obtained. Further results are obtained on a different generalization of Sylow tower groups, called Generalized Sylow Tower Groups (GSTG) by J. Derr. It is shown that the class of all GSTG's of a fixed complexion form a saturated formation, and a structure theorem for all such groups is given. NOTATIONS The following notations will be used throughont this paper: N<]G N is a normal subgroup of G ΛΓCharG N is a characteristic subgroup of G ΛΓ OG N is a minimal normal subgroup of G M< G M is a proper subgroup of G M<- G M is a maximal subgroup of G Z{G) the center of G #>-part of the order of G, p a prime set of all prime divisors of \G\ Φ(G) the Frattini subgroup of G — the intersec- tion of all maximal subgroups of G [H]K semi-direct product of H by K F(G) the Fitting subgroup of G — the maximal normal nilpotent subgroup of G C(H) = CG(H) the centralizer of H in G N(H) = NG(H) the normalizer of H in G PeSy\p(G) P is a Sylow ^-subgroup of G P is a Sy-subgroup of G PeSγlp(G) Core(H) = GoreG(H) the largest normal subgroup of G contained in H= ΓioeoH* KG) the nilpotence length (Fitting height) of G h(G) p-length of G d(G) minimal number of generators of G c(P) nilpotence class of the p-group P some nonnegative power of prime p OP(G) largest normal p-subgroup of G 453 454 A.
    [Show full text]
  • LM2931-N SNOSBE5G –MARCH 2000–REVISED APRIL 2013 LM2931-N Series Low Dropout Regulators Check for Samples: LM2931-N
    LM2931-N www.ti.com SNOSBE5G –MARCH 2000–REVISED APRIL 2013 LM2931-N Series Low Dropout Regulators Check for Samples: LM2931-N 1FEATURES DESCRIPTION The LM2931-N positive voltage regulator features a 2• Very Low Quiescent Current very low quiescent current of 1mA or less when • Output Current in Excess of 100 mA supplying 10mA loads. This unique characteristic and • Input-output Differential Less than 0.6V the extremely low input-output differential required for • Reverse Battery Protection proper regulation (0.2V for output currents of 10mA) make the LM2931-N the ideal regulator for standby • 60V Load Dump Protection power systems. Applications include memory standby • −50V Reverse Transient Protection circuits, CMOS and other low power processor power • Short Circuit Protection supplies as well as systems demanding as much as • Internal Thermal Overload Protection 100mA of output current. • Mirror-image Insertion Protection Designed originally for automotive applications, the LM2931-N and all regulated circuitry are protected • Available in TO-220, TO-92, TO-263, or SOIC-8 from reverse battery installations or 2 battery jumps. Packages During line transients, such as a load dump (60V) • Available as Adjustable with TTL Compatible when the input voltage to the regulator can Switch momentarily exceed the specified maximum operating voltage, the regulator will automatically shut down to protect both internal circuits and the load. The LM2931-N cannot be harmed by temporary mirror-image insertion. Familiar regulator features such as short circuit and thermal overload protection are also provided. The LM2931-N family includes a fixed 5V output (±3.8% tolerance for A grade) or an adjustable output with ON/OFF pin.
    [Show full text]
  • Quasi P Or Not Quasi P? That Is the Question
    Rose-Hulman Undergraduate Mathematics Journal Volume 3 Issue 2 Article 2 Quasi p or not Quasi p? That is the Question Ben Harwood Northern Kentucky University, [email protected] Follow this and additional works at: https://scholar.rose-hulman.edu/rhumj Recommended Citation Harwood, Ben (2002) "Quasi p or not Quasi p? That is the Question," Rose-Hulman Undergraduate Mathematics Journal: Vol. 3 : Iss. 2 , Article 2. Available at: https://scholar.rose-hulman.edu/rhumj/vol3/iss2/2 Quasi p- or not quasi p-? That is the Question.* By Ben Harwood Department of Mathematics and Computer Science Northern Kentucky University Highland Heights, KY 41099 e-mail: [email protected] Section Zero: Introduction The question might not be as profound as Shakespeare’s, but nevertheless, it is interesting. Because few people seem to be aware of quasi p-groups, we will begin with a bit of history and a definition; and then we will determine for each group of order less than 24 (and a few others) whether the group is a quasi p-group for some prime p or not. This paper is a prequel to [Hwd]. In [Hwd] we prove that (Z3 £Z3)oZ2 and Z5 o Z4 are quasi 2-groups. Those proofs now form a portion of Proposition (12.1) It should also be noted that [Hwd] may also be found in this journal. Section One: Why should we be interested in quasi p-groups? In a 1957 paper titled Coverings of algebraic curves [Abh2], Abhyankar conjectured that the algebraic fundamental group of the affine line over an algebraically closed field k of prime characteristic p is the set of quasi p-groups, where by the algebraic fundamental group of the affine line he meant the family of all Galois groups Gal(L=k(X)) as L varies over all finite normal extensions of k(X) the function field of the affine line such that no point of the line is ramified in L, and where by a quasi p-group he meant a finite group that is generated by all of its p-Sylow subgroups.
    [Show full text]
  • A Fast Diffeomorphic Image Registration Algorithm
    www.elsevier.com/locate/ynimg NeuroImage 38 (2007) 95–113 A fast diffeomorphic image registration algorithm John Ashburner Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London, WC1N 3BG, UK Received 26 October 2006; revised 14 May 2007; accepted 3 July 2007 Available online 18 July 2007 This paper describes DARTEL, which is an algorithm for diffeo- Many registration approaches still use a small deformation morphic image registration. It is implemented for both 2D and 3D model. These models parameterise a displacement field (u), which image registration and has been formulated to include an option for is simply added to an identity transform (x). estimating inverse consistent deformations. Nonlinear registration is considered as a local optimisation problem, which is solved using a Φ x x u x 1 Levenberg–Marquardt strategy. The necessary matrix solutions are ð Þ ¼ þ ð Þ ð Þ obtained in reasonable time using a multigrid method. A constant In such parameterisations, the inverse transformation is sometimes Eulerian velocity framework is used, which allows a rapid scaling and approximated by subtracting the displacement. It is worth noting that squaring method to be used in the computations. DARTEL has been this is only a very approximate inverse, which fails badly for larger applied to intersubject registration of 471 whole brain images, and the deformations. As shown in Fig. 1, compositions of these forward resulting deformations were evaluated in terms of how well they encode and “inverse” deformations do not produce an identity transform. the shape information necessary to separate male and female subjects Small deformation models do not necessarily enforce a one-to-one and to predict the ages of the subjects.
    [Show full text]
  • Deep Compositing Using Lie Algebras
    Deep Compositing Using Lie Algebras TOM DUFF Pixar Animation Studios Deep compositing is an important practical tool in creating digital imagery, Deep images extend Porter-Duff compositing by storing multiple but there has been little theoretical analysis of the underlying mathematical values at each pixel with depth information to determine composit- operators. Motivated by finding a simple formulation of the merging oper- ing order. The OpenEXR deep image representation stores, for each ation on OpenEXR-style deep images, we show that the Porter-Duff over pixel, a list of nonoverlapping depth intervals, each containing a function is the operator of a Lie group. In its corresponding Lie algebra, the single pixel value [Kainz 2013]. An interval in a deep pixel can splitting and mixing functions that OpenEXR deep merging requires have a represent contributions either by hard surfaces if the interval has particularly simple form. Working in the Lie algebra, we present a novel, zero extent or by volumetric objects when the interval has nonzero simple proof of the uniqueness of the mixing function. extent. Displaying a single deep image requires compositing all the The Lie group structure has many more applications, including new, values in each deep pixel in depth order. When two deep images correct resampling algorithms for volumetric images with alpha channels, are merged, splitting and mixing of these intervals are necessary, as and a deep image compression technique that outperforms that of OpenEXR. described in the following paragraphs. 26 r Merging the pixels of two deep images requires that correspond- CCS Concepts: Computing methodologies → Antialiasing; Visibility; ing pixels of the two images be reconciled so that all overlapping Image processing; intervals are identical in extent.
    [Show full text]
  • Integer Discrete Flows and Lossless Compression
    Integer Discrete Flows and Lossless Compression Emiel Hoogeboom⇤ Jorn W.T. Peters⇤ UvA-Bosch Delta Lab UvA-Bosch Delta Lab University of Amsterdam University of Amsterdam Netherlands Netherlands [email protected] [email protected] Rianne van den Berg† Max Welling University of Amsterdam UvA-Bosch Delta Lab Netherlands University of Amsterdam [email protected] Netherlands [email protected] Abstract Lossless compression methods shorten the expected representation size of data without loss of information, using a statistical model. Flow-based models are attractive in this setting because they admit exact likelihood optimization, which is equivalent to minimizing the expected number of bits per message. However, conventional flows assume continuous data, which may lead to reconstruction errors when quantized for compression. For that reason, we introduce a flow-based generative model for ordinal discrete data called Integer Discrete Flow (IDF): a bijective integer map that can learn rich transformations on high-dimensional data. As building blocks for IDFs, we introduce a flexible transformation layer called integer discrete coupling. Our experiments show that IDFs are competitive with other flow-based generative models. Furthermore, we demonstrate that IDF based compression achieves state-of-the-art lossless compression rates on CIFAR10, ImageNet32, and ImageNet64. To the best of our knowledge, this is the first lossless compression method that uses invertible neural networks. 1 Introduction Every day, 2500 petabytes of data are generated. Clearly, there is a need for compression to enable efficient transmission and storage of this data. Compression algorithms aim to decrease the size of representations by exploiting patterns and structure in data.
    [Show full text]
  • 7 Homomorphisms and the First Isomorphism Theorem
    7 Homomorphisms and the First Isomorphism Theorem In each of our examples of factor groups, we not only computed the factor group but identified it as isomorphic to an already well-known group. Each of these examples is a special case of a very important theorem: the first isomorphism theorem. This theorem provides a universal way of defining and identifying factor groups. Moreover, it has versions applied to all manner of algebraic structures, perhaps the most famous being the rank–nullity theorem of linear algebra. In order to discuss this theorem, we need to consider two subgroups related to any group homomorphism. 7.1 Homomorphisms, Kernels and Images Definition 7.1. Let f : G ! L be a homomorphism of multiplicative groups. The kernel and image of f are the sets ker f = fg 2 G : f(g) = eLg Im f = ff(g) : g 2 Gg Note that ker f ⊆ G while Im f ⊆ L. Similar notions The image of a function is simply its range Im f = range f, so this is nothing new. You saw the concept of kernel in linear algebra. For example if A 2 M3×2(R) is a matrix, then we can define the linear map 2 3 LA : R ! R : x 7! Ax In this case, the kernel of LA is precisely the nullspace of A. Similarly, the image of LA is the column- space of A. All we are doing in this section is generalizing an old discussion from linear algebra. Theorem 7.2. Suppose that f : G ! L is a homomorphism. Then 1.
    [Show full text]
  • Infinite General Linear Groups
    INFINITE GENERAL LINEAR GROUPS BY RICHARD V. KADISONO 1. Introduction. In this paper, we carry out the investigation, promised in [4](2), of the general linear group 5W9of a factor 5W(3), that is, the group of all invertible operators in 9ïC. The results of this investigation have much in common with the results of [4]; and, where there is some basis for com- parison, namely, in the case where M is a finite factor, there is a great deal of similarity between the present results and those of the classical finite-di- mensional case. There is much greater resemblance between *Mg and 7^, with M a factor of type IL and N a factor of type ln, than there is between the corresponding unitary groups, M« and 7^u, of such factors. This circum- stance results from the fact that the determinant function of [3] takes effect on "M, but not on 5WU(cf. [3, Theorem 4]). The spectral theoretic and ap- proximation techniques of [4] are stock methods in the present paper, and, for this reason, many of the proofs appearing here are reminiscent of those in [4]. There are, however, some striking differences in the methods required for the investigation of the general linear groups and those required for the investigation of the unitary groups. The most minor difference is the fact that the scalars involved do not form a compact set. This, in combination with the fact that the groups we handle are not locally compact, forces us, occasionally, to detour around what we have come to regard, through our experience with locally compact groups, as the most natural approach to the results obtained.
    [Show full text]
  • Math 412. Homomorphisms of Groups: Answers
    (c)Karen E. Smith 2018 UM Math Dept licensed under a Creative Commons By-NC-SA 4.0 International License. Math 412. Homomorphisms of Groups: Answers φ DEFINITION:A group homomorphism is a map G −! H between groups that satisfies φ(g1 ◦ g2) = φ(g1) ◦ φ(g2). DEFINITION: An isomorphism of groups is a bijective homomorphism. φ DEFINITION: The kernel of a group homomorphism G −! H is the subset ker φ := fg 2 G j φ(g) = eH g: φ THEOREM: A group homomorphism G ! H is injective if and only if ker φ = feGg, the trivial group. THEOREM: A non-empty subset H of a group (G; ◦) is a subgroup if and only if it is closed under ◦, and for every g 2 H, the inverse g−1 is in H. A. EXAMPLES OF GROUP HOMOMORPHISMS × 1 (1) Prove that (one line!) GLn(R) ! R sending A 7! det A is a group homomorphism. Find its kernel. (2) Show that the canonical map Z ! Zn sending x 7! [x]n is a group homomorphism. Find its kernel. × (3) Prove that ν : R ! R>0 sending x 7! jxj is a group homomorphism. Find its kernel. (4) Prove that exp : (R; +) ! R× sending x 7! 10x is a group homomorphism. Find its kernel. (5) Consider 2-element group {±} where + is the identity. Show that the map R× ! {±} sending x to its sign is a homomorphism. Compute the kernel. (6) Let σ : D4 ! {±1g be the map which sends a symmetry the square to 1 if the symmetry preserves the orientation of the square and to −1 if the symmetry reserves the orientation of the square.
    [Show full text]