Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering

Jongmin Baek and Andrew Adams

Stanford University

The aim of this self-contained supplement is to rigorously Definition 1.1. A lattice L is a discrete additive subgroup of establish the mathematical properties of the permutohedral a Euclidean space. lattice, to adapt it for the use in Gaussian filtering, and to An additive subgroup of a Euclidean space is closed under further justify its use. For discussion of a CPU implementa- addition and subtraction, and contains the origin. Hence, one tion, applications and empirical comparison to other meth- can alternatively characterize a lattice as all linear combina- ods, see [ABD10]. tions of a set of vectors called the frame (akin to the basis The rest of the paper is organized as follows: in Section 1, of a vector space) with integer weights. For example, the lat- d we give preliminary definitions of relevant terms and no- tice Z corresponds to the frame consisting of standard basis tation; in Section 2, we formally define the permutohedral vectors ~u0,~u1,.... lattice in several ways and prove useful structural proper- The fact that a lattice is an additive group immediately ties stemming from the definition; in Section 3, we suggest gives rise to many useful properties, including translational criteria for picking the ideal lattice with which to perform symmetry; the local structure of a lattice around each lattice Gaussian filtering, and present an argument that the permu- point is invariant. A typical way to study the local structure tohedral lattice is the most appropriate choice within a large of any discrete point set is to examine its Voronoi cells: family of lattices; in Section 4, we discuss how to perform Rd each step of the Gaussian filter using the permutohedral lat- Definition 1.2. Let~x be a point in a lattice L . Then the Rd ⊂ tice. Finally, in Section 5, we analyze the overall algorithm Voronoi cell of ~x is the set of points in nearer to~x than to and compare it to the implementation on Zd in [PD09]. any other lattice point, i.e. ~y Rd ~x′ L, ~x ~y 2 ~x′ ~y 2 . ∈ ∀ ∈ k − k ≤ k − k 1. Preliminaries n o It is clear that the Voronoi cells of a lattice are uniform and We begin by declaring the notation used hereafter in Table 1. related to one another by translation. They also tessellate the Rd Rd Other standard notation such as R,Z applies. underlying space , since any point in can be associated with a lattice point nearest to it. d The dimensionality of the underlying space. Definition 1.3. A Delaunay cell of a lattice is the convex ~x A vector of dimensionality d +1, unless spec- hull of all lattice points whose Voronoi cells share a common ified otherwise: x0,x1,...,x . vertex. { d} ~1 The d + 1-dimensional vector 1,1,...,1 . { } The Delaunay cells are the dual of the Voronoi cells, and Hd The d-dimensional hyperplane yield a natural notion of “nearby” lattice points for an arbi- ~x ~x ~1 = 0 Rd+1. trary point in space. They also tessellate the space, but they { | · }⊂ need not be uniform. Figure 1 demonstrates the Voronoi and ∗ Ad The permutohedral lattice of dimensionality Delaunay cells for a pair of two-dimensional lattices. d, lying inside H . d Note that Rd in the above definitions can be replaced with T( ) The projection mapping Rd+1 onto H along d any set isometric to Rd . For instance, H is a d-dimensional · the vector~1. (See Proposition 2.3.) d hyperplane in Rd+1 and is therefore isometric to Rd. Any N(~x;θ) Multi-variate Gaussian of covariance matrix discrete additive subgroup of H is a lattice, and its Voronoi θI, evaluated at ~x. d and Delaunay cells resides in H . (This is the case with the ~u The k-th standard basis vector, indexed from d k permutohedral lattice, as we shall see shortly in Section 2.) zero, i.e. ~u0 = 1,0,0,...,0 . { } Table 1: Common notation used in the paper. 2. The Permutohedral Lattice The permutohedral lattice has seen applications in many Now we may review the basic definitions that will be use- fields ranging from crystallography to communication. It ful in the subsequent sections. Our first definition is, not sur- arises ubiquitously in nature; its two-dimensional and three- prisingly, the standard definition of a lattice: dimensional instances are the hexagonal honeycomb—see 2 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering

d A∗ := ~x H ~x is a remainder-k point. , (2.3) d { ∈ d | } k[=0 where we call ~x H a remainder-k point for some k ∈ d ∈ 0,...,d iff all coordinates are congruent to k modulo { } d + 1. Proof. (2.2) (2.3): Let~x (d + 1)Zd+1. Then we can write ⊆ ∈ ~x = (d + 1)~y for some ~y Zd+1. This yields, ∈ (d + 1)(~y ~1) T(~x) = (d + 1)~y · ~1 − d + 1 = (d + 1)~y (~y ~1)~1. − · We see that each component of T(~x) is an integer and has a Figure 1: Voronoi and Delaunay tessellations of some two- consistent remainder modulo d + 1, namely (~y ~1). Z2 ∗ · dimensional lattices. Top: the Voronoi cells of and A2 are shown, along with lattice points. The two lattices have (2.3) (2.1): Let ~x be a remainder-k point for some k ⊆ ∈ equal density. Bottom: the Delaunay cells induced by the 0,...,d . Then we can write ~x = (d + 1)~y + k~1 for some { } Voronoi cells above. All cells of Z2 are squares, whereas A∗ ~y Zd+1. To show that ~x/(d + 1) is in the dual of A , note 2 ∈ d has hexagonal Voronoi cells and triangular Delaunay cells. that for all~z A , ∈ d ~x k ~z = ~y + ~1 ~z d + 1 · d + 1 · Figure 1—and the face-centered cubic lattice seen in the k =~y ~z + ~z ~1 packing structure of diamond, respectively. We will define · d + 1 · the permutohedral lattice in several ways and derive its struc- =~y ~z Z, since~z ~1 = 0. tural properties. · ∈ · (2.1) (2.2): Let ~x/(d + 1) be in the dual lattice of A . Note ⊆ d that for all i = j, the vector ~ui ~u j belongs to A , because 2.1. Definition 6 − d (~ui ~u j) ~1 = 0. Hence, Traditionally, the permutohedral lattice is defined as the dual − · ~x xi x j of the root lattice Ad. Note that the dual of a lattice is not the (~ui ~u j) = Z. vertices of its Voronoi cells, as typically intuited; see Defini- d + 1 · − d + 1 − d + 1 ∈ tion 2.2. Both Ad and the permutohedral lattice are typically This implies that all pairs of coordinates xi,x j differ by mul- d+1 embedded in H R for ease of manipulation. tiples of d +1. Writing xi = (d +1)yi +k for some yi Z,k d ∈ ∈ ⊂ R, we see that~x is the projection of (d +1)~y obtained by sub- Definition 2.1. The root lattice A is d tracting a multiple of (1,...,1). Because~x H , this projec- ∈ d d+1 tion must be equivalent to T. ~x = (x0,...,x ) Z ~x ~1 = 0 . d ∈ · According to Proposition 2.3, the simplest way to charac- d+1 ∗ Zd+1 In other words, Ad = Z Hd . Quite clearly, Ad Hd. terize Ad is to project along the long diagonal vector ∩ ⊂ ~ Z ∗ 1, with a scalar factor of d + 1 to keep the coordinates in , Definition 2.2. The permutohedral lattice, denoted by Ad , is as seen in Figure 2. From now on, we will assume the scaled the dual lattice of Ad inside Hd : ∗ version of Ad ,i.e.(2.2)or(2.3), rather than the original (2.1). A∗ := ~x H ~y A , ~x ~y Z . (2.1) d ∈ d ∀ ∈ d · ∈

For example, A∗ is spanned by 1 , 1 , 2 and 1 , 2 , 1 . 2 3 3 − 3 3 − 3 3 ∗ We shall state two equivalent definitions of Ad scaled up by d + 1 in each dimension. The scale factor ensures that all coordinates will be integers, and these definitions more clearly elucidate the properties of the lattice. ∗ Proposition 2.3. The following two definitions of Ad are equivalent to (2.1) scaled up by d + 1: Figure 2: Construction of A∗ in case d = 2. Left: a regular A∗ := T(~x) ~x (d + 1)Zd+1 , (2.2) d d ∈ grid. The xy-plane is shown in red, along with the unit cube n o in blue. Right: the same figure in orthographic projection, where T is the projection of Rd+1 onto H , namely d seen along the vector~1. Note that many of the points line up, ~ indicating that they will project to the same point on H . The ~x 1 ~ d T :~x ~x · 1. set of projected points form A∗, with appropriate scaling. 7→ − ~1 ~1 ! d · J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering 3

2.2. Structural Properties In particular, 2 2 As with other lattices, A∗ induces a tessellation of H with its argmin ~y ~x = argmin ∑ (d + 1)zi + k xi d d ~zk − k ~z | − | Voronoi cells. The Voronoi cells of A∗ are polytopes called 2 2 d = argminz ∑(d + 1) zi + 2(d + 1)zi(k xi) permutohedra because they are obtained by permuting the ~ − 2 2 = argmin ∑(d + 1) z 2(d + 1)zixi coordinates of any single vertex. The name of these poly- ~z i − topes also lends itself to this lattice. = argmin (d + 1)~z (d/2,..., d/2) 2 ~zk − − k Proposition 2.4. [CS98] The Voronoi cell of the origin in = (0,...,0, 1,... 1). ∗ − − Ad is a permutohedron with vertices d+1−k k d d 2 d + 2 d Plugging this expression| {z } | back{z into} ~y = (d + 1)~z + k~1, we ρ , − ,..., − , − ρ S , 2 2 2 2 ∈ d+1 obtain ~y = (k,...,k,k (d + 1)...,k (d + 1)), showing − − that there is a unique nearest lattice point of remainder k. where Sd+1 is the symmetric group on d + 1 elements, i.e. Next, we can check in a straightforward manner that the the set of all permutations. nearest remainder-k points for all k are all equidistant from ~x. Hence, varying k over 0,...,d yields the canonical The permutohedra in the first three dimensons are a line simplex as stated above. { } segment, a hexagon (Figure 1) and a uniform truncated octahedron (Figure 3), respectively. ρ d −d If~x is some permutation of 2 ,..., 2 , the above deriva- tion commutes fully with ρ, yielding a simplex that is related to the canonical simplex via ρ. Corollary 2.6. Each Delaunay cell contains exactly one lattice point of remainder k, for each k 0,...,d . ∈ { } Proof. By construction, the canonical simplex contains exactly one lattice point of remainder k. Because other simplices are obtained by permuting the components of its ver- ∗ Figure 3: Left: the Voronoi cell of A3 (a permutohedron in tices and translating them, the same holds. R3), which is also the uniform truncated octahedron. Right: ∗ In fact, we can significantly strengthen Corollary 2.6: the Delaunay cell of A3 , which is an isosceles tetrahedron. The red sides are longer than the blue sides. Corollary 2.7. For every ~x Hd, the Delaunay cell containing it is the set of the closest∈ remainder-k point for each k 0,...,d . ∈ { } To demonstrate this, we first show a lemma characterizing ∗ Theorem 2.5. The Delaunay cells of Ad are d-simplices, and points near an arbitrary lattice point: are related via permutation and translation to the canonical Lemma 2.8. (The Range Lemma) Let ~x Hd, and let ~y simplex whose vertices are the following: ∗ ∈ ∈ Ad be a remainder-k point. Then,~y is the closest remainder-k (0,0,...,0,0,0), point to ~x iff 1 1 1 1 d ( , ,..., , , ), max(xi yi) min(xi yi) d + 1. − i − − i − ≤ . . Proof. Because of translational symmetry, it suffices to prove (k,...,k,k (d + 1),...,k (d + 1)), − − the claim for~y = (0,...,0). By inspection, all vertices of the d+1−k k canonical simplex obey the following inequalities: . | {z } | . {z } x0 x1 x2 x , and x0 x d + 1. (2.4) ≥ ≥ ≥···≥ d − d ≤ (d, 1,..., 1, 1, 1). − − − − Since any point in the simplex is a convex combination of the vertices, the above inequality must also hold for ∗ Proof. Because of the translational symmetry in Ad , it any point inside. In particular, for any of the d + 1 in- suffices to characterize only the Delaunay cells containing equalities in (2.4), all but one vertex of the canonical the origin. Let ~x be the vertex shared by a set of Voronoi simplex satisfy it with equality. Since a linear inequality cells, one of which is the Voronoi cell of the origin. By is a (d 1)-dimensional hyperplane, which is completely d d−2 −d − Proposition 2.4,~x is some permutation of ( 2 , 2 ,..., 2 ). determined by d points, it must be that the face formed by WLOG, assume ~x = ( d , d−2 ,..., d ). the d vertices corresponds to the hyperplane. Therefore, the 2 2 − 2 region bounded by the faces (i.e. the simplex) corresponds The Delaunay cells are defined by the lattice points closest exactly to the set of points satisfying (2.4). ∗ to~x. By Proposition 2.3, lattice points in Ad are remainder-k d+1 points, having the form ~y = (d + 1)~z + k~1 where ~z Z . Hence, for any ~x in the canonical simplex, maxi xi ∈ − Since ~y ~1 = 0, consequently~z ~1 = k. mini xi = x0 x d + 1. Likewise, for any simplex that · · − − d ≤ is a permutation of the canonical simplex, we have x j x − k ≤ By choice of ~z, the distance between ~x and ~y is minimized. d + 1 for some j,k where x j = maxi xi and xk = mini xi. 4 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering

Corollary 2.7 then follows: if~x is in the canonical simplex, 3. Evaluating Lattices for High-Dimensional Filtering we can easily check that the vertices given in Theorem 2.5 There are various spatial data structures for accelerating satisfy the premise of the Range Lemma. Therefore, they high-dimensional Gaussian filtering, such as the bilateral must be the closest remainder-k points to ~x as desired. Note grid [CPD07], the Gaussian KD-tree [AGDL09] and point that the expression maxi(xi yi) mini(xi yi) is invariant − − − clusters [GS91]. However, their efficacy can be limited by under permutation of the components or translation of~x and poor scaling behavior in both time and memory, numerical ~y, so the argument applies to other simplices in A∗d. instability and complexity. Corollary 2.7 is of main interest to us because it suggests These approaches bifurcate into two branches: one re- an efficient way to calculate the vertices of the Delaunay cell lies on discretization of data and predicable placements of containing an arbitrarily specified point. It suffices to simply points—a prime example is a lattice. The latter is more data- find the closest remainder-k point for each k 0,...,d . ∈ { } dependent in its organization, e.g. trees and clusters. We Lemma 2.9. [CS98] Given ~x Hd , the closest remainder-0 shall restrict ourselves to the former class. The use of the ∗ ∈ permutohedral lattice can then be motivated by examining points in Ad can be found with the following procedure: how a Gaussian filter might be applied on lattices and iden- 1. Define ~y where yi is the multiple of d + 1 closest to xi. tifying useful criteria. ∑d y 2. Compute the difference ~∆ =~x ~y, and h = i=0 i . − d + 1 3. If h < 0, add d + 1 to each y j where ∆ j is one of the h - 3.1. Framework for Arbitrary Lattices | | largest differences. If h > 0, subtract d + 1 from each y j We first review the bilateral filter on a regular grid Zd, gener- ∆ where j is one of the h-smallest differences. alizing to an arbitrary lattice L at the same time. This allows Proof. Note that the set of remainder-0 points in A∗ is equiv- us to identify properties that are critical for efficient and ac- d curate Gaussian filtering. In doing so, we adopt the terminol- alent to Ad scaled up by d + 1. Conway gives an analogous ogy of [AGDL09] for the steps in Gaussian filtering: splat- algorithm for finding the nearest lattice point in Ad with proof [CS98]. ting, blurring and slicing. For more detailed overview of Gaussian filtering on particular data structures, see [CPD07] Theorem 2.10. Given ~x H , the Delaunay cell containing [PD09][AGDL09][YDGD03]. ∈ d it can be found by the following procedure in O(d2): Splatting 1. Find the nearest remainder-0 point ~y. 2. Compute the differential ~∆ =~x ~y. The first step in Gaussian filtering is to splat, or embed, the 3. Find the permutation ρ that sorts−~∆ in decreasing order. input signal into a higher-dimensional Euclidean space. For 4. For each k, the nearest remainder-k point is given by each sample of the input signal, its value is accumulated at −1 ρ (~vk) +~y where ~vk is the remainder-k point in the one or more spatially nearby lattice points belonging to L. canonical simplex. In case only one point in L is chosen, the process is effectively equivalent to quantization. Choosing multiple points Proof. The theorem follows directly from Lemma 2.9 and with nonnegative weights that sum to 1 corresponds to an 2 Theorem 2.5. Each step runs in O(d log d) or O(d ), so the interpolation (for example, bilinear weights in 2D), and gen- 2 overall time complexity is O(d ). erally improves accuracy. In summary, we have explored the structure of the permu- For Zd, the standard multi-linear weights can be used to tohedral lattice in terms of its Voronoi and Delaunay cells, splat the sample point into 2d positions. More generally, for along with useful results on how to quickly locate nearby arbitrary lattices, the sample can be splatted onto the ver- lattice points or verify them. tices of the Delaunay cell containing it, with an appropriate scheme to assign weights. See Figure 4 for examples. We end the section by briefly discussing the volumes of the Voronoi and Delaunay cells, as it will be necessarily in Blurring the subsequent sections: The value stored at each location is then blurred with those Proposition 2.11. The volume of a permutohedron is given d−1/2 at nearby locations in L, with weights that decay as the Gaus- by (d + 1) . The volume of the canonical simplex is sian of their L -distance. This step is referred to as blurring, d−1/2 2 (d + 1) /d!. as it accomplishes blurring of the data held at the lattice points. In case of Zd, its separable nature enables us to blur Proof. By inspection, A∗ is spanned by any d of the per- d each dimension in isolation, requiring access only to neigh- mutations of (1,1,..., d). The volume of the parallelop- bors along each axis. For other lattices, we may consider as iped constructed by a− basis is given by the square root of neighbors all vertices that share at least one Delaunay cell its Gram determinant, which can be easily computed to be with the given vertex. Unfortunately, most lattices are not (d + 1)d−1/2. The volume of the permutohedron must equal separable, complicating this process. this number, because both the parallelopiped and the permutohedron tessellate the space with equal density, namely the Slicing density of the lattice points in space. Lastly, a simple count- ing argument shows that there are d! simplices for every per- Slicing is the process of sampling the data structure at the mutohedron. original input locations, and is exactly analogous to splat- J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering 5

Lattice # of vertices in Delaunay cell d+1 Ad k for k 1,...,d ∗ ∈ { } Ad d + 1 1 1 ∗ Zd d Bd = Bd = 2 1 d−1 1 Dd 2d or 2 + 1 ∗ ⌊d/2⌋ Dd 4 E6 27 2/9 1/9 1/6 E7 8 or 56 1/6 E8 9 or 16 1 1 F4 8 G 3 2/3 2 4/9 2/9 Table 2: Classiﬁcation of irreducible root lattices in ﬁnite dimensions. The dual lattices are also given for ones that Z2 ∗ generalize across dimensions. The right-hand column gives Figure 4: The splatting process on and A2 . The Delaunay tessellations are shown in dotted guidelines. Top: a sample the number of vertices in the Delaunay cells; some lattices point is rounded to the nearest lattice point. Bottom: a sam- may have more than one type of Delaunay cells that occur. ∗ ple point is spread onto the vertices of the Delaunay cell con- Note that Ad is the only family of root lattices that general- taining it, using bilinear coordinates for Z2 and barycentric izes across d, tessellates the space with a uniform Delaunay ∗ cell, and features a small number of vertices in each cell. coordinates for A2 .

ting: each sample of the output signal is reconstructed from crystallograpy (the 14 Bravais lattices) can be decomposed nearby positions, with the same set of weights as in splatting. into root lattices and their duals up to affine transform. Theorem 3.1. (Witt, 1941) [CS98] Let L be an integral lat- 3.2. Criteria for the Ideal Lattice tice, meaning that ~x,~y L, ~x ~y Z. Then, all sublattices ∀ ∈ · ∈ The framework in the previous section guides the desig- of L generated by vectors of norm 1 and 2 are direct sums of nation of appropriate criteria for selecting the ideal lattice root lattices. for Gaussian filtering. In forming the criteria, we concern Formally, root lattices are lattices generated by the roots ourselves with numerical accuracy, computational efficiency of semisimple Lie algebras [Ser87]. Fortunately, irreducible and generality: root lattices in finite dimensions have been fully enumerated. (Translational symmetry) For every point ~x Rd, the • ∈ splatting, blurring and slicing kernels are spatially invari- Theorem 3.2. [Sam69] [Ser87] There exist exactly the fol- ant. This also ensures that the calculations will be uniform lowing irreducible root lattices: across the space, simplifying the algorithms required and ensuring numerical stability. By definition, lattices exhibit Ad,Bd,Dd,E6,E7,E8,F4 and G2. this property. d Note that Bd is equivalent to the standard lattice Z . (Even distribution) The distribution of the points in space • is as even and isotropic as possible. This requirement is in We may then restrict our search to these irreducible root place to ensure that the kernels will resemble the Gaussian lattices and their duals. The Voronoi and Delaunay cells of as much as possible. irreducible root lattices are given in [MP92]. Those of the (Fast splat and slice) Given ~x Rd, one must be able to dual lattices can be found without much difficulty. Table 2 • quickly and systematically locate∈ nearby lattice points, i.e. lists the root lattices along with their duals, and the number find the Delaunay cell containing ~x. The number of ver- of vertices in each of their Delaunay cells. This number must tices in the Delaunay cell should scale slowly with d. be small in order to support fast splatting and slicing, and the (Fast blur) Given an arbitrary lattice point, one must be lattice should generalize across dimensions. As shown, only • ∗ able to quickly and systematically locate nearby lattice Ad , the permutohedral lattice, meets these criteria. (In fact, points with which to blur. We prefer that the blurring step d + 1 is the fewest number of vertices possible for any d- be separable if at all possible. dimensional Delaunay cell.) (Generality) There should exist an analogous construction Moreover, A∗ has been shown to exhibit very efficient • of discrete sets for each dimensionality d. d packing; it is the best known lattice for sphere covering up to d = 22 [CS98]. The problem of sphere covering seeks 3.3. Finding the Ideal Lattice to place uniform spheres at lattice point to cover the entire space, while minimizing the relative density of the lattice. The classification of lattices in arbitrary dimensions is an This loosely indicates that the distribution of neighbors of a open research problem, and is beyond the scope of this pa- lattice point in A∗ is as even and isotropic as possible. per. However, it does happen that an important class of lat- d ∗ tices can be constructed from a rather simple set of lattices Finally, we claim that Ad enables fast blurring by being called root lattices. For instance, all significant lattices in 3D separable in a weak sense, as will be demonstrated in Sec- 6 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering tion 4.2. We have already shown that nearest lattice points xd−1 xd xd−2 xd−1 = − + 2 − + are easily located in A∗. d + 1 d + 1 ··· d ∗ d In summary, the permutohedral lattice A is the only root x0 x1 d + d − (d + 1) ∑ b lattice or dual of a root lattice that meets the criteria spelled d + 1 − k " k=d− j+1 # out in Section 3.2. It is computationally efficient (fast splat- xd xd−1 x1 + dx0 ting, blurring and slicing), exhibits good packing behavior, = − − −···− (x0 x j) d + 1 − − and generalizes across dimensions. While this does not rule xd xd−1 x1 x0 out lattices not covered by Witt’s Theorem from being supe- = − − −···− − + x j ∗ d + 1 rior, it does strongly suggest the optimality of Ad . = x j, Remark 3.3. Permutohedra fall under the family of poly- as desired. Lastly, the weights b ,...,b sum to 1 by con- topes called parallelohedra, which are polytopes that give 0 d struction. a face-to-face tiling of Euclidean space by translation. Par- allelohedra whose Delaunay cells are simplices are called Lemma 4.1 easily generalizes to arbitrary points in Hd by primitive. Voronoi proved that any primitive parallelohedon utilizing the fact that other simplices are given by permuta- is affinely equivalent to the Voronoi cell of some integral tion and translation of the canonical simplex. lattice [Vor52][Dol07][Rys98]. This justifies our focus on Proposition 4.2. Let~x be an arbitrary point in Hd, and let~vk lattices, which are translation-invariant, and in particular, on be the remainder-k point in the Delaunay cell containing ~x, integral lattices. for each k 0,...,d . Also, let ρ S be the permutation ∈ { } ∈ d+1 that sorts the components of ~∆ =~x ~v0 in decreasing order. − Then, the barycentric coordinates for ~x is given by~b where 4. Algorithms ρ(~∆) ρ(~∆) Now we proceed to describe in full how to perform Gaus- d−k − d+1−k , k = 0, ∗ b = d + 1 6 sian filtering using the permutohedral lattice Ad . Each of k  ρ ~∆ ρ ~∆  ( )0 ( )d splatting, blurring, slicing steps is discussed in detail, with 1 − , k = 0. − d + 1 pseudocode given when applicable. Proof. The claim follows from Lemma 4.1 and Theo-  rem 2.10. 4.1. Splatting Hence the barycentric coordinates of a point in Hd can be ∗ computed in O(d log d), and if ρ and~v0 are already available, Splatting a point involves computing its neighbors in Ad , namely the vertices of the Delaunay cell containing it, in O(d). and accumulating the value at the vertices with appropri- The splatting step is complete once every sample from the ate weights. For simplices, the most natural interpolation input signal has been splatted. Algorithm 1 summarizes the scheme is barycentric coordinates. process. Note that at each lattice point, nearby samples from

Lemma 4.1. Denote by ~vk the remainder-k vertex of the Algorithm 1 The splatting algorithm. Given the input signal canonical simplex. Let ~x be an arbitrary point in the canoni- n I : X R , where X is the discrete subset of Hd, it embeds cal simplex, and let b0,...,bd be its barycentric coordinates → ∗ the signal in Ad and returns the result. in the simplex, i.e. Require: I : X Rn. → d d V (~y) 0, ~y A∗ ← ∀ ∈ d ~x = ∑ bk~vk, and ∑ bk = 1. for all ~x X do k=0 k=0 ∈ ~y the closest remainder-0 point to ~x (Lemma 2.9) ← Then, ~∆ ~x ~y ρ ← the− permutation that sorts ~∆ in decreasing order xd−k xd+1−k ← − , k = 0, for k = 0 to d do b = d + 1 6 k  x0 xd ~v the closest remainder-k point (Lemma 2.10) 1 − , k = 0. k  − d + 1 b ← the barycentric coordinate (Proposition 4.2) k ← V(~y) V (~y) + b I(~x) Proof. Because the vertices of a (non-degenerate) simplex ← k span the underlying Euclidean space, a barycentric decom- end for position exists and is unique. Thus it sufﬁces to show that end for ∗ Rn ∑d return V : Ad the given weights do yield ~x. Let~y = k=0 bk~vk. Then, →

d the input are accumulated with weights that depend on the y j = ∑ bk (vk) j ~∆ k=0 · offset vector . So the splatting process is a convolution with d− j d a kernel in Hd (followed by sampling at the lattice points.) = ∑ bkk + ∑ bk(k (d + 1)) Proposition 4.3. The splatting process is equivalent to con- − "k=0 # "k=d− j+1 # volution with the following kernel: d d maxi xi mini xi = ∑ bkk (d + 1) ∑ bk Ks :~x 1 − . "k=0 # − " k=d− j+1 # 7→ − d + 1 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering 7

Proof. It sufﬁces to examine the values accumulated at the fore, origin. For each Delaunay cell containing the origin, which 1 is obtained by permuting the dimensions of the canoni- λ ~y [0,d + 1]d+1 T(~y) =~x √ { ∈ | } cal simplex, the weight associated to the origin is 1 (d + 1) d + 1 − ρ(~x)0 ρ(~x)d 1 − , where ρ sorts ~x in decreasing order. (See = √d + 1(d + 1 (maxxi minxi)) d + 1 (d + 1)√d + 1 − i − i max x min x i i i i maxi xi mini xi Proposition 4.2.) Hence Ks(~x) = 1 − as de- = 1 − − d + 1 d 1 sired. − + = Ks(~x) by Proposition 4.3, as desired.

See Figure 5 for illustration of Proposition 4.4. This “un- ﬂattened” representation facilitates numerical analysis of the splatting kernel, as seen in the following two results. 6 Corollary 4.5.

3 d− 1 K (~x)d~x = (d + 1) 2 . Z s Hd 0 0 0 Proof. We can “unproject” Ks back into a cube with uniform 3 3 density, and the total mass would remain constant. Hence 1 d+1 d− 1 6 6 it becomes (d + 1) = (d + 1) 2 as de- (d + 1)√d + 1 sired. Figure 5: A plot of the 3-dimensional cube [0,3]3, along with points in 3Z3. The plot is oriented such that (1,1,1) is nor- d(d + 1)2 Proposition 4.6. The variance of Ks is . mal to the plane of the figure. Note that the flattening of the 12 cube along 1 1 1 yields the union of six simplices, which ( , , ) Proof. By definition, equals the support for Ks for d = 2. ~x 2K (~x)d~x Hd s Var(Ks) = k k . R K (~x)d~x Hd s R The denominator is given by Corollary 4.5. For the numer- The splatting kernel Ks can be described as a tent whose 2 ator, the sum of ~x over Hd weighted by the density of support is the union of the Delaunay cells meeting at the k k 2 Ks is equivalent to an unweighted sum of T (~y) over origin. There is also a surprisingly elegant way of character- d+1 k k ∗ [0,d + 1] . Hence, izing Ks, based on one of the definitions of Ad in Section 2.1. 2 1 [0,d+1]d+1 T(~y) d~y R k k (d + 1)√d + 1 d+1 Var(K ) = Proposition 4.4. Let T : R H be the projection onto s d−1/2 → d (d + 1) Hd as before. Then 1 2 = d 1 T(~y) d~y (d + 1) + Z[0,d+1]d+1 k k 1 d+1 K (~x) = λ ~y [0,d + 1] T(~y) =~x , 2 s 1 ∑y (d + 1)√d + 1 { ∈ | } = ~y i ~1 d~y (d + 1)d+1 Z d+1 − d + 1 [0,d+1] where λ is the 1-dimensional Lebesgue measure, i.e. the 2 2 1 2 (∑yi ) 2(∑yi) length of the set. Equivalently, Ks is the density of a uni- = d 1 ∑yi + d~y (d + 1) + Z[0,d+1]d+1 d + 1 − d + 1 form cube [0,d + 1]d+1 after it is flattened onto H , up to a d ∑ 2 ∑ y y multiplicative factor. 1 d yi i6= j i j = d 1 d~y (d + 1) + Z[0,d+1]d+1 d + 1 − d + 1 d+1 2 d+1 1 d d y Proof. The set ~y [0,d + 1] T(~y) = ~x contains ele- = ∑ (d + 1) · i dy { ∈ | } d+1 d+1 Z i ments of the form ~y = ~x + q ~1. Since ~y [0,d + 1] , it (d + 1) i 0 d + 1 · ∈ must be that 0 x q d 1 for all i. This is equivalent to d+1 i + + 1 yiy j ≤ ≤ ∑ (d + 1)d−1 dy dy d+1 ZZ i j − (d + 1) i6= j 0 d + 1 minxi q d + 1 maxxi. − i ≤ ≤ − i d(d + 1)2 d(d + 1)2 = 3 − 4 The set is a linear segment parallel to ~1, so its measure is d(d + 1)2 given by its projected length along any of the dimensions = , 12 multipled by ~1 = √d + 1. Its projected length is simply k k the extent of q, which is d + 1 (maxi xi mini xi). There- as desired. − − 8 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering

4.2. Blurring Meanwhile, 2 A standard multi-variate Gaussian blur in a d-dimensional ~y 2 = ∑s ~w k k k · k space is separable into d Gaussian blurs along each of the d 2 Zd = ∑si ~wi ~w i + ∑ sis j~wi ~w j independent axes. The advantage of using as the lattice · · is that the choice of the d axes is straightforward, and that i i6= j Rd 2 sis j each of the d directional blurs is simple: for all~x ,~x can = ∑si ∑ ∈ − d be blurred with points of the form ~x q ~uk, where q Z. i i6= j ∗ ± · ∈ For Ad , however, the choice of d principal axes in Hd with 2 such property is not clear. In fact, there is no natural set of d d + 1 2 1 = ∑si ∑si independent axes that align with the spatial arrangement of d i − d i ! ∗ points in Ad . (See Figure 1.) d + 1 = ~s 2. ∗ d k k Recall Proposition 2.3, which states Ad is obtained by ap- d+1 d+1 plying the projection T on (d +1)Z R . The original θ d ∈ It follows that Kb(~y) = N ~y; d+1 as claimed. space is spanned by the vectors in the set (d + 1)~uk . Be- { ∗ } cause T is linear, we can equivalently state that Ad is spanned Proposition 4.7 demonstrates that a Gaussian blur can be by their projections ~wk, i.e. performed on Hd in a “separable” fashion. To adapt this approach onto A∗, simply define Kk : H R as the discretiza- T(~uk) 1 d b d → ~wk := = ( 1,..., 1,d, 1,..., 1). tion of k: T(~uk) d(d + 1) − − − − k k G k Q p k θ d δ The set w represents a natural choice of d 1 axes along Kb :~x ∑ N ~x; (~x q ~t), ~ k | {z } + 7→ d + 1 · − · which to{ blur.} q=−Q k Of course, ~w is not linearly independent, since its car- where ~t = ( 1,..., 1,d, 1,..., 1). Note that Kb is { k} − − − − dinality exceeds the rank of the space. Nevertheless, d +1 di- k rectional blurs along these axes compose to form a standard nonzero only at lattice points along the projection of the k-th | {z } multi-variate Gaussian blur in Hd, as shown in the following standard axis. See Algorithm 2 for the exact description of proposition. the steps. Proposition 4.7. Define to be the operator that performs Gk Gaussian blur along ~wk with variance θ: Algorithm 2 The blurring algorithm. Data stored in the per- ∗ Rn mutohedral lattice in the form V : Ad is blurred with θ θ ∗ Rn→ k( f ) :~x f (~x t ~wk)N(t; )dt, variance and stored in W : Ad . In practice, the do- G 7→ ZR − · → ∗ main of V and W is restricted to some finite subset Y Ad . where f : H R. Then, = performs the stan- ⊂ d 0 d Require: V : A∗ Rn dard multi-variate→ GaussianG blurG ◦···◦G in H with variance θ d . d d d+1 Require: Q N → ∈ Proof. Composing 0,..., k yields Require: θ R+ G G ∈ for k = 0 to d do ( f ) :~x f ~x ∑tk ~wk N(~t; θ)d~t. Z d+1 ~t (1,...,1, d,1...,1) G 7→ R − · ← − is a sequence of convolutions, and thus corresponds to a k ∗ G R for all ~x A do convolution kernel Kb : Hd : | {z∈ }d → ∑Q ~ ~ θ W(~x) q=−Q V(~x + q t) N(q t; (d + 1)/d) ( f ) :~x f (~x ~y)K (~y)d~y. end for ← · · · G 7→ Z − b Hd V W Equating the two expressions above, we obtain end for← return I : X Rn Kb(~y) = N(~t; θ)δ ∑tk ~wk ~y d~t. → ZRd+1 · − Since the equation ∑t ~w =~y is underconstrained by a sin- One may realize that the composition of K0,...,Kd does k · k b b gle dimension, the set of solutions has one degree of free- not equal Kb, a true Gaussian blur. This is the discretization dom. By inspection,~t =~s + q ~1 where ~s Hd is a solution error inherent to any discrete spatial data structures such as R · ∈ 0 d and q . Hence we can write lattices. Kb Kb is akin to approximating a multi-variate ∈ ◦···◦ ∗ d~t Gaussian kernel by sampling it at points in Ad , up to some Kb(~y) = N(~s + q ~1; θ) dq discretization error. ZR · dq

In practice, one can use a simpler and more compact set = N(~s; θ)N(q ~1; θ)√ d + 1dq ZR · of weights for the directional blur, e.g., ~ because ~s and 1 are orthogonal, V (~x ~t) V (~x) V (~x +~t) W (~x) − + + , (4.1) = N(~s; θ) N(q ~1; θ)√d + 1dq ← 4 2 4 ZR · ln√2 = N(~s; θ). Equation (4.1) amounts to setting Q = 1 and θ = . (d + 1)2 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering 9

4.3. Slicing Ks The output signal can be reconstructed from the permutohedral lattice by sampling from nearby lattice points, as in Rotate Sample I into H at A∗ splatting. The splatting algorithm, given in Algorithm 1, can d d be adapted with very little change. 0 d Kb ,...,Kb Ks

Algorithm 3 The slicing algorithm. Given data stored in the Rotate ∗ Rn into Rd permutohedral lattice in the form W : Ad , the output n → I : X Hd R is reconstructed. ⊂ → ∗ n Require: W : A R . Figure 6: Summary of Gaussian ﬁltering with A∗. The in- d → d I(~x) 0, ~x X Rd ← ∀ ∈ put signal I resides in , and is reparametrized into Hd for all~x X do ′ Rn ∈ via an appropriate rotation, yielding a map I : Hd . ~y the closest remainder-0 point (Lemma 2.9) ′ → ← Each component of I is then convolved with Ks in the splat- ~∆ ~x ~y ∗ ← − ting step, is sampled at positions in Ad , is convolved with ρ the permutation sorting ~∆ in decreasing order 0 d ← Kb ,...,Kb and Ks in the blurring step and the slicing step, for k = 0 to d do respectively, and rotated back into Rd to yield an output sig- v~ the closest remainder-k point (Lemma 2.10) ′′ d n k ← nal I : R R . b the barycentric coordinate (Proposition 4.2) → k ← I(~x) I(~x) + bkW (~y) end for← end for plot of Figure 7.) Analogously, let Z~x be the convolution ker- d return I : X Rn nel at ~x for Z . For convenience, we presume that the blur → 1 1 1 kernel in each dimension is 4 2 4 , as in Equation (4.1). h i Note that for all~x X, the closest remainder-0 point~y and Proposition 5.1. ρ ∈ the permutation were previously computed in the splatting d algorithm. By storing these results during the splatting step, Z~x(~y ~x) = ∏ ∑ Bp(xi)Bq(yi)b(p q), we can reduce the runtime complexity of the slicing step to − i=1 p,q∈Z − O( X d), paying storage cost of O( X d). | | | | where The slicing process is again equivalent to a convolution. 1 , x = 0, 2 2 d(d + 1) 1 The kernel is simply Ks as before, with variance . b :~x , x = 1, , Bp : x max(1 p x ,0). 12 7→  4 ± 7→ − | − | 0, otherwise.

4.4. Summary Proof. The input signal at ~y is splatted onto each lattice ∏d point ~q with multi-linear weight i=1 Bqi (yi), which is All three steps are linear in the input, so their composition then blurred onto every other lattice point ~p with weight can be described as a spatially-varying convolution shown d ∏ b(pi qi). Then the output at ~x is reconstructed by in Figure 6. To control the variance of the Gaussian blur, i=1 − summing over each lattice point ~p with weight ∏d B (x ). one may scale the input signal appropriately before feeding i=1 pi i The claim in the proposition then follows once we rewrite it into the pipeline. Figure 7 visualizes the components for the sum of product terms as the product of sums. d = 2. Proposition 5.2.

5. Analysis K~x(~y ~x) = ∑ B~p(~x)B~q(~y)b(~p ~q) − ~p,~q∈A∗ − For analysis of time and space complexity of Gaussian filter- d ing on the permutohedral lattice, see [ABD10], which gives where B~p(~x) is the barycentric weight assigned to point ~p an implementation using a sparse permutohedral lattice built for the interpolation of~x, and b( ) is the appropriate blurring 0 · d on a hash table. [ABD10] further demonstrates that it is the kernel given by convolving Kb ,...,Kb . first known method for Gaussian filtering with runtime that Proof. The proof is analogous to that of Proposition 5.1. is linear in the number of samples and polynomial in the dimensionality, and showcases real-time applications. 5.1. Kernel Distance The rest of this section is concerned with measuring the numerical accuracy of the filter. Ideally, we would like our One conceivable error metric is the squared L2-norm be- process to mimic a Gaussian blur as much as possible. De- tween K~x (or Z~x) and an isotropic Gaussian kernel, where the Gaussian kernel has the same total variance as K (or Z ). note by K~x the convolution kernel used to evalute the output ~x ~x at ~x, i.e. using the terminology in Figure 6, This measure is the continuous equivalent of the Frobenius norm for matrices, and bounds from above the relative er- ′′ I (~x) = K~x(~y) I(~x +~y)d~y, ror on arbitrary input signals. As the kernels depend on ~x, ZH · d the L2-norm should be power-averaged over all possible ~x. assuming I has already been rotated into Hd . (See the last It should also be normalized by the power of the Gaussian 10 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering

kernel. We can consider this to be the inverse of the “signal- to-noise ratio” (SNR) where K~x is the observed signal and the Gaussian is the ground truth. Recall that SNR is conven- tionally defined as, Power of the signal . Power of the noise = observation - signal Our definition is fully analogous, and is formalized below: Definition 5.3. Given a family of kernels f : Rd R ~x → parametrized by ~x Rd that approximates a Gaussian blur, its signal-to-noise ratio∈ (SNR) is 2 Rd N(~y;θ) d~y η := , R θ 2 E~x Rd ( f~x(~y ~x) N(~y ~x; )) d~y R − − − θ h i where is the expected variance of f~x over all ~x. Also, it is Rd assumed that f~x is normalized to sum to 1 over .

Lemma 5.4. The expected variance of Z~x where ~x is chosen Rd randomly in is 5d/6. The expected variance of K~x where 2 ~x is chosen randomly in Hd is 2d(d + 1) /3. Proof. For Zd, the multi-linear splatting and slicing are simply the d-dimensional tent functions with variance d/6. The 1 1 1 d blur stage has variance d ( 1)2 + (0)2 + (12) = . · 4 − 2 4 2 Hence the total variance is (d/6) 2 + (d/2) = 5d/6. · ∗ For Ad , Proposition 4.6 tells us that the variance of the splatting and slicing step is d(d + 1)2/12 each. The variance of the blurring step is, 1 1 1 (d + 1) ( d(d + 1))2 + (0)2 + ( d(d + 1))2 , · 4 − 2 4 p p which equals d(d + 1)2/2. Together, the total expected variance is 2d(d + 1)2/3. The term expected variance is used because the actual variance of a particular kernel Z~x or K~x depend on ~x, but the additive nature of the variances of each step is clear from the proofs of Propositions 5.1 and 5.2. Note that the expected variance is the sum of variance over all dimensions, so it should be divided by d to yield the variance of a comparable isotropic multi-variate Gaussian.

The calculation of SNR for Z~x and K~x are given in Ap- pendix B and Appendix C, respectively. For Z~x, a closed form as a function of the dimensionality d can be derived; for K~x, a recipe for numerically estimating the SNR is pro-

0.15 vided. 0.1 The SNRs for the two kernels are plotted in Figure 8. The 0.05 SNRs of the two lattices are comparable, with K~x outper- 0 6 −6 4 forming Z~x at low dimensionality and Z~x outperforming K~x −4 2 −2 at higher dimensionality. This is explained by the fact that a 0 0 d −2 2 sample in Z is splatted to exponentially many lattice points, 4 −4 ∗ 6 −6 whereas in Ad it is splatted only to few points (d + 1). We Figure 7: Visualization of convolution kernels for each step also compare the SNR with that of true Gaussian kernels that in Gaussian ﬁltering for d = 2. Top: the splatting or slicing are truncated at certain distances from the origin. 0 1 2 kernel Ks. Next three: the three blur kernels Kb ,Kb ,Kb . Bot- tom: the overall kernel for ~x = 0. All ﬁgures have the same 6. Conclusion scale except for the bottommost plot. We have demonstrated interesting structural properties of the permutohedral lattice and used them to argue that the permutohedral lattice represents an appropriate choice of data J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering 11

[Sam69] SAMELSON H.: Notes on Lie Algebras. Van Nostrand Reinhold Company, 1969. Trunc. Gaussian (@ 1.5 std) Trunc. Gaussian (@ 2.0 std) [Ser87] SERRE J.-P.: Complex Semisimple Lie Algebras. Trunc. Gaussian (@ 2.5 std) Springer-Verlag, 1987. Trunc. Gaussian (@ 3.0 std) Z x [Vor52] VORONOI G. F.: Studies of Primitive Parallelotopes, K x vol. 2. Ukrainian Academy of Sciences Press, 1952, pp. 239– 30 368.

[YDGD03] YANG C., DURAISWAMI R., GUMEROV N.A., 25 DAVIS L.: Improved fast gauss transform and efﬁcient kernel density estimation. In Proc. ICCV ’03 (2003), vol. 1, pp. 664– 20 671.

15 Appendix A: Useful Lemmas SNR (dB) 10 The appendix contains proofs of several miscellaneous facts about simplices, the Gaussian distribution and the permuto- 5 hedral lattice, used to support Appendix B and C.

0 Lemma A.1. Let be a d-dimensional simplex defined 1 2 3 4 5 6 7 8 9 10 S Dimensionality by vertices ~s0,...,~sd, and let B0(~x),B1(~x),...,Bd(~x) be the barycentric coordinates of ~x for the vertices in the given order. Then, k, j 0,...,d , Figure 8: The signal-to-noise ratio of the kernels K~x and ∀ ∈ { } 2 Z~x for d = 1,...,10, along with truncated Gaussian kernels. , k = j, Note that K exhibits SNR that is similar to that of Z , de- S Bk(~x)B j(~x)d~x (d + 1)(d + 2) ~x ~x R =  1 spite touching only polynomially-many vertices; Z touches S d~x , k = j. ~x  exponentially many vertices. Because of this difference, the R (d + 1)(d + 2) 6 SNR for K suffers slightly more when d grows high.  ~x Proof. Let P be the face defined by all vertices except ~sk. Then is a pyramid with base P and apex ~s . Let h be the S k height of the altitude and let~y be the foot of the altitude. Con- structure for performing Gaussian filtering. We have also de- sider slicing along a hyperplane parallel to P that contains S scribed how to efficiently perform Gaussian filtering on the (1 t) ~s +t ~y, for some t [0,1]. The slice will be similar − · k · ∈ permutohedral lattice, and its numerical stability relative to to P with a scale factor of t, and by definition of barycentric that of a regular lattice. coordinates, Bk( ) will evaluate to a constant, namely 1 t, across the whole· slice. Therefore, − References 1 B (~x)2d~x = (1 t)2d~x(h dt) Z k Z Z [ABD10] ADAMS A.,BAEK J.,DAVIS M. A.: High-dimensional S 0 (1−t)~sk+tP − · filtering with the permutohedral lattice. To appear, Eurographics, 1 2010. = (1 t)2 d~x (h dt) Z0 − ZtP · [AGDL09] ADAMS A., GELFAND N., DOLSON J.,LEVOY M.: 1 Gaussian kd-trees for fast high-dimensional filtering. ACM 2 d−1 Transactions on Graphics (Proc. SIGGRAPH ’09) (2009), 1–12. = (1 t) t Vol(P)h dt Z0 − · [CPD07] CHEN J.,PARIS S.,DURAND F.: Real-time edge-aware 1 image processing with the bilateral grid. ACM Transactions on = Vol(P) h (1 t)2td−1dt Graphics (Proc. SIGGRAPH ’07) (2007), 103. · · Z0 − 2 [CS98] CONWAY J. H.,SLOANE N. J. A.: Sphere Packings, Lat- = Vol(P) h . (A.1) tices and Groups. Springer, 1998. · · d(d + 1)(d + 2) [Dol07] DOLBILIN N. P.: Minkowski’s theorems on parallelohe- Note that the same integral without the weight term (1 t)2 dra and their generalizations. Russian Mathematical Surveys 62 − (2007), 793–795. will simply yield the volume of : S [GS91] GREENGARD L. F., STRAIN J. A.: The fast gauss trans- 1 1 form. SIAM Journal on Scientific and Statistical Computing 12 d~x(h dt) = Vol(P) h td−1dt (1991), 79–94. Z0 ZtP · · · Z0 1 [MP92] MOOD R. V., PATERA J.: Voronoi and delaunay cells of = Vol(P) h . (A.2) root lattices: classification of their faces and facets by coxeter- · · d dynkin diagrams. Journal of Physics A: Mathematical and Gen- Dividing (A.1) by (A.2) yields the first of the two cases. eral 25 (1992), 5089–5134. When k = j, we consider applying the same decomposition [PD09] PARIS S., DURAND F.: A fast approximation of the bi- to P: let 6P′ be the face of P spanned by all vertices of ex- lateral filter using a signal processing approach. Int. J. Comput. S′ Vision 81, 1 (2009), 24–52. cept ~sk and ~s j. Then P is again a pyramid with base P and height h′, and y′ the foot of the altitude. Each slice of P par- [Rys98] RYSHKOV S. S.: The structure of primitive parallelohe- ′ ~′ dra and voronoi’s last problem. Russian Mathematical Surveys allel to P cutting the altitude at (1 r) ~s j + r y is again a ′ − · · 53 (1998), 403–405. copy of P scaled by r, and the barycentric weight B j( ) is · 12 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering uniformly 1 r. Therefore, The next pair of lemmas illustrate how to integrate arbi- − 1 1 trary functions over the canonical simplex with particular Bk(~x)B j(~x)d~x = (1 t) t(1 r) weighting schemes. ZS Z0 Z0 − · − Vol(P′)td−1rd−2 (h′ dr)(h dt) Lemma A.4. Let X be the union of Delaunay cells contain- ∗ R · · · ing the origin in Ad , and let f : X be a function. Then, 1 1 → = hh′Vol(P′) td td+1dt rd−2 rd−1dr Z Z 1 0 − 0 − f (~x) Ks(~x)d~x = f T((d + 1)~x) d~x. 1 ZX · (d + 1)3/2 Z[0,1]d+1 = hh′Vol(P′) . (A.3) (d + 1)(d + 2)(d 1)d − Proof. This is a direct consequence of Proposition 4.4. Without the barycentric weights, In short, weighted sampling corresponding to Ks can be 1 1 achieved by uniformly sampling in the unit cube and pro- ′ d−1 d−2 ′ d~x = Vol(P )t r (h dr)(h dt) jecting the sample onto Hd . ZS Z0 Z0 · · 1 1 ∗ R ∗ ′ ′ d−1 d−2 Lemma A.5. Let f : Ad be a function on Ad . Let b : = hh Vol(P ) t dt r dr ∗ → Z0 Z0 A R and the vectors w~ be as in Section 4.2. Then, d → k 1 = hh′Vol(P′) . (A.4) d(d 1) 1 ∑ f (~q)b(~q) = ∑ f ∑tk w~k ∏ . − 1+|tk| ∗ d+1 · 2 Dividing (A.3)by(A.4) yields the second of the two cases, ~q∈Ad ~t∈{−1,0,1} k ! k as desired. ∗ Proof. The blur kernel b(~x) is given by the convolution of Lemma A.2. Let ~x,~y Ad . Then the number of Delaunay 0 d ∈ Kb ,...,Kb . Therefore, it is the accumulation of the weights cells containing both ~x and ~y is given by corresponding to offsets in the d + 1 axes that sum to ~x. The if ~x ~y is some permutation of the weight equals 1 if the offset t in a given axis is zero, and 1 − 2 k 4 (d + 1 k)!k!, remainder-k vertex of the canonical 1  − if it is tk = 1. Hence we can write it succinctly as . simplex, 1+|tk|  ± 2 0, otherwise. The claim follows. Proof. Since every Delaunay cell is given by permuting the  coordinates of the canonical simplex and translating the Appendix B: Evaluation of SNR for Zd vertices, if ~x ~y cannot be written as a permutation of a − vertex of the canonical simplex, then they must not belong The kernel Z~x repeats periodically because of the transla- to any common Delaunay cell. So we can assume that ~x ~y tional symmetry of Zd. In order to evaluate the expected − is a permutation of some vertex of the canonical simplex. value of an expression involving Z~x, it suffices to consider ~x [0,1]d. In other words, ∈ WLOG let ~x = 0. Then any simplex containing ~x has a cor- 2 Rd N(~y; 5/6) d~y responding permutation in Sd+1. Among those, the number η = R 2 . of simplices also containing~y is the number of permutations d Rd (Z (~y ~x) N(~y ~x; 5/6)) d~yd~x [0,1] ~x − − − that sends the remainder-k vertex of the canonical simplex R R to ~y. Because ~y has d + 1 k coordinates of the same value, Expanding the quadratic term in the integrand, we obtain − (B.2) and the remaining k coordinates also have the same value, η = , containing the following three (B.1) (B.2) 2(B.3) the number of possible permutations is (d + 1 k)!k!. + − expressions: − Lemma A.3. The power of a d-dimensional Gaussian kernel θ πθ −d/2 2 with covariance I is (4 ) . Z~x(~y ~x) d~yd~x, (B.1) Z[0,1]d ZRd − Proof. N(~y; 5/6)2d~y, (B.2) ∞ d ZRd N(~y; θ)2dy = N(y; θ)2dy ZRd Z −∞ Z~x(~y ~x)N(~y ~x; 5/6)d~yd~x. (B.3) Z[0,1]d ZRd − − 2 d 2 ∞ exp y /(2θ) − = dy π −d/2 Z−∞ 2πθ  It follows from Lemma A.3 that (B.2) is (10 /3) .   (B.1) is similarly separable:   d 2 θ ∞ exp y / 2 − · 2 2 = dy Z~x(~y ~x) d~yd~x Z−∞ π θ πθ  Z[0,1]d ZRd − 2 2 √4 2  q d  d 1 ∞ θ = ∏ ∑ Bp(xi)Bq(yi)b(p q) d~yd~x = N(y; /2)dy Z[0,1]d ZRd − √4πθ Z−∞ "i=1 p,q∈Z # 2 d = (4πθ)−d/2, = ∑ Bp(x)Bq(y)b(p q) dydx Z ZR as desired. " [0,1] "p,q∈Z − # # J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering 13

Lemma C.1. = ∑ Bp(x)Bp′ (x)Bq(y)Bq′ (y) Z ZR " [0,1] p p′ q q′ Z d 1 2 , , , ∈ K (~y ~x)d~y = (d + 1) − / . Z ~x d Hd − b(p q)b(p′ q′)dydx · − − # Proof.

′ ′ K (~y ~x)d~y = ∑ B (~x)B (~y)b(~p ~q)d~y = ∑ b(p q)b(p q ) Z ~x Z ~p ~q ′ ′ − − Hd − Hd ∗ − " p,p ,q,q ∈Z ~p,~q∈Ad d by Proposition 5.1, Bq(y)Bq′ (y)dy Bp(x)Bp′ (x)dx . · ZR Z[0,1] # = ∑ B (~x) B (~y)d~y b(~p ~q) ~p Z ~q ∗ Hd − ~p,~q∈Ad Note that Bp(x) and Bp′ (x) are both nonzero only if d−1/2 ′ ′ = ∑ B~p(~x)(d + 1) b(~p ~q) p = p or p p = 1. In the former case, the integral of ∗ − | − | ~p,~q∈Ad Bp(x)Bp′ (x) with respect to x over R is 2/3; in the latter by Corollary 4.5, case, 1/6. Simiarly, Bq(y) and Bq′ (y) are both nonzero over ′ ′ y [0,1] iff q,q = 0 or 1. Setting q = q yields 1/3 for the = ∑ B (~x)(d + 1)d−1/2 ∈ ′ ~p value of the integral, and q = q yields 1/6. Summing over ~p∈A∗ 6 d all possible values of p′,q,q′ with nonzero summands, we since ∑ b(~p ~q) = 1, obtain that (B.1) equals ∗ − ~q∈Ad d d b(p)2 4b(p)b(p 1) b(p)b(p 2) 29 = (d + 1)d−1/2. ∑ + − + − = . " p Z 2 9 18 # 96 ∈ The last line follows because the sum of splatting weights The cross term (B.3) remains. We can similarly expand for~x over all lattice points is also 1. the expression and collect product terms into the product of K~x, like the lattice, is translation-invariant and symmetric sums, and exploit the fact that only few values of p,q yield with respect to permutation, so it sufﬁces to iterate ~x over nonzero summands. Unfortunately, because of the Gaussian the canonical simplex, denoted by C∗, in order to compute term, there is no succinct analytic expression. the expected value of the L2 distance. Recall from Proposi- tion 2.11 that the volume of C∗ is given by (d +1)d−1/2/d!. (B.3) = Z~x(~y ~x)N(~y ~x; 5/6)d~yd~x Z[0,1]d ZRd − − Therefore, 1 d (d + 1)d−1/2/d! N(~y; 2(d + 1)2/3)2d~y = ∏ ∑ Bp(xi)Bq(yi) Hd Z ZRd η = . 0 i=1 p,q∈Z R 2 K~x(~y−~x) 2 ∗ − N(~y ~x; 2(d + 1) /3) d~yd~x C Hd (d+1)d 1/2 b(p q)N(yi xi; 5/6)d~yd~x R R − − · − − 1 = ∑ Bp(x)Bq(y) (C.2) Z0 ZR As before, η can be expressed as , p,q∈Z (C.1) + (C.2) 2(C.3) d where the components are as follows: − b(p q)N(y x; 5/6)dydx · − − d! 2 K~x(~y ~x) d~yd~x, (C.1) 1 (d + 1)3d−3/2 ZC∗ ZH − = ∑ (1 x)b(q) + xb(q 1) d Z ZR { − − } q∈{−1,0,1,2} 0 N(~y; 2(d + 1)2/3)2d~y, (C.2) Z d Hd d! Bq(y)N(y x; 5/6)dydx K (~y ~x) · − 2d−1 Z ∗ Z ~x (d + 1) C Hd − 0.30456. 2 ≃ N(~y ~x; 2(d + 1) /3)d~yd~x. (C.3) · −

Corollary B.1. The SNR term for Zd is −d/2 8(d + 1)2π d By Lemma A.3,(C.2) equals . 3/(10π) 3 ! η = d d (29/96) + p3/(10π) 2 0.30456d Let us compute (C.1) without the constant term. The inte- − · grand may be expanded and be simpliﬁed as follows: p ∗ Appendix C: Evaluation of SNR for Ad K (~y ~x)2d~yd~x Z ∗ Z ~x ∗ Zd C Hd − The error analysis for Ad is analogous to that of , with 2 a few subtle changes. First, K~x does not sum to 1 over its domain, as seen in Lemma C.1 and should be normalized = ∑ Bp(~x)Bq(~y)b(~p ~q) d~yd~x Z ∗ Z ~ ~ C Hd  ∗ −  before being compared to a Gaussian. ~p,~q∈Ad   14 J. Baek, & A. Adams / Some Useful Properties of the Permutohedral Lattice for Gaussian Filtering

= ∑ B~p1 (~x)B~p2 (~x)Bq~1 (~y)Bq~2 (~y) ZC∗ ZH d ~p1,~p2, ∗ q~1,~q2∈Ad

b(p~1 q~1)b(p~2 q~2)d~yd~x · − −

= ∑b(q~1)b(q~2) Bp (~x)Bp (~x)d~x Z ∗ ~1 ~2 ... C B (~y)B (~y)d~y d~yd~x, · Z ~p1−q~1 ~p2−q~2 Hd via change of variable ~qi ~pi ~qi. ← − Our expression is a weighted sum with weights b(q~1)b(q~2). By Lemma A.5, it is equivalent to 1 1 ∑ ∑ [ ][ ]∏ , (C.4) 1+|tk | 1+|sk| ∗ d 1 ··· ··· 2 2 p~1,~p2∈C ~t∈{−1,0,1} + k ~s∈{−1,0,1}d+1 where q~1 = ∑t w~ and q~2 = ∑s w~ . k · k k · k Note that the ﬁrst bracketed term is effectively given by Lemma A.1. The second term is similar, but the integral is over Hd, so we must additively consider each simplex whose vertex contains both p~1 q~1 and p~2 q~2. The number of such simplices is given by− Lemma A.2−. (C.4) is more attractive than its original form because its summands are now simple numerical quantities. There is no succinct analytic expression for the sum, but it can be computed numerically in a straightforward manner. The number of summands is (d + 1)2 32(d+1). · That leaves us with (C.3). Once again, the integral is expanded and manipulated:

K (~y ~x)N(~y ~x; 2(d + 1)2/3)d~yd~x Z ∗ Z ~x C Hd − −

= ∑ Bp(~x)Bq(~y) Z ∗ Z ~ ~ ∗ C Hd ~p,~q∈Ad b(~p ~q)N(~y ~x;...)d~yd~x · − − Using Lemma A.5, we obtain

∑ ∑ B (~x)B (~y) ∗ ~p ~p+~q ∗ Z Z ~p∈C ~t∈{−1,0,1}d+1 C Hd 1 ∏ N(~y ~x;...)d~yd~x, 1+|tk| · k 2 − ∗ where ~q = ∑tk w~k. By symmetry, we can swap out C with the union of Delaunay· cells containing the origin. Then we have an integral weighted by B~p(~x) and B~p+~q(~y), which can be computed by Monte-Carlo sampling the (d + 1)- dimensional unit cube, by Lemma A.4. In summary, we iterate over ~p C∗ and~t 1,0,1 d+1, and for each pair ∈ ∈ {− } of vectors, sample ~x and ~y via Lemma A.4, and evaluate the 1 Gaussian of ~y ~x, weighted by ∏k . Multiplying the − 21+|tk| result by an appropriate constant yields (C.4).