Sifting Convolution on the Sphere Patrick J

Sifting Convolution on the Sphere Patrick J. Roddy∗ and Jason D. McEwen∗ ∗ Mullard Space Science Laboratory (MSSL), University College London (UCL), Surrey RH5 6NT, UK [email protected], [email protected]

Abstract—A novel spherical convolution is defined when both inputs are directional. Moreover, the convolution through the sifting property of the Dirac delta on the is efficient to compute, and is commutative up to a complex sphere. The so-called sifting convolution is defined by conjugate. the inner product of one function with a translated version of another, but with the adoption of an alter- The remainder of this letter is as follows. Section II native translation operator on the sphere. This trans- includes some mathematical preliminaries and reviews lation operator follows by analogy with the Euclidean existing spherical convolutions in the literature. Section III translation when viewed in harmonic space. The sifting introduces the proposed sifting convolution. Section IV convolution satisfies a variety of desirable properties presents a demonstration of the convolution with a direc- that are lacking in alternate definitions, namely: it supports directional kernels; it has an output which tional kernel. Lastly, Section V sets out some concluding remains on the sphere; and is efficient to compute. An remarks. illustration of the sifting convolution on a topographic map of the Earth demonstrates that it supports directional kernels to perform anisotropic filtering, while its II. Mathematical Background and Problem output remains on the sphere. Formulation Index Terms—Convolution, 2-sphere, spherical harmonics. A. Mathematical Preliminaries 1) Signals on the Sphere: Consider a complex valued I. Introduction square-integrable function f(ω) on the 2-sphere 2 = {ω ∈ ANY fields in science and engineering measure S 3 : kωk = 1}. Here ω = (θ, φ) parameterise a point on data on spherical manifolds, such as computer R M the unit sphere, where θ ∈ [0, π] is the colatitude and graphics [1], planetary science [2], geophysics [3], quantum φ ∈ [0, 2π) is the longitude. The functions f(ω) form the chemistry [4], cosmology [5], and computer vision [6], [7], Hilbert Space L2( 2). The complex inner product induces [8]. Possible extensions to signal processing techniques S a norm kfk = phf|fi. Signals on the sphere are functions developed in the Euclidean domain may be transferred with a finite induced norm. to the spherical domain. The convolution is an important 2) Spherical Harmonics: The spherical harmonics are signal processing technique between two signals defined on the complete orthonormal set of basis functions of the the 2-sphere, which is central to filtering — an integral Hilbert space L2( 2). By the completeness of spherical part of spherical analyses. S harmonics, any f ∈ L2( 2) may be decomposed as Many definitions of spherical convolutions exist in the S

literature. A spherical convolution operator would ideally ∞ ` exhibit a variety of desirable properties — such a spherical X X f(ω) = f`mY`m(ω), (1) convolution would accept directional inputs (i.e. functions `=0 m=−` that are not invariant under azimuthal rotation), whilst arXiv:2007.12153v2 [cs.IT] 29 Oct 2020 having the output remain on the sphere. Moreover, the where f`m are the spherical harmonic coefficients given convolution would be efficient to compute. Existing con- by f`m = hf|Y`mi. The phase convention adopted here is ∗ m ∗ m volutions such as the isotropic convolution (e.g.[9], [10], Y`m = (−1) Y`(−m) such that f`m = (−1) f`(−m) for a [11]) and the left convolution [11], [12] restrict themselves real field. One often considers signals on the sphere with a to an axisymmetric kernel (i.e. kernels that are invariant bandlimit of L, i.e. signals such that f`m = 0, ∀` ≥ L; and P PL−1 P` under azimuthal rotation). The directional convolution has adopts the shorthand notation `m = `=0 m=−`. an output which is not on the sphere (e.g.[9], [13]). Lastly, 3) Dirac Delta: The Dirac delta on the sphere sat- the commutative anisotropic convolution [14], [15] and the isfies the following normalisation and sifting properties, directional convolution are computationally demanding. No R ∗ 0 respectively: 2 dΩ(ω) δ(ω) = 1 and hδω0 |f i = f(ω ), existing spherical convolution satisfies all three desirable S where δω0 (ω) represents the Dirac delta rotated to some properties. ω0 = (θ0, φ0). The harmonic expansion of the Dirac delta is This letter presents an alternative spherical convolution, the sifting convolution, defined through the sifting property X ∗ 0 δω0 (ω) = Y`m(ω )Y`m(ω), (2) of the Dirac delta — in analogy to the Euclidean definition. `m The convolution is anisotropic in nature and supports directional kernels. The output remains on the sphere, even which follows trivially by the sifting property. 2

4) Rotation of a Signal on the 2-Sphere: The Euler 2) Left Convolution: The definition of the left convolu- angles may parameterise three-dimensional rotations with tion [11], [12] in real space is ρ = (α, β, γ) ∈ SO(3), where α ∈ [0, 2π), β ∈ [0, π], Z −1 and γ ∈ [0, 2π). The rotation operator Rρ consists of (f g)(ω) = dρ(ρ) f(ρη)g(ρ ω), (7) the sequence of rotations: (i) γ rotation about the z-axis; SO(3) (ii) β rotation about the y-axis; and (iii) α rotation about the z-axis. The rotation of a function on the sphere is where η is the north pole, and dρ(ρ) = sin β dα dβ dγ −1 defined by (Rρf)(ω) = f(Rρ ω), where Rρ is the three- is the usual invariant measure on SO(3). The harmonic dimensional rotation matrix corresponding to Rp. The representation of this convolution is spherical harmonic coefficients of a rotated function read r 4π (f g)`m = 2π f`mg`0, (8) ` 2` + 1 X ` (R f) = D 0 (ρ)f 0 , (3) ρ `m m m `m As the harmonic representations suggest, the isotropic and 0 m =−` left convolutions are closely related, as elaborated in [11]. ` Hence, the properties are similar. The left convolution has where Dm0m(ρ) are Wigner D matrices which form the 2` + 1-dimensional representation of the rotation group for the following properties: (i) does not support directional a given `. kernels since g(ω) must be axisymmetric; (ii) an output which remains on the sphere; and (iii) efficient computation since it is a product in harmonic space. B. Spherical Convolutions 3) Directional Convolution: Rotations on the sphere are the spherical counterpart of translations in the Euclidean The conventional convolution between two functions on domain in real space. Hence, the standard directional n two-dimensional Euclidean space R is convolution is Z Z ∗ (f ? g)(x) = dy f(x − y)g(y), (4) (f ~ g)(ρ) = dΩ(ω) f(ω)(Rρg) (ω). (9)

2 2 R S Upon expanding in harmonic space, this becomes (e.g.[9], where x, y ∈ Rn. The convolution is commutative f ? g = g ? f. A spherical counterpart of the convolution [13]) is required for functions defined on the sphere. Alternative ` X X ` ∗ definitions of such a convolution exist in the literature but, (f ~ g)(ρ) = f`m Dm0m(ρ)g`m0 , (10) while already useful, lack certain desirable properties. `m m0=−` The properties desired in the spherical extension of the and hence, the output is on SO(3). Fast algorithms exist [9], convolution include: (i) the support of directional kernels; [13], [17], [18] but the convolution remains less efficient than (ii) an output which remains on the sphere; and (iii) efficient a spherical harmonic transform. The directional convolution computation. A convolution is considered computationally has the following properties: (i) does support directional efficient here if its computational cost is no greater than kernels; (ii) an output which does not remain on the sphere the cost of fast spherical harmonic transforms, i.e. O(L3) due to the 3D rotation of the kernel; and (iii) expensive (e.g.[12], [16]). Formulations of spherical convolutions computation. exist in the literature but none satisfy all these properties. 4) Commutative Anisotropic Convolution: The definition A summary of existing spherical convolutions and their of the commutative anisotropic convolution [14], [15] is properties follows. Z 1) Isotropic Convolution: In real space the isotropic (f ⊕ g)(ω) = dΩ(ω0)(R f)(ω0)g(ω0), (11) convolution (e.g. [9], [10], [11]) is (φ,θ,π−φ) 2 S Z 0 0 ∗ 0 which on expansion reads (f g)(ω) = dΩ(ω ) f(ω )(Rωg) (ω ), (5)

2 ` S X X ` ∗ (f ⊕ g)(ω) = Dm0m(φ, θ, π − φ)f`m0 g`m. (12) which in harmonic space becomes (e.g. [9]) `m m0=−`

r 4π The limitation here is that one must specify the initial (f g) = f g∗ . (6) `m 2` + 1 `m `0 rotation as γ = π − α in order for the convolution to be commutative. The complexity of the convolution is The isotropic convolution has the following properties: O(L3 log L), and hence, it is less eﬃcient than a spherical (i) does not support directional kernels since g(ω) must harmonic transform. The convolution has the following be axisymmetric; (ii) an output which remains on the properties: (i) it supports directional kernels; (ii) an sphere; and (iii) eﬃcient computation since it is a product output which remains on the sphere; and (iii) expensive in harmonic space. computation. 3

TABLE I B. Convolution Operator Properties of spherical convolutions. With a translation operator to hand, one may define the 2 2 2 sifting convolution on the sphere of f, g ∈ L ( ) in the Anisotropic S Output Efficient S usual manner by the inner product Isotropic Left (f } g)(ω) ≡ hTωf|gi , (16) Directional noting the use of the alternative translation operator Commutative Anisotropic defined in Section III-A. Sifting (this work) In harmonic space this simplifies to the product ∗ (f } g)`m = f`mg`m, (17) C. Problem Formulation as Table I presents a summary of the spherical convolutions (f g)(ω) = hT f|gi discussed and their properties. No existing definition of } ω Z a spherical convolution has all the desired properties dis- 0 0 ∗ 0 = dΩ(ω )(Tωf)(ω )g (ω ) cussed in Section II-B. In this work, the sifting convolution, 2 which satisfies all desirable properties, is presented. SZ 0 X 0 X ∗ ∗ 0 = dΩ(ω ) f`mY`m(ω )Y`m(ω) g`0m0 Y`0m0 (ω ) 0 0 2 `m ` m III. Sifting Convolution S X ∗ This work defines the sifting convolution which permits = f`mg`mY`m(ω). (18) directional kernels, whose output remains on the sphere and `m is efficient to compute. Moreover, it is commutative up to a Since the harmonic representation of the convolution is complex conjugate. The sifting convolution is constructed simply a product (again by analogy with the harmonic using a novel translation operator defined on the sphere. representation of the Euclidean convolution), it is efficient to compute. Note that harmonic multiplication has been A. Translation Operator considered before [11]; although it has been used here to define a new anisotropic convolution operator, introducing In real space, the rotation operator on the sphere is the a conjugation and elaborating a real space interpretation. usual analogue of the translation operator in the Euclidean setting. One may define an alternative operator, T , which ω C. Translation Interpretation follows as the analogue of the Euclidean setting but in harmonic space. This translation is in contrast to the One may show that the translation operator is simply a standard rotation as it considers two angles rather than (sifting) convolution of a function with the shifted Dirac three and thereby its output remains on the sphere. In delta function: practice the translation operator is defined as a product X 0 (f } δω0 )(ω) = f`mY`m(ω )Y`m(ω) of basis functions. In the Euclidean setting, e.g. R, the `m complex exponentials φ (x) = exp{iux}, with x, u ∈ u R = (Tω0 f)(ω), (19) form the standard orthonormal basis. A shift of coordinates 0 by noting Eq. (17) and where the final equality follows by defines the translation of the basis functions: φu(x + x ) = 0 0 Eq. (14). The sifting convolution and translation are thus φu(x )φu(x), with x ∈ R and where the final equality follows by the standard rule for exponents. The definition natural analogues of the respective operators defined in of the translation of the spherical harmonics on the sphere Euclidean space. follows by analogy with the representation as a product of basis functions: D. Properties

0 The sifting convolution has all the desired properties (T 0 Y )(ω) ≡ Y (ω )Y (ω), (13) ω `m `m `m discussed in Section II-B, namely, the convolution accepts where ω0 = (θ0, φ0). directional inputs, has an output which remains on the This leads to a natural harmonic expression for the sphere, and is eﬃcient to compute. Table I summarises the translation of a general arbitrary function f ∈ L2(S2) properties of the sifting convolution and compares them to the properties of alternative spherical convolutions. X 0 (Tω0 f)(ω) = f`mY`m(ω )Y`m(ω), (14) The translation preserves symmetries, which means that `m any symmetry that exists in the initial kernel deﬁnition will implying be present after the translation. Thus, one must be careful 0 when choosing a kernel for a convolution to ensure it has the (Tω0 f) = f`mY`m(ω ). (15) `m desired properties when translated, e.g. spatial localisation. This translation operator is considered further in Sec- To perform anisotropic smoothing that is localised (the tion III-C to build greater intuition. usual interpretation), the translated kernel also needs to be 4

1 1 1

0 0 0 Fig. 2. EGM2008 dataset centred on a view of South America (a) Re{(Tω0 fA)(ω)} (b) Re{(Tω0 fB )(ω)} (bandlimited at L = 128). The colour is between zero and one, Fig. 1. A harmonic Gaussian translated to some ω0 = (θ0, φ0) reflecting the scaled intensity of the field. (bandlimited at L = 128). Panel (a) corresponds to a more elongated 2 1 kernel fA, where (σ`, σm) = (10 , 10 ); whereas panel (b) corresponds 1 1 to a more symmetric kernel fB , where (σ`, σm) = (10 , 10 ). The colour is between zero and one, reflecting the scaled intensity of the 1 1 field.

localised. If the kernel has, say, even azimuthal symmetry, when it is translated to ω0 = (θ0, φ0) it will have a localised component both at φ0 and −φ0. While this is not a problem per se, for the usual interpretation of smoothing one would 0 desire a localised component at φ only. This can be 0 0 achieved by ensuring the original kernel does not exhibit a (a) Re{(fA } g)(ω)} (b) Re{(fB } g)(ω)} symmetry that would lead to multiple localised components Fig. 3. The real part of the sifting convolution between the EGM2008 once translated. The harmonic Gaussian introduced in dataset and the harmonic Gaussian, rotated to view of South America (bandlimited at L = 128). Panel (a) corresponds to a more elongated Section IV satisfies the desired property. 2 1 kernel fA, where (σ`, σm) = (10 , 10 ); whereas panel (b) corresponds 1 1 to a more symmetric kernel fB , where (σ`, σm) = (10 , 10 ). As IV. Numerical Illustration expected, the resultant sifting convolved Earth map exhibits greater anisotropic smoothing in panel (a) than in panel (b). It is clear that the This section demonstrates the effect of the sifting sifting convolution supports directional kernels to perform anisotropic convolution through the application of a directional kernel filtering (smoothing), while the output remains on the sphere. The to an example signal on the sphere. colour is between zero and one, reflecting the scaled intensity of the Define the harmonic Gaussian as a two-dimensional field. Gaussian in harmonic space by 2 2 ` m symmetric kernel, as shown in the right panel. It is clear f`m = exp − 2 + 2 . (20) 2σ` 2σm that the sifting convolution supports directional kernels to In effect, this function is the standard axisymmetric perform anisotropic filtering (smoothing), while the output Gaussian in ` modulated by a Gaussian in m. Note this remains on the sphere. function is not real — if required one can define only the positive m components and impose reality by the conjugate symmetry relationship in harmonic space. The function is V. Conclusion directional, and hence, is useful in illustrating the effect of the sifting convolution on the sphere. All later computations use the SSHT1 code [16]. This work presents the sifting convolution on the sphere Consider two differently sized harmonic Gaussians on and demonstrates its application. The convolution accepts the sphere to see the effect on the sifting convolution. Fig. 1 directional functions as inputs, has an output which shows both an elongated (left panel) and symmetric (right remains on the sphere, and is efficient to compute. The panel) translated harmonic Gaussian. sifting convolution is defined in the usual manner through To study the effect of the sifting convolution, consider the inner product but with an alternative translation the Earth Gravitational Model EGM2008 dataset [19]. operator on the sphere. This follows by analogy with This dataset is the topographic map of the Earth. Fig. 2 the Euclidean translation when viewed as a convolution presents the dataset up to an order of L = 128. The sifting with a shifted Dirac delta function. An illustration of convolution is then performed between the Earth represen- the sifting convolution on the topographic map of the tation and the harmonic Gaussian with the resultant plot Earth demonstrates that it supports directional kernels to given in Fig. 3. As expected, when the elongated kernel is perform anisotropic filtering, while its output remains on considered, as shown in the left panel, the result exhibits the sphere. Convolutions are an important part of signal greater anisotropic smoothing than when considering the processing techniques, hence, the sifting convolution can play an integral role in constructions of alternate spherical 1http://astro-informatics.github.io/ssht/ analysis techniques, which is the focus of future work. 5

References [1] R. Ramamoorthi and P. Hanrahan, “A Signal-Processing for Reflection,” ACM Trans. Graph., vol. 23, no. 4, pp. 1004–1042, 2004. [2] D. L. Turcotte, R. J. Willemann, and W. F. Haxby, “Role of Membrane Stresses in the Support of Planetary Topography,” J. Geophys. Res., vol. 86, pp. 3951–3959, 1981. [3] F. J. Simons, F. A. Dahlen, and M. A. Wieczorek, “Spatiospectral Concentration on a Sphere,” Soc. Ind. Appl. Math., vol. 48, no. 3, pp. 504–536, 2006. [4] C. H. Choi, J. Ivanic, M. S. Gordon, and K. Ruedenberg, “Rapid and Stable Determination of Rotation Matrices between Spherical Harmonics by Direct Recursion,” J. Chem. Phys., vol. 111, no. 19, pp. 8825–8831, 1999. [5] C. L. Bennett, A. J. Banday, K. M. Górski, G. Hinshaw, P. Jackson, P. Keegstra, A. Kogut, G. F. Smoot, D. T. Wilkinson, and E. L. Wright, “Four-Year COBE DMR Cosmic Microwave Background Observations: Maps and Basic Results,” Astrophys. J., vol. 464, no. 1, pp. L1–L4, 2002. [6] T. S. Cohen, M. Geiger, J. Koehler, and M. Welling, “Spherical CNNs,” jan 2018. [7] C. Esteves, C. Allen-Blanchette, A. Makadia, and K. Daniilidis, “Learning SO(3) Equivariant Representations with Spherical CNNs,” Int. J. Comput. Vis., vol. 128, no. 3, pp. 588–600, nov 2017. [8] O. J. Cobb, C. G. R. Wallis, A. N. Mavor-Parker, A. Marignier, M. A. Price, M. D’Avezac, and J. D. McEwen, “Efficient Generalized Spherical CNNs,” ArXiv:2010.11661, oct 2020. [9] J. D. McEwen, M. P. Hobson, D. J. Mortlock, and A. N. Lasenby, “Fast directional continuous spherical wavelet transform algorithms,” IEEE Trans. Signal Process., vol. 55, no. 2, pp. 520– 529, feb 2007. [10] L. Wei, R. A. Kennedy, and T. A. Lamahewa, “Quadratic Variational Framework for Signal Design on the 2-Sphere,” IEEE Trans. Signal Process., vol. 59, no. 11, pp. 5243–5252, nov 2011. [11] R. A. Kennedy, T. A. Lamahewa, and L. Wei, “On azimuthally symmetric 2-sphere convolution,” in Digit. Signal Process. A Rev. J., vol. 21, no. 5. Elsevier Inc., sep 2011, pp. 660–666. [12] J. R. Driscoll and D. M. Healy, “Computing Fourier Transforms and Convolutions on the 2-Sphere,” Adv. Appl. Math., vol. 15, no. 2, pp. 202–250, jun 1994. [13] B. D. Wandelt and K. M. Górski, “Fast convolution on the sphere,” Phys. Rev. D, vol. 63, no. 12, p. 123002, 2001. [14] P. Sadeghi, R. A. Kennedy, and Z. Khalid, “Commutative Anisotropic Convolution on the 2-Sphere,” IEEE Trans. Signal Process., vol. 60, no. 12, pp. 6697–6703, dec 2012. [15] Z. Khalid, R. A. Kennedy, and P. Sadeghi, “Efficient Computa- tion of Commutative Anisotropic Convolution on the 2-sphere,” in 2012 6th Int. Conf. Signal Process. Commun. Syst. IEEE, dec 2012, pp. 1–7. [16] J. D. McEwen and Y. Wiaux, “A Novel Sampling Theorem on the Sphere,” IEEE Trans. Signal Process., vol. 59, no. 12, pp. 5876–5887, dec 2011. [17] Y. Wiaux, J. D. McEwen, and P. Vielva, “Complex data processing: Fast wavelet analysis on the sphere,” in J. Fourier Anal. Appl., vol. 13, no. 4. Springer, aug 2007, pp. 477–493. [18] J. D. McEwen, P. Vandergheynst, and Y. Wiaux, “On the computation of directional scale-discretized wavelet transforms on the sphere,” in Wavelets Sparsity XV, D. Van De Ville, V. K. Goyal, and M. Papadakis, Eds., vol. 8858. SPIE, sep 2013. [19] N. K. Pavlis, S. A. Holmes, S. C. Kenyon, and J. K. Factor, “The development and evaluation of the Earth Gravitational Model 2008 (EGM2008),” J. Geophys. Res. Solid Earth, vol. 118, no. 5, p. 2633, 2013.