Fast Computation of 3D Spherical Fourier Harmonic Descriptors - A Complete Orthonormal Basis for a Rotational Invariant Representation of Three-Dimensional Objects Henrik Skibbe Marco Reisert Qing Wang, Olaf Ronneberger, Hans Burkhardt Dept. of Diagnostic Radiology, Medical Physics Department of Computer Science University Medical Center, Freiburg Center for Biological Signalling Studies (bioss) [email protected] Albert-Ludwigs-Universitat¨ Freiburg, Germany [email protected] Abstract a dense set of nested spheres is necessary to represent an object sufficiently. The descriptors representing different In this paper we propose to extend the well known spheres are then invariant to inner object rotations (see [6] spherical harmonic descriptors[6] (SHD) by adding an for examples.), which in many cases weakens the ability additional Fourier-like radial expansion to represent volumetric data. Having created an orthonormal basis of object discrimination. We can overcome this limitation on the ball with all the gentle properties known from by using an additional radial expansion as proposed for in- the spherical harmonics theory and Fourier theory, we stance in [7, 17, 2]. Experiments in [17] have already shown are able to compute efficiently a multi-scale represen- that the consideration of a radial expansion increases the tation of 3D objects that leads to highly discrimina- performance in object classification tasks. In contrast to tive rotation-invariant features, which will be called [7, 17, 2], we propose a radial expansion which is based spherical Fourier harmonic descriptors (SFHD). Exper- on a Fourier-like basis that has already been widely used iments on the challenging Princeton Shape Benchmark in a similar way for two-dimensional image representation (PSB[16]) demonstrate the superiority of SFHD over and feature generation. (Known as angular radial transfor- the ordinary SHD. mation, see e.g. [3, 12]). The transformation for volumetric data which we propose here has some very nice properties: 1. Introduction fast algorithms are available for both, the spherical harmon- With the increasing performance of modern computer ics transformation[8] and the Fourier transformation[4]. We systems the number of volumetric images derived from further can make use of the spherical tensor product[11] stereo vision systems, laser scanning devices or microscop- to preserve the phase information of expansion coefficients ical recordings has drastically increased in recent years. In in the final object descriptors. The descriptors will be many applications three-dimensional objects must be de- called spherical Fourier harmonic descriptors (SFHD). We scribed and classified in a space- and time-saving man- conduct experiments on the challenging Princeton Shape ner. These descriptions should be robust or invariant to Benchmark (PSB[16]) containing 1814 objects by compar- certain transformations, and discriminative. One possible ing the performance of ordinary SHD to our SFHD leading solution are object descriptors which rely on the idea of to very promising results. group integration, where certain features are averaged over The paper is organised in four further sections. Section 2 the whole group to become invariant [15, 9]. For the rota- gives a short introduction to the mathematical definitions tion group the spherical harmonics can directly be utilised and notations we use. In Section 3, we introduce an or- to obtain rotation invariant object representations in an an- thonormal basis system on the ball, which will be utilised alytical way, as shown e.g. in [6, 10, 13]. The major to derive rotation invariant descriptors from volumetric im- disadvantage of this approach is the restriction to spheri- age data. Our Experiments are discussed in section 4 and cal functions. This means, that the spherical harmonic de- our conclusion is given in section 5. scriptors for three-dimensional scalar fields are based on a function expansion done for an (theoretically) infinite set of nested spheres. Hence, from a practical point of view, 2. Preliminaries function f on the unit sphere can thus be expanded as: We assume that the reader has basic knowledge in the 1 m=` X X ` theory and notations of the harmonic analysis of SO(3), f(θ; φ) = b`mYm(θ; φ) meaning he or she should have knowledge both in spherical `=0 m=−` harmonics and in Wigner D-Matrices. The reader should 1 X T ` also know how and why we can obtain rotation invariant = (b`) Y (θ; φ) (6) features from spherical harmonic coefficients [6]. `=0 A good start for readers who are completely unfamiliar with with expansion coefficients b 2 2`+1. A very important the theory might be [5], where a basic understanding of ` C property which has been utilised for deriving rotation invari- spherical harmonics is given, focused on a practical point of ant descriptors in an analytical way from three-dimensional view. The design of rotation invariant spherical harmonic scalar fields [6] is the rotation behaviour of the spherical features was first addressed in [6]. Deeper views into the harmonics expansion coefficients. A rotation g 2 SO(3) theory are given in [14, 11, 1]. However, we first want to re- with corresponding Wigner D-Matrix D acting on a spher- capitulate the mathematical notations and definitions which g ical function f transforms the expansion coefficient b to we will use in the following sections. ` b0 with ` ` Let Dg be the unitary irreducible representation of a g 2 SO(3) of order ` 2 , acting on the vector space 0 ` N b ` = Dgb` (7) C2`+1. They are widely known as Wigner D-Matrices [14]. n We denote the complex conjugate of a vector a 2 C by a 2.2. 1D Fourier Basis Functions and the transpose of a by aT . Depending on the context we T will express the coordinate vector r = (x; y; z) 2 R3 in The Fourier basis spherical coordinates (r; θ; φ), which is closer to the com- R 1 2πikr 1 monly used notation of spherical harmonics, where: w (r) = p e R (8) k R p r = krk = x2 + y2 + z2 (1) is a countably infinite complete orthonormal basis system for functions on the interval [0;R], where k 2 and R 2 z Z θ = arccos (2) R>0 with px2 + y2 + z2 Z R φ = atan2(y; x) (3) R R R R hwk ; wk0 i = wk (r)wk0 (r) 0 Ç å Z R 2.1. Spherical Harmonics Basis Functions R R = wk (r)w−k0 (r) = δkk0 (9) 0 In our work we use the following definition of the spher- ical harmonics: 2.3. Spherical Tensor Product 2`+1 We further need to define the bilinear form •0 : C × ` 2` + 1 (` − m)! m imφ 2`+1 ! (a special case of the spherical tensor prod- Ym(θ; φ) = P` (cos θ)e (4) C C 4π (` + m)! uct [11]) which we use to couple expansion coefficients of equal rank : with ` 2 N0; m 2 Z; jmj ≤ ` and associated Legendre m ` m=` functions P` . TheÊ functions Ym build a countably infinite X m complete orthonormal basis for representing functions on (u •0 v) := (−1) umv−m (10) the 2-sphere, m=−` with u; v 2 C2`+1. Assume a rotation g 2 SO(3) is acting 0 0 on two expansion coefficients b` and b`, where b`; b` 2 Z π Z 2π 2`+1 ` `0 ` `0 C . The bilinear form has one very important property: hYm;Ym0 i = Ym(θ; φ)Ym0 (θ; φ)dΩ 0 0 coupling two expansion coefficients preserves the rotation. In our case: = δ``0 δmm0 (5) ` ` 0 0 0 (D b`) •0 (D b ) = D (b` •0 b ) h· ; · i dΩ = sin θdφdθ g g ` g ` where denotes the scalar product, 0 and δ denotes the Kronecker symbol. A square-integrable = (b` •0 b`) (11) H. Skibbe et al.,in Proc. of the 3DIM 2009, part of the 12th IEEE ICCV 2009 Rk ` R 1 Figure 1. Spherical Fourier harmonics basis functions E`m (θ; φ, r) = Ym(θ; φ)wk (r) r for ` 2 f0 ::: 2g and k 2 f0 ::: 4g. Negative 1 values are depicted in blue, positive values in yellow. The term r is indicated by darkening the colour for higher (inner) values. 3. An Orthogonal System on the Closed Ball The first 36 basis functions for ` 2 f0 ::: 2g and k 2 f0 ::: 4g are depicted in figure 1. We propose to expand a function f which is defined on the ball with given radius R 2 R>0 by a set of orthonor- 3.1. Fast Computation of Expansion Coefficients mal basis functions derived by combining the spherical har- monic expansion describing the angular part, and a damped The computation of expansion coefficients can be per- Fourier expansion describing the radial part. The expansion formed in a very efficient manner. The expansion can be is given by separately performed for the angular part using a spheri- k=1 `=1 m=` cal harmonic transformation and for the radial part using X X X f(r) = aRkERk(r) an ordinary 1D Fourier transformation. Fast algorithms `m `m exist for both transformations [4, 8]. A brief sketch of k=−∞ `=0 m=−` the fast computation of expansion coefficients aRk with k=1 `=1 ` T Rk Rk X X Rk Rk a`m := hf; E`m i is given in the following: = (a` ) E` (r) (12) k=−∞ `=0 Rk hf; E`m i = Rk 2`+1 with expansion coefficients a` 2 C and orthonormal Z R Z π Z 2π Rk 2 basis functions = f(r; θ; φ)E`m (r; θ; φ)r dΩdr 0 0 0 Rk ` R 1 ZZZ E`m (r) = Ym(θ; φ)wk (r) (13) ` R r = f(r; θ; φ)Ym(θ; φ)w−k(r)rdΩdr (16) where Rk Rk0 Weighting each voxel by its distance to the object centre hE ;E 0 0 i = `m ` m f 0(r; θ; φ) = rf(r; θ; φ) leads to: Z R Z π Z 2π Rk Rk0 2 = E`m (r)E`0m0 (r)r dΩdr hf; ERki = ··· 0 0 0 `m ZZZ = δ``0 δmm0 δkk0 (14) 0 ` R = f (r; θ; φ)Ym(θ; φ)w−k(r)dΩdr Given a real-valued function, the coefficients are symmetri- Z ZZ R 0 ` cally related by = w−k(r) f (r; θ; φ)Ym(θ; φ)dΩdr (17) Rk m R;−k | {z } a`m = (−1) a`;−m (15) b`m(r) H.