Phase in signals and images

Brigitte Forster

Fakultat¨ fur¨ Informatik und Mathematik Universitat¨ Passau

Brigitte Forster (Passau) September 2013 1 / 47 Marie Curie Team MAMEBIA at the IBB Research Topic

Mathematical Methods in Biological Image Analysis MAMEBIA Development of theoretical and concrete Researchm Topic:athematical methods Development of theoretical and concrete mathematical methods To model and analyse biological image data To model and analyse biomedical image data With emphasisWith em onph complex-valuedasis on com methodsplex-v andalue phased methods informationand phase information

Biological Mathematical Numerical Algorithms & Image Theory Analysis Software Analysis

Brigitte Forster (Passau) September 2013 2 / 47 1. Phase in signals and images

Mostly real-valued methods used for (biological) image analysis, because “Images are real-valued.” Real-valued methods need less storage space.

Exception: Fourier transform.

Brigitte Forster (Passau) September 2013 3 / 47 1. Phase in signals and images

But: There are good reasons to consider complex-valued methods.

First reason: Well interpretable Fourier transform:

f f where f (ω) = f (t)e−iωt dt. 7→ ZR Interpretation: f (ω)bamplitudeb to the frequency ω. | | bCoefficient = Amplitude Phase term · f (ω) = f (ω) eiϕ(ω) | | · b b

Brigitte Forster (Passau) September 2013 4 / 47 1. Phase in signals and images

Similar amplitude-phase decomposition for signals in time domain: Analytic signal (Dennis Gabor, 1946)

fa(t) = f (t) + iH(f )(t),

where H denotes the Hilbert transform. Then there is the decomposition

iφ(t) fa(t) = a(t)e

with instantaneous amplitude a(t) and instantaneous frequency φ0(t),

Brigitte Forster (Passau) September 2013 5 / 47 1. Phase in signals and images

Second reason: Dogma “Images are real-valued” is just not true.

There are complex-valued data and tasks: e.g. Magnetic resonance images as Fourier inverses of non-symmetric measurements Phase retrieval in digital holography, . . .

(Left: Johnson & Becker, The whole brain atlas, Harvard / MIT. Right: Bob Mellish, Wikipedia.)

Brigitte Forster (Passau) September 2013 6 / 47 1. Phase in signals and images

Third reason: Real bases cannot analyze single-sided frequency bands:

1.5 0.8

1.0 0.6

0.5 0.4

0.2 -4 -2 2 4

-0.5 2 4 6 8 10

-0.2 -1.0

-0.4 -1.5 Left: Mexican hat wavelet. Right: Its Fourier transform and spectral envelope.

This is another reason, why Gabor invented the analytical signal.

Brigitte Forster (Passau) September 2013 7 / 47 1. Phase in signals and images

Fourth reason: The phase of images contains edge and detail information. Reconstruction from Fourier amplitude and phase:

(Cf. A. V. Oppenheim and J. S. Lim, The importance of phase in signals, Proc. IEEE, 1981.)

(Left image: Laurent Condat’s Image Database http://www.greyc.ensicaen.fr/∼lcondat/imagebase.html.)

Brigitte Forster (Passau) September 2013 8 / 47 1. Phase in signals and images

Fourth reason: The phase of images contains edge and detail information. Reconstruction from Fourier amplitude and phase (denoised):

(Cf. A. V. Oppenheim and J. S. Lim, The importance of phase in signals, Proc. IEEE, 1981.)

(Left image: Laurent Condat’s Image Database http://www.greyc.ensicaen.fr/∼lcondat/imagebase.html.)

Brigitte Forster (Passau) September 2013 9 / 47 1. Phase in signals and images

Another example: Interchanging amplitude and phase

Brigitte Forster (Passau) September 2013 10 / 47 1. Phase in signals and images

Fifth reason: The dogma “Images are real-valued” neglects various classes of powerful mathematical transforms, e.g. complex-valued wavelet transforms, analytic and monogenic signals instantaneous frequency, → conformal monogenic signals curvature term, and others. →

Brigitte Forster (Passau) September 2013 11 / 47 Outline

1 Introduction

2 Signal analysis: B-splines and their complex relatives — Interpolation

3 Image Analysis: Phase information in higher dimensions

Brigitte Forster (Passau) September 2013 12 / 47 2. B-splines

Cardinal B-splines Bn of order n, n N, with knots in Z are defined recursively: ∈

B1 = χ[ 0,1 ], Bn = B − B for n 2. n 1 ∗ 1 ≥

Isaac J. Schoenberg (Source: The MacTutor History Piecewise polynomials of degree at most n and of Mathematics archive) − smoothness Cn 1(R). Brigitte Forster (Passau) September 2013 13 / 47 2. B-splines

Why are splines useful? Approximation: Approximation of a function f C[a, b] with sums of translated ∈ and dilated B-Splines Bn(( τ)/α) ∈ ∈ . { • − }τ N,α D Sharp estimates of the best approximation with splines are known for numerous function spaces and topologies. Well interpretable coefficients.

Multiresolution analyses

Interpolation: Spline interpolation does not show oscillations — in contrast to interpolation with polynomials.

Brigitte Forster (Passau) September 2013 14 / 47 2. B-splines

Classical B-splines

B = χ , Bn = B − B for n 2, 1 [ 0,1 ] n 1 ∗ 1 ≥ n 1 e−iω B (ω) = − n iω   have a discrete indexing.b have a discrete order of approximation. are real-valued. have a symmetric spectrum.

Brigitte Forster (Passau) September 2013 15 / 47 2. Complex B-splines

Idea: Complex-valued B-splines defined in Fourier domain:

z 1 e−iω B (ω) = B (ω) = − , z α+iγ iω   1 where z C, withb parametersb Re z > , Im z R. ∈ 2 ∈ (BF, Blu, Unser. ACHA, 2006)

Brigitte Forster (Passau) September 2013 16 / 47 2. Complex B-splines

Why is this definition reasonable and useful? z 1−e−iω z Bz (ω) = Bα+iγ(ω) = iω = Ω(ω) is well defined, since Ω(R) R− = .  b b ∩ ∅ z Re z −Im z arg(Ω(ω)) Bz (ω) = Ω(ω) = Ω(ω) e . | | | | | | 6

b 5 Im 4 0.5 3

2 Re 0.5 1

-0.5 -15 -10 -5 5 10 15

Ω: R C B3+iγ for γ = 0, 1, 2. → | | Approximately single-sided frequencyb analysis!

Brigitte Forster (Passau) September 2013 17 / 47 2. Complex B-splines

What do they look like? Representation in time-domain

1 k z z−1 Bz (t) = ( 1) (t k) Γ(z) − k − + kX≥0   pointwise for all t R and in the L2(R) norm. ∈ ——

Compare: The cardinal B-spline Bn, n N, has the representation ∈ n 1 k n n−1 Bn(t) = ( 1) (t k) (n 1)! − k − + − k=0   ∞ X 1 n = ( 1)k (t k)n−1. Γ(n) − k − + kX=0  

Brigitte Forster (Passau) September 2013 18 / 47 2. Complex B-splines

Complex B-splines Bz are piecewise polynomials of complex degree. Smoothness and decay tunable via the parameter Re z.

Recursion: Bz Bz = Bz +z 1 ∗ 2 1 2 They are scaling functions and generate multiresolution analyses and wavelets. Simple implementation in Fourier domain Fast algorithm. → Nearly optimal time frequency localization. Relate fractional or complex differential operators with difference operators.

(Here Re z, Re z1, Re z2 > 1.)

Brigitte Forster (Passau) September 2013 19 / 47 2. Interpolation with complex B-splines

How about interpolation with complex B-splines? Some interpolating complex splines for Re z = 4: (Real and imaginary part)

1.0

0.15 0.8 0.10

0.6 0.05

0.4 -6 -4 -2 2 4 6

- 0.2 0.05

-0.10 -6 -4 -2 2 4 6 -0.15

Joint work with Peter Massopust, Ramunas¯ Garunkstis,ˇ and Jorn¨ Steuding. (Complex B-splines and Hurwitz zeta functions. LMS Journal of Computation and Mathematics, 16, pp. 61–77, 2013.)

Brigitte Forster (Passau) September 2013 20 / 47 2. Interpolation with complex B-splines

Aim: Interpolating splines Lz of fractional or complex order. Interpolation problem:

(z) Lz (m) = c Bz (m k) = δm,0, m Z, k − ∈ Xk∈Z Fourier transform:

B (ω) 1/ωz L (ω) = z = , Re z 1. z 1 Bz (ω + 2πk) ≥ z k∈ b (ω + 2πk) b XZ k∈Z b X Question: When is the denominator well-defined?

Brigitte Forster (Passau) September 2013 21 / 47 2. Interpolation with complex B-splines

Rescale denominator. 1/ωz Lz (ω) = , Re z 1 1 ≥ (ω + 2πk)z b ∈ Xk Z Let ω = 0 and a := ω/2π. 6 ∞ ∞ 1 1 1 = + e±iπz (k + a)z (k + a)z (k + 1 a)z Xk∈Z kX=0 kX=0 − = ζ(z, a) + e±iπz ζ(z, 1 a). − This is a sum of Hurwitz zeta functions ζ(z, a), 0 < a < 1.

Brigitte Forster (Passau) September 2013 22 / 47 2. Interpolation with complex B-splines

Theorem: (Spira, 1976) If Re z 1 + a, then ζ(z, a) = 0. ≥ 6

Thus: For 0 < a < 1 and for σ R with σ 2 and σ / 2N + 1: ∈ ≥ ∈ ζ(σ, a) + e±iπσ ζ(σ, 1 a) = 0 . − 6

Brigitte Forster (Passau) September 2013 23 / 47 2. Interpolation with complex B-splines

1

0.5

2 4 6 8

-0.5

Interpolating spline Lσ in time domain for σ = 2.0, 2.1,..., 2.9. σ = 2: Classical case of linear interpolation.

Brigitte Forster (Passau) September 2013 24 / 47 2. Interpolation with complex B-splines

Question: Are there complex exponents z C with Re z > 3 and Re z / 2N + 1, such that the denominator of the∈ Fourier representation of the∈ fundamental spline satisfies

ζ(z, a) + e±iπz ζ(z, 1 a) = 0 − 6 for all 0 < a < 1.

Consequence: The interpolation construction also holds for certain B-splines Lz of complex order z.

Brigitte Forster (Passau) September 2013 25 / 47 2. Interpolation with complex B-splines

Why was the problem on zero-free regions so resisting?

±iπz f±(z, a) := ζ(z, a) + e ζ(z, 1 a) − Involves the Hurwitz zeta function aside the “interesting domain” 1 around Re z = 2 . Rouche’s´ theorem seems to be suitable to find zeros, but not to exclude them. The problem is that the estimates have to be uniform in a. Numerical examples fail, because the Hurwitz zeta function is implemented via meromorphic approximations. However, there exist classes of interpolating B-splines of complex order.

Brigitte Forster (Passau) September 2013 26 / 47 2. Interpolation with complex B-splines

Curves of zeros: Denote

±iπz t f±(z, a) := ζ(z, a) + e ζ(z, 1 a). − 4 Consider the zero trajectories 2

z(a) ∂f±(z,a) ∂ ∂a Σ = , 1 2 3 4 5 6 7 8 9 ∂a − ∂f±(z,a) ∂z -2

where z = z(a) and f±(z(a), a) = 0 with initial conditions provided at the -4 zeros z = 2n + 1, n N, a = 1/2. 0.001 < a < 0.999 ∈

Brigitte Forster (Passau) September 2013 27 / 47 2. Interpolation with complex B-splines

Functional equation:

±iπz f±(z, 1 a) = e f∓(z, a). − 1 t We can reduce the problem to 0 < a 2 . 4 Define ≤

2 z z ±iπz a Σ g (z, a) = a f (z, a) = 1+e +B 1 2 3 4 5 6 7 8 9 ± ± 1 a  −  -2 Basic estimations of the first two terms and B yield squares of zero-free regions: -4

1 S = z C : z = 2(n+1)+s, Re s X < , Im s Y . { ∈ | | ≤ 2 | | ≤ }

Brigitte Forster (Passau) September 2013 28 / 47 2. Interpolation with complex B-splines

t

4

Consider again the first terms: 2

Σ α z 1 2 3 4 5 6 7 8 9 1 + e±iπz = 0 for 0 < α < 1, 1 α -2  −  describes circles: -4 1 1 z s : s k + = k + , k Z . ∈ | − 2| | 2| ∈  

Brigitte Forster (Passau) September 2013 29 / 47 2. Interpolation with complex B-splines

t

Crescent shaped zero-free regions 10

For 8 2k + 2 + δ 2k + 2 + δ s : s = , 0 < δ < δ , 6 − 2 2 | | 0  4

σ Re s 2n + 2 + δ 0 ≤ ≤ 0} 2 with δ < 1 and 2 < σ < 2n + 2 + δ , Σ 0 0 0 38 40 41 42 43 the first two terms are located between -2 the circles of zeros, α s 1 -4 1 + e±iπs , 1 α ≥ 2 -6  −  1 and the term B satisfies B < . -8 | | 2 -10 Zero Free Regions. ⇒ Brigitte Forster (Passau) September 2013 30 / 47 2. Interpolation with complex B-splines

Example:

1.0

0.8 0.05

0.6

0.4 -6 -4 -2 2 4 6

0.2

-0.05 -6 -4 -2 2 4 6

-0.2

Interpolating complex B-splines with Re z = 10 (Real and imaginary part).

Brigitte Forster (Passau) September 2013 31 / 47 3. Phase information in higher dimensions

How to extend the notion of phase from C to higher dimensions? Problem closely related to William Rowan Hamilton’s work (1805–1865, Dublin) trying to extend the complex numbers to triplets, leading to quaternions with three complex units. Today: Approach via Clifford algebras (William Kingdon Clifford, 1845–1879, London).

Hamilton and Clifford

(Images:Brigitte Magnus Forster Manske, (Passau) wikipedia) September 2013 32 / 47 3. Phase information in higher dimensions

Firstly, let f L2(R, R) be some function/signal. ∈ Which amplitude and which frequency are active at the moment t R? ∈ Gabor’s Idea: Analytic signal:

iϕ(t) fa(t) = f (t) + iHf (t) = a(t)(cos ϕ(t) + i sin ϕ(t)) = a(t)e

such that ϕ0(t) is the instantaneous frequency of the signal f . Here, H is the Hilbert transform

2 2 ω H : L (R) L (R), Hf (ω) = i f (ω). → ω | | c b

Brigitte Forster (Passau) September 2013 33 / 47 3. Phase information in higher dimensions

Phase in higher dimensions Riesz transform → 2 n 2 n n R : L (R ) (L (R )) , Rf = (R1f ,..., Rnf ) → with ω c d d R f (ω) = i k f (ω), ω = (ω , . . . , ω ). k ω 1 k k k Question: Representationd b

n+1 fm(x) = (f (x), R1f ,..., Rnf ) = a(x) exp iϕ(x) R ∈ for x Rn? ∈ + + + No ad hoc multiplicative structure · : Rn 1 Rn 1 Rn 1. × → Solution: Multiply anyway!

Brigitte Forster (Passau) September 2013 34 / 47 3. Phase information in higher dimensions

2 n n+1 Let f L (R , R), ek k=0,...,n ONB of R . ∈ { } n Define Clifford algebra Rn or Cn of dimension 2 via the multiplication of the basis vectors

e e = e e , e e = e e , e2 = e , e2 = e k j − j k k 0 0 k k − 0 0 0 ie = e i, i2 = e . k k − 0 (k = j, k, j = 1,..., n). 6 + For x, y Rn 1, ∈ n n n n xy = ek xk el yl = ek el xk yl . ! ! kX=0 Xl=0 kX=0 Xl=0 n+1 Note that R Cn and C = C1. ⊂ Identification 1 e . ↔ 0 Brigitte Forster (Passau) September 2013 35 / 47 3. Phase information in higher dimensions

Amplitude and phase + Let x Rn 1. ∈ n x = x0 + ~x, where ~x = ek xk . kX=1 x ~x ~x ~x x = x 0 + k k = x cos ϕ + sin ϕ . k k x ~x · x k k ~x k k k k k k  k k  ~x x = x exp ϕ . ⇒ k k ~x k k  ϕ angle between scalar and vector part: tan ϕ = ~x /x . k k 0 Terminology: Amplitude x Phase ϕ k k ~x Phase direction k~xk

Brigitte Forster (Passau) September 2013 36 / 47 3. Phase information in higher dimensions

Let f L2(Rn, R). ∈ Monogenic signal:

n fm = fe0 + Rf = e0f + ek Rk f . kX=1 Phase and phase direction:

Rf (x) fm(x) = fm(x) cos ϕ(x)e + sin ϕ(x) k k 0 Rf (x)  k k  with tan ϕ(x) = kRf (x)k for f (x) = 0. f (x) 6

Brigitte Forster (Passau) September 2013 37 / 47 3. Phase information in higher dimensions

Riesz transform intertwins with translations scaling/dilation, 656 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010 thus with multiresolution transforms, e.g. the wavelet transform.

Fig. 1. Commutative diagram for Riesz and wavelet transform. For Brigitte Forstera multiscale (Passau) decomposition of the details of amplitude and phase, weSeptember can use 2013 38 / 47 either way or , since they are equal.

Equipped with this, in general, noncommutative multiplica- Fig. 2. Phase angle and phase direction (direction of the dashed tion, is extended to a so-called Clifford algebra of di- vector) of the 3-D vector . mension . In the same way, complex vector spaces can be equipped with a multiplication; for more details we refer to Ap- pendix A. In analogy with our identification of with in Ex- ample 2.2, in all what follows we use the identification . Then each element viewed as element of the sub- space of the algebra has a representation in form of a scalar and a vector part

Let denote the angle between the scalar and the vector part: . Then

We call the modulus amplitude, the phase, and the normalized vector part the phase direction of (see Figs. 2 and 4). To establish a decomposition as in (2) we define an exponen- tial function on . We do this via the usual exponential se- ries. It can be shown that the subspace of the algebra is invariant under power operations

Fig. 3. Wavelet for . (a) Real part of the monogenic wavelet , i.e., the real wavelet . (b) Riesz transforms , of the wavelet . (c) Am- plitude . (d) Phase and phase direction (arrows) of the hypercomplex wavelet . The isotropy is clearly visible. The direction of the phase points This is due to the fact that for any and any from high to low phase values. it holds that . Note that is both a subalgebra of and a subset of , e.g., for the square Due to this fact, we can define an exponential function by

Authorized licensed use limited to: T U MUENCHEN. Downloaded on August 04,2010 at 12:31:18 UTC from IEEE Xplore. Restrictions apply. 3. Phase information in higher dimensions

2 A frame is a family of functions F = f ∈ in an L -space, that { k }k N spans the space, and allows for stable analysis and reconstruction with respect to the family: A f 2 f , f 2 B f 2 k k ≤ |h k i| ≤ k k kX∈N (f L2, A, B > 0). ∈

Advantage: Riesz-transformed frames stay frames: (S. Held, 2010) 2 n Let F = fk k∈N be a frame for L (R ) with frame bounds A and B. { } 2 n Then F is a Clifford frame for L (R , Cn) with the same frame bounds. 2 n The Riesz transformed frame RF is a Clifford frame for L (R , Cn) with the same frame bounds.

Brigitte Forster (Passau) September 2013 39 / 47 HELD et3. al.: STEERABLE Phase WAVELET information FRAMES BASED ON THE RIESZ TRANSFORMin higher dimensions 657

Radial wavelet ψ, Riesz transform R1ψ, R2ψ yield steerable wavelet frame.

Fig. 4. First row: Original image and Riesz transforms in both directions. Second to fourth row: Wavelet coefficients of the monogenic wavelet decomposition on first three(S. scales Held, (band M. pass Storath, component). P.Fifth Massopust, row: Approximation and BF: coefficients IEEE (low Trans. pass component). Image Proc.,The first column 2010.) displays the real part of the wavelet coefficients, i.e., the component, the second and third column the imaginary parts, i.e., the and components. We observe that the Riesz transform acts as weak directional filter (see, e.g., the table leg in the image). The fourth and fifth column are the amplitude and the phase of the wavelet coefficients. The descending sizes of the imagesBrigitte represent Forster the sub-sampling (Passau) scheme of our filter-bank. September 2013 40 / 47

It holds that for all . analytic signal of a real-valued function Furthermore, note that . Since into components with respect to an appropriate frame with well-interpretable frame elements

We construct our wavelet frame in the Fourier domain. Let , where is the Clifford algebra with complex coefficients defined by and the multiplication rules , and . Then we write in short

where . The Fourier transform on is defined by

we finally derive . This is equivalent to (2) for the case . Since the Fourier transform maps to , IV. CLIFFORD FRAMES we define the concept of Clifford frames on . We now combine the concept of monogenic signal with the As a motivation, we first give the definition of a classical concept of wavelet frame. That is, we aim at decomposing the frame in a Hilbert space.

Authorized licensed use limited to: T U MUENCHEN. Downloaded on August 04,2010 at 12:31:18 UTC from IEEE Xplore. Restrictions apply. 3. Phase information in higher dimensions

Algorithm for contrast enhancement Image decomposition with monogenic wavelet transform Reconstruction from phase only

Brigitte Forster (Passau) September 2013 41 / 47 3. Phase information in higher dimensions

Decreening

SUBMITTED TO IEEE TRANSACTIONS ON IMAGE PROCESSING 12

(a) Original (b) Screened (c) Descreened Fig. 13: Descreening with our parameter free descreening algorithm: The screening effect of (b) is nicely interpolated and details are preserved (c). The abrupt change of color (e.g. the heaven in image (c)) is induced by the algorithm used for producing the screening effect in image (b), which can resolve only 16 gray values.

The Clifford algebra Rn can be complexified by setting Analogous to complex conjugation, we define a conjugation Brigitte Forster (Passau) September 2013 42 / 47 ieα1 eα2 eαν = eα1 eα2 eαν i, for all ordered index tuples : Cn Cn by setting · · · · · · → 1 α1 < α2 < < αν n, 0 ν n. The resulting ≤ · · · ≤ ≤ ≤ n i = i, e0 = e0, eα = eα, for all α = 1, . . . , n, Clifford algebra is denoted by Cn. It is a 2 -dimensional − − vector space over the complex numbers C: Every x Cn and xy = y x, for all x, y Cn. It then follows that ∈ has a unique representation of the form ∈ x = xαν eαν

x = xα1α2 αν eα1 eα2 eαν , (20) αν Iν ··· · · · 0 !ν∈ n 1 α1<α2< <αν n ≤ ≤ ≤ 0 !ν ···n ≤ ν(ν+1)/2 ≤ ≤ and e = ( 1) e , for all 0 ν n. The αν − αν ≤ ≤ where the coefficients xα1α2 αν are now complex numbers. conjugation can now be used to define a norm and thus ··· The real part of the coefficient x0, which corresponds to the a metric on Cn: empty product, is called the real part of x and denoted by 2 2 x = Re xx = xαν , x Cn. (22) Re x = Re x0. Note that this definition agrees with the usual & &Cn & &C ∀ ∈ αν Iν definition of real part when we consider R1 = C. 0 !ν∈ n To simplify notation, we henceforth use multi-indices. For ≤ ≤ 1 α < α < < α n, set α := α α α and APPENDIX B ≤ 1 2 · · · ν ≤ ν 1 2 · · · ν denote the product e e e by e . We then denote the CLIFFORD MODULES α1 α2 · · · αν αν corresponding index set by Iν := αν : 1 α1 < . . . < In order to define a Clifford frame decomposition of a { ≤ n αν n Thus, representations of the form (20) or (21) can function f : R Cn, we need to introduce an inner prod- ≤ } 2 n→ 2 n be written more succinctly as uct space, L (R , Cn), which extends the known L (R , C) Hilbert space to the Clifford setting. x = xαν eαν , xαν R or C. ∈ To discuss decompositions of the form αν Iν 0 !ν∈ n ≤ ≤ f = xαν fαν , (23)

Elements x Rn or Cn are called paravectors if they are αν Iν ∈ 0 !ν∈ n of the form ≤ ≤ n 2 n where xαν Cn and fαν L (R , Cn), we need to first x = x e + x e =: x + x . (21) ∈ ∈ 2 n 0 0 α α 0 →− define a vector space structure on L (R , Cn) and interpret α=1 ! the product xαν fαν as the Clifford algebra Cn operating on 2 n 2 n Note that for paravectors the sum in (20) is taken only over the vector space L (R , Cn), i.e., L (R , Cn) is a so-called 2 n 0 ν 1. It follows directly from (21) that a paravector Clifford module. The inner product on L (R , Cn) is then ≤ ≤ n+1 x Rn or Cn can be identified with an element in R , used to obtain the coefficients x in the expansion (23). ∈ αν respectively, Cn+1. To this end, let H be an arbitrary real or complex Hilbert Remark A.1: Note that the vector space of paravectors is space. Define Hn to consist of all elements of the form not a subalgebra of Rn. However for any paravector x the h = hα eα , space e0R + #xR is a subalgebra of Rn isomorphic to C via ν ν #x αν Iν e0 1, #x i. 0 !ν∈ n %→ # # %→ ≤ ≤ 3. Phase information in higher dimensions

Software: Monogenic Wavelet Toolbox

Freely available as ImageJ plugin. www.mamebia.de/Software

Brigitte Forster (Passau) September 2013 43 / 47 Steerable wavelet frames based on the Riesz transform S. Held, M. StorathBrief Article, P. Massopust, and B. Forster

The Author 1. Analytic wavelets Brief Article 5. Applications are widely used to separateBriefMay amplitude 31, Article 2011 and phase information in 664 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010 signals662 via a multiscale approachThe Author. 5.1.IEEE Descreening TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010 The Author Analytic signal of a real-valued function f: May 31, 2011 May 31, 2011 iϕ(t) fa(t)=f(t)+iHf(t)=a(t)e TABLE III COMPUTATION TIME AND ACCURACY OF THE RIESZ WAVELET FILTER B ANK FOR VARIOUS SAMPLE SIZES.THE TIME FOR THE CREATION OF where H denotesTHE F ILTERthe -B HilbertANK IS transformDENOTED AS. !P‘RECALCULATION is the instantaneousTIME.THE RECONSTRUCTION ERROR LAYS IN THE RANGE OF MACHINE ACCURACY. 2 n 2 n n frequency. An: L analytic(R ) wavelet(L (R ))is the, analyticfT=(HE EXAMPLES 1signalf,...,iϕ(t) ofW naEREf wavelet) COMPUTED. ON AN INTEL CORE 2DUO WITH 2.33 GHz R fa(t)=→ f(t)+iHf(Rt)=a(Rt)e R iϕ(t) 2. Monogenic waveletsfa(t)=f ( t)+iHf(￿t)=a(t￿)e ￿ 2 n 2 n ξα for higher dimensionsα2: Ln(R )via 2theL ( nRRieszn), transformαf(ξ):= Ri: f(ξ). :RL (R ) (→L (R )) , R f =( 1f,...,ξ nf) R 2 n → 2 n n R R ￿ ￿ R : L (R ) (L (R )) , f =( 1f,..., nf) Fig. 13. Descreening with our parameter free descreening algorithm: The screening effect of (b) is nicely interpolated and details are preserved (c). The abrupt R → R￿ R ￿R Descreeningchange of color (e.g., with the heaven parameter-free in image (c)) is induced descreening by the algorithm usedalgorithm. for producing The the screeningscreening effect in effect image (b), is which nicely can resolve interpolated only 16 gray ￿ ￿ ￿ values. 2 n 2 n ξα and details are preserved. The abrupt change of color is induced by the screening procedure, which : L ( ) L ( ), f(ξ):=i f(ξ). can only resolve 16 gray values. α R R ￿α ￿ ξ ￿ R 2 n → 2 n R ξα Clifford modules over complex Hilbert spaces as an extension where the coefficients are real numbers. It follows α : L (R ) L (R ), αf(ξ):=i ￿ ￿f(ξ). Here656, " = R1,...,n. Note: →H = R for n=1R. ξ IEEE of TRANSACTIONS the concept of Hilbert spaces ON IMAGE that is compatible PROCESSING, with the mul- VOL.from the 19, above NO. representation 3, MARCH that 2010 the dimension of is . 1 ￿ ￿ 5.2.tiplication Equalization by vectors and Clifford framesof Brightness to extend the concept Important special cases of Clifford algebras are obtained by Brief Article ￿ ￿ of frames to Clifford modules. setting , 1, and 2: , , and , the The Riesz transform R commutes with￿ the wavelet transform￿ W. We use a method of frame construction applicable to arbitrary algebra of quaternions. Note that for , the multiplication dimensions. The implementation was done via a filter bank as is noncommutative. ImageJ plug-in for 2-D and 3-D images. It is freely available at The Clifford algebra can be complexified by setting The Author [25]. , for all ordered index tuples The implemented applications include demodulation of , . The resulting amplitude modulated signals, steerable directional filters, Clifford algebra is denoted by . It is a -dimensional vector descreening and adjusting brightness by phase only recon- space over the complex numbers : Every has a unique May 31, 2011 struction. The presented applications show that the monogenic representation of the form wavelet decomposition is a useful tool in image reconstruction. Future plans include improving the robustness of the phase by (20) amplitude thresholding, optimization of the wavelets to reduce 656 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010 artifacts, develop advanced algorithms for edge detection and denoising. Another approach would is the study of higher Riesz where the coefficients are now complex numbers. transforms [19] which would for example allow us to extend the The real part of the coefficient , which corresponds to the iϕ(t) approach used in [10] and [11] from 2-D to n-D. empty product, is called the real part of and denoted by fa(t)=f(t)+iHf(t)=a(t)e . Note that this definition agrees with the usual definition APPENDIX A of real part when we consider . CLIFFORD ALGEBRAS To simplify notation, we henceforth use multi-indices. For Let be any orthonormal basis of .We , set and 2 n 2 n n define a multiplication between the elements by denote the product by . We then denote the : L (R ) (L (R )) , f =( 1f,..., nf) corresponding index set by R → R R R . Thus, representations of the form (20) or (21) can be written Left: Artificially corrupted image with parts of verymore low succinctly contrast. as Right: Reconstruction from the Clifford frame‘s phase only. Even fine image structures are recovered. ￿ ￿ ￿ Fig. 10. (a) Artificially corrupted image. Gray levels are multiplied by a ramp Fig. 1. Commutative2 n diagram2 for Rieszn and wavelet ξαtransform. For The Clifford algebra over has as its basis the set 3. Phaseα interpretation: L ( ) L: ( ), αf(ξ):=i f(ξ). 3. Phasefunction information and by 0.05 in the in center. higher (b) Reconstruction dimensionss from phase only. The a multiscale decompositionR of theR details of amplitude and phase, we can use . Vectors indexed by boldface as given above represent Fig. 1. CommutativeR diagram for Riesz andn+1 wavelet→ transform. For R ξ Elements or are called paravectors if they are of a multiscaleEacheither decomposition element way of the details in ofR amplitudeor has and phase, ,a we since representation can use they are equal. in form of a scalar and elementsbrightness of this basis. is now The emptybalanced, product isin defined particular as the the difference of brightness between the either way or , since they are equal. ￿ ￿ 5.3. Mathematical analysis of thethe visual form perception of optical A curiositymultiplicativedark found center identity and by: its outside M. Storath:, for disappears. all , Even fine image structures are recovered. a vector part. ￿ n ￿ . Hence, every element can be written uniquelyDecomposition illusions in the form was computed using sub-channels (compare the quality(21) n+1 Equipped withx this, in general, noncommutative: x = x multiplica-0 + ￿x,Fig. 2.where Phase angle and￿x phase= direction eα(directionxα. of the dashed with the one sub-channel reconstruction in Fig. 5, lower right image). R vector) of the 3-D vector . tion, is extended to a so-called Clifford algebra of di- Note that for paravectors the sum in (20) is taken only over 656 IEEE TRANSACTIONS∈ ON IMAGE PROCESSING, VOL. 19, NO. 3, MARCH 2010 Fig. 2. Phase angle and phase direction (direction of the dashed mension .Fig. InEquipped the9. same Filter way, complexwith bank vectorthis, for spaces decompositionin general, can be noncommutative and reconstructionα=1 multiplica- of signal by the . It follows directly from (21) that a paravector equipped with a multiplication; for more details we refer to Ap- 1 ￿ vector) of the 3-D vector . tion,monogenicis wavelet extended frame. to Herea so-called Cliffordhigh pass algebra filters are used,ofdi- namely pendix A. In analogy with our identification of with in Ex- Authorized licensed use limited to: T U MUENCHEN. Downloaded on August 04,2010 at 12:31:18 UTC from IEEE Xplore. Restrictions apply. ample 2.2,mension inand all what follows. we.is useIn the the the approximation identification same way,. filter complex [see Fig. vector 8(a) for spaces the filter can masks]. be Then each elementdenotes the Fourierviewed as element filter of themultiplied sub- by the Fourier multiplier of the Riesz phase of the wavelet coefficients of the image, and not for the space equippedof the algebra withhas a a representation multiplication; in form of a for more details we refer to Ap- scalar and a vectortransform part [see Fig. 8(b) for case ]. and are the amplitude and phase decomposition of the image itself. (com- pendixreal and A. hypercomplex In analogy with detail our coefficients, identification respectively. of Thewith first indexin Ex-denotes pare Fig. 4). Hence, reconstruction by phase only delivers an amplethe scale 2.2, of in the all coefficients. what follows we1 use the identification . 1 image with balanced brightness (Fig. 10). Details are conserved Then each element viewed as element of the sub- Let denote the angle between the scalar and the vector part: or even enhanced by equalization of brightness. As mentioned space. Then of the algebra has a representation in form of a scalarcompared and a vector to the part dyadic case, e.g., for the application “Equal- above the results become better when using -expansive di- Fig. 1. Commutative diagram for Riesz and wavelet transform. For lations instead of dyadic ones (we chose ). The algorithm a multiscale decomposition of the details of amplitude and phase, we can use ization of Brightness” (see Section VIII-A), turned out either way or , since they are equal. works as follows for an image of dimension . to be a sensible trade-off between frequency resolution and

1) Decompose image via the filter bank algorithm into de- Equipped with this, in general, noncommutative multiplica- Fig. 2. Phaseruntime. angle and phase direction (direction of the dashed vector) of the 3-D vector . tion, is extended to a so-called Clifford algebra of di- WeLeft call: the Phase modulus angle andamplitude phase, directionthe phase,. andRight the: (a) Real-valued isotropic wavelet. (b) The Riesz tail coefficients , where is the sub-channel mension . In the same way, complex vector spaces can be We computed images of different sizes with the ImageJ im- normalized vector part the phase direction of (see equipped with a multiplication; for more details we refer to Ap- transforms of the wavelet yield steerable generators of a Clifford frame, which is ideal for image In the Hermann grid, the observer perceives illusory gray smudges at the intersections of the white Figs. 2 and 4). index and the scale index. pendix A. In analogy with our identification of with in Ex- analysisplementation. Let denote the of angle our fast between algorithm. the scalar The and runtime the vector and part: accuracyLeft: Opticalbars. The illusion illusion disappears and a, if the variant. bars are sine- Right:shaped Reconstruction. Both effects are reproduced from accurately phase by only. To establish a decomposition as in (2) we define an exponen- ample 2.2, in all what follows we use the identification . 2) Compute the phase as in (17). tial function onresults. We do of this the via the experiments usual. Then exponential se- are given in Table III. The exam- the equalization of brightness algorithm (right column). Then each element viewed as element of the sub- ries.4. It canTheorem: be shown that the subspace(S. Held)of the algebra 3) Compute the new wavelet coefficients for space of the algebra has a representation in form of a is invariant underples power were operations calculated using dyadic dilations. In exampleThe VIII-A mathematical method for contrast enhancement from phase scalar and a vector part 2 n 2 n (i) A frame in L (R ) is also a Clifford frame for L (R , Cn) with the Reference:all scales and all sub-channels , except the approxima- below, we used -expansiveFig. 3. Wavelet dilations. for . (a) Real Note part of the monogenic that wavelet thereproduces filters, i.e., S.Held the, M. Storath optical, P. Massopust illusion, and ofB. Forster the, HermannSteerable Wavelet grid. Frames Based on the Riesz same frame bounds. the real wavelet . (b) Riesz transforms , of the wavelet . (c) Am- plitude . (d) Phase and phase direction (arrows) of the hypercomplex Transform.tion IEEE component Trans. Image Proc., Vol. 19,[that No. 3, March is, (18) 2010 with ]. have to be computed only oncewavelet because. The isotropy is clearly they visible. are The direction used of theboth phase pointsBrigitte for Forster (Passau) September 2013 44 / 47 (ii) The Riesz transform of an L2(Rn)-frame is a Clifford frame in This is due to the fact that for any and any from high to low phase values. 4) Reconstruct from the new real wavelet coefficients it holds that decomposition2 n . Note that and reconstruction.is both a Note: M. Storath received the Serendipity Award 2011 from the Universität Bayern e.V. for his Let denote the angle between the scalar and the vector part: L (R , Cn) with the same frame bounds. . Then subalgebra of and a subset of , e.g., for the square contribution to the. mathematical analysis of the visual effects of the Hermann grid illusion. Due to this fact, we can define an exponential function by VIII. APPLICATIONS The algorithm also sets very low amplitudes to one resulting in We now apply the monogenic wavelet decomposition to an amplification of noise. Therefore, we added a simple noise Institute of Biomathematics and Biometry – Marie Curie suppressionExcellence based Team on the magnitude of the amplitude: Let AuthorizedWeimagecall licensed use the reconstruction limited modulusto: T U MUENCHEN. Downloaded problems.amplitude on August 04,2010 at ,12:31:18 We UTCthe from will IEEEphase Xplore. present ,Restrictions and apply. the two ap- Mathematical Methods in Biomedical Imaging MAMEBIAbe the mean value and be the standard deviation of . normalizedplications: vectorEqualization part of1 the brightnessphase directionand parameterof (see free and Zentrum Mathematik, Technische Universität München. We call the modulus amplitude, the phase, and the We set a threshold . Then the coefficients normalized vector part the phase direction of (see This workFigs.descreening was 2 partially and 4). funded. The by the grant former MEXT-CT-2004-013477of exploits the the phase, European theCommission. later The the grant was awarded to B. Forster. Figs. 2 and 4). amplitude.To establish a decomposition as in (2) we define an exponen- of step 3) are computed as To establish a decomposition as in (2) we define an exponen- tial function on . We do this via the usual exponential se- tial function on . We do this via the usual exponential se- if ries. It can be shown that the subspace of the algebra ries.A. It Equalization can be shown of that Brightness the subspace of the algebra is invariant under power operations if . is invariant under power operations Fig. 3. Wavelet forLooking. (a) Real part at of the monogenic monogenic wavelet , i.e., wavelet decomposition we observe the real wavelet . (b) Riesz transforms , of the wavelet . (c) Am- plitude . (d) Phase and phase direction (arrows) of the hypercomplex wavelet . Thethat isotropy the is clearly amplitude visible. The direction contains of the phase points information about the brightness of Thus, insignificant coefficients stay small. This is due to the fact that for any and any from high to low phase values. Fig. 3. Wavelet for . (a) Real part of the monogenic wavelet , i.e., it holds that . Note that is both a the image, whereas the phase contains the structure of the image.the real waveletEqualization. (b) Riesz transforms of brightness, canof also the wavelet be applied. (c) Am- to color im- subalgebra of and a subset of , e.g., for the square Note that this observation holds true only for the amplitude andplitude ages.. (d) The Phase colorand image phase directionis transformed (arrows) of to the the hypercomplexcolor space Due to this fact, we can define an exponential function by wavelet . The isotropy is clearly visible. The direction of the phase points This is due to the fact that for any and any from high to low phase values. it holds thatAuthorized licensed use. Notelimited that to: T U MUENCHEN.is bothDownloaded a on August 04,2010 at 12:31:18 UTC from IEEE Xplore. Restrictions apply. Authorized licensed use limited to: T U MUENCHEN. Downloaded on August 04,2010subalgebra at 12:31:18 UTC from ofIEEE Xplore. and Restrictions a subsetapply. of , e.g., for the square Due to this fact, we can define an exponential function by

Authorized licensed use limited to: T U MUENCHEN. Downloaded on August 04,2010 at 12:31:18 UTC from IEEE Xplore. Restrictions apply. Summary

Five reasons to consider phase information Complex B-splines: Approximately single-sided frequency analysis Families of complex interpolating functions Phase in higher dimensions via Riesz transform and just-do-it multiplication Applications/Software Image Restoration Descreening Optical illusions

Brigitte Forster (Passau) September 2013 45 / 47 Literature

Main references:

BF, T. Blu and M. Unser: Complex B-splines. Appl. Comput. Harmon. Anal. 20, 2006, pp. 261–282. BF and P. Massopust: Statistical Encounters with Complex B-Splines. Constr. Approx., 29 (3), pp. 325–344, 2009. BF, Ramunas¯ Garunkstis,ˇ Peter Massopust, and Jorn¨ Steuding: Complex B-splines and Hurwitz zeta functions. LMS Journal of Computation and Mathematics, 16, pp. 61–77, 2013. S. Held, M. Storath, P. Massopust, and BF Steerable wavelet frames based on the Riesz transform. IEEE Transactions on Image Processing. Volume: 19 , Issue: 3, pp. 653–667, 2010.

Brigitte Forster (Passau) September 2013 46 / 47 Brigitte Forster (Passau) September 2013 47 / 47