Outline Multidimensional and Directional Filter Banks
1. Multidimensional filter banks • Sampling lattices • Filter design
Minh N. Do 2. Laplacian pyramid frames Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign 3. Iterated directional filter banks
www.ifp.uiuc.edu/∼minhdo [email protected]
Joint work with Martin Vetterli (EPFL and UC Berkeley), Arthur Cunha, Yue Lu, Jianping Zhou (UIUC)
1. Multidimensional Filter Banks 1
1. Multidimensional Filter Banks Filter Banks
Motivation Wavelets and filter banks... fundamental link between continuous and discrete domains
Looking for... analysis synthesis y1 x1 Mallat H1 2 2 G1 • Discrete-domain expansions for sampled data x + xˆ
H 2 2 G0 0 y x • Structured transforms with fast algorithms 0 0
• Seamless connection with continuous-domain expansions
Answer: Filter banks and associated bases and frames. 2 G 1 Daubechies 2 G1 +
+ 2 G0
2 G0
1. Multidimensional Filter Banks 2 1. Multidimensional Filter Banks 3 Multidimensional Filter Banks “True” Multidimensional Filter Banks
ω2 π • Common approach: use 1-D techniques in a separable fashion HH1L Q
x −π π ω1
HH 0Η Q −π
• More flexibility: – True multidimensional filters → directional filters – Sampling via lattices → discrete rotations • Limitations: – very constrained filter design (separable filters) – only rectangular frequency partition possible
1. Multidimensional Filter Banks 4 1. Multidimensional Filter Banks 5
Sampling via Lattices Multidimensional Signals and Filtering
• Discrete-time d-dimensional signal: In 1D (downsample by 2):
def T d n1 x[n], n = (n1,...,nd) ∈ Z
In 2D (downsample by 2): • The z-transform:
d −n n ni X(z)= x[n]z where z = zi d i=1 nX∈Z Y
• Filtering with h[n] yields
y[n]= x[n] ∗ h[n]= x[k]h[n − k] d kX∈Z Y (z)= X(z)H(z)
1. Multidimensional Filter Banks 6 1. Multidimensional Filter Banks 7 Multidimensional Sampling Examples of Sampling Lattices
Sampling operation is represented by a d × d nonsingular integer matrix M. Separable lattice in two dimensions:
Downsampling x [n]= x[Mn]. d 2 0 D = 1 0 2 Upsampling d 2 2 x[k] if n = Mk, k ∈ Z D = xu[n]= 2 0 2 (0 otherwise.
Sampling lattice Quincunx lattice in two dimensions:
def d LAT (M) = n : n = Mk, k ∈ Z 1 1 = all linear combinations of columns of M with integer coefficients D = 1 1 −1 1 2 D = def 2 1 0 Sampling density is equal to |M| = |det M|
1. Multidimensional Filter Banks 8 1. Multidimensional Filter Banks 9
Sampling Lattices and Matrices Examples of Multidimensional Sampling
Theorem 1. LAT (A) = LAT (B) if and only if A = BE where E is a unimodular (i.e. det E = ±1) integer matrix. Example: Four basic unimodular matrices (shearing operators)
1 1 1 −1 1 0 1 0 R = , R = , R = , R = . 0 0 1 1 0 1 2 1 1 3 −1 1 Theorem 2. [Smith form] Any integer matrix M can be factorized as M = UDV where U and V are unimodular integer matrices and D is an integer diagonal matrices. Example: For quincunx matrices
1 −1 1 1 ↓ R ↓ Q Q = , Q = 0 1 1 1 −1 1 1 0 1 1 Q0 = R1D0R2 = R2D1R1 and Q1 = R0D0R3 = R3D1R0, R = Q = = R D R 3 −1 1 1 −1 1 0 0 3 where D0 = diag(2, 1) and D1 = diag(1, 2)
1. Multidimensional Filter Banks 10 1. Multidimensional Filter Banks 11 Multidimensional Sampling in Frequency-Domain (1/2) Multidimensional Sampling in Frequency-Domain (2/2)
Upsampling by M Downsampling by M T Xu(ω)= X(M ω), 1 X (ω)= X(M −T (ω − 2πk)). M d |det(M)| Xu(z)= X(z ). T k∈NX(M ) M def m m T Here z = (z 1,..., z d) , where mi is the i-th column of M. Here N (M) is the set of integer vectors of the form Mt, t ∈ [0, 1)d.
1. Multidimensional Filter Banks 12 1. Multidimensional Filter Banks 13
Polyphase Representation Examples of Polyphase Representations Separable lattice
• Polyphase components with respect to the sampling matrix M:
xi[n]= x[Mn + li] 2 0 M = 0 2 d where {li} is the set of |M| integer vectors of the form Mt, t ∈ [0, 1)
• Each polyphase component “lives” on a coset. 2 2 −1 2 2 −1 2 2 −1 −1 2 2 X(z1, z2)= X00(z1, z2)+z1 X10(z1, z2)+z2 X01(z1, z2)+z1 z2 X11(z1, z2) def d Ci = m : m = Mn + li, n ∈ Z , 0 ≤ i ≤|M|− 1 Quincunx lattice |M|−1 d Ci = Z , Ci Cj = ∅, i 6= j. i=0 [ \ 1 1 M = −1 1 • In the z-domain: |M|−1 −li M X(z)= z Xi(z ) −1 −1 −1 i=0 X(z1, z2)= X0(z1z2, z1z )+ z X1(z1z2, z1z ) X 2 1 2
1. Multidimensional Filter Banks 14 1. Multidimensional Filter Banks 15 Polyphase-Domain Analysis Filter Bank in the Polyphase-Domain
Filter and downsample ANALYSIS SYNTHESIS ANALYSIS SYNTHESIS
−l0 D D l0 H0 D D G0 z z x[n] −→ H(z) −→ (↓ M) −→ c[n] −l D l H1 D D G1 z 1 D z 1 + X + Xˆ X Hp Gp Xˆ |M|−1
C(z)= Hi(z)Xi(z)= H(z)x(z) −l l HN−1 D D GN−1 z |D|−1 D D z |D|−1 i=0 X def T def where x(z) = (X0(z),...,X|M|−1(z)) , and H(z) = (H0(z),...,H|M|−1(z)) • Critically sampled: |D| = N Upsample and filter
• Perfect reconstruction: Gp(z)Hp(z)= I|D| c[n] −→ (↑ M) −→ G(z) −→ p[n] • Biorthogonal: Perfect reconstruction + critically sampled
Pi(z)= Gi(z)C(z) or p(z)= G(z)C(z) T −1 T −1 • Orthogonal: Hp(z)= Gp (z ) and Gp(z)Gp (z )= I def T def T where G(z) = (G0(z),...,G|M|−1(z)) , and p(z) = (P0(z),...,P|M|−1(z)) (i.e. Gp(z) is a paraunitary matrix)
1. Multidimensional Filter Banks 16 1. Multidimensional Filter Banks 17
A Reduction Theorem for Biorthogonal Filter Banks Example: Quincunx Filter Bank Theorem 3. [ZhouD:04] Suppose G(z) is an N × N matrix and y0 GN−1(z) is its submatrix obtained by deleting the last row of G(z). H0 Q Q G0 Suppose H(z) is another N × N matrix and HN−1(z) is its submatrix x + xˆ obtained by by deleting the last column of H(z). y1 H1 Q Q G1 Then G(z)H(z)= IN if and only if Possible frequency partitions: GN−1(z)HN−1(z)= IN−1, (1) ω1 ω1 i+N (π, π) (π, π) GN,i(z)= α(z)(−1) det Hi,N−1 (z), (2) 1 0 −1 i+N 0 1 H z α z G z , i,N ( )= ( )(−1) det N−1,i ( ) (3) ω0 ω0 where α(z) = det G(z) is an arbitrary nonzero filter, Hi,N−1 (z) is the submatrix of HN−1(z) obtained by deleting its ith row, and GN−1,i (z) is G z the submatrix of N−1( ) obtained by deleting its ith column. Equivalent biorthogonal condition for FIR filters: If G(z) and H(z) have FIR entries, then additionally α(z)= αzk H0(z)G0(z)+ H0(−z)G0(−z) = 2, and N Corollary 1. A -channel biorthogonal FIR filter bank is completely −k k determined by its first N − 1 channels, and a scaled delay in the last H1(z)= z G0(−z), G1(z)= z H0(−z). channel.
1. Multidimensional Filter Banks 18 1. Multidimensional Filter Banks 19 Filter Design Problem Design via Mapping
1. Start with a 1D solution: Filter design amounts to solve P (1D)(z)+ P (1D)(−z) = 2 and P (1D)(z)= H(1D)(z)G(1D)(z) P (z)+ P (−z) = 2, (easy) 2. Apply a 1D to MD mapping to each filter: where F (1D)(z) → F (MD)(z)= F (1D)(M(z)) P (z)= H0(z)G0(z) (hard) • If M(z) = −M(−z) then the perfect reconstruction condition is preserved Fundamental problem in multi-dimensions: no factorization theorem. P (MD)(z)+ P (MD)(−z)= P (1D)(M(z)) + P (1D)(M(−z))=2 How to avoid MD factorization? • Can design the mapping to preserve and obtain other properties, like One possibility: Map 1D filters to MD filters... linear phase, vanishing moments, directional vanishing moments, ... (McClellan:73, TayK:93, ChenV:93, CunhaD:04)
This only works for biorthogonal filter banks...
1. Multidimensional Filter Banks 20 1. Multidimensional Filter Banks 21
MD Orthogonal FB Design using Cayley Transform Example: Two-Channel Filter Banks
Cayley Orthogonal Paraunitary Transform Para-skew-Hermitian Polyphase-domain: Cayley-domain: Filter Banks matrices matrices
U (z) U (z) H (z) H (z) U(z)= 00 01 H(z)= 00 01 . Definition. [Cayley transform] The Cayley transform of a matrix U(z) is U (z) U (z) H (z) H (z) 10 11 10 11 −1 H(z)= I + U(z) I − U(z) . The paraunitary condition The PSH condition T T U(z)U (z−1)= I H(z−1)= −H (z) Definition. H(z) is a para-skew-Hermitian (PSH) matrix if it satisfies becomes becomes
−1 T −1 −1 −1 H(z )= −H (z), for real coefficients. U00(z)U00(z )+ U01(z)U01(z ) = 1, H00(z )= −H00(z), −1 −1 −1 U00(z)U10(z )+ U01(z)U11(z ) = 0, H11(z )= −H11(z), U z N N −1 −1 −1 Proposition 1. [ZhouDK:04] Suppose ( ) is an × matrix. Then U10(z)U10(z )+ U11(z)U11(z ) = 1. H01(z )= −H10(z). there exists at least one N × N diagonal matrix Λ whose diagonal entries are either 1 or −1 such that I +ΛU(z) is nonsingular. → nonlinear equations → linear equations Theorem 4. [ZhouDK:04] The Cayley transform of a paraunitary matrix is a PSH matrix. Conversely, the Cayley transform of a PSH matrix is a paraunitary matrix.
1. Multidimensional Filter Banks 22 1. Multidimensional Filter Banks 23 Outline 2. Laplacian Pyramid Frames
1. Multidimensional filter banks
2. Laplacian pyramid frames • Framing pyramids • New reconstruction (dual frame)
3. Iterated directional filter banks decomposition reconstruction
• Reason: avoid “frequency scrambling” due to (↓) of the HP channel.
• Laplacian pyramid as a frame operator → tight frame exists.
• New reconstruction: efficient filter bank for dual frame (pseudo-inverse).
2. Laplacian Pyramid Frames 24 2. Laplacian Pyramid Frames 25
Decomposition in the Laplacian Pyramid Usual Reconstruction in the Laplacian Pyramid c
x p d c xˆ H M M G _ + M G +
H G d
xˆ(z)= G(z)C(z)+ d(z)
C(z)= H(z)x(z) Or, C(z) d(z)= x(z) − G(z)H(z)x(z)=(I − G(z)H(z))x(z). xˆ(z)= G(z) I . d(z) S1(z) y(z) Combining gives | {z } | {z } Note that S1(z)A(z) = I (perfect reconstruction) for any H(z) and C(z) H(z) G z = x(z). ( ). d(z) I − G(z)H(z) But... what about noisy pyramids: yˆ = y + e ? y(z) A(z) | {z } | {z }
2. Laplacian Pyramid Frames 26 2. Laplacian Pyramid Frames 27 Frame Analysis A Tight Frame Pyramid
Proposition. The Laplacian pyramid with orthogonal filters is a tight • LP is a frame operator (A) with redundancy. frame with frame bounds equal to 1.
∗ ∗ • It admits an infinite number of left inverses. Proof: Orthogonal condition means G (z)G(z) = 1 and H(z) = G (z). With this, we can directly verify that • Let S be an arbitrary left inverse of A, H(z) A∗(z)A(z)= H∗(z) I − H∗(z)G∗(z) = I. I − G(z)H(z) xˆ = Syˆ = S(y + e)= x + Se. Geometrical interpretation: • The optimal left inverse (minimizing kSk) is the pseudo-inverse (or dual frame) of A: x † T −1 T p[n]= hx[•], g[•− Mk]i g[n − Mk]. A =(A A) A . d kX∈Z c[k] Using the Pythagorean theorem: d • If the noise is white, then among all left inverses, the pseudo-inverse | {z } V minimizes the reconstruction MSE. kxk2 = kpk2 + kdk2 = kck2 + kdk2. p
2. Laplacian Pyramid Frames 28 2. Laplacian Pyramid Frames 29
New Reconstruction for the Laplacian Pyramid Laplacian Pyramids as an Oversampled Filter Bank From the polyphase representation of A(z) and S(z) c H M M G For tight frame with frame bounds 1, the pseudo-inverse of A(z) is d K MM0 F T x 0 0 xˆ H(z) + A†(z)= AT (z)= = G(z) I − G(z)GT (z) . I − G(z)GT (z) d|M|−1 K|M|−1 M M F|M|−1 Which leads to the optimal reconstruction: d c For the usual reconstruction (REC-1)
d xˆ F [1](z)= z−ki, for i = 0,..., |M|− 1. H M_ + M G + i
For the new reconstruction (REC-2)
[2] −ki M This also works for biorthogonal filters H and G. Fi (z)= z − G(z)Hi(z ), for i = 0,..., |M|− 1.
2. Laplacian Pyramid Frames 30 2. Laplacian Pyramid Frames 31 Multilevel Laplacian Pyramids PSfrag Experimental Results (1/2) xˆ 2 G + 2 G + usual xˆ1 rec. 2 2 x LP y yˆ F0 F0 + dec. new xˆ2 2 F1 2 F1 threshold rec.
Frequency responses of equivalent synthesis filters: REC-1 and REC-2
3.5 3.5 Nonlinear approximation: SNR’s of the reconstructed images from the M
3 3 most significant LP coefficients (after 6 levels of decomposition)
2.5 2.5 M 212 214 216 2 2 Barbara REC-1 9.68 12.56 20.94 1.5 1.5 Magnitude Magnitude REC-2 9.87 13.18 21.75 1 1 Goldhill REC-1 12.30 15.79 21.55 0.5 0.5 REC-2 12.60 16.23 22.19 0 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Normalized Frequency (×π rad/sample) Normalized Frequency (×π rad/sample) Peppers REC-1 15.06 20.81 26.77 REC-2 15.62 21.33 27.32 The new reconstruction is crucial for building multiscale directional frames.
2. Laplacian Pyramid Frames 32 2. Laplacian Pyramid Frames 33
PSfrag Experimental Results (2/2) Outline
usual xˆ1 rec. x LP y yˆ 1. Multidimensional filter banks + dec. xˆ new 2 2. Laplacian pyramid frames e rec.
3. Iterated directional filter banks With additive uniform white noise in [0, 0.1] (non-zero mean) • How to obtain directional decomposition with 2D tree-structured filter banks • Local directional bases
usual new rec. rec. SNR = SNR = 17.42 dB 6.28 dB
2. Laplacian Pyramid Frames 34 3. Iterated Directional Filter Banks 35 3. Iterated Directional Filter Banks Our Simplified DFB: Two Building Blocks
• Feature: division of 2-D spectrum into fine slices using tree-structured filter banks. • Frequency splitting by the quincunx filter banks (Vetterli’84). ω2 (π, π) y0 0 1 2 3 Q Q 7 4 6 5 x + xˆ ω 5 6 1 y1 Q Q 4 7
3 2 1 0 (−π, −π) • Shearing by resampling
• Background: Bamberger and Smith (’92) cleverly used quincunx FB’s, modulation and shearing.
• We propose: – a simplified DFB with fan FB’s and shearing – use DFB to construct directional bases
3. Iterated Directional Filter Banks 36 3. Iterated Directional Filter Banks 37
Illustration: The First Two Levels of the DFB How Frequency is Divided into Finer Direction?
Q1 0 Q (l+1) 0 H2k Q1 1 (l+1) (l) Q M Hk Q1 2 Q l 0 M ( ) ≡ Q1 3 Q M (l+1) Using the multirate identity: H(l+1) H ω T 2k+1 M ( ) ≡ H(M ω) M The support configurations of equivalent filters are: 0 1 ω1 ω1 ω1 (π, π) (π, π) (π, π) 0 0 1 0 1 1 2 3 ω0 ω0 ω0 3 2 1 0 (l) l−2 sl(k) Overall sampling: M k = [2 · Di ] · R First level Second level Overall filters ≡ separable sampling, then shearing (doesn’t change lattice
3. Iterated Directional Filter Banks 38 3. Iterated Directional Filter Banks 39 Multichannel View of the Directional Filter Bank Why Critical Sampling Works? H0 G0 y0 S0 S0 ω2 H1 G1 y1 S1 S1 H x + xˆ k π
Sk π ω1 H2l−1 G2l−1 y2l−1 S2l−1 S2l−1
Use two separable sampling matrices: Frequency tiling in the MDFB: l−1 2 0 l−1 k < T l 0 ≤ 2 (“near horizontal” direction) δ (ω − 2πS− m) = 1 for all k = 0,..., 2 − 1; ω ∈ 2. " 0 2# Rk k R Sk = m∈Z2 2 0 X l−1 k < l l−1 2 ≤ 2 (“near vertical” direction) "0 2 # where Rk is the ideal frequency region for Hk.
3. Iterated Directional Filter Banks 40 3. Iterated Directional Filter Banks 41
General Bases from the DFB Example of DFB Impulse Responses
An l-levels DFB creates a local directional basis of l2(Z2): 32 equivalent filters for the first half channels (basically horizontal directions) (l) (l) of a 6-levels DFB that use the Haar filters gk [·− Sk n] 0≤k<2l, n∈Z2 n o (l) • Gk are directional filters:
• Sampling lattices (spatial tiling): 2 2l−1
2 2l−1
3. Iterated Directional Filter Banks 42 3. Iterated Directional Filter Banks 43 References • Multidimensional filter banks – P. P. Vaidyanathan, Multirate Systems and Filter Banks, Prentice- Hall, 1993. – M. Vetterli and J. Kovaˇcevi´c, Wavelets and Subband Coding, Prentice-Hall, 1995. – J. Zhou, M. N. Do, and J. Kovaˇcevi´c, Multidimensional orthogonal filter bank characterization and design using the Cayley transform, IEEE Trans. on Image Proc., to appear.
• Laplacian pyramid frames – M. N. Do and M. Vetterli, “Framing pyramids,” IEEE Trans. on Signal Proc., pp. 2329-2342, Sep. 2003.
• Iterated directional filter banks – M. N. Do,“Directional multiresolution image representations,” Ph.D. thesis, EPFL, December 2001. (Chapter 3)
• Software and downloadable papers: www.ifp.uiuc.edu/∼minhdo
44