Image Convolution Image Boundaries: “Valid” Convolution If I is m n and H is k `, then J is (m k + 1) (n ` + 1) ⇥ ⇥ − ⇥ − E he i m COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 14 / 26 Image Convolution Image Boundaries: “Full” Convolution If I is m n and H is k `, then J is (m+k 1) (n+` 1) ⇥ ⇥ − ⇥ − [Pad with either zeros or copies of boundary pixels] I I COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 15 / 26 Image Convolution Image Boundaries: “Same” Convolution If I is m n and H is k `, then J is m n ⇥ ⇥ ⇥ i t i COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 16 / 26 Filters Filters What is convolution for? • Smoothing for noise reduction • Image differentiation • Convolutional Neural Networks (CNNs) • ... • We’ll see the first two next, CNNs later • Smoothing and differentiation are examples of filtering: • Local, linear image image transformations ! COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 17 / 26 Filters Smoothing for Noise Reduction if 4 jez Hlj ft0 otherwise Assume: Image varies slowly enough to be locally linear • Assume: Noise is zero-mean and white • x x x x fusex x x x x x x x x x x x x x x x x x x x x th c X 1/5 A r rn N MEAS Mlc I c t N c j j J c j I I KL Is q Eine t IN H n c j I c COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 18 / 26 Filters 2 Dimensions: The Pillbox Kernel BOX FILTER Ig µ a Ft COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 19 / 26 Filters Issues with the Pillbox f o COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 20 / 26 Filters The Gaussian Kernel I o DISCRETE version Hca aye ER.EEEuEEiIEEDA mn zf zfEY trTIttFe e gcx RANK 1 TRUNCATED GAUSSIAN COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 21 / 26 Filters Gaussian versus Pillbox COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 22 / 26 Filters Truncation 1 u2+v2 G(u, v)=e− 2 σ2 The larger σ, the more smoothing • u, v integer, and cannot keep them all • Truncate at 3σ or so • tH COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 23 / 26 Filters Normalization 1 u2+v2 G(u, v)=e− 2 σ2 We want I G I • ⇤ ⇡ For I = c (constant), I G = I • get ⇤ Normalize by computing γ = 1 G, and then let G G/γ • D⇤ COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 24 / 26 Separable Convolution Separability A kernel that satisfies H(u, v)=h(u)`(v) is separable • The Gaussian is separable withIh = `: E.IE.El • 2 2 1 u +v 1 u 2 2 ( ) G(u, v)=e− 2 σ = g(u) g(v) with g(u)=e− 2 σ A separable kernel leads to efficient convolution: EI 7 • h k J(r, c)= H(u, v) I(r u, c v) − − u= h v= k X− X− h k = h(u) `(v) I(r u, c v) − − u= h v= k X− X− h HOOh = h(u) φ(r u, c) where φ(r, c)= `(v)I(r, c v) − − u= h v= h − − H X X COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 25 / 26 Separable Convolution Computational Complexity ympheduots1pixee 2mnops h k General: J(r, c)= u= h v= k H(u, v) I(r u, c v) − − − − Separable: J(r, c)= h h(u) φ(r u, c) where P u= Ph F h − − φ(r, c)= v= h `(v)I(r, c v) − P − = + = + Let m 2Ph 12mops1pixee_r and n 2k 1 General: About 2mn operations perX2n.o pixel pslpixel Separable: About 2m + 2n operations per pixel Example: a When m = n (square kernel), the gain is 2m2/4m = m/2 With m = 20: About 80 operations per pixel instead of 800 COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 26 / 26.