Image Convolution Image Boundaries: “Valid” Convolution

If I is m n and H is k `, then J is (m k + 1) (n ` + 1) ⇥ ⇥ ⇥ E he i m

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 14 / 26 Image Convolution Image Boundaries: “Full” Convolution

If I is m n and H is k `, then J is (m+k 1) (n+` 1) ⇥ ⇥ ⇥ [Pad with either zeros or copies of boundary pixels]

I I

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 15 / 26 Image Convolution Image Boundaries: “Same” Convolution

If I is m n and H is k `, then J is m n ⇥ ⇥ ⇥ i

t i

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 16 / 26 Filters Filters

What is convolution for? • Smoothing for noise reduction • Image differentiation • Convolutional Neural Networks (CNNs) • ... • We’ll see the first two next, CNNs later • Smoothing and differentiation are examples of filtering: • Local, linear image image transformations !

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 17 / 26 Filters Smoothing for Noise Reduction if 4 jez Hlj ft0 otherwise Assume: Image varies slowly enough to be locally linear • Assume: Noise is zero-mean and white • x x x x fusex x x x x x x x x x x x x x x x x x x x x th c X 1/5 A r rn N MEAS Mlc I c t N c j j J c j I I KL Is q Eine t IN H n c j I c

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 18 / 26 Filters 2 Dimensions: The Pillbox Kernel

BOX FILTER Ig µ

a Ft

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 19 / 26 Filters Issues with the Pillbox

f

o

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 20 / 26 Filters

The Gaussian Kernel I o DISCRETE version Hca aye ER.EEEuEEiIEEDA mn zf zfEY trTIttFe e gcx RANK 1 TRUNCATED GAUSSIAN

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 21 / 26 Filters Gaussian versus Pillbox

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 22 / 26 Filters Truncation

1 u2+v2 G(u, v)=e 2 2 The larger , the more smoothing • u, v integer, and cannot keep them all • Truncate at 3 or so •

tH

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 23 / 26 Filters Normalization

1 u2+v2 G(u, v)=e 2 2 We want I G I • ⇤ ⇡ For I = c (constant), I G = I • get ⇤ Normalize by computing = 1 G, and then let G G/ • D⇤

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 24 / 26 Separable Convolution Separability A kernel that satisfies H(u, v)=h(u)`(v) is separable • The Gaussian is separable withIh = `: E.IE.El • 2 2 1 u +v 1 u 2 2 ( ) G(u, v)=e 2 = g(u) g(v) with g(u)=e 2 A separable kernel leads to efficient convolution: EI 7 • h k J(r, c)= H(u, v) I(r u, c v) u= h v= k X X h k = h(u) `(v) I(r u, c v) u= h v= k X X h HOOh = h(u) (r u, c) where (r, c)= `(v)I(r, c v) u= h v= h H X X

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 25 / 26 Separable Convolution

Computational Complexity ympheduots1pixee 2mnops h k General: J(r, c)= u= h v= k H(u, v) I(r u, c v) Separable: J(r, c)= h h(u) (r u, c) where P u= Ph F h (r, c)= v= h `(v)I(r, c v) P = + = + Let m 2Ph 12mops1pixee_r and n 2k 1 General: About 2mn operations perX2n.o pixel pslpixel Separable: About 2m + 2n operations per pixel Example: a When m = n (square kernel), the gain is 2m2/4m = m/2 With m = 20: About 80 operations per pixel instead of 800

COMPSCI 527 — Computer Vision Correlation, Convolution, Filtering 26 / 26