Robert Collins Robert Collins CSE486, Penn State CSE486, Penn State Administrivia I have put some Matlab image tutorials on Angel. Please take a look if you are unfamiliar with Matlab Lecture 3: or the image toolbox. Linear Operators I have posted Homework 1 on Angel. It is due next Friday at beginning of class. Robert Collins Robert Collins CSE486, Penn State Recall: 2D Gradient CSE486, Penn State Numerical Derivatives See also T&V, Appendix A.2 Gradient = vector of partial derivatives of image I(x,y) = [dI(x,y)/dx , dI(x,y)/dy] Finite forward difference Gradient vector field indicates the direction and slope of steepest ascent (when considering image pixel values as a surface / height map). Finite backward difference Finite central difference More accurate Robert Collins Robert Collins CSE486, Penn StateExample: Spatial Image Gradients CSE486, Penn StateExample: Spatial Image Gradients Ix=dI(x,y)/dx Ix=dI(x,y)/dx I(x+1,y) - I(x-1,y) I(x+1,y) - I(x-1,y) 2 2 I(x,y) I(x,y) Partial derivative wrt x Partial derivative wrt x I(x,y+1) - I(x,y-1) I(x,y+1) - I(x,y-1) 2 Iy=dI(x,y)/dy 2 Iy=dI(x,y)/dy Partial derivative wrt y Partial derivative wrt y Note: From now on we will drop the constant factor 1/2. We can divide by it later. 1 Robert Collins Robert Collins CSE486, Penn State More Specifically CSE486, Penn State More Specifically Image I I2,3 I2,1 dI/dx Image I I2,4 I2,2 dI/dx I1,1 I1,2 I1,3 I1,4 I1,5 I1,1 I1,2 I1,3 I1,4 I1,5 -1 +1 -1 +1 I2,1 I2,2 I2,3 I2,4 I2,5 I2,1 I2,2 I2,3 I2,4 I2,5 I3,1 I3,2 I3,3 I3,4 I3,5 I3,1 I3,2 I3,3 I3,4 I3,5 I4,1 I4,2 I4,3 I4,4 I4,5 I4,1 I4,2 I4,3 I4,4 I4,5 I5,1 I5,2 I5,3 I5,4 I5,5 I5,1 I5,2 I5,3 I5,4 I5,5 Robert Collins Robert Collins CSE486, Penn State More Specifically CSE486, Penn State More Specifically Image I I2,5 I2,3 dI/dx Image I I3,3 I3,1 dI/dx I1,1 I1,2 I1,3 I1,4 I1,5 I1,1 I1,2 I1,3 I1,4 I1,5 -1 +1 I2,1 I2,2 I2,3 I2,4 I2,5 I2,1 I2,2 I2,3 I2,4 I2,5 -1 +1 I3,1 I3,2 I3,3 I3,4 I3,5 I3,1 I3,2 I3,3 I3,4 I3,5 I4,1 I4,2 I4,3 I4,4 I4,5 I4,1 I4,2 I4,3 I4,4 I4,5 I5,1 I5,2 I5,3 I5,4 I5,5 I5,1 I5,2 I5,3 I5,4 I5,5 And so on … Robert Collins Robert Collins CSE486, Penn State Linear Filters CSE486, Penn State Image Filtering • General process: • Example: smoothing by – Form new image whose averaging pixels are a weighted sum – form the average of pixels in a of original pixel values, neighbourhood using the same set of • Example: smoothing with a weights at each point. Gaussian • Properties – form a weighted average of – Output is a linear function pixels in a neighbourhood of the input • Example: finding a derivative – Output is a shift-invariant – form a weighted average of function of the input (i.e. pixels in a neighbourhood shift the input image two pixels to the left, the output is shifted two pixels to the left) Note: The “Linear” in “Linear Filters” means linear combination of neighboring pixel values. Freeman, MIT 2 Robert Collins Robert Collins CSE486, Penn State Linear Filtering CSE486, Penn State Linear Filtering think of this as a weighted sum (kernel specifies the weights): We don’t want to only do this at a single pixel, of course, but want instead to “run the kernel 10*0+5*0+3*0+4*0+5*.5+1*0+1*0+1*1+7*.5 = 7 over the whole image”. Freeman, MIT Freeman, MIT Robert Collins Robert Collins CSE486, Penn State Convolution (2D) CSE486, Penn State Convolution Given a kernel (template) f and image h, the convolution f*h is defined as 1 2 (p,q) 5 f(u,v) h(u,v) 1) Note strange indexing into neighborhood of h(x,y). As a result, f behaves as if rotated by 180 degrees before 3 4 combining with h. f(u,v) h(x-u,y-v) (-p,-q) (x-p,y-q) 2) That doesn’t matter if f has 180 deg symmetry Integral of red area is the 3) If it *does* matter, use cross correlation instead. h(-u,-v) h(x-u,y-v) convolution for this x and y Adapted from Rosenfeld and Kak, Digital Picture Processing Robert Collins Robert Collins CSE486, Penn State Convolution Example CSE486, Penn State Convolution Example f h f * h f 1 1 1 f * h 1 -1 -1 1 -1 -1 -1 2 1 2 2 2 3 ? ? ? ? 2 2 2 3 5? ? ? ? 1 2 -1 1 2 -1 -1 -1 1 2 1 3 3 ? ? ? ? 2 1 3 3 ? ? ? ? 1 1 1 1 1 1 2 2 1 2 ? ? ? ? 2 2 1 2 ? ? ? ? Rotate 1 3 2 2 ? ? ? ? Rotate 1 3 2 2 ? ? ? ? 1 1 1 1 1 1 2*2+1*2+(-1)*2+1*1 = 5 -1 2 1 Apply -1 2 1 Apply -1 -1 1 -1 -1 1 adapted from C. Rasmussen, U. of Delaware 3 Robert Collins Robert Collins CSE486, Penn State Convolution Example CSE486, Penn State Convolution Example f 1 1 1 f * h f 1 1 1 f * h 1 -1 -1 -1 2 1 1 -1 -1 -1 2 1 2 2 2 3 5 4 ? ? 2 2 2 3 5 4 4 ? 1 2 -1 -1 -1 1 1 2 -1 -1 -1 1 2 1 3 3 ? ? ? ? 2 1 3 3 ? ? ? ? 1 1 1 1 1 1 2 2 1 2 ? ? ? ? 2 2 1 2 ? ? ? ? Rotate 1 3 2 2 ? ? ? ? Rotate 1 3 2 2 ? ? ? ? 1 1 1 1 1 1 2*2+1*2+(-1)*2+1*1 = 5 2*2+1*2+(-1)*2+1*1 = 5 -1 2 1 Apply -1 2 1 Apply -1*2+2*2+1*2-1*2-1*1+1*3= 4 -1*2+2*2+1*2-1*2-1*1+1*3= 4 -1 -1 1 -1 -1 1 -1*2+2*2+1*3-1*1-1*3+1*3= 4 Robert Collins Robert Collins CSE486, Penn State Convolution Example CSE486, Penn State Convolution Example f 1 1 1 f * h f f * h 1 -1 -1 -1 2 1 1 -1 -1 1 1 1 2 2 2 3 5 4 4 -2 2 2 2 3 5 4 4 -2 1 2 -1 -1 -1 1 1 2 -1 -1 2 1 2 1 3 3 ? ? ? ? 2 1 3 3 9 ? ? ? 1 1 1 1 1 1 -1 -1 1 2 2 1 2 ? ? ? ? 2 2 1 2 ? ? ? ? Rotate 1 3 2 2 ? ? ? ? Rotate 1 3 2 2 ? ? ? ? 1 1 1 1 1 1 2*2+1*2+(-1)*2+1*1 = 5 2*2+1*2+(-1)*2+1*1 = 5 -1 2 1 Apply -1 2 1 Apply -1*2+2*2+1*2-1*2-1*1+1*3= 4 -1*2+2*2+1*2-1*2-1*1+1*3= 4 -1 -1 1 -1*2+2*2+1*3-1*1-1*3+1*3= 4 -1 -1 1 -1*2+2*2+1*3-1*1-1*3+1*3= 4 -1*2+2*3-1*3-1*3 = -2 -1*2+2*3-1*3-1*3 = -2 1*2+1*2+2*2+1*1-1*2+1*2= 9 Robert Collins Robert Collins CSE486, Penn State Convolution Example CSE486, Penn State Convolution Example f f * h 1 -1 -1 1 1 1 2 2 2 3 5 4 4 -2 1 2 -1 -1 2 1 2 1 3 3 9 6 ? ? 1 1 1 -1 -1 1 2 2 1 2 ? ? ? ? and so on… Rotate 1 3 2 2 ? ? ? ? 1 1 1 2*2+1*2+(-1)*2+1*1 = 5 -1 2 1 Apply -1*2+2*2+1*2-1*2-1*1+1*3= 4 -1 -1 1 -1*2+2*2+1*3-1*1-1*3+1*3= 4 -1*2+2*3-1*3-1*3 = -2 1*2+1*2+2*2+1*1-1*2+1*2= 9 1*2+1*2+1*2-1*2+2*1+1*3-1*2-1*2+1*1= 6 From C. Rasmussen, U. of Delaware 4 Robert Collins Robert Collins CSE486, Penn StatePractical Issue: Border Handling CSE486, Penn StatePractical Issue: Border Handling • Problem: what do we do for border pixels • Different border handling methods specify different ways where the kernel does not completely of defining values for pixels that are off the image. • One of the simplest methods is zero-padding, which we overlap the image? used by default in the earlier example. ??? 0 0 0 0 ... 0 0 0 0 for interior pixels where but what values do we there is full overlap, we use for pixels that are know what to do.