Administrivia Scale Invariant Features Why Extract Features?
Total Page:16
File Type:pdf, Size:1020Kb
Administrivia • Exam review session in next class CMPSCI 370: Intro to Computer Vision • Midterm in class (Thursday) Image processing • All topics covered till Feb 25 lecture (corner detection) Scale Invariant Feature Transform (SIFT) • Closed book University of Massachusetts, Amherst March 03, 2015 • Grading issues Instructor: Subhransu Maji • Include all the information needed to grade the homework • Keep the grader happy :-) • Candy wrapper extra credit for participation (5%) 2 Scale invariant features Why extract features? • Motivation: panorama stitching “blob detection” • We have two images – how do we combine them? Source: L. Lazebnik 3 Slide credit: L. Lazebnik 4 Why extract features? Why extract features? • Motivation: panorama stitching • Motivation: panorama stitching • We have two images – how do we combine them? • We have two images – how do we combine them? Step 1: extract features Step 1: extract features Step 2: match features Step 2: match features Step 3: align images Slide credit: L. Lazebnik 5 Slide credit: L. Lazebnik 6 Feature detection with scale selection Scaling • We want to extract features with characteristic scale that matches the image transformation such as scaling and translation (a.k.a. covariance) Corner All points will be classified as Matching regions across scales edges Corner detection is sensitive to the image scale! Source: L. Lazebnik 7 Source: L. Lazebnik 8 Blob detection: basic idea Blob detection: basic idea • Convolve the image with a “blob filter” at multiple scales Find maxima and minima of blob filter response in space minima • Look for extrema (maxima or minima) of filter response in and scale the resulting scale space • This will give us a scale and space covariant detector * = maxima Source: L. Lazebnik 9 Source: N. Snavely 10 Blob filter Recall: sharpening filter Laplacian of Gaussian: Circularly symmetric operator for I = blurry(I)+sharp(I) sharp(I)=I blurry(I) − blob detection in 2D = I e I g ⇤ − ⇤ σ = I (e g ) ⇤ − σ 2 2 2 ∂ g ∂ g ∇ g = + unit impulse ∂x2 ∂y 2 Gaussian Laplacian of Gaussian Source: L. Lazebnik 11 12 Recall: edge detection Edge detection using a Laplacian f signal f Edge 2 Second derivative g edge filter, e.g. d of Gaussian Derivative of a 2 g (Laplacian) Gaussian dx 2 edge response d Edge = zero crossing f*g f ∗ g of second derivative dx2 Source: S. Seitz 13 Source: S. Seitz 14 From edges to blobs Scale selection • edge = ripple • We want to find the characteristic scale of the blob by convolving it with Laplacians at several scales and looking • blob = superposition of two ripples for the maximum response • However, Laplacian response decays as scale increases: maximum Spatial selection: the magnitude of the Laplacian response will achieve a maximum at the center of original signal increasing σ the blob, provided the scale of the Laplacian is (radius=8) “matched” to the scale of the blob Source: L. Lazebnik 15 Source: L. Lazebnik 16 Scale normalization Scale normalization • The response of a derivative of Gaussian filter to a perfect • The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases step edge decreases as σ increases • To keep response the same (scale-invariant), must multiply Gaussian derivative by σ 1 • Laplacian is the second Gaussian derivative, so it must be σ 2π multiplied by σ2 Source: L. Lazebnik 17 Source: L. Lazebnik 18 Effect of scale normalization Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for Original signal Unnormalized Laplacian response blob detection in 2D Scale-normalized Laplacian response & ∂ 2 g ∂ 2 g # Scale-normalized: ∇2 g = σ 2 $ + ! norm $ ∂x2 ∂y 2 ! maximum % " Source: L. Lazebnik 19 Source: L. Lazebnik 20 Scale selection Scale selection • At what scale does the Laplacian achieve a maximum • At what scale does the Laplacian achieve a maximum response to a binary circle of radius r? response to a binary circle of radius r? • To get maximum response, the zeros of the Laplacian have to be aligned with the circle • The Laplacian is given by (up to scale): 2 2 2 (x2 + y 2 − 2σ 2 ) e−(x + y )/ 2σ • Therefore, the maximum response occurs at σ = r / 2. circle r r 0 Laplacian image Laplacian image Source: L. Lazebnik 21 Source: L. Lazebnik 22 Characteristic scale Scale-space blob detector • We define the characteristic scale of a blob as the scale 1. Convolve image with scale-normalized Laplacian at that produces peak of Laplacian response in the blob several scales center characteristic scale T. Lindeberg (1998). "Feature detection with automatic scale selection." International Journal of Computer Vision 30 (2): pp 77--116. Source: L. Lazebnik 23 Source: L. Lazebnik 24 Scale-space blob detector: Example Scale-space blob detector: Example 25 26 Scale-space blob detector Scale-space blob detector: Example 1. Convolve image with scale-normalized Laplacian at several scales 2. Find maxima of squared Laplacian response in scale- space Source: L. Lazebnik 27 Source: L. Lazebnik 28 Efficient implementation Efficient implementation • Approximating the Laplacian with a difference of Gaussians: 2 L = σ (Gxx (x, y,σ ) + Gyy (x, y,σ )) (Laplacian) DoG = G(x, y,kσ ) − G(x, y,σ ) (Difference of Gaussians) 2 2 2 (x2 + y 2 − 2σ 2 ) e−(x + y )/ 2σ Is the Laplacian separable? David G. Lowe. "Distinctive image features from scale-invariant keypoints.” IJCV 60 (2), pp. 91-110, 2004. Source: L. Lazebnik 29 30 From feature detection to description Eliminating rotation ambiguity • Scaled and rotated versions of the same neighborhood will • To assign a unique orientation to circular image windows: give rise to blobs that are related by the same transformation • Create histogram of local gradient directions in the patch • What to do if we want to compare the appearance of these • Assign canonical orientation at peak of smoothed histogram image regions? • Normalization: transform these regions into same-size circles • Problem: rotational ambiguity 0 2 π Source: L. Lazebnik 31 Source: L. Lazebnik 32 SIFT features From feature detection to description • Detected features with characteristic scales and orientations: how should we represent the patches? David G. Lowe. "Distinctive image features from scale-invariant keypoints.” IJCV 60 (2), pp. 91-110, 2004. Source: L. Lazebnik 33 Source: L. Lazebnik 34.