Introduction to Computer Vision
Total Page:16
File Type:pdf, Size:1020Kb
Edges and Features - 2 CAP 5415, FALL 2019 Credits and Sources ▪This lecture contains open source materials authored by ▪ Dr. Mubarak Shah ▪ Dr. Ajit Rajwade ▪ Dr. David Lowe (inventor of SIFT) ▪ Dr. George Bebis ▪ Dr. Cornelia Fermüller Aspects of Classical Computer Vision Edges & Features Tracking Classification Shape From 3D Edges and Features are precursors to other functions such as tracking, recovering 3D shape from 2D images, and object classification Uses of Features - •Automate object tracking •Point matching for computing disparity •Stereo calibration •Estimation of fundamental matrix •Motion based segmentation •Recognition •3D object reconstruction •Robot navigation •Image retrieval and indexing 푤 푥, 푦 window SIFT: Motivation The Harris operator is not invariant to scale and correlation is not invariant to rotation1. For better image matching, Lowe’s goal was to develop an interest operator that is invariant to scale and rotation. Also, Lowe aimed to create a descriptor that was robust to the variations corresponding to typical viewing conditions. The descriptor is the most-used part of SIFT. 1But Schmid and Mohr developed a rotation invariant descriptor for it in 1997. 4/28/2020 30 Idea of SIFT Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters SIFT Features 4/28/2020 31 Overall Procedure at a High Level 1. Scale-space extrema detection Search over multiple scales and image locations. 2. Keypoint localization Fit a model to detrmine location and scale. Select keypoints based on a measure of stability. 3. Orientation assignment Compute best orientation(s) for each keypoint region. 4. Keypoint description Use local image gradients at selected scale and rotation to describe each keypoint region. 4/28/2020 35 A blob detector for different sizes Lowe’s Scale-space Interest Points Laplacian of Gaussian kernel ◦ Scale normalised (x by scale2) ◦ Proposed by Lindeberg Scale-space detection ◦ Find local maxima across scale/space ◦ A good “blob” detector [ T. Lindeberg IJCV 1998 ] 4/28/2020 40 Lowe’s Scale-space Interest Points: Difference of Gaussians Gaussian is an ad hoc solution of heat diffusion equation Hence k is not necessarily very small in practice 4/28/2020 41 Lowe’s Pyramid Scheme • Scale space is separated into octaves: • Octave 1 uses scale • Octave 2 uses scale 2 • etc. • In each octave, the initial image is repeatedly convolved with Gaussians to produce a set of scale space images. • Adjacent Gaussians are subtracted to produce the DOG • After each octave, the Gaussian image is down-sampled by a factor of 2 to produce an image ¼ the size to start the next level. 4/28/2020 42 Keypoint localization with orientation 832 233x189 initial keypoints 536 729 keypoints after keypoints after gradient threshold ratio threshold 4/28/2020 54 4. Scale Invariant Feature Transform (SIFT) 1. Take a 16 x16 window around interest point (i.e., at the scale detected). 2. Divide into a 4x4 grid of cells. 3. Compute histogram of image gradient directions in each cell (8 bins each). 16 histograms x 8 orientations = 128 features 5. Feature matching For each feature in A, find nearest neighbor in B A B Example: Planar recognition Training images Example: Planar recognition • Reliably recognized at a rotation of 60° away from the camera • Affine fit is an approximation of perspective projection • Only 3 points are needed for recognition 3D object recognition Training images 3D object recognition Only 3 keys are needed for recognition, so extra keys provide robustness Affine model is no longer as accurate Recognition under occlusion Illumination invariance Robot Localization Credits and Sources ▪This lecture contains open source materials authored by ▪ Dr. Mubarak Shah ▪ Dr. Ajit Rajwade ▪ Dr. David Lowe (inventor of SIFT) ▪ Dr. George Bebis ▪ Dr. Cornelia Fermüller.