Edges

Edge Detection, Corner Detection Lines [Nice to see you again] Introduction to Computer Vision CSE 152 1. Object boundaries Lecture 10 2. Surface normal discontinuities 3. Reflectance (albedo) discontinuities 4. Lighting discontinuities (shadow boundaries) CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Effects of Noise Edge is Where Change Occurs: 1-D • Change is measured by derivative in 1D • Consider a single row or column of the image – Plotting intensity as a function of position gives a signal Ideal Edge Smoothed Edge

First Derivative

Second Derivative

• Biggest change, derivative has maximum magnitude Where is the edge?? • Or 2nd derivative is zero. CSE152, Winter 2014 (from Srinivasa Narasmihan) Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Implementing 1-D Edge Detection Canny Edge Detector

1. Filter out noise: convolve with Gaussian 1. Smooth image by filtering with a Gaussian 2. Compute gradient at each point in the image. 2. Take a derivative: convolve with [-1 0 1] 3. At each point in the image, compute the direction – We can combine 1 and 2. of the gradient and the magnitude of the gradient. 4. Perform non-maximal suppression to identify 3. Find the peak of the magnitude of the convolved candidate edgels. image: Two issues: 5. Trace edge chains using hysteresis tresholding. – Should be a local maximum. – Should be sufficiently high.

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

1 2D Edge Detection: Canny Gradient • Given a function f(x,y) -- e.g., intensity is f 1. Filter out noise • Gradient equation: – Use a 2D Gaussian Filter. J = G * I 2. Take a derivative -- gradient • Represents direction of most rapid change in intensity – Compute the magnitude of the gradient – Compute the direction of the gradient € • Gradient direction:

• The edge strength is given by the gradient magnitude

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Gradients: $∂ I /∂x' ∇I = & ) %∂ I /∂y( Is this dI/dx or dI/dy?

€ σ = 1 σ = 2

There are three major issues: 1. The gradient magnitude at different scales is different; which scale should we choose? 2. The gradient magnitude is large along thick trail; how do we identify the significant points? y 3. How do we link the relevant points up into curves? x CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Non-maximum suppression Magnitude of Image Gradient Loop over every point q in the image, decide whether q p is a candidate edge point Tangent: ∇I(p) Orthogonal Using gradient direction at to gradient direction q, find two points p and r on Normal: q adjacent rows (or columns). € In direction ∇I(q) of grad ∇I(q) > ∇I(p) and If ∇I(q) > ∇I(r) We wish to mark points along a curve where the magnitude is biggest. r We can do this by looking for a maximum along a slice orthogonal to the curve € ∇I(r) then q is a candidate edgel (non-maximum suppression). These points should form a curve. € €

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

2 Non-maximum suppression The Canny Edge Detector Loop over every point q in the image, decide whether q p is a candidate edge point

∇I(p) Using gradient direction at q, find two points p and r on q adjacent rows (or columns). € p & r are found by ∇I(q) interpolation ∇I(q) > ∇I(p) and If r ∇I(q) > ∇I(r) € ∇I(r) then q is a candidate edgel

€ € original image (Lena)

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 (Example from Srinivasa Narasmihan) Intro Computer Vision

The Canny Edge Detector The Canny Edge Detector

magnitude of the gradient After non-maximum suppression

CSE152, Winter 2014 (Example from Srinivasa Narasmihan) Intro Computer Vision CSE152, Winter 2014 (Example from Srinivasa Narasmihan) Intro Computer Vision

An Idea: Single Threshold An OK Idea: Single Threshold

T=15 T=5

1. Smooth Image 1. Smooth Image 2. Compute gradients & Magnitude 2. Compute gradients & Magnitude 3. Non-maximal supression 3. Non-maximal supression 4. Compare to a threshold: T 4. Compare to a threshold: T CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

3 A Better Idea: Linking + Two Tresholds A Better Idea: HysteresisThresholding

• Define two thresholds τlow and τhigh. Linking: Assume the • Starting with output of nonmaximal supression, find a marked point q is an point q where ∇ I ( q ) > τ , and ∇ I ( q ) is a local maximum. edge point. Then we 0 o high o • Start tracking an edge chain at pixel location q in one of construct the tangent to 0 the two directions q the edge curve (which is normal to the gradient at • Stop when€ gradient magnitude < τlow. that point) and use this – i.e., use a high threshold to start edge curves and a low to predict the next threshold to continue them. ∇I(q) points (either r or s). q0 τhigh

€ τlow

Position along edge curve CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Single Threshold

T=15 T=5

Hysteresis

Th=15 Tl = 5

Hysteresis thresholding

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

coarse scale, fine scale High threshold high threshold

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

4 Why is Canny so Dominant

• Still widely used after 20 years. coarse 1. Theory is nice scale 2. Details good (magnitude of gradient, non-max Low high threshold suppression). 3. Hysteresis thresholding an important heuristic. 4. Code was distributed.

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Feature extraction: Corners and blobs

Corner Detection

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Why extract features? Why extract features? • Motivation: panorama stitching • Motivation: panorama stitching – We have two images – how do we combine them? – We have two images – how do we combine them?

Step 1: extract features Step 2: match features

CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

5 Why extract features? Finding Corners • Motivation: panorama stitching – We have two images – how do we combine them?

Step 1: extract features Intuition: Step 2: match features • Right at corner, gradient is ill-defined. Step 3: align images • Near corner, gradient has two different values. CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

The Basic Idea Distribution of gradients for different • We should easily recognize the point by image patches looking through a small window dI/dy dI/dy dI/dy

dI/dx dI/dx dI/dx

“flat” region: “edge”: “corner”: no change in no change along significant all directions the edge change in all Source: A. Efros CSE152, Winter 2014 direction directions Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Finding Corners Because C is a symmetric positive definite matrix, it For each image location (x,y), we create a can be factored as: matrix C(x,y): " % " % Gradient with respect to x, −1 λ1 0 T λ1 0 Sum over a small region times gradient with respect to y C = R $ 'R = R $ 'R 0 0 #$ λ2 &' #$ λ2 &' # I 2 I I & ∑ x ∑ x y where R is a 2x2 rotation matrix and λ1 and λ2 C(x,y) = % ( are non-negative. I I I 2 $%∑ x y ∑ y '( 1. λ1 and λ2 are the Eigenvalues of C. 2. The columns of R are the Eigenvectors of C. 3. Eigenvalues can be fond by solving the characteristic equation det(C-λ I) = 0 for λ. Matrix is symmetric WHY THIS? CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision €

6 Example: Assume R=Identity (axis aligned) So, to detect corners What is region like if: • Filter image with a Gaussian.

1. λ1 = 0, λ2 > 0? • Compute the gradient everywhere.

2. λ2 = 0, λ1 > 0? • Move window over image, and for each

3. λ1 = 0 and λ2 = 0? window location:

4. λ1 >> 0 and λ2 >> 0? 1. Construct the matrix C over the window.

2. Use linear algebra to find λ1 and λ2. 3. If they are both big, we have a corner.

1. Let e(x,y) = min(λ1(x,y), λ2(x,y)) 2. (x,y) is a corner if it’s local maximum of e(x,y) and e(x,y) > τ

Parameters: Gaussian std. dev, window size, threshold CSE152, Winter 2014 Intro Computer Vision CSE152, Winter 2014 Intro Computer Vision

Corner Detection Sample Results

Threshold=25,000 Threshold=10,000

Threshold=5,000

CSE152, Winter 2014 Intro Computer Vision

7