Edges 09-Oct-2018 1 and Ranjay and Krishna Ranjay
Niebles
Stanford Vision and LearningStanford Lab Juan Carlos Carlos Juan Lecture: Edge Detection Edge Lecture: Stanford University Stanford Edges 09-Oct-2018 2
Canny Canny edge detector Hough Transform A simple edge detector Sobel detector edge Edge detection Gradients Image
Forsyth andPonce, Computer Vision, Chapter 8 Somebackgroundreading: • • • • • • What we will learn today Stanford University Stanford Edges 09-Oct-2018 3
Canny Canny edge detector Hough Transform A simple edge detector Sobel detector edge Edge detection Gradients Image
Somebackgroundreading: Forsyth andPonce, Computer Vision, Chapter 8 • • • • • • What we will learn today Stanford University Stanford Edges 09-Oct-2018 4 Stanford University Stanford Edges 09-Oct-2018 5 - -
200 B.C.;
– Lleras , Peru, Peru, , , France, France, , Zhou (1427 Zhou
Line drawing by 7 by Line drawing
Cave Cave painting at Aerial photograph of
Shen (2010 A.D.). A.D.). (2010 year old I. paper, China; paper, mountain top, ink on 1509 A.D.): onPoet a about 700 about 700 geoglyphs monkey as part of Lines Nazca the the picture of a picture the Chauvet about 30,000 B.C.; Stanford University Stanford
(D) (C)
(B) (A)
Edges 09-Oct-2018 6 Hubel & Wiesel, 1960s Wiesel, & Hubel Stanford University Stanford Edges 09-Oct-2018 7 vision studies
Stanford University Stanford
(mammalian) We know edges are special from human human from special are edges know We Hubel & Wiesel, 1960s Wiesel, & Hubel Edges 09-Oct-2018 8
vision studies
(mammalian) We know edges are special from human human from special are edges know We Stanford University Stanford Edges 09-Oct-2018 9 PNAS, 2011 , Fei - Fei , Beck & & , Beck Caddigan Walther, Chai, Stanford University Stanford Edges 09-Oct-2018 10 Source: D. Lowe level knowledge) level - Identify sudden changes changes sudden Identify artist’s line (but artistdrawing
information from the image can be be can image the from information edges the in encoded pixels than compact More Intuitively, most semantic and shapeIntuitively,
– –
is also using object using also is Ideal: Goal: image an in (discontinuities)
Stanford University Stanford Edge detection
• • Edges 09-Oct-2018 11 point Source: J. Hayes point Vanishing (at infinity)(at Vertical vanishing Vertical line point Vanishing Vanishing
Recover geometry and viewpoint Extract information, recognize objects recognize information, Extract
•
• Why doWhy we care about edges? Stanford University Stanford Edges 09-Oct-2018 12
surface color discontinuity color surface illumination discontinuity surface normal discontinuity normal surface discontinuity depth
Stanford University Stanford Origins of edges Edges 09-Oct-2018 13
Surface Surface normal discontinuity
Stanford University Stanford Closeup Closeup of edges Edges 09-Oct-2018 14 discontinuity
Depth
Stanford University Stanford Closeup Closeup of edges Edges 09-Oct-2018 15
Surface Surface color discontinuity
Stanford University Stanford Closeup Closeup of edges Edges 09-Oct-2018 16
Canny Canny edge detector Hough Transform A simple edge detector Sobel detector edge Edge detection Gradients Image
• • • • • • What we will learn today Stanford University Stanford
Edges 17 09-Oct-2018 Derivatives Derivatives in 1D Stanford University Stanford Edges 09-Oct-2018 18
example - Derivatives Derivatives in 1D Stanford University Stanford Edges 09-Oct-2018 19
example - Derivatives Derivatives in 1D Stanford University Stanford
Edges 20 09-Oct-2018 Discrete Derivative Discrete in 1D Stanford University Stanford Edges 09-Oct-2018 21
Central Forward Backward Types of Discrete derivative in 1D Stanford University Stanford Edges 09-Oct-2018 22 1] 1] - - 1 1 0] -
Central: Central: [ 1 0 Forward: [ Backward filter:Backward [0 1
• •
• 1D discrete derivate filters derivate 1D discrete Stanford University Stanford Edges 09-Oct-2018 23 1] -
Backward filter:Backward [0 1
• 1D discrete derivate filters derivate 1D discrete Stanford University Stanford Edges 09-Oct-2018 24 1] - 1 1 0] -
Forward: [ Backward filter:Backward [0 1
•
• 1D discrete derivate filters derivate 1D discrete Stanford University Stanford
Edges 25 09-Oct-2018 1D discrete derivate example derivate 1D discrete Stanford University Stanford
Edges 26 09-Oct-2018 Discrete derivate inderivate Discrete 2D Stanford University Stanford
Edges 27 09-Oct-2018 Discrete derivate inderivate Discrete 2D Stanford University Stanford
Edges 28 09-Oct-2018 Discrete derivate inderivate Discrete 2D Stanford University Stanford
Edges 09-Oct-2018 29 What does this filter do? 2D discrete derivative filters derivative 2D discrete Stanford University Stanford
Edges 09-Oct-2018 30 What about this filter? 2D discrete derivative filters derivative 2D discrete Stanford University Stanford Edges 09-Oct-2018 31
example - 2D discrete derivative derivative 2D discrete Stanford University Stanford Edges 09-Oct-2018 32 this filter?
example - What happens when we apply 2D discrete derivative derivative 2D discrete Stanford University Stanford Edges 09-Oct-2018 33
example - What happens when we apply this filter? 2D discrete derivative derivative 2D discrete Stanford University Stanford Edges 09-Oct-2018 34
example Now let’s try the other filter! - 2D discrete derivative derivative 2D discrete Stanford University Stanford Edges 09-Oct-2018 35 0 0 0 0 0 0 0 0 0 0 10 10 10 10 10 10 10 10 10 10 0 0 0 0 0
example - What happens when we apply this filter? 2D discrete derivative derivative 2D discrete Stanford University Stanford
Edges 36 09-Oct-2018 3x3 image gradient filters gradient 3x3 image Stanford University Stanford Edges 09-Oct-2018 37
Canny Canny edge detector Hough Transform A simple edge detector Sobel detector edge Edge detection Gradients Image
• • • • • • What we will learn today Stanford University Stanford Edges 09-Oct-2018 38 of derivative of firstderivative edges correspond to correspond edges extrema intensity function (alonghorizontal scanline) image An edge is a place of rapid change in the image intensity function • Characterizing edges Characterizing Stanford University Stanford Edges 09-Oct-2018 39 Source: Steve Seitz is given by the gradient magnitude how does this relate to the direction of the edge? the of direction the to relate this does how
• edge strength edge
The gradient of an image: of an image: The gradient The The gradient direction is given by given is direction gradient The The gradient vector points in the direction of most rapid increase in intensity in increase rapid most of direction the in points vector gradient The • Image gradient Image Stanford University Stanford Edges 09-Oct-2018 40 direction? direction - - Gradient Gradient magnitude y direction? about How y - Original Image direction - x Which one is the gradient in the x • Finite differences: example Finite differences: Stanford University Stanford
Edges 09-Oct-2018 41
Gradient Intensity Intensity profile Intensity Stanford University Stanford
Edges 42 09-Oct-2018 Effects of noise Stanford University Stanford Edges 09-Oct-2018 43 Source: S. Seitz S. Source:
Plotting intensity as a function of position gives a signal a gives position of function a as intensity Plotting –
Consider a single orrow column of the image Where is the edge? • Effects of noise Stanford University Stanford Edges 09-Oct-2018 44 Source: D. Forsyth D. Source:
pixels different to their pixels different neighbors (=noise pixels?) to look more like neighbors Smoothing the image should the help,image Smoothing by forcing different from theirfrom different neighbors Generally, the larger the noise the stronger the response Image noise results in pixels that look very very look that in pixels results noise Image
– – – What is to be done? noise Finite difference filters respond strongly to to strongly filters respond Finite difference
•
• Effects of noise Stanford University Stanford Edges 09-Oct-2018 45 Source: D. Forsyth D. Source:
Smoothing the image should help, by forcing pixels different to their to different pixels Smoothingshould forcing theimage help, by neighbors look like more to (=noise neighbors pixels?) Image noise results in pixels lookthat from their very different neighbors response the stronger the noise the larger the Generally,
– – – What is to be done? Finite filtersdifference to noisestrongly respond
Stanford University Stanford •
• Effects of noise Edges 09-Oct-2018 46 Slide credit: Steve Seitz Steve credit: Slide
Gaussian (smoothing * derivative) Mean smoothing
•
• Smoothing with different filters Smoothing with different Stanford University Stanford
Edges 09-Oct-2018 47 Slide credit: Steve Seitz Steve credit: Slide Smoothing with different filters Smoothing with different Stanford University Stanford Edges 09-Oct-2018 48 Source: S. Seitz S. Source: ) g * f ( d dx ) g f g * f * g f * f ( d dx To find edges, look for peaks in • Solution: smooth first Stanford University Stanford Edges 09-Oct-2018 49 Source: S. Seitz S. Source: g d dx * f = ) g * f ( d dx g g f d d dx dx * f saves us one operation: us one saves
This This theorem gives us a very useful Thisproperty: theorem
Stanford University Stanford •
• Derivative Derivative theorem of convolution Edges 09-Oct-2018 50 derivative - x 1] = - * [1 0 * 0 [1 gaussian - 2D Derivative Derivative of Gaussian filter Stanford University Stanford Edges 09-Oct-2018 51 direction - y
direction - x Derivative Derivative of Gaussian filter Stanford University Stanford Edges 09-Oct-2018 52 Derivative Derivative of Gaussian filter Stanford University Stanford Edges 09-Oct-2018 53 Source: D. Forsyth D. Source: 7 pixels 3 pixels 1 pixel Smoothed derivative removes noise, derivative removes Smoothed but blurs edge. Also “scales”. different at edges finds • Tradeoff between smoothing at different scales Stanford University Stanford Edges 09-Oct-2018 54 the optimal detector must minimize the probability of of probability the minimize must detector optimal the
of false negatives (missing real edges) Good detection: false positives (detecting spurious edges caused by noise), as well as that
–
Criteria for an “optimal” edge detector: Designing an edge detector
Stanford University Stanford • Edges 09-Oct-2018 55 the edges detected must be as close as possible to the the to possible as close as be must detected edges the the optimal detector must minimize the probability of of probability the minimize must detector optimal the
of false negatives (missing real edges) Good localization: edges true Good detection: false positives (detecting spurious edges caused by noise), as well as that
– –
Criteria for an “optimal” edge detector: Designing an edge detector
Stanford University Stanford • Edges 09-Oct-2018 56 the edges detected must be as close as possible to the the to possible as close as be must detected edges the the detector must return one point only for each true true each for only point one return must detector the the optimal detector must minimize the probability of of probability the minimize must detector optimal the
Single response: point;edge that is, the ofthe minimize number localtruearound maxima edge of false negatives (missing real edges) Good localization: edges true Good detection: false positives (detecting spurious edges caused by noise), as well as that
– – –
Criteria for an “optimal” edge detector: Designing an edge detector
Stanford University Stanford • Edges 09-Oct-2018 57
Canny Canny edge detector Hough transform A simple edge detector detector Sobel Edge Edge detection Gradients Image
• • • • • • What we will learn today Stanford University Stanford Edges 09-Oct-2018 58 with the original image to derivatives convolved 3 kernels which are ×
one for horizontal changes, and one for vertical uses 3 two calculate approximations of the
• • Sobel Operator Stanford University Stanford Edges 09-Oct-2018 59 differentiation Gaussian smoothing
Smoothing + Smoothing differentiation
• Sobel Operation Stanford University Stanford Edges 09-Oct-2018 60
Angle or direction of the gradient: Magnitude:
•
• Sobel Operation Stanford University Stanford
Edges 09-Oct-2018 61
Stanford University Stanford Sobel Filter example Edges 09-Oct-2018 62
Can miss oblique edges more than horizontal or vertical edges False negatives
– – Poor Localization (Trigger response in multiple adjacent pixels)others certain directions over favors value Thresholding
• • Sobel Filter Problems Stanford University Stanford Edges 09-Oct-2018 63
Canny Canny edge detector Hough Transform A simple edge detector detector Sobel Edge Edge detection Gradients Image
• • • • • • What we will learn today Stanford University Stanford Edges 09-Oct-2018 64 , IEEE Trans. Pattern Pattern Trans. , IEEE 714, 1986. - and and localization edges corrupted additiveby corrupted edges - noise ratio - to - signal A Computational Edge DetectionApproach To
Canny Canny has shown that the first derivative of the Gaussian the that optimizes the operator closely approximates product of Theoretical model: step model: Theoretical Gaussian noise This is probably the most widely used edge detector in detector widely edge used This isthe most probably computer vision
Stanford University Stanford
Analysis and Machine Intelligence, 8:679 J. Canny, J. Canny,
• •
• Canny edge detector edge Canny Edges 09-Oct-2018 65 Maximum Suppression -
Assures minimal Assures response
– Use hysteresis and connectivity analysis to detect edges Apply Non Suppress Suppress Noise Compute gradient magnitude and direction
• • • • Canny edge detector edge Canny Stanford University Stanford Edges 09-Oct-2018 66 original image • Example Stanford University Stanford Edges 09-Oct-2018 67 direction - y
direction - x Derivative Derivative of Gaussian filter Stanford University Stanford Edges 09-Oct-2018 68 Source: J. Hayes Gradient Magnitude Gradient ) Derivative of Gaussian of Derivative DoG - Y Derivative of Gaussian of Derivative - X Compute gradients ( gradients Compute Stanford University Stanford Edges 09-Oct-2018 69 Source: J. Hayes Get orientation at each pixel at orientation Get Stanford University Stanford Edges 09-Oct-2018 70 Gradient Magnitude Gradient ) DoG Derivative of Gaussian of Derivative - Y Compute gradients ( gradients Compute Derivative of Gaussian of Derivative - X Stanford University Stanford Edges 09-Oct-2018 71 Maximum Suppression -
Assures minimal Assures response
– Apply Non Suppress Suppress Noise Compute gradient magnitude and direction
• • • Canny edge detector edge Canny Stanford University Stanford Edges 09-Oct-2018 72 maxima gradient even if it passes threshold -
maximum suppression -
Do this in each marked pixel neighborhood pixel Dothis in eachmarked Suppress all pixels in each direction which are notmaxima which inare eachdirection all pixels Suppress
– – Only eight angle directions possible Edge occurs where gradient reaches a maxima Suppress non
• • • Non Stanford University Stanford Edges 09-Oct-2018 73 %′ %′′ , , #′ #′′ " " ! ! > > % % , , # # " " ! ! % , # " !
is the gradient at pixel (x, y) (x, pixel at gradient the is % ,
# " ! Remove spurious gradients Stanford University Stanford Edges 09-Oct-2018 74 maxima gradient even if it passes threshold -
maximum suppression -
Do this in each marked pixel neighborhood pixel Dothis in eachmarked Suppress all pixels in each direction which are notmaxima which inare eachdirection all pixels Suppress
– – Only eight angle directions possible Edge occurs where gradient reaches a maxima Suppress non
• • • Non Stanford University Stanford Edges 09-Oct-2018 75 Source: D. Forsyth D. Source: Interpolate to get get to Interpolate values. these At q, we have a maximum if the value is larger than p both at those and at r.
maximum suppression - Non Stanford University Stanford Edges 09-Oct-2018 76 Before Before After max max Suppression - Non Stanford University Stanford Edges 09-Oct-2018 77 Maximum Suppression -
Assures minimal Assures response
– Use hysteresis and connectivity analysis to detect edges Apply Non Suppress Suppress Noise Compute gradient magnitude and direction
• • • • Canny edge detector edge Canny Stanford University Stanford Edges 09-Oct-2018 78 Stanford University Stanford Edges 09-Oct-2018 79
If between Low and High, weak edge If less than Low, not an edge If less than Low, thanIf greater High, edgestrong
– – – Avoid streaking near threshold value Define two thresholds: Low and High
• • Hysteresis thresholding Hysteresis Stanford University Stanford Edges 09-Oct-2018 80 pixel” - edge -
Consider its neighbors iteratively then declare it an “edge pixel” if it is connected to an an to connected is it if pixel” “edge an it declare then iteratively neighbors its Consider ‘strong edge pixel’ directly or via pixels between Low and High
– below Low, declare it as a “non below Low, and Low between High above High,above declare it pixel’ edge as an ‘strong
• • • If the gradient at a pixel is pixel a at gradient If the Hysteresis thresholding Hysteresis Stanford University Stanford Edges 09-Oct-2018 81 Source: S. Seitz S. Source: strong edge pixel edge pixels edge weak but connected connected but weak strong edge pixel Hysteresis thresholding Hysteresis Stanford University Stanford Edges 09-Oct-2018 82 Final Canny FinalEdges Canny Stanford University Stanford Edges 09-Oct-2018 83 down to singledown pixel width ridges pixel wide - maximum suppression: -
Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them Thin multi
Thresholding and linking (hysteresis): Find magnitude and orientation of Findgradient and orientation magnitude Non Filter image with Filter x, of image y Gaussian derivatives
– – –
4. 3. 2. 1. Canny edge detector edge Canny Stanford University Stanford Edges 09-Oct-2018 84 Source: S. Seitz S. Source: Canny with Canny with depends depends on desired behavior s Gaussian kernel spread/size) detects fine features detects large scale edges ( s s s large large small • • original The The choice of Effect of Stanford University Stanford Edges 09-Oct-2018 85 Color Human Texture Gradients Combined (e.g.Canny) Stanford University Stanford Edges 09-Oct-2018 86 , Maire, Fowlkes, and Malik. TPAMI 2011 (pdf) 2011 TPAMI Malik. and Fowlkes, , Maire, Arbelaez Source:
years of boundary detection
Stanford University Stanford 45 Edges 09-Oct-2018 87
Canny Canny edge detector Hough Transform A simple edge detector detector Sobel Edge Edge detection Gradients Image
• • • • • • What we will learn today Stanford University Stanford Edges 09-Oct-2018 88 1972). Duda : Hough transform can detect lines, circles and other structures ONLY if ONLY structures other and circles lines, detect can transform : Hough
It can give robust detection under noise and partial occlusion partial and noise under detection robust give can It Caveat is known. equation their parametric decade decade later ( isThe goal find to of the lineslocation in images. The Hough transform (HT) can be used to detect lines. detect be (HT) used to can transform The Hough a images in lines find to used first and 1962) (Hough 1962 in introduced was It
• • • • • Intro to Hough transform Hough to Intro Stanford University Stanford Edges 09-Oct-2018 89
Thus, we have some pixels that may partially describe may of the that boundary pixels some have Thus, we objects. some Assume that we have performed some edge detection, and a thresholding of the edge magnitude image.
•
• Prior to Hough transform Hough to Prior Stanford University Stanford Edges 09-Oct-2018 90 b + i *x a = i ). y i y , i ) i ;y i x
Common to them is theythat satisfy the equation some for set of parameters b) (a, There are many lines passing through the point (x point the through passing lines many are There
– – Straight linesStraight that pass thethat point form have We wish to find sets of pixelsConsider a point of known coordinates ( that make up straight lines.
• • • Detecting lines using Hough transform Stanford University Stanford Edges 09-Oct-2018 91 space. ) a,b space. ( ) a,b ( space gives a line in in line a gives space - 1 ,y 1 ) will) anotherrise line give to space parameterized by x and y. y. x and by parameterized space 2 , y 2 (a, b) b) (a, i y + as variables. i b *x a -
= and
So: a singleSo: ina x point Another(x point We can now consider x and y as parameters a b
– – – – – This is a line in This equation can obviously be rewritten as follows: as follows: obviously be rewritten Thiscan equation
•
• Detecting lines using Hough transform Stanford University Stanford
Edges 92 09-Oct-2018 Detecting lines using Hough transform Stanford University Stanford
Edges 93 09-Oct-2018 Detecting lines using Hough transform Stanford University Stanford Edges 09-Oct-2018 94 space. space. ) a,b ( ) in (x, y) space will will space ) y) (x, in space. 2 ) , y , 2 a,b ( ) and (x ) and 1 , y , 1 ) define a line in the (x, y) plane. (x, y) the in line a ) define 2 y 2 ) and(x 1 , y , 1 space these lines will intersect in(a’ lines these a point b’) space will intersect )
a,b (
parameterize lines parameterize b’) that intersectin in (a’, In In All points on the line defined by (x Two points (x lines in different give risepoints two to These two
• • • • Detecting lines using Hough transform Stanford University Stanford Edges 09-Oct-2018 95 ’) a’,b ) detected as an edge, find the intersection ( 2 by dividing it into cells , y 2 (a b) (a ) and (x and ) 1 ) space.) , y 1 x,y ]] that (a’, b’) belongs to. belongs b’) (a’, that ]] max ,b min b ],[ max space. a ,
min (a, b) (a, a
Cells receiving more than a certain number of counts (also called ‘votes’) are assumed to to assumed are ‘votes’) called (also counts of number certain a than more receiving Cells correspond to lines in ( in in Increase the value of a cell in the range [[ For each pair of points (x
– – – Count the number of times a line intersects a given cell. Quantize the parameter space cells. as the to accumulator space is referred often This quantized
• • • Algorithm for Hough transform Algorithm for Stanford University Stanford Edges 09-Oct-2018 96
Here are the top 20 most voted lines in the image:
• Output of Hough transform Stanford University Stanford Edges 09-Oct-2018 97 b *x + a ρ = θ ? ) θ ρ ( + y*sin + θ
(x y) and (xand y) x*cos
– – Can you figure out the relationship between Polar coordinate representation: We can represent lines as polar coordinates instead of y =
• •
• Other Hough transformations Stanford University Stanford Edges 09-Oct-2018 98 equal ρ equal to ρ . . =0 =0 and axis - θ axis =90 and =90 - x θ line will have have will line line will have have will line ) space, unlike (a b) space. b) (a unlike ) space,
θ vertial horizontal horizontal
ρ A the with intercept the to the intercept with the y A Note that lines in (x y) space are not lines in (
• •
• Other Hough transformations Stanford University Stanford Edges 09-Oct-2018 99 fFIlI?t=3m35s -
https://youtu.be/4zHbI
• Example video Example Stanford University Stanford Edges 09-Oct-2018 100 linear line segments cannot be separated. be cannot segments line linear
-
The length and the position of a line segment cannot be determined. Co Looks for only one single type of object object of type single one only for Looks Can be lines”. “fooled” by “apparent Can be adapted to many types of forms, not just lines just not forms, of types many to adapted be Can Computationally complex objects for with parameters. many Easy implementation gracefully. very data occluded and missing Handles Conceptually simple. Conceptually
– – – – – – – – – Disadvantages: Advantages:
•
• Concluding remarks Stanford University Stanford Edges 09-Oct-2018 101
Canny Canny edge detector Hough Transform A simple edge detector detector Sobel Edge Edge detection Gradients Image
• • • • • • What we will learn today Stanford University Stanford