Image Processing Image Processing - Low Level

• Edge detection masks • Gradient Detectors Image Processing representation, compression,transmission • Second Derivative - Laplace detectors

• Edge Linking image enhancement • edge/feature finding

image "understanding" Computer Vision

High Level

UFO - Unidentified Flying Object Origin of Edges

surface normal discontinuity

depth discontinuity

surface color discontinuity

illumination discontinuity

Edges are caused by a variety of factors

1 Edge detection Profiles of image intensity edges

Step Edge Line Edge

How can you tell that a pixel is on an edge? gray value gray gray value gray

x x edge edge edge edge

Edge Detection by Differentiation Edge detection

1D image f(x) gray value gray 1. Detection of short linear edge segments (edgels) x

2. Aggregation of edgels into extended edges 1st derivative f'(x)

threshold |f'(x)| - threshold

Pixels that passed the threshold are Edge Pixels

2 Image gradient The discrete gradient The gradient of an image: How can we differentiate a digital image f[x,y]? • Option 1: reconstruct a continuous image, then take gradient • Option 2: take discrete derivative (finite difference) The gradient points in the direction of most rapid change in intensity

How would you implement this as a convolution?

The gradient direction is given by:

The edge strength is given by the gradient magnitude

Effects of noise Solution: smooth first Consider a single row or column of the image • Plotting intensity as a function of position gives a signal

Where is the edge? Where is the edge? Look for peaks in

3 Derivative theorem of convolution Laplacian of Gaussian

Consider This saves us one operation:

Laplacian of Gaussian operator

Where is the edge? zero crossing

2D edge detection filters Discrete derivatives The Sobel operators below are very commonly used • Better approximations of the derivatives exist

Laplacian of Gaussian 1 -1 0 1 1 1 2 1 8 -2 0 2 8 0 0 0 -1 0 1 -1 -2 -1

Gaussian derivative of Gaussian

Irwin Sobel Palo Alto,2007 is the Laplacian operator:

4 Gradient operators Example Edge

Original Gradient-X Gradient-Y

(a): Roberts’ cross operator (b): 3x3 (c): (d) 4x4 Prewitt operator Gradient-Magnitude Gradient-Direction

Discrete Laplacian Variations on Laplacian Operators:

2 2 2 ∂ ∂ Laplacian Operator ∇ = ( 2 + 2 ) ∂x ∂y 1 -2 1 0 -1 0 -1 -1 -1 -2 4 -2 -1 4 -1 -1 8 -1 Approximation of second derivative (horizontal): 1 -2 1 0 -1 0 -1 -1 -1 ∂f2(x,y) ∂2x = f''(x,y) = f'(x+1,y) - f'(x,y) = = [f(x+1,y) - f(x,y)] - [f(x,y) - f(x-1,y)] All are approximations of: = f(x+1,y) - 2 f(x,y) + f(x-1,y)

= convolution with: [ 1 -2 1] 1 Approximation of second derivative (vertical): convolution with -2 1

1 0 -1 0 [ 1 -2 1] * -2 = -1 4 -1 1 0 -1 0

5 Example of Laplacian Edge Detection Laplacian ~

- =

DOG = Difference of Gaussians

DOG in 2D Edge detection by subtraction

Gaussian delta function

original Laplacian of Gaussian

6 smoothed – original smoothed (5x5 Gaussian) (scaled by 4, offset +128)

Derivatives by Spectral Analysis The above maximization amounts to finding the maximum eigen-vector of a 2x2 matrix: The gradient vector at location (x,y) is: ⎛ ∂f x, y()⎞ ⎡ f 2 f f ⎤ ⎜ ⎟ ∑ x ∑ x y ⎛ f x ()x, y ⎞ ∂x T ⎢ x, y x,y ⎥ ⎜ ⎟ ⎜ ⎟ arg max u 2 u = u =1 ⎢ ⎥ ⎜ ()⎟ ∂f ()x, y ∇f f x f y f y ∇f f y x, y ⎜ ⎟ ⎢∑ ∑ ⎥ ⎝ ⎠ ⎜ ⎟ u ⎣ x, y x, y ⎦ u ⎝ ∂y ⎠ Each entry in the 2x2 matrix can be implemented by a convolution

For a local neighborhood Nx,y this direction also maximizes: λ2

2 edge ⎡ f x ()()x1, y1 f y x1, y1 ⎤ ⎢ ()()⎥⎛ux ⎞ arg max f x x2 , y2 f y x2 , y2 ⎜ ⎟ u =1 ⎢ ⎥⎜ ⎟ corner ⎝u y ⎠ ⎣⎢ M M ⎦⎥

smooth edge

λ1

7 Optimal Edge Detection: Canny Canny Edge detection – Step 1

• Convolve with derivatives (x and y) of Gaussians. Assume: • Calculate the magnitude of gradients. • Linear filtering • Additive iid Gaussian noise

Original Edge detector should have: • Good Detection. Filter responds to edge, not noise. • Good Localization: detected edge near true edge. • Single Response: one per edge.

J. Canny: A Computational Approach to Edge Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 8, No. 6, Nov. 1986.

F =f*D *G F =f*D *G 2 2 1/2 A x B y ((FA ) + (FB) )

Canny Edge detection – Step 2 • Non-Maximum suppression (Thinning): Check if pixel is local maximum along gradient direction Magnitude NonMaximum Supression • requires checking interpolated pixels p and r

8 Gradient thresholding thinning (non-maximum suppression)

Canny Edge detection – Step 3: Hysteresis thresholding • Predicting the next edge point • Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s). Edge Pixel • Check that maximum value of gradient value is High Threshold sufficiently large • drop-outs? use hysteresis Possible Edge Pixel • use a high threshold to start edge curves and a low Low Threshold threshold to continue them.

Non-Edge Pixel

9 Effect of Scale Effect of σ (Gaussian kernel size) Magnitude (NonMax Suppressed)

original

Low High Hysteresis Threshold Threshold Threshold

Canny with σ = 1 Canny with σ = 2

An edge is not a line... Finding lines in an image

Option 1: • Search for the line at every possible position/orientation • What is the cost of this operation?

Option 2: • Use a voting scheme: Hough transform

How can we detect lines ?

10 The Hough Transform Finding lines in an image

y b y b

b0 y0

x m0 m x0 x m image space Hough space image space Hough space

• A line in the image space corresponds to a point in Hough space • A point in the image space corresponds to a line in Hough space

Demo: http://www.rob.cs.tu-bs.de/content/04-teaching/06-interactive/Hough.html

Hough transform algorithm Image Domain Hough Domain Typically use a different parameterization (why?)

• d is the perpendicular distance from the line to the origin • θ is the angle this perpendicular makes with the x axis d y (θ0,d0) d θ x θ Image space Hough space

y d θ x d y

x θ

11 Hough Transform Algorithm Hough Transform Example

y d Original square image

x θ Basic Hough transform algorithm 1. Initialize H[d, θ]=0 2. for each edge point I[x,y] in the image for θ = 0 to 180 H[d, θ] += 1 3. Find the value(s) of (d, θ) where H[d, θ] is maximum 4. The detected line in the image is given by What’s the running time (measured in # votes)? Hough Transform Reconstructed (s,θ) space line segments

Hough Transform Example Extensions Original Edges Hough Transform • Use the gradient directions. • give more votes for stronger edges. • Distributed votes. • The same procedure can be used with circles, squares, or any other shape.

Results

12 Hough Transform for Circles

Image Domain Hough Domain

r

(x ,y ) 0 0 (x ,y ,r) y r 0 0 T H E E N D

y x 0 Image space Hough space x0

2 2 2 r = (x-x0) +(y-y0)

13