Robert Collins CSE486, Penn State
Lecture 16: Planar Homographies Robert Collins CSE486, Penn State Motivation: Points on Planar Surface
y
x Robert Collins CSE486, Penn State Review : Forward Projection
World Camera Film Pixel Coords Coords Coords Coords U X x u M M V ext Y proj y Maff v W Z
U X Mint u V Y v W Z
U M u V m11 m12 m13 m14 v
W m21 m22 m23 m24
m31 m31 m33 m34 Robert Collins CSE486, PennWorld State to Camera Transformation
PC PW W
Y
X U R Z C V Rotate to Translate by - C align axes (align origins)
PC = R ( PW - C ) = R PW + T Robert Collins CSE486, Penn State Perspective Matrix Equation X (Camera Coordinates) x = f Z Y X y = f x' f 0 0 0 Z Y y' = 0 f 0 0 Z z' 0 0 1 0 1
p = M int ⋅ PC Robert Collins CSE486, Penn State Film to Pixel Coords
2D affine transformation from film coords (x,y) to pixel coordinates (u,v): X u’ a11 a12xa'13 f 0 0 0 Y v’ a21 a22 ya'23 = 0 f 0 0 w’ Z 0 0z1' 0 0 1 0 1 M M aff proj
u = Mint PC = Maff Mproj PC Robert Collins CSE486, Penn StateProjection of Points on Planar Surface Perspective projection y Film coordinates x
Point on plane Rotation + Translation Robert Collins CSE486, Penn State Projection of Planar Points Robert Collins CSE486, Penn StateProjection of Planar Points (cont)
Homography H (planar projective transformation) Robert Collins CSE486, Penn StateProjection of Planar Points (cont)
Homography H (planar projective transformation)
Punchline: For planar surfaces, 3D to 2D perspective projection reduces to a 2D to 2D transformation.
Punchline2: This transformation is INVERTIBLE! Robert Collins CSE486, Penn State Special Case : Frontal Plane
What if the planar surface is perpendicular to the optic axis (Z axis of camera coord system)?
Then world rotation matrix simplies: Robert Collins CSE486, Penn State Frontal Plane
So the homography for a frontal plane simplies:
Similarity Transformation! Robert Collins CSE486, Penn State Convert to Pixel Coords
Internal camera a11 a12 a13 v pixels parameters a21 a22 a23 0 0 1 u Robert Collins CSE486, Penn State Planar Projection Diagram
Here’s where transformation groups get useful! Robert Collins CSE486, Penn State General Planar Projection
H1
H A1 Robert Collins CSE486, Penn State General Planar Projection
H1
H-1 A1 Robert Collins CSE486, Penn State Frontal Plane Projection
S1
A A1 Robert Collins CSE486, Penn State Frontal Plane Projection
S1
A-1 A1 Robert Collins CSE486, Penn State General Planar Projection
H1 H2
H Robert Collins CSE486, Penn State Summary: Planar Projection Pixel coords Internal u params
Perspective v projection y Homography x
Point on plane Rotation + Translation Robert Collins CSE486, Penn StateApplying Homographies to Remove Perspective Distortion
from Hartley & Zisserman 4 point correspondences suffice for the planar building facade Robert Collins CSE486, Penn State Homographies for Bird’s-eye Views
from Hartley & Zisserman Robert Collins CSE486, Penn State Homographies for Mosaicing
from Hartley & Zisserman Robert Collins CSE486, Penn State Two Practical Issues
How to estimate the homography given four or more point correspondences (will derive L.S. solution now)
How to (un)warp image pixel values to produce a new picture (last class) Robert Collins CSE486, Penn State Estimating a Homography
Matrix Form:
Equations: Robert Collins CSE486, Penn State Degrees of Freedom?
There are 9 numbers h11,…,h33 , so are there 9 DOF?
No. Note that we can multiply all hij by nonzero k without changing the equations: Robert Collins CSE486, Penn State Enforcing 8 DOF
One approach: Set h33 = 1.
Second approach: Impose unit vector constraint
Subject to the constraint: Robert Collins CSE486, Penn State L.S. using Algebraic Distance
Setting h33 = 1
Multiplying through by denominator
Rearrange Robert Collins CSE486, Penn State Algebraic Distance, h33=1 (cont) 2N x 8 8 x 1 2N x 1 Point 1
Point 2
Point 3
Point 4
additional points Robert Collins CSE486, Penn State Algebraic Distance, h33=1 (cont)
Linear 2Nx8 8x1 2Nx1 equations A h = b
8x2N 2Nx8 8x1 8x2N 2Nx1 Solve: AT A h = AT b 8x8 8x1 8x1 (AT A) h = (AT b) -1 h = (AT A) (AT b)
Matlab: h = A \ b Robert Collins CSE486, Penn State Caution: Numeric Conditioning
R.Hartley: “In Defense of the Eight Point Algorithm”
Observation: Linear estimation of projective transformation parameters from point correspondences often suffer from poor “conditioning” of the matrices involves. This means the solution is sensitive to noise in the points (even if there are no outliers).
To get better answers, precondition the matrices by performing a normalization of each point set by: • translating center of mass to the origin • scaling so that average distance of points from origin is sqrt(2). • do this normalization to each point set independently Robert Collins CSE486, Penn State Hartley’s PreConditioning
-1 -1 PointSet1 H = T2 S2H? Hnorm S1 T1 PointSet2
Translate T center of mass T2 1 to origin
Scale so average S1 point dist is sqrt(2) S2
Estimate Hnorm Robert Collins CSE486, Penn State A More General Approach
What might be wrong with setting h33 = 1?
If h33 actually = 0, we can’t get the right answer. Robert Collins CSE486, Penn State Algebraic Distance, ||h||=1
||h|| = 1
Multiplying through by denominator
Rearrange = 0 = 0 Robert Collins CSE486, Penn StateAlgebraic Distance, ||h||=1 (cont)
2N x 9 9 x 1 2N x 1
4 P O I N T S
additional points Robert Collins CSE486, Penn StateAlgebraic Distance, ||h||=1 (cont)
Homogeneous 2Nx9 9x1 2Nx1 equations A h = 0
Solve: 9x2N 2Nx9 9x1 9x2N 2Nx1 AT A h = AT 0
9x9 9x1 9x1 (AT A) h = 0 SVD of ATA = U D UT
Let h be the column of U (unit eigenvector) associated with the smallest eigenvalue in D. (if only 4 points, that eigenvalue will be 0)