Homography, Transforms, Mosaics
Credits: S. Seitz, R. Collins, J. Hays, K. Grauman, C. Choi, C. Brunner, R. Szeliski, L. Zitnick Robert Collins CSE486, Penn StateApplying Homographies to Remove Perspective Distortion
from Hartley & Zisserman 4 point correspondences suffice for the planar building facade Robert Collins CSE486, Penn State Homographies for Bird’s-eye Views
from Hartley & Zisserman Robert Collins CSE486, Penn State Homographies for Mosaicing
from Hartley & Zisserman What are 2D geometric transformations? Translation
Preserves: Orienta on Translation and rotation Scale Similarity transformations
Similarity transform (4 DoF) = transla on + rota on + scale Preserves: Angles Aspect ratio Shear Affine transformations
Affine transform (6 DoF) = transla on + rota on + scale + aspect ra o + shear
Preserves: Parallelism What is missing?
Canale o
Are there any other planar transforma ons? General affine
We already used these
How do we compute projec ve transforma ons? Homogeneous coordinates
One extra step: Projective transformations a.k.a. Homographies
“keystone” distor ons
Preserves: Straight Lines Finding the transformation
Transla on = 2 degrees of freedom Similarity = 4 degrees of freedom Affine = 6 degrees of freedom Homography = 8 degrees of freedom
How many corresponding points do we need to solve? Image warping with homographies
image plane in front image plane below Richard Szeliski 17 Finding the transformation
- How can we find the transforma on between these images? - How many corresponding points do we need to solve?
Planar Homography
• A projective mapping from one plane to another
• e.g. mapping of points on a ground plane to the image of the camera
camera xi xw center
image plane world (ground) plane
xi Hxw 1 ' H xi xw '44 Homography Projective – mapping between any two PPs with the same center of projection • rectangle should map to a quadrilateral. – straight lines are conserved. – parallel lines are not conserved. PP2 • same as: project, rotate, reproject. called Homography
PP1
p’ H p To apply a homography H • Compute p’ = Hp (regular matrix multiplication) • Convert p’ from homogeneous to image coordinates Planar Homography z key idea: Z = 0 xi xw X x f sc 1000 x x Rt Y y fy cy 0100 2 3 2 3 ' 2 3 2 3 0| 1 Z 1 1 0010 6 1 7 4 5 4 5 4 5 6 7 r r 4 r 5 t X f sc 1000 11 21 31 1 x x r r r t Y = f c 0100 12 22 32 2 y y 2r r r t 3 2Z 3 2 1 3 200103 13 23 33 3 6 00017 6 1 7 4 5 4 5 6 7 6 7 4r r t 5 4 5 f sc 1000 11 21 1 X x x r r t = f c 0100 12 22 2 Y y y 2r r t 3 2 1 3 200103 13 23 3 2 1 3 6 0017 4 5 4 5 6 7 4 5 4 5 homography fx scx r11 r21 t1 X = f c r r t Y matrix 2 y y3 2 12 22 23 2 3 1 r13 r23 t3 1 4 5 4 5 4 5 h11 h21 h31 X = h h h Y 2 12 22 323 2 3 h13 h23 h33 1 4 5 4 5 45 Robert Collins CSE486, Penn State Projection of Planar Points Fitting a homography • Recall: homogeneous coordinates
Converting to homogeneous Converting from homogeneous image coordinates image coordinates
• Equation for homography: Robert Collins CSE486, Penn State Estimating a Homography
Matrix Form:
Equations: Robert Collins CSE486, Penn State Degrees of Freedom?
There are 9 numbers h11,…,h33 , so are there 9 DOF?
No. Note that we can multiply all hij by nonzero k without changing the equations: 10/14/2015
Homography x, y x1, y1 1 1
x2 , y2
x2 , y2
… …
xn , yn xn , yn
To compute the homography given pairs of corresponding points in the images, we need to set up an equation where the parameters of H are the unknowns…
Solving for homographies p’ = Hp wx' a b c x wy' d e f y w g h i 1 Can set scale factor i=1. So, there are 8 unknowns. Set up a system of linear equations: Ah = b where vector of unknowns h = [a,b,c,d,e,f,g,h]T Need at least 8 eqs, but the more the better… Solve for h. If overconstrained, solve using least-squares: min Ah b 2
>> help lmdivide BOARD
13 Image Stitching, Alignment, Blending
Image from http://graphics.cs.cmu.edu/courses/15-463/2010_fall/ 10/14/2015
Mosaics
. . . image from S. from Seitzimage
Obtain a wider angle view by combining multiple images.
Main questions
Alignment: Given two T images, what is the transformation between them?
Warping: Given a T source image and a transformation, what does the transformed output look like?
7 Why Mosaic? Are you getting the whole picture? • Compact Camera FOV = 50 x 35°
Slide from Brown & Lowe Why Mosaic? Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135°
Slide from Brown & Lowe Why Mosaic? Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135° • Panoramic Mosaic = 360 x 180°
Slide from Brown & Lowe Image Mosaics
+ + +
virtual wide-angle camera Goal: Stitch together several images into a seamless composite A pencil of rays contains all views
real synthetic camera camera
Can generate any synthetic camera view as long as it has the same center of projection! Image reprojection
mosaic PP The mosaic has a natural interpretation in 3D • The images are reprojected onto a common plane • The mosaic is formed on this plane • Mosaic is a synthetic wide-angle camera How to do it? Basic Procedure 1. Take a sequence of images from the same position. – Rotate the camera about its optical center. 2. Compute transformation between second image and first. 3. Transform the second image to overlap with the first. 4. Blend the two together to create a mosaic. 5. If there are more images, repeat. Full-view Panorama