Image Rectification

ECE 181b Homework 3 Image Rectiﬁcation

April 20, 2006

The goal of this project is to explore some fundamental concepts of projective geometry related to the problem of image rectification. We will rectify the image of one of the facades of the Bren School of Environmental Science & Management at UCSB (see Figure 1) and verify how points at infinity can be mapped to finite points via homographies. Caveat. To receive full credit your answers must be clearly justified. Make sure to include the relevant steps that were required to obtain the numerical answer.

1 Calculating the Homography

A transformation from the projective space P2 to itself is called homography. A homography is represented by a 3 × 3 matrix with 8 degrees of freedom (scale, as usual, does not matter):

 0      x h1 h4 h7 x 0  y  =  h2 h5 h8   y  0 w h3 h6 h9 w

To estimate the coeﬃcients of the homography H we will use the approach described in class (see the notes that can be found at http://www.ece.ucsb.edu/~manj/ece181b/ homography_zuliani.pdf). The test images can be downloaded from the course website. For sake of convenience we will adopt the coordinate system convention displayed in Figure 2 (for a more detailed image go to http://vision.ece.ucsb.edu/~zuliani/Code/lattice. png).

Question 1 How many point correspondences are necessary to compute the homography that relates Image 1(a) to Image 1(b) under the constrain that the vector h obtained stacking the components of H one on top of the other has unit norm (i.e. khk = 1)? Why?

Answer 1 We need at least 4 point correspondences. In fact each point correspondence provides two equations (for a total of 8 constraints) and we have 9 unknowns. The extra degree of freedom is ﬁxed by imposing khk = 1.

1 (a) (b) Figure 1: Image (a) shows a perspective view of one of the (almost) planar facades of the Bren School. Our goal is to the calculate the warping that will transform image (a) as if the picture was taken with the camera image plane parallel to the facade plane (e.g. image (b)). Thorough all this homework we will refer to the quantities related to Image (b) using the superscript 0.

Question 2 With the help of Matlab, answer the following questions:

• Manually select N point correspondences and estimate the homography that maps the points in Image 1(a) to the points in Image 1(b) (you are expected to provide the numerical values of the homography that you estimated). You may want to use the Matlab function that was described in class (see the course webpage to download the code).

• How is the number of points that are selected aﬀecting the accuracy of the homography estimate?

• Explain why it is crucial that the chosen points do not form a “singular conﬁguration”.

• Provide an example of a singular conﬁguration. 305 Where does the estimated homography map the point x = ? • 379

Suggestion. We suggest to pick a reasonable large number of points, not just 4. To select these points you may want to give a look to the matlab function ginput or, if the image processing toolbox is available, to the fancier function cpselect.

2 Figure 2: The coordinate system convention that is to be adopted in this homework.

Answer 2 The estimate of the homography (calculated using 42 points, under the constraint khk = 1) is given by:  −0.0022 0.0014 −0.9466  H =  0.0002 −0.0019 −0.3224  0.0000 0.0000 −0.0060 The larger is N the more accurate is the estimate of the homography (provided that the set of point correspondences does not include any outliers). If the points were chosen in a singular configuration (e.g. along a line), then the matrix associated to the homogeneous linear system of the DLT algorithm becomes ill conditioned and small amounts of noise in the position of the points will produce large fluctuations in the coefficients of the homography. To calculate the position of the point x we just need to convert to cartesian coordinates the following homogenous expression:

 −0.0022 0.0014 −0.9466   305  Hx =  0.0002 −0.0019 −0.3224   379  0.0000 0.0000 −0.0060 1.0000

298.7273 We therefore obtain x0 = 280.0202

Question 3 With the help of Matlab, answer the following questions:

• After reading the help, use the function WarpHomography (which can be downloaded from the course website) to compute the rectiﬁed version of Image 1(a). Include the images in your answer.

• Comment on the diﬀerences that you notice between the two output images that are aligned to the same coordinate system.

3 (a) (b) Figure 3: The images rendered by the function WarpHomography.

Answer 3 The images produced by the function WarpHomography are shown in Figure 3. The diﬀerences in the overlapping portions of the images are due to the fact that the hypoth- esis that the scene is planar does not hold everywhere (note the portion of the grass in the bottom right corner of Image 1(a) and the depth discontinuities in correspondences of the windows).

2 Homographies and Points at Inﬁnity

In the projective space parallel lines intersect at a point at infinity. An homography can map a point at infinity into a point that has finite cartesian coordinates. Using the images in Figure 1 we will show how this can happen.

Question 4 Consider the two lines shown in Figure 4 (corresponding to Image 1(b)), whose 0 T projective representation is lR = −11 0 3300 (line containing the red segment) and 0 T lG = −11 0 3383 (line containing the green segment). With the help of Matlab, answer the following questions:

• Compute the projective representation q0 of the intersection between the two lines.

• Where is this point located?

• What are the expressions lR and lG (both analytical and numerical) of the lines in the coordinate system of Figure 1(a)?

• Compute their intersection and call it q. What is the cartesian representation of q?

• When you try to position q in Image 1(a), does your result make sense?

• How is q analytically related to q0?

4 0 0 Figure 4: The segments that identify the lines lR and lG.

Answer 4 In the projective space P2, the intersection between two lines can be easily computed taking the cross product between the vectors that represent the lines:   −11 0 3300 0 0 0 0 q = lR × lG = −11 0 3383 ∼  1 

i j k 0 Since the third component of the vector is equal to zero, we are dealing with a point at inﬁnity (as expected for the intersection of two parallel lines). The expression for the lines in the coordinate frame of Image 1(a) can be obtained via the transpose of the homography computed T 0 T 0 in Question 2: lR = H lR and lG = H lG (remember the duality principle). In matlab: 0001 % apply mapping 0002 l R = transpose(H)*l R prime; 0003 % normalize 0004 l R = l R/l R(3); Thus we obtain:  −0.0026  0 lR ∼  −0.0006  1.0000 and:  −0.0019  0 lG ∼  −0.0007  1.0000 The intersection can be once again computed via the cross product:   −0.0026 −0.0006 1.0000 183.2369

q = lR × lG = −0.0019 −0.0007 1.0000 ∼  947.3679 

i j k 1.0000 where the cartesian representation corresponds to the ﬁrst two components of the homogeneous vector. The results makes sense, since q is located on the top left of Image 1(a), where

5 0 the remapped lines lR and lG intersect. Finally q is related to q via the inverse of the homography H, so that q = H−1q0. As mentioned in class, for numerical conditioning reasons, rather than using the Matlab function inv, it is better if we use the backslash operator:

0001 % use backslash operator rather than inv function 0002 q = H\q prime; 0003 % normalize 0004 q = q/q(3);