Geometric Structure Computation from Conics

Pawan Kumar Mudigonda C. V. Jawahar P. J. Narayanan Department of Computing Center for Visual Information Technology Oxford Brookes University International Institute of Information Technology

Oxford, UK Hyderabad, India

[email protected] jawahar, pjn ¡ @iiit.net

Abstract properties of parallel and perpendicular lines have been used for metric rectification of planes [7]. Multiple views This paper presents several results on images of various from uncalibrated cameras have been used for projective configurations of conics. We extract information about the reconstruction of the scene using dense correspondence of plane from single and multiple views of known and un- points; correspondences between the scene and the image known conics, based on planar homography and conic cor- can lead to metric reconstruction of the scene [4]. Vanish- respondences. We show that a single conic section cannot ing points, i.e., the images of points at infinity, have been provide sufficient information. Metric rectification of the used for single view affine reconstruction [2]. plane can be performed from a single view if two conics Point and line correspondences are not available in many can be identified to be images of circles without knowing situations or could be noisy. Higher order parametric curves their centers or radii. The homography between two views can, however, be recovered robustly from images due to the of a planar scene can be computed if two arbitrary conics large number of points defining them. Algorithms for pro- are identified in them without knowing anything specific jective and metric reconstruction of images of conics in the about them. The scene can be reconstructed from a sin- world from two or three views are presented in [8, 11]. The gle view if images of a pair of circles can be identified in problem of reconstruction of quadrics from multiple views two planes. Our results are simpler and require less infor- is addressed in [9]. We discuss the case of single view re- mation from the images than previously known results. The construction in this paper. A method to recover the planar results presented here involve univariate polynomial equa- homography from conics and other curves given the epipo- tions of degree 4 or 8 and always have solutions. Appli- lar geometry can be found in [12]. A discussion on the re- cations to metric rectification, homography calculation, 3D covery of homography and fundamental matrix using corre- reconstruction, and projective OCR are presented to demon- spondences of higher order parametric curves can be found strate the usefulness of our scheme. in [5]. Their results can work for conics, but do not ex- ploit any special property of conics. Their solution involves 1. Introduction a set of complex multivariate non-linear equations which The geometry of multiple views has been a subject of ex- cannot be easily solved. For instance, homography calcu- tensive research in the past few years. Relationships exist lation from conics using their formulation will result in 5 between corresponding entities of images of a scene in mul- non-linear equations in 8 unknowns with infinite solutions. tiple viewpoints. These relationships are well understood We show in this paper that the problem can be reduced to a

for points, lines, and simple primitives [3, 4]. The images univariate, fourth-degree polynomial equation for two con- ¢¤£ ¢ ics. An earlier work [13] that has dealt with homography and in two views of any point ¥ lying on a plane are

related as ¢¦£¨§ © ¢ in homogeneous coordinates. The non- calculation from conics used a minimum of 7 conic cor- © respondences. They used a transformation relating singular projective transformation matrix is called a homography. Homography can be recovered from sufﬁcient conics and derived the homography in a linear fashion us-

number of point and line correspondences [4]. Non-planar ing it. We show that only two conic correspondences are ¢§ points are related by an algebraic constraint ¢£ , enough for calculating the homography and present an efﬁ-

cient algorithm for doing so. A very recent paper describes where is a matrix of rank 2 called the fundamental matrix. Corresponding points are similarly related by a a method to recover camera calibration given two parallel trifocal tensor when three views of a scene are given. (i.e., not necessarily coplanar) circles [14]. Metric rectification of planes involves finding its fronto- We present results for metric rectification, homography parallel view from a projectively transformed view. The estimation, and 3D reconstruction for various configura- A

tions of conic correspondences in this paper. First, we show the conic ¡ are given as

£¤D¥ © §

that the minimum number of conics required for recovering

¡ §

the homography matrix, without any point correspondence, 9

is two. Conic correspondences can be classiﬁed into four §

£¤ ©

cases: (a) single view of coplanar conics, (b) multiple views ¥

£ © £E§ £

of coplanar conics, (c) single view of non-coplanar conics, 7

>¨ >

@ © £F¨ £ £

¡ § © ¡B9 © §

and (d) multiple views of non-coplanar conics. We describe (1)

£ £G¡£

an approach to perform metric rectiﬁcation from a single §

02 0 0

view of coplanar conics using the information that the con- §

9 9

0+2 2 0 2 0

¥IHC¥ H ¥

3 § 3

ics are circles. We also present a method for homography §

0 2 0

§ 3 ¡

estimation from multiple views of general coplanar conics. §

£¤B¥ ©

£ © £ £E§ £ £

Two conic correspondences are all that is required for it. We £

>¨ >

A @

¡ § © ¡ © § § then present a reconstruction algorithm from conics. Unlike

(2)

£ £ £ £ £ £

¡ previous approaches, our method works for a single view of §

non-coplanar conics. O

JLKMNPORQM"S TN¦UM)S M1NWQXMYS N[Z\M"S T]N^UXMYS T S QXMNWUXM"S T

T V

O O

M M M T M T M M M M M

N[Q S N^Z S N^U S S NWO`Z S NWU S Z NaU S

O O _ O O

The next section establishes the minimum number of O

QXMNWUXM)S T ZMNWUXM"S UXMcb conic correspondences required for homography calcula- O tion as two. Section 3 presents metric rectification from There are infinite number of ellipses which undergo dif- a single view of two circles. Section 4 extends this method ferent pure projective transformations to result in any given to finding the homography from two general conic corre- conic. This is because the number of equations that an spondences. Section 5 presents 3D reconstruction from a ellipse-conic correspondence offers is 5 (Eq. 2) and the single view of two planes with two circles each. Section 6 number of unknowns is 7 (5 for the ellipse and 2 for the discusses an application of using conic correspondences in projective transform). Each of these ellipses can be gener- the form of projective OCR. Section 7 provides some con- ated by a unit circle by a similarity transformation followed cluding remarks. by an affine transformation. Therefore, an infinite number of possible homographies differing by a general projective 2. Number of Coplanar Conics transform can exist between two given conics. In the case of imaging an unknown circle, we get only 5 equations in

Conic sections are planar curves defined by a second de- 7 unknowns (3 for ¡#9 , 2 for affine, 2 for projective or 5 @ gree parametric equation. Conics are abundantly available for ¡ , 2 for projective) which again have infinite possible

as many man-made structures follow them. A general conic solutions. Thus, given only one conic correspondence no

¢¡ §

section can be expressed as , where is a point concrete information can be gathered about the projective,

expressed in homogeneous coordinates and afﬁne or similarity part of the homography.

£¤¦¥ © §

§ ¡ From Eq. 2, we get the following for one of the conics.

¥ ¥ Hed H

£ £ £ £

§ § 67 ¡56

§ ¡

H H H

£ £ £ £ £

£ £ £

¡567 § §

Any projective transformation can be split into a sim- §

9 H

ilarity transform , a pure afﬁne transform and a pure Hed

£ £ £ £

¨ § ¨ 6:9 ¡56

projective transform © such that [7]. In

£ £ £

¡56:9 §

general © , and can respectively be written as

£ £

¡ § ¡

£¤ "!#%$'&

£¤10 2

£ © =<

( )!* + $',

-43

Twelve equations in fourteen unknowns are obtained using

- 6786:9;- two sets of the above equations. If a circle getting trans-

formed as an ellipse using a pure afﬁne transform, we can ©

A conic ¡ gets transformed by a homography as get from Eq. 1,

9h 9 9 9ih 9 9

0 2 0 0 2 0

H ¥ H ¥ ¥

¡£§ ©?> ¡©?> ¡

£ £

§ -3 3^g § -43 3g §

. A circle ﬁrst undergoes a simi- 3:¨ const

0 2 0 02

¥ ¥ ¥

£ £

§ 3 § © § 3

larity transform to become another circle , then an 3

. .

afﬁne transform to become an ellipse , and a projec-

0 2

¥ ¥ ¥ ¥

7 9 9 7 7 9 9

tive transform to become conic section . Let Therefore, 7 = and = , which

6:9 ©C>

and be some parameters which deﬁne the matrices gives two more equations. We have 14 equations in 14 un-

©?> ¡#9 ¡ and . The equation of the circle , the ellipse , and knowns. It can be shown that these equations reduce to a

univariate polynomial equation of degree 8 which has at conjugate to each other. Thus, two or four of the points of

calculate the entire homography between two views using both) as imaginary. We get one or two pairs of imaginary ¢ two conic correspondences. Having more than two conic points such that within each, the and coordinates are correspondences would result in an over-determined set of conjugates of each other. equations which could provide more robust solutions. It can Let the two conics present in the view be given by Eq. 3. also be shown that one conic correspondence with a point Since they are obtained from imaging circles, solving them

correspondence on the conic is sufﬁcient for homography would give us the circular points. These can be used to ﬁnd

¡(' § © ©

) * calculation. However, it is difﬁcult to obtain a point corre- the dual conic [4]. The matrix spondence on a general conic and therefore, this method is can be obtained using the dual conic [7]. This degenerate of purely theoretical interest. dual conic is invariant to similarity transform and is given

In summary, no information can be obtained from one or as

£¤ ©

multiple views of one general conic. However, identiﬁca- -

- §

tion of two circles is sufﬁcient to obtain the homography up ¡

)

to similarity.

h h h

' £ '

g`¡ § g © © © ¡ g © © ©

) ) h

3. Single View of Coplanar Conics h

§ g © © ¡ g © ©

)

£++ +

Many documents and billboards contain circles which result J

-, §

in conics when imaged. The exact equation of the circles is +

not known. We show that a fronto-parallel view of the scene 2

can be generated given 2 circles on a plane. Two conics, in 0

§0/

021

. -43

general, intersect in four points, real or imaginary. Consider where deﬁnes the afﬁne transformation

, §43 6 6

7 95

the conic sections .

and deﬁnes the projective transformation. The

9 9

0 2

Hed H¥¤ H H d§¦ H©¨

¡ £¢ ¢ ¢ §

g projective and afﬁne components are determined directly

¡6'

9 9 h

)

2 0

H d§¦ d H ¤ H H ¨

H from the image of but the similarity component is un-

£ £ £ £ £ £

g ¢ ¢ ¢ §

(3) determined. The circular points get projected to points at inﬁnity if the transformation is either similarity or afﬁne.

Eliminating from these equations results in an equation of Solving the above equations will not yield the correct re- the fourth degree in ¢ , giving four values, real or imaginary,

9 sult. For this case, consider Eq. 3 of the conics using ho- ¢ for . Eliminating from these two equations, we see that

mogeneous coordinates. Solve for points of intersection at

there is only one value of for each value of . Thus, there §

. . . .

inﬁnity by setting 7 . This results in the following h

are only four points of intersection. All circles pass throughh

equations.

§ g'- § g -

the circular points and [4].

9 9

2 0

Hed H

¢ ¢ §

Therefore, two of the points of intersection of any two cir-

9 9

2 0

H d H

£ £ £

¢ § ¢

cles are the circular points. Consider the case where the

circular points are subjected to a general projective trans-

7 9

This would give either one or two pairs of circular points.

form © Thus, we obtain a maximum of four pairs of circular points,

. .

h two for the projective case and two for handling the afﬁne

. .

H H H

Consider the points h and similarity cases. The degenerate dual conic is obtained

and h . Multiplying in each case to rectify the image. One of the four possible

by and by h we get resultant images is the required rectiﬁed image. The exact

H H H H

g¡ g 7 9 g¡ [9 7 h

h equations or the relative positions of the circles in the scene

9 9

. .

H H

g¡ [9" © § g 7! 9

h h

h and need not be known. The ease of specifying a conic section

9 9

H H

A A

g¡ " 7! . These makes it possible to obtain the result with minimal corre-

circular points cannot get mapped onto points with both real spondences. This method does not result in complex multi-

¢ ©

and coordinates because then the matrix becomes variate non-linear equation and guarantees a result for every

rank deﬁcient. This is because then 9 and input. We cannot determine which of the four possible solu-

[A § 3# §$ [A43§ §% 3#

7 9

which implies . tions is the correct one without additional information about

§& § © (Or, , in which case is afﬁne, which is han- the scene such as the ratio of the radii of the circles or the dled later.) That is, the ﬁrst two columns are related by a distance between their centers.

scale, which implies that the matrix is rank deficient. Also, The algorithm for metric rectification can be summarized ¢ the and coordinates of the projected circular points are as follows. 1. Obtain the equations of the two conics from the image. 4 points because any two circles can intersect at a maximum 2. Solve them using non-homogeneous coordinates and homo- of 2 real points. The method works even when the conics £ are not obtained by imaging circles. Even though the recti- geneous coordinates with ¢¡ . fication would be incorrect, both the views are transformed 3. Obtain a maximum of four pairs of imaginary points with

incorrectly in a similar way and the ﬁnal homography is cal- ¥ conjugate ¤ and coordinates.

culated using this rectiﬁed views.

7 9

¡ ¡ 7 4. Treating each pair as the circular points, obtain four different 7

Given a view with conics and , obtain7 the ho-

7 7 9

> >

¦¨§©¦

7 7 7 7

matrices for . 7

mography 7 such that and

9 7 9

> >

¡ © © ¡ ¡

7 7 7 7

5. One of these four matrices results in the actual rectiﬁcation. 7 where and are circles. Similarly, for

7 9

¡ ¡

9 9

the second view containing conics and , we7 obtain the

7 7 9

> >¨

9 9 9 9 Figures 1 shows the single view of some of the images 9

homography7 such that and

9 7 9

> >

¡ © © ¡ ¡

9 9 9 9 containing unknown coplanar circles and their rectiﬁed im- 9

where and are circles. If the views ages using the above algorithm. In all our experiments, the © are of the same scene, there exists a similarity transform equations of the conics in the image were computed from such that

their boundary points by the method described in [6]. The

7 7 7 9 9 7

>¨ > > >

9 7 9 method has the advantage of explicitly modelling noise and 7 and

as the centers of the circles in the rectiﬁed view as similar- .

ity transform preserves the centers. The center of a circleh

9 9

0 0 0 0

Hcd"¤ H d ¦ H ¨ H ¤ ¦

¢ ¢ § g 3 3

is given as .

The homography7 between the two views is the product

©7 © © © 9 . We can obtain a unique if the radii of the two circles are different. If the radii are the same, we cannot determine which center in one view corresponds to which center in the other. This results in the rotation and translation being unknown. If the circles are concentric, the scale and translation can be determined from the radii and the center, but the rotation ambiguity will remain. This approach involves simple univariate polynomial equations of degree 4 which always have a solution. In summary, the algorithm for homography computation from 2 views is Figure 1: Circles in the left images were used to rectify them to get the right images. The circles representing the 1. Obtain the equations of the conics in both views. nodes of the graph in the book and the ’O’s in the classroom 2. Rectify both views assuming that the conics are images of board were used for rectiﬁcation by projectively transform- circles. The results are correct even if this is not so. Let ¦ ing them to result in circles. The rectiﬁcation is quite good; and ¦ be the rectifying homographies.

the images on the right show the quadrilaterals to which the 3. Calculate the similarity transform ¦ between the two recti- rectangular images on the left map. ﬁed views using two point correspondences obtained by ﬁnd- ing the centers of the two circles. 4. Multiple Views of Coplanar Conics

4. The homography between the two views is obtained as

¦ ¦ ¦ Multiple views of a plane containing conics can help in . recovering the homography between the views. Previous approaches to homography calculation from conics require Figure 2 shows the results of homography calculation for a minimum of 7 correspondences [13]. Other approaches a real image. The two ’O’s in the sign were assumed to be to homography calculation from higher order parametric circles. The image obtained by applying the homography to curves fail in the case of conics as they result in insufficient the left image is shown at the bottom. number of equations [5]. Two given conics intersect at 4 points. If these points can 5. Single View of Non-coplanar Conics be recovered and corresponded, the homography can be cal- Lastly, we consider the case of imaging non-coplanar con- culated directly. If not, both images can metric rectified by ics, specifically the case where there are two planes con- assuming that the given conics are images of circles. Note taining at least two conics each. We show that we can ob- that this assumption is not valid when the conics intersect at tain the 3D reconstruction of the scene from them, provided Figure 2: The homography between the top images was computed. The bottom image is the result of applying it Figure 3: The 3D reconstruction of the top image was used to the top left one. to render the bottom views. Figure 3 shows the results of reconstructing the image using they are images of circles. The previous approaches to 3D the two circles on the wall. The lines on the floor were reconstruction from conics [8, 11] are restricted to the case used to get its vanishing line. Two different views generated of multiple views. Our approach is similar to the one in [2], from the 3D model starting with the first view are shown. which finds 3D affine measurements from minimal geomet- The results are similar to those reported in [2] which used ric information such as the vanishing line of a reference parallel lines to obtain vanishing lines. Our approach gave plane and a vanishing line for another plane not parallel to

6 the height of the leftmost wall as 16.1 units compared to 16 the previous plane . To calculate the height of an object,

2 units from [2]. ¢

we use a reference height ¡£¢ speciﬁed by a base point

$ ¤

and a top point ¢ , a parameter such that

$ 2 ¥¦¥

¥¦¥ 6. Application to Projective OCR

¢ ¢

2 $

¥ ¥ ¥¦¥

g 6

©¨ ¢ ¢

¡§¢ Optical Character Recognition (OCR) is the process of con- 2& & verting a document image to text. Normally, the input im-

The height ¡ of an object speciﬁed by the base point

$ & age is scanned and presented to the OCR system. The doc-

and the top point is given as ument might be skewed or scaled slightly. Projective OCR

2 $

& & ¥ ¥

¥ ¥ is one that identiﬁes the text from the image of a document

2]& $'&

¥¦¥ ¥ ¥

¡ 6

g taken from a general view using a perspective camera. Con-

¨ ¤ ventional OCR engine can be used on the document image The measurements are restricted to the ratio of the heights after metric rectification of the projectively distorted one. of objects, ratio of the lengths along parallel lines on the Earlier approaches, such as [10], find the transformation as- plane and the ratio of areas on the plane. The complete suming that the imaging system has the vertical vanishing reconstruction of the 3D scene containing conics can be ob- point at infinity. Others map the quadrilateral containing the tained from its multiple views using the method described text in the image to a rectangle [1]. We assume that there in [11]. are two circles – either from character ‘O’ or otherwise – The vanishing line is the last row of the matrix used for in the text of the document. These can be identified auto- rectification. Metric rectification can be performed as de- matically by looking for conics in the image. Rectification scribed in Section 3. The user has to specify a reference is then performed on the image by assuming ‘O’s to be cir- height used for the reconstruction. The algorithm is sum- cles. The recognition is independent of the aspect ratio; thus marized as follows. this assumption produces good results. Figure 4 shows two sample pages along with their rectified image. The filled 1. Obtain the equations of the conics on each plane and rectify ‘O’s in the heading were used to rectify the image. We took each plane. 10 different views of 7 pages, rectified them and fed them 2. Obtain the vanishing lines of the two planes from the rectifi- to an existing OCR system with and without rectification. cation transformation. An overall accuracy of 94.75% was achieved with rectifi- 3. Use the procedure in [2] to obtain the 3D reconstruction of cation when compared to an accuracy of 42.68% without the scene from these vanishing lines. rectification. Figure 4: Right images give the rectified versions of the left ones.

Type Conﬁguration #Views Application Reference 4 Conics Non-coplanar 2 Fundamental Matrix Computation [5] 2 Circles Coplanar 1 Metric Rectiﬁcation In this paper 7 Conics Coplanar 2 Homography Computation [13] 2 Conics Coplanar 2 Homography Computation In this paper Conics Non-coplanar 2 3D Reconstruction [8, 11] Circles Non-coplanar 1 3D Reconstruction In this paper

Table 1: Summary of applications of various conﬁgurations of conics

7. Conclusions and Future Work [3] O. Faugeras and Q. Luong. The Geometry of Multiple Im- ages. MIT Press, 2001. This paper discussed various configurations of conics in [4] R. Hartley and A. Zisserman. Multiple View Geometry. the scene. We presented several applications of geomet- Cambridge University Press, 2000. ric structure estimation from conics in the image, includ- [5] J. Kaminski and A. Shashua. Multiple view geometry of ing metric rectification of a single view of unknown circles, algebraic curves. IJCV, 2003. homography calculation from multiple views of unknown [6] Y. Kanazawa and K. Kanatani. Optimal conic fitting and conics or single view of known conics, and 3D reconstruc- reliability evaluation. IEICE Trans. on Information and Sys- tems, E79-D(9):1323–1328, 1996. tion from single view of unknown non-coplanar conics. The [7] D. Liebowitz and A. Zisserman. Metric rectification for per- algorithms are simple and do not require correspondences spective images of planes. In IEEE Conference on Computer of a large number of conics or the solution of multivariate Vision and Pattern Recognition, pages 482–488, June 1998. polynomial equations. A summary of the results is shown [8] S. Ma and X. Chen. Quadric reconstruction from its occlud- in Table 1. Extensions to these methods which handle over ing contours. In International Conference of Pattern Recog- determined sets of equations, obtained due to the presence nition, pages 27–31, 1994. of more than two coplanar conics, need to be explored. [9] S. Ma, S. Si, and Z. Chen. Quadric curve based stereo. In ICPR, volume I, pages 1–4, 1992. Acknowledgment [10] G. Myers, R. Booles, Q. Luong, and J. Herson. Recognition of text in 3d scenes. In SDIUT, pages 85–100, 2001. We thank Andrew Zisserman for fruitful discussions on [11] L. Quan. Conic reconstruction and correspondence from two methods for performing metric rectification using images views. IEEE Transactions on Pattern Analysis and Machine of circles. Intelligence, 18:151–160, february 1996. [12] C. Schmid and A. Zisserman. The geometry and matching References of curves in multiple views. IJCV, 40(3):199–233, 2000. [13] A. Sugimoto. A linear algorithm for computing the homog- [1] P. Clark and M. Mirmehdi. On the recovery of oriented raphy from conics in correspondence. Journal of Mathemat- documents from single images. Technical Report CSTR- ical Imaging and Vision, 12, 2000. 01-004, Univ. of Bristol, Nov 2001. [14] Y. Wu, H. Zhu, Z. Hu, and F. Wu. Camera calibration from [2] A. Criminisi, I. Reid, and A. Zisserman. Single view metrol- quasi-affine invariance of two parallel circles. In ECCV, ogy. IJCV, 40(2):123–148, Nov. 2000. 2004.