Wu YH, Lan T, Hu ZY. Degeneracy from twisted cubic under two views. JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY 25(5): 916{924 Sept. 2010. DOI 10.1007/s11390-010-1072-9 Degeneracy from Twisted Cubic Under Two Views Yi-Hong Wu (吴毅红), Tian Lan (兰 添), and Zhan-Yi Hu (胡占义) National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China E-mail: fyhwu, tlan, [email protected] Received November 9, 2009; revised May 20, 2010. Abstract Fundamental matrix, drawing geometric relationship between two images, plays an important role in 3- dimensional computer vision. Degenerate con¯gurations of the space points and the two camera optical centers a®ect stability of computation for fundamental matrix. In order to robustly estimate fundamental matrix, it is necessary to study these degenerate con¯gurations. We analyze all the possible degenerate con¯gurations caused by twisted cubic and give the corresponding degenerate rank for each case. Relationships with general degeneracies, the previous ruled quadric degeneracy and the homography degeneracy, are also reported. Keywords fundamental matrix, twisted cubic, ruled quadric, degenerate con¯guration 1 Introduction can give more geometric intuition, which could be of guidance for placing cameras to avoid degeneracy in Fundamental matrix describes geometric relation practice. Furthermore, if we can judge the degeneracy between two 2-dimensional views and is a key concept by applying geometric knowledge, RANSAC will work in 3-dimensional computer vision. It plays an impor- much easier. tant role in image matching, epipolar geometry, camera Due to the importance of degeneracy analyses, many motion determination, camera self-calibration and 3- of such works have been reported previously. The dimensional reconstruction[1¡9]. Robust and accurate planar scene is a trivial degenerate con¯guration for estimation for fundamental matrix has been the re- computing fundamental matrix, where the images can search focus of extensive researchers[10¡17]. provide only six independent constraints[1;16] but the From at least seven pairs of point-point correspon- general fundamental matrix has seven degrees of free- dences between two views, the fundamental matrix dom in a nonlinear space. Degeneracy from twisted can be estimated. Sometimes, no matter how many cubic con¯guration has also been discussed. In [22], correspondences are used, accuracy of the estimation Buchanan stated that camera calibration from known is low. One of the main reasons is that the cameras space points under a single view is not unique if the and the scene lie on a degenerate or quasi-degenerate optical center and the space points lie on a twisted con¯guration. If a space con¯guration is degenerate cubic. The corresponding detection as well as emen- mathematically but the noise from the measured image dations including other unreliability was given by Wu makes it non-degenerate, any estimation under such a et al.[23] Then, under two views, Maybank[24] analyzed con¯guration would be useless[7;18-20]. It follows that characterizations of horopter curve and relations be- we should know what con¯gurations might cause de- tween the curve and ambiguous case of reconstruc- generacy for estimating the fundamental matrix. More- tion. The horopter curve is regarded as a twisted cu- over in order for a robust random sample consensus bic, which intersects the plane at in¯nity at three par- (RANSAC) like [16, 21], we still need to know how ticular points. The ambiguous case of reconstruction great the degenerate degree is, namely, to know the de- implies ambiguity of fundamental matrix. Luong and generate rank of the coe±cient matrix of the equations Faugeras reported stability of computing fundamental for computing fundamental matrix. In [12], RANSAC matrix caused by quadric critical surface in [25]. Hart- loop to estimate relation from quasi-degenerate data is ley and Zisserman[1] also gave systematic discussions reported, where the degenerate con¯gurations need not on degeneracy of camera projection estimation from be known. This does not mean the studies on the degen- twisted cubic under a single view and on degeneracy erate con¯gurations are useless. At least, such studies from ruled quadric surface under two views. Under Regular Paper Supported by the National Natural Science Foundation of China under Grant Nos. 60835003 and 60773039. ©2010 Springer Science + Business Media, LLC & Science Press, China Yi-Hong Wu et al.: Twisted Cubic Degeneracy 917 three views, critical con¯gurations are provided in [26], Section 4 and Section 5 is some conclusions. which is an extension of the critical surface under two views. Degeneracy under a sequence of images is also 2 Preliminaries investigated[2;27]. Maybank and Shashua[27] pointed The camera model used is a perspective camera. A out there is a three-way ambiguity for reconstruction space point or its homogeneous coordinates is denoted from images of six points when the six points and the by M , an image point or its homogeneous coordinates camera optical centers lie on a hyperboloid of one sheet. i is denoted by m , P denotes the camera projection In [2], Hartley and Kahl presented a classi¯cation of all i matrix, and O denotes the camera optical center. If possible critical con¯gurations for any number of points two views are used, P 0 denotes the second camera pro- from three images and showed that in most cases, the jection matrix, O0 denotes this second camera optical ambiguity could extend to any number of cameras. center, and m0 denotes the corresponding image point Relative to the above works on degenerate con¯g- i of M under this camera. Let F be the fundamen- urations, there are fewer in-depth studies on degener- i tal matrix between these two views. Other vectors or acy degrees of degenerate con¯gurations. Torr et al.[16] matrices are also denoted in boldface. The symbol ¼ catalogued all two-view non-degenerate and degenerate means equality up to a scale. cases in a logical way by dimensions of the right null ² Camera Projection Matrix space of equations on fundamental matrix and then M ; i = 1::N are N 3-dimensional space points. proposed a PLUNDER-DL method to detect degene- i And, their 2-dimensional image points under a single racy and outliers. Chum et al.[21] also analyzed those view are m , i = 1::N. The camera projection matrix dimensions when the two views or most of the point i P is a 3 £ 4 matrix such that m ¼ PM for all i. correspondences are related by a homography and pre- i i P is equal to K(R; t) up to a scale, where K is the sented an algorithm to estimate fundamental matrix 3 £ 3 camera intrinsic parameter matrix, R is the 3 £ 3 through detecting the homography degeneracy. They camera rotation matrix, t the camera translation vec- all[16;21] generalized the robust estimator RANSAC[28]. tor relative to the world coordinate system. For the The plane degeneracy in [16, 21] is consistent with camera optical center O, we have the equation: the ruled quadric degeneracy proposed by Hartley and Zisserman[1] because a plane and two camera optical PO = 0: (1) centers always lie on a degenerate ruled quadric. What are the degeneracy degrees when estimating the fun- ² Fundamental Matrix damental matrix for other non-trivial degenerate con¯- Fundamental matrix describes the geometric rela- gurations? In this paper, we discuss all possible dege- 0 tionship between two 2-dimensional views. Let mi, nerate situations caused by twisted cubic and give the i = 1::N be the 2-demensional image points of the same corresponding degeneracy degrees. Let S be a set of 0 space points M i under another view. Then, mi and mi space points and the two camera optical centers. We are related by the 3£3 fundamental matrix F through: ¯nd that if all the points in S lie on a twisted cubic, the con¯guration is degenerate for estimating fundamental 0T mi F mi = 0; i = 1::N: (2) matrix and the corresponding rank of the coe±cient 0 1 matrix is ¯ve; if all the points other than one in S lie f1 f2 f3 on a twisted cubic, the corresponding rank is six; if all We denote F as F = @ f4 f5 f6 A. If mi ¼ the points other than two in S lie on a twisted cubic, the f7 f8 f9 corresponding rank is seven. The previous general de- T 0 0 0 0 T (ui; vi; wi) and mi ¼ (ui; vi; wi) , we expand (2) generacies are twisted cubic degeneracy, ruled quadric and have: degeneracy, and homography degeneracy. However, few 0 1 studies are given on relationships among them. We in- ¢ ¢ ¢ @ 0 0 0 0 0 0 0 0 0 A vestigate relationships of the twisted cubic degeneracy uiui; uivi; uiwi; viui; vivi; viwi; wiui; wivi; wiwi f ¢ ¢ ¢ with the pervious ruled quadric degeneracy and homog- N£9 raphy degeneracy, where some interesting theoretical = 0 (3) results are obtained such as a complete homography T relation between two views. where f = (f1; f2; f3; f4; f5; f6; f7; f8; f9) . The N £ 9 Organization of the paper is as follows. Some pre- coe±cient matrix of f is denoted by G. liminaries are listed in Section 2. The complete and uni- ² Plane Homography ¯ed degeneracy study from twisted cubic is elaborated When scene points lie on the same plane in space, 0 in Section 3. Some experimental results are reported in their corresponding image points mi and mi under two 918 J. Comput. Sci. & Technol., Sept. 2010, Vol.25, No.5 views are related by a 3 £ 3 matrix H: then F can be determined uniquely by linear 8-point algorithm[13].