3-D Scene Reconstruction from Line Correspondences Between Multiple Views
Total Page:16
File Type:pdf, Size:1020Kb
3-D SCENE RECONSTRUCTION FROM LINE CORRESPONDENCES BETWEEN MULTIPLE VIEWS A dissertation proposal submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science and Engineering By Michael Linger Ph.D., Computer Science and Engineering, Wright State University, 2014 M.S., Computer Science, Wright State University, 2009 B.S., Computer Science, Wright State University, 2005 B.S., Mathematics, Wright State University, 2005 A.A.S., Electronics Engineering Technology, ITT Technical Institute, 1996 2014 Wright State University WRIGHT STATE UNIVERSITY GRADUATE SCHOOL December 1, 2014 I HEREBY RECOMMEND THAT THE DISSERTATION PREPARED UNDER MY SUPERVISION BY Michael Linger ENTITLED 3-D Scene Reconstruction from Line Correspondences between Multiple Views BE ACCEPTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Doctor of Philosophy. Arthur A. Goshtasby, Ph.D. Dissertation Director and Director, Computer Science and Engineering Graduate Program Robert E. W. Fyffe, Ph.D. Vice President for Research and Dean of the Graduate School Committee on Final Examination Arthur A. Goshtasby, Ph.D. Mateen Rizki, Ph.D. Bin Wang, Ph.D. Thomas Wischgoll, Ph.D. Fred Garber, Ph.D. ABSTRACT Linger, Michael Ph.D. Computer Science and Engineering, Wright State University, 2014. 3-D Scene Reconstruction from Line Correspondences between Multiple Views. Three-dimensional scene reconstruction from 2-D images has many applications, such as surveillance, mission planning, autonomous navigation systems, cartography, and target recognition. Of specific interest to this research is the reconstruction of urban scenes containing man-made structures, such as roads and buildings, to support the burgeoning surveillance industry. Using 3-D maps to augment existing mission planning cartography products (DTED/SRTM, CADRG, CIB), mission and event planners will be able to compute strategic line- of-sight coverage for threat avoidance or threat prosecution. Forensic video analysts can use these models to recreate crime scenes, while law enforcement can build flight plans to minimize occlusions from tall structures in their persistent surveillance systems. Traditional methods of 3-D scene reconstruction leverage image points as a primitive element. Various approaches detect and correlate points for use in triangulation and 3-D reconstruction. Little work has been done in 3-D reconstruction using lines as primitives. In my research, I detect line segments and their associated planar surfaces. Lines detected in 2-D images are back projected to corresponding planar patches and triangulated via linear incident relations resulting in a reconstructed 3-D wireframe model. This research uses high resolution imagery at close range as could be collected by autonomous drones. It reduces data by an order of magnitude by exploiting the point-line duality of projective geometry. iii TABLE OF CONTENTS 1 Introduction .............................................................................................................................. 1 1.1 Purpose ............................................................................................................................. 1 1.2 Problem Description ........................................................................................................ 2 1.3 Scope ................................................................................................................................ 4 1.4 Document Overview ........................................................................................................ 5 2 Background .............................................................................................................................. 6 2.1 Introduction to Geometry ................................................................................................. 6 2.2 Geometric Primitives in Cartesian and Homogeneous Coordinates ................................ 7 2.3 Introduction to 2-D Projective Geometry ...................................................................... 10 2.4 Introduction to 1-D Projective Geometry ...................................................................... 12 2.5 Pinhole Camera Model .................................................................................................. 14 3 Method ................................................................................................................................... 16 3.1 Overview ........................................................................................................................ 16 3.2 Line Detection ................................................................................................................ 17 3.3 Camera Computation ..................................................................................................... 25 3.3.1 Calibration Box ...................................................................................................... 25 3.3.1.1 Edge Detection ................................................................................................... 26 3.3.1.2 Hough Transform ............................................................................................... 26 3.3.1.3 Connected Graph ............................................................................................... 27 3.3.1.4 Camera Parameters ............................................................................................ 30 3.3.1.5 Correlation ......................................................................................................... 31 iv 3.3.2 Surveyed Camera ................................................................................................... 32 3.4 Planar Surface Detection ................................................................................................ 35 3.5 Homographic Registration ............................................................................................. 38 3.5.1 Global Rotation ...................................................................................................... 39 3.5.2 Putative Correspondence........................................................................................ 40 3.5.3 Point Descriptor ..................................................................................................... 42 3.5.4 Template Matching Via Point Descriptor .............................................................. 43 3.5.5 2-D Homographic Registration from Correspondences ......................................... 45 3.5.5.1 Registration from Point Correspondences ......................................................... 45 3.5.5.2 Registration from Line Correspondences .......................................................... 47 3.5.5.3 Registration from Line Correspondences and Their 1-D Homographies ......... 49 3.6 Line Correspondence ..................................................................................................... 53 3.7 3-D Wireframe ............................................................................................................... 54 3.8 Polygon Boundary ......................................................................................................... 57 4 Results .................................................................................................................................... 58 4.1 Edge Detection ............................................................................................................... 58 4.2 Line Detection ................................................................................................................ 60 4.3 3-D Models .................................................................................................................... 62 4.3.1 Model Doll House .................................................................................................. 62 4.3.2 House Reconstruction ............................................................................................ 65 4.4 Error Analysis ................................................................................................................ 68 5 Conclusion ............................................................................................................................. 70 5.1 Summary of Contributions ............................................................................................. 70 v 5.2 Challenges Encountered ................................................................................................. 72 5.3 Recommendations for Future Research ......................................................................... 73 Appendix A Geometric Parameterization of a 2-D Homography .................................................. 75 Appendix B Orthogonal Regression .............................................................................................. 80 Appendix C Multiple Regression ................................................................................................... 84 Appendix D Bilinear Interpolation ................................................................................................ 85 Appendix E Line Templates .......................................................................................................... 87 Appendix F 2-D Image Registration with Points ........................................................................... 89 Appendix G 2-D image Registration with Lines