Purdue University Purdue e-Pubs Open Access Theses Theses and Dissertations January 2016 Spatial Reconstruction of Biological Trees from Point Cloud Jayakumaran Ravi Purdue University Follow this and additional works at: https://docs.lib.purdue.edu/open_access_theses Recommended Citation Ravi, Jayakumaran, "Spatial Reconstruction of Biological Trees from Point Cloud" (2016). Open Access Theses. 1190. https://docs.lib.purdue.edu/open_access_theses/1190 This document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries. Please contact [email protected] for additional information. SPATIAL RECONSTRUCTION OF BIOLOGICAL TREES FROM POINT CLOUDS A Thesis Submitted to the Faculty of Purdue University by Jayakumaran Ravi In Partial Fulfillment of the Requirements for the Degree of Master of Science May 2016 Purdue University West Lafayette, Indiana ii Dedicated to my parents and my brother who always motivate me to give my best in whatever I choose to do. iii ACKNOWLEDGMENTS Firstly, I would like to thank Dr. Bedrich Benes - my advisor - for giving me the opportunity to work at the High Performance Computer Graphics (HPCG) lab. He gave me the best possible support during my stay here and was kind enough to overlook my mistakes. These past years have been one of the best learning opportunities of my life. I learnt a great deal from all my former and current lab members who are exceptionally talented and smart. I thank Dr. Peter Hirst for giving me the opportunity to work with his team and for also sending us apples from the farm during harvest season. I thank Biying Shi and Fatemeh Sheibani for going with me to the field and helping me with the tree scanning under various weather conditions. I thank Alejandro for his initial guidance when I started working here at HPCG lab and the times we went to the sports center. Thanks to my lab mates, Jorge Garcia, Sonali Patil and my friend Sai Gurumurthy, I was able to figure out the calculations to compute error in reconstruction. Thanks to Vojtech for helping me with the Gizmo tool in the skeleton editor. Scanning apple trees would not have been possible without my friends Jebaraj Vasudevan, Anand Samuel and Richard Shenbagaraj who gave me their GPU enabled laptops from time to time. I also thank all my friends here at Purdue for making my life here pleasant. It would not have been easy without them. I thank all members of my thesis committee - Dr. Bedrich Benes, Dr. Peter Hirst, Dr. Tim McGraw and Dr. Victor Chen - for providing valuable inputs and feedback to complete this thesis. Finally, I want to thank my mother, father and my brother for constantly giving me moral support and motivating me to do my best during my stay at Purdue. iv TABLE OF CONTENTS Page LIST OF TABLES :::::::::::::::::::::::::::::::: v LIST OF FIGURES ::::::::::::::::::::::::::::::: vi ABSTRACT ::::::::::::::::::::::::::::::::::: viii CHAPTER 1. INTRODUCTION :::::::::::::::::::::::: 1 1.1 Significance ::::::::::::::::::::::::::::::: 2 1.2 Research Question ::::::::::::::::::::::::::: 3 1.3 Assumptions ::::::::::::::::::::::::::::::: 4 1.4 Limitations ::::::::::::::::::::::::::::::: 4 1.5 Delimitations :::::::::::::::::::::::::::::: 4 1.6 Summary :::::::::::::::::::::::::::::::: 5 CHAPTER 2. REVIEW OF RELEVANT LITERATURE :::::::::: 6 2.1 Tree modeling :::::::::::::::::::::::::::::: 6 2.1.1 Rule and simulation based tree modeling ::::::::::: 6 2.1.2 Sketch-based tree modeling :::::::::::::::::: 10 2.1.3 Image-based tree modeling ::::::::::::::::::: 12 2.1.4 Laser scan-based tree modeling :::::::::::::::: 15 2.2 Summary :::::::::::::::::::::::::::::::: 17 CHAPTER 3. FRAMEWORK AND METHODOLOGY ::::::::::: 18 3.1 System overview :::::::::::::::::::::::::::: 18 3.2 Capture point cloud :::::::::::::::::::::::::: 18 3.3 De-noising and merging point cloud :::::::::::::::::: 19 3.4 Skeletonization and skeleton editor :::::::::::::::::: 22 3.5 Sweeping reconstruction :::::::::::::::::::::::: 23 3.6 Apply subdivision surface ::::::::::::::::::::::: 28 3.7 Evaluation :::::::::::::::::::::::::::::::: 28 3.8 Summary :::::::::::::::::::::::::::::::: 30 CHAPTER 4. IMPLEMENTATION AND RESULTS ::::::::::::: 31 4.1 Implementation ::::::::::::::::::::::::::::: 31 4.2 Results :::::::::::::::::::::::::::::::::: 33 4.3 Future work ::::::::::::::::::::::::::::::: 40 4.4 Conclusion :::::::::::::::::::::::::::::::: 40 LIST OF REFERENCES :::::::::::::::::::::::::::: 42 v LIST OF TABLES Table Page 4.1 Skeleton editor keyboard/mouse combination :::::::::::::: 32 4.2 Residual error calculated for seven apple trees. ::::::::::::: 33 4.3 Increase in error after sweeping reconstruction in percentage. :::::: 34 vi LIST OF FIGURES Figure Page 1.1 Golden Delicious Apple Trees. ::::::::::::::::::::::: 3 2.1 Space colonization algorithm (Runions, Lane, & Prusinkiewicz, 2007). 8 2.2 Section of a tree evolving and growing around a fence obstacle (Kratt et al., 2015). :::::::::::::::::::::::::::::::::: 10 2.3 2D sketches producing complete 3D polygonal meshes of three different tree using Okabe, Owada, and Igarashi (2005)'s sketching interface. :: 11 2.4 Overview of image-based plant modeling (Quan et al., 2006). ::::: 13 2.5 Single image tree modeling (Tan, Fang, Xiao, Zhao, & Quan, 2008). : 14 2.6 Five different trees reconstructed by Livny et al. (2010)'s algorithm. :: 16 2.7 3D reconstruction by intrusive acquisition and modeling (Yin et al., 2015). 16 3.1 Reconstruction pipeline :::::::::::::::::::::::::: 18 3.2 Scanning. :::::::::::::::::::::::::::::::::: 19 3.3 Cleaning raw scan. ::::::::::::::::::::::::::::: 20 3.4 Merging and down-sampling :::::::::::::::::::::::: 21 3.5 Skeletonization output. ::::::::::::::::::::::::::: 23 3.6 Use of skeleton curve editor. Operations in view: dragging nodes, bend, split and extend curves ::::::::::::::::::::::::::: 24 3.7 2D slice sampling. Left to right: Frenet-frame, cutting plane, neighhorhood search, samples projected to plane. :::::::::::::::::::: 25 3.8 Result of applying parallel transport frames. ::::::::::::::: 26 3.9 Depiction of a graph tree data structure. BN - Branch Node :::::: 27 3.10 Cross-sections connected as triangle strips. :::::::::::::::: 28 3.11 Applying subdivision surfaces in Autodesk Maya. :::::::::::: 29 4.1 Graphical user interface of our reconstruction application. ::::::: 32 4.2 Direct sampling of trees produces unrealistic reconstructed tree model. 35 vii Figure Page 4.3 Sweeping reconstruction fixes inconsistent branch radii using prior knowledge of trees. ::::::::::::::::::::::::::::::::::: 36 4.4 Branch reconstructed without point cloud samples. ::::::::::: 37 4.5 A branch reconstruction. :::::::::::::::::::::::::: 37 4.6 Reconstruction of a Golden Delicious apple tree. ::::::::::::: 38 4.7 Difference in radii between direct sampling and sweeping reconstruction. Blue - zero difference. Red - maximum difference. White - not enough sample points. :::::::::::::::::::::::::::::::: 39 4.8 A reconstructed apple tree rendered in Autodesk Maya. ::::::::: 39 4.9 Inadvertent scans of supporting steel poles along with the actual tree. : 41 viii ABSTRACT Ravi, Jayakumaran M.S., Purdue University, May 2016. Spatial Reconstruction of Biological Trees from Point Clouds. Major Professor: Bedrich Benes. Trees are complex systems in nature whose topology and geometry are influenced by environmental factors. Tree geometry is extremely complicated and its capturing poses a challenging problem. Horticulturists require captured data of tree geometry to analyze regulation of resources. Traditionally, 3D digitizers and calipers are used to record position and orientation of every branch. While these are accurate, they are fundamentally time consuming. This thesis is an extension of our paper, Apple tree scanning and reconstruction using Kinect submitted to Acta Horticulturae. In our work we propose to reconstruct spatial data of Golden Delicious apple trees with user assistance. Our system requires a point cloud input to reconstruct the base tree. Our approach involves the use of Microsoft Kinect v2 sensor for scanning Golden Delicious apple trees. We extract a curve skeleton from the given point cloud and attach 2D shape profiles along axes to generate a triangular mesh. Incomplete skeletal structures can be completed using skeleton editor tools provided in our GUI based application. Branch organs are reconstructed by sampling local points in the vicinity of curve skeleton obtained by the skeletonization algorithm. Direct sampling without establishing topological information can produce unrealistic visual results. We propose a sweeping reconstruction method which is capable of reconstructing branches starting from the root branch. Our method is capable of reconstructing branches which do not have enough sample points by using neighboring branches as reference. Results show that error in sweeping reconstruction is higher than directly ix sampled reconstruction. But this produces better visual results without any gaping holes in the 3D mesh model from a computer graphics perspective. 1 CHAPTER 1. INTRODUCTION Trees are complex systems whose topology and geometry are influenced by endogenous and exogenous processes. In the field of horticulture, the geometry and topology of a tree is connected to plant architectural studies (Barthlmy & Caraglio, 2007). Plant architecture analysis involves identifying factors that influence the plasticity of trees. These have an effect on crop yield and fruit quality. Apart from genetic traits, environmental factors such as sun-light, temperature, wind