Point Cloud Framework for Rendering 3D Models Using Google Tango Maxen Chung Santa Clara University, [email protected]
Total Page:16
File Type:pdf, Size:1020Kb
Santa Clara University Scholar Commons Computer Engineering Senior Theses Engineering Senior Theses 6-13-2017 Point Cloud Framework for Rendering 3D Models Using Google Tango Maxen Chung Santa Clara University, [email protected] Julian Callin Santa Clara University, [email protected] Follow this and additional works at: https://scholarcommons.scu.edu/cseng_senior Part of the Computer Engineering Commons Recommended Citation Chung, Maxen and Callin, Julian, "Point Cloud Framework for Rendering 3D Models Using Google Tango" (2017). Computer Engineering Senior Theses. 84. https://scholarcommons.scu.edu/cseng_senior/84 This Thesis is brought to you for free and open access by the Engineering Senior Theses at Scholar Commons. It has been accepted for inclusion in Computer Engineering Senior Theses by an authorized administrator of Scholar Commons. For more information, please contact [email protected]. Point Cloud Framework for Rendering 3D Models Using Google Tango by Maxen Chung Julian Callin Submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering School of Engineering Santa Clara University Santa Clara, California June 13, 2017 Point Cloud Framework for Rendering 3D Models Using Google Tango Maxen Chung Julian Callin Department of Computer Engineering Santa Clara University June 13, 2017 ABSTRACT This project seeks to demonstrate the feasibility of point cloud meshing for capturing and modeling three dimensional objects on consumer smart phones and tablets. Traditional methods of capturing objects require hundreds of images, are very slow and consume a large amount of cellular data for the average consumer. Software developers need a starting point for capturing and meshing point clouds to create 3D models as hardware manufacturers provide the tools to capture point cloud data. The project uses Googles Tango computer vision library for Android to capture point clouds on devices with depth-sensing hardware. The point clouds are combined and meshed as models for use in 3D rendering projects. We expect our results to be embraced by the Android market because capturing point clouds is fast and does not carry a large data footprint. Table of Contents 1 Introduction 1 1.1 Motivation . .1 1.2 Solution . .1 2 Literature Review and Assessment of Previous Work 2 2.1 Introduction . .2 2.2 Stitching Images to Create 3D Objects . .3 2.3 Project Tango: Beyond Imagery . .3 2.4 Google's Contributions to Tango on Android . .3 2.5 OpenCV for Processing Point Clouds . .4 2.6 An Application Perspective . .4 2.7 Mesh Construction Process . .5 2.8 Global vs Local . .5 2.9 Visual Contrast Sensitivity for 3D Meshes . .6 3 Requirements 8 3.1 Functional . .8 3.2 Non-Functional . .8 3.3 Design Constraints . .8 4 Use Cases 9 4.1 Diagram . .9 4.2 Explanation . 10 5 Activity Diagram 12 5.1 Sample App . 12 6 Conceptual Model 14 6.1 Front End Application . 14 6.1.1 User Interface . 14 6.2 Back End Application . 17 6.2.1 PCL on AWS . 17 6.2.2 Processing and Meshing Point Clouds . 18 7 Technologies Used 19 8 Design Rationale 20 8.1 Technologies Used . 20 8.1.1 Android . 20 8.1.2 Google Tango . 20 8.1.3 Java . 20 8.1.4 OpenCV . 20 8.1.5 Lenovo Phab2 Phone . 21 iv 8.1.6 Github . 21 8.2 User Interface and Framework . 21 9 Test Plan 22 9.1 Unit Testing . 22 9.2 Alpha Testing . 22 9.3 Beta Testing . 22 10 Cost Estimates 23 10.1 Cost Table . 23 11 Risk Analysis 24 11.1 Risk Table . 24 11.2 Mitigation Strategies . 24 12 Development Timeline 25 12.1 Gantt Chart . 25 13 Results 26 13.1 Test object: Mouse from side . 26 13.2 Improving the mesh . 29 13.3 Creating a 3D Object . 30 13.3.1 Single-sided Mirroring Approach . 30 13.3.2 Double-sided Capture Approach . 32 13.4 Benefits of using Point Clouds to Capture 3D Objects . 33 13.4.1 Usability . 33 13.4.2 Bandwidth . 33 13.4.3 Total Processing Time . 34 13.5 Limitations of Point Cloud Capture . 34 13.5.1 Computer Vision in 3D . 34 13.5.2 Fixed Viewpoint . 34 13.5.3 RGB Support . 34 13.5.4 Point Resolution . 35 14 Societal Impact 36 14.1 Ethical . 36 14.2 Social . 36 14.3 Political . 37 14.4 Economic . 37 14.5 Health and Safety . 37 14.6 Manufacturability . 37 14.7 Environmental Impact . 38 14.8 Sustainability . 38 14.9 Usability . 38 14.10Lifelong Learning . 39 14.11Compassion . 39 15 Conclusion 40 A Source Code 43 A.1 Backend . 43 A.1.1 Server: HTTP endpoint . 43 A.1.2 Server: Point cloud processing and meshing . 44 A.2 Repository . 52 v List of Figures 2.1 A point cloud mesh model of a dolphin . .2 6.1 Starting State of the Main Menu . 14 6.2 Input Filename Dialog . 15 6.3 Capture Screen . 15 6.4 Main menu after front has been captured . 16 6.5 Main menu after both have been captured . 16 6.6 Standard Deviation Input . 17 6.7 Finished Capture . 17 12.1 Development Timeline Gantt Chart . 25 13.1 Computer Mouse . 26 13.2 Computer mouse from another angle . 27 13.3 Mouse with outlier and plane removal . 28 13.4 Mouse isolated using cluster selection . 29 13.5 Side of shoe captured using improved techniques . 30 13.6 Original and mirrored cloud combination . 31 13.7 Original and mirrored cloud combination(2) . 31 13.8 Shoe mesh using Poisson.