A Web-Based Software Platform for Data Processing Workflows and its Applications in Aerial Data Analysis A thesis submitted to the graduate school of the University of Cincinnati in partial fulfilment of the requirements for the degree of Master of Science in the department of Electrical Engineering and Computer Science of the College of Engineering and Applied Sciences June, 2019 by Niranjan R Krishnan B. Tech, Cochin University of Science and Technology, Cochin, India 2013 Committe Chair: Dr. Arthur Helmicki, PhD Committee Members: Dr. Victor Hunt, PhD Dr. Nan Niu, PhD Abstract Given the rapid advances in development of unmanned aerial vehicles (UAV), em- ployment of drones in various business functions becomes reasonable and afford- able. Usage of drones as data collection tools will give us access to a new set of geo-referenced images and videos that were not easily accessible in the past. The ultimate objective of this web based platform for data processing workflows, from here on referred to as the common operating platform, is to enable users to archive, process and visualize aerial data without a need for advanced hardware and software locally. This work details the development of the the common operating platform which consist of a web-based frontend and a backend. The frontend is a web app developed based on Django, Twitter Bootstrap and Javascript where the user authenticates, uploads data, submit processing tasks and visualizes the results. The back end, developed using Python 3, is where data is being stored and various processing tasks are being done based on commercial(Pix4D), open source(OpenDroneMap) and custom(Traffic Monitoring) processing engines. First, the intricacies of the data processing workflow is discussed and this in- cludes diving into detail the various steps related to processing workflows using proprietary, open software and custom software. Second, procedures for integrating commercial processing engines as well as development of the in house traffic param- eter extraction system will be shown and the result of running various case studies and their processing performance will be discussed. And finally, the system archi- tecture design and implementation will be detailed given the scalability, modularity, extensibility and reliablity requirements will be discussed. The idea is to have a secure system, which is accessible to a broad audience, that can receive and service all of their processing requirements. In doing so, it dismisses the need for an unini- tiated audience to install highly specialized software on their personal computers as well as relieves the need for them to maintain sophisticated hardware. i Acknowledgements I wish to acknowledge the expertise and feedback of my professors Dr. Arthur Helmicki and Dr. Victor Hunt. Special thanks to my colleagues, Arjun Chiddarwar and Dr. Mahdi Norouzi, for all their help. I would also like to thank the Ohio De- partment of Transportation(ODOT) for funding the research project. Their support gave us all the resources we could ask for to accomplish this task. I would also like to acknowledge the work of the pilots from the UAV Master Labs at the University of Cincinnati for flying the data collection missions for the uninterrupted traffic flow calculations, Aswin Balasubramaniam for his help with testing and benchmarking the platforms capabilities, Sudhamsh Korrapati for his help with implementing the index page on the website and Ganesh Vathyam and ODOT IT for helping migrate the platform to ODOT's internal servers. Finally, I would like to thank my Mom and Dad for their love, endless support and encouragement. ii Contents 1 Introduction 1 1.1 Problem Statement . .1 1.2 Motivation of Research . .2 1.3 Basic Concepts and Definitions . .2 1.3.1 3D Modeling . .3 1.3.2 Docker and Containerization . .4 1.3.3 Deep Learning Frameworks for Computer Vision . .6 1.3.4 Object Detection . .8 1.3.5 Object Tracking . 10 1.3.6 GPU Accelerated Processing . 10 1.3.7 Traffic Metrics . 11 1.4 Proposed Approach . 13 2 Common Operating Platform 15 2.1 Distributed System Architecture . 15 2.1.1 The Web Interface . 17 2.1.2 Database Design . 21 2.1.3 Backend Systems . 24 2.2 Installation . 26 2.2.1 System Requirements . 26 2.2.2 Special Configurations . 27 2.3 Templatizing Workflows . 33 iii CONTENTS 2.4 Implementation Issues . 36 2.4.1 Changing IT Eco-Systems . 36 2.4.2 Maximum file upload size . 37 2.4.3 Scalability . 38 2.5 Results . 38 2.5.1 Register and Login . 39 2.5.2 Navigating the Dashboard . 39 2.5.3 Processing 3D models . 40 2.5.4 Visualize Point Clouds . 42 2.5.5 Process Traffic Monitoring Projects . 44 2.5.6 Change Password . 46 2.5.7 Create API Keys . 46 2.5.8 Benchmarks . 47 3 Custom Software Design: Traffic Monitoring 50 3.1 Objectives . 50 3.2 Region of Interest . 51 3.3 Feature Extraction . 54 3.4 Lane Detection . 59 3.4.1 Detecting Lane Lines using Hough Transforms . 59 3.4.2 Tracing Lane Lines . 63 3.5 Vehicle Trajectory Analysis . 64 3.6 Results . 67 3.6.1 Counts of Vehicles . 68 3.6.2 Flow Rate, Average Travel Speed and Density . 70 4 Conclusion 81 4.1 Future Scope . 81 References 83 iv CONTENTS CONTENTS A Point Clouds on Potree Viewer 91 CONTENTS v List of Tables 2.1 Defintion of Table: UploadedFiles . 22 2.2 Defintion of Table: DirectoryStructure . 22 2.3 Defintion of Table: APIAuth . 23 2.4 Defintion of Table: TaskBucket . 23 2.5 Defintion of Table: PostProcessedData . 24 2.6 Specifications of Production Server . 38 2.7 Specifications of Production and Test Servers . 48 2.8 Pix4D vs ODM Benchmarks . 48 3.1 Human Counts from April 18, 2018 . 69 3.2 Machine Counts from April 18, 2018 . 69 3.3 Error(%) from April 18, 2018 . 70 3.4 Human Counts from June 20, 2018 . 71 3.5 Continued - Human Counts from June 20, 2018 . 72 3.6 Machine Counts from June 20, 2018 . 73 3.7 Continued - Machine Counts from June 20, 2018 . 74 3.8 Error(%) from June 20, 2018 . 75 3.9 Continued - Error(%) from June 20, 2018 . 76 3.10 Total Counts with Error(%) . 76 3.11 Aggregate of human and machine counts for all videos with error(%) 77 3.12 Results of experiment conducted to verify average travel speed . 77 3.13 Average Metric Values for Lane 6 and Lane 4 . 79 3.14 Average Metric Values for Lane 5 and Lane 1 . 79 vi LIST OF TABLES 3.15 Average Metric Values for Lanes 1, 4, 5 and 6 . 80 LIST OF TABLES vii List of Figures 1.1 A point cloud of a building . .4 1.2 Docker Architecture[7] . .5 1.3 Hypervisor Architecture[7] . .6 1.4 Relationships among Speed, Density and Flow Rate on Uninterrupted Flow Facilities [36] [37] . 14 2.1 System Architecture Diagram . 18 2.2 Entity Relationship Diagram . 21 2.3 Register a user account . 39 2.4 Login Page . 39 2.5 Navigation . 40 2.6 Create Pix4D/ODM Project . 41 2.7 Upload Image Files . 41 2.8 Process Project . 41 2.9 Backend Jobs . 42 2.10 Create a visualization project . 42 2.11 Upload Point Cloud Files . 43 2.12 Process Project . 43 2.13 Backend Jobs . 44 2.14 Create Traffic Monitoring Project . 44 2.15 Upload Video Files for Traffic Monitoring Project . 45 2.16 Process Traffic Monitoring Project . 45 2.17 Status of jobs in Backend Jobs . 46 viii LIST OF FIGURES 2.18 Change Password . 47 2.19 Create / Delete API Keys . 47 2.20 Potree viewer displaying the results of the small project . 48 2.21 Potree viewer displaying the results of the medium project . 49 3.1 Traffic Monitoring Architecture . 51 3.2 Object detection using YOLO . 52 3.3 Region of interest scatter plot . 52 3.4 Region of interest scatter plot with cluster centers . 53 3.5 Region of interest applied to a frame . 54 3.6 Intersection over Union of two bounding boxes from consecutive frames 55 3.7 Original Frame . 60 3.8 Original Frame in HSV Space . 60 3.9 Apply Gaussian Blur . 61 3.10 Filter Yellow and White Colors . 62 3.11 Canny Edge Detection . 62 3.12 Hough Transform Lane Detection . 63 3.13 Lane Detection Block Diagram . 63 3.14 Tracing Lane Lines . 64 3.15 Trajectory of a vehicle recorded by the feature extraction module . 65 3.16 Lane labels assigned by the lane detection module . 68 3.17 Comparison of Lane 6 with Lane 4 . ..