A Data-Driven Approach to Object Classification Through Fog
Total Page:16
File Type:pdf, Size:1020Kb
A Data-Driven Approach to Object Classification through Fog by Alisha Saxena Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degree of Master of Engineering in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology June 2018 ã 2018 Alisha Saxena. All rights reserved. The author hereby grants to M.I.T. permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole and in part in any medium now known or hereafter created. Author: ____________________________________________ Department of Electrical Engineering and Computer Science May 25, 2018 Certified by: ____________________________________________ Ramesh Raskar, Associate Professor, Thesis Supervisor May 25, 2018 Accepted by: ____________________________________________ Katrina LaCurts, Chair, Master of Engineering Thesis Committee 2 A Data-Driven Approach to Object Classification Through Fog by Alisha Saxena Submitted to the Department of Electrical Engineering and Computer Science May 25, 2018 In Partial Fulfillment of the Requirements for the Degree of Master of Engineering in Electrical Engineering and Computer Science ABSTRACT Identifying objects through fog is an important problem that is difficult even for the human eye. Solving this problem would make autonomous vehicles, drones, and other similar systems more resilient to changing natural weather conditions. While there are existing solutions for dehazing images occluded by light fog, these solutions are not effective in cases of very dense fog. Hence, we present a system that uses a combination of time resolved sensing, specifically using Single Photon Avalanche Photodiode (SPAD) cameras, and deep learning with convolutional neural networks to detect and classify objects when imaged through extreme scattering media like fog. This thesis describes our three-pronged approach to solving this problem: (1) building simulation software to gather sufficient training data, (2) verifying and benchmarking output of simulation with real-life fog data, (3) training deep learning models to classify objects occluded by fog. Thesis Supervisor: Ramesh Raskar Title: Associate Professor 3 4 Acknowledgements I would firstly like to thank Professor Ramesh Raskar for giving me the opportunity to work on this project for my MEng. A huge thanks to Guy Satat for providing invaluable guidance for my research and patiently helping me overcome roadblocks in my work. His mentorship has allowed me to become a better researcher and learner. Without his support, I would not have been able to start, let alone complete, this thesis. I’d also like to thank my fellow members of the Camera Culture group, in particular, Tristan Swedish, Otkrist Gupta, and Matt Tancik, for providing useful insights for a variety of tasks. Every single one of the group members has been helpful to me in some way throughout my time in the lab. It was a privilege to work with the Camera Culture group, and the whole team made this masters an amazing learning experience. Thanks to Maggie Church and Maggie Cohen, who have been instrumental in managing the logistics of completing this thesis. A special thanks also to Anne Hunter for helping me through my BS and MEng degrees with her encyclopedic knowledge about the nuances of the EECS department. Lastly, I would like to thank my family, who have always supported me and helped me dream big. This milestone would not have been possible without their years of sacrifice and dedication, and for that I am very grateful. 5 6 Table of Contents 1 Introduction ................................................................................................ 11 2 Related Work ............................................................................................. 12 2.1 Fog Properties .............................................................................................. 12 2.2 Simulation Software ...................................................................................... 12 2.3 Combined Approach ..................................................................................... 13 3 Overview of Approach ................................................................................ 15 4 Simulation Design ....................................................................................... 16 4.1 Technical Details .......................................................................................... 16 4.2 Setup ............................................................................................................. 17 4.3 Input Parameters .......................................................................................... 18 4.3.1 Fog properties ............................................................................................................. 19 4.3.2 Light sources ............................................................................................................... 20 4.3.3 Target ......................................................................................................................... 21 4.3.4 SPAD Camera ............................................................................................................ 23 4.3.5 Lens ............................................................................................................................ 23 4.4 Algorithm ..................................................................................................... 24 4.5 Output .......................................................................................................... 24 4.6 Challenges ..................................................................................................... 25 4.6.1 Lens ............................................................................................................................ 25 4.6.2 Speed .......................................................................................................................... 26 5 Verification ................................................................................................. 27 5.1 Unit testing ................................................................................................... 27 5.2 Visual testing ................................................................................................ 28 5.3 Verifying with lab measurements ................................................................. 29 7 6 Data Generation ......................................................................................... 32 7 Deep Learning ............................................................................................ 34 7.1 Input ............................................................................................................. 34 7.2 VGG ............................................................................................................. 35 7.2.1 Architecture ................................................................................................................ 36 7.2.2 Training ...................................................................................................................... 37 7.2.3 Results ........................................................................................................................ 37 7.3 AlexNet ........................................................................................................ 39 7.3.1 Architecture ................................................................................................................ 39 7.3.2 Training ...................................................................................................................... 40 7.3.3 Results ........................................................................................................................ 40 7.4 C3D .............................................................................................................. 41 7.4.1 Architecture ................................................................................................................ 41 7.4.2 Training ...................................................................................................................... 42 7.4.3 Results ........................................................................................................................ 43 8 Future Work .............................................................................................. 45 8.1 Simulation ..................................................................................................... 45 8.1.1 Target ......................................................................................................................... 45 8.1.2 Speed .......................................................................................................................... 46 8.2 Data Generation ........................................................................................... 47 8.3 Deep Learning ............................................................................................... 48 9 Contributions ............................................................................................. 49 10 Conclusion ................................................................................................ 50 Bibliography ................................................................................................... 51 8