Purdue University Purdue e-Pubs Open Access Theses Theses and Dissertations Spring 2014 Moving object detection for interception by a humanoid robot Saltanat B. Tazhibayeva Purdue University Follow this and additional works at: https://docs.lib.purdue.edu/open_access_theses Part of the Artificial Intelligence and Robotics Commons Recommended Citation Tazhibayeva, Saltanat B., "Moving object detection for interception by a humanoid robot" (2014). Open Access Theses. 270. https://docs.lib.purdue.edu/open_access_theses/270 This document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries. Please contact [email protected] for additional information. *UDGXDWH6FKRRO(7')RUP 5HYLVHG 0114 PURDUE UNIVERSITY GRADUATE SCHOOL Thesis/Dissertation Acceptance 7KLVLVWRFHUWLI\WKDWWKHWKHVLVGLVVHUWDWLRQSUHSDUHG %\ Saltanat B. Tazhibayeva (QWLWOHG Moving object detection for interception by a humanoid robot Master of Science )RUWKHGHJUHHRI ,VDSSURYHGE\WKHILQDOH[DPLQLQJFRPPLWWHH Eric T. Matson Julia M. Taylor Anthony H. Smith 7RWKHEHVWRIP\NQRZOHGJHDQGDVXQGHUVWRRGE\WKHVWXGHQWLQWKHThesis/Dissertation Agreement. Publication Delay, and Certification/Disclaimer (Graduate School Form 32)WKLVWKHVLVGLVVHUWDWLRQ adheres to the provisions of 3XUGXH8QLYHUVLW\¶V³3ROLF\RQ,QWHJULW\LQ5HVHDUFK´DQGWKHXVHRI FRS\ULJKWHGPDWHULDO Eric T. Matson $SSURYHGE\0DMRU3URIHVVRU V BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB $SSURYHGE\Computer and Information Technology 4/25/2014 +HDGRIWKHDepartment *UDGXDWH3URJUDP 'DWH MOVING OBJECT DETECTION FOR INTERCEPTION BY A HUMANOID ROBOT A Thesis Submitted to the Faculty of Purdue University by Saltanat B. Tazhibayeva In Partial Fulfillment of the Requirements for the Degree of Master of Science May 2014 Purdue University West Lafayette, Indiana ii I dedicate this thesis work to my parents, Bulatbek and Balgaisha - for your endless love and belief in me, for no matter what. To my husband, Mukhtar - for your encouragement and support, I would not have done that without you. To my smartest brother, Yedyge - for raising me in love and playing logic games. iii ACKNOWLEDGMENTS I would like to gratefully thank my thesis committee for their useful comments and guidance. Doctor Eric Matson, Doctor Julia Taylor, and Doctor Tony Smith provided me with valuable help and assistance. Assistance given by Doctor John A. Springer, Amy Wagoner, and Mauricio Gomez is greatly appreciated. I wish to thank my family and friends for their support. iv TABLE OF CONTENTS Page LIST OF TABLES :::::::::::::::::::::::::::::::: vi LIST OF FIGURES ::::::::::::::::::::::::::::::: vii GLOSSARY :::::::::::::::::::::::::::::::::::: ix ABSTRACT ::::::::::::::::::::::::::::::::::: x CHAPTER 1. INTRODUCTION :::::::::::::::::::::::: 1 1.1 Problem statement ::::::::::::::::::::::::::: 1 1.2 Research Question ::::::::::::::::::::::::::: 2 1.3 Scope :::::::::::::::::::::::::::::::::: 2 1.4 Significance ::::::::::::::::::::::::::::::: 2 1.5 Assumptions ::::::::::::::::::::::::::::::: 3 1.6 Limitations ::::::::::::::::::::::::::::::: 3 1.7 Delimitations :::::::::::::::::::::::::::::: 4 1.8 Summary :::::::::::::::::::::::::::::::: 4 CHAPTER 2. REVIEW OF RELEVANT LITERATURE :::::::::: 5 2.1 Applications of intercepting robots :::::::::::::::::: 5 2.1.1 Industrial - Robotic hand-eye system ::::::::::::: 5 2.1.2 Human adapted games ::::::::::::::::::::: 7 2.2 Movement trajectory :::::::::::::::::::::::::: 8 2.3 Methods for target tracking and interception ::::::::::::: 9 2.3.1 Optic-flow calculation ::::::::::::::::::::: 9 2.3.2 Statistical background subtraction ::::::::::::::: 10 2.3.3 Kalman filters :::::::::::::::::::::::::: 11 2.3.4 Novel gaze-based approach ::::::::::::::::::: 12 2.3.5 Updating while navigating ::::::::::::::::::: 13 2.3.6 Line of sight ::::::::::::::::::::::::::: 15 2.3.7 Fifo graphs ::::::::::::::::::::::::::: 16 2.3.8 Object detection based on a \boosted cascades" ::::::: 17 2.3.9 Neural networks ::::::::::::::::::::::::: 18 2.3.10 2D spatial operation into 3D :::::::::::::::::: 20 2.4 Summary :::::::::::::::::::::::::::::::: 21 CHAPTER 3. FRAMEWORK AND METHODOLOGY ::::::::::: 22 3.1 Study Design :::::::::::::::::::::::::::::: 22 v Page 3.2 Unit & Sampling :::::::::::::::::::::::::::: 25 3.2.1 Hypotheses ::::::::::::::::::::::::::: 25 3.2.2 Population :::::::::::::::::::::::::::: 26 3.2.3 Sample :::::::::::::::::::::::::::::: 26 3.2.4 Variables ::::::::::::::::::::::::::::: 29 3.2.5 Measure for Success ::::::::::::::::::::::: 29 3.3 Summary :::::::::::::::::::::::::::::::: 29 CHAPTER 4. SYSTEM OVERVIEW AND DESIGN ::::::::::::: 30 4.1 System overview :::::::::::::::::::::::::::: 30 4.2 Intercepting Robot Controller ::::::::::::::::::::: 31 4.3 Vision processing system :::::::::::::::::::::::: 33 4.4 Summary :::::::::::::::::::::::::::::::: 36 CHAPTER 5. RESEARCH DATA AND RESULTS :::::::::::::: 37 5.1 DARwIn humanoid robot cascade classifiers data :::::::::: 37 5.2 Contour threshold parameter adjustment ::::::::::::::: 40 5.3 Interception test results :::::::::::::::::::::::: 42 5.3.1 First 10 test results for Interceptor and Target scenarios :: 42 5.4 Statistical analysis ::::::::::::::::::::::::::: 47 5.4.1 The test of H0 hypothesis ::::::::::::::::::: 47 5.4.2 Performance analysis of tracking system ::::::::::: 49 5.4.3 Performance analysis of detection system ::::::::::: 50 5.4.4 Simple statistics for the parameters of test outputs ::::: 50 5.5 Summary :::::::::::::::::::::::::::::::: 51 CHAPTER 6. CONCLUSIONS AND FUTURE WORKS :::::::::: 52 6.1 Conclusions ::::::::::::::::::::::::::::::: 52 6.2 Future works :::::::::::::::::::::::::::::: 53 6.3 Summary :::::::::::::::::::::::::::::::: 53 LIST OF REFERENCES :::::::::::::::::::::::::::: 54 APPENDIX A. PURSUER AND TARGET CONFIGURATIONS :::::: 57 APPENDIX B. INTERCEPTOR AND TARGET INTERCEPTION TEST RESULTS CONTINUED ::::::::::::::::::::::::::::::::: 60 APPENDIX C. CREATETRAINSAMPLES.PL BY NAOTOSHI SEO (2009) 67 vi LIST OF TABLES Table Page 3.1 Simulation environment configurations :::::::::::::::::: 28 5.1 Data set sizes for target detection system ::::::::::::::::: 38 5.2 Expanded positive data set sizes for target detection system :::::: 39 5.3 Training time of classifiers for indoor sample sets :::::::::::: 40 5.4 T - test procedure table :::::::::::::::::::::::::: 47 5.5 One sample proportion z - test :::::::::::::::::::::: 49 5.6 False negative and false positive :::::::::::::::::::::: 51 5.7 Simple Statistics :::::::::::::::::::::::::::::: 51 A.1 Pursuer and Target initial heading angle configurations (in radians) :: 57 A.2 Target and Ball x y coordinate values (in meters) :::::::::::: 58 vii LIST OF FIGURES Figure Page 2.1 Trajectory of pursuer and interceptor (Manchester, Low, & Savkin, 2008, p. 494) :::::::::::::::::::::::::::::::::::: 15 2.2 Haar-like features (Bradski & Kaehler, 2008, p. 509) :::::::::: 17 3.1 Cascade classifier training procedure ::::::::::::::::::: 23 3.2 Webots Robotic Simulation Enviroment for the DARwIn-OP robot :: 24 3.3 Example of traincascade configuration :::::::::::::::::: 27 3.4 Target and Ball random positions ::::::::::::::::::::: 28 4.1 Basic system diagram ::::::::::::::::::::::::::: 30 4.2 Flowchart diagram of Intercept Controller :::::::::::::::: 32 4.3 Flowchart diagram of \Find DARwIn" process :::::::::::::: 34 4.4 Detected DARwIn window with contour mask and sparse optical flow points :::::::::::::::::::::::::::::::::::: 35 5.1 Simulation data set for DARwIn object :::::::::::::::::: 37 5.2 Data set of images taken from indoor lab for DARwIn object :::::: 38 5.3 Generated training set image samples for real world ::::::::::: 39 5.4 Generated training set image samples for simulation ::::::::::: 39 5.5 Threshold testing results :::::::::::::::::::::::::: 41 5.6 Threshold with blurring window results :::::::::::::::::: 41 5.7 Test #1 ::::::::::::::::::::::::::::::::::: 42 5.8 Test #2 ::::::::::::::::::::::::::::::::::: 43 5.9 Test #3 ::::::::::::::::::::::::::::::::::: 43 5.10 Test #4 ::::::::::::::::::::::::::::::::::: 44 5.11 Test #5 ::::::::::::::::::::::::::::::::::: 44 5.12 Test #6 ::::::::::::::::::::::::::::::::::: 45 viii Figure Page 5.13 Test #7 ::::::::::::::::::::::::::::::::::: 45 5.14 Test #8 ::::::::::::::::::::::::::::::::::: 46 5.15 Test #9 ::::::::::::::::::::::::::::::::::: 46 5.16 Test #10 :::::::::::::::::::::::::::::::::: 46 5.17 95% C.I for the time of interception :::::::::::::::::::: 48 5.18 Interception time(s) versus distance(m) graph :::::::::::::: 48 B.1 Test #11 :::::::::::::::::::::::::::::::::: 60 B.2 Test #12 :::::::::::::::::::::::::::::::::: 60 B.3 Test #13 :::::::::::::::::::::::::::::::::: 60 B.4 Test #14 :::::::::::::::::::::::::::::::::: 61 B.5 Test #15 :::::::::::::::::::::::::::::::::: 61 B.6 Test #16 :::::::::::::::::::::::::::::::::: 61 B.7 Test #17 :::::::::::::::::::::::::::::::::: 62 B.8 Test #18 :::::::::::::::::::::::::::::::::: 62 B.9 Test #19 :::::::::::::::::::::::::::::::::: 62 B.10 Test #20 :::::::::::::::::::::::::::::::::: 63 B.11 Test #21 :::::::::::::::::::::::::::::::::: 63 B.12 Test #22 :::::::::::::::::::::::::::::::::: 63 B.13 Test #23 :::::::::::::::::::::::::::::::::: 64 B.14 Test #24 :::::::::::::::::::::::::::::::::: 64 B.15 Test #25 :::::::::::::::::::::::::::::::::: 64 B.16 Test #26 :::::::::::::::::::::::::::::::::: 65 B.17 Test #27 :::::::::::::::::::::::::::::::::: 65 B.18 Test #28 :::::::::::::::::::::::::::::::::: 65 B.19 Test #30 :::::::::::::::::::::::::::::::::: 66 B.20 Test #31 :::::::::::::::::::::::::::::::::: 66 B.21 Test