A Novel Approach to Complex Human Activity Recognition Md Osman Gani Marquette University
Total Page:16
File Type:pdf, Size:1020Kb
Marquette University e-Publications@Marquette Dissertations (2009 -) Dissertations, Theses, and Professional Projects A Novel Approach to Complex Human Activity Recognition Md Osman Gani Marquette University Recommended Citation Gani, Md Osman, "A Novel Approach to Complex Human Activity Recognition" (2017). Dissertations (2009 -). 701. https://epublications.marquette.edu/dissertations_mu/701 A NOVEL APPROACH TO COMPLEX HUMAN ACTIVITY RECOGNITION by Md Osman Gani, B.S., M.S. A Dissertation submitted to the Faculty of the Graduate School, Marquette University, in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Milwaukee, Wisconsin May 2017 ABSTRACT A NOVEL APPROACH TO COMPLEX HUMAN ACTIVITY RECOGNITION Md Osman Gani, B.S., M.S. Marquette University, 2017 Human activity recognition is a technology that offers automatic recognition of what a person is doing with respect to body motion and function. The main goal is to recognize a person’s activity using different technologies such as cameras, motion sensors, location sensors, and time. Human activity recognition is important in many areas such as pervasive computing, artificial intelligence, human-computer interaction, health care, health outcomes, rehabilitation engineering, occupational science, and so- cial sciences. There are numerous ubiquitous and pervasive computing systems where users’ activities play an important role. The human activity carries a lot of information about the context and helps systems to achieve context-awareness. In the rehabilitation area, it helps with functional diagnosis and assessing health outcomes. Human activity recognition is an important indicator of participation, quality of life and lifestyle. There are two classes of human activities based on body motion and function. The first class, simple human activity, involves human body motion and posture, such as walking, running, and sitting. The second class, complex human activity, includes function along with simple human activity, such as cooking, reading, and watching TV. Human activity recognition is an interdisciplinary research area that has been active for more than a decade. Substantial research has been conducted to recognize human ac- tivities, but, there are many major issues still need to be addressed. Addressing these issues would provide a significant improvement in different aspects of the applications of the human activity recognition in different areas. There has been considerable re- search conducted on simple human activity recognition, whereas, a little research has been carried out on complex human activity recognition. However, there are many key aspects (recognition accuracy, computational cost, energy consumption, mobility) that need to be addressed in both areas to improve their viability. This dissertation aims to address the key aspects in both areas of human activity recognition and eventually focuses on recognition of complex activity. It also addresses indoor and outdoor local- ization, an important parameter along with time in complex activity recognition. This work studies accelerometer sensor data to recognize simple human activity and time, location and simple activity to recognize complex activity. i ACKNOWLEDGMENTS Md Osman Gani, B.S., M.S. I thank Almighty Allah for giving me the opportunity and strength to complete my PhD studies. I would like to express my heartfelt gratitude to my supervisor, Dr. Sheikh Iqbal Ahamed, for his continuous support, encouragement, and guidance. His thoughtful directions and supervision were invaluable. I am grateful to Dr. Roger O. Smith for his enthusiasm, innovative ideas, guidance, and continuous support to address research challenges throughout my PhD studies. His critical thinking about the methodologies helped me keep the focus on my research work. Special thanks to Dr. Richard Povinelli for his guidance with a deep understanding of the time-delay embedding and continuous guidance and support in simple human activity recognition research. I am also grateful to Dr. Dennis Brylow, and Dr. Mehdi Maadooliat for their invaluable time, advice, and inspiration. I would like to thank all committee members for taking their valuable time and serving on my PhD committee. I would like to thank Dr. Rebecca Sanders, Dr. Gary Krenz, and Dr. Stephen Merrill for their continuous support during my graduate studies. I would like to acknowledge the continuous support from the department during my graduate studies. Being a recipient of the Computational Sciences Summer Research Fellowship in 2012, 2013, and 2015, I want to acknowledge the opportunity to strengthen my research experience and progress my research works. Through this prestigious opportunity, I was able to outline the approach to complex activity recognition, and develop indoor localization and simple human activity recognition techniques. This is worth mentioning that all three works are part of my PhD dissertation. Being diagnosed with a chronic illness at the beginning of the PhD studies, it was indeed a difficult journey for me. I would like to thank my physicians, Dr. A. J. Gorve and Dr. Brandt Mylott, from Marquette University Clinic, and my physiotherapist, Dr. Albojay Deacon, from Marquette University Sports Rehabilitation Clinic, for taking care of my health. I am grateful to my friends Imran Reza Ananta, Amit Kumar Saha, G. M. Tanim, Mohammad Adibuzzaman, Ferdaus Kawsar, Ishmat Zerin, Jahangir Majumder and Casey O’Brien for supporting me in every possible way. ii My special thanks go to my wonderful wife, Taskina Fayezeen Disha, for her never ending support, encouragement, and unfailing love. Thanks for always being there for me. And last but not the least, I would like to take this opportunity to thank my parents, sister, and brothers for their unconditional love and support. They always tried their best to arrange the best possible education for me. My sincere gratitude goes to my in-laws who were very supportive during every step of my PhD studies. I am grateful to my undergraduate thesis advisor Dr. Hasan Sarwar. I was fortunate enough to have an excellent mentor like him. iii TABLE OF CONTENTS ACKNOWLEDGMENTS i LIST OF TABLES viii LIST OF FIGURES ix 1 INTRODUCTION 1 1.1 Dissertationfocus............................. 3 1.2 DissertationStatement . .. .. .. .. .. .. .. 4 1.3 ResearchChallenges ........................... 5 1.4 Dissertationorganization . 5 1.5 Publications................................ 7 1.5.1 Conferences/Journals. 7 1.5.2 Poster............................... 9 2 HUMANACTIVITYANDHUMANACTIVITYRECOGNITION 10 2.1 Terminology................................ 10 2.2 TaxonomyofHumanActivities. 11 2.2.1 SimpleHumanActivityorSimpleActivity . 11 2.2.2 ComplexHumanActivityorComplexActivity . 11 2.3 Taxonomy of Human Activity Recognition Approaches . ..... 12 2.3.1 ComputerVision ......................... 13 2.3.2 EnvironmentalSensor . 13 2.3.3 WearableSensor ......................... 14 2.3.4 TimeGeography ......................... 14 iv 3 BACKGROUND 16 3.1 ExponentialRegression. 16 3.2 DelayReconstructionofthePhaseSpace . .. 17 3.2.1 PhaseSpace............................ 17 3.2.2 DynamicalSystem . .. .. .. .. .. .. 17 3.2.3 ReconstructedPhaseSpace. 18 3.3 GaussianMixtureModels. 19 3.4 MaximumLikelihoodClassifier . 19 3.5 MarkovModel .............................. 19 3.6 HiddenMarkovModel .......................... 20 4 INDOORANDOUTDOORLOCALIZATION 21 4.1 Introduction................................ 21 4.1.1 Contributions........................... 23 4.2 RelatedWork ............................... 24 4.3 OurApproach............................... 32 4.3.1 Localization of Mobile Wi-Fi Node with Smartphone . ... 32 4.3.2 LocalizationofSmartphonewithWi-Fi Routers . ... 37 4.4 Evaluation................................. 39 4.4.1 Scenario1............................. 40 4.4.2 Scenario2............................. 41 4.4.3 Scenario3............................. 42 4.5 Discussion................................. 44 4.6 Conclusion ................................ 45 4.7 RelatedPublications ........................... 46 4.7.1 Publications............................ 46 4.7.2 Poster............................... 47 v 5 SIMPLEHUMANACTIVITYRECOGNITION 48 5.1 Introduction................................ 48 5.2 RelatedResearch ............................. 51 5.3 Background................................ 55 5.3.1 ReconstructedPhaseSpace. 55 5.3.2 GaussianMixtureModels . 56 5.3.3 MaximumLikelihoodClassifier . 58 5.4 Data.................................... 58 5.4.1 DevelopmentoftheDataCollectionTool . 59 5.4.2 DataCollection.......................... 62 5.4.3 PublicDataset .......................... 64 5.5 Method .................................. 65 5.5.1 Training.............................. 65 5.5.2 Testing .............................. 66 5.6 Evaluation................................. 67 5.6.1 ExperimentwithOurApproach . 67 5.6.2 Experiment with Time-Domain Features and Classification Al- gorithms ............................. 71 5.6.3 Experiment with Time and Frequency Domain Features . .. 72 5.6.4 Results .............................. 72 5.7 Discussion................................. 78 5.8 Conclusion ................................ 83 5.9 RelatedPublications ........................... 84 5.9.1 Publications............................ 84 5.9.2 Poster............................... 84 6 COMPLEXHUMANACTIVITYRECOGNITION 86 6.1 Introduction................................ 86 vi 6.2 RelatedWork ............................... 88