Analysis and Categorization of 2D Multi-Touch Gesture Recognition Techniques THESIS Presented in Partial Fulfillment of the Requ
Total Page:16
File Type:pdf, Size:1020Kb
Analysis and Categorization of 2D Multi-Touch Gesture Recognition Techniques THESIS Presented in Partial Fulfillment of the Requirements for the Degree Master of Science in the Graduate School of The Ohio State University By Aditi Singhal Graduate Program in Computer Science and Engineering The Ohio State University 2013 Master's Examination Committee: Dr. Rajiv Ramnath, Adviser Dr. Jay Ramanathan, Co-Adviser Copyright by Aditi Singhal 2013 Abstract Various techniques have been implemented for wide variety of gesture recognition applications. However, discerning the best technique to adopt for a specific gesture application is still a challenge. Wide variety, complexity in the gesture and the need for complex algorithms make implementation of recognition difficult even for basic gesture applications. In this thesis, different techniques for two dimensional (2D) touch gestures are reviewed, compared and categorized based on user requirements of the gesture applications. This work introduces two main paradigms for gesture applications: a) gestures for direct manipulation and navigation, b) gesture-based languages. These two paradigms have specific and separate roles in 2D touch gesture systems. To provide the clear distinction between two paradigms, three different algorithms are implemented for basic to complex 2D gestures using simple sometimes, as well as complex techniques such as linear regression and Hidden Markov Models. Thereafter, these algorithms are analyzed based on their performance and their fit with application requirements. ii Dedication I lovingly dedicate this thesis to my husband Shashnk Agrawal, who supported me each step of the way, my son Aryan Agrawal who in his 3 year old wisdom knew that mommy should not be disturbed while studying, and my late father Dr. Devendra Kumar Singhal whose passion for education encouraged me to increase my knowledge and was a driving force in my decision to pursue a graduate degree. iii Acknowledgments I would like to express my deepest gratitude to my advisor, Dr. Rajiv Ramnath, for his excellent guidance, patience, and provision of wonderful atmosphere to excel. His continuous support and ease of interaction when asked for any work or non-work related assistance were greatly appreciated. I would also like to thank my co-adviser, Dr. Jay Ramanathan for believing in me. Her guidance helped me in doing research and in writing my thesis. In addition, a special thank you to Tom Lynch for all his help and support, and also the encouragement and insightful comments, which helped me shape my thesis. iv Vita 2001-2005........................................................BS, Information Technology, Galgotia college of Engineering and Technology, India April 2006 to May 2007…………………… Tata Consultancy Services, India Sep 2010 to Present………………………….Graduate student at The Ohio State University Fields of Study Major Field: Computer Science and Engineering v Table of Contents Abstract……………………………………………………………………………….…...ii Dedication………………………………………………………………………………...iii Acknowledgments………………………………………………………………………..iv Vita…………………………………………………………………………...…….……..v List of Tables…………………………………………………………………………......ix List of Figures……………………………………………………………………………..x Chapter 1: Introduction...................................................................................................... 1 1.1 Gesture Recognition Overview ................................................................................. 1 1.2 Motivation................................................................................................................. 3 1.3 Thesis contribution and scope................................................................................... 4 Chapter 2: Related Work .................................................................................................... 6 Chapter 3: Architectural Framework and Requirements .................................................. 12 3.1 Architecture Overview ............................................................................................ 12 3.2 Windows 7 Touch Framework................................................................................ 13 3.2.1 Touch Events .................................................................................................... 13 vi 3.2.2 Manipulation Events......................................................................................... 14 3.2.3 Touch Frame-Reported Event .......................................................................... 15 Chapter 4: Direct Manipulation Algorithm ...................................................................... 17 4.1 Algorithm Overview ............................................................................................... 17 4.2 Experimental Procedure and Software Design ....................................................... 18 4.3 Results and Findings ............................................................................................... 22 Chapter 5: Normalization and Gesture Recognition......................................................... 24 5.1 Algorithm Overview ............................................................................................... 24 5.2 Algorithm concepts ................................................................................................. 24 5.2.1 Input Homogenization and Normalization ....................................................... 24 5.2.2 Gesture Recognition ......................................................................................... 27 5.3 Experimental Procedure and Software Design ....................................................... 31 5.3.1 Normalization and Heuristic............................................................................. 31 5.3.2 Normalization and Pattern Matching................................................................ 33 5.4 Results and Findings ............................................................................................... 36 Chapter 6: Generalized Gesture using The Hidden Markov Model ................................. 40 6.1 Introduction ............................................................................................................. 40 6.2 Hidden Markov Model overview ............................................................................ 40 6.2.1 Specification and parameter of an HMM ......................................................... 41 vii 6.2.2 Learning Problem of Hidden Markov Model................................................... 42 6.3 The Hidden Markov Model in Gesture Recognition............................................... 44 6.4 Experimental Procedure and Software Design ....................................................... 47 6.5 Results and Findings ............................................................................................... 50 Chapter 7: Comparison, Conclusion and Future Work..................................................... 55 7.1 Algorithm selection and distinction based on two paradigms of gesture recognition: ....................................................................................................................................... 55 7.2 Findings................................................................................................................... 57 7.3 Conclusions............................................................................................................. 58 7.4 Future Work ............................................................................................................ 58 References......................................................................................................................... 60 viii List of Tables Table 1: Line circle interception....................................................................................... 26 Table 2: Heuristic two finger gesture results .................................................................... 36 Table 3: Pattern matching 2 finger gesture results ........................................................... 37 Table 4: Results for digit database.................................................................................... 53 Table 5: Results for customized database......................................................................... 53 Table 6: Results for DragZoomRotate database ............................................................... 54 ix List of Figures Figure 1: Gesture Recognition Systems and Techniques. .................................................. 2 Figure 2: Tabletop using SmartSkin Technology [10]. ...................................................... 3 Figure 3: Touch sensor technology in tabletops [10]. ........................................................ 7 Figure 4: Camera mounted hat and plastic markers to project gesture information [16]. .. 8 Figure 5: Image feature extraction and gesture recognition [18]........................................ 8 Figure 6: Various hand gloves in hand tracking systems [9].............................................. 9 Figure 7: Gesture recognition using classification [20].................................................... 10 Figure 8: Use of the Wii remote with other applications for gesture recognition [21, 22]11 Figure 9: Ordered flow of the touch events. ..................................................................... 14 Figure 10: Ordered flow of the touch events. ..................................................................