COST-EFFICIENT VIDEO INTERACTIONS FOR VIRTUAL TRAINING ENVIRONMENT Arsen Gasparyan A Thesis Submitted to the Graduate College of Bowling Green State University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE August 2007 Committee: Hassan Rajaei, Advisor Guy Zimmerman Walter Maner ii ABSTRACT Rajaei Hassan, Advisor In this thesis we propose the paradigm for video-based interaction in the distributed virtual training environment, develop the proof of concept, examine the limitations and discover the opportunities it provides. To make the interaction possible we explore and/or develop methods which allow to estimate the position of user’s head and hands via image recognition techniques and to map this data onto the virtual 3D avatar thus allowing the user to control the virtual objects the similar way as he/she does with real ones. Moreover, we target to develop a cost efficient system using only cross-platform and freely available software components, establishing the interaction via common hardware (computer, monocular web-cam), and an ordinary internet channel. Consequently we aim increasing accessibility and cost efficiency of the system and avoidance of expensive instruments such as gloves and cave system for interaction in virtual space. The results of this work are the following: the method for estimation the hand positions; the proposed design solutions for the system; the proof of concepts based on the two test cases (“Ball game” and “Chemistry lab”) which show that the proposed ideas allow cost-efficient video-based interaction over the internet; and the discussion regarding the advantages, limitations and possible future research on the video-based interactions in the virtual environments. iii This work is dedicated to my ancestors, teachers, colleagues and other kind people. iv ACKNOWLEDGMENTS This study could not be completed without my scientific advisor Dr. Rajaei, who had enough patience and goodwill to guide me through the whole process of scientific research. I would also like to thank Dr. Maner for his persistent help and care and Dr. Zimmerman for his support and precious comments on the committee meetings. A special acknowledgement goes to ACM Digital Library, Wikipedia contributors and the Open Source community members who made all the necessary information and software components available for this research. v TABLE OF CONTENTS Page CHAPTER 1. INTRODUCTION ...................................................................................................1 1.1 Hypothesis......................................................................................................................1 1.2 Goals ..............................................................................................................................1 1.3 Research questions.........................................................................................................3 1.4 Preliminary Research.....................................................................................................3 CHAPTER 2. LITERATURE REVIEW ........................................................................................5 2.1 Virtual Reality/Environment..........................................................................................5 2.2 Visual Aspects ...............................................................................................................6 2.3 Computer Vision..........................................................................................................10 2.4 Preliminary Conclusions..............................................................................................12 CHAPTER 3. METHODOLOGY ................................................................................................14 3.1 The interaction paradigm .............................................................................................14 3.2 Test cases .....................................................................................................................14 3.2.1 Test Case 1: Simple Ball Game ....................................................................15 3.2.2 Test Case 2: Virtual Chemistry Lab..............................................................16 3.2.3 Observations .................................................................................................17 3.3 Preliminary gesture tracking algorithm .......................................................................17 3.4 Proposed methods ........................................................................................................19 CHAPTER 4. IMPLEMENTATION............................................................................................21 4.1 Problem Statement.......................................................................................................22 4.1.1 Problems and solutions .................................................................................23 vi 4.1.2 Test cases ......................................................................................................24 4.2 Overview of available platforms..................................................................................25 4.2.1 Graphics engines...........................................................................................26 4.2.2 Physics engines .............................................................................................27 4.2.3 Sound engines ...............................................................................................28 4.2.4 Video acquisition ..........................................................................................29 4.2.5 Network engines ...........................................................................................29 4.3 Networking ..................................................................................................................30 4.3.1 Preliminary estimation of channel usage ......................................................30 4.3.2 Network architecture.....................................................................................31 4.4 Prototype 1...................................................................................................................32 4.4.1 Prototype 1 Goals..........................................................................................32 4.4.2 Prototype 1 Design........................................................................................32 4.4.3 Prototype 1 Implementation..........................................................................34 4.5 Prototype 2...................................................................................................................36 4.5.1 Prototype 2 Goals..........................................................................................36 4.5.2 Prototype 2 Design........................................................................................36 4.5.3 Prototype 2 Implementation..........................................................................37 4.6 Prototype 3...................................................................................................................39 4.6.1 Prototype 3 Goals..........................................................................................39 4.6.2 Prototype 3 Design........................................................................................40 4.6.3 Prototype 3 Implementation..........................................................................42 4.7 Prototype 4...................................................................................................................43 vii 4.7.1 Prototype 4 Goals..........................................................................................43 4.7.2 Prototype 4 Design........................................................................................43 4.7.3 Prototype 4 Implementation..........................................................................43 CHAPTER 5. RESULTS AND DISCUSSION............................................................................46 5.1 Proof of concept...........................................................................................................46 5.2 Identified problems ......................................................................................................47 5.2.1 Physical avatar ..............................................................................................47 5.2.2 Mapping movements to virtual body ............................................................47 5.2.3 Object representation ....................................................................................48 5.3 Proposed solutions .......................................................................................................49 5.3.1 Interaction paradigm .....................................................................................49 5.3.2 Human communication.................................................................................50 5.3.3 Scalability and networking ...........................................................................50 5.4 Developed algorithm....................................................................................................51 5.5 Estimated feasibility of created interface.....................................................................51 5.6 Integration with web services ......................................................................................52