City Research Online
Total Page:16
File Type:pdf, Size:1020Kb
City Research Online City, University of London Institutional Repository Citation: Hadjiminas, N. and Child, C. H. T. (2012). Be The Controller: A Kinect Tool Kit for Video Game Control - Recognition of Human Motion Using Skeletal Relational Angles. Paper presented at the 5th Annual International Conference On Computer Games, Multimedia And Allied Technology (CGAT 2012), 2012, Bali, Indonesia. This is the unspecified version of the paper. This version of the publication may differ from the final published version. Permanent repository link: https://openaccess.city.ac.uk/id/eprint/2996/ Link to published version: Copyright: City Research Online aims to make research outputs of City, University of London available to a wider audience. Copyright and Moral Rights remain with the author(s) and/or copyright holders. URLs from City Research Online may be freely distributed and linked to. Reuse: Copies of full items can be used for personal research or study, educational, or not-for-profit purposes without prior permission or charge. Provided that the authors, title and full bibliographic details are credited, a hyperlink and/or URL is given for the original metadata page and the content is not changed in any way. City Research Online: http://openaccess.city.ac.uk/ [email protected] Be The Controller: A Kinect Tool Kit for Video Game Control Recognition of human motion using skeletal relational angles Nicholas Hadjiminas Christopher Child Department of Computing, Department of Computing, School of Informatics, City University School of Informatics, City University London, United Kingdom London, United Kingdom [email protected] [email protected] Abstract — As technology evolves, more interactive video game industry by storm becoming a prerequisite for new generation controllers are being developed in order to provide players with game consoles [1]. Motion controllers use different types of greater interaction with games. Motion control is one of the most mechanisms such as, accelerometers, infra-red (IR) sensors and challenging and exciting topics in today's games industry and the cameras. These help detect the player’s movements and related controllers have taken the industry by storm becoming a identify his gestures in order to allow him to interact with the prerequisite for new generation game consoles. game and manipulate items on screen through real-time We propose a new method to quickly and accurately recognize movement and gestures. Contemporary video game controllers human motion using skeletal relation angle recognition rather use a camera, IR projector and IR sensors to track the user’s than body parts Cartesian co-ordinates position recognition in motion without the need for the user to hold a controller. order to provide a more flexible, accurate and efficient way Microsoft’s Kinect holds the Guinness World Record of being tracking human motion. This method was used to develop a tool the "fastest selling consumer electronics device”. It sold an kit that aims to help game designers easily identify and recognise average of 133 units per day with a total of 8 million units sold a user’s pose and gestures using Microsoft’s Kinect motion within the first 60 days of its release [2]. controller, in order to then link these movements with actions in a game for example key presses can be liked to poses or gestures in order to produce key events. Despite the success of motion controllers they have not resulted in the significant and radical change to the game Finally, we evaluated the usability of the tool kit and the success industry expected. This can be explained with a brief look at rate of skeletal relation angle recognition through an experiment. the current games market and the use of motion tracking Ten representative users were selected and asked to complete a controllers. One could describe the market for motion tracking set of tasks using the “Be the Controller” project and a game games as casual, consisting of people with little or no (World of Warcraft) in order to simulate real conditions of use of the software and evaluate its usability. The data from this experience with games, who do not like playing games for experiment helped to form some important conclusions: users extended periods and who will probably not read the found it easy to create their own poses and gestures; they were instruction manual before playing a game [3]. Probably the enthusiastic about the fact that they were able to bind their reason behind this is that motion controllers support more actions to key/mouse events; and were satisfied with the success accessible simple motions compared to the traditional rate of pose recognitions gamepads with their many action buttons. However, other game markets, such as the more traditional core games, appear Keywords- Pose and gesture recognition; Kinect project; Be the not to share the immense success of casual games with motion controller tool kit;motion control; human motio; skeletal relational tracking . There are only a few releases of core game (like Frist angles recognition Pearson Shouter games) with motion tracking control and these I. INTRODUCTION are not as successful as their casual competitors. This is in direct contrast to recent statistics illustrating traditional core As technology evolves, more interactive video game controllers are being developed in order to provide players with market games as being very successful. For example, Call of Duty: Black Ops has sold over 18 million copies [4]. Finally, a greater interaction with games and thus offering more enjoyment and the ability for a player to become more cursory analysis of the casual market’s released games, shows immersed in the game on many different levels, (immersed, that most users use the motion controller’s capabilities in a refers to how much the user is drawn into the game’s world and restricted, controlled fashion. This observation regarding the current state of the motion controllers’ market gives the feels as though he is actually in the gaming environment). impression of market stagnation and there is no obvious Motion control is one of the most challenging topics in today's games industry whereas the related controllers have taken the explanation as to why this is happening. Nevertheless, several theories try to explain this stagnation. programing experience to use the system to control The first of these is that it is caused by the limitations of the contemporary games with the Kinect controller. motion controllers [5]. However, this explanation does not seem to be the case as there are several impressive open source II. HUMAN BODY MOVEMENTS implementations that use motion controllers developed by This section looks at the method we suggest to track the programmers’ communities. Another reason considered more movements of a human for this project. First research about the valid is that the motion controller market is small, thus the movements of the human body is presented and then a designing of games aimed at this market need considerably description of the technique used to recognise the user’s more experiences and the available Software Development Kits different actions using the positioning of human body parts is provide developers with only basic functionalities. For provided. example, available SDKs provide developers with a A. Understanding Body Movements mechanism to track the user’s position in the environment but they do not provide them with a mechanism to identify user’s According to Bartlett, the human body movements are usually actions. Subsequently, the conclusion is that not enough described in three dimensional space using a system of planes research has been carried out regarding motion controllers and and axis [7]. The three planes of motion the human body passes which mechanisms are needed in order to allow developers to through are: use the motion tracking controllers to their full capabilities. The lack of research regarding the full capabilities of motion • The sagital plane controllers and ways to use them in order to identify players’ • actions were the motivation of the “Be The Controller: A The frontal plane Kinect Tool Kit for Video Game Control” project [6]. The • The transverse (horizontal) plane current paper summarizes the work presented in that project The sagital plane is the plane that lies vertically and bisects the A. What is the “Be the Controller tool kit” ? body into right and left parts. Sagital plane motions are the The primary goal of the Be the Controller tool kit is to help movements of human parts around the frontal axis. game designers easily identify and recognise a user’s pose and gestures using Microsoft’s Kinect motion controller, in order to The frontal plane also lies vertically however it bisects the then link these movements with actions in a game. The project body into the front and back parts. Frontal plane motions are is divided into two parts: the first part of the tool kit is a the movements of human parts around the sagital axe. windows application that allows a developer to create a set of gestures and poses and then export these gestures and pose lists The transverse is the plane that lies horizontally and bisects the to files. The second part of the tool kit is a software library (dll body into upper and lower parts. Transverse plane motions are file) that provides the developer with the infrastructure to load the movements of human parts around the vertical axe [7]. lists of poses and gestures to an application, which are then recognised and linked to the appropriate action in the application. The tool uses a skeletal relation angle recognition method rather than body parts Cartesian co-ordinates position recognition method, which provide a more easy and flexible way to create and recognise pose and gesture. By using this method the tool can recognise poses unaffected by the positioning of the body’s parts in the virtual world environment because it only compares the relationship of the parts, (if the Figure 1.