A Novel Gesture Driven Fuzzy Interface System for Car Racing Game

A Novel Gesture Driven Fuzzy Interface System For Car Racing Game Chiranjib Saha∗, Debdipta Goswami∗, Sriparna Saha∗, Amit Konar∗, Anna Lekovay and Atulya K. Nagarz ∗Department of of Electronics and Telecommunication Engineering, Jadavpur University, Kolkata-700050, India Email: [email protected], [email protected], [email protected] and [email protected] yInstitute of System Engineering and Robotics, Bulgarian Academy of Sciences, Sofia, Bulgaria Email: [email protected] zDepartment of Mathematics and Computer Science, Liverpool Hope University, United Kingdom Email: [email protected] Abstract—The recently developed Kinect sensor has opened a HCI devices, have experienced a paradigm shift in developing new horizon to Human-Computer Interface (HCI) and its native advanced control by adding more degrees of freedom into the connection with Microsoft’s product line of Xbox 360 and Xbox gaming environment. Considering the driving based games, One video game consoles makes completely hands-free control in next generation of gaming. Games that requires a lot of degree gesture tracking can lead to a new level of experience beyond of freedoms, especially the driving control of a car in racing classical joystick control, enabling the player to move his/her games is best suitable to be driven by gestures, as the use of limbs seemingly controlling an imaginary steering wheel or simple buttons does not scale to the increased number of assistive, joystick to move through the game. Also the tracking of comfort, and infotainment functions. In this paper, we propose feet movements would yield speed controls. Keeping in mind a Mamdani type-I fuzzy inference system based data processing module which effectively takes into account the dependence of the simulation of virtual reality by head-mounted holographic actual steering angle with the distance of two palm positions displays of by Oculus Rift or Microsoft HoloLens, gesture and angle generated with respect to the sagittal plane. The FIS controlled steering seems an indispensible part of driving, output variable controls the duration of a virtual “key-pressed” eleminating joystick or steering wheel and pedals. However the event which mocks the users pressing of actual keys assigned to challenge lies in the fact that gesture-inputs would be ridden control car direction in the original game. The acceleration and brake(deceleration) of the vehicle is controlled using the relative with more inaccuracy, redundancy and noise than that occur in displacement of left and right feet. The proposed experimental a classical contact-based input device, and must be processed setup, interfacing Kinect and a desktop based racing game, has with intelligent data-processing and recognition algorithms. shown that the virtual driving environment can be easily applied The gesture recognition algorithms and applications have to any games belonging to this particular genre. been taken the centre field of HCI in the recent decade. The development of an SDK by Microsoft for Kinect sensors I. INTRODUCTION has made it an unparalleled motion sensing device used in The rapid growth of computer applications for normal users research and development. Bonanni [2] developed a gesture in the second half of 20th century called for a robust and driven interaction with a mobile robot using Kinect sensors reliable system to interact with the machine, thus initiating a where he employed hidden Markov model [3]. Using of Kinect new subject area: Human Computer Interaction, also known as sensor in Hand Gesture recognition is carried out by Li [4]. HCI. The evolution in input devices to achieve more intricate Oszust et al. [5] have proposed a method for recognition man-machine interaction with physical feedback has been of signed expressions observed by the Kinect sensor. Here made possible by the recent advancement in sensors, display skeletons of human body with shape and position of hands and rendering technology. The principle used to recognize the are taken into account for polish sign language recognition. gestures may be based on the mimicry of human vision system: The recognition is implemented using k-nearest neighbour the reception of photons on the retina and the processing of (kNN) classifier. Burba et al [6] have used Microsoft Kinect to the signal by visual cortex [1]. A line of motion sensing monitor respiratory rate is estimated by measuring the visual devices have been developed by different corporations that expansion and contraction of the users chest cavity. This work can track the human motion and basic gestures using infrared also lightens the area of measuring fidgeting behavior. This is sensing. The most popular motion sensing devices are Mi- also done by Kinect sensor focusing on vertical oscillations of crosoft Kinect, Nintendo Wii, Sony PlayStation Eye and Sega the users´ knees. Dreameye. The Microsoft Kinect gives a skeleton of a human The feedback-actuator system for steering wheel driving has standing in its field of view with the coordinate of 20 joints that been improved and modified with different interfaces in the can be used for gesture recognition algorithms. Equipped with past years to provide more stability and robustness. Zhang et such revolutionary improvement of input devices, computer al [7] experimented on driving control of electric vehicles and games, which have been an integral part of computer based proposed a new H1 robust control stratgy. Wada and Kimura entertainment and solely depends of the features offered by [8] analysed the stability of a joystick interface for driving where a steering wheel and gas/brake pedals are actuated more precise and comfortable control of the car movement by electric motors which rotations are controlled by micro- and the incorporation of acceleration and brake under gesture- computers based on 2DOF joystick commands. control. Inspired from the driving system presented in the Recent works have been focussing on integrating gesture Forza Motorsport 4 [11] where a Kinect driven mode is in racing video games, specially in the form of touch-screen also included, we have chosen the normal steering wheel gesture or dummy steering wheel. Pfleging et al [9] has com- control for our model. However, in the said game the player’s bined both the speech and gesture on a multi-touch steering gesture is constrained by the fact that he has to keep the wheel. Using gestures for manipulation, they provided a fine- hands in right angle with the body-axis for movement, which grained control with immediate feedback. Siversten [10] used may be tiresome as referred by the critics [11]. Moreover, a steering wheel input device with pointer event detection the acceleration and brake control is automatic, cutting short and angle compensation technology that can serve as input the degrees of freedom. We have eliminated that need by signals for vehicular control. However, due to the extremely considering only the line joining both hand joint coordinates high degree of freedom required by a driving control system, so that the hands can be placed comfortable at any angle with free-air gesture is best suited for it. folded elbow-joint. Another major improvement is that in the The proposed model of driving control for a racing game proposed model acceleration and brake can also be controlled mimics the natural driving of an auto-transmission car with using gesture which were not available for Forza Motorsport two hands kept in such a position that they seem holding 4. The most important of all is that in our model, even the an imaginary steering wheel. The player sits in a chair and wild-turning of the imaginary steering wheel will not cause moves his/her right and left feet forward to actuate acceleration disastrous effect as it does in Forza Motorsport 4 since the and brake. The steering angle is determined by the angle steering angle is processed by a Fuzzy Inference System (FIS) between axis created by joining both hand-coordinates and the along with the steering diameter. Another improvement is the transverse plane. As in the free-air gesture, the player may get additional feature of speed control with feet movements which the perception of linear displacement as the governing quantity simulates real-time driving experience. and may turn the imaginary wheel in a greater angle when The rest of the paper is organised as follows. Section II steering radius is small and vice-versa, the latter quantity is gives a brief overview of Kinect for Xbox 360 sensor and also taken into account while generating the control signal. the description of its skeleton tracking activity. Section III The control signal is generated as the keyboard interrupts gives a vivid analysis of proposed driving model and the for “keydown” and “keyup” events with the duration between insight of the Fuzzy Inference System. We have described them as the “Keypress duration”. The duration is controlled the experimental setup in Section IV. The relevant simulation by the steering angle and radius using a type-I Mamdani results and discussion on them constitutes Section V. Section Fuzzy Inference System to make it precise and robust. The VI discusses the future work and concludes the paper. acceleration and brake is controlled by the detection of relative feet position from the skeleton tracked by Kinect. II. KINECT SENSOR AND ITS SPECIFICATIONS J Head Kinect is a line of motion sensing input devices originally 4 developed for Xbox and Xbox 360 video game consoles J Shoulder Center 3 and later extended for non-gaming uses in Windows-PC. The J Shoulder right J Shoulder 9 5 Kinect sensor for Xbox 360 [12]- [14] consists of an RGB left J camera, an IR camera, an IR projector and a microphone array. 6 J spine J Elbow right Elbow left 2 10 It basically looks like a webcam (8-bit VGA resolution of J hip left J hip center 13 1 J Wrist right 640×480 pixels with a Bayer color filter).

Load more