Wireless Assistive Head Controlled Mouse with Eye-Blink Detection for Enhanced Actions
Total Page:16
File Type:pdf, Size:1020Kb
Turkish Journal of Physiotherapy and Rehabilitation; 32(2) ISSN 2651-4451 | e-ISSN 2651-446X WIRELESS ASSISTIVE HEAD CONTROLLED MOUSE WITH EYE-BLINK DETECTION FOR ENHANCED ACTIONS ALAN FRANCIS CHEERAMVELIL1, ARJUN ALOSHIOUS2, N. DHARANIDEVI3, S. VARUN4, ABEY ABRAHAM5 5Assistant Professor, Department of Information Technology, Rajagiri School of Engineering & Technology, Kerala, India- 682039 1,2,3,4Student, Department of Information Technology, Rajagiri School of Engineering & Technology, Kerala, India-682039 [email protected], [email protected], [email protected], [email protected], [email protected] ABSTRACT The Wireless Assistive Head Controlled Mouse with Eye-Blink Detection for enhanced actions is an assistive Human Interface Device aimed at quadriplegic people. The current pandemic has made the world more reliant on digital means of communications than ever before and people with motor disability have no means to access these resources. A majority of them have good head movement but no torso movement. The proposed device uses a Gyroscope sensor to accurately map the user’s head movements to the corresponding mouse coordinates. The device works with Bluetooth Low Energy technology enabling the user to control the digital devices at a comfortable range without the hassle of wires. The plug-N-play feature allows the use of the device without additional drivers. For more sophisticated usage scenarios, the user can choose between the various traditional mouse operations using a desktop software with the help of the eye-blink detection using image processing. Keywords— Assistive technologies; Persons with disability; Gyroscope sensor; Human Computer Interaction; Eye-blink Detection; I. INTRODUCTION According to the Census conducted in 2011, persons with disability (PwD) constitute 2.21% of the total population in India. Assistive technologies make their life easier by creating and redesigning devices which are easily accessible by that community. People who are quadriplegic have limited options for modern communication techniques through computers and mobiles due to lack of input methods, so an alternative mode of giving inputs is required. In the present scenario of COVID-19 pandemic the world is reliant on technology more than ever. We now use technology for work, keeping in touch with our loved ones etc. In this situation, one should not be left behind because of their disability and a method should be devised to help them lead a normal life during this tough time. People who are quadriplegic have limited options for modern communication techniques through computers and mobiles due to lack of input methods, so an alternative mode of giving inputs is required. The Wireless Assistive Head controlled Mouse with Eye-Blink Detection for Enhanced Actions makes use of a person’s head movements to move the cursor on a computer screen or a mobile device accordingly. It performs a Click operation when the device detects no head movement. This helps in making communication easier for PwDs. It also helps them in implementing various tasks in their everyday lives with less effort and more efficiency. The Wireless Assistive Head Controlled Mouse with Eye-blink Detection for Enhanced Actions helps in empowering individuals with disabilities by allowing those with motor disabilities to interact with computers. The device should be able to emulate all the basic functionalities of an input device so that a person with disability can get the maximum out of his digital devices. It should enable a disabled user to easily access and switch between the various functionalities of a mouse with the help of a reliable and user-friendly Graphical User www.turkjphysiotherrehabil.org 2378 Turkish Journal of Physiotherapy and Rehabilitation; 32(2) ISSN 2651-4451 | e-ISSN 2651-446X Interface with enough accessibility options to assist them. Since it is not possible to emulate all the functionalities of a normal mouse with this device, given the lack of inputs, we give the user the option to use the device in various mouse modes including left-click, right-click, drag, scroll, double-click, no click & hover and give the person the option to choose between these modes at any point of time. The device translates the head movements of the user to the pointer coordinates on the screen. It offers the feature of modular attachment to existing devices such as head phones. It can be customized and developed according to the requirements of the individual, as each disabled individual has different need depending on his disability. One of the major reasons for a disabled individual to withdraw from an assistive device is its high cost. So, the product must be cost effective. The device provides an excellent user experience and opens up a plethora of opportunities in the digital world regardless of their physical impairments. There are a lot of assistive technologies available in the market that allow quadriplegic users to operate computer devices. Some of the most popular devices include Gaze Trackers [3] , Video-Based head tracking [4] , Optical Sensor-Based head tracking [5] and Gyroscope-Based head tracking [6] . Gaze trackers track the user’s eye movement by using an IR light source to illuminate the eye causing highly visible reflections, and a camera to capture an image of the eye showing these reflections. The image captured by the camera is then used to identify the reflection of the light source on the cornea (glint) and in the pupil. We can then calculate a vector formed by the angle between the cornea and pupil reflections—the direction of this vector, combined with other geometrical features of the reflections, is then used to calculate the gaze direction. These are one of the most precise technologies for pointer movement in the market but they are extremely sensitive to external lighting noise and reflections on the eye from other IR sources. They also come in the price range of $300 - $10.000 which is not affordable by most individuals. Video-Based head tracking uses the live camera feed from a generic webcam and applies image processing techniques on the video to track the head movement of the user and convert them to the corresponding mouse coordinates on the screen. This is the most popular solution in the market owing to the lack of external hardware required and the cost effectiveness ranging from free to use product to software costing only up to a few hundred dollars. The major drawback of this technology is its poor precision and its heavy dependence on external lighting conditions and variable lighting can make the video unintelligible rendering image processing almost impossible. The optical sensor-based head tracking has an optical reflector placed on the user’s face and an IR emitter sensor couple detects the motion of the reflector thus identifying the user’s head movement and converting it into the corresponding mouse coordinates. The tracking precision is very high as compared to video-based head tracking and the external lighting conditions do not affect the precision too much as there is no need to identify patterns and it has its own source of illumination. The product is also much more affordable as compared to the gaze trackers and costs only a few hundred dollars. The Gyroscope based head- tracking uses a gyroscope sensor in order to detect the user’s head movement and convert it to the corresponding mouse coordinates. It is very accurate and it is completely independent of external lighting as gyroscope is not a sensor that depends on external illumination. The technology also comes in an affordable cost range of about a few hundred dollars. II. BACKGROUND The different modules included in the working of the proposed device are: 1. Bluetooth Module 2. Microcontroller Module 3. Gyroscope Module 4. Eye Blink Detection Module 1. Bluetooth Module The Bluetooth module is used to establish a connection between a user system and the device. The Bluetooth module can be provided as an additional hardware on the device itself or a Bluetooth enabled microcontroller module can be used to establish the connection. Once a connection has been established the user can start using the device. This is made possible with the introduction of Bluetooth Low Energy (BLE) Technology which has an in-built Human Interface Device (HID) mode which can run on a host system as a regular HID device without the need for additional drivers. The BLE technology also reduces the power consumption dramatically which is a www.turkjphysiotherrehabil.org 2379 Turkish Journal of Physiotherapy and Rehabilitation; 32(2) ISSN 2651-4451 | e-ISSN 2651-446X requirement for any battery powered device. These allow the user to simply connect the device using Bluetooth and start using it making it a Plug-N-Play device. 2. Microcontroller Module Once the device is connected and ready to use, the user can start moving his head in order to work on his computer. These head movements are converted into mouse pointer movements by the combined efforts of the gyroscope module and microcontroller module. The proposed device uses an ESP32 series microcontroller module [7][9][10][11]. It is a dual-core system with two Harvard Architecture Xtensa LX6 CPUs. All embedded memory, external memory and peripherals are located on the data bus and/or the instruction bus of these CPUs. With some minor exceptions (see below), the address mapping of two CPUs is symmetric, meaning that they use the same addresses to access the same memory. Multiple peripherals in the system can access embedded memory via DMA. The two CPUs are named “PRO_CPU” and “APP_CPU” (for “protocol” and “application”), however, for most purposes the two CPUs are interchangeable. Inside the ESP32, it has a 1296 KB embedded memory address space, 19704 KB external memory address space, 512 KB peripheral address space and a 328 KB DMA address space.