BRAINPORT VISION DEVICE By D.SWATHI, Final Year Computer Science And Engineerng, Sri Venkateshwara Engineering College, Contact [email protected] CONTENTS Statistics on the Blind: 37 million: People in the world are blind India (9 million), Africa (7 million) and China (6 million) Every 5 seconds: One person in our world goes blind 75 million: People will be blind by 2020 (if trends continue) Cybernetics Cybernetics is about having a goal and taking action to achieve that goal. "Cybernetics" comes from a Greek word meaning "the art of steering“. Ironically but logically, AI and cybernetics have each gone in and out in the search for machine intelligence So “I Can Read” can be termed as a “Cybernetics System For Disabled (blind)” DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING,SVEC,Suryapet 3 What is Brainport Vision Device? "BrainPort device" is a technology developed in US, which is making the world visible to the ones who lose their sight due to some accidental incidents. Neuroscientists at Wicab, Inc. has developed the BrainPort Vision Device that allows the blinds to “see” using their tougues. Craig Lundberg, 24, is the first British soldier to test the BrainPort system, which is billed as the next best thing to sight. The technology has made the dark-dependent world come alive and independent to Craig Lundberg who completely lost his sight after a grenade attack in Iraq, as he is now able to sense the visuals with his tongue. The soldier admits that his world has been transformed because of the technology. The device which sends visual input through tongue in much the same way that seeing individuals receive visual input through the eyes is called the “Brainport Vision Device”. BrainPort could provide vision-impaired people with limited forms of sight. Technically, this device is underlying a principle called “electrotactile stimulation for sensory substitution”. To produce tactile vision, BrainPort uses a camera to capture visual data. DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING,SVEC,Suryapet 4 ABSTRACT “BRAINPORT DEVICE” The device which sends visual input through tongue in much the same way that seeing individuals receive visual input through the eyes is called the “Brainport Vision Device”. BrainPort could provide vision-impaired people with limited forms of sight. To produce tactile vision, BrainPort uses a camera to capture visual data. The optical information -- light that would normally hit the retina -- that the camera picks up is in digital form, and it uses radio signals to send the ones and zeroes to the CPU for encoding. Each set of pixels in the camera's light sensor corresponds to an electrode in the array. The CPU runs a program that turns the camera's electrical information into a spatially encoded signal. The encoded signal represents differences in pixel data as differences in pulse characteristics such as frequency, amplitude and duration. Technically, this device is underlying a principle called “electrotactile stimulation for sensory substitution”, an area of study that involves using encoded electric current to represent sensory information and applying that current to the skin, which sends the information to the brain. The brain is capable of major reorganization of function at all ages, and for many years following brain damage. It is also capable of adapting to substitute sensory information following sensory loss (blindness; tactile loss in Leprosy; damaged vestibular system due to ototoxicity, or general balance deficit as result of stroke or brain trauma), providing a suitable human-machine interface is used (reviewed in Bach-y-Rita, 1995; in press). One such interface is the tongue BrainPort interface (Bach-y-Rita, et al 1998; Tyler, et al, 2003). The major objective of this study was to estimate feasibility and efficacy of an electro-tactile vestibular substitution system (ETVSS) in aiding recovery of posture control in patients with bilateral vestibular loss (BVL) during sitting and standing. Subjects used the BrainPort balance device for a period from 3 to 5 days. Subjects readily perceived both position and motion of a small 'target' stimulus on the tongue display, and DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING,SVEC,Suryapet 5 interpreted this information to make corrective postural adjustments, causing the target stimulus to become centered. With two twenty minute sessions a day significant functional improvement lasts the whole day. 1.INTRODUCTION A blind woman sits in a chair holding a video camera focused on a scientist sitting in front of her. She has a device in her mouth, touching her tongue, and there are wires running from that device to the video camera. The woman has been blind since birth and doesn't really know what a rubber ball looks like, but the scientist is holding one. And when he suddenly rolls it in her direction, she puts out a hand to stop it. The blind woman saw the ball through her tongue. Well, not exactly through her tongue, but the device in her mouth sent visual input through her tongue in much the same way that seeing individuals receive visual input through the eyes. In both cases, the initial sensory input mechanism -- the tongue or the eyes -- sends the visual data to the brain, where that data is processed and interpreted to form images. Braille is a typical example of sensory substitution -- in this case, you're using one sense, touch, to take in information normally intended for another sense, vision. Electrotactile stimulation is a higher-tech method of receiving somewhat similar (although more surprising) results, and it's based on the idea that the brain can interpret sensory information even if it's not provided via the natural channel. An electric lollipop that allows the blind to ‘see’ using their tongue has been developed by scientists. DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING,SVEC,Suryapet 6 Fig.1 Position of device The machine is called the Brain Port vision device and is manufactured by Wicab, a biomedical engineering company based in Middleton, Wis. It relies on sensory substitution, the process in which if one sense is damaged, the part of the brain that would normally control that sense can learn to perform another function. About two million optic nerves are required to transmit visual signals from the retina— the portion of the eye where light information is decoded or translated into nerve pulses—to the brain’s primary visual cortex. With Brain Port, the device being developed by neuroscientists at Middleton, Wisc.–based Wicab, Inc. (a company co-founded by the late Back-y-Rita), visual data are collected through a small digital video camera about 1.5 centimeters in diameter that sits in the center of a pair of sunglasses worn by the user. Bypassing the eyes, the data are transmitted to a handheld base unit, which is a little larger than a cell phone. This unit houses such features as zoom control, light settings and shock intensity levels as well as a central processing unit (CPU), which converts the digital signal into electrical pulses—replacing the function of the retina. “Part of the challenge of Brain Port is to train the brain to interpret the DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING,SVEC,Suryapet 7 information it receives through the stimulation device and use it like data from a natural sense. Research from prototype devices showed such training is possible, as patients with severe bilateral vestibular loss could, after time, maintain near-normal posture control while sitting and walking, even on uneven surfaces. ELECTROTACTILE STIMULATION FOR VISUAL SUBSTITUTION When a human looks at an object, the optical image entering the eyes does not go beyond the retina. Instead, it would turn into spatio-temporal nerve patterns of impulse along the optic nerve fibres. By analysing the impulse patterns, the brain recreates the images. Indeed, the channels such as eyes, ears and skin those carry sensory information to the brain are set up in a similar manner to perform similar activities. To substitute one sensory input channel for another, the big challenge to the scientists is how to correctly encode the nerve signals for the sensory event and send them to the brain through the alternate channel as the brain appears to the flexible when it comes to interpreting sensory input. The concepts at work behind electrotactile stimulation for sensory substitution are complex. The idea is to communicate non-tactile via electrical stimulation of the sense of touch. In practice, this typically means that "an array of electrodes receiving input from a non- tactile information source (a camera, for instance) applies small, controlled, painless currents (some subjects report it feeling something like soda bubbles) to the skin at precise locations according to an encoded pattern." For a blind person, it means the encoding of the electrical pattern essentially attempts to mimic the input that would normally be received by the non-functioning sense – vision. So patterns of light picked up by a camera to form an image are replacing the perception of the eyes and converted into electrical pulses that represent those patterns of light. In other words, when the encoded pulses are applied to the skin, the skin is actually receiving image data which would be then sent to the brain in the forms of impulse. Under normal circumstances, the parietal lobe in the brain receives touch information, while the occipital lobe receives vision information. When the nerve fibers forward the image-encoded touch signals to the parietal lobe, "the electric field thus generated in subcutaneous tissue directly excites the afferent nerve fibers responsible for touch sensations". Within the system, arrays of electrodes can be used to communicate non-touch information through pathways to the brain normally used for the touch related impulses.