The Past, Present and Future of Bionic Vision Elliot Whaley Computing Research Department of Computing Sciences Villanova University, Villanova, Pa, 19085 [email protected] November 19, 2008 Abstract The WHO, world health organization, estimated that in 2002 there were 161 million (about 2.6% of the world population) visually impaired people in the world. This staggering number leads many researchers and scientist to develop methods and devices to aid or cure blindness. This paper goes over the current and past methods used to help the blind as well as future research into methods that will not only help the blind but enable people with normal vision to see beyond 20/20. We also propose a method to obtain energy one such device more efficiently. 1. Introduction

This research surveys the development stages of bionic vision and proposes a new line of research that explores new energy solutions. In the beginning, attempts to improve human vision were made by connecting head pieces to the brain in order to cure blindness. Later research led to the development of more advanced glasses, called EyeTaps, that went beyond correcting vision. The latest stage of development involves contact lenses that can aid a human to see better than 20/20 and link their vision to computer devices. Bionic vision is an important area of research with a significant impact on blind people who, with the help of bionic vision, are capable of seeing again.

Existing techniques include implanting computer chips in the human brain and using bionic glasses. The military can use these lenses to improve the vision of pilots or soldiers, a most important aid in battle. Video games, movies, iphones and computers could be brought right to a persons’ field of vision. Special contacts could be used by doctors to zoom in on a patient in the operating table.

Researchers at the University of Washington have developed a prototype for a bionic contact lens [4]. The lenses have been tested on rabbits, however testing on humans has not yet been approved. This device includes light-emitting diodes, basic wiring for electronic circuits and a tiny antenna. The idea is to use the lenses as a flexible plastic platform that can display interfaces from other devices, such as a web browser or an ipod on the contact lens. Bionic contact lenses are in the trial stages because there are many health and technical issues that still need to be fixed. One of those issues is being able to adequately power the contact lenses with cumbersome batteries. I propose using photoelectric cells implanted in the lens to take power from solar energy to allow the contacts to function anywhere.

Section 2 will cover the history of bionic vision leading up to its state today. Then in section 3 will cover various ways of implementing bionic technology. Finally, Section 4 describes in detail my proposed work.

2. Development Stages of Bionic Vision

Bionic vision refers to the application of biological methods in nature combined with study of engineering and computer science, working together towards the common goal to correct vision and cure blindness.

Bionic vision began with the creation of technology to help cure blindness in patients that lost their vision. That goal to cure and help the blind evolved into other areas of development in bionic vision. Now computers and special glasses are used in procedures that help people see again as well as improve vision beyond 20/20.

Medical procedures to cure blindness were first done in 1978 by William Dobelle. He implanted electrodes into a man’s visual cortex. The procedure used by Dobelle successfully treated 16 patients in Portugal [1].

A newer method of helping the blind see is by using the SWAN system which stands for System for Wearable Audio Navigation [11]. This system was originally developed to help the visually impaired, firefighters and soldiers navigate their way in unknown territory, particularly when vision is obstructed or impaired.

Another approach to the vision problem was developed by Electrical Engineering doctoral student Ninad Thakoor who is currently working to provide the blind and visually impaired with a wearable, interactive system that will give them better information on their indoor and outdoor surroundings. They system is referred to as IntelligentEyes. They describe the system as it “consists of four major components: eyeglass frame-mounted cameras providing real-time pattern recognition and distance information; a wireless, ultra-wideband transceiver sending video and voice signals to and from a reconfigurable processor; a battery-powered image processor analyzing information from the cameras; and voice recognition and generation software providing ease-of-use communications from and to the user. The prototype contains everything but the voice capabilities for the system” [12].

A more advanced solution to helping the blind and going beyond that is the EyeTap system. The EyeTap, a lifelong cyborglog, is a device originally designed to continuously record a persons’ life. A person would wear glasses that would do the recording and could be played back at anytime. Future developments of this device include helping correct vision. This device started as a big clumpy device over the eyes however, in the past 30 years its development has led to smaller more visually attractive device resembling everyday eyeglasses (FIGURE 1b). As a form of electronic visual aid, surveillance system, and wearable camera phone, this device functions as natural extensions of the mind and body.

A research team at University of Washington is working on a pair of contact lenses, the latest step in bionic vision (FIGURE 3). These contact lenses combined with a computer chip have the capability to connect to a wireless device and provide a visual image of the data that would normal display on the device into your field of vision. High – resolution images will be displayed before the user such as video games, texts, and music.

3. Bionic Vision Technology and Methods

The technology ranges from an entire bionic eye to just contact lenses with computer chips. Depending on the need different technologies are created and used in bionic vision. This section will go through not only what these technologies are but also how they work.

3.1 Technology to help the blind see There was a procedure created to give sight to people who lost it by using glasses connected to the human brain and a computer. This procedure uses a reverse Brain Computer Interface (BCI). BCI measures minute differences in the voltage between neutrons. The signal is then amplified and filtered. Then the signal interpreted by a computer program or device (FIGURE 4). The device functions as a prosthetic cornea, by directing light to the interior of the eye. An eye piece is placed on a pair of sunglasses and connected to an electrode inside her skull. Then a small camera on the eye piece sends video signals to a computer where it interprets the information from the signal and sends it through two cables plugged into her skull. That information informs the electrodes to now stimulate the brain, which creates a dot matrix image. Until her brain adapts to the technology, she can only see outlines.

FIGURE 4[1]

A device that does not involve surgery but main goal is still to help the blind is the SWAN system. The SWAN (System for Wearable Audio Navigation) system utilizes a small laptop, a proprietary tracking chip, GPS, a digital compass, a head tracker, four cameras and light sensor and bone-conduction headphones. Bone phones send auditory signals in the form of vibrations through the skull without plugging the user’s ears. These tools provide audio guidance to a person in their surrounding with or without vision. The laptop is worn as a backpack; the sensors and tracking chip are worn on the head which send data to the SWAN application on the laptop. The computer determines the user’s location and the direction they are going and maps out a travel route, then sends 3-D audio signals to the headphones to guide the user. SWAN uses two different types of auditory displays. It can direct a user toward a sound for example, “walk 100 yards and turn left.” The system can also point out items of interest such as doors, benches and steps.

3.2 Technology of the EyeTap

The EyeTap is an experience capturing system that functions as both a camera and display. The EyeTap started as a wearable telephone/computer satisfying the two most used senses, sound and sight. Then a Cathode Ray Tube attached to a helmet presenting both text and graphics. Once a made small enough to fit on glasses it was implemented in the system as in figure1a (figure 1a). Rays of light which normally enter the eye are now reflected by the diverter, which is a double-sided mirror. The rays of light are collected by a sensor such as a CCD (charge coupled device) camera. The camera data is processed and the aremac, a device that generates a synthetic ray of light, redisplays the image as rays of light. The rays are again reflected off the diverter and the user perceives the virtual light. The virtual light can be either an image or a computer mediated version of the real world. The EyeTap differs from device such as head-mounted-displays (HMDs). HMD normally are used to provide or add information into what a user perceives. However, “In EyeTap devices, the diverter places the centre of projection of the camera at the centre of projection of the lens of an eye of the wearer. When no computer mediation is used, EyeTap video can be displayed to the user in such a way that the user perceives what he/she would otherwise have in the absence of the device. EyeTap mediates a portion of the user's vision, in such a way that it is integrated with the un-mediated portion of the user's field of view, without any mismatch between the mediated area and the real world.”[13] Figure 1a[13]

Figure 1b[2]

3.3 Scene specific imaging method for improving bionic vision

Bionic vision can be adjusted depending on the scene type. Different scenes and visual aid devices can be suited to increase their effectiveness. For a scene to be translated into an algorithm it first must be categorized. There are some automatic scene categorization models, for instance Chernyak and Stark created one using Bayes’ theorem [5]. Segment features such as aspect ratio and average color are obtained from a test image. Once the scene has been accurately determined by the user the authors want to apply context dependent importance weighting to the image. Image mapping, finding the most important object in an image, is used through several algorithms like lossless compression, military target detection and advertising. The authors performed an experiment where they proposed that scene weighted processing can improve perception of low quality images. Figure 2 was presented to 20 normally sighted people. Results show that scene-dependent importance mapping is a good tool to use in automatic optimization of low quality images. The authors work applies to importance map methods which bring together aspects of image that are known to influence the attention of the human viewer. Such features include shape, size and contrast.

Figure 2[5]

3.4 Technology of the Contact Lenses

A prototype has been created by Babax Parviz at the University of Washington. The lenses combined with a computer chip have the capability to connect to a wireless device and provide a visual image of the data that would normal display on the device into your field of vision. High –resolution images will be displayed before the user such as video games, texts, and music. Researchers at the University of Washington felt they could make a really tiny functional device that could be embedded into a contact lens. A metal circuit and light –emitting diodes are placed in a polymer-based lens where the lens is biologically compatible with the eye. Ultra-thin antennas, a few nanometers thick, are used to send information wirelessly to devices.

Figure 3[4]

Prototype of the contact lens

Fused with computer chip

3.41 Adaptive Optics

Adaptive optics is a technique originally designed to sharpen images for military surveillance devices and astronomical telescopes. In relation to the human eye, adaptive optics allows people to see at high resolution, but it also works in reverse allowing researches to capture detailed images of the eye’s retina. David R. Williams of the University of Rochester has developed an approach to obtain this vision [3]. The goal of adaptive optics is to improve human vision, but presently the focus is on preventing vision loss and correcting eyesight problems. This is done by collecting light waves with a deformable mirror which can be shaped to compensate for distortions in an image; then the mirror is coupled with a high resolution camera that takes images of the retina to correct the distortions produced by the imperfections in the eye. This method was used to help cure bad vision, however when people with normal vision used the adaptive optics they experienced up to six fold improvement in sight.

4. Proposed Work

My proposal involves the use of photoelectric cells implanted in the contact lens developed at the University of Washington [4], to take power from solar energy and an antenna to allow the contacts to function anywhere. An important feature of these contact lenses is that they are wireless and hardware less device. It does not need to be physically connected to anything to work. In order to keep this device stand alone it needs a way to power itself without an accessory that needs to be carried around to do it. Taking solar energy to power these lenses solves this problem. The lenses would need to have photoelectric cells, which convert solar energy to electric. Just one square meter of a solar panel has the ability to power a 100W light bulb. A polar panel will need to be scaled down to fit inside the contacts lenses with material that is compatible with the eye. Some advantages of using solar energy are that solar energy is free, needs no fuel and produces no waste or pollution. Some disadvantages are it cost a lot of money to make and will not work well at night. However, once solar energy is no longer available researches at Washington University developed a way to extract minimal power from the antenna which could compensate for no light [4]. The antenna does this by collecting radio frequency waves and turning them into useful energy. The drawback of this method is if the user is in a place with no frequencies to pick up like the Amazon. So a system combined of both methods solar and antenna the contacts could be powered at all times.

I plan to hire experts in the fields of solar power, biology and computer science to assist with the project. With the proper research and development team this proposal can be done and will help advance the device already in progress at University of Washington.

Timeline: Two months

Week 1&2: Research areas that I am unfamiliar with.

Week 3: Develop materials that can convert solar energy and are compatible with the human eye.

Week 4 &5: Use those materials to create a tiny solar panel that will fit in the lens

Week 6: Implement an algorithm that will let the antenna and solar panel work together to power the lenses in any situation.

Week 7: Testing of device on animals or prosthetic eye.

Week 8: Final release. 5. Conclusion

We conclude that even though blindness is a large problem in the world, there have been major advances that may one day lead to everyone having beyond perfect vision. From the medical procedure involving a lot of machinery and time bionic vision has come very far. A person can now record their entire life via the EyeTap or possibly connect their contact lenses with a wireless device. References

 [1] Jennifer Anderson, “Bionic Vision: Rare Operation Brings Sight to Blind Woman” Ergonomics Today, 2005.

 [2] Steve Mann, “Continuous lifelong capture of personal experience with EyeTap,” Proceedings of the 1st ACM workshop on Continuous archival and retrieval of personal experiences, p.1-21, October 15-15, 2004, New York, New York, USA

 [3] Corinna Wu,” Supernormal Vision, A focus on adaptive optics improves images of the eye and boosts vision,” Science News, 1997, pp.1-3.

 [4] Alexi Mostrous, “And next- the contact lens that lets e-mail really get in your face,” TimesOnline, 2008.

 [5] Boyle, J.; Maeder, A.; Boles, W., "Scene specific imaging for bionic vision implants," Image and Signal Processing and Analysis, 2003. ISPA 2003. Proceedings of the 3rd International Symposium, pp. 423-427 Vol.1, 18-20 Sept. 2003

 [6] Gregg J Suaning, Nigel H Lovell, Klaus Schindhelm, Minas T Coroneo,” The bionic eye (electronic visual prosthesis): A review, ” Clinical and Experimental Ophthalmology, 1998

 [7] Brendan Z Allison, Elizabeth Winter Wolpaw, Jonathan R Wolpaw. (2007) Brain–computer interface systems: progress and prospects. Expert Review of Medical Devices 4:4, 463-474 online publication date: 1-Jul-2007.

 [8] Lotfi BMerabet , Joseph FRizzo , AlvaroPascual-Leone , EduardoFernandez. (2007) ‘Who is the ideal candidate?’: decisions and issues relating to visual neuroprosthesis development, patient testing and neuroplasticity. Journal of Neural Engineering 4:1, S130-S135 Online publication date: 1-Apr-2007.

 [9] Normann R, Maynard E, Rousche P, Warren D, A neural interface for a conical vision prosthesis, Vision Research 39(15), pp. 2577-2587, 1999

 [10] 121 Suaning G, Lovell N, CMOS Neurostimulation System with 100 Electrodes and Radio Frequency Telemetry, Inaugural Conference of the IEEE EMBS (Vic), Melbourne, pp.37-40, Feb 1999  [11] Walker, B. N., & Lindsay, J. (2006). Navigation performance with a virtual auditory display: Effects of beacon sound, capture radius, and practice. Human Factors, 48(2), 265-278.

 [12] Ninad Thakoor, “Aiming to help the Blind See,” University of Texas Arlington College of Engineering.

 [13] no author, “ EyeTap: The eye itself as display and camera ,”EyeTap.