Visual Prosthetics Wwwwwwwwwwwww Gislin Dagnelie Editor

Visual Prosthetics wwwwwwwwwwwww Gislin Dagnelie Editor

Visual Prosthetics

Physiology, Bioengineering, Rehabilitation Editor Gislin Dagnelie Lions Vision Research & Rehabilitation Center Johns Hopkins University School of Medicine 550 N. Broadway, 6th floor Baltimore, MD 21205-2020 USA [email protected]

ISBN 978-1-4419-0753-0 e-ISBN 978-1-4419-0754-7 DOI 10.1007/978-1-4419-0754-7 Springer New York Dordrecht Heidelberg London

Library of Congress Control Number: 2011921400

© Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com) Preface

Visual Prosthetics as a Multidisciplinary Challenge

This is a book about the quest to realize a dream: the dream of restoring sight to the blind. A dream that may have been with humanity much longer than the idea that disabilities can be treated through technology – which itself is probably a very old idea. Long ago, when blindness was still considered a curse from the gods, some- one must have had the inspiration of building a wooden leg to replace one that had been crushed in a natural calamity or in battle. Many centuries lie between the concept of creating such a crude prosthesis to treat disability and today’s endeavors to replace increasingly complex bodily functions, but the wish to restore useful function and the researchers’ creative spirit remain the same. Around 1980, the developers of the cochlear implant were performing the first modest clinical trials of a technology to make the deaf hear again, or even hear for the first time. From those humble first attempts sprang a field that has become a model for modern neuroprosthetics, with tens of thousands of cochlear implants used successfully around the world. The development of the cochlear prosthesis illustrates the importance of bringing together professionals from a wide range of disciplines, from basic biology and engineering to rehabilitation, to create a func- tional substitute for a human sensory organ. In 1995, the editor of IEEE Spectrum magazine determined that artificial vision might be the next technological frontier, and that it should be the topic of a special issue. He invited a half dozen vision researchers to contribute articles about their expectations in two areas, visual prosthetics and machine vision, combined under the title “Towards an Artificial Eye.” He instructed the authors not to feel con- strained by existing technology, but rather to envision the steps that would be required to replace natural vision. Most of the ideas presented in that May 1996 issue have not yet been realized, especially those for prosthetic vision. Machine vision has made larger strides, which just goes to show that biology is more stubborn than technology - but also more resourceful, as machine vision researchers realize on a daily basis: Segmenting and recognition tasks that our visual system performs effortlessly can pose formidable problems for a computer-based image analysis system. Yet, encouragingly, some visual prosthesis designs predicted in that 1996 magazine are now being tested in clinical trials.

v vi Preface

This is an exciting time for the field of visual prosthetics. Obviously, it is exciting for the hope it brings that vision can be restored. It is exciting for its challenge to researchers, technicians, clinicians, rehabilitation workers, and people in many other fields to commit their talents to the solution of a problem with so many dimensions. It is exciting for the experimenters when, seemingly against all odds, a blind study participant with a few dozen electrodes on the retina recognizes an object or letter “E” and finds a path around traffic cones in the lab without a cane or guidance. It is exciting for the participants in these trials, who feel they can play an active role in realizing the dream. It is exciting for their loved ones and the public at large, for whom the developments can’t come quickly enough. And it is, unfortunately, too exciting for some media types who can’t stop themselves from running ahead of the facts. This is also a field of setbacks, as when the new electrode coating that was sup- posed to withstand conditions inside the body for 20 years starts peeling off during its initial high-temperature soak test; of unpleasant surprises, as when the simple idea of putting together many small phosphenes to create an image runs up against the reality that phosphenes overlap and blur the image beyond recognition; and of patience put to the test, as when investors and the public do not get the miracle cure they may have been expecting. But mostly this is a field of great dedication by hundreds of researchers in doz- ens of labs in countries on four continents; of amazing tenacity by study participants learning to make sense of a way of seeing that is so different from the vision they lost; and of true collegial spirit among all who share the dream, despite the realities of commercial interest. This collegial spirit was evident even in the days of the IEEE Spectrum issue: Throughout the 1990s, the National Institute of Neurological Disorders and Stroke sponsored an annual neural prosthesis workshop that was attended by all researchers competing for the scarce development funds then available for neuroprosthetics. Although the competition could be fierce, the annual workshop attendees formed a community that collectively solved stubborn problems of interfacing technology and biology, and attracted many new and tal- ented researchers to the field. Looking back, I feel that these workshops had a limi- tation: They were, by the nature of the research contracts given out, strongly geared towards technology, and less towards integration with physiology or rehabilitation. This was inherent in NINDS’s mission to foster development of devices with broad application, but non-engineers were less likely to attend these highly technical gatherings. In the year 2000, Dr. Philip Hesburg at the Detroit Institute of Ophthalmology had the inspiration to foster a new collaboration among visual prosthesis researchers, clinicians, and workers in low vision rehabilitation by creating and sponsoring a series of biennial meetings that he calls “The Eye and the Chip.” Successful beyond Dr. Hesburg’s expectations, these meetings have become the premier gath- ering place for researchers from all parts of the world and from very different backgrounds. Invited speakers are scientists who are advancing the field, yet the scale and atmosphere allow all researchers, patients, and the media to come and be updated about progress over the past 2 years. More perhaps than at other scientific Preface vii meetings, where investigators tend to gather within disciplines, participants at The Eye and the Chip are challenged to be open-minded, learn about and critique each other’s work, and return home with fresh ideas for interdisciplinary approaches. The interdisciplinary character of this book reflects that same spirit. This book is also a reality check, an assessment of where we stand in 2010, almost 50 years after G.S. Brindley put the first revolutionary electrode assemblies under a blind patient’s skull, yet in a field that is still very young. And this book is an introduction for people outside the field who may want to join the quest, or just be better informed. The book is unusual in being aimed at a readership as diverse as the disciplines contributing to the field: basic scientists, tissue and biomedical engineers, clinical researchers, and rehabilitation specialists. Most of all, this book is a tribute to the visionaries, the inventors, the creators of devices, the biomedical engineers, the surgeons and medical staff, the research psychophysicists, the occupational therapists, and the patient pioneers and their loved ones. In the chapters that follow, a few dozen workers in the field present their work and that of many colleagues. Each of their accounts conveys a passion for this multidisciplinary journey of discovery, a sense of urgency, a precise and meticulous effort to get it right and to learn – from the damaged visual system and from study participants – how to further improve the technology. If the reader comes away from this book with a sense of the breadth of the enter- prise, the hope for solutions that will truly help blind individuals, and the excite- ment shared by so many working in the field, then it has accomplished much of what the authors set out to do. If it allows practitioners in one discipline participat- ing in this development to get a better appreciation for what their colleagues in other disciplines are trying to accomplish, then the authors have clearly hit the right notes. And if it inspires enthusiastic young minds to join the quest, and to help turn the visual prosthesis into the next cochlear implant, then we will truly have succeeded. Baltimore, MD Gislin Dagnelie September 2010 wwwwwwwwwwwww Acknowledgments

This book reflects a group effort. Each contributor embraced the concept of a book that would span many disciplines, and reaching a consensus about what should be covered, and by which authors, proved surprisingly easy. I thank the authors for making time in their busy schedules to share their knowledge and create this overview. I appreciate the encouragement of my colleagues at the Lions Vision Research and Rehabilitation Center of the Wilmer Eye Institute at Johns Hopkins, who encouraged me to take on the challenge of creating this book and who helped in large and small ways to bring it to completion. I am deeply grateful to Maryam Khan, M.D., who helped me turn a stack of diverse manuscripts into polished chapters that not only met the publisher’s technical standards but are a pleasure to read. But most of all, I am grateful to the study participants who give meaning to our research, and who are an ongoing source of inspiration. This book is dedicated to them. Baltimore, MD Gislin Dagnelie

ix wwwwwwwwwwwww Contents

Preface...... v

Part I Structure and Function of the Visual System

1 The Human Visual System: An Engineering Perspective...... 3 Gislin Dagnelie

2 Vision’s First Steps: Anatomy, Physiology, and Perception in the Retina, Lateral Geniculate Nucleus, and Early Visual Cortical Areas...... 23 Xoana G. Troncoso, Stephen L. Macknik, and Susana Martinez-Conde

3 Retinal Remodeling and Visual Prosthetics...... 59 Bryan W. Jones, Robert E. Marc, and Carl B. Watt

4 Cortical Plasticity and Reorganization in Severe Vision Loss...... 77 Eduardo Fernández and Lotfi B. Merabet

5 Visual Perceptual Effects of Long-Standing Vision Loss...... 93 Ava K. Bittner and Janet S. Sunness

Part II Neural Stimulation of the Visual System

6 Structures, Materials, and Processes at the Electrode-to-Tissue Interface...... 113 Aditi Ray and James D. Weiland

7 Delivery of Information and Power to the Implant, Integration of the Electrode Array with the Retina, and Safety of Chronic Stimulation...... 137 James Loudin, Alexander Butterwick, Philip Huie, and Daniel Palanker

xi xii Contents

8 Retinal Cell Excitation Modeling...... 159 Carlos J. Cela and Gianluca Lazzi

9 Neurotransmitter Stimulation for Retinal Prosthesis: The Artificial Synapse Chip...... 173 Raymond Iezzi and Paul G. Finlayson

10 Synthetic Chromophores and Neural Stimulation of the Visual System...... 193 Elias Greenbaum and Barbara R. Evans

11 Biophysics/ Engineering of Cortical Electrodes...... 207 Philip R. Troyk

Part III Prosthetic Visual Function: Acute and Chronic

12 The Response of Retinal Neurons to Electrical Stimulation: A Summary of In Vitro and In Vivo Animal Studies...... 229 Shelley I. Fried and Ralph J. Jensen

13 Findings from Acute Retinal Stimulation in Blind Patients...... 259 Peter Walter and Gernot Roessler

14 The Perceptual Effects of Chronic Retinal Stimulation...... 271 Alan Horsager and Ione Fine

15 Findings from Chronic Optic Nerve and Cortical Stimulation...... 301 Edward M. Schmidt

Part IV Towards Prosthetic Vision: Simulation, Assessment, Rehabilitation

16 Simulations of Prosthetic Vision...... 319 Michael P. Barry and Gislin Dagnelie

17 Image Analysis, Information Theory and Prosthetic Vision...... 343 Luke E. Hallum and Nigel H. Lovell

18 Simulations of Cortical Prosthetic Vision...... 355 Nishant R. Srivastava

19 Phosphene Mapping Techniques for Visual Prostheses...... 367 H. Christiaan Stronks and Gislin Dagnelie

20 Prosthetic Vision Assessment...... 385 Marilyn E. Schneck and Gislin Dagnelie Contents xiii

21 Activities of Daily Living and Rehabilitation with Prosthetic Vision...... 413 Duane R. Geruschat and James Deremeik

Author Index...... 425

Subject Index...... 447 wwwwwwwwwwwww Contributors

Michael P. Barry Lions Vision Research & Rehabilitation Center, Johns Hopkins University School of Medicine, Baltimore, MD 21205-2020, USA [email protected] Ava K. Bittner Lions Vision Research & Rehabilitation Center, Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, MD 21205, USA [email protected] Alex Butterwick Department of Applied Physics, Stanford University, Stanford, CA, USA [email protected] Carlos J. Cela Department of Electrical and Computer Engineering, University of Utah, 50 S. Central Campus Drive, Room 3280, Salt Lake City, UT 84112-9206, USA [email protected] Gislin Dagnelie Lions Vision Research & Rehabilitation Center, Johns Hopkins University School of Medicine, Baltimore, MD, 21205-2020 USA [email protected] James Deremeik Johns Hopkins University, Baltimore, MD, USA [email protected] Barbara R. Evans Oak Ridge National Laboratory, Oak Ridge, TN, USA [email protected] Eduardo Fernández Instituto de Bioingeniería, Universidad Miguel Hernández, Avda de la Universidad s/n, 03202 Elche (Alicante), Spain [email protected] xvi Contributors

Ione Fine University of Washington, Seattle, WA, USA [email protected] Paul G. Finlayson Departments of Otolaryngology and Ophthalmology, Wayne State University, 550 E. Canfield [email protected] Shelley I. Fried VA Boston Healthcare System, 150 South Huntington Avenue, Boston, MA 02130, USA and Massachusetts General Hospital & Harvard Medical School, 429 Their, 50 Blossom Street, Boston, MA 02114, USA [email protected] Duane R. Geruschat Lions Vision Research & Rehabilitation Center, Wilmer Eye Institute, Johns Hopkins University School of Medicine, 550 N, Baltimore, MD 21205, USA [email protected] Elias Greenbaum Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA [email protected] Luke E. Hallum Graduate School of Biomedical Engineering, University of New South Wales, ANZAC Parade, Sydney 2052, Australia and Center for Neural Science, New York University, New York, NY 10003, USA [email protected] Alan Horsager Eos Neuroscience, Inc., 2100 3rd Street, 3rd floor, Los Angeles, CA 90057, USA and Department of Ophthalmology, University of Southern California, Los Angeles, CA 90089, USA [email protected] Philip Huie Department of Ophthalmology, Stanford University, 450 Serra Mall, Stanford, CA, 94305, USA [email protected] Raymond Iezzi Department of Ophthalmology, Mayo Clinic, 200 First Street, SW, Rochester, MN 55905, USA [email protected] Contributors xvii

Ralph J. Jensen VA Boston Healthcare System, Boston, MA, USA [email protected] Bryan W. Jones Moran Eye Center, University of Utah, 65 Mario Capecchi Drive, Salt Lake City, UT 84132, USA [email protected] Gianluca Lazzi Department of Electrical and Computer Engineering, University of Utah, Salt Lake City, UT, USA [email protected] James Loudin Department of Applied Physics, Stanford University, 450 Serra Mall, Stanford, CA 94305, USA [email protected] Nigel H. Lovell University of New South Wales, Sydney, Australia [email protected] Stephen L. Macknik Barrow Neurological Institute, Phoenix, AZ, USA [email protected] Susana Martinez-Conde Barrow Neurological Institute, 350 W. Thomas Road, Phoenix, AZ 85013, USA [email protected] Robert E. Marc Moran Eye Center, University of Utah, Salt Lake City, UT, USA [email protected] Lotfi B. Merabet Harvard Medical School, Boston, MA, USA [email protected] Daniel Palanker Department of Applied Physics, Stanford University, Stanford, CA, USA [email protected] Aditi Ray Department of Biomedical Engineering, 1042 Downey Way, Denney Research Building (DRB) 140, Los Angeles, CA 90089, USA [email protected] Gernot Roessler Department of Ophthalmology, RWTH Aachen University, Aachen, Germany [email protected] xviii Contributors

Edward M. Schmidt National Institutes of Health (retired) [email protected] Marilyn E. Schneck Rehabilitation Engineering and Research Center, The Smith-Kettlewell Eye Research Institute, 2318 Fillmore Street, San Francisco, CA 94115, USA and Vision Sciences Program School of Optometry-2020, University of California at Berkeley, Berkeley, CA 94720-2020, USA [email protected] Nishant R. Srivastava Department of Biomedical Engineering, Pritzker Institute of Biomedical Science and Engineering, Illinois Institute of Technology, 3255 S. Dearborn, WH 314, Chicago, IL 60616, USA [email protected] H. Christiaan Stronks Lions Vision Research and Rehabilitation Center, Wilmer Eye Institute, Johns Hopkins University School of Medicine, 550 N. Broadway, 6th floor, Baltimore, MD 21205, USA [email protected] Janet S. Sunness Greater Baltimore Medical Center, Baltimore, MD, USA [email protected] Xoana G. Troncoso California Institute of Technology, Pasadena, CA, USA [email protected] Philip R. Troyk Department of Biomedical Engineering, Pritzker Institute of Biomedical Science and Engineering, Illinois Institute of Technology, 3255 S. Dearborn, WH 314, Chicago, IL 60616, USA [email protected] Peter Walter Department of Ophthalmology, RWTH Aachen University, Pauwelsstr. 30, 52074 Aachen, Germany [email protected] Carl B. Watt Moran Eye Center, University of Utah, Salt Lake City, UT, USA [email protected] James D. Weiland Department of Biomedical Engineering, University of Southern California, Los Angeles, CA, USA [email protected]