Elicitation of Retinal Neural Circuitry with Vision Prosthetic Devices

Elicitation of retinal neural circuitry with vision prosthetic devices

David Tsai

Submitted in partial fulﬁlment of therequirementsforthedegreeof Doctor of Philosophy in Biomedical Engineering at Graduate School of Biomedical Engineering University of New South Wales

March, 2011

Supervisor: Professor Nigel H. Lovell Co-supervisor: Professor John W. Morley

PLEASE TYPE THE UNIVERSITY OF NEW SOUTH WALES Thesis/Dissertation Sheet

Surname or Family name: Tsai

First name: David Other name/s:

Abbreviation for degree as given in the University calendar: 1710

School: Graduate School of Biomed Eng Faculty: Engineering

Title: Elicitation of retinal neural circuitry with vision prosthetic devices

Abstract 350 words maximum: (PLEASE TYPE)

Vision prostheses currently under development by several research groups aim to restore functional sight to the profoundly blind suffering from retinal neural degenerative diseases. Human clinical trials in the last decade have demonstrated the ability of these devices to elicit simple percepts, such as bright spots of light. However, further improvements in implant perceptual efficacy will critically depend on improved understanding of the retinal neural mechanisms underlying the electrically evoked responses, and on how these mechanisms could be controlled artificially.

In the first part of this thesis I quantitatively study, using a new statistical analysis technique, the temporal response properties of the retinal ganglion cells (RGCs) following electrical stimulation of the retina. I also demonstrate conclusively, for the first time, that small electrodes placed in the subretinal space could reliably elicit direct RGC spiking responses.

In the second part of the thesis I investigate the mechanisms underlying the previously observed RGC response depression during repeated electrical stimulation of these cells. The experimental findings lead me to the development of a new stimulation method for preventing the response depression.

The image processor is a crucial component of a vision prosthesis. It replaces some of the neural computations that occur in a healthy retina by converting visual stimuli into electrical stimuli. In the final part of the thesis I implement an image processor for a vision prosthesis. I show that such devices could be built with appropriate embedded hardware. Benchmark testing suggests that, depending on the complexity of the image processing strategies, care should be exercised in generalising the performance of algorithms developed on standard computers to these embedded devices.

Declaration relating to disposition of project thesis/dissertation

I hereby grant to the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or in part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all property rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation.

I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstracts International (this is applicable to doctoral theses only).

…………………………………………………………… ……………………………………..……………… ……….……………………...…….… Signature Witness Date

The University recognises that there may be exceptional circumstances requiring restrictions on copying or conditions on use. Requests for restriction for a period of up to 2 years must be made in writing. Requests for a longer period of restriction may be considered in exceptional circumstances and require the approval of the Dean of Graduate Research.

FOR OFFICE USE ONLY Date of completion of requirements for Award:

THIS SHEET IS TO BE GLUED TO THE INSIDE FRONT COVER OF THE THESIS

‘I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International (this is applicable to doctoral theses only). I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restriction of the digital copy of my thesis or dissertation.'

Signed ……………………………………………......

Date ……………………………………………......

AUTHENTICITY STATEMENT

‘I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the conversion to digital format.’

Signed ……………………………………………......

Date ……………………………………………......

Originality Statement

“I hereby declare that this submission is my own work and to the best of my knowledge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project’s design and conception or in style, presentation and linguistic expression is acknowledged.”

David Tsai March 2011

Front Matters

This thesis is presented as a series of publications, on approval by the Faculty of Engineering Higher Degree Committee. A few notes regarding the format of this thesis:

• All Chapters, except for the Introduction and Conclusion, are works either published or submitted for publication in peer–reviewed journals.

• Each publication is presented as a chapter.

• A statement regarding my contribution to each publication is stated at the beginning of each chapter.

• The status of each publication is noted at the beginning of each chapter.

• A consolidated Bibliography including all references in each paper is provided at the end of the thesis.

Permission has been granted by the journals to include the published papers, either in full or in part, in this thesis.

Abstract

Vision prostheses currently under development by several research groups aim to restore functional sight to the profoundly blind suffering from retinal neural degenerative diseases. Human clinical trials in the last decade have demonstrated the ability of these devices to elicit simple percepts, such as bright spots of light. However, further improvements in implant perceptual efficacy will critically depend on improved understanding of the retinal neural mechanisms underlying the electrically evoked responses, and on how these mechanisms could be controlled artificially. In the first part of this thesis I quantitatively study, using a new statistical analysis technique, the temporal response properties of the retinal ganglion cells (RGCs) following electrical stimulation of the retina. I also demonstrate conclusively, for the first time, that small electrodes placed in the subretinal space could reliably elicit direct RGC spiking responses. In the second part of the thesis I investigate the mechanisms underlying the previously observed RGC response depression during repeated electrical stimulation of these cells. The experimental findings lead me to the development of a new stimulation method for preventing the response depression. The image processor is a crucial component of a vision prosthesis. It replaces some of the neural computations that occur in a healthy retina by converting visual stimuli into electrical stimuli. In the final part of the thesis I implement an image processor for a vision prosthesis. I show that such devices could be built with appropriate embedded hardware. Benchmark testing suggests that, depending on the complexity of the image processing strategies, care should be exercised in generalising the performance of algorithms developed on standard computers to these embedded devices.

Contents

1 Introduction & Overview 13 1.1 Artificial control of the retinal neural network – the challenge of prosthetic vision...... 15 1.1.1 Thresholdofactivation...... 16 1.1.2 Spatialspecificityofactivation...... 16 1.1.3 Temporalaccuracyandtemporalspecificityofactivation...... 17 1.1.4 Selectiveexcitationofcelltypes...... 17 1.1.5 Coding of visual information into electrical stimuli ...... 18 1.1.6 A note on neural degeneration and the effects thereof ...... 19 1.2Surveyofcurrentprogressinprostheticcontrolofretinalneurons..... 19 1.2.1 Thresholdofactivation...... 19 1.2.2 Spatialspecificityofactivation...... 21 1.2.3 Temporalaccuracyandtemporalspecificityofactivation...... 23 1.2.4 Selectiveexcitationofcelltypes...... 24 1.2.5 Coding of visual information into electrical stimuli ...... 25 1.3 Aims of the thesis ...... 27

2 RGC temporal response properties following subretinal stimulation 31

3 Frequency–dependent reduction of INa modulates RGC responses to repetitive stimulation 45

4 Real-time image processing and stimulus coding 59

5 Conclusion 73

A List of publications 79

Chapter 1

Introduction & Overview

Sight is arguably one of our most important senses. Approximately a third of the human brain is devoted to vision [126]. The first stages of visual information processing begin at the retina, a thin sheet of neural tissue lining the back of the eyes. To a first degree of approximation, the retina contains five principal types of neurons, which are structured in a highly stereotypic organisation (Fig 1.1A). The outer layer of the retina consists of the photoreceptors, which are responsible for capturing photo–energy from the light entering the eyes. The middle layer of the retina contains the horizontal cells, the bipolar cells, and all but one subtype of amacrine cells (the displaced amacrine cells). Together, these cells are responsible for the first steps of information processing. For instance, the visual input is segregated into the ON and OFF streams, which codes for an increase and decrease in light intensity, respectively. More specific visual attributes, such as the color information and frequency components, are extracted and conveyed via different pathways. The retinal ganglion cells (RGCs) occupy the inner layer of the retina. These cells, with the amacrine cells, further the processing started by the preceding neurons and transmit the resulting signals to the brain via the optic nerve that emerges from each eye. The retinal functions outlined above may be disrupted as a result of neural degenerative diseases. In many such conditions, the outer retina is lost but the inner retina remains relatively intact (Fig 1.1B and C), thus opening the possibility for therapeutic interven- tions to restore visual percepts by artificially activating the surviving neurons. Indeed, throughout the ages investigators have examined such a possibility. In 1755, the French physician Charles LeRoy [66] wrapped a wire around a blind man’s head just above the eyes. Electric pulses were applied in the hope that the patient might see again. The patient reportedly saw flashes of light. The device, however, was far from portable. It involved a 14

ABC Epiretinal electrode

RGCs

Amacrine cells

Bipolar cells

Horizontal cells

Cones and rods

Subretinal electrode

Healthy Retina Degenerate Retina

Figure 1.1: Schematic diagram of the retina. Neurons in the retina are organised in a highly stereotypic fashion. A and C(A). A healthy retina has photoreceptors, horizontal cells, bipolar cells, amacrine cells, and retinal ganglion cells (RGCs). B and C(B). In a degenerate retina the photoreceptors, horizontal cells and a large proportion of bipolar cells are lost. Many amacrine cells and RGCs survive. These remaining neurons are the targets of electrical stimulation, which could be delivered via either epiretinal or subretinal electrodes. C. Adapted from [73], besides the loss of outer retinal layers, signiﬁcant remodeling of the surviving retinal network also occurs at advanced disease states. Here the structure of the retina was “mapped” by immunohistochemical markers. bank of Leyden jars for storing electricity. In more recent times, the development of an implantable retinal stimulator was entertained by Tassicker [120]. The design involved a photosensitive metal inserted between the retina and the choroid. It was claimed that the device was “successful in giving the patient appreciation of light and large objects where there was no previous sensation of vision” [120]. Since these early attempts, modern researchers have utilised increasingly more sophisticated technologies in their attempts to restore functional sight to the profoundly blind. Over the last two decades a series of retinal implants were trialed on blind human patients. The devices were able to induce some, albeit simple, percepts of light (phosphenes) [16, 32, 71, 129, 137]. While the details vary between the designs, modern retinal implants generally consist of a number of components (Fig 1.2). A miniature camera, often depicted as a glasses– mounted apparatus, captures visual images and transmits them to a wearable image processor. The image processor may apply additional image manipulation algorithms before converting each image into commands and energy (to power the implant) for controlling an ASIC (application speciﬁc integrated circuit) “chip”. Finally the ASIC chip delivers an 15

Retinal Implant Wearable Camera Via RF Link

Transceiver

Image Processor

Remote Supervising Tool

Figure 1.2: Components of a retinal prosthesis. An image processor converts images captured by a camera into stimulation commands and transmits them wirelessly to an implant, which delivers electrical stimuli to the remaining retinal neurons. A remote supervising tool allows the investigator and clinicians to control the entire system externally. Figure adapted from [124]. appropriate amount of electrical stimuli via a set of electrodes implanted adjacent, or in close proximity, to the retina. The transmission of stimulus commands from the external components (the camera and image processor) to the implanted components (ASIC chip and electrodes) is generally through a wireless radio frequency (RF) link. The retinal prosthesis described in this thesis uses electric pulses to restore visual percepts. Other sight restoration strategies are also being tested. For example, these include genetic approaches using channelrhodopsin–2 [63] or melanopsin [68], transplantation of the neural retina [96], and neuromorphic chips [61].

1.1 Artiﬁcial control of the retinal neural network – the challenge of prosthetic vision

Irrespective of the implementation details, the success, and ultimately the efficacy of, electrical stimulation as a therapeutic strategy for neural degenerative diseases depends critically on our understanding of the neural mechanisms elicited by electrical stimulation, and how these mechanisms may be controlled precisely to evoke the intended visual percepts. In this respect, five important questions regarding electrical elicitation of the neural mechanisms have to be addressed. They are all of importance, but I will introduce each below in order of increasing difficulty. 16

1.1.1 Threshold of activation

First, given a stimulus conﬁguration, which encompasses both the physical aspects of the electrodes (e.g. dimension and placement location) as well as the electrical parameters (e.g. pulse waveform shape and delivery frequency), one must determine the threshold for eliciting responses from the retinal neurons. The importance of the answer to this question is clear: not only does it dictate our ability to reliably evoke neural responses within a set of device–implementation constraints, but also aﬀects the safety associated with the use of retinal implants. Finally, the threshold for eliciting neural activities may be a dynamic parameter with use–dependency. For instance, this could occur in the short–term as a result of ion channel inactivation following repeated stimulation, and in the long–term due to electrochemical reactions occuring at the tissue–electrode interface.

1.1.2 Spatial speciﬁcity of activation

Second, the spatial specificity of the applied electrical stimuli, and more importantly, the specificity of the resulting neural responses have to be considered. The primate retina contains 100 million photoreceptors. The information thus relayed is “condensed” down to all that could be conveyed to the higher centres by approximately 1 million RGCs [77]. To a first approximation, the RGCs tile the inner retina and each cell encodes a specific aspect of the visual stimuli for a specific region in the visual space [101]. Therefore, our ability to confine neural activation has a direct impact on the spatial acuity of electrically evoked percepts. But it should be noted that a spatially confined electrical stimulus does not necessarily imply a correspondingly specific neural response. The cell body (the soma) of mammalian retinal neurons is generally less than 25 μm in diameter [49]. However, a cell’s dendritic tree could in some cases span up to a few millimetres [93], and the axon of all RGCs must traverse the distance between the soma and the optic nerve head, where it leaves the eye for the brain. Given a small region in the retina, say of 25 μm diameter, the general upper bound of somatic size and well within the limit of current electrode fabrication technologies, one would encounter structures from many neurons, be it the soma, the dendrites, or the axons. If they are activated simultaneously by an electrical stimulus confined entirely within this region, the resulting extent of neural activation could potentially be far more widespread. 17

1.1.3 Temporal accuracy and temporal speciﬁcity of activation

The third question pertains to 1) the temporal accuracy and 2) the temporal specificity of the electrically evoked responses. The RGCs are known to operate (fire action potentials) transiently at high frequencies in excess of 200 Hz, depending on the cell type and the visual stimuli presented [7, 80, 132], followed by brief periods of silence [6]. These suggest that an ideal retinal implant should be able to reliably elicit responses, in the form of action potentials from the RGCs, at high frequencies and with high temporal accuracy by having minimal spike time jitter (variation of response latency). Following electrical stimulation, the neural responses occur both in space (the second question above) and in time. The discussion thus far implicitly assumes that a single stimulus presentation elicits only one RGC action potential, the smallest unit of information a RGC could convey to the brain. However, in addition to the RGCs, the stimulus could also activate the surviving retinal network. Any such responses eventually, after some delay, feed into the RGCs, which may have concomitantly responded with a preceding spiking response. Thus the temporal specificity of electrically evoked responses is determined by the direct RGC responses as well as the responses of neurons presynaptic to the RGCs. Traditionally (and as described in the preceding section), individual RGCs are thought to fire independently following light stimulation. Emerging evidence suggests that the be- haviour of neighbouring RGCs are strongly correlated (or anti–correlated, depending on the cells’ receptive field structures) [21, 78, 81, 122]. Encoding at the population level provides signals that are more noise–resistant and conveys more information than by individual neurons [95]. Higher centres may use these features, in addition to the information provided by individual RGCs, to interpret the incoming visual information. These two issues (responses of neurons along the vertical retinal pathway and later- ally among the RGCs) raise concerns regarding the temporal specificity of the electrically evoked responses. Temporal specificity may be the more challenging of the two temporal considerations, because it also relates to the coding of neural information. But this issue will be deferred until the fifth question.

1.1.4 Selective excitation of cell types

The mammalian retina contains three to four types of photoreceptors, two types of horizontal cells, around twelve types of bipolar cells, two dozen types of amacrine cells and a dozen types of RGCs [77]. Much of the outer retina is lost following disease progression 18

[73]. This still leaves most of the RGCs and a substantial fraction of the amacrine cells intact [119]. The fourth question is thus whether these remaining cells could be selectively stimulated by a retinal implant. In the healthy retina, these cells encode or play specific roles in extracting features (e.g. increase or decrease of light intensity, color, motion, and directionality of movements) from the visual stimuli. An ideal retinal implant would selectively activate the individual cell types, eliciting responses resembling those of normal vision. On the other hand, if selective excitation were not achieved, it would be informative to investigate how this might affect the artificial generation of percepts, and ultimately also its ramification on the efficacy of retinal implants.

1.1.5 Coding of visual information into electrical stimuli

The fifth and final question involves the encoding of visual information into appropriate electrical stimuli, which when presented will hopefully evoke percepts in patients that reasonably approximate the original visual information. Putting it more simply, how do we portray an arbitrary image to a blind human patient with only electric pulses? The first step of this process involves decomposing an image, most likely from a camera, into pixelised components. Image–processing techniques could be performed here to maximally utilise the limited number of electrodes available in implants of the near future.

The second step involves transforming the pixelised images into electric pulses to elicit percepts that approximate the original pixelised components. Following light stimulation, the RGCs respond (as judged from their spike trains) in a highly precise manner and with little variability from trial to trial [6, 7]. How one might approach the second step with an aim of maximising the implant visual performance is far from trivial. Because this step involves matters such as what constitutes a “correct” retinal neural code, and what it entails for us to stimulate the individual retinal cells such that the correct signals are conveyed to the higher centres. In addition to encoding at the single cell level [85], further visual information is probably conveyed through the concerted activities of several neighbouring RGCs. Therefore an ideal encoding may require some control over the spike trains from several RGCs simultaneously. Understanding the retinal neural code, in particular at the population level, is a topic of ongoing research [97]. Progress in this area will likely also beneﬁt the development of prosthetic vision. 19

1.1.6 A note on neural degeneration and the eﬀects thereof

In the foregoing discussions the effects of neural degeneration were not explicitly considered. As apparent in Figure 1.1C, by the time advanced degeneration has occurred, the neurons migrate and synaptic connections rewire throughout what is left of the retina [94, 119]. These disease processes will further complicate the above considerations. For instance, thinning of the retina and appearance of the glial seal [73] may affect the activation threshold, any changes in neuronal intrinsic properties may cause them to behave differ- ently from the healthy state, and rewiring of the retinal circuit may redefine the stimulus encoding procedure. Furthermore, given the gradual progression of degenerative conditions such as retinitis pigmentosa, changes to the higher visual pathways may also occur over time. Thus what is known about the healthy visual system may not translate directly to a diseased system, further complicating issues already present at the retina.

1.2 Survey of current progress in prosthetic control of retinal neurons

In the last decade many studies have addressed aspects of the above questions. I will outline below for each question what has been reported and point out the issues addressed by my work.

1.2.1 Threshold of activation

Of the five questions previously listed, the first (determination of threshold as a function of the stimulation configuration) is by far the most extensively investigated. Papers reporting threshold data from in vitro preparations of the retina are listed in Table 1.1. Also listed are the animal models, the stimulus configurations and the recording techniques. Sekirnjak and colleagues [107] have provided a review of the previously reported charge density values, so these will not be replicated here. The use of voltage-pulses is exceedingly rare (Table 1.1, 5th column, abbreviation = V). There are several disadvantages associated with this approach, namely: 1) poor control of the total charge injected and 2) maximum stimulation only occurs at the beginning of pulse delivery [82]. It is also apparent that early work, primarily from Jensen and colleagues, often used monophasic pulses (5th column, abbreviation = mono). Such stimuli may have 20 imtr p prtnl u urtnl rn rnrtnl i=bpai,mn oohsc urn tmlto,V stimulation, current = I monophasic, = mono biphasic, = bi array. transretinal, multielectrode = = trans MEA subretinal, stimulation, = voltage sub = epiretinal, = epi diameter, al 1.1: Table esnadRzo20 mice 2009 Rizzo and Jensen al. et Tsai eina ta.20 rat 2009 al. et Jensen al. et Sekirnjak esnadRzo20 rabbit 2005 Rizzo and Jensen al. et Jensen re tal. et Fried ie al. et Li al. et Jensen al. et Jensen al. et Stett esnadRzo20 rabbit 2006 Rizzo and Jensen ’er ta.20 mice 2006 al. et Stett al. et Jensen al. et O’Hearn ( Electrode Model Animal Year Grumet Investigators hj tal. et Ahuja agltadToeo 06salamander 2006 Thoreson and Margalit eina ta.20 ie unapg okyei 6-25 epi, monkey pig, guinea mice, 2006 al. et Sekirnjak eina ta.20 monkey 2008 al. et Sekirnjak al. et Jensen re tal. et Fried al. et Shyu umr fprevious of Summary 09rabbit 2009 09rabbit 2009 03rabbit 2003 05frog rabbit 2005 rabbit 2005 2005 chicken 2000 07rabbit 2007 09rabbit 2009 00rabbit 2000 08mice 2008 07chicken 2007 08salamander 2008 06rabbit 2006 06rabbit 2006 nvitro in lcrclsiuainsuisweetrsodvle eerpre.Abbreviations: reported. were values threshold where studies stimulation electrical u,400 sub, u,25 sub, u,10 sub, p,sb 125 sub, epi, u,400 sub, u,1-0lnt 5 x length 10-30 sub, rn,5 50 x 50 trans, u,500 2 sub, x legnth 5 epi, u,400 sub, p,7-16 epi, p,15 500 125, epi, 125 epi, u,1 p 10 epi ; 5 1, sub, p,200 epi, p,3 eghx30 x length 35 epi, u,500 sub, p,10 epi, p rtnlsie,2-143 slice), (retinal epi p,9-15 epi, p,15lnt 15 x length 125 epi, p,sb 5 125 25, sub, epi, μm tmlto Recording Stimulation ) ooIExtracellular I mono I bi I bi ooIExtracellular I mono iI bi I bi ooIExtracellular Extracellular Extracellular I mono I mono I mono V bi iI bi iI bi ooIExtracellular I mono iI bi iI bi I bi ooVCl-tahd MEA Cell-attached, V mono I bi iI bi ooadb hl-elpatch Whole-cell I bi and mono iI bi iI bi iI bi Patch-clamp Extracellular MEA Patch-clamp Cell-attached MEA MEA Extracellular MEA Extracellular MEA Patch-clamp Extracellular MEA Extracellular = 21

Table 1.2: The effects of neural degeneration on stimulation threshold. The latencies are in ms. Threshold comparisons are relative to healthy retinas of the same animal species under identical experimental conditions. Refer to Table 1.1 for the stimulation and recording configurations. Investigators Year Animal Model Stimulation Latency Threshold O’Hearn et al. 2006 mice(rd1) epiretinal ≤ 2.69 increase subretinal ≤ 3.01 no difference Sekirnjak et al. 2009 rat(P23H, S334Ter) epiretinal < 0.5 no difference Jensen and Rizzo 2009 mice(rd1) subretinal ≤ 20 increase limited clinical usefulness due to the potential for accelerated electrode degradation and possibly also tissue damage by the lack of charge recovery [82]. Also apparent in Table 1.1, the thresholds of epiretinal biphasic current stimulation have been studied over a range of electrode sizes (2 – 500 μm diameter). However, reports of threshold with subretinal biphasic stimulation have been limited to relatively large electrodes. In particular, Shyu et al. [111] reportedly could not elicit responses consistently with small (25 μm diameter) subretinal electrodes. In [123] (Chapter 2) I provided the first detailed investigation of biphasic current stimulation with small subretinal electrodes (25 μm diameter). I found, contrary to Shyu and colleagues, that this configuration could reliably elicit neural responses in almost every RGC examined. Recent works have investigated the effects of neural degeneration on the stimulation threshold (Table 1.2). The results are inconsistent. For epiretinal stimulation, O’Hearn et al. [88] reported an increase in threshold following degeneration. Sekirnjak and colleagues [109], on the contrary, found the threshold to be stable and show no change over time with increasing severity of disease progression. Similar discrepancy also applies for subretinal stimulation, with O’Hearn et al. finding no difference in threshold between the normal and diseased retinas, while Jensen and Rizzo [56] observed an increase in threshold. Other than the differences in animal models, which may be a contributing factor, it is also worthwhile to compare the response latencies in these reports. The latencies of O’Hearn et al. and Jensen and Rizzo were much longer than those of Sekirnjak et al. It is possible that responses of shorter latencies (due to direct RGC activation) were present in the former studies, but were obscured by stimulus artifacts, leading to biased threshold values.

1.2.2 Spatial speciﬁcity of activation

The second question regarding the spatial specificity of electrically evoked responses requires two pieces of knowledge. To begin with, one needs to determine the electric field 22 associated with a particular stimulus configuration (both the physical aspects of the electrode geometry and placement, and the electrical characteristics such as the waveform shape and pulse duration). Then given the electric field, one needs to determine the extent of neural activation across different cell types, and at the single-cell level, activation of the cellular components. Traditionally, these problems have been investigated by mathematical modelling [5, 19, 22, 37, 79, 136]. There have been several attempts to understand the extent of neural activation by recording directly from the retina following electrical stimulation. Two approaches are commonly used: record from a single cell and position the stimulation electrode(s) at different locations relative to this cell [28, 29, 57], or fix the stimulation electrode(s) and concurrently record from a population of cells around the stimulation site [1, 40, 107, 108, 118, 117]. The results from both techniques are qualitatively similar. Neural activity reduced with increasing distance from the stimulation site. A number of reports have also looked at the neural responses using a multi–electrode array (MEA) when the retina was simultaneously stimulated with several electrodes arranged in different patterns [107, 117]. Not surprisingly, the responses varied with the patterns of active electrodes. However, the more important question not addressed (and significantly more challenging) is whether the responses were coding for the different electrode patterns. A number of early works have noted varying activation thresholds despite placing a stimulation electrode equidistant from the soma [40, 57]. It was postulated that axonal activation might have been the cause. Using a combination of electrophysiological recordings and immunohistochemistry, Fried et al. [29] found the region of lowest threshold for the RGCs to be along the axonal sodium-channel band. Consequently, this is likely to be the activation site by electrical stimulation in general. Furthermore, they found regions of the axon to have threshold higher (approximately three times) than that of the axon initial segment (AIS). It was thus suggested that electrical stimulation, at least under appropriate conditions with the commonly used biphasic rectangular pulses, could also activate passing axons in addition to the targeted cells. This would potentially degrade the spatial acuity of electrically evoked percepts. In contrast, Sekirnjak and colleagues [107, 108] showed that, with small electrodes (≤ 10 μm) and small current, it was possible to confine activation to only the immediate surrounding of the stimulation electrode. However, large stimulus current did activate cells as far away as 160 μm. It is worthwhile noting that Sekirnjak used the peripheral retina for their work, while Fried used the central region containing high cell density and 23 passing axons. Thus the issue regarding the extent of axonal activation is far from settled.

1.2.3 Temporal accuracy and temporal speciﬁcity of activation

Numerous papers have addressed the temporal response properties of retinal cells to electrical stimulation. Most reports measured this by the latency of the first action potential to a single stimulus presentation [53, 55, 56, 74, 88, 107, 108, 109, 111]. Others have shown in greater detail that the RGC spiking responses changed over time following stimulus presentation [1, 28, 67, 117, 123]. However, prior to [123] (Chapter 2) all reports only qualitatively demonstrated, with a few example figures, that changing the stimulus parameter varied the neural responses. No attempts were made to rigorously quantify the RGC temporal responses as a function of stimulus configuration (pulse amplitude and pulse width). To do this, I developed (Chapter 2) a statistical testing procedure to assess whether a change in RGC spiking responses, at any point in time, was above chance (non–random) and stimulus driven. With careful control of the stimulus parameters, an electrode placed epiretinally could reliably activate only the RGCs. By increasing the stimulus, one could then activate the neural network presynaptic to the RGCs [28, 107]. On the contrary, the responses following subretinal stimulation have been attributed to retinal network activation [53, 117]. The possibility of direct RGC activation with subretinal stimulation was not conclusively demonstrated until my work that comprises Chapter 2 [123]. The RGCs could respond to light stimuli with high frequency spiking. This held for both artificial laboratory stimuli and natural scenes. To maximise the perceptual efficacy, retinal prostheses may need to drive the RGCs at comparable frequencies. Several studies have investigated the ability of RGCs to follow high frequency stimulation (up to 400 Hz). Sekirnjak and colleagues [107] reported a frequency dependent decline in rat RGC spiking response rate with epiretinal stimuli above 50 Hz. I tested the spiking response rate of rabbit RGCs to subretinal stimulation at 50 – 200 Hz (Chapter 2 and 3). Consistent with Sekirnjak et al., a statistically significant decline in RGC spiking response rate was observed. The inability to drive the RGCs artificially at frequencies comparable to those of light evoked responses may limit the implant perceptual efficacy. Recent clinical studies also reported a frequency–dependent “fading” of visual percepts [17, 138]. The reason(s) for the fading in human patients is presently unknown. However, the observations made in the isolated retina suggest that retinal mechanisms could, at least in part, be responsible. I investigated the cause of the RGC spiking response depression in Chapter 3, and found 24 it to be primarily a result of RGC voltage–gated sodium channel inactivation. Chapter 3 also presented a technique to prevent RGC spiking response depression during repetitive stimulation. It should be noted that RGC spiking response depression was not universally observed by all investigators (see [1] and [28]). In particular, Ahuja et al. [1] reported the RGCs to follow stimuli faithfully up to 400 Hz. This discrepancy is difficult to reconcile. My whole– cell recording and cell–attached recording techniques ruled out the possibility of missing spikes hidden in stimulus artifacts. It is possible, however, that the other studies using extracellular recording techniques, as a result of the limited signal–to–noise ratio, may have mistaken minute stimulus artifact waveform variations as short–latency responses partially obscured by the artifacts. I also demonstrate in Chapter 3 that under identical conditions, the RGCs were capable of spiking at high frequencies through intracellular depolarization, and did so when driven by light stimuli. Therefore the spiking response depression was not an artifact of the experimental conditions. The above discussion focused exclusively on the responses of direct RGC stimulation. Electrical stimulation could also activate the other retinal cell types, such as the bipolar cells and the amacrine cells. If evoked, these cells give rise to responses that would eventually feed into the RGCs and potentially generate spikes. Jensen and Rizzo [54] investigated the ability of the retinal network to follow repetitive stimuli (up to 65 Hz). The stimuli were delivered via large (400 μm diameter) subretinal electrodes and consisted of anodic first biphasic pulses. The retinal network was unable to respond to stimulus rates greater than a few Hz. Similarly, I observed (Chapter 2) a severe decline in these network–originating responses during cathodic–first biphasic pulse stimulation with small electrodes (25 μm). The inability of the retinal network to respond faithfully to repetitive stimulation may thus be a general phenomenon with little stimulus configuration dependency.

1.2.4 Selective excitation of cell types

The ON and OFF RGC pathways convey two elementary streams of visual information to the brain. Several investigations have looked for differences in the activation threshold between the ON and OFF RGCs. Jensen and colleagues [58] showed that epiretinally applied cathodic stimuli preferentially activated the OFF cells, while anodic stimuli were equally effective at evoking both cell types. In a later report [53] they showed subretinally applied cathodic stimuli to preferentially activate the ON cells. These findings remain to be confirmed by other groups. Nonetheless, as mentioned previously, monophasic pulses may 25 have limited applications in chronic settings, due to the potential for accelerated electrode and tissue damage. Sekirnjak et al. [108] and my findings [123] (Chapter 2) suggest that biphasic stimuli were equally likely to elicit ON and OFF cells. Intriguingly, Fried [29] found the threshold between the brisk transient RGCs, the directionally selective RGCs and the local edge detectors to be significantly different. These results open the possibility for selective excitation on the basis of this classification scheme. However, it should be emphasised that a brisk transient RGC could either be the ON–type or the OFF–type. Similarly, there are two directionally selective cells, the ON–type or the ON-OFF type. While these results were promising, they still do not provide distinction between the ON and OFF categories (or the ON and ON–OFF categories for the directionally selective cells), and hence do not contradict the finds from either Sekirnjak or myself. The above work pertains to the RGCs only. Some bipolar cells and probably a large proportion of amacrine cells could be expected to survive the neural degeneration process. Following subretinal stimulation with biphasic pulses, the threshold for eliciting spikes of network origin (that is, activation of the bipolar cells and/or photoreceptors in the healthy retina) was similar to that of the RGCs [123] (Chapter 2), despite the former being closer to the stimulation electrodes than the latter. These results indicate that, at least with biphasic subretinal stimulation with 25 μm diameter electrodes, the bipolar cells, amacrine cells, and the RGCs could not be preferentially activated, irrespective of their proximity to the stimulation electrode(s). Recently, Freeman et al. [26] found sinusoidal stimuli of varying frequencies, but not biphasic pulses (in line with the above observations), could be used to preferentially drive the RGCs, bipolar cells, or photoreceptors. While the work does not suggest that the sinusoidal stimuli were exclusively targeting particular cell types, an important criteria for true selective excitation, it does nonetheless demonstrate the potentials of using non–conventional stimulus waveforms to preferentially activate different cell types.

1.2.5 Coding of visual information into electrical stimuli

The first step of this process involves decomposing an image into pixelised vision. The first paper describing this procedure appeared in 2007 by Asher and colleagues [3]. They presented the algorithms for image tracking, cropping, geometric transformation, and spatial– temporal filtering. The work was implemented in Matlab and ran on a desktop computer. It was postulated that the implementation could be translated to an embedded system and still maintaining real–time performance. In 2009 I published the first paper describing the 26 algorithms, architecture and implementation of a wearable real–time image processor for vision prostheses [124]. I showed that depending on the processing strategies used, general– purpose microprocessors (of the time) may be inadequate for implementing such devices, and that care should be exercised in generalising the performance of algorithms running on desktop computers to these embedded systems. Since these two early reports, several papers have appeared [25, 41, 91], describing possible processing strategies to improve the efficacy of retinal prostheses. Possibly due to the technical complexity, the next step involving conversion of pixelised images into electrical stimulus parameters is perhaps the least reported topic of retinal prosthesis research. Existing work generally assumed, often implicitly, that either the diameter or brightness (or both) of the elicited phosphenes could be modulated as a function of stimulus strength. That is, stronger stimuli equated to larger/brighter phosphenes. This may be a reasonable approximation in some patients using one particular retinal implant [38]. However, the generality of this observation remains to be tested. A more physiologically oriented approach to the conversion step is to consider it a spike– encoding problem [97]. Fried and colleagues [28] showed that the RGCs could be stimulated epiretinally to elicit one action potential per stimulus. By evoking these action potentials in quick succession, they were able to artificially drive the spiking patterns of a RGC to mimic that of the light evoked spike trains from the same cell. This approach requires all RGCs to faithfully follow repetitive electrical stimuli at physiologically relevant frequencies (up to 200 Hz or more). As indicated above, the RGCs generally showed a decline in spiking response rate upon stimulation at ≥ 50 Hz. Using the current stimulation techniques, it may not be possible to faithfully drive the RGC spikes artificially at high frequencies. However, the “adaptive current scaling” technique presented in Chapter 3 may be able to alleviate this problem. I also proposed a strategy whereby high frequency stimulus trains were interleaved with a quiescent period, allowing the RGC voltage–gated sodium channels to recover from inactivation. With the aim of developing an encoding strategy for prosthetic vision, Ryu et al. [105] reported that the firing rate of RGCs could be modulated by the stimulus amplitude. It was concluded that through stimulus amplitude modulation, one could encode and deliver useful percepts to the implant recipient. The work assumed the RGCs to use only rate code (integrated over a timescale of seconds in [105]) for conveying information to the brain. That is, the number of spikes fired within a time period of seconds is the only relevant coding parameter. However, both the number of spikes fired as well as the timing 27 between the spikes are used by the RGCs for coding [7, 80]. While it is useful being able to modulate the RGC firing rates, at the seconds time scale as demonstrated by the authors, the ability to control timing between spikes will also be important. Beyond these works, little else has been reported on neural coding for prosthetic vision.

1.3 Aims of the thesis

This thesis aims to better understand the retinal neural mechanisms elicited by electrical stimulation and how these mechanisms may be precisely controlled through artiﬁcial stimulation. In doing so, this thesis attempts to address the following questions.

1. Can subretinal stimulation reliably evoke RGC responses directly? In particular, could this be achieved with small electrodes having dimensions close to that of the RGC soma? The work of Fried [28] and Sekirnjak [107] have experimentally demonstrated the ability of epiretinal stimulation to elicit direct RGC responses, evoking spikes with short latency. Previous subretinal stimulation studies have failed to observe these short–latency stimulus–locked RGC spikes. The latencies reported were generally in excess of 5 ms [53]. These ﬁndings may have, at least partially, given rise to the notion that subretinal stimulation is associated with responses of network origin (for example, see [23, 73]). To address this question, I stimulated isolated rabbit retinas with 25 μm diameter electrodes, a size similar to the somatic dimensions of many RGCs. I then recorded the RGC responses using either cell–attached extracellular recording or whole-cell current clamp.

2. How do the RGCs respond temporally to subretinal stimulation as a function of stimulus conﬁguration? Following electrical stimulation with an array of electrodes, the retina responds both in space and in time to the stimuli. In epiretinal stimulation, carefully controlled stimuli could elicit a single action potential from only one, or a handful of, RGCs [28, 107]. Several previous reports (for example see [53, 65, 117]) have noted the temporal diversity of subretinally evoked RGC spikes. However, no attempt was made to quantitatively assess how the RGC temporal spiking response proﬁle varied as a function of the stimulus parameters, and whether generalisations could be made regarding the spiking patterns. To answer this question, I began by developing a 28

statistical analysis technique to ascertain if a RGC spike at a particular point in time is non–random and thus stimulus–driven. I then applied this technique to RGC spikes following a variety of stimulus conﬁgurations.

3. Can the RGCs reliably follow electrical stimulation at repetition rates comparable to that of the light evoked RGC spiking frequencies?

The RGCs are able to, and do, convey visual information with high frequency bursts of action potentials. To maximise the perceptual efficacy of retinal implants, it may be necessary to drive the RGCs artificially at similar frequencies. The work of Sekirn- jak [107] revealed a decline in rat RGC spiking response rate when these cells were stimulated epiretinally at ≥ 50 Hz. The question thus arises regarding the generality of these results with respect to the animal species and stimulation paradigm (that is, epiretinal verses subretinal stimulation). Furthermore, what retinal mechanisms underlie this phenomenon and are there ways to circumvent this problem? The answers to these questions are all the more pertinent after recent reports of fading percepts in human patients during repeated stimulation of the retina [17, 138]. I addressed this question by stimulating rabbit retinas subretinally at frequencies 50 – 200 Hz. The findings also led me to the development of a new stimulation technique for preventing the spiking response depression.

4. Using conventional cathodic–ﬁrst biphasic pulses, is it possible to preferentially activate: 1) the ON and OFF RGCs and 2) the retinal layers?

As the first steps of image processing, the retina extracts salient visual features and transmits them via several parallel pathways to the brain [103]. Selective excitation of different cell types would afford retinal implants the ability to utilise the existing neural architecture for conveying visual information. On the contrary, if selective excitation were not possible, it would be useful to understand how this might affect the implant efficacy. I addressed this question by examining, under a variety of stimulus configurations, whether the ON, OFF, and ON–OFF RGCs, as well as neurons in different layers of the retina were preferentially excited by the electrical stimuli.

5. Can an image processor for coding visual information into electrical stimuli be adequately built using embedded system technologies, and what are the ramiﬁcations of using such hardware to implement these devices? 29

The image processor is a core component of vision prostheses. It needs to replace, or at least approximate, the computation that normally occurs in the distal layers of a healthy retina. Psychophysical studies of simulated pixelised vision on sighted subjects suggest that non–trivial image processing strategies may be necessary to maximally utilise the limited electrode count in implants of the near future [15, 43]. The question thus arises whether embedded hardware are adequate for implementing such devices, while still retaining adequate conﬁgurability (for the investigators) and portability (for the implant recipients). To address this question, I implemented an image processor using a Texas Instruments OMAP5912 dual–core microprocessor, which is commonly used in portable embedded multimedia devices. 30 Chapter 2

RGC temporal response properties following subretinal stimulation

This chapter contains the published article: Tsai D, Morley J W, Suaning G J, Lovell N H. Direct activation and temporal response properties of rabbit retinal ganglion cells following subretinal stimulation. Journal of Neurophysiology. 2009(102): 2982- 2993 (cover article).

Current status: Published.

Author contributions: The contribution of D. T. to this paper was 85%, consisting of designing the experiments, conducting all aspects of data collection, and writing the manuscript. J Neurophysiol 102: 2982–2993, 2009. First published September 9, 2009; doi:10.1152/jn.00545.2009.

Direct Activation and Temporal Response Properties of Rabbit Retinal Ganglion Cells Following Subretinal Stimulation

David Tsai,1 John W. Morley,2,3 Gregg J. Suaning,1 and Nigel H. Lovell1 1Graduate School of Biomedical Engineering and 2School of Medical Sciences, University of New South Wales; and 3School of Medicine, University of Western Sydney, Sydney, New South Wales, Australia

Submitted 22 June 2009; accepted in ﬁnal form 7 September 2009

Tsai D, Morley JW, Suaning GJ, Lovell NH. Direct activation and mechanisms involved during electrical activation and how temporal response properties of rabbit retinal ganglion cells following these mechanisms may be controlled artificially to produce subretinal stimulation. J Neurophysiol 102: 2982–2993, 2009. First clinically useful vision. published September 9, 2009; doi:10.1152/jn.00545.2009. In the last Stimulating electrodes may be placed at any one of the decade several groups have been developing vision prostheses to several locations relative to the retina. The subretinal approach restore visual perception to the profoundly blind. Despite some promising results from human trials, further understanding of the involves placing the electrodes between the photoreceptor Downloaded from neural mechanisms involved is crucial for improving the efficacy of layer and retinal pigment epithelium. Several authors have these devices. One of the techniques involves placing stimulating previously studied the effect of subretinal stimulation on retinal electrodes in the subretinal space between the photoreceptor layer and ganglion cells (RGCs) using a variety of animal models the pigment epithelium to evoke neural responses in the degenerative (Jensen and Rizzo 3rd 2006, 2007, 2008; Li et al. 2005; retina. This study used cell-attached and whole cell current-clamp O’Hearn et al. 2006; Shyu et al. 2006; Stett et al. 2000). With recordings to investigate the responses of rabbit retinal ganglion cells the exception of Stett et al. (2000), Li et al. (2005), and Jensen (RGCs) following subretinal stimulation with 25-␮m-diameter elec- and Rizzo 3rd (2008), existing subretinal studies have primar- Յ jn.physiology.org trodes. We found that direct RGC responses with short latency ( 2 ily focused on the thresholds for eliciting neural responses. It ms using 0.1-ms pulses) could be reliably elicited. The thresholds for has been observed previously (Li et al. 2005; Stett et al. 2000) these responses were reported for ON, OFF, and ON– OFF RGCs over pulse widths 0.1–5.0 ms. During repetitive stimulation these direct that the temporal response profiles of RGCs following subreti- activation responses were more readily elicited than responses arising nal stimulation were diverse and varied with stimulus config- from stimulation of the retinal network. The temporal spiking char- uration. Thus the question arises whether there exist patterns in acteristics of RGCs were characterized as a function of stimulus the RGC response profile and, if so, how these patterns relate configurations. We found that the response profiles could be general- to the stimulus configurations. Furthermore, studies using on November 12, 2009 ized into four classes with distinctive properties. Our results suggest epiretinal stimulations, in which the electrodes were placed that for subretinal vision prostheses short pulses are preferable for adjacent to the RGC layer, have shown that short-latency efficacy and safety considerations, and that direct activation of RGCs stimulus-locked RGC spikes could be reliably elicited (Fried et will be necessary for reliable activation during high-frequency al. 2006; Sekirnjak et al. 2006). Such responses have not been stimulation. conclusively demonstrated for the subretinal approach. Here we studied the responses of RGCs following subretinal INTRODUCTION stimulation using cell-attached and whole cell current-clamp recordings in the rabbit retina. We began by demonstrating that Retinal degenerative diseases such as retinitis pigmentosa subretinal stimulation with small electrodes (25-␮m diameter) and age-related macular degeneration are currently the leading can reliably elicit short-latency direct RGC spikes. The robust- cause of blindness in developed countries (Klaver et al. 1998). ness of these direct responses during repetitive stimulation was As the population ages, it is expected that the prevalence of examined at a range of frequencies and pulse widths. We also these diseases will increase significantly in the years to come quantitatively investigated the temporal response profiles of (Friedman et al. 2004). Several research groups are actively RGCs following electrical stimulation, as a function of stimu- developing strategies to restore vision in profoundly blind lus amplitude and duration. individuals through electrical stimulation of the retina (Fu- jikado et al. 2007; Gerding et al. 2002; Humayun et al. 1999; METHODS Vo¨lker et al. 2004; Wong et al. 2009; Zhou et al. 2008) and trials on blind subjects have shown that simple percepts, such Retinal whole-mount preparation as bright spots, could be generated (Fujikado et al. 2007; Gekeler et al. 2006; Humayun et al. 2003; Rizzo et al. 2003a). All procedures were reviewed and approved by the UNSW Animal Care and Ethics Committee. New Zealand White rabbits (2–3.5 kg) Although promising, the elicited percepts were rudimentary were anesthetized with an intramuscular injection of ketamine (70 and, in many cases, also unexpected (Rizzo et al. 2003). Our mg/kg) and xylazine (10 mg/ml) and supplemented intravenously as ability to improve the efficacy of these prosthetic devices will needed. An eye was enucleated. The animal was then killed by sodium depend critically on further understanding of the retinal neural pentobarbital overdose. The eye was hemisected 2–3 mm behind the ora serrata. The front portion of the eye and the vitreous were Address for reprint requests and other correspondence: N. H. Lovell, discarded. Three pieces of the inferior retina inclusive of the visual Graduate School of Biomedical Engineering, University of New South Wales, streak were dissected free with the underlying sclera and placed in an Sydney, NSW 2125, Australia (E-mail: [email protected]). incubation chamber containing Ames’ medium (Sigma–Aldrich) with

1% penicillin/streptomycin (Invitrogen) equilibrated to pH 7.4 at 66.7, 100, and 200. At each frequency three pulse widths were tested either 25 or 31°C for 1 h before being transferred to a room temper- (ms): 0.1, 0.2, and 0.5. For each frequency and pulse-width configu- ature holding chamber and kept for Յ10 h before recording. ration pair, the stimulus train was repeated 20 times, with a 1-s delay To assist visualization, prior to recording the RGCs were labeled by between repeats. immersing a piece of the retina, with the attached sclera, in Ames’ medium containing 1 mg/ml of Azure B (Sigma–Aldrich) (Amthor et al. 2002; Hu et al. 2000) for 45 s. The neural retina was then separated Recording of retinal ganglion cell responses from the pigment epithelium and mounted photoreceptor side up in an RGC responses were recorded with either cell-attached recording or imaging chamber on an inverted microscope with poly-L-lysine (Sigma– whole cell current clamp (I ϭ 0 nA) using a Multiclamp 700B Aldrich) at 1 mg/ml. The retinas were perfused with Ames’ medium h amplifier (Molecular Devices). Data were low-pass filtered at 3 kHz at 5 ml/min, equilibrated with 95% O -5% CO to pH 7.4 and heated 2 2 and digitized at 10 kHz with a Digidata 1440A and pCLAMP 10 to 34–35°C. software (Molecular Devices). All recordings were carried out under mesopic lighting. The retinas were visualized with either Hoffman Light stimulation Modulation Contrast or Nomarski differential interference contrast optics under near-infrared illumination. RGCs were selected for re- A stationary spot of light emitted by a white LED behind a pinhole cording by targeting cells with soma Ͼ10 ␮m (Vaney 1980) and lying was projected onto the retina via the ϫ10 objective of the microscope. proximal to the inner limiting membrane. The spot was centered over the soma of the targeted RGC and had a Electrodes were pulled from borosilicate glass with outer and inner diameter of 250 ␮m when focused on the photoreceptor layer. diameters of 1.5 and 0.86 mm, respectively. All electrodes were To determine the light response type of the recorded cell, a 4-s coated with Sylgard (Dow Corning) and flame polished before use. alternating sequence consisting of ON– OFF– ON– OFF, each of 1 s, was For whole cell recordings electrodes were filled with an intracellular Downloaded from presented. This was repeated five times with 5 s between each (pipette) solution containing (in mM): 116 KMeSO4, 10 KCl, 0.5 presentation. The entire process was then repeated for the alternating EGTA, 1 MgCl2, 10 HEPES, 4 ATP-Na2, 0.5 GTP-Na3, adjusted to sequence OFF– ON– OFF– ON. pH 7.2 with KOH. The electrodes had resistance 3–6 M⍀ with this Cells were considered transient if their responses returned to base- solution. To study RGC spikes due to direct activation, in some line by the end of the 1-s light stimulus; otherwise, they were ␮ recordings synaptic transmissions were blocked with 250 M CdCl2. considered sustained (Roska and Werblin 2001). Tetrodotoxin (TTX) at 1 ␮M was used to block RGC spikes. In both cases, the pharmacological agents were added directly to Ames’ jn.physiology.org Electrical stimulation medium and perfused over the retina. All chemicals were acquired from Sigma–Aldrich. Constant-current charge-balanced biphasic stimuli were generated by a No systematic difference was found for electrical elicitation thresh- custom-made neural stimulator capable of delivering Յ200 ␮A of cur- olds between the cell-attached mode and whole cell mode recording rent. The stimulation electrodes were fabricated from a platinum–iridium techniques. We thus pooled the results from both techniques during (Pt-Ir) wire (A-M Systems) of 25-␮m diameter coated with Teflon, with analysis. only the transversely cut circular tip exposed. Electrodes were placed at on o em er the photoreceptor side under visual guidance to a lateral distance of 55 Ϯ 10 ␮m from the soma of the target cell (Fig. 1A) to avoid mechanical Data analysis disturbance of the patched cell during positioning. The stimulation return Spikes were detected using pCLAMP10 (Molecular Devices) with electrode consisted of a distant platinum wire loop in the perfusion bath threshold crossing. Data were analyzed with UNIX command line about 2 cm away from the stimulation electrode (monopolar configura- utilities on MacOS X (Apple), MATLAB (The MathsWorks), and tion). All biphasic pulses were cathodic-first, followed immediately by

Prism (GraphPad Software). The statistical tests used will be stated as the anodic component without interpulse delay. they appear. The confidence level has been set at 95% (two-tailed). In this study we defined the stimulation threshold to be the current required to elicit RGC action potential(s) in Ն10 of 20 consecutive trials (50%) with a 1-s delay between trials to minimize potential Fitting of direct RGC elicitation probability to long-duration effects of repetitive stimulation (Jensen and Rizzo 3rd stimulus strength 2008). Response latencies were measured from the onset of the stimulus artifact to the peak of the action potentials. Assuming that direct elicitation of RGC spikes was due to activa- For repetitive stimulation with pulse trains, the stimulus amplitudes tion of independent voltage-gated channels by the stimulus, we used were set at 110 Ϯ 10% of the cell’s threshold (as defined earlier) at the the following Boltzmann equation to describe the probability of given pulse duration. Each pulse train consisted of four identical eliciting RGC spikes as a function of stimulus current amplitude, in biphasic pulses. The repetition frequencies attempted were (Hz): 50, accordance with Hodgkin–Huxley style analysis of neural excitation

FIG. 1. Typical subretinal stimulation con- ABfiguration. A: a schematic diagram of the stimu- Stimulation Electrode lation and recording arrangement. The whole- mount retina was placed photoreceptor side up Patch Electrode ␮ 55 ± 10 m on an inverted microscope. The 25- m platinum–iridium (Pt-Ir) stimulation electrode was placed at the photoreceptor side under visual Photoreceptors guidance to a lateral distance of 55 Ϯ 10 ␮m from the soma of the targeted retinal ganglion cell (RGC). B: a photograph (ϫ100 magnification) of the setup. The arrow marks the location of the cell and the circle denotes the point of RGCs contact of the stimulation electrode at the photo- ϭ ␮ Chamber Base receptor-side. Scale bar 100 m. Inset: view (ϫ400 magnification) of the targeted RGC.

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org 2984 D. TSAI, J. W. MORLEY, G. J. SUANING, AND N. H. LOVELL

100 parameter (see Fig. 6 for example). A star symbol (*) marks all P͑I͒ ϭ Ϫ͑ Ϫ␣͒ ␤ segments where the difference from the control is statistically signif- ϩ e I / 1 icant at the 95% confidence level. The colors indicate the spiking where P(I) is the probability of eliciting a spike and I is the applied probability for a given time segment aggregated over 20 repetitions current as a percentage of the threshold current. The fitting parameters with identical stimulus, expressed as a percentage. Therefore a value are ␣ and ␤. of 100 at a particular time segment would indicate that every stimulus presentation successfully elicited a spike within that time interval. In this report we call these figures “temporal activity maps.” Creation of temporal activity maps

To quantitatively assess the time series data recorded from RGCs in RESULTS electrically stimulated retina as a function of stimulus strength, we analyzed the homogeneity of repeated stimulation runs against the Subretinal stimulation could elicit short-latency direct corresponding repeated control runs without stimulus presentation. RGC responses We begin here by formally defining the methodology for the general case. To study the effects of subretinal electrical stimulation on the Assuming weak stationarity, the time series data were first decom- retina, we placed a stimulation electrode at the photoreceptor side posed into orderly stochastic point processes. Let S and C be the set such that its tip visibly contacted the distal boundary of the of trials with and without stimulus presentation, respectively. The photoreceptor layer, as depicted schematically in Fig. 1A.A trials were indexed k ϭ 1,...,K, so that k ϭ 1 for the first trial, and photograph of the typical configuration viewed from the RGC k ϭ K for the last trial. For each trial we had the recording interval (0, side at ϫ100 magnification is shown in Fig. 1B. The cell is Downloaded from T], where T is the recording duration. Recording intervals were marked with an arrow and the circle denotes the position of the divided into small equal segments each with duration ⌬t and indexed ϭ ϭ ⌬ stimulation electrode at the photoreceptor side. The inset shows as n 1,...,N, such that T N t. So we may denote a particular the targeted RGC, which has been labeled with Azure B. segment for a particular trial in the stimulation set as Sk,n and similarly ⌬ Using cell-attached or whole cell current-clamp recordings for the control set as Ck,n. We chose t to be sufficiently small such that each interval of each trial contained either one spike or no spike. we found that with sufficient stimulus strength subretinally Thus a 2 ϫ 2 contingency table could be constructed for the following placed electrodes were able to elicit short-latency stimulus- pairs, for each segment n locked action potential(s) in almost all (97.9%) RGCs tested jn.physiology.org n ϭ K ( 46/47). Importantly, these responses were easily distin- ϭ ഫ guished from the stimulus artifacts. Figure 2A shows ten Sn Sk,n kϭ1 superimposed cell-attached recordings of an OFF–RGC on stim- K ulation with a 125-␮A, 0.1-ms biphasic pulse. Six of the ten ϭ ഫ Cn Ck,n trials elicited an action potential. As shown in the inset of Fig. kϭ1 ␮ 2A, these short-latency spikes were blocked by TTX (1 M, on o em er With this formalization we then statistically tested each segment with n ϭ 3/3 cells). Spike responses were equally clear with long- stimulus presentation Sn against its corresponding segment without duration stimuli. In Fig. 2B an OFF–RGC was successfully stimulation Cn (the control) for homogeneity using Fisher’s exact test activated in seven of the ten trials using a 7-␮A, 5.0-ms (two-tailed, 95% CI). biphasic pulse. The spike latency and temporal jitter were In practice, unless stated otherwise, we performed 20 repetitions for larger than those for 0.1-ms pulses in Fig. 2A. Also apparent in both the control trials and the stimulation trials (K ϭ 20), with each ϭ both figures are spontaneous spikes not correlated to the T trial having a duration of 100 ms ( 100). The control trials were C always performed before the stimulation trials. Based on previous electrical stimuli. As demonstrated in Fig. 2 , these stimulus- locked short-latency responses remained in the presence of reports of RGC firing frequency (O’Brien et al. 2002) and our ␮ ϭ observations, the segments were set to 2 ms (⌬t ϭ 2), such that all bath-applied CdCl2 (250 M, n 7/7 cells). This suggests that segments contained either one spike or none. they were not of presynaptic origin. The results are presented here as bivariate figures with the horizon- Taken together, these results indicated that subretinal stim- tal axis being the time and the vertical axis being the stimulation ulation could reliably activate RGCs directly, evoking short-

Standard Bath A success (spikes) C FIG. 2. Elicitation of short-latency stimu- 2 mV 20 2 ms lus-locked RGC responses with subretinal 15 stimulation. A: 10 superimposed traces of an TTX OFF–RGC stimulated with a single 125-␮A, std. aCSF 10 0.1-ms biphasic pulse delivered at the 0-ms failure

Sweep # time point. Six of the trials successfully elicited 5 a short-latency spike, which was clearly distin- 0 guished from the biphasic stimulus artifact. In- -10 10 30 50 70 90 Time (ms) set: short-latency responses were blocked by B tetrodotoxin (TTX, 1 ␮M, n ϭ 3/3). B:10 CdCl2 Bath 20 superimposed traces of an OFF–RGC stimulated with a 7-␮A, 5.0-ms biphasic pulse. A 15 short-latency response was elicited in 7 trials. success (spikes) These spikes were also easily seen within the 10 artifacts. C: raster plots of a cell stimulated 20 Sweep # ␮ failure 5 times with a 105- A, 0.1-ms pulse. The short- 20mV latency responses remained in the presence of ␮ ϭ 0 CdCl2 (250 M, n 7/7), suggesting that they -5 0 5 10 15 20 -10 10 30 50 70 90 Time (ms) Time (ms) were not of presynaptic origin.

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org RGC RESPONSES FOLLOWING ELECTRICAL STIMULATION 2985 latency stimulus-locked responses that were visible with cell- 4.5 attached and whole cell current-clamp recordings. 4 Thresholds of short-latency direct RGC responses

) 3.5 2 To determine the threshold of these short-latency spikes we recorded from ON (n ϭ 14), OFF (n ϭ 23), and ON– OFF (n ϭ 5) 3 cells over pulse durations 0.1–5.0 ms. Figure 3 summarizes the results for these cells. For all cell types the threshold decreased 2.5 with increasing pulse width. However, there was no statistically significant difference in thresholds between the cell types Charge Density (mC/cm 2 for eliciting short-latency direct activation responses with subretinal stimulation (two-way ANOVA, P ϭ 0.6561). Figure 4 shows the charge densities on the 25-␮m Pt-Ir 1.5 electrode at the median threshold for eliciting short-latency 1 stimulus-locked spikes over pulse widths 0.1–5.0 ms. The data 0 1 2 3 4 5 from all cell types (ON, OFF, ON– OFF; n ϭ 42) have been pooled Pulse Width (ms) for the calculation. The median charge density was lowest with FIG. 4. Charge density for direct activation of RGCs. The median charge Downloaded from 0.2-ms pulses, at 1.43 mC/cm2, and increased with pulse width density was lowest with 0.2-ms pulses and increased with pulse width there- Ն0.2 ms. after. The data from all cell types (ON, OFF, and ON-OFF; n ϭ 42) have been combined in this graph. Temporal response properties of RGCs were cell and RGC activation probability increased nonlinearly with stimulus strength dependent stimulus strength In addition to the short-latency stimulus-locked spikes de- jn.physiology.org We examined the probability of eliciting direct RGC re- tailed thus far, subretinal stimulations generally also evoked sponses with respect to the variation in stimulation strength. responses with longer latencies, some in the order of tens of Using 20 consecutive trials of 0.1-ms biphasic pulses on 12 milliseconds or more. We attempted to gain a quantitative cells (n ϭ 4 for each of ON, OFF, ON– OFF) we determined the understanding of how the stimulus strength modulated both of percentage of trials with successful activation against the these response types. Temporal activity maps were created stimulus strength, expressed as a percentage of the cells’ from the 0.1-ms biphasic stimulation data sets for all RGCs. on o em er threshold current (Fig. 5). The electrical stimuli were always delivered at the 0-ms time The fitting parameters in the Boltzmann equation, their point. In all maps stimulus strength is shown on the vertical corresponding confidence intervals, and the goodness-of-fit axis and time progresses from left to right. We found the measures are listed in Table 1. For all three RGC types temporal response profiles of RGCs following subretinal stim- investigated the probability of activation increased nonlinearly ulation to be highly heterogeneous. The responses varied between cells and for a given cell generally also varied with with stimulus current. In all cases there was a critical current range in which the chance of directly evoking RGC spikes rose stimulus strength. This is demonstrated with the temporal dramatically with increasing stimulus amplitude. Below or 100 above this region the rate of increase was markedly reduced. ON OFF ON−OFF 80 200 ON OFF ON-OFF 150 60 A)

100

% Elicitation 40 Threshold ( 50

0 20 0.1 0.2 0.5 1.0 2.0 5.0 Pulse Width (ms)

FIG. 3. Strength–duration plot for short-latency direct RGC activation. The 0 0 40 80 120 160 200 thresholds for ON (n ϭ 14), OFF (n ϭ 23), and ON– OFF (n ϭ 5) cells were determined for pulse widths 0.1–5.0 ms. The median current amplitude % Threshold Current reduced for all cells with increasing pulse width. No significant difference was FIG. 5. Modulation of elicitation probability with current amplitude. The found for the thresholds between the 3 cell types over the pulse widths tested probability of eliciting short-latency RGC responses increased nonlinearly (2-way ANOVA, P ϭ 0.6561). The box-and-whisker plot shows (from top to with stimulus strength. Boltzmann curves were fitted to the data points for each bottom) the maximum, third quartile, median, first quartile, and minimum. cell types (n ϭ 12).

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org 2986 D. TSAI, J. W. MORLEY, G. J. SUANING, AND N. H. LOVELL

TABLE 1. Relationship between spiking probability Classiﬁcation of RGC temporal response proﬁles and stimulus amplitude Although the temporal responses of RGCs following sub- Cell Type ␣ (95% CI) ␤ (95% CI) R2 RMSE retinal stimulation were heterogeneous, we found that the responses could be generalized into four classes: primary ON 98.0 (96.8, 99.2) 5.6 (4.2, 6.9) 94.5% 8.6 OFF 93.7 (90.7, 96.6) 9.6 (6.1, 13.0) 98.5% 12.2 A ON–OFF 96.1 (90.6, 101.6) 16.2 (9.5, 22.9) 93.9% 8.9 80 75uA The relationship between spiking probability and stimulus current, ex- 70

pressed as a percentage of threshold, can be described by a Boltzmann curve. 60 The fitting parameters and 95% confidence intervals are shown here. Also 70uA listed are the goodness-of-fit measures. 50 40 Stimulus 60uA activity map of four cells in Fig. 6. It should be emphasized 30

that the temporal activity patterns presented here are not 20 50uA exhaustive. 10

In some cells electrical stimulation elicited only a single 0 −10 10 30 50 70 90 short-latency spike. Besides enhancing the presence of these Time (ms) responses, further increase of the stimulation current had no

B 90 Downloaded from other statistically signiﬁcant effect on spiking activities. Figure 6A shows an example of such cells. The probability of eliciting 75uA 80 short-latency spikes increased with the current amplitude. Of 70 60 all 2-ms time segments immediately after stimulus presenta- 70uA tion, only the 70- and 75-␮A cases were statistically different 50 40 Stimulus from the control without stimulus delivery. 67uA 30

Electrical stimulation had a suppressive effect on the jn.physiology.org spontaneous activities of some cells. As demonstrated in 20 65uA Fig. 6B, by using current Ն70 ␮A, the spiking activities of 10 0 this cell were sufﬁciently reduced around the 36-ms time −10 10 30 50 70 90 Time (ms) point such that the differences compared with the control

were statistically signiﬁcant. The threshold for initiating this C 90

suppressive effect (70 ␮A) was higher than the threshold for 100uA 80 on o em er

eliciting short-latency spikes (65 ␮A). Both the short- 70

latency responses and the spike suppression persisted for 60 95uA current Ͼ75 ␮A (data not shown). 50

The cell in Fig. 6C exhibited spikes with three categories of 40 Ͻ Ͼ Stimulus 90uA latency: 2 ms, 4–6 ms, and 35 ms. Their occurrences were 30

stimulus strength dependent. Although activities of all catego- 20 ␮ ries were apparent at 80 A, only the segments belonging to 80uA 10 the second category were statistically different from the con- 0 −10 10 30 50 70 90 trol. Increasing the current further made the second category Time (ms) spikes more robust. The presence of the third category became signiﬁcant at 90 ␮A. The ﬁrst category, with latency Ͻ2 ms, D 90 had the highest threshold at 95 ␮A. 170uA 80

Somewhat similar is the cell in Fig. 6D, which exhibited 70

two categories of response latency, one of Ͻ2msand 60 150uA another with Ͼ33 ms. Stimulating at 100 ␮A elicited a 50

cluster of long-latency spikes at the 50- to 60-ms time 40 Stimulus 125uA points. The presence of short-latency responses became 30 ␮ statistically signiﬁcant at 125 A. Thus this is the opposite 20

of Fig. 6C, where short-latency responses had a lower 100uA 10 threshold than that of the long-latency responses. Further 0 −10 10 30 50 70 90 increase of stimulus strength raised the probability of elic- Time (ms) iting both the short-latency responses and the long-latency FIG. 6. Temporal activity map of 4 RGCs stimulated with 0.1-ms biphasic spike cluster. pulses. The color in each time segment indicates the response rate (%), over 20 We did not observe any correlation between the RGC types consecutive trials. Stars mark all segments statistically different from the (ON, OFF, ON– OFF) and the temporal response patterns elicited by control (without stimulus). A: a cell that exhibited only short-latency re- electrical stimulation. In summary, the temporal activity pro- sponses. B: an example of a cell that had a period of reduced activity in addition to the short-latency responses. C: a cell that exhibited 3 classes of ﬁles of RGCs following subretinal stimulation varied from cell responses with different latencies (in order of increasing threshold): 4–6, Ͼ40, to cell. Furthermore, for a given cell the responses were and Ͻ2 ms. D: similar to the previous cell, but with only 2 classes of response, dependent on stimulus strength. one with latency Ͼ40 ms and another with latency Ͻ2 ms.

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org RGC RESPONSES FOLLOWING ELECTRICAL STIMULATION 2987

TABLE 2. Properties of the four RGC temporal response classes stimulus current levels tested. Only ﬁve RGCs (n ϭ 5/47) following subretinal stimulation displayed this class of response. However, the present statistical comparison technique may have underestimated the actual Response Class Latency, ms Duration, ms Frequency prevalence of this response class (see DISCUSSION). The late onset cluster class encompassed the spike cluster Primary spike Յ2 1 spike 98% Secondary spike(s) 4–20 1–2 spikes 36% with latencies of 25–50 ms (calculated to the ﬁrst spike of the Latent period 2–45 2–30 ms 11%† cluster). The durations were variable, ranging from 10 to 40 Late onset cluster 25–50 10–40 ms 15% ms. In some cases (n ϭ 3) the durations were stimulus strength dependent, increasing with pulse duration. The threshold for The values here are based on data from stimulation of RGCs with 0.1-ms pulses.† See text for details. eliciting this response could be higher (Fig. 6C) or lower (Fig. 6D) than the primary and secondary spike classes. Of the cells spike, secondary spike(s), latent period, and late onset cluster. examined, seven (n ϭ 7/47) had this class of response. Together these classes completely described the responses of Using data from all three cell types (ON, OFF, and ON– OFF; every RGC studied. The temporal activity maps of RGCs n ϭ 42 cells), we compared the thresholds for eliciting primary following 0.1-ms biphasic pulse stimulation were used to spikes, secondary spikes, and late onset clusters (Fig. 8). There quantitatively identify and categorize the response patterns. was no systematic difference in thresholds of these response Table 2 summarizes the pertinent features of the four response classes over the pulse width tested (two-way ANOVA, P ϭ classes when RGCs were stimulated with 0.1-ms pulses. Also 0.071).

listed is the prevalence of each class in our sample of RGCs. Downloaded from Յ Primary spikes had response latency of 2 ms and showed Long stimulus pulses increased latency and onset jitter of little temporal jitter (Fig. 7). These were the short-latency, primary and secondary spikes stimulus-locked action potentials described earlier. Almost all RGCs studied exhibited this class of response (n ϭ 46/47). Using temporal activity maps of RGCs stimulated with Pharmacological blocking of synaptic inputs with CdCl2 sug- 0.1-ms pulses, we have shown that the temporal response gested that these spikes were not of presynaptic origin (Fig. profiles could be generalized into four classes despite the 2C). Increasing the stimulus amplitude increased the probabil- observed diversity both within and between cells. However, the jn.physiology.org ity of eliciting this response. question arises whether RGC responses were also pulse width The secondary spike(s) class consisted of either one or two dependent (in addition to being pulse amplitude dependent, as action potentials. The onset latencies were 4–20 ms and had demonstrated earlier) and, more generally, whether the pro- more temporal jitter than that of the primary spikes. This is posed response classification scheme is valid for long-duration demonstrated by the raster plot of a cell with both classes of electrical stimuli. response (Fig. 7A). The latency range (Fig. 7B) of the second- We began by comparing the temporal activity maps of on o em er ary spikes was longer than that of the primary spikes. Further- stimulation trials over pulse widths 0.1, 1.0, and 2.0 ms for more for cells exhibiting secondary spikes, the threshold for each RGC. In general, the temporal response characteristics for these responses could be lower than, higher than, or equal to each cell were preserved across pulse widths. Thus if a cell the threshold of the primary spikes. The secondary spikes of demonstrated (or lacked) particular temporal response classes the cell in Fig. 6C provide an example of the first case. when stimulated with 0.1-ms pulses, then the same response

Secondary spikes were observed in 17 of 47 cells. Applications classes would also be present (or absent) at longer stimulus ␮ of CdCl2 (250 M) abolished spikes of this response class in pulses. Figure 9 shows the temporal activity maps of three cells all cells examined (n ϭ 3/3), indicating that these were likely (Fig. 9, A and B; C and D; and E and F) stimulated with 0.1- to be of presynaptic origin. This class of response became more and 2.0-ms pulses. The primary spikes remained as the pulse robust with increasing stimulus amplitude. width was increased. Furthermore, consistent with the obser- The latent period was characterized by a time interval within vations of Fig. 6, the probability of eliciting primary spikes which the RGC spiking activity was signiﬁcantly lower than increased with stimulus strength for all three cells at both pulse that of the control. The onset latencies were variable (4–20 widths. Nevertheless, even with 2-ms time segments the in- ms). The observed durations ranged from 2 to 30 ms for the crease in temporal jitter due to longer pulses was readily A B 20 primary spikes 8

15 6 FIG. 7. Jitter comparison of the primary and secondary spikes. A: raster plot of a cell stimulated with 150-␮A, 0.1-ms biphasic pulses. The primary spikes had latency Յ2

10 4 ms after stimulus onset, whereas the secondary spikes had latency of 4.4–6.2 ms. B: the median latency and range of Sweep # the 2 response classes in the raster plot. The secondary

Spike Peak Time (ms) spikes exhibited larger temporal jitter than the primary 5 2 spikes.

secondary spikes 0 0 Primary Spikes Secondary Spikes -10 0 10 20 30 Time (ms) Response Type

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org 2988 D. TSAI, J. W. MORLEY, G. J. SUANING, AND N. H. LOVELL

200 Primary spike and secondary response classes as a function of pulse widths. Secondary spike(s) Plotted in Fig. 10A are the median and range of RGC primary 150 Late onset cluster spike latencies for pulse widths (ms): 0.1, 1.0, and 2.0, when

A) the cells (n ϭ 12) were stimulated at the threshold for primary

100 spikes. The differences in median latencies were highly significant (Kruskal–Wallis, P Ͻ 0.0001). The median latency of

Threshold ( 0.1-ms pulses was significantly different from the median 50 latency of 1.0- and 2.0-ms pulses (Dunn’s posttest, P Ͻ 0.05). Also apparent in the figure is the increase in temporal jitter of the 0 response spikes, as indicated by the increased interquartile ranges, 0.1 0.2 0.5 1.0 2.0 5.0 when longer stimuli (1.0 and 2.0 ms) were used. The differences Pulse Width (ms) in latency variance between those elicited by 0.1-ms stimuli and FIG. 8. Thresholds of RGC response classes: primary spike, secondary the longer stimuli were highly significant (Levene’s test; 0.1 vs. spike(s), and late onset cluster. For all response classes the threshold decreased 1.0 ms, P Ͻ 0.001; 0.1 vs. 2.0 ms, P Ͻ 0.001). with pulse width. However, there was no systematic difference in threshold Figure 10B shows the median and range of RGC (n ϭ 9) between the classes over the pulse widths tested (n ϭ 42; 2-way ANOVA, P ϭ 0.071). secondary spike latencies for pulse widths (ms): 0.1, 1.0, and 2.0. For cells that responded with more than one secondary apparent in two of the three cells. In the first (Fig. 9, A and B) spike, we used the time-of-peak of the first spike to calculate and second (Fig. 9, C and D) cells, onset of the primary spikes the latency. The median latency increased significantly with Downloaded from was delayed by one time segment (2 ms) when the stimulus pulse width (Kruskal–Wallis, P Ͻ 0.0001, Dunn’s posttest, pulse width was increased from 0.1 to 2.0 ms. P Ͻ 0.05). By comparing Fig. 10A with 10B it is also apparent To further investigate the effect of pulse width on RGC that the temporal jitter of primary spikes was smaller than that spiking responses, we compared the latencies of the primary of the secondary spikes at all pulse widths tested. Pulse Width = 0.1ms Pulse Width = 2.0ms jn.physiology.org A B 60 80

170uA 70 17uA 50

60 40

165uA 16uA 30 on o em er Cell 1 40 Stimulus Stimulus

30 20

20 160uA 13uA 10 10

0 0 −10 10 30 50 70 90 −10 10 30 50 70 90 Time (ms) Time (ms)

FIG. 9. Comparison of temporal activity C 90 D 50 maps of RGCs stimulated with 0.1- and 80 45 40uA 4.0uA 2.0-ms pulses. Shown here are the temporal 40 70 activity maps of 3 RGCs stimulated with 35 60 0.1-ms (left column) and 2.0-ms (right col- 30 umn) biphasic pulses. The temporal response 50 Cell 2 30uA 3.5uA 25 characteristics were preserved across differ- 40

Stimulus Stimulus 20 ent pulse widths. However, long-duration 30 15 pulses generally increased the response la- 20 tencies, as apparent from the temporal activ- 10 27uA 3.0uA ity maps of the ﬁrst (A and B) and second (C 10 5 and D) cell. 0 0 −10 10 30 50 70 90 −10 10 30 50 70 90 Time (ms) Time (ms) E F 70 60 15uA 180uA 60 50

40 40 Cell 3 170uA 12uA 30 Stimulus 30 Stimulus

20 20 10uA 150uA 10 10

0 0 −10 10 30 50 70 90 −10 10 30 50 70 90 Time (ms) Time (ms)

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org RGC RESPONSES FOLLOWING ELECTRICAL STIMULATION 2989

ABPrimary Spikes Secondary Spikes * * * FIG. 10. The effect of pulse width on the latency of primary * * and secondary spikes. A: the median latency of primary spikes 5 elicited by 0.1-ms stimulus pulses was significantly shorter than 20 that by 1.0- and 2.0-ms pulses (n ϭ 11; Kruskal–Wallis, P Ͻ 4 0.0001, Dunn’s posttest, P Ͻ 0.05). The jitter of spike onset 15 also increased with pulse width. The latency variances of 3 responses elicited by the 0.1-ms pulses were significantly dif- 10 ferent from those elicited by longer pulses (Levene’s test; 0.1 2 vs. 1.0 ms, P Ͻ 0.001; 0.1 vs. 2.0 ms, P Ͻ 0.001). B: the Latency (ms) Latency Latency (ms) Latency increase in median latency of secondary spikes due to increas- 1 5 ing pulse width was significant (n ϭ 9; Kruskal–Wallis, P Ͻ 0.0001, Dunn’s posttest, P Ͻ 0.05). 0 0 0.1 1.0 2.0 0.1 1.0 2.0 Pulse Width (ms) Pulse Width (ms)

In summary, the four temporal response classes remained consisting of four identical biphasic stimuli. For each cell, four valid for long-stimulus pulses. However, some characteristics pulse train frequencies were tested (50, 66.7, 100, and 200 Hz) of the response classes varied with stimulus pulse width. In the and, at each frequency, three different stimulus pulse widths Downloaded from case of the primary spikes, increasing the pulse width increased were tested (0.1, 0.2, and 0.5 ms). the median latency and temporal jitter. For the secondary Figure 11A shows the mean primary spike response rate (%) spikes, we observed an increase in median latency with in- of the cells to each of the four 0.1-ms pulses over 20 repeti- creasing pulse width. tions. The results for all four stimulation frequencies are presented together. Figure 11, B and C illustrates the results for Depression of RGC primary spikes by repetitive stimulation pulse width of 0.2 and 0.5 ms, respectively. For all stimulus jn.physiology.org Given the prolonged activities of many RGCs subsequent to frequencies and every pulse width tested there was a trend for a single subretinally applied electrical stimulus, as apparent decreasing response rate with repetitive stimulation. The dif- from the temporal activity maps, and the presence of primary ferences in response rate to the pulses in the stimulus train were spikes in almost every cell investigated, we asked whether this signiﬁcant (repeated-measures two-way ANOVA) for all pulse class of response could be reliably elicited during repetitive widths tested (0.1 ms, P Ͻ 0.0001; 0.2 ms, P ϭ 0.0059; 0.5 ms, ϭ

stimulations using the subretinal paradigm. We performed P 0.0005). However, for a given pulse width, there was no on o em er repetitive stimulations on RGCs (n ϭ 11) with a pulse train signiﬁcant difference in response rate among the four stimula-

ABCPulse Width = 0.1ms Pulse Width = 0.2ms Pulse Width = 0.5ms 100 100 100 50Hz 66.7Hz 100Hz 80 80 80 200Hz

60 60 60 Response Rate (%) Rate Response Response Rate (%) Response Rate (%)

40 40 40 1st 2nd 3rd 4th 1st 2nd 3rd 4th 1st 2nd 3rd 4th Pulse Count Pulse Count Pulse Count D

10 Sweep #

0 −10 10 30 50 70 90 110 130 150 170 190 Time (ms)

FIG. 11. The response rate of RGC spikes reduced with repetitive subretinal stimulation. There was a trend for decreasing response rate with each subsequent stimulus pulse. For all pulse widths and all repetition frequencies, the differences in primary spike response rate between each successive pulse in the train were statistically signiﬁcant (n ϭ 11; repeated-measures 2-way ANOVA). However, given a pulse width, there was no signiﬁcant difference between the stimulation frequencies tested. A: four 0.1-ms pulses, between pulses: P Ͻ 0.0001; between frequencies: P ϭ 0.4200. B: four 0.2-ms pulses, between pulses: P ϭ 0.0059; between frequencies: P ϭ 0.7429. C: four 0.5-ms pulses, between pulses: P ϭ 0.0005; between frequencies: P ϭ 0.8154. D: raster plot of a cell exhibiting both primary and secondary spikes when the cell was stimulated at 50 Hz with 4 identical 0.1-ms biphasic pulses. The secondary spikes were less robust than the primary spikes.

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org 2990 D. TSAI, J. W. MORLEY, G. J. SUANING, AND N. H. LOVELL tion frequencies (0.1 ms, P ϭ 0.4200; 0.2 ms, P ϭ 0.7429; 0.5 RGC responses. Earlier reports have not distinguished between ms, P ϭ 0.8154). these different classes and reported thresholds as the current These results suggested that repetitive subretinal stimulation levels that evoked the first instance of any one or more of the increased the failure rate of eliciting RGC primary spikes. This RGC response classes. Due to their prevalence and for consis- was true for all pulse widths tested. Furthermore, the response tency we have reported the threshold for eliciting primary RGC rates were not significantly different between the slowest spikes. As noted previously, the threshold for these responses repetition frequency (50 Hz) and fastest repetition frequency for a given cell could be higher than, lower than, or equal to the (200 Hz) attempted. threshold of any of the other response classes. Third, existing Four of the cells (n ϭ 4/11) also exhibited secondary spikes subretinal reports have not been able to unambiguously resolve when stimulated with a single biphasic stimulus. Shown in Fig. the short-latency stimulus-locked responses, potentially bias- 11D is one of the cells stimulated at 50 Hz with four 78-␮A, ing the reported thresholds. Last, thresholds have not been 0.1-ms biphasic pulses. Primary spikes were successfully elic- defined consistently in the literature. Some investigators have ited (100%) for all four stimuli during the trial. The first used the definition as the current required for evoking RGC stimulus also elicited secondary spikes in all 20 trials. How- responses in 50% of the trials, whereas others used the defini- ever, the response rate of secondary spikes diminished faster tion as the current needed for successfully eliciting spikes in than those of the primary spikes with each subsequent stimu- 90% of the trials. However, in practice this may not be a lus. This was observed in all four cells for all pulse widths at significant issue. Figure 5 shows that, at least for ON and OFF all repetition rates. This suggests that, in our sample of RGCs, type RGCs, within the current range required for eliciting secondary spikes were less robust during repetitive stimulation direct RGC responses in 20–80% of the trials, a small increase Downloaded from than the primary spikes. in stimulus amplitude resulted in a disproportionally large increase in probability of evoking responses. DISCUSSION Quantitative assessment of RGC temporal response profiles Using cell-attached and whole cell current-clamp recordings this report investigated the responses of rabbit RGCs to sub- The primary spikes were characteristically conspicuous and retinal stimulation with small-diameter electrodes. thus easily correlated with the stimuli. The other response jn.physiology.org classes were more difficult to assess in relation to the stimulus Activation of RGCs with subretinal stimulation parameters. We statistically analyzed the time series data to ensure that the presence of these responses was above chance Subretinal stimulation could elicit RGC responses directly. and stimulus driven. We found assessment on the basis of The effects of subretinal stimulation on RGCs have been raster plots and peristimulus time histograms, as has been done studied by several investigators using a variety of species, previously, to be subjective and thus unreliable. This is espe- on o em er including: frogs (Li et al. 2005), chickens (Stett et al. 2000), cially true when evaluating the latent periods and late onset rabbits (Jensen and Rizzo 3rd 2006, 2007; Shyu et al. 2006), clusters, resulting from their large temporal jitter of onset and and mice (Jensen and Rizzo 3rd 2008; O’Hearn et al. 2006). variability in duration. However, previous reports have not provided conclusive evi- It should be noted that the temporal activity maps used dence for the existence of short-latency responses (primary spikes to analyze differences between two time series data sets.

spikes) with subretinal stimulation. The reason for this may be Therefore the accuracy of this technique in measuring spike due to the recording techniques used. In all previous studies inhibition due to electrical stimulation relied on the cell having RGC responses were recorded extracellularly with metal or some level of sustained spontaneous activities. The median carbon-fiber electrodes. Large stimulus artifacts, lasting several spontaneous firing rate of the RGCs in our study was 4.5 Hz milliseconds or more, have been observed in these studies. (approximately Poisson distribution; with 52% of cells Յ4 Shyu et al. (2006) were unable to elicit RGC responses Hz), which is equivalent to 0.45 spike per 100 ms. The consistently with 25-␮m-diameter electrodes for pulse widths temporal activity maps were divided into 2-ms time segments. Ͻ1.0 ms, when the electrodes were placed either subretinally Thus in general the segments would be very sparse under the or epiretinally. We were able to evoke short-latency direct control condition. For this reason, the true prevalence of the responses in all but one of the RGCs. For this cell the highest latent period could have been underestimated. current amplitude tested at 0.1 ms was 200 ␮A, the maximum output of our stimulator. At this level the cell clearly exhibited Efficacy and safety of subretinal stimulation responses belonging to the secondary spike(s) class. This cell may have had an unusually high threshold for primary spikes. Temporal accuracy of RGC spikes. To simulate natural-light It is possible that we would have evoked short-latency re- responses, vision prostheses should be capable of eliciting sponses had we been able to deliver higher currents with the spikes over a wide range of frequencies. Jensen and Rizzo 3rd stimulator. (2008) previously reported that RGC response rate declined Threshold of direct RGC activation. Attempts to compare with each successive pulse when stimulating subretinally at the present thresholds to previous reports are confounded by frequencies Ն4 Hz and that most RGCs did not respond to several factors. First, in a number of previous subretinal stim- frequencies Ն60 Hz. Their response latencies were of the order ulation studies the activation thresholds were not systemati- of 5–32 ms and thus most likely the equivalent of our second- cally investigated or reported, whereas others have used volt- ary spikes and possibly also the late onset clusters. Our results age stimuli or monophasic current pulses. Second, we have indicated that, in addition to a reduction of secondary spike noted that subretinal stimulation gives rise to four classes of response rate (Fig. 11D), there was also a decreasing trend in

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org RGC RESPONSES FOLLOWING ELECTRICAL STIMULATION 2991 response rate of the primary spikes during repetitive subretinal When stimulating with cathodic first biphasic pulses, the stimulation at all frequencies tested (Fig. 11, A–C). However, charge injection limits of Pt and Pt-Ir were reported to be with the exception of the 0.1-ms, 200-Hz case, the response 0.7–3.3 mC/cm2 (Robblee et al. 1983) and, more recently, as rates of all other configurations were still Ͼ50% (above thresh- low as 0.1–0.15 mC/cm2 (Brummer and Robblee 1983; Rose old) by the fourth pulse. The stimulation frequencies used in and Robblee 1990). Therefore the Pt-Ir electrodes used here the present study were also much higher (50–200 Hz). These would be close to, or even over, the reported limits. To results suggest that the primary spikes were less susceptible to overcome this, one could use electrode materials with higher repetitive stimulation failure than the other response classes. charge injection limits, such as activated iridium (Brummer Margalit and Thoreson (2006) and Fried et al. (2006) previ- and Robblee 1983). Alternatively, larger electrodes may be ously found that activation of the amacrine cells resulted in a used. Previous subretinal studies using 125-␮m electrodes large sustained inhibitory postsynaptic current (IPSC) lasting have found the charge densities to be below the safety limits Ն500 ms, which suppressed RGC excitability. Compared with (O’Hearn et al. 2006; Shyu et al. 2006). the IPSC, bipolar cell excitatory postsynaptic currents (EPSCs) were of smaller amplitude and duration. Presumably then, Implications for vision prosthesis design when RGC spikes were repetitively evoked via stimulation of the presynaptic pathway during subretinal stimulation, the Comparison of subretinal and epiretinal stimulation. In inhibition by amacrine cells onto the bipolar cells and the most aspects the present subretinal findings are similar to those RGCs far outweighed the excitatory inputs from the bipolar reported for the epiretinal approach using cathodic-first charge- cells. Thus the RGCs were unable to follow high-frequency balanced biphasic stimuli. The strength–duration curves (Fig. Downloaded from stimulation. However, when the RGCs were recruited directly 3) were comparable; thresholds fell with increasing pulse width (Ahuja et al. 2008; Fried et al. 2006; Sekirnjak et al. 2006), (Ahuja et al. 2008; Sekirnjak et al. 2006). The relationship spiking responses of the RGCs no longer depended critically between probability of direct elicitation of RGC responses and on the bipolar cell inputs, thus making repetitive stimulation stimulus amplitude (Fig. 5) were both sigmoidal (Fried et al. more robust than the scenario involving elicitation via the 2009; Sekirnjak et al. 2008). Short-latency responses by direct network. Taken together, the results suggest that to achieve RGC activation and long-latency responses by stimulation of reliable activation during repetitive stimulation Ͼ4 Hz, sub- the retinal network (Table 2) could be elicited in both para- jn.physiology.org retinal vision prostheses may need to activate RGCs directly, digms (Fried et al. 2006; Sekirnjak et al. 2006). In the epireti- rather than through activation of the retinal network. nal case, the latency of short-latency responses appeared to be Electrical stimulation of the retina should ideally elicit slightly lower on average (approximately Յ0.7 ms). Finally, responses with high temporal precision. This will be particu- the responses elicited by direct stimulation of the RGCs were larly important during high-frequency stimulation. It is appar- also found to be more robust during high-frequency stimulation ent in Fig. 10 that short pulses resulted in shorter response than those arising from activation of the retinal network for on o em er latencies and for the primary spikes also smaller temporal jitter both subretinal (Fig. 11) and epiretinal (Ahuja et al. 2008; of onset. Therefore short pulses would be preferred in this Fried et al. 2006; Sekirnjak et al. 2006) stimulation. In the context. epiretinal case RGC responses were reliably up to 50 Hz, Selective excitation of RGCs. To imitate natural-light re- whereas a moderate decline was noted with subretinal stimu- sponses, retinal stimulation should ideally excite the RGCs lation at the same frequency.

selectively. We did not observe any significant difference in The greatest difference between subretinal and epiretinal threshold when classifying RGCs on the basis of ON, OFF, and stimulation is the minimum electrode size for safe stimulation. ON– OFF types. This is consistent with previous findings using It has been demonstrated that epiretinal stimulation using Pt biphasic stimuli (Sekirnjak et al. 2008). We also did not find electrodes of 10–15 ␮m diameter could effectively evoke RGC any correlation between the light response properties (ON, OFF, responses at charge injection densities that were well within the 2 ON– OFF, transient/sustained and peak firing rate) and the result- safety limit (0.03 mC/cm ; Sekirnjak et al. 2006). In contrast, ing electrical response classes evoked on the RGCs (data not the present subretinally placed 25-␮m-diameter electrodes shown). Fried et al. (2009) found the difference in threshold were close to, or over, the safety limit. This implies that, when between brisk transient cells, direction selective cells, and local fabricated from materials such as Pt or Pt-Ir, electrodes placed edge detectors to be significantly different when stimulating subretinally are fundamentally more constrained in their min- epiretinally. However, the light stimulus used here did not imum size than those placed epiretinally. This may potentially allow us to functionally distinguish between RGCs to the same have an impact on the visual resolution achievable on subreti- extent as in their study. nal implants. Furthermore, the lowest charge density required Safety. Although small electrodes in theory provide more to elicit RGC responses with the present subretinal electrode focused stimuli, the charge density has to remain within the was about 70-fold higher than the values reported for epiretinal safety limit of the electrodes for chronic applications. It can be electrodes of comparable dimensions (Sekirnjak et al. 2006). seen in Fig. 4 that, except for the slight increase with 0.1-ms The higher charge density may lead to further loss of focal pulses, short pulses had lower charge density than that of the stimulation. long pulses. Since no significant difference in thresholds was Further considerations for subretinal implants. There are observed between the RGC response classes (Fig. 8), this two methods of eliciting RGC responses with subretinal implants, finding also applies for the other spiking response classes. either through indirect activation by stimulating the retinal net- Given the reduced charge density and the higher temporal work, which has been the primary focus of previous subretinal accuracy mentioned earlier, short pulses are therefore the studies, or through direct stimulation of the RGCs as demon- configuration of choice for subretinal applications. strated here. Direct activation has a number of advantages over

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org 2992 D. TSAI, J. W. MORLEY, G. J. SUANING, AND N. H. LOVELL network stimulation. The response latencies were shorter and had Fried SI, Lasker ACW, Desai NJ, Eddington DK, Rizzo JF 3rd. Axonal less temporal jitter and, importantly, these responses were more sodium channel bands shape the response to electric stimulation in retinal robust during repetitive stimulation. However, spikes evoked via ganglion cells. J Neurophysiol 101: 1972–1987, 2009. Friedman DS, O’Colmain BJ, Munõz B, Tomany SC, McCarty C, de Jong direct activation of RGCs (with high temporal precision) were PTVM, Nemesure B, Mitchell P, Kempen J, and Eye Diseases Preva- generally also accompanied by long-latency activity arising from lence Research Group. Prevalence of age-related macular degeneration in activation of the network (with comparatively poor temporal the United States. Arch Ophthalmol 122: 564–572, 2004. accuracy). Furthermore, selective excitation of either the RGCs Fujikado T, Morimoto T, Kanda H, Kusaka S, Nakauchi K, Ozawa M, only or the network does not appear to be possible using the Matsushita K, Sakaguchi H, Ikuno Y, Kamei M, Tano Y. Evaluation of present stimulation technique (Fig. 8), due to similarity in thresh- phosphenes elicited by extraocular stimulation in normals and by suprachoroidal-transretinal stimulation in patients with retinitis pigmentosa. Graefes olds. It is not clear how these long-latency activities will influence Arch Clin Exp Ophthalmol 245: 1411–1419, 2007. the percepts generated initially by direct activation. In contrast, Gekeler F, Messias A, Ottinger M, Bartz-Schmidt KU, Zrenner E. Phos- with short-duration pulses and carefully controlled stimulus phenes electrically evoked with DTL electrodes: a study in patients with strength, epiretinal stimulation can be reliably confined to activate retinitis pigmentosa, glaucoma, and homonymous visual field loss and only the RGCs (Fried et al. 2006; Sekirnjak et al. 2006). normal subjects. Invest Ophthalmol Vis Sci 47: 4966–4974, 2006. Gerding H, Eckmiller RE, Hornig R, Ortmann V, Kolck A, Taneri S. Safety assessment and acute clinical tests of epiretinal retina implants. Invest Experimental considerations Ophthalmol Vis Sci 43: E-Abstract 4488, 2002. Hu EH, Dacheux RF, Bloomfield SA. A flattened retina-eyecup prepara- This study investigated the neurophysiological responses of tion suitable for electrophysiological studies of neurons visualized with RGCs following electrical stimulation. The complexities of the trans-scleral infrared illumination. J Neurosci Methods 103: 209–216, Downloaded from electrical field associated with the placement of both the 2000. Humayun MS, de Juan E Jr, Weiland JD, Dagnelie G, Katona S, Green- working and return electrodes and thus the effect of this field berg R, Suzuki S. Pattern electrical stimulation of the human retina. Vision on the retina and the subsequent interpretation of the results Res 39: 2569–2576, 1999. were simplified by the use of a monopolar stimulation config- Humayun MS, Weiland JD, Fujii GY, Greenberg R, Williamson R, Little uration. Similar to previous in vitro studies, the retinas were J, Mech B, Cimmarusti V, Boemel GV, Dagnelie G, de Juan E Jr. Visual isolated and placed in an imaging chamber. The lack of perception in a blind subject with a chronic microelectronic retinal prosthesis. Vision Res 43: 2573–2581, 2003. jn.physiology.org vitreous space and the nonconductive chamber glass base Jensen RJ, Rizzo JF 3rd. Thresholds for activation of rabbit retinal ganglion would have influenced the current field arising from the elec- cells with a subretinal electrode. Exp Eye Res 83: 367–373, 2006. trical stimuli. This most likely had some impact on the evoked Jensen RJ, Rizzo JF 3rd. Responses of ganglion cells to repetitive electrical responses. stimulation of the retina. J Neural Eng 4: 1–6, 2007. This study used healthy rabbit retina. The loss of outer Jensen RJ, Rizzo JF 3rd. Activation of retinal ganglion cells in wild-type and rd1 mice through electrical stimulation of the retinal neural network. Vision neural layers in the diseased retina would bring subretinally Res 48: 1562–1568, 2008. on o em er placed electrodes closer to the RGCs, thus potentially reducing Klaver CCW, Wolfs RCW, Vingerling JR, Hofman A, de Jong PTVM. the activation threshold. However, a high-resistance fibrotic Age-specific prevalence and causes of blindness and visual impairment in an seal forms between the remnant neural retina and the pigment older population. Arch Ophthalmol 116: 653–658, 1998. Li L, Hayashida Y, Yagi T. Temporal properties of retinal ganglion cell epithelium during disease progression (Marc et al. 2003). This responses to local transretinal current stimuli in the frog retina. Vision Res may reverse the benefit of reduced electrode–cell distance. 45: 263–273, 2005. Studies have begun to investigate the activation thresholds of Marc RE, Jones BW, Watt CB, Strettoi E. Neural remodeling in retinal

the degenerative retina (Jensen and Rizzo 3rd 2008; O’Hearn et degeneration. Prog Retin Eye Res 22: 607–655, 2003. al. 2006), although little is known about the underlying neural Margalit E, Thoreson WB. Inner retinal mechanisms engaged by retinal electrical stimulation. Invest Ophthalmol Vis Sci 47: 2606–2612, 2006. mechanisms involved and how these compare with healthy O’Brien BJ, Isayama T, Richardson R, Berson DM. Intrinsic physiological retinas. Future investigations will need to address these issues. properties of cat retinal ganglion cells. J Physiol 538: 787–802, 2002. O’Hearn TM, Sadda SR, Weiland JD, Maia M, Margalit E, Humayun ACKNOWLEDGMENTS MS. Electrical stimulation in normal and retinal degeneration (rd1) isolated mouse retina. Vision Res 46: 3198–3204, 2006. We thank P. Byrnes-Preston for designing and assistance in implementing Rizzo JF 3rd, Jensen RJ, Loewenstein J, Wyatt J. Unexpectedly small the neural stimulator. percepts evoked by epiretinal electrical stimulation in blind humans. Invest Ophthalmol Vis Sci 44: E-Abstract 4207, 2003. GRANTS Rizzo JF 3rd, Wyatt J, Loewenstein J, Kelly S, Shire D. Methods and perceptual thresholds for short-term electrical stimulation of human This work was supported by National Health and Medical Research Council retina with microelectrode arrays. Invest Ophthalmol Vis Sci 44: 5355– of Australia. 5361, 2003. Robblee LS, Lefko JL, Brummer SB. Activated Ir: an electrode suitable for REFERENCES reversible charge injection in saline solution. J Electrochem Soc 130: 731–733, 1983. Ahuja AK, Behrend MR, Kuroda M, Humayun MS, Weiland JD. An in vitro Rose TL, Robblee LS. Electrical stimulation with Pt electrodes. VIII. Elec- model of a retinal prosthesis. IEEE Trans Biomed Eng 55: 1744–1753, 2008. trochemically safe charge injection limits with 0.2 ms pulses. IEEE Trans Amthor FR, Keyser KT, Dmitrieva NA. Effects of the destruction of Biomed Eng 37: 1118–1120, 1990. starburst-cholinergic amacrine cells by the toxin AF64A on rabbit retinal Roska B, Werblin FS. Vertical interactions across ten parallel, stacked directional selectivity. Vis Neurosci 19: 495–509, 2002. representations in the mammalian retina. Nature 410: 583–587, 2001. Brummer SB, Robblee LS. Criteria for selecting electrodes for electrical Sekirnjak C, Hottowy P, Sher A, Dabrowski W, Litke AM, Chichilnisky stimulation: theoretical and practical considerations. Ann NY Acad Sci 405: EJ. Electrical stimulation of mammalian retinal ganglion cells with multi- 159–171, 1983. electrode arrays. J Neurophysiol 95: 3311–3327, 2006. Fried SI, Hsueh HA, Werblin FS. A method for generating precise temporal Sekirnjak C, Hottowy P, Sher A, Dabrowski W, Litke AM, Chichilnisky patterns of retinal spiking using prosthetic stimulation. J Neurophysiol 95: EJ. High-resolution electrical stimulation of primate retina for epiretinal 970–978, 2006. implant design. J Neurosci 28: 4446–4456, 2008.

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org RGC RESPONSES FOLLOWING ELECTRICAL STIMULATION 2993

Shyu J-S, Maia M, Weiland JD, O’Hearn T, Chen S-J, Margalit E, Suzuki ment of subretinally implanted microphotodiode arrays in cats by optical S, Humayun MS. Electrical stimulation in isolated rabbit retina. IEEE coherence tomography and ﬂuorescein angiography. Graefe’s Arch Clin Exp Trans Neural Syst Rehabil Eng 14: 290–298, 2006. Ophthalmol 242: 792–799, 2004. Stett E, Barth W, Weiss S, Haemmerle H, Zrenner E. Electrical Wong YT, Chen SC, Seo JM, Morley JW, Lovell NH, Suaning GJ. Focal multisite stimulation of the isolated chicken retina. Vision Res 40: activation of the feline retina via a suprachoroidal electrode array. Vision 1785–1795, 2000. Res 49: 825–833, 2009. Vaney D. A quantitative comparison between the ganglion cell populations Zhou JA, Woo SJ, Park SI, Kim ET, Seo JM, Chung H, Kim SJ. A and axonal outﬂows of the visual streak and periphery of the rabbit retina. suprachoroidal electrical retinal stimulator design for long-term animal J Comp Neurol 189: 215–233, 1980. experiments and in vivo assessment of its feasibility and biocompatibil- Vo¨lker M, Shinoda K, Sachs H, Gmeiner H, Schwarz T, Kohler K, ity in rabbits. J Biomed Biotechnol 2008: Article 547428 (1–10), Inhoffen W, Bartz-Schmidt KU, Zrenner E, Gekeler F. In vivo assess- 2008. Downloaded from jn.physiology.org on o em er

J Neurophysiol • VOL 102 • NOVEMBER 2009 • www.jn.org

Chapter 3

Frequency–dependent reduction of

INa modulates RGC responses to repetitive stimulation

This chapter contains the article: Tsai D, Morley J W, Suaning G J, Lovell N H. Frequency-dependent reduction of voltage-gated sodium current modulates retinal ganglion cell response rate to electrical stimulation Journal of Neural Engineering. 2011 in press (recommended reader)

Current status: In press, September 2011

IOP PUBLISHING JOURNAL OF NEURAL ENGINEERING J. Neural Eng. 8 (2011) 000000 (12pp) UNCORRECTED PROOF Frequency-dependent reduction of voltage-gated sodium current modulates retinal ganglion cell response rate to electrical stimulation

David Tsai1, John W Morley1,2, Gregg J Suaning1 and Nigel H Lovell1,4

1 Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW 2052, Australia 2 School of Medicine, University of Western Sydney, Sydney, NSW 1797, Australia Q1 3 School of Medical Sciences, University of New South Wales, Sydney, NSW 2052, Australia E-mail: [email protected]

Received 5 August 2011 Accepted for publication 27 September 2011 Published DD MMM 2011 Online at stacks.iop.org/JNE/8/000000

Abstract The ability to elicit visual percepts through electrical stimulation of the retina has prompted numerous investigations examining the feasibility of restoring sight to the blind with retinal implants. The therapeutic efficacy of these devices will be strongly influenced by their ability to elicit neural responses that approximate those of normal vision. Retinal ganglion cells (RGCs) can fire spikes at frequencies greater than 200 Hz when driven by light. However, several studies using isolated retinas have found a decline in RGC spiking response rate when these cells were stimulated at greater than 50 Hz. It is possible that the mechanism responsible for this decline also contributes to the frequency-dependent ‘fading’ of electrically evoked percepts recently reported in human patients. Using whole-cell patch clamp recordings of rabbit RGCs, we investigated the causes for the spiking response depression during direct subretinal stimulation of these cells at 50–200 Hz. The response depression was not caused by inhibition arising from the retinal network but, instead, by a stimulus-frequency-dependent decline of RGC voltage-gated sodium current. Under identical experimental conditions, however, RGCs were able to spike at high frequency when driven by light stimuli and intracellular depolarization. Based on these observations, we demonstrated a technique to prevent the spiking response depression. Q2 (Some figures in this article are in colour only in the electronic version)

Introduction as bright light spots (Gerding et al 2002, Rizzo et al 2003, Gekeler et al 2006, Fujikado et al 2007, Greenwald et al There has been a long-standing interest in restoring sight to the 2009, Horsager et al 2011). When driven by laboratory blind. Many research groups have been developing implants visual stimuli and natural scenes, retinal ganglion cells (RGCs) that target neurons in the retina. These implants operate are able to ﬁre spikes at frequencies above 200 Hz (Meister by activating the surviving neurons following the loss of 1999, Baccus 2007), and for some cells over 300 Hz under photoreceptors due to conditions such as retinitis pigmentosa. appropriate conditions (Zeck and Masland 2007). Thus, these Human trials over the last decade have demonstrated the ability high-frequency spike trains may be the fundamental coding of these retinal implants to elicit simple visual percepts such blocks with which the retina conveys visual information (Berry et al 1997). To maximize the quality of electrically evoked 4 Author to whom any correspondence should be addressed. percepts, retinal implants may need to activate these cells at

1741-2560/11/000000+12$33.00 1 © 2011 IOP Publishing Ltd Printed in the UK J. Neural Eng. 8 (2011) 000000 DTsaiet al similar frequencies. However, several investigations using pipette solution. Furthermore, starburst amacrine cells in the isolated retinas have found a decline in spiking response rate adult rabbit retina do not fire somatic action potentials (Zhou while electrically stimulating RGCs directly (as opposed to and Fain 1996, Zheng et al 2004). Together, these rule out indirect responses of synaptic origin) at 50 Hz (Sekirnjak the possibility of contamination by amacrine cells. Current et al 2006,Tsaiet al 2009). Similarly, others described failures clamp electrodes were filled with (mM) 120 KMeSO4,10KCl, to generate full action potentials (Cai et al 2011) and increasing 0.008 CaCl2, 0.5 EGTA, 1 MgCl2, 10 HEPES, 4 ATP-Na2 and RGC threshold (Freeman and Fried 2011) during repeated 0.5 GTP-Na3, adjusted to pH 7.2 with KOH. Final pipette stimulation. Recent human clinical studies have reported a resistances were 2.5–5.0 M. All chemicals were purchased frequency-dependent ‘fading’ of electrically evoked percepts from Sigma Aldrich. A 5 mV liquid junction potential (Zrenner et al 2010, Christopher et al 2010). In some subjects, correction has been applied for all results. For cell-attached the percepts lasted for less than a second. While the reason(s) recordings, the pipettes were filled with Ames’ medium. for the fading percepts is unknown, the spiking response A Multiclamp 700B (Molecular Devices) amplifier, Digidata depression observed in studies using isolated retinas suggests 1440A (Molecular Devices) and custom-written software were that retinal mechanisms could be responsible, at least in part, used for data acquisition. Data were low-pass filtered at for the response fading. This is particularly plausible given 10 kHz and digitized at 50 kHz. Current clamp series studies demonstrating use-dependent inactivation of voltage- resistance was compensated with the bridge balance control. gated sodium channels (Debanne 2004), which is the primary Voltage clamp series resistance was compensated by 33–50% mechanism behind RGC direct responses following electrical at 5–7 kHz bandwidth. stimulation (Fried et al 2009). In this study, we investigate the cause of RGC Data were analyzed in pClamp 10 (Molecular Devices), spiking response depression when these cells are electrically Matlab 2009 (Mathworks Inc.), Minitab 15 (Minitab Inc.) and stimulated at 50 Hz. We found the effects of retinal Prism 5.0a (GraphPad Software). The statistical significance network inhibition to be insignificant in suppressing RGC level was set at 95% (two-tailed). Data are presented as the ± spiking probability during repetitive stimulation. The evoked mean standard error of mean. RGC voltage-gated sodium current (INa), however, decreased exponentially. We then demonstrate that RGC spiking Pharmacology response rate decline was prevented by counteracting the decline of electrically evoked sodium current. Finally we show The AMPA/kainate, NMDA, mGluR6, GABAa/c and that under identical conditions, these RGCs were intrinsically glycine receptors were blocked with (in mM) 0.075 capable of high-frequency repetitive spiking and did so when 6-cyano-7-nitroquinoxaline-2,3-dione (CNQX), 0.06 (+)-MK stimulated by light. 801 hydrogen maleate (MK-801), 0.02 L-(+)-2-Amino-4- phosphonobutyric acid (L-AP4), 0.1 picrotoxin (pic) and Methods 0.01 strychnine (stry), respectively. The blocking efficacy was ascertained by the lack of light-evoked RGC responses. Preparation Voltage-gated sodium channels were blocked with 0.5 μM All procedures were approved and monitored by the Animal tetrodotoxin (TTX). Calcium currents were non-selectively μ Care and Ethics Committee at University of New South Wales. blocked with 250 MCdCl2. All drugs were bath-applied. NZ White rabbits were anesthetized intramuscularly with All pharmacological agents were purchased from either Sigma ketamine(70mgkg−1) and xylazine (10 mg kg−1), then Aldrich or Tocris Bioscience. euthanized with an intravenous overdose of pentobarbital. Inferior retinas, encompassing the visual streak, were Electrical stimulation isolated and kept in Ames’ Medium supplemented with 1% / penicillin streptomycin and saturated with 95% O2 and 5% We stimulated the isolated neural retinas at the photoreceptor CO2 at room temperature. The retinas were kept under this side (subretinal stimulation). The stimuli were delivered via a condition in darkness for 1–12 h. Prior to recording, a piece single 40 × 40 μm2 planar electrode in a multielectrode array of the retina was transferred RGC-side up to a fixed-stage (MEA; Ayanda Biosystems), with 60 platinum electrodes on upright microscope. The retinas were perfused continuously a glass substrate and arranged in a square grid with 200 μm with equilibrated Ames’ Medium warmed to 34–35 ◦C electrode center-to-center distance. The stimulus return was (pH 7.4) at ∼5mLmin−1. via eight distant electrodes at the MEA perimeter connected in parallel. Patch clamp electrophysiology Symmetric, charge-balanced, constant-current, biphasic, The RGCs were selected for recording by targeting cells rectangular pulses were generated from a custom-built with somatic diameter >10 μm and lying proximal to the stimulus generator. All biphasic pulses were cathodic-first inner limiting membrane (Vaney 1980). In most experiments, and without inter-phase delay. Here, we defined the pulse we also ascertained that the targeted cells were RGCs, and amplitude and duration as the height and width, respectively, not displaced starburst amacrine cells, by epi-fluorescence for one phase of the biphasic pulse. Each stimulus presentation microscopy of Alexa Fluor 488 (75 μM; Invitrogen) in the was separated by at least a 1 s delay.

2 J. Neural Eng. 8 (2011) 000000 DTsaiet al

(a) (b) 20 sweeps 20

10 Sweep

5 4 mV

0 20 40 60 80 100 0 20 40 60 80 100 Time (ms) Time (ms)

) 200 3 60 mV ) ( Action potential 2 40 litude litude A) p μ 20 Am 100 2 Spikelet 0 Failure APs Spikelets Threshold ( Charge Density (C/cm -64 mV 20 mV 0 1 0 0.5 1 25 ms Pulse Width (ms)

Figure 1. RCG spikes evoked by repetitive stimulation. (a) Superimposed cell-attached extracellular recordings of a RGC stimulated with a 66.6 Hz train consisting of four 100 μA 0.1 ms biphasic pulses. The stimulus delivery times are marked by the triangles. The spikes are easily distinguished from the stimulus artifacts (vertical lines). The ﬁrst sweep is denoted in black. (b) Raster plot for the spikes in (a). Marked in black are the short-latency spikes due to direct RGC stimulation. (c) In some cells, stimulation occasionally evoked spikelets with amplitude much smaller than that of full action potentials (p < 0.0001, t-test, n = 5 cells). Spikelets were apparent during whole-cell current clamp recordings. The stimuli were 0.1 ms at 110 μA. (d) Mean threshold current (red) that elicited a direct RGC spike and the equivalent charge density values (green) for evoking direct RGC responses.

Measuring RGC response rate depression strength dependent. Thus, when measuring INa,weused stimuli that are close to the average RGC threshold values We characterized RGC spiking response rate depression during (figure 1(d)). We used six consecutive 0.1 ms pulses at repetitive, direct activation by stimulating the cells with a train three pulse train frequencies (50, 100 and 200 Hz). Finally, of four identical biphasic pulses. We defined direct activation instead of counting spikes, whole-cell voltage clamp was used as RGC spiking responses (2 ms latency) that are resistant to to measure the evoked INa. The calcium-mediated currents synaptic input blocking, as determined in a previous study were blocked with 250 μMCdCl2 in the perfusion bath. The (Tsai et al 2009). We note that varying direct activation mean evoked INa for each pulse was determined by repeating latencies have been reported under different conditions and the same pulse train ten times. The I reduction rate was < Na stimulation configurations ( 0.7 ms in Fried et al;0.35msin described with a one-phase exponential decay equation: < Sekirnjak et al; 7 ms in Ahuja et al 2008). For each RGC, −kn we began by determining the stimulus threshold that evoked In = (1 − α) × e + α, a direct RGC response in approximately 80% of trials (over where In is the evoked INa for the nth pulse, relative to the INa 20 repetitions). All subsequent stimuli used this threshold for the first pulse (n = 0). Least-squares fitting was used to current. Four pulse train frequencies were tested (50, 66.6, determine the parameters α and k, which are the exponential 100, and 200 Hz). For each frequency, we tested three pulse decay plateau limit and rate constant, respectively. widths (0.1, 0.2 and 0.5 ms), for a total of 12 test combinations. To determine the mean spiking response rate for each stimulus Light stimulation of the pulse train, each combination was repeated at least ten times. RGCs were visualized with near-infrared (>820 nm) The INa decline rate during repetitive, direct RGC illumination. All recordings were performed under mesopic stimulation was determined analogously. Unlike the all-or- ambient lighting (∼7 cd.sr m−2) with light-adapted retinas. A none action potentials, the evoked INa amplitude is stimulus- stationary light spot from a white light-emitting diode behind

3 J. Neural Eng. 8 (2011) 000000 DTsaiet al a pinhole was focused onto the photoreceptor layer using the were not conventional RGC action potentials evoked by microscope’s 40× objective lens (Nikon NIR APO 0.8 N.A). intracellular depolarization (e.g. figure 7) and light stimulation The spot covered a 160 μm diameter circular area. We took (e.g. figure 8). Thus, in the responses described below, the soma as the center of the cell’s receptive field. To we excluded these spikelets from analysis. To facilitate characterize the response type (ON, OFF and ON-OFF) and comparison with existing studies, we also determined the the inter-spike intervals (ISIs) of light-evoked responses, we thresholds and charge densities for eliciting direct RGC spikes presented the cells with a 4 s light sequence consisting of ON- across a range of pulse widths (figure 1(d)) with the present OFF-ON-OFF, each of 1 s duration. This was repeated five 40 μm × 40 μm subretinal electrodes. times with 5 s delay between presentations. The mean light- evoked ISIs were calculated by averaging the ISIs across the RGC intrinsic properties underlie spiking response five repetitions. Only cells with a clearly defined response depression during repetitive direct stimulation classification (ON, OFF and ON-OFF) were included for analysis. To calculate the mode for each ISI, we fitted the RGCs are capable of high-frequency spiking when driven by duration distribution data for each ISI with a log-normal light. A number of previous prosthetic stimulation studies probability density function (PDF): have observed a decline in RGC spiking response rate during repetitive direct stimulation at 50 Hz. Electrical stimuli 1 −(ln x−μ)2 f(x|μ, σ) = √ e 2σ 2 . could activate amacrine cells (Fried et al 2006, Margalit and xσ 2π Thoreson 2006), in addition to RGCs; thus, inhibition from The parameters (μ and σ ) for each log-normal PDF were these cells could potentially underlie the response depression. determined by maximum likelihood estimation (Matlab). In all To test this hypothesis under a variety of stimulation cases, the goodness of fit was ascertained using the Anderson– configurations, we stimulated RGCs (n = 10) directly with Darling test (Stephens 1974; p 0.107). Finally, the mode a train of four identical biphasic pulses at four frequencies was then given by (50, 66.6, 100 and 200 Hz), and for each frequency at three μ−σ 2 pulse widths (0.1, 0.2 and 0.5 ms). We first stimulated RGCs e . under the control condition without the blockers (figure 2, red; left group). We then repeated the stimuli while blocking Results all presynaptic inputs (figure 2, green; right group). The differences in response rate across the four pulses were Depression of RGC spiking responses during repeated statistically significant, with or without blocking (0.1 ms: p = stimulation 0.001, 0.2 ms: p = 0.0003, 0.5 ms: p < 0.0001, repeated measure two-way ANOVA). More importantly, blocking the RGCs could be directly activated by extracellular electrical presynaptic inputs did not improve the RGC response rate stimulation, eliciting spikes with latency 2ms(Friedet al during repetitive stimulation (0.1 ms: p = 0.761, 0.2 ms: p = 2006, Sekirnjak et al 2006). Other neurons in the retinal 0.271, 0.5 ms: p = 0.810). These results suggest that inhibition network (e.g. bipolar cells and amacrine cells) could also from amacrine cells following electrical stimulation did not be activated by electrical stimulation. However, these cells play a significant role in suppressing the response rate of respond poorly to stimuli greater than a few Hz (Sekirnjak directly activated RGCs, at least not for the stimulus duration et al 2006, Jensen and Rizzo 2007, Ahuja et al 2008). used here. Figure 1(a) shows 20 superimposed cell-attached recordings of a cell stimulated directly with a train of four 0.1 ms biphasic Evoking I with extracellular stimulation while voltage pulses at 66.6 Hz. The spiking response rate decreased from Na clamping RGCs at rest 70% following the first pulse to 10–20% for the subsequent pulses (figure 1(b)). In the following work, we investigated the Voltage-gated sodium channels play an important role mechanisms underlying the decrease in RGC spiking response in determining neuronal excitability during extracellular rate during repetitive direct RGC stimulation. electrical stimulation. We therefore asked if these channels Electrical stimulation occasionally failed to elicit a full were responsible for the response depression observed during action potential in some RGCs. Instead, it evoked a small- high-frequency stimulation. Whole-cell voltage clamp with amplitude depolarization, which we will refer to as ‘spikelets’. simultaneous extracellular stimulation of presynaptic cells During whole-cell current clamp recordings, these spikelets has been used extensively to study excitatory and inhibitory were characteristically different from action potentials and postsynaptic currents (EPSCs and IPSCs). Before quantifying complete elicitation failures (figure 1(c)). In five RGCs (one the sodium current evoked by electrical stimulation, we began ON cell, two OFF cells and two ON-OFF cells; determined by verifying that the voltage clamp technique (clamping by light stimulation), where these responses were observed, at resting potential) could also be applied to quantify the spikelet mean amplitude was significantly smaller than RGC voltage-gated sodium current evoked by extracellular that of action potentials (64.6 ± 2.3 mV versus 24.0 ± stimulation. When RGCs were clamped at rest (n = 3, 2.7 mV, p < 0.0001, t-test). Spikelets are known features Vhold =−65 mV), electrical stimulation consistently evoked of neurons in other CNS regions, for instance in hippocampal a single inward current with fast activation kinetics neurons (Kandel and Spencer 1961) and in inferior (figure 3(a), red trace, arrow). The mean peak current olive neurons (Llinas et al 1974). However, these amplitude and latency (measured from the stimulus artifact

4 J. Neural Eng. 8 (2011) 000000 DTsaiet al

0.1 ms Pulse 0.2 ms Pulse 0.5 ms Pulse Control 50 Hz 1.0 1.0 1.0 66.6 Hz 0.8 0.8 0.8 100 Hz 200 Hz 0.6 0.6 0.6 Presyn. blocker 0.4 0.4 0.4 50 Hz 66.6 Hz

Spiking Probability Spiking 0.2 0.2 0.2 100 Hz 200 Hz 0.0 0.0 0.0 1st 2nd 3rd 4th 1st 2nd 3rd 4th 1st 2nd 3rd 4th Pulse Count Pulse Count Pulse Count Figure 2. Blocking presynaptic inputs did not improve RGC spiking response rate during repetitive stimulation. Ten RGCs were stimulated with a train of four biphasic pulses (50–200 Hz, 0.1–0.5 ms pulse width), and the probability of eliciting a spike through direct activation of the RGCs was determined over ten trials. For both the control condition without the blockers (red; left group) and with presynaptic input blocked (green; right group; CNQX + MK-801 + L-AP4 + pic + stry), the decline in response rates was signiﬁcant (0.1 ms: p = 0.001, 0.2 ms: p = 0.0003, 0.5 ms: p < 0.0001; repeated measure two-way ANOVA). However, blocking the presynaptic inputs did not improve the response rates (0.1 ms: p = 0.761, 0.2 ms: p = 0.271, 0.5 ms: p = 0.810). Thus, presynaptic inhibition did not play a signiﬁcant role in suppressing the response rates. In all cases, we used stimulus amplitudes that evoked a spike in approximately 80% of the trials.

(a)

1 nA 2 ms Vhold = -65 mV

Control (b) 4 ns Presyn. Block

Presyn + TTX 3 Washout 2 1 nA

1 1 ms Mean Evoked INa (nA) stimulus 0 Control Presyn. Block (c) Condition

ON OFF ON OFF

Presyn. Block Washout 100 pA

200 ms

Figure 3. Evoking INa with extracellular electrical stimulation while voltage clamping RGCs at rest. (a) While clamping a RGC at rest, a 130 μA 0.1 ms extracellular pulse evoked a short-latency inward current (red, arrow), which was not an EPSC by its lack of sensitivity to presynaptic input blockers: CNQX + MK-801 + L-AP4 + pic + stry (green). But it was abolished by TTX (blue), suggesting voltage-gated sodium channels as the carrier. Washout partially reversed the effect of TTX (gray). All four traces were the average of ten repetitions. Inset: the same traces over a longer duration. (b) The amplitude of the inward current was not affected by presynaptic input blockers (p = 0.3385, repeated measure two-way ANOVA, n = 3 cells). (c) The effectiveness of the presynaptic input blockers was ascertained by the lack of light responses after wash in (green). Light-evoked responses were restored after washout (gray). Both traces were the average of five repetitions. onset to the current peak) across the three cells were trace). The block was partially reversed after washing out for 2.83 ± 0.42 nA and 0.45 ± 0.003 ms, respectively. 20 min in this example (gray trace). These observations These characteristics are reminiscent of the sodium current suggest that the voltage-gated sodium channels were responsible for action potentials. TTX, a potent and specific responsible for the inward currents, ruling out the possibility of blocker for the voltage-gated sodium channels, completely oscillating stimulus artifacts or poorly compensated electrode eliminated the short-latency fast inward current in every and cell capacitive transients. Moreover, these inward currents case. This is demonstrated for one cell in figure 3(a) (blue always appeared as a single peak and were never followed by

5 J. Neural Eng. 8 (2011) 000000 DTsaiet al

400 pA

800 pA

1 ms 5 ms

Figure 4. Measuring RGC INa evoked by repetitive extracellular electrical stimulation. Ten superimposed traces of a RGC stimulated with a 100 Hz train of six 110 μA 0.1 ms biphasic pulses. The electrically evoked INa was clearly separated from the stimulus artifacts (insets). The INa amplitude declined with repeated stimulation. This consistently occurred across all ten trials.

any outward current (figure 3(a) red, green and gray traces), Table 1. Fitting the decline of INa during repetitive stimulation with which, had they occurred, would be indicative of insufficient a one-phase exponential decay function. voltage control and thus generation of action currents in Frequency (Hz) α K SS R2 unclamped regions. We next ascertained that the inward currents were not due 50 0.5043 0.2380 0.000 683 0.9922 to EPSCs from retinal network activation. In each cell, we 100 0.3354 0.2382 0.006 171 0.9630 200 0.4152 0.6607 0.003 124 0.9867 repeated the electrical stimulus while blocking all presynaptic inputs (CNQX + MK-801 + L-AP4 + pic + stry; figure 3(a), The parameters α and k are the plateau limit and rate constant of the green trace). There was no significant difference in the exponential decay function, respectively. The SS and R2 values are evoked current amplitude before and during blocking across the sum-of-squares and goodness-of-fit measures. the three cells (figure 3(b); p = 0.3385, repeated measure two-way ANOVA), indicating that EPSC did not contribute immediately following each stimulus artifact (figure 4 inset). to the short-latency fast inward current following electrical A decline in the evoked INa peak, relative to the first response, stimulation. Effective presynaptic input blocking was crucial was evident following the second stimulus, and by the sixth for this reasoning. This was confirmed by the absence of light- stimulus the evoked INa amplitude had fallen to 85.95 ± 0.01% evoked EPSC during drug perfusion (figure 3(c), green trace). relative to the first response. The light-evoked EPSCs subsequently returned upon washout We quantified the change in evoked I during repetitive (figure 3(c), gray trace). Na direct stimulation with 16 RGCs (figure 5). Consistent with the In summary, despite voltage clamping the RGCs near observations in figure 4, the decline in relative I following their resting potentials, extracellular electrical stimulation Na each pulse of the stimulus train was significant (p < 0.0001, could activate the voltage-gated sodium channels, resulting two-way ANOVA) and also increased at higher stimulation in large-amplitude I of several nA. In particular, the Na < inward currents were not stimulus artifacts, uncompensated frequencies (p 0.0001). The relative INa for the 100 Hz ± capacitive transients, artifacts of poor space clamp, nor stimuli (green) increased slightly at the end, from 0.54 0.03 ± EPSCs. at the fifth pulse to 0.58 0.03 at the sixth pulse. However, the difference was not significant (95% CI of difference =−0.14 to 0.06, Tukey’s post-test). The decline in relative INa during I declines during repetitive stimulation Na repetitive stimulation was well described by an exponential

Because RGC INa could be evoked by extracellular stimulation decay function (figure 5(a), dotted lines). The parameters and while voltage clamping the cells at rest, we next used this goodness-of-fit values are listed in table 1. Since the short- technique to investigate the effect of repetitive stimulation on latency stimulus-locked RGC spikes are due to activation of INa. Figure 4 shows ten superimposed current responses of a voltage-gated sodium channels (Sekirnjak et al 2006,Fried RGC stimulated with a train of six 0.1 ms pulses at 100 Hz et al 2006), the declining INa should therefore influence RGC (Vhold =−65 mV). The electrically evoked INa was apparent excitability during repetitive stimulation.

6 J. Neural Eng. 8 (2011) 000000 DTsaiet al

(a) (b) 50 Hz 50 Hz 100 Hz 200 Hz 1.0 1.0 1.0 1.0 100 Hz ON 200 Hz OFF 0.8 0.8 0.8 0.8

0.6 0.6 0.6 0.6 Relative Na Current Relative Na Current 0.4 0.4 0.4

0.4 0.2 0.2 0.2 1st 2nd 3rd 4th 5th 6th 1st 2nd 3rd 4th 5th 6th 1st 2nd 3rd 4th 5th 6th 1st 2nd 3rd 4th 5th 6th Pulse Count Pulse Count Pulse Count Pulse Count

Figure 5. INa decline during repetitive stimulation of RGCs (n = 16) with 0.1 ms biphasic pulses. (a) The decline in relative INa was significant (p < 0.0001, two-way ANOVA). The decline also increased with higher stimulation frequencies (p < 0.0001). The dotted line for each frequency represents the one-phase exponential decay fit for the relative INa over the six pulses. (b) Comparison of relative INa between ON (n = 5) and OFF (n = 8) RGCs. The decline in relative INa between the pulses was significant (50 Hz: p < 0.0019, 100 Hz: p < 0.0001, 200 Hz: p < 0.0001), but there was no difference between the ON and OFF RGCs (50 Hz: p = 0.4295, 100 Hz: p = 0.6762, 200 Hz: p = 0.2232).

Comparison between ON and OFF cells For each frequency, we used the rate constant (k) and plateau limit (α) determined previously (table 1). The evoked I While several lines of evidence suggest that the behavioral Na is assumed to increase linearly with stimulus strength in differences between the ON and OFF RGCs may be largely this equation. Figure 6(a) illustrates the relative scaling determined by the afferent inputs they receive (Pang et al of stimulus amplitude over the six repetitive pulses, after 2003, Murphy and Rieke 2006), Margolis and Detwiler substituting k and α into the modified exponential equation. (2007) recently reported intrinsic differences between ON The scaling increased with stimulus frequency. For ease of and OFF alpha RGCs in mice (see also Myhr et al 2001, discussion, we will refer to this procedure as ‘adaptive current Margolis et al 2008, Sekirnjak et al 2011). To check if scaling’ (ACS). When tested on four RGCs, the ACS stimulus there are differences in I decline rate between these two Na trains improved the spiking response rate during repetitive functionally distinct populations of RGCs following repetitive stimulation compared to the control stimulus trains. This electrical stimulation, we compared their relative evoked I Na observation held for all frequencies (figure 6(b ); 50 Hz: p = (figure 5(b), from the same group of cells as figure 5(a), 1 0.0004, 100 Hz: p = 0.0087, 200 Hz: p < 0.0001, two-way ON = 5 and OFF = 8). Consistent with the data from ANOVA, n = 4). Furthermore, the differences in response rate figure 5(a), the differences in relative I between the pulses Na between pulses in the ACS stimulus train were not significant were significant (50 Hz: p < 0.0019, 100 Hz: p < 0.0001, (figure 6(b ) green traces; 50 Hz: p = 0.7040, 100 Hz: p = 200 Hz: p < 0.0001) and increased with higher stimulation 1 0.8158, 200 Hz: p < 0.8771, one-way ANOVA). frequencies. There were more variations in the ON cell relative To quantify the effect of ACS on I , we again measured I , possibly due to the smaller sample size. But we found no Na Na the evoked I during repetitive stimulation while voltage significant difference between these two classes across all three Na clamping the RGCs (V =−65 mV). Comparing to stimulation frequencies (50 Hz: p = 0.4295, 100 Hz: p = hold the control pulse trains, the ACS pulse trains significantly 0.6762, 200 Hz: p = 0.2232). Therefore, any differences increased the relative I at all stimulation frequencies between the ON and OFF cells did not have appreciable effects Na (figure 6(c ); 50 ∼ 200 Hz: p < 0.0001, two-way ANOVA, on the response rates of these cells during repetitive electrical 1 n = 3). With increasing stimulus frequencies, ACS began to stimulation, at least in the rabbit retina using the stimulus overestimate the scaling required to counteract I decline, configurations tested here. Na as indicated by the relative INa values greater than unity (1.0). This is likely a consequence of the assumption that INa Increasing evoked INa prevents RGC spike response increases linearly with stimulus strength (see the discussion depression section). Nevertheless, these results strongly suggest that INa has a strong influence over RGC spiking response rate during A decline in INa during repetitive stimulation would reduce the RGCs’ excitability, thus decreasing their response rate to repetitive stimulation. A decrease in evoked INa was associated repetitive stimulation. Notwithstanding this inference, as a with a decline in spiking response rate, while increasing the evoked INa, by applying progressively larger stimulus further test, we reasoned that if INa reduction was the primary cause of RGC spiking response depression, then preventing amplitudes, prevented the spiking response rate decline. INa decline during repetitive stimulation should prevent the decrease of the response rate. Instead of using fixed pulse amplitude, we scaled the pulse amplitude to progressively RGC intrinsic and light-evoked spiking frequencies increase the driving force for I . We derived the scaling factor Na To ascertain that the stimulation frequencies used (50– (sf ) for each stimulus frequency by modifying the original one- 200 Hz) were within the RGC physiological operating range, phase exponential decay equation (see the methods section): and importantly, that the response depression was not simply −kn sf = 1+[1− ((1 − α) × e + α)]. an artifact of our ex vivo experimental conditions; we examined

7 J. Neural Eng. 8 (2011) 000000 DTsaiet al

(a) 50 Hz 100 Hz 200 Hz (b2) 1.6 1.6 1.6 ***** * * *

1.4 1.4 1.4

1.2 1.2 1.2 ACS

Relative Stim Amp Stim Relative 1.0 1.0 1.0 123456 123456 123456 Pulse Count Pulse Count Pulse Count * **** (b1) 2 mV 1.5 1.5 1.5 ACS Control 10 ms

1.0 1.0 1.0 Control

0.5 0.5 0.5 (c2) Relative Spike Rate Spike Relative

ACS 0.0 0.0 0.0 1st 2nd 3rd 4th 5th 6th 1st 2nd 3rd 4th 5th 6th 1st 2nd 3rd 4th 5th 6th Pulse Count Pulse Count Pulse Count (c1) * 5 5 5 ACS Control 4 4 4 * * * 3 3 3 * 1 nA * 10 ms 2 2 2 Relative INa Relative

1 1 1 Control ** 0 0 0 1st 2nd 3rd 4th 5th 6th 1st 2nd 3rd 4th 5th 6th 1st 2nd 3rd 4th 5th 6th * Pulse Count Pulse Count Pulse Count * * *

Figure 6. Increasing the evoked INa reduced spiking response depression. To facilitate comparison, all measurements were normalized relative to the response of the first stimulus. (a) Progressive increase in stimulus amplitude by ACS. (b1) Comparing to the control, the spiking rates were i by ACS (50 Hz: p = 0.0004, 100 Hz: p = 0.0087, 200 Hz: p < 0.0001, two-way ANOVA, n = 4 cells). The differences in response rates between the pulses in the ACS stimulus trains were insignificant (50 Hz: p = 0.7040, 100 Hz: p = 0.8158, 200 Hz: p = 0.8771, one-way ANOVA). As in figure 2, we used for the first pulse, stimulus amplitudes that evoked a spike in approximately 80% of trials. (b2) Spiking responses (stars) of an RGC to a train of six 100 Hz pulses, with (green) and without (red) ACS. (c1) ACS increased the relative INa when compared to control (50–200 Hz: p < 0.0001, two-way ANOVA, n = 3 cells). (c2) Electrically evoked INa (marked by stars) of a RGC to six 100 Hz pulses with (green) and without (red) ACS. the RGCs’ ability to spike at high frequency using two conditions RGC ISIs not only depend on their intrinsic techniques. properties but also on the properties and connections of the First we measured the ISIs evoked by injecting a 500 ms, presynaptic pathway. Therefore, we next measured the RGC 180 pA depolarizing pulse during current clamp. RGCs ISIs during light stimulation. responded to the depolarization with a burst of action In figure 8(a), an RGC fired a volley of action potentials potentials. Consistent with previous findings (O’Brien et al in response to a 1 s light stimulus (first sweep in black). The 2002), almost all RGCs tested (n = 43/49) repeatedly spikes from all repetitions are plotted together in the raster plot fired action potentials during the entire depolarizing pulse below the membrane potential traces. We presented the light (figure 7(a)). A few RGCs (n = 6/49) showed accommodation stimuli to 45 RGCs. The ISIs were strongly biased toward after a few spikes (figure 7(b)) and thus were unable to sustain short durations (figure 8(b)), and could be as short as 2 ms repetitive spiking for prolonged periods. Nevertheless, to throughout the first six ISIs. To more precisely quantify the gain a quantitative measure of the firing frequencies across ISIs, we fitted the distribution for the first six ISIs with a log- our RGC sample for comparison to repetitive electrical normal PDF and calculated the mode for each ISI. The mode stimulation, we determined the duration of the first six ISIs (marked with ‘x’ in figure 8(b)) was lowest for the first ISI during depolarizing pulse injection. When the cells were at 4.20 ms and highest at 5.22 ms for the sixth ISI. These depolarized with a 180 pA pulse (figure 7(c)), the median modes approximately correspond to spiking frequencies of durations (marked with ‘x’) of the first six ISIs ranged from 192–238 Hz. To provide an indication for the spikes elicited 6.82 ms for the first ISI to 14.34 ms for the last ISI. These values by the 1 s light stimulus in these RGCs, we also plotted the correspond to spiking frequencies 70–147 Hz. To examine elicitation probability for the first six ISIs (figure 8(c)). The the effects of depolarization amplitude, we tested stronger probability of having the fifth spike was 0.688. Therefore, depolarization at 320 pA (n = 26 cells). The durations of 68.8% of the cells spiked at least six times when stimulated by the first six ISIs decreased with stronger depolarization. The the light for 1 s. median duration for the first ISI was 5.17 ms and 10.27 ms by In summary, we evaluated the RGCs’ ability to spike at the sixth ISI (figure 7(d)), corresponding to spiking frequencies high frequencies with two complementary techniques. Despite 97–193 Hz. some variability from cell to cell, possibly due to variations While RGCs were generally capable of repetitive firing between different RGC classes, most cells were intrinsically at 50–200 Hz upon depolarization by intracellular current capable of high-frequency spiking for several consecutive injection, the ISI durations were pulse-strength dependent. spikes and did so when driven by light stimuli. Therefore, the Thus, the question arises whether the depolarizing pulse we response depression observed during repetitive extracellular injected was physiologically relevant, since under normal stimulation at 50–200 Hz was unlikely to be an artifact of the

8 J. Neural Eng. 8 (2011) 000000 DTsaiet al

(a) (c) Icmd = 180 pA 40 8 35

20 4 Count 15 Duration (ms)

-68 mV 10 180 pA 5 0 1 2 3 4 5 6 ISI #

20 mV (b) (d) Icmd = 270 ~ 320 pA 40 100 ms 8 35

20 4 Count 15 Duration (ms)

-72 mV 10 180 pA 5 0 1 2 3 4 5 6 ISI # Figure 7. RGC ISIs during depolarizing pulse injection. (a) Most RGCs (n = 43/49) spiked continuously throughout the duration of the 180 pA depolarizing pulse. (b) A small number of cells (n = 6/49) showed spike accommodation. (c) Duration distribution summary of the first six ISIs of 49 RGCs to 180 pA pulse injection. The median duration for each ISI (marked by ‘x’) ranged from 6.82 ms for the first ISI to 14.34 ms by the last ISI. The dashed line designates the 5 ms time point. (d) Duration distribution summary for 26 of the 49 RGCs, where we also tested 320 pA pulse injection. The medians decreased to 5.17 and 10.27 ms for the first and last ISI, respectively. ex vivo conditions. Failure of electrical stimulation to reliably 1.2 sodium channel subtypes (Lai and Jan 2006), which are elicit spikes at similar frequencies may thus have important thought to be responsible for action potential initiation (Kole consequences on the perceptual efficacy of electrically evoked et al 2008,Huet al 2009). This site also has the percepts. lowest threshold for spike generation during extracellular electrical stimulation (Fried et al 2009; see also Jensen et al 2003; Sekirnjak et al 2008). Therefore, these two sodium Discussion channel subtypes likely play a critical role in determining RGC electrically evoked responses. The present results We showed that the spiking response depression of directly are consistent with previous findings of sodium channel activated RGCs during electrical stimulation was not mediated inactivation following repeated depolarization (Bishop et al by retinal network inhibition. Instead, INa played a key role 1959, Sefton and Swinburn 1964, Tsutsui and Oka 2002, in determining RGC spiking probability. Under identical Debanne 2004). To avoid these conditions, retinal implants conditions, RGCs were intrinsically capable of high-frequency may need to adopt a stimulation strategy consisting of brief spiking and did so when driven by light. A decline in INa stimulus bursts interleaved by periods of inactivity, a pattern was associated with decreasing RGC spiking response rate. akin to light-evoked RGC responses under physiological Conversely, counteracting the INa decline with progressively conditions (Meister 1999). stronger stimuli prevented the response depression. INa inactivation may not be the only mechanism responsible for spiking response depression. Some factors implicated in use-dependent failure of axonal Underlying causes for INa decline spikes and action potential propagation include activity- In this study, we demonstrated a causal relationship between dependent hyperpolarization by calcium-mediated potassium INa decline and spiking response depression. The question conductance (IK(Ca); Bielefeldt and Jackson 1993), thus arises regarding the underlying mechanisms responsible extracellular potassium accumulation following repeated for the electrically evoked decline of INa. The RGC axon initial depolarization (Grossman et al 1979) and modulation by segment (AIS) contains a high density of Nav 1.6 and Nav the hyperpolarization-activated non-selective cationic current

9 J. Neural Eng. 8 (2011) 000000 DTsaiet al

(a) (b) 80 ON

20 mV 60 400 ms

8 -68 mV 50

3 40 4 Count Sweep

1 Duration (ms)

30 0 (c) 1 0.688

0.8 20

0.6

0.4 10 Probability

0.2

0 1 2 3 4 5 6 1 2 3 4 5 6 ISI # ISI # Figure 8. RGC ISIs during 1 s light stimulation. (a) Five superimposed responses of an ON-type RGC. The cell fired a train of action potentials on a sustained depolarization. The first trial is in black. The raster plot (below) shows the spike times for all five trials. (b) Duration distribution for the first six ISIs of 45 RGCs to light stimulation. The mode (marked by ‘x’) ranged from 4.20 ms for the first ISI to 5.22 ms for the sixth ISI. The dashed line designates the 5 ms time point. (c) The elicitation probability of the first six ISIs for the RGCs in (b), 68.8% of the cells spiked at least six times when stimulated by the light.

(Ih; Soleng et al 2003). Since the AIS, and thus more generally Why does prosthetic stimulation appear less effective than the axon, is targeted by electrical stimulation, these factors, if light at driving high-frequency spikes? present in RGC axons, could also contribute to the response As a mechanism intrinsic to RGCs, the declining I would depression observed. Na affect light-evoked responses as much as electrically evoked responses. Why then are the cells able to fire reliably at high frequencies (50–200 Hz) for several spikes or more during light Spiking response depression during repetitive stimulation (figure 8), while extracellular electrically evoked stimulation—does it matter? responses showed an immediate decline in spiking probability Some RGCs may be inherently unable to spike repetitively at stimulation frequency as low as 50 Hz? Two observations at high frequencies. During intracellular depolarizing current could reconcile this disparity. First, there are mechanistic injection, accommodation occurred after a few spikes in a differences between light-evoked and electrically evoked RGC small number (n = 6/49) of cells. It would be inappropriate responses. Light stimuli activate (in the following order) the ligand-gated currents, the voltage-gated currents and finally to expect these cells to follow high-frequency electrical sodium spikes at the AIS. Activating the ligand-gated and the stimulation beyond a few pulses. While we did not voltage-gated currents could result in sustained depolarization, morphologically identify these cells, the spiking patterns for instance, by virtue of the NMDA receptors (Copenhagen et al (figure 7(b)) were very similar to the theta cells in O’Brien 2003) and L-type calcium channels (Ishida 2003) present in (2002). Nevertheless, consistent with previous reports (Berry RGCs, respectively. This sustained depolarization plateau (see and Meister 1998, Zeck and Masland 2007), most RGCs figure 8(a)) could provide sufficient drive for repeated spike (∼88%) were intrinsically capable of high-frequency spiking generation even in the presence of gradual INa inactivation, when presented with appropriately strong stimuli. Given the a situation similar to repeated spiking through depolarizing high temporal precision and trial-to-trial reproducibility of current injection (figure 7). In contrast, electrical stimulation light-evoked RGC spikes (Meister and Berry 1999, Baccus of RGCs recruits the INa directly (Fried et al 2006, Sekirnjak 2007, Gollisch and Meister 2010), a reduction in spike et al 2006) and bypassing the ligand-gated currents entirely. elicitation reliability during repetitive electrical stimulation The sustained voltage-gated currents, with slower activation will likely have important ramifications for the perceptual kinetics than INa, may be poorly recruited by the short- efficacy of retinal prostheses. duration (0.1–0.5 ms) biphasic pulses. The lack of sustained

10 J. Neural Eng. 8 (2011) 000000 DTsaiet al depolarization thus comparatively decreases the excitability of Comparison to recent reports RGCs, especially with concomitant INa inactivation. Second, it is worthwhile comparing the definition of threshold for There are conflicting reports regarding the extent to which light-evoked and electrically evoked responses. In the period RGCs could follow high-frequency extracellular pulse trains following a visual event in which a RGC fires a burst of action (Sekirnjak et al 2006,Tsaiet al 2009, Ahuja et al 2008). In potentials, the EPSP is sufficiently powerful and sustained to an attempt to reconcile these discrepancies, a recent report by ensure high-frequency spiking. However, the definition of Cai et al (2011) found that, using extracellular recordings, threshold for electrical stimulation is typically defined as the when a pulse in a high-frequency stimulus train failed to current amplitude that elicits spike(s) in some percentage of the elicit an action potential, a small-amplitude biphasic response trials (often 50%). This definition, in effect, sets a low safety was often observed. If these small-amplitude responses were margin for spike elicitation in the presence of INa inactivation. included, then some RGCs could follow stimulus rates up Given prior studies with similar conclusions (e.g. to 600 Hz. These small-amplitude responses are likely the Sekirnjak et al 2006,Tsaiet al 2009), it seems unlikely spikelets we observed during whole-cell patch recordings that the stimulation configuration was responsible for the (figure 1(c)). These spikelets are presumably the AIS spikes response depression. For instance, Sekirnjak and colleagues that failed to result in full action potentials (Cai et al 2011,Hu used epiretinal stimulation with pulse configurations different et al 2009). The ability of RGCs to following high-frequency from the present study, and in a previous study, we used a pulse trains ultimately depends on the responses, whether full single monopolar electrode rather than a multielectrode array. action potentials or spikelets, propagating down the axon to Furthermore, a range of species have been used across these higher visual centers. It is not clear if the spikelets could studies (rabbits, mice, rat and primates). do so successfully, because they are not observed following light stimulation or intracellular depolarizing pulse injection. Preventing RGC spiking response depression Nonetheless, the ACS technique presented here could be used to circumvent potential spikelet propagation failures by The present ACS implementation assumed the evoked INa to scale linearly with stimulus amplitude. Two observations eliciting full RGC action potentials instead. suggest that this approximation overestimated the stimulus increase needed for counteracting INa decline. First, as we Acknowledgments increased the scaling factor with stimulus frequency, it was evident in figure 6(c) that the relative differences in evoked We thank Professor William R Levick (Australian National INa between the control pulse trains and the ACS pulse trains University) for discussion. This research was supported in part also increased. Second, previous reports have found RGC by an Australian Research Council Special Research Initiative spike elicitation probability to scale in a sigmoidal fashion in Bionic Vision Technologies. with stimulus amplitude (Sekirnjak et al 2006,Tsaiet al 2009,Friedet al 2009). Indeed, the INa voltage-dependent activation and inactivation kinetics are well described by the References Q3 Boltzmann equation, not the linearity assumed in the modified one-phase exponential equation. ACS prevented the response Ahuja A K, Behrend M R, Kuroda M, Humayun M S and depression in the isolated retina. A similar methodology Weiland J D 2008 An in vitro model of a retinal prosthesis IEEE Trans. Biomed. Eng. 55 1744–53 may be useful for preventing fading responses in clinical Baccus S A 2007 Timing and computation in inner retinal circuitry applications. While the overestimation here is inconsequential Annu. Rev. Physiol. 69 217–90 for demonstrating the role of INa in RGC response depression, Berry M J and Meister M 1998 Refractoriness and neural precision stimulus amplitude overestimation in clinical applications does J. Neurosci. 18 2200–11 reduce the safety margin to the electrode charge injection Berry M J, Warland D K and Meister M 1997 The structure and limit, which if exceeded, will reduce the electrode lifetime precision of retinal spike trains Proc. Natl Acad. Sci. USA 94 5411–16 (Merrill et al 2005). In this study, we determined the scaling Bielefeldt K and Jackson M B 1993 A calcium-activated potassium function at fixed stimulation frequencies. Retinal implants channel causes frequency-dependent action-potential failures will likely need dynamically changing stimulation frequencies, in a mammalian nerve terminal J. Neurophysiol. 70 284–98 as determined by the visual inputs. Further work will be Bishop P O, Burke W and Hayhow W R 1959 Repetitive stimulation necessary to derive the general form for the scaling function of optic nerve and lateral geniculate synapses Exp. Neurol. under such conditions. 1 534–55 Cai C, Ren Q, Desai N, Rizzo J and Fried S 2011 Response The causes of fading percepts in human subjects are variability to high rates of electric stimulation in retinal likely multifaceted. In a small number of patients, percepts ganglion cells J. Neurophysiol. Q4 disappeared in less than a second when stimulating at greater Christopher P, McMahon M J, Patel U, Wei J, Greenberg R J than a few Hz (Christopher et al 2010, Zrenner et al 2010). and Dorn J D 2010 The temporal dynamics of percepts The findings made in the isolated retinal studies, where RGC generated by the ArgusTM II epiretinal prosthesis spiking response rate declined when stimulating at >50 Hz, Neuroscience Meeting Planner, Program no 20.3 2010 (San Diego, CA: Society for Neuroscience) online Q5 do not account for the severe fading in these individuals. Copenhagen D 2003 Excitation in retina: the flow, filtering, and Additional factors such as the degenerate retina and visual molecules of visual signalling in the glutamatergic pathways cortical activities may have added further complexities to the from photoreceptors to ganglion cells The Visual Neurosciences issue. ed L M Chalupa and J S Werner (Cambridge, MA: MIT Press)

11 J. Neural Eng. 8 (2011) 000000 DTsaiet al Debanne D 2004 Information processing in the axon Nat. Rev. Margolis D J, Newkirk G, Euler T and Detwiler P B 2008 Functional Neurosci. 5 304–16 stability of retinal ganglion cells after degeneration-induced Freeman D K and Fried S I 2011 Multiple components of ganglion changes in synaptic input J. Neurosci. 2008 6526–36 cell desensitization in response to prosthetic stimulation Meister M and Berry M J 1999 The neural code of the retina Neuron J. Neural Eng. 8 016008 22 435–50 Fried S I, Hsueh H A and Werblin F S 2006 A method for Merrill D R, Bikson M and Jefferys J G R 2005 Electrical generating precise temporal patterns of retinal spiking using stimulation of excitable tissue: design of efficacious and safe prosthetic stimulation J. Neurophysiol. 95 970–8 protocols J. Neurosci. Methods 141 171–98 Fried S I, Lasker A C W, Desai N J, Eddington D K and Murphy G J and Rieke F 2006 Network variability limits Rizzo J F III 2009 Axonal sodium channel bands shape the stimulus-evoked spike timing precision in retinal ganglion cells response to electric stimulation in retinal ganglion cells Neuron 52 511–24 J. Neurophysiol. 101 1972–87 Myhr K L, Lukasiewicz P D and Wong R O L 2001 Mechanisms Fujikado T et al 2007 Evaluation of phosphenes elicited by underlying developmental changes in the firing patterns of ON extraocular stimulation in normals and by and OFF retinal ganglion cells during refinement of their suprachoroidal-transretinal stimulation in patients with retinitis central projections J. Neurosci. 21 8664–71 pigmentosa Graefes Arch. Clin. Exp. Ophthalmol. O’Brien B J, Isayama T, Richardson R and Berson D M 2002 245 1411–9 Intrinsic physiological properties of cat retinal ganglion cells Gekeler F, Messias A, Ottinger M, Bartz-Schmidt K U and Zrenner J. Physiol. 538 787–802 E 2006 Phosphenes electrically evoked with DTL electrodes: a Pang J J, Gao F and Wu S M 2003 Light-evoked excitatory and Q6 study in patients with retinitis pigmentosa, glaucoma, and inhibitory synaptic inputs to ON and OFF α ganglion cells in homonymous visual field loss and normal subjects Invest. the mouse retina J. Neurosci. 22 6063—73 Ophthalmol. Vis. Sci. 47 4966–74 Rizzo J F, Jensen R J, Loewenstein J and Wyatt J 2003 Methods and Gerding H, Eckmiller R E, Hornig R, Ortmann V, Kolck A perceptual thresholds for short-term electrical stimulation of and Taneri S 2002 Safety assessment and acute clinical tests of human retina with microelectrode arrays Invest. Ophthalmol. epiretinal retina implants Invest. Ophthalmol. Vis. Sci. 43 4488 Vis. Sci. 44 5355–61 (E-Abstract) Sefton A J and Swinburn M 1964 Electrical activity of lateral Gollisch T and Meister M 2010 Eye smarter than scientists believed: geniculate nucleus and optic tract of the rat Vision Res. 4 315–28 neural computations in circuits of the retina Neuron 65 150–64 Sekirnjak C, Hottowy P, Sher A, Dabrowski W, Litke A M Greenwald S H, Horsager A, Humayun M S, Greenberg R J, and Chichilnisky E J 2006 Electrical stimulation of mammalian McMahon M J and Fine I 2009 Brightness as a function of retinal ganglion cells with multielectrode arrays current amplitude in human retinal electrical stimulation Invest. J. Neurophysiol. 95 3311–27 Ophthalmol. Vis. Sci. 50 5017–25 Sekirnjak C, Hottowy P, Sher A, Dabrowski W, Litke A M Grossman Y, Parnas I and Spira M 1979 Ionic mechanisms involved and Chichilnisky E J 2008 High-resolution electrical in differential conduction of action potentials at high frequency stimulation of primate retina for epiretinal implant design in a branching axon J. Physiol. 295 307–22 J. Neurosci. 28 4446–56 Horsager A, Boynton G M, Greenberg R J and Fine I 2011 Temporal Sekirnjak C, Jepson L H, Hottowy P, Sher A, Dabrowski W, interactions during paired-electrode stimulation in two retinal Litke A M and Chichilnisky E J 2011 Changes in physiological prosthesis subjects Invest. Ophthalmol. Vis. Sci. 52 549–57 properties of rat ganglion cells during retinal degeneration Hu W, Tian C, Li T, Yang M, Hou H and Shu Y 2009 Distinct J. Neurophysiol. 2011 2560–71 contributions of Nav1.6 and Nav1.2 in action potential Soleng A, Chiu K and Raastad M 2003 Unmyelinated axons in the initiation and backpropagation Nat. Neurosci. 12 996–1004 rat hippocampus hyperpolarize and activate an H current when Ishida A T 2003 Retinal ganglion cell excitability The Visual spike frequency exceeds 1 Hz J. Physiol. 552 459–70 Neurosciences ed L M Chalupa and J S Werner (Cambridge, Stephens M A 1974 EDF statistics for goodness of fit and some MA: MIT Press) comparisons J. Am. Stat. Assoc. 69 730–7 Jensen R J and Rizzo J F III 2007 Responses of ganglion cells to Tsai D, Morley J W, Suaning G J and Lovell N H 2009 Direct repetitive electrical stimulation of the retina J. Neural Eng. activation and temporal response properties of rabbit retinal 4 1–6 ganglion cells following subretinal stimulation J. Neurophysiol. Jensen R J, Rizzo J F, Ziv O R, Grumet A and Wyatt J 2003 102 2982–93 Thresholds for activation of rabbit retinal ganglion cells with Tsutsui H and Oka Y 2002 Slow removal of Na+ channel an ultrafine, extracellular microelectrode Invest. Ophthalmol. inactivation underlies the temporal filtering property in the Vis. Sci. 44 3533–43 teleost thalamic neurons J. Physiol. 539 743–53 Kandel E R and Spencer W A 1961 Electrophysiology of Vaney D 1980 A quantitative comparison between the ganglion cell hippocampal neurons: IV. Fast prepotentials J. Neurophysiol. populations and axonal outflows of the visual streak and 24 272–85 periphery of the rabbit retina J. Comp. Neurol. 189 215–33 Kole M H P, Ilschner S U, Kampa B M, Williams S R, Ruben P C Zeck G M and Masland R H 2007 Spike train signatures of retinal and Stuart G J 2008 Action potential generation requires a high ganglion cell types Eur. J. Neurosci. 26 367–80 sodium channel density in the axon initial segment Nat. Zheng J-J, Lee S and Zhou Z J 2004 A developmental switch in the Neurosci. 11 178–86 excitability and function of the starburst network in the Lai H C and Jan L Y 2006 The distribution and targeting of neuronal mammalian retina Neuron 44 851–64 voltage-gated ion channels Nat. Rev. Neurosci. 7 548–62 Zhou Z J and Fain G L 1996 Starburst amacrine cells change from Llinas R, Baker R and Sotelo C 1974 Electrotonic coupling between spiking to nonspiking neurons during retinal development neurons in cat inferior olive J. Neurophysiol. 37 560–71 Proc. Natl Acad. Sci. USA 1996 8057–62 Margalit E and Thoreson W B 2006 Inner retinal mechanisms Zrenner E, Benav H, Bruckmann A, Greppmaier U, Kusnyerik A, engaged by retinal electrical stimulation Invest. Ophthalmol. Stett A, Stingl K and Wilke R 2010 Electronic implants Vis. Sci. 47 2606–12 provide continuous stable percepts in blind volunteers only if Margolis D J and Detwiler P B 2007 Different mechanisms generate the image receiver is directly linked to eye movement 2010 maintained activity in ON and OFF retinal ganglion cells ARVO Meeting Planner, Program no 444 (Fort Lauderdale, FL: J. Neurosci. 27 5994–6005 ARVO) (online)

12 QUERIES

Page 1 Q1 Author: The third afﬁliation address has not been linked with any author. Please check.

Q2 Author: Please be aware that the colour ﬁgures in this article will only appear in colour in the Web version. If you require colour in the printed journal and have not previously arranged it, please contact the Production Editor now. If you choose not to pay for colour printing, please amend reference to colour in the text and in the captions to ﬁgures.

Page 11 Q3 Author: Please check the details for any journal references that do not have a blue link as they may contain some incorrect information. Pale purple links are used for references to arXiv e-prints.

Q4 Author: Please provide the volume number and page number(s) in reference [7].

Q5 Author: Please check whether references [8] and [51] are okay as set.

Page 12 Q6 Author: Please check whether the year is okay as corrected in reference [37]. Chapter 4

Real-time image processing and stimulus coding

This chapter contains the published article: Tsai D, Morley J W, Suaning G J, Lovell N H. A wearable real-time image processor for a vision prosthesis. Computer Methods and Programs in Biomedicine. 2009(95): 258-269.

Current status: Published.

Author contributions: The contribution of D. T. to this paper was 85%, consisting of designing and developing the system and writing the manuscript. computer methods and programs in biomedicine 95 (2009) 258–269

journal homepage: www.intl.elsevierhealth.com/journals/cmpb

A wearable real-time image processor for a vision prosthesis

D. Tsai a, J.W. Morley b,c, G.J. Suaning a, N.H. Lovell a,∗ a Graduate School of Biomedical Engineering, University of New South Wales, Sydney, NSW 2052, Australia b School of Medicine, University of Western Sydney, Sydney, NSW 1797, Australia c School of Medical Sciences, University of New South Wales, Sydney, NSW 2052, Australia article info abstract

Article history: Rapid progress in recent years has made implantable retinal prostheses a promising ther- Received 3 July 2008 apeutic option in the near future for patients with macular degeneration or retinitis Received in revised form pigmentosa. Yet little work on devices that encode visual images into electrical stimuli have 10 December 2008 been reported to date. This paper presents a wearable image processor for use as the exter- Accepted 13 March 2009 nal module of a vision prosthesis. It is based on a dual-core microprocessor architecture and runs the Linux operating system. A set of image-processing algorithms executes on the dig- Keywords: ital signal processor of the device, which may be controlled remotely via a standard desktop Embedded image processing computer. The results indicate that a highly ﬂexible and conﬁgurable image processor can Retinal prosthesis be built with the dual-core architecture. Depending on the image-processing requirements, Bionic eye general-purpose embedded microprocessors alone may be inadequate for implementing Macular degeneration image-processing strategies required by retinal prostheses. Retinitis pigmentosa Crown Copyright © 2009 Published by Elsevier Ireland Ltd. All rights reserved.

1. Introduction from the above that the image processor plays an integral role in converting the visual scene into a representation for mod- Several research groups are developing retinal prostheses as ulating electrical stimuli. Such a device is not unique to the a potential treatment for retinal degenerative diseases such described epi-retinal prosthesis. Vision prostheses proposed as retinitis pigmentosa and age-related macular degeneration by several other groups also necessitate the use of an image [1–4]. Many proposed designs require an image-processing processor. Notwithstanding its importance, little work has device for acquiring images from a camera and transforming been reported on the hardware and software for such devices. them into stimulus commands to conﬁgure the stimulation The design of such an image processor presents consid- to be delivered by the implant. Details of the epi-retinal pros- erable challenges. The device should provide support for all thesis being developed in the authors’ laboratory have been essential activities that vision impaired subjects may need reported previously [5]. In brief, a camera worn by a patient during their daily life, including: navigation [6], object recogni- captures an image, sends it to a portable processor for decom- tion [7], facial recognition [8] and even reading [9–11]. However, posing into stimulus command sequences, which are then vision prostheses of the near future are expected to contain transmitted to an implanted device located within the eye via only limited numbers of electrodes, thus constraining the a transcutaneous radio frequency (RF) link. Electronics in the visual resolution implant recipients are likely to perceive. Psy- implant decode the signal and use the incipient energy of the chophysical studies of simulated pixelized vision on sighted transmitter to activate the microelectrodes (Fig. 1). It is clear patients suggest that non-trivial image processing may be

∗ Corresponding author at: Graduate School of Biomedical Engineering, University of New South Wales, Sydney 2052, Australia. Tel.: +61 2 9385 3922; fax: +61 2 9663 2108. E-mail address: [email protected] (N.H. Lovell). 0169-2607/$ – see front matter. Crown Copyright © 2009 Published by Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.cmpb.2009.03.009 computer methods and programs in biomedicine 95 (2009) 258–269 259

whereas the image processor can be built from commercially available components, many of which are already optimized for performance, power efficiency and size, savings in both hardware and software development time and effort will be significant. An external image processor is also more amend- able to upgrades. In this paper we present a wearable image processor based on a Texas Instruments OMAP processor running a cus- tomized version of Linux. A set of purpose-built software libraries and programs operate on the device to acquire images from the camera in real-time, perform image processing, and produce outputs, which can then be sent to a transmitter for subsequent delivery to the implant via an RF link. The software components have been carefully designed to provide real-time performance. The image processor is a stand-alone device capable of independent operation while being worn by a patient. It supports Bluetooth technology so that subjects can control various image-processing parameters using other Bluetooth-enabled devices. A complete suite of remote con- Fig. 1 – Components of a vision prosthesis. The image figuration tools have also been incorporated into the system, processor encodes images from the camera into stimulus allowing researchers and clinicians to control the image pro- commands and transmits them to the retinal implant via a cessor as well as to experiment with various settings without wireless RF link. The remote supervising tool allows physically accessing the device. Our work also indicates that clinicians and researchers to configure the image processor depending on the image-processing algorithms involved, the using a separate computer. hardware platform used needs to be selected carefully in order to attain real-time performance. required in order to maximally utilize the limited number of 2. The image processor available electrodes [12–14]. Additionally, the image processor hardware needs to be portable yet able to perform possi- 2.1. Hardware bly complex image-processing algorithms in real-time. The software on the device needs to be optimized for efficiency The hardware was manufactured by Spectrum Digital Incor- in power consumption and execution speed. Image acquisi- porated (Stafford, TX, USA) and subsequently modified in tion and processing parameters should be easily configured to our laboratory for further functionality. The device compo- adapt to individual requirements of the patients. It has also nents relevant to the current discussion are summarized in been suggested that devices to be used in preliminary human Fig. 2. At the heart of the image processor is an OMAP5912 trials should be equipped with a number of image-processing microprocessor (Texas Instruments, Dallas, TX, USA), which is strategies in order to validate the efficacy of each [15]. There- primarily designed for embedded multimedia systems such as fore it is prudent to have a scalable software architecture to personal digital assistants (PDAs), or portable medical devices. support changes as the need arises. The OMAP5912 has a dual-core architecture comprised of an Despite the aforementioned design considerations, only ARM9TDMI general-purpose reduced instruction set micro- a few studies have hitherto addressed the issues. Buffoni processor designed by ARM Limited (Cambridge, UK) and a et al. [15] reviewed psychophysical findings of pixelized vision, discussed the key constraints of image processing for vision prostheses and described a number of possible image-processing strategies. More recently, as part of their implementation of a sub-retinal vision prosthesis, Asher et al. [16] have presented a set of software algorithms for tracking, cropping, geometrically transforming, and filtering images from a camera into stimulation commands. There are several advantages associated with stand-alone wearable image processors. Powerful hardware may be essential for executing the algorithms involved. This in turn leads to more complex circuitry, higher power consumption, bigger physical dimensions and higher heat dissipation, all of which are at odds with the requirements of an implantable device. Fig. 2 – System hardware of the image processor. By delegating image-processing tasks to an external proces- Peripherals are connected through USB. The image sor the implant is freed from these additional complications. processor can communicate with other devices via Furthermore, given that the implant will most certainly be a Ethernet, RS232 and Bluetooth. Furthermore, flash memory custom-designed application specific integrated circuit (ASIC), and SDRAM are available for data storage. 260 computer methods and programs in biomedicine 95 (2009) 258–269

latter runs on a remote computer (e.g. a laptop), while the former two programs run on the image processor itself. The two cores on the OMAP have different instruction set archi- tectures and hence separate binary executables are required to implement the on-board image-processing software. The ARM program is the master element of the system in that it performs the following tasks:

1. Initializes hardware/software resources on start up. 2. Acquires image frames from the camera continuously. 3. Delegates image-processing tasks to the DSP program. 4. Transfers stimulus commands to the output interface(s). 5. Handles network communication requests from the Blue- tooth interface. 6. Listens and responds to commands from the connected remote supervising tool. Fig. 3 – Photograph of the image processor with only the 7. Releases resources on shutdown. camera attached. The ARM program runs continuously unless terminated by the user. TMS320C55x digital signal processor (DSP,Texas Instruments). The purpose of the DSP program is to process camera For simplicity, we will refer to the ARM9TDMI core on an images when instructed by the ARM program. The ARM OMAP5912 microprocessor as the ARM core. During normal program can also reconfigure the processing parameters as operation the ARM core runs in host mode. It interfaces with required. most hardware components, executes the operating system, The main purpose of the remote supervising tool is to con- and delegates mathematically intensive operations to the DSP. trol the image-processing parameters of the DSP program, The DSP has a separate clock, so it operates independently of including: the ARM core, allowing for parallel processing. The image processor has SDRAM and flash memory for data • The application of image manipulation routines (e.g. storage. An Ethernet port allows the system to be connected to smoothing) on the camera output. a network. Peripherals may be connected to the device using • Selection of sampling technique for computing electrode RS232 serial or USB. While the OMAP5912 has dedicated hard- stimulus value. ware pins for attaching a CCD camera, we have opted for a • Panning the camera over both X and Y axes. common PC webcam through the USB bus instead, as CCD • Sampling field size adjustment (zooming). cameras with the appropriate wiring format are unavailable. • Fine-tuning of electrode stimulus intensity. The Bluetooth transceiver was also connected to the system via USB. At present, subjects can control various features of the image processor (e.g. zoom and smoothing) using a Bluetooth- In addition, the remote supervisor can display both enabled cell phone. A photo of the device is shown in Fig. 3. raw camera images and processed results from the image- processing device in either real-time or off-line mode. System 2.2. Operating system statistics are also displayed to aid development and diagnos- tics. It should be noted that the supervisor does not need to A version of the Linux kernel (the core of an operating system) be in operation for the remainder of the system to function. for the ARM architecture has been under active development In fact, it can be connected and disconnected at any time. The by the open source community for a number of years. The ARM program on the image processor handles such transitions image processor uses a recent version (2.6.18) of the kernel without affecting the rest of the system. in order to support specialized features and hardware. The The DSP and ARM program can be divided into several mod- source code was patched for the OMAP processor and parts ules as illustrated in Fig. 4. The Main module conglomerates rewritten to fix hardware driver issues. We used the “DSP Gate- functionalities of all other system components. The Blue- way” device driver [17] to facilitate communication between tooth and Camera modules are for accessing Bluetooth and the two microprocessor cores. Some code modifications were Camera hardware, respectively, while the Linux kernel pro- necessary to make it compatible with the version of Linux ker- vides a handful of system calls for accessing hardware. These nel used. The root file system containing all software libraries, modules present a more convenient programming interface. utilities, and applications was specially prepared such that it Buffering is also performed to minimize delays associated only required 6.1 MB of storage space. with hardware access. The DSP module facilitates communication between the 2.3. Software architecture DSP program and the other ARM modules. The Supervisor module services commands from the remote supervisor. This The system consists of three major components: the DSP pro- is assisted by Net Operations, which performs all the read- gram, the ARM program, and the remote supervising tool. The ing and writing functions over a TCP/IP network. The Output computer methods and programs in biomedicine 95 (2009) 258–269 261

Fig. 4 – Components of the image processor software. The DSP and ARM programs run on the DSP and ARM cores of the OMAP processor, respectively. The former, being responsible for image-processing tasks, is relatively small, consisting of only two modules. The latter handles the remaining tasks, including DSP control and interfacing with external entities.

Interfaces module is responsible for sending processed results gram reconfigures the DSP program settings appropriately from the DSP to one or more system outputs, such as a RF such that changes are reflected on the next image processed. link transmitter. The Frame Processor is the main module of If changing sampling points were all that the subject could do the DSP program, performing all image-processing operations. then an efficient and simple implementation would be eas- The Memory Allocator is a purpose built memory manager for ily achieved by storing the points in a lookup table. However, the DSP. because users can dynamically zoom and pan the sampling field, the centers of each point are calculated dynamically. The camera produces images by impulsively sampling val- 2.4. Image processing ues arranged in a two-dimensional array. Zooming will likely cause the sampling points to fall somewhere between the dis- Image processing for vision prostheses requires a number of crete points of the image array. Bilinear interpolation [18] is fundamental procedures, as outlined in Fig. 5. The first step used to approximate the image value in such cases. Essen- involves acquiring an image from the camera. The second tially, it approximates the value by taking a distance-weighted step performs several enhancement algorithms on the image. average of the quasi-point’s four nearest real neighbors, whose Because the image would most certainly have far more pix- value we know (from the image). For instance, suppose a point els than electrodes available on an implant, appropriate down P =(x, y) is the sampling point and the value of its four nearest sampling and pixelization is necessary. In the final step the neighbors are Q1, Q2, Q3 and Q4. Then P’s value is given by: pixelized image is converted into stimulation commands for transmission to the prosthesis in the patient’s eye. Q1 × (1 − xr) × (1 − yr) + Q2 × xr × (1 − yr) + Q3 × (1 − xr) × yr We have been developing stimulation arrays whereby the electrodes are arranged as a hexagonal lattice [5]. This is +Q4 × xr × yr demonstrated in the center window shown in Fig. 7. While our stimulating electrode arrays have fixed positioning rela- where tive to each other, clinicians can configure the image processor xr = x −| x | to sample points away from the default locations. This is achieved by assigning new sampling coordinates using the yr = y −| y | “sampling point re-mapping” utility in the remote supervising tool. The DSP program will then produce appropriate outputs All coordinate calculations in the DSP program are car- by calculating the center of each sampling point using the ried out in floating point representation to minimize rounding offsets supplied by the user. errors. Subjects can also zoom in or out (non-optical zoom per- If desired, image smoothing can be applied. Two smoothing formed in software), and pan the field of sampling points algorithms are available. When 3 × 3 neighbored averaging [18] horizontally and vertically. Whenever a subject changes the is used, the returned value is simply the average of the sam- sampling parameters, the Supervisor module of the ARM pro- pling point and its surrounding eight neighbors. Although less

Fig. 5 – Summary of the image-processing procedure. Images acquired by a camera are enhanced, pixelized, then converted into stimulation commands for transmission to the implanted device. 262 computer methods and programs in biomedicine 95 (2009) 258–269

Table1–Performance (in frames per second) of smoothing schemes on various processing/computation platforms. The “Impulse” column represents the baseline value without smoothing. The “3 × 3” column and “Gaussian” column are values derived when performing 3 × 3 neighbor averaging and Gaussian-weighted averaging, respectively. Platform Impulse 3 × 3 Gaussian

OMAP 5912 (ARM9 only) 24 3 0.5 OMAP 5912 (ARM9 + C55x) 30 21 9 PC (camera) 30 30 30 PC (software) 54 51 48

than optimal, it is fast to compute. Gaussian-weighted averaging [18] returns a value based on the weighted average of the center point and its surrounding points. The weighting is based on the profile of a two-dimensional Gaussian surface, which is approximated by a 5 × 5 convolution kernel in the spatial domain. The second algorithm is slower than the first due to the almost threefold increase in the number of processing points (see Table 1), but the resulting output is more representative of the original image by taking into account more information from it. After sampling and smoothing, the value for each electrode can be further modulated by a scaling factor. The last operation that can optionally be performed is inverting the value of each sampling point to create a negative effect. In summary Fig. 6 illustrates the procedures executed by the DSP program for each frame of camera image. Fig. 6 – Image-processing steps of the DSP program. The program begins by determining the center sampling point 3. Results for each electrode. Fine-tuning of the stimulus intensity of each electrode, and selection of image smoothing methods 3.1. Image processing and stimulus generation are possible. In addition, the final output can be inverted to produce a negative effect not unlike those in photography. Fig. 7 is a screen capture of the remote supervising tool running on a Linux desktop PC. Arranged down the right hand side are various controls for sending commands to the image in Fig. 10.InFig. 10A the camera was fixated on the ‘B’ key. processor. The child window towards the top left (partially cov- Without any zoom the white letter appeared as a dot in one of ered) displays the raw images from the camera as well as the the phosphene pixels in Fig. 10B. The camera was then zoomed stimulus output for each electrode rendered graphically. The in 5.3 times in Fig. 10C, Where the outline of the character is dialog in the center is the “sampling point remapping” utility clearly visible. The negative effect is demonstrated in Fig. 10D. described previously. In some situations it enhances image contrast. Smoothing was The zooming functionality is demonstrated in Fig. 8.For not used in any of these cases. comparison, Fig. 8A is the view as seen by the camera. Fig. 8B is the default output without any zoom. Fig. 8C and D used a 3.2. System performance zoom factor of 1.6 and 2.6 times, respectively. Smoothing was disabled in all three instances. The side profile of the monitor In our implementation the smoothing algorithms have the is barely visible in the default view due to aliasing. It becomes greatest impact on the overall system performance. For this progressively clearer as the zoom factor increases. reason we have benchmarked the speed of the system with: Fig. 9 illustrates the effect of Gaussian smoothing. Fig. 9A smoothing off, 3 × 3 neighborhood averaging, and Gaussian- is the view seen by the camera. The image was taken in a weighted averaging. The unit of measure is frames per second dark room with the desk lamp being the only source of light. (FPS). Table 1 summarizes the results on different hardware Smoothing was disabled in Fig. 9B but enabled in Fig. 9C. The platforms. An identical camera was used in all instances to softening effect is clearly visible. Also available in the image provide an objective comparison. The camera is only able to processor is smoothing with 3 × 3 neighborhood averaging acquire 30 frames per second at 176 × 144 pixels resolution. (not shown). Thus the maximum FPS achievable by the image processor is An attempt at using the image processor for locating and limited to 30 as well. In all cases the ARM9 microprocessor and reading keys on a black keyboard with white prints is shown the C55x DSP were both clocked at 192 MHz. computer methods and programs in biomedicine 95 (2009) 258–269 263

Fig. 7 – The remote supervising tool. Researchers and clinicians have full control of the image processor through this tool. It runs on a standard PC with an Ethernet connection.

The first row shows the result of using only the ARM9TDMI avoid transient fluctuations the measurements were recorded microprocessor on the OMAP. The values were obtained by and averaged over 5 min of execution. In all cases the Ethernet re-implementing the entire DSP program within the ARM pro- cable was plugged in with the remote supervisor connected, gram. The second row represents the default configuration, and the image processor was also connected to the devel- namely, the image processor setup as described in this paper. opment PC via the RS232 serial port. The “Idle” condition is For comparison both the ARM and DSP programs were defined as when the device has booted up and is standing ported for use on a x86 machine. The FPS for a Pentium 4 by to begin image processing. Under this condition, the CPU 2.13 GHz desktop machine with 1 GB of RAM is shown in the cycles and memory requirements are primarily dictated by the last two rows of the table. The third row shows the speed when operating system’s needs. images were acquired with the camera used on the image processor. The FPS was limited by the throughput of the camera (30 frames). To obtain the absolute speed of the software when Table2–Powerconsumption of the image processor under different conditions. The current values were running on a desktop PC, we arranged for pre-recorded images measured at the power input socket of the device and to be fed into the program. Without the camera speed bottle- averaged over 5 min. The second and third column neck the program was able to deliver a FPS of approximately indicates whether the Camera and Bluetooth module 50, as listed in the last row. were attached to the device, respectively. Condition Camera Bluetooth Current (mA) 3.3. Power consumption Idle No No 60 Idle Yes No 340 Table 2 provides power consumption measurements under Idle Yes Yes 360 various conditions. The values are total current draw of the Processing Yes Yes 425 image processor as measured at the 5 V power input socket. To 264 computer methods and programs in biomedicine 95 (2009) 258–269

Fig. 8 – Zooming with the image processor. The effect of zooming at 1× (B), 1.6× (C) and 2.6× (D) is demonstrated. The original image (A) is shown at top left for comparison. Notice that the left portion of the monitor becomes more distinguishable as zoom factor increases.

It is noteworthy to highlight the amount of power the device control. While many high-performance ultra-portable USB webcam used. As indicated by the first and second row PCs currently available will undoubtedly have enough process- of Table 2, when connected it consumed several times the ing power, as suggested by Table 1, they would be far more combined power requirement of all remaining electronics on cumbersome to carry around, and are certainly by no means the device. Under the “Processing” condition (third row), the wearable. image processor was converting camera images into stimulus A number of kernels exist for use in embedded devices, commands and performing Gaussian smoothing to simulate ranging from the large monolithic types like Linux, to the small real-life scenarios. With the current hardware configuration application specific microkernels. Our choice to use Linux was and a battery similar to those available for laptops (e.g. a prompted primarily by three reasons: 5110 mAh lithium ion battery), the image processor can be expected to operate continuously for over 12 h. 1. By building on the existing code-base available for Linux, the device can support a large feature set from the incep- 4. Discussion tion. This is an advantage for devices that are undergoing constant research and development, and where the clinical 4.1. Choice of platform and development tools requirements are as yet not thoroughly understood. As the device matures over time, unnecessary features can then Previous quantitative performance measurements of image- be excluded. processing algorithms pertaining to vision prosthesis suggest 2. The Linux kernel has extensive support for the ARM archi- that DSPs may be essential for a successful image proces- tecture. sor implementation [19]. While they excel at mathematically 3. Drivers for a large number of peripheral devices currently intensive computations, DSPs generally have minimal sup- exist for Linux. port for peripheral hardware and are certainly not designed to run general-purpose operating systems. For these two rea- With the Linux operating system as the basis, a variety sons, the Texas Instruments OMAP microprocessors are ideal of programming languages may be used to implement the off-the-shelf candidates. Within one chip, the DSP core can ARM program. The C language was chosen for a balance perform all numerical computations while the ARM core han- between system performance and development effort. Due dles all other tasks such as communication and peripheral to the minimalist nature of the device, both hardware- and computer methods and programs in biomedicine 95 (2009) 258–269 265

Fig. 9 – Gaussian smoothing. The original image is shown in (A). Outputs without and with Gaussian smoothing are shown in (B) and (C). Averaging gives the output a soft appearance.

software-wise, it was difficult to program from within the sending them to the DSP, or the DSP program should use device itself. Instead, the GNU GCC cross-compilation tool bit masking in software whenever reading 8-bit blocks from chain for the ARM platform was used to compile the source the image. We have chosen the former technique. Rewriting code on a PC-based development machine. The resulting exe- images into 16-bit blocks results in twice the amount of mem- cutable binaries were then transferred onto the device. All the ory requirements and the most significant 8 bits of each 16-bit system-level libraries and utilities required for basic functions block are always set to 0. However, this adds a constant – hence of the image processor were also produced in the same way. predictable – amount of run time per image frame processed, A number of time-critical sections of the DSP program were a particularly important consideration for real-time systems hand optimized using assembly code while the remaining such as the image processor. Furthermore, the DSP program majority was written in C. In both cases the C55x C compiler implements a series of image-processing filters, each of which and assembler provided by Texas Instruments were used to requires numerous read operations into the image. Performing generate the binary. a bit masking operation for every read would add significant The remote supervisor was written in C++ with Qt version 4 overhead and hence degrade performance. (Qt Software, Oslo, Norway), a graphical user interface toolkit, The DSP has 32k word of dual-access RAM and 48k word of primarily for its extensive support of cross-platform compati- single-access RAM, each word being 16 bits in length. Image bility. The authors developed and used the remote supervisor rewriting as described above doubles the memory require- on Linux as a matter of convenience. It can however be ported ment. Clearly complete images (approximately 100 kB after to other popular operating systems with ease. rewrite) cannot fit into the DSP’s RAM. It is also preferable to keep as much of the DSP’s RAM free as possible, because 4.2. Implementation considerations many image-processing algorithms require memory for storing intermediate results during computation. To resolve the Each raw image from the camera is of the order of 50 kB in issue we used a feature available on the OMAP whereby the size, packed into a contiguous stretch of 8-bit memory blocks. SDRAM normally only used by the ARM core is mapped into This is problematic for a number of reasons. The smallest data the virtual address space of the DSP. In other words, by sharing type supported by the ARM architecture and the TMS320C55x some of the system RAM used by the ARM processor, one can architecture is 8 and 16 bits, respectively. Thus either the ARM extend the amount of memory available to the DSP. In theory, program should rewrite all images into 16-bit blocks before up to 64 MB of SDRAM can be mapped into the DSP. But for rea- 266 computer methods and programs in biomedicine 95 (2009) 258–269

Fig. 10 – Reading a keyboard character. The processing parameters can be tuned to enhance one’s ability to read ﬁne prints, demonstrated here with keyboard labels. (A) The scenery as seen by the camera. (B) The ﬁxated keyboard character is not recognizable without any zoom. (C) Zooming at 5.3×, the letter is easily readable. (D) At the same zoom level, but with negative effect turned on.

sons related to the C compiler supplied by Texas Instruments whenever it blocks waiting for hardware I/O, and executes and the Memory Management Unit on the DSP, only 128 kB another thread (for example, the main module) suspended can be shared in the current setup. Nevertheless, this is more previously. Consequently, delays caused by Bluetooth do not than adequate. Sharing memory also provides an additional affect the remainder of the program. benefit. Namely, it is no longer necessary to copy data from The DSP program uses the Memory Allocator module (see the SDRAM into DSP’s RAM for each image to be processed, Fig. 4) to manage the SDRAM it shares with the ARM program. thus improving overall efficiency of the program. This is necessary because the memory allocation functions As mentioned previously, the ARM and DSP core have sep- provided by the C standard library could only allocate from the arate clocks and run concurrently. The two programs were DSP’s own internal RAM. A custom implementation can also arranged to coordinate access to the shared memory to ensure exploit memory requirement characteristics of the DSP pro- data consistency. Access to the shared memory is arbitrated gram to provide a level of efficiency that would not have been by a signaling mechanism. It would be wasteful to allow the possible with a general-purpose implementation. The current ARM program to sit idle while the DSP is working. Therefore memory allocator operates by returning the first stretch of the ARM program is arranged to perform image acquisition I/O free memory it can find. No attempt is made to minimize during this time. It is a relatively lengthy process taking up to memory fragmentation and a compaction algorithm is never several tens of milliseconds per frame. Furthermore, the ARM used. For most situations this would quickly lead to small program buffers two frames ahead so that occasional delays “gaps” in the free memory pool as numerous allocations and in hardware I/O would not unnecessarily stall the rest of the deallocations are made for memory of varying sizes. With- system. out further treatment it would eventually fail prematurely. To prevent hardware I/O delays from the Bluetooth module However, the current implementation works for the image slowing down the entire system, it is implemented as a sepa- processor and is extremely fast. This is because it is known rate thread. On multi-CPU systems threads allow several parts a priori that for every frame of image processed, three or less of a program to be executed concurrently by different CPUs, memory allocation requests will be made, with each being for thus increasing throughput. On the image processor there is a relatively large block of memory in the order of tens of kilo- only one ARM core. However, benefit is derived from allow- bytes. Moreover, when the program completes a frame, the ing the kernel to suspend execution of the Bluetooth module state is reset. The next frame will start the process afresh. computer methods and programs in biomedicine 95 (2009) 258–269 267

Therefore the DSP program will never run to the stage where repeated allocations and deallocations may start to cause problems.

4.3. Image processing

Electrical stimulation of the retina will elicit activity in multiple ganglion cells and possibly other cell types in layers of the retina [20,21], which in turn may lead to the activation of ganglion cells at sites distant to the intended position. Con- sequently the position of any perceived phosphene(s) may be displaced from the intended location. The “sampling point re- mapping” tool corrects any such discrepancies. As the image processor acquires images from the camera and turns them into electrical stimuli, it samples each image at locations that correspond to the arrangement of the stimulation electrode array by default. If it is subsequently found that the subject’s perceived phosphene position does not match the physical position of the light stimuli (as captured by the camera), then the sampling point can be moved with the re-mapping tool. An example is demonstrated in Fig. 11. Bilinear interpolation was chosen because it is reasonably inexpensive to calculate. Depending on the level of scaling, images up-scaled (zoomed in) using bilinear interpolation suffer from some blurring, jagged edges (aliasing) and sometimes halo effects around edges. When used for down-scaling (zooming out) moiré patterns tend to occur (another form of aliasing). It has been proposed and is hoped that the number of electrodes will increase substantially in the not too distant Fig. 11 – Correction of positional discrepancy between a future. As such, the scaling implementation on the current camera image sampling point and a phosphene position. image processor may no longer be ideal with respect to the Coordinates for the Image Space and Electrode Space are resolution offered by new devices. A number of alternative written in capital and small letters, respectively. (A) In the algorithms are known to produce images with better quality, ideal case, the subject sees the phosphene at the location but they tend to incur more resources than bilinear interpola- where the light stimulus appears. (B) It is possible that the tion. One can also reduce interpolation artifacts by increasing stimulating electrodes may activate cells at slightly the resolution of the camera output at the expense of higher different location ((x3, y3) instead of (x1, y1)), causing the memory consumption, which increases roughly by the square wrong phosphene to be generated for that location. (C) The of the image dimensions. However given the rather limited re-mapping tool adds an offset (I, J) to the default sampling number of electrodes available on current vision prostheses, location so as to generate the correct phosphene. the computational overhead is not warranted. The reason for having scaling of individual sampling points is motivated by concerns regarding variations at the Buffoni et al. proposed that maximum control should electrode–tissue interface after implantation. Electrodes with be given to the patient [15]. We fulfill this requirement non-ideal positioning may have higher impedance, hence by letting patients control the image processor using a requiring stronger intensity levels to evoke the same response. Bluetooth-enabled cell phone, which in essence functions as Sometimes an electrode may fail to elicit an action potential an omni-directional remote control. The limited number of regardless of stimulus intensity. In such situations the elec- buttons with simple arrangement allows patients to issue trode can be disabled all together by setting the scaling factor commands by “feeling” for the right key using their fingertips. to 0. For example, the joystick (four directions of movement) con- It has been suggested that user-controllable zooming will trols the zoom level and camera panning, while the numeric help to increase the efficacy of vision prostheses, especially keypad toggles various image-processing effects. This said, a when only a limited number of electrodes are available [22]. purpose built remote control could also be constructed and This is not surprising, since zooming allows sampling den- interfaced with the image processor using Bluetooth. sity to be matched to the dimensions of the visual field of interest such that aliasing is reduced to a level where recogni- 4.4. System performance tion becomes possible. Psychophysical studies by Fu et al. [23] involving reading printed texts support this assertion. Here it Comparing the first two rows of Table 1, it is clear that can be seen in Fig. 10 that readability of a keyboard character the TMS320C55x DSP improves performance. As expected, is enhanced by filling the electrode array with the character of the more mathematical operations performed, the greater interest at a zoom factor of 5.3×. the gain. ARM9TDMI plus TMS320C55x performs 19 times 268 computer methods and programs in biomedicine 95 (2009) 258–269

better than with ARM9TDMI alone when Gaussian-weighted rigorously followed during the development of the current smoothing is used. Comparing with some of the image- wearable image processor. These however will be essential processing techniques proposed for vision prostheses, the for any device intended for clinical trial on vision-impaired real-time algorithms used in the present device are demand- patients. While the use of open source software (Linux and ing for embedded systems but by no means complex. associated device drivers) has many advantages as described Therefore DSPs will be of great benefit, if not crucial, for most previously, it poses special regulatory clearance considera- implementations. tions. Regulatory agencies such as the FDA generally decree Our algorithms are able to deliver approximately 50 frames that it is the device manufacturer’s responsibility to ensure per second on a desktop PC. Incidentally, Asher and col- the safety of all components, including software, acquired leagues [16] had similar throughput on a PC, even though from a third-party [24]. Linux is a system designed primar- the algorithms are very different. Their image pre-processing ily for general-purpose computing, where consequences of algorithms were developed in Matlab while the time-critical failure are likely to be more tolerable, the level of correct- components were written in C. A throughput of 50 frames per ness verification by the original developers may not suffice for second or higher was achieved using a Pentium 2.6 GHz PC. medical devices. Additionally, unused kernel features increase Although untested, they believe their implementation is capa- complexity of the system, thus the likelihood of faults. While ble of providing 25 frames per second or more when ported to Linux is open for white box testing without restrictions, the a pocket PC. When demanding processing is performed, it is of process can be cumbersome due to the large code base. In significance to note that our apparently fast program (on a PC) recent years a number of open source kernels designed specif- in fact performs sub-optimally when executed on an embed- ically for mission- and life-critical embedded systems have ded device. Therefore one should exercise caution when using appeared. One of which, the seL4 [25], is particularly promising desktop PCs to estimate performance of programs intended for devices such as the present image processor. Its support for for use on embedded systems. the ARM architecture is already in place, has a small code base, Powered by a Texas Instruments OMAP microprocessor the and most important of all, a project is currently in progress to wearable image processor achieved satisfactory performance mathematically verify the correctness of the entire kernel [26]. under most configurations. When image smoothing is per- An increasing number of vision prosthesis prototypes are pro- formed by way of convolution with a 5 × 5 Gaussian kernel, gressing towards human trials. Considerations like these will the throughput dropped to approximately 10 frames per sec- become even more pertinent for those involved in the design ond. While still usable, it is no longer ideal. The reduction in and development process. performance is due to dynamic zooming. With a 5 × 5 kernel, the DSP needs to compute 25 bilinear interpolations in 5. Conclusion order to approximate the stimulus value for an electrode. In contrast, impulse sampling only needs one. However, the inde- We have demonstrated the implementation of an image pendent nature of these operations means that they can be processor for use with vision prostheses, with particular performed in parallel. Additionally, in the current implemen- emphasis on the approach taken to maximize software per- tation the DSP program never modifies the raw camera image formance. The results indicate that a highly flexible and acquired. These two characteristics imply that if the image configurable device can be built using an OMAP micropro- processor were to employ multiple DSPs then significant per- cessor with dual-core architecture. However, depending on formance improvements – approaching 30 frames per second the image-processing algorithms involved, more powerful sig- as limited by the camera CCD circuitry – can be expected. An nal processing hardware than those available on the OMAP alternative to the multi-DSP scheme is to use a more powerful may be required to deliver real-time performance. Further- DSP, such as processors belonging to the Texas Instruments more, general-purpose embedded microprocessors alone (for TMS320C6x family. Although much more capable, these DSPs example, ARM) are unlikely to be adequate for implement- also consume prohibitive amounts of power. A TMS320C6711 ing image-processing strategies required by vision prostheses. for example requires as much as 1.1 watt, which can be any- Our benchmark comparisons also indicate that care should where from 7 to 17 times more than a TMS320C55x performing be taken when using desktop PCs to judge the performance the equivalent algorithm [19]. If an image processor were to be of algorithms intended for use on embedded devices, which built with such a DSP, the battery requirement and the weight are constrained by processing capability, memory availability, would make them rather inconvenient to use. power consumption, size, and weight. A consequence of low frame rate is that patients will Little work on image-processing devices for vision pros- receive intermittent stimuli delivery. A possible remedy theses has been reported to date. Given the importance of involves delivering identical stimuli multiple times while image processing and contemporaneously the complex nature the image processor is computing the next frame. It should of some proposed techniques, design and implementation of be noted, however, that while this technique ensures con- such devices is a non-trivial task requiring careful consider- stant visual perception, the patient would still experience the ations between several conflicting requirements. More effort latency due to processing delays. should be devoted to this important topic. 4.5. Regulatory considerations Conflict of interest statement Besides general good software engineering techniques, no particular software process life cycle and associated steps were None declared. computer methods and programs in biomedicine 95 (2009) 258–269 269

[11] J. Sommerhalder, B. Rappaz, R. de Haller, A.P. Fornos, A.B. Acknowledgements Safran, M. Pelizzone, Simulation of artificial vision. II. Eccentric reading of full-page text and the learning of this We thank Philip Byrnes-Preston for expert technical assis- task, Vision Research 44 (2003) 1693–1703. tance. This research was supported by funding from the [12] J.R. Boyle, A.J. Maeder, W.W. Boles, Challenges in digital imaging for artificial human vision, in: Proceedings of the Australian Federal Government (Department of Education, SPIE, vol. 4299, 2001, pp. 533–543. Science, and Training and the Australian Research Council) [13] S. Chen, L.E. Hallum, N.H. Lovell, G.J. Suaning, Visual acuity and Retina Australia. measurement of prosthetic vision: a virtual–reality simulation study, Journal of Neural Engineering 2 (2005) references 135–145. [14] L.E. Hallum, G.J. Suaning, D.S. Taubman, N.H. Lovell, Simulated prosthetic visual fixation, saccade, and smooth pursuit, Vision Research 45 (2005) 775–788. [1] M.S. Humayun, J.D. Weiland, G.Y. Fujii, R. Greenberg, R. [15] L. Buffoni, J. Coulombe, M. Sawan, Image processing Williamson, J. Little, B. Mech, V. Cimmarusti, G.V. Boemel, G. strategies dedicated to visual cortical stimulators: a survey, Dagnelie, E. de Juan Jr., Visual perception in a blind subject Artificial Organs 29 (2005) 658–664. with a chronic microelectronic retinal prosthesis, Vision [16] A. Asher, W.A. Segal, S.A. Baccus, L.P. Yaroslavsky, D.V. Research 43 (2003) 2573–2581. Palanker, Image processing for a high-resolution [2] W. Liu, M. Sivaprakasam, P.R. Singh, R. Bashirullah, G. Wang, optoelectronic retinal prosthesis, IEEE Transactions on Electronic visual prosthesis, Artificial Organs 27 (2003) Biomedical Engineering 54 (2007) 993–1004. 986–995. [17] T. Kobayashi, Linux DSP Gateway Specification Rev 3.3, [3] Y. Pan, T. Tokuda, A. Uehara, K. Kagawa, M. Nunoshita, J. Nokia Corporation, 2005. Ohta, Flexible and extendible neural stimulation device with [18] R.C. Gonzalez, R.E. Woods, Digital Image Processing, 2nd ed., distributed multichip architecture for retinal prosthesis, Prentice Hall, New Jersey, USA, 2002. Japanese Journal of Applied Physics 44 (2005) 2099–2103. [19] N.J. Parikh, J.D. Weiland, M.S. Humayun, S.S. Shah, G.S. [4] Y.T. Wong, N. Dommel, P. Preston, L.E. Hallum, T. Lehmann, Mohile, DSP based image processing for retinal prosthesis, N.H. Lovell, G.J. Suanning, Retinal neurostimulator for a in: Proceedings of the 26th Annual International Conference multifocal vision prosthesis, IEEE Transactions on Neural of the IEEE EMBS, vol. 2, 2004, pp. 1475–1478. Systems and Rehabilitation Engineering 15 (2007) [20] R.J. Jensen, J.F. Rizzo III, Thresholds for activation of rabbit 425–434. retinal ganglion cells with a subretinal electrode, [5] N.H. Lovell, L.E. Hallum, S. Chen, S. Dokos, P. Byrnes-Preston, Experimental Eye Research 83 (2006) 367–373. R. Green, L. Poole-Warren, T. Lehmann, G.J. Suaning, [21] R.J. Jensen, J.F. Rizzo III, Responses of ganglion cells to Advances in retinal neuroprosthetics, in: M. Akay (Ed.), repetitive electrical stimulation of the retina, Journal of Handbook of Neural Engineering, Wiley/IEEE Press, Neural Engineering 4 (2007) 1–6. Hoboken, NJ, USA, 2007 (Chapter 20). [22] L.E. Hallum, G. Dagnelie, G.J. Suaning, N.H. Lovell, Simulating [6] K. Cha, K. Horch, R.A. Normann, Mobility performance with auditory and visual sensorineural prostheses: a comparative a pixelized vision system, Vision Research 32 (1992) review, Journal of Neural Engineering 4 (2007) 58–71. 1367–1373. [23] L. Fu, S. Cai, H. Zhang, G. Hu, X. Zhang, Psychophysics of [7] J.S. Hayes, V.T. Yin, D. Piyathaisere, J.D. Weiland, M.S. reading with a limited number of pixels: towards the Humayun, G. Dagnelie, Visually guided performance of rehabilitation of reading ability with visual prosthesis, simple tasks using simulated prosthetic vision, Artificial Vision Research 46 (2006) 1292–1301. Organs 27 (2003) 1016–1028. [24] Office of Device Evaluation, Center for Devices and [8] R.W. Thompson Jr., G.D. Barnett, M.S. Humayun, G. Dagnelie, Radiological Health, Food and Drug Administration, General Facial recognition using simulated prosthetic pixelized Principles of Software Validation; Final Guidance for vision, Investigative Ophthalmology and Visual Science 44 Industry and FDA Staff, 2002. (2003) 5035–5042. [25] G. Heiser, K. Elphinstone, I. Kuz, G. Klein, S.M. Petters, [9] X. Chai, W. Yu, J. Wang, Y. Zhao, C. Cai, Q. Ren, Recognition Towards trustworthy computing systems: taking of pixelized Chinese characters using simulated prosthetic microkernels to the next level, ACM Operating Systems vision, Artificial Organs 31 (2007) 175–182. Review 41 (2007) 3–11. [10] J. Sommerhalder, E. Oueghlani, M. Bagnoud, U. Leonards, [26] K. Elphinstone, G. Klein, P. Derrin, T. Roscoe, G. Heiser, A.B. Safran, M. Pelizzone, Simulation of artificial vision. I. Towards a practical, verified kernel, in: Proceedings of the Eccentric reading of full-page text and the learning of this 11th Workshop on Hot Topics in Operating Systems, San task, Vision Research 43 (2003) 269–283. Diego, CA, USA, 2007.

Chapter 5

Conclusion

This thesis aims to address ﬁve questions. Here I shall summarise how my work has contributed to answering these questions.

1. Can subretinal stimulation reliably evoke RGC responses directly? In particular, could this be achieved with small electrodes having dimensions close to that of the RGC soma?

In Chapter 2 I showed that subretinal stimulation was able to elicit RGC spikes that 1) had latency ≤ 2ms; 2) were insensitive to blocking of presynaptic inputs; 3) were dependent on the voltage–gated sodium channels. These observations conclusively demonstrated for the first time that subretinal stimulation is able to evoke RGC responses directly. The reason previous studies failed to recognise these short–latency direct responses might be due to the recording techniques. These studies generally recorded responses with metallic extracellular microelectrodes, which do not offer the signal–to–noise ratio of patch clamp recordings. Consequently, short latency responses may have been difficult to resolve from the stimulus artifacts.

The threshold of eliciting RGC spikes with 25 μm diameter subretinal electrodes is approximately 70–fold higher than that of comparable epiretinal electrodes [107]. The charge density required to elicit RGC responses with 25 μm platinum–iridium electrodes is close to, or over, the safety limit of platinum–iridium at that dimension. Therefore materials with higher charge injection limits, for example activated iridium oxide [100], may be necessary when fabricating small electrodes for subretinal implants. 74

2. How do the RGCs respond temporally to subretinal stimulation as a function of stimulus conﬁguration?

Similar to earlier studies, I also observed RGC spikes with latencies ranging from a few ms to a few hundred ms following subretinal stimulation. Unlike the short latency spikes, which were characteristically conspicuous due to the small jitter of latency (median = 0.5 ms with 0.1 ms biphasic pulses; Chapter 2), the long latency responses were more difficult to assess in relation to the stimulus configurations. In existing papers, the effects of stimulus configuration on these spiking responses were judged qualitatively using either raster plots or peri–stimulus time histogram, or both. I found assessments based on these tools to be subjective and thus unreliable. The problem was particularly prominent when judging spiking response variations due to small stimulus configuration variations. I developed a statistical analysis technique (temporal activity maps) in Chapter 2 to circumvent this difficulty, and to provide a quantitative assessment of the RGC spiking responses over time as a function of stimulus configurations. With the aid of this technique, I was able to generalise the RGC spiking responses following subretinal stimulation into four classes. Together these classes completely described the temporal responses of every RGC examined. Finally, temporal activity maps are not limited to assessment of neural activities driven by electrical stimulation. It is applicable to any time series data that could be decomposed into orderly stochastic point processes and where weak stationarity could be assumed.

3. Can the RGCs reliably follow electrical stimulation at repetition rates comparable to that of the light evoked RGC spiking frequencies?

The RGCs convey visual information using bursts of high frequency spikes. However, studies using isolated retinas have found a decline in RGC spiking response rate during repetitive stimulation. Electrical stimulation could activate the amacrine cells, which provide inhibitory inputs to the RGCs, thus potentially reducing their excitability. Blocking these network inputs (Chapter 3) did not improve the response rates, suggesting that inhibition from the amacrine cells did not have a signiﬁcant

eﬀect on the spiking response rates of directly activated RGCs. The evoked RGC INa,

however, decreased exponentially during repetitive stimulation. Preventing the INa decline also prevented the decrease in response rates. Together, these observations strongly suggest that the voltage–gated sodium channels exert a strong inﬂuence over 75

the response rates of RGCs during repetitive stimulation. I also showed in Chapter 3 that under identical conditions the RGCs were intrinsically capable of high frequency spiking upon intracellular depolarising pulse injection and when driven by light stimuli. Therefore the spiking response depression was not an artifact of the experimental conditions, but a true phenomenon of electrically evoked responses. In this work, while demonstrating the association between spiking response rates and the evoked RGC sodium current, I developed a new stimulation technique where, instead of using ﬁxed stimulus amplitude during repetitive stimulation, the amplitude

was scaled progressively to increase the driving force for INa. The scaling factors

were derived from previously recorded RGC INa inactivation rates. The technique was termed adaptive current scaling in Chapter 3. It signiﬁcantly improved the RGC spiking response rates during repetitive stimulation. It may therefore be useful for preventing the fading of electrically evoked responses in human patients.

4. Using conventional cathodic-first biphasic pulses, is it possible to preferentially activate: 1) the ON and OFF RGCs and 2) the retinal layers? Any difference in stimulus threshold could potentially be exploited to selectively excite different cell types. Of particular interest is the ability to discriminate between the ON and OFF RGCs. I compared the threshold for ON and OFF cells in Chapter 2. Consistent with Sekirnjak’s [107] epiretinal results, no significant difference was found. Therefore selective excitation of either RGC types is not possible with conventional cathodic–first biphasic stimuli for both epiretinal and subretinal stimulation. More elaborate stimulus waveforms may be necessary to achieve this goal. Selective excitation is not restricted to the ON and OFF cells (lateral comparison), but also applies to various retinal layers (vertical comparison). With short duration pulses and appropriate control of the pulse strength, epiretinal stimuli could be confined to RGCs activation only [28, 107]. By increasing the pulse strength, one could then recruit cells deeper within the retina [28], in addition to the RGCs. An analogous scheme does not appear to be possible with subretinal stimulation, due to the similarities in thresholds of direct RGC activation verses retinal network activation (Chapter 2). Thus despite their close proximity to the subretinal electrode(s), the distal retinal neurons (photoreceptors, bipolar cells, and amacrine cells) were not preferentially activated prior to the RGCs. Presumably, this could be due to the 76

low level of voltage–gated sodium channel expression in these neurons relative to the RGCs [130], thus making them comparatively diﬃcult to excite electrically. These observations imply that subretinal stimulation, at least with 25 μm diameter electrodes and biphasic pulses, would concurrently activate both the network neurons as well as the RGCs. Consequently, it may be diﬃcult to achieve temporally precise spiking patterns.

Several lines of evidence suggest that the behavioural distinction between the ON and OFF RGCs is primarily due to the presynaptic inputs they receive [83, 90]. However, Margolis and Detwiler [75] reported differences in the intrinsic mechanisms that generate the repetitive spikes of mice ON and OFF alpha RGCs following light stimulation. I therefore asked if there are also variations in the response rates of rabbit RGCs during repetitive electrical stimulation. Any such variation could potentially be used to favour excitation to particular RGC types. In Chapter 3, the differences in declining response rates between the ON and OFF cells were insignificant. This could be a consequence of species variations or that extracellular electrical stimulation did not have the specificity to adequately target the intrinsic mechanisms unique to the ON and OFF cells.

In Chapter 4 I described the algorithm, architecture, and implementation of a wearable real–time image processor for vision prostheses. This work is the ﬁrst report of such a stand–along system. The image processor decomposes images from a camera, transforms the information into pixelised images based on the geometry of the stimulation electrode array, and converts them into commands for driving an implanted ASIC. Based on the ﬁndings of psychophysical study with normally sighted human subjects, a number of image sampling and processing algorithms were also implemented. The work showed that such devices could be implemented with embedded hardware. But depending on the image processing strategies used, general–purpose embedded processors may be inadequate for implementing a retinal implant image processor.

An underlying assumption made by the image processor, and indeed by most existing implementations of retinal prostheses, is that either the brightness or the size of the 77 phosphene (or both) increases as a function of stimulus strength. This approximation is probably reasonable for generating simple percepts such as spots of light [38]. To advance beyond the rudimentary, and often unexpected, percepts produced by existing implants [99], more sophisticated stimulation strategies will most certainly be required. 78 Appendix A

List of publications

A list of the author’s publications and provisional patent during the course of the PhD candidature, including those not presented as Chapters 2, 3, and 4, are listed below.

Patent

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Stimulation method for maintaining the responsiveness of electrically excitable cells to repeated electrical stimulation. United States Patents and Trademarks Oﬃce, Application: 61435696, provisional ﬁling January 2011.

Journal Articles

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Frequency-dependent reduction of voltage-gated sodium current modulates retinal ganglion cell response rate to electrical stimulation Journal of Neural Engineering, 2011 in press (recommended reader).

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Direct activation and temporal response properties of rabbit retinal ganglion cells following subretinal stimulation, Journal of Neurophysiology, 2009(102) 2982–2993 (cover article).

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, A wearable real–time image processor for a vision prosthesis, Computer Methods and Programs in Biomedicine, 2009(95) 258–269.

Refereed Proceedings 80

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Responses of starburst amacrine cells to prosthetic stimulation of the retina, Proceedings of the 33rd Annual Interna- tional Conference of the IEEE EMBS, Boston USA, 2011 (4 pages), submitted.

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Sodium channel inactivation reduces retinal ganglion cell responsiveness to repetitive prosthetic stimulation, The 5th international IEEE EMBS Conference on Neural Engineering, Cancun, Mexico, 2011 (4 pages), in press.

• G. J. Suaning, S. Kisban, S. C. Chen, P. J. Byrnes-Preston, C. Dodds, D. Tsai,P. Matteucci, S. Herwik, J. W. Morley, N.H. Lovell, O. Paul, T. Stieglitz, P. Ruther, discrete cortical responses from multi–site supra–choroidal electrical stimulation in the feline retina, Proceedings of the 32nd Annual International Conference of the IEEE EMBS, Buenos Aires, Argentina, 2010 (4 pages).

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Direct activation of retinal ganglion cells with subretinal stimulation, Proceedings of the 31st Annual International Conference of the IEEE EMBS, Minneapolis USA, 2009 (4 pages).

Conferences

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Temporal response proﬁles of amacrine cells following electrical stimulation of the retina, Neuroscience 2011, Washington DC, USA, 2011.

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Activation of voltage-gated ion channels in rabbit retinal ganglion cells following electrical stimulation of the retina, Australian Neuroscience Society Meeting 2011, Auckland New Zealand, 2011.

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Inactivation of sodium current modulates retinal ganglion cell response rate during high frequency stimulation, Neuroscience 2010, San Diego USA, 2010.

• V. Tatarinoﬀ, D. Tsai, How Deep is deep Enough? Australian and New Zealand Laboratory Animal Assocation Conference, 2010 (Best presentation award).

• D. Tsai, J. W. Morley, G. J. Suaning, N. H. Lovell, Retinal ganglion cell responses following subretinal electrical stimulation in the isolated rabbit retina, Neuroscience 2009, Chicago USA, 2009. Bibliography

[1] A. K. Ahuja, M. R. Behrend, M. Kuroda, M. S. Humayun, and J. D. Weiland. An in vitro model of a retinal prosthesis. IEEE Transactions on Biomedical Engineering, 55:1744–1753, 2008.

[2] F. R. Amthor, K. T. Keyser, and N. A. Dmitrieva. Eﬀects of the destruction of starburst-cholinergic amacrine cells by the toxin AF64A on rabbit retinal directional selectivity. Visual Neuroscience, 19:495–509, 2002.

[3]A.Asher,W.A.Segal,S.A.Baccus,L.P.Yaroslavsky,andD.V.Palanker.Image processing for a high–resolution optoelectronic retinal prosthesis. IEEE Transactions on Biomedical Engineering, 54:993–1004, 2007.

[4] S. A. Baccus. Timing and computation in inner retinal circuitry. Annual Reviews of Physiology, 69:271–90, 2007.

[5] M. R. Behrend, A. K. Ahuja, and J. D. Weiland. Dynamic current density of the disk electrode double-layer. IEEE Transactions on Biomedical Engineering, 55:1056–1062, 2008.

[6] M. J. Berry and M. Meister. Refractoriness and neural precision. The Journal of Neuroscience, 18:2200–2211, 1998.

[7] M. J. Berry, D. K. Warland, and M. Meister. The structure and precision of retinal spike trains. Proceedings of The National Academy of Sciences of The United States of America, 94:5411–5416, 1997.

[8] K. Bielefeldt and M. B. Jackson. A calcium-activated potassium channel causes frequency-dependent action-poitential failures in a mammalian nerve terminal. Jour- nal of Neurophysiology, 70:284–298, 1993. 82

[9] P. O. Bishop, W. Burke, and W. R. Hayhow. Repetitive stimulation of optic nerve and lateral geniculate synapses. Experimental Neurology, 1:534–555, 1959.

[10] J. R. Boyle, A. J. Maeder, and W. W. Boles. Challenges in digital imaging for artiﬁcial human vision. In Proceedings of the SPIE, vol. 4299, pp. 533–543, 2001.

[11] L.-X. Buﬀoni, J. Coulombe, and M. Sawan. Image processing strategies dedicated to visual cortical stimulators: A survey. Artiﬁcial Organs, 29:658–664, 2005.

[12] C. Cai, Q. Ren, N. J. Desai, J. F. Rizzo, and S. I. Fried. Response variability to high rates of electric stimulation in retinal ganglion cells. Journal of Neurophysiology, 106:153–162, 2011.

[13] K. Cha, K. Horch, and R. A. Normann. Simulation of a phosphene-based visual ﬁeld: Visual acuity in a pixelized vision system. Annals of Biomedical Engineering, 20:439–449, 1992.

[14] X. Chai, W. Yu, J. Wang, Y. Zhao, C. Cai, and Q. Ren. Recognition of pixelized chinese characters using simulated prosthetic vision. Artiﬁcial Organs, 31:175–182, 2007.

[15] S. Chen, L. E. Hallum, N. H. Lovell, and G. J. Suaning. Visual acuity measurement of prosthetic vision: A virtual-reality simulation study. Journal of Neural Engineering, 2:135–145, 2005.

[16] A. Y. Chow, V. Y. Chow, K. H. Packo, J. S. Pollack, G. A. Peyman, and R. Schuchard. The artiﬁcial silicon retina microchip for the treatment of vision loss from retinitis pigmentosa. Archives of Ophthalmology, 122:460–469, 2004.

[17] P. Christopher, M. J. McMahon, U. Patel, J. Wei, R. J. Greenberg, and J. D. Dorn. The temporal dynamics of percepts generated by the ArgusTM II epiretinal prosthesis. In Program No. 20.3. 2010 Neuroscience Meeting Planner. San Diego, CA: Society for Neuroscience, 2010. Online, 2010.

[18] D. Copenhagen. Excitation in retina: the ﬂow, ﬁltering, and molecules of visual signalling in the glutamatergic pathways from photoreceptors to ganglion cells. In L. M. Chalupa and J. S. Werner, editors, The Visual Neurosciences. The MIT Press, 2003. 83

[19] N. P. Cottaris and S. D. Elfar. How the retinal network reacts to epiretinal stimulation to form the prosthetic visual input to the cortex. Journal of Neural Engineering, 2:74–90, 2005.

[20] D. Debanne. Information processing in the axon. Nature Reviews Neuroscience, 5:304–316, 2004.

[21] S. H. DeVries. Correlated ﬁring in rabbit retinal ganglion cells. Journal of Neuro- physiology, 81:908–920, 1999.

[22] S. Dokos, G. J. Suaning, and N. H. Lovell. A bidomain model of epiretinal stimulation. IEEE Transactions on Biomedical Engineering, 13:137–146, 2005.

[23] J. Dowling. Current and future prospects for optoelectronic retinal prostheses. In Eye Cambridge Symposium, 2008.

[24] K. Elphinstone, G. Klein, P. Derrin, T. Roscoe, and G. Heiser. Towards a practical, veriﬁed kernel. In Proceedings of the 11th Workshop on Hot Topics in Operating Systems, San Diego, CA, USA, 2007.

[25] W. Fink, C. X. You, and M. A. Tarbell. Microcomputer-based artiﬁcial vision support system for real-time image processing for camera-driven visual prostheses. Journal of Biomedical Optics, 15:016013, 2010.

[26] D. K. Freeman, D. K. Eddington, J. F. Rizzo III, and S. I. Fried. Selective activation of neuronal targets with sinusoidal electric stimulation. Journal of Neurophysiology, 104:2778–2791, 2010.

[27] D. K. Freeman and S. I. Fried. Multiple components of ganglion cell desensitization in response to prosthetic stimulation. Journal of Neural Engineering, 8:016008, 2011.

[28] S. I. Fried, H. A. Hsueh, and F. S. Werblin. A method for generating precise temporal patterns of retinal spiking using prosthetic stimulation. Journal of Neurophysiology, 95:970–978, 2006.

[29] S. I. Fried, A. C. W. Lasker, N. J. Desai, D. K. Eddington, and J. F. Rizzo III. Axonal sodium channel bands shape the response to electric stimulation in retinal ganglion cells. Journal of Neurophysiology, 101:1972–1987, 2009. 84

[30] D. S. Friedman, B. J. O’Colmain, B. Munoz, S. C. Tomany, C. McCarty, P. T. V. M. de Jong, B. Nemesure, P. Mitchell, J. Kempen, and Eye Diseases Prevalence Research Group. Prevalence of age–related macular degeneration in the united states. Archives of Ophthalmology, 122:564–572, 2004.

[31] L. Fu, S. Cai, H. Zhang, G. Hu, and X. Zhang. Pychophysics of reading with a limited number of pixels: Towards the rehabilitation of reading ability with visual prosthesis. Vision Research, 46:1292–1301, 2006.

[32] T. Fujikado, T. Morimoto, H. Kanda, S. Kusaka, K. Nakauchi, M. Ozawa, K. Mat- sushita, H. Sakaguchi, Y. Ikuno, M. Kamei, and Y. Tano. Evaluation of phosphenes elicited by extraocular stimulation in normals and by suprachoroidal–transretinal stimulation in patients with retinitis pigmentosa. Graefe’s archive for clinical and experimental ophthalmology, 245:1411–1419, 2007.

[33] F. Gekeler, A. Messias, M. Ottinger, K. U. Bartz-Schmidt, and E. Zrenner. Phosphenes electrically evoked with dtl electrodes: A study in patients with retinitis pigmentosa, glaucoma, and homonymous visual ﬁeld loss and normal subjects. Investigative Ophthalmology & Visual Science, 47:4966–4974, 2006.

[34] H. Gerding, R. E. Eckmiller, R. Hornig, V. Ortmann, A. Kolck, and S. Taneri. Safety assessment and acute clinical tests of epiretinal retina implants. In Investigative Ophthalmology & Visual Science, volume 43 (E Abstract 4488), 2002.

[35] T. Gollisch and M. Meister. Eye smarter than scientists believed: Neural computations in circuits of the retina. Neuron, 65:150–164, 2010.

[36] R. Gonzalez and R. Woods. Digital Image Processing, 2nd ed. Prentice Hall, 2002.

[37] R. J. Greenberg, T. J. Velte, M. S. Humayun, G. N. Scarlatis, and J. Eugene de Juan. A computational model of electrical stimulation of the retinal ganglion cell. IEEE Transactions on Biomedical Engineering, 46:505–514, 1999.

[38] S. H. Greenwald, A. Horsager, M. S. Humayun, R. J. Greenberg, M. J. McMahon, and I. Fine. Brightness as a function of current amplitude in human retinal electrical stimulation. Investigative Ophthalmology & Visual Science, 50:5017–5025, 2009. 85

[39] Y. Grossman, I. Parnas, and M. E. Spira. Ionic mechanisms involved in diﬀerential conduction of action potentials at high frequency in a branching axon. Journal of Physiology, 295:307–322, 1979.

[40] A. E. Grumet, J. L. Wyatt, and J. F. Rizzo. Multi-electrode stimulation and recording in the isolated retina. Journal of Neuroscience Methods, 101:31–42, 2000.

[41] H. Guo, R. Qin, Y. Qiu, Y. Zhu, and S. Tong. Conﬁguration-based processing of phosphene pattern recognition for simulated prosthetic vision. Artiﬁcial Organs, 34:324–330, 2010.

[42] L. E. Hallum, G. Dagnelie, G. J. Suaning, and N. H. Lovell. Simulating auditory and visual sensorineural prostheses: a comparative review. Journal of Neural Engineering, 4:58–71, 2007.

[43] L. E. Hallum, G. J. Suaning, D. S. Taubman, and N. H. Lovell. Simulated prosthetic visual ﬁxation, saccade, and smooth pursuit. Vision Research, 45:775–788, 2005.

[44] J. S. Hayes, V. T. Yin, D. Piyathaisere, J. D. Weiland, M. S. Humayun, and G. Dag- nelie. Visually guided performance of simple tasks using simulated prosthetic vision. Artiﬁcial Organs, 27:1016–1028, 2003.

[45] G. Heiser, K. Elphinstone, I. Kuz, G. Klein, and S. Petters. Towards trustworthy computing systems: taking microkernels to the next level. ACM Operating Systems, 41:3–11, 2007.

[46] A. Horsager, G. M. Boynton, R. J. Greenberg, and I. Fine. Temporal interactions during paired-electrode stimulation in two retinal prosthesis subjects. Investigative Ophthalmology & Visual Science, 52:549–557, 2011.

[47] E. H. Hu, R. F. Dacheux, and S. A. Bloomﬁeld. A ﬂattened retina–eyecup preparation suitable for electrophysiological studies of neurons visualized with trans-scleral infrared illumination. Journal of Neuroscience Methods, 103:209–216, 2000.

[48] W. Hu, C. Tian, T. Li, M. Yang, H. Hou, and Y. Shu. Distinct contributions of nav1.6 and nav1.2 in action potential initiation and backpropagation. Nature Neuroscience, 12:996–1004, 2009.

[49] A. Hughes and D. Vaney. Coronate cells: Displaced amacrines of the rabbit retina? Journal of Comparative Neurology, 198:169–189, 1980. 86

[50] M. S. Humayun, E. de Juan Jr., J. D. Weiland, G. Dagnelie, S. Katona, R. Greenberg, and S. Suzuki. Pattern electrical stimulation of the human retina. Vision Research, 39:2569–2576, 1999.

[51] M. S. Humayun, J. D. Weiland, G. Y. Fujii, R. Greenberg, R. Williamson, J. Little, B. Mech, V. Cimmarusti, G. V. Boemel, G. Dagnelie, and E. de Juan Jr. Visual perception in a blind subject with a chronic microelectronic retinal prosthesis. Vision Research, 43:2573–2581, 2003.

[52] A. T. Ishida. Retinal ganglion cell excitability. In L. M. Chalupa and J. S. Werner, editors, The Visual Neurosciences. The MIT Press, 2003.

[53] R. J. Jensen and J. F. Rizzo III. Thresholds for activation of rabbit retinal ganglion cells with a subretinal electrode. Experimental Eye Research, 83:367–373, 2006.

[54] R. J. Jensen and J. F. Rizzo III. Responses of ganglion cells to repetitive electrical stimulation of the retina. Journal of Neural Engineering, 4:1–6, 2007.

[55] R. J. Jensen and J. F. Rizzo III. Activation of retinal ganglion cells in wild-type and rd1 mice through electrical stimulation of the retinal neural network. Vision Research, 48:1562–1568, 2008.

[56] R. J. Jensen and J. F. Rizzo III. Activation of ganglion cells in wild-type and rd1 mouse retinas with monophasic and biphasic current pulses. Journal of Neural Engineering, 6:035004, 2009.

[57] R. J. Jensen, J. F. Rizzo III, O. R. Ziv, A. Grumet, and J. Wyatt. Thresholds for activation of rabbit retinal ganglion cells with an ultraﬁne, extracellular microelectrode. Investigative Ophthalmology & Visual Science, 44:3533–3543, 2003.

[58] R. J. Jensen, O. R. Ziv, and J. F. Rizzo III. Responses of rabbit retinal ganglion cells to electrical stimulation with an epiretinal electrode. Journal of Neural Engineering, 2:16–21, 2005.

[59] C. C. W. Klaver, R. C. W. Wolfs, J. R. Vingerling, A. Hofman, and P. T. V. M. de Jong. Age-speciﬁc prevalence and causes of blindness and visual impairment in an older population. Archives of Ophthalmology, 116:653–658, 1998.

[60] T. Kobayashi. Linux DSP gateway speciﬁcation rev 3.3. Technical report, Nokia Corporation, 2005. 87

[61] C. Koch and B. Mathur. Neuromorphic vision chips. IEEE Spectrum, 33:38–46, 1996.

[62] M. H. P. Kole, S. U. Ilschner, B. M. Kampa, S. R. Williams, P. C. Ruben, and G. J. Stuart. Action potential generation requires a high sodium channel density in the axon initial segment. Nature Neuroscience, 11:178–186, 2008.

[63] P. S. Lagali, D. Balya, G. B. Awatramani, T. A. M¨unch, D. S. Kim, V. Busskamp, C. L. Cepko, and B. Roska. Light-activated channels targeted to on bipolar cells restore visual function in retinal degenerationi. Nature Neuroscience, 11:667–675, 2009.

[64] H. C. Lai and L. Y. Jan. The distribution and targeting of neuronal voltage-gated ion channels. Nature Reviews Neuroscience, 7:548–562, 2006.

[65] S. Lee and Z. J. Zhou. The synaptic mechanism of direction selectivity in distal processes of starburst amacrine cells. Neuron, 51:787–799, 2006.

[66] C. LeRoy. Oùl’onrend compte de quelques tentatives quel’ on a faites pour guérir plusieurs maladies par l’électricité.hist acad roy sciences (paris). Mémoire Math Phys, 60:87–95, 1755.

[67] L. Li, Y. Hayashida, and T. Yagi. Temporal properties of retinal ganglion cell responses to local transretinal current stimuli in the frog retina. Vision Research, 45:263–273, 2005.

[68] B. Lin, A. Koizumi, N. Tanaka, S. Panda, and R. H. Masland. Restoration of visual function in retinal degeneration mice by ectopic expression of melanopsin. Proceedings of The National Academy Of Sciences Of The United States Of America, 105:16009– 16014, 2008.

[69] W. Liu, M. Sivaprakasam, P. R. Singh, R. Bashirullah, and G. Wang. Electronic visual prosthesis. Artiﬁcial Organs, 27:986–995, 2003.

[70] R. Llinas, R. Baker, and C. Sotelo. Electrotonic coupling between neurons in cat inferior olive. Journal of Neurophysiology, 37:560–571, 1974.

[71] J. I. Loewenstein, S. R. Montezuma, and J. F. Rizzo. Outer retinal degeneration: an electronic retinal prosthesis as a treatment strategy. Archives of Ophthalmology, 122:587–596, 2004. 88

[72] N. H. Lovell, L. E. Hallum, S. Chen, S. Dokos, P. Byrnes-Preston, R. Green, L. Poole- Warren, T. Lehmann, and G. J. Suaning. Advances in retinal neuroprosthetics. In M. Akay, editor, Handbook of Neural Engineering. Wiley–IEEE Press, 2007.

[73] R. E. Marc, B. W. Jones, C. B. Watt, and E. Strettoi. Neural remodeling in retinal degeneration. Progress In Retinal and Eye Research, 22:607–655, 2003.

[74] E. Margalit and W. B. Thoreson. Inner retinal mechanisms engaged by retinal electrical stimulation. Investigative Ophthalmology & Visual Science, 47:2606–2612, 2006.

[75] D. J. Margolis and P. B. Detwiler. Diﬀerent mechanisms generate maintained activity in on and oﬀ retinal ganglion cells. The Journal of Neuroscience, 27:5994–6005, 2007.

[76] D. J. Margolis, G. Newkirk, T. Euler, and P. B. Detwiler. Functional stability of retinal ganglion cells after degeneration-induced changes in synaptic input. The Journal of Neuroscience, 28:6526–6536, 2008.

[77] R. H. Masland. The fundamental plan of the retina. Nature Neuroscience, 4:877–886, 2001.

[78] D. N. Mastronarde. Correlated ﬁring of retinal ganglion cells. Trends in Neuro- sciences, 12:75–80, 1989.

[79] C. C. McIntyre and W. M. Grill. Finite element analysis of the current–density and electric ﬁeld generated by metal microelectrodes. Annals of Biomedical Engineering, 29:227–235, 2001.

[80] M. Meister and M. J. Berry. The neural code of the retina. Neuron, 22:435–450, 1999.

[81] M. Meister, L. Lagnado, and D. A. Baylor. Concerted signaling by retinal ganglion cells. Science, 270:1207–1210, 1995.

[82] D. R. Merrill, M. Bikson, and J. G. R. Jeﬀerys. Electrical stimulation of excitable tissue: Design of eﬃcacious and safe protocols. Journal of Neuroscience Methods, 141:171–198, 2005.

[83] G. J. Murphy and F. Rieke. Network variability limits stimulus-evoked spike timing precision in retinal ganglion cells. Neuron, 52:511–524, 2006. 89

[84] K. L. Myhr, P. D. Lukasiewicz, and R. O. L. Wong. Mechanisms underlying developmental changes in the firing patterns of on and off retinal ganglion cells during refinement of their central projections. The Journal of Neuroscience, 21:8664–8671, 2001.

[85] S. Nirenberg, S. M. Carcieri, A. L. Jacobs, and P. E. Latham. Retinal ganglion cells act largely as independent encoders. Nature, 411:698–701, 2001.

[86] B. J. O’Brien, T. Isayama, R. Richardson, and D. M. Berson. Intrinsic physiological properties of cat retinal ganglion cells. Journal of Physiology, 538:787–802, 2002.

[87] Office of Device Evaluation, Center for Devices and Radiological Health, Food and Drug Administration. General principles of software validation, final guidance for industry and FDA staff, 2002.

[88] T. M. O’Hearn, S. R. Sadda, J. D. Weiland, M. Maia, E. Margalit, and M. S. Hu- mayun. Electrical stimulation in normal and retinal degeneration (rd1) isolated mouse retina. Vision Research, 46:3198–3204, 2006.

[89] Y. Pan, T. Tokuda, A. Uehara, K. Kagawa, M. Nunoshita, and J. Ohta. Flexible and extendible neural stimulation device with distributed multichip architecture for retinal prosthesis. Japanese Journal of Applied Physics, 44:2099–2103, 2005.

[90] J.-J. Pang, F. Gao, and S. M. Wu. Segregation and integration of visual channels: Layer-by-layer computation of on–oﬀ signals by amacrine cell dendrites. The Journal of Neuroscience, 22:4693–4701, 2002.

[91] N. Parikh, L. Itti, and J. Weiland. Saliency-based image processing for retinal prostheses. Journal of Neural Engineering, 7:016006, 2010.

[92] N. J. Parikh, J. D. Weiland, M. S. Humayun, S. S. Shah, and G. S. Mohile. DSP based image processing for retinal prosthesis. In Proceedings of the 26th Annual International Conference of the IEEE EMBS, 2004.

[93] B. B. Peterson and D. M. Dacey. Morphology of human retinal ganglion cells with intraretinal axon collaterals. Visual Neuroscience, 15:377–387, 1998.

[94] M. J. Phillips, D. C. Otteson, and D. M. Sherry. Progression of neuronal and synaptic remodeling in the rd10 mouse model of retinitis pigmentosa. The Journal of Comparative Neurology, 518:2071–2089, 2010. 90

[95] J. W. Pillow, J. Shlens, L. Paninski, A. Sher, A. Litke, E. J. Chichilnisky, and E. P. Simoncelli. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature, 454:995–999, 2008.

[96] N. D. Radtke, M. J. Seiler, R. B. Aramant, H. M. Petry, and D. J. Pidwell. Trans- plantation of intact sheets of fetal neural retina with its retinal pigment epithelium in retinitis pigmentosa patients. American Journal of Ophthalmology, 133:544–550, 2002.

[97] F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W. Bialek. Spikes: Exploring the Neural Code. The MIT Press, 1996.

[98] J. F. Rizzo III, R. J. Jensen, J. Loewenstein, and J. Wyatt. Unexpectedly small percepts evoked by epi–retinal electrical stimulation in blind humans. In Investigative Ophthalmology & Visual Science, volume 44 (E Abstract 4207), 2003.

[99] J. F. Rizzo III, J. Wyatt, J. Loewenstein, S. Kelly, and D. Shire. Methods and perceptual thresholds for short–term electrical stimulation of human retina with microelectrode arrays. Investigative Ophthalmology & Visual Science, 44:5355–5361, 2003.

[100] L. S. Robblee, J. L. Lefko, and S. B. Brummer. Activated Ir: An electrode suitable for reversible charge injection in saline solution. Journal of the Electrochemical Society, 130:731–733, 1983.

[101] R. W. Rodieck. The First Steps in Seeing. Sinauer Associates, 1998.

[102] T. L. Rose and L. S. Robblee. Electrical stimulation with Pt electrodes. VIII. elec- trochemically safe charge injection limits with 0.2 ms pulses. IEEE Transactions on Biomedical Engineering, 37:1118–1120, 1990.

[103] B. Roska, A. Molnar, and F. S. Werblin. Parallel processing in retinal ganglion cells: How integration of space–time patterns of excitation and inhibition form the spiking output. Journal of Neurophysiology, 95:3810–3822, 2006.

[104] B. Roska and F. Werblin. Vertical interactions across ten parallel, stacked representations in the mammalian retina. Nature, 410:583–587, 2001. 91

[105] S. B. Ryu, J. H. Ye, J. S. Lee, Y. S. Goo, and K. H. Kim. Characterization of retinal ganglion cell activities evoked by temporally patterned electrical stimulation for the development of stimulus encoding strategies for retinal implants. Brain Research, 1275:33–42, 2009.

[106] A. J. Sefton and M. Swinburn. Electrical activity of laternal geniculate nucleus and optic tract of the rat. Vision Research, 4:315–328, 1964.

[107] C. Sekirnjak, P. Hottowy, A. Sher, W. Dabrowski, A. M. Litke, and E. J. Chichilnisky. Electrical stimulation of mammalian retinal ganglion cells with multielectrode arrays. Journal of Neurophysiology, 95:3311–3327, 2006.

[108] C. Sekirnjak, P. Hottowy, A. Sher, W. Dabrowski, A. M. Litke, and E. J. Chichilnisky. High-resolution electrical stimulation of primate retina for epiretinal implant design. The Journal of Neuroscience, 28:4446–4456, 2008.

[109] C. Sekirnjak, C. Hulse, L. H. Jepson, P. Hottowy, A. Sher, W. Dabrowski, A. M. Litke, and E. J. Chichilnisky. Loss of responses to visual but not electrical stimulation in ganglion cells of rats with severe photoreceptor degeneration. Journal of Neurophysiology, 102:3260–3269, 2009.

[110] C. Sekirnjak, L. H. Jepson, P. Hottowy, A. Sher, W. Dabrowski, A. M. Litke, and E. J. Chichilnisky. Changes in physiological properties of rat ganglion cells during retinal degeneration. Journal of Neurophysiology, 105:2560–2571, 2011.

[111] J.-S. Shyu, M. Maia, J. D. Weiland, T. O’Hearn, S.-J. Chen, E. Margalit, S. Suzuki, and M. S. Humayun. Electrical stimulation in isolated rabbit retina. IEEE Trans- actions on Neural Systems and Rehabilitation Engineering, 14:290–298, 2006.

[112] A. F. Soleng, K. Chiu, and M. Raastad. Unmyelinated axons in the rat hippocampus hyperpolarize and activate an H current when spike frequency exceeds 1 Hz. Journal of Physiology, 552:459–470, 2003.

[113] J. Sommerhalder, E. Oueghlani, M. Bagnoud, U. Leonards, A. B. Safran, and M. Pelizzone. Simulation of artiﬁcial vision I. eccentric reading of full–page text and the learning of this task. Vision Research, 43:269–283, 2003. 92

[114] J. Sommerhalder, B. Rappaz, R. de Haller, A. P. Fornos, A. B. Safran, and M. Peliz- zone. Simulation of artiﬁcial vision II. eccentric reading of full–page text and the learning of this task. Vision Research, 44:1693–1703, 2003.

[115] W. A. Spencer and E. R. Kandel. Electrophysiology of hippocampal neurons: iv. fast prepotentials. Journal of Neurophysiology, 24:272–285, 1961.

[116] M. A. Stephens. Edf statistics for goodness of ﬁt and some comparisons. Journal of the American Statistical Association, 69:730–737, 1974.

[117] A. Stett, W. Barth, S. Weiss, H. Haemmerle, and E. Zrenner. Electrical multisite stimulation of the isolated chicken retina. Vision Research, 40:1785–1795, 2000.

[118] A. Stett, A. Mai, and T. Herrmann. Retinal charge sensitivity and spatial discrim- ination obtainable by subretinal implants: key lessons learned from isolated chicken retina. Journal of Neural Engineering, 4:7–16, 2007.

[119] E. Strettoi, V. Pignatelli, C. Rossi, V. Porciatti, and B. Falsini. Remodeling of second–order neurons in the retina of rd/rd mutant mice. Vision Research, 43:867– 877, 2003.

[120] G. E. Tassicker. United States Patent 2,760,483, 1956.

[121] R. W. Thompson, Jr., G. D. Barnett, M. S. Humayun, and G. Dagnelie. Facial recognition using simulated prosthetic pixelized vision. Investigative Ophthalmology & Visual Science, 44:5035–5042, 2003.

[122] P. K. Trong and F. Rieke. Origin of correlated activity between parasol retinal ganglion cells. Nature Neuroscience, 11:1343–1351, 2008.

[123] D. Tsai, J. W. Morley, G. J. Suaning, and N. H. Lovell. Direct activation and temporal response properties of rabbit retinal ganglion cells following subretinal stimulation. Journal of Neurophysiology, 102:2982–2993, 2009.

[124] D. Tsai, J. W. Morley, G. J. Suaning, and N. H. Lovell. A wearable real-time image processor for a vision prosthesis. Computer Methods and Programs in Biomedicine, 95:258–269, 2009. 93

[125] H. Tsutsui and Y. Oka. Slow removal of Na+ channel inactivation underlies the temporal ﬁltering property in the teleost thalamic neurons. Journal of Physiology, 539:743–753, 2002.

[126] D. C. van Essen. Organization of visual areas in macaque and human cerebral cortex. In L. M. Chalupa and J. S. Werner, editors, The Visual Neurosciences. MIT Press, 2003.

[127] D. Vaney. A quantitative comparison between the ganglion cell populations and axonal outﬂows of the visual streak and periphery of the rabbit retina. Journal of Comparative Neurology, 189:215–233, 1980.

[128] M. Völker, K. Shinoda, H. Sachs, H. Gmeiner, T. Schwarz, K. Kohler, W. Inhoffen, K. U. Bartz-Schmidt, E. Zrenner, and F. Gekeler. In vivo assessment of subretinally implanted microphotodiode arrays in cats by optical coherence tomography and flu- orescein angiography. Graefe’s archive for clinical and experimental ophthalmology, 242:792–799, 2004.

[129] J. D. Weiland, W. Liu, and M. S. Humayun. Retinal prosthesis. Annual Review of Biomedical Engineering, 7:361–401, 2005.

[130] D. A. Wollner, R. Scheinman, and W. A. Catterall. Sodium channel expressionand assembly during development of retinal ganglion cells. Neuron, 1:727–737, 1988.

[131] Y. T. Wong, S. C. Chen, J. M. Seo, J. W. Morley, N. H. Lovell, and G. J. Suaning. Focal activation of the feline retina via a suprachoroidal electrode array. Vision Research, 49:825–833, 2009.

[132] G. M. Zeck and R. H. Masland. Spike train signatures of retinal ganglion cell types. European Journal of Neuroscience, 26:367–380, 2007.

[133] J. Zheng, S. Lee, and Z. J. Zhou. A developmental switch in the excitability and function of the starburst network in the mammalian retina. Neuron, 44:851–864, 2004.

[134] J. A. Zhou, S. J. Woo, S. I. Park, E. T. Kim, J. M. Seo, H. Chung, and S. J. Kim. A suprachoroidal electrical retinal stimulator design for long-term animal experiments and in vivo assessment of its feasibility and biocompatibility in rabbits. Journal of Biomedicine and Biotechnology, 2008:1–10, 2008. 94

[135] Z. J. Zhou and G. L. Fain. Starburst amacrine cells change from spiking to nonspiking neurons during retinal development. Proceedings of The National Academy Of Sciences Of The United States Of America, 93:8057–8062, 1996.

[136]O.R.Ziv,J.F.RizzoIII,andR.J.Jensen.Invitroactivationofretinalcells: estimating location of stimulated cell by using a mathematical model. Journal of Neural Engineering, 2:S5–S15, 2005.

[137] E. Zrenner. Will retina implants restore vision? Science, 295:1022–1025, 2002.

[138] E. Zrenner, H. Benav, A. Bruckmann, U. Greppmaier, A. Kusnyerik, A. Stett, K. Stingl, and R. Wilke. Electronic implants provide continuous stable percepts in blind volunteers only if the image receiver is directly linked to eye movement. In Program No. 444. 2010 ARVO Meeting Planner. Fort Lauderdale, FL: ARVO, 2010. Online, 2010.