sign in sign up
<<
Home , Optogenetics, Visual prosthesis

OptoNeuro

Optogenetic visual cortical prosthesis

Conflicts of interest: I have filed a number of patents and CANDO may have commercial impact. But currently no commercial interests. Dr. Patrick Degenaar Reader in Neuroprosthesis @ Newcastle

Epilepsy prosthesis http://www.cando.ac.uk

Visual prosthesis http://www.optoneuro.eu

Spinal cord injury Stimulate Record

Hand/Leg bionics

Electrodiagnostics Process Multi-EMG recording Closed loop neural interfaces!! The optogenetics revolution

Blue light is used to activate expressing . Other can silence activity Optogenetic control of behaviour. Gradinaru et al. J Neurosci (2007) Optoelectronics for optogenetics

µlens Gallium Nitride µLED GaN MQW 3D bump bond p-contact Solder bond CMOS driver CMOS

64x64, 64x1 16 x 16 MPW CMOS 90x90 CANDO 1 CANDO 3 passive µLED arrays chip driven µLED arrays wafer Optrode Wafer

2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 High radiance optoelectronics for optogenetics: Grossman N et al. IEEE TBME 2011, 58(6), 1742-1751. McGovern B et al IEEE TBCAS 2010, 4(6, part 2), 469-476. Grossman N et al, J. 2010, 7(1), 016004. … The visual system

Optics (/lens)

Sense/code Temporal (retina)

Transmission Pulvinar nucleus () Lateral Geniculate Nucleus Sorting Superior (Thalamus) Colliculus Optical radiation

Interpretation Primary visual (Visual Cortex) cortex

5 Visual loss: statistics

Corneal opacities (5%) Diabetic retinopathy (5%) According to WHO, about Prevalence Childhood blindness (4%) 180 million people worldwide have a AMD (9%) Others Glaucoma Legal blindness: (16%) (12%)

“medically diagnosed central visual acuity of 20/200 or less in the better eye with the best Cataracts possible correction, and/or a (48%) RP (1%) visual field of 20 degrees or Trauma less” (?%) From: WHO Primary visual prosthesis target conditions Drugs and gene therapy becoming increasingly useful.. But cannot restore lost vision 6 Disorders of the visual system Optics (cornea/lens) Sub/epi prosthesis Sense/code (retina) Optic nerve prosthesis (<1% of blindness)

Transmission (Optic Nerve)

LGN Sorting Glaucoma (Thalamus) prosthesis Trauma Cortical Other severe blindness prosthesis (~30% of blindness)

For invasive procedures only! Interpretation Very expensive! (Visual Cortex) Will be enormously difficult! But can treat much larger cohort Implementing a visual prosthesis Optics (cornea/lens) Image Sense/code acquisition (retina)

Transmission Visual (Optic Nerve) processing

Sorting (Thalamus) Wireless power/data

Stimulator Interpretation (Visual Cortex)

Easy?? Brindley and Lewin did it in 1968!!! • Brindley,G.S.,Lewin,W.S.,1968.The visual sensations produced by electrical stimulation of the medial occipital cortex. J Physiol.19454-5P. • Brindley,G.S.,Lewin,W.S.,1968.The sensations produced by electrical stimulation of the visual cortex.J.Physiol.196,479–493. Challenge 1: Sheer scale of challenge

1960’s: You could build a device on a Monday and put it in a patient on a Friday

Now: A project to create an active implantable device requires £10-50M and 5-10 years worth of regulatory effort to demonstrate safety before implantation is allowed

As such, almost the entire field moved to retinal prosthesis in the 1990’s as it was a much easier target!! Challenge 2: Visual acuity

Asian/pictographic scripts: ~ 1000 pixels

European/phonetic scripts: ~ 5000 pixels

1001033705 xxx 1001033705Normal (10,000)(100)(1000)(5000)(25) vision pixels European/phonetic characters: ~ 100 pixels

11 Lessons from retinal prosthetics

1500 electrodes – but only a few dozen can be on at any moment in time.

From Retina AG Challenge 3: Effective contrast Assume greyscale retina Add noisy retina Original image @ 64 x 64 resolution effect

With insufficient stimulus contrast it is like whispering through a typhoon!

100:1 contrast ratio 50:1 contrast ratio 13 Challenge 4: The eye is not a camera!

The retina Rods/cones 1. Image capture Ligtht

Horizontal cells 2. smoothing

5. Spike coding Bipolar cells Ganglion cells 3. Derivative

OFF ON Amacrine cells 4. Temporal edge

The retina has 60 different cell types. If we are bypassing the natural processing, we need to replicate the key features! 14 Challenge 5: Biocompatibility

Glial scarring

Electrode insertion point

Journal of The Electrochemical Society, 152 (7) J85-J92 ~2005! Accelerated ageing, 1000 IEEE Transactions on Neural Systems and cycles at 1.7V/s Rehabilitation , Vol. 12, No. 2, June 2004 Ideal Real Electrode degradation and adverse tissue reactions are a major issue for smaller stimulating electrodes because of high charge density requirements Annu. Rev. Biomed. Eng. 10:275–309, 2008 15 Where to start? Developing the stimulator

Surface stimulation (Brindley, Dobelle)

0.18 mm

0.18 mm

0.46 mm

0.41 mm 2.2 mm

0.31 mm `

0.68 mm

From: Atlas of Cytoarchitectonics - Von Economo

Penetrating stimulation Can we make the communication bi-directional? Can we treat communication as a control problem? Optogenetics & closed loop control

Optical 0.18 mm

0.18 mm

0.46 mm Electricall Light stimulus 0.41 mm 2.2 mm Electrical recording 0.31 mm

0.68 mm

From: Atlas of Cytoarchitectonics - Von Economo

Penetrating stimulation Can we make the communication bi-directional? Can we treat communication as a control problem? Controlling Abnormal Network Dynamics with Optogenetics

£10M project: Clinical trials aimed for 2021

19 Noise cancellation of epileptic seizures – but adaptable to vision Adapting to visual brain prosthetics

Brain unit (Actually implant unit different position)

Control unit Connecting lead

Central control unit inc wireless comms External headset CANDO - Epilepsy Visual brain prosthesis

Principles: 2mm 1mm 4mm 1. Custom brain implant starting as 4 x 4 x 4 LEDs can be utilised TX Skin Fat Muscle RX 2. The control unit is quite different – higher power, no battery Average 5 mm 3. Cabling can be simpler Brain stimulator unit

Communications Power

Digital control unit (FSM)

Recording Diagnostic Stimulation Circuits Circuits Circuits

Interface unit Optrode head Recording Active electronics Electrode

Optrode Mounting Photonic plate Stimulation site

1. Ramezani et al. submitted to IEEE TCAS-I, June 2017 Optrode 2. Zhao et al. Implantable optrode GB1410886.4 shaft 3. Dekhoda et al. Temperature sensor GB1522790.3 4. Degenaar Optical stimulation arrangement GB1616725.6 5. Constandinou et al On Chip Random ID generation GB1702623.8 Recording microelectronics C

Vin VON VOP 50C LNA VIP VIN 6C Gm VOUT Vref VIP VIN

C=100fF

C

Front end amplifier Programmable gain amplifier

Recordings from non-human primate brain LED driver circutry

• Amplitude modulation • PWM modulation • Diagnostic and methods

LED drive power LED efficiency

DAC How much light do we need?

Lambertian emitter Shaped emitter Pseudo-collimated emitter 2 2 2 Threshold = 1mW/mm Threshold = 1mW/mm Threshold = 1mW/mm 100 )

W 10 m (

r e 1 w

o

P

l

a 0.1 c i t p

(a) O 0.01 (d) (e) (f) 0.001 0 200 400 600 800 1000 0 200 400 600 800 1000 0 200 400 600 800 1000 Tissue penetration depth (μm) LED diameter 120μm 100μm 80μm 60μm 40μm

Threshold = 1mW/mm2 Threshold = 0.1mW/mm2 Threshold = 0.01mW/mm2 100Key take home message:

) 101. Size doesn’t matter to penetration depth! W

(b) m ( 12. Need to penetrate at least 100-200µm for long term implantables due to gliosis r e

w 0.13. More collimated light will give better penetration o P

l 0.01 a c i t 0.001 p Dong et al. For submission to J. Biomedical Photonics 2017 O 0.0001 (g) (h) (i) 0.00001 0 200 400 600 800 1000 0 200 400 600 800 1000 0 200 400 600 800 1000 Tissue penetration depth (μm) LED type (c) Lambertian Shaped Pseudo-collimated ΔT (°C) ΔT (°C) 10 10 10mW 1 LED on 10mW 8 LEDs on 5mW 2.5mW 5mW 1 1

2.5mW 0.1 0.1

0.01 0.01

Optical stimulus Optical stimulus

LED thermal impact (a) 1ms 10ms100ms 1s 10s (b) 1ms 10ms100ms 1s 10s 0 1.0 2.0 3.0 Dimension (μm) 2.0 ΔT (K) 0 . 0 0 0 ΔT (°C) 0 . 1 0 0 0 ΔT(K) 200 ΔT (°C) 2.0 0 . 2 0 0 0 (e) II 0 . 3 0 0 0 3 1.5 0 . 4 0 0 0 Optrode 10 P 0 . 5 0 0 0 0 1.5

L 0 0 . 6 0 0 0 1P LEDp rofilne 10mW 8 LEDs on 10mW

1.0 2 0 . 7 0 0 0

0 . 8 0 0 0 Δ 0.5 ) T (K) II 0 . 9 0 0 0 ) K 0 . 0 0 0 1 . 0 0 0 (

0 m 10. . 10 0 0 0 S S 1.0 T

0 . 2 0 0 0 μ Δ III ( 8 00. . 38 0 0 0

0 . 4 0 0 0 n o 0 . 5 0 0 0

0.6 i 0 . 6 0 0 0 s 0.5 00. . 74 0 0 0 2 n e 00. . 82 0 0 0 (d) I 0 . 9 0 0 0 I m i 6 1 . 0 0 0 0 100μm (b) D 0 5mW 2.0 5mW ΔT (K) (f) III L 2.0 4 P 1.5 1.5 1 ) E 0.000 1.0 K ( 2.5mW 0.5 T 2.5mW 0.1000 L 0 Δ 2 0.2000E 1.0 0.3000 S 0.40000 0.5 0 0.5000

0.6000 0 m Optical stimulus Optical stimulus

0.7000

μ Dimension (μm) 0 1000 0 0.8000 0 1 No coating 20um polymer coating 0.9000 100μm (a) (c) 100μm On-optrode On-optrode 1.000 1ms 10ms100ms 1s 10s 1ms 10ms100ms 1s 10s L: LED E: Electrode P: Polymer coating 25um in the tissue S: Silicon substrate (c) 100um in the tissue (d) Maximum allowed pulse width Maximum allowed duty ratio (%) 10s 100 Target LED characteristics: Typical micro-LED efficiencies from literature ~ 1-5% 1s 80 Driving current: 1mA Driving voltage: 4-5V Wu et60 al. 2015: 0.8% 0.1s Kim et al. Science 2013: 0.2% 1 LED on 1 LED on Light requirement: 1mW McAlinden40 al: Opt Lett 2013: 5% 10ms Efficiency: >20% McGovern et al: PLOS ONE 2013: 5% 8 LEDs on McGovern et al:8 L IEEEEDs o nTBCAS 2010:1% 20 1ms 0 10 20 30 40 50 60 0 10 20 30 40 50 60 (e)Dong et al. ForThe submissionrmal power (tom WJ.) Biomedical(f )Photonics 2017Thermal power (mW) LED efficiencies

Wu et al Neuron 2015

Our LED 100µm 30µm 10µm @ 1mA 320 x 240! Dong et al. For submission to J. Biomedical Photonics 2017 Light requirement in-vivo

“Official” threshold is 0.7mW/mm2 for dissociated culture. But what about in practice???

In NHP experiments, We saw responses at 1mm with emission intensities of only 0.5mW. i.e. At that depth the irradiance << 0.1mW/mm2

Caveat: we cannot separate neural transmission!

Modelling indicates a lower effective threshold of 0.1 - 0.3 mW/mm2

Currently Unpublished work Luo et al. Submitted to JNE 2017 Assembling the optrodes

Schematic design CAD layout Wafer fabrication

Probe assembly 3D assembly

Control electronics Lightb)- emitting diodes

Recording sites

We are developing a fully manufacturable implant!! Developing the control system

Ontogenetically Sensitized tissue

Optrode

Implant Base plate

Internal control unit Camera Receiver Transmitter Power and antennae To receiver antennae Control unit antennae

Batt and cam External control unit Internal control unit Optrode unit

Power Power Power RX/ Voltage TX RX 3.7V module trans/Amp AC->DC regulator

USB Embedded BLE BLE Embedded SoC module module SoC Optrodes

Fattah & Soltan et al. in preparation for IEEE TBME submission System software - preprocessing

Visual augmentation The vision that we will return to patients with visual prosthesis will be rudimentary. We are therefore using patients with partially degenerate vision as a simulation of Simplification Cartoonization what can be achieved.

Song of the Machine! Full software process

Ontogenetically Sensitized tissue

Optrode

Implant Base plate

Internal control unit Camera Receiver Transmitter Power and antennae To receiver antennae Control unit antennae

External control unit Internal control unit Compression

Video/Image acquisition

Decompression Pulse coding Transmission

Simplification Retinal processing LED reconstruction

Fattah & Soltan et al. in preparation for IEEE TBME submission Visual cortical Headset Implantable control unit

44% 61% 75% 86% 80 80 70 70 60 (fps) 60 50 40 50 30 40 20 (mW) Power 30

MC Perfromance Perfromance MC 10 0 20 60 40 20 60 40 20 Frequency (MHz) Frequency (MHz) Timeline

Visual cortical trial??

CANDO trial

CANDO Engineering

CANDO First in human trial CANDO Current Regulatory & funding ends We are here ethical (Aug 2021) submission Acknowledgments OptoNeuro

CANDO project Tim Constandinou (Imperial College), Nick Donaldson (UCL), Andrew Sims (Newcastle RVI (NHS)), Andrew Jackson (Newcastle IoN), Mark Cunningham (Newcastle IoN), Anthony O’Neil (Newcastle EEE), + Others:

OptoNeuro FP7 project Mark Neil (Imperial College), (Max Planck Institute), Christian Bamann (Max Planck Institute), Botond Roska (FMI, Switzerland), Pleun Maaskant (Tyndall Institute), David Rogerson (Scientifica), Mark Johnson (Scientifica)