Contributions to Bionic Eye ResearchDARIEN PARDINAS DIAZ

2016/06/03

OutlineOverview of the Bionic Eye project

◦ First prototype and operation principle◦ Neurostimulator and electrode configurations◦ Existing psychophysics experiments

First prototype – Contributions◦ From single electrode stimulation to image recognition psychophysics◦ Pulse dose quantification and patient safety◦ Outcomes of the first trials

Second prototype – Contributions◦ Development of a Portable Vision Processor◦ Stimulation strategies and real-time pulse dose quantification algorithms◦ Stimuli validation framework◦ Psychophysics application redesigned

Summary

2

First prototype“Device intended to restore functional vision”

Joint effort between the Bionics Institute and CERA produced the first Australian prototype of a bionic eye tested in humans

3

Operation principle

IR Image: Centre for Eye ResearchAustralia (CERA)

4

NeuroBi – The neurostimulator

Common GroundMonopolar

Pseudo-Hex(Quasi-Monopolar)

Hexagonal

Electrode buffer (active/return) Pulse buffer (PW, PPD, IPG, etc.) Sequence buffer (µA, n)

5

Psychophysics Experiments Determine stimulation

thresholds Dynamic range Phosphene shape and

size characterisation SOP - Impedance

measurement

Characterise safe and effective electrical stimulation levels

Evaluate implant stability over the course of the clinical trials

6

EyeSee app

First prototype – Contributions

There were many experiments in place to study single-electrode stimulation, but not much work had been done to study the image recognition capabilities of the prototype

7

From single electrode stimulation to image recognition psychophysics

1. Integration of a vision processing library developed by NICTA CRL

Implemented with OpenCV

Provided different ways to sample the input image – NN, Lanczos2

C#/WPF -> C++/CLR wrapper to convert WPF image data into OpenCV and call the library

8

From single electrode stimulation to image recognition psychophysics

2. Development of a real-time stimulation algorithm capable of transforming phosphene intensities into pulse sequences

Other experiments used ImPreSS (latency)

NeuroBi had a time overhead to load the stimulation sequence

Implemented algorithm to convert phosphene intensities into interleaved train of pulses and upload with minimum overhead

Ability to play input movies at 10 FPS

9

From single electrode stimulation to image recognition psychophysics

3. Generation of synthetic static images and GIF movies of shapes of different sizes calculated in FOV units

Moving gratings, bars, circles, etc. multiple directions and speeds (°/sec)

6°width moving gratings speed = 4°/sec

3°width moving barspeed = 4°/sec

10

From single electrode stimulation to image recognition psychophysics

11

Eye gaze affects percept location

Data collation, transformation and visualisation

12

Pulse dose quantification and patient safety

NeuroBi had the ability to acquire and send voltage waveform, but this data was never recorded during patient sessions

Challenges:

-16bits resolution/100kHz sample rate for long periods of time

-Raw data didn’t specify from which electrode configuration

Proposed solution:

-Implemented an efficient data logging module that allowed recording all the stimulation activity for detailed analysis of the patient session

Allowed researchers to:

-Verify that established safety limits of pulse charge were observed and that stimulation pulses were delivered as intended

-Explore new dose limits by stimulating the eye while running a live retinal OCT

-Study impedance dependency on the electrode configuration, pulse parameters, stimulation period

13

Pulse dose quantification and patient safety

14

Automated report generation of patient sessions for analysis

Outcomes of the first trialsProof of concept that a suprachoroidal visual prosthesis can provide basic vision

Larger electrode surface area are more effective – less stimulation power

Simple monopolar proved to be the most effective electrode configuration

Percutaneous connector is not suitable for a take-home scenario

Continuous stimulation caused the phosphenes to fade after a few seconds

Phosphene location changes with eye gaze angle – eye tracker can help

Phosphenes overlap significantly with one another – brain plasticity can help

Neurostimulator needs to provide better real-time stimulation capabilities

Challenges to overcome

15

Second prototype – 44ChFIOperation Modes: Take-home system Stimulator for psychophysics

research

Pulse Streaming

PulseGeneration PIC

CI

CI

SPI

RF

Vision Processing

ImageCapture

Video

framesPhosphene

intensities

Pulse

train

Pulse

stream

ARM based processorRF

Implanted

EyeSeeApp

TCP

16

Second prototype – test bed

Load Board – LEDs version

Implant Emulators

PVP Unit

17

Stimulation strategies to mitigate phosphene fading effect“Continuous electrical stimulation causes the retina to desensitize and in humans it leads to a reduction in the brightness of phosphenes and also increases threshold” - Davuluri (2014)

• Phase width

• Interphase gap

• Anodic vs. cathodic first

• Phase amplitude attenuation

• Pulse period

18

Stimulation strategies design

• Suitable for both basic psychophysics and real-time video feed

• Producer-consumer paradigm

• Run efficiently on embedded hardware

class Stimulation Strategy

StimulusConfiguration

+ PhospheneDefinitions

+ PulseStrategies

PhospheneDefinition

+ ElectrodeConfigurations

+ MaxRating

+ MinRating

+ PulseStrategy

PulseStrategy

+ AnodicFirst

+ AttenuationFactor

+ InterPhaseGap

+ PhaseWidth

+ PulsePeriod

ElectrodeConfiguration

+ ActiveElectrodes

+ GammaFactor

+ MaximumCurrent

+ MinimumCurrent

+ ReturnElectrodes

Defines what electrodes

are involved in the

stimulation and the

range of currents that

could be used for

stimulation

Pulse parameters are

potentially different for

each pulse to be

generated

1..2

1

11

1..*

1

1..*

19

Proposed stimulation algorithmDesigned a generic algorithm capable of producing a pulse stream from any possible variation of pulse parameters (~3kpps)

Implemented as a priority queue of pulses with variable time slot

Algorithm very efficient with little memory allocation

act Stimulus Generation Algorithm

Initialise Phosphene

Requests

stimulation time

consumed?

Retriev e

first

phosphene

request

time_to_stimulate <= 0

Try

Generate

Pulse

A pulse can only be generated if

for all electrode configurations the

charge and energy dose are

below the allowed limits

pulse was

generated?

Append to

pulse train

Reschedule

Phosphene

Generat Null

frames to cov er

time_to_stimulate

Done with this

stimulation

request

Commit

pulse

Phosphene requests are

kept in a priority queue

sorted by their time to

generate the next pulse

[no]

[no]

[yes]

[yes]

[no]

[yes]

20

A real time dose tracker to ensure patient safety

Multi-window size dose trackers running over the same pre-allocated circular buffer

Efficient non-blocking binary pulse stream logger

Charge

Energy

Time

Charge

Energy

Time

Charge

Energy

Time

Charge

Energy

Time

Charge

Energy

Time

Now33ms ago1s ago

𝑄,𝐸 ≤ 𝐿𝑖𝑚𝑖𝑡33𝑚𝑠

𝑄,𝐸 ≤ 𝐿𝑖𝑚𝑖𝑡1𝑠

Charge

Energy

Time

Charge

Energy

Time

?

21

Automated validation of stimuli delivered

• 32 Independent Channels• 16 bits resolution• Fs = 250kHz

Integration test to stimulate and record the pulse waveforms

Extraction of pulse parameter from recorded signal (resolution error ≤ 4µs)

Algorithm to match extracted pulses with generated pulse train

Purpose: Validate pulse streaming from PVP to the electrode array

22

Psychophysics application redesigned No automated tests in place

Anti-patterns (God classes, code duplicates, no abstractions, etc.)

Highly coupled to the stimulation principle of NeuroBi and to predefined electrode configuration modes

Designed and implemented a device agnostic application based on generic stimulation conceptsIncreased application maintainability by reusing common functionality across experiment procedures – e.g. stimulus output visualisation, etc.Implementing unit tests for critical components, etc.

23

Stimulus Configuration Editor

24

Reusable infrastructure

25

Stimulator output visualisation

SummaryVery exciting project and great team of researchers

Very dynamic project – there is not much that can be anticipated

Got to apply a broad range of technical skills

Practiced to work effectively with legacy code

26

Thank you!

Questions?

27