SPECIAL ISSUE ON BIOMEDICAL CIRCUITS AND SYSTEMS 1 1 2 3

SPECIAL ISSUE ON BIOMEDICAL CIRCUITS AND SYSTEMS 1

1 2 3


Visual Neuroprosthesis: A Non Invasive System forStimulating the Cortex

M. Piedade, J. Gerald, Member, IEEE, L. Sousa, Senior Member, IEEE, G. Tavares, Member, IEEE,P. Tomas, Student Member, IEEE

Abstract— This paper describes a complete visual neuropros-thesis wireless system designed to restore useful visual sense toprofoundly blind people. This visual neuroprosthesis performsintracortical microstimulation through one or more arrays of mi-croelectrodes implanted into the primary visual cortex. The wholesystem is composed by a primary unit located outside the bodyand a secondary unit, implanted inside the body. The primaryunit comprises a neuromorphic encoder, a forward transmitterand a backward receiver. The developed neuromorphic encodergenerates the spikes to stimulate the cortex by approximatingthe spatio-temporal receptive fields characteristic response ofganglion cells. Power and stimuli information are carried toinside the cranium by means of a low-coupling transformer,which establishes a wireless inductive link between the two units.The secondary unit comprises a forward receiver, microelectrodestimulation circuitry and a backward transmitter that is used tomonitor the implant. Address Event Representation (AER) isused for communicating spike events. Data is modulated withBinary Frequency Shift Keying (FSK) and Differential BinaryPhase Shift Keying (DBPSK) in the forward and in the backwarddirections, respectively. A prototype of the proposed systemwas developed and tested. Experimental results show that thespikes to stimulate the visual cortex are accurately generatedand that the efficiency of the inductive link is relatively high,about 28% in average for 1 cm intercoil distance providing apower of about 50 milliwatts to the secondary implanted unit.Application Specific Integrated Circuits (ASIC) were designedfor this secondary unit, showing that, with current technology,it is possible to implement such a unit, respecting the powerconstraints.

Index Terms— Biomedical implant, wireless neuroprosthesis,artificial retina, neuromorphic encoder, smart implant, inductivecommunication link, cortical stimulation and monitoring.

I. INTRODUCTION

IN the last few years there has been a huge effort todevelop implantable integrated stimulators for bio-medical

applications. Most of these stimulators are in the area ofmuscular stimulation (for instance, for heart or limbs diseases)[1]–[5] and in the area of cortical or nerve stimulations (e.g.for blind/partially blind people or hearing diseases) [6]–[12].In all these stimulators, information has to be coded in a formatsomewhat similar to the way stimulation is performed prior to

Manuscript received February 1, 2005.Authors are with the Electrical and Computers Engineering Dept.

at Instituto Superior Tecnico, Technical University of Lisbon andwith INESC-ID, Rua Alves Redol 9, P-1000-029 Lisboa, Portugal;email:msp,jabg,las,goncalo.tavares,[email protected]

This work was partially supported by the Portuguese Foundation for Scienceand Technology (FCT) through Program FEDER, by FCT and FSE (FundoSocial Europeu) in the context of III European Framework of Support,and by the European Commission through the project ”Cortical VisualNeuroprosthesis for the Blind” (CORTIVIS), QLK6-CT-2001-00279.

the development of each of the particular diseases. Moreover,with the exception of a few applications (e.g. pacemakers),long time performance of the implanted stimulator impliesthe power to be furnished by the outside unit. It avoidsfuture operations to replace old batteries, reduces the risk ofinfections and therefore improves the patient health level. Thisposes the need for wireless delivery of both power and data[10]–[12].

A lot of research has been performed in the last fiftyyears for understanding the structure of the mammalian visualsystem, its main components, and the relationship betweenelectrical stimulation of any part of the visual pathwaysand the resulting visual sensations. It has been shown thatelectrical stimulation of individual components of the visualpathway such as the retina, the optic nerve or, directly, thevisual cortex, evokes the perception of points of light spatiallylocated, denominated by phosphenes. Based on this nuclearobservation, research labs and projects have been dedicatedto the study of the human visual system, researching accuratemodels for the visual system and developing visual prosthesescapable of partially restoring vision to blind people.

Microelectronic prostheses which interact with the remain-ing healthy retina have been developed to restore some visionto those who suffer from eye diseases, such as retinis pigmen-tosa. This type of prosthesis can use sub-retinal devices [13] —to replace the photoreceptors — or more complex epi-retinaldevices [14] — for capturing and processing images which aretransmitted to the ganglion cells through an electrode array—, but require that output neurons of the eye and the opticalnerves are in a healthy state. When this is not the case, thesemicroelectronic prostheses are not useful. The stimulation hasto be performed in the primary visual cortex, directly tothe neurons in higher visual regions of the brain. This is achallenge, because visual information has to be encoded ina format somewhat similar to the way stimulation was doneprior to the development of total blindness. It is expectedthat neurons will adapt to the stimulus in a way that a blindindividual will be able to extract from it information on thephysical world [15].

Brindley [16] and Dobelle [17] showed that simultaneousstimulation of multiple electrodes allowed blind volunteers torecognise simple patterns. This research however also showedthat a cortical prosthesis based on a relatively large numberof superficial implanted electrodes requires high currents toproduce phosphenes — more than 1 mA — which leads toproblems such as epileptic seizures. In this cases, deep intra-cortical neuro-stimulation should be used, exciting the neurons


PowerDataRF Link

NeuromorphicEncoder

Electrode Stimulator

DigitalCamera

Secondary coil

Primary coil

Fig. 1: Cortical Visual Prosthesis.

at a depth between 1 mm and 2 mm, which corresponds to thecortical layers 4 (namely, Cα and Cβ), where signals fromthe Lateral Geniculate Nucleus (LGN) arrive.

This paper presents a cortical visual neuroprosthesis forprofoundly blind people which has been developed within thescope of the CORTIVIS project and supported by the EuropeanCommission [18], [19]. CORTIVIS started in January 2002within the scope of the European program Quality of Life andManagement of Living Resources, and required the work ofmulti-disciplinary research groups, specialised in areas such asneuro-anatomy, neuro-surgery, signal processing or microelec-tronics. The project aims at developing prototypes in the fieldof visual rehabilitation and at demonstrating the feasibilityof a cortical neuroprosthesis to interface the visual cortexthrough intracortical microstimulation, as a means throughwhich a limited but useful visual sense may be restored toprofoundly blind people. The designation “profoundly blind”means that the output neurons of the eye, the ganglion cells,often degenerate in many retinal blindnesses and therefore aretinal prosthesis would not be helpful. It is expected thatneural plasticity will help to associate phosphenes with thecorresponding physical world. In parallel with the CORTIVISproject other labs and projects are performing research withthe same objective namely the Meister Lab [20] which isresearching models for the visual and auditive systems. TheJohn Moran Labs in Applied Vision and Neural Sciences [21]at the University of Utah is another example of the currenteffort to develop an intracortical visual neuroprosthesis.

Figure 1 illustrates the basic components of our CorticalVisual Neuroprosthesis approach. The whole system uses abioinspired visual processing front-end, the NeuromorphicEncoder, which generates the electrical signals that are trans-ferred to inside the skull through an RF link, for stimulatingan array of penetrating electrodes implanted into the primaryvisual cortex. Power to operate the stimulating circuitry is alsoprovided from the outside through the RF link. This visualneuroprosthesis is expected to recreate a limited, but usefulvisual sense in blind individuals, by applying the electricalsignals to up to about 1000 microelectrodes provided by

several Utah Electrode Arrays (UEAs). One UEA has 100microelectrodes, each 1.5 mm long, arranged in a square gridand contained in a package covering an area of approximately4 mm×4 mm (see figure 2). This UEA has to be priorlyinserted into area 17, the primary visual cortex, by using adedicated system developed to implant the array of microelec-trodes in a manner that minimises dimpling and compressionof the subjacent structures. The microelectrode array is in-serted to stimulate up to layer 4C, where the LGN axons enter.To the best of our knowledge, this paper is the first one tocover the design and implementation of all the components ofan intracortical visual neuroprosthesis. It implements severalinnovative features, namely at the architecture level of theneuromorphic encoder and on the solutions adopted to conveydata and energy through the wireless communication link tothe brain.

In what concerns the neuromorphic encoder, the simplestmodels assume that the neural coding can be reduced topredicting the firing rate as a function of the sensory stimulus.This assumption may be justified in certain brain areas, e.g.deep in the cortex [22]. On the other hand, neurons inthe early visual system, for example from the retina to theLGN, can deliver reproducible spike trains, whose trial-to-trial variability is clearly lower than the one predicted fromthe simple firing rate approach [23]. To predict individualspikes, spike patterns with higher timing accuracy and also toaccount for the stochastic variability of these responses otherapproaches have to be considered. The first retina model toprovide a quantitative output was proposed in [24]. The spatialprocessing was modelled with a Difference of Gaussians(DoG), representing the receptive field’s centre and surroundareas. After convolving the stimulus with this spatial kernel,the signal is processed by a temporal highpass filter thatoutputs the firing rate. Another block is required to convertthis firing rate into spike trains. In 2001, a new model wasproposed that introduced a non-linear feedback loop at theoutput of the temporal filter [25], performing a Contrast GainControl (CGC). The loop is composed by a temporal lowpassfilter followed by a non-linear function and feedback througha multiplier, introducing a modulation factor in the response.Also in 2001, [22] reported a stochastic model to predictthe variability in the number of spikes and on its time ofoccurrence, introducing a spike generation block instead ofusing firing rate as output. It uses only temporal processing,but spatial processing can be included considering time-spaceseparability. Stimuli are processed by a temporal filter, basedon the spike triggered average — the first Wiener kernel of anexpansion series of the ganglion cell response [26]. Variabilityin spike occurrence is done by adding two noise sources tospecific points of the model.

A first experimental work comparing the performance ofthese models has been recently published by our researchgroup [27]. Based on this study the model with the CGC [25]was adopted in the CORTIVIS project. Particular aspects of theretina organisation are considered in this work. For example,retinotopic gradients and magnification factors are config-urable parameters of the system, which allow to compensatethe fact that sampling across the retina is not uniform [19].


The wireless radio frequency (RF) link must provide bothpower and bidirectional transmission of data between theoutside and the implant. While the visual neuroprosthesisrequires a few tenths of milliwatts of power, the actual powerprovided from the outside must be somewhat higher, in orderto account for the coupling losses. The most common solutionsproposed in literature for the RF link are Amplitude ShiftKeying (ASK) modulation [28]–[33], Frequency Shift Keying(FSK) [28], [34]–[37] and Differential Binary Phase ShiftKeying (DBPSK) modulation [28], [38]. The ASK signalis usually demodulated in a noncoherent fashion, thereforerequiring a very simple receiver. However, the performanceof the ASK receiver is highly dependent on the amplitudeof the received signal, which is unknown and may vary intime. Therefore, an ASK solution requires high-efficient gain-monitoring and level-controlling devices. This is not the casewith either FSK or DBPSK, which are constant amplitudemodulations. Within reasonable limits, the FSK and DBPSKreceivers are insensitive to the amplitude of the received signal,making these circuits more robust regarding the high variabil-ity of the RF channel. Demodulation of FSK and DBPSKis usually done coherently, thus requiring more complex andpower consuming receivers.

The choice of the modulation format has impact on twoother important features of the implant receiver, namely therequirement for wireless powering of the implant and the needfor recovery of a master clock (from the incoming data signal).Carrier frequency recovery is important because it allows thedigital circuitry in the implant to operate with a clock which isfrequency-synchronised with the received signal. This avoidsintermodulation noise components and contributes to increasethe overall system performance. With ASK modulation carrierrecovery is somewhat easier since, to allow for nonchoerentdemodulation, a fraction of the unmodulated carrier has tobe sent together with the information signal. With FSK orDBPSK, the transmitted spectrum does not contain a specificcarrier signal. Consequentially, carrier recovery is more dif-ficult and requires a narrower Phase Locked Loop (PLL) orbandpass filter. On the other hand, power extraction is easierfrom the constant amplitude FSK or DBPSK signals than fromthe amplitude-varying ASK signal. Another important issue isthat power recovery by the circuitry in the secondary systemdegradates the received signal, changing both its waveformand envelope amplitude therefore introducing a kind of noisedependency on the time varying power consumption of thecortical prosthesis. So, this paper proposes the use of a datamodulator type that does not uses amplitude variation. Aftercareful analysis of the different modulation characteristics,FSK was chosen as the modulation to use in the forwardlink. For monitoring purposes a reverse data link has alsobeen implemented. It uses DBPSK, which is implementedwith a very simple transmitter. DBPSK was not selected forthe forward link because the receiver (Costas Loop) is morecomplex than the FSK receiver.

This paper is organised as follows. In section II, the maincomponents of the system are specified and an architec-ture is proposed to fulfil those specifications. Sections IIIand IV present the neuromorphic encoder and the wireless

communication link, respectively. A prototype of the systemwas implemented for which some experimental results arereported in Section V. Section VI concludes the paper anddiscusses the future work to continue the development ofvisual neuroprostheses.

II. SYSTEM ARCHITECTURE

The system architecture of the proposed visual neuropros-thesis is represented in Figure 2. It is physically divided intwo units: a primary unit located outside the body and asecondary unit consisting of an implant located inside thebody. The units are connected by means of a low-couplingtwo-coil transformer, which establishes a magnetic RF linkbetween the two devices. The purpose of this RF link istwofold: firstly, it is used to remotely power the secondaryunit, a mandatory requirement to avoid the use of batteries;secondly to allow bidirectional data communication betweenthe units. The primary unit interfaces with a miniature digitalvideo camera and with a personal computer used for systemconfiguration, patient visual training and prosthesis perfor-mance analysis purposes.

The Neuromorphic Encoder translates the visual signalcaptured in the miniature digital camera into a sequence ofelectrical pulses, a spike train, capable of being recognised bythe brain. It is composed by two main modules connected inseries. The Early Layers, based on models previously proposedin [25], is responsible for processing the visual signal and forits conversion into a spike rate.This rate is then taken as inputby the Neuromorphic Pulse Coding module and translated intothe actual spike events by using a simplified version of theintegrate-and-fire spiking neuron [39]. The access to the RFlink is then arbitrated by the latest module using a First-InFirst-Out (FIFO) buffer to store the events until the link isable to send more spike events. In the Visual Neuroprosthesispresented in this paper, the Neuromorphic Encoder is requiredto process visual images at a rate of 30 frames per second(fps), generating spikes at a rate up to 100 Hz and stimulatinga number of microelectrodes equivalent to an array with up to32× 32.

The Forward Transmitter receives the multiplexed spikedata from the Neuromorphic Pulse Coding in a synchronousserial bit stream format at a rate of fb = 1 Mbps. This datais modulated using Frequency Shift Keying (FSK) modulationwith a centre frequency of fc = 10.333 MHz and a frequencydeviation ∆f = ±333.3 kHz. To understand this modulationchoice we need to digress briefly and characterise the com-munication channel between the primary and secondary units.While high level noise is not expected (either thermal or orig-inated from magnetic disturbances), it is reasonable to assumethat the channel will exhibit a high amount of attenuationand, due to the relative movement/placement between the twotransformer coils, severe amplitude fluctuations. With FSKmodulation, information is coded in carrier frequency changes,not in amplitude levels (it is a constant-amplitude modulation)and therefore it offers significant robustness regarding to theprevious channel impairments. On the secondary unit, power isextracted from the received signal by a power supply generator


NEUROMORPHIC CODER

EARLY LAYERS

NEUROMORPHIC PULSE CODER

CONTROL

FORWARD TRANSMITTER

BACKWARD RECEIVER

FORWARD RECEIVER

BACKWARDTRANSMITTER

ELECTRODES STIMULATOR& SENSING

RF link

PRIMARY UNIT SECONDARY UNIT

PERSONALCOMPUTER

CAMERA

Fig. 2: Intracortical Prosthesis System Architecture.

circuit (considered to be part of the Forward Receiver inFigure 2) which will be described later in Section IV. Becausethe integrity of brain tissue does not allow the use of highpower stimuli, the secondary unit requires only tenth’s ofmilliwatts of remotely delivered power. The spike data isrecovered by the Forward Receiver which consists on anFSK demodulator, bit synchroniser and frame disassemblycircuit. The recovered data and clock are then forwarded tothe microelectrodes stimulator circuit.

To accomplish implant monitoring (e.g. electrodeimpedance measurement and calibration) a reverse datalink has been developed. In the secondary unit, maintenancedata is modulated and transmitted using DBPSK at a datarate of fb = 156.25 kbps using low amplitude (≈ 1 V)and a carrier frequency fc = 5 MHz. This modulation waschosen because the transmitter is simple and consumes verylow power. Being also a constant-amplitude modulation, itexhibits the same robustness of FSK with respect to thechannel adversities. In addition, for the same average bit errorrate, it offers a signal-to-noise-ratio (SNR) gain of 3 dB.The disadvantage of DBPSK is that it requires a relativelycomplex, power-hungry receiver (significantly more complexthan its FSK counterpart) and this is why DBSPK was not thechosen modulation for the forward link (note that the forwardreceiver is located in the remotely-powered secondary unit).In the case of the backward receiver, located on the primaryunit, complexity (or, for that matter, power consumption)is not an issue and the previously mentioned advantages ofDBPSK may be exploited.

The main objective of the Electrode Stimulators and Sensingmodule is to stimulate the primary visual cortex and tomeasure the connectivity between the electrodes and cortex.Extensive experiments have shown that in order to safelyinduce phosphenes a current of 20 µA is required. Howeverit expected for small variations in this value to be requiredfrom microelectrode to microelectrode and patient to patient.The actual current value should therefore be set during thetraining phase. Electrode sensing is achieved by injecting afixed current value to a microelectrode and then measuringthe induced current value. This allows for measuring theconnectivity between the microelectrode and the visual cortex

cells.The Electrode Stimulators and Sensing module is composed

of a set of Digital to Analog Converters (DACs) which areused to stimulate up to 32× 32 microelectrodes in the visualcortex with the spike events received from the NeuromorphicEncoder. Each DAC has a current-steering type architecture,using scaled currents added at an output node, in order toensure the desired linear digital-to-analog conversion. Themain difference with respect to the common current-steeringDAC is that it allows for the removal of the charge from thebrain: it can inject current (positive sign) or remove current(negative sign) from the electrode. The DAC reference currentis around 20 µA and the overall power consumption forstimulating and sensing the 1024 electrodes is expected to be50 mW. The amplitude and duration of the stimulus is pre-recorded in dedicated registers on the Electrodes Stimulatorand Sensing module, which can be accessed by means of theRF link for both reading and writing.

The overall system is presently under development in VLSItechnology. The Electrode Stimulator and Sending block is tobe implemented in a modular way in AMS CMOS 0.35 µmtechnology: each module (except for the power supply genera-tor) drives a 100 microelectrode array, which will be attacheddirectly to the VLSI chip (flip chip technology). The Forwardand backward emitters and receivers are also being developedusing AMS CMOS 0.35 µm technology. Moreover, in orderto evaluate the circuit requirements when implementing thein deep sub-micron technology, the Neuromorphic Encoderis now being synthesised using the UMC 0.13 µm CMOStechnology process [40]. The UMC L130E SG-HS 1P8Mprocess is a single poly, 8 metal layer with an operating voltageof 1.2 V.

However, to rapidly prove the concepts involved in the RFlink, by using conventional integrated circuits and discretecomponents, a scaled frequency prototype operating 10 timesslower was built: the forward data rate is 100 kbps with a car-rier frequency of 1 MHz; the backward data rate is 15.625 kbpswith carrier frequency of 500 kHz. Also a NeuromorphicEncoder was developed using FPGA technology.

The visual neuroprothesis system was planed to support thedriving of 1024 electrodes. However, the stimulator chip that


VisualStimulus

Spike Multiplexing RF LinkSpike

GenerationVisual

Stimulus

Contrast Gain

Control

Motion detection

+

R

G

B

Edge detection

Early Layers Neuromorphic Pulse Coding

Rectifier

Fig. 3: Neuromorphic Encoder.

is being developed will be flipped directly over the back of onemicroelectrodes array (UEA with 100 microelectrodes). Thissolution will solve the electrode wiring problem. Each one ofthe 100 ADC converters are being designed around each padimplanted in a matrix form like the UEA. Each module will bepacked in a ceramic hermetic package. The set flip chip/UEAwill be placed over a thin film printed circuit over the packagebase besides the power supply recovering circuitry. Secondarycoil may be inside the ceramic package or in a separated placeunder the skin.

III. NEUROMORPHIC ENCODER

The Neuromorphic Encoder herein presented and shown inFigure 3, is an approximation of the spatio-temporal receptivefields characteristic response of the retina ganglion cells. Theencoder is composed of two main blocks: the Early Layersblock, that performs both spatial and temporal processingof the visual signal; the Neuromorphic Pulse Coder block,responsible for the conversion of the visual information toa sequence of pulses capable of being interpreted by thebrain. It uses the Address Event Representation (AER) [41]protocol to communicate information without timestamps tothe microelectrode stimulator placed inside the human brain.The architecture of the system has been designed taking intoaccount the specifications of the problem, namely the require-ment of low power real-time processing of visual stimuliand stimulation of up to 1024 microelectrodes. To this end,the Neuromorphic Encoder translates a visual stimulus s(r, t)described by the intensity of each of the three basic colourcomponents - red, green and blue - as a function of spacer = [ x y ]

T and time t

s(r, t) = [sR(r, t) sG(r, t) sB(r, t) ] (1)

into a sequence of pulses. The encoder was designed to allowa blind individual to recognise patterns of 32× 32 points.

A. Early LayersThe Early Layers block, responsible for spatio-temporal

filtering, is based on research published in [25] and extendedto the chromatic domain by considering independent filters foreach basic colour component. The first filtering element of theEarly Layers is an edge detector composed by a set of twoGaussian spatial filters per colour channel (see Figure 4(d)1):

gij =aij

2πσ2ij

e− ||r||

2

2σ2ij . (2)

1In this figure, signals are represented in the discrete time domain where,s[q, n] corresponds to s(r, t) in the continuous time domain.

The Gaussian filters are parameterised by a gain aij and astandard deviation σij , where i represents the colour channeland j the number of the filter in the set.

In order to perform edge detection, Gaussian filters withinthe same colour component are parameterised with oppositesigns, ai1 = −ai2, and different standard deviations, σi1 6=σi2. The output is then processed by a set of temporal first-order lowpass filters with impulse response

hij(t) = H(t) · βij · e−βij ·t (3)

where H(·) represents the Heaviside step function and βij thedecay rate. The output of the edge detection module will thenbe as in (4), where ∗ represents the convolution operation.

m(r, t) =∑

i=R,G,B

si(r, t) ∗

2∑

j=1

(gij(r) · hij(t))

. (4)

The m(r, t) signal is then convolved with the impulse responseof a first order highpass filter with the pole at −α rad/s asexpressed in (5), to perform motion detection

hHP (t) = δ(t)− α ·H(t) · e−α·t. (5)

The resulting activation function u(r, t) = m(r, t)∗hHP (r, t)is then modulated by the Contrast Gain Controller (CGC)which models the strong modulatory effect exerted by stimuluscontrast. The CGC non-linear approach is also used in order tomodel the motion anticipation effect observed on experimentswith a continuous moving bar [24]. The CGC loop works asfollows. First, the CGC output y(r, t) is convolved with theimpulse response of a lowpass temporal filter with a pole at−γ rad/s

hLP (t) = H(t) · γ · e−γ·t. (6)

The resulting signal, v(r, t) = y(r, t) ∗ hLP (r, t), is thenprocessed by a non-linear function

k (v(r, t)) =1

1 + [v(r, t) ·H (v(r, t))]4 (7)

before being applied to modulate the amplitude of the CGCinput signal as expressed in (8)

y(r, t) = k(v(r, t)) · u(r, t). (8)

Finally, the last processing step of the Early Layers block isa rectifier operation represented in (9) which yields the firingrate f of the ganglion cells’ response to the input stimuli,where Ψ and θ define the scale and baseline values of thefiring rate.


f(r, t) = Ψ ·H(y(r, t) + θ) · [y(r, t) + θ]. (9)

Two main restrictions were taken into consideration whendesigning the circuits to implement the Early Layers: lowpower consumption and reduced circuit area. In order tocomply with these restrictions, folding techniques were appliedto the architecture originally obtained from the signal flowgraph (SFG) [42]. The complete Early Layers circuit wasfolded 1024 times and the firing rate for each microelectrodeis computed in series. Moreover, assuming a Gaussian kernelof 7 × 7, edge detecting takes a total of 98 multiplicationsand 98 additions. This requires the folding technique to beapplied to the Gaussian filters by a factor equal to the kernelsize. It allows to save computational units since only onemultiply-and-accumulate (MAC) unit is needed to computeeach Gaussian filter (see Figure 4(a)).

In order to implement the temporal filters presented in (3),(5) and (6), the bilinear approximation was used, resulting inInfinite Impulse Response (IIR) digital filters of the form:

y[n] = b · y[n− 1] + c · (x[n] + d · x[n− 1]) (10)

where b > 0 and c > 0 represent the temporal filter coeffi-cients, d = 1 for the lowpass filters in (3) and (6) and d = −1for the highpass filter in (5). Also, to reduce the amount ofmemory needed, the temporal filters were implemented in atransposed form where the output is calculated by adding theinput x[n] to a stored value l[n] which is computed in theprevious cycle as

y[n] = l[n− 1] + c · x[n] (11a)l[n] = b · y[n] + c · d · x[n] (11b)

Figure 4(b) shows the implementation diagram of the temporalfilters. The input scaling c in (10), existent in temporal filters(3), (5) and (6), was moved and applied simultaneously withthe calculation of the Gaussian filters and with the non-linear function, for (3) and for (5) and (6), respectively. Theremaining modules, the non-linear function presented in (7)and the rectifier shown in (9) were implemented by means ofa look-up table and a comparator, respectively.

B. Neuromorphic Pulse Coding

The Neuromorphic Pulse Coding (NPC) block performstwo operations. Firstly it converts the continuous time-varyingrepresentation of the signal produced in the Early Layers of theretina into a neural pulse representation: in this representation,the signal provides information only when a new pulse begins.This block then stores the information about spike events andsends them to the implant at the maximum rate allowed bythe channel. The second operation corresponds to arbitrate theaccess of the generated spikes to the serial bus which is locatedat the input of the Forward Emitter (see Figure 3).

The model adopted for the Spike Generation is a simplifiedversion of an integrate-and-fire spiking neuron [39]. Theneuron accumulates input values from the respective receptivefield (output firing rate determined by the Early Layers) until itreaches a threshold φ. Then it fires a pulse and discharges the

accumulated value. A leakage term is included to force theaccumulated value to diminish for low or null input values.The pulses are then generated accordingly to

Pacc[q, n] = F [q, n] + γ · Pacc[q, n− 1]−pulse[q, n− 1]− d (12a)

pulse[q, n] = H(Pacc[q, n]− φ) (12b)

where Pacc is the accumulated value, γ sets the decay rate ofP (decay = 1 − γ) and H(·) represents the Heaviside stepfunction. The implementation of the pulse generation circuitis shown in Figure 4(c). It works in a two stage pipeline: inthe first stage the input firing rate F [q, n] is added to theaccumulated value; in the second stage, the leakage value issubtracted and, if the result is bigger than the threshold φ, apulse is fired and the accumulator returns to zero. This blockis connected to the Early Layers by means of a dual portmemory bank. This allows for the Early Layers block to writedata onto one port while the Neuromorphic Pulse Coding blockreads data from the other port. The Spike Multiplexing blockuses a first in first out (FIFO) buffer to arbitrate the access ofthe spikes generated in the Spike Generating block to the RFlink, as shown in Figure 4(d). When a spike is generated it isstored in the buffer until the channel becomes available. Thebuffer allows the system to respond well to short periods inwhich the spike rate is high.

C. Model Evaluation

To evaluate the performance of this model, a study wasmade by one of the authors based on real experimental datafrom salamander retinal responses [27]. This study comparedthe performance of the presented model to the one of arecently published stochastic model which attempts to predictthe temporal occurrence of spikes and spike patterns [22].Model performance was assessed based on the mean squarederror (MSE) of the firing rate. This measure is defined by [23]

MSE =

∫(f(t)− r(t))2 dt∫

(f(t)− r)2 dt(13)

where f(t) is the firing rate produced by the model, andr(t) is the firing rate obtained from the peristimulus timehistogram (PSTH). This rate is calculated from the observedreal data by histogramming the spike times occurrence fromall experimental trials, and dividing its amplitude by the widthof the time bin and by the total trials number. Formally, for aspike train described by ρ(t), the firing rate in the time interval[t1; t2] averaged amongst M trials, is given by (14).

r(t) =1

M

∫ t2t1ρ(t) dt

t2 − t1(14)

The MSE for the firing rate of the presented model isabout 1.11, while for the stochastic model is 1.13 [27].These results show that by using a deterministic model, it ispossible to approximate the real neural retina response withan accuracy similar to that of the stochastic model. However,deterministic models require lower computational power, thusmaking them a more suitable approach for developing a visualneuroprosthesis.


x

Q

Memory bank: stores the kernel coefficients

++

+

Input

(a) Gaussian Filter Block.

+

Q D

x

+

Coef.+/-

+

+

+l[n-1] l[n]

x[n] y[n]

Memory Bank

(b) Infinite ImpulseResponse filter.

Memory Bank

+

d

f[q,n] + -

Φ

0

≥

MU

X

Q

D

pulse[q,n]xγ

(c) Spike Generation Block.

Low pass (IIR2)

Low pass (IIR1)

Gaussian1

++

-

High pass (IIR3)

x

Non Linear

Low pass (IIR4)V

isual stimulus

Gaussian2 Rectif.

QA

DBMemory bank:implements a FIFO

D Q

Early Layers Neuromorphic Pulse Coding

QA

DB

Memory Bank

Shift registerLook-Up table

m[q,n]

Spike GenerationQA

DB

Dual port RAM

v[q,n]

y[q,n]

f[q,n]

x3

u[q,n]

(d) Global architecture.

Fig. 4: Detailed diagram of the visual encoder computational architecture.

D. Operating considerations

Experiments using transcranial magnetic stimulation [43]have shown that the patterns to evoke phosphenes differ fromperson to person. Exciting the visual cortex by means of amicroelectrode array should also require an adaptation of boththe stimulation patterns and the spike amplitude and length.Thus some level of programming is required at the encoderlevel. An interface with a personal computer allows changingthe value of each register storing the parameters of theNeuromorphic Encoder model. It is also possible to configurethe amplitude and duration of spikes at each microelectrode bysending a configuration command to the Electrode Stimulatorlocated on the secondary unit (the detailed communicationprotocol in presented in subsection IV-D).

Another operating consideration for the Neuromorphic En-coder comprises the mapping of the spike trains to themicroelectrodes, which can only be made by reviewing mapsafter implanting the visual neuroprosthesis, during the trainingphase. In this sense, full re-configuration is achieved by chang-ing the parameters of a look-up table. Again, this configurationis made through the interface with the Personal Computershown in Figure 2.

IV. WIRELESS COMMUNICATION LINK

The RF link block diagram is shown in Figure 5. The RFlink circuitry can be divided into three parts: the primary RFunit (located outside the body), the secondary RF unit (locatedinside the body, but not necessarily inside the head) andthe transformer which establishes inductive coupling betweenthe two units (one coil in the primary unit and another inthe secondary unit). This wireless communication channelimplements the bidirectional RF link: (i) the forward link(the main link: data bit rate is up to 1 Mbps), in which apower/data signal is transmitted using FSK modulation witha 10 MHz frequency carrier and (ii) the backward link (thesecondary link: data bit rate is up to 156.25 kbps, which

suffices for maintenance and initial configuration purposes)in which data is transmitted in the reverse direction usingDBPSK modulation with a 5 MHz frequency carrier. Toimplement a very compact low-power wireless communicationlink, some integrated circuits were initially designed usingAMS CMOS 0.8 µm technology but the new circuits, that arein development phase, use AMS CMOS 0.35 µm technology.

A. Primary RF Unit

The primary RF unit is shown in Figure 5. The transmitter(forward link) is represented in Figure 6 and includes a FSKmodulator and a signal amplifier. FSK was chosen for itsrobustness to amplitude distortions [44]. The FSK modulatoris implemented by means of a counter, driven by an oscillatorfCLK . According to the transmitted data bits, 0 or 1, thecounter divides the clock signal by 16 or 15, respectively. Inthis way, data is modulated at fCLK16 and at fCLK15 frequencies.This signal feeds a class E switching-mode tuned poweramplifier. The class E switching-mode tuned power amplifierconfiguration was chosen to optimise the efficiency at thetransmitter [45]–[47].

The primary RF unit receiver (backward link) is a Costas-Loop [28] coherent demodulator placed after a 6th orderbandpass filter and an Automatic Gain Control (AGC) unit.

B. The Transformer

The transformer is of major importance in the RF linksince it has a strong influence on the overall performance ofthe secondary system. It has to be designed to allow propersystem operation regardless of the intercoil distance (withinreasonable limits, say 1 to 2 cm). The coupling transformeris represented in Figure 7. It should be noticed that at thedesired operation frequencies the real transformer exhibits adistributed parameter behaviour (the distributed elements arerepresented in Figure 7). Due to the coil separation, there


FORWARD RECEIVER

FSKMODULATOR

BANDPASSFILTER

DBPSKDEMODULATOR

&BIT TIMING RECOVERY

data fromNPC

TX clock

RX clock

RX data

POWER SUPPLY GENERATOR

MASTERCLOCK

RECOVERY

FSKDEMODULATOR

DATA PROCESSING

& CONTROL

DBPSKMODULATOR

BIT

S

YN

CH

RO

NIZ

ER

POWER AMPLIFIER

ELECTRODES STIMULATOR

&SENSING

POWER AMPLIFIER

AGC

Electrode array

FORWARD TRANSMITTER

BACKWARD RECEIVER

clock

raw data

BACKWARD TRANSMITTER

RF link

Fig. 5: RF link circuitry diagram.

Modulated Data

FSKModulatorClock

Data In

RFTransformer

C1

L1

9 V

Fig. 6: Primary RF unit transmitter.

is an high magnetic flux dispersion (not connected with thesecondary coil), the coupling is weak and, as a consequence,a significant amount of energy is lost. To maximise theefficiency, an appropriate design of the coils is important. Inorder to compensate the equivalent inductances relative to theprimary and secondary magnetic fluxes dispersions, capacitors(C1, C2) are placed in series with the corresponding coil,resonating at the 10 MHz carrier frequency. This originatesa double tuned bandpass filter behaviour for the RF trans-former. Both circular coils have inside a cylindrical powerfulneodymium magnet used for coil self-attracting and fixingpurposes.

Experiments concerning the transformer confirmed the the-oretical results with the single exception of the bandwidthcentre frequency. This is explained by the fact that the trans-former equivalent model used in the literature assumes a highmagnetic coupling factor (parameter k ≈ 1) and it does nottake into account neither the distributed capacitance nor theskin effect. This assumption does not hold in this applicationbecause the absence of an iron core makes it impossible tohave a strong magnetic coupling: the measured coupling factorwas 0.3 for an intercoil distance of 1 cm using carefullydesigned planar coils of Litz wire with about 3 cm of diameter.

C. Secondary RF Unit

The secondary RF unit architecture is depicted in Figure 5.The power supply generator is comprised of a half-wave

C

RL

ri

C1

L2

Class EAmplifier

2

L1

PrimaryReceiver

ImplantOutside

TRANSFORMER

l ici

rj

rk

rlcj cl

ck

l j

lk

ll

Fig. 7: The coupling transformer with distributed parametermodel.

rectifier, protection circuits and a series regulator; it is ableto recover the required power (stabilising at 5 V or 3.3 VDC) from the received signal with 28% power efficiency atan intercoil distance of 1 cm. The binary FSK Demodulatoris built around a Phase Locked Loop (PLL) circuit and acomparator, and provides a stream of Non-Return-to-Zero(NRZ) data at a bit rate of 1 Mbps. This bit stream is fed tothe Bit Synchroniser, which provides a synchronised clock andretimed data to the data processing unit. The latest performs bitand frame synchronisation and frame disassembly. Formatteddata is then forwarded to the Electrode Stimulator and SensingBlock. The master clock recovery task is accomplished in theMaster Clock Recovery block (see Figure 5) by means of anarrow-band PLL designed to produce a reference 10 MHzclock from the received signal.

The signal received from the primary system is usedto extract the system master clock, with frequencyfCLK = 1

TCLK= N × Rb where Rb = 1

Tb= 10 Mbit/s is

the raw bit-rate and N = 10 (corresponding to a RF carrierfrequency of 1MHz). The following point is worth noting:since the master clock is derived from the transmitted signal,it follows that the data stream is frequency synchronised(i.e., frequency-locked) with the master clock; a data clockcould therefore be obtained by suitable division (by a factorN ) of fCLK . This is because there is no frequency offset


!#" $&% $"" &

' &&$""( $

) % *

+&, - ./ 021354

678:9 ; <>=

?@BAC

%

$% $"D"E' &$"" &D$

FG HIJKMLNOPJG QR

S T T S

PUKMEOKMV

NMKOPWI

)MUVNXOPNYNOP

KMEOKV ZNKG OPJ

Fig. 8: Data clock recovery and bit synchroniser.

between transmitter and receiver in this system. However,the (lead or lag) phase difference between the positive-goingclock transitions and the optimum time epoch for samplingthe data, which is the central point in the time frame ofeach bit, is unknown and varies significantly. In fact, evensmall disturbances in the relative position of both coils lead toimportant phase offsets which have to be properly estimatedand compensated by the bit synchroniser. The task performedby the bit synchroniser is thus of fundamental importance toestablish a proper time reference in the receiver. The positive-going transitions in this reference clock should accuratelysignal the optimum instants to sample and detect the receiveddata bits. The part of the receiver interacting with the bitsynchroniser is shown in Figure 8.

As can be seen in Figure 8, the new bit synchroniser hasa feedforward structure; this important feature avoids the an-noying loop behaviour known as hang-up, typical in feedbacksynchroniser operation, which manifests itself has an unac-ceptably long synchroniser acquisition period, compromisingreceiver operation. The operation of the Bit Synchroniser isexplained next. Suppose that we have a binary counter beingdriven with the master clock frequency fCLK ; then, it willadvance N states within each bit period. If, at time t0, thecounter is in state i then, at time t0 + Tb

2 , it will have advancedTb

2×TCLK = fCLK2·Rb = N

2 and be in state i+N2 (on average); this

is the time epoch at which the recovered clock should havea positive transition, marking the middle of the data bit. Thisreasoning justifies the block diagram represented in Figure 9:the positive-going transition on the raw, unsynchronised datasignal latches the counter state i, at reference time t0, andmarks the start of a bit. When the counter reaches the stateS = i+ N

2 , then (S + N2 ) mod N = (i+N) mod N = i and

the comparator will signal this event to the final processingblock, which in turn samples the raw data and produces aclock pulse synchronised with the master clock.

Note that after a positive-going transition of the raw data,the synchroniser operates in a free-running fashion. The phaseoffset which eventually accumulates after this event is cor-rected when the next positive-going transition occurs. Thusthe raw data should not have long sequences of equal bits.This is guaranteed by the use of self-synchronising scrambleran descrambler circuits.

In summary, this new bit synchroniser has the following

[\ ][^

_

` a b c d

eMfg h iPfkjjUlUg l

mMlg l

no pMqkr s

t(lUu g fv[k\ ][^

eMlUwxjlUg l

m_

y

z

y| zy

z~g v l [ g fjjUlg l [\ ]U[k^

&

MMe

y MNt( y e y e

y mMmMe y t(eH MMe y e

Fig. 9: Block diagram of the new feedforward bit synchroniser.

Header 0 0Type V Data1 0 ... 0 V Data8 0

12 bits 3 bits 10 bits 10 bits

(a) Structure of the transmitted packets.

Load Header

SendingLoad Type

SendingLoad Data

Sending

(b) Data packager for the transmitters.

Detect Header

ReceivingProcess

TypeReceiving

Process Data

(c) Data unpackager for the receivers.

Fig. 10: Communication protocol.

desirable properties: i) due to the absence of frequency offsetbetween transmitter and receiver the recovered clock is jitter-free (no dynamic phase error), ii) it has a feedforward structurethat avoids the hang-up disabling phenomenon and finally, iii)it is easily implemented with very simple digital logic.

A reverse link used for monitoring tasks, electrode cal-ibration and voltage measurement is established from thesecondary to the primary system at a lower data rate of156.25 kbps, using DBPSK modulation. The DBPSK modula-tion format was chosen because the transmitter is very simpleand requires very low power. Operation of this reverse linkis in half-duplex mode i.e., while data is being sent on thislink to the primary system, no useful data is transmitted inthe forward link. However, to keep the secondary system con-veniently powered, the primary system continually transmitsan unmodulated carrier (at 10 MHz). In the primary system,the DBPSK data is recovered by means of a Costas-Loopdemodulator [28] which performs satisfactorily with intercoildistances up to 2 cm.

D. Communication Protocol

Both forward and backward communication links use thesame data protocol, developed specifically for this system. Thedata packet structure is represented in Figure 10(a). To transmita set of up to 8 data values, it uses a synchronisation headercomposed by a sequence of twelve 1’s and a 3-bit type field


indicating the current operation mode. Three operation modesare specified for the forward transmission. In the normal modethe RF link is being used to transmit spike information to theElectrodes Stimulator and Sensing (see Figure 2), and eachdata field contains the address of the electrode to be stimulated.The other operation modes are used to read/write the internalregisters, for controlling the amplitude and duration of theelectrical pulses. The write operation mode is accomplishedby sending two pairs of data: the address of the register beingconfigured plus the new value. The read operation is intendedto measure the impedance of the microelectrodes (throughthe developed voltage and known drive current), in order toevaluate the microelectrode-cortex interface. It works in a twostage transmission. In the first stage, a forward transmissionsends a request to the microelectrode controller for the voltagemeasurement of up to 8 microelectrodes (again the data fieldsare used as the address of the microelectrode). In the secondstage, a backward transmission sends to the control modulein the primary unit (see Figure 2) the voltage information ofthe requested microelectrodes in the data fields. In order toincrease the efficiency of the operation, the address of themicroelectrodes is not repeated in the backward transmission.Instead, the measured voltage of each of the requested elec-trodes is sent in the exact same data field as the addresswas sent. Optionally, the system allows the reading of theconfiguration of the internal registers with the read operationmode. The verification of which data fields are valid is madeby a valid bit (marked with V in Figure 10(a)), precedingeach data field. Also, in order to avoid the repetition ofthe synchronisation header within the transmitted packages,trailing zeros are sent at the end of each packet field: header,type and data.

Four blocks were developed for implementing the describedprotocol: a transmitter and a receiver for both primary andsecondary units. The transmitters, or data packagers, are builtaround a 13-bit shift register with parallel load and serialoutput. The output is sent to the RF link through a scramblercontrolled by the state machine presented in Figure 10(b).The state machine works as follows. In the Load states, theshift-register is loaded with the Header/Type/Data valuesfollowed by a trailing zero. Afterwards, during the Sendingstates, the loaded values and trailing zeros are transmitted.On the other side, data unpackagers for the receivers weredesigned. These are built around 12 bit shift registers withserial input (the bits received from the RF link, after passingby the unscrambler) and parallel output. The control of thiscircuit is made by a state machine as shown in Figure 10(c).The state machine is started on Detect Header where itremains until the 12 bit header is found. Afterwards the packetinformation, type and data, are read in the states ProcessType and Process Data, respectively. These states are precededby Receiving states which last while the next package field isbeing loaded into the shift register. After the whole packageis read, a new cycle begins with the state machine returningto Detect Header.

Fig. 11: Neuromorphic Encoder Prototype.

Register Configuration

Serialization and Data Packing

SDA

SCL

Input Image

Data Packets

HS

VS

R,G,B

Image Capture and Resize

Image Display

Resized Image

Neuromorphic Encoder

Spikes

Processed Image

Fig. 12: Diagram of the blocks implemented in the processingcore.

V. EXPERIMENTAL RESULTS AND SYSTEM PROTOTYPE

In order to perform the experimental evaluation of the Neu-romorphic Encoder, a system prototype was developed [48].The processing core is based on a XILINX SPARTANXC3S400 FPGA [49] with 400 k system gates, a total memoryof 288 kbits distributed in 16 RAM blocks, 16 bit multipliersand fast carry look-ahead logic. The prototype was devel-oped on a 4-layer Printed Circuit Board (PCB), where the2 inner layers are dedicated to the power circuit while the2 outer layers are used for signal routing (see Figure 11).This structure decreases the coupling noise between signals,therefore improving the overall performance of the prototype.The system also includes 3 I/O ports: i) an I/O interface toconnect to a digital camera for capturing the images to beprocessed by the FPGA; ii) a 40-pin digital interface to the RFlink; and iii) a standard VGA output used for visualizing bothinput and processed images. For image capture a miniaturedigital camera sensor [50] is used. This sensor is capableof providing a color image with a resolution up to XVGA(1280× 1024) at a frame rate of 15 fps consuming 50 mW ofpower. The operation of the camera and the resolution of theacquired images can be configured through the sensor internalregisters, which are accessed by using the Serial CameraControl Bus (SCCB) [51] protocol.

Besides the Neuromorphic Encoder, the prototype process-ing core includes 4 other modules (Figure 12). The RegisterConfiguration module is used to program camera operation in


TABLE I: Resource occupancy on the processing core.Slice R A M

M o d u les O ccu p a t io n b lo ck s M u lt .

To ta l a v a ila b le re s o u rc e s 3584 1 00% 1 6 1 6

I m a g e C a p tu r e a n d R e s iz e 1 08 3% 0 2

R e g is te r C o n fi g u r a tio n 1 42 4% 0 0

C la s s ic M o d e l 420 1 2% 5 3

S e r ia liz a tio n a n d D a ta Pa c k in g 6 0 2% 2 0

Im a g e D is p la y 21 8 6 % 8 9

C o m p le te S y s te m 948 26 % 1 5 1 2

order to acquire frames in YUV QVGA format at 30 fps. TheImage Capture and Resize module deals with the synchroniza-tion with the image sensor and with resizing the input image toa size of 32×32 pixels. This module takes the 128×128 pixelscentre part of the input image and, after applying a lowpassfilter based on a Gaussian kernel in the horizontal dimension,decimates the image to a size of 32 × 32 pixels. The ImageDisplay module implements a standard VGA graphic interfaceto display any of the input or processed images on a standardmonitor. It allows for the verification of the correct operationof the Neuromorphic Encoder. Finally, the Serialization andData Packing module corresponds to the forward transmitterdata packager, as described in Section IV-D.

The circuits were described in VHDL and, by using theXILINX WEBPACK 6.2 tool, synthesized and mapped to thechosen XILINX SPARTAN XC3S400 FPGA. The resourceoccupancy of the FPGA for the implemented modules isshown in Table I. As it can be seen, the complete systemwas able to be mapped in the chosen FPGA using only 26%of the total resources, operating at a maximum frequency of85 MHz. This largely exceeds the required ≈ 1.5 MHz for theimplementation of the Early Layers block (processing of animage of size 32×32 at a rate of 30 fps executing 49 operationsper cycle due to hardware folding) and ≈ 1 MHz for theNeuromorphic Pulse Coding block (generation of spikes at amaximum rate of 100 Hz to an equivalent microelectrode arrayof size 32× 32).

To garantee the correct operation of all the modules, exhaus-tive tests were performed. In the case of the NeuromorphicPulse Coding block and the Serialization and Data Packingmodule, the evaluation of these circuits was made by imple-menting i) a Data Unpacking block to decode the transmitteddata packets and ii) an Image Recovery block responsible forthe translation of a sequence of pulses into a image capableof being displayed on a standard monitor. Both these blockswere implemented on a second FPGA, a XILINX SPARTANXC2S300 FPGA [52] connected to the prototype by a 40pin header. The Image Recovery Block is based on a 4-order lowpass filter where the output increases when a newspike is generated and decreases otherwise. Figure 13 showsa photograph of the monitor on an experimental test of theNeuromorphic Pulse Coding block and Serialization and DataPacking module. In this test, a colour image, captured directlyfrom the camera, was presented as input to the NeuromorphicPulse Coding block (top left corner of Figure 13). The valueof each pixel, treated as a firing rate, is converted into asequence of pulses, packed into structures as those of Figure

input image

spatial filtering only

retina model output

recovered from spike information

Fig. 13: Capture of the image displayed on a standard monitor.

12 and sent to a second FPGA by a serial communicationlink (which simulates the real wireless link). In the secondFPGA, a SPARTAN XC2S300, the packets are decoded, andthe spikes sent to the Image Recovery block which attempts torecover the initial image (bottom right corner of Figure 13).The person in the images was moving at the time of the phototo allow displaying an image on the retina model output.

The prototype shown in Figure 11 is operated at a frequencyof 50 MHz and consumes ≈ 500 mW at 5 V, where 50 mWare due to the image sensor. The DAC’s only operate whenimages are being displayed on a standard monitor. In this casethe prototype consumes an extra power of 250 mW.

In order to study the involved phenomena, in particularthe important magnetic coupling issues, an RF link boardlevel prototype was implemented [53]. This prototype is a 10times frequency scaled version of the system operating with aforward data rate 100 kbps and carrier frequency 1 MHz andwith backward data rate 15.625 kbps with carrier frequency500 kHz. The RF link results presented in this section wereobtained using this prototype.

The primary RF unit maximum power consumption is180 mW at a power supply of 9 V. Figure 14 illustrates theamplitude response of the transformer for different intercoildistances. The transformer pass band is centred at 1 MHz, asdesired. As one can see, a significant attenuation is introducedby the transformer which results from its very weak magneticcoupling [44]. However, the signal delivered to the receiverstill allows satisfactory power and data recovery.

-120

Gai

n (d

B)

0

-20

-40

-60

-80

-100

Frequency (MHz)0.2 0.4 1.0 2.0 4.0 10.00.10.02 20.0

3 cm

0 cm

1 cm

2 cm

Fig. 14: Amplitude characteristics for distances of 0, 1, 2 and3 cm between primary and secondary coil.


Distance (cm)

1.0 1.5 2.0 2.50.50.0 3.00

5

10

15

20

30

25

Eff

icie

ncy

(%)

R = 220 Ω

R = 330 Ω

R = 390 Ω

R = 1000 Ω

Fig. 15: Coupling efficiency as a function of the intercoildistance and load resistor.

Distance (cm)

1.0 1.5 2.0 2.50.50.0 3.0

R = 220 Ω

R = 330 Ω

R = 390 Ω

R = 1000 Ω

0

20

30

Loa

d po

wer

(mW

) 40

50

60

10

Fig. 16: Load power as a function of intercoil distance andload resistor.

Some experiments were conducted to determine the powerefficiency that is magnetically induced from the primary tothe secondary system. It was determined that it is possibleto obtain an efficiency of about 28% (about 50 milliwattshave been measured at the secondary receiver output and180 milliwatts power consumption in the primary RF unit forintercoil distances up to 1 cm) as can be seen in Figure 15.

Figure 16 depicts the load power as a function of theintercoil distance. As can be seen, there is a degradationin power transfer between primary and secondary RF units.However, t is not significant for the equivalent load resistorvalues expected for a secondary RF unit with maximumpower consumption of 50 mW. This degradation augmentswith increased distance or when the load resistances decrease.The maximum output capability occurs for intercoil distancesbelow 1 cm (this is the target distance for which the RFtransformer design was optimised). The overall transmittedand received data are illustrated in Figure 17 (bit rate is100 kbps and the intercoil distance is 1 cm). As one can seefrom this figure, scrambled data is properly recovered at thereceiver after crossing all the RF link: primary modulator, RFtransformer and FSK secondary demodulator. In this experi-ment, 50 mW of a 3.3 V DC power supply are being extractedby the secondary power recovering circuitry, simultaneouslywith the secondary data demodulation. The oscilloscope traces

Fig. 17: RF link FSK demodulator operation: transmitted NRZdata (top) and recovered data (bottom).

correspond to the data from the NPC and the recovered data(raw data), respectively (see Figure 5).

VI. CONCLUSIONS

This paper proposes the architecture for a visual neuro-prosthesis and demonstrates its feasibility using nowadaystechnology. The primary goal of this system is to restore alimited but useful visual sense to profoundly blind people.This neuroprosthesis is based on a Neuromorphic Encoderand a non-invasive system for intracortical stimulation of thevisual cortex. The developed Neuromorphic Encoder receivesthe visual stimulus from a miniature digital video camera,performs the spatial and temporal processing correspondingto the ”early layers” of the retina and implements a simplifiedversion of an integrate-and-fire spiking neuron to model thespike generation.

To achieve a non-invasive system for intracortical stimula-tion a bidirectional RF link was designed. The system usesFSK modulation in the forward link and DBPSK modulationin the backward link. These modulation formats were cho-sen since, due to their constant-amplitude characteristic, theycan offer significant robustness against the severe amplitudevariations which are expected in this system. Because of itshigh efficiency, a class E tuned power amplifier was chosento transmit at 1 Mbps over a 10 MHz carrier.

The Neuromorphic Encoder, which generates the spikesdelivered to the RF link, was implemented in a video pro-cessing board based on a XILINX Spartan XC3S400 FPGA.The encoder occupies about 26% of the FPGA resources,exhibiting a power consumption of about 500mW for a clockfrequency of 50MHz. A frequency scaled prototype was alsobuilt in order to test the performance of the data and powercommunication system. The transmitted signal also carries thepower for the secondary unit (≈ 50 milliwatts) with an averageefficiency of 28% for 1 cm intercoil distance. The system


operates very well with power, data and clock recovery fordistances up to 2 cm.

The Neuromorphic Encoder, which generates the spikesdelivered to the RF link, was also implemented in a video pro-cessing board based on a XILINX Spartan XC3S400 FPGA.The encoder occupying about 26% of the FPGA resourcesand operating at 50 MHz, has a power consumption of about500 mW.

Based on the developed prototype, a complete system isnow being designed in ASIC technology. The NeuromorphicEncoder is being synthesised using the UMC 0.13 µm CMOStechnology process [40]. For the Wireless CommunicationLink and the Electrode Stimulators and Sensing modules itis required a 0.35 µm AMS CMOS technology to allowfor the high voltage levels being used. The work presentedhere contains several significative innovations, mainly: i) itis the first known complete architecture designed and im-plemented of the intracortical visual neuroprosthesis; ii) anew full directional RF link carrying data and power wasproposed with significative contributions, namely: a new dataclock feedforward bit synchroniser and a new communicationprotocol appropriated for cortical prosthesis; iii) a new videoneuromorphic encoder system was developed. It is expectedthat this innovations contribute to help the development offuture practical and compact visual prosthesis.

ACKNOWLEDGMENTS

Acknowledgements are due to all people participating inthe CORTIVIS research project. The scientific discussions andresearch results obtained contributed significantly to improvethis work. A very special acknowledgement is due to Prof.Jorge Fernandes and to Prof. Marcelino Santos for their con-tribution for CORTIVIS project in particular for supervisingthe students that are developing the chips for the integratedvisual prosthesis.

REFERENCES

[1] M. R. Shah, R. P. Phillips, and R. A. Normann, “A transcutaneous powerand data link for neuroprosthetic applications,” Paper presented at theannual meeting of the IEEE-EMBS, pp. 1357–1358, 1993.

[2] T. Cameron, G. Loeb, R. Peck, J. Schulman, P. Strojnik, and P. Troyk,“Micromodular implants to provide electrical stimulations of paralyzedmuscles and limbs,” IEEE Transactions on Biomedic Engineering,vol. 44, no. 9, pp. 781–790, 1997.

[3] B. Smith, Z. Tang, M. Johnson, S. Pourmehdi, M. Gazdik, J. Buckett,and P. Peckham, “An externally powered, multichannel, implantablestimulator-telemeter for control of paralyzed muscle,” IEEE Transactionson Biomedic Engineering, vol. 45, no. 4, pp. 463–475, 1998.

[4] Q. Bai and K.D.Wise, “Single-unit neural recording with active micro-electrode arrays,” IEEE Transactions on Biomedic Engineering, vol. 48,pp. 911–920, August 2001.

[5] R. A. Andersen and C. A. Buneo, “Intention maps in posterior parietalcortex,” Annual Revision of Neuroscience, vol. 25, pp. 189–220, 2002.

[6] W. H. Dobelle, “Artificial vision for the blind by connecting a televisioncamera to the brain,” American Society for Artificial Internal Organs(ASAIO) Journal, no. 46, pp. 3–9, 2000.

[7] W. Liu, E. McGucken, K. Vitchiechom, and M. Clements, “Dual unitvisual intraocular prosthesis,” in Proceedings on the 19th InternationalConference on IEEE-EMBS, 1997, pp. 2303–2306.

[8] T. Stieglitz, H. Beutel, R. Keller, C. Blau, and J.-U. Meyer, “Devel-opment of flexible stimulation devices for a retina implant system,” inProceedings on the 19th International Conference on IEEE-EMBS, 1997,pp. 2307–2310.

[9] W. Dobelle, J. L. Antunes, D. Coiteiro, and J. Girvin,“The first artificial vision system in commercial distribution,”http://www.dobelle.com/news.html.

[10] G. A. Kendir, W. Liu, R. Bashirullah, G. Wang, M. Humayun, andJ. Weiland, “An efficient inductive power link design for retinal pros-thesis,” in IEEE ISCAS 2004, 2004, pp. 41–44.

[11] M. Mojarradi, D. Binkley, B. Blalock, R. Andersen, N. Ulshoefer,T. Johnson, and L. D. Castillo, “A miniaturized neuroprosthesis suitablefor implantation into the brain,” IEEE Transactions on Neural Systemsand Rehabilitation Engineering, vol. 11, no. 1, pp. 38–42, March 2003.

[12] “Cochlear,” http://www.cochlear.com/.[13] E. Zrenner, A. Stett, S. Weiss, R. Aramant, E. Guenther, K. Kohler,

K. Miliczek, M. Seiler, and H. Haemmerle, “Can subretinal micropho-todiodes successfully replace degenerated photoreceptors?” Vision Re-search, vol. 39, no. 15, pp. 2555–2567, 1999.

[14] R. Eckmiller, “Learning retina implants with epiretinal contacts,” Oph-thalmic Research, vol. 29, no. 5, pp. 281–289, 1997.

[15] J. Eggermont, “Is there a neural code?” Neuroscience & BiobehavioralReviews, vol. 22, no. 2, pp. 355–370, 1998.

[16] G. Brindley, “Effects of electrical stimulation of the visual cortex,”Human Neurobiology, vol. 1, no. 4, pp. 281–283, 1982.

[17] W. Dobelle and M. Mladejovsky, “Phosphenes produced by electricalstimulation of human occipital cortex, and their application to thedevelopment of a prosthesis for the blind,” The Journal of Physiology,vol. 243, no. 2, pp. 553–576, 1974.

[18] “Cortical Visual Neuroprosthesis for the Blind (CORTIVIS),”http://cortivis.umh.es.

[19] P. Ahnelt, J. Ammermller, F. Pelayo, M. Bongard, D. Palomar, E. Anger,M. Piedade, J. Ferrandez, L. Borg-Graham, and E. Fernandez, “Neuro-scientific Basis for the Design and Development of a Bioinspired VisualProcessing Front-End,” in Proceedings of the 2nd European Medical andBiological Engineering Conference, December 2002, pp. 1692–1693.

[20] Harvard University, “The Meister Lab,” http://rhino.harvard.edu.[21] Utah University, “Vision Team,”

http://biomed.brown.edu/Courses/BI108/BI108 1999 Groups/Vision Team.[22] J. Keat, P. Reinagel, R. C. Reid, and M. Meister, “Predicting Every

Spike: A Model for the Responses of Visual Neurons,” Neuron, vol. 30,pp. 803–817, June 2001.

[23] M. Merry and M. Meister, “Refractoriness and neural precision,” TheJournal of Neuroscience, vol. 18, no. 6, pp. 2200–2211, March 1998.

[24] M. J. B. II, I. H. Brivanlou, T. A. Jordan, and M. Meister, “Anticipationof moving stimuli by the retina,” Nature, vol. 398, pp. 334–338, March1999.

[25] S. D. Wilke, A. Thiel, C. W. Eurich, M. Greschner, M. Bongard,J. Ammermuller, and H. Schwegler, “Population coding of motionpatterns in the early visual system,” Journal of Comparative PhysiologyA, vol. 187, no. 7, pp. 549–558, March 2001.

[26] Chichilnisky, “A simple white noise analysis of neuronal light re-sponses,” Network, May 2001.

[27] J. Martins and L. Sousa, “Performance comparison of computationalretina models,” in 5th IASTED International Conference on Visualiza-tion, Imaging and Image Processing, Benidorm, Spain, September 2005.

[28] J. G. Proakis, Digital Communications. McGraw-Hill, 1983.[29] J. A. V. Arx and K. Najafi, “A Wireless Single-Chip Telemetry-Powered

Neural Stimulation System,” in Proceedings of the 1999 InternationalSolid-State Circuits Conference, 1999, pp. 214–215.

[30] W. Liu, K. Vichienchom, M. Clements, S. DeMarco, C. Hughes,E. McGucken, M. Humayun, E. D. Juan, J. Weiland, and R. Greenberg,“A neuro-stimulus chip with telemetry unit for retinal prosthetic device,”IEEE Journal Solid-State Circuits, vol. 35, pp. 1487–1497, October2000.

[31] A. Abrial, J. Bouvier, M. Renaudin, P. Senn, and P. Vivet, “A newcontactless smart card IC using an on-chip antenna and an asynchronousmicrocontroller,” IEEE Journal Solid-State Circuits, vol. 36, pp. 1101–1107, July 2001.

[32] M. Schwarz, L. Ewe, N. Hijazi, B. J. Hosticka, J. Huppertz, S. Kolns-berg, W. Mokwa, and H. K. Trieu, “Micro Implantable Visual Prosthe-ses,” in Proceedings of the 1st Annual International IEEE-EMBS SpecialTopic Conference on Microtechnologies in Medicine and Biology, Oc-tober 2000, pp. 461–465.

[33] Q. Huang and M. Oberle, “A 0.5-mW Passive Telemetry IC forBiomedical Applications,” IEEE Journal of Solid-State Circuits, vol. 33,no. 7, pp. 937–946, July 1998.

[34] D. Mar, M. Troosters, I. Martinez, E. Valderrama, and J. Aguillo,“New Developments for High Performance Implantable Stimulators :first 3 Mbps up to 4.46 Mbps demodulator chip through a wirelesstranscutaneous link - - Esprit Project ’Microsystems for Visual Prosthesis


MIVIP LTR-22527,” in Proceedings of the MicroNeuro 1999, 1999, pp.120–126.

[35] M. Ghovanloo and K. Najafi, “A high data-rate frequency shift keyingdemodulator chip for the wireless biomedical implants,” in Proceedingsof the IEEE International Symposium on Circuits and Systems, vol. 5,May 2003, pp. 45–48.

[36] P. Troyk and G. DeMichele, “Inductively-coupled power and data linkfor neural prostheses using a class-E oscillator and FSK modulation,” inProceedings of the 25th Annual International Conference of the IEEEEngineering in Medicine and Biology Society, vol. 4, September 2003,pp. 33/6–33/9.

[37] K. N. M. Ghovanloo, “A wideband frequency-shift keying wireless linkfor inductively powered biomedical implants,” in IEEE Transactions onCircuits and Systems - I: Regular Papers, vol. 52, no. 12, December2004, pp. 2374–2383.

[38] M. Sawan, Y. Hu, and J. Coulombe, “Wireless smart implants dedicatedto multichannel moritoring and microstimulation,” IEEE Circuits andSystems Magazine, vol. 5, no. 1, pp. 21–39, first quarter 2005.

[39] W. Gerstner and W. Kistler, Spiking Neuron Models. CambridgeUniversity Press, 2002.

[40] P. Malisse, UMC L130E FSG-HS 1P8M Logic Process with VSTLibraries, Europractice IC Service, December 2003.

[41] L. J. and W. J., “A multi-sender asynchronous extension to the addressevent protocol,” in Proc. of 16th Conference on Advanced Research inVLSI, 1995, pp. 158–169.

[42] K. K. Parhi, VLSI Digital Signal Processing Systems. John Wiley &Sons, Inc., 1999.

[43] E. Fernandez, A. Alfaro, J. M. Tormos, R. Climent, M. Martinez,H. Vilanova, V. Walsh, and A. Pascual-Leone, “Mapping of the humanvisual cortex using image-guided transcranial magnetic stimulation,”Brain Research Protocols, vol. 10, no. 2, pp. 115–124, October 2002.

[44] G. Tavares, J. Gerald, M. Piedade, and R. Ribeiro, “Evaluation ofDifferent Wireless Communication Systems For Intracortical Prosthesis,”Cortivis Deliverable D4, INESC-ID, Tech. Rep., 2003.

[45] M. Kazimierczuk, “Collector amplitude of the class E tuned poweramplifier,” IEEE Transactions on Circuits and Systems, vol. 31, no. 6,pp. 543–549, June 1984.

[46] N. O. Sokal and A. D. Sokal, “Class E - A new class of high-efficiencytuned single-ended switching power amplifiers,” IEEE Journal Solid-State Circuits, vol. 10, pp. 168–176, June 1975.

[47] F. Raab, “Idealized operation of the class E tuned power amplifier,” IEEETransactions on Circuits and Systems, vol. 24, pp. 725–735, December1977.

[48] J. Germano, R. Baptista, and L. Sousa, “Configurable platform for realtime video processing and vision systems,” in XX Conference on Designof Circuits and Integrated Systems, Lisbon, Portugal, November 2005.

[49] Xilinx, “Spartan-3 FPGA Family: Complete Data Sheet,”http://direct.xilinx.com/bvdocs/publications/ds099.pdf.

[50] O. T. Inc., “OV9650 Color CMOS SXGA CameraChip with OmniPixelTechnology,” http://www.ovt.com/pdfs/pb 9650.pdf.

[51] ——, “The Serial Camera Control Bus Functional Specifications,”http://www.ovt.com/pdfs/ds note.pdf.

[52] X. Inc., “Spartan-2 Complete Data Sheet,”http://direct.xilinx.com/bvdocs/publications/ds077.pdf.

[53] J. Gerald, G. Tavares, M. Piedade, E. Varela, and R. Ribeiro, “RF-link system for cortical neuroprosthesis,” CORTIVIS Project, INESC-ID,Tech. Rep., 2004.

Moises Piedade received the Ph.D. degree in Elec-trical and Computer Engineering from Instituto Su-perior Tecnico (IST), Technical University of Lis-bon, Lisbon, Portugal in 1983. He is Professorwith the Department of Electrical and ComputerEngineering and the leader of Signal ProcessingSystems research group at Instituto de Engenharia deSistemas e Computadores–R&D (INESC-ID), Lis-bon. His main research interests include electronicsystems, signal acquisition and processing systemsand circuits and systems for biomedical applications.

Jose Gerald was born in Lisbon in 1956. Hereceived the B.S. degree in Electrical Engineeringfrom the Instituto Superior Tecnico (IST), TechnicalUniversity of Lisbon, in 1980. In 1992 he receivedthe Ph.D. degree from IST, with a thesis on adaptiverecursive structures for data communication systems.Since 1980, he has been with IST in the Elec-trical Engineering Department, teaching both elec-tronics and telecommunication engineering. Since1988 he has been a research engineer at Institutode Engenharia de Sistemas e Computadores–R&D

(INESC-ID). His research interests include the areas of Adaptive Filter-ing, Spread-Spectrum Communications and Power Line Communications.Recently he was also involved in a wireless visual system for corticalneuroprodthesis.

Leonel Augusto Sousa (M’ 01-SM’ 03) received thediploma degree in Electronic and TelcommunicationEngineering from Universidade de Aveiro (Portugal)in 1984. He received the MSc and PhD degreesin Electrical and Computer Engineering from theInstituto Superior Tecnico (IST), Technical Univer-sity of Lisbon (Portugal), in 1989 and 1996, respec-tively. He is currently a member of the Electricaland Computer Engineering Department at IST, anda Senior Researcher at Instituto de Engenharia deSistemas e Computadores–R&D (INESC-ID). His

research interests include computer architectures, parallel and distributed com-puting, and VLSI architectures for multimedia and biomedical systems. Hehas contributed more than 70 papers to international journals and conferencesand he is a member of HiPEAC European Network of Excellence.

He is a member of the IEEE Computer Society and a member of ACM.

Goncalo Tavares (M’ 02) received the Ph.D. de-gree in Electrical and Computer Engineering fromInstituto Superior Tecnico (IST), Technical Uni-versity of Lisbon, Lisbon, Portugal in 2002. Heis a Professor with the Department of Electricaland Computer Engineering of the IST and a seniorresearcher at Instituto de Engenharia de Sistemase Computadores–R&D (INESC-ID), Lisbon. Hismain research interests include system architecturefor signal processing, statistical signal processing,spread spectrum systems and synchronization issues

in digital communication.

Pedro Tomas received his 5-year degree in electricaland computer engineering from the Instituto Supe-rior Tecnico (IST), Technical University of Lisbon,Portugal, in 2003. Currently, he is pursuing thePh.D. degree in electrical engineering at IST and isdeveloping his research with the Signal ProcessingResearch Group (SIPS) at Instituto de Engenhariade Sistemas e Computadores–R&D (INESC-ID). Hisresearch interests include high-level synthesis of dig-ital signal processing algorithms, image processingand modelling of the human visual system.

Documents

SPECIAL ISSUE ON BIOMEDICAL CIRCUITS AND SYSTEMS 1 1 2 3