Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Advanced Neural Implantsand Control
Daryl R. KipkeAssociate ProfessorDepartment of BioengineeringArizona State UniversityTempe, AZ [email protected]
Approved for Public Release, Distribution Unlimited: 01-S-1097
The Underlying Premise…
The ability to engineer reliable, high-capacity direct interfaces to the brain and then integrate these into a host of new technologies will cause the world of tomorrow to be much different than that of today.
However…
� There are some serious scientific barriers between where we stand today and where we can stand in the future. • How do we establish permanent and reliable interfaces to
selected areas of the central nervous system?
• How do we use these interfaces to directly and reliably communicate at high rates with the brain?
Applied Neural Implants and Control
Project Director Kipke (BME)
AdvisoryCommittee Raupp,Hoppensteadt,Farin
Visualization & Modeling Farin (CSE) Nelson (CSE) Razdan (CSE) Smith (Math)
Systems Science &Signal Processing He (BME) Hoppensteadt (Math &EE) Kipke (BME) Si (EE)
Neural & Tissue Engineering Kipke (BME) Massia (BME) Panitch (BME) Rousche (BME)
Tissue Culture & Analysis Capco (Bio) Massia (BME) Pauken (Bio)
Materials Synthesis& Bioactive Coatings Ehestraimi (BME) Massia (BME) Panitch (BME) Raupp (ChemE)
MEMS Shen (EE) Pivin (EE) Li (EE)
INFO
BIO
MICRO
Primary Goals of theBIO:INFO:MICRO Project
� Develop new neural implant technologies to establish reliable, high-capacity, and long-term information channels between the brain and external world.
� Develop real-time signal processors and system controllers to optimize information transmission between the brain and the external world.
SysSci VizMod
NeuEng MEMS
TisClt
Mat'lSyn
Systems-level Approach…
Feedback control signals
Subject Neural system
(global)
Controlled neural plasticity
local
Neural Implant
Adaptive Controller
External World
Objective 2: Optimize Adaptive Controller
Objective 1: Optimize neural interface
Topics
� Project overview
� Towards the Development of Next-Generation Neural Implants (BIO, MICRO, and INFO)
� Bioactive Coatings to Control the TissueResponses to Implanted Microdevices
� Modeling the Device-Tissue Interface
� Direct Cortical Control of an Actuator
� Neural Control of Auditory Perception
� Wrap-up
Focus on Next-GenerationNeural Implants
Feedback signals: local
Subject Neural system
(global)
Controlled neural plasticity
local
Neural Implant
Neural Controller
External World
host response
Info. Signals: electrical & chemical
Objective 2: Optimize Adaptive Controller
Objective 1: Optimize neural interface to achieve reliable, two-way, high-capacity information channels. …and “self-diagnostic”
Fundamental Problem of ImplantableMicroelectrode Arrays
� Brain often encapsulates the device with scar tissue � Normal brain movement may cause micro-motion at the tissue-
electrode interface � Proteins adsorb onto device surface � Useful neural recordings are eventually lost
Electrode 1
Electrode N
Implant Failure
Month 1Implant Month N
3rd-Generation Neural Implants
Technology Spectrum
1st-generation
Microwires
2nd-generation
Silicon arrays 3rd-generation Neural Implants
Desired Properties • Very high channel count
(<1000)
• Bioactive coatings
• Flexible
• Engineered surfaces
• Controlled biological response
• Integrated electronics
“Brain-centered” Design of Neural Implants
Initial conceptual designs
B
B
A
A
A A B B
through hole connecting channel
recording site
bioactive gel
Standard Perforated Probe
Simple Bioactive Probe
Differential Bioactive Probe
through holerecording site
bioactive gel
flexible polyimide substrate
bond pads
e.g. corticosteroid NGF e.g. GABA
cross-section (A-A) cross-section (B-B)
Polymer-substrate Neural Implants
• 2-D planar devices can be bent into 3-D structures • Increases insertion complexity
Holes to promote integration with neuropil
90 degree angles
Recordings From Polymer-substrateNeural Implants
Chan. 9
Chan. 10
One Day Post-op
Lost most unit activity after 7 days – Most likely due to failure to properly close dural opening.
Flexible Neural Implants PresentSurgical Challenges
� While the “micro-motion” hypothesis suggests that flexible neural implants should be more stable, the same flexibility presents significant new surgical challenges.
“Difficult” insertion “Easy” insertion
Rdr2, 9-00 Rdr3, 9-00
Using Dissolvable Coatings toStiffen the Neural Implant
� Dip-coat microdevice with polyethylene glycol (PEG) • Provides mechanical stiffening prior to implant • Quickly dissolves when in contact with tissue
First insertion of coated microdevice into Second insertion of coated microdevice
gelatin -- Device easily penetrates into gelatin – The device is too flexible to
material penetrate material because the PEG has dissolved.
Micromachined Surgical Devices
Vacuum nozzle
Flexible probe
Insertion aid
Vacuum Actuated Knife/Inserter
PEG
Silicon Knife/Inserter
Exploratory Functionality
Bioactive Component Storage Structures
Passive Surface Engineering
Active FET Devices, ChemFETs
Electrical Recording/Stimulating
Surfaces
Other Active Devices (Thermal, Magnetic, Strain, etc.)
Fluid Microchannels
Polymer Substrate
• Magnetic/thermal stimulation
• Drug delivery channels
• Active micro-manipulation of probes
Currently...
Internal Review
Feasibility Studies Insertion Aids
Mechanical Transfer Structures
Signal Processing Termination
Multiple Dimensions and Forms
Implant Coatings and Surface Modifications
Parylene-N,C Photo-crosslinked PolyimidesCl
Cl O O
O
n
C
C
C
C
smoothsmoothsmooth porousporousporous
N N
OO
Surface Plasma Treatments NH2 NH2 NH2 NH2
(NH3 - Amination)
Advanced Neuro-Device Interfaces
Passive
Chemical/Electronic NHNH22 NHNH22 NHNH22NHNH22
Amplification
ion beammetal
modified regionsite or interdigits
release layer polymer (PI/P-C)
or substrate
Active
Silicon FETs?
Topics
� Project overview � Towards the Development of 3rd-Generation
Neural Implants (BIO, MICRO, and INFO)
� Bioactive Coatings for Controlled Biological Response (BIO, MICRO, and INFO)
� Modeling the Device-Tissue Interface
� Direct Cortical Control of an Actuator
� Neural Control of Auditory Perception
� Wrap-up
Approach
Advanced biomaterials and micro-devices for long-term
implants (BIO, MICRO, INFO)
Cellular and biochemical response characterization
(BIO, MICRO)
Models and 3-D visualization of device-tissue dynamics
(BIO, INFO)
Engineer the neural implant surface in order to control both the material response and the host response.
Factors Limiting ChronicSoft Tissue Implants
� Inability to control cellular interactions at biomaterial-tissue interface
� Initial adsorption of biological proteins • Non-selective cellular adhesion
� Unavoidable “generic” foreign body reactions • Inflammation
• Fibrous capsule formation
Potential Solution
� Engineer surface for minimal protein adsorption and selective cell adhesion • Cell-resistant polymer coatings
• Synthetic: Polyethylene Glycol, Polyvinyl Alcohol
• Natural: Polysaccharides, Phospholipids
• Surface immobilization of biologically active molecules
• Mimic biochemical signals of extracellular matrix
• Cell binding domains for integrin receptors
Biomimetic Surface Modification
NH2 NH2 OH
O
HO
N
O O
OH
OH
O O
OH
OH
O
HO HOHO
O
OH
OH
OO
HO
N OHHO
O
HO
O
NTF NTF
Material Surface
Recombinant NGF Fusion Protein
Factor IIIa
Active or inactive plasmindegradable substrate
Degraded plasminsubstrate
substrate
Human b-NGF
plasmin
Fibrin
Plasmin cleavage Human b-NGF
Fibrin
Bioactive Functionality
Methods 6-hour diffusion in rat cortex
Fluorescence Intensity Prof i le
250
NeuroTrace� DiI tissue-labeling paste, inverted fluorescent microscope with FITC/rhodamine filter cube
200
150
Pix
el
Va
lue
100
5 0
0
0 2 0 4 0 6 0 8 0 100 120 140 160
D i s t a n c e ( m i c r o n s )
Topics
� Project overview � Towards the Development of 3rd-Generation
� Bioactive Coatings to Control the Tissue Neural Implants (BIO, MICRO, and INFO)
Responses to Implanted Microdevices (BIO, MICRO, and INFO)
� Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)
� Direct Cortical Control of a Motor Prosthesis � Neural Control of Auditory Perception � Wrap-up
The Device-Tissue Interface
Neural Interface: Micro-device, Neurons, Glia, Extracellular Space
The Goal is to Characterize, Predict, and Controlthe Device-Tissue Interface
Tissue State
(e.g., encapsulation,excitability)
Biophysical Model of the
Device-Tissue Interface
Device Function
(e.g., impedancespectrum)
• Integrate bioelectrical, histological and biochemical data • Optimize electrode specifications
Visualization of the Chronic Device-TissueInterface With Confocal Microscopy
A B
C D
In vivo Visualization of the ChronicDevice-Tissue Interface
Multi-Domain Continuum Model
( ) ( )
( )
/
/
/
At each "point" in space:
volume fraction
potential ,
conductivity tensor
membrane parameters
, , , etc.
e i
e i
e i
L
r
f
r t
G
C ga
F
r
r
r
r
• Tissue is two (or more) coupled volume-conducting media
• Electrode is boundary condition
r
r
Equations for a Multi-DomainContinuum Model
Volume conductor equations (conservation of current)
- fe��(Ge�Fe ) = +� Imemi + Iapp
i
- fi��(Gi�Fi ) = -Imemi i = index over intracellular domains
Membrane potential(s) and membrane current(s)
� ¶V �Vi = F i - F e Imemi = ai �
Ł Ci ¶t
i + Iioni �ł -1
F = potential (mV) a i = surface to volume ratio (cm ) Vi = membrane potential (mV)/e i 3 2Ge i = conductivity (mS/cm) Imemi
= membrane current (mA/cm ) Ci = membrane capacitance (mF/cm )/
3 2fe i = volume fraction Iapp = applied current (m A/cm ) Iioni = membrane current (mA/cm )/
Levels of Modeling
NumericalMultiple intracellular domains
Voltage-dependent conductances
ioni = � g ij � qijk (Vi - E j )
j k
¥¶qijk qijk -q Vijk ( i )= -¶t t ijk (Vi )
Complex electrode geometry
Tissue inhomogeneous and anisotropic
under construction
AnalyticalA single intracellular domain
Passive membrane conductance
I ion = g L (V EL )-
Simple electrode geometry
Tissue assumed homogenous and isotropic
much progress
I
Bi-domain Model for theMicrocapillary Bioreactor
Calculate profiles
F1 e i (x;w)/
in bioreactor
...and impedance...
Z(w) =
F1 e ( L;w) -F1
e (0;w) j1
100 Hz
1 1 / Write BCs and assume: ( , ) ( ; )i t i t
e i e i j j e x t xw w= � F = F
Z
w
( )
/
...and predict
as tissue parameters
, , , , ,
are experimentally
manipulated
e i e i L
Z
f G C g E
w
a
V
e F
i F L E
/ ew
/ L
Recap
� Focused & integrated effort • BioMEMS…Neural
Engineering…Materials… Computational Neuroscience…Cellular Biology…Visualization
� Why are we so excited?• We have the very real
potential of characterizing the biological responses to neural implants and then engineering new classes of microdevices to provide a permanent high-capacity interface to the brain
BIO
INFO
MICRO
Why the BIO, INFO, andMICRO Program?
� Wide-open Challenges • Characterizing and modeling the biological (cellular and
chemical) responses around a neural implant • Controlling the dynamic biological responses around a neural
implant. • Designing, fabricating, and using “advanced” neural implants
� Collaboration Possibilities • Additional functionalities for implantable microdevices of the
class that we are working on. • Exploring fundamentally new types of tissue-device interfaces. • Complementary studies of the neural interface (experimental
and analytical) • Confocal microscopy of the neural interface • Sharing technologies, procedures, insights, etc… • New emergent ideas…
Systems-level Analysis of AdvancedNeuroprosthetic Systems
Feedback control signals
Subject Neural system
(global)
Controlled neural plasticity
local
Neural Implant
Adaptive Controller
External World
Objective 2: Optimize Adaptive Controller
Objective 1: Optimize neural interface
Systems-level Approach for AdvancedNeuroprosthetic Systems
Subject Neural system
(global)
Controlled neural plasticity
local
Neural Implant
Adaptive Controller
External World
Feedback control signals
Objective 2: Develop Objective 1: Optimize neuraladaptive controller to interface
optimize systemperformance.
Advanced Neuroprosthetic Systems
High-Level Neural Computation
Sensory Transduction & Pre-processing Motor
Commands Movement
Perception, Decision, Detection
Sensory Integration
External World
Neuroprosthetic System
� Underlying System Principles •Two-way communication with targeted neural systems
•Harness neural plasticity to our advantage
•Appropriately balanced “wet-side” and “dry-side” computation
Approach
� Four Project Areas �Direct neural control of actuators
�Detection of novel sensory stimuli through monitoring neural activity
�Neural control of behavior
�Investigate signal transformations from ensembles of single neurons to local field potentials to EEG.
Topics
�
�
�
�
Project overview
Towards the Development of 3rd-Generation Neural Implants (BIO, MICRO, and INFO)
Bioactive Coatings to Control the Tissue Responses to Implanted Microdevices (BIO, MICRO, and INFO)
Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)
� Direct Cortical Control of a Motor Prosthesis(BIO, MICRO, and INFO)
� Neural Control of Auditory Perception
� Wrap-up
Direct Cortical Control of Actuators
High-Level Neural Computation
Sensory Transduction & Pre-processing Motor
Commands Movement
Perception, Decision, Detection
Sensory Integration
External World
NeuroprostheticSystem
Goal: Control arm-related actuator
External Actuator Robotic Arm or Virtual Reality
Fundamental Questions
� What are “optimal” real-time signal processing strategies for precise 3-D control of external, arm-related actuators in the presence of sensory distractions and/or physical perturbations to the arm?
� To what extent can we use composite neural signals [neuronal (unit) recordings, local field potentials, and brain-surface recordings] for control signals?
� How do we take advantage of inherent or controlled neural plasticity in order to optimize system performance?
Experimental Preparation
• Train monkeys to perform tracking and/or reaching tasks.
• Record cortical responses with multichannel neural
implants.
• Measure arm movement in 3-D space.
Chronic Neural Recordings
� Multi-channel neural implants in motor and sensorimotor cortical areas.
� Eventually: Sub-dural electrodes for local potentials
-0.2 0 0.2 0.4 0.6
0
10
dsp009b
-0.2 0 0.2 0.4 0.6
0
20
40
dsp012a
-0.2 0 0.2 0.4 0.6
0
10
20dsp018a
-0.2 0 0.2 0.4 0.6
0
20
40
dsp024a
-0.2 0 0.2 0.4 0.6
0
20
40dsp025a
-0.2 0 0.2 0.4 0.6Time (sec)
0
10
dsp030a
-0.2 0 0.2 0.4 0.6
0
100
dsp034a
-0.2 0 0.2 0.4 0.6
050
100150
dsp037a
-0.2 0 0.2 0.4 0.6
05
10
15dsp040a
-0.2 0 0.2 0.4 0.6
0
10
20
dsp042a
-0.2 0 0.2 0.4 0.6
0
20
dsp042b
-0.2 0 0.2 0.4 0.6Time (sec)
010
20
30dsp045a
-0.2 0 0.2 0.4 0.6
0
20
40
dsp046a
-0.2 0 0.2 0.4 0.6
0
5
1015
dsp051a
-0.2 0 0.2 0.4 0.6
0
20
40
dsp057a
-0.2 0 0.2 0.4 0.6
0
40
80
dsp058a
Perievent Histograms Target 1, reference = C_rel, bin = 20 ms
NeuralRecordingSystem
OfflineAnalysis
Real-timeSignalProcessing
ActuatorControl
Extracellular recordings
Direct Cortical Control of Movement
Green ball: Target Yellow ball: Actual hand position, or hand position estimated from cortical responses
m0602pa
Topics
� �
�
�
�
Project overview Towards the Development of 3rd-Generation Neural Implants (BIO, MICRO, and INFO) Bioactive Coatings to Control the Tissue Responses to Implanted Microdevices (BIO, MICRO, and INFO)
Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)
Direct Cortical Control of a Motor Prosthesis (BIO, MICRO, and INFO)
� Neural Control of Auditory Perception(BIO, MICRO, and INFO)
� Wrap-up
Neural Control of Auditory Perception
High-Level Neural Computation
Sensory Transduction & Pre-processing Motor
Commands Movement
Perception, Decision, Detection
Sensory Integration
External World
NeuroprostheticSystem
Goal: Control auditory perception
Fundamental Questions
� To what extent can we control auditory-mediated behavior using intra-cortical microstimulation (ICMS) through the neural interface?
Transmitter Channel Receiver Source Signal
Received Signal
Stimulator Neural Interface
Auditory Cortex
� What are the information transmission characteristics of the multichannel neural implant in high-level cortical areas using ICMS? � Channel capacity (bits per second) � Channel reliability � Channel resolution
� How can we optimize information transmission � Implant designs, Neural implant locations, Signal encoding strategies,
Controlled neural plasticity
Chronic Neural Recordings
� Multi-channel neural implants in primary auditory cortex
Extracellular recordings in auditory cortex
Estimation of
Neural Recording System
Offline Analysis
Neuronal ResponseProperties
Algorithm Selection
Signal Encoder
SoundsElectrical Stimulation to Aud. Ctx.
Behavioral performance to both sounds and cortical electrical stimulation
Auditory Behavior
• Lever-press sound or ICMS discrimination task
• Center paddle hit starts trial, 2-tone pair presented
• Reward obtained by signaling the correct stimulus sequence
centerleft right
rat
Frequencyresponse areas
Frequency Selectivity in Auditory Cortex
1 2 5 10 20 30
40
60
80dsp024b
28.
56.
1 2 5 10 20 30
40
60
80dsp018d
11.
22.
1 2 5 10 20 30
40
60
80dsp020a
10.
20.
1 2 5 10 20 30
40
60
80dsp024a
10.
20.
1 2 5 10 20 30
40
60
80dsp012a
21.
42.
1 2 5 10 20 30
40
60
80dsp018b
10.5
21.
1 2 5 10 20 30
40
60
80dsp018c
22.
44.
1 2 5 10 20 30
40
60
80dsp002a
3.
6.
1 2 5 10 20 30
40
60
80dsp002b
5.5
11.
1 2 5 10 20 30
40
60
80dsp010b
12.
24.
Freq.
SoundLevel
Signal Encoding Algorithm:Frequency Selectivity
ICMS pattern is based solely on frequency selectivity of neurons recorded on an electrode
dB
80
60
40
u5b 8
6
Spik
es
4
2
01 5 10 30
kHz
u32a
0
2
4
6
8
kHz
1 5 10 30
Spik
es
80
60dB
40
Behavioral Performance
Ricms6
Rat Behavioral Performance
RICMS 6
100
09/06
/00
09/16
/00
09/26
/00
10/06
/00
10/16
/00
10/26
/00
Training day Implanted
90
80
Per
cen
t Co
rrec
t
70
60
50
40
30
20
10
0
Cortical Electrodes
D
Expected Results to ICMS Stimuli
Begin ICMS
100
% D% due to ICMS
Trial #
Auditory trial =
ICMS Algorithm1 =
ICMS Algorithm2 =
Behavioral Curve
RICMS 6 10/25 (Only Session)
100
80
Per
cen
tag
e
audPercent, icmsPercent,
60
40
20
0
0 100 200
Trial
Alternative Signal Encoding Algorithm:Cortical Activation Pattern
For a given electrode, the unit firing pattern is used as a template for ICMS delivery
Auditory Stimulus
Sound on
Response Raster
Matching ICMS ‘pattern’
***Procedure is simultaneously duplicated on each active electrode
Recap
� Focused & integrated effort • Neural Engineering…Signal
Processing…Systems Neurophysiology…Visualization
� Why are we so excited? • We have the very real
potential of developing new classes of neuroprosthetic systems to explore our ability to interact directly with the brain.
BIO
INFO
MICRO
BIO, INFO, and MICRO…
� Wide-open Challenges • Appropriate mathematical constructs for describing neural
encoding and decoding.
• Advanced data visualization techniques for understanding this new class of neural data.
• Understanding signal transformations as a function of the spatial and temporal scale of the neural data.
� Collaboration Possibilities • Exploring new signal encoding and decoding strategies for
particular neuroprosthetic applications.
• Sharing technologies, procedures, insights, etc…
• New emergent ideas…
Topics
�
�
�
�
�
�
Project overview
Towards the Development of 3rd-Generation Neural Implants (BIO, MICRO, and INFO)
Bioactive Coatings to Control the Tissue Responses to Implanted Microdevices (BIO, MICRO, and INFO)
Modeling the Device-Tissue Interface (BIO, MICRO, and INFO)
Direct Cortical Control of a Motor Prosthesis (BIO, MICRO, and INFO)
Neural Control of Auditory Perception(BIO, MICRO, and INFO)
� Wrap-up
Project Challenges
� Scientific • Overcoming engineering and scientific hurdles.
• Identifying and fostering strategic alliances with appropriate external groups.
• Crossing disciplines
� Management • Strategic planning
• Resource allocation
• Open and effective communication among the diverse project team
• Team-building: Maintaining enthusiasm, energy, and focus after the initial “honeymoon” period
“Insanely IntenseInterdisciplinary” Research
“pieces of a puzzle” “easy synergism”
BIO
INFO MICRO
BIOMICRO
INFO
Breakthrough Science
•Hard work •Open minds •Honesty •Top-notch research
--
What Does the Future Hold?
“Perhaps within 25 years there will be some new ways to put information directly into our brains. With the implant technology that will be available by about 2025, doctors will be able to put something like a chip in your brain to prevent a stroke, stop a blood clot, detect an aneurysm, help your memory or treat a mental condition. You may be able to stream (digital) information through your eyes to the brain. New drugs may enhance your memory and fire up your neurons.” Dr. Arthur Caplan,
Director of the Center of Bioethics, University of Pennsylvania Arizona Republic, Dec 27, 1998.
Acknowledgments
� ASU Colleagues • 13 co-PI’s, 5 research faculty, numerous graduate
and undergraduate students.
� Arizona State University administration • Seed funding from Department, College, and
University
• Significant cost-share on this project
� DARPA Program Managers • Eric Eisenstadt, Abe Lee, and Gary Strong