Embed Size (px)
Citation preview
Agenda
Introduction to IMEC
Wearable Healthcare
Introduction
Analog Sensor Interface
Digital Signal Processing
Implantable Neural Interface
Introduction
ASIC Architecture
Recording / Stimulating Circuits
Spike Sorting
Joonsung Bae 2
IMEC – Leuven, Belgium
Joonsung Bae 3
CMOS and beyond CMOS
Joonsung Bae 4
Industry’s partner ecosystem
R&D 300mm cleanroom
Next-generation logic/memory devices
3D system integration
Advanced patterning
Wearable
Chronic disease management
Lifestyle and preventive care
Bio-medical sensor SoC
Wearable prototypeshadware/firmware/sotware/mechanics
Joonsung Bae 5
Life Sciences
Personalized medicine
Neuro-Electronics
DNA analysis
Single-molecule interaction
Joonsung Bae 6
Data Science and Data Security
Data extraction
Data mining
Data visualization
Machine learning
Data security
Joonsung Bae 7
Wearable Healthcare
Market Trends
Joonsung Bae 9
1990s:
• PCs
2000s:
• Notebooks
2010s:
• Smartphones/Tablets
2020s:
• Wearable devices
From portable devices to wearable devices
Growing Industry Interest
Joonsung Bae 10
Technology Enablers for Intelligence
Joonsung Bae
Cloud computing, Big data analysis
Flexible electronics
Autonomous wireless connectivity
Advanced sensor platform
Software
Material
Protocol
Hardware
Smart sensor
SensorPhysical property
Quantitative indication
Sensing element
Signal conditioning
Generate response
Signal processing
Electrode
Antenna
Transducer
Microphone
Photo-detector
Wireless data transmission
Actuator response
User feedback
Compression
Feature extraction
Signal analysis
security
Analog interface circuitry
Analog to Digital Conversion
12Joonsung Bae
Sensor Hardware : SIMBAND
Joonsung Bae 13
Tizen OS
Sensors
Cloud Sensor Platform : SAMI
Like Apple HealthKit Platform
Open API to analyze the sensed signal
Continuous and Non-invasive
Joonsung Bae 14IT대학 교수 세미나
Challenges
Joonsung Bae 15
Long Term Battery Autonomy
High Quality and Reliable Data
Low Cost
Desirable form factor
Personalized
Many sensors
Smart Sensor Devices
Joonsung Bae 16
Sensor interfaces
Local processing
Data collection
Comm. interface
Radio
Storage
Controller
Power management
data stream
Man
y se
nso
rs
µProcessorDSP
Memory
Accelerators
DMA
Local processing Cloud processing
Clocking
Typical Architecture
Joonsung Bae 17
AHB
Processor
ARM, ARC
Memory
SRAM, Flash
Accelerators
Filters
Sample Rate Conversion
Encryption
Compression
Matrix/Vector calculation
FFT
Timestamp
DMA
ABP
Sensor
read-out
ECG, BioZ,
GSR, PPG, GP
Sensor drivers
LED
CS (BioZ)
Oscillator
CMUPMU
LDO, DC-DC
Interfaces
SPI, I2C, UART, GPIOTimers
RTC
Analog Digital
Analog Sensor Interface
Signal Characteristics
19
[Webster1992]
10mV
1mV
100uV
10uV
1uV
100nV
1Hz 10Hz 100Hz 1kHz 10kHz
ECG
Action Potentials
LFP
ECoG
EEG
EMG EMG
ECG
EEG
ECoG&
LFPAP
Joonsung Bae
Sensing Challenges
Joonsung Bae 20
Cbo
dy-t
o-
main
s
BIASVBZbias
Zelec2
Zelec1
V2
V1
VB
MAINS
Sensor
Interface
DC-offset:
Mains interference (VCM):
Electrode Impedance Mismatch converts VCM
into VDM
21 VVVDC
BIASCCM ZIV
V V CM
ZIN
Z ZIN E2
Inherent CMRR
V V CM
ZIN
Z ZIN E1
DV
VCM=
DZ
ZIN
General Implementation
Joonsung Bae 21
Filters PGAIA
AA
ADC Filters
sensor
signal Digital samples
#bits@sample rateOther phases / frequencies
DC Control
Application requirements
High input impedance
Low thermal noise
Low 1/f noise
Large dynamic range
Low power
High CMRR
Low offset
Design techniques
Chopper modulation
Correlated double-sampling
DC-servo
Bootstrapping
Design in weak inversion
Digital Signal Processing
Accelerators - Motivation
Joonsung Bae 23
A processor (like e.g., ARM Cortex M0) is popular for embedded applications
Ease of programming
Availability of tools and hardware
However:
optimized for controlling tasks.
DSP tasks (like data filters) not well supported needs many more cycles
Latency due to software (Real-time issue)
Might be less accurate
Accelerators
Joonsung Bae 24
For sensor modules several accelerators might make sense:
Digital filtering (LPF, HPF, CIC)
Sample Rate Conversion
Motion Artefact Reduction
Time-stamping
Encryption
Compression
Vector/Matrix calculations
FFT
....
They off-load the processor and can do the calculations faster, more accurate and with less energy
Motion Induced Signal Artifacts
Joonsung Bae 25
Ambulatory ECG monitoring
Enabling continuous health monitoring under user’s daily routine
Body movement occurs significant Motion Artifacts on ECG signal, which suffers from ..
Poor signal quality
Potentially wrong clinical diagnosis
Steady Motion induced
PCA Processing for Artifact Reduction
Condense Measured data set into a few “principal components”
Principal components are a linear combination of the data set, with weights chosen so that they are mutually uncorrelated
Joonsung Bae F. Castellas et. al. “Principal Component Analysis in ECG Signal Processing”
8 channelsMeasured ECG Processed ECG
4 principalcomponents
Mutuallyuncorrelated
PCAProcessing
PCA is powerful algorithm for MA reduction Because MA and ECG are uncorrelated
PCA Algorithm
Measure M-channel ECGs with N-samples
Define the data matrix X
Calculate the covariance matrix Sx
Calculate the Eigenvectors and Eigenvalues of the Sx
Chose feature vector (P) with eigenvectors
Final Data = P ∙ X
Joonsung Bae F. Castellas et. al. “Principal Component Analysis in ECG Signal Processing”
Mch
an
nels
N samples
X∙XT (multiply a large matrix with its transposed matrix)
P∙X (multiply a small matrix with a large matrix)
X-E (column-wise subtraction for eigenvalue calculation)
Huge matrix operations
H/W accelerator for vector / matrix operations is needed to
reduce the burden on M0
Vector / Matrix Operations
Principal Component Analysis (PCA) and Independent Component Analysis (ICA) to separate any noise from ECG signal by making use of N-channel ECG signal
Joonsung Bae 28
▸ Transposed
multiplication
▸ Matrix
multiplication
▸ Matrix row
summation
Rnn = Anm . AT
mn
n = 1..4, m = 1 ... 5120
Rm =
Processor
Memory MAU accelerator
AHB
Matrix Processor Kernel
Joonsung Bae 29
Matrix to Matrix multiplication
Accumulator in the inner loop
Define ResultR[COUNT_2][COUNT_1];
For INDEX_1=0 to COUNT_1 -1 { // for each col (B)
For INDEX_2=0 to COUNT_2 -1 { // for each row (A)
Rval=0;
For INDEX_3=0 to COUNT_3 -1 {
Rval = Rval + (operandA[INDEX_2][INDEX_3] *
operandB[INDEX_3][INDEX_1]);
} // for each col(A) & row(B)
ResultR[INDEX_2][INDEX_1] = Rval
}
}
Pseudo code
CO
UN
T_
2
COUNT_3 COUNT_1
CO
UN
T_
3
CO
UN
T_
2
COUNT_1
Benefits
Joonsung Bae 30
Perform X*XT
Software realization running on the Cortex M0 vs. Matrix Processor
ECG with Motion Artifact Reduction
Joonsung Bae
PCA processing with concurrent 3-lead ECG signal
3-channel ECG (512 S/s)
SoC Implementation
Joonsung Bae 32
0.18mm CMOS
7.2 mm x 8mm
10.4MHz core clock
On-chip 192kB SRAM
Continuous data collection with analog and digital
Future Direction
ISSCC 2017
Joonsung Bae 33
EEG Artifact Reduction in Analog
Joonsung Bae 34
Analog Signal Processing
Joonsung Bae 35
Linear Transforms
Joonsung Bae 36
Dot Product Unit
Joonsung Bae 37
Fully Autonomous System
Acoustic Sensing and Object Recognition System
Joonsung Bae 38
Future Directions
Joonsung Bae 39
Machine learning!! Context-aware sensing, Motion artifact, Personalization
Dedicated Hardware with M0 or
Software Implementation with upgraded core like M4
Implantable Neural Interface
Central Nervous System
Joonsung Bae 41
Neural Interface – Closed Loop
Joonsung Bae 42
Peripheral Nervous System
Joonsung Bae 43
Sensory perception
Motor functionality
CMOS Chip in the Nerve
Joonsung Bae 44IT대학 교수 세미나
x8 Readout Channels
x8 Readout Channels
x16 Stimulation Units
x32 Recording Pixels
Switch Matrix
x32 Recording Pixels
Dig
ita
l U
nit
500 μm
Time [ms]
Delicate surgery procedure
MultidisciplinaryCooperation
Chip designer
Processing engineer
Neural scientist
Neural surgeon
Where to sense in Vivo
Joonsung Bae 45
45
Electrode
Skull
Stratum Corneum
SubcutaneousLayers
Brain
Electrode
Electrode
Skin
EEGSubcutaneous
Scalp
EEG ECoG LFPAction
Potentials
Non-Invasive Invasive
Macroscopic
Microscopic
ASIC Architectures
System Architecture
Joonsung Bae 47
Ele
ctr
odes
Analog amplification
& filtering
A/D Signal processing
Data transmission
Data
analy
sis
Stimulation circuitry
D/A Digital control
Data transmission
How to partition the system
Signal Processing and Integration
Joonsung Bae 48
What to do with all
that data??
Local signal processing
Data reduction/compaction
Feature extraction
Reduced output data rate and consequently reduces TX power
But, algorithms can be easily computationally-intensive
Need trade-off between TX power vs processing power
Algorithm and hardware optimization is required
Transmission Data
Processing Power
Passive Arrays
Joonsung Bae 49
Integrated External
Ele
ctr
odes Analog
ampl.A/D Signal
processingData
transmission
Data
analy
sis
Stimulation circuitry
D/A Digital control
Data transmission
[Campbell et. al. 1991]
Active Arrays
Joonsung Bae 50
[Lopez et. al. 2013, Lopez et. al. 2016, Raducanu et. al. 2016]
Integrated External
Ele
ctr
odes Analog
ampl.A/D Signal
processingData
transmission
Data
analy
sis
Stimulation circuitry
D/A Digital control
Data transmission
Active Arrays
Joonsung Bae 51
Target: reduce output data rates and power
Spike detection => only transmit spike data and timestamps
Spike sorting => only transmit cluster information
Lossless data compression
Other advantages:
Simple close-loop functionality
Increased number of readout channels (> 1000)
Integrated External
Ele
ctr
odes Analog
ampl.A/D Signal
processingData
transmission
Data
analy
sis
Stimulation circuitry
D/A Digital control
Data transmission
Wireless Active Probes
Joonsung Bae 52
Advantages:
Autonomous system => no cables
Reduced packaging effort
Reduced tissue damaged => untethered probe
Versatile freely-moving animal experiments
Challenges:
Power required for wireless transmission
Bandwidth
Integrated External
Ele
ctr
odes Analog
ampl.A/D Signal
processingData
transmission
Data
analy
sis
Stimulation circuitry
D/A Digital control
Data transmission
Fully Autonomous Active Probes
Joonsung Bae 53
Dream solution:
On-chip data analysis for advanced close-loop applications
Minimum data transmission: only configuration or status
Challenges:
Design of machine learning algorithms
Minimize size of integrated system
Integrated
Ele
ctr
odes Analog
ampl.A/D Signal
processing
Data
analy
sis
Stimulation circuitry
D/A Digital control
Recording / Stimulating Circuits
Neural Recording Circuits
Joonsung Bae 55
How to design a good neural amplifier
Recording
Electrode
Reference
Electrode
Low-Noise
Neural
Amplifier Filter
Programmable
Gain Amplifier ADC
Neural Amplifier Specs
Joonsung Bae 56
Neuro-modulation
Joonsung Bae 57
Non-invasive
E.g. tENS (Transcranial Electrical Nerve Stim.)
E.g tMS (Transcranial Magnetic Stimulation)
Invasive
E.g cochlear implants
E.g. Deep Brain Stimulation
Various stimulation methods
Electrical
Optical
Chemical
Ultrasound
Electrical Stimulation
Joonsung Bae 58
Electrical Stimulation
Joonsung Bae 59
IStim
Tissue
WE
CEIStim
IStimIStim
S1
S1
S2
S2
S3 S3
VSS
VDD VDD
VSS(b)
ON OFFS1
S2
S3
IStim
Tissue
WE CE
IStim
IStimIStim
S1
S1
S2
S2
VSS
VDD VDD
VSS(a)
ON OFFS1
S2
Passive charge balancing
Series capacitor to block DC current into the tissue (avoid charge buildup)
Short to ground (usually limits stimulation frequency)
Spike Detection & Sorting
Neural Probe Data Bottleneck
Joonsung Bae 61
Upscaling will generate ever more data.
Example:
Sampling Rate = 30kSps
Resolution = 10bits
Number of Channels = 1000
Total Data Rate = 300Mbps
What to do with all that data??
Neural Systems
Joonsung Bae 62
Transmitting raw data over wireless is too power hungry!
Mostly wired solutions are used today
Bulky and unreliable
Impose serious limits on the experiments that can be performed
7.2 mm
20 mm
3 mm
Spike detection
63
Raw signal
Filtering
Emphasis
Threshold
Window
Goal:
Find the regions in the raw signal
where a spike is present and
discard the rest.
Joonsung Bae
Spike detection
64
Raw signal
Filtering
Emphasis
Threshold
Window
BPF removes undesired signals:
baseline wander
LFP
High-frequency noise
[S. Gibson 2012]
Joonsung Bae
Spike detection
65
Raw signal
Filtering
Emphasis
Threshold
Window
Filtered data
NEO
Amplify spike for more reliable thresholding
Avoid erroneous spike detection
Several methods
Non-linear energy operator
Template matching
[S. Gibson 2012]
Joonsung Bae
Spike detection
66
Raw signal
Filtering
Emphasis
Threshold
Window
Retain only spike signals
Significant data reduction
NEO
[S. Gibson 2012]
Joonsung Bae
Spike alignment
Threshold crossings are typically noisy
Need to find a better way to properly align the detected spikes for efficient feature extraction and spike clustering.
Align based on local maximum
Align based on maxim slope
67
After spike detection After spike alignment
[S. Gibson 2012]
Joonsung Bae
Spike Sorting
Joonsung Bae 68
Recorded signal = superposition of signals originating from various neurons close to the recording site.
Which neuron fires when?
Different neurons generate different spike shapes
Spike Sorting
Joonsung Bae 69
Spike sorting:
identify spike times of individual neurons
“which neuron fires when”
Obviously reduces data rate significantly
But is also often the first step required for proper data interpretation
Neuron A
Neuron B
Neuron C
Spike sorting
Feature extraction
Map time-domain waveform onto multi-dimensional feature space
Simple example:
Peak-to-peak amplitude
Spike width
More complex algorithms extract much more features:
Principle Component Analysis (PCA)
Discrete Wavelet Transform (DWT)
70
AmplitudeW
idth
Joonsung Bae
Clustering
Identify clusters in the multi-dimensional feature space
Clusters indicate similar spikes spikes originating from the
same neuron
Very active field of ongoing research (cfr. machine learning)
71
NEURON A spikes
NEURON B spikes
NEURON C spikes
Joonsung Bae
Neural Signal Processing
Joonsung Bae 72
[Gibson 2012]
Future Directions
Joonsung Bae 73
Incredible progress, yet still such a long way to go
High-density analog
Digital signal processing
Material science & processing technology
Wireless power and data
Algorithm for efficient spike
sorting
Protocol for autonomous connectivity
Any Suggestions?
배준성 (裵俊盛)
http://sites.google.com/view/kwbics
