Basics of Brain Computer

Interface

Chapter 2

BING 8995

Dr. Vidya Manian

BCI, BMIs

• Direct neural interface, or mind-machine interface: bypass typical neural pathways (nerves and muscles) by providing novel output pathways to interact with a variety of applications that replace, improve, enhance, restore, and supplement the human user’s central nervous system output

• Important for individuals with compromised neural tracts

• 2 million people suffer from neural disabilities• 500,000 people suffer from locked-in syndrome

BCI

• Used by people with motor impairment to write sentences, control an unmanned aerial vehicle, prosthetic, play video games and perform collaborative work

• BCI system has following steps: brain activity pattern generation, signal acquisition, feature extraction, and classification

• Brain activity: electrical activity, magnetic fields created by electrical activity, blood oxygenation

• Differs in spatial and temporal characteristics depending on stimulus type, stimulus intensity, mental effort, and mental status

BCI challenges

• Low BCI signal strength: Brain signals have low strength,

signal amplification is required

• Data transfer rate (bandwidth): Best transfer rate from a subject

is 3 characters. BCI suffers from fast response as well as

accurate control

• High error rate: Error percentage is high. Brain signals have

high variability

• Inaccurate signal classification: Classifying the captured

signals have high interference and inaccurate classification

Brain Anatomy

Brain Anatomy

• 5 basic parts: Cerebrum, diencephalon, cerebellum, mesencephalon, and medulla oblongata

• Cerebrum functions: initiation of complex movement, speech and language understanding and production, memory and reasoning

• Diencephalon forwards sensory information to other brain areas• Cerebellum coordinates all movement• Mesencephalon controls vigilance and sleep-wake rhythm• Medulla oblongata connects brain with spinal cord• Cortex consists of two hemispheres connected by the corpus

collosum• Right hemisphere is activated more during recognition of geometric

patterns, spatial orientation, non-verbal memory and noises• Left hemisphere is activated more for recognition of letters and

words, auditory perception of words and language

Hemisphere partitioned into 4 lobes and functionally defined cortical areas

Mapping of motor cortex with motor functions

Ventral view of the brain and spinal cord

Scales of cortical tissue

Spatial scale of cortical tissue structure

CNS/Muscle and CNS/BCI systems

Design and operation of a BCI system

Classification of BCI systems

BCI categories

• BCIs are categorized by brain signal pattern, stimulus modality, mode of operation, operation strategy, and recording method

• A brain signal is a set of electrical impulses that flows on groups of active neurons. Eg., P300 Event Related Potentials (ERPs) that reach a maximum peak in voltage about 300ms after stimulus onset (oddball), requires no initial training

• P300 can be evoked by visual, auditory, tactile, and even olfactory or gustatory stimulus

• SSVEP BCIs are based on visual stimulus. Steady State Visual Evoked Potential (SSVEP) are recorded from the scalp originating at the brainstem level

• SSVEP located within the occipital lobe of the brain caused by focusing on regularly flashing light

Brain signals for BCI

• Slow Cortical Potentials (SCPs) are lesser used brain signals

• SCPs are shift in the cortical electrical activity lasting from several ms to seconds.

Sensorimotor Rhythm (SMR) involves event-related desynchronization/synchronization (ERD/ERS), brain activity associated with imagining motor behavior

Methods for recording brain signals

• Noninvasive: uses equipment that touches the scalp such as EEG or near Infrared Spectroscopy (fNIRS), functional Magnetic Resonance Imaging (fMRI), Magnetoencephalography (MEG), or Positron Emission Tomography (PET).

• Invasive: require surgery in order to place signal receptors within the brain (intracortical neuronal recording and electrocochleography (ECoG))

Operation Strategy

• BCIs that use selective attention presents users with auditory, visual or tactile stimulation that can elicit brain signal responses

• Cognitive efforts: relies on biofeedback to train users to maintain a desirable level of brainwave frequency amplitude

• Motor Imagery allows the user to imagine muscle movements, to cause a spike in neuronal activity in the motor cortex

BCI mode of operation

• Brain signals are elicited synchronously or asynchronously

• Synchronous BCIs are cue based, information is presented to the user in order to elicit certain brain signal responses

• Asynchronous BCIs are self-paced, they are controlled through user intention in the user’s desired timing

Stimulus modality

• Visual, tactile, or hybrid which combines multiple BCIs such as imagined movement and visual attention, or P300 and SSVEP

Overview of a general BCI system framework

Signal acquisition methods

Signal Acquisition

Noninvasive recording methodsEEG

• Direct measures that detect electrical (EEG) or magnetic activity (MEG) of the brain

• Indirect measures reflecting brain metabolism or hemodynamics of the brain (fMRI, fNIRS, and PET)

• EEG – measures “the flow of electric current during synaptic excitations of the dendrites of many pyramidal neurons in the cerebral cortex

• EEG picks local current flows on groups of active neurons within the cerebral cortex

• Neural oscillations picked by EEG are called “brainwaves”

• Range of µVolts

Five categories of brainwave patterns

Brainwaves

• Delta brainwaves reflect slow, loud, and functional mental states that prevail during late sleep

• A headset consisting of an EEG cap with atleast 3 electrodes (ground, reference, and a recording electrode) is needed

• An amplifier, an A/D converter and a computing device• Electrodes are made of Ag, AgCl, or gold, could be wet (needs gel) or dry• Active and reference electrodes serve as the signal receptors for potential

difference comparisons• Ground electrode serves as the baseline of brainwave signals that helps remove

irrelevant data from the active and reference signals• Correct electrode placement is important: 10/20 system internationally recognized

standard system with 21 electrodes• Skull is divided into 6 areas with interval rates of 10%, 20%, 20%, 20%, 20%, and

10% (Fp: frontopolar, F: frontal, C: central, P:parietal, and O: occipital)• Left to right preauricular points (T3: temporal, C3, central, Cz, C4, and T5)• With multi channel EEG acquisition the 10/20 system is extended to 10/10 and

10/5 systems allowing more than 500 electrode positions

• Anatomical landmarks for positioning the electrodes: (1) the nasion: point between forehead and nose, (2) the inion: lowest point of the skull from the back of the head, indicated by a prominent bump; (3) pre-auricular points anterior to the ear

• 10 and 20 refer to the distance between adjacent electrodes

• ‘z’ refers to an electrode placed on the midline, and a number to identify the hemisphere location

• O1 identifies left occipital, C4 identifies right central, P3 –left parietal, A1 – left ear reference

• EEGs are most popular technique for BCI• It is noninvasive, inexpensive, and portable• Does not have high spatial resolution, mathematically

difficult to accurately compute the distribution of currents within the brain that generated these signals “inverse problem”

10/20 international system of electrode placement

fNIRS: functional Near Infrared Spectroscopy

• Images blood oxygenation and deoxygenation in the cerebral cortex; these absorb light at different rates

• fNIRS uses difference in light absorption to detect neuronal activity

• Infrared light source, a light detector, signal processing device, computer

• The penetrated light undergoes attenuation based level of blood oxygenation

• Temporal resolution is slower in fNIRS than EEG• fNIRS has motion, pulse and heartbeat artifacts that

need to be removed by preprocessing using ICA, wavelets, to improve SNR before feature extraction

PET: Positron Emission Tomography

• Noninvasive 3D radiation or nuclear medicine imaging technique that is used to measure the functional processes within body including neural activity

• When a positron passes through matter, two photons are simultaneously emitted in almost exactly opposite directions

• A positron emitting tracer atom is introduced into the bloodstream in fludeoxyglucose, which concentrates in areas with higher metabolic needs

• This tracer molecules emits positrons, which are detected by a sensor

• A 3D image is constructed of the areas of the brain that have the highest metabolic needs

• PET machine consists of coincidence detectors, scintillating crystals, and block detectors

Invasive methods

• Electrocorticography

• Intracortical neuron recording

BCI research articles

Brain signal patterns for BCI operation

• Classified according to physiological bases, initial training requirement for use, and rate at which information is transferred from brain to application

• P300 ,SSEP and ERD/ERS all use neuroelectricsignals (EEG)

Brain signal patterns for BCI operationP300 ERP and Sensorimotor rhythm

P300

• P300 is an ERP component of the EEG that reaches a maximum positive peak in voltage about 300ms after stimulus onset

• Appears at central and parietal scalp regions

• Healthy individuals reach high accuracy with little training time

Face SSVEP and Slow cortical potential

Artifacts in EEG signals: eye blink, eye movement 50Hz, muscle activity, pulse

Artifacts and noise sources, causal factors, and removal methods

Spatial Filtering: Referencing methods

Referencing methods

• Bipolar reference: electrode placed in an ‘inactive’ zone such as the mastoid, earlobe, nose or base of the neck that has 0 potential

• EEG reference is critical: it can improve SNR of the electrical activities at target and reference sites

• Vbipolar(i,j) = Vi-Vj (i is recording electrode, j is reference electrode

• Advantages: detect spatial differences, signal may cancel out when brain activities at the 2 positions are similar

Surface Laplacian Reference

• S(i) is subset of 4 adjacent electrodes surrounding the target electrode i, di,j is distance from target electrode i to an adjacent electrode j

• It can be used as a spatial filter

Common Average Reference

• The potential is averaged over all electrodes

• N is the total number of electrodes, Vi is the potential between the recording electrode i and reference electrode j

• The advantage is that it can measure spatially broad activities that cannot be measured by bipolar or surface Laplacian reference methods

Data dependent spatial filtering

• Principal component analysis

• Independent component analysis

• Common spatial pattern

• Feature extraction: SSVEPs

Feature classification methods

• Linear classifiers: LDA, SVM

• Artificial neural network classifiers: mUltilayerperceptron, LVQ

• Hidden Markov Model classifiers

Neuron• Neuron- leaky bag of charged liquid, membrane is a lipid bi-

layer with openings called ionic channels that allows specific ions

• Aqueous medium with sodium, chloride and calcium on outside of cell and potassium and organic anions inside

• When neurons receive an input, there is influx of sodium, and rise of potential causes outflux of potassium ions

• Rise and fall of membrane potential: Action potential or spike

• Information is transferred by firing rate: number of spikes per second and/or timing of spikes

• Neurons emit a 0 or 1 digital output

Ions in a neuron

Action potentials or spikes

Dendrites and axons

• Neuron structure includes a cell body (called soma),connected to a tree-like structure with branches called dendrites and a single branch called the axon that emanates from the soma and conveys the output spike to other neurons

• The spike is initiated near the junction of the soma and axon and propagates down the length of the axon

• Many axons are covered by myelin, a white sheath that significantly boosts the speed of propagation of the spike over long distances

• White matter and gray matter: myelinated axons connecting different brain regions and the regions containing the cell bodies

Synapses

• Neurons communicate with each other through connections called synapses

• Synapses are chemical, a gap or a cleft between the axon of one neuron (presynaptic neuron) and a dendrite of another neuron (postsynaptic neuron)

• When an action potential arrives, neurotransmitters (chemicals) are released into the cleft. They bind with ionic channels( or receptors) on postsynaptic neuron, causing channels to open, changing local membrane potential

• Synapses are excitatory or inhibitory• EPSP: excitatory postsynaptic potential: increase potential• IPSP: inhibitory postsynaptic potential: causes potential

decrease

Dendrites, soma, axon, and synapse

Spike generation

• When a neuron receives strong inputs that cross a neuron-specific threshold, a spike is emitted

• A hybrid analog-digital computing device: digital 0/1 inputs are converted to analog changes in the local membrane potential, followed by summation of these changes at the soma, and a spike if the summation of changes exceeds threshold

• Threshold model of neuron: artificial neural networks

Adapting the connections: synaptic plasticity

• Neurons change strength of connections through synaptic plasticity

• LTP: long term potentiation

• LTD: long term depression

• Both involve changes to a synapse that last for hours or even days

LTP, LTD and STDP

• LTP involves increase in synaptic connection between two neurons caused by correlated firing of the two neurons

• Also, called Hebbian learning or plasticity: if a neuron A is consistently causing another neuron B to fire, the strength of connection from A to B should be increased. Found in Hippocampus and neocortex

• LTD: involves decrease in strength of a synaptic connection caused, by uncorrelated firing between two neurons: found in hippocampus, neocortex and other areas

• STDP: timing of pre- and postsynaptic spikes can determine whether the change in synaptic strength is positive or negative

Synaptic plasticity

Which brain area for BCI ?

• Central and Peripheral Nervous system (CNS, PNS)• PNS: somatic nervous system-neurons connected to skeletal muscles, skin, and

sense organs. Automatic nervous system: visceral functions such as pumping of the heart, breathing

• CNS: brain and spinal cord• spinal cord conveys information to and from the brain• Brain stem: medulla, pons and midbrain• Pons: regulatory functions - breathing, muscle tone, blood pressure, sleep and

arousal• Tectum in midbrain: composed of inferior and superior colliculus controls eye

movement, visual and auditory reflexes• Tegmentum: has reticular formation and other nuclei: modulates muscle reflexes,

pain perception, breathing • Cerebellum: coordination of movements• Diencephalon: thalamus and hypothalamus- Thalamus is the relay station conveys

information from sensory organs to neocortex

Cerebrum

• Cerebral hemisphere consists of neocortex, basal ganglia, amygdalia, and hippocampus

• Basal ganglia: motor control• Amygdalia: regulation of emotion• Hippocampus: critical for memory and learning, spatial

cognition• Neocortex has 30 billion neurons arranged in 6 layers, each

making 10,000 synapses, yielding 300 trillion connections• Pyramidal neurons: arranged in columns perpendicular to

the cortical surface• Cortex is convoluted with fissures known as sulci and ridges

known as gyri

Lobe functions

• Occipital lobe: basic visual processing

• Parietal lobe: spatial reasoning and motion processing

• Temporal areas: visual and auditory recognition

• Frontal areas: planning and higher cognitive functions

• Cortex: hierarchically organized network of sensor and motor areas

Visual processing

• Information from retina reaches cortex via lateral geniculate nucleus (LGN in thalamus)

• Reaches V1 in primary visual cortex that contains neurons selective for primitive features such as moving bars and edges

• More complex processing involves V2, V4, and IT (inferotemporalcortex) along ‘ventral stream’, and MT, MST, and parietal cortex along ‘dorsal stream’

• Ventral stream processes color and form of objects involved in object and face recognition

• Dorsal stream involve din reasoning about spatial relations• Areas of cortex are similar in anatomical organization, suggesting

that it employs a prototypical algorithm for processing information, and specialization occurs through differences in the types of inputs received in each area

Review questions• What is a typical resting potential difference across the membrane of a cortical

neuron? Explain the biochemical mechanisms that allow the neuron to maintain this potential difference.

• 2. Describe the sequence of events that gives rise to an action potential. Start from a volley of action potentials arriving along the input axons to the neuron and trace the biochemical and electrical consequences leading to an output action potential.

• 3. What are four prominent types of synaptic plasticity observed in the brain?• Explain how they serve to modify the effect of a presynaptic spike on the postsynaptic

neuron.• 4. What are the major components of the CNS and the PNS?• 5. Describe the functions that have been ascribed to the brain stem and the

cerebellum.• 6. What are the major components and functions of the diencephalon?• 7. What are some of the functions thought to be carried out by the basal ganglia and

the hippocampus?• 8. Approximately how many cells does the neocortex contain? How many synapses on

average does a cortical neuron have with other neurons?• 9. What are the major areas of the neocortex and what are some of their functions?• 10. Is the cortex hierarchically organized? Discuss evidence for and against this

hypothesis.