Upload
others
View
11
Download
0
Embed Size (px)
Citation preview
Towards a Wireless EEG System for Ambulatory Mental Health Applications
by
Gregory Jackson
A thesis submitted in conformity with the requirements for the degree of Master of Health Science in Clinical Biomedical Engineering
Institute of Biomaterials & Biomedical Engineering University of Toronto
© Copyright by Gregory Jackson 2013
ii
Towards a Wireless EEG System for Ambulatory Mental Health
Applications
Gregory Jackson
Master of Health Science in Clinical Biomedical Engineering
Institute of Biomaterials & Biomedical Engineering
University of Toronto
2013
Abstract
The purpose of this thesis was to create and test a proof-of-concept novel ambulatory EEG
system to monitor emotional valence in real-time. A qualitative comparison of a wireless EEG
acquisition system by the imec group to a gold standard laboratory EEG system was successfully
performed. A new wireless transmission system was created using the Texas Instruments’
ADS1299 EEG front-end chip and quantitatively compared to the gold standard system. This
system and the ADS1299 performance demonstration kit were used to evaluate several equations
for emotional valence classification. Three of these equations were able to correctly classify
emotional valence on a positive-neutral vs. negative basis over 90% of the time on the
performance demonstration kit and over 90% of the time on the wireless system. The wireless
data was acquired and saved on a novel BlackBerry application that also allowed emotional self-
assessment by the user during testing.
iii
Acknowledgments
I would like to thank my supervisor, Dr. Joseph Cafazzo, and the other members of my
committee Dr. Paul Ritvo and Dr. Jeff Daskalakis for their guidance and support during this
project.
I would also like to thank my main collaborators on the system development side of this project:
Kevin Tallevi for his hardware design and tireless work in getting the wireless system up and
running despite repeated and unexpected difficulties; John Li for his work on the BlackBerry 10
software and significant work in troubleshooting all aspects of the final system; Kevin Armour
for his design work on the headset prototype; and Nathaniel Hamming for his help in
troubleshooting unexpected issues while trying to get the final system working.
Additionally, I would like to thank Natasha Radhu and Yinming Sun at the CAMH for their help
with scheduling and assisting with EEG testing, all volunteers who gave their time to be tested
on all systems in this thesis, and the imec group for their support in testing their wireless EEG
system and for providing the background for the emotional valence work.
Finally, I would like to thank my fiancée Ruth for her infinite patience and support during the
ups and downs of this project. I would also like to thank all of my family and friends for their
support and encouragement through this whole educational journey, and specifically thank
Kenneth Dodd for his editing expertise in getting this thesis to print.
BlackBerry (Research in Motion) and Healthcare Support through Information Technology
Enhancements (hSITE) provided generous financial support to this project.
iv
Table of Contents
Acknowledgments.......................................................................................................................... iii
Table of Contents ........................................................................................................................... iv
List of Figures ............................................................................................................................... vii
List of Tables ................................................................................................................................. ix
List of Appendices ......................................................................................................................... xi
List of Abbreviations .................................................................................................................... xii
Chapter 1. Introduction ..............................................................................................................1
1.1 Research Rationale...............................................................................................................1
1.1.1 The Cost of Mental Illness in Canada ......................................................................1
1.1.2 Quantifying Mental Illness with EEG......................................................................4
1.1.3 Ambulatory Physiological Monitoring Systems ......................................................6
1.2 Problem Statement and Objectives ......................................................................................8
1.3 Scope ....................................................................................................................................9
1.4 Overview of Thesis ............................................................................................................10
Chapter 2. Relevant Literature.................................................................................................11
2.1 Wireless EEG and Capacitive EEG Electrodes .................................................................11
2.1.1 Development of Wireless EEG Systems................................................................11
2.1.2 Dry and Capacitive EEG Electrode Technology ...................................................15
2.2 Emotional Valence, Brain Asymmetry, and Mental Illness ..............................................19
2.2.1 A Psychological Perspective on Emotions ............................................................19
2.2.2 A Neurological Perspective on Emotional Valence...............................................21
2.2.3 Emotional Valence and Asymmetry Related to Mental Illnesses ..........................27
v
Chapter 3. Technical Background ...........................................................................................32
3.1 EEG Background and Specifications .................................................................................32
3.1.1 Digital EEG Requirements ....................................................................................32
3.1.2 Standard Electrode Placement and Electrode Montages .......................................33
3.1.3 EEG Signal Processing ..........................................................................................35
3.1.4 Expanding on Clinical EEG Bands ........................................................................38
3.2 Statistical Analysis and Correlation Methods ....................................................................39
Chapter 4. Comparitive Evaluation of an MBAN EEG Platform vs. Clinical Gold
Standard [23] .............................................................................................................................41
4.1 Introduction ........................................................................................................................41
4.2 Participants .........................................................................................................................42
4.3 Equipment ..........................................................................................................................42
4.4 Testing Method ..................................................................................................................43
4.5 Analysis Method ................................................................................................................44
4.6 Results ................................................................................................................................45
4.7 Conclusion .........................................................................................................................46
Chapter 5. System Development .............................................................................................47
5.1 Hardware Development .....................................................................................................47
5.1.1 Acquisition System ................................................................................................47
5.1.2 Capacitive Electrodes.............................................................................................49
5.1.3 Wireless Cap and Electrode Placement .................................................................50
5.2 Software and Signal Processing Development ..................................................................51
Chapter 6. Testing and Validation ...........................................................................................54
6.1 Participants .........................................................................................................................54
6.2 Testing ADS1299 Demonstration Kit ................................................................................54
6.3 Testing Electrodes ..............................................................................................................56
vi
6.4 Emotional Valence Testing Protocol .................................................................................57
6.5 Emotional Valence Data Analysis .....................................................................................59
6.6 Testing Full Mobile System ...............................................................................................60
Chapter 7. Results ....................................................................................................................61
7.1 Validation of Performance Demonstration Kit ..................................................................61
7.2 Emotional Valence Testing ................................................................................................65
7.3 Proof-of-Concept Data Validation Results ........................................................................68
7.4 Proof-of-Concept Emotional Valence Results ...................................................................72
Chapter 8. Discussion and Conclusions ..................................................................................73
8.1 Discussion of Results .........................................................................................................73
8.1.1 Validation of ADS1299 Performance Demonstration Kit .....................................73
8.1.2 Comparison of Emotional Valence Calculation Methods .....................................74
8.1.3 Proof-of-Concept System Testing..........................................................................75
8.2 Limitations and Difficulties Encountered ..........................................................................77
8.3 Future Directions ...............................................................................................................79
8.4 Conclusions ........................................................................................................................81
8.5 Summary of Contributions .................................................................................................82
References ......................................................................................................................................83
Appendices .....................................................................................................................................97
vii
List of Figures
Figure 1 - Estimated Wage-Based Productivity Impact for Mental Illnesses in Canada (left);
Estimated Total Cost of Mental Illnesses in Canada (right) [4] ..................................................... 2
Figure 2 - Estimated Reduction in Total Direct Costs for Mental Illnesses in Annual Future
Value Terms, Four Scenarios [4] .................................................................................................... 3
Figure 3 - Examples and Definitions of EEG Bands ...................................................................... 4
Figure 4 - Medical Body Area Networks [15] ................................................................................ 6
Figure 5 - Epidermal Electronics [18] ............................................................................................ 7
Figure 6 - Diagram of Proposed System ......................................................................................... 9
Figure 7 - Annotated Image of Batteryless 19 uW Energy Harvesting BSN [39] ........................ 14
Figure 8 - a) Conventional gel electrode functionality b) Dry electrode with microtips [51] ...... 16
Figure 9 - 12-Point Circumplex Model of Affect [67] ................................................................. 20
Figure 10 - Self-Assessment Manikin: Valence, Arousal, Dominance [70]................................. 21
Figure 11 - Sectors of the prefrontal cortex: lateral view (left), ventral view (right) [74] ........... 22
Figure 12 - Statistical maps showing blood oxygen signal intensity associated with (left) valence
ratings, and (right) arousal ratings [77]......................................................................................... 24
Figure 13 - Standard 10-20 Electrode Positioning with ACNS Modification [103] .................... 33
Figure 14 - Equipment setup (left to right) a) Imec 8-channel EEG ASIC; b) NeuroScan
SynAmps connection; c) NeuroScan QuikCap 64-channel EEG cap; d) Custom-made 80-pin
connection board ........................................................................................................................... 43
Figure 15 - Electrodes from 10-20 system used for testing .......................................................... 43
Figure 16 - 3D Representation of Wireless EEG Acquisition System ......................................... 47
viii
Figure 17 - Functional Block Diagram of ADS1299 [122] .......................................................... 48
Figure 18 - Design of Active Capacitive Electrode ...................................................................... 49
Figure 19 - Electrode Placement for EEG Cap ............................................................................. 50
Figure 20 - Images of Headset Prototype ..................................................................................... 51
Figure 21 - Screenshots of EEG App on BlackBerry 10 - Bluetooth Device Selection (left);
Graph of time signals (centre); Emotional Self-Assessment (right) ............................................. 53
Figure 22 - Self-Assessment Manikin (Pleasure) [70] .................................................................. 57
Figure 23 - Emotional Valence Experimental Protocol: ............................................................... 57
Figure 24 - Emotional Valence vs. Emotional Arousal Scatterplot of IAPS Images ................... 58
ix
List of Tables
Table 1 - Estimated 12-month prevalence of any mental illnesses in Canada [4] .......................... 2
Table 2 - Relative Risk of Adult Mental Illnesses Given Prior Adolescent Illness [4] .................. 3
Table 3 - Information on EEG Bands [109].................................................................................. 38
Table 4 - Significance of Different Correlation Coefficients [114] .............................................. 39
Table 5 - Correlation for 2-second windows: Pearson’s Correlation Coefficient and Confidence
Valuesa .......................................................................................................................................... 45
Table 6 - Correlation for 10-second windows: Pearson’s Correlation Coefficient and Confidence
Values b
......................................................................................................................................... 45
Table 7 - Number of Segments Used for Correlation Analysis .................................................... 56
Table 8 - Emotional Valence Equations Tested ............................................................................ 59
Table 9 - Pearson's Correlation (r, p) for Amplitude and Power Spectra of 4-second and 20-
second windows with Averaged and Scaled ADS1299 Data ....................................................... 61
Table 10 - Pearson's Correlation (r, p) Repeated with Raw ADS1299 Data ................................ 61
Table 11 - Lin's Covariance Correlation Coefficient (Rc) for Amplitude and Power Spectra of 4-
second and 20-second windows with Averaged and Scaled ADS1299 Data ............................... 62
Table 12 - Lin's Covariance Correlation Coefficient Repeated with Raw ADS1299 Data .......... 62
Table 13 - Intraclass Correlation Coefficients (ICC) for Amplitude and Power Spectra of 4-
second and 20-second windows with Averaged and Scaled ADS1299 Data ............................... 63
Table 14 - Intraclass Correlation Coefficients repeated with Raw ADS1299 Data ..................... 63
Table 15 - Emotional Valence Testing on ADS1299 Demonstration Kit .................................... 65
Table 16 - Testing Emotional Valence Equations on DEAP Dataset ........................................... 66
x
Table 17 - Pearson's Correlation for Subject 1 on Proof of Concept System (Averaged and Scaled
Data) – Effective Sampling Rate Calculated as 234 Hz ............................................................... 68
Table 18 - Pearson's Correlation for Subject 2 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 234 Hz ................................................................ 68
Table 19 - Pearson's Correlation for Subject 3 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 240 Hz ................................................................ 68
Table 20 - Pearson's Correlation for Subject 4 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 240 Hz ................................................................ 69
Table 21 - Pearson's Correlation for Subject 5 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 240 Hz ................................................................ 69
Table 22 - Lin's Correlation for Subject 1 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 234 Hz ................................................................ 69
Table 23 - Lin's Correlation for Subject 2 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 234 Hz ................................................................ 70
Table 24 - Lin's Correlation for Subject 3 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 240 Hz ................................................................ 70
Table 25 - Lin's Correlation for Subject 4 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 240 Hz ................................................................ 70
Table 26 - Lin's Correlation for Subject 5 on Proof of Concept System (Averaged and Scaled
Data) - Effective Sampling Rate Calculated as 240 Hz ................................................................ 71
Table 27 - Emotional Valence Classification for Proof-of-Concept System Test ........................ 72
xi
List of Appendices
Appendix 1 - Register Settings for ADS1299............................................................................... 97
Appendix 2 - IAPS Images Used in Emotional Valence Study .................................................... 98
Appendix 3 - Screening Form for EEG Study Participants at CAMH ....................................... 100
xii
List of Abbreviations
ACNS American Clinical Neurophysiology Society
ADC Analog-Digital Converter
ADHD Attention Deficit Hyperactivity Disorder
AgCl Silver chloride (Silver-silver chloride electrodes)
ASIC Application Specific Integrated Circuit
ATM Acquisition and Transmission Module
BCI Brain-Computer Interface
BTLE Bluetooth Low Energy (Bluetooth 4.0)
bps Bits per second (also kbps, Mbps, etc.)
CAMH Centre for Addiction and Mental Health
CMRR Common-Mode Rejection Ratio
DFT Discrete Fourier Transform
ECG Electrocardiogram / Electrocardiography
EEG Electroencephalogram / Electroencephalography
EOG Electrooculogram / Electrooculography
ERG Electroretinography
FCC Federal Communications Commission (United States)
FFT Fast Fourier Transform
fMRI Functional Magnetic Resonance Imaging
GB Gigabytes (230
bytes or 2,000,000,000 bytes; also MB, kB, etc.)
Hz Hertz, unit of frequency equal to 1/s (also kHz, MHz, GHz, etc.)
IAPS International Affective Picture System
ICC Intraclass Correlation Coefficient
IFFT Inverse fast Fourier transform
KNN K-Nearest Neighbour
mAh Milliamp hours (measure of battery power)
MBAN Medical Body Area Network
MDD Major Depressive Disorder
MEG Magnetoencephalography
MHCC Mental Health Commission of Canada
xiii
NIH National Institute of Health (United States)
OCD Obsessive Compulsive Disorder
ODD Oppositional Defiant Disorder
OSET International Organisation of Societies for Electrophysiological Technology
pF Picofarad (capacitance; also F, etc.)
PET Positron Emission Tomography
PTSD Post-Traumatic Stress Disorder
RF Radio Frequency
SAM Self-Assessment Manikin
SoC System on Chip
SSVEP Steady-state visually evoked potentials
SUD Substance Abuse Disorder
SVM Support Vector Machines
USB Universal Serial Bus
V Volts (also mV, kV)
µW Microwatts (also mW, W)
1
Chapter 1. Introduction
1.1 Research Rationale
1.1.1 The Cost of Mental Illness in Canada
In 2002, Health Canada produced A Report on Mental Illnesses in Canada [1], which included
detailed statistics on mental illnesses in Canada. Health Canada investigated mood disorders,
schizophrenia, anxiety disorders, personality disorders, eating disorders, and suicidal behaviour
in particular. Their report also stated that over 10% of all hospitalizations in general hospitals
for patients between 15 and 44 are due to one of seven mental illnesses, and 3.8% of
hospitalizations across all ages are due to mental illnesses [1]. The Government of Canada
followed up on this report in 2006 with a report titled The Human Face of Mental Health and
Mental Illness in Canada [2], which referenced Health Canada’s report in producing further
statistics.
Following the 2006 research, the MHCC was created by Health Canada to “create a mental
health strategy, work to reduce stigma, advance knowledge exchange in mental health, and to
help people who are homeless and living with mental health problems” [3]. The MHCC
commissioned a study in 2010 to fill a gap in information about people living with mental
illnesses and the associated costs of these illnesses today [4], which was published in 2013. The
study showed that the economic cost to Canada of mental health problems and illnesses is at least
$50 billion per year, or 2.8% of Canada’s 2011 gross domestic product, and that over the next 30
years the total economic cost will add up to more than $2.5 trillion. The MHCC also determined
that the cost to business was at least $6 billion in lost productivity from absenteeism,
presenteeism, and turnover in 2011, and that the cumulative cost over the next 30 years may add
up to nearly $200 billion. The MHCC explain that over the next 30 years the total economic cost
of mental illnesses in Canada will add up to more than $2.5 trillion.
In figure 1, shown below from the MHCC report, the left side shows the estimated annual future
value and cumulative present value of wage-based productivity loss due to mental illnesses. The
right side shows the cumulative present value and annual future value of the total cost of mental
illnesses in Canada.
2
Figure 1 - Estimated Wage-Based Productivity Impact for Mental Illnesses in Canada
(left); Estimated Total Cost of Mental Illnesses in Canada (right) [4]
The report states that, in any given year, one in five people in Canada experience a mental health
problem or illness, including more than 28% of people aged 20-29. By the age of 40, roughly
half of people in Canada will have had or have a mental illness. The impact of mental illnesses
is greatest in workplaces and working aged people, as 21.4% of the working population in
Canada currently experiences mental health problems and illnesses, which account for
approximately 30% of short- and long-term disability claims.
The MHCC produced updated numbers of 12-month prevalence of mental illnesses including
mood and anxiety disorders, schizophrenia, SUD, ADHD, oppositional defiant disorder ODD,
conduct disorders, and dementia, which are shown with future estimates in the table below with
no interventional changes.
Table 1 - Estimated 12-month prevalence of any mental illnesses in Canada [4]
Total (%) 2011 2021 2031 2041
Males 3,178,446 (18.7%) 3,415,276 (18.3%) 3,736,764 (18.6%) 4,044,688 (18.9%)
Females 3,619,181 (20.9%) 3,994,881 (21.0%) 4,448,014 (21.6%) 4,866,402 (22.2%)
Total 6,797,627 (19.8%) 7,410,157 (19.7%) 8,184,778 (20.1%) 8,911,090 (20.5%)
Furthermore, many mental illnesses start in the young (those from ages 9-19). Mood and anxiety
disorders affect 11.7% of people in Canada across all ages, and 12.1% of those aged 9-19, while
substance use disorders affect 5.9% of people across all ages, and 6.8% of those aged 9-19.
Adolescents who suffer from mental illnesses are at significantly increased risk of suffering more
mental illnesses as adults. Table 2, below, shows the specific relative risk ratios of adults
developing specific mental illnesses if they suffered specific disorders as adolescents.
3
Table 2 - Relative Risk of Adult Mental Illnesses Given Prior Adolescent Illness [4]
Male Female
Prior Adolescent Illness Anxiety Mood
Disorders
SUD Anxiety Mood
Disorders
SUD
ADHD 2.21 1.23 2.23 2.00 1.22 2.52
Anxiety - 2.43 1.04 - 2.33 1.05
Conduct Disorders 1.78 1.74 2.81 1.67 1.69 3.46
Mood Disorders 3.05 - 1.38 2.70 - 1.44
ODD 2.74 2.08 1.84 2.40 1.99 2.03
SUD 2.59 1.88 - 2.31 1.81 -
The MHCC estimates that by reducing the number of people experiencing a new mental illness
in a given year by 10%, there would be an economic saving of at least $4 billion per year after 10
years, and over $20 billion in 30 years. Providing early access to healthcare to keep people out
of hospitals or the criminal justice system can generate cost savings, and improving mental
health management in the workplace can significantly reduce losses in productivity. The
estimates are shown in figure 2 below, from the MHCC report.
Figure 2 - Estimated Reduction in Total Direct Costs for Mental Illnesses in Annual Future
Value Terms, Four Scenarios [4]
The Canadian Mental Health Association’s Framework for Support [5] discusses the three pillars
for recovery for people with mental illness, which are a Personal Resource Base, a Community
Resource Base, and a Knowledge Resource Base. The Personal Resource Base refers to the
person being in control of his or her own life, and the ability to understand the illness and
perform self-care behaviour greatly improves the lifestyle of someone living with mental illness.
Ambulatory monitoring can contribute significantly to this pillar.
4
1.1.2 Quantifying Mental Illness with EEG
EEG is the measurement of scalp electrical activity generated by neuronal activity in the brain.
In particular, the electrical activities of large, synchronously firing groups of neurons are
measured with electrodes placed on the scalp [6]. The activity is measured in real time and is
converted to the frequency domain for clinical applications. The frequency data is
conventionally divided into five bands as specified in a set of example EEG signals in figure 3,
below. There is occasional divergence on the precise EEG band definitions, but these values are
commonly used in major publications including Brain [7] and Journal of Neurophysiology [8].
Figure 3 - Examples and Definitions of EEG Bands
Research has demonstrated that each frequency band is related to different levels and types of
arousal. In particular, alpha waves are seen most prominently during early sleep stages, low
arousal, and when the subject’s eyes are closed. Beta waves are most prominent during resting
wakefulness [9]. In healthy subjects, delta and theta rhythms are greatest during deep sleep, as
they are normal signs of deactivated brain networks. When they are prominent in waking states,
they are considered abnormal. They can be caused by structural cortical lesions including stroke,
tumors and scars, and concentrations of slow wave activities (delta and theta, particularly) can be
found in individuals with psychiatric disorders who do not have obvious structural brain damage
[10].
5
In schizophrenia in particular, a study found that resting EEG readings can contain abnormalities
including increased power in low frequency waves (delta and theta) and diminished alpha-band
power. These findings may indicate the presence of thalamic and frontal lobe dysfunction which
appears to be unique to schizophrenia [11]. Another study examining the distribution of slow
wave activity in a group of healthy subjects, patients with schizophrenic or schizoaffective
diagnoses and patients with affective or neurotic/reactive diagnoses using resting MEG (which
contains similar features to EEG) found significantly more intense slow wave activity,
particularly in frontal and central areas. By comparison, affective disorder patients showed
fewer slow wave generators in frontal and central regions compared to both healthy subjects and
schizophrenia patients. The regions of abnormal slow wave activity corresponded to gray matter
loss in schizophrenic patients, suggesting that this activity may be used as a measure of altered
neuronal network architecture [12].
Another recent study evaluated whether the abnormal frequency composition of the resting state
EEG in schizophrenia and bipolar disorder showed similarities to first-degree biological relatives
of patients. Schizophrenia patients and their relatives showed increased beta frequency activity,
which suggests that disturbances in resting state brain activity may be specific to genetic liability
for schizophrenia. This similarity was not, however, seen in bipolar patients or their relatives.
Furthermore, schizophrenia patients had increased low-frequency activity, as mentioned
previously, which was not seen in bipolar patients or either group of relatives. The study
determined that excessive high-frequency EEG activity in frontal brain regions may reflect
genetic vulnerability to schizophrenia, while low-frequency abnormalities are more related to
disorder-specific pathophysiology [13]. Furthermore, patients with schizophrenia were found to
have significant deficits of cortical inhibition of gamma waves in the dorsolateral prefrontal
cortex compared to healthy subjects and patients with bipolar disorder, but no significant deficits
were found in the motor cortex. This finding suggests that there may be an important frontal
neurophysiological deficit that contributes to symptoms of schizophrenia [7].
Quantitative EEG analysis of patients with OCD revealed greater slow-wave activity and less
alpha activity in left frontotemporal locations compared to control subjects. Furthermore, the
dysfunction was found to be greater in female patients, and correlated positively with the
severity of the OCD. There was also greater dysfunction found in patients who responded to
OCD treatment [14].
6
1.1.3 Ambulatory Physiological Monitoring Systems
The rapid progression of computing technology and wireless handheld devices in particular, has
allowed for the development of small sensors to monitor illnesses and chronic conditions.
MBANs, which have recently been approved by the United States’ FCC for protected wireless
spectrum, are networks of physiological sensors worn on or implanted in the body. These
sensors can monitor different vital signs, such as temperature, blood pressure, and glucose levels,
and transit the information to a central node such as a smartphone [15]. Shown in the figure
below is an example of an MBAN with sensors to monitor glucose, toxins, blood pressure, ECG,
EEG, hearing, vision, positioning, and more.
Figure 4 - Medical Body Area Networks [15]
A transition document released by the FCC [16] discusses some demonstrations of MBAN
technology, including Fetal Telemetry to noninvasively monitor fetal health while allowing a
mother to move freely; LifeLine Home Care Pendants, which collect health information for
elderly patients and patients with chronic diseases, allowing them to live independently; and
Predictive and Early Warning Systems to provide continuous monitoring for the prevention of
sudden and acute deterioration of patients’ conditions. The FCC estimates that prevention of
unplanned transfers of patients could save up to $1.5 million USD per month in healthcare costs.
Furthermore, disposable sensors could save an estimated $2,000-$12,000 per patient, or over $11
billion USD in the U.S. Remote monitoring of patients with congestive heart failure alone could
save over $10 billion USD per year in the U.S. The FCC also found that patients accessing their
7
health data on mobile phones increased by 125 percent from 2010 to 2012, and that mHealth
(mobile health) could be a $2-$6 billion USD industry by 2013. Finally, and most significantly,
industry estimates show that remote monitoring of four chronic conditions could save just under
$200 billion USD from 2008 to 2033 [17]. These 4 conditions, and their expected savings, are:
Congestive Heart Failure $102.5 billion USD
Diabetes $54.4 billion USD
Chronic Obstructive Pulmonary Disease $24.1 billion USD
Chronic Skin Ulcers $16.0 billion USD
Sensors are also shrinking to the point that there are now sensors that have similar thicknesses,
elastic moduli, bending stiffness and mass densities to skin [18]. The epidermal electronics
created by Kim et al. incorporate electrophysiological, temperature, and strain sensors, as well as
transistors, light-emitting diodes, photodetectors, radio frequency inductors, capacitors,
oscillators, and rectifying diodes. Solar cells and wireless coils provide the power supply. The
sensors measure electrical activity produced by the heart, brain, and skeletal muscles, and are
shown in the figure below.
Figure 5 - Epidermal Electronics [18]
Monitoring EEG with MBANs is relatively new, as the earliest literature result was published in
2004 by Berka et al. [19], in a study that presents an integrated hardware and software solution
for acquisition and real-time analysis of EEG to monitor indices of alertness, cognition, and
memory. Berka et al. use a six-channel EEG headset with a sampling rate of 256 Hz and find it
to be a robust and reliable method for monitoring alertness and cognitive workload. Berka et al.
also discuss future developments including more complex monitoring systems that connect to a
wider range of receivers.
8
1.2 Problem Statement and Objectives
The purpose of this thesis is to create and test a proof-of-concept novel ambulatory EEG system
to monitor emotional valence in real-time. The system would take the concepts put forward by
Brown et al. in their paper on wireless emotional valence [20] and attempt to reproduce the
results in a more fully ambulatory system, as their work required the wireless receiver to be
attached to a computer. To assess the performance of emotional valence monitoring, it is
important to verify signal quality in the acquisition platform, and to have the user complete an
emotionally affective task and calculate the response.
From a clinical perspective, this system would be a stepping stone to a full MBAN system used
to monitor patients with diagnosed mental illnesses and improve their daily lives by providing
them with a means for self-care at home and at work. Presently, there is a shortage of
ambulatory monitoring for people with mental illnesses. In order to be monitored and receive
care, they need to be in a hospital or clinical setting. Providing a means of detecting adverse
events may allow a patient to perform self-care and self-regulating behaviour as well as notifying
their responsible clinician. As with other conditions such as heart disease or diabetes,
preventative care can help reduce the economic cost to the healthcare system and the personal
cost to people living with illnesses. It can also provide them with a means of continuing with
their normal daily lives.
In order to measure emotional response in a repeatable way, the presentation of affective stimuli
is often used. In particular, one of the most common systems is the IAPS [21], which is a large
series of images that have been rated by a large number of users on their emotional valence and
emotional arousal score and an average score is provided. These images can be used to provide
an emotional stimulus with an expected response to the user. The user can prove his or her
response to the stimulus using a feedback method that can be matched to the appropriate window
of measured EEG data.
The objective of this thesis is to determine whether emotional valence can be monitored with
acceptable accuracy in real-time using an ambulatory EEG system connected to a smartphone
while allowing the user to annotate emotional events.
9
1.3 Scope
It was decided that the scope of this thesis would focus on the creation of an ambulatory EEG
system that could transmit signal to a smartphone for data storage and analysis and that could
monitor emotional valence in real-time. The system diagram shown in figure 6 summarizes the
components and basic operational principles of the ambulatory EEG system. The headband
contains four EEG measurement electrodes as well as bias and reference electrodes. These
electrodes are connected to an acquisition board using an analog-to-digital conversion chip that
transmits the digitized EEG signal by BTLE to a smartphone. The smartphone stores the EEG
signal and calculates the emotional valence value while also allowing the user to annotate
emotional events to compare self-assessments to measured values in post-processing.
+
-
+
-
R
1
2
B
+
-
+
-
ADC
AVDD/2
CONTROL
ADS1299
Instrumentation AmplifierG=100
Instrumentation AmplifierG=100
IN A333
IN A333
Direct Connection
Bluetooth Low Energy Connection
Provide Emotional Valence feedback to User
Annotate emotional events
Figure 6 - Diagram of Proposed System
For the hardware portion of the project, after testing several wireless EEG acquisition systems, it
was decided that a custom system would be created as most available wireless systems used
wireless protocols that were not directly compatible with smartphones. As part of the testing, a
complete qualitative evaluation of an ultra-low-power wireless EEG acquisition system by the
imec group [22] was performed [23] with volunteers who passed a screen process that ensured
their eligibility as healthy control subjects. This testing protocol was also used to validate the
analog-to-digital front-end chip used in the custom-made EEG acquisition system. The system
10
uses Bluetooth Low Energy communication which is compatible with a number of cutting edge
smartphones. Furthermore, it was decided that dry electrodes would be preferred as they are
easier to apply and are less untidy. While research was done into capacitive electrodes and
prototypes were created, the technology available in the budget and the timeline for this thesis
were not able to provide a sensitive enough response to adequately measure EEG signals. This
situation meant that conventional gel electrodes had to be used.
On the software side, a basic signal processing application for BlackBerry 10 was created in
order to take the EEG signal in its raw time-domain form and convert it to frequency domain in
order to measure the emotional valence in short time windows. The application also allowed the
user to annotate events with an emotional valence rating to be read in post-processing.
Finally, the whole system and emotional valence measurement were tested using more volunteer
test subjects. This testing involved the acquisition of baseline readings followed by the
presenting of IAPS images with the subject rating each image after its presentation. The
emotional valence was measured in real-time and verified afterwards and compared to the
subjects’ self-assessment ratings.
1.4 Overview of Thesis
In Chapter 2, an overview of literature is presented on the subjects of wireless EEG technology
and emotional valence as a monitor of mental illness. Chapter 3 provides technical background
on EEG technology, signal processing for EEG data, and statistical analysis and correlation
methods. Chapter 4 is a comparative evaluation of a wireless EEG system to a gold standard
laboratory system, which was published as a conference proceeding. Chapter 5 describes system
development, including hardware development as well as software and signal processing. The
hardware development was led by technologist Kevin Tallevi. The software development was
led by programmer John Li. My responsibilities were to provide EEG-specific hardware
requirements, signal processing code, and to test the system and its components. Chapter 6
describes the testing and validation of the individual components of ambulatory EEG system,
including the emotional valence score calculations, as well as the full ambulatory system.
Chapter 7 presents the results of testing and validation. Finally, Chapter 8 contains discussion,
summary of contributions of this work, conclusions, difficulties encountered, and future
directions of this work.
11
Chapter 2. Relevant Literature
2.1 Wireless EEG and Capacitive EEG Electrodes
2.1.1 Development of Wireless EEG Systems
Electroencephalography of humans was first performed by German psychiatrist Hans Berger in
1924 [24], who published some 20 scientific papers on EEG afterward. In recent years, digital
EEG recording has become the most popular method, leading to guidelines for digital EEG being
created by the OSET [25]. Guidelines and ideal settings for EEG are discussed further in section
3.1 This section also includes recommendations on analog to digital conversion, reference and
channel placements, analysis, and advantages and cautions of digital EEG.
The newest evolution of EEG has been the creation of wireless acquisition systems, which allow
more freedom of movement to the subject and provide the opportunity for ambulatory
monitoring. The imec group from the Netherlands has produced significant research into ultra-
low-power wireless EEG ASIC. In 2006 [26], the imec group presented a custom-built 300 µW
eight-channel front-end ASIC to implement a portable EEG acquisition system. Each channel of
the ASIC contained an instrumentation amplifier, spike filter, and variable gain stage with very
low noise and very low power consumption. The system was capable of operating more than
seven months from two AA batteries. In 2008 [27], the system was refined to 200 µW power
consumption. Calibration and Electrode Impedance Measurement Modes were added to the
ASIC to increase the ease of use of the system. The ASIC had a wireless radio added to it for
signal transmission. The signal quality of the EEG is evaluated in Chapter 4 of this thesis by
quantitatively comparing it to a clinical gold standard system [23].
In the literature published since 2010, much of the research into wireless EEG systems is often
focused on either brain-computer interface devices or on patient monitoring. Brain-computer
interfaces are becoming more sophisticated in the research field. While commercial systems
often focus on gaming or control of computers, research systems are more concerned with
offering communication and control to motor-disabled subjects. One study by Filipe et al. in
2011 presents a custom ASIC supporting RF transmission of 32 channels of EEG [28]. This
system records EEG at 1 kHz sampling rate with 12-bit resolution and transmits using the
Medical Implant Communication Service at 402-405 MHz.
12
A system implemented by Kim et al. in 2012 [29] included multi-channel EEG and head motion
signal acquisition by adding a Bluetooth communication module and head motion tracker to a
commercial EEG BCI headset. The system was tested indoors and outdoors to prove the validity
of a portable multi-channel signal acquisition system.
Another wireless EEG based BCI system created in 2013 by Lin and Huang [30] to control
electric wheelchairs through a Bluetooth interface for paralyzed patients. This system used two
EEG channels and a signal processing unit to extract EEG and winking signals to transform them
into control signals that drive the electric wheelchairs.
From a monitoring standpoint, in 2010 Verma et al. created a micro-power EEG acquisition SoC
to focus on seizure detection [31]. This chip corresponds to one EEG channel, but up to 18
channels can be worn by a single patient to detect seizures as part of a chronic treatment system.
This system focuses on amplification, filtering, feature extraction, and minimizing power
consumption.
From an MBAN perspective, Chen et al. [32] published research in 2010 on a flexible wireless
body area network node platform using ZigBee wireless technology for EEG monitoring. Their
EEG conditioning system includes pre-amplification and filtering that is able to pass signals with
minimal attenuation from 2-50 Hz with notch filtering for electrical noise. This system connects
to a centralized MBAN node, which sends signals a ZigBee wireless internet gateway that can
connect to a database as well as clinicians and relevant parties.
In 2010, Dilmaghani et al. presented a wireless multi-channel EEG recording device [33]. This
system includes analog filtering and gain amplifiers to filter noise and amplify the EEG signals.
The microcontroller digitizes the analog signal and digital filtering removes power-line
interference. The system transmits data with Bluetooth to an end device where the data can be
post-processed. The system was tested with sample voltage rather than live subjects.
Chen and Wang from the University of British Columbia created a wearable, wireless EEG
acquisition and recording system in 2011 [34]. Their system was broken down into a data
acquisition circuit and a data transmission and receiving unit. The acquisition circuit includes a
voltage follower, instrumentation amplifier, DC restorator, right-leg drive or bias circuit, band
pass filter, and power supply on a PCB. The data transmission and receiving unit consists of an
13
ADC and a ZigBee wireless module. Preliminary testing showed that high quality physiological
signals could be acquired while the subject performed daily-life tasks.
In 2012, Thie et al. created a system with four bipolar channels for biomedical signal acquisition
usable for EEG, ECG, and ERG [35]. Their focus was on consistent data transmission for
reliable recording of visually evoked potentials. Data was continuously streamed at 915 MHz
and a constant delay of 20 ms was added to remove distortion and minimize error rates.
Wang et al. at the University of California, San Diego and National Chiao-Tung University in
Taiwan created a cell phone based drowsiness monitoring system in 2012 [36]. This system uses
non-prep dry EEG sensors and Bluetooth transmission protocol with an Android smartphone.
The headset has four channels with a microprocessor, pre-amplifier and battery charger, 24-bit
ADC, Bluetooth module, and dry spring-loaded EEG sensors. The Bluetooth module transmits
data to an Android smartphone with no loss in signal processing performance. The Drowsiness
Monitoring and Management system continuously observes EEG dynamics and delivers arousing
feedback to users if they are experiencing drowsiness or a cognitive lapse. The system is then
able to assess the effect of the feedback in near real-time.
In 2012, Boquete et al. presented an 8-channel system for capturing bioelectric signals using
ZigBee wireless transmission protocol that can be used for ECG, EEG, EOG, and more [37].
This system focused on an ATM and a PC host to process, store, and display the data sent by the
ATM. The PC host can also set the parameters of the ATM. The ATM contains an analog front
end, a microcontroller, and a ZigBee transceiver. The analog front end has eight differential
inputs connected to a multiplexer which is then connected to an ADC chip. Boquete et al. were
able to sample 8 channels at 400 Hz each with 12 bit resolution and run the system for 68 hours
on a 6800 mAh battery.
Sawan et al. presented a wireless recording system for NIRS and scalp EEG for non-invasive
monitoring or intracerebral EEG for invasive monitoring in 2013 [38]. The system uses
Bluetooth to transmit at 2 kHz with 24-bit resolution. The system can run 8 EEG and 32 NIRS
channels simultaneously. The wireless signals were compared to a wired system and normalized
root-mean square deviation was under 2%. Sawan et al. also presented a wireless EEG recording
system for epilepsy.
14
In 2013, Zhang et al. presented an ultra-low power batteryless energy harvesting body sensor
node SoC capable of acquiring, processing, and transmitting ECG, EMG and EEG data using RF
transmission [39]. This node is fabricated on commercial 130 nm CMOS technology and is
designed so that multiple chips can be integrated to create a flexible and reconfigurable wireless
system with autonomous power management and operation from harvested power. The chip was
shown to perform ECG heart rate extraction and atrial fibrillation detection while consuming just
19 µW, allowing it to run only on harvested energy. The system is reconfigurable for EEG and
MEG applications and a four-channel front-end with ADC, filtering and data memory is
contained on a 2.5mm x 3.3mm board. An image of this chip is shown below, in figure 7.
Figure 7 - Annotated Image of Batteryless 19 uW Energy Harvesting BSN [39]
From a commercial perspective, there are several systems that have been created primarily for
BCI and game design applications. One of the most commonly used commercial systems in
research is the Emotiv Epoc, a 14-channel EEG headset that acquires and digitizes EEG signals
using felt covered sensors and a proprietary USB based wireless communication protocol. To
date there are over 30 papers published using the Emotiv system [40] and it is being used for BCI
and gaming design. The Emotiv Epoc system was qualitatively tested during this project, but
was ultimately not used due to the USB requirement in the wireless communication. The Epoc
samples at 2048 Hz initially and then downsamples the data to 128 Hz for output and BCI
applications at 14 bits. The system is able to resolve to 0.51µV at a bandwidth of 0.2-45 Hz.
NeuroSky is a company that has several one-channel EEG headsets, including the MindWave,
MindWave Mobile, and MindSet, as well as the ThinkGear ASIC Module for EEG acquisition.
Their products are used in mobile gaming and BCI applications and have had several academic
15
papers make primary use of them [41]. As they are only single-channel forehead systems, they
are not ideal for clinical applications; however, the ThinkGear ASIC can be used to acquire EEG
channels at 512 Hz as part of a full system.
Muse by InterAxon is a flexible four-sensor headband designed to work with applications for
brain health, fitness training, stress management, and more. The Muse headset samples at up to
600Hz. InterAxon is aiming to have the device commercially ready by late 2013 [42].
The X-Series EEG Headset Systems from Advanced Brain Monitoring come in 4-, 10-, and 24-
electrode configurations. The systems can be used for neurofeedback, BCI applications, ERP
analysis and more [43]. The X-Series headsets sample at 256 Hz with 16-bit resolution and have
at least six hours of battery life in the 10- and 24-electrode configurations. The data can be
transmitted via Bluetooth Class 2 or saved to an onboard SD card for longer battery life.
In a partnership between imec, Holst Centre and Panasonic, an eight-channel wireless EEG
headset was created using the ultra-low-power EEG monitoring chipset mentioned previously
[22]. This system is currently being made available to industrial research partners [44]. The
battery lasts up to 22 hours with 8 channels of EEG and impedance levels can be monitored.
The g.Nautilus from g.tec is a 32-, 16-, or 8-channel wireless EEG platform available with
g.tec’s own active or active dry electrodes and contactless charging using a 2.4GHz wireless
transmission band [45]. The system samples at 250 or 500 Hz with 24-bit resolution and can
record continuously for up to eight hours.
Enobio from Neuroelectrics is an 8- or 20-channel EEG system using Bluetooth communication
with MicroSD data storage capability [46]. It samples data at 500 Hz with 24-bit resolution and
very low noise. The battery can last 16 hours on Bluetooth or over 24 hours with SD storage.
2.1.2 Dry and Capacitive EEG Electrode Technology
In order for an EEG system to be fully ambulatory for non-clinical use, dry electrodes will be an
essential component. Conventionally, EEG electrodes are made of one of silver-silver chloride,
gold plated silver, tin, silver, sintered silver-silver chloride (AgCl), platinum, or stainless steel.
These electrodes are used with gel, which reduces the impedance so that a clean signal can be
detected from the scalp [47]. The NeuroScan Quik-Cap used for the comparative evaluation of
16
the imec system in Chapter 4 and the validation testing of the ADS1299 Performance
Demonstration Kit in section 6.2 has silver-silver chloride electrodes that provide the lowest
offset voltage, rate of drift, noise level, and have the best suitability for DC-coupled recording
and long time-constant AC-coupled recording.
Dry electrodes are less common than wet electrodes, but come in several different forms. An
early example of a dry EEG electrode was presented by Taheri et al. in 1994 [48] that used a 3
mm stainless steel disk as a sensing element with a nitride coating on the contact side. The
power spectra of the prototype electrode compared well to conventional gel electrodes.
A common format of dry electrode being used recently is silver-silver chloride resistive
electrodes with special contact posts [49]. The contact posts allow the electrode to make contact
with the scalp through hair, and the electrodes provide reasonably good performance compared
with conventional electrodes using gel and skin preparation. The electrodes are, however, not
very comfortable as they make contact with the skin. Another similar dry contact electrode is the
g.Sahara from g.tec. The electrodes are designed with long pins to make contact through the hair
to the skin, and provide good performance for SSVEPs despite having significantly higher
impedance than traditional gel electrodes [50].
Figure 8 - a) Conventional gel electrode functionality b) Dry electrode with microtips [51]
An example of how these electrodes work is shown in figure 8, above. On the left, a
conventional gel electrode is shown, where the gel allows the surface of the electrode to be
connected to the stratum corneum and the epidermal layer. On the right, a dry electrode has
small sharp microtips that penetrate to the epidermis to read the EEG potentials. These
electrodes have equal frequency response in conductivity and permittivity to traditional silver
chloride electrodes [51].
17
A similar electrode created by Salvo et al. in 2012 [52] contained 180 conical needles 250 µm
apart on a circular base. This electrode showed comparable results to conventional wet
electrodes for ECG and EEG applications. Similarly, Liao et al. in 2011 designed, fabricated and
validated a dry-contact EEG sensor [53] that contained 17 spring contact probes on each sensor.
Each probe included a probe head, plunger, spring, and barrel and was inserted into a flexible
substrate. Liao et al. found that the sensor was reliable in measuring EEG signals with no skin
preparation or gel when compared to conventional wet electrodes.
A study by Mihajlovic et al. on replacing conductive gel electrodes with dry and water based
electrodes [54] used SSVEPs to evaluate and compare the electrode performance. Mihajlovic et
al. found that the dry and water based electrodes had acceptable classification accuracy with
somewhat lower communication speed compared to the gel electrodes, and could be used in
brain-computer interface application if the lower communication speed was acceptable. A
methodological review of dry-contact and noncontact biopotential electrodes by Chi et al. [55] in
2010 compared the performance of silver-silver chloride wet electrodes to dry electrodes and
non-contact electrodes for EEG and ECG applications. Chi et al. found no clinical dry or
noncontact EEG devices on the market at the time, though there were commercial devices
focusing on entertainment, sleep, and marketing. They concluded that there was a need for
greater emphasis on materials, packaging, and signal processing and systems development.
A different form of dry electrode in a number of sources uses textile sensors or conductive
threads. In 2010, Löfhede et al. presented textile electrodes for use in EEG monitoring [56].
They tested three different types of textile electrodes. The first type was metallic silver thread
knitted in a circular knitting machine into a mesh fabric. The second type was woven from a
nylon substrate coated by pure silver, from Less EMF Inc. The third type was 15% nylon, 30%
conductive fibers, 20% Spandex, and 35% polypropylene from Textronics, Inc. Their results
showed that type I produced poor signal quality with both saline solution and gel. Electrode type
II produced good signal quality using gel (nearly equivalent to standard electrodes) but less well
with saline. Electrode type III showed equally good results with both gel and saline. Löfhede et
al. concluded that soft conductive textile materials can be used in EEG applications, and may be
particularly useful for long-term monitoring.
18
Another approach to gel-free electrodes that has been developed recently is capacitively coupled
active electrodes. These electrodes make use of an extremely high input impedance operational
amplifier (1013
Ω in one study [57]) with a very low input capacitance (1 pF in the same study)
that allows the electrode to acquire EEG signals through hair without a conductive contact to the
copper plate. These electrodes have the advantage of not needing scalp preparation and of not
causing discomfort with sharp probes. The electrodes do show a high susceptibility to motion
artifact, however, though more research is being done in terms of deformable electronics that can
fit more closely to the head.
Researchers at the University of California, San Diego produced significant developments in
capacitive electrodes. In 2007, these researchers presented a non-contact EEG/ECG sensor [58]
with on board electrode to capacitively couple to the skin. They produced the next generation of
the non-contact electrode in 2009 [59], followed by further improvements in 2010 [60]. In 2011,
they reported on an ultra-high impedance front-end for capacitive electrodes [61] that produced
stable frequency response to below 0.05 Hz with extremely low noise.
In 2012, with Chi et al., the researchers developed a mobile brain-computer interface system
using the ADS1298 analog-to-digital conversion chip from Texas Instruments to compare dry
electrodes and capacitive electrodes to a reference wet electrode [62]. Their dry electrodes used
spring-loaded pins to go through the subject’s hair and make contact with the scalp. The
noncontact electrode achieved extremely high input impedances through a custom integrated
circuit design. The system transmitted the data with Bluetooth to a cellular phone for processing.
They found that the dry electrode performed better than the non-contact electrode, but the non-
contact electrodes still had a mean correlation of 0.80 with the wet electrodes, and a similar
signal-to-noise ratio. They also found that while the noncontact electrode showed more signal
degradation and susceptibility to movement artifacts than the dry electrode or the wet electrode,
it could still be successfully utilized for controlled BCI applications.
19
2.2 Emotional Valence, Brain Asymmetry, and Mental Illness
2.2.1 A Psychological Perspective on Emotions
Emotions are often categorized in literature on the basis of valence and arousal, often placed on a
polar scale as by Russell in 1980 [63]. Russell suggested that the affective dimensions of
emotion are interrelated in a systematic fashion: that is, emotions have a positive and negative
dimension, and an activation dimension. In the simplest terms, the relationship can be
represented by a spatial model with the affective concepts on the following points of a circle:
pleasure (0o), excitement (45
o), arousal (90
o), distress (135
o), displeasure (180
o), depression
(225o), sleepiness (270
o), and relaxation (315
o). This model was updated by Russell and Barrett
in 1999 [64], where the Y-axis of the circle is labeled Activation – Deactivation and the X-axis is
Unpleasant – Pleasant, and four different emotions are in each quadrant. Russell and Barrett
mention that the pleasure-displeasure dimension is also called “valence, hedonic tone, utility,
good-bad mood, pleasure-pain, approach-avoidance, rewarding-punishing, appetitive-aversive,
positive-negative”.
In 2003, Russell discusses the psychological construction of emotion [65], where he calls core
affect the “neurophysiological state consciously accessible as the simplest raw (nonreflective)
feelings evident in moods and emotions.” Russell describes it as similar to what others call
activation, affect, mood, and most commonly feeling. The raw feeling can at any time be a blend
of two dimensions, pleasure-displease (or valence) and arousal.
Posner, Russell, and Peterson in 2005 examine the circumplex model of affect as a way to
integrate affective neuroscience, cognitive development, and psychopathology [66]. Posner,
Russell, and Peterson propose that in the circumplex model of affect, all affective states come
from interpretations of core neural sensations that are produced by two independent
neurophysiological systems. While most theories of basic emotions suggest that every emotion
is created by a discrete and independent neural system, many findings in behavioral, cognitive
neuroscience, neuroimaging, and developmental studies of affect are more consistent with this
circumplex model.
Most recently, Yik, Russell, and Steiger produced an updated 12-point circumplex model of core
affect to integrate major dimensional models of mood and emotion [67]. Yik, Russell, and
20
Steiger examine emotions from a psychological perspective, and propose a 12-point model that is
plotted around a circle where the vertical dimension remains activation or arousal; the horizontal
dimension is pleasure-displeasure, or valence; and each quadrant is separated into three sections.
The model is displayed in figure 9, below.
Figure 9 - 12-Point Circumplex Model of Affect [67]
Colombetti’s Appraising Valence in 2005 [68]
describes emotional valence as the “positive” or
“negative” character of an emotion. Colombetti’s study explores the uses in the term valence in
psychology and emotion theory, the problems with these uses, and the utility of the notion of
valence. The first use of “valence” in psychological literature was used mainly as a synonym of
“charge”. In different studies, valence is used to describe objects or directions of behavior, while
in others (particularly later studies), it is used more for positive and negative emotions. Most
often, it is now used as “affect valence”, or how good or bad an emotion experience feels.
Another way to rate emotions is the SAM [69], which is used to rate affective emotional
dimensions of valence, arousal, and dominance. The SAM is used to measure emotional
responses to pictures, sounds, advertisements, painful stimuli, and more. It has also been used
with children, anxiety patients, analogue phobics, psychopaths, and other clinical populations. A
continuous nine-point scale can be used, and is shown in figure 10, below. The top row is
valence, the second row is arousal, and the third row is dominance. It has also become a useful
21
and easy to implement tool for measuring responses in marketing and advertising research as it is
usable in cross-cultural contexts [70].
Figure 10 - Self-Assessment Manikin: Valence, Arousal, Dominance [70]
2.2.2 A Neurological Perspective on Emotional Valence
A number of different approaches are used to calculate emotional valence. One of the more
common tools is the Valence Hypothesis, which states that there is “hemispheric specialization
for positive and negative emotions.” The valence hypothesis holds that the left hemisphere of
the brain is dominant for processing positive emotions while the right hemisphere is dominant
for processing negative emotions [71]. Furthermore, anatomical studies show that emotions are
processed primarily in the pre-frontal cortex with high asymmetry, though this may vary due to
varying underlying structures. The frontal lobe regulates voluntary movement, consciousness,
emotional response and more. Observed problems in the frontal lobe include inability to focus
on task, mood changes, change in personality, and changes in social behavior [72].
A major article discussing brain asymmetry as it relates to emotions is Davidson from 1992 [73],
which assigns primary roles in approach and withdrawal behaviours to the left and right frontal
and anterior temporal regions of the brain. Individual differences in emotional reactivity are
associated with stable differences in baseline asymmetry measurements in the anterior regions.
Davidson associates increased activation or decreased alpha activity in the left frontal region (F3
electrode, for example) with happy or positive emotions, and increased activation in the right
22
frontal region (F4 electrode, for example) with unhappy or disgust emotions. The individual
differences in asymmetry patterns are found to be stable over time and can predict features such
as an individual’s dispositional emotional profile, emotional reactivity and mood.
In Davidson and Irwin’s 1999 review [74], they look at the functional neuroanatomy of
emotions. Davidson and Irwin emphasize the prefrontal cortex and the amygdala as key
components of emotional circuitry. In figure 11, below, the brain is shown with the dorsolateral
region highlighted in blue, the orbitofrontal region in green, and the ventromedial region in red,
which are all sectors of the prefrontal cortex. In the ventral view on the right, the amygdalae are
identified by the yellow arrows.
Figure 11 - Sectors of the prefrontal cortex: lateral view (left), ventral view (right) [74]
Davidson and Irwin explain that many studies show that the left anterior areas are activated by
positive emotions and the right anterior areas are activated by negative emotions, though spatial
resolution is limited in electrophysiological measurements. The regions highlighted in figure 11,
above, are found to be important in human affective response, though very few studies have been
designed to manipulate subcomponents of emotions specifically in order to investigate the
individual areas in more depth. Davidson and Irwin summarize evidence for the lateralization of
emotional valence, particularly that the right prefrontal cortex is responsible for aversive
emotional responses.
Davidson’s article on affective neuroscience and psychophysiology, based on the Presidential
Address to the Society for Psychophysiological Research, was published in 2003 [75]. Here,
23
Davidson emphasized the role of asymmetry in the prefrontal cortex for approach and
withdrawal and the role of the amygdala in direction attention to affectively salient stimuli.
In 2004, Davidson published a commentary on what the prefrontal cortex specifically does in
affect [76]. Davidson discusses that research in frontal EEG asymmetries have made
considerable progress since the first reports 25 years before, but that there has been an absence of
connection with neuroscience research on the structure and function of the prefrontal cortex. He
also states that while most research into emotions have focused on alpha band power,
asymmetrical effects have been seen in other bands including theta, beta and gamma, and that
further investigation into these frequency bands may provide additional information in emotional
processing. Other issues discussed are bilateral variations in the prefrontal cortex, the problem
of inconsistent reference electrode placement.
Colibazzi et al. reported on the neural systems that are responsible for valence and arousal in a
2010 report [77]. The study used fMRI to identify the neural networks subserving valence and
arousal by assessing the associations of blood-oxygen level-dependent response, which is an
indirect index of neural activity, with ratings of valence and arousal during induced emotional
experiences. In particular, Clibazzi et al. found that unpleasant emotional experiences were
associated with increased blood-oxygen intensity in the supplementary motor, anterior
midcingulate, right dorsolateral prefrontal, occipito-temporal, inferior parietal, and cerebellar
cortices. Furthermore, high arousal was associated with increased blood-oxygen intensity in the
left thalamus, globus pallidus, caudate, parahippocampal gyrus, amygdala, premotor cortex, and
cerebellar vermis. Further analysis found that pleasant emotions involved the midbrain, ventral
striatum, and caudate nucleus, which are portions of a reward circuit. The findings suggest that
distinct networks subserve valence and arousal. In particular, arousal is mediated by midline and
medial temporal lobe structures, and valence is mediated by dorsal cortical areas and mesolimbic
pathways. In figure 12, below, statistical maps show where blood-oxygen intensity is associated
with valence values on the left, and arousal ratings on the right.
24
Figure 12 - Statistical maps showing blood oxygen signal intensity associated with (left)
valence ratings, and (right) arousal ratings [77]
In 2009, the United States NIH produced a review of measures of emotion [78]. In the review,
they explore self-reported measures of emotion, autonomic measure of emotion, startle response
magnitude, brain states including EEG and neuroimaging such as fMRI and positron emission
tomography (PET), and behaviour. Of most interest is the review of EEG measures of emotion.
The measures of emotion with EEG are typically contrasting activation in large regions of the
brain. This contrast can be anterior vs. posterior in combination with the distinction between
left-sided and right-sided hemispheric activation. It is common to measure valence using alpha
band power (8-12 Hz), which is inversely related to regional activation: that is, increased alpha
band power in one region of the brain suggests less emotional activity. The review focuses
particularly on “frontal asymmetry”, comparing alpha power in the left frontal region of the brain
vs. the right frontal region. Studies in the review found that greater activation in the left side
suggests greater positive emotional response. Further studies showed that frontal EEG
asymmetry may more accurately represent approach (left side) vs. avoidance (right side) rather
than emotional valence.
Brown et al.’s study on wireless emotional valence detection [20] describes emotional arousal as
the “level of physiological activation in response to a stimulus,” and emotional valence as
“commonly attributed as the psychological appraisal given to the stimulus”. Using an approach
based on the Valence hypothesis, Brown et al. quantify emotional valence based on positive
emotions being processed primarily in the left hemisphere and negative emotions being
processed primarily in the right hemisphere. Their approach uses alpha power ratio features in
25
asymmetrical electrode pairs, particularly F3/F4 and F7/F8. In particular, they calculate
maximum and kurtosis of alpha power ratio and the number of peaks in the alpha power ratio to
create a profile of the dynamics of the alpha left/right ratio during the period of recording
including maximum, kurtosis, and peaks per minute where peaks are two standard deviations
from the mean. The subjects were tested using emotionally affective film clips. Using QDC,
SVM, and KNN classifiers to compare results, they were able to achieve 82% correct emotional
classification with KNN classifiers when viewing films clips.
Using the Valence Hypothesis Ramirez and Vamvakousis created an equation to quantify
emotional valence using EEG electrodes on the prefrontal cortex [79]. Cited research shows that
beta waves are associated with an alert mental state while alpha waves are associated with a
relaxed state or less mental activity. In particular, an increase in alpha activity together with a
decrease in beta activity demonstrates cortical inactivation. Since the prefrontal lobe plays an
important role in emotion regulation, the F3 and F4 electrode positions are mostly used to
measure alpha activity. By comparing hemispheric activation, emotional valence can be
computed by comparing the alpha and beta power in channels F3 (left side) and F4 (right side).
In particular,
Equation 1 - Emotional Valence [79]
Using SVM classification, they were able to classify high vs. low arousal with 77.82% accuracy
and positive vs. negative valence with 80.11% accuracy while collecting responses to 12
different sound stimuli from 6 different subjects.
Schmidt and Trainor presented results of frontal brain activity distinguishing valence and
intensity of emotions elicited by music in 2001 [80]. Schmidt and Trainor recorded EEG during
60-second musical excerpts from asymmetrical frontal (F3, F4) and parietal (P3, P4) electrode
positions referenced to the Cz position. They saw significantly lower left frontal alpha power vs.
right frontal alpha power during positive emotions and the opposite during negative emotions,
confirming the usual Valence hypothesis. They also found that the overall frontal activity was
directly related to the intensity of the emotion. In particular, there was significantly greater
overall activity in the frontal area as the intensity of the stimuli increased.
26
In 2004, Rusalova and Kostyunina produced extensive spectral correlation studies on emotional
states related to different EEG electrode positions [81]. In their study, emotional responses were
measured by 14 electrodes and activity between electrode pairs was correlated for baseline, anger
and fear states. Fear conditions showed an extensive distribution of intracortical connections in
the δ range including the frontal, central, temporal, parietal, and occipital areas. Rusalova and
Kostyunina found that the anger emotion produced changes in the pattern of the distribution of
intracortical connections in the α-frequency range, with strong connections formed in the frontal
areas. An even larger number of connections was found in the high β (20-30 Hz) range.
Winkler et al. classified emotional valence using frontal EEG asymmetry in a 2010 article [82],
but found that affective pictures did not reliably cause changes in frontal asymmetry. Winkler et
al. were unable to replicate predicted asymmetry averaging within subjects or on a single trial
basis, and only found better-than-chance performance in two of nine subjects. They suggest that
stronger emotional elicitation might be necessary as some of the brain activity related to
emotions is created by deeper brain structures, so images alone may not produce sufficient
emotional engagement.
In 2011, Petrantonakis and Hadjileontidis proposed a novel emotion elicitation index using
frontal brain asymmetry [83]. Petrantonakis and Hadjileontidis created an AsI by analyzing
information from FP1, FP2 and F3/F4 sites. To evaluate the asymmetry index, they created a
classification process using two feature-vector extraction techniques and a SVM classifier in the
valence/arousal space. They reported up to 62.58% classification in user independent cases and
94.40% classification in the user-dependent case.
In 2012, Schuster et al. presented findings on EEG-based valence recognition and the influence
of individual specificity [84]. Schuster et al. recorded event-related potentials from subjects
stimulated by pictures from the International Affective Picture System. They found support
vector machine classifications based on intraindividual data showed significantly higher
classification rates than global ones, showing that classification accuracy can be boosted with
individual specific settings.
27
2.2.3 Emotional Valence and Asymmetry Related to Mental Illnesses
A variety of literature on emotional valence, emotional processing, and cerebral asymmetry in
different mental illnesses exists. In particular, this section will include examples of depression,
PTSD, schizophrenia, bipolar disorder, dementia, and other studies of interest.
In 2001, Knott et al. published a report on EEG characteristics in male depression [85]. In
particular, they focused on EEG power, frequency, asymmetry and coherence. They compared
resting EEG from 70 male, unmedicated, unipolar major depressive disorder outpatients and 23
normal control male subjects. The patient population showed greater overall relative beta power
and greater absolute beta power at bilateral anterior regions with a faster mean total spectrum
frequency. Knot et al. also noted inter-hemispheric alpha power asymmetry differences, as
controls showed relatively lower left hemispheric activation as well as widespread reduction of
delta, theta, alpha and beta coherence indices. Patients showed intra-hemispheric theta power
asymmetry reduction, and right hemisphere dominant beta power asymmetry. Using
discriminant analysis, 91.3% of both patients and controls were correctly classified.
Mathersul et al. investigated EEG alpha asymmetry in depression and anxiety in a 2008 study
[86]. Mathersul et al. categorized 428 participants on the basis of both negative mood and alpha
EEG to investigate the relationships in nonclinical depression or anxiety and lateralized
frontal/parietotemporal activity. They found that anxious participants showed greater right
frontal lateralization, depressed or comorbid participants showed symmetrical frontal activity,
and healthy control subjects showed increased left frontal lateralization. They also saw right
frontal lateralization in anxious arousal participants, and left frontal and right parietotemporal
lateralization in anxious apprehension. Finally, they found a bilateral increase in frontal and
increased right parietotemporal activity in depressed or comorbid participants. These findings
supported the valence-arousal predictions for frontal but not posterior regions.
An fMRI study of depressed patients by Herwig et al. in 2010 [87] studied the effect of
pessimism-related emotion processing in major depression. Herwig et al. cued depressed
patients and healthy control subjects to expect and then perceive pictures of known emotional
valences as well as stimuli of unknown valence in order to compare the brain activation in the
unknown expectations with the known expectations. They found that the brain activation in
depressed patients with unknown expectation was comparable to known negative expectation,
28
but not to known positive expectation, in contrast with healthy control subjects. This finding
was in line with the assumption that a cognitive feature of depression is the expectation of a
negative outcome. There was also increased activity in the dorsolateral prefrontal cortex and
medial prefrontal cortex correlated with the grade of depression, and also differed significantly
from the activity in healthy control subjects.
Kemp et al’s study in 2010 [88] compared resting EEG data in patients with MDD and PTSD
relative to healthy control subjects. The purpose of the study was to determine the specificity of
brain laterality in both disorders. Kemp et al. found reduced left-frontal activity and an overall
increase in alpha power in MDD. They also saw positive correlation between the severity of
PTSD and right-frontal lateralization and greater activity in the right-parietotemporal region in
PTSD relative to MDD. The increased alpha power in MDD was unexpected, as was the right-
frontal lateralization in PTSD. Their findings suggested that activation in the right-
parietotemporal region in particular may distinguish between the disorders in resting EEG.
From a treatment perspective, Rosenblau et al.’s 2012 study [89] investigated the effects of
successful antidepressant therapy on major depressive disorder. Rosenblau et al. used fMRI to
study activation during the presentation and anticipation of negative stimuli on a group of MDD
patients and healthy control subjects before and after an eight-week antidepressant treatment.
The patient group had greater amygdala activation during negative anticipation and greater
prefrontal activation without anticipation. Post-treatment, the amygdala and prefrontal activation
was significantly decreased in the patient population relative to the controls. The results indicate
that dysfunctions in emotional regulation mechanisms are present in depressed patients, but that
these dysfunctions may be at least partially reversible.
In 2013, Groenewold et al. reviewed fMRI studies of depression to determine whether emotional
valence modulates any of the abnormalities in brain function [90]. In a systematic literature
review, they found that opposing effects were seen in the amygdala, striatum, parahippocampal,
cerebellar, fusiform and anterior cingulate cortex. In particular, depressed subjects showed
hyperactivation to negative stimuli and hypoactivation for positive stimuli. Anterior cingulate
activity varied with facial vs. non-facial stimuli as well as to emotional valence. There was
reduced left dorsolateral prefrontal activity with negative stimuli and increased activity in the
29
orbitofrontal cortex with positive stimuli. Groenewold et al. determined that emotional valence
does moderate neural abnormalities in depression.
Focusing on post-traumatic stress disorder, Shankman et al. studied resting EEG asymmetry in
2008 by comparing a PTSD group and a control group [91]. Against their expectations,
Shankman et al. did not find significant differences in resting EEG asymmetry between the two
groups, nor did they relate specific aspects of PTSD to hemispheric differences. They concluded
that PTSD may be associated with different processes than conditions normally studied with
relation to brain asymmetry. Their study may have been limited by taking only one reading per
subject, and possibly by some of the PTSD group taking psychiatric medications.
In a 1995 study, Grosh et al. studied abnormal laterality in schizophrenia [92] by looking at both
schizophrenic patients and their parents. The study paired neutral words with words of positive
emotional valence in one test and negative emotional valence in another test. The results of the
study suggested that schizophrenics and their parents had similar abnormalities in hemispheric
activation only at baseline, but negative emotional stimuli caused a greater decrease in left
hemisphere activation only in the patient group. Grosh et al. suggest that dysfunction of the left
hemisphere may be a marker of vulnerability to schizophrenia, and the severity of the
dysfunction may distinguish those who do develop schizophrenia.
Burbridge and Barch studied the impact of emotional valence on reference disturbance inpatients
schizophrenia in a 2002 report [93]. The study found that schizophrenic patients had more
reference errors in their language for affectively negative topics vs. neutral topics. There is a
possibility that negative valence increases arousal levels, which can negatively impact the clarity
of language production, which is consistent with prior research on increased arousal negatively
influencing cognitive function. Burbridge and Barch suggest that further studies should include
more measures of affective arousal including skin conductance and heart rate. In 2007, Phillips
et al. [94] produced a similar study that also showed impairment in speech referencing of
schizophrenic patients during high arousal conditions. Phillips et al. found that patients with
depressive symptoms showed an even higher reactivity to stimuli with negative valence and high
arousal. Their findings demonstrated the importance of considering emotional context and
content in these patients.
30
In 2011, Lepage et al. studied the idea that schizophrenia can cause difficulties in facial
emotional processing [95]. Lepage et al. performed an fMRI study on a group of schizophrenia
patients and a group of healthy controls presented with unhappy, happy and neutral faces. Both
groups were able to rate the emotional valence of the faces similarly, and exhibited increased
brain activity when presented with emotional faces compared to neutral ones in multiple brain
regions. There were differences in specific areas, as the schizophrenia group showed a
correlation between flat affect and activity in the amygdala and bilateral parahippocampal
regions. The healthy group showed more activity in brain regions involved in early visual
processing compared to the patient group.
Pavuluri et al. performed an fMRI study on pediatric bipolar patients in 2008 [96] to determine
how attentional control and affect processing are integrated. A patient group and a healthy
control group were given an emotional valence task matching the colour of affective words to
coloured circles. The patient group showed functional alteration compared to the healthy control
group in affective and cognitive brain circuitry which may contribute to difficulty with affect
regulation and behavioural self-control in patients with pediatric bipolar disorder.
In 2010, Drago et al. [97] studied the intensity of emotional processing in patients with
Alzheimer’s disease. Drago et al. presented a group of patients with Alzheimer’s disease and a
healthy control group a series of emotionally affective pictures and asked them to rate these
pictures on a linear scale from happy to sad, based on how pleasant or unpleasant they found the
image. The patient group scored lower intensities of emotional valence than the control subjects
and had more inconsistency in the valence ratings.
Schiffer et al. studied the hemispheric emotional valence response to auditory evoked potentials
in a 2007 study [98] by comparing a group of healthy control subjects to a group who were
victims of childhood maltreatment. The results showed that 62% of controls and 67% of
maltreated subjects had right negative hemispheric emotional valence with a strong relationship
to the gender of the subject. Schiffer et al. suggest that the laterality of emotional valence may
be an important factor for guiding lateralized treatments such as transcranial magnetic
stimulation.
31
In 2011, Flo et al. studied emotional and physical distress during sleep [99] by examining frontal
EEG alpha asymmetry. Flo et al. found that even during sleep, measurable changes in alpha
symmetry were seen to aversive stimulation, as well as galvanic skin response, and REM sleep.
No frontal beta asymmetry was seen during sleep conditions.
32
Chapter 3. Technical Background
3.1 EEG Background and Specifications
3.1.1 Digital EEG Requirements
While EEG was originally recorded on paper in analog form, the advancements in computer
technology made digital EEG the new standard. Advantages of digital EEG include efficiency,
nondestructive processing, precision, archiving, transmission and comparison, increased
frequency range, reliability, portability, and the use of digital signal processing [100].
Recording digital EEG has a number of requirements for clinical use, including amplifier
specifications, analog-to-digital conversion specifications, and filtering. The OSET [25]
recommends a minimum of 25 electrode inputs including 21 on the scalp, the system reference,
ground, and two extra electrodes, which will be described more fully in the next section. The
input impedance of the amplifier is recommended to be greater than 10 MΩ with a common
mode rejection ratio (CMRR) of greater than 100 dB for each input. Common-mode-rejection
ratio refers to how well the differential amplifier can reflect the difference between inputs [100].
The OSET recommends a minimum sampling rate of 200 Hz, with 256 or more being preferable,
but over 500 Hz is not required for cortical EEG. In terms of vertical resolution, 12 bits or
higher is preferred. In the area of filtering, a wide bandpass of 0.1-100 Hz is suggested with a
notch filter at 50 or 60 Hz, depending on the line power used.
The ACNS guidelines from 2006 [101] modify these recommendations slightly. In particular,
they suggest that the sampling rate should be at least three times higher than the high-frequency
filter setting, for example, 100 Hz for 35-Hz high filter, or 200 Hz for 70 Hz high filter, though
higher rates are preferable. The ACNS recommends a minimum of 11 bits per sample though
prefer 12 bits or more to resolve EEG down to 0.5 µV and up to plus or minus several mV
without clipping. The minimum CMRR is set at 80 dB, though again with a preference for
higher, and noise in the recording of less than 2 µV peak-to-peak from 0.5-100 Hz.
33
3.1.2 Standard Electrode Placement and Electrode Montages
The standard EEG electrode positioning is referred to as the International 10-20 System which
was first published in 1958 [102]. In this system, the subject’s head is measured from nasion to
inion (top of nose to base of skull) and from middle of ear to middle of ear. The main electrode
lines laterally are Fp (frontal pole), F (frontal), C (central), P (parietal), and O (occipital). The
central line is exactly half the distance from nasion to inion, while the frontal pole and occipital
lines are 10% of the distance from the nasion and inion, respectively, and twice this distance, or
20% of the total, separates each of the other lines. This positioning allows the system to be used
on any subject universally.
Since the original definitions, more lines have been added between the original five. The ACNS
Guidelines for Standard Electrode Position Nomenclature [103] suggest a slight modification of
the original 10-20 system as there was an inconsistency in T3/T4 and T5/T6 electrodes. The
standard definitions used now are shown in the figure below.
Figure 13 - Standard 10-20 Electrode Positioning with ACNS Modification [103]
34
The additional lines are AF (anterior frontal), FT and FC (frontotemporal and frontocentral), TP
and CP (temporal-posterior temporal and centroparietal), and PO (parieto-occipital). The
electrodes are evenly spaced still as percentages of the subject’s head size.
The voltages at each of these electrode sites are measured as a voltage difference. Where the
differences are set is referred to as an electrode montage. In their guideline to standard montages
[104], the ACNS discusses three main classes of montage: longitudinal bipolar, transverse
bipolar, and referential. While there are several different types of longitudinal and transverse
bipolar montages, the type used in this thesis and in EEG studies at CAMH is the referential
montage. In this system, a single reference electrode is chosen, and the voltage at each
measurement electrode is recorded as VElectrode - VReference.
In Pivik et al.’s guidelines for EEG [105], the placement of the reference should be as
electrophysiologically silent as possible. The convention is to use a contralateral or unilateral
earlobe or mastoid reference, designated A1/A2 or M1/M2. In the United States NIH’s EEG and
ERP guidelines [6], the commonly suggested options are the tip of the nose, single or linked
mastoid, or single or linked earlobes. In NeuroScan’s 64-channel cap, the reference electrode is
located between the CZ and CPZ electrodes.
In order to minimize electrical noise, a ground or bias electrode is used. The NIH suggests that
this electrode can be placed anywhere, but a forehead or ear location is often used. Teplan’s
Fundamentals of EEG Measurement [106] suggests that a forehead or ear location can be used,
but sometimes a wrist or leg (similar to the right leg drive in ECG) can be used. The NeuroScan
cap places the ground electrode at the AFZ position.
35
3.1.3 EEG Signal Processing
An EEG is originally recorded as a signal of voltage (or power) vs. time. In order to analyze the
signal using the clinical bands described in section 1.1.2 and further in the next section, the
signal must be converted to the frequency domain. This section will discuss the techniques used
to convert a time-domain EEG signal to a frequency domain signal, though the same technique
could be used for any time-domain signal.
In particular, signals can be examined in the frequency domain using Fourier analysis. A
detailed description of Fourier series is not given here, as it is not required for basic signal
analysis, however, it can be found in Signal Processing for Neuroscientists [107] or Discrete-
Time Signal Processing [108], among other resources. The majority of the signal processing is
carried out using MATLAB software, though some of the techniques are later converted to C++
for the ambulatory system.
The primary technique used is the discrete Fourier transform (DFT) [107], which is defined as:
Equation 2 - Discrete Fourier Transform
In this equation, x(n) is the time signal represented by a discrete (or sampled) series, and X(k) is
the discrete Fourier series. is a notational simplification of the exponential value shown in
the first equation, which comes from the continuous Fourier transform. This value is also called
the twiddle factor. The discrete inverse Fourier transform, which returns the Fourier series to a
discrete time series, is defined as:
Equation 3 - Discrete Inverse Fourier Transform
In MATLAB and in many other programming languages, the DFT is optimized using a periodic
twiddle factor and called an FFT and the inverse DFT is called an IFFT. This periodicity is
limited by sampling frequency, FS, where the largest frequency that can be represented is one
half of the sampling frequency, defined as the Nyquist limit [107].
36
The initial output of the FFT is a series of N complex values where the frequency spacing is
given by
. In order to obtain a real value in frequency analysis, this output can be interpreted
as an amplitude spectrum or a power spectrum. The power spectrum is the power at each
frequency, which is obtained by multiplying the FFT output X with its complex conjugate X*.
Equation 4 - Power Spectrum of Signal
This power spectrum is normalized by dividing by the number of data points, N, which ensures
that the energy of the time series equals the sum of elements in the power spectrum, which
comes from Parseval’s theorem [107]. The DFT consists of even (real) and odd (imaginary)
parts. The power spectrum is even, so the part of the spectrum relating to negative frequencies is
identical to the part with positive frequencies. It is therefore common to depict only the first half
of the spectrum, which is up to the Nyquist limit.
The other approach to spectral analysis is the amplitude spectrum, which is used commonly in
this project. The amplitude spectrum corresponds with the amplitude of sinusoidal signals in the
time domain, or the square root of the power.
Equation 5 - Amplitude Spectrum of Signal
In practical terms, the following scripts with example numbers are used to calculate these
elements in MATLAB. The time signal will be represented by x.
Fs = 250; % Sampling rate
L = 1000; % Length of FFT
x % time signal
y = fft(x,L)/L; % create frequency signal
S = y(1:L/2).*conj(y(1:L/2)); % Power spectrum of signal
AS = abs(y(1:L/2)); % Amplitude spectrum
Furthermore, in order to use the clinical EEG bands as defined in section 1.1.2, there are several
different approaches that can be taken [105]. Most commonly, the bands are quantified by
37
summing up the amplitudes within the band or the power components within each band. This
summation is referred to as absolute band power or total amplitude. However, relative amplitude
or power can also be used, where the sum of amplitudes or power in each band is divided by the
total amplitude or power across all bands. This type of summation gives a value in percentage,
which can then be easily compared between different EEG systems.
Finally, when performing spectral analysis, it is common to use windows in the time domain in
order to gain a sense of the change in the amplitude or power spectrum with time. For example,
a single channel of five minutes of EEG data sampled at 250 Hz would contain 75000 data points
in the time domain. One could perform an FFT on the entire signal and have an amplitude
spectrum over five minutes of time. If, however, the user performed different tasks over these
five minutes, it might be useful to view the amplitude spectrum in different segments, or
windows, of time. This view might involve taking an FFT of every 10 seconds of data, or every
2500 points. In some applications, it would be ideal to overlap these windows by some portion
of the window length. In this example, a five second, or 1250 point overlap would provide
spectral analysis of time windows from 0-10s, 5-15s, 10-20s, and so on. The ideal length for
windows will be discussed further in section 3.1.4.
There are a number of different windows that can be used in signal processing, including
Bartlett, Hamming, and Hann windows, but for the purposes of this research, the simple
rectangular window is used, where:
Equation 6 - Definition of Rectangular Data Window
While the rectangular window can come with a cost of ripple effects in the spectrum [107], the
limits of the clinical EEG bands still make it a useful tool, particularly if the signal is not being
converted back to the time domain.
38
3.1.4 Expanding on Clinical EEG Bands
The standard EEG bands are explained briefly in section 1.1.2, but there are more details with
regards to amplitudes, areas of occurrence, subject condition, and reliability of quantitative
features. Detailed explanations of different EEG waveforms are provided in several reference
texts, including Standard Electroencephalography in Clinical Psychiatry [109] and
Niedermayer’s Electroencephalography [110]. An expansion of the clinical bands taken and
slightly modified from Standard Electroencephalography is shown in the table below. Note that
neither source mentions gamma rhythms beyond a cursory explanation.
Table 3 - Information on EEG Bands [109]
Band
Name
Frequency
Range
Amplitude
Range
Main area Condition
Delta 1-3.5 Hz 50-350 µV Variable Drowsiness, deep sleep, hyperventilation,
infancy & childhood
Theta 4-7 Hz 10-150 µV Variable Drowsiness, deep sleep, hyperventilation,
infancy & childhood
Alpha 8-12 Hz 20-100 µV Posterior Relaxed wakefulness, eyes closed
Beta 12.5-28 Hz 10-30 µV Frontal or
Diffuse
Increase during cognitive efforts,
drowsiness and light sleep
Gamma 30-50 Hz Low No discussion
As mentioned in the previous section, the power in each band can be measured both absolutely
and relatively, though measuring relative power may reduce the ability to interpret variations in
bands, so absolute power measures are recommended for clinical use and research [105].
In 2007, Gudmundsson et al. produced a report on the reliability of different quantitative EEG
features [111]. Among their findings, they determined that the highest reliability was obtained
with an average montage, where the “reference” electrode is an average of all electrodes, and
that the EEG signals became increasingly reliable with epochs or windows up to 40 seconds in
length, with longer epochs not providing significant improvement. Gudmundsson et al. found
that quantitative EEG features were most reliable in power spectral parameters. Among the EEG
bands, they found that the theta, alpha and beta bands were most reliable in repeated readings,
while delta and gamma bands were somewhat less reliable.
39
3.2 Statistical Analysis and Correlation Methods
In Chapter 4 of this thesis, as well as sections 6.2 and 7.1, ambulatory EEG acquisition systems
are quantitatively compared to a gold standard system. A common tool for comparing numerical
measurements, which is used particularly in Chapter 4, is Pearson’s correlation [112] [113]. In
particular, Pearson’s correlation provides a correlation coefficient (R or ρ) and p-value, which is
an indicator of the significance of the results. The following table explains the significance of
some correlation coefficients.
Table 4 - Significance of Different Correlation Coefficients [114]
Correlation Coefficient (R or ρ)
Value Meaning
1.0 Complete positive linear relationship
0.7 Strong positive linear relationship
0.5 Average positive linear relationship
0.0 No relationship between data
-0.7 Strong negative linear relationship
-1.0 Complete negative linear relationship
The p-value is the probability of getting a result more extreme than the one that was observed
(the correlation coefficient in this case) if we assume that there is no relationship between the
data (null hypothesis). A significance level can be chosen, usually 0.01 or 0.05, which indicates
that the result would be very unlikely if the data was unrelated [114]. Pearson’s correlation
coefficient is explained in many statistical textbooks including Probability, Statistics and
Random Processes for Electrical Engineering [115] and can be calculated in MATLAB using
the [R,P] = corr(X,Y) command. Other statistical software programs also commonly use
Pearson’s correlation.
It has been suggested in literature that evaluating different observation methods with Pearson’s
correlation coefficient is inappropriate because the data may be linearly similar but have little or
no agreement [116]. Suggested alternative approaches include Lin’s concordance correlation
coefficient and intraclass correlation coefficients (ICC). Pearson’s correlation coefficient
describes the relationship of two variables to a line of best fit. Lin’s coefficient modifies this
approach by also assessing how close the line is to a 45-degree line drawn through the origin if
the two variables are plotted on a scatter diagram. Lin’s coefficient is calculated as:
40
Equation 7 - Lin's Concordance Correlation Coefficient
where r is the estimated Pearson coefficient between n pairs of results (xi, yi), and are the
sample means of x and y, and and
are
times the estimated variance of x and y,
respectively.
An ICC is an index of reliability that is used to measure reproducibility and repeatability which
is very similar to Lin’s coefficient. It is similar to calculating Pearson’s correlation between the
data sets, however instead of centering and scaling each group by its own mean and standard
deviation, a combined mean and standard deviation is used. In the simplest terms, the ICC is a
measurement of the proportion of variance that is due to the object of measurement, or target
[117]. An ICC is calculated as the part of the total variance that is due to the differences in
paired measurements obtained by two or more methods [118]. It can be calculated as:
Equation 8 - Intraclass correlation
where is the variance between the methods being compared and
is the error variance.
This variance can be calculated using analysis of variance (ANOVA) tables. McGraw and Wong
[117] discuss different versions of ICC, two of which may be appropriate in comparing two
measurement systems. To compare two sets of scores, a two-way ICC model can be used. One
two-way ICC measures the degree of consistency among measurements, which excludes column
variance, or variance between the particular measurement systems. This variance can be the case
if one of the two systems may consistently measure or rate higher or lower than the other. The
other ICC measures the absolute agreement between measurements, where both measurements
are assumed to come on the same scale with no significant difference in anchor point.
Calculations for ICC are based on this paper, and use a MATLAB function available on the
Mathworks File Exchange [119].
41
Chapter 4. Comparitive Evaluation of an MBAN EEG Platform vs. Clinical Gold Standard [23]
4.1 Introduction
Monitoring EEG in ambulatory environment is becoming more important not only in clinical
domains but also as an extra parameter for various life-style, brain computer interface (BCI), and
entertainment applications. In order to address a wide variety of clinical applications, it is
important to have a system that is miniaturized, wearable, wireless and capable of providing
flexibility and comfort to the user.
The imec group [22] [27] [120] [26] has created an eight-channel ultra-low-power wireless EEG
system that acquires EEG data and wirelessly transmits to a USB-connected receiver. In order to
analyze data quality, it was determined that the ideal method would be to connect a single EEG
cap to both the wireless system and to the gold standard NeuroScan SynAmps system, which is
used extensively in clinical research applications. NeuroScan lists many articles in applied
neuroscience, in research involving MRI/EEG recordings, and in sensory neuroscience in where
NeuroScan equipment is used [121].
There is a lack of data available on the quality of ambulatory MBAN EEG systems compared to
clinical standard systems. NeuroSky, a manufacturer of single-lead dry EEG systems, published
their own white paper [112] in which they gave correlation coefficients for Fourier-transformed
EEG signals from their dry sensor EEG system to the wet electrode Biopac system, which is
used in medical and research applications. Signals were simultaneously recorded from side-by-
side electrodes, and they provided results for a single subject tested with approximately 30
seconds of data, with no correlation coefficient above 0.858 recorded in the frequency domain.
Matthews et al. [113] of QUASAR produced a study comparing novel hybrid EEG electrodes to
conventional wet electrodes in side-by-side testing which produced high levels of correlation
(>99% for seated subjects in the frequency domain).
Neither of these studies, however, tested two systems using the same headgear, which eliminates
any differences resulting from discrepancies in electrode positioning, material, and stimulus
42
effects. In this study, all testing was done simultaneously with the wireless MBAN EEG system
and a gold standard NeuroScan SynAmps system using a standard 64-channel EEG cap.
4.2 Participants
Nine healthy control subjects were tested in the EEG laboratory at the Centre for Addiction and
Mental Health (CAMH) in Toronto, Canada. All subjects passed a screening process that
included collecting their basic medical history to ensure their eligibility as healthy controls. We
excluded subjects with a psychiatric history, as well as any mental health history of first-degree
relatives. All subjects gave their written informed consent and the protocol was approved by
CAMH in accordance with the Declaration of Helsinki. The consent and screening form is
including in Appendix 3.
4.3 Equipment
The eight-channel wireless EEG system developed by imec is shown in figure 14a below, at far
left. The system builds on an EEG Application-Specific Integrated Circuit (ASIC) that achieves
high-performance at low power consumption [27]. The system has a low-noise (62 nV/Hz),
high common mode rejection ration (120dB) and has been optimized for low power
consumption, consuming between 3.3mW and 14mW depending on the mode of operation [120].
The system’s packaging included connectors for EEG DIN cables. In order to evaluate the signal
quality, it was compared to NeuroScan’s SynAmps amplifier system, which can be connected to
a 64-channel EEG cap. The SynAmps connector is shown in the figure 14b below, at middle
left. A 64-channel Quik-Cap from NeuroScan was used, with only eight channels plus reference
and ground being prepared with EEG gel to minimize their impedances. The specific channels
used are shown in figure 2. The Quik-Cap is shown in the figure 14c below, at middle right. A
pin-out board was created so that the 80-pin connector on the Quik-Cap could be connected both
to the SynAmps system and to the imec EEG ASIC. The connector board is shown in the figure
14d below, at far right.
43
Figure 14 - Equipment setup (left to right) a) Imec 8-channel EEG ASIC; b) NeuroScan
SynAmps connection; c) NeuroScan QuikCap 64-channel EEG cap; d) Custom-made 80-pin
connection board
4.4 Testing Method
The subjects completed baseline EEG readings, including 10 minutes of resting EEG (eyes
closed), and 5 minutes of watching an emotionally neutral video clip (eyes open) from Disney’s
“Silly Symphonies” without audio. These steps were completed in a counterbalanced order.
After the baseline readings, the subjects completed N-back working memory tests (N=0, 1, 2) in
random order and counterbalanced. Each of these tests was 13-15 minutes long.
In order to obtain meaningful correlation, simultaneous testing was required. Since only eight
channels could be used by both systems, four pairs of parallel electrodes were chosen for testing;
they are shown in the figure below. Frontal polar (FP1, FP2), anterior frontal (AF3, AF4),
frontal (F5, F6) and central (C3, C4) electrodes were used. Each electrode was prepared with
electrode gel as electrode impedances were lowered to < 5kΏ. Channels were referenced to an
electrode placed posterior to the CZ electrode.
Figure 15 - Electrodes from 10-20 system used for testing
44
4.5 Analysis Method
Acquired data was post-processed using MATLAB. The NeuroScan acquired signals were
sampled at 1000 Hz, and the imec acquired signals were sampled at 1024 Hz. To correct this
discrepancy, a native resampling function in MATLAB was used to down-sample the Imec data
to 1000 Hz to match the NeuroScan sampling rate. Next, the data was aligned using an original
function that checked the correlation between the data sets by moving the first 5 seconds of the
imec data by milliseconds against the NeuroScan data and finding the correct offset. After
matching the offset, the beginning and end of each set was trimmed to make them each even
multiples of five seconds and to remove data that may have been affected by the down-sampling.
After aligning the data and dropping the front and back, there was a total of over 7.5 hours of
data used.
The data was then transformed from time domain to the frequency domain using overlapping
two-second windows using MATLAB’s FFT function at 0.25 Hz resolution. The overlap was
one second, so the time domain data was converted for segments from 0-2 s, 1-3 s, 2-4 s, etc. for
each test. To match clinical EEG use, the frequency domain data from 1-50 Hz was used for
correlation, with 201 total points from the 0.25Hz resolution (i.e. 1.00 Hz, 1.25 Hz, 1.50 Hz, etc).
This frequency domain data was also split into δ (1-3.5Hz), θ (4-7Hz), α (8-12Hz), β (12.5-
28Hz) and γ (30-50Hz) bands. The same analysis was also done for 10-second time windows
with 5-second overlaps (0-10 s, 5-15 s, etc.). A correlation analysis returned the Pearson’s
correlation coefficient (R) and confidence value (P) for each set, and the means and medians of
different subsets were isolated. The results are presented below.
45
4.6 Results
Table 5 - Correlation for 2-second windows: Pearson’s Correlation Coefficient and
Confidence Valuesa
Band
(# of Points)
Pearson’s Coefficient (R) P-value
Mean Median Variance R2>0.5 Mean Median Variance
All (201) 0.9303 0.9757 0.00001 94.68% 0.0018 <1E-50 3.8E-7
Delta (11) 0.8110 0.9528 0.00016 80.62% 0.0148 1.6E-6 3.1E-6
Theta (13) 0.8383 0.9552 0.00011 83.76% 0.0197 1.7E-9 4.7E-6
Alpha (17) 0.8357 0.9612 0.00021 82.87% 0.0205 8.2E-12 1.3E-5
Beta (63) 0.8325 0.9414 0.00024 82.30% 0.0110 7.5E-36 7.3E-6
Gamma (83) 0.7839 0.8768 0.00011 78.07% 0.0121 3.7E-34 4.7E-6
Time (2001) 0.5520 0.6688 0.00015 41.73% 0.0084 <1E-50 3.0E-6
a. 215,880 total sets for correlation from 7.5 hrs. of testing
Table 6 - Correlation for 10-second windows: Pearson’s Correlation Coefficient and
Confidence Values b
Band
(# of Points)
Pearson’s Coefficient (R) P-value
Mean Median Variance R2>0.5 Mean Median Variance
All (201) 0.9570 0.9848 0.00001 97.83% 0.0018 <1E-50 3.8E-7
Delta (11) 0.9070 0.9647 0.00009 93.95% 0.0148 1.6E-6 3.1E-6
Theta (13) 0.9124 0.9834 0.00009 92.84% 0.0197 1.7E-9 4.7E-6
Alpha (17) 0.9064 0.9797 0.00018 91.15% 0.0205 8.2E-12 1.3E-5
Beta (63) 0.8922 0.9637 0.00014 90.10% 0.0110 7.5E-36 7.3E-6
Gamma (83) 0.8467 0.9215 0.00006 86.96% 0.0121 3.7E-34 4.7E-6
Time (2001) 0.5804 0.7063 0.00008 45.27% 0.0084 <1E-50 3.0E-6
b. 42,888 total sets for correlation from 7.5 hrs. of testing
Over the range of clinical EEG bands, very high correlation was seen between the two systems.
The correlation for each of the Delta, Theta, Alpha, Beta and Gamma bands was very high as
well, with significant p-values. The correlation values improved with larger windows,
suggesting that small errors were not as significant depending on the size of window used. It
was noted that the results of all bands together are not an average of the individual bands. This
46
discrepancy is likely due to the algorithm used by the correlation function in MATLAB, which
would be more forgiving to a 201-point data set.
The time domain signal was not as well correlated. This problem may be due in part to electrical
noise, as the NeuroScan system used AC power while the Imec system ran on a DC battery. The
NeuroScan system also excelled at eliminating offset or drift, but these issues were largely
removed by conversion to the frequency domain.
4.7 Conclusion
While the imec system was susceptible to some noise in the time domain, its frequency domain
information compared favourably to the gold standard NeuroScan system. In particular, for the
1-50 Hz range, if nearly 95% of values had a coefficient of determination (R2) above 0.5, then in
a 60-second sample where moving 2-second windows were compared, approximately 57 seconds
of the data are well correlated. For clinical EEG that is used for evaluation of emotional state,
this result would provide more than sufficient information. By reanalyzing the data with 10-
second windows, the results were improved across all bands. It is possible that changing the
time window further would improve results, as small errors and noise would be reduced further
with longer windows. Depending on the application of the system, a large enough window
would provide near-perfect results. In the future, this testing will be used to validate a full
wireless system for ambulatory monitoring of subjects with mental illness. Results indicate that
a fully ambulatory EEG system has comparable fidelity to clinical gold standard system.
47
Chapter 5. System Development
5.1 Hardware Development
5.1.1 Acquisition System
In the early stages of this project, several commercial and research systems were tested,
including the Emotiv Epoc, imec’s eight-channel wireless EEG ASIC and the Enobio EEG
system. The main problem with these systems was a lack of direct compatibility with
smartphones. In particular, none of them used Bluetooth or Bluetooth Low Energy, though the
newest iteration of the Enobio system does use Bluetooth communication. Due to this limitation,
it was decided that a new custom-made system should be built with chosen specifications. The
design and build of the system was led by technologist Kevin Tallevi. I provided requirements
for the system and helped test at each step of development. A 3D representation of the wireless
acquisition system is shown below, in figure 16. The major components are discussed below.
Figure 16 - 3D Representation of Wireless EEG Acquisition System
In order to perform straightforward EEG signal acquisition, the system was based around Texas
Instruments’ ADS1299 analog front-end chip [122]. The ADS1299 is a low-noise, 8-channel,
24-bit analog front-end chip designed for biopotential measurements. The chip is designed for
EEG measurements with very low input-referred noise of 1.0 µVPP and low 5 mW/channel
48
power requirements. It uses an input bias current of 300 pA, and has a common-mode rejection
ratio of -110dB, can provide a programmable gain between 1 and 24 and digitizes the analog
data at rates between 250 Hz and 16 kHz. The maximum input voltage is +4.5V (VREF)/gain.
The ADS1299 also has a built-in bias drive amplifier, lead-off detection, and test signals, as well
as internal or external reference signals. A functional block diagram of the ADS1299 is shown
in figure 17, below. The channel inputs can be bipolar or single-ended with a common
reference. For this system, a common reference is used, which is the format used by many other
EEG systems including the NeuroScan EEG system and the imec wireless system.
Figure 17 - Functional Block Diagram of ADS1299 [122]
In Appendix 1, the register settings used for the final system is provided. In particular, the
programmable gain is set to 12 for each channel, the reference and bias buffers are enabled, the
data rate is set to 250 Hz, and the single reference option is chosen.
Next, the wireless communication portion of the acquisition system is provided by the Texas
Instruments’ CC2541 Bluetooth Low Energy chip [123]. This chip is Bluetooth Low Energy
compliant and supports 250 kbps-2Mbps data rates with programmable output power of up to 0
dBm.
49
The power is provided by a 3.7V 500 mAh lithium battery, which is isolated with the Analog
Devices ADuM6201 5 kV isolator [124] to protect the circuit from ground when the charging
adapter is plugged in and isolates the patient from the power of the circuit. A 3V regulator is
used to provide power to the Bluetooth module. The 3.7V from the battery is also stepped up to
5V, and then split into +2.5V bipolar supply to power the ADS1299.
5.1.2 Capacitive Electrodes
After researching dry and capacitive electrodes as documented in section 2.1.2, an active
capacitive electrode was designed for EEG measurements. Kevin Tallevi designed and built the
electrode. I provided requirements and assisted with testing. The design is shown in figure 18,
below. The electrode was composed of four layers. The top layer contained the electronic
components, particularly the instrumentation amplifier as well as the ribbon connector; the
second layer contained a ground plane; the third layer was the bias layer of the electrode; and the
bottom layer was an insulated metal plane that acted as the electrode surface.
Figure 18 - Design of Active Capacitive Electrode
Unfortunately, when the actual electrode was built, it was unable to acquire small enough signals
to be used for EEG purposes. When connected to a signal generator, it was able to resolve
signals down at the 1 mV range, but EEG signals are typically in the 10-100µV range. When it
was connected to a 10-100 µV sine wave, there was no response, and there was also no response
when it was connected to a live test subject. As noted by Chi et al. [62], there is significant
attention required to the circuit design using custom components in order to acquire an
acceptable EEG signal, and there was not sufficient time or resources to fulfill this particular
50
portion of this project. Due to these limitations, the final proof-of-concept system uses
conventional wet electrodes.
5.1.3 Wireless Cap and Electrode Placement
In Brown et al.’s study [20], they began with eight electrodes at Fp1, Fp2 F3, F4, F7, F8, C3 and
C4. However, when data was analyzed, Fp1 and Fp2 were removed due to significant EOG
artifacts, and C3 and C4 were removed due to lack of emotional processing in this part of the
brain. As other studies including Ramirez and Vamvakousis [79] used F3 and F4 in particular,
and general consensus focuses on frontal asymmetry, the F3, F4, F7 and F8 positions were
chosen. Keeping the headset to four electrodes also helps to minimize the data rate and improve
the performance of the wireless transmission. The electrode placement is highlighted below in
figure 19. The signal electrodes are in green, and reference and ground electrodes are in blue.
As discussed in section 3.1.2, standard placement for ground electrodes varies, but forehead is
often used, and the NeuroScan cap uses the AFz position. To simplify the design, an Fz position
was chosen for the ground electrode so that a single line between the four signal electrodes and
the ground could be created. The common single earlobe/mastoid position was chosen for the
reference electrode (both sides are highlighted).
Figure 19 - Electrode Placement for EEG Cap
51
Images of the headset prototype are shown in figure 20, below. Kevin Armour assisted with
designing and building this headset prototype.
Figure 20 - Images of Headset Prototype
5.2 Software and Signal Processing Development
To create an ambulatory system, the BlackBerry Z10 smartphone was used for signal acquisition
and basic analysis. The Z10 uses the new BlackBerry 10 operating system, and has 2 GB of
RAM, 16 GB of internal storage, expandable MicroSD storage, 4.2 inch touchscreen, and a
Qualcomm Snapdragon S4 processor with 1.5 GHz dual-core CPUs which allows for multi-
threaded processes. The Z10 has USB 2.0 for charging and data synchronization, and is
Bluetooth 4.0 Low Energy (BTLE) compatible [125].
BlackBerry 10 applications can be written in C++, so the initial signal processing code was
written in Visual Studio and then migrated to BlackBerry Native SDK with Cascades for User
Interface components. Initially, I wrote and tested the signal processing code with CSV files of
data acquired during lab testing sessions. The data was windowed and transformed to the
frequency domain using the open-source Kiss FFT source code [126]. The size of the window
was easily adjustable using minimal variable changes, and was set generally at 5000 samples, or
20 seconds. These windows were used for calculating emotional valence using the simple
formula Valence = αF4 – αF3. The valence value was scaled by a multiplication factor so that the
graphed value would be on a similar scale to the self-reported emotion ratings.
The BlackBerry application was fully written by programmer John Li. John took the signal
processing code and translated it to the Cascades framework to be able to visually represent the
data and to add the user annotation portion of the software. The application was set up so that
52
the user could start an acquisition session when the headgear was applied, and that emotional
events could be annotated with a score ranging from -4 to + 4, using the same Self-Assessment
Manikin used in section 6.4 with a normalized scoring scale. These emotional events can be
roughly time-stamped from a menu that allows people to choose “When did this happen?”.
The data provided by the ADS1299 came in a 3-byte signed two’s complement format that had
to be translated to voltage. The least significant bit (LSB) is weighted as (VREF/Gain) / (2(Nbits-1)
)
= (4.5/12) / (223
) = 4.47035 x 10-8
. The data output from the ADS 1299 is converted to voltage in
the following form:
Equation 9 - Voltage Conversion Formula (24-bit)
For the purposes of Bluetooth data streaming, 16 bits were used. The calculation above remains
similar except that the last (least significant) byte is dropped, and then the multiplications are
based on 16 rather than 24 bits.
In the figure below, screenshots of the three main parts of the application are shown. On the left
is the device pairing screen, where the EEG device is selected and data streaming can begin. In
the middle is a graph of the 4 channels in near real-time and where the option to log the data is
selected. On the right is the emotional self-assessment screen, where the user grades his or her
emotion, adds a comment if required, and also adds a time-stamp.
53
Figure 21 - Screenshots of EEG App on BlackBerry 10 - Bluetooth Device Selection (left);
Graph of time signals (centre); Emotional Self-Assessment (right)
54
Chapter 6. Testing and Validation
6.1 Participants
The testing of the ADS1299 demonstration kit was completed with two healthy control subjects
(one male, one female). The testing of emotional valence calculations using the ADS1299
demonstration kit was completed with four healthy control subjects (two males, two females).
The full mobile system was tested with five healthy control subjects (two males, three females).
All of these tests were carried out at CAMH TMS-EEG labs. All subjects were between the ages
of 20 and30
The subjects were required to fill out a screening form in order as part of the REB approval
obtained by CAMH. The screening form is the same form that subjects in the comparative
evaluation of imec’s wireless EEG system in Chapter 4. This screening included collecting their
basic medical history. Subjects with a psychiatric history or a significant brain injury were
excluded, as well as those with any mental health history of first-degree relatives. All subjects
gave their written informed consent and the protocol was approved by CAMH in accordance
with the Declaration of Helsinki. The consent and screening form is included in Appendix 3.
6.2 Testing ADS1299 Demonstration Kit
In order to verify the performance of the Texas Instruments’ (TI) ADS1299, a simultaneous
testing protocol, similar to the one completed in Chapter 4, was performed with the NeuroScan
SynAmps2 system. As in Chapter 4, the subjects completed baseline EEG readings. In
particular, these readings included 10 minutes of resting EEG (eyes closed), and 5 minutes of
watching an emotionally neutral video clip (eyes open) from Disney’s “Silly Symphonies”
without audio. After the baseline readings, the subjects completed N-back working memory
tests (N=0, 1) in random order. Each of these tests was 13-15 minutes long. For one of the two
subjects, the 0-back condition data was lost due to technical difficulties.
In order to calculate the correlation, simultaneous signal acquisition was performed. For the first
subject, eight EEG channels were used, as in Chapter 4, specifically frontal polar (FP1, FP2),
anterior frontal (AF3, AF4), frontal (F5, F6) and central (C3, C4) positions. For the second
subject, new electrode positions had been chosen for the proof-of-concept system, as outlined in
55
section 5.1.3. These electrodes were all in frontal positions (F3, F4, F7 and F8). Each electrode
was prepared with electrode gel as electrode impedances were lowered to < 5kΏ. Channels were
referenced to an electrode placed posterior to the CZ electrode. The NeuroScan Quik-Cap was
connected to a pin-out board that split the connection to the NeuroScan SynAmps amplifier and
to the ADS1299 Performance Demonstration Kit.
The NeuroScan data was sampled at 1000 Hz, and the TI data was sampled at 250 Hz. In order
to align the data laterally, two tests were performed, with the NeuroScan data downsampled in
MATLAB to 250 Hz and the TI data upsampled to 1000 Hz. To align the data vertically, the TI
data was centred at 0 by subtracting all data points in a single channel from the mean of that
channel, and multiplying them by a scaling factor of 1,000,000 to qualitatively match the
amplitude of the NeuroScan data. After the data was aligned and scaled, the following steps
were performed to analyze the fidelity of the ADS1299 compared to the NeuroScan system:
1. Both data sets were separated into 4-second, non-overlapping windows (1000 samples for
TI data, 4000 samples for NeuroScan) and 20-second, non-overlapping windows (5000
samples for TI, 20000 samples for NeuroScan). The number of data sets is summarized
at the end of this section in table 7. The subsequent steps were done for both window
lengths and on individual channels.
2. The windows were converted to the frequency spectrum with a 1000-point FFT for TI
data and a 5000-point FFT for NeuroScan data, which corresponded to a 0.25 Hz
resolution for both systems.
3. The amplitude spectrum and power spectrum of each window was calculated for the
range of 1-50Hz.
4. The absolute and relative powers of each EEG band were calculated from the power
spectrum data.
5. For both the full amplitude spectrum (1-50Hz) and full power spectrum (1-50Hz),
Pearson’s correlation coefficient and confidence value, Lin’s correlation coefficient, and
ICC’s for consistency and absolute agreement were calculated. Each of these values was
correlated for the spectrums from each time window. Each correlation was done on 197
points (0.25 Hz resolution).
56
6. These correlations were repeated for the alpha and beta range combined (8-28.5 Hz), as
these ranges are of primary interest for calculating emotional valence. Each correlation
was done on 81 points.
Table 7 - Number of Segments Used for Correlation Analysis
Subject Test Time 4-second segments 20-second segments
Subject 1
(8 channels)
Eyes Open 5:10 (310 s) 616 128
Eyes Closed 10:10 (610 s) 1216 248
0-Back 14:00 (840 s) 1680 336
1-Back 15:00 (900 s) 1800 360
Subject 2
(4 channels)
Eyes Open 5:10 (310 s) 308 64
Eyes Closed 10:10 (610 s) 608 124
1-Back 15:00 (900 s) 900 180
Total 74:40 (4480 s) 7128 1440
The results are presented in section 7.1. Where means, standard deviations, minima, maxima
and percentages are used, they refer to the totals from the table above. For the sake of
comparison, all correlations were also repeated using the unprocessed data from the ADS1299.
6.3 Testing Electrodes
As outlined in section 5.1.2, a prototype of an active capacitive electrode design was ordered and
built. It was initially tested by hooking it up to a signal generator that was able to create sine
waves with amplitudes as low as 10 µV and in the 1-50Hz frequency range. Unfortunately, the
prototype was not able to read signals below 1 mV, which suggested that it would be unable to
acquire EEG signals. To ensure that the result was correct, it was placed on the forehead of test
subject and an exaggerated blink test was attempted, but no result was seen.
Next, conventional gel electrodes were tested. First, basic blink tests were attempted which
produced a clean result. Next, a prototype headset was made with gel electrodes, and they were
attached the NeuroScan SynAmps amplifier, and the impedance was successfully lowered to
below 5 kΩ, which is ideal for clinical EEG. This test determined that the electrodes were
usable.
57
6.4 Emotional Valence Testing Protocol
The subjects completed baseline EEG readings, including 5 minutes of watching an emotionally
neutral video clip (eyes open) from Disney’s “Silly Symphonies” without audio and 5 minutes of
resting EEG (eyes closed), at the end of which a chime sounded to alert the subject to open his or
her eyes for the next section.
Next, a series of 50 emotionally stimulating images was presented from the IAPS [21]. For each
image, the following timing sequence occurred: first, 4 seconds of blank screen, followed by
countdown slides from 4 to 1 lasting 1 second each. Then the emotional image was presented for
4 seconds, and then the subject was given 8 seconds to mark down his or her emotional rating of
the image on the Pleasure portion of the SAM by Bradley and Lang [69], a version of which is
shown in figure 22, below. This SAM is a modified version of the protocol used by
Petrantonakis and Hadjileontiadis [83]. Images were projected in groups in Brown et al. [20],
but time was not provided for the subjects to self-assess the valence after each image or video.
The full SAM also contains figures for arousal and dominance, but these were not deemed
necessary for this exercise. Each picture segment lasted for 20 seconds. A block figure of the
experimental protocol is shown in figure 23, below.
Figure 22 - Self-Assessment Manikin (Pleasure) [70]
Blank
Screen
Emotionally
Neutral Video
Instructions to
close eyes
Eyes Closed Instructions to
open eyes
Emotional
Pictures
Blank
Screen
0:08 5:00 0:04 5:00 0:04 16:40 0:04
Blank Screen 4 3 2 1 Picture Presentation Instruction to rate image
0:04 0:01 0:01 0:01 0:01 0:04 0:08
Figure 23 - Emotional Valence Experimental Protocol:
a) Overall protocol (top); b) Breakdown for each image (bottom)
58
Pictures from the IAPS database were chosen and grouped into five classes based on scores
reported by IAPS testers. This approach is similar to the protocol used by Brown et al. in the
image section [20], Petrantonakis and Hadjileontiadis [83], and others. Valence and arousal
were each rated on a scale of 1-9, with 5 considered average. These groups were, in order:
low valence (all below 4.0, average 3.38), low arousal (all below 4.0, average 3.81)
low valence (all below 4.0, average 2.57), high arousal (all above 6.0, average 6.40)
average valence (all between 4.5-5.5, average 5.00), average arousal (all between 4.5-5.5,
average 4.93)
high valence (all above 6.0, average 7.21), low arousal (all below 4.0, average 3.37)
high valence (all above 6.0, average 7.50), high arousal (all above 6.0, average 6.29)
A scatter plot of the emotional valence vs. emotional arousal of the emotional pictures is shown
in figure 24, below. A detailed breakdown of the images used is included in Appendix 2.
Figure 24 - Emotional Valence vs. Emotional Arousal Scatterplot of IAPS Images
For the first four subjects, data was acquired using the ADS1299 demonstration kit. Further
testing of emotional valence was done during the full mobile system test in section 6.5. Signal
electrodes were placed at F3, F4, F7 and F8. The bias electrode was placed at approximately
position Fz, and the reference electrode was placed at the mastoid. The simple prototype headset
pictured in section 5.1.3 was used for this testing. Data was sampled at 250 Hz.
1
3
5
7
9
1 2 3 4 5 6 7 8 9
Emo
tio
nal
Aro
usa
l
Emotional Valence
Emotional Arousal vs Emotional Valence (IAPS Averages)
59
6.5 Emotional Valence Data Analysis
For data analysis, the time data was converted to the frequency domain in 20-second (5000
sample) windows, which matches the length of time given for each emotional stimulus picture.
Different equations for calculating emotional valence were tested, and these are shown in table 8,
below. The variables represent the absolute power in the band at each electrode location. The
basis for these equations was taken from Ramirez and Vamvakousis [79] who calculate
emotional valence using equation 1 in the table below, and Brown et al. [20] who calculate
emotional valence based on equation 4 in the table below. The left hemisphere of the brain, with
electrodes F3 and F7, and the right hemisphere of the brain, with electrodes F4 and F8, are used in
different combinations.
Table 8 - Emotional Valence Equations Tested
Type (Right – Left) Type (Right / Left) - 1
1
2 (
) - 1
3 - 4 / - 1
5 - 6 / - 1
7
8
– 1
9
10 (
) - 1
11 - 12 / - 1
In order to obtain a more complete picture of these emotional valence calculations, this
processing was repeated on the Database for Emotion Analysis using Physiological Signals
(DEAP) [127] which is a set of physiological signals (EEG, skin conductance, EOG, EMG,
temperature, respiration) that were collected while subjects watched emotionally affective video
clips and then provided self-assessment ratings. There were 32 channels of EEG collected and
processed to a sampling rate of 128 Hz with EOG artifacts removed and bandpass filtered from
4.0-45.0 Hz. The data was split into 40 segments, each 63 seconds long, to match the videos
presented to the subjects, and a 512-point FFT was taken for 0.25 Hz resolution. The data had
already been pre-processed when it was acquired.
To test the accuracy of each emotional valence equation, the subjects’ self-assessment ratings of
the emotional stimuli were compared to the calculated emotional valence value after the mean of
the emotional valence ratings from the baseline period was subtracted from each value. In the
case of the DEAP datasets, the mean of the readings in the subject set was used. The self-
60
assessments were split into two categories, neutral-positive and negative, as in one version of
Brown et al.’s protocol [20]. For a simple binary outcome, if the self-assessment rated as 4.5 or
higher and the calculated value was positive or within one variance of 0 (as calculated through
MATLAB or Excel), this was considered a correct classification. Similarly, if the self-
assessment rated as 4.5 or lower, and the calculated value was negative or within one variance of
0 this was also considered a correct calculation. In summary:
IF SA > 4.5 AND EV > [mean(baseline) - variance(baseline)] Correct classification
IF SA < 4.5 AND EV < [mean(baseline) + variance(baseline)] Correct classification
6.6 Testing Full Mobile System
In order to minimize sources of error for the mobile acquisition system and software, the
NeuroScan Quik-cap EEG cap was used for mobile system testing at CAMH. The same
electrode locations were used as in section 6.4, though the Quik-cap reference at the top of the
head was used. The cap was connected to a breakout board so that signals could be collected
both by the wireless acquisition system and the NeuroScan SynAmps system.
The same evaluation testing protocol was used as in section 6.2, however, only 20 second
windows were calculated. The NeuroScan data was downsampled in the NeuroScan software to
250 Hz and FFT lengths were the same for both systems. The same testing protocol was used as
in section 6.4, as the subjects were presented with an emotionally neutral video clip for eyes
open baseline readings, followed by an eyes closed baseline reading, then the emotional stimuli.
The subjects rated each picture in the BlackBerry application with the SAM on a scale of -4 to 4.
Emotional valence was calculated by the BlackBerry application in real-time. Afterward, the
frequency components of the signal were correlated to the NeuroScan data as in section 6.2, and
the emotional valence measurements were analyzed as in section 6.5. In order to verify the real-
time calculations, the data was processed offline and emotional valence measurements were
compared to the participants’ ratings. Due to the packet streaming protocol used with Bluetooth
Low Energy, the transmitted sampling frequency did not exactly match the 250 Hz used by the
ADS1299 chip. The effective sampling frequency of the BlackBerry data was computed for
each subject in MATLAB, and data was resampled in MATLAB to match the NeuroScan data.
61
Chapter 7. Results
7.1 Validation of Performance Demonstration Kit
Table 9 - Pearson's Correlation (r, p) for Amplitude and Power Spectra of 4-second and 20-
second windows with Averaged and Scaled ADS1299 Data
Set Pearson’s Coefficient (r) P-value
Mean Median Variance r2>0.5 Mean Median Variance p<0.01
Amplitude
(4-second) 0.9550 0.9800 0.0039 98.91% 2.4E-14 9.3E-139 4.1E-24 100%
Power
(4-second) 0.9354 0.9801 0.0112 94.98% 5.7E-7 7.3E-139 6.4E-10 100%
Amp: α-β
(4-second) 0.9292 0.9633 0.0097 96.11% 4.4E-4 7.1E-47 1.3E-4 99.76%
Pow: α-β
(4-second) 0.9428 0.9733 0.0085 96.86% 4.2E-4 2.9E-52 1.1E-4 99.68%
Amplitude
(20-second) 0.9533 0.9792 0.0042 98.89% 6.0E-17 4.4E-137 5.2E-30 100%
Power
(20-second) 0.9336 0.9802 0.0116 94.65% 1.8E-6 3.8E-139 2.3E-9 100%
Amp: α-β
(20-second) 0.9298 0.9635 0.0084 96.46% 4.3E-6 5.9E-47 9.4E-9 100%
Pow: α-β
(20-second) 0.9423 0.9724 0.0073 96.88% 2.0E-6 1.0E-51 1.6E-9 100%
Table 10 - Pearson's Correlation (r, p) Repeated with Raw ADS1299 Data
Set Pearson’s Coefficient (r) P-value
Mean Median Variance r2>0.5 Mean Median Variance p<0.01
Amplitude
(4-second) 0.9548 0.9798 0.0039 98.92% 2.4E-14 2.9E-138 4.1E-24 100%
Power
(4-second) 0.9348 0.9799 0.0112 94.95% 5.7E-7 1.7E-138 6.4E-10 100%
Amp: α-β
(4-second) 0.9287 0.9632 0.0098 96.06% 4.4E-4 8.0E-47 1.3E-4 99.76%
Pow: α-β
(4-second) 0.9425 0.9734 0.0086 96.80% 4.2E-4 2.7E-52 1.1E-4 99.68%
Amplitude
(20-second) 0.9526 0.9789 0.0043 98.89% 6.0E-17 1.6E-136 5.2E-30 100%
Power
(20-second) 0.9319 0.9798 0.0119 94.51% 1.8E-6 2.7E-138 2.3E-9 100%
Amp: α-β
(20-second) 0.9288 0.9634 0.0088 96.25% 5.7E-6 6.3E-47 1.2E-8 100%
Pow: α-β
(20-second) 0.9415 0.9723 0.0076 96.67% 2.0E-6 1.2E-51 1.6E-9 100%
62
Table 11 - Lin's Covariance Correlation Coefficient (Rc) for Amplitude and Power Spectra
of 4-second and 20-second windows with Averaged and Scaled ADS1299 Data
Set Mean Median Variance Rc2 > 0.5
Amplitude
(4-second) 0.9456 0.9766 0.0062 97.69%
Power
(4-second) 0.9014 0.9659 0.0215 90.01%
Amp: α-β
(4-second) 0.9186 0.9611 0.0146 94.56%
Pow: α-β
(4-second) 0.9260 0.9688 0.0152 94.88%
Amplitude
(20-second) 0.9438 0.9767 0.0065 97.78%
Power
(20-second) 0.8988 0.9649 0.0218 89.65%
Amp: α-β
(20-second) 0.9197 0.9608 0.0132 94.65%
Pow: α-β
(20-second) 0.9258 0.9679 0.0139 94.65%
Table 12 - Lin's Covariance Correlation Coefficient Repeated with Raw ADS1299 Data
Set Mean Median Variance Rc2 > 0.5
Amplitude
(4-second) 1.2E-6 1.3E-6 1.5E-13 0%
Power
(4-second) 1.9E-12 1.8E-12 1.0E-24 0%
Amp: α-β
(4-second) 5.7E-7 5.6E-7 2.2E-14 0%
Pow: α-β
(4-second) 1.3E-12 1.3E-12 4.6E-25 0%
Amplitude
(20-second) 1.2E-6 1.2E-6 1.8E-13 0%
Power
(20-second) 2.0E-12 1.8E-12 3.1E-24 0%
Amp: α-β
(20-second) 5.7E-7 5.6E-7 2.1E-14 0%
Pow: α-β
(20-second) 1.3E-12 1.3E-12 9.7E-25 0%
63
Table 13 - Intraclass Correlation Coefficients (ICC) for Amplitude and Power Spectra of 4-
second and 20-second windows with Averaged and Scaled ADS1299 Data
Set ICC (C-1) ICC (A-1)
Mean Median Variance R2>0.5 Mean Median Variance R
2<0.5
Amplitude
(4-second) 0.9473 0.9772 0.0057 97.91% 0.9458 0.9767 0.0062 97.71%
Power
(4-second) 0.9024 0.9663 0.0211 90.17% 0.9018 0.9660 0.0214 90.08%
Amp: α-β
(4-second) 0.9261 0.9625 0.0109 95.64% 0.9193 0.9615 0.0145 94.65%
Pow: α-β
(4-second) 0.9312 0.9701 0.0127 95.62% 0.9267 0.9692 0.0150 94.99%
Amplitude
(20-second) 0.9456 0.9771 0.0060 97.92% 0.9441 0.9768 0.0064 97.78%
Power
(20-second) 0.8998 0.9653 0.0214 89.93% 0.8992 0.9650 0.0217 89.79%
Amp: α-β
(20-second) 0.9267 0.9628 0.0097 96.04% 0.9204 0.9613 0.0131 94.79%
Pow: α-β
(20-second) 0.9305 0.9688 0.0118 95.35% 0.9265 0.9682 0.0137 94.86%
Table 14 - Intraclass Correlation Coefficients repeated with Raw ADS1299 Data
Set ICC (C-1) ICC (A-1)
Mean Median Variance R2>0.5 Mean Median Variance R
2<0.5
Amplitude
(4-second) 2.0E-6 2.0E-6 7.0E-14 0% 1.3E-6 1.3E-6 1.5E-13 0%
Power
(4-second) 2.2E-12 2.0E-12 1.2E-24 0% 2.0E-12 1.9E-12 1.1E-24 0%
Amp: α-β
(4-second) 1.9E-6 1.9E-6 4.9E-14 0% 5.8E-7 5.7E-7 2.3E-14 0%
Pow: α-β
(4-second) 2.0E-12 1.9E-12 1.1E-24 0% 1.3E-12 1.3E-12 1.1E-24 0%
Amplitude
(20-second) 2.0E-6 2.0E-6 1.1E-13 0% 1.3E-6 1.3E-6 1.8E-13 0%
Power
(20-second) 2.2E-12 2.0E-12 3.3E-24 0% 2.0E-12 1.8E-12 3.2E-24 0%
Amp: α-β
(20-second) 1.9E-6 1.9E-6 6.8E-14 0% 5.7E-7 5.7E-7 2.2E-14 0%
Pow: α-β
(20-second) 2.0E-12 1.9E-12 2.5E-24 0% 1.3E-12 1.3E-12 9.9E-25 0%
64
A thorough set of correlations were done to compare the ADS1299 data to the NeuroScan data.
The data was separated into 4-second windows and 20-second windows and converted to the
frequency domain and the amplitude spectra and power spectra were calculated. Correlations
were performed for the full 1-50 Hz range and alpha-beta only (8-28.5 Hz) in both amplitude and
power spectra. The correlations were also done for both the raw ADS1299 data and with the
data that was averaged and scaled to match the NeuroScan data.
First, Pearson’s correlation coefficients were calculated, as in Chapter 4. Next, Lin’s covariance
correlation coefficients were calculated, which are a measure of both linear correlation and
agreement. Finally, intraclass correlation coefficients were calculated for both consistency (C-1)
and agreement (A-1).
The Pearson’s correlation coefficients were very high (mean above 0.928 for all sets) for both
averaged and scaled data and raw data, as seen in tables 9 (averaged and scaled) and 10 (raw),
which was expected. For 1-50 Hz frequencies, the amplitude spectrum showed slightly higher
correlation, but for the alpha-beta range, the power spectrum showed slightly higher correlation.
The Lin’s coefficients were very high for averaged and scaled data, shown in table 11, (mean
above 0.898 for all sets) with slightly higher correlation for amplitude spectra in the 1-50 Hz
range and slightly higher correlation for power spectra for alpha-beta frequencies. For the raw
data, shown in table 12, the correlation was extremely poor (mean on the order of 10-6
or less).
The intraclass correlation coefficients were very similar to Lin’s coefficients. The values were
very high for averaged and scaled data, shown in table 13, similar to the Pearson’s coefficients.
For raw data, shown in table 14, the correlation was extremely low.
These differences come primarily from the fact that the NeuroScan system automatically
averages the data so that each channel is centred on zero, and writes the values in microvolts,
while the ADS1299 writes the values in volts and has an offset for each channel. The NeuroScan
system also uses 32-bit data resolution, while the ADS1299 uses 24-bit data resolution, which
may introduce a level of difference as well. When the data is compared on equal footing, the
results are excellent, and are similar to those seen with the comparative evaluation of the imec
system performed in Chapter 4. Of particular note, the power spectra for the alpha-beta
frequency ranges were very well correlated.
65
7.2 Emotional Valence Testing
The results of the emotional valence testing on healthy control subjects at CAMH on the
ADS1299 demonstration kit are presented in the table below. Four subjects were tested on 50
images each. Based on the self-assessments completed by the four subjects, 129 of 200 images
were rated as neutral-positive, with the remaining 71 rating as negative.
Table 15 - Emotional Valence Testing on ADS1299 Demonstration Kit
Equation Positive
(of 129)
Negative
(of 71)
Total
(of 200)
Average
(of 50)
Maximum
(of 50)
1 -
91
(70.54%)
54
(76.06%)
145
(72.5%)
36.25 50
2 - (
) - 1 89
(68.99%)
58
(81.69%)
147
(73.5%)
36.75 50
3 - - 129
(100%)
69
(97.18%)
198
(99.0%)
49.50 50
4 - / - 1 129
(100%)
68
(95.77%)
197
(98.5%)
49.25 50
5 - - 120
(93.02%)
70
(98.59%)
190
(95.0%)
47.50 50
6 - / - 1 109
(93.02%)
71
(100%)
180
(90.0%)
45.00 50
7 -
81
(62.79%)
44
(61.97%)
125
(62.5%)
31.25 40
8 -
– 1 73
(56.59%)
50
(70.42%)
123
(61.5%)
30.75 40
9 -
100
(77.52%)
31
(43.66%)
131
(65.5%)
32.75 39
10 - (
) - 1 94
(72.87%)
36
(50.70%)
130
(65.0%)
32.50 40
11 - - 115
(89.15%)
66
(92.96%)
181
(90.5%)
45.25 50
12 - / - 1 99
(76.74%)
52
(73.24%)
151
(75.5%)
37.75 49
The results of testing the same emotional valence calculations on the DEAP dataset are presented
in the table below. A total of 32 subjects were tested on 40 images each. Based on the self-
66
assessments completed by the 32 subjects, 808 out of 1280 video clips were rated as neutral-
positive, with the remaining 472 rating as negative.
Table 16 - Testing Emotional Valence Equations on DEAP Dataset
Equation Positive
(of 808)
Negative
(of 472)
Total
(of 1280)
Average
(of 40)
Maximum
(of 40)
1 -
608
(75.25%)
366
(77.54%)
974
(76.09%)
30.4 40
2 - (
) - 1 572
(70.79%)
367
(77.75%)
939
(73.36%)
29.3 40
3 - - 808
(100%)
472
(100%)
1280
(100%)
40.0 40
4 - / - 1 558
(69.06%)
353
(74.79%)
911
(71.17%)
28.5 40
5 - - 808
(100%)
472
(100%)
1280
(100%)
40.0 40
6 - / - 1 527
(65.22%)
337
(71.40%)
864
(67.50%)
27.0 40
7 -
608
(75.25%)
346
(73.31%)
954
(74.53%)
29.8 37
8 -
– 1 529
(65.47%)
335
(70.97%)
864
(67.50%)
27.0 40
9 -
642
(79.46%)
373
(79.03%)
1015
(79.30%)
31.7 40
10 - (
) - 1 771
(95.42%)
180
(38.14%)
951
(74.30%)
29.7 39
11 - - 808
(100%)
472
(100%)
1280
(100%)
40.0 40
12 - / - 1 564
(69.80%)
350
(74.15%)
914
(71.41%)
28.6 40
The emotional valence testing showed very good results. By adding in an averaging mechanism
for each individual subject, some equations were able to correctly match the self-assessment on a
positive-neutral vs. negative basis nearly 100% of the time.
67
In particular, the simple - equation correctly identified the valence range of subjects
tested on the ADS1299 performance demonstration kit 99% of the time (198 of 200 ratings), and
100% of the time for DEAP subjects.
Using - , the valence range of subjects tested on the ADS1299 performance
demonstration kit was correctly identified 90.5% of the time, and 100% of the time for DEAP
subjects, which provides a useful secondary measure given the four electrodes being used by the
proof-of-concept system. This secondary measure may be particularly useful if one electrode
malfunctions.
Another alternative is the - equation, which correctly identified
valence in 95.0% of ratings on the ADS1299, and for 100% of DEAP subjects. This equation
makes use of the alpha-band power in all four electrodes.
In the simplest terms, these results show that positive and negative emotional valence can be
effectively classified for different subjects using a baseline average. While the equation
proposed by Ramirez and Vamvakousis [79] did not perform as well as those above, the
equations that did perform best were variations of the approach taken by Brown et al. [20],
focusing on power in the alpha band only. The difference between the two hemispheres
produced better results than the ratio between the two hemispheres.
68
7.3 Proof-of-Concept Data Validation Results
Table 17 - Pearson's Correlation for Subject 1 on Proof of Concept System (Averaged and
Scaled Data) – Effective Sampling Rate Calculated as 234 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.6640 0.6694 0.6158 0.6541 0.7140 0.6987 0.7567 0.7757
Power
(20-second) 0.5789 0.5954 0.5140 0.5076 0.6735 0.6996 0.7307 0.7513
Amp: α-β
(20-second) 0.1474 0.1123 0.2419 0.2525 0.1960 0.1438 0.2240 0.1849
Pow: α-β
(20-second) 0.0585 0.0001 0.2109 0.2219 0.1018 0.0424 0.1619 0.1020
Table 18 - Pearson's Correlation for Subject 2 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 234 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.5071 0.4720 0.4798 0.4493 0.5399 0.5242 0.2878 0.2728
Power
(20-second) 0.3941 0.3309 0.3404 0.2946 0.4301 0.3649 0.1197 0.0822
Amp: α-β
(20-second) 0.0466 0.0472 0.0631 0.0617 0.0357 0.0200 0.0194 0.0252
Pow: α-β
(20-second) 0.0034 -0.0133 0.0050 -0.0106 -0.0047 -0.0301 0.0065 -0.0114
Table 19 - Pearson's Correlation for Subject 3 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 240 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.7121 0.7114 0.6129 0.6663 0.7463 0.7392 0.6704 0.6689
Power
(20-second) 0.6243 0.6202 0.5368 0.5686 0.6778 0.7399 0.5771 0.5799
Amp: α-β
(20-second) 0.3772 0.4061 0.2979 0.2667 0.4262 0.4236 0.4346 0.4248
Pow: α-β
(20-second) 0.2544 0.2624 0.1961 0.1667 0.3199 0.3164 0.3763 0.3453
69
Table 20 - Pearson's Correlation for Subject 4 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 240 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.7089 0.7272 0.7642 0.7749 0.7483 0.7519 0.6431 0.6957
Power
(20-second) 0.6839 0.7157 0.7513 0.7865 0.7329 0.7523 0.5754 0.6553
Amp: α-β
(20-second) 0.5689 0.5925 0.6380 0.6646 0.6043 0.6250 0.5076 0.5409
Pow: α-β
(20-second) 0.5304 0.5406 0.5968 0.6327 0.5873 0.6093 0.5091 0.5224
Table 21 - Pearson's Correlation for Subject 5 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 240 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.7378 0.7327 0.6997 0.7154 0.7840 0.7843 0.7446 0.7679
Power
(20-second) 0.6881 0.7145 0.6355 0.6472 0.7217 0.7730 0.6891 0.7195
Amp: α-β
(20-second) 0.4784 0.4802 0.4988 0.5097 0.4092 0.3773 0.4530 0.4492
Pow: α-β
(20-second) 0.3699 0.3288 0.3842 0.3726 0.3744 0.3072 0.3534 0.3180
Table 22 - Lin's Correlation for Subject 1 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 234 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.5951 0.6073 0.0628 0.0484 0.6371 0.6395 0.7169 0.7290
Power
(20-second) 0.4452 0.4251 0.0085 0.0020 0.5159 0.5434 0.6156 0.6530
Amp: α-β
(20-second) 0.1143 0.0733 0.0177 0.0118 0.1571 0.1133 0.2045 0.1634
Pow: α-β
(20-second) 0.0451 0.0000 0.0045 0.0014 0.0654 0.0098 0.1278 0.0658
70
Table 23 - Lin's Correlation for Subject 2 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 234 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.4781 0.4534 0.4611 0.4297 0.5066 0.4766 0.0496 0.0443
Power
(20-second) 0.3122 0.2589 0.2873 0.2223 0.3443 0.2574 0.0040 0.0009
Amp: α-β
(20-second) 0.0391 0.0.0392 0.0565 0.0494 0.0293 0.0167 0.0029 0.0042
Pow: α-β
(20-second) 0.0032 -0.0042 0.0052 -0.0041 -0.0005 -0.0054 0.0004 -0.0004
Table 24 - Lin's Correlation for Subject 3 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 240 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.6708 0.6809 0.4567 0.5737 0.7049 0.6963 0.4986 0.5061
Power
(20-second) 0.5368 0.4998 0.3623 0.3197 0.5809 0.5664 0.3702 0.3269
Amp: α-β
(20-second) 0.3176 0.3004 0.2026 0.1449 0.3853 0.3734 0.3004 0.2349
Pow: α-β
(20-second) 0.1607 0.0981 0.1031 0.0255 0.2463 0.2014 0.2032 0.0956
Table 25 - Lin's Correlation for Subject 4 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 240 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.6540 0.7128 0.7566 0.7641 0.7255 0.7350 0.5699 0.6445
Power
(20-second) 0.6042 0.6627 0.7146 0.7578 0.6593 0.6700 0.4628 0.5025
Amp: α-β
(20-second) 0.5288 0.5710 0.6260 0.6574 0.5911 0.6245 0.4271 0.4631
Pow: α-β
(20-second) 0.4698 0.4823 0.5577 0.5928 0.5546 0.5553 0.3713 0.3583
71
Table 26 - Lin's Correlation for Subject 5 on Proof of Concept System (Averaged and
Scaled Data) - Effective Sampling Rate Calculated as 240 Hz
Set F3 F4 F7 F8
Mean Median Mean Median Mean Median Mean Median
Amplitude
(20-second) 0.7239 0.7225 0.6874 0.7002 0.7626 0.7645 0.7300 0.7446
Power
(20-second) 0.6397 0.6630 0.6019 0.6006 0.6542 0.6733 0.6435 0.6552
Amp: α-β
(20-second) 0.4518 0.4606 0.4713 0.4679 0.3816 0.3539 0.4306 0.4246
Pow: α-β
(20-second) 0.3108 0.2563 0.3233 0.3072 0.3274 0.2972 0.3073 0.2509
The results of comparing the data from the wireless system to the NeuroScan SynAmps2 system
are shown in the four previous tables. The results were separated by test subject due to different
recurring errors during each test; namely, an unexpected pattern occurring in a single channel
(channel 2 for subject 1, and channel 4 for subject 2). The errors are more noticeable in Lin’s
correlation (tables 19 and 20) compared to Pearson’s correlation (tables 17 and 18). Since Lin’s
correlation and intraclass correlation were so similar in section 7.1, only Lin’s was reported here
to minimize data overload.
The results for subjects 3 to 5 were significantly improved compared to the first two subjects.
After analyzing the data from the first two subjects, changes were made to the BTLE code to
produce a higher and more consistent sampling rate.
The results overall are significantly worse than when testing the ADS1299 performance
demonstration kit. While this situation will be discussed more in section 8.1.3, two potential
culprits of the problem are the variation in sampling rate due to Bluetooth streaming, and the
dropping of 1 byte (8 bits) from the transmitted data. The sampling rate for subjects 1 and 2 was
calculated at approximately 234 Hz based on calculation and inspection of the data in MATLAB,
but there was some variation during the testing itself. The sampling rate for subjects 3 to 5 was
calculated at approximately 240 Hz after some changes to the BTLE code were completed
between testing sessions. Note that the results in the alpha-beta range were also significantly
worse than in the full clinical range (1-50 Hz).
72
7.4 Proof-of-Concept Emotional Valence Results
Due to the discrepancies in data between the two systems outlined in the previous section,
emotional valence was calculated separately using both the wireless system and the NeuroScan
system. Self-assessment values were taken from the BlackBerry self-assessment window. In
this case, positive-neutral was counted as any rating from 0 to 4 and negative was any rating
from -4 to -1. This data was post-processed in MATLAB. While the BlackBerry application
contained a section to calculate emotional valence, the discrepancy in the sampling rate would
produce significant errors at this time. Since equations 3, 5, and 11 were shown to be the best in
section 7.2, only those are compared here.
Table 27 - Emotional Valence Classification for Proof-of-Concept System Test
Equation Wireless System NeuroScan
Positive
(154)
Negative
(96)
Total
(250)
Positive
(154)
Negative
(96)
Total
(250)
3 - - 153 96 249 154 95 249
5 - - 154 96 250 154 96 250
11 - - 153 88 231 152 96 248
It is useful to note that despite the problems with the data correlation and the different errors seen
with both subjects, especially in the alpha-beta frequency range, the emotional valence response
was correctly classified in almost every case. Using the baseline readings for an average as
described in section 6.5 probably allowed for the discrepancies to be worked around.
73
Chapter 8. Discussion and Conclusions
8.1 Discussion of Results
8.1.1 Validation of ADS1299 Performance Demonstration Kit
All three types of correlation (Pearson, Lin, ICC) showed that the ADS1299 could produce
frequency domain results very similar to NeuroScan’s SynAmps2 EEG amplifiers. Power
spectra and amplitude spectra for both 4-second and 20-second windows had very high
correlations when the ADS1299 data was averaged and scaled to make the signal amplitudes
roughly the same as the NeuroScan data.
Both 4-second and 20-second windows produced very similar results, with generally the same
mean to at least 2 decimal points. This result shows that both window lengths can produce valid
results at 250 Hz sampling frequency on the ADS1299. While there were some differences in
mean correlation values between amplitude spectra and power spectra, the median values were
very close across both window lengths and over all clinical frequency bands as well as the alpha-
beta range only. The lowest median value across all different correlation coefficients was 0.9608
and the highest was 0.9801. This result suggests that certain windows may have seen errors that
affected the mean much more than the median value.
The ADS1299 appears to acquire and digitize EEG signals with an acceptable fidelity. The
ADS1299 did prove to be somewhat difficult to work with in the proof-of-concept system
development, although this challenge was related to the microcontroller programming involved.
As it is a relatively new chip from Texas Instruments, more development on the documentation
and software side would make it easier to use for EEG applications.
As discussed in section 3.2, the value of using Lin’s correlation and intraclass correlation was
demonstrated by the differences in the values for the raw ADS1299 data compared to the
averaged and scaled data. The Pearson’s correlation coefficients were nearly identical for both
types of data, but the raw ADS1299 data had extremely low Lin’s correlation coefficients and
ICC’s due to an amplitude difference on the order of 1,000,000 (V vs. µV).
The results overall compare very well to the results presented in Chapter 4 of this thesis.
74
8.1.2 Comparison of Emotional Valence Calculation Methods
While many papers regarding emotional valence measurement use frontal alpha-band
asymmetry, they take a wide range of approaches, including support vector machines,
asymmetry indices, and other machine-learning classifications. Some publications also choose
to keep their approaches proprietary as they may be used for marketing or advertising research.
The approach taken here was to compare a number of different combinations of power in the
alpha and beta frequency ranges over the four frontal electrodes being used in the proof-of-
concept system. Equation 1 was taken directly from Ramirez and Vamvakousis [79], and
equations 3 and 4 were versions of the approach used by Brown et al. [20], but the rest were
developed as modifications of those using the F3 / F4 and F7 / F8 electrode pairings alone or in
combination.
The three most accurate equations for classification were based only on alpha-band power, which
does hold with common theories of alpha asymmetry being an appropriate measure of emotional
valence. That any combination of F4 – F3, F8 – F7, or (F4+F8) – (F3+F7) seemed acceptable is
a good result for simple ambulatory monitoring.
The use of a baseline average came after evaluating the calculated emotional valences and
noticing that some subjects had consistently positive or consistently negative measurements,
regardless of their self-assessment scores. Since (in the case of subjects tested at CAMH), there
were baseline EEG readings available, those were used to calculate an average. In the subjects in
the DEAP dataset, an average of all readings for the individual subject was used as there were no
baseline readings. The variance was added to account for slight differences around the average
value. This variance also allowed the approach to be used for all test subjects with a simple
individualized approach. Anecdotally, one subject said that most of the images did not cause any
strong emotion, which was reflected in relatively small changes in measured emotional valence.
Nevertheless, the emotions were still correctly classified on a binary level.
It is a standard EEG procedure to acquire baseline readings, including in Brown et al.’s study on
wireless emotional valence detection [20]. In this way, the emotional valence fluctuations can be
seen more clearly in each individual subject. A simplified binary emotion approach (positive-
neutral and negative) was used due to the limited electrodes and the noted limitations of emotion
elicitation using images only, which was also one of the approaches used in Brown et al. As the
75
results in section 7.2 showed, it was possible to correctly classify the subjects’ classified
emotions using this binary approach in nearly 100% of cases. The use of the variance may
account in some part for the particularly high classification percentage, as a very slightly positive
value for a reported negative emotion could still count as a correct classification, and vice versa.
While this approach may not be perfect because of this slightly soft definition, it seems to
demonstrate that strongly felt emotions would be classified correctly, as a slight difference in
positive or negative in relatively neutral emotions might not be especially noteworthy, especially
in a clinical application.
These results also suggest that using a straightforward emotional valence calculation with the
benefit of a baseline reading might be very useful as a monitoring tool, since strong emotions
could be measured reliably. The results also compare well to the results in Brown et al., which
was the basis of this study. Brown et al. were able to achieve 82% accuracy for classification of
positive, negative and neutral valence while viewing film clips. They achieved up to 85%
accuracy using the same binary emotion approach used here (positive-neutral vs. negative) with
KNN using multiple classifiers using the maximum and kurtosis of maximum of the alpha power
ratio between electrodes F3 and F4. The equations used here are simpler to implement on a
smartphone compared to support vector machines or KNN classifiers. The results also compare
well to the results in Ramirez and Vamvakousis [79], who were able to achieve 80.11%
classification accuracy for positive vs. negative emotional valence in response to sound stimuli.
8.1.3 Proof-of-Concept System Testing
The creation of the wireless EEG acquisition system was a difficult process for the team working
on it (Kevin Tallevi, John Li, Nathaniel Hamming, and myself). The Bluetooth Low Energy
transmission protocol used required interrupt requests to send signal compared to previous
versions of Bluetooth that could be switched on for as long as required. This requirement meant
that the proper interrupt interval had to be found to prevent the Bluetooth transmission from
crashing or shutting down, which created a sampling frequency different than the one being used
by the ADS1299 chip. The data was also transmitted only using its two most significant bytes
(16 bits) instead of the full three bytes created by the chip so that all information for four
channels could be transmitted in the Bluetooth packets.
76
The system was tested first using a low-frequency, low-voltage signal generator to mimic EEG
signals (1-100 Hz at 10-100 µV), which was created by Kevin Tallevi. Once a reasonable signal
could consistently be seen from the signal generator, the system was tested using a standard EEG
cap to look for eye blink peaks and other typical EEG signal qualities. Due to time constraints,
the final system was tested simultaneously with the NeuroScan SynAmps2 system at the same
time as the emotional valence classification was being tested. Anecdotally, certain channels
exhibited unexpected behavior during some of the tests, which is seen most in the Lin’s
correlation coefficients (tables 19-20).
As noted in section 7.3, the correlation between the two systems was not ideal. The most likely
reasons for this are the difference in sampling frequency and the dropping of the third byte of
data. The effective sampling frequency of the wireless system was not absolutely consistent, and
had slight fluctuations. In order to match the data from the NeuroScan system, the data from the
wireless system was processed in MATLAB and an effective average sampling frequency was
calculated based on comparing the length of the time signals and locating the 60 Hz noise signal
with a spectral analysis. Some differences in the sampling frequencies over different windows
may have adversely affected the frequency elements that were being correlated. The data was
resampled in MATLAB, and to avoid some of the signal changes that occur with the resampling
process, the beginning and end of each test (2-4 seconds total) of data was removed on both
systems after the data was aligned properly. A simple analysis on the same test signal with two
bytes and three bytes showed that minimal information loss should have occurred due to this
choice, but it is not impossible that this had an impact.
It should also be emphasized that, while the prototype headset was used for the emotional
valence testing in section 6.4 (shown in figure 20), the standard NeuroScan 64-channel Quik-Cap
was used for the wireless system testing to try to minimize any sources of error. Future testing
should use a separate four-channel headset, though there was not enough time to focus on the
final design and implementation of this headset.
It was worth noting that emotional valence classification showed strong results despite the
differences in data fidelity. This result probably owes a great deal to the use of the baseline
readings in particular, as recurring errors in any channel would be consistent through both the
baseline readings and the subsequent emotional readings. The classification was higher even
77
than that seen in Brown et al’s study, despite the use of images only. This result is probably due
to two-class emotional valence used as well as the individual mean and variance, as discussed
previously.
It should also be noted that the emotional valence calculations and classifications were carried
out in MATLAB during post-processing rather than in real-time on the BlackBerry device. This
approach was due in part to time constraints and in part because of the inconsistencies with the
sampling rate from the Bluetooth device. Most of the time had to be spent producing a reliable
signal on the phone, and the real-time signal processing was kept to a minimum. The application
does calculate the frequency response every 5000 points (20 seconds with a consistent 250 Hz
sampling rate), and can use this data to calculate the emotional valence using all three equations
shown in section 7.4. These results are saved in a database file at this time to be viewed offline,
and are not shown immediately on the BlackBerry.
The emotional self-assessment screen proved to be very easy for the subjects to use during
testing. John Li created it so that it was as simple as moving a slider from -4 to +4
(corresponding to the 1 to 9 scale on the self-assessment manikin), with an image of the SAM
shown above the slider, as in figure 21. At this time, separate user profiles have not been
created, so all self-assessments were saved together, and were compared after testing to the
emotional valence data.
8.2 Limitations and Difficulties Encountered
There were a number of limitations in this study, though there were still quality results in the
end. The primary limitations included:
The lack of time to focus on headset design including the difficulty in designing or
obtaining dry electrodes;
The problems with Bluetooth Low Energy data transmission and the problems with
programming the ADS1299 and using the evaluation software;
Emotional valence testing being limited to a lab setting using image stimuli only.
78
First of all, though a prototype headset was created for some of the tests, it had to be made with
conventional wet electrodes. Since the signal quality of the NeuroScan cap was known, this cap
was used in the analysis of the final wireless system. Some time was spent on the design of
capacitive electrodes, but they were unable to acquire signal of small enough amplitude for EEG.
We also attempted to acquire dry or capacitive electrodes, but they are still a very new
technology. The dry electrodes that were available were priced too high for this project, so a
more standard approach had to be used. As mentioned, Chi et al. noted [62] that significant
focus needs to be given to the electrode design alone to obtain a useable signal. In the future,
this design may be its own project to add on to the wireless EEG acquisition system.
As discussed in the previous section, there were many difficulties encountered while building the
wireless acquisition system. The BTLE data transmission protocol forced a reduction in
sampling rate and bit-rate, and the time it took to resolve these issues to an acceptable level
prevented a more significant testing protocol as well as ways to fix those specific problems. The
ADS1299 also proved difficult to work with on the programming and hardware design side.
While it does incorporate useful EEG-specific features, the ADS1299 also had limited
documentation and support. As the chip was only released in 2012, it is likely for such a young
technology that further development is needed for ideal use. In the future, Kevin Tallevi has
suggested that it might be simpler to create a custom front end with an analog-to-digital
converter and microcontroller that may have fewer inherent difficulties involved.
Finally, the testing protocol was limited to a lab setting with image stimuli. The main reason for
this protocol was the time constraints due to technical difficulties. While the testing did help to
demonstrate the efficacy of the system as a proof-of-concept, it would be very helpful for the
system to be tested outside of a lab setting. This additional testing would evaluate the system
with increased movement as well as real-life emotional stimuli. These stimuli would be harder
to measure for comparison, but the goal of this research is to provide a stepping-stone to a real-
life clinical monitoring application.
79
8.3 Future Directions
This project has a number of future directions that would greatly add to its field of research. In
particular, a dry electrode headset should be developed so that the system can be easily and
comfortably worn on a daily basis without assistance. While wet electrodes provide the best
EEG signal, the system degrades over time and requires gel preparation every time it is worn,
which is messy and somewhat uncomfortable. Dry electrodes with good signal quality would
maintain a consistent signal and would not be difficult to apply properly. Capacitive electrodes
in particular are such a new technology that there is room to focus on the electronics and
shielding design for a wireless system. Also, some research suggests that to read more depth in
emotions rather than just a binary positive-negative scale, a larger electrode montage may be
useful. This requires further testing and research to determine which areas of the brain and
which electrodes can provide meaningful information on specific emotions.
The wireless transmission system may also require refinement. For example, it is possible that
designing an EEG front end from known components rather than using a single chip like the
ADS1299 would provide more flexibility in signal acquisition and analysis. It is also worth
exploring whether another wireless transmission protocol that can handle constant transmission
more simply than BTLE might be useful. A change like this would require some way of
transmitting it to the smartphone, but it is possible that some onboard memory on the acquisition
system could be used for data buffering.
On the software side, future refinements would increase the functionality of the system as a
whole. For example, creating a unique user profile would be helpful, though presumably only
one user would use one system in a real-life setting. It would also be useful to have a setting for
acquiring a baseline reading at the beginning of each wearing period that could be used to create
an average for the emotional valence calculation and classification. A screen for displaying the
user’s emotional valence could provide him or her with useful feedback during the day as well.
On a system diagnostic side, depending on the chips used, it would be a useful feature to
incorporate either an impedance measurement or a lead-off detection setting to ensure that all
electrodes are making sufficient contact to provide good EEG readings.
80
On the testing side, as mentioned in the previous section, it would be helpful to add more testing
modalities. These modalities could include adding sound stimuli, or video stimuli, as well as
scenarios where a user could wear the system at home or at work for an extended period of time.
This testing could be part of a clinical trial when the wireless transmission and headset have been
sufficiently developed. If the system is able to accurately monitor EEG and emotional state, it
may provide a useful tool for ambulatory monitoring of people living with mental illnesses. A
comfortable and unobtrusive headset connected to a smartphone that can be carried in a user’s
pocket would allow them to integrate this monitoring into their daily activities. They could be
provided feedback by the smartphone to promote self-care and self-regulatory behaviour.
Furthermore, if their clinician can be notified of any potential adverse events, more serious
complications and hospitalizations may be avoidable.
Finally, to further improve the emotional recognition, this system could be grown to incorporate
sensors on the autonomic nervous system, such as heart rate, blood pressure, or skin
conductance, which are used in different emotional response studies. This development would
create a more complete MBAN-type system. Alternatively, in an MBAN system, the EEG
sensors could be used alongside other monitoring tools for ECG, blood sugar, and others
depending on the requirements of each individual user. This more complete approach would
provide better non-clinical monitoring for patients who may not be able to readily access a
hospital and to improve patient-care outcomes by detecting problems before they become serious
enough to require hospitalization.
81
8.4 Conclusions
The purpose of this thesis, as stated in section 1.3, was to create and test a proof-of-concept
novel ambulatory EEG system to monitor emotional valence in real-time. This purpose was
mostly achieved, as an ambulatory EEG acquisition and transmission system was created that
sends data to a BlackBerry Z10 smartphone for storage and basic analysis. During testing, users
were able to perform emotional valence self-assessments on the smartphone. Using the acquired
data, the emotional valence of subjects was correctly classified on a positive-neutral vs. negative
basis in post-processing.
In pursuit of the main objective of the thesis, a comparative evaluation of a wireless EEG system
from the imec group to a gold-standard laboratory EEG system was performed prior to the
development of a novel ambulatory system. The novel system was developed using the Texas
Instruments’ ADS1299 EEG front-end chip. This chip was evaluated using the performance
demonstration kit in the same way as the imec system. A number of emotional valence
calculation methods were also compared using the performance demonstration kit, the final
wireless system, and data from the DEAP. Three of these equations using alpha asymmetry in
frontal EEG electrodes (F3 to F4 and F7 to F8) were able to correctly classify the user’s self-
reported emotional valence nearly 100% of the time using a baseline reading as an average.
The wireless acquisition and transmission system was tested using a standard laboratory EEG
cap rather than with a simple ambulatory headset, as dry electrodes proved too costly to acquire
and too difficult to build in the required timeframe. The data also had to be post-processed using
MATLAB as Bluetooth transmission requirements affected the sampling rate of the data to the
point that the emotional valence calculations on the BlackBerry would have been affected. The
quality of the data was not excellent, but the emotional valence classification was still very
successful. Furthermore, while emotional valence was only tested using image stimuli in a lab
setting, the classification was very accurate across all participants.
The work completed is a step towards an ambulatory monitoring system for people living with
mental illnesses. Further refinements to this work may provide a useful tool to monitor and
prevent the escalation of adverse events in mental illnesses to lessen the impact of mental
illnesses to individuals, to the community, and to the economy.
82
8.5 Summary of Contributions
This study was designed to replicate research by Brown et al. [20] by measuring emotional
valence with EEG signals on a mobile system. Several contributions to the research field were
accomplished during this project.
1. This study was one of the first to perform a direct quantitative comparison of a wireless
EEG system to a gold-standard EEG system used in mental health applications. A direct
comparison of two systems using the same cap had not been done in this way before.
Some studies had placed electrodes very close together for two different systems but this
study provided the most direct comparison possible for what would be the same signal
source. This comparison method was repeated for both the ADS1299 performance
demonstration kit and the wireless acquisition and transmission system.
2. This study was also one of the first to compare a wide range of simple emotional valence
equations using a combination of asymmetrical alpha and beta power in frontal
electrodes, particularly F3, F4, F7, and F8. By averaging the emotional valence during
baseline (neutral eyes open and eyes closed) EEG readings, several equations, in
particular [ - ], [ - ], and [ - ] were able to
correctly classify positive-neutral vs negative emotional valence nearly 100% of the time
compared with the subjects’ self-assessments. Other studies used machine learning
algorithms with a number of different signal characteristics, particularly in the alpha
band, but these approaches used simple ratios and differences of absolute power over the
alpha and beta frequency ranges.
3. The study produced an EEG acquisition and transmission system using the ADS1299
EEG front-end for analog-to-digital conversion and multiplexing, and Bluetooth Low
Energy for transmission to a BlackBerry smartphone.
4. This study also produced a BlackBerry application that could acquire and analyze EEG
data to calculate emotional valence given a consistent sampling rate. The application also
allows users to perform self-assessment of emotional valence that can be compared to the
emotional valence measured at different times.
83
References
[1] P. Stewart, T. Lips, C. Lakaski, and P. Upshall, "A Report on Mental Illnesses in Canada,"
Health Canada, Ottawa, Canada, 2002.
[2] P. Stewart and eds, "The Human Face of Mental Health and Mental Illness in Canada,"
Government of Canada, Minister of Public Works and Government Services, Ottawa,
Canada, 2006.
[3] Mental Health Commission of Canada. (2013) About MHCC. [Online].
http://www.mentalhealthcommission.ca/English/who-we-are
[4] Mental Health Commission of Canada, "Making the Case for Investing in Mental Health in
Canada," Mental Health Commission of Canada, Calgary, AB, Document 2013.
[5] J. Trainor, E. Pomeroy, and B. Pape, "A Framework for Support, Third Edition," Canadian
Mental Health Association, Toronto, Canada, 2004.
[6] G.A. Light et al., "Electroencephalography (EEG) and Event-Related Potentials (ERP's)
with Human Participants," Current Protocols in Neurocience (NIH), vol. 6, pp. 1-32, July
2010.
[7] F. Farzan et al., "Evidence for gamma inhibition deficits in the dorsolateral prefrontal
cortex of patients with schizophrenia," Brain, vol. 133, pp. 1505-1514, 2010.
[8] F. Farzan et al., "Reliability of Long-Interval Cortical Inhibition in Healthy Human
Subjects: A TMS-EEG Study," Journal of Neurophysiology, vol. 104, pp. 1339-1346, June
2010.
[9] György Buzsáki, Rhythms of the Brain. Oxford, New York: Oxford University Press, 2006.
[10] C. Wienbruch, "Abnormal slow wave mapping (ASWAM) - A tool for the investigation of
abnormal slow wave activity in the human brain," Journal of Neuroscience Methods, vol.
84
163, pp. 119-127, 2007.
[11] S.R. Sponheim, B.A. Clementz, W.G. Iacono, and M. Beiser, "Clinical and Biological
Concomitants of Resting State EEG Power Abnormalities in Schizophrenia," Biological
Psychiatry, vol. 48, no. 11, pp. 1088-1097, Dec 2000.
[12] B.S. Rockstroh, C. Wienbruch, W.J. Ray, and T. Elbert, "Abnormal oscillatory brain
dynamics in schizophrenia: a sign of deviant communication in neural network?," BMC
Psychiatry, vol. 7, no. 44, 2007.
[13] N.C. Venables, E.M. Bernat, and S.R. Sponheim, "Genetic and Disorder-Specific Aspects
of Resting State EEG Abnormalities in Schizophrenia," Schizophrenia Bulletin, vol. 35,
no. 4, pp. 826-839, 2009.
[14] S. Tot, A. Özge, Ü Çömelekoglu, K. Yazici, and N. Bal, "Association of QEEG Findings
With Clinical Characteristics of OCD: Evidence of Left Frontotemporal Dysfunction,"
Canadian Journal of Psychiatry, vol. 47, no. 6, pp. 538-545, August 2002.
[15] T. Harbert. (2012, June) FCC Gives Medical Body Area Networks Clean Bill of Health.
[Online]. http://spectrum.ieee.org/tech-talk/biomedical/devices/fcc-gives-medical-body-
area-networks-clean-bill-of-health
[16] Federal Communications Commission. (2012, May) Federal Communications
Commission. [Online].
http://transition.fcc.gov/Daily_Releases/Daily_Business/2012/db0529/DOC-314146A1.pdf
[17] Litan, R.E. and the Kaufman Foundation. (2008) Vital Signs via Broadband: Remote
Health Monitoring Transmits Savings. [Online].
https://www.corp.att.com/healthcare/docs/litan.pdf
[18] D.-H. et al Kim, "Epidermal Electronics," Science, vol. 333, pp. 838-843, August 2011.
[19] C. Berka et al., "Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory
Acquired with a Wireless EEG Headset," International Journal of Human-Computer
85
Interaction, vol. 17, no. 2, pp. 151-170, 2004.
[20] L. Brown, B. Grundlehner, and J. Penders, "Towards Wireless Emotional Valence
Detection from EEG," in 33rd Annual International Conference of the IEEE EMBS,
Boston, MA, USA, 2011, pp. 2188-2191.
[21] P.J. Lang, M.M. Bradley, and B.N. Cuthbert, "International affective picture system
(IAPS): Affective Ratings of pictures and instruction manual.," University of Florida,
Gainesville, FL, Technical Report A-8, 2008.
[22] B. et al Büsze, "Ultra low power programmable biomedical SoC for on-body ECG and
EEG processing," in IEEE Asian Solid-State Circuits Conference, Beijing, CN, 2010, pp.
341-344.
[23] G. Jackson et al., "Comparitive Evaluation of an Ambulatory EEG Platform vs. Clinical
Gold Standard," in 35th Annual International Conference of the IEEE Engineering in
Medicine and Biology Society, Osaka, JP, 2013.
[24] J. Malmivuo and R. Plonsey, Bioelectromagnetism. New York, NY: Oxford University
Press, 1995.
[25] International Organization of Societies for Electrophysiological Technology, "Guidelines
for Digital EEG," 1999.
[26] R.F. Yazicioglu, P. Merken, R. Puers, and C. Van Hoof, "Low-Power Low-Noise 8-
Channel EEG Front-End ASIC for Ambulatory Acquisition Systems," in Proceedings of
the 32nd European Solid-State Circuits Conference, Montreux, CH, 2006, pp. 247-250.
[27] R.F. Yazicioglu, P. Merken, R. Puers, and C. Van Hoof, "A 200 μW Eight-Channel EEG
Acquisition ASIC for Ambulatory EEG Systems," IEEE Journal of Solid-State Circuits,
vol. 43, no. 12, pp. 3025-3038, 2009.
[28] S. Filipe et al., "A wireless multichannel EEG recording platform," in 33rd Annual
Conference of the IEEE EMBS, Boston, MA USA, 2011, pp. 6319-6322.
86
[29] D.-G. Kim, K.-S. Hong, and K.-W. Chung, "Implementation of Portable Multi-Channel
EEG and Head Motion Signal Acquisition System," in 8th International Conference on
Computing and Network Technology, Gueongju, KR, 2012, pp. 370-375.
[30] J.-S. Lin and S.-M. Huang, "An FPGA-Based Brain-Computer Interface for Wireless
Electric Wheelchairs," Applied Mechanics and Materials, vol. 284-287, pp. 1616-1621,
2013.
[31] N. Verma et al., "A Micro-Power EEG Acquisition SoC with Integrated Feature Extraction
Processor for a Chronic Seizure Detection System," IEEE Journal of Solid-State Circuits,
vol. 45, no. 4, pp. 804-816, April 2010.
[32] H. Chen, W. Wu, and J. Lee, "A WBAN-based Real-time Electroencephalogram
Monitoring System: Design and Implementation," Journal of Medical Systems, vol. 34, pp.
303-311, 2010.
[33] R. Dilmaghani et al., "Design and Implementation of a Wireless Multi-Channel EEG
Recording," in IEEE, IET International Symposium on Communication Systems, Networks
and Digital Signal Processing, Newcastle, UK, 2010, pp. 741-746.
[34] X. Chen and J. Wang, "Design and Implementation of A Wearable, Wireless EEG
Recording System," in 5th International Conference on Bioinformatics and Biomedical
Engineering, Wuhan, CN, 2011, pp. 1-4.
[35] J. Thie, A. Klistorner, and S.L. Graham, "Biomedical signal acquisition with streaming
wireless communication for recording evoked potentials," Documenta Ophthalmologica,
vol. 125, pp. 149-159, 2012.
[36] Y.T. Wang et al., "Cell-phone Based Drowsiness Monitoring and Management System," in
IEEE Biomedical Circuits and Systems Conference, Hsinchu, TW, 2012, pp. 200-203.
[37] L. Boquete et al., "A portable wireless biometric mullti-channel system," Measurement,
vol. 45, pp. 1587-1598, 2012.
87
[38] M. Sawan et al., "Wireless Recording Systems: From Noninvasive EEG-NIRS to Invasive
EEG Devices," IEEE Transactions on Biomedical Circuits and Systems, vol. 7, no. 2, pp.
186-195, April 2013.
[39] Y. Zhang et al., "A Batteryless 19 uW MICS/ISM-Band Energy Harvesting Body Sensor
Node SoC for ExG Applications," IEEE Journal of Solid-State Circuits, vol. 48, no. 1, pp.
199-213, January 2013.
[40] Emotiv. (2013) Published Papers: EEG. [Online]. http://www.emotiv.com/ideas/eeg.php
[41] NeuroSky. (2012) Academic Papers. [Online].
http://www.neurosky.com/AcademicPapers.aspx
[42] InterAxon. (2010) Muse - FAQ. [Online]. http://interaxon.ca/muse/faq.php
[43] Advanced Brain Monitoring. (2013) X-Series EEG Systems. [Online].
http://advancedbrainmonitoring.com/xseries/
[44] Imec. (2012, October) Imec, Holst Centre and Panasonic Present Wireless Low-power
Active-Electrode EEG Headset. [Online]. http://www2.imec.be/be_en/press/imec-
news/imeceeg2012.html
[45] g.tec. (2013) g.Nautilus - g.tec's wireless EEG system with active electrodes. [Online].
http://www.gtec.at/Products/Hardware-and-Accessories/g.NAUTILUS-Specs-Features
[46] Neuroelectrics. (2011) Features. [Online]. http://www.neuroelectrics.com/enobio/features
[47] P. Tallgren, S. Vanhatalo, K. Kaila, and J. Voipio, "Evaluation of commercially available
electrodes and gels for recording of slow EEG potentials," Clinical Neurophysiology, vol.
116, pp. 799-806, 2005.
[48] B.A. Taheri, R.T. Knight, and R.L. Smith, "A dry electrode for EEG recording,"
Electroencephalography and Clinical Neurophysiology, vol. 90, pp. 376-383, 1994.
88
[49] L. Brown et al., "A low-power, wireless, 8-channel EEG monitoring headset," in 32nd
Annual International Conference of the IEEE EMBS, Buenos Aires, AR, 2010, pp. 4197-
4200.
[50] G. Edlinger, G. Krusz, and C. Guger, "A dry electrode concept for SMR, P300 and SSVEP
based BCIs," in Proceedings of 2012 International Conference on Complex Medical
Engineering, Kobe, JP, 2012, pp. 186-190.
[51] N.S. Dias, J.P. Carmo, P.M. Mendes, and J.H. Correia, "Wireless instrumentation system
based on dry electrodes for acquiring EEG signals," Medical Engineering & Physics, vol.
34, pp. 972-981, 2012.
[52] P. Salvo et al., "A 3D printed dry electrode for ECG/EEG recording," Sensors and
Actuators A: Physical, vol. 174, pp. 96-102, 2012.
[53] L.-D. Liao, I.-J. Wang, S.-F. Chen, J.-Y. Chang, and C.-T. Lin, "Design, Fabrication and
Experimental Validation of a Novel Dry-Contact Sensor for Measuring
Electroencephalography Signals without Skin Preparation," Sensors, vol. 11, pp. 5819-
5834, 2011.
[54] V. Mihajlovic, G.G. Molina, and J. Peuscher, "To what extent can dry and water-based
EEG electrodes replace conductive gel ones?," in Biodevices, Vilamoura, PT, 2012, pp. 14-
26.
[55] Y.M. Chi, T.-P. Jung, and G. Cauwenberghs, "Dry-Contact and Noncontact Biopotential
Electrodes: Methodological Review," IEEE Reviews in Biomedical Engineering, vol. 3, pp.
106-119, 2010.
[56] J. Löfhede, F. Seoane, and M. Thordstein, "Soft Textile Electrodes for EEG Monitoring,"
in 10th IEEE International Conference on Information Technology and Applications in
Biomedicine (ITAB), Corfu, GR, 2010, pp. 1-4.
[57] H.J. Bake, H.J. Lee, Y.G. Lim, and K.S. Park, "Investigations of capacitively-coupled EEG
89
electrode for use in brain-computer interface," in IEEE International Conference on
Systems, Man, and Cybernetics, Seoul, KR, 2012, pp. 278-282.
[58] T.J. Sullivan, S.R. Deiss, and G. Cauwenberghs, "A Low-Noise, Non-Contact EEG/ECG
Sensor," in Biomedical Circuits and Systems Conference, Montreal, QC, 2007, pp. 154-
157.
[59] Y.M. Chi and G. Cauwenberghs, "Micropower Non-contact EEG Electrode with Active
Common-Mode Noise Suppression and Input Capacitance Cancellation," in 31st Annual
International Conference of the IEEE EMBS, Minneapolis, MN, USA, 2009, pp. 4218-
4221.
[60] Y.M. Chi et al., "Wireless Non-contact Cardiac and Neural Monitoring," in Wireless
Health, San Diego, CA, 2010, pp. 15-23.
[61] Y.M. Chi, C. Maier, and Cauwenberghs, "Integrated Ultra-High Impedance Front-end for
Non-contact Biopotential Sensing," in IEEE Biomedical Circuits and Systems Conference,
San Diego, CA, 2011, pp. 456-459.
[62] Y.M. Chi et al., "Dry and Noncontact EEG Sensors for Mobile Brain-Computer
Interfaces," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol.
20, no. 2, pp. 228-235, March 2012.
[63] J.A. Russell, "A Circumplex Model of Affect," Journal of Personality and Social
Psychology, vol. 39, no. 6, pp. 1161-1178, 1980.
[64] J.A. Russell and L.F. Barrett, "Core Affect, Prototypical Emotional Episodes, and Other
Things Called Emotion: Dissecting the Elephant," Journal of Personality and Social
Psychology, vol. 76, no. 5, pp. 805-819, 1999.
[65] J.A. Russell, "Core Affect and the Psychological Construction of Emotion," Psychological
Review, vol. 110, no. 1, pp. 145-172, 2003.
[66] J. Posner, J.A. Russell, and B.S. Peterson, "The circumplex model of affect: An integrative
90
approach to affective neuroscience, cognitive development and psychopathology,"
Development and Psychopathology, vol. 17, pp. 715-734, 2005.
[67] M. Yik, J.A. Russell, and J.H. Steiger, "A 12-Point Circumplex Structure of Core Affect,"
Emotion, vol. 11, no. 4, pp. 705-731, 2011.
[68] G. Colombetti, "Appraising Valence," Journal of Consciousness Studies, vol. 12, no. 8-10,
pp. 103-126, 2005.
[69] M.M. Bradley and P.J. Lang, "Measuring emotion: The self-assessment manikin and the
semantic differential," Journal of Behavior Therapy and Experimental Psychiatry, vol. 25,
no. 1, pp. 49-59, 1994.
[70] J.D. Morris, "Observations: SAM: The self-assessment manikin," Journal of Advertising
Research, vol. 35, pp. 63-68, 1995.
[71] N.T. Alves, S.S. Fukusima, and J.A. Aznar-Casanova, "Models of Brain Asymmetry in
Emotional Processing," Psychology & Neuroscience, vol. 1, no. 1, pp. 63-66, 2008.
[72] Queensland Government. (2012, March) Acquired Brain Injury: Brain Map. [Online].
http://www.health.qld.gov.au/abios/asp/bfrontal.asp
[73] R.J. Davidson, "Anterior Cerebral Asymmetry and the Nature of Emotion," Brain and
Cognition, vol. 20, pp. 125-151, 1992.
[74] R.J. Davidson and W. Irwin, "The functional neuroanatomy of emotion and affective
style," Trends in Cognitive Sciences, vol. 3, no. 1, pp. 11-21, January 1999.
[75] R.J. Davidson, "Affective neuroscience and psychophysiology: Toward a synthesis,"
Psychophysiology, vol. 40, pp. 655-665, 2003.
[76] R.J. Davidson, "What does the prefrontal cortex "do" in affect: perspectives on frontal
EEG asymmetry research," Biological Psychology, vol. 67, pp. 219-233, 2004.
91
[77] T. Colibazzi et al., "Neural Systems Subserving Valence and Arousal During the
Experience of Induced Emotions," Emotion, vol. 10, no. 3, pp. 377-389, 2010.
[78] I.B. Mauss and M.D. Robinson, "Measures of emotion: A review," Cognition & Emotion,
vol. 23, no. 2, pp. 209-237, February 2009.
[79] R. Ramirez and Z. Vamvakousis, "Detecting Emotion from EEG Signals Using the
Emotive Epoc Device," in Brain Informatics, Macau, China, 2012, pp. 175-184.
[80] L.A. Schmidt and L.J. Trainor, "Frontal brain electrical activity (EEG) distinguishes
valence and intensity of musical emotions," Cognition and Emotion, vol. 15, no. 4, pp.
487-500, 2001.
[81] M.N. Rusalova and M.B. Kostyunina, "Spectral Correlation Studies of Emotional States in
Humans," Neuroscience and Behavioral Physiology, vol. 34, no. 8, pp. 803-808, 2004.
[82] I. Winkler, M. Jäger, V. Mihajlovic, and T. Tsoneva, "Frontal EEG Asymmetry Based
Classification of Emotional Valence using Common Spatial Patterns," World Academy of
Science, Engineering and Technology, vol. 45, pp. 373-378, 2010.
[83] P.C. Petrantonakis and L.J. Hadjileontiadis, "A Novel Emotion Elicitation Index Using
Frontal Brain Asymmetry for Enhanced EEG-Based Emotion Recognition," IEEE
Transactions on Information Technology in Biomedicine, vol. 15, no. 5, pp. 737-746,
September 2011.
[84] T. Schuster, S. Gruss, S. Rukavina, S. Walter, and H.C. Traue, "EEG-based Valence
Recognition: What do we Know About the influence of Individual Specificity," in The
Fourth International Conference on Advanced Cognitive Technologies and Applications,
Nice, FR, 2012, pp. 71-76.
[85] V. Knott, C. Mahoney, S. Kennedy, and K. Evans, "EEG power, frequency, asymmetry
and coherence in male depression," Psychiatry Research: Neuroimaging, vol. 106, pp.
123-140, 2001.
92
[86] D. Mathersul, L.M. Williams, P.J. Hopkinson, and A.H. Kemp, "Investigating Models of
Affect: Relationships Among EEG Alpha Asymmetry, Depression, and Anxiety,"
Emotion, vol. 8, no. 4, pp. 560-572, 2008.
[87] U. Herwig et al., "Neural correlates of 'pessimistic' attitude in depression," Psychological
Medicine, vol. 40, pp. 789-800, 2010.
[88] A.H. Kemp et al., "Disorder specificity despite comorbidity: Resting EEG alpha
asymmetry in major depressive disorder and post-traumatic stress disorder," Biological
Psychology, vol. 85, pp. 350-354, 2010.
[89] G. Rosenblau et al., "Functional neuroanatomy of emotion processing in major depressive
disorder is altered after successful antidepressant therapy," Journal of
Psychopharmacology, vol. 26, no. 11, pp. 1424-1433, 2012.
[90] N.A. Groenewold, E.M. Opmeer, P. de Jonge, A. Aleman, and S.G. Costafreda,
"Emotional valence modulates brain functional abnormalities in depression: Evidence from
a meta-analysis of fMRI studies," Neuroscience and Biobehavioral Reviews, vol. 37, pp.
152-163, 2013.
[91] S.A. Shenkman et al., "Resting Electroencephalogram Asymmetry and Posttraumatic
Stress Disorder," Journal of Traumatic Stress, vol. 21, no. 2, pp. 190-198, April 2008.
[92] E.S. Grosh, N.M. Docherty, and B.E. Wexler, "Abnormal laterality in schizophrenics and
their parents," Schizophrenia Research, vol. 14, pp. 155-160, 1995.
[93] J.A. Burbridge and D.M. Barch, "Emotional Valence and Reference Disturbance in
Schizophrenia," Journal of Abnormal Psychology, vol. 111, no. 1, pp. 186-191, 2002.
[94] L.K. Phillips, M.M. Voglmaier, and P.J. Deldin, "A preliminary study of emotion
processing interference in schizophrenia and schizoaffective disorder," Schizophrenia
Research, vol. 94, pp. 207-214, 2007.
93
[95] M. Lepage et al., "Emotional face processing and flat affect in schizophrenia: functional
and structural neural correlates," Psychological Medicine, vol. 41, pp. 1833-1844, 2011.
[96] M.N. Pavuluri, M.M. O'Connor, E.M. Harral, and J.A. Sweeney, "An fMRI study of the
interface between affective and cognitive neural circuitry in pediatric bipolar disorder,"
Psychiatry Research: Neuroimaging, vol. 162, pp. 244-255, 2008.
[97] V. Drago et al., "Emotional Indifference in Alzheimer's Disease," Journal of
Neuropsychiatry and Clinical Neuroscience, vol. 22, no. 2, pp. 236-242, Spring 2010.
[98] F. Schiffer et al., "Determination of hemispheric emotional valence in individual subjects:
A new approach with research and therapeutic implications," Behavioral and Brain
Functions, vol. 3, no. 13, March 2007.
[99] E. Flo et al., "Transient changes in frontal alpha asymmetry as a measure of emotional and
physical distress during sleep," Brain Research, vol. 1367, pp. 234-249, 2011.
[100] D. Maus, C.M. Epstein, and S.T. Herman, "Digital EEG," in Niedermeyer's
Electroencephalography: Basic Principles, Clinical Applications, and Related Fields, 6th
ed., D.L. Schomer and F.H.L. da Silva, Eds.: Lippincott Williams & Wilkins, 2011, ch. 7,
pp. 119-141.
[101] American Clinical Neurophysiology Society, "Guideline 8: Guidelines for Recording
Clinical EEG on Digital Media," Journal of Clinical Neurophysiology, vol. 23, no. 2, pp.
122-124, April 2006.
[102] H. Jasper, "Appendix: The ten-twenty electrode system of the International Federation,"
Electroencephalography and Clnical Neurophysiology, vol. 10, pp. 371-375, 1958.
[103] American Clinical Neurophysiology Society, "Guideline 5: Guidelines for Standard
Electrode Position Nomenclature," Journal of Clinical Neurophysiology, vol. 23, no. 2, pp.
107-110, April 2006.
94
[104] American Clinical Neurophysiology Society, "Guideline 6: A Proposal for Standard
Montages to Be Used in Clinical EEG," Journal of Clinical Neurophysiology, vol. 23, no.
2, pp. 111-117, April 2006.
[105] R.T. Pivik et al., "Guidelines for the recording and quantitative analysis of
electroencephalographic activity in research contexts," Psychopyhsiology, vol. 30, pp. 547-
558, 1993.
[106] M. Teplan, "Fundamentals of EEG Measurement," Measurement Science Review, vol. 2,
no. 2, pp. 1-11, 2002.
[107] W. van Drongelen, Signal Processing for Neuroscientists. London, UK: Academic Press,
2010.
[108] A.V. Oppenheim and R.W. Schafer, Discrete-Time Signal Processing, 3rd ed. Upper
Saddle River, NJ, USA: Prentice Hall, 2010.
[109] P. Bucci, A. Mucci, and S. Galderisi, "Normal EEG Patterns and Waveforms," in Standard
Electroencephalography in Clinical Psychiatry: A Practical Handbook, N. Boutros et al.,
Eds.: John Wiley & Sons, Ltd, 2011, ch. 4, pp. 33-57.
[110] B.S. Chang, D.L. Schomer, and E. Niedermeyer, "Normal EEG and Sleep: Adults and
Elderly," in Niedermeyer's Electroencephalography: Basic Principles, Clinical
Applications, and Related Fields, 6th ed., D.L. Schomer and F.H.L. da Silva, Eds.:
Lippincott Williams & Wilkins, 2011, ch. 11, pp. 183-214.
[111] St. Gudmundsson, T.P. RUnarsson, S. SIgurdsson, G. Eiriksdottir, and K. Johnsen,
"Reliability of quantitative EEG features," Clinical Neurophysiology, vol. 118, pp. 2162-
2171, 2007.
[112] NeuroSky. (2009, December) Brain Wave Signal (EEG) of NeuroSky, Inc. [Online].
http://www.neurosky.com/AcademicPapers.aspx
95
[113] R. et al Matthews, "Novel Hybrid Bioelectrodes for Ambulatory Zero-Prep EEG
Measurements Using Multi-Channel Wireless EEG System," in Foundations of Augmented
Cognition: Lecture Notes in Computer Science, vol. 4565. Heidelberg, Germany: Springer
Berlin Heidelberg, 2007, pp. 137-146.
[114] R. Taylor, "Interpretation of the Correlation Coefficient: A Basic Review," Journal of
Diagnostic Medical Sonography, vol. 1, pp. 35-39, January/February 1990.
[115] A. Leon-Garcia, Probability, Statistics, and Random Processes for Electrical Engineering,
3rd ed. Upper Saddle River, NJ, USA: Pearson Education, Inc., 2009.
[116] P.F. Watson and A. Petrie, "Method agreement analysis: A review of correct
methodology," Theriogenology, vol. 73, pp. 1167-1179, 2010.
[117] K.O. McGraw and S.P. Wong, "Forming Inferences About Some Intraclass Correlation
Coefficients," Psychological Methods, vol. 1, no. 1, pp. 30-46, 1996.
[118] A. Indrayan, "Clinical Agreement in Quantitative Measurements," in Methods of Clinical
Epidemiology, S.A.R. Doi and G.M. Williams, Eds. Berlin Heidelberg, DE: Springer-
Verlag, 2013, ch. 2, pp. 17-27.
[119] A. Salarian. (2008, Nov) Intraclass Correlation Coefficient - Mathworks File Exchange.
[Online]. http://www.mathworks.com/matlabcentral/fileexchange/22099-intraclass-
correlation-coefficient-icc
[120] S. Patki, B. Grundlehner, T. Nakada, and J. Penders, "Low Power Wireless EEG headset
for BCI Applications," in 14th International Conference, HCI International, Orlando, FL,
2011, pp. 481-490.
[121] NeuroScan. (2012, November) Peer-Reviewed Articles by Solution. [Online].
http://www.neuroscan.com/articles.cfm
[122] Texas Instruments, Low-Noise, 8-Channel, 24-Bit Analog Front-End for Biopotential
96
Measurements, August 2012.
[123] Texas Instruments, 2.4-GHz Bluetooth™ low energy and Proprietary System-on-Chip,
June 2013.
[124] Analog Devices, Dual-Channel, 5 kV Isolators with Integrated DC-to-DC Converter, June
2012.
[125] BlackBerry. (2013) BlackBerry Z10 Specs. [Online].
http://ca.blackberry.com/smartphones/blackberry-z10/specifications.html
[126] (2013) Kiss FFT. [Online]. http://sourceforge.net/projects/kissfft/
[127] S. Koelstra et al., "DEAP: A Database for Emotion Analysis using Physiological Signals,"
IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18-31, January-March 2012.
97
Appendices
Appendix 1 - Register Settings for ADS1299
Register Address Value D7 D6 D5 D4 D3 D2 D1 D0
ID 0x00 0x3E 0 0 1 1 1 1 1 0
CONFIG1 0x01 0x96 1 0 0 1 0 1 1 0
CONFIG2 0x02 0xC0 1 1 0 0 0 0 0 0
CONFIG3 0x03 0xFC 1 1 1 1 1 1 0 0
LOFF 0x04 0x00 0 0 0 0 0 0 0 0
CH1SET 0x05 0x50 0 1 0 1 0 0 0 0
CH2SET 0x06 0x50 0 1 0 1 0 0 0 0
CH3SET 0x07 0x50 0 1 0 1 0 0 0 0
CH4SET 0x08 0x50 0 1 0 1 0 0 0 0
CH5SET 0x09 0xD0 1 1 0 1 0 0 0 0
CH6SET 0x0A 0xD0 1 1 0 1 0 0 0 0
CH7SET 0x0B 0xD0 1 1 0 1 0 0 0 0
CH8SET 0x0C 0xD0 1 1 0 1 0 0 0 0
BIASSENSP 0x0D 0x06 0 0 0 0 0 1 1 0
BIASSENSN 0x0E 0x02 0 0 0 0 0 0 1 0
LOFF_SENSP 0x0F 0x00 0 0 0 0 0 0 0 0
LOFF_SENSN 0x10 0x00 0 0 0 0 0 0 0 0
LOFF_FLIP 0x11 0x00 0 0 0 0 0 0 0 0
LOFF_STATP 0x12 0x00 0 0 0 0 0 0 0 0
LOFF_STATN 0x13 0x00 0 0 0 0 0 0 0 0
GPIO 0x14 0x00 0 0 0 0 0 0 0 0
MISC1 0x15 0x20 0 0 1 0 0 0 0 0
MISC2 0x16 0x00 0 0 0 0 0 0 0 0
CONFIG4 0x17 0x00 0 0 0 0 0 0 0 0
98
Appendix 2 - IAPS Images Used in Emotional Valence Study
Included below are the IAPS image identifications of the images used in the emotional valence
portion of this thesis. Valmn is the mean valence score reported by the IAPS, Valsd is standard
deviation of the valence score. Aromn is the mean arousal score, and arosd is the standard
deviation of arousal. Though it was not used in this study, dom1mn and dom1sd refer to the
dominance score of the images.
The images are grouped as follows: High valence, high arousal; high valence, low arousal; low
valence, high arousal, low valence, low arousal, medium valence, medium arousal.
desc IAPS valmn Valsd aromn arosd dom1mn dom1sd
Jaguar 1650 6.65 2.25 6.23 1.99 4.29 1.99
Astronaut 5470 7.35 1.62 6.02 2.26 4.96 2.47
Cupcakes 7405 7.38 1.73 6.28 2.16 5.67 2.4
Sailboat 8170 7.63 1.34 6.12 2.3 5.72 2.15
Skier 8190 8.1 1.39 6.28 2.57 6.14 2.74
WaterSkier 8200 7.54 1.37 6.35 1.98 6.17 1.61
Rafting 8370 7.77 1.29 6.73 2.24 5.37 2.02
Gymnast 8470 7.74 1.53 6.14 2.19 6.17 2.09
RollerCoaster 8490 7.2 2.35 6.68 1.97 5.37 2.46
Rollercoaster 8499 7.63 1.41 6.07 2.31 . .
PolarBears 1441 7.97 1.28 3.94 2.38 . .
Gannet 1450 6.37 1.62 2.83 1.87 6.75 1.87
Rabbit 1610 7.82 1.34 3.08 2.19 6.77 2.19
Antelope 1620 7.37 1.56 3.54 2.34 6.82 2.34
Binoculars 2314 7.55 1.24 4 2.01 6.17 1.78
Flower 5010 7.14 1.5 3 2.25 7.4 2.25
Nature 5780 7.52 1.45 3.75 2.54 6.05 2.3
Clouds 5891 7.22 1.46 3.29 2.57 5.2 2.57
IceCream 7340 6.68 1.63 3.69 2.58 6.32 2.33
Violin 7900 6.5 1.72 2.6 2.08 6.48 2.22
Snake 1052 3.5 1.87 6.52 2.23 3.36 2.26
Gun 2811 2.17 1.38 6.9 2.22 . .
AimedGun 6200 2.71 1.58 6.21 2.28 3.35 2.28
Attack 6370 2.7 1.52 6.44 2.19 3 1.87
Attack 6510 2.46 1.58 6.96 2.09 2.81 2.12
Shipwreck 9620 2.7 1.64 6.11 2.1 3.29 1.95
Bomb 9630 2.96 1.72 6.06 2.22 2.98 2.13
CarAccident 9904 2.39 1.36 6.08 2.06 3.4 2.21
CarAccident 9910 2.06 1.26 6.2 2.16 3.02 1.89
99
Fire 9921 2.04 1.47 6.52 1.94 3.57 2.41
Woman 2039 3.65 1.44 3.46 1.94 5.06 1.85
Woman 2399 3.69 1.4 3.93 2.01 . .
Man 2490 3.32 1.82 3.95 2 4.72 2.03
ElderlyWoman 2590 3.26 1.92 3.93 1.94 4.31 2.14
Jail 2722 3.47 1.65 3.52 2.05 5.34 2.34
Jail 6010 3.73 1.98 3.95 1.87 5.08 2.53
Bucket 7078 3.79 1.45 3.69 1.86 5.41 1.73
Cemetery 9001 3.1 2.02 3.67 2.3 3.47 1.9
Puddle 9110 3.76 1.41 3.98 2.23 4.88 1.68
Cemetery 9220 2.06 1.54 4 2.09 3.13 1.97
Wolf 1645 4.99 1.64 5.14 1.99 4.74 1.91
TongueOut 2122 5.15 1.82 4.59 1.91 5.49 1.81
MaleFace 2220 5.03 1.39 4.93 1.65 5.32 1.77
Soldiers 2704 4.85 1.89 5.3 2.16 . .
Actor 2780 4.77 1.76 4.86 2.05 . .
Coach 3550.2 4.92 1.62 5.13 2.24 5.38 2.02
Stove 7077 5.12 1.46 4.61 2.06 5.6 1.85
Ramen 7476 4.99 2.24 4.63 2.02 5.45 1.99
Crowd 7497 5.19 1.55 4.97 2.16 4.26 2.1
Battleship 9422 4.95 1.72 5.09 1.92 4.89 2.25
100
Appendix 3 - Screening Form for EEG Study Participants at CAMH
TMS - Screening and Demographic Information Form
Completed by: Date:
Subject Name: Subject #:
Gender: Male Female DOB: _____/_____/_____ Age: ________
d m y
Status: Control Patient Diagnosis: _____________________
Address: ____________________________________________________________
____________________________________________________________
____________________________________________________________
Telephone # - Home: ______________________ Work: ___________________
Primary Language: Other Languages:
DRUG USAGE? SMOKER?
ETHNICITY:
Type of Education: 1 – grade 6 or less
2 – grade 7 to 12 (w/o completing high school), 3 – graduated high school
4 – part college
5 – graduated 2 year college
6 – graduated 4 year college
7 – part graduate / professional school
8 – completed graduate / professional school
Years of Education:
Current Main Occupation and for How Long:
Father’s Highest Education (see prev. scale): Occupation:
Mother’s Highest Education (see prev. scale): Occupation:
101
Handedness:
L R
1. Writing
2. Drawing
2. Throwing
3. Scissors
4. Toothbrush
5. Knife (without fork)
6. Spoon
7. Broom (upper hand)
8. Striking a match
Kick a ball
Eye when using only one
+, ++ when preference is so strong that you would never try to use the other hand unless
absolutely forced to
MEDICAL HISTORY
I would like to ask you some questions about your health.
1. Weight: __________ Height: __________
2. Have you ever been hit on the head and lost consciousness for more than one hour?
(probe for when head injury occurred and if hospitalization was required, how long PTA
lasted, etc.) Y N
If yes, describe: ________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
102
3. Do you have a history of (check ALL that apply):
Seizures ______ Stroke ______
Hypertension ______ Diabetes ______
Heart Attack ______ Thyroid Disease ______
Pulmonary ______ Allergies ______
Other (specify): ________________________________________________________________
______________________________________________________________________________
Details (explain): _______________________________________________________________
Are you pregnant?
Are you currently on birth control?
4. Have you ever seen a psychologist, psychiatrist, clinical social worker, or other mental
health professional? Y N
Under what circumstances? _______________________________________________________
______________________________________________________________________________
Is there a history of mental illness in your family? Y N
Which illness and what family member?_____________________________________________
______________________________________________________________________________
Have you been hospitalized for psychiatric reasons? Y N
If yes, roughly how many times? 1 2 3 4 5 6 ___
Where and when were you admitted?
Date Reason/Dx In/Out Pt. Length Tx
Emerg?
___/___/___ _____________________ _________ _________________ _________
d m y
103
___/___/___ _____________________ _________ _________________ _________
d m y
___/___/___ _____________________ _________ _________________ _________
d m y
___/___/___ _____________________ _________ _________________ _________
d m y
___/___/___ _____________________ _________ _________________ _________
d m y
___/___/___ _____________________ _________ _________________ _________
d m y
5. Do you need/use glasses for reading? Y N
6. How many times have you seen a doctor in the past year (excluding times/ hospitalized)?
___________
Dr. Reason Tx
_______________________ _____________________________ _______________________
_______________________ _____________________________ _______________________
_______________________ _____________________________ _______________________
7. What medicines are you currently taking (prescription and non-prescription)?
Time on Change
Name Reason Dose How Often Medication in Dose
_________________ _________________ _____ ___________ _______________________
_________________ _________________ _____ ___________ _______________________
_________________ _________________ _____ ___________ _______________________
_________________ _________________ _____ ___________ _______________________
_________________ _________________ _____ ___________ _______________________
_________________ _________________ _____ ___________ ___________ ___________
104
Contraindications to Magnetic Exposure
No If Yes, Explain
Surgical clips in the brain
Cardiac pacemaker OR valves
Cochlear implant
Metal rods, plates, screws, or nails
Shrapnel/metal fragments in head/eyes/body
Dentures
Have you ever had an adverse reaction to TMS?
Have you ever had an EEG?
Do you suffer from frequency or severe headaches?
Have you ever had any other brain-related condition?
Have you ever had any illness that caused brain injury?
Does anyone in your family have epilepsy?
Do you need further explanation of TMS and its associated risks?