Embed Size (px)
Citation preview
IMPROVING ATTENTION FOR LEARNING TASKS WITH QUANTIFIED-SELF
FEEDBACK FROM A BRAIN-COMPUTER INTERFACE
By
MARVIN ANDUJAR
A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
UNIVERSITY OF FLORIDA
2017
© 2017 Marvin Andujar
To my parents for teaching me the value of education, hard work, risk taking, and for laying the
path of success for my myself and my sisters
4
ACKNOWLEDGMENTS
My research journey started in my undergraduate career, and there were different groups
at my undergraduate institution that contributed towards my research training. I thank the Ronald
E. McNair Program and LSAMP for providing funding and advice on how to get accepted into
graduate school. I also want to thank my undergraduate advisor, Dr. Patricia Morreale for
providing constant advice and support through my five years of undergraduate.
I want to thank my Ph.D. advisor, Dr. Juan E. Gilbert for selecting me as one of his Ph.D.
students and for believing in my potential to complete this degree. He was the main person to
encourage me to apply for the NSF fellowship as he strongly believed, I could receive the award.
Also, he has provided a vast amount of advice on pursuing the Professor career route, and I know
he will continue advising me throughout my career journey. I thank my committee members,
Anton Nijholt, Shaundra Daily, Jesse Dallery, and Kristy Boyer for agreeing to be a committee
member and to provide feedback to my work.
My parents are the source of my diligence, motivation, and willingness to pursue a
doctoral degree. They made their life purpose of making sure I get an education and have a better
life that they could not have when they were younger. Their sacrifices, hard work, and
overcoming adversity results to me completing this dissertation and obtaining a Ph.D. I thank
them for teaching me the value of hard work and discipline, which helped me become the first
person in my entire family lineage to become a Doctor.
Lastly, I want to thank NSF, Intel, GEM and Google for funding my Ph.D. work.
5
TABLE OF CONTENTS
page
ACKNOWLEDGMENTS ...............................................................................................................4
LIST OF TABLES ...........................................................................................................................7
LIST OF FIGURES .........................................................................................................................9
LIST OF ABBREVIATIONS ........................................................................................................11
ABSTRACT ...................................................................................................................................13
CHAPTER
1 INTRODUCTION ..................................................................................................................15
2 BACKGROUND ....................................................................................................................20
Brain-Computer Interfaces .....................................................................................................20 BCI Cycle ........................................................................................................................21 BCI Society and Enhance Roadmap ................................................................................24 Engagement .....................................................................................................................26
Personal Informatics/Quantified-Self .....................................................................................31 Reflective Thinking ................................................................................................................34
3 ATTENTION VISUALIZATION ..........................................................................................41
Experimental Design ..............................................................................................................41 Participants .............................................................................................................................42 Procedure ................................................................................................................................42 Results and Discussion ...........................................................................................................43
4 ENGAGEMENT APPLICATION PROTOTYPE .................................................................62
Brainstorming .........................................................................................................................62 Low-Fidelity ...........................................................................................................................62 High-Fidelity ...........................................................................................................................62 Application Architecture.........................................................................................................63 User-Mobile Interaction ..........................................................................................................64
5 QUANTIFIED-SELF ATTENTION FEEDBACK ................................................................69
6 CONCLUSION .....................................................................................................................119
6
APPENDIX
A VISUALIZATIONS CREATED BY PARTICIPANTS IN FOCUS GROUP ....................122
B FOCUS GROUP DEMOGRAPHIC SURVEY ....................................................................128
C FOCUS GROUP FIRST TASK ............................................................................................130
D FOCUS GROUP SECOND TASK ......................................................................................131
E FOCUS GROUP THIRD TASK ..........................................................................................134
F QUANTFIED-SELF ATTENTION FEEDBACK SURVEY ..............................................135
G INTERVIEW QUESTIONS .................................................................................................151
H GLOSSARY .........................................................................................................................152
LIST OF REFERENCES .............................................................................................................154
BIOGRAPHICAL SKETCH .......................................................................................................160
7
LIST OF TABLES
Table page
2-1 Wireless BCI apparatus specifications...............................................................................38
2-2 EEG bands descriptions .....................................................................................................40
3-1 Focus group steps ...............................................................................................................58
3-2 Visualization: descriptive statistics ....................................................................................60
3-3 Visualizations: Wilcoxon Signed-Rank test results. Significant values are noted with
the symbol * .......................................................................................................................61
3-4 Scales: descriptive statistics ...............................................................................................61
3-5 Scales: Wilcoxon Signed-Rank test results. Significant values are noted with the
symbol * .............................................................................................................................61
5-1 aBCI/HCI EEG studies with sample sizes, number of conditions, and number of
channels used in the study..................................................................................................71
5-2 Groups descriptions ...........................................................................................................75
5-3 Positive and Negative Affect Schedule (PANAS) - descriptive statistics for each
group. PA = Positive Affect, NA = Negative Affect .......................................................109
5-4 Positive and Negative Affect Schedule (PANAS) - descriptive statistics for each
affect per group. PA = positive affect, NA = negative affect ..........................................110
5-5 Positive and Negative Affect Schedule (PANAS) – Cronbach’s Alpha overall result.
PA = Positive Affect, NA = Negative Affect ..................................................................111
5-6 Positive and Negative Affect Schedule (PANAS) - Cronbach’s Alpha for each affect
per group. PA = Positive Affect, NA = Negative Affect .................................................111
5-7 Descriptive statistics of the quiz scores ...........................................................................111
5-8 Quiz understanding and clearness ....................................................................................112
5-9 Descriptive statistics of the quiz completion time. The time is represented in seconds ..112
5-10 Descriptive statistics of Engagement indices for all the groups ......................................113
5-11 Descriptive statistics of engagement indices for control group (NFNF) .........................113
5-12 Descriptive statistics of engagement indices for feedback-no feedback group (FNF) ....113
8
5-13 Descriptive statistics of engagement indices for no feedback-feedback group (NFF) ....114
5-14 Descriptive statistics of engagement indices for feedback-feedback group (FF) ............114
5-15 Multivariate analysis of variance results of attention levels effect on quiz scores ..........114
5-16 Multivariate analysis of variance results of attention levels effect on quiz scores ..........115
5-17 Correlation between quiz scores and attention levels. P-value with the symbol *
represents statistical significance .....................................................................................116
9
LIST OF FIGURES
Figure page
2-1 BCI areas specified by quadrants. ......................................................................................37
2-2 BCI Cycle introduced by Wolpaw. ....................................................................................37
2-3 Non-Invasive Wireless Ubiquitous EEG BCIs ..................................................................38
2-4 BCI Society Roadmap for Enhance Application. ..............................................................39
2-5 Roadmap Legend ...............................................................................................................40
3-1 Raw EEG from Emotiv test bench .....................................................................................57
3-2 Focus group’s room design ................................................................................................57
3-3 Bar graph ............................................................................................................................58
3-4 Filling meter .......................................................................................................................58
3-5 Bulls eye.............................................................................................................................59
3-6 Light bulb ...........................................................................................................................59
3-7 Eyes ....................................................................................................................................59
3-8 Speedometer .......................................................................................................................60
3-9 Line graph ..........................................................................................................................60
4-1 Low-fidelity wireframes ....................................................................................................66
4-2 High-fidelity wireframes ....................................................................................................66
4-3 Focus group’s room design Engagement application prototype architecture ....................67
4-4 EEG channel good signal qualities from Emotiv application except for channel Pz ........67
4-5 Bar Graphs used in the mobile application to showcase attention levels ..........................68
5-1 Five channels Bluetooth-based non-invasive BCI device – Emotiv Insight......................74
5-2 Emotiv channel locations in the 10-20 international system identified by red circles ......74
5-3 Study steps .........................................................................................................................75
5-4 Collection methods used to collect data. Adopted from [68] ............................................75
10
5-5 Study steps Scatter plot of quiz 2 and total time taken to complete the quiz ..................112
11
LIST OF ABBREVIATIONS
aBCI Affective Brain-Computer Interfaces
ADHD Attention-deficit/hyperactivity disorder
BCI Brain-Computer Interfaces
CNS Central Nervous System
CPT Continuous Performance Test
ECoG Electrocorticogram
EEG Electroencephalography
EMG Electromyography
EOG Electrooculography
ERP Event-Related Potentials
FFT Fast Fourier Transform
fMRI Functional Magnetic Resonance Imaging
FF Feedback-Feedback
FNF Feedback-No Feedback
fNIRS Functional Near-Infrared Spectroscopy
HCI Human-Computer Interaction
GSR Galvanic Skin Response
ICA Independent Component Analysis
MEA Multielectrode arrays
MEG Magnetoencephalography
NFF No Feedback-Feedback
NFNF No Feedback-No Feedback
PI Personal Informatics
RFQ Reflective Questionnaire
12
R-SPQ Revised Study Process Questionnaire
SPQ Study Process Questionnaire
SSVEP Steady State Visually Evoked Potential
QS Quantified-Self
Q-Selfers Quantified-Selfers
13
Abstract of Dissertation Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Doctor of Philosophy
IMPROVING ATTENTION FOR LEARNING TASKS WITH QUANTIFIED-SELF
FEEDBACK FROM A BRAIN-COMPUTER INTERFACE
By
Marvin Andujar
August 2017
Chair: Juan E. Gilbert
Major: Human-Centered Computing
In the era of Millennials, people track many aspects of their daily activities to gain self-
awareness of their performance. The data acquisition is accomplished manually by inputting data
in a mobile application, notebook, or website. Additionally, data is obtained by automatic
logging using wearable devices for recording heart rate, calories burned, step counts, and other
fitness information. These objective data are usually correlated with manual data log like calories
burned obtained from a wearable with calories consumed and by inputting the food eaten that
day. Users can see their data visualized as soon as they finish their task and in the future for short
term and long term reflection. Currently, data quantification is done mainly for fitness and
health, but not necessarily in the learning space. There is a discussion of expanding quantified-
self for learning and its implications, but it has not been formally studied to the best of our
knowledge. Also, wearables that people use for self-quantification do not gather brain data.
Therefore, people do not record brain data for any activity unless they are researchers in a neuro
field.
This dissertation explores the utilization of a wireless wearable electroencephalographic
(EEG) device to measure attention and provide visual feedback to the learner after they have
14
watched a ted-Ed video for short-term improvement. Also, it explores the perception and
preference of college students on visualizing attention data with their desired scale. This
dissertation demonstrates that there is an increase of attention from the first video task to the
second video task when learners receive attention feedback gathered from their brain or know
they are being monitored in the form of a bar graph in a 0-100 scale. Furthermore, this work
shows that most learners are willing and interested in utilizing a Brain-Computer Interface to
measure their attention for learning activities, to detect or confirm if they have attention deficit
hyperactivity disorder detection (ADHD), or for other non-learning activities such as
entertainment.
This dissertation opens the door for further investigation on the use of EEG devices for
self-quantification from a personal informatics perspective to self-improve student’s learning
habits.
15
CHAPTER 1
INTRODUCTION
Knowledge of the self is the mother of all knowledge. So, it is incumbent on me to know
myself, to know it completely, to know its minutiae, its characteristics, its subtleties, and its very
atoms.
—Khalil Gibran
The Prophet
Khalil Gibran, a Lebanese-American artist, poet, and writer understood the importance of
obtaining knowledge about his behavior. Knowing more about your mental and physical
characteristics can enable you to make better, more focused, personal improvements. The
quantified-self (QS) movement utilizes data acquisition technologies to measure people’s daily
life activities. The data acquisition constitutes of the user’s state (i.e. emotional, behavioral) and
performance (mental or physical). Visualizations of the self-obtained data can help users reflect
and self-regulate by comparing their state and performance to a goal in a specific environment
[1].
There exist various self-tracking tools and methods such as physical and digital journals,
wearable/ubiquitous devices, physiological sensors, manual data entry in a mobile application or
web dashboard, and regular paper [2]. These methods have primarily been used for physical
activities and calorie consumption/burn comparisons. There have been discussions of
incorporating QS techniques and tools towards the learning space, but it has not been formally
studied [69]. Furthermore, there has been a lack of attempts in the use of quantified-self
techniques in learning, and the use of any self-tracking with a BCI has not been investigated
from a QS perspective. QS in learning can open several doors to other related fields and answer
several research questions on how students learn [3]. When combined with BCI, it can provide
data that no other physiological, ubiquitous device or sensor can provide [4]. BCI allows people
to quantify their affective, emotional, and cognitive state to an extent directly from the brain.
16
Motivation
The question many people face is, “Who am I?” [5]. The QS movement thinks the answer
lies in what the user does in his/her daily activities. This view associates itself with Khalil’s
statement on self is the mother of all knowledge. The more people learn about themselves, the
more they can improve in activities within specific environments. However, people do not tend
to recognize events and experiences right away or in detail. Also, they may not know how to
assess themselves without a tool. Therefore, the QS movement hopes to address this by
incorporating self-logging through ubiquitous devices, physiological and non-physiological
sensors (i.e. GPS, cameras, heart rate, etc.), and manual logging.
Quantified-self approaches to collect user’s data could support reflection amongst
learners about their learning [3,6]. Reflection effectiveness among learners has been attempted in
earlier years in the field of Educational Psychology on relating reflection with academic
performance through journal keeping (manual logging) [7-9]. Also, the use of
neurophysiological devices such as EEG has not been researched in the form of quantified-self,
but it has been investigated in the educational domain.
This work is motivated by the need to further explore how to adapt non-invasive EEG
devices on educational tasks to help learners improve how they learn, explore ways to measure
attention from the brain for self-knowledge while learning, explore ways to introduce self-
quantification for attention improvement for learning tasks and make it as efficient and user-
friendly as methods used for fitness and health, and investigate the preferred visualization for
specific data sets, in this case, attention levels for an educational task.
Research Questions, Hypothesis and Methodology
The main goals of this work are to understand college student’s preferences of visualizing
attention for self-quantification and evaluate the effectiveness of quantified-self attention
17
feedback obtained from a BCI in the form of the preferred visualization for self-regulation in a
learning task.
RQ1: what is the preferred visualization and scale to represent attention among
college students?
This question explores which static visualization and data scale do college students prefer
by ranking them and providing qualitative data on their reason for their ranking. This experiment
was in form of a focus group. Also, during the focus group, the participants produced different
visualizations based on their perception on how attention should be visualized.
The following research questions were investigated in the second study after the focus
group with different participants. The participants firstly watched the video, “How Chronic
Stress Affects Your Brain (video 1)”, then watched, “The Effects Of Sleep Deprivation (video
2).” While they were watching the videos, their attention levels were measured and decoded
from an electroencephalography (EEG) device. Participants were randomly placed in one of the
four groups: control group/no feedback-no feedback (NFNF) and the experimental groups,
feedback-no feedback (FNF), no feedback-feedback (NFF), feedback-feedback (FF).
RQ2: What are the effects of quantified-self attention feedback from a BCI on the
short-term increment of attention from the first video to the second educational video
among college students?
H1: Participants who were not introduced to attention feedback have a lower
improvement of attention in the second task than the participants who obtained feedback.
RQ3: What is the effect of attention levels per index on quiz scores?
H2: Participants receive attention feedback will have a higher assessment score than the
participants who did not obtain feedback.
18
RQ4: What is the correlation between quiz scores and attention levels per index
when presented feedback to the user?
H3: There is a negative correlation between quiz scores and attention levels for at least
one index.
This research aims to help learners at the college level to become self-aware of their
attention levels when they perform a learning task through videos. The learner will be involved
with the interpretation of their data and learn more about themselves through the visualizations
provided by the QS application [3]. This process can let learners improve certain learning
behaviors, in this case, the amount of attention they put into learning the content of a material.
Furthermore, the goal of this research is to evaluate the usefulness, usability and short-term
effectiveness of the given feedback for sparking self-awareness and self-improvement of the
user’s attention.
Dissertation Overview and Organization
This dissertation is organized in the following format: Chapter 2 surveys background
work in the areas associated with this work: Brain-Computer Interfaces, Personal Informatics,
and Reflective Learning. Chapter 3 discusses the college student’s preferences of visualizing
attention with a specific scale. The preference discussed in this chapter is based on results from a
focus group. The results discussed in Chapter 3 are then implemented in the design and
development process discussed in Chapter 4. Chapter 4 provides the wireframes used to develop
the mobile application prototype and mobile application architecture. Chapter 5 follows up with
the understanding of effectiveness of brain-based quantified-self attention feedback. Finally,
Chapter 6 provides a summary, contributions, and insights on future work based on the
qualitative and quantitative results from both user studies. This dissertation also contains
19
appendices containing a glossary of terminologies, survey questions, and visualization drawn by
participants in the focus group.
20
CHAPTER 2
BACKGROUND
This dissertation is comprised of the fields of Brain-Computer Interfaces (BCI), Personal
Informatics (PI) (also known as Quantified-Self), and reflective learning. This section briefly
introduces each area to provide enough background for the understanding of this research and
discusses related work in each field related to the goals of this research (discussed in Chapter 1).
Brain-Computer Interfaces
The term Brain-Computer Interfaces (BCI) was introduced by Jacques Vidal in 1973
[10], although there were electroencephalography (EEG) research studies before its introduction.
BCI focuses on the use of a neurophysiological apparatus for controlling digital (i.e. In Virtual
Reality, simulations, or video games) or physical objects (described as active or reactive BCI) or
to measure and decode the affective, cognitive, and emotional state of a user while they perform
different activities in each environment, known as passive BCI [11]. In a more medical
perspective, it is the measurement of the central nervous system (CNS) activity in which
translate its output to an application: replace, enhance, restore, supplement, or improve [12-13].
It also changes the continuing interaction between the CNS and the given environment.
BCI research is categorized into four different quadrants [14-15]. As seen in Figure 2-1,
Quadrant I represents the active BCI sub-field that concentrates on the control of machines using
the affect-based method or motor imagery control. Quadrant II describes another form of control,
reactive BCI. This area focuses on even-related potentials (ERPs) such as P300-based selection
and the steady-state visually evoked potentials (SSVEP) based selection control. Quadrant II and
III reflect passive BCI research. This area does not focus on control but on the measurement and
translation of brain data to affective, cognitive, or emotional state. The five methods of passive
BCI are workload monitoring, affect monitoring, P300-base lie detection, workload probing, and
21
media-induced affect estimation. Passive BCI is considered the parent field of affective BCI
(aBCI), which focuses on the measurement of user’s affect during a specific interaction in a
specific environment [11].
BCI Cycle
The process to obtain affective data from the human brain is derived from the BCI cycle
[12-13]. It consists of obtaining a raw signal from the user with a neurophysiological device,
performing signal processing on the acquired data for the removal of noise and artifacts, and
translating the signal to suit a specific application area for the user to receive feedback (Figure 2-
2).
The first phase of the cycle is the measurement of the user’s brain activity with a non-
invasive, invasive, or semi-invasive neurophysiological apparatus [16]. Typical brain imaging
technologies used for brain data acquisition are electroencephalography (EEG), Functional
Magnetic Resonance Imaging (fMRI), Functional-Near Infrared Spectroscopy (fNIRS),
Magnetoencephalography (MEG), electrocorticogram (ECoG), and microelectrode array (MEA)
[17]. Also, neural lace is a new brain data acquisition device that has recently emerged.
Electroencephalography (EEG). EEG is a device or cap typically placed on the user’s
scalp to measure the brain’s electrical activity. It is highly portable, low cost, and has a good
temporal resolution, but poor spatial resolution. Due to its portability and cost to build most of
the modern wireless BCIs are EEG-based (Figure 2-3). Each of these EEG apparatuses are based
on the 10-20 international system [18]. Table 2-1 describes the specifications of the EEG BCIs
showcased by Figure 2-3.
Functional Magnetic Resonance Imaging (fMRI). fMRI (Figure 2-4) measures
hemodynamic brain activity by detecting changes in the blood flow. It is a good method to
visualize brain activity in different regions of the brain. This brain sensing technology is
22
considered to have a low temporal resolution, but good spatial resolution. Also, it is not
considered to be portable as it is a big machine and is highly expensive. The use of an fMRI
machine tends to cost around $500 per hour and to buy it costs around one million dollars.
Functional-Near Infrared Spectroscopy (fNIRS). fNIRS (Figure 2-5) is a similar
measurement to fMRI. However, fNIRS is a small device that is portable and offered at a lower
cost; it is still not as affordable as off-the-shelf EEG devices. It also tends to be placed on the
pre-frontal cortex (forehead) of the user, and it is a good method to measure cognitive workload.
Magnetoencephalography (MEG). MEG (Figure 2-6) maps brain activity by recording the
magnetic fields created by electrical currents using magnetometers. This technique is
noninvasive, with a good temporal resolution, but low spatial. Similarly, to fMRI, it is not
portable as it tends to be a big machine and it has a very high cost to use and buy.
Electrocorticography (ECoG). ECoG (Figure 2-7) is a well-known invasive measurement
that reads electrical signal from the brain with high temporal and spatial resolution. It is portable
(especially if the user is in a mechanical wheelchair) and highly expensive. This measurement is
mainly used for patients with mental or physical disabilities because it requires the skull to be
opened for placement.
Microelectrode array (MEA). MEA (Figure 2-8), which is also an invasive measurement
technique that reads electrical activity with an electronic circuitry or chip. This device is highly
surgical and can be considered as the most robust technique to measure brain activity as it is
placed directly on the brain. It is also considered to have high temporal and spatial resolution.
MEA is extremely portable, but it is very expensive to implement and not currently
recommended for healthy users as it requires surgery.
23
Syringe-injectable electronic. It is also known as neural lace. It is a flexible neural
technology that is placed on top of the species brain through a syringe (Figure 2-9). This neural
recording method can adapt to the shape of non-planar surfaces such as the brain. This brain data
acquisition technique is the newest and tested with mice where it successfully detected neuronal
activity reliably [19].
Signal Processing. The next phase of the cycle is the signal processing of the recorded
brain signals. The first step of signal processing is feature extraction, which is the process of
identifying the applicable signal characteristics from inappropriate content and representing them
in an appropriate form for human or computer interpretation. This process is conducted once the
signals have been acquired from the user. During this phase, signals are prepared for translation
into a BCI output [20]. This phase is accomplished by isolating important features of the signals
from corrupted information or interference, also known as noise. Noise originates from within
the neurological activity. Another form of contaminated information is known as artifacts, which
it is any content in the signal that is not neurological like muscle movements, eye blinks, heart
rate, sweat, and bad channel placement or condition.
The second step of signal processing is the adaptation of a translation algorithm. This
algorithm is necessary because features represent indirect measurements of the user’s intent.
Therefore, it must be translated into an appropriate command to convey that intent. A translation
algorithm is considered a mathematical procedure that is comprised of an equation, set of
equations, or mapping mechanism like a lookup table [21].
Applications. Once the signal is successfully cleaned, it is sent as an output to a given
application for different purposes. In BCI, one of the purposes of an application is to detect affect
to enrich the human-machine interaction. An example of this would be the ability to adjust the
24
user interface (i.e. video games, virtual reality scenes, etc.) based on the user’s affect at that
specific time. Once the system has adapted, the user receives feedback. This change in the
environment corresponds to the user’s affective intent.
BCI Society and Enhance Roadmap
Recently, a new BCI society has emerged to provide a global perspective and to establish
a consensus of where BCI research and applications are headed for the year of 2025. The
founders developed a roadmap for each identified BCI application described in the cycle:
replace, improve, restore, enhance, supplement, and research tool [23]. The closest application
that relates to a BCI directly is enhance. The purpose of the enhance application is to monitor the
user’s affective and cognitive state and stimulate the state to improve their experience. Figure 2-
4 shows the roadmap assembled by the BCI Society for the enhancement application. It shows
that the end-user and the advancement of BCI research and technology are equally important.
Figure 2-5 illustrates the meaning of each object/symbol. This dissertation fits in the enhance
category with the goal of the BCI society. By 2025, the roadmap shows there will be an
adaptability of user state monitoring applications for several purposes.
We provide the following descriptions as an explanation of the purpose of each category
within the roadmap. It is good to note that the founders of the society did not explain these in
detail, therefore we provide our perception of what they represent.
Science and Research. Decoding the affective and cognitive state needs further
optimization. This needs to be an iterative process for robust achievement and in combination
with an intuitive user interface. It can also be observed that the decoding of the user’s state
should interact with other scientific fields. The fields of behavioral Psychology,
Psychophysiology, Cognitive Psychology, Neuroscience, and others can help the field advance
the user’s state decoding. These areas study the functionalities of the different regions of the
25
brain in detail through user studies. Several trials are needed in the utilization of
neurophysiological analysis algorithms such as independent component analysis (ICA) and the
adaptability in several user studies differing by applications to advance the decoding phase
efficiently in the next ten years.
Industry. Due to the vast release of non-invasive wireless BCI devices for consumers
and researchers, industry has become vital for the advancement of the field. The consumer based
devices need to become more robust in their data acquisition to achieve the same or close
resolution to the wired BCIs. Besides the BCI hardware robustness, the user interface to obtain
feedback on their mental state is vital. This process requires easiness to log brain data and see the
visual representation of it from both a mobile device and on a computer. The goal is to deliver
robust and adapted user interface BCIs at a low cost for several applications like gaming.
End User. The advancement of BCI applications and the decoding of the user’s state also
rely on BCI adoption by end users. The more data that can be gathered from end users, the more
the field can advance the decoding of the user’s state. Researchers would be able to interpret the
user’s affective and cognitive state in different physical and virtual environments. Also, the idea
is for the user to accept BCIs as a wearable computer for their daily routine to log their mental
data. Rigorous studies should be performed to understand how this technology may be adapted
by users.
Clinical Trials. The understanding of the user’s state can be applied in the clinical
domain as well. The user state monitoring can be utilized for both healthy people and people
with physical or mental disabilities. In the clinic, the identification of the user’s state can help
medical doctors comprehend the mental state of their patient in different situations and clinical
procedures. The enhance application roadmap does not have a clinical trial process as they
26
related it to learning and education. However, the medical domain can benefit from the
advancement of this decoding as well.
Product and Evidence. This phase in the roadmap specifies that evidence of success
needs to be provided through data acquisition to form a product. This step is performed in a
research lab for internal validity and tested in several locations outside the lab for external
validity and robustness improvement.
User Group. This focuses on the adaptability of BCIs to different types of users. The
huge market represents the adaptability of BCIs in healthy users and people with mental or
physical activities. Within healthy users, the effectiveness and adaptability of BCIs depend on
the physical properties of each user. People’s heads differ in shape and size, and their hair
differentiate in length and thickness. The purpose of a BCI is to be multimodal and to be used by
a wide variety of users. Therefore, the robustness improvement is important for users to use these
devices without limitations.
Engagement
Engagement is the user’s level of attention while they perform a specific task in a specific
environment [24,25]. Attention refers to the focus of cognitive resources on a specific stimulus.
In HCI, measuring attention could estimate how much information users perceive [26,27].
Engagement has also been defined as a real-time indicator of a learner’s motivation [28] hold
attention [29,30], or when a person is immersed in a task by blocking out internal or external
factors [31]. It tends to be affected by the user’s motivation and interest in performing the task,
and it is also influenced by task’s difficulty in which affects the individual’s attention allocation
[29].
27
A user’s engagement increment could improve learning outcome in problem-solving,
critical thinking and memorization [32]. This signifies that regardless of what the student is
trying to learn, it requires engagement [33].
The investigation of measuring engagement with EEGs has been pursued using different
EEG waves ratios. The ratios are analyzed through different channels to study their significance
in the changes of emotions, depression, consciousness, motivation, and other criteria that help
increase engagement or cause it to decrease.
Spectral-Based EEG Engagement indices. There are various engagement indices
evaluated to determine the most appropriate method to measure and decode engagement from
brain waves. There is not a main index to be used for all occasions. Different indices need to be
compared to determine significance utilizing specific EEG channels for a given event. For the
following studies, we provide summaries that tested different indices to measure engagement.
Also, some of these studies utilized an engagement index developing engagement monitoring
tools. The indices tend to consist of the EEG bands, Alpha (8-12 Hz), Beta (13-30 Hz), and
Theta (4-7 Hz), where they differentiate by frequency and amplitude [66] as seen in Table 2-2.
The bands are decoded in the post-processing step from the raw EEG and are used to understand
the mental state of the user at the moment, in this case, the user’s engagement.
Indices based on the band powers have been evaluated to test the effectiveness of
measuring mental engagement in a task using the electrode sites Cz, T5, P3, Pz, P4, O1, and O2
referred to an average of left and right earlobes [34]. The indices were tested on six participants
with beta/alpha, beta/(alpha + theta), left temporal alpha/central alpha, and left occipital
alpha/right occipital alpha (arbitrary defined and was not expected to be related to engagement).
Participants interacted with a modified version of the Multi-Attribute Battery (MAT), which is a
28
computer-based task to evaluate operator performance and workload. In the study, the tracking
task switched between manual and automatic mode based on the behavior of an EEG index. The
switch was based on the decrease of an index in the negative or positive feedback. During
negative feedback, the task mode would switch to manual, and for the positive feedback, it
would switch to automatic mode. The beta/(alpha + theta) index had the most significant
difference between both feedback conditions, which reflects task engagement best.
The measurement and decoding of the beta/(alpha + theta) index have been adapted in
domains such as learning/education, Human-Robot Interaction (HRI), and video games. In HRI,
an adaptive agent was designed to monitor student’s attention levels in real time with a wireless
non-invasive low-cost EEG device consisted of one channel, Fp1. In the study, the participants
were provided a memory task to assess their recall of details of 10 minutes long Japanese story
on "My Lord Bag of Rice" narrated by the robotic teacher. The adaptive agent demonstrated
different behavioral techniques to regain user’s attention when the attention dropped. The user’s
attention improved by 43% over the baseline after executing the task [35]. A further study was
conducted to monitor a student’s attention during educational presentations to determine which
of the lectures’ content was more beneficial for students. The students’ attention levels were
measured with a non-invasive wireless BCI with one EEG sensor (Fp1) mounted in the forehead
as well. During the recording, participants watched four minutes’ lectures of different art periods
and answered questions about them. Students could recall 29% over the baseline and were able
to match recall gains achieved by a full lesson review in a shorter amount of time [36]. Similarly,
there was a discussion of the possibilities of measuring attention levels to identify when the
user’s attention drops at a moment when they are studying a specific content. Once the drop is
detected, a video related to the reading would be shown to stimulate engagement to improve it
29
after showing a visual and “more interesting” way of explaining the content of the reading [37].
Afterward, there was an implementation like the discussion by creating a system called FOCUS
that monitors children’s engagement levels in real time to provide contextual BCI training
sessions to improve their reading engagement. The engagement was measured using a wireless
14-channel (AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4) EEG device. The
authors found that by identifying low levels of engagement and displaying a BCI training session
on top of the book, children could concentrate better after that session was completed [38]. There
is also an interest of incorporating EEG devices in the classroom [61] for providing feedback to
the instructor on student’s engagement or obtaining other data that lectures would be interested
in obtaining. A system called EngageMeter obtains engagement data from the Neurosky
Mindwave (channel: Fp1) and presents the presenter of the lecture real-time access to the data
[58]. This method allows lecturers to assess the time and part in their lecture, the students were
and were not engaged. Instructors can adjust in their future lectures in the way they present the
material to enhance student’s engagement.
The measurement of engagement has also been applied to study different learning
phenomena instead of improving attention using external tools, media sources, or building
quantifiable tools. Engagement levels measured from a BCI was compared while learners
learned the Lewis and Clark adventure through an educational video game or a handout. The
engagement was measured with a proprietary algorithm from Emotiv. Although there were no
statistically significant results between the two techniques from a neurophysiological
perspective, there were useful questions that emerged from the study [65]. These questions were
based on how dependent should researchers be on traditional statistics on brain data and
evaluating the limitations of wireless BCIs.
30
Another approach for understanding the user’s engagement is while they watch movies or
videos. There are a lot of emotions within movies or videos that stimulate the user’s affective
state and changes the mental state. A hybrid approach was used with galvanic skin response,
EEG, and facial tracking. The work focused on measuring the viewer’s engagement levels while
the participants watched a sequence of clips from a short movie. Their results entail that
combining the given modalities is beneficial to classifying levels of engagement. Also, each
modality significantly encodes the levels of engagement for the viewers in correspondence to the
movie clips [64].
The efficiency of measuring engagement through EEG in combination with sustained
attention has also been studied utilizing beta/alpha index. The EEG signals were recorded with
19 Ag/AgCI electrodes (Fp1, Fp2, F3, F4, F7, F8, T3, T4, C3, C4, T5, T6, P3, P4, O1, O2, Fz,
Cz, Pz). The signals were referenced to A1 and A2 electrodes. A 32-channel AC/DC amplifier at
a sampling rate of 500 Hz was used for the EEG recordings. Nine healthy subjects participated in
a continuous performance test (CPT) while EEG brain activity was acquired during the task. It
was deduced that there is a relationship between task engagement and ratio of the alpha and beta
rhythms including a correlation between reaction time and engagement levels variations. It also
provides evidence that obtaining EEG data from the frontal cortex is accurate for sustained
attention and vigilance [39].
User’s engagement measurement research has concentrated on identification of its level
and performance when users perform a task. Further investigation is needed to understand the
changes in the bands when different types of stimulus (i.e. audio, video) are presented. Although,
it has been documented as described formerly in this chapter that a ratio of Beta over Alpha
performs well for the identification of attention levels. Also, understanding the alpha power can
31
help researchers measure engagement [63], especially obtaining data from the channels located
in the pre-frontal cortex. Application areas and domains like the studies show the different
possibilities of how engagement measurement can be adapted for research studies or
applications. However, there are limitations to these studies and their implementation of the
engagement index. The index beta/(alpha + theta) have been adapted to the HCI community, but
the channels used in HCI work do not match the ones tested in [34]. The main problem is that
off-the-shelf devices have limited channels and do not necessarily contain the channels that other
researchers use. Further validity studies are needed to test the index in channels used in HCI
research studies, and further investigation is necessary for an index that works for channels used
in off-the-shelf. Another limitation in the studies is that the sample size is respectively small. It is
hard to conclude that one index is acceptable for specific channels when the number of subjects
is less than 30. Therefore, further studies with bigger sample size are needed for further
validation.
Personal Informatics/Quantified-Self
Quantified-Self is also known as Personal Informatics, the logging of the self, living by
numbers, self-surveillance, self-tracking, personal analytics, and others. This area is responsible
for recording the personal information of a user through a different set of tools for self-reflection
to achieve a specific goal [2, 40-41]. Self-reflection is the phase when the user sees the acquired
data and notices how s/he performed in that activity and thinks of ways to improve. Self-
regulation is the following step where the user takes in mind their previous performance and tries
to improve at the given task. Users can log their self-data in various ways; they can use
wearables (i.e. Fitbit, Jawbone, smartwatches), notepads (i.e. journal keeping), mobile
applications, web-dashboards, etc. A wide variety of tools and methods designed, developed and
32
researched for the purpose obtaining self-data for multiple purposes can be found in the personal
informatics repository (www.personalinformatics.org/tools).
Five stages for QS have been composed. These five stages comprise the process user’s
take from obtaining the data with their preferred tool to self-regulation [40]. These following
steps also serve as a guideline of what a QS application or logging method should allow a user to
do to be a successful tool.
Stage I: Preparation - Focuses on people’s motivation to collect the self and how they
determine what information they will record, along with how they will record it.
Stage II: Collection - The user collects the self. They observe their inner thoughts,
behavior, and their environment. There are different time frequencies of collection (i.e. per hour,
day, week, month, etc.).
Stage III: Integration - The stage that lies between collection and reflection. The collected
data is prepared, combined, and transformed for the user to reflect on.
Stage IV: Reflection - The user reflects on their data. This reflection includes looking at a
list of collected information or exploring and interacting with the visualizations.
• Users may reflect on their info immediately after recording it (short-term). It makes
the user aware of their status.
• Users may reflect on their data after several days or weeks involving extensive self-
reflection (long-term). This reflection is useful because it allows users to compare
personal information between different times and it reveals trends and patterns.
Stage V: Action - This is the stage users choose what they are going to do with their
newfound understanding of themselves. Some people reflect on the information to track their
progress towards goals.
33
Barriers and Pitfalls
The implementation of quantified-self methodologies has multiple barriers and
challenges. Reported barriers include fragmentation of data spread in different platforms,
unappropriated or complicated visualization, lack of knowledge in analyzing the data, lack of
motivation to utilize these tools, unsuitable analytic tools, and lack of time for utilization [2,40].
Some of the other identified pitfalls are the amount of recording data, “tracking too many things
often led users to either stop tracking entirely due to tracking fatigue or failure to do data
analysis due to too much data in different formats [2].” These barriers and pitfalls provide a poor
user experience where the user stops using the quantified-self tools. One of the main reasons for
the existence of the barriers and pitfalls above is because of the lack of automatic quantification.
Users tend to forget or are too busy to log their data on an hourly, daily, weekly, or monthly
basis. Sensors incorporated within wearables are needed to log self-data automatically and
visualize the information for analysis towards improvement.
Moving Forward in the Quantified-Self Movement
To address the barriers and challenges that quantified-selfers encounter in their data
logging, implementation of technology that automatically quantifies data is needed. Therefore,
self-monitoring has been incorporated in the design process of sensing and monitoring
applications [2]. This self-monitor is incorporated because sensors are becoming smaller and
more accurate through every iteration, which makes it possible for quantified-selfers to track
several types of data at once. Therefore, the use of wearable devices like Fitbit uses
accelerometer sensors to track the number of steps and translate them to calories burned and uses
heart rate sensors on the wrist. The heart rate technology is also known as photoplethysmography
(PPG), which illuminates the human skin and measures the change in light absorption [42]. In
34
the health domain, it is believed that users are only interested in easily understanding the data on
their health and not necessarily interested in the technology and the data [43].
In summary, quantified-self research concentrates on the evaluation of several methods
and tools when the user logs their data for a specific task. Further, research has concentrated on
understanding how and what the user thinks about the data. This kind of information helps
researchers identify pitfalls and barriers and what researchers need to do to move the field
forward [2]. The information also includes how users perceive their data specifically with
ubiquitous technologies [41].
Based on reviewing the personal informatics repository and literature in QS the use of
other wearables like neurophysiological devices needs to be studied. This technology can
provide affective information about the user that cannot be provided by other physiological
sensors. Therefore, the implementation of wearable BCI devices like those in Figure 2-3 for data
acquisition of levels of attention and other affective and cognitive data from the user while they
perform a task may be beneficial towards their self-regulation.
Reflective Thinking
Reflection has been defined in different ways. In 1985, reflection was defined as, “those
intellectual and affective activities that individuals engage into exploring their experience, which
leads to new understanding and appreciations” [44]. Six years later, Dewey defined self-
reflection or reflection as the “active, persistent and careful consideration of any belief or
supposed form of knowledge in the light of the grounds that support it and the further conclusion
to which it tends” [45]. Eight years afterward, Moon related reflection with learning and
incorporated reflection into the learning process. The author provides the following definition for
reflection, “a form of mental processing with a purpose and participated outcome that is applied
to relatively complex or unstructured ideas for which there is not an obvious solution” [46].
35
Based on these definitions on reflection, there have been several studies conducted to
assess the effectiveness of reflection on learning outcomes. The impact of reflection journals has
been utilized in college level math courses. The purpose of the reflection journal writing
includes: to critically review the behaviors like strengths and weaknesses, learning styles and
strategies. The reflection journal has been an effective method to assist students developing
abstract thinking, but it did not necessarily assist students to obtain higher grades [47]. The
utilization of the journal method was also used on 40 undergraduates in a first-year biology
course. Half of the students were randomly selected to learn from the journal, and the other half
were chosen to learn from a scientific report (today’s way of learning). The students who used
the journals showed greater awareness of cognitive strategies than when they were learning from
text [48]. They also performed significantly better on the final examination for the course.
Another study further investigated relationships between student’s self-reflection and
academic achievement through journal logging. A methodology similar to the studies above was
applied. In this case, several coding categories were implemented in comparison with grades
acquired by the students. There was no statistical difference between the academic performance
in the classroom and the different journal coding categories [49].
Although several studies have been conducted utilizing reflective journals as a way to
improve student’s self-reflection for academic achievement, it still theoretical. Further studies
are needed for both internal and external validation. It requires moving away from just reflective
journals and adopting wearable devices for reflective learning and thinking.
Learning and Reflective Questionnaires
There have been investigations on adapting surveys to assess learning and reflective
thinking for any educational task. A well-known and widely adapted survey called the Study
36
Process Questionnaire (SPQ) assesses how the students approach learning [50]. Throughout the
years the survey was updated, shortened, and conveniently designed for teachers to use for
evaluation purposes [51].
Another survey introduced by Biggs is the Revised Study Process Questionnaire (R-SPQ-
2F) [51]. This survey consists of 20 questions that measure two main scales: surface learning
approach and deep learning approach. The students respond to the questions on a five-point
Likert scale ranging from “this item is never or only rarely true of me” to “this item is always or
almost always true of me.” The scores range from 5 – 25, where the score determines the
learning used.
A reflection-centric questionnaire, named Reflection Questionnaire consists of 16
questions, with four items in four different scales: habitual action, understanding, reflection, and
critical reflection. The students also fill out the questionnaire on a five-point Likert scale.
However, this one ranges from “strongly agree” to “strongly disagree” [52-53].
These questionnaires help researchers assess how students tried to learn the material and
how much reflection they put into critical thinking and understanding the content. For the
proposed research, the integration of some of these questionnaires may be valuable towards
understanding how much reflection the application encourages the learners to do.
37
Figure 2-1. BCI areas specified by quadrants [14-15].
Figure 2-2. BCI Cycle introduced by Wolpaw [13, 22].
38
Figure 2-3. Non-Invasive Wireless Ubiquitous EEG BCIs
Table 2-1. Wireless BCI apparatus specifications
Apparatus Min.
Channels
# of
channels
Sampling
Rate
Network
Protocol
Noise Suppression
(reference)
Neurosky
Mindwave
FP1 1 512 Hz Bluetooth Located in ear clip
Muse TP9,
AF7,
AF8,
TP10
4-6 220Hz or
500 Hz,
10 or 16
bits /
sample
Bluetooth
2.1
DRL-REF feedback
with 2V (RMS), FPz
(CMS/DRL)
Emotiv
Epoc+
AF3, F7,
F3,
FC5,
T7,
P7,
O1,
O2,
P8,
T8,
FC6,
F4,
F8,
AF4
14 128 SPS or
256 SPS
(2048 Hz
internal)
Bluetooth
Smart
CMS/DRL, P3/P4
locations
Emotiv
Insight
AF3,
AF4,
T7,
T8, Pz
5 128 SPS Bluetooth
4.0 LE
CMS/DRL
39
Apparatus Min.
Channels
# of
channels
Sampling
Rate
Network
Protocol
Noise Suppression
(reference)
OpenBCI,
Cyton
Board
Custom 8 24-bit
channel
data
resolution
RFduino
Low
Power
Bluetooth
Custom
B-Alert X10 Fz, F3,
F4,
Cz,
C3,
C4,
POz,
P3, P4
10/24 256 SPS Bluetooth
Class 2
+4dBm
~ +2uV @ 10 Hz and
50 kO impedance @
256 s/s
G.tec Fz, Cz,
Pz,
P3,
P4,
P07,
P08,
Oz
8/16/32/64 24-bit
accuracy
at 500 Hz
Wireless CMS/DRL
Enobio P7, P4,
Cz,
Pz,
P3,
P8,
O1,
O2
8/20/32 500 SPS Bluetooth
2.1
CMS/DRL
Figure 2-4. BCI Society Roadmap for Enhance Application [23].
40
Figure 2-5. Roadmap Legend
Table 2-2. EEG bands descriptions
EEG Band
Frequency (Hz)
Amplitude (uV)
Specification
Beta 13 – 30 Lowest Mental activity, alert
Alpha 8 – 12 Low Relaxation
Theta 4- 7 High Drowsy, creativity
41
CHAPTER 3
ATTENTION VISUALIZATION
Quantified-Self mobile applications visualize data in various ways. There is no
consistency graphing logged data. The user needs to disseminate the acquired data easily and at a
quick glance. However, the existing tools have not formally studied people’s preference of static
visualizations for respective datasets. QS researchers tend to recommend visualizations for
different activities, therefore we recommend understanding how users perceive visualizations for
specific data set [70-71].
Raw EEG (Figure 3-1) cannot be understood by everyday users. They contain a lot of
information, but do not show a pattern that can be understood by non-trained individuals. A
focus group was conducted to understand the preference of static visualization for understanding
attention to represent attention from the brain in a static form. This study addresses the research
question, “which static visualization and scale is preferred to represent attention among college
students?”
Experimental Design
This study was a focus group of up to four people at a time. The participants had to
complete three tasks with a discussion following each activity. The first task consisted of
drawing a visualization that helps them understand their attention levels. The purpose of this task
was to explore how college students perceive attention, see if there are any patterns across
students, and explore other visualization possibilities that the quantified-self field has not
formulated. In the second task, the participants consisted of choosing and ranking their top three
from six static visualizations demonstrating attention. The third and last task consisted of the
participants choosing and ranking their top three scales from five given scales. All the given
42
tasks forms are provided in Appendix C, D and E. In Figure 3-2, we can see the layout of the
room of where the focus group sessions were conducted.
Participants
Forty-one participants (23 males, 18 females), graduate and undergraduate students from
the University of Florida participated in the focus group at different times. 39.02% of the
participants aged from 18-21, 29.927% aged from 22-24, 24.39% aged from 25-28, 4.88% age
range was 29-31, and 2.44% aged 32 or above. Among the participants 31.71% were Caucasian,
21.95% were African-American, 17.07% were Hispanic, 17.07% were Asian, and the remainder
12.20% identified themselves as either Black, West Indian, Haitian, or multiethnic.
The participants were students from different academic years; 7.32% were freshman and
sophomore, 34.15% were junior, 17.07% were senior, 7.32% were master students, and 26.83%
were Ph.D. students. These students majored in Computer Science, Industrial and Systems
Engineering, General Business, Criminology, Psychology, Anthropology, Statistics, Electrical
Engineering, Human-Centered Computing, Computer Engineering, Information Systems,
Biomedical Engineering, and Digital Arts and Sciences.
All the participants arrived at the focus group location with no knowledge of the tasks
descriptions and logistics. The demographic survey can be seen in Appendix B.
Procedure
Each focus group session consisted of 45-60 minutes depending on the number of
participants participating in a specific session. The length also differs by the amount of time each
participant took to share their views during each discussion. Each task had a predetermined time
frame: the first task lasted 10 minutes, second and third task lasted a maximum of 5 minutes. For
the second and third tasks, the length was relatively short as participants chose their top choices
quickly.
43
Table 3-1 shows the steps that were taken during the focus group for each session. It
consisted of nine steps. In step 1, when the participants arrived at the study location and took a
seat, they were given the consent form and instructed to read it and sign after agreeing to the
conditions of the study. In step 2, participants were instructed to complete a short survey of their
demographic information (paper survey). Steps 3-5 consisted of the given tasks and their
discussion. For step 3, they had to describe the visualization they formulated and how it helps
them think of attention. For steps 4.5 and 5.5, the discussions were based on identifying their
ranked visualizations and scales and reason for their choice. The final step (step 6) consisted of
any final questions or comments the participants may have that it was not discussed in the
previous steps.
Results and Discussion
The results consist of qualitative data for the first task, which consists of reasons for
drawing the visualization for attention. The second and third tasks consist of both quantitative
(score means and statistics) and qualitative data (comments on choosing visualizations and their
descriptions).
First task: visualization drawing
The participants formulated various visualizations to represent attention. There were
similarities on visualizations between participants. There were more than 100 visualizations
formulated during the task. In these results, the ones formulated multiple times by different
participants are chosen to be discussed. The other visualizations are showcased in Appendix A.
The bar graph (Figure 3-3) is a popular visualization to graph data from quantified-self
applications. Wearables like Fitbit, moto 360, apple watch and consumer based BCI devices use
bar graphs. Therefore, several participants formulated the bar graph to represent attention may be
because of their familiarity with the graph type. It is a straightforward graph where users can at a
44
quick glance see their data score (low or high) over time. “a vertical bar that goes from red (low
attention) to green (high attention)”, “bar graphs with attention levels against time. When you
look at it, you can easily find the information”, “it can be good when you are trying to
multitask”, “in bar graph you can see when attention is higher.”
The filling meter (Figure 3-4) is like a cup filled with water that shows the quantity of its
content. This visualization provides at a quick glance the level of attention (low, mid or high). It
helps users understand how their attention performed. “it is like when you go to the fair, it goes
up with intensity. The power level is the power of focus”.
The smallest circle within the other two circles in the bullseye (Figure 3-5) represent the
highest attention level. Therefore, a marker in the visualization will be in the small circle when
the student’s attention is very high, also described as narrow focused. The circle in between
represented with stripes shows that the student is attentive, but it is not very high. The big circle
encompassing the other two circles represent that the student is not paying attention. “when
focus increase is represented when the dark dot expands.”
The light bulb (Figure 3-6) illustration has been associated with coming up with ideas
abruptly. This visualization is usually demonstrated in cartoons shows. Participants indicated that
light bulb would be a good indicator for attention where the absence of light represents no
attention at all, dimmed light is low attention, bright yellow light is most attentive. “when the
light bulb is off means there is no attention at all, when it is dimmed mid attention, and really
bright is high attention. As the light becomes brighter then. I know I am paying more attention. If
the light bulb is off, then I think what I can do to increase my attention.”
The human eyes are a body part that, we use consistently to recognize the shape and color of
objects and people. It may seem natural that participants thought of the eye as to pay attention to
45
learning activities need the use of eyes. Figure 3-7 shows the eye at a descriptive five-liker scale:
need to wake up, not attentive, somewhat attentive, attentive, and very attentive. Each liker level
differs by how open the eye is from completely closed to fully opened. How awake the eye is
going hand in hand with how we humans use our eyes when we are reading or watching videos
to absorb specific information.
This graph is a speedometer (Figure 3-8) on a scale from 0-100 like what we see in
automotive. When the line indicator is far to the right is higher the user’s attention level. The
graph also has a timer to indicate the time it takes the learner to complete the task. “speedometer
can show how long you have been focusing for. I can see every information at once.”
The line graph (Figure 3-9) is also a known visualization as the bar graph and used in data
reports. Participants found this graph simple and easy to understand their attention. They were
also interested to see their attention level along with their interest level in the same graph. Line
graphs with multiple data of attention can show when the value increased or dropped between
interest, time, or tasks in which help them compare their performance within the same graph.
“preferred the line graph because of its simplicity”, “it is always good to see quantity, use form
of metric with time and you can see it over time”, “progression shows low and high attention.”
Second task: visualization ranking
In the second task, participants selected their top three from six static visualizations. The
given visualizations are the typical type of graphs used in quantified-self applications and graphs
that college students are used to seeing. Table 3-2 shows the descriptive statistics of the
visualization ranking scores. Bar graph ranked first with the highest mean score, line graph
second, and clustered graph third.
46
The Friedman rank sum test was performed to evaluate the significant difference of the
overall scores between the visualizations. The test showed that there is an overall significance
difference between all the visualizations, X2 (2) = 35.546, p = 0.000. Afterward, because
Friedman test does not provide unique differences, a Wilcoxon signed-rank test was performed.
The Wilcoxon signed-rank test shows that there was a statistical difference between most the
visualizations except for the ones without an asterisk in Table 3-3.
Each visualization was provided feedback (statements with (N) is annotated as a negative
comment) from participants on preferring a graph or for disliking it. Bar graph was the only
visualization that did not receive a negative comment. Therefore it further supports that bar
graph is the preferred visualization between the six given ones.
A bar graph is liked by college students because it perceived to be clear, easy to understand,
specific, and considered helpful without distractions. Its simplicity is the strongest aspect of this
visualization, hence the preference. “when it is going up you know high value and when its going
low you know the low value”, “it is very straight forward to see”, “pretty clear”, “breaks
everything out evenly”, “normal and regular graph, easy to understand”, “similar to the clustered
bar, words are in the x-axis”, “show me every task is independent of each other and shows me
that clearly in the data”, “show exactly what level of attention for each task, attention should
depend on the task”, “it is specific”, “perceive better the formatting”, “it goes with what
convention goes, task is changing, pretty easy to look at a task and easy to figure out”, “it is
easier to look at”, “it is a histogram, getting to know the first data for attention”, “I like bar
graphs and can compare each task and like more vertical. It is the traditional one”, “you can
easily read it, we read from left to right. That is something that I do every day. You can easily
tell what is the lowest”, “properly represents like a trial, this is the person attention level”, “felt
47
more comfortable to me, like concentration in y-axis”, “it is very traditional, because one of the
first graph people see in their schooling”, “like the height more than the width”, “like bar
graphs”, “bar is very helpful, and it is not too distracting.”
The clustered bar is a vertical representation of the bar graph is not perceived as positive as
the horizontal bar. It is liked because users can compare data from top to bottom, it is closely
related to the bar graph, and it is easy to understand. “horizontal takes more time to interpret
(N)”, “the words in y axis, I read from left to right, easier to read for lefties”, “I can see the entire
graph and get a feeling for it, and I don’t have to move my eyes to see what’s going on”, “not
used to reading graphs from right to left. Rather avoid (N)”, “simple for understanding and all
the tasks are listed, fluctuates”, “same metric as the bar graph, I personally like the formatting of
the bar graph better (N)”, “it is the same as bar graph, I prefer going up like bar graph though
(N)”, “the layout is better than bar graph, when you think how things are loaded in a PC, it goes
from left to right”, “naturally read from top to bottom and understand things from ascending
over. Not being a good visual person going from left to right confuses me on first glance (N)”,
“another way to read data from bar graph”, “you can compare it up and down, you can see task
#3 is the lowest attention”, “easy way to see which was the lowest or highest”, “right to left
makes more sense to me than up and down (personal preference)”, “easy to understand”, “don’t
like how this graph is organized, would like inverted of how tasks are organized (N)”.
The pie chart appeared to be confusing for the participants to analyze the data. However, the
use of colors make it more visually appealing, and the size of each pie indicate how big the
attention. One of the main issues that participants had with this visualization was that it takes a
longer time to understand what is going on with the data. It was not clear for them how the
attention differed per task, because there was too much going on and did not know what the
48
whole pie represented. “confusing (N)”, “it seems part of a whole (N)”, “grouped the tasks and
see the attention level with each task”, “I am able to see the motion (waves), the high, middle, or
low”, “pie chart has too many colors and too many options (N)”, “don’t like the score next to
task number, have to look more specifically to get an idea what it is. The size of each section
doesn’t help (N)”, “shows the attention level and the slices indicate the levels of attention. I like
the different colors that represent”, “I don’t know what value each correspond to, but I can see
the proportion”, “I like the pie chart how it distributes the different attention levels, nice
visualization of levels of attention”, “it does not provide numerical data, but easy
representation”, “when I look at the data, I can see what size is bigger that shows higher value”,
“it shows the task that has the most attention”, “it splits up the different tasks between colors and
segments”, “the equal proportion of tasks, every task has certain weights. Each task has certain
involvement in the whole experiment”, “it is the most confusing, because multiple tasks are in
the same pie. You cannot tell if task 2 and 7 are the same or different (N)”, “although like the
visual, what is the whole pie? When I saw multiple numbers in there, there are a whole bunch of
questions when I see this graph. It is a lot more confusing. If it has what the whole pie represents,
may be better (N)”, “it doesn’t make sense as it is part of a whole (N)”, “it is pretty much the
same thing as the clustered graph, but less fluid than the line graph”, “don’t know what I’m
looking at. It is not a good representation. It seems messy (N)”, “what is the whole pie supposed
to represent? (N)”, “it doesn’t make any sense (N)”, “the colors are useful. I am not sure if the
tasks are related. If they are, then it’s a good presentation, if they are not, then not useful (N)”, “it
can distinguish the difference”, “highlight the difference between datasets in more visual way”,
“too many colors too much going on, when think of attention, don’t think of pie chart think of
line graphs (N)”, “cannot see which one is bigger to me (N).”
49
The area graph was generally received positive, considered easy to follow, can see the peaks
and lows easily. It was also perceived as an ugly design and less clear than other graphs. “filled
in makes it less clear (N)”, “like line graph”, “pretty easy to read”, “I can see which task is
where”, “see my peaks and lows of attention easily”, “very intuitive, there is progression spatial
sense to it, from task 1 to 10. Very easy to take all the information with just a glance”, “it is
similar to the bar graph because it is easy to follow, its ambiguous that each task is different from
each other”, “I like is very variable depending on the activity”, “design is ugly. Makes me think
of integral (N)”, “It is too jagged (N)”, “it goes over multiple tasks over time. A broader
approximation it makes more sense. It seems a combination of the line graph and the bar graph”,
“It does not give me a feeling of any correlation, find it abrupt (N)”, “it is the easiest to read. I
feel it shows peak of attention. How the bottom half is filled make it easy to read”, “it is kind of
the same as the line graph, but less curvy. It is drawn from one dot to the next”, “I thought the
attention levels with the shade under, gives it a greater grade of scale”, “I like it is less curvy
than the line graph”, “it is more straightforward”, “sharp smooth is more clear.”
The line graph was the second preferred visualization for its clean visual presentation, how
it connects the points, and it is to understand. Unlike bar graph, this graph was slightly disliked
due to be sometimes misleading. “curve is clean”, “clean visual”, “easily see which task have
more attention, depth in attention within the task”, “similar to the area graph can see the trends of
the task and attention levels per each task”, “clearly see the data points, go from x to y values
very quickly”, “the gridlines are a little confusing. It has all the other good benefits; it has spatial
awareness to it. Each dot connects very well”, “it looks like a sine wave, it should be rounded to
the next point (N)”, “looks very steady, doesn’t have as much variables as the other ones, it
keeps going up and down (N)”, “some issues with it, implies linear relationship, makes it easy to
50
read”, “like the design better, fluid between the tasks”, “x-axis in respect to time because of
constant line”, “I like how it is connecting the points, you can see where it fluctuates”, “also, the
same as line graph, but prefer the bottom half to be colored”, “attention is fluid and graph is
more fluid than other ones. Over the course of each task. If the task is one after the other”, “it is
like brainwaves, fluid”, “it could be a little misleading sometimes (N)”, “the tasks are not
connected at all, so why a line? (N)”, “it is more straightforward”, “connection of the points
highlight the change in the data. Easy to interpret”, “easy on x hard on y, line easy to follow”,
“less line in the graph than area graph. Smooth pattern.”
The dots line was the least preferred graph with few positive feedback. The main issue
participants encountered was the time it took them to get the information they would want to get
from it. They wanted to glance at the graphs and see the attention levels quickly, they felt with
the dots graph it was not possible. “take more time to see what it was (N)”, “don’t know if I am
supposed to be looking at a trend (N)”, “difficult to understand the diagram, which tasks to see
which attention is low (N)”, “hard to see exactly where they are, like trends (N)”, “the dots don’t
tell me anything, it is hard to go to each point. I want to have something I can instantly see of
peaks and lows (N)”, “have to focus a lot, harder to trace as well. More tedious (N)”, “it’s the
same as the bar graph, really good representation. It’s better than the line graph and the area”,
“there is no such thing as tasks 2.5 or 3.5”, “hard for me to read and follow (N)”, “look at 2
different points, hard to see their levels (N)”, “it is scatted (N)”, “visual impartment may be
difficult to see (N)”, “at first glance, you wouldn’t know and have to carefully track to find the
information (N)”, “because they are just points, you have to look to see which one is really and it
can be a little confusing (N)”, “very difficult to read (N)”, “I cannot see anything. It is less
visual, it is a whole bunch of dots in a grid (N)”, “for me visually, it doesn’t say anything (N)”,
51
“because the other ones I didn’t select did not make sense to me”, “when I look at a scatter plot, I
automatically think of aviation (N)”, “had to compare different tasks (N)”, “data scattered on the
graph, hard to see any correlation (N)”, “cleanest to look and less lines. It is in a graph paper,
you can trace back”, “grid points are distracting (N).”
Third task: scales ranking
Table 3-4 shows the descriptive statistics of the results of the participants ranking the
scales. The scale 0-100 obtained the highest score followed by 1-5. Participants liked the 0-100
scale because it is easy to read and is like percentages. On the other hand, participants are used to
seeing the scale 1-5 in surveys and applications (like the five-star scale). Also, they like the scale
has a middle number (3), which helps them determine where they stand (low, middle, or high).
Therefore, if they are below the 3, then they know they should put more attention to get to the
positive side, which it is after 3. Based on the mean scores, 0-100 ranked first, 1-5 ranked
second, and 0-1 ranked third.
The Friedman rank sum test was performed to evaluate the significant difference in the
overall scores between the mean ranks of the scales, X2 (2) = 34.049, p = 0.000. Because the
Friedman test tells us that there are overall differences, but no the difference between scales, a
post hoc test was run to pinpoint the differences between scales. The Wilcoxon signed-rank test
shows that there was a significant difference in the preference students have in the scales to read
their attention levels in a visualization (Table 3-5). There were no significant differences
between the scales 0-1 and 1-5, between 0-1 and 1-7, and between 0-100 and 1-5. Nevertheless,
the comments provided by the participants on liking or disliking a specific scale helped
distinguish the preference among the non-significant differences.
52
The 0-1 scale had mixed feedback. The main reason for not liking this scale is because
many participants did not like decimals, which make it more complex. On the other hand, it is
liked because it is percentage and is like the scale 0-100. It is important to note, that
physiological data are mainly scaled from 0 to 1. However, participants did not find this scale to
be the best indicator for representing attention. “it is like decimals (N)”, “don’t like decimals
(N)”, “clear numerical progression, a scale not quite familiar because it is like stats, counter
intuitive with decimals”, “you know where it begins and ends. They are even in both sides”,
“more decimal places are unnecessary (N)”, “just as 100, but in decimals”, “decimals are not the
best scales (N)”, “everyone would try to avoid the lower spectrum”, “it is similar to 0-100, but
using decimals than whole numbers”, “doesn’t provide big enough range (N)”, “if I really want
to know what my attention really is, it is like percentage”, “it is overly difficult because its
decimal (N)”, “I like how many options there are. I like the whole 0-1 scale”, “decimals make it
seem more complex (N)”, “it is percentages makes more sense than everything else”, “it is the
same thing as 0-100”, “preference rather see decimals than whole numbers”, “it is equivalent to
0-100”, “felt the best, having a scale from 0-1 seems standard. Easy to convert to a fraction. Felt
right”, “pretty similar to percentage, its useful”, “remind me of percentages”, “really like
decimals gives you lots of options”, “because in programming I am used to 0-1.”
The 0-100 scale is the preferred scale with its highest ranking among all the presented
scales. “it is a longer range”, “the gap between 0-10 can be too big (N)”, “it is similar to a letter
grade”, “this reminds me when I used to get a grade, I’m 100% paying attention”, “I can easily
visualize that as a percentage, we use that for many other things”, “get annoyed by 0, equal from
each side in terms of interval. Easy to navigate and represent percentage”, “people who are
equally attentive, may choose different values when the scales are larger. 0-1, 0-100, and F-A are
53
the same because they are 11 points, they deal with less decimal places and get integer values. It
is easier than 0-1 to interpret”, “because it is out of 100, there is a lot of scale values”, “100 is a
good number”, “it is like percentage format”, “it has a wider range between each set”, “it is
similar to 0-1, but in values of 10”, “It is easily reading out, people can associate themselves
whether they spend 10% of a task paving attention. It has more meaning than the rest of them. It
is not a negative connotation to it. It is the most obvious and makes the most sense. Like 100%
means full attentive level. Most user friendly”, “it is like percentages”, “it is pretty obvious”,
“can relate to rounded numbers”, “similar to decimals, didn’t feel scientific”, “intuitive scale,
because of percentage. I feel in the USA that’s how people are graded”, “0-100 similar to 1-10”,
“it is really easy to understand”.
The F-A scale the least preferred scale because of the negative emotion that elicits to the
participants as it is related to school experiences. Few participants did not express concern over
this scale as they are used to it. “F has such a negative connotation (N)”, “like the letter grade
and used to it”, “it is like grading system, easy scale to understand”, “I didn’t like it, because I
don’t want them to grade me. I’m fine with the number grades not letter grades. Reminds me of
school too much (N)”, “don’t like letters, don’t like letter grade scales, hard to memorize what it
really means (N)”, “it is too detailed. It is too much, without the minus or plus would be better.
Too much to remember (N)”, “letter grade should not be given to one single assignment or level.
It is a good summary (of combination of grades). I have a hard time comparing grades. It is not a
good scale to measure that stuff (N)”, “scale should not vary from culture to culture, I don’t think
it fulfills its purpose. They are subjective (N)”, “the discrete values are not continuous therefore
liked the least (N)”, “it doesn’t give an actual value of where you stand (N)”, “not everyone does
well in their classes and it is related to graded system, I may not feel good if see a C- (N)”,
54
“don’t like the grading system (N)”, “like more numerical type analysis (N)”, “letter have a
familiarly with me and others (a means great job, F means terrible)”, “it contains F, D, D+, and
Cs, you feel a bit low if you get these letters. I see percentages as a Good figure in this manner
then I have proven certain number (N)”, “it has a negative connotation. Don’t like the F, it has
negative connotation. You automatically associate it with grades. It is confusing (N)”, “F has a
negative connotation. It is confusing with the minus and the plusses (N)”, “it is more relative to a
person in terms of good or bad”, “the letters are judgmental. It is like a grade you get in school.
Some people cannot pay attention, you cannot give them an F for that. Attention is based on
interest (N)”, “it is not telling the person what their attention really is (N)”, “if I were to get a C+
then I feel I didn’t do a good job. Cultural issues try to send that info to people (N)”, “don’t
know how it correlates to numerical values. What necessarily does this mean?”, “it is stupid.
Letter grades is not very quantitative (N)”, “don’t want to be graded. I associated it to school and
testing (N)”, “reminds me of school and never liked the grading system. Negative emotion
connection (N)”, “looks like grades. Attention shouldn’t be graded. Don’t like to be graded.
Starts with F goes to A (start backward). Include a D- and an A+ and inverse may change
perception a bit (N).”
The 1-5 scale is the second most preferred. Users are used to seeing this scale in surveys and
ratings of applications and other items on the web (i.e. movies, songs, etc.). They consider it
simple, easy to understand, and to follow. “very easy to understand”, “don’t have to think much”,
“it is a typical rating”, “reminds me of liker scale, 1 not paying attention, 3 neutral states, 5
extremely paying attention”, “I like scales from 1-5, it's easier to figure out where the middle is”,
“it is a bit restrictive because is only 5. I liked it more than the other options, but not really good
reason why I liked it”, “the scale is easy to follow”, “self-reporting their attention, you can say
55
three is neutral. Similar how you would rank on Yelp from 1-5 stars. Ranking good something
is”, “when you take a survey there is always 5 options”, “it has a low, medium, and high”,
“keeps it simple”, “because it is like 0-1”, “it gives you the low and high spectrum”, “it is a small
number and a fixed scalable number. If I get 4 out 5, it would feel good. If I get 1 still It is good,
because it is not 0”, “because you don’t have so many numbers to choose from. You have a
middle number”, “it is easy. When you got to a Doctor, they ask you about your state from a
scale from 1-5 or 1-10”, “it is more even than 1-7”, “1-5 has a middle number”, “many places
use 1-5”, “it might not give big enough range. 5 numbers are limited (N)”, “it would be easy to
follow. didn’t have enough options (N).”
The 1-7 scale is well received, because of its similarly to the scale 1-5 and provides more
options for further depth in attention score. “very concrete”, “wider range than 1-5”, “starts with
1 and ends with 7. it is a simple option”, “reminds me of liker scale”, “it is close to 1-5 scale,
where you can see the middle, don’t like the too many numbers”, “larger range can take people
off”, “it is the same concept as 0-5”, “more in depth than 1-5”, “gives more option than 1-5”, “it
has an abrupt number (N)”, “because there is a lot of option there, and there is a middle”, “I like
the scale more than 1-7 than having decimals”, “argument in liker scales, do more tests to
increase the fidelity from scale 1-5 or 1-7. Having 7 would be ok, having only 5 different
attention would have enough differences (N)”, “didn’t have enough options (N)”, “has more
levels than 1-5”, “like 1-7, because I have been working with scales 1-7. I have been dealing
with music a lot lately.”
Limitations
The limitation for this focus group is the usual limitation known for conducting this type
of study. There was difficulty to encourage some participants to provide as much feedback as
56
others were providing. Some participants found the focus group intimidating as they are not very
good at communicating with others. Also, the number of participants in the focus group varied
due to scheduling conflicts and some participants not showing up. Although, the focus group was
designed for the discussion between participants is necessary for good results, there was not a lot
of commenting on other’s feedback during the discussion.
Summary
The different datasets visualized in various quantified-self applications need to be easily
understood at a quick glance. This study answers the research question, “which static
visualization is preferred to represent attention among college students?” Students prefer the bar
graph to visualize attention with the 0-100 scale to see the exact value of their attention levels.
Although there was no statistical difference between the bar graph and line graph and the
clustered graph, the qualitative responses were more positive towards the bar graph. Also, during
the first task, the bar graph was the only graph that was drawn multiple times compared to the
other static visualizations. The preference for visualizing attention is implemented in the mobile
application to test the effectiveness of the quantified-self attention feedback from the brain for
improving attention during a learning task discussed in Chapter 5.
Chapter Highlights
The following lists are the main takeaways of this chapter:
The bar graph is the most preferred visualization, but there is not a significant difference
between the line graph and the clustered graph. However, the bar graph did not receive negative
feedback.
The 0-100 scale is preferred between the given choices based on the mean score and the
qualitative responses.
57
The formulation of visualization is based on participants’ perception on how they see the
changes of data, instead of researcher’s recommendations.
College students formulated and selected their visualization and scales based on previous
experience and how they read information.
Figure 3-1. Raw EEG from Emotiv test bench
Figure 3-2. Focus group’s room design
58
Table 3-1. Focus group steps
Steps Task Length
Step 1 Consent Form N/A
Step 2 Demographic Survey N/A
Step 3 Visualization Drawing 10 minutes
Step 3.5 Visualization Drawing
Discussion
N/A
Step 4 Static Visualizations Ranking 3-5 minutes
Step 4.5 Static Visualizations Ranking
Discussion
N/A
Step 5 Scales Ranking 3-5 minutes
Step 5.5 Visualization Scales
Discussion
N/A
Step 6 Open Discussion: final
comments and questions
N/A
Figure 3-3. Bar graph
Figure 3-4. Filling meter
59
Figure 3-5. Bulls eye
Figure 3-6. Light bulb
Figure 3-7. Eyes
60
Figure 3-8. Speedometer
Figure 3-9. Line graph
Table 3-2. Visualization: descriptive statistics
Visualization Median Values Mean STD
Bar Graph 2.0 1.73 1.20
Clustered Graph 1.0 1.12 1.14
Pie Chart .00 0.66 0.91
Area .00 1.02 1.19
Line 1.0 1.34 1.15
Dots Line .00 0.27 0.74
61
Table 3-3. Visualizations: Wilcoxon Signed-Rank test results. Significant values are noted with
the symbol *
Visualization Comparison Z-Value P-Value
Bar Graph-Clustered Bar -2.076 0.038*
Bar Graph-Pie Chart -3.902 0.000*
Bar Graph-Area -2.208 0.027*
Bar Graph-Line -1.071 0.284
Bar Graph-Dots Line -4.602 0.000*
Clustered Bar-Pie Chart -1.876 0.061
Clustered Bar-Area -0.350 0.726
Clustered Bar-Line -0.758 0.448
Clustered Bar-Dots Line -3.026 0.002*
Pie Chart-Area -1.627 0.104
Pie Chart-Line -2.466 0.014*
Pie Chart-Dots Line -2.113 0.035*
Area-Line -1.411 0.158
Area-Dots Line -2.688 0.007*
Line-Dots Line -3.818 0.000*
Table 3-4. Scales: descriptive statistics
Scales Median Values Mean STD
0 – 1 1.0 1.37 1.26
0 – 100 2.0 1.83 1.89
F – A .00 0.39 0.83
1 – 5 1.0 1.51 1.12
1 – 7 .00 0.90 1.16
Table 3-5. Scales: Wilcoxon Signed-Rank test results. Significant values are noted with the
symbol *
Scales Comparison Z-Value P-Value
(0-1) - (0-100) -2.143 0.032*
(0-1) - (Z-A) -3.029 0.002*
(0-1) - (1-5) -.559 0.576
(0-1) - (1-7) -1.349 0.177
(0-100) - (F-A) -4.784 0.000*
(0-100) - (1-5) -1.244 0.213
(0-100) - (1-7) -3.026 0.002*
(F-A) - (1-5) -3.756 0.000*
(F-A) - (1-7) -2.225 0.026*
(1-5) - (1-7) -2.410 0.016*
62
CHAPTER 4
ENGAGEMENT APPLICATION PROTOTYPE
The mobile application design process consisted on brainstorming, low-fidelity
prototype, and high-fidelity prototype. These steps were done before the development phase. The
designs were not formally tested in a usability study. However, it was informally tested with
some college students for additional feedback on user friendliness.
Brainstorming
During the brainstorming session, we considered different QS mobile application designs.
The position of the buttons, color schemes, and user interaction for selected mobile application in
the market were considered and used the common qualities of the applications for designing the
prototype. After the brainstorming session, we proceeded to design low-fidelity wireframes.
Low-Fidelity
The low-fidelity wireframes (Figure 4-1) were designed in Balsamiq. The purpose of this
step is to serve as a blueprint that dictates the interface’s information architecture for the
positions of the buttons, illustrations, user’s instructions, space allocation for each item, and flow
of user interaction. Also, to show the connection between the screens when the user navigates
through the interface.
High-Fidelity
In this step, we included more details, made type choices, and incorporated the low-
fidelity wireframes to wireframes that symbolize the actual look and specific dimensions of the
application when it's developed (Figure 4-2). The wireframes were designed on Adobe Illustrator
and created a prototype in Invision to test the look, feel, and user interaction with the application.
The test was done informally asking random users their opinion. Also, to test the complexity of
63
user interaction for immediate changes before development. Users would follow the following
steps according to Figure 4-2 (from top left to bottom right):
1. Sign-up or login with their e-mail and password.
2. If sign-up, input their e-mail and password.
3. The relaxation instruction screen shows up, where the user click start once s/he is
ready.
4. A countdown of the relaxation (alpha) phase is shown.
5. After the countdown reaches zero, the user can select one of the pre-defined tasks
and start recording.
6. A recording screen is displayed with a stop button.
7. A visualization of the user’s attention level for a task is showcased for feedback.
Application Architecture
The application architecture (Figure 4-3) starts with the user wearing a BCI device for
data acquisition. The device communicates with the Emotiv application via Bluetooth to
establish good signal quality between the user’s scalp and the EEG sensors. The Emotiv
application sends connection and signals quality information to the prototype once the
engagement application has been opened. The prototype also receives the EEG data from the
BCI and stores it in excel sheet files for further analysis.
Within the prototype, there are computations done by the EEG wave converter and
information inputted by the user in the user interface. In the user interface, the user can input
their name, log their relaxation data during the alpha phase, select a learning task they are about
to perform, and see the bar graph of their attention levels (Figure 4-5) after completing a task. In
the back-end, there are calculations performed by the EEG waves converter. It is converting the
raw EEG signals to spectral bands (alpha, low beta, high beta, and theta) with Fast Fourier
64
Transform (FFT). Afterward, it obtains the absolute value of the EEG bands and normalizing the
values to the scale from 0 to 100 to show the attention value in the bar graph.
User-Mobile Interaction
The application was developed in the iOS environment after it was designed. The user
must use both the Emotiv mobile application developed by them to establish the connection
between the device and the phone via Bluetooth. Also, to obtain good signals from the channels
when they are touching the scalp of the user. The application shows different colors portraying
the quality of channel placement for data acquisition (Figure 4-4). Black represents no
connection at all, red is a poor connection, orange is a good connection, and green is a perfect
connection. The colors change when the user moves the hair around and move the channels to
touch the scalp through the hair. Furthermore, the connection of the reference on the bone behind
the left ear can be seen in the head map.
Once the connection has been well established, the user proceeds to our engagement
application where they input their name, perform the alpha calibration, record their engagement
during the learning tasks, and see the visualization of a bar graph of their engagement level after
the completed task.
65
66
Figure 4-1. Low-fidelity wireframes
Figure 4-2. High-fidelity wireframes
67
Figure 4-3. Focus group’s room design Engagement application prototype architecture
Figure 4-4. EEG channel good signal qualities from Emotiv application except for channel Pz
68
Figure 4-5. Bar Graphs used in the mobile application to showcase attention levels
69
CHAPTER 5
QUANTIFIED-SELF ATTENTION FEEDBACK
The goal of this experiment is to assess the effect of presenting quantified-self attention
feedback gathered from an EEG apparatus in the form of a visualization to a college student after
completing a learning task. This includes improvement of attention from the first learning task to
the second learning task. Also, assessing their self-initiative to try to improve their attention even
if their attention does not significantly improve. This study seeks to answer the following
research questions identified with their hypothesis:
RQ2: What are the effects of quantified-self attention feedback from a BCI on the
short-term increment of attention from the first video to the second educational video
among college students?
H1: Participants who were not introduced to attention feedback have a lower
improvement of attention in the second task than the participants who obtained feedback.
H0: Participants that were not introduced to quantified-self feedback will have the same
or higher improvement of attention in the second task than the participants who obtained
feedback.
RQ3: What is the effect of attention levels per index on quiz scores?
H2: Participants who receive attention feedback will have a higher assessment score than
the participants who did not obtain feedback.
H0: Participants who did not receive attention feedback will have the same or higher
assessment score than the participants who obtained feedback.
RQ4: What is the correlation between quiz scores and attention levels per index
when feedback is presented to the user?
H3: There is a correlation between quiz scores and attention levels for at least one index.
70
H0: There is no correlation between the quiz scores and attention levels for any index.
Experimental Design
Participants
Seventy participants (51 males, 19 females) from University of Florida participated in the
study. The participants were randomly placed in one of the four groups. 74.29% of the
participant’s age ranged from 18-21, 18.57% were 22-24, 5.71% were 25-28, and 1.43% were 32
or above. They were from different ethnicities, 31.43% were Caucasian, 8.57% were African-
American, 22.86% were Hispanic or Latino, 28.57% were Asian, and 8.57% identified as other.
The others were identified as Black or Biracial. The college students were both undergraduate
and graduate. When identified by colleges year, 24.29% were Freshman, 22.86% were
Sophomore, 24.29% were Junior, 20% were Seniors, and 8.57% were graduate students.
21.13% of the participants used a BCI before in a different context. Some participated in
the previous Brain-Drone Race in 2016 [67], some participated in a study, while others used it
for a school project, or previously attached electrodes to their head.
Some of the participants received extra credit from their courses, others were interested in
winning the apple watch, and other participants were just interested in the study. They all
completed the same exact task; the difference was the option to see their attention feedback,
which was dependent on the group. The demographic questions along with the entire survey can
be seen in Appendix F.
Table 5-1 shows sample BCI studies from the CHI conference with their sample size.
This dissertation’s sample size is 70, which is relatively larger than the ones indicated in Table 5-
1. Also, it is larger than traditional BCI studies, which in many cases consists of fewer than 10
subjects.
71
Table 5-1. aBCI/HCI EEG studies with sample sizes, number of conditions, and number of
channels used in the study
Ref
# Authors N
No. of
Conditions
No. of
electrodes
Electrodes
Used BCI Device Venue
58 Hassib, M., et
al.
11 1 1 FP1 Neurosky
Mindwave
CHI’17
38 Huang, et al. 24 1 14 All 14 Emotiv
EPOC
CHI’14
36 Szafir and
Mutlu
48 4 1 FP1 Neurosky
Mindwave
CHI’13
35 Szafir and
Mutlu
30 3 4 FP1 Neurosky
Mindset
CHI’12
59 Vi and
Subramania
n
12 14 F3, F4, F8,
FC6, FC5,
AF4
Emotiv
EPOC
CHI’12
Apparatus
The attention was obtained from the participant’s brain electrical signals with the non-
invasive wireless wearable Emotiv Insight (Figure 5-1) (www.emotiv.com). Emotiv Insight is a
5-channel EEG device that acquires electrical signals from the brain. The five channels’ (AF3,
AF4, T7, T8, Pz) locations are based on the 10-20 international system (Figure 5-2). Its
references are in the CMS/DRL noise cancellation configuration. The signal resolution of the
device has a data transmission rate of 128 samples per second per channel. The minimum voltage
resolution is 0.51μV with a frequency response of 1-43 Hz. The device connects to the computer
or mobile phone via Bluetooth (4.0 LE).
The Emotiv Insight was used among the other devices because of its ability to obtain
EEG data wirelessly. As the purpose of this research is to investigate the effectiveness of
quantified-self attention feedback from the brain for healthy students, it is important to use a
consumer-based BCI instead of a medical based BCI to maximize positive user experience.
The second apparatus is the iPhone 7 mobile device. The phone was used to save the
attention levels of the participants and visualize it when they finish the learning task as a bar
72
graph. The third apparatus was the MacBook Pro laptop computer used to play the relax melody
sounds during the alpha step and to answer the survey questions along with watching the videos.
Procedure
The experiment was performed in a quiet room where no internal ambient noise was
present. Once the participants arrived at the study room,they were seated in front of a monitor
where they would be answering the survey questions and watching the videos. Once they were
seated, a consent form was given to them to read about the study logistics, risks, and benefits.
Once the participant signed the consent form, they proceeded to complete the pre-survey. The
pre-survey consisted of questions regarding their current affect (PANAS survey) as their
attention performance may vary based on what they are experiencing, and their experience using
quantified-self tools. Afterward, the BCI device was mounted on their head assuring the channels
in the frontal lobe (AF3, AF4) had a good connection to receive good data. Once good signals
had been established, the alpha phase was conducted where it lasted 10 minutes. During the
alpha step, the participants were instructed to relax and not think about anything while listening
to an ocean wave and Alpha white noise. Also, they were instructed not to make any muscle
movement. A questionnaire was given to the participants once they completed the alpha step to
understand if they were thinking about anything during the relaxation procedure. Afterward, the
first learning task was given where they had to watch the video from Ted-Ed, “How Chronic
Stress Affects Your Brain.” They were instructed not to make any muscle movement while they
were watching the video. Group 1 and Group 3 saw their attention in a bar graph on the mobile
device after watching the first video. Then, they completed a quiz regarding the video to assess
their acquired knowledge. The quiz questions consisted of 5 multiple-choice questions provided
by the creators of the video. The second video task consisted of the same process, but they had to
73
watch the video, “The Effects of Sleep Deprivation.” The last two steps consisted of a post-
survey asking their current affect after completing the study and demographic information. Also,
a brief interview (audio recorded) was conducted to obtain insight into their perception and
mental strategy, if any. Figure 5-3 shows a diagram of the study process.
Design
The experiment was a counterbalanced between-subjects study where the quantified-self
attention feedback was presented at different times, both times, or not at all. The study consisted
of four groups: one control group and three experimental groups. The control group represented
the traditional and current way of self-learning where learners do not receive attention feedback.
The experimental groups received attention feedback for both tasks or one task. Table 5-2
describes when attention feedback was presented for each group along with the ID used to
represent them in this work. The dependent variables were attention levels and quiz scores,
where the independent variable is the times the quantified-self attention feedback was presented.
Both videos from Ted-Ed were chosen because they adhered to recommendations on their
characteristics from [54]. These characteristics are the length, the intensity, core themes,
presence and number of human figures, brightness, and picture motion. Also, the videos “How
Chronic Stress Affects Your Brain” [55] and “The Effects of Sleep Deprivation” [56] were
relatable to college students because they tend to experience these situations; specifically, during
exam season.
Both objective and subjective measurements (surveys managed by Qualtrics) were used to
obtain data as seen in Figure 5-4. EEG was the physiological measurement used to obtain
attention levels from the learner when they were watching the videos. Audial feedback of the
learner’s experiences was recorded during the interview step. The interview questions included
74
some reflective questions from [60]. The questionnaire (survey) asked questions related to
demographics, experience with other quantified-self technologies, assessed affect with PANAS,
asked task assessment questionnaire, and included the quiz questions related to the videos.
Figure 5-1. Five channels Bluetooth-based non-invasive BCI device – Emotiv Insight
Figure 5-2. Emotiv channel locations in the 10-20 international system identified by red circles
75
Figure 5-3. Study steps
Table 5-2. Groups descriptions
Groups ID Description
Control Group NFNF No Feedback-No Feedback.
Do not receive feedback
for both tasks
Experiment Group 1 FNF Feedback-No Feedback.
Receive feedback only for
the first task.
Experiment Group 2 NFF No Feedback-Feedback.
Receive feedback only for
the second time.
Experiment Group 3 FF Feedback-Feedback. Receive
feedback for both tasks
Figure 5-4. Collection methods used to collect data. Adopted from [68]
76
Results and Discussion
The statistical analysis for all the tests presented in this section was conducted with the
statistical tool, SPSS.
Alpha Phase
Before the participants started the learning tasks, the participants proceeded to do an
alpha/relaxation phase. The purpose of this phase is to relax the participant so they can have
better concentration during the tasks. Daily activities before the study may affect the ability of
the participants to perform the tasks. Therefore, the relaxation step is performed for relaxing the
students to get them ready to learn new content.
During the relaxation step, the participants are supposed to not think of anything. However,
it could be difficult for some college students to blank their mind. After the Alpha phase, they
answered the question, “did you think of anything while doing the alpha calibration?” 77.14% of
the participants said they were thinking of something.
The list shows that most of the participants had a hard time trying to relax and blank their
mind. The main thoughts can be categorized as things that they need to do (studying for an
exam), family and friend experience, thoughts related to the beach, other daily routines, and a
combination of multiple thoughts:
1. I cannot stop thinking about seas, then about cartoons, computers.
2. Although I attempted to think about “nothing”, my mind still thought about what I had to
complete today or what is needed to get down.
3. My plans for the future and my exam tomorrow.
4. Memories of nights at the beach.
5. Depends – usually pleasant situations.
77
6. My day today, things I have to do, friends, how pretty the sound is, or location/origin of
the sound.
7. Thoughts from family, recent events, friendships and all sorts of daily things.
8. The sound of the wave made me think of water.
9. Being on a beach at night as the ocean waves crash onto the sand. Staring at the night
sky watching the stars on top of a hill.
10. Thinking about sleeping.
11. Thoughts of today, what I did and accomplished. Other small things such as breathing
and noises in the environment.
12. I was thinking about the beach and times my friends and family had gone there. Also,
about current situations that were on my mind before coming to this study.
13. I had some slight thoughts about upcoming assignments, a previous exam, and what I
was going to get for dinner tonight.
14. They ranged from various things going on in my life at the moment including looking for
housing for the summer or my plans for this weekend.
15. I went to a beach with my friends.
16. Random things, but mostly beach related experiences.
17. Thought about upcoming things I have to do, I thought about what I had done with my
friends recently. I thought about being on a beach.
18. I thought about things pertaining to the ocean, about background noise in the room, and
about a very irritating itch I had on the back of my neck.
19. I was thinking of how to not move my muscles. After a while I began to think about my
programming projects and my assignments due.
78
20. Trying to not think. Trying to focus on something relaxing like the beach.
21. Random thoughts ranging from trying to focus on my breath and clear my mind, and
thoughts and homework I need to do.
22. I sang one or two songs in my head for most of the time. other times, I thought about my
breathing, sometimes while still singing a song. When my nose and mouth itched, I
thought about whether I should scratch them. When I scratched my nose I hoped I
wouldn't get in trouble. At the beginning, I thought about how I don't really like these
headphones and would prefer different ones, then later I thought about how they weren't
too bad. At one point, I thought about how it sounded like the ocean was just continually
coming onto the shore (like it wasn't waiting long enough between waves for the water
to go back into the ocean). Then later I thought maybe I heard the sound of the waves
going back. At one point near the beginning, I tried to imagine myself sitting on the
beach. Multiple times I thought about how the head thing hurts, both near my left ear and
how it feels very uncomfortable having the pressure on my forehead.
23. How much I wanted to move.
24. I was thinking about the sounds I was hearing during the calibration as well as random
thoughts that just appeared in my mind, such as homework and my friends.
25. Breaking up with my girlfriend.
26. I was thinking of school-related assignments and upcoming internship related tasks.
27. I thought about the seaside and general classrooms programs I attend here at UF.
28. I was thinking about the beach and being at the beach. I also thought about things are
going on in my life, especially what I have to do today.
29. Random thoughts about the day and memories that reminded me of the beach.
79
30. Sporadic short thoughts before trying to think about nothing again.
31. Random thoughts of what I need to do after this. The ocean as well.
32. I don’t remember. There were just thoughts and memories about things and people that
flashed through my mind.
33. Laying in bed with my girlfriend at night because that is usually when I am most relaxed.
I also thought about laying on the beach in the hot sun.
34. School stress, relationship stress.
35. Just relaxing on a beach.
36. I thought of sitting by the beach with my girlfriend. Other random thoughts, such as what
this study would be about popped into my head, but I didn’t really focus on them too
much.
37. Travel plans, work, what I have to do after this.
38. I was trying to figure out how much time had passed.
39. Water, the beach, waves, and the moon.
40. I have anxiety so when there’s no stimulation for my brain it tends to go towards
thinking about anxious thoughts.
41. They were random, I had thoughts of fishing, swimming, a movie, what type of data this
BCI is gathering, memories of friends.
42. Girl problems and football.
43. I thought my nose itched a lot but I couldn’t move to scratch it.
44. Down and out of my head. Deep breaths. Separate.
45. I was wondering why I couldn’t fall asleep.
80
46. Just idle things about my surroundings, the study itself, and my previous uses of BCI. I
find the study very interesting; my interest was the sole reason I came here to participate.
As such, I found myself occasionally thinking about the implications of BCI, and other
such things. Nothing major or concrete formed in my head, and it was mostly just
physical sensations.
47. Standing in the sea.
48. Visions of water, a beach with waves crashing, being near the shore by rocks with my
feet in the water only seeing my legs and feet. A memory from childhood at the beach
with my mom, and the later deeper memories of my late grandparents by the beach
lighthouse in Key Biscayne Florida we used to visit. Intentionally focused back on
sounds to clear mind.
49. I thought of being at the beach watching waves break on the shore from left to right
while the sun was rising.
50. Prayed. Ocean blue. A Little bit of white with a little bit of green. The memory of zip
lining in Colombia meditated/remembered cruise voyage I took from Cartagena to
Lisbon in the summer gratitude. Humility. (Which are established weaknesses of mine
according to positive psych) Peace. Questions about artificial intelligence popped into
my mind at some point in the beginning. Love.
51. A lot of it was at the beach imagining the ocean waves other thoughts included things
that happened throughout my day like going to class.
52. During the alpha calibration, I was thinking of beach because of the ocean sound and my
trips there during spring break. I was also thinking of what I was going to do throughout
the week.
81
53. I was thinking about the tasks for the next day. Random thoughts about my home flashed
in between.
54. Initially, I was successful at having a blank mind and relaxing. Once I got relaxed, and I
felt my body lose tension, I would often find myself thinking/day dreaming about
scenarios and pending tasks in school and relationships. It was a lot of imagination, very
similar to dreaming.
Emotion Assessment
Students experience both positive and negative activities throughout the day that may
result in mental distractions when they are doing learning tasks. Therefore, the PANAS results
show how positive and negative their affect was before starting the study and once finishing the
study. Table 5-3 represents the descriptive data for each group. Pre-PANAS was assessed at the
beginning of the study session when the participant started filling out the pre-survey. It assessed
their affective state at that specific time. The Post-PANAS was given to assess the participant’s
affect after they have experienced the given task. It is to deduce if there was a change of emotion
both positively or negatively. Table 5-4 shows the descriptive statistics for each individual affect
before the study and after completing both tasks.
We conducted a Cronbach’s alpha test to measure the internal consistency of the emotion
pre and post responses for each group. In Table 5-5, we can see a high level of internal
consistency for the PANAS scale overall for all groups. Table 5-6 shows high level reliability of
each specific affect per group.
Knowledge Assessment
The knowledge acquired from the participants based on the video content was acquired through a
five-question, multiple choice quiz. The quiz’s questions were formulated by the experts of the
82
topics and provided in Ted-Ed. For validity purposes, these questions were used as experts
prepared them. The quiz’s score ranged from 0-5 with each question weighting one point. Table
5-7 shows the descriptive statistics of the quiz scores and time taken to complete the quiz for
both videos for each group. A one-way ANOVA was conducted to determine if there is a
difference of quiz scores between groups. There is no significant difference in quiz scores across
the four groups (F1,3 = .874, p = .459).
The questions designed by the experts were also assessed in the survey for their clarity
and readability so the participants can answer them. After the quiz, the participants selected their
agreement from a 1 (strongly disagree) – 5 (strongly agree) scale for the statements provided in
Table 5-8, which shows the responses per group for each quiz. We can see that the questions
were clear to understand, easy to answer, were related to the videos, and were useful to test the
knowledge of the participants.
Table 5-9 shows the time taken for each participant to complete the quiz for each video.
The time was recorded by a function within Qualtrics. The functions recorded three sets of time,
first click, last click, and process time. The first click refers to the first answer choice made by
the participant, last click is the last choice made, and process time is when the quiz’s answers are
submitted. The purpose of the completion time logging is to see if there is a correlation between
the score and the amount of time taken to complete the questions. This comparison helps to
determine if the participants thought of their answers or simply guessed [57]. A Pearson’s
Product-Moment Correlation test was conducted to determine if there is a correlation between
quiz scores and the amount of time taken. The quiz for video 1 and its completion time were not
correlated (r = -.137, p = .261). However, there is a negative correlation between the second quiz
and its completion time (r = -.602, p = .000). The longer it takes the learners to complete the quiz
83
the smaller the quiz score. Figure 5-5 shows a scatterplot of the negative linear relationship
between the second quiz and its completion time (process time data of Table 5-9).
Effect of Quantified-Self Attention Feedback
Attention levels were measured from EEG signals through post-processing to obtain
spectral band waves of alpha, beta (low and high), and theta through Fast Fourier Transform
(FFT). The data was processed through a high-pass filter prior to applying FFT. The quantified-
attention feedback consisted of the values obtained from the index beta/(alpha + theta). This
index has been utilized in several Human-Computer Interaction (HCI) studies for education and
learning tools [35, 37-38, 58]. In addition, we also evaluated the significance with other indices
tested in previous work [39]: beta/alpha (low beta (at 12-16 Hz)/alpha, high beta (at 16-25
Hz)/alpha), and frontal alpha asymmetry (log (alpha power right [channel]/alpha power left
[channel])). The alpha asymmetry extracts the alpha power from the raw electrical signals and
computes its power and divides the left hemisphere by the right hemisphere of the brain. The
right/left ratios of alpha tend to be higher for verbal tasks, which engage in the left hemisphere,
than spatial tasks in the right hemisphere [60]. Also, beta (low and high) bands were measured as
they have been related to attention from the frontal lobe [67]. In this work, the indices were
applied to both channel locations AF3 and AF4. In Table 5-10, we can see the mean and standard
deviation (STD) of the engagement indices for both tasks for all the groups. Mean1 and STD1
correspond to the first video (task 1), and mean2 and STD2 correspond to the second video (task
2). Respectively, Table 5-11 (NFNF), Table 5-12 (FNF), Table 5-13 (NFF), and Table 5-14 (FF)
show the mean and standard deviations values for each group separately.
RQ2: What are the effects of quantified-self attention feedback from a BCI on the
short-term increment of attention from the first video to the second task among college
84
students? An analysis of covariance was done to see if there is an increment of attention
between the first and second tasks. There is a significant difference in the index 𝐻𝑖𝑔ℎ 𝛽/𝛼 in
AF4, P = .033. Table 5-15 shows the One-Way ANCOVA results for each engagement indices.
After finding a significance difference in the one-way ANCOVA for 𝐻𝑖𝑔ℎ 𝛽/𝛼 in AF4, a
post-hoc test was conducted to distinguish the difference in attention levels between each group.
However, before the post-hoc test, a Shapiro test was conducted to test for normality in the data
for each group. Groups NFNF (P = .955) and NFF (P = .854) are normally distributed and
Groups FNF (P = .000) and FF (P = .005) are not normally distributed.
Because there was not a normal distribution for all groups, a Wilcoxon signed rank test
was conducted. It showed that there is a significant statistical difference in increment of attention
between not presenting feedback at all (NFNF) and presenting feedback the first time (FNF) (Z =
-3.292, p = .001) and not presenting feedback at all (NFNF) and presenting feedback the second
time (NFF) (Z = -2.341, p = .019). Nevertheless, there was not a statistical significance in
increment of attention between not presenting feedback at all (NFNF) and presenting feedback
twice (FF) (Z = -.770, P = .441), presenting feedback the first time (FNF) and the second time
(NFF) (Z = -1.885, p = .059), feedback the first time (FNF) and feedback both times (FF) (Z = -
.287, p = .774), and presenting feedback the second time (NFF) and both times (FF) (Z = -.650, p
= .516). In Figure 5-6, we can see the difference between groups in the box plot.
RQ3: What is the effect of attention levels per index on quiz scores? To evaluate the
effect of attention levels on quiz scores a multivariate analysis of variance was run. We tested to
see if the higher the attention, the higher the quiz scores. In Table 5-16, we can see the results
from the One-Way MANOVA per index for each channel. Overall, there is no statistical
significance. Therefore, we fail to reject the null hypothesis where participants did not receive
85
attention feedback will have the same or higher assessment score than the participants who
obtained feedback. Therefore, the attention level increment did not influence the quiz scores.
RQ4: What is the correlation between attention levels per index and quiz scores
when presented feedback to the user? The Pearson’s product-moment correlation test was
conducted to determine if there is a correlation between the quiz scores and attention levels
across groups. There is a negative correlation between quiz 2 and low beta AF3 2 (r = -.288, P =
0.016) as seen in Table 5-17. Therefore, we reject the null hypothesis. This entails that there is a
correlation between quiz scores and attention levels for at least one index, in this case, low beta
at the AF3 location.
Qualitative Responses
The purpose of the interviews was to obtain further feedback on the participant’s views
and experiences on utilizing a BCI for monitoring their attention to improve it. The control group
had fewer questions than the experimental groups because the participants did not obtain any
feedback during the study. The responses are described below (statements with (N) is annotated
as a negative comment) and denoted by participant’s IDs (PNFNF1, PFNF1, PNFF1, PFF1…).
Responses categories are noted with group IDs (NFNF (no feedback, no feedback), FNF
(feedback and no feedback), NFF (no feedback and feedback), FF (feedback and feedback))
indicating that the question was asked in those specific groups. The list of questions can be seen
in Appendix G.
What was going through your mind when you were watching the second video?
(NFNF, FNF, NFF, FF). The thought in the participant’s mind while watching the second video
provide insight on the distraction of thinking about other things and any mental strategy to pay
more attention. As the control group did not receive any feedback, they did not have an extra
86
thought that they would be thinking about compared to the other groups. Participants could relate
to the content of the videos because these are occurrences that college students experience each
semester. Especially during exam season, students tend to sleep less and get more stressed when
the dates start getting closer and they need to manage between finishing projects, studying for a
test, and other activities they are involved in.
When participants were watching the video, they were distracted several times by
thinking about the times when they are stressed or sleep deprived or when they know someone
who does. “More empathy to myself, I was thinking how sleep deprived and the people I know
are, got sidetracked as I started thinking about my friend” (PNFNF1), “tried to relate it to my
own experience and reasons of why I can’t fall asleep” (PNFNF4), “whenever I was distracted, it
was because of the music and the new knowledge, but I was like “oh oh” I am getting
distracted”, (PNFNF7, “should pay attention to it, often I don’t get a lot of sleep and related to
my life” (PNFNF8), “was thinking about the amount of sleep I got last night” (PNFNF9), “it was
distracting because it relates to myself and my family” (PNFNF10), “related to how I don’t get a
lot of sleep, worried me of the consequences” (PNFNF12), “thinking how I can fix my sleeping
habit and relate to it” (PNFNF15), “during the first video, I thought of finals coming up and
stress I was feeling about it. During the second video, I was just listening because my roommate
has insomnia” (PNFNF16), “how I can retain much information for the quiz and how I can keep
distraction away” (PFNF2), “trying to think more, change the way I try to pay attention”
(PFNF3), “tried to concentrate a little bit more after saw my attention level” (PFNF4), “I was
thinking less about other things than the first video. Trying to focus more and be more prepared
in the second set of questions” (PFNF5), “I didn’t pay attention to the first video and spaced out.
I was trying to focus on the content of the second video” (PFNF6), “recalled some articles and
87
videos I have seen about the topic” (PFNF7), “related to it, I experience sleep deprivation”
(PFNF8), “I don’t sleep enough (that’s me), how negatively is affecting my health” (PFNF9),
“tried not sleeping for 3 days and felt asleep” (PFNF12), “kind of surprised how sleep
deprivation cause on some people, try to sleep to do better in my classes” (PFNF14), “dealing
with some problem sleeping because of exams, taking caffeine pills, afraid of sleep deprivation,
had my own thoughts as well” (PFNF16), “just the content of the video, sleep deprivation relates
to me personally, I am interested to learn about it” (PNFF1), “I didn’t sleep so much and
probably going to die soon because of it” (PNFF3), “thinking about myself, I should really
change the way of life” (PNFF8), “thinking about how much sleep I usually get and the effects of
not having much sleep. Started to think what I should or shouldn’t do more often” (PNFF9),
“related to me as I only got 5 hours of sleep” (PNFF11), “a video of interest of mine, I
experience sleep deprivation. Just want to see how it affects me and my stress level” (PNFF12),
“I was able to think of times I lose sleep myself” (PNFF14), “Sleep deprivation is something I
can relate to” (PFF4), “thinking about the number of hours of sleep I get, I feel good about that”
(PFF7), “thinking how much sleep I got, must of the time related to the video” (PFF10), “got
distracted by drawing connections to myself” (PFF14), “I am lucky that I sleep 7-8 hours”
(PFF18).
Another type of thought that serves as a distraction was keeping up with the scientific
jargons presented in a short amount of time in the videos. However, it was also a motivation for
some to pay more attention as they did not know the terms and knew they were going to be
quizzed on the content. There was also an attempt to focus on learning the content of the video,
especially after some participants thought the first video contained a lot of jargon, they expected
it from the second video. “Tried to put more focus on the video” (PNFNF2), “tried to get as
88
much information as I could” (PFNF16), “in the first video, I was less focused on the
information. Focused more on the information after seeing the questions from the first video”
(PFNF17).
Finally, the device served as a distraction to some participants as it felt tight on their head
and gave some users a mild headache. People have different head shape and sizes, which leads to
different ergonomic experiences while wearing a BCI. Despite the mindset to design a more
aesthetic and comfortable device by the BCI startups, current devices still do not cover all the
head’s shapes and sizes. “thinking about the video, nearing the end getting distracted thinking
about the head gear. Started to get pressure in the ear from Emotiv” (PNFNF3).
Is there anything that can help you become more aware of your attention for a
learning task? (NFNF). Self-awareness of performance during an activity can be important to
perform a task well and obtain the results the person is striving to achieve. Self-awareness of
attention level could aid learners to improve their attention during or after they perform their
learning task over time. Students do not receive feedback in their attention levels when they are
studying. In this era of digital media, students participate in more self-learning by watching
online videos on YouTube, Ted Talks, and other online sites to obtain further knowledge or to
understand a material better. Participants were not aware that attention could be measured and
visualized using a BCI or with other apparatus until their participation in the study. Therefore,
they were not able to give a concrete response that would help improve their attention or make
them self-aware. “Have 1 or 2 questions every 10 minutes or so in a lecture, for short term
memory” (PNFNF1), “I should be able to control my thinking. Focus one thing at a time”
(PNFNF2), “the idea of the assessments afterwards helps or even talk about it and describe what
you just saw.” (PNFNF6), “I don’t use anything, I would be interested about something. I am
89
interested into self-improvement” (PNFNF7), “the fact that I knew that my attention was being
monitored, it made me aware” (PNFNF8), “sleeping more often. Relaxing was really good, I
have a lot of noise in my mind. Hear myself thinking, hear my own voice. Doing the alpha
calibration the noise went away” (PNFNF9), “trying to become more mindful of everything in
general” (PNFNF11), “definitely get more sleep. It’s hard to keep attention right now”
(PNFNF12), “I don’t see how awareness would help you” (PNFNF13), “definite see if someone
has disorders or not like ADHD through screening and testing” (PNFNF15), “if I notice
distraction and go back to paying attention” (PNFNF16), “what helps your attention span is
closely related to interest, then your attention will be good. You can tell you are not
concentrating, but you can’t tell you are focused until you are done” (PNFNF17), “if it is
something of interest to you then it’s easier to stay engaged. It is always looking at the key
concept” (PNFNF18).
Do you think if I show you your attention gathered from your brain on a mobile
device, you would be more aware of your attention and help you improve it? How?
(NFNF). The control group did not see their attention levels as it is how students perform
learning tasks, without any provided feedback. The participants thought that such feedback
would be useful to improve their attention either by seeing the feedback during the task or
afterward. 14 of the participants agreed that it would be helpful, two said maybe, one didn’t
know, and one did not think it would be helpful. “It is good to have such thing. Would not be
good to use it while watching the video, I would prefer it if I am done with a study section. I
think it will help me improve my attention” (PNFNF1), “I don’t have a mean to measure my
attention. A list of questions is the only mean. After the video show the attention feedback. If it
is during the video, it would distract me. I can use it for the next task if you show it after the
90
video” (PNFNF2), “it would be interesting, to see at what point I was more alert and if I was
more focused in the first video or second. During the video, I would like to see the attention. It
was easier to think of any other things because I just had to listen” (PNFNF3), “I can keep
messing around with the attention. I would try different approaches to learn from the video like
reading the subtitles or looking that animation. I would like to see the attention after the video.
During is distracting” (PNFNF4), “my mind wander a lot, so it will help me get back on track.
After the video, I feel like it’s better, no particular reason” (PNFNF5), “it may likely tell me how
attentive I was. I would like to see the graph after the video, so I am not distracted. I would be
too focused on the graph if it is during” (PNFNF6), “relating it back to yourself makes it more
interesting. It provides another dimension. I can write, think and talk, but seeing it is another
dimension. I would like to see the graph after the video, it would be distracting during the video”
(PNFNF7), “if it shows I am not paying a lot of attention, I would try to pay more attention. I
would like to see the graph after watching the video, if it is while then I would be more
distracted” (PNFNF8), “become aware of what is helping and distracting, maybe I will be able to
alter the attention to obtain an optimal attention. The graph during the video would be helpful,
both implications have pros. I can adjust in real time” (PNFNF9), “it is interesting to know both
during and after” (PNFNF10), “I think it would be very interesting. Having a new quantifiable
perspective on that, even if it is definite or not, it is still good. Having an objective measurement
can be helpful. I definitely do not want to see the graph during the video, I may be tempted to
playing with the attention level. I can be distracted. Even if you show me the results before the
second video, I would be thinking about it. It would mess with my attention during the second
video for better or for worst” (PNFNF11), “if it is lower than the average or what I expected it
would motivate me to improve. If it is higher, then I don’t if I feel good. I am not good at paying
91
attention, so if the attention is above average then I will feel skeptical” (PNFNF12), “I would be
interested to see what the bar graph says. I don’t think you can influence your attention by just
seeing the data. I don’t see it changing my behavior or my strategy towards completing a task”
(PNFNF13), “show more statistics than those training simulators. I would like to see the
attention after, if it is during I would be focused on the graph than on the video itself”
(PNFNF14), “it will have a lot of influence and help me grasp a better understanding of it. I can
see which part of the video I am more attentive. I can see I was not attentive after the video”
(PNFNF15), “it is interesting to see it, really cool you can actually measure it, probably realize
where I was less focused” (PNFNF16), “I don’t know, it depends if it is really positive or
negative. If it is a good feedback it can help me improve my attention span. If the attention is
bad, then I would be confused if I thought it was good” (PNFNF17), “you will be able to
associate visuals and the time when you were not paying attention, especially you don’t notice
when you are not paying attention” (PNFNF18).
When you saw your attention visualized in the mobile device, what did you think?
(FNF, NFF, FF). Participants received the attention feedback either once or twice. It is
important to give instructions or some sort of guidance on the scale of the attention levels in the
visualization. The scale was not discussed to the participants in the beginning because
quantified-self applications do not provide a guideline on interpreting their data and how it is
recorded. We wanted to provide a similar experience based on state of the art. College students
would like to know how they perform compared to everybody else, especially those who are very
competitive. “what is the scale” (PFNF1), “I was it was half way of the spectrum, made me think
if it was low or high, for the second task I tried to get it higher” (PFNF2), “I was shocked how
low it was. Thinking who it was so low and thinking ways how to improve it for the next video”
92
(PFNF4), “it seems kind of low. I thought I had better attention than that” (PFNF5), “it is a good
indicator that you need to pay more attention of what you are doing. It indicates that you are
spacing out. It helped me out for the second one” (PFNF6), “nothing in particular, I couldn’t
compare to anything else” (PFNF7), “it was pretty low. I have been up since 8:30 AM” (PFNF8),
“I thought it was low” (PFNF9), “it reinforces how much attention I was paying to whatever
content I am watching. It is good to see it as well instead of thinking how attentive I was”
(PFNF10), “I really don’t understand how the scoring work. I can probably concentrate more
than this. I worked harder in the second one” (PFNF11), “I was sad. I wanted it to be better, I
thought it was a mistake” (PFNF12), “wondering how that compared to other people” (PFNF13),
“it could have been higher, I don’t know what could be the average or the highest score”
(PFNF14), “I thought it was low, I was more determined in the next round to try to get it up”
(PFNF15), “it is interesting to see your own electrical activity in a mobile device. It makes you
feel more alive and gave me more interest on the study” (PFNF16), “I didn’t know what it was in
comparison to what I have done before or compared to other people” (PFNF17), “I wondered if
it was high or not. I have never done anything like this before” (PNFF1), “how is that compared
to other people and the scale” (PNFF2), “there wasn’t a scale, I didn’t know if it was a good
thing or bad thing. I paid more attention in the second video” (PNFF3), “need to buy a mobile
device to track my brain activity” (PNFF4), “it didn’t mean much because I didn’t have anything
to relate it to. If it was good or bad compared to other people or first video” (PNFF5), “I really
didn’t understand what it was representing, but I don’t know what it was considered normal, high
or low” (PNFF6), “I didn’t know what was the scale and what that meant” (PNFF7), “little
surprised, not really sure how everything plays out like during an exam or walking from class to
class. I was thinking what my highest would have been if I would be taking an exam” (PNFF9),
93
“I thought it was lower than I thought it would be. I expected it to be around 60” (PNFF10), “I
thought it was inaccurate, I was paying attention at least 60%” (PNFF11), “I wanted to know
what is a good average attention. I didn’t feel if I was not paying attention at all to the video. I
don’t know what else my mind is doing to see that low score” (PNFF12), “I thought I could have
done better to be more attentive” (PNFF13), “I am not sure what it means is it good or bad?
Probably means below 50 I should try to increase it more” (PNFF14), “I don’t know why it’s so
low. Maybe it is not very correct. I would like to know why is less than 20%” (PNFF15), “to me
it was just a number, didn’t have anything reference it to, it wasn’t too important to me”
(PNFF16), “I was wondering what was the scale and how it was compared to everyone else and
the mean attentiveness where I ranked compare to others” (PNFF17), “very disappointed in
myself because of the score. This is something I would definitely buy. Interested to see how the
mind works” (PFF1), “I felt I did more attention for the second video, therefore it confirmed for
the second video” (PFF2), “how it measures that and happy that it improved” (PFF3), “is that
good or bad I thought. After second video, I thought what happened the first time” (PFF4), “I
thought it was low, then I thought I could do better if I pay more attention. I did pay more
attention during the second video” (PFF5), “didn’t think much at all as didn’t have point of
reference. It was significantly higher kind of made me wonder what I did different” (PFF6), “I
didn’t know what it meant, I saw almost doubled the second time” (PFF7), “it was interesting, I
was slightly curious what it meant. It was interesting it was lower” (PFF8), “wasn’t sure how low
or high, it was higher but wasn’t sure where it landed” (PFF9), “it is an arbitrary number”
(PFF10), “I was wondering if it was a good score. I felt a bit disappointed. I was happy it was a
lot higher the second time, I felt that reading the subtitles was helpful” (PFF11), “I was it was
low. It was a low better but needs to be improved. There is a lot of room for improvement”
94
(PFF12), “a bit shocked, it was surprising to me it was so low. The second time, I felt a little
better. I thought my attention went higher” (PFF13), “I have no idea what that means. Then I
thought, the bigger the number the more attentive, then I saw more attentive and I agree. Don’t
care what the actual value is, it’s all about improving” (PFF14), “I was wondering if it was good
or not. It look a lot better than the first time” (PFF15), “for the first video, I thought I was kind of
dumb. In the second, I thought it was low but did not do as bad as the first time” (PFF16), “really
didn’t understand it, I didn’t know how it compares to other people and what it really means.
Saw it was a little bit higher, but I still don’t know. Would like to know how I compare to other
people” (PFF17).
During the second video task, did you use any mental strategy to pay more
attention? If yes, can you please describe it? (NFNF, FNF, NFF, FF). To improve the
learning experience and increase attention, adjustments are needed in how to proceed. Learners
considered a mental strategy on the characteristics of the video they paid more attention to and
repeated the information in their head while they watched the videos. In other cases, participants
tried not to get distracted by not thinking of other things. Every time they noticed they were
thinking of something else, they started to concentrate again on the video. “I tried to connect
things between two items” (PNFNF2), “I was paying attention to animation more than anything
else in the first video, then second video changed tactics. I noticed I forgot some stuff, so tried
different ways” (PNFNF4), “decided to read the subtitles” (PNFNF5), “mental strategy for both
videos were equal, so no” (PNFNF6), “just grounding myself” (PNFNF7), “spent less time trying
to remember each word” (PNFNF8), “I thought to myself don’t try to remember and don’t think
too much, just listen” (PNFNF9), “graphics were changing too fast and it was distracting to my
process of learning. The terms they used are not applied in the graphics, so topped looking at the
95
graphics. Just paid attention to the audio” (PNFNF10), “I kind of watched video two just like
video one” (PNFNF11), “I tried to read the caption while following what they were saying”
(PNFNF12), “I tried reading the subtitles for a while” (PNFNF13), “see if I can make more
connections between the video and actual life” (PNFNF15), “when I noticed getting distracted
tried to focus on video” (PNFNF16), “I tried not to force anything, don’t force yourself to focus
very hard and just watch it” (PNFNF17), “after I noticed what type of questions were going to be
asked in the second video I paid more attention to the terminologies” (PNFNF18), “started
picturing what they described in my head” (PFNF1), “saying the same thing they were saying in
my head” (PFNF12), “tried to focus on the text than actual animation” (PFNF13).
What did you learn about the experience about how you react? (RFQ) (FNF, NFF,
FF). Some learners do not know how they react in learning activities. The best indicator they
have is exam results, but there is no indicator before the exam. Students that saw their attention
learned that by seeing their attention level, they could use it as an indicator to improve the next
time. Also, learners felt that they could adapt their learning style after obtaining experience and
seeing their results. “the fact the attention level was shown to me, it gave me the motivation to
concentrate more. I don’t know if I paid more attention the second time because I didn’t see my
attention” (PFNF1), “for the first task I wasn’t aware of the tracking, I was not focused on my
attention. During the second task, obtaining a higher attention became my goal” (PFNF2), “I
guess I am not very attentive when I watch stuff” (PFNF3), “definitely wanted to improve my
attention. I paid attention to the terms more, after the first quiz I wasn’t sure about the terms, so I
was trying to pay attention more for the quiz” (PFNF4), “seeing where I can improve, I can see
that as guidance. My reaction to the visualization impacted the way I saw the second video”
(PFNF5), “I can see a precise value of my attention help me a lot more. Something tangible to
96
look at” (PFNF6), “I think I pay attention a lot if it interests me. I can pay attention for a long
period of time” (PFNF7), “shocked, tried to focus more” (PFNF8), “I think I tried to do better. I
don’t know if I did better. I have stuff going on personally right now, hard to focus on videos”
(PFNF9), “you can control how you react based how you relax before hand. I can control what I
was doing” (PFNF10), “a lot of the focusing in the second video, not just the content. I was kind
of a challenge to improve my concentration” (PFNF11), “I don’t think I learned anything”
(PFNF12), “start quantifying my performance, you start paying attention to task. Terrible to
writing essays, get sidetrack, having a numeric value of my attention, it gives me a goal of trying
to improve that number” (PFNF13), “not really sure. When you see the results, I try to improve
and increase performance” (PFNF14), “I already know that I zone out a lot” (PFNF15), “I
always knew I had a problem with attention span. Had neglected meditating, should do it more
often. It solidifies what I knew about my own learning methods” (PFNF16), “I react to small
insignificant details. Some animations were distracting, was thinking how they were done and
animated. I tried to snap myself of distraction, tell myself is insignificant and should not be
thinking about it” (PNFF1), “I have adaptive attention, I pay more attention to what’s more
important” (PNFF5), “having a clear mind is easier to concentrate” (PNFF6), “if I know what I
should be looking for, I pay more attention” (PNFF7), “if it is something I don’t know about, I
will be more focus on it. If it is more applicable, I listen to it but thinking about how it applies to
me. I may not be 100% focused on it” (PNFF8), “the length of the video plays a part with your
attention span” (PNFF10), “I might not be paying as much attention as I thought I was”
(PNFF11), “sleep deprivation does affect my skills as a student. I might not do as well in a class.
I may have to go to sleep to perform better” (PNFF12), “thinking about focusing, may have cut
my focus” (PNFF13), “for the second video I was ready to take more info, coz of experience
97
with the first video” (PNFF14), “I learned if there is some fear of losing attention, I would be
involved in the task” (PNFF15), “just saw the visualization after the second video, so not much
there” (PNFF16), “I need to try to focus more. For one, I don’t know if I have ADHD, but I have
noticed I drift in class” (PFF1), “made me do better, and I am a visual person” (PFF2), “if I
really focus I can improve it” (PFF3), “if I am paying attention, it’s going to increase my ability
to memorize and learn” (PFF4), “if you know you are not paying more attention, and you want to
pay more attention, you can. It would be more difficult if you don’t find the videos interesting”
(PFF5), “I have to focus on paying attention, unless I just can’t” (PFF6), “I learned I don’t pay as
much as attention as much I thought I was” (PFF7), “if something is more interesting I am more
likely to pay attention” (PFF8), “I learned after seeing my attention I wanted it to go higher. I
wasn’t aware that I was paying attention” (PFF9), “if I am not as devoted to the information, I
cannot concentrate” (PFF10), “you can make yourself pay more attention. I never thought about
that I can make myself pay more attention” (PFF11), “very calm person in general, after the
attention stressed me a bit to pay more attention the second time” (PFF13), “I like to improve
myself. I didn’t do good answering the questions, so if I see something I try to improve it instead
of getting mad” (PFF15), “if there is some proof that you don’t pay attention and you want to get
better, you will actually do better” (PFF16), “I felt a bit more motivated coz I knew I had to take
a quiz” (PFF17).
What does the experience suggest to you about your strengths? (RFQ) (FNF, NFF,
FF). The purpose of receiving feedback after completing an activity is to become self-aware of
one’s performance and to improve performance in future tasks. To improve, users may need to
learn about their strengths and weaknesses. Users found that they can associate the information
presented in the videos with images, visualizations, and other aspects of the video. Some users
98
did not learn about their strengths. “I don’t know about my strength” (PFNF1), “it shows me that
I have the capability to learn very quickly. For the first video, I watched it for entertainment
purposes, I not that serious” (PFNF2), “I am not super attentive, but I am not all over the place. I
am able to remember stuff. I definitely cannot think about anything for 10 minutes. It is hard to
do” (PFNF3), “I am not sure it displayed my strength, I think it displays my weakness” (PFNF4),
“picking up on trends, after watching the first video and the questions. I needed to concentrate
more on names than general ideas” (PFNF5), “I associate information with images, so I can
retain much more” (PFNF6), “I am pretty good at synthesizing information” (PFNF7),
“analyzing the outcome, and try to do something new to make it higher” (PFNF8), “I always
knew paying attention is not my strong asset. I am diagnosed with ADHD. I try to deal with it”
(PFNF9), “Fairly competitive in a sense. Having a number attached to focus, motivates me in a
sense, because I challenge myself to focus more” (PFNF11), “given the right motivation you can
improve your focus. Having a value of your concentration, gives you immediate feedback you
can try to concentrate more” (PFNF13), “I guess I am not very attentive” (PFNF14), “I am good
when it comes to visual, when you show me visuals I am able to pay attention better” (PFNF15),
“attention span hinders me. My brain is all wired up” (PFNF16), “I have pretty good learning
abilities and test taking abilities” (PFNF17), “I am an auditory learner, very easy for me to take
details from what I hear. I didn’t notice the subtitles, I was just listening” (PNFF1), “I can work
on my attention. Something like this can help to enhance it” (PNFF2), “I am still pretty mentally
strong” (PNFF3), “my ability to adapt to if I feel overwhelm” (PNFF5), “Not sure, I can’t do a
full day of study, only can do 1 hour. Cannot do 3-4 hour straight” (PNFF6), “I believe I can
focus very well. I can improve by getting more sleep” (PNFF7), “now I know how I learn more.
I like to see it when people start explaining things to me” (PNFF8), “I have to realize I am not
99
focused, I need to correct myself a little bit” (PNFF13), “the calibration helped me calm my
mind. Once you are at a good mental state you are ready to get more information” (PNFF14), “I
am not very good at memorizing things. If you put a lot of words in short amount of time, I’m
not good memorizing. If I understand, I can answer correctly” (PNFF15), “hard to concentrate, it
is not an easy task. There is more work I can do to remember things better” (PNFF16), “I am
competitive in most of things I do. I try to compare my quizzes and test with other people. I
knew what they were looking for so I did better in the second quiz” (PNFF17), “that my
attentiveness increases over time, if I am doing similar studies” (PFF1), “push myself to do
better, not really sure” (PFF2), “if I am invested in it, I can pay more attention” (PFF5), “I need
medication to pay attention better. I am Diagnosed with ADD” (PFF6), “strong abilities to
concentrate on what I want to” (PFF7), “I don’t like the info I won’t be able to pay attention to
it” (PFF8), “adapting, once I go through something the second time, I will be able to do better.
Repetition is key for me” (PFF9), “I am passionate about what I want to learn, I can focus
heavily” (PFF10), “I am not a picture person, but more a word and finding the meaning behind it
when it comes to learning” (PFF11), “persistence, being able to focus on things now that I know
how to improve it” (PFF12), “I am an auditory learner” (PFF13), “some things I can work on. I
start thinking about other things when I am learning” (PFF14), “improving myself” (PFF15), “I
don’t think I have a long attention span, I have always known this because I don’t listen well. It
was more real because I actually saw it. Different knowing it and being there on paper” (PFF16),
“I am not sure if it is that strong. I feel I could have done better” (PFF17), “I don’t know if I have
any strengths” (PFF18).
What might you do differently as a result of the experience and your reflections on
it? What actions do your reflections lead you to? (RFQ) (FNF, NFF, FF). Students are not
100
use to receiving feedback for their study tasks. Nevertheless, when they are introduced to new
experiences, they think of different ways to use what they learned from the experience for
different activities. Users would adapt based on their experience by receiving their feedback and
the content of the video (or other educational activities). “Get back to my sleep schedule.
Certain times I prioritize sleep and do very well, but when. If I don’t, things just go downhill”
(PFNF1), “if there is ever content I need to retain, I may watch it more than once. I will try to
keep distractions away and just focus on the task” (PFNF2), “now that I am aware of my
concentration level, hopefully try to improve my concentration when trying to learn things”
(PFNF4), “definitely listen attentively. I think I can do something else while I try to listen to my
professors, I need to really focus on what I’m doing inside and outside of class” (PFNF5),
“visualize more while reading something, to associate it with something better to arrange the
information better” (PFNF6), “in a given task, try to focus on it and try to block everything out.
Try to use the wave sound process” (PFNF8), “made me realized more that I don’t pay attention
as good as I think. I guess try harder” (PFNF9), “definitely consider how stress impacts the way I
go about life. Improve how I work and do meditation” (PFNF10), “meditate prior studying may
be effective” (PFNF11), “definitely going to try to do more meditation before hand, just like the
alpha and try to get more sleep” (PFNF13), “sleep more and probably exercise to release stress
while maintaining a good study” (PFNF14), “try to relax, being stressed out to pay attention it
worsens my attention. Breath and take sleeping and meditating more seriously” (PFNF16), “try
to figure out the information I’m going to be tested on” (PFNF17), “I think I will try to pay less
attention to insignificant detail and pay more attention to actual content” (PNFF1), “try to make
myself learn things and limit distractions” (PNFF2), “try to be less stress and do better on sleep”
(PNFF3), “be more aware of negative impact of stress and sleep deprivation. Specifically, as an
101
Engineer it happens to everyone. It is important to try to control it” (PNFF5), “when I feel I am
losing concentration try to meditate for 10 minutes instead of watching TV” (PNFF6), “get better
sleep after seeing the important of it. Counter the stress that is in my life” (PNFF7), “probably
read more. I depend on the visual” (PNFF8), “keep improving sleep and managing my stress.
Probably do more meditation and go for a simple walk everyday” (PNFF9), “concentrate on the
main problem to try to understand why the content is being shown” (PNFF10), “if I can get a
device like that I might use it more to see if my attention improves” (PNFF11), “get more sleep
and eat breakfast in the morning, now I am aware of negative effects of sleep deprivation”
(PNFF12), “definitely try more focusing activities. Alpha helps to clear my mind” (PNFF13), “I
would buy this headset and measure my attention progressively” (PNFF14), “try to learn how to
concentrate better. and pay more attention to the things I am doing” (PNFF16), “its possible I
would watch the video full instead of pausing” (PNFF17), “if possible try to get more sleep, do
more to expand and benefit my focus and attentiveness” (PFF1), “I learned if I want to
concentrate on something, from just absorbing the information, I should repeat what they are
saying as they are saying it. I am sure I tried this before, because I read it in the internet” (PFF2),
“really trying to focus on whatever I am learning or when I am in class” (PFF3), “be more aware
of my environment and my attention of what I am doing. I put the same amount of time in the
first video and second, in the second video I retained more info in the second” (PFF4), “for
lectures, I may try to pay more attention to details” (PFF5), “not really sure, perhaps taking a
break to relax before I start learning may be helpful” (PFF6), “I pay more attention to my study
strategies and see what is working and what is not” (PFF7), “probably nothing, I tried to pay
attention to the videos much as possible” (PFF8), “try to relax before watching the video”
(PFF9), “definitely caught myself from drifting away from focusing. Tried to clear my mind
102
more often, because my mind was going everywhere, cluttered mind during alpha” (PFF10), “try
to get rid of my distractions, play something outload and listen to it. Try writing out the words to
see which one is more beneficial to pay more attention” (PFF11), “get a decent amount of sleep
and apply myself into paying more attention like in lectures or classes” (PFF12), “put away other
distractions and put time in the beginning to center my thoughts” (PFF13), “force myself to be
more attentive. Even before this session tried to do that” (PFF14), “if I am not understanding
something, trying to understand why I’m not paying attention” (PFF15), “I want to say I am
going to pay more attention, there is always that motivation and you doze off and try again but it
is too late” (PFF16), “I wasn’t focusing on the facts because I didn’t know about the quiz. I
would be more attentive to remember those details” (PFF17), “probably sleep well and handle
my stress and do exercise and meditation” (PFF18).
What do you think of this new learning process? (FNF, NFF, FF). The new learning
process consists of seeing the attention feedback right after they perform a learning task to obtain
awareness of their attention to try to self-improve. Participants found this tool useful and
valuable towards their educational experience. “It may be more powerful feedback than grades.
Sometimes teacher uses a grade to encourage students do better, but sometimes students perceive
those as the teacher gave them the grade With BCI attention, machine has nothing against you,
so you can’t blame it. It gives you a harsh reality that you don’t pay that much attention”
(PFNF1), “it could help me improve, it puts you in a different mindset thanks to the relaxation. If
the experience becomes more natural, then it can be helpful” (PFNF2), “it would be useful to
know what the data means. If I am able to sync up the data along with video, I can see where I
didn’t pay attention” (PFNF3), “it’s helpful to see where you are in the spectrum and it is
frustrating because it is up to you to improve it. It didn’t tell me how to improve it” (PFNF4),
103
“it’s kind of helpful and informative. It shows how much you are getting out of what you are
doing” (PFNF5), “it’s a pretty good way to help me see where I went wrong. It can be used as a
motivation to improve. Without it I wouldn’t have a benchmark” (PFNF6), “if you know how
your attention in the scale, you can see if it is effective or not. I didn’t think about it because I
don’t know what the number means” (PFNF7), “I think it is pretty effective, you can analyze
what you are doing to see if it is right or wrong” (PFNF8), “It can be helpful. I don’t know how
practical it is to put the headset every time. If you can put it on your head and forget you are
wearing it” (PFNF9), “it would be helpful what my average attention level is for a specific task. I
would try to repeat the task to try to meet my average” (PFNF10), “if it was not cost prohibiting,
it can be very helpful” (PFNF11), “it is a good metric to see how well I am performing. Half
along the way of the task you can notice that you are thinking on other stuff” (PFNF13), “I think
is very effective that you can see how attentive you were while watching the video, then you
answer the quiz to see how much you learned” (PFNF14), “I think it might be useful if you are
trying to see your study habit and how to improve it. I think it is up to you at the end. I wouldn’t
need one, I would try my best to focus and I know how to focus” (PFNF15), “it can be really
helpful for students wondering why they are not doing well. It would not be based on their ability
to remember, but on how well they focused” (PFNF17), “it reminds me a lot of gamification
such as fitness. I didn’t know this was a thing. This can be a useful tool. Incrementally improving
and the way we learn and pay attention and trying to improve it” (PNFF1), “I see the benefit of
it. It is a concrete number of your attention and makes you think about how you can make
yourself being more attentive” (PNFF2), “the idea on being tested on your attention, it kinds of
makes you want to be more attentive on what you are learning” (PNFF3), “it could be useful, if
you have a lot of feedback and you know what the number means. You need to wear it a lot to
104
get a lot of useful data. There is a lot of adaptive problems in terms of interference and comfort”
(PNFF5), “it is a good way to do that and you can get immediate feedback” (PNFF6), “the way
the future is going people like to see how they progress” (PNFF8), “I think is neat, a new way to
try to gauge how much attention someone is giving to specific activities. Also, I think is a good
exercise for small activities” (PNFF9), “it can be really informative” (PNFF10), “by seeing my
score, I could pay attention or learn more stuff. I feel is kind of different from what you think”
(PNFF11), “I think it would help in the long run. People would know what they need to do to be
more attentive. They can go through different tasks to see how attention improves” (PNFF12),
“try to move your attention up over time would be of benefit” (PNFF13), “it could help a lot
because, as soon when you answer some questions, you can see your attention. It can boost
confidence if it goes up” (PNFF14), “a very informative thing of attention. If it goes down, I can
pay more attention to the task and be more engaged in my daily tasks. It can be beneficial during
studies” (PNFF15), “its pretty cool. It could be a good way to measure your attention span. If it
says what the number represents and correlate to the grades getting in school” (PNFF16), “I
think is really interesting. It can be used for small children to figure out what classes they are
going to excel at. To some extent for people with ADHD, especially if they have not been
diagnosed. Caffeine versus no caffeine with people” (PNFF17), “I think it would be very useful
if they see how they process and retain information. How you do well in some subjects versus
others” (PFF1), “I think is really cool, it is helpful to see the visualization of your attention
levels” (PFF2), “it helps you understand, not necessarily what it is wrong, but how you can fix or
improve your grade or ability to give information back from what you just learned” (PFF3), “I
think is cool and can be very useful. At the same time is a little bit weird incorporating the BCI
into the study habit. Maybe down the road a way making it less conspicuous, then it can be more
105
adapted to students. Right now, it is more a novelty” (PFF4), “in a class setting they may end up
grading you on that. At home they may help people” (PFF5), “has a potential to be useful as long
as it is managed. Just because you are paying attention, doesn’t mean you are retaining
information. They may not be strong correlation between quiz scores and attention” (PFF6), “I
think is very beneficial, and people can see what their strong points are and come up with study
strategies” (PFF7), “if it is low, you can do trial and error to see how you can pay attention more
effectively” (PFF8), “the attention part brings another level, it makes you aware of where you are
at some point. I can possibly unlock my attention awareness” (PFF9), “it depends how accurate it
is. Because I was being monitored, I tried to pay more attention” (PFF10), “I think it can be
helpful, I don’t know how quickly it can be helpful. I have been learning for 10 years specific
way” (PFF11), “it is interesting, really valuable to me because I learned about the learning
process. You can learn about different ways and perspectives” (PFF12), “it is helpful until an
extent. It helps you evaluate your levels and compare to actual numbers with perception. If I use
it in a daily basis you may get sick of it. Short term use can be better than long term” (PFF13), “I
like it, I think its beneficial, people use watches to see their health. You want to get a good grade
in the exam and people don’t know their best study habit” (PFF15), “it is really interesting, you
may think you didn’t pay attention and its different when you see the results. I wonder if a
repetition may improve. I am not quite sure if you do this in a daily basis if your attention
improves” (PFF16), “it is pretty cool, never experienced it before. It would be helpful, it would
show you, you need to aim higher” (PFF17), “I tried to pay attention to the video, but I still
couldn’t focus on it. The distraction happens when you can’t notice it. It will help me improve. I
really want to buy the device” (PFF18).
106
How do you see yourself using this tool? (FNF, NFF, FF). Learners have different
ways and locations to perform their learning activities. Each would have their perception on how
to use the quantified-self attention feedback application along with the BCI device. Students
would like to use this tool in their daily life. Some ADHD students would like to use it to
monitor their attention. “While watching videos, reading books and papers and see what is my
attention level. Still want to see if the grades from the quiz correlate with the engagement. Use it
as an encouragement” (PFNF1), “if the calibration is an everyday process and no movement is
removed, then I would use it” (PFNF2), “it would be hard to see it apply outside school, but it
would help me to study to get better results” (PFNF3), “it would be helpful to record my
attention during an online lecture” (PFNF4), “if I have the ability to use it while being in class, I
can see when I am paying attention” (PFNF5), “calibrate my attention span with a short video
with short content and see my attention and aim for a higher number” (PFNF6), “I haven’t
thought about using it for daily life” (PFNF7), “measure attention for each subject” (PFNF8), “I
would use it if it doesn’t look weird and it would be affordable. It is more worthwhile than a
Fitbit that measures exercise, especially because it could help with my ADHD” (PFNF9), “I
would need to use it when an upcoming deadline is coming and when I need to learn a particular
information” (PFNF10), “maybe reading a textbook and see what I’m focused on and for regular
studying, but not for studying for an exam” (PFNF11), “I am more interested in brain technology
not for data gathering, but something to store data for me” (PFNF12), “another tool to help me
focus on whatever tasks on projects or homework. To make me self-aware of my current
attention at the moment” (PFNF13), “definitely to study for a class I thought it was difficult”
(PFNF14), “I would probably use this tool a lot. I would try the thing on when I start studying,
study for 12 hours, when I am in a good mood, and sleep deprived. I would use it during
107
different seasons and while I am menstruating because of my hormones” (PFNF16), “measuring
how I pay attention in the classroom or when watching a class video. I would use it at home as
well when reading and going over information. Making yourself responsible when reading
things” (PFNF17), “if I am watching a video lecture, I can measure my performance and
attention and try to incrementally bring my scores up” (PNFF1), “if I have tool on my desk can
put it on and do work and check my attention levels after 10 mins of work. If I am watching a
lecture video, I can see my attention, if not high, then re-watch” (PNFF2), “I would use it during
studies and see where I am paying more attention or not” (PNFF3), “if it fits consumer needs and
it is comfortable enough, I might wear it as a Fitbit. It can be useful for pretty much any task”
(PNFF5), “if I have to read sections of data I would check if my attention is high, then I would
feel comfortable that I learned the information” (PNFF6), “while reading non-fiction, taking an
exam, or just to meditate” (PNFF9), “I don’t believe it was accurate. If feedback would be better
I would like to use this tool” (PNFF10), “I can see myself using it. If the device is more discrete
for me to use it. After some time, there is some pressure” (PNFF11), “for tasks like this, it could
be helpful. It can be applicable to a lot of things we use now days” (PNFF12), “try to use it while
doing any readings or lecture videos. It would help me use my time wisely” (PNFF13), “I would
use it during meditation. Different things start coming up in my mind, this tool makes me aware
that my meditation is doing good” (PNFF14), “I would use it when I am studying or during my
free time” (PNFF15), “I don’t think I have ways of affording it. I also don’t think I would carry it
on my head where I go, if I am home alone I may use it” (PNFF16), “I would not buy it for
myself, because it was uncomfortable and had to adjust my hair. If it is modular, I would”
(PNFF17), “try to use it in a daily basis or every other day. If I am able to increase my
attentiveness can help my academic and professional career” (PFF1), “I have been wanting to
108
meditate I would use it for that and when trying to study to have a accountability if I am paying
attention or not” (PFF2), “if it is affordable I would probably use it. If you see you are losing
focus I can try to improve it” (PFF3), “maybe using it to study before final. I don’t see myself
using it day to day” (PFF4), “I like playing video games and reading, I may use it for that. It is
interesting to see data between dates” (PFF5), “it would be interesting, to see what level of
attention I pay, also when I haven’t had any caffeine vs no caffeine” (PFF6), “I can use it during
study, to see what its helping me and use it in lecture halls to see when I’m not paying attention”
(PFF7), “if there are certain classes that complete bored me, I may use it to pay attention better”
(PFF8), “I think I would use it, if I have the money. It seems simple enough like Bluetooth
headphones” (PFF9), “measure interest in the topic in correlation with attention, If I get a low
score in something I should be studying, maybe do it again” (PFF10), “for a test, learn something
for a job like working for Starbucks and learning all the drinks. Learning tangible information”
(PFF11), “trying to improve myself more and more in paying attention” (PFF12), “the days I am
learning a new material. It helps me to see how much time I need to study certain things and
come up with strategies” (PFF13), “I treat it as a game just like anything I do” (PFF14), “if I am
not paying attention, I will try to see what works” (PFF15), “maybe when I watch some lecture
online or in person or a video for class. For reading as well, but I may do this just for myself or
fun, not necessarily for school” (PFF16), “if I am feeling inattentive I can use it to motivate
myself when I need to study or do work” (PFF17), “When I have to prepare for exams, if I have
to study for 4 hours every day I would check my focus to see if I am distracted” (PFF18).
Limitations
The limitations encountered in this study are like other BCI studies using a wireless non-
invasive wearable EEG device. Some participants felt mild pain from the device as it was tight
109
for some people this is dependent on their head size and shape. There was one potential
participant who could not do the experiment due to the amount of hair on her scalp. This is a
typical issue encountered in female participants participating in neurophysiological studies.
The application is in the prototype phase. Therefore it crashed during the study at random
times. However, despite the crashes, it did not happen during the data acquisition, but after the
data had been gathered. These crashes caused the studies to be prolonged for several minutes.
There is no standard in performing statistical analysis in BCI. The statistical tests
conducted in BCI studies are from what is known from statistics until the community finds an
effective way to check the validity of the acquired brain data. The hypothesis in the second study
was tested 70 times. This means that there is a 97% chance of obtaining one significant result,
even if all the other tests are not significant. Correction tests are needed to address multiple
testing by adjusting alpha so that the probability of obtaining at least one significant result due to
chance is below the desired significance level. A Bonferroni correction test should be conducted
to cut-off at alpha/n. Therefore, in the second study of this dissertation, the null hypothesis
should have been rejected if the p-value is less than 0.00071. In this dissertation, a correction test
was not performed. Therefore p-values below 0.05 are not reliable. In future studies, Bonferroni
should be conducted to address these false positives.
Table 5-3. Positive and Negative Affect Schedule (PANAS) - descriptive statistics for each
group. PA = Positive Affect, NA = Negative Affect
Groups N Pre-PANAS
PA Scale
M SD
Pre-PANAS
NA Scale
M SD
Post-PANAS
PA Scale
M SD
Post-PANAS
NA Scale
M SD
NFNF 18 33.65 6.46 13.47 3.61 34.06 8.84 11.88 2.91
FNF 17 27.06 7.62 14.31 4.83 29.75 9.35 13.50 4.77
NFF 17 31.47 7.10 14.33 4.19 33.00 8.28 13.40 5.14
FF 18 27.25 9.90 13.00 2.19 27.63 11.00 11.94 1.88
110
Table 5-4. Positive and Negative Affect Schedule (PANAS) - descriptive statistics for each affect
per group. PA = positive affect, NA = negative affect
Affects/Groups NFNF
Pre Post
FNF
Pre Post
NFF
Pre Post
FF
Pre Post
Interested PA M
SD
3.94
0.24
3.78
1.00
3.35
0.86
3.88
0.70
3.47
0.80
3.82
0.73
3.33
0.84
3.50
1.20
Distressed NA M
SD
1.17
0.51
1.28
0.83
1.71
0.85
1.82
1.19
1.53
0.72
1.47
0.87
1.89
1.32
1.50
0.71
Excited PA M
SD
3.33
0.59
3.28
1.07
2.88
0.99
3.00
1.12
2.88
0.86
3.24
1.09
2.44
1.04
2.78
1.44
Upset NA M
SD
1.22
0.43
1.11
0.32
1.35
0.86
1.24
0.75
1.24
0.56
1.18
0.53
1.17
0.51
1.44
0.70
Strong PA M
SD
3.06
0.87
3.11
1.18
2.53
0.87
2.71
1.05
2.88
0.99
2.82
1.24
2.39
1.14
2.17
1.29
Guilty NA M
SD
1.22
0.73
1.06
0.24
1.18
0.39
1.12
0.33
1.12
0.33
1.24
0.56
1.11
0.32
1.06
0.24
Scared NA M
SD
1.33
0.49
1.17
0.38
1.24
0.66
1.18
0.53
1.12
0.33
1.24
0.44
1.11
0.32
1.11
0.47
Hostile NA M
SD
1.11
0.32
1.00
0.00
1.24
0.66
1.24
0.66
1.12
0.33
1.24
0.75
1.11
0.32
1.06
0.24
Enthusiastic PA M
SD
3.67
1.08
3.50
1.04
3.06
1.09
3.06
1.20
3.00
1.00
3.24
0.97
2.67
1.19
2.83
1.42
Proud PA M
SD
2.83
0.86
3.06
1.30
2.35
1.06
2.53
1.33
2.76
1.03
2.82
1.19
2.17
1.25
2.39
1.09
Irritable NA M
SD
1.28
0.46
1.28
0.57
1.71
1.10
1.47
0.87
1.59
0.87
1.41
0.80
1.44
0.62
1.22
0.43
Alert PA M
SD
3.22
1.26
3.28
1.36
2.59
1.18
2.82
1.55
3.12
0.93
3.18
1.01
2.78
1.17
2.78
1.35
Ashamed NA M
SD
1.06
0.24
1.11
0.47
1.12
0.33
1.12
0.33
1.06
0.24
1.35
0.70
1.00
0.00
1.17
0.38
Inspired PA M
SD
3.17
1.25
3.22
1.44
2.24
1.09
3.06
1.34
2.94
1.14
3.06
1.20
2.11
1.08
2.56
1.10
Nervous NA M
SD
2.00
1.19
1.22
0.55
1.71
0.77
1.24
0.44
2.99
0.92
1.59
0.94
1.67
0.91
1.44
0.62
Determined PA M
SD
3.67
0.84
3.61
0.98
2.82
1.33
3.00
1.41
3.41
1.00
3.29
1.05
3.00
1.19
2.61
1.33
Attentive PA M
SD
3.83
0.71
3.67
1.24
2.94
1.25
3.29
1.31
3.53
0.87
3.53
1.01
3.11
1.37
3.17
1.25
Jittery NA M
SD
1.89
1.08
1.44
0.70
1.71
1.21
1.71
1.21
1.94
0.83
1.59
0.94
1.83
1.10
1.39
0.85
Active PA M
SD
3.22
1.00
3.33
1.24
2.65
1.17
2.88
1.17
3.12
0.86
3.47
0.87
2.61
1.04
2.61
1.24
Afraid NA M
SD
1.17
0.38
1.11
0.32
1.18
0.39
1.18
0.39
1.29
0.69
1.18
0.53
1.11
0.47
1.06
0.24
111
Table 5-5. Positive and Negative Affect Schedule (PANAS) – Cronbach’s Alpha overall result.
PA = Positive Affect, NA = Negative Affect
Affect/Groups NFNF
Pre Post
FNF
Pre Post
NFF
Pre Post
FF
Pre Post
Cronbach’s Alpha .814 .814 .831 .825 .863 .813 .879 .879
Table 5-6. Positive and Negative Affect Schedule (PANAS) - Cronbach’s Alpha for each affect
per group. PA = Positive Affect, NA = Negative Affect
Affects/Groups NFNF
Pre Post
FNF
Pre Post
NFF
Pre Post
FF
Pre Post
Interested PA .815 .790 .849 .834 .856 .814 .869 .859
Distressed NA .818 .830 .826 .823 .865 .829 .899 .885
Excited PA .810 .797 .838 .814 .850 .780 .859 .864
Upset NA .826 .816 .826 .828 .858 .808 .885 .887
Strong PA .797 .798 .810 .799 .848 .788 .857 .866
Guilty NA .804 .814 .828 .829 .864 .803 .885 .881
Scared NA .806 .816 .815 .831 .861 .806 .881 .882
Hostile NA .814 N/A .815 .823 .864 .805 .880 .881
Enthusiastic PA .778 .784 .819 .808 .847 .781 .857 .861
Proud PA .788 .794 .808 .808 .855 .822 .861 .862
Irritable NA .823 .822 .849 .829 .862 .816 .885 .877
Alert PA .795 .803 .809 .807 .863 .806 .858 .869
Ashamed NA .810 .820 .823 .829 .864 .811 N/A .882
Inspired PA .778 .804 .821 .795 .840 .796 .860 .865
Nervous NA .815 .821 .833 .831 .879 .827 .873 .878
Determined PA .792 .776 .822 .799 .836 .772 .866 .861
Attentive PA .812 .785 .816 .795 .851 .787 .860 .863
Jittery NA .816 .819 .819 .813 .860 .840 .880 .885
Active PA .799 .786 .818 .797 .849 .789 .868 .868
Afraid NA .812 .812 .824 .829 .856 .800 .883 .881
Table 5-7. Descriptive statistics of the quiz scores
Groups N Video 1 Quiz
M SD
Video 2 Quiz
M SD
NFNF 18 3.94 0.87 4.56 0.78
FNF 17 3.47 1.12 4.59 0.51
NFF 17 3.41 0.80 4.24 1.03
FF 18 3.67 0.97 4.28 0.83
112
Table 5-8. Quiz understanding and clearness
NFNF FNF NFF FF Videos
The questions in
the quiz were
clear and
understandable
M 4.22 4.53 4.29 4.33 V1
SD 0.79 0.61 0.67 0.94
M 4.83 4.88 4.82 4.67 V2
SD 0.37 0.32 0.38 0.58
The questions
were easy to
answer
M 3.44 3.76 3.59 3.94 V1
SD 0.83 0.94 0.69 0.91
M 4.28 4.65 4.47 4.39 V2
SD 0.73 0.59 0.70 1.11
The questions
were relative
with the video
M 4.72 4.76 4.59 4.78 V1
SD 0.56 0.42 0.49 0.53
M 4.89 4.88 4.82 4.67 V2
SD 0.31 0.32 0.51 0.75
The questions
were useful to
test my
understanding
M 4.28 4.29 4.29 4.44 V1
SD 0.87 0.75 0.57 0.68
M 4.78 4.88 4.76 4.67 V2
SD 0.53 0.32 0.55 0.58
Table 5-9. Descriptive statistics of the quiz completion time. The time is represented in seconds
Groups N Video 1
First Click
M SD
Video 1
Last Click
M SD
Video 1
Process Time
M SD
Video 2
First Click
M SD
Video 2
Last Click
M SD
Video 2
Process Time
M SD
NFNF 18 11.35 6.46 90.21 41.16 99.07 41.82 9.90 7.17 40.52 14.71 42.75 20.40
FNF 17 13.98 7.65 84.21 21.77 89.60 24.84 6.50 2.02 32.14 9.60 36.06 11.23
NFF 17 9.48 2.44 83.59 15.01 97.13 33.57 7.85 3.86 39.97 16.11 43.90 16.83
FF 18 13.07 7.35 74.83 33.29 81.63 33.20 11.04 14.03 38.23 17.92 41.97 19.81
Figure 5-5. Study steps Scatter plot of quiz 2 and total time taken to complete the quiz
113
Table 5-10. Descriptive statistics of Engagement indices for all the groups
Engagement Indices Channel Mean1 STD1 Mean2 STD2
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 .089 .057 .078 .047
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 .084 .062 .072 .050
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 .051 .042 .042 .034
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 .051 .045 .041 .037
Low beta engagement index - 𝛽/𝛼 AF3 .417 .133 .404 .122
Low beta engagement index - 𝛽/𝛼 AF4 .467 .140 .453 .131
High beta engagement index - 𝛽/𝛼 AF3 .547 .150 .512 .147
High beta engagement index - 𝛽/𝛼 AF4 .278 .134 .264 .141
Alpha Asymmetry AF3/AF4 1.157 .599 1.068 .559
Low Beta AF3 4.995 4.492 5.520 6.362
Low Beta AF4 4.722 4.742 4.949 5.626
High Beta AF3 2.709 2.538 2.717 3.032
High Beta AF4 2.723 2.953 2.712 3.121
Table 5-11. Descriptive statistics of engagement indices for control group (NFNF)
Engagement Indices Channel Mean1 STD1 Mean2 STD2
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.093 0.075 0.084 0.072
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.089 0.067 0.069 0.063
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.050 0.026 0.041 0.023
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.049 0.026 0.032 0.021
Low beta engagement index - 𝛽/𝛼 AF3 0.405 0.347 0.391 0.344
Low beta engagement index - 𝛽/𝛼 AF4 0.449 0.395 0.417 0.360
High beta engagement index - 𝛽/𝛼 AF3 0.536 0.341 0.466 0.314
High beta engagement index - 𝛽/𝛼 AF4 0.259 0.164 0.201 0.127
Alpha Asymmetry AF3/AF4 1.170 1.150 1.027 0.969
Low Beta AF3 4.82 15.95 6.94 33.90
Low Beta AF4 4.46 14.75 5.48 29.88
High Beta AF3 2.37 4.08 2.73 11.30
High Beta AF4 2.27 3.66 2.35 10.35
Table 5-12. Descriptive statistics of engagement indices for feedback-no feedback group (FNF)
Engagement Indices Channel Mean1 STD1 Mean2 STD2
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.094 0.078 0.074 0.048
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.082 0.055 0.071 0.053
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.057 0.044 0.038 0.019
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.053 0.029 0.041 0.025
Low beta engagement index - 𝛽/𝛼 AF3 0.426 0.385 0.396 0.348
Low beta engagement index - 𝛽/𝛼 AF4 0.484 0.415 0.440 0.400
High beta engagement index - 𝛽/𝛼 AF3 0.563 0.462 0.493 0.348
High beta engagement index - 𝛽/𝛼 AF4 0.309 0.221 0.256 0.176
Alpha Asymmetry AF3/AF4 1.098 0.909 1.132 1.161
Low Beta AF3 5.56 16.96 5.45 14.78
Low Beta AF4 5.03 12.03 5.18 15.35
High Beta AF3 3.25 9.88 2.84 5.56
114
Engagement Indices Channel Mean1 STD1 Mean2 STD2
High Beta AF4 3.22 6.86 3.14 6.75
Table 5-13. Descriptive statistics of engagement indices for no feedback-feedback group (NFF)
Engagement Indices Channel Mean1 STD1 Mean2 STD2
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.086 0.050 0.081 0.072
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.090 0.065 0.082 0.078
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.049 0.017 0.048 0.038
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.054 0.022 0.050 0.038
Low beta engagement index - 𝛽/𝛼 AF3 0.435 0.316 0.435 0.423
Low beta engagement index - 𝛽/𝛼 AF4 0.470 0.358 0.484 0.410
High beta engagement index - 𝛽/𝛼 AF3 0.503 0.294 0.525 0.411
High beta engagement index - 𝛽/𝛼 AF4 0.265 0.138 0.280 0.204
Alpha Asymmetry AF3/AF4 1.364 1.001 1.178 1.132
Low Beta AF3 5.06 12.35 5.49 11.34
Low Beta AF4 5.12 12.81 5.78 12.56
High Beta AF3 2.45 2.92 3.02 6.35
High Beta AF4 2.67 3.30 3.24 6.58
Table 5-14. Descriptive statistics of engagement indices for feedback-feedback group (FF)
Engagement Indices Channel Mean1 STD1 Mean2 STD2
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.082 0.058 0.072 0.043
Low beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.077 0.049 0.062 0.030
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF3 0.048 0.019 0.042 0.014
High beta engagement index - (𝛽/(𝛼 + 𝜃)) AF4 0.046 0.017 0.041 0.011
Low beta engagement index - 𝛽/𝛼 AF3 0.402 0.313 0.396 0.295
Low beta engagement index - 𝛽/𝛼 AF4 0.467 0.347 0.472 0.355
High beta engagement index - 𝛽/𝛼 AF3 0.584 0.342 0.566 0.329
High beta engagement index - 𝛽/𝛼 AF4 0.282 0.125 0.318 0.153
Alpha Asymmetry AF3/AF4 1.005 1.028 0.885 0.716
Low Beta AF3 4.58 12.00 4.04 7.47
Low Beta AF4 4.32 10.21 3.31 4.08
High Beta AF3 2.78 4.19 2.30 2.31
High Beta AF4 2.76 3.63 2.15 1.43
Table 5-15. Multivariate analysis of variance results of attention levels effect on quiz scores
Engagement Indices Channel One-Way ANCOVA P-Value
Low beta engagement index
(𝛽/(𝛼 + 𝜃))
AF3 .657
Low beta engagement index
(𝛽/(𝛼 + 𝜃))
AF4 .785
High beta engagement index (𝛽/(𝛼 + 𝜃)) AF3 .632
High beta engagement index (𝛽/(𝛼 + 𝜃)) AF4 .531
Low beta engagement index
𝛽/𝛼
AF3 .754
115
Engagement Indices Channel One-Way ANCOVA P-Value
Low beta engagement index
𝛽/𝛼
AF4 .348
High beta engagement index
𝛽/𝛼
AF3 .082
High beta engagement index
𝛽/𝛼
AF4 .033*
Alpha Asymmetry AF3/AF4 .647
Low Beta AF3 .440
Low Beta AF4 .493
High Beta AF3 .656
High Beta AF4 .604
Figure 5-6. Box plot for engagement index 𝐻𝑖𝑔ℎ 𝛽/𝛼 for AF4
Table 5-16. Multivariate analysis of variance results of attention levels effect on quiz scores
Quiz and Engagement Channel One-Way MANOVA
Quiz 1 and low beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF3 F(6,130) = .637, p = .700
Quiz 1 and low beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF4 F(6,130) = .639, p = .699
Quiz 1 and high beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF3 F(6,130) = .647, p = .693
Quiz 1 and high beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF4 F(6,130) = .606, p = .725
116
Quiz and Engagement Channel One-Way MANOVA
Quiz 1 and low beta engagement
index 1 - 𝛽/𝛼
AF3 F(6,130) = .652, p = .688
Quiz 1 and low beta engagement
index 1 - 𝛽/𝛼
AF4 F(6,130) = .652, p = .688
Quiz 1 and high beta engagement
index 1 - 𝛽/𝛼
AF3 F(6,130) = 1.013, p = .420
Quiz 1 and high beta engagement
index 1 - 𝛽/𝛼
AF4 F(6,130) = .830, p = .549
Quiz 1 and Alpha Asymmetry 1 AF3/AF4 F(6,130) = 1.119, p = .355
Quiz 1 and Low Beta 1 AF3 F(6,130) = .664, p = .679
Quiz 1 and Low Beta 1 AF4 F(6,130) = .649, p = .690
Quiz 1 and High Beta 1 AF3 F(6,130) = .807, p = .566
Quiz 1 and High Beta 1 AF4 F(6,130) = .761, p = .602
Quiz 2 and low beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF3 F(6,130) = .653, p = .688
Quiz 2 and low beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF4 F(6,130) = .682, p = .664
Quiz 2 and high beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF3 F(6,130) = .541, p = .777
Quiz 2 and high beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF4 F(6,130) = .772, p = .593
Quiz 2 and low beta engagement
index 2 - 𝛽/𝛼
AF3 F(6,130) = .673, p = .672
Quiz 2 and low beta engagement
index 2 - 𝛽/𝛼
AF4 F(6,128) = 1.037, p = .404
Quiz 2 and high beta engagement
index 2 - 𝛽/𝛼
AF3 F(6,130) = 1.213, p = .303
Quiz 2 and high beta engagement
index 2 - 𝛽/𝛼
AF4 F(6,128) = 1.679, p = .131
Quiz 2 and Alpha Asymmetry 2 AF3/AF4 F(6,128) = .922, p = .481
Quiz 2 and Low Beta 2 AF3 F(6,130) = 1.021, p = .414
Quiz 2 and Low Beta 2 AF4 F(6,128) = .928, p = .477
Quiz 2 and High Beta 2 AF3 F(6,130) = .551, p = .768
Quiz 2 and High Beta 2 AF4 F(6,128) = .749, p = .612
Table 5-17. Correlation between quiz scores and attention levels. P-value with the symbol *
represents statistical significance
Correlation Channel r N P-Value
Quiz 1 and low beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF3 .008 70 .948
Quiz 1 and low beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF4 -.016 70 .894
Quiz 1 and high beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF3 -.016 70 .897
117
Correlation Channel r N P-Value
Quiz 1 and high beta engagement
index 1 - (𝛽/(𝛼 + 𝜃))
AF4 .017 70 .892
Quiz 1 and low beta engagement
index 1 - 𝛽/𝛼
AF3 -.057 70 .641
Quiz 1 and low beta engagement
index 1 - 𝛽/𝛼
AF4 .014 70 .910
Quiz 1 and high beta engagement
index 1 - 𝛽/𝛼
AF3 .076 70 .531
Quiz 1 and high beta engagement
index 1 - 𝛽/𝛼
AF4 .096 70 .432
Quiz 1 and Alpha Asymmetry 1 AF3/AF4 .010 70 .932
Quiz 1 and Low Beta 1 AF3 .124 70 .306
Quiz 1 and Low Beta 1 AF4 .098 70 .419
Quiz 1 and High Beta 1 AF3 .111 70 .362
Quiz 1 and High Beta 1 AF4 .110 70 .363
Quiz 2 and low beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF3 -.053 70 .661
Quiz 2 and low beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF4 .004 70 .975
Quiz 2 and high beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF3 -.084 70 .489
Quiz 2 and high beta engagement
index 2 - (𝛽/(𝛼 + 𝜃))
AF4 -.028 70 .821
Quiz 2 and low beta engagement
index 2 - 𝛽/𝛼
AF3 -.009 70 .944
Quiz 2 and low beta engagement
index 2 - 𝛽/𝛼
AF4 .084 70 .491
Quiz 2 and high beta engagement
index 2 - 𝛽/𝛼
AF3 -.021 70 .863
Quiz 2 and high beta engagement
index 2 - 𝛽/𝛼
AF4 .056 69 .646
Quiz 2 and Alpha Asymmetry 2 AF3/AF4 .077 69 .531
Quiz 2 and Low Beta 2 AF3 -.288 70 .016*
Quiz 2 and Low Beta 2 AF4 -.194 69 .109
Quiz 2 and High Beta 2 AF3 -.181 70 .134
Quiz 2 and High Beta 2 AF4 -.147 69 .228
Summary
We discussed a user study to primarily evaluate the effect of quantified-self attention
feedback from a BCI on college students. There was an increment of attention on the students on
the right side of the frontal lobe. In the right side of the frontal lobe, there was an increment of
118
student’s attention. This information shows that utilizing the engagement index, high beta
divided by alpha can be useful to obtain attention levels from the AF4 channel. Specifically,
there was a significant difference on attention increment between participants that did not receive
any attention feedback and those who received it the first time, but not the second time. Also,
there was a significant difference when attention increased between the control group (no
attention) and the group who saw their attention the second time, but not the first time. The
increment of attention did not influence quiz scores. There was also a negative correlation in quiz
scores and taken time. When students take more time to complete the quiz, then the score will be
lower.
The students who participated in the study have never used a BCI to receive feedback on
their attention during a learning task. By the end of the study, most participants saw the benefit
of utilizing a BCI to obtain attention feedback. Regardless of the attention score, they will strive
to perform better every time. Learners diagnosed with Attention Deficit/Hyperactivity Disorder
(ADHD) think this tool is a good way to measure their attention and improve it over time. The
ADHD students mentioned they are diagnosed during the interview, although this was not asked
in the questions. The result of this study contributes towards the understanding of using of
specific engagement indices on channels AF3 and AF4 using a low-cost BCI for learning
activities.
119
CHAPTER 6
CONCLUSION
In this dissertation, we presented a user-centered approach to understand user’s
perception and preference of visualizing attention levels. In addition to visualization
recommendations for researchers, this provides a different approach to understanding
visualization types to be used for showcasing specific datasets. Users prefer and perceive bar
graphs with a 0-100 scale to be more user-friendly to visualize attention levels. Users also have
their way of visualizing data based on what they have experienced and seen.
We evaluated the effectiveness of quantified-self attention feedback in improving
attention during short-term self-regulation. There was no significant effect in the attention levels
increment for all identified engagement indices in quiz scores. Nevertheless, there was a
significant increase of attention in the AF4 channel position for beta/alpha index. Specifically,
between the control group (NFNF) and the experimental group (FNF) and between NFNF and
NFF. There was also a significant difference in quiz scores between all groups. Also, the
correlation between attention scores and quiz scores was tested. There was a negative correlation
between the quiz scores of second video and attention from EEG wave, low beta at the AF3
location in the right side of the frontal lobe.
Users also perceive the feedback to be useful and see themselves utilizing a BCI for their
learning activities. Students diagnosed with ADHD think this tool could be useful for tracking
their attention. Also, those who are not diagnosed but know they do not have a good attention
span see this tool beneficial to improve their attention.
This work aligns with the enhance roadmap of the BCI Society by utilizing brain imaging
devices to measure and enhance affect or cognition. This dissertation can serve as the start for
120
future studies towards understanding the effect of providing feedback to learners from a BCI and
the appropriate visualization for learners.
Contributions
This dissertation contributes to the understanding of obtaining spectral attention levels
from AF3 and AF4 channels (frontal lobe) using specific engagement indices with a wireless
non-invasive EEG device. This work also shows that college students find the utilization of a
BCI to quantify attention useful to improve their attention for their learning activities.
Furthermore, this research demonstrates how users perceive visualizations to graph attention and
how they prefer to scale the dataset. There are also contributions by areas described below:
Human-Computer Interaction/User Experience. A design process for neurophysiological
mobile applications that visualize affective data was presented. The design process can be used
as a guideline for future mobile applications for quantifying affective data from the brain.
Brain-Computer Interfaces (BCI). This research demonstrates a different method to
incorporate BCIs outside the traditional venues such as medicine, gaming, and machine control.
A new sub-field within BCI may emerge with a focus on quantifying attention affective state of
users using quantified-self methodologies for different domains like education.
Quantified-Self. It shows how EEG technology can be incorporated in personal informatics for
learning purposes. So far, there have only been discussions on combining quantified-self in the
learning space, but not a formal study, specifically, with a BCI. This study can serve as the
gateway of incorporating wearables for self-logging in the learning domain.
Society. This work presents a new method for college students to measure their attention during
their study activities with a Brain-Computer Interface. Also, according to students with
121
diagnosed ADHD, the tool presented in this dissertation can help them monitor their attention for
the possibility to increment their attention or learn strategies to gather more information.
Future Work
Off-the-shelf BCI devices are limited by the number of channels they have and are
designed with EEG sensors based on companies’ decisions. Interaxon created the muse with the
purpose of meditation, although it can be used to measure other activities. Further work on
measuring attention with different indices for different activities can further help us determine
the proper channels for such activity. Such findings can help with the decision of choosing EEG
channels for new wireless brain devices for measuring attention.
Follow-up studies consist of measuring the effectiveness of attention feedback with a
wireless BCI with users diagnosed with ADHD. Adopt principles and methods from
neurofeedback studies for ADHD and correlate it with quantified-self methodologies. These next
steps are based on participant’s comments and positive view towards receiving feedback for their
learning activities. Also, ADHD students do not have a tool they can use at home to monitor
their attention. The tool presented in this dissertation may be beneficial to ADHD students.
122
APPENDIX A
VISUALIZATIONS CREATED BY PARTICIPANTS IN FOCUS GROUP
Figure A-1. Visualizations drawn by participants
123
Figure A-1. Continued
124
Figure A-1. Continued
125
Figure A-1. Continued
126
Figure A-1. Continued
127
Figure A-1. Continued
128
APPENDIX B
FOCUS GROUP DEMOGRAPHIC SURVEY
BCI Focus Group Demographic Survey
Note: There is a minimal risk that security of any online data may be breached, but since no
identifying information will be collected, and the online host (Qualtrics) uses several forms of
encryption other protections, it is unlikely that a security breach of the online data will result in
any adverse consequence for you. You may review the privacy statement of Qualtrics at:
http://www.qualtrics.com/privacystatement
Please select your demographic specification:
Q1 What is your gender?
Male
Female
Q2 What is your age range?
18 - 21
22 - 24
25 - 28
29 - 31
32+
Q3 What is your ethnicity?
Caucasian
African-American
Hispanic or Latino
Asian
Native American
Other (Please Specify) ____________________
Q4 What is your primary hand?
Left-Handed
Right-Handed
Ambidextrous
129
Q5 What is your academic year?
Freshman
Sophomore
Junior
Senior (Including 5th and 6th year senior)
Master Student
Ph.D. Student
Other (Please Specify) ____________________
130
APPENDIX C
FOCUS GROUP FIRST TASK
Formulate different visualizations with scales that helps you understand your attention levels.
Draw anything that comes to your mind. There is no right or wrong answer.
#1
#2
131
APPENDIX D
FOCUS GROUP SECOND TASK
Please rank from 1 (first preference) to 3 (third preference) the type of visualization that helps
you understand your attention levels better:
#1
Figure D-1. Bar Graph
#2
Figure D-2. Clustered Graph
132
#3
Figure D-3. Pie Chart
#4
Figure D-4. Area Graph
133
#5
Figure D-5. Line Graph
#6
Figure D-6. Dots Graph
134
APPENDIX E
FOCUS GROUP THIRD TASK
Please rank from 1 (first preference) to 3 (third preference) the scale that helps you understand
your attention levels better when combining them with a visualization.
#1
0 | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | 1
0 - Not paying attention at all
1 – Extremely paying attention
#2
0 | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100
0 - Not paying attention at all
100 – Extremely paying attention
#3
F | D | D+ | C- | C | C+ | B- | B | B+ | A- | A
F - Not paying attention at all
A – Extremely paying attention
#4
1 | 2 | 3 | 4 | 5
1 - Not paying attention at all
5 – Extremely paying attention
#5
1 | 2 | 3 | 4 | 5 | 6 | 7
1 - Not paying attention at all
7 – Extremely paying attention
135
APPENDIX F
QUANTFIED-SELF ATTENTION FEEDBACK SURVEY
The survey contains the pre-survey, quiz questions for each videos, and post-survey.
QSAF Study
Note: There is a minimal risk that security of any online data may be breached, but since no
identifying information will be collected, and the online host (Qualtrics) uses several forms of
encryption other protections, it is unlikely that a security breach of the online data will result in
any adverse consequence for you. You may review the privacy statement of Qualtrics
at:http://www.qualtrics.com/privacystatement
Participant ID
Group Number
Group 0
Group 1
Group 2
Group 3
136
Pre-PANAS: Indicate to what extent you feel this way right now, that is, at the present moment.
Very Slightly
or Not at all
A Little Moderately Quite a Bit Extremely
Interested
Distressed
Excited
Upset
Strong
Guilty
Scared
Hostile
Enthusiastic
Proud
Irritable
Alert
Ashamed
Inspired
Nervous
Determined
Attentive
Jittery
Active
Afraid
137
Do you use an activity tracker? (i.e. jawbone, Fitbit, apple watch, other smartwatches, etc.)
Yes
No
If you use a tracker, what do you use it for?
Prior arriving to the study location, did you consume any energy drink or caffeine (coffee, coke,
etc.)?
Yes
No
Approximately, how many hours of sleep did you have last night?
Do you currently feel sleepy or drowsy?
Yes
No
What kind of drink did you consume?
Have you ever used a Brain-Computer Interface device (BCI)?
Yes
No
How have you used the Brain-Computer Interface device?
Do you use an application on the computer or phone to track your learning performance?
Yes
No
Learning - Which application do you use?
Learning - Please describe how you use the application and for what purpose?
Do you use an application on the computer or phone to track your attention performance?
Yes
No
Attention - Which application do you use?
Attention - Please describe how you use the application and for what purpose?
138
Please select the agreement for each statement:
Strongly
disagree
Somewhat
disagree
Neither agree
nor disagree
Somewhat
agree
Strongly agree
I know a lot
of about the
effects stress
has on my
brain
I know a lot
on the effects
of sleep
deprivation
139
LET THE RESEARCHER KNOW YOU HAVE COMPLETED THE QUESTIONS.
Did you think of anything while doing the alpha calibration?
Yes
No
Please describe your thoughts during the alpha calibration
Did you start falling asleep at any point during the alpha calibration?
Yes
No
140
LET THE RESEARCHER KNOW YOU HAVE COMPLETED THE QUESTIONS.
Please watch this video
141
QUESTIONNAIRE V1
V1Q1 - Where are the endocrine glands that control the HPA axis?
Brain and spinal cord
Skin and heart
Brain and kidneys
Stomach and kidneys
Brain and heart
V1Q2 - How does increased cortisol make you more afraid?
It increases the activity of the neurons in the fear center of your brain, the amygdala
It decreases the activity in your frontal cortex, which controls courage
It tricks your brain into thinking there is danger
It increases connections between neurons
It tricks your brain into thinking there is not danger and everything is fine
V1Q3 - What function does the hippocampus NOT control?
Learning
Fear
Memory
Stress
It controls them all
V1Q4 - How does chronic stress affect your brain's size?
It makes a swell like a balloon
It reduces synaptic connections between neurons, causing the brain to shrink
It increases synaptic connections, and therefore shrinks
It doesn't, your brain size never changes
None of the above
V1Q5 - What are epigenetic changes?
Changes to your genetic code, caused by environmental factors
Changes in which genes are expressed, caused by disease
Changes in which genes are expressed, caused by environmental factors
Changes in your genetic code, caused by disease
None of the above
142
V1 - Please select the scale in each statement based on the quiz you have just taken:
Strongly
disagree
Somewhat
disagree
Neither agree
nor disagree
Somewhat
agree
Strongly agree
The questions
in the quiz
were clear and
understandable
The questions
were easy to
answer
The questions
were relative
with the video
The questions
were useful to
test my
understanding
V1 - Have you watched this video prior to this study?
Yes
No
143
Please watch this video
144
QUESTIONNAIRE V2
V2Q1 - What has losing sleep has been linked to?
Increased inflammation
High blood pressure
Obesity
All of the above
None of the above
V2Q2 - What did Randy Gardner experience by the end of his experiment?
Hallucinations
Concentration problems
Short-term memory problems
Paranoia
All of the above
V2Q3 - How much sleep do adolescents need?
5-6 hours
9-10 hours
7-8 hours
12-13 hours
2-3hours
V2Q4 - What is the approximate percent of adolescents that are sleep deprived according to
research studies?
95
66
75
80
30
V2Q5 - Which substance in the video builds up and causes 'sleep pressure'?
Adenosine
Glutamine
Histamine
Glycine
None of the above
145
V2 - Please select the scale in each statement based on the quiz you have just taken:
Strongly
disagree
Somewhat
disagree
Neither agree
nor disagree
Somewhat
agree
Strongly agree
The questions
in the quiz
were clear and
understandable
The questions
were easy to
answer
The questions
were relative
with the video
The questions
were useful to
test my
understanding
V2 - Have you watched this video prior to this study?
Yes
No
146
Post-PANAS: Please indicate your current emotion:
Very Slightly
or Not at all
A Little Moderately Quite a Bit Extremely
Interested
Distressed
Excited
Upset
Strong
Guilty
Scared
Hostile
Enthusiastic
Proud
Irritable
Alert
Ashamed
Inspired
Nervous
Determined
Attentive
Jittery
Active
Afraid
147
Please select the agreement for each statement regarding the BCI:1
Strongly
disagree
Somewhat
disagree
Neither agree
nor disagree
Somewhat
agree
Strongly agree
After
participating
in this study,
would you
buy a BCI
device at a
low cost to
measure your
attention?
I felt
discomfort
from the BCI
1 Questions with liker scales better to have declarative statements and not in interrogative form.
148
Please select the agreement for each statement regarding the visualization:2
Strongly
disagree
Somewhat
disagree
Neither
agree nor
disagree
Somewhat
agree
Strongly
agree
N/A
Would you
use this
application
to log your
attention
levels to use
during your
learning
tasks?
I can
improve my
attention by
using this
tool
consistently
The
attention
visualization
was difficult
to
comprehend
I felt aware
of my
attention
during my
second task
after I saw
my attention
visualization
When you saw the visualization of your attention, what did you think about?
2 Questions with liker scales better to have declarative statements and not in interrogative form.
149
DEMOGRAPHICS
What is your gender?
Male
Female
What is your age range?
18 - 21
22 - 24
25 - 28
29 - 31
32+
What is your ethnicity?
Caucasian
African-American
Hispanic OR Latino
Asian
Native American
Other (Please Specify) ____________________
What is your primary hand?
Left-Handed
Right-Handed
Ambidextrous
What is your current grade level?
Freshman
Sophomore
Junior
Senior (Regardless of Senior Year)
Masters
PhD
Other (Please Specify) ____________________
What is your major?
Does your family have a history of Alzheimer's Disease?
Yes
No
I don't know
150
Does your family have history of ADHD/ADD?
Yes
No
I don't know
Do you have ADHD/ADD?
Yes
No
I don't know
How attractive is getting the apple watch for you?
Not attractive at all
It is somewhat attractive
Neither attractive or non-attractive
Attractive
Very Attractive
Please provide any other opinions or feedback you may have:
151
APPENDIX G
INTERVIEW QUESTIONS
Interview Questions
Experimental Groups Questions
1. What was going through your mind when you were watching the second video?
2. When you saw your attention visualized on the mobile device, what did you think?
3. During your second video task, did you use any mental strategy to pay more attention? If yes,
can you please describe it?
4. What did you learn about the experience about how you react? (RFQ)
5. What does the experience suggest to you about your strengths? (RFQ)
6. What might you do differently as a result of the experience and your reflections on it? What
actions do your reflections lead you to? (RFQ)
7. What do you think of this new learning process?
8. How do you see yourself using this tool?
Control Group Questions
1. What was going through your mind when you were watching the second video?
2. Is there anything that can help you become more aware of your attention for a learning task?
3. Do you think if I show you your attention gathered from your brain on a mobile device, you
would be more aware of your attention and help you improve it? How?
152
APPENDIX H
GLOSSARY
Constructs Definition How Measured
Affective Brain-Computer
Interfaces
The field that measure the
user’s affective state with
a brain imaging
technology
N/A
Attention Increment Strategy The strategy that participants
put into practice to
increment their attention
during the second task
after they see their
attention feedback
In the interview a question is
asked, “did you use any
mental strategy to pay
more attention?”
EEG Waves A set of waves that construct
the EEG signals. For this
dissertation, alpha, beta,
and theta are used
Obtained from the user’s
frontal lobe (AF3 and AF4
channels) with a non-
invasive Brain-Computer
Interface
Engagement Levels of attention Check engagement index
Engagement Index Formulas consisted of EEG
waves to obtain
engagement values from
brain activity
Beta/(Alpha+Theta),
Beta/Alpha
Self Behavioral or physical
information about the user
N/A
Self-Aware When the user is aware of
their current state, in this
care their attention after
they perform a learning
task
Asked in the post-survey if
the visualization made
them self-aware of their
attention
Self-Reflection The step when the user
reflects on their attention
level once they see their
feedback
In the post-survey, a question
is asked, if they thought
how much attention they
put into the learning task
and how to improve
Self-Regulation When users observe and
record their attention
where users reflect by
looking at the visualization
to achieve a particular goal
Measured in the interview
with a question, what went
through their mind when
they saw the visualization
Short-term Reflection When users reflect on their
attention right after they
finish their learning task
and see the visualization
See self-reflection
Quantified-Self The logging of self-data for
self-improvement
N/A
153
Constructs Definition How Measured
Quantified-Self Attention
Feedback
A visualization that represent
the user’s attention level
after performing a task
The value from the
engagement index,
Beta/(Alpha+Theta)
154
LIST OF REFERENCES
[1] I. Li, A. K. Dey, and J. Forlizzi, “Understanding my data, myself: supporting self-reflection
with ubicomp technologies,” Proceedings of the 13th international conference on
Ubiquitous computing - UbiComp’11, pp. 405-414, 2011.
[2] E. K. Choe, N. B. Lee, B. Lee, W. Pratt, and J. A. Kientz, “Understanding quantified-selfers'
practices in collecting and exploring personal data,” In Proceedings of the 32nd annual
ACM conference on Human factors in computing systems, pp. 1143-1152, 2014.
[3] R. Eynon, “The quantified self for learning: critical questions for education,” Learning,
Media and Technology, vol. 40, no. 4, pp. 407–411, 2015.
[4] D. P. O. Bos, B. Reuderink, B. van de Laar, H. Gurkok, C. Muhl, M. Poel, A. Nijholt, and D.
Heylen. "Brain-Computer Interfacing and Games" In: D. S. Tan and A. Nijholt, editors,
"Brain-Computer Interfaces: applying our minds to human-computer interaction,"
London: Springer, pp. 149-178, 2010.
[5] A. Marcengo, and A. Rapp, “Visualization of human behavior data: the quantified self,”
Innovative approaches of data visualization and visual analytics, 1, pp. 236-265, 2014.
[6] V. Rivera-Pelayo, V. Zacharias, L. Müller, and S. Braun, “Applying quantified self
approaches to support reflective learning,” Proceedings of the 2nd International
Conference on Learning Analytics and Knowledge - LAK ’12, 2012.
[7] C.L. Selfe, B. T. Petersen, and C. L. Nahrgang, “Journal writing in mathematics,” In A. Y. T.
Fulwiler (Ed.), Writing across the disciplines. Upper Montclair, NJ: Boynton/Cook, pp.
192-207, 1986.
[8] A. R. McCrindle and C. A. Christensen, “The impact of learning journals on metacognitive
and cognitive processes and learning performance,” Learning and Instruction, 5, pp. 167-
185, 1995.
[9] M. D. Lew and H. G. Schmidt, “Self-reflection and academic performance: is there a
relationship?” Advances in Health Sciences Education, 16(4), pp. 529-545, 2011.
[10] J. J. Vidal, “Toward direct brain-computer communication,” Annual review of Biophysics
and Bioengineering, 2(1), pp. 157-180, 1973.
[11] M. Andujar, C. S. Crawford, A. Nijholt, F. Jackson, and J. E. Gilbert, “Artistic brain-
computer interfaces: the expression and stimulation of the user’s affective state,” Brain-
Computer Interfaces, vol. 2, no. 2–3, pp. 60–69, 2015.
[12] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan,
“Brain–computer interfaces for communication and control,” Clinical neurophysiology,
113(6), pp. 767-791, 2002.
155
[13] J. R. Wolpaw and E. W. Wolpaw, “Brain-Computer Interfaces: something new under the
sun,” In: J. R. Wolpaw and E. W. Wolpaw, editors, Brain-Computer Interfaces:
Principles and Practice. New York: Oxford, pp. 3-12, 2012.
[14] C. Mühl, B. Allison, A. Nijholt, and G. Chanel, “A survey of affective brain computer
interfaces: principles, state-of-the-art, and challenges,” Brain-Computer Interfaces, 1(2),
pp. 66-84, 2014.
[15] C. Mühl, D. Heylen, and A. Nijholt, “Affective brain-computer interfaces: neuroscientific
approaches to affect detection,” R. Calvo, S. D'Mello, G. Gratch, and A. Kappas, The
oxford handbook of affective computing. Oxford University Press, 2015.
[16] S. Ortiz, “Brain-computer interfaces: Where human and machine meet,” Computer, 40(1),
pp. 17-21, 2007.
[17] L. F. Nicolas-Alonso and J. Gomez-Gil, “Brain computer interfaces,” a review.
Sensors,12(2), pp. 1211-1279, 2012.
[18] H. H. Jasper, “Report of the committee on methods of clinical examination in
electroencephalography,” Electroencephalography and Clinical Neurophysiology, vol.
10, no. 2, pp. 370–375, 1958.
[19] J. Liu, T.-M. Fu, Z. Cheng, G. Hong, T. Zhou, L. Jin, M. Duvvuri, Z. Jiang, P. Kruskal, C.
Xie, Z. Suo, Y. Fang, and C. M. Lieber, “Syringe-injectable electronics,” Nature
Nanotechnology, vol. 10, no. 7, pp. 629–636, Jun. 2015.
[20] H. Scherberger. "BCIs that use signals recorded in parietal or premotor cortex" In: J. R.
Wolpaw and E. W. Wolpaw, editors, "Brain-Computer Interfaces: Principles and
Practice," New York: Oxford, pp. 289, 2012.
[21] D. J. McFarland and D. J. Kruenski. "BCI signal processing: feature extraction" In: J. R.
Wolpaw and E. W. Wolpaw, editors, "Brain-Computer Interfaces: Principles and
Practice," New York: Oxford, pp. 147, 2012.
[22] C. Brunner, N. Birbaumer, B. Blankertz, C. Guger, A. Kübler, D. Mattia, J. del R. Millán, F.
Miralles, A. Nijholt, E. Opisso, N. Ramsey, P. Salomon, and G. R. Müller-Putz, “BNCI
Horizon 2020: towards a roadmap for the BCI community,” Brain-Computer Interfaces,
vol. 2, no. 1, pp. 1–10, 2015.
[23] BNCI Horizon 2020 Roadmap. [Online]. Available: http://bnci-horizon-2020.eu/roadmap.
[Accessed: July 24, 2017]
[24] R. P. Rao, “Brain-computer interfacing: an introduction,” Cambridge University Press,
2013.
[25] M. Tops and M. A. S. Boksem, “Absorbed in the task: Personality measures predict
engagement during task performance as tracked by error negativity and asymmetrical
156
frontal activity,” Cognitive, Affective, and Behavioral Neuroscience, vol. 10, no. 4, pp.
441–453, 2010.
[26] J. Frey, M. Daniel, J. Castet, M. Hachet, and F. Lotte, “Framework for
Electroencephalography-based Evaluation of User Experience,” Proceedings of the 2016
CHI Conference on Human Factors in Computing Systems - CHI’16, 2016.
[27] J.D.R. Millán, R. Rupp, G. Mueller-Putz, R. Murray-Smith, C. Giugliemma, M.
Tangermann, C. Vidaurre, F. Cincotti, A. Kubler, R. Leeb, and C. Neuper, “Combining
brain–computer interfaces and assistive technologies: state-of-the-art and challenges,”
Frontiers in neuroscience, 4, pp. 1-15, 2010.
[28] M. Ainley, “What do we know about student motivation and engagement?” Communication
presented at the Annual meeting of the Australian Association for Research in Education,
Melbourne, Australia, 2004.
[29] W. J. Horrey, M. F. Lesch, A. Garabet, L. Simmons, and R. Maikala, “Distraction and task
engagement: How interesting and boring information impact driving performance and
subjective and physiological responses,” Applied Ergonomics, vol. 58, pp. 342–348,
2017.
[30] P. R. Pintrich and E. V. de Groot, “Motivational and self-regulated learning components of
classroom academic performance,” Journal of Educational Psychology, vol. 82, no. 1, pp.
33–40, 1990.
[31] M. Csikszentmihalyi, “Flow: the Psychology of Optimal Experience,” Harper and Sons,
New York, 1990.
[32] D. Zhang, L. Zhou, R. O. Briggs, and J. F. Nunamaker, “Instructional video in e-learning:
Assessing the impact of interactive video on learning effectiveness,” Information and
Management, 43(1), 15-27, 2006.
[33] R. C. Clark, and R. E. Mayer, “E-learning and the Science of Instruction” Pfeiffer: San
Francisco, 2011.
[34] A. T. Pope, E. H. Bogart, and D. S. Bartolome, “Biocybernetic system evaluates indices of
operator engagement in automated task,” Biological Psychology, vol. 40, no. 1–2, pp.
187–195, 1995.
[35] D. Szafir and B. Mutlu, “Pay attention! Designing Adaptive Agents that Monitor and
Improve User Engagement,” Proceedings of the 2012 ACM annual conference on Human
Factors in Computing Systems – CHI’12, 2012.
[36] D. Szafir and B. Mutlu, “ARTFul: Adaptive review technology for flipped learning,”
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems -
CHI’13, 2013.
157
[37] M. Andujar, and J. E. Gilbert, “Let's learn!: enhancing user's engagement levels through
passive brain-computer interfaces,” In CHI'13 Extended Abstracts on Human Factors in
Computing Systems, pp. 703-708, 2013.
[38] J. Huang, C. Yu, Y. Wang, Y. Zhao, S. Liu, C. Mo, ... and Y. Shi, “FOCUS: enhancing
children's engagement in reading by using contextual BCI training sessions,” In
Proceedings of the 32nd annual ACM conference on Human factors in computing
systems, pp. 1905-1908, 2014.
[39] S. Coelli, R. Sclocco, R. Barbieri, G. Reni, C. Zucca, and A. M. Bianchi, “EEG-based index
for engagement level monitoring during sustained attention,” 2015 37th Annual
International Conference of the IEEE Engineering in Medicine and Biology Society
(EMBC), Aug. 2015.
[40] I. Li, A. Dey, and J. Forlizzi, “A stage-based model of personal informatics systems,”. In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.
557-566, 2010.
[41] I. Li, A. K. Dey, and J. Forlizzi, “Understanding my data, myself: supporting self-reflection
with ubicomp technologies,” In Proceedings of the 13th international conference on
Ubiquitous computing, pp. 405-414, 2011.
[42] K. Shelley, and S. Shelley, “Pulse oximeter waveform: photoelectric plethysmography”,
Clinical Monitoring, Carol Lake, R. Hines, and C. Blitt, Eds.: WB Saunders Company,
pp. 420-428, 2001.
[43] J. Meyer, S. Simske, K. A. Siek, C. G. Gurrin, and H. Hermens, “Beyond quantified self:
data for wellbeing,” In CHI'14 Extended Abstracts on Human Factors in Computing
Systems, pp. 95-98, 2014.
[44] D. Boud, R. Keogh, and D. Walker, “Reflection: Turning experience into learning,”
London: Kogan Page, 1985.
[45] J. Dewey, “How we think”, Buffalo, NY: Prometheus Books (Originally published:
Lexington, MA: D.C. Heath, 1910), 1991.
[46] J. A. Moon, “A handbook of reflective and experiential learning,” London: Routledge,
1999.
[47] Selfe, C. L., Petersen, B. T., and Nahrgang., C. L. (1986). Journal writing in mathematics. In
A. Y. T. Fulwiler (Ed.), Writing across the disciplines. Upper Montclair, NJ:
Boynton/Cook, pp. 192-207, 1986.
[48] A. R. McCrindle and C. A. Christensen, “The impact of learning journals on metacognitive
and cognitive processes and learning performance,” Learning and Instruction, 5, pp. 167-
185, 1995.
158
[49] M. D. Lew and H. G. Schmidt, “Self-reflection and academic performance: is there a
relationship?” Advances in Health Sciences Education, 16(4), pp. 529-545, 2011.
[50] J. Biggs, “Student approaches to learning and studying,” Melbourne: Australian Council for
Educational Research, 1987.
[51] J. Biggs, D. Kember, and D. Y. Leung, “The revised two‐factor study process questionnaire:
R‐SPQ‐2F.” British journal of educational psychology,71(1), pp. 133-149, 2001.
[52] D. Kember, D. Y. Leung, A. Jones, A. Y. Loke, J. McKay, K. Sinclair, ... and E. Yeung,
Development of a questionnaire to measure the level of reflective thinking. Assessment
and evaluation in higher education, 25(4), pp. 381-395, 2000.
[53] D. Y. Leung, and D. Kember, “The relationship between approaches to learning and
reflection upon practice,” Educational Psychology,23(1), pp. 61-71, 2003.
[54] J. Rottenberg, and R. D. Ray, and J. J. Gross, “Emotion elicitation using films,” In J. A.
Coan and J. J. B. Allen (Eds.), The handbook of emotion elicitation and assessment.
London: Oxford University Press, pp. 9-28, 2007.
[55] TED-Ed Talks, Madhumita Murgia: How Chronic Stress Affects your Brain. [Online].
Available: http://ed.ted.com/lessons/how-stress-affects-your-brain-madhumita-murgia.
[Accessed: July 24, 2017]
[56] TED-Ed Talks, Claudia Aguirre: The Effects of Sleep Deprivation. [Online]. Available:
http://ed.ted.com/lessons/what-would-happen-if-you-didn-t-sleep-claudia-aguirre.
[Accessed: July 24, 2017]
[57] C. Binder, “Behavioral fluency: Evolution of a new paradigm,” The Behavior Analyst,
19(2), p.163, 1996.
[58] M. Hassib, S. Schneegass, P. Eiglsperger, N. Henze, A. Schmidt, and F. Alt, “EngageMeter:
A System for Implicit Audience Engagement Sensing Using Electroencephalography,”
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems -
CHI ’17, 2017.
[59] C. Vi, and S. Subramanian, “Detecting error-related negativity for interaction design,” In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.
493-502, 2012.
[60] D. Galin, J. Johnstone, and J. Herron, “Effects of task difficulty on EEG measures of
cerebral engagement,” Neuropsychologia, 16(4), pp.461-472, 1978.
[61] S. Dikker, L. Wan, I. Davidesco, L. Kaggen, M. Oostrik, J. McClintock, J. Rowland, G.
Michalareas, J. J. Van Bavel, M. Ding, and D. Poeppel, “Brain-to-brain synchrony tracks
real-world dynamic group interactions in the classroom,” Current Biology, 27(9),
pp.1375-1380, 2017.
159
[62] T. Bourner, “Assessing reflective learning,” Education+ Training, 45(5), pp.267-272, 2003.
[63] A.-M. Brouwer, J. van Erp, D. Heylen, O. Jensen, and M. Poel, “Effortless Passive BCIs for
Healthy Users,” Lecture Notes in Computer Science, pp. 615–622, 2013.
[64] M. K. Abadi, J. Staiano, A. Cappelletti, M. Zancanaro, and N. Sebe, “Multimodal
Engagement Classification for Affective Cinema,” 2013 Humaine Association
Conference on Affective Computing and Intelligent Interaction, pp. 411-416, Sep. 2013.
[65] M. Andujar, J. I. Ekandem, J. E. Gilbert, and P. Morreale, “Evaluating Engagement
Physiologically and Knowledge Retention Subjectively through Two Different Learning
Techniques,” Lecture Notes in Computer Science, pp. 335–342, 2013.
[66] S. Jia-Wei and C. Siew Wen, “A study on non-invasive brainwave optimization,”
International Conference on Software Intelligence Technologies and Applications and
International Conference on Frontiers of Internet of Things, pp. 27-32, 2014.
[67] University of Florida brain-drone race. [Online]. Available:
https://www.youtube.com/watch?v=C0s3w-wqcI8andt. [Accessed: July 24, 2017]
[68] R. L. Mandryk, K. M. Inkpen, and T. W. Calvert, “Using psychophysiological techniques to
measure user experience with entertainment technologies,” Behaviour and Information
Technology, 25(2):141–158, 2006b.
[69] V. R. Lee, “The Quantified Self (QS) movement and some emerging opportunities for the
educational technology field. Educational Technology”, ITLS Faculty Publications, pp.
39-42, 2013.
[70] S. O’Neill, “Stripe painting: A Method of Expressing the Experience of Cycling Through
‘Quantified Self’ Data Visualization,” Proceedings of the 2016 ACM International Joint
Conference on Pervasive and Ubiquitous Computing Adjunct - UbiComp’16, pp. 600-
601, 2016.
[71] J.E. Larsen, A. Cuttone, and S.L. Jørgensen, “QS Spiral: Visualizing periodic quantified self
data,” In CHI 2013 Workshop on Personal Informatics in the Wild: Hacking Habits for
Health & Happiness, 2013.
160
BIOGRAPHICAL SKETCH
Marvin Andujar was born in Santo Domingo, Dominican Republic. In 2004, he migrated
to the United States and grew up in New Jersey. He graduated from Perth Amboy High School in
2007 and received his B.S. in computer science and B.A. in mathematical sciences from Kean
University in 2012. During his undergraduate journey, he became a Ronald E. McNair Scholar,
LSAMP Scholar, and an NSF-STEM computer science Scholar. He also held multiple leadership
positions such as: President of the Association for Computing Machinery and Vice-President of
the Senior Class. He also initiated and led a team of undergraduate students once he and his
undergraduate advisor acquired a research grant of $15,000 from the Computing Research
Association (CRA) for the Collaborative Research Experience for Undergraduates.
Marvin started his Ph.D. in human-centered computing with a research concentration on
Brain-Computer Interfaces (BCI) in 2012 at Clemson University. In 2014, he transferred to the
University of Florida along with his advisor and doctoral peers to finish his Ph.D. During his
doctoral journey, he became an NSF Graduate Research Fellow, a Google Generations Scholar,
an Intel Scholar, and a GEM Fellow. He was also inducted to Tau Beta Pi and the Alpha Epsilon
Lambda Honor Societies. He along with two of his colleagues received a research grant from
Intel of $300,000 for BCI work after a meeting with the CEO and a Senior Vice-President. The
grant funded his dissertation and led him to become one of the co-founders of the world’s first
Bra-Drone Race showcased in more than 550 media news outlets after the race was hosted.
After completing his Ph.D., he accepted an offer from University of South Florida to
become an Assistant Professor in the Computer Science and Engineering Department. You can
check his updates in his website at www.marvinandujar.com or by searching for him online.
![,f${%e0` ]e§مرأة من... · 2018. 9. 9. · ,f${%e0` ]e,f${%e0` ]e,f${%e0` ]e](https://img.pdfslide.net/doc/110x75/611ec51e4d8b02131e3d0467/fe0-e-2018-9-9-fe0-efe0-efe0-e.jpg)
