Classifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns

You-un Lee, Shulan HsiehPLOS ONE, 2014

Emotion Classification

What is emotion classification?

Motivation

Business aspect: Assist in the decision-making process

Research opportunity: Help us to understand other behaviours such as intentions

Emotion Classification - Textual information

I am very excited today :)

I am very excited today

I am very excited today... but also feeling very tired

Multilingual Emotion Classification

Informality is a major issue

Imbalanced data

Alternatives

Brainwaves(Universal)

(Sophisticated)

Audio(tones)

Brainwaves as data

Pros:

Universal - reliable data source (language not an issue)

Sophisticated - opportunity to mine gems of knowledge (lots of data)

Cons:

Universal - difficult to interpret

Sophisticated - data preprocessing (e.g., noise filtering)

First thing is first...

How to collect the neuroimaging data?

● Functional magnetic resonance imaging (fMRI)

● Positron emission tomography (PET)

● Electroencephalography (EEG)

● Magnetoencephalography (MEG)

EEG(Electroencephalography)

EEG - Fundamental Concepts

Monitors electrical activity in the brain through electrodes placed along the scalp

EEG - Fundamental Concepts

EEG Bands - are defined by the frequency of brainwaves.

5 different types of brainwaves:

❖ Gamma

❖ Beta

❖ Alpha

❖ Theta

❖ Delta

Source: http://psychedelic-information-theory.com/eeg-bands

EEG - Fundamental Concepts

Each band can be associated with different emotional and mental states

Examples:❖ Rapid eye movement (REM) sleep (slower frequencies involved)❖ ADHD (too much theta, not enough alpha and beta)

EEG - Fundamental Concepts

Artifacts - electrical activities usually not originating from the brain.

EEG - Fundamental Concepts

Brain Maps - illustrates the electrical powerat each frequency

Green region - normal electrical activity

Red region - abnormal electrical activity

ObjectiveCapture the relationship between brain activity and emotional states.

Important considerations

Single-electrode level vs. Functional connectivity

Emotion is a complex behaviour Electrical activity is usually dispersed

How to estimate EEG functional connectivity?

Using three popular connectivity indices:

Correlation - (independent of amplitude)

Coherence - (amplitude and phase important)

Phase synchronization - (phase important)

Combination is important since each connectivity index is sensitive to different characteristics of EEG signals (phase, polarity, and amplitude).

http://predictablynoisy.com/correlation-and-coherence-whats-the-difference/

Basic intuition

A particular connectivity index might be better at recognizing a particular emotion

No such thing as a perfect measure

Materialand

Method

Participants

40 healthy studentsNo psychiatric illness

24 hour away from caffeine or tobaccoNT $1000

For 6 hours

Clips use for Emotional stimuli

Standard Chinese Emotional Film Clips Database (not for free; need to pay)

Clips consideration

Overpowering of a particular emotion was counterbalanced using Latin Square Design

Sad Joy Anticipation

Anticipation Sad Joy

Joy Anticipation Sad

EEG Measurement

Electrooculography (EOG) was measured to capture ocular artifacts. The eye component was later removed.

EOG and EEG amplified (500 Hz per channel)

NeuroScan 4.3.1

Feature selection

Functional connectivity in four bands for all pairs of 19 electrodes: Theta band (4-7 Hz)Alpha band (8-12 Hz)Beta band (13-30 Hz)Gamma band (31-50 Hz)

Transformation of raw EEG signals: Fast Fourier Transformation (FFT)

The connectivity indices for all pairs of electrodes at each frequency band were selected

as features. Features where ANOVAs results was significant (p<= 0.05) were kept.

Capture relevant interactions within the brain

Pattern Classification

Quadratic Discriminant Analysis (QDA)

❖ Reason: performs extremely faster evaluations compared to other algorithms

Two 2-fold cross validation

❖ Reason: each data point used for training and testing on each fold

Accuracy as an evaluation metric

❖ Reason: Imbalanced dataset problem.

Experiments

Experimental procedure

1. 60-s go/nogo task to keep participants in a neutral state.

2. Two 90-s baseline resting EEGs (eyes open, then closed)

3. The film was then shown to participant

4. Spacebar when emotion changes or is triggered

5. 60-s resting period

6. SAM self-assessment

16s (8192 data-points) signal

Experimental setup

Scale in terms of valence:

Negative (surprising / amusing clips)

Positive - (disgust / fear clips)

Neutral - (no emotion clips)

Data cleaning: remove data of users that did not felt the correct emotion when viewing

the clips (29 out of 40 got it right!)

1 2 3 4 5 6 7 8 n

Valence scores

Negative Neutral Positive

Better dataset or more participants?

Evaluation metric

“Balanced accuracy” across 50 trials

½(TP/P + TN/N) Where P = TP+FN, N = TN+FP

Confusion Matrix

Actual Prediction

Malignant Benign

Benign Benign

Benign Benign

Benign Benign

Malignant Benign

Malignant Benign

Benign Benign

Benign Benign

Benign Benign

Problem with imbalanced data

Connectivity Indices - Correlation

Significant only in:

Theta

Neg-N → T,O

N-P → T,P,O

Alpha

Neg-N → F7-P7

Neg-P → P,O

N-P → RT

Result (Correlation)

Connectivity Indices - Coherence

Significant results in Theta, Alpha, Beta

Any other patterns?

Result (Coherence)

Connectivity Indices - Phase synchronization (PSI)

Significant results in all bands

Any other patterns?

Result (PSI)

Result - Multiple bands

All frequency bands combined as features

Conclusion

● Did other stimuli affect the results? SAM helps to remove this concern.

● Better results with feature selection

● All bands can help towards emotion analysis and not just one in particular.

● PSI performs better in most cases

● Gender was not an underlying factor in the study

Research Opportunities

Utilize other emotion-eliciting stimuli such as music and picture viewing.

Analysis of other emotions (e.g., anticipation)

Deep Learning algorithms for automatic feature learning

References

http://neurosky.com/2015/04/reading-your-brainwaves-understanding-the-basics-of-eeg

/

https://addyssey.wordpress.com/2013/09/27/qeeg-as-a-diagnostic-tool-in-the-assessme

nt-of-addadhd/

http://mentalhealthdaily.com/2014/04/15/5-types-of-brain-waves-frequencies-gamma-b

eta-alpha-theta-delta/