Audio consciousness shift for Brain-ComputerInterface

Lior Golgher, Roni More, Ran Halprin

November 1, 2007

1 Introduction

Brain Computer Interface by EEG

Establishing a complete BCI (Brain Computer Interface) is one of the mostprominent research fields in Neuroscience. Various methods have been at-tempted to allow humans to control machines by thought alone, some quitesuccessful [5]. Alas, in order to achieve high precision in deciphering brainactivity, it appears that a direct connection to the brain is needed. Creatingsuch a connection requires surgery that is expensive, dangerous and frighten-ing to most people, and the results are still not very unreliable. This has ledto vast research attempting to create non-intrusive BCI methods, many ofwhich are based on scalp EEG (electroencephalogram). While giving muchlower resolution and much higher noise then direct contact with the brain,EEG provides a simple method to analyze brain activity.

Binaural stimulus frequency difference creates frequencycomponent

In one scalp EEG experiment set, Schwarz and Taylor showed that an audi-tory binaural stimulus of 40Hz difference could give a marked enhancementof phase-locked 40Hz activity in Cz, C3 and C4 electrode sites [8]. They stim-ulated each ear with a distinct tone, both differing in frequency by 40Hz, torecord a 40 Hz binaural beat Auditory Steady State Response (ASSR). Themaximal ASSR for a 40 Hz stimulus is well known since 1981 [7], but its cause

1

is still intensely debated (see [3][1][2][4][6]). Schwarz and Taylor note thatthe binaural ASSR vanished in specific subjects that were recruited from thelocal music department, when they attended to the two input tones ratherthan to one beating combination tone. It consistently reappeared when theyaverted from attending to any ear in particular.

We suggest attempting to use this phenomena for binary BCI. This caneasily be accomplished if the 40Hz component is learned to be detected inthe EEG stream. This method does not appear to allow better EEG basedBCI than currently existing, but we find it an interesting endeavor becausemost BCI research is pointed at detecting muscle control neural activity,while this method is based on detecting attention shifts. If this method isproven successful, it might be possible to expand it and develop better BCI,or perhaps to understand the neurological correlates of attention better.

Structure of this article

This article mainly discusses an experiment we have conducted on a singlesubject with limited musical experience, attempting to reproduce Schwarzand Taylor’s experiment and isolate the phenomena. In section 2, we describethe methodology of the experiment and data analysis. In section 3, we presentthe results and our interpretations of them. In section 4 we present ourconclusions.

2 Methodology

The experiment

In the experiment, we attempted to reproduce the 40Hz component discov-ered by Schwarz and Taylor on a single subject (Lior Golgher). After twoshort warmup sessions, the following 220 seconds long protocol was performedtwice:

• 10 seconds silence

• 10 seconds 380Hz right ear

• 10 seconds 420Hz left ear

2

• 10 seconds 40Hz both ears

• 30 seconds 380Hz right ear, 420Hz left ear, subject concentrates on leftear

• 30 seconds 380Hz right ear, 420Hz left ear, subject does not concentrateon either ear

• 30 seconds 380Hz right ear, 420Hz left ear, subject concentrates onright ear

• 30 seconds 380Hz right ear, 420Hz left ear, subject concentrates on leftear

• 30 seconds 380Hz right ear, 420Hz left ear, subject does not concentrateon either ear

• 30 seconds 380Hz right ear, 420Hz left ear, subject concentrates onright ear

During the experiment, EEG was used to capture electrical scalp activity inthe relevant electrode sites (C3, Cz and C4), as well as other electrode sitesfor control.

Data analysis

The desired result of the experiment was a strong 40Hz component when thesubject listens to 40Hz, or when he ears the combined signal of 380Hz and420Hz and does not concentrate on a specific ear. In order to test for thishypothesis, we extracted the 40Hz component from every electrode data, aswell as surrounding frequencies of 32-48Hz. We cleaned noise from the signalby cutting extreme peaks, and smoothed it with a low pass filter to easevisualization and calculations. We then normalized each extracted frequencyto mean 0 and standard deviation 1 in order to eliminate natural strengthdifferences between them.

In order to detect the relation between the EEG data and the desired re-sults, we compared each electrode (as well as a mean of all three relevantelectrodes) to a ”ground truth” vector created to be 1 when the componentis expected to be expressed strongly, and -1 when it is not expected to beexpressed.

3

3 Results

Graphs

All the graphs in this section come from the first experiment. In each graph,the EEG results are given in blue (amplitude vs. time), while the groundtruth vector is plotted on them in red.

Figures 1a, 1b

Figures 2a, 2b

4

Figures 3a, 3b

Figures 4a, 4b

5

Respective correlation in other electrode sites:

Figures 5a, 5b

Some control graphs for Cz on other frequencies:

Figures 6a, 6b

6

Interpretation

If we consider the data from neighboring electrodes (Namely Pz and Fz, seefigures 5a, 5b), we see that they also give high correlation with the groundtruth, though lower than that found in central electrode sites. This might in-dicate that the high correlation does not stem from a single source, but rathera general 40Hz activity during these times - although it is still possible thatthe same central source was recorded over parietal and frontal electrodes,due to scalp conductance.

It appears on the surface that the relatively high correlation implies thatthe experiment results were a moderate success. In all relevant electrodes,both in 40Hz and in the surrounding area of 38-42Hz, we see correlations inthe range (0.1, 0.22). When centering on the 40Hz frequency only, we seecorrelations in the range (0.11, 0.21). In addition, when we move away from40Hz and test nearby frequencies, we see a sharp drop in correlation (seefigures 6a, 6b). Consider the correlation against the frequencies tested, givenin the following figures 7a-7d (note that some frequencies around 50Hz havebeen removed during the experiment to eliminate power grid noise):

Figures 7a, 7b

7

Figures 7c, 7d

While in each electrode separately 40Hz is not the highest peak (althoughin all graphs it is a local maximum), when we combine all electrodes we see40Hz becomes the largest peak in correlation. The consistency of the 40Hzpeak hints that the other peaks are local or noise, although it appears not tobe significant enough to determine anything. The consistent negative corre-lation around 98 Hz might stem from the second harmonic of 50 Hz powergrid noise. The negative correlation around 20 Hz may be attributed to betaband activity, but its analysis is beyond the scope of this project. We haveno straightforward explanation for the marked correlation at about 90 Hz.

Hampering the positive interpretation of these results is the absence of clearpatterns during the expected 40 Hz activity durations, despite the relativelyhigh and consistent correlations found. Without known ground truth, itseems that it would have been very hard to guess what the subject is con-centrating on (even more so in real time analysis). The highest peaks andlowest troughs in the graph do not always correlate with the ground truth.Specifically, high correlations with the ground truth vector are occasionallyobserved when the subject was asked to concentrate on the sound in one of hisears, thereby attenuating the 40 Hz activity. Thus, the suggested techniquemay be of low usability for BCI applications.

The second experiment

In the second experiment, the results were much worse. For all possiblecombinations, the correlations were in the range (-0.12, 0.03). One possi-

8

ble explanation is high noise activity that penetrated the system outside theground truth activity area (this is easily visible in the original data beforereducing noise, but traces of this noise are still visible in figures 8 and 9 below)

Figure 8 Figure 9

There are two possible ways to explain this. One is that there was somefactor that altered the results of the experiment, possibly electrical noiseor subject distraction. Another option is that the subject did not performcorrectly from personal reasons.

4 Summary

The results are inconclusive. It seems as though a large set of experimentson several subjects needs to be done in order to evaluate the consistency ofthis results. Even taking the most optimistic view of the results, using themfor BCI is still difficult in the sense that it appears very hard to determinein real time what the subject is concentrating on. In addition, this methodof BCI appears limited to binary (or trinary at most) control, as a subjectcan only concentrate on one or two ears. This naturally leads to differentfuture work on the topic of concentration-based BCI, such as visual stimulus(e.g. subject concentrates on a specific visual stimuli out of several, or evenvisualizes a specific stimuli in their mind) or tactile stimulus (e.g. the subjectconcentrates in a specific tactile area).

9

Acknowledgements

The authors would like to thank Yuval Kaminka for suggesting EEG dataanalysis techniques, and Ilan Lewitus for his technical assistance in conduct-ing the EEG experiment.

References

[1] J. Artieda. Potentials evoked by chirp-modulated tones: a new tech-nique to evaluate oscillatory activity in the auditory pathway. ClinicalNeurophysiology, Volume 115, Issue 3, Pages 699-709.

[2] A. T. Herdman B. Ross and C. Pantev. Stimulus induced desynchroniza-tion of human auditory 40-hz steady-state responses. Journal of Neuro-physiology 94: 4082-4093, August 2005.

[3] Rossitza Draganova Larry E. Roberts Christo Pantev Bernhard Rob,Christian Borgmann. A high-precision magnetoencephalographic studyof human auditory steady-state responses to amplitude-modulated tones.The Journal of the Acoustical Society of America – August 2000 – Volume108, Issue 2, pp. 679-691.

[4] P. Teale M. Kleman T. Benkers J. Carlson M. Reite D. Rojas, K. Ma-harajh. Development of the 40hz steady state auditory evoked magneticfield from ages 5 to 52. Clinical Neurophysiology, Volume 117, Issue 1,Pages 110-117.

[5] A. Nurmikko et al. Donoghue, J. P. Development of neuromotor prosthe-ses for humans. Suppl Clin Neurophysiol 57: 592-606. (2004).

[6] Lior Golgher. Synchronous neural oscillations and motor ve-hicle vibrations - a preliminary literature survey, june 2006.http://www.weizmann.ac.il/home/liorg/4.pdf.

[7] Scott Makeig Robert Galambos and Peter J. Talmachoff. A 40-hz audi-tory potential recorded from the human scalp. Proceedings of the Na-tional Academy of Sciences, vol. 78 no. 4, April 1981, pages 2643-2647.

[8] D.W.F. Schwarz and P. Taylor. Human auditory steady state responsesto binaural and monaural beats. Clinical Neurophysiology, Volume 116,Issue 3, March 2005, Pages 658-668.

10