10
Delineating the Operational Envelope of Mobile and Conventional EDA Sensing on Key Body Locations Panagiotis Tsiamyrtzis Dept. of Statistics Athens Univ. of Econ. & Bus. [email protected] Malcolm Dcosta Computational Physiology Lab Univ. Of Houston [email protected] Dvijesh Shastri Computer Science Univ. of Houston-Downtown [email protected] Eswar Prasad Computational Physiology Lab Univ. of Houston [email protected] Ioannis Pavlidis Computational Physiology Lab Univ. of Houston [email protected] ABSTRACT Electrodermal activity (EDA) is an important aective indi- cator, measured conventionally on the fingers with desktop sensing instruments. Recently, a new generation of wearable, battery-powered EDA devices came into being, encouraging the migration of EDA sensing to other body locations. To investigate the implications of such sensor/location shifts in psychophysiological studies we performed a validation exper- iment. In this experiment we used startle stimuli to instan- taneously arouse the sympathetic system of n = 23 subjects while sitting. Startle stimuli are standard but minimal stres- sors, and thus ideal for determining the sensor and location resolution limit. The experiment revealed that precise mea- surement of small EDA responses on the fingers and palm is feasible either with conventional or mobile EDA sensors. By contrast, precise measurement of small EDA responses on the sole is challenging, while on the wrist even detection of such responses is problematic for both EDA modalities. Given that aective wristbands have emerged as the dominant form of EDA sensing, researchers should beware of these limitations. Author Keywords Electrodermal Activity (EDA); skin conductance; aective sensors; wearable sensors; startle ACM Classification Keywords H.5.2 Information Interfaces and Presentation (e.g. HCI): User Interfaces; Input devices and strategies INTRODUCTION Sympathetic signals are fundamental indicators of emotional state in aective human studies [1]. Peripheral sensing of sympathetic signals primarily targets transient perspiratory responses, also known as electrodermal activity (EDA) [2]. Conventionally, EDA is measured with clinical instruments Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. CHI 2016, May 7–12, 2016, San Jose, California, USA. Copyright © 2016 ACM ISBN 978-1-4503-3362-7/16/05 ...$15.00. http://dx.doi.org/10.1145/2858036.2858536 that are grid-powered, have limited Internet connectivity, and feature tethered probes. These probes are attached to the sub- ject’s fingers. Such conventional sensing schemes restrict mobility, and in some cases interfere with the subject’s task (e.g., dexterous action studies [10]). The availability of more autonomous and unobtrusive forms of EDA sensing has in- creased with the proliferation of mobile computing [11]. There are several mobile EDA sensors in wristband or other wearable forms. These include the Q Sensor (Aectiva, Waltham, Massachusetts), the Shimmer3 GSR + Optical Pulse Development Kit (Shimmer, Dublin, Ireland), and the E4 Wristband (Empatica, Milan, Italy). There are also several conventional EDA instruments built around desktop data acqui- sition modules. These include the Powerlab (ADInstruments, Bella Vista, New South Wales), the MP 150 (Biopac, Goleta, California), and others. The technology within each family of EDA instruments (con- ventional vs. mobile) is largely the same, but there are dier- ences across families. Specifically, both instrument families use Ag/AgCl disc electrodes with contact areas of 1.0 cm 2 for their recordings, as recommended in the literature [4]; they dier, however, in terms of power mode (AC vs. DC), pack- aging (large vs. small form factor), standard communication capabilities (wired vs. wireless), and wearable options. The power mode has some implications in measurement accuracy (AC is better [2]), while the wearable options in the mobile family of sensors oer flexibility but also present a challenge. The challenge stems from the fact that sympathetically in- duced EDA responses dier in their strength among various body locations [2, 18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body locations. The combination of lesser accuracy with lesser re- sponses may bias results in mobile studies of subject aect. Such biases have not been suciently appreciated or studied in the literature. M. van Dooren et al. investigated the relative strength of EDA responses on 16 body locations [18]. They aroused sub- jects via emotional film segments, measuring responses with a conventional EDA device. The study concluded that EDA measurements at the foot were most similar with those of the finger. In contrast, arm, back, and armpit traces diered most

Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

Delineating the Operational Envelope of Mobile andConventional EDA Sensing on Key Body Locations

Panagiotis TsiamyrtzisDept. of Statistics

Athens Univ. of Econ. & [email protected]

Malcolm DcostaComputational Physiology Lab

Univ. Of [email protected]

Dvijesh ShastriComputer Science

Univ. of [email protected]

Eswar PrasadComputational Physiology Lab

Univ. of [email protected]

Ioannis PavlidisComputational Physiology Lab

Univ. of [email protected]

ABSTRACTElectrodermal activity (EDA) is an important affective indi-cator, measured conventionally on the fingers with desktopsensing instruments. Recently, a new generation of wearable,battery-powered EDA devices came into being, encouragingthe migration of EDA sensing to other body locations. Toinvestigate the implications of such sensor/location shifts inpsychophysiological studies we performed a validation exper-iment. In this experiment we used startle stimuli to instan-taneously arouse the sympathetic system of n = 23 subjectswhile sitting. Startle stimuli are standard but minimal stres-sors, and thus ideal for determining the sensor and locationresolution limit. The experiment revealed that precise mea-surement of small EDA responses on the fingers and palm isfeasible either with conventional or mobile EDA sensors. Bycontrast, precise measurement of small EDA responses on thesole is challenging, while on the wrist even detection of suchresponses is problematic for both EDA modalities. Given thataffective wristbands have emerged as the dominant form ofEDA sensing, researchers should beware of these limitations.

Author KeywordsElectrodermal Activity (EDA); skin conductance; affectivesensors; wearable sensors; startle

ACM Classification KeywordsH.5.2 Information Interfaces and Presentation (e.g. HCI): UserInterfaces; Input devices and strategies

INTRODUCTIONSympathetic signals are fundamental indicators of emotionalstate in affective human studies [1]. Peripheral sensing ofsympathetic signals primarily targets transient perspiratoryresponses, also known as electrodermal activity (EDA) [2].Conventionally, EDA is measured with clinical instruments

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from [email protected] 2016, May 7–12, 2016, San Jose, California, USA.Copyright© 2016 ACM ISBN 978-1-4503-3362-7/16/05 ...$15.00.http://dx.doi.org/10.1145/2858036.2858536

that are grid-powered, have limited Internet connectivity, andfeature tethered probes. These probes are attached to the sub-ject’s fingers. Such conventional sensing schemes restrictmobility, and in some cases interfere with the subject’s task(e.g., dexterous action studies [10]). The availability of moreautonomous and unobtrusive forms of EDA sensing has in-creased with the proliferation of mobile computing [11].

There are several mobile EDA sensors in wristband or otherwearable forms. These include the Q Sensor (Affectiva,Waltham, Massachusetts), the Shimmer3 GSR + Optical PulseDevelopment Kit (Shimmer, Dublin, Ireland), and the E4Wristband (Empatica, Milan, Italy). There are also severalconventional EDA instruments built around desktop data acqui-sition modules. These include the Powerlab (ADInstruments,Bella Vista, New South Wales), the MP 150 (Biopac, Goleta,California), and others.

The technology within each family of EDA instruments (con-ventional vs. mobile) is largely the same, but there are differ-ences across families. Specifically, both instrument familiesuse Ag/AgCl disc electrodes with contact areas of 1.0 cm2 fortheir recordings, as recommended in the literature [4]; theydiffer, however, in terms of power mode (AC vs. DC), pack-aging (large vs. small form factor), standard communicationcapabilities (wired vs. wireless), and wearable options. Thepower mode has some implications in measurement accuracy(AC is better [2]), while the wearable options in the mobilefamily of sensors offer flexibility but also present a challenge.The challenge stems from the fact that sympathetically in-duced EDA responses differ in their strength among variousbody locations [2, 18]. Unfortunately, the strongest responsesdo not necessarily correlate with the most ‘wearable’ bodylocations. The combination of lesser accuracy with lesser re-sponses may bias results in mobile studies of subject affect.Such biases have not been sufficiently appreciated or studiedin the literature.

M. van Dooren et al. investigated the relative strength ofEDA responses on 16 body locations [18]. They aroused sub-jects via emotional film segments, measuring responses witha conventional EDA device. The study concluded that EDAmeasurements at the foot were most similar with those of thefinger. In contrast, arm, back, and armpit traces differed most

Page 2: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

from the finger trace. Ranogajec and Geršak also found thatEDA finger responses are comparable to EDA foot responses[14].

Poh et al. [13] compared the measurements of mobile EDAsensors on subjects’ wrists against measurements taken withconventional EDA sensors on subjects’ fingers; the stimuli in-cluded physical activity, mental arithmetic, Stroop, and horrormovies. Then, Poh and other researchers used mobile EDAsensors to collect affective signals from daily activities includ-ing reading, walking, and sleeping [13, 16]. They also usedmobile EDA sensors in an epilepsy study [12]. The range ofapplications kept expanding. Chaspari et al. [3] used mobileEDA sensing to model verbal response latencies in autistic chil-dren. McDuff et al. [8] used mobile EDA sensing to quantifyuser emotions in a reflection study of past events. Overwhelm-ingly, in all these studies researchers used the wristband formof mobile EDA sensing and reported interesting findings.

To summarize, some researchers investigated the strength ofEDA responses at several body locations using a conventionalEDA device for the measurements; thus, they studied the lo-cation effect but not the sensor effect [18]. Other researchersattempted to investigate the sensor effect by using conven-tional EDA sensors on the subjects’ fingers and mobile EDAsensors on the subjects’ wrists; in doing so, they confoundedsensor effect with location effect [13]. Importantly, the stim-uli used in all these studies were emotionally complex andthus, non-ideal for inter-subject comparison. This leaves theinvestigation of mobile EDA sensing incomplete. We aim tofill this gap by answering two questions: (a) What is the EDAsensor effect at body locations with wearability potential? (b)What are the body’s EDA responses in these locations whensympathetic arousal takes place in its most primal form? Stim-uli used in prior studies were capable of generating complexemotions, which might depend on the subject’s genotype andphenotype. In contrast, we use auditory startle stimuli in ourstudy. Auditory startle is known to invoke a threshold responseon the fingers of healthy subjects [6]. This is the least commondenominator when it comes to subject arousal, allowing us toestablish a firm basis for comparisons.

An EDA device that successfully captures startle responseson the fingers, meets the ‘gold standard’, as it has the capa-bility of measuring a ubiquitous sympathetic arousal of lowintensity and minimal duration at a prime neurophysiologicalsite. Conventional EDA devices belong to this category. Evena gold standard device, however, may not capture startle re-sponses on a different body location (e.g., wrist), where theremay be lower concentration of perspiration glands and sparserinnervation. Hence, an EDA device should be validated forevery intended body location.

Using EDA sensors on body locations that have not been vali-dated against the gold standard may introduce bias into sym-pathetic measurements in the form of underestimation. Withthe proliferation of mobile affective studies, this is becominga pressing question.

METHODSEthics Statement. The experimental procedures were ap-proved by the University of Houston’s Institutional ReviewBoard (IRB), and were performed in accordance with the ap-proved guidelines. Informed consent was obtained from eachsubject before conducting the experiments.

Subjects. We recruited subjects through email solicitationsand flyer posting in the university campus community (popu-lation about 35, 000). We excluded children (< 18), subjectswith hearing impairments, and subjects on medications. Agebrings psychological and physiological changes that are espe-cially prominent during developmental and late years [2]; forthis reason, we did not include children and older adults (> 59)in the subject pool. The use of auditory startle stimuli in theexperiment necessitated the exclusion of subjects with hearingproblems. Certain medications affect sympathetic responses[5]; to simplify screening and minimize confounding factors,we excluded all medication cases.

Experimental Design. After each subject consented, s/hefilled out a biographic questionnaire, the Trait Anxiety In-ventory (TAI), and the State Anxiety Inventory (SAI) [17].The last two meant to check if any pathological or extraor-dinary conditions were biasing responses. Next, we askedeach subject to wash her/his hands and feet prior to sensorattachment. We waited 10 min for normal moisture levels toreestablish on their skin, before attaching the sensors. Thispreparation ensured optimal and uniform skin conditions inthe attachment areas for all subjects. We used Galvanic SkinResponse probes connected to an ADInstruments PowerLabdata acquisition unit (ADInstruments, Bella Vista, Australia)as conventional EDA sensors (C EDA); we used Q Sensorsets (Affectiva,Waltham, Massachusetts) [15] as mobile EDAsensors (M EDA).

This was a validation experiment that aimed to isolate thesensor effect from the location effect in EDA measurements.Ideally, for the sensor validation part we needed to test if theC EDA sensor and the M EDA sensor give the same mea-surement on the same body location at the same time. Therecommended test locations are the fingers and palm, wherethe sympathetic responses are known to be strong. Becauseit is impossible to attach the C and M sensors on the samelocation, however, we used the bilateral symmetry of the hu-man body as a workaround. The results show that the C sensormeasurements on the fingers and palm of the right hand werein agreement with theM sensor measurements on the fingersand palm of the left hand, respectively (see RESULTS section).For this to happen both of the following conditions had to betrue: (a) The C and M sensors had to have similar capabil-ity and (b) the subject pool had to have bilateral symmetry.Hence, our subject pool had bilateral symmetry and this is notsurprising, because the great majority of the human popula-tion has bilateral symmetry in its functional responses undernon-extreme conditions [7].

A total of n = 25 subjects fulfilling the inclusion/exclusioncriteria volunteered for the experiment. We attached seven

Page 3: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

0.66

1.87

3.09

4.30

180 240 300 360 420

µ S

Time [s]

S003 C Fingers - Raw Signal

0.66

1.87

3.09

4.30

180 300 360 420

µ S

Time [s]

S003 C Fingers - Noise-�ltered Signal

A

tOn

tP

tOff

HeadsetC Wrist

C Fingers

M WristM Palm

M Fingers

M Sole

C Palm

A B

C

Figure 1. Experiment and sample outcome. A, Experimental setup, demonstrating all seven nodes. B, Raw EDA signal on the fingers of subject S003,captured via the conventional sensing device. The dotted lines mark the occurrence of the three stimuli. Only the last minute of the baseline period isdepicted to economize space. C, The signal after noise filtering. There are multiple arousals after each stimulus; circles mark onsets, triangles markpeaks, and crosses mark offsets; ton, denotes the time of Onset occurrence, tp the time of Peak occurrence, and to f f the time of Offset occurrence; Astands for the arousal’s amplitude.

EDA sensors to each subject: three sets of a conventionalEDA sensor (C EDA) on the subject’s right half and four setsof a mobile EDA sensor (M EDA) on the subject’s left half.Specifically, we had the following sensor-location pairs (akanodes - Fig. 1A):

• Conventional EDA on the right hand fingers (C Fingers)

• Conventional EDA on the right palm (C Palm)

• Conventional EDA on the right wrist (CWrist)

• Mobile EDA on the left hand fingers (M Fingers)

• Mobile EDA on the left palm (M Palm)

• Mobile EDA on the left wrist (MWrist)

• Mobile EDA on the left sole (M Sole)

One of the 25 subjects had missing conventional signals dueto a technical problem. For another subject the recorded EDAresponses across body locations had a low signal to noiseratio, rendering analysis impossible. Hence, we excludedthese two subjects from any further consideration. The usabledata set included n = 23 subjects (11 males / 12 females; age:23.91 ± 8.12). Figure 2, Fig. 3, and Fig. 4 depict the analyzeddata set for the finger, wrist, and sole locations, respectively,facilitating cross-sensor and cross-location comparisons.

The main experiment lasted seven minutes for each subject.During these seven minutes, the subject listened to soothingmusic via a headset in a near dark room. This soothing musicwas interrupted three times with a glass breaking sound, which

served as the auditory startle stimulus. The first stimuluswas delivered in the fourth minute. After that, two morestimuli were delivered at one-minute interval each. All sensormeasurements were synchronized and recorded throughout theexperiment. Hence, our usable data bank accrued 161 EDAsignals (23 subjects × 7 EDA signals per subject - one for eachnode).

Signal Processing. The C sensor samples at the rate of 25points per second, while theM sensor samples at the rate of 32points per second. For this reason, we resampled theM signalsdown to 25 points per second to establish uniformity across thesignal bank. Then, we applied a moving window filter twiceto reduce noise. We set the window size at W = 125 points.We arrived at this selection by performing sensitivity analysiswith window sizes W = 25 × k, k = 1, 2, 3, . . . We chosethe size of the increment to be 25 points because it matchesthe minimum resolution (i.e., rate of sampling). At k = 5 weattained optimal performance by substantially reducing noisewithout destructing signal information; at k > 5 the signalsexhibited over-smoothing, which affected the performance ofthe Peak and Onset detection algorithms.

Even under the best circumstances, EDA signals are not onlynoisy (Fig. 1B) but also variable. An electrodermal responsemay feature more than one Peak. Such peaks correspondto multiple neurophysiological firings provoked by a singlestimulus [2]. Hence, for each stimulus a subject can experiencenone, one, or several peaks (Fig. 1C). For this reason, signalabstraction is quintessential to fair comparisons; noise filtering

Page 4: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

8.29

10.67

13.05

15.43

180 240 300 360 420

1.39

1.95

2.51

3.06

µ S

S002

0.57

1.67

2.77

3.87

180 240 300 360 420

2.67

3.06

3.46

3.85S003

−2.02

0.14

2.31

4.47

180 240 300 360 420

1.64

2.23

2.82

3.41S004

0.79

1.89

2.99

4.08

180 240 300 360 420

0.87

0.94

1.01

1.09

µ S

S005

−0.45

0.64

1.72

2.81

180 240 300 360 420

1.28

1.59

1.90

2.21

µ S

S006

−1.57

1.86

5.29

8.72

180 240 300 360 420

2.89

3.52

4.16

4.79S007

0.15

1.11

2.08

3.05

180 240 300 360 420

4.97

5.36

5.76

6.15S008

−3.04

−1.74

−0.44

0.85

180 240 300 360 420

µ S

S009

−0.77

3.00

6.78

10.55

180 240 300 360 420

3.38

4.56

5.74

6.93

µ S

S010

−1.65

1.68

5.00

8.33

180 240 300 360 420

2.77

3.85

4.93

6.01S011

−2.22

−0.76

0.70

2.15

180 240 300 360 420

1.66

1.97

2.28

2.59S012

5.97

8.29

10.61

12.93

180 240 300 360 420

1.55

2.21

2.87

3.53

µ S

S013

7.25

9.12

10.99

12.86

180 240 300 360 420

3.03

3.63

4.23

4.83

µ S

S014

0.83

1.47

2.11

2.76

180 240 300 360 420

2.37

2.63

2.89

3.15S015

10.22

15.97

21.71

27.46

180 240 300 360 420

2.63

3.76

4.89

6.02S016

5.73

7.43

9.13

10.84

180 240 300 360 420

2.67

3.28

3.89

4.49

µ S

S017

9.70

11.74

13.78

15.82

180 240 300 360 420

4.19

5.38

6.58

7.78

µ S

S019

0.78

1.57

2.35

3.14

180 240 300 360 420

0.58

0.65

0.73

0.81S020

−5.11

0.73

6.57

12.41

180 240 300 360 420

2.05

3.16

4.27

5.38S021

4.12

4.58

5.04

5.51

180 240 300 360 420

0.82

0.85

0.88

0.92

µ S

Time [s]

S022

5.20

6.54

7.89

9.23

180 240 300 360 420

0.83

0.92

1.00

1.09

µ S

Time [s]

S023

3.74

5.81

7.88

9.95

180 240 300 360 420

2.49

3.00

3.51

4.02

Time [s]

S024

4.11

5.55

6.99

8.42

180 240 300 360 420

2.14

2.31

2.49

2.66

µ S

Time [s]

S025

C F

ing

ers

M F

ing

ers

Figure 2. C and M EDA responses on the fingers per subject. From the four minutes of baseline only the last minute is depicted in the graphs toeconomize space. Vertical dotted lines identify stimuli times and inverted triangles denote peaks, facilitating arousal comparison. TheM EDA signalfor subject S009 was not collected due to technical reasons. The graphs confirm qualitatively the high responsiveness of the fingers location and theagreement between the two sensor modalities.

alone is not sufficient. What is of interest here is the ability ofthe sensor to measure the essence of the stimulus’ responseat the specific site. A normal neurophysiological response(arousal) can be reconstructed to a good approximation fromthree key points in the corresponding EDA signal [2]: Onset,Peak, and Offset. The Onset point represents the start of theEDA activation; the Peak point represents the culmination ofthe activation; and, the Offset point represents the ebbing ofthe activation.

On the noise-filtered signals, we focused on the non-baselineperiod from t = 4 to 7 min. We divided this period into threesegments: S 1 = [4, 5) min, S 2 = [5, 6) min, and S 3 = [6, 7]min; each segment included the respective stimulus S 1 , S 2, S 3 and the period up to the delivery of the next stimulus orthe end of the experiment. Within each segment we sought

arousals (firings), each characterized by a Peak and an Onset.An Offset might or might not have existed, depending on therecovery rate and the timing of the next arousal. To facilitatecurve analysis we performed averaging on the noise-filteredsignals reducing the resolution down to two points per second.Then, we executed the following algorithmic steps:

1. Peak Detection: We applied a Peak detection algorithm onthe non-baseline portion of each signal - S 1 ∪ S 2 ∪ S 3. Thetemplate we used for Peak detection was a 5-tuple structurefulfilling the following condition: P(t − 2) < P(t − 1) <P(t) > P(t + 1) > P(t + 2), where P(t) is the signal intensityP recorded at time t. In essence, this template is the discretedefinition of the derivative test for locating curve maxima.Figure 2, Fig. 3A, and Fig. 4 depict the peaks located bythe template matching algorithm for all legitimate signals in

Page 5: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

C W

rist

M W

rist

A

2.37

2.42

2.46

2.50

180 240 300 360 420

0.03

0.03

0.03

0.03

µ S

S002

0.03

0.10

0.17

0.24

180 240 300 360 420

S003

2.80

3.05

3.31

3.56

180 240 300 360 420

0.09

0.10

0.10

0.11S004

180 240 300 360 420

0.02

0.02

0.03

0.03S005

180 240 300 360 420

µ S

S006

180 240 300 360 420

µ S

S007

180 240 300 360 420

S008

180 240 300 360 420

0.14

0.14

0.14

0.15S009

2.06

2.64

3.22

3.80

180 240 300 360 420

0.08

0.09

0.10

0.11S010

180 240 300 360 420

µ S

S011

1.87

2.01

2.15

2.29

180 240 300 360 4200.06

0.06

0.07

0.07

0.07

µ S

S012

12.84

13.76

14.67

15.59

180 240 300 360 420

S013

180 240 300 360 420

S014

2.14

2.21

2.27

2.33

180 240 300 360 420

S015

180 240 300 360 420

µ S

S016

180 240 300 360 420

µ S

S017

180 240 300 360 420

S019

8.06

8.36

8.65

8.94

180 240 300 360 420

S020

3.54

3.65

3.75

3.85

180 240 300 360 420

0.07

0.07

0.08

0.08S021

3.68

3.81

3.94

4.07

180 240 300 360 420

0.13

0.14

0.15

0.16

µ S

Time [s]

S022

0.52

0.54

0.57

0.59

180 240 300 360 4200.05

0.06

0.06

0.07

0.07

µ S

Time [s]

S023

−0.53

−0.09

0.36

0.80

180 240 300 360 420

Time [s]

S024

3.59

4.13

4.68

5.22

180 240 300 360 420

0.34

0.35

0.37

0.38

µ S

Time [s]

S025

C W

rist

M W

rist

B

180 240 300 360 420

µ S

S002

180 240 300 360 420

0.07

0.08

0.09

0.09S003

180 240 300 360 420

S004

0.47

0.69

0.90

1.11

180 240 300 360 420

S005

0.69

0.91

1.14

1.36

180 240 300 360 420

0.14

0.20

0.26

0.32

µ S

S006

1.16

1.33

1.50

180 240 300 360 420

0.18

0.21

0.25

0.28

µ S

S007

4.94

6.22

7.50

8.79

180 240 300 360 420

0.12

0.15

0.17

0.20S008

2.67

2.77

2.88

2.98

180 240 300 360 420

S009

180 240 300 360 420

S010

1.74

2.43

3.13

3.82

180 240 300 360 420

0.10

0.12

0.14

0.16

µ S

S011

180 240 300 360 420

µ S

S012

180 240 300 360 420

0.22

0.30

0.37

S013

10.16

11.84

13.52

15.20

180 240 300 360 420

0.39

0.63

0.88

1.12S014

180 240 300 360 420

S015

11.77

14.99

18.20

180 240 300 360 420

1.16

1.78

2.40

µ S

S016

5.15

6.52

7.89

9.27

180 240 300 360 420

0.87

1.41

1.96

2.51

µ S

S017

3.69

4.98

6.27

7.56

180 240 300 360 420

0.42

0.53

0.65

0.76S019

180 240 300 360 420

0.34

0.47

0.60

0.73S020

180 240 300 360 420

Time [s]

S021

180 240 300 360 420

µ S

Time [s]

S022

180 240 300 360 420

µ S

Time [s]

S023

180 240 300 360 420

0.14

0.18

0.23

0.27

Time [s]

S024

180 240 300 360 420

µ S

Time [s]

S025

1.66

21.40 3.020.45

Figure 3. C andM EDA responses on the wrist per subject. From the four minutes of baseline only the last minute is depicted in the graphs to economizespace. Vertical dotted lines identify stimuli times and inverted triangles denote peaks, facilitating arousal comparison. A, Signals manifesting sympa-thetic responses on the wrist. B, Signals manifesting the absence of sympathetic responses on the wrist (complement to panel A), which were excludedfrom further processing and analysis. The graphs confirm qualitatively the low responsiveness of the wrist location, a result that was documentedquantitatively in the text.

the finger, wrist, and sole nodes. For the palm node, signalsand results were quite similar to the finger node. The Peakdetection algorithm accurately located almost all legitimatepeaks in our signal bank. It failed to locate peaks only in a

couple of very weak wrist signals, where there is inherentlyambiguity as to what constitutes a Peak - see S003 and S013in Figure 3A.

Page 6: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

180 240 300 360 420

1.71

2.03

2.34

2.66S002

180 240 300 360 420

3.01

3.34

3.67

4.00S003

180 240 300 360 420

1.63

2.05

2.47

2.88S004

180 240 300 360 420

0.47

0.50

0.54

0.57

µ S

S005

180 240 300 360 420

1.03

1.39

1.74

2.10S006

180 240 300 360 420

0.82

0.87

0.93

0.98S007

180 240 300 360 420

1.09

1.36

1.64

1.91S008

180 240 300 360 420

0.47

0.54

0.62

0.69

µ S

S009

180 240 300 360 420

0.99

1.40

1.82

2.23S010

180 240 300 360 420

0.75

1.05

1.35

1.65S011

180 240 300 360 420

1.08

1.40

1.71

2.03S012

180 240 300 360 420

0.69

1.18

1.67

2.15

µ S

S013

180 240 300 360 420

0.52

0.83

1.14

1.45S014

180 240 300 360 420

0.90

0.94

0.99

1.03S015

180 240 300 360 420

1.30

2.13

2.97

3.80S016

180 240 300 360 420

1.95

2.41

2.86

3.32

µ S

S017

180 240 300 360 420

1.03

1.43

1.83

2.24S019

180 240 300 360 420

1.76

2.08

2.40

2.73S020

180 240 300 360 420

0.76

1.31

1.86

2.42S021

180 240 300 360 420

0.53

0.56

0.59

0.62

µ S

Time [s]

S022

180 240 300 360 420

1.13

1.55

1.98

2.41

Time [s]

S023

180 240 300 360 420

0.76

0.84

0.92

1.00

Time [s]

S024

180 240 300 360 420

2.39

2.70

3.01

3.33µ

S

Time [s]

S025

M S

ole

Figure 4. M EDA responses on the sole per subject. From the four minutes of baseline only the last minute is depicted in the graphs to economize space.Vertical dotted lines identify stimuli times and inverted triangles denote peaks, facilitating arousal comparison. The graphs confirm qualitatively theresponsiveness of the sole location, a result that was documented quantitatively in the text. However, amplitude correlations with the finger and palmlocations are not strong, suggesting that precise EDA measurements on the sole may be challenging.

2. Detection of First Onset: Between each stimulus andthe first Peak that followed it within the respective seg-ment, we determined the corresponding Onset; this wasthe Onset of the first arousal in response to the specificstimulus. We computed the intensity of this first Onset asPS + 10%(Peak1 − PS ), where PS denotes the signal inten-sity at the time of the stimulus and Peak1 denotes the firstPeak.

3. Detection of Additional Onsets: When multiple firingstook place in response to a stimulus, then more than onePeak points existed within the respective segment S i, i =1, 2, 3. We located the Onset point corresponding to suchan additional Peak in the valley between this additionalPeak and the preceding Peak. The template we used for this

was a 3-tuple structure fulfilling the following condition:P(t − 1) > P(t) < P(t + 1), where P(t) is the signal intensityP recorded at time t.

4. Offset Detection: For each Peak we determined the match-ing Offset as the 50% × (Peak - Onset) drop-off point. Ifthis point occurred after the next Onset or stimulus, then itwas rejected and the Offset was treated as a missing value;this is an indication that the subject had not recovered at thetime a new arousal set in.

RESULTSPeaks are the most characteristic points of an electrodermalresponse. Hence, our analysis proceeds at two levels:

Page 7: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

Detection level We pay attention to the occurrence of peaks,as proxies of neurophysiological responses; the absence ofpeaks signifies the absence of a response at a node.

Measurement level We pay attention to the parameters of theneurophysiological responses recorded at the various nodes.These parameters include the times of occurrence for Onset(ton), Peak (tp), and Offset (to f f ) as well as the amplitudeof the recorded response (A = Peak - Onset). The timeston, tp, and to f f quantify the arousal’s evolution, while theamplitude quantifies the arousal’s intensity (Fig. 1C).

To ensure that the subjects’ trait and state anxiety levels didnot affect manifestation of sympathetic activation [9], we ob-tained their trait (TAI) and state (SAI) anxiety psychometrics[17]. Then, we examined in each node the correlations be-tween the number of peaks detected per subject versus thecorresponding TAI and SAI scores. None of these correlationswas statistically significant (p > 0.05).

Detection Level AnalysisThe number of peaks denote responsiveness to stimuli, withzero peaks signifying a non-responsive subject in the specificnode. Different subjects exhibited different levels of respon-siveness at different nodes. As we observe in Fig. 5, thewrist locations had the highest number of non-responsive sub-jects irrespective of the sensor type. This indicates that wristsresponded poorly to stimuli. The rest of the locations hadminimal numbers of non-responsive subjects, indicating theregular presence of arousals in response to stimuli.

We are interested to examine if indeed the proportion of non-responsive subjects differed significantly among the nodes.To do so we use the Binomial distribution. In each node wesampled 23 subjects, where each subject presented a binaryoutcome: failure, if s/he was non-responsive or success, if s/hehad at least one arousal. Hence, for each node if we call Yi therandom variable that denotes the outcome of subject i and θ theunknown probability of success, we have Yi|θ ∼ Bernoulli(θ),giving:

Yi =

{0 if no peaks in the entire experiment1 if at least one peak in the entire experiment

(1)

We count the total number of successes in the 23 subjects,forming the random variable X|θ ∼ Binomial(n, θ), n = 23:

X =

n∑i=1

Yi (2)

As we observe in Fig. 5, the maximum likelihood estimates(MLE) at the wrists are quite smaller compared to the otherlocations. In fact, running a seven-sample test for equalityof proportions shows that there are significant differencesamong the seven nodes (p < 0.01). Next, we exclude the twowrist nodes that appear to be the culprit and we compare theproportions in the remaining five nodes. This time the testreturns a non-significant number (p > 0.05), which indicatesthat with the exception of the wrist nodes, all the other nodesare statistically equivalent in terms of peak presence.

0

2

4

6

8

10

12

14

0 2 4 6 8 10 12 0

2

4

6

8

10

12

14

0 2 4 6 8 10 12 0

2

4

6

8

10

12

14

0 2 4 6 8 10 12 0

2

4

6

8

10

12

14

0 2 4 6 8 10 12

0

2

4

6

8

10

12

14

0 2 4 6 8 10 12 0

2

4

6

8

10

12

14

0 2 4 6 8 10 12 0

2

4

6

8

10

12

14

0 2 4 6 8 10 12

1.00 0.96 0.44

0.96 0.83 0.35 0.91

M Fingers M Palm M Wrist M Sole

C Fingers C Palm C Wrist

# Su

bjec

ts#

Subj

ects

# Peaks # Peaks # Peaks # Peaks

Figure 5. Responsiveness of subjects per node. Each bar chart depictsthe distribution of the number of subjects over different levels of re-sponse (i.e., number of recorded peaks in the node). Red bars indicatethe number of completely non responsive subjects for the specific node.The maximum likelihood estimates (MLE, θ̂ = X/n ) for getting at leastone peak appear on the upper right corner of the node’s panel.

Measurement Level AnalysisWe start the measurement level analysis by studying the rela-tionship of arousal timing between each pair of nodes, takinginto account all three stimuli. Figure 6 shows a matrix that issplit along its diagonal. The matrix portion below the diagonalshows the scatterplots of Onset (ton), Peak (tp), and Offset(to f f ) times for all node pairs. The matrix portion above the di-agonal shows the correlation coefficients for the correspondingscatterplots. As indicated both visually and numerically, thetiming agreement between nodes is exceptionally high. Hence,the arousal’s evolution is captured accurately at the locationsof interest (fingers, palm, wrist, sole), irrespective of the typeof sensor used. One has to note, however, that in the case ofwrist nodes, the number of points is small. This is consistentwith the finding in Fig. 5. Arousal detection at the wrist is rare,but when it occurs, both the C andM sensors track equallywell its evolution.

Next, we study the relationship of arousal intensity betweeneach pair of nodes, taking into account all three stimuli. Fig-ure 7 also shows a matrix that is split along its diagonal. Thematrix portion below the diagonal shows the amplitude (A)scatterplots for all node pairs. The matrix portion above the di-agonal shows the correlation coefficients for the correspondingscatterplots. As indicated both visually and numerically, theagreement is poor for pairs involving one wrist node, irrespec-tive of the sensor type attached to this wrist node. Agreementis also poor for pairs involving the mobile sole node. Agree-ment gets at least moderately strong for pairs involving fingers,palms, or finger-palm, irrespective of the sensing modality.

This brings to the fore a problem with EDA sensing: whilethe timing of the phenomenon is accurately captured across

Page 8: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

250 350

0.991 0.983

250 350

0.995 0.985

250 350

0.987

250

350

0.993

250

350

0.983 0.992 0.991 0.986 0.989

0.983 0.978 0.984

250

350

0.984

250

350

0.991 0.985 0.993

0.977

250

350

0.988

250

350

0.988

250 350 250 350 250 350 250 350

250

350

ton tp toff

C Finger

C Palm

C Wrist

M Finger

M Palm

M Wrist

M Sole

Time [s]

Tim

e [s

]

Figure 6. Arousal timing relationships between nodes. BELOW THE DIAGONAL: Scatter-plots of Onset (ton), Peak, (tp), and Offset (to f f ) times for thevarious node pairs. ABOVE THE DIAGONAL: The Pearson correlation coefficients that correspond to the strength of the linear relationships depictedin the scatter-plots below the diagonal (all are significant, p < 0.01).

locations and sensor types, the magnitude of the phenomenonposes a challenge. This challenge is bigger for the sole andinsurmountable for the wrist location, irrespective of the sensortype.

DISCUSSIONWe recognize the importance of quantifiable and objective in-formation the EDA responses can provide to studies of subjectaffect. Traditionally, EDA sensing is performed on the fingerswith conventional EDA devices. Although, this has serious

usability problems, it captures well minimal bursts of sym-pathetic activation. The new mobile EDA sensors, attachedon various body locations, have obvious usability advantages.However, one has to be careful not to compromise measure-ment accuracy or if s/he does, s/he should at least be aware ofit.

In this study we found that in response to minimal standardizedsympathetic stimuli, conventional EDA devices are in mod-erate agreement with mobile EDA devices on the fingers andpalm. At the same time, we found that both conventional and

Page 9: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

0 5 10

0.815 -0.064

0.0 1.5 3.0

0.739 0.559

0.0 0.2 0.4

0.245

05

15

0.169

05

10 -0.049 0.828 0.757 -0.079 0.141

0.067 0.254 0.624

0.0

1.0

2.0

0.394

0.0

1.5

3.0

0.6 -0.054 0.171

-0.328

0.0

0.4

0.8

0.043

0.0

0.2

0.4

0.096

0 5 15 0.0 1.0 2.0 0.0 0.4 0.8 0.0 1.5 3.00.

01.

53.

0

C Finger

C Palm

C Wrist

M Finger

M Palm

M Wrist

M Sole

AμS

μS

Figure 7. Arousal intensity relationships between nodes. BELOW THE DIAGONAL: Scatter-plots of amplitude (A) for the various node pairs. ABOVETHE DIAGONAL: The Pearson correlation coefficients that correspond to the strength of the linear relationships depicted in the scatter-plots belowthe diagonal (coefficients in bold are significant, p < 0.01).

mobile EDA devices give significantly inferior measurementswhen attached to the wrist. Actually, not only measurementbut also mere detection of sympathetic responses on the wristis quite challenging. Given the proliferation of mobile EDAdevices, often in the form of affective wristbands, and the ac-companying marketing hype, these results are a waking call fora more careful examination of operational limitations; other-

wise, the introduction of measurement bias in affective studiesappears likely. A simple way for researchers to guard againstthis, is to screen their subjects through a startle experiment,ensuring that they produce sympathetic responses on the wristbefore embarking on their study.

Page 10: Delineating the Operational Envelope of Mobile and ... · body locations [2,18]. Unfortunately, the strongest responses do not necessarily correlate with the most ‘wearable’ body

Another body location - the sole - that is a candidate site forwearable EDA sensing provides strong detection capability,but relatively poor measurement capability.

This study tested sensors and locations in a stationary context.While this was necessary to minimize confounding factors andestablish a clear first-level comparison, it certainly does notaccount for additional effects that are present in practice. Themain such effect is ambulatory motion that needs to be studiedin the future.

ACKNOWLEDGMENTSThis material is based upon work supported by the NationalScience Foundation via grant # IIS-1249208, entitled ‘EAGER:The Effect of Stress and the Role of Computer Mediation onExam Performance’. Any opinions, findings, and conclusionsor recommendations expressed in this paper are those of theauthors and do not necessarily reflect the views of the fundingagency.

REFERENCES1. John L Andreassi. 2000. Psychophysiology: Human

behavior & physiological response. Psychology Press,New York, New York.

2. Wolfram Boucsein. 2012. Electrodermal activity.Springer Science & Business Media, New York, NewYork.

3. Theodora Chaspari, Daniel Bone, James Gibson,Chi-Chun Lee, and Shrikanth Narayanan. May 26-31,2013. Using physiology and language cues for modelingverbal response latencies of children with ASD. InProceedings of the 2013 IEEE International Conferenceon Acoustics, Speech and Signal Processing (ICASSP).IEEE, Vancouver, BC, Canada, 3702–3706.

4. Don C Fowles, Margaret J Christie, Robert Edelberg,William W Grings, David T Lykken, and Peter HVenables. 1981. Publication recommendations forelectrodermal measurements. Psychophysiology 18, 3(1981), 232–239.

5. Alan S Horn, Joseph T Coyle, and Solomon H Snyder.1971. Catecholamine uptake by synaptosomes from ratbrain structure-activity relationships of drugs withdifferential effects on dopamine and norepinephrineneurons. Mol. Pharmacol. 7, 1 (1971), 66–80.

6. Peter J Lang, Margaret M Bradley, and Bruce N Cuthbert.1990. Emotion, attention, and the startle reflex. Psychol.Rev. 97, 3 (1990), 377–395.

7. AL MacMillan and JM Spalding. 1969. Human sweatingresponse to electrophoresed acetylcholine: A test ofpostganglionic sympathetic function. J Neurol NeurosurgPsychiatry 32, 2 (1969), 155–160.

8. Daniel McDuff, Amy Karlson, Ashish Kapoor, AstaRoseway, and Mary Czerwinski. May 5-10, 2012.AffectAura: An intelligent system for emotional memory.

In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems (CHI ’10). ACM, Austin,TX, 849–858.

9. Richard S Neary and Marvin Zuckerman. 1976.Sensation seeking, trait and state anxiety, and theelectrodermal orienting response. Psychophysiology 13, 3(1976), 205–211.

10. I Pavlidis, P Tsiamyrtzis, D Shastri, A Wesley, Y Zhou, PLindner, P Buddharaju, R Joseph, A Mandapati, BDunkin, and others. 2012. Fast by nature-how stresspatterns define human experience and performance indexterous tasks. Sci. Rep. 2 (2012), 305; doi:10.1038/srep00305.

11. Rosalind W Picard. 2010. Emotion research by thepeople, for the people. Emot Rev 2, 3 (2010), 250–254.

12. Ming-Zher Poh, Tobias Loddenkemper, Nicholas CSwenson, Shubhi Goyal, Joseph R Madsen, andRosalind W Picard. September 1-4, 2010a. Continuousmonitoring of electrodermal activity during epilepticseizures using a wearable sensor. In Proceedings of the32nd Annual International Conference of the IEEEEngineering in Medicine and Biology Society (EMBC2010). IEEE, Buenos Aires, Argentina, 4415–4418.

13. Ming-Zher Poh, Nicholas C Swenson, and Rosalind WPicard. 2010b. A wearable sensor for unobtrusive,long-term assessment of electrodermal activity. IEEETrans. Biomed. Eng. 57, 5 (2010), 1243–1252.

14. Slaven Ranogajec and Gregor Geršak. September 22-24,2014. Measuring site dependency when measuring skinconductance. In Proceedings of the Twenty-ThirdInternational Electrotechnical and Computer ScienceConference ERK 2014, Vol. 23. IEEE Region 8, Portorož,Slovenia, 155–158.

15. Rozalind W Picard et al., inventors; MassachusettsInstitute of Technology, assignee. 2012 Mar 20. Washablewearable biosensor. (March 20 2012 Mar 20). UnitedStates patent US 8,140,143.

16. Akane Sano and Rosalind W Picard. August30-September 3, 2011. Toward a taxonomy of autonomicsleep patterns with electrodermal activity. In Proceedingsof the 33rd Annual International Conference of the IEEEEngineering in Medicine and Biology Society (EMBC2011). IEEE, Boston, MA, 777–780.

17. Charles D Spielberger. 2010. State-Trait AnxietyInventory. In The Corsini Encyclopedia of Psychology,Irving B Weiner and Edward W Craighead (Eds.). JohnWiley & Sons, Inc., Hoboken, New Jersey.

18. Marieke van Dooren, Joris H Janssen, and others. 2012.Emotional sweating across the body: Comparing 16different skin conductance measurement locations.Physiol Behav 106, 2 (2012), 298–304.