of 23/23
Assessing listening effort (processing load) during speech perception in noise, using the method of pupillometry Sophia E. Kramer, Adriana A. Zekveld, Thomas Koelewijn Dept of ENT/ Audiology EMGO Institute for Health and Care Research VU University medical center, Amsterdam, The Netherlands

Assessing listening effort (processing load) during · PDF fileAssessing listening effort (processing load) during speech perception in noise, using the method of pupillometry Sophia

  • View
    220

  • Download
    0

Embed Size (px)

Text of Assessing listening effort (processing load) during · PDF fileAssessing listening effort...

  • Assessing listening effort (processing load) during speech perception in noise, using the method of pupillometry

    Sophia E. Kramer, Adriana A. Zekveld, Thomas Koelewijn

    Dept of ENT/ Audiology EMGO Institute for Health and Care Research VU University medical center, Amsterdam, The Netherlands

  • An individuals attempts to compensate and communicate optimally require vigilance: a constant effort to hear and pay attention (Demorest and Erdman, 1986)

  • Netherlands Longitudinal Study on Hearing (NL-SH) *Need for Recovery (NfR) (cohort of 1000 employees)

    - Degree to which employee recovers from stressful work activities- Acute, short-term reaction- Predictor of health complaints and sick-leave in the long term

    Its difficult to concentrate in the hours after working I find it difficult to relax at the end of a working day

    With every dB increase in hearing loss, NfR increased with percent points

    Hearing loss & sick-leave in a sample of 210 employees) (Kramer et al., 2006)

    Employees with hearing loss more likely to report sick leave due to mental distress

    Self-reported hearing problems & long-term stress (Hasson et al., 2009)

    Negative relationship ability to recover from stress & self-reported hearing difficulty * (Nachtegaal et al., 2009)

  • Objective test

  • Task evoked pupil dilation is a sensitive measure of cognitive processing load- Digit list recall (Granholm et al., 1996)

    - Recall of lists in reversed order (Taylor, 1981)

    - Arithmetic test complexity (Ahern & Beatty, 1981)

    - Syntactic complexity sentences (Piquado et al., 2010)

    Pupillometry

    Cognitive processing frontal activation reticular formation pupil dilatation

    (parasympathetic & sympathetic system) (Siegle et al., 2004; Recarte et al., 2008)

  • Speech Reception Threshold (SRT) in noise test (Plomp & Mimpen, 1979)

    Intelligibility level- Adaptively estimate the Speech-to-Noise Ratio (SNR) required for: 50% correct, 71% correct , 84% correct

    Type of background noise- Stationary noise, fluctuating noise, or interfering speaker

    Task demand- Word identification (in stationary noise, 79% correct)- Noise-burst-in-stationary-noise detection (79% correct)- No demand, just listening to noise alone, with/without responding

    Test modality - Auditory versus visual

    Relation to self-report- Subjective effort ratings after each test

    Hearing impaired vs. Normal hearing

    How does the pupil respond?

  • Pupil dilation is recorded during the tests

  • Pupil response: methods

    PPD

  • 0,00

    0,00

    0,06

    0,12

    0,12

    0,13

    0,17

    0,18

    0,22

    0,24

    0,30

    0,00 0,10 0,20 0,30

    noisealone,notask

    noisealone+response("yes")

    noiseburstdetectionstat 79%

    sentencesstat84%

    wordident.stat79%

    sentencesstat71%

    sentencesfluc84%

    sentencesstat50%

    sentencesfluc50%

    sentencessingletalker84%

    sentencessingletalker50%

    Peakpupilresponseamplituderelativetobaseline(mm)

    Overview of the results of 4 published studies: Intelligibility, background noise, task demandNormally hearing adults Intelligibility effect

    -Zekveld et al. (2010), Ear and Hearing 31-Zekveld et al. (2011), Ear and Hearing 32-Kramer et al. (2012), J Lang Cogn Processes, in press -Koelewijn et al. (2012), Ear and Hearing 33

  • 0,00

    0,00

    0,06

    0,12

    0,12

    0,13

    0,17

    0,18

    0,22

    0,24

    0,30

    0,00 0,10 0,20 0,30

    noisealone,notask

    noisealone+response("yes")

    noiseburstdetectionstat 79%

    sentencesstat84%

    wordident.stat79%

    sentencesstat71%

    sentencesfluc84%

    sentencesstat50%

    sentencesfluc50%

    sentencessingletalker84%

    sentencessingletalker50%

    Peakpupilresponseamplituderelativetobaseline(mm)

    Noise effect (dark bars)

    Overview of the results of 4 published studies

    -Zekveld et al. (2010), Ear and Hearing 31-Zekveld et al. (2011), Ear and Hearing 32-Kramer et al. (2012), J Lang Cogn Processes, in press -Koelewijn et al. (2012), Ear and Hearing 33

  • 0,00

    0,00

    0,06

    0,12

    0,12

    0,13

    0,17

    0,18

    0,22

    0,24

    0,30

    0,00 0,10 0,20 0,30

    noisealone,notask

    noisealone+response("yes")

    noiseburstdetectionstat 79%

    sentencesstat84%

    wordident.stat79%

    sentencesstat71%

    sentencesfluc84%

    sentencesstat50%

    sentencesfluc50%

    sentencessingletalker84%

    sentencessingletalker50%

    Peakpupilresponseamplituderelativetobaseline(mm)

    Task effect

    Overview of the results of 4 published studies

    (mm)

    -Zekveld et al. (2010), Ear and Hearing 31-Zekveld et al. (2011), Ear and Hearing 32-Kramer et al. (2012), J Lang Cogn Processes, in press -Koelewijn et al. (2012), Ear and Hearing 33

  • 0,00

    0,00

    0,06

    0,12

    0,12

    0,13

    0,17

    0,18

    0,22

    0,24

    0,30

    0,00 0,10 0,20 0,30

    noisealone,notask

    noisealone+response("yes")

    noiseburstdetectionstat 79%

    sentencesstat84%

    wordident.stat79%

    sentencesstat71%

    sentencesfluc84%

    sentencesstat50%

    sentencesfluc50%

    sentencessingletalker84%

    sentencessingletalker50%

    Peakpupilresponseamplituderelativetobaseline(mm)

    Overview of the results of 4 published studies:

    .30 mm: corresponds to pupil response during6-digit memory task

    -Zekveld et al. (2010), Ear and Hearing 31-Zekveld et al. (2011), Ear and Hearing 32-Kramer et al. (2012), J Lang Cogn Processes, in press -Koelewijn et al. (2012), Ear and Hearing 33

  • Peak latency (relative to stimulus onset)

    Mean pupil trace in four listening conditions. SRT = speech reception threshold. Data are from Zekveld et al.2010, 2011 and Koelewijn et al.2012(SRT conditions) and from Kramer et al. In press (noise detection and word identification conditions).

  • Pattern of results of pupillometric data and effort ratings differ

    Relatively small range in effort ratings

    Relation with subjective effort ratings

    0,00

    0,00

    0,06

    0,12

    0,12

    0,13

    0,17

    0,18

    0,22

    0,24

    0,30

    0,00 0,10 0,20 0,30

    noisealone,notask

    noisealone+response("yes")

    noiseburstdetectionstat 79%

    sentencesstat84%

    wordident.stat79%

    sentencesstat71%

    sentencesfluc84%

    sentencesstat50%

    sentencesfluc50%

    sentencessingletalker84%

    sentencessingletalker50%

    Peakpupilresponseamplituderelativetobaseline(mm)

    6,1

    4,8

    5,6

    5,6

    4,8

    5,9

    6,7

    5,5

    7,0

    0,0 2,0 4,0 6,0 8,0

    Subjectiveeffortrating[0..10]No effort Very much effort

    50%

    84%

  • Effect of stimulus modality: TRT vs SRT

    Text Reception Threshold testZekveld et al. (2007)

    Main effects of test modality (p < 0.001) and difficulty level (p < 0.05), No interaction effect

    Zekveld, Festen, Kramer (2012), under review0.0

    0.1

    0.2

    0.3

    0.4

    0.5

    29% 71%

    peak dilation

    dila

    tion

    rela

    tive

    to b

    asel

    ine

    (mm

    )SRTTRT

    Similar performance differences in the visual and auditory modality affected the pupil response similarly!

  • 0.00

    0.05

    0.10

    0.15

    0.20

    0.25

    0.30

    0.35

    0.40

    0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

    pupi

    l res

    pons

    e (m

    m)

    % words correct

    0.000.050.100.150.200.250.300.350.400.45

    0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

    pupi

    l res

    pons

    e (m

    m)

    % sentences correct

    Relation intelligibility and PPD: Inverse U-shape

    -Speech masked by single-talker- < 50% overload conditions?

    Zekveld et al., (2012) under reviewZekveld & Kramer, under review

    Inte

    rindi

    vidu

    al d

    iffer

    ence

    s

  • Listening effort as function of SNR

    N = 24, 47-63 y, M 55 y N = 32, 31-76 y, M 59 y

  • Summary of results

    Evidence that the pupil response is a promising, precise, robust objective measure of processing load

    - results have been replicated- hypotheses have been confirmed

    several questions remain.- Individual differences in the pupil response?

    Partly explained by cognitive abilities (Zekveld et al., 2011, 2012; Koelewijn et al., 2012)

  • Future

    - Effects of hearing loss: perhaps strategy differences, habituation?

    - Relation with other physiological measures (fMRI)

    - Age effects on pupil response?

    - Translation of laboratory based pupil response to daily life stress

  • Kramer et al. (1997) Audiology 36, 155-164

    Zekveld et al. (2010) Ear and Hearing 31, 480-490

    Zekveld et al. (2011) Ear and Hearing, 32, 498-510.

    Kramer, et al. (2012). Lang Cogn Processes, in

    press (Epub available).

    Koelewijn, et al. (2012). Ear and Hearing 32, 291-

    300.

    Koelewijn, et al. (2012). Int J Otolaryng, in press.

    Zekveld et al. (2012) INCE conference proceedings

    or: se [email protected] nl

    For more information...

  • Thank you for your listening effort!

  • 00,020,040,060,080,1

    0,120,140,160,18

    5 4 3 2 1 0 1 2 3 4

    SRT(dBSNR)

    peakpup

    ildilatio

    n(m

    m)

    YNH

    MANH

    MAHI