Upload
jacekg
View
53
Download
4
Embed Size (px)
Citation preview
Neuro-‐Physiological Evidence as a Basis for Studying Search
Dr. Jacek Gwizdka
Information eXperience (IX) Lab, Co-Director School of Information, University of Texas at Austin
http://gwizdka.com/research www.neuroinfoscience.org | www.neuroir.org
December 8, 2015
http://bit.ly/ix_lab http://www.ischool.utexas.edu
Understanding Search
2
Psycho–physiological (PP) States
3
� Two-way interaction ◦ PP states affect human information interaction (HII) ◦ HII induces changes in PP states
Search intent Relevance
Engagement
Peeking Inside a Searcher’s Brain
© Jacek Gwizdka
Brain activation Word vs. Concept search
Brain activation responding to relevant vs. partially relevant document
fMRI
Eye gaze patterns • reading vs. scanning
à task type • cognitive load
à task difficulty • “mindless” reading …
EEG Brain activity measured during information-interaction • cognitive load • stress • engagement …
Eye tracking
Peeking Inside a Searcher’s Brain Using Neuro-Physiological Methods
© Jacek Gwizdka
Outline
© Jacek Gwizdka 6
Neuro-Physiological (NP) Methods Three modalities: eye-tracking, EEG, and fMRI
Correlating NP data with Information Relevance
1. fMRI on static text documents 2. Eye-tracking -- ,, -- 3. Eye-tracking + EEG -- ,, -- 4. Eye-tracking on web pages
Challenges and opportunities in using NP
Experimental Design � Mixed-design with 2 blocks (types of tasks, balanced)
� Document corpus – small subset from AQUAINT ◦ a corpus of English-language news, international topics, text only
� Task type 1: WS – word search: ◦ Find target word in a short news story – press yes/no ◦ 21 trials composed of a target word 1 documents ◦ Order of trials randomized
7
xmx ssms nsns snsns jsdjsd djdjd djdj dkke ekek kdkddk dkdkdk dkdkdkd kkdkd d d dd d djdj djdjdj rjrjr rjr jweje ejejej ejej kekekek ekeke wej e eej eje j
+ + target: word
21 x
WS task instruc-
tions +
30s 6s 6s 6s 20s max 6s
WS
Experimental design with Dr. Michael Cole © Jacek Gwizdka
Experimental Design � Task type 2: IS – information search ◦ Find information that answers a given question – press yes/no. ◦ 21 trials composed of a question (task) and 3 documents at different
levels of relevance: Relevant (R), Topical (T), Irrelevant (I) à next slide
(pseudo-randomized – subset of possible combinations) ◦ Order of trials randomized ◦ To minimize memory load, the question was repeated before each stimulus
xmx ssms nsns snsns jsdjsd djdjd djdj dkke ekek dkdkdkkd kdkddk dkdkdk dkdkdkd kkdkd d d dd d djdj djdjdj rjrjr rjjweje ejejej ejej kek ekeke wej e ejej eje j
xmx ssms nsns snsns jsdjsd ke ekek dkdkdkk kdkddk dkdkdkdkdkdkd kkdkd d rjr jweje ejeje ekeke wej e ejej fjfjf fjfjfjfjf fjfjrjr rreje j
xmx ssms nsns snsns jsdjsd ke ekek dkdkdkkd
kdkddk dkdkdk dkdkdkd kkdkd d rjr jweje ejejej ejej kekekek ekeke wee ejej fjfjf fjfjfjfjf fjfjrjr rreje j
+ target: infoQ
target: infoQ
target: infoQ
21 x
IS task instruc-
tions
+ + + + + + 30s 6s 8s 6s 20s max 6s 4s 6s 20smax 6s 4s 6s 20smax
8
© Jacek Gwizdka
IS
© Jacek Gwizdka
Example IS Question and Text Docs
© Jacek Gwizdka 9
Does the next news story contain the following information: Russian submarine Kursk sinks: Which Russian fleet was the submarine part of?
Relevant News Story (R)
Irrelevant Story (I)
Partially Relevant Story (T)
© Jacek Gwizdka 10
Are brain areas involved in processing relevant and irrelevant texts different?
Experiment 1: fMRI
TYPE controlled lab study PARTICIPANTS 8 students (out of 18 total) (19-26 years); 1 fMRI session each SEARCH TASKS fact-finding task (Q&A); 21 topics STUDY DESIGN within-subjects DOC CORPUS AQUAINT - a corpus of English-language news (TREC Q&A) DATA COLLECTION binary relevance judgments
fMRI Siemens 3T and eye-tracking (EyeLink 1000)
11 © Jacek Gwizdka
NeuroIS’2013 “Looking for information relevance in the Brain” Most visionary paper award! Experimental design and data collection with Dr. Michael Cole
fMRI – functional magnetic resonance imaging
© Jacek Gwizdka 12
� fMRI uses BOLD – Brain Oxygen Level Dependent signal ◦ an indirect measure of neural activity – disputed ◦ oxygenated and de-oxygenated blood differ in magnetic properties ◦ increased neural activity à increased blood flow à more freshly oxygenated
blood à increased BOLD signal � Allows for precise location of activated brain regions, but: ◦ Poor temporal resolution (> 6 seconds)
◦ “Harsh” conditions for participants (noise, no movement, tiny space) ◦ Very expensive and difficult to use ◦ Difficult data analysis
� Danger of “reverse” inference ◦ For example � executive function à medial frontal gyrus � medial frontal gyrus à executive function
X
Pilot Study: fMRI + Eye-tracking � RUBIC = RUtgers Brain Imaging Center
Prof. Steve Hanson lab director � Lab Equipment: ◦ fMRI: 3T Siemens TRIO ◦ Eye-tracker: Eyelink-1000 � non-ferromagnetic optimized design; up to 2000 Hz sampling rate
13 © Jacek Gwizdka
fMRI + Eye-tracking Setup
Eye-tracking imposes additional constraints on projection (geometry)
projected screen
mirror
eye-tracker
14
Infrared cam
projected screen
Hypotheses
© Jacek Gwizdka 15
1. Brain activations associated with fact-finding (IS task) relevance judgments are different from activations associated with visual word search (WS task)
2. Brain activations associated with judging relevant, partially relevant and irrelevant information are different
◦ no hypothesis about specific ROIs (i.e. where the brain activity is located)
Model
Text stimulus
Text stimulus
Text stimulus
Response Response Response
6s Fixation
+
6s Fixation
+
6s Fixation
+
xmx ssms nsns snsns jsdjsd djdjd djdj dkke ekek dkdkdkkd fsdf sdfsdf kdkddk dkdkdk dkdkdkdff fsdf sdf kkdkd d d dd d djdj djdjdj rjrjr rjjweje ejejej ejej jddddd hfisdfh h osifh oishd foosh foih
xmx ssms nsns snsns ddjsdjsd ke ekek dkdkdkk kdkd sss
xmx ssms nsns snsns jsdjsd ke ekek dkdkdkkd
kdkddk dkdkdk dkdkdkd kkdkd d rjr jweje ejejej e kekekek ekeke wee ejej fjfjf fjfjfjfjf fjfjrjr rreje j
target: infoQ Q
8s 6s 20s max 6s 4s 4s 20smax 6s 4s 4s 20smax 6s 4s 4s IS
++ + ++ Q ++ Q
© Jacek Gwizdka
fMRI Data Pre-Processing
© Jacek Gwizdka 17
� Anatomical (structural) scans manually extracted brains (FSL/BET)
� motion correction � spatial smoothing 5mm � temporal filtering
Analysis – 1st Level (individual)
© Jacek Gwizdka 18
� Analysis of individual sessions using GLM (FSL/FEAT) � Activations for WS, IS (R,T,I) � Voxel thresholding (p=.05) ◦ select voxels activated at a given significance level ◦ input square waveform was convoluted using Gaussian kernel ◦ slice timing corrected by shifting the model achieved by adding temporal
derivative
Text stimulus
Text stimulus
Text stimulus
Response Response Response
6s Fixation
6s Fixation
6s Fixation
Analysis – 2nd Level (group)
© Jacek Gwizdka 19
� Group analysis using individual analysis results � GLM: t-tests and ANOVAs with repeated measures ◦ Mixed-effects analysis
� 2 factors : IS and WS ◦ WS – 1 level ◦ IS – 3 levels (R,T,I)
� IS vs. WS � IS R vs. T, R vs. I, T vs. I
� Clusters of voxels thresholded at two levels: ◦ clusters1 (C1): less restrictive Z=1.65, p=.05 ◦ clusters 2 (C2): more restrictive: Z=2.33, p=.001
Results: IS vs. WS � Simple repeated measures ANOVA, 2 factors ◦ 1-fixed (the two conditions: IS, WS), and 1 random-effect (subject)
� IS>WS significant, but WS>IS not significant ◦ C1: 1 cluster p<10-9, Zmax=3.89; ◦ C2: 2 clusters p<10-4, Zmax=3.89, � Left Occipital lobe/pole, middle occipital lobe, (near visual cortex) � Right Occipital lobe/pole, lingual gyrus ◦ (X,Y,Z)= (-25,-95,6) & (17,-93,-6) [mm] @ voxel: (115, 31, 78) & (73, 33,66)
© Jacek Gwizdka
Results IS-R, T, I
© Jacek Gwizdka 21
� Repeated measures ANOVA, 1-factor with 3-levels ◦ mixed- effects: 1 factor: fixed-effect; subject: random-effect
� F-test ◦ C1: p=.008; Zmax=3.62; Left Frontal lobe, (Middle Frontal Gyrus) � lower probability of: Precentral G, Superior FG, BA: 6 � (X,Y,Z)= (-39, 4, 53) [mm]; @ voxel (129,130,125) à Region 1 ◦ C2: not sig
Region 1
Results IS-R, T, I
� ANOVA w. repeated measures - individual contrasts: ◦ IS R-T (Relevant-Topical) sig ◦ IS R-I (Relevant-Irrelevant) sig ◦ IS-T-I (Topical-Irrelevant) not-sig
© Jacek Gwizdka
IS Relevant-Topical
© Jacek Gwizdka 23
� C1: 3 clusters ◦ p<.001 Zmax=3.58 � Left Parietal lobe � Lateral Occipital Cortex, AG, BA39 � (X,Y,Z)=(-42,51,9) [mm] ◦ p=.007 Zmax=3.18 � Right Frontal Lobe/Pole � MFG ◦ p=.02 Zmax=2.99 � Left Frontal Lobe/Pole � MFG, BA46
� C2 – none left
Region 2 Left
IS Relevant- Irrelevant
© Jacek Gwizdka 24
� C1: 3 clusters ◦ p =10-12 Zmax=4.13 � Left Frontal lobe (MFG)
◦ p=.0013, Zmax=3.25 � Right Temporal lobe � Middle & Inferior TG, BA20 ◦ p=.04, , Zmax=3.31 � Right Parietal lobe � Lateral OC, Angular Gyrus, BA39
� C2: first cluster left ◦ p=10-6 Zmax=4.13 ◦ Left Frontal lobe ◦ X,Y,Z= -39, 4, 53 [mm] ◦ Middle Frontal Gyrus � PG, SFG, BA6
Region 1
Region 2 Right
Results Summary: IS-WS; IS R–T R-I T– I
� Significant difference in activation between IS-WS
� Pattern of significant and insignificant differences in activation between IS R-T, R-I, T-I
� Brain activations similar to fMRI study with images by Moshfeghi et al. Moshfeghi, Y., Pinto, L. R., Pollick, F. E., & Jose, J. M. (2013). Understanding Relevance: An fMRI Study. ECIR 2013.
C1. Less restrictive C2. More restrictive R-T yes
(L parietal–R2L, R&L frontal) no
R-I yes (L frontal-R1, R temporal, R parietal-R2R )
yes (L frontal-R1)
T-I no no
© Jacek Gwizdka
Experiment 2: Eye-tracking + EEG + static text stimuli
TYPE controlled lab study PARTICIPANTS 24 students (9 females); 1 search session each SEARCH TASKS fact-finding task (Q&A); 21 topics STUDY DESIGN within-subjects, more in the next slides DOC CORPUS AQUAINT - a corpus of English-language news (TREC Q&A) DATA COLLECTION binary relevance judgments, eye-tracking (Tobii T60)
26 © Jacek Gwizdka
IIIX’2014 “Characterizing Relevance with Eye-tracking Measures” ETRA 2014: “News Stories Relevance Effects on Eye-Movement” Experimental design and data collection with Dr. Michael Cole
Eye-tracking History – A Digression
© Jacek Gwizdka 27
� Mechanical registration: Delabarre (1898) and Huey (1898) � Writing lever secured to eyeball using ring of plaster of Paris, later
using suction cups � Cornea was cocainized to prevent pain and winking.
writing lever light rod
eye
cornea cocainized
ring of plaster, suction cup
Eye-tracking
28
� Eye-mind link hypothesis: attention is where eyes are focused* (Just & Carpenter, 1980; 1987)
� Modern eye-trackers computerized and “easy to use” ◦ infrared light sources and cameras (low-accuracy possible using web cams)
◦ eye-trackers use relationship between pupil and corneal light reflection to calculate where a person is looking – eye-gaze location
Example Tobii eye-‐trackers © Jacek Gwizdka
mobile /wearable eye-tracker stationary (“remote”)
Research Questions
© Jacek Gwizdka 29
RQ1. Does the degree of relevance of a text document affect how it is read?
RQ2. Does the degree of relevance affect cognitive effort invested in reading it?
RQ3. Could the degree of relevance be plausibly inferred in an non-intrusive way from eye-tracking data?
Independent and Dependent Variables
© Jacek Gwizdka 30
� Independent: ◦ document relevance ◦ perceived relevance
� Dependent: ◦ time on a document (reaction time) ◦ reading state probability transitions, ◦ eye-tracking-based cognitive effort measures, ◦ pupil dilation (relative change in pupil diameter)
Two-State Reading Model
31
◦ Filter fixations < 150ms (min time required for lexical processing) ◦ Model states characterized by: � probability of transitions; number of lexical fixations; duration � length of eye-movement trajectory, amount of text covered
Scan Read
1-q
p
1-p
q isolated fixations fixation
sequences in one line of text
© Jacek Gwizdka
Cole, M. J., Gwizdka, J., Liu, C., Bierig, R., Belkin, N. J., & Zhang, X. (2011). Task and user effects on reading patterns in information search. Interacting with Computers, 23(4), 346–362
Results: Document relevance vs. perceived relevance
© Jacek Gwizdka 32
Document Relevance
I T R Total
Perceived Relevance
I 258 178 36 470
R 2 29 229 260
Total 260 205 265 730
Mean participant accuracy
99.2% 85.9% 86.4% Overall
accuracy: 91.1%
Results: Example Eye-Movement Patterns
© Jacek Gwizdka 33
What kind of document is read: Relevant, Topical, Irrelevant? 1.
Results: Example Eye-Movement Patterns
© Jacek Gwizdka 34
What kind of document is read: Relevant, Topical, Irrelevant? 2.
Results: Example Eye-Movement Patterns
© Jacek Gwizdka 35
What kind of document is read: Relevant, Topical, Irrelevant? 3.
Results: Example Eye-Movement Patterns
© Jacek Gwizdka 36
1. 2. 3.
R Relevant document T Topical document I Irrelevant document
Results: Example Eye-Movement Patterns
© Jacek Gwizdka 37
1. Irrelevant document 2. Topical document 3. Relevant document
Results: Reading Transition Probabilities
© Jacek Gwizdka 38
Degree of text’s relevance affects:
◦ how the text is read
Scan Read
1-q
p
1-p
q
R relevant T part.relevant I irrelevant
Results: Cognitive Effort & Relevance
© Jacek Gwizdka 39
Eye-tracking derived cognitive effort measures (per document)
Document relevance
Perceived relevance
Variable I T R I R Reaction time (RT) [s] L H M not. sig.
Reading speed [pixels] L H H L H Reading speed [words] not. sig. not. sig.
Duration of longest reading seq [s] L M H L H Max fixation duration in a reading seq [ms] L M H L H
Number of words fixated upon L H M not. sig. Number fixations on words L H M not. sig.
Proportions of words fixated on [%] L H M not. sig.
All significant at p<.001 H-high; M-medium; L-low
Results: Cognitive Effort & Relevance
© Jacek Gwizdka 40
Degree of text’s relevance affects:
◦ Cognitive effort
◦ Maximum cognitive effort
(not to scale)
I irrelevant T part.relevant R relevant
(not to scale)
Results: Pupilometry
© Jacek Gwizdka 41
� Pupil dilation is controlled by the Autonomic Nervous System � Dilation associated with cognitive functions: ◦ mental effort, interest, making a decision à attention
� Relative change in pupil diameter:
Relative pupil dilation measured during the whole text stimulus exposure
I T R ANOVA
Document relevance - 1.2% (.02%) - 0.4% (.02%) +0.8%(.02%) F(2, 505183)=3439, p<.001
Perceived relevance - 0.9% (.02%) +1.1% (.02%) F(1,505186)=8689, p<.001
Relative pupil dilation during the last 1000 ms before a participant’s response
Document relevance - 0.3% (.05%) +1% (.05%) +2.9% (.05%) F(2, 47498)=1211, p<.001
Perceived relevance +0.1% (.03%) +3% (.05%) F(1, 47499)=2696, p<.001
prit = (pt-pi
avg)/piavg
Results: Classification
© Jacek Gwizdka 42
� Perceived relevance: two classes: R & I ◦ Decision tress C4.5 algorithm with 10-fold cross-validation � as implemented in Weka 3.7 under the name J48 ◦ Chance probability for two classes: 50%
Variables Accuracy Reading transition probabilities 72%
Eye-tracking cognitive effort 72% Reaction time 64%
Eye-tracking + reaction time 74% Pupil diameter – whole text stimulus exposure 65%
Pupil diameter – 1000 ms before response 67%
Summary of Experiment Results at Text Stimulus Level
© Jacek Gwizdka 43
� RQ1. Does the degree of relevance of a text document affect how it is read? ◦ yes… Reading patterns differ between R, T, I text documents
� RQ2. Does it affect cognitive effort invested in reading it? ◦ yes…
� RQ3. Could the degree of relevance be plausibly inferred in an non-intrusive way from eye-tracking data? ◦ yes… possibility of inferring degree of relevance from eye-movement
patterns and from pupil dilation
� Limitations ◦ one type of texts (news stories) and only one type of a search task (fact
finding)
Experiment 2 continued Eye-tracking + EEG + static text stimuli
44 © Jacek Gwizdka
Results – within text stimulus
Unpublished. Collaboration on data analysis with Dr. Shouyi Wang, UT Arlington
EEG – Electro-encephalography
© Jacek Gwizdka 45
� Captures electrical signals at the skull – neural activity in cortex ◦ very good temporal resolution (1ms) ◦ not good at identifying location ◦ need to clean signal � electricity; facial muscles
� Two types of analysis ◦ signal properties � spectral analysis (α θ β δ) � signal power ◦ event-related potentials ERP
� Correlated with: ◦ cognitive load ◦ attention ◦ engagement ◦ meditation…
Research Questions
© Jacek Gwizdka 46
RQ1. Could the text document relevance be plausibly inferred from EEG signals obtained from a low-cost device alone and in combination with eye-tracking data?
RQ2. Does the text document relevance affect EEG signals and eye-tracking data differently at early, middle, late stages of reading and during reading relevant words?
Classification: EYE features
© Jacek Gwizdka 47
Classification: EEG features
© Jacek Gwizdka 48
Classification: Algorithms
© Jacek Gwizdka 49
� Feature Selection ◦ Minimum redundancy maximum relevance (mRMR)
� Classification Method ◦ Binary classification model Proximal Support Vector Machine (PSVM)
Classification: Data Segmentation - Epochs
© Jacek Gwizdka 50
t 200ms fixations on relevant words
1s-2s 1s-2s 1s-2s
BEGINNING MIDDLE END RELEVANT
0
Classification Results…
© Jacek Gwizdka 51
Classification Results
© Jacek Gwizdka 52
Feature Set
Epochs (pairwise for perceived relevance)
AUC Accuracy
EEG beg; mid; end 0.57; 0.60; 0.59 0.55; 0.60; 0.65
EYE beg; mid; end 0.56; 0.70; 0.80 0.40; 0.66; 0.72
EEG Fixations on relevant vs. other (beg/mid/end)
0.76-0.78 0.74-0.76
EYE Fixations on relevant vs. beg; mid; end 0.88; 0.79; 0.77 0.79 0.70; 0.71
EYE* Fixations on relevant vs. beg for 1000ms epoch 0.95 0.86
EYE+EEG* Fixations on relevant vs. beg for 1000ms epoch 0.96 0.87
Classification improvement from adding EEG features • to best EYE classification: 0.01-0.02 • to comparable EYE classification: 0.05-0.07
Classification Results Interpretation
© Jacek Gwizdka 53
� Reading relevant and not-relevant texts ◦ Eye-tracking data reflects how texts are read ◦ Relevant and irrelevant texts are initially read similarly, but then reading
style diverges
t 200ms 1s-2s 1s-2s 1s-2s
BEGINNING MIDDLE END reading relevant text
reading irrelevant text
Eye-tracking features
Classification Results Interpretation
© Jacek Gwizdka 54
� Making relevance judgment and reading other parts of texts ◦ EEG signals reflect (mainly) cognitive processes involved in relevance
judgment – plausibly decision making ◦ Eye-tracking data also shows differences in reading relevant passages
vs. all other parts
t 200ms 1s-2s 1s-2s 1s-2s
BEGINNING MIDDLE END
reading relevant text
reading irrelevant text
fixations on relevant words
RELEVANT
EEG features
© Jacek Gwizdka 55
So far: static text stimuli
Can we find similar results on web search?
Can we infer degree of relevance from eye-tracking data collected on web search?
Experiment 3: Eye-tracking + web search
TYPE controlled lab study PARTICIPANTS 32 participants (15 females); 1 search session each SEARCH TASKS 4 tasks at two levels of complexity STUDY DESIGN within-subjects DOC CORPUS Search on Wikipedia using searchtechnologies.com
search engine DATA COLLECTION binary relevance judgments, screen cam, interaction
logs, eye-tracking (Tobii T60) collected using iMotions software and our own YASFIIRE IIR framework
56 © Jacek Gwizdka
SIGIR’2015 “Differences in Eye-tracking Measures Between Visits and Revisits to Relevant and Irrelevant Web Pages“ with masters student Yinglong Zhang
Hypotheses
© Jacek Gwizdka 57
1. Pupil dilation and eye-tracking measures will differ between relevant and irrelevant pages.
2. Pupil dilation and eye-tracking measures will differ between first and subsequent visits to Web pages.
3. Pupil dilation and eye-tracking measures will differ between visits to relevant pages when a page relevance was decided compared to other visits to the same relevant pages, when the pages were not judged as relevant yet.
Eye-tracking Measures
© Jacek Gwizdka 58
Variable DescriptionFixation duration Duration of an eye fixation, in milliseconds
Saccade duration Duration of a saccade, that is of a fast eye movement between eye fixations, in milliseconds
Saccade length Length of a saccade, in pixels
Saccade angle Angle of a saccade relative to the horizontal axis, in degrees
Relative pupil dilation
The relative change in pupil diameter: A difference between pupil size at a time t and the average pupil size for a participant, normalized by that average
Results: Web Page Visit Types
© Jacek Gwizdka 59
# Page and visit types Count
1Irrelevant page
First visit306
215
2 Revisit 91
3Relevant page
First visit697
323
4 Revisit 374
5 Relevant page visit with relevance judgment 68
Comparisons Between 5 Visit Types
© Jacek Gwizdka 60
Variable 1-2 1-3 1-4 1-5 2-3 2-4 2-5 3-4 3-5 4-5
Fixation duration + +
Saccade duration + + + + + +
Saccade length + + + +
Saccade angle
Relative pupil
dilation+ + + + + + + +
Results: pupilometry
© Jacek Gwizdka 61
� Pupils dilated more on visits to relevant pages ◦ this suggests higher mental effort and attention paid to relevant pages
� Pupil dilation did not differentiate between revisits and relevance judgment visits to relevant pages. ◦ However, an increased pupil dilation in the last 2 seconds before the
relevance judgment was made allows to differentiate visits to web pages when relevance judgment was made ◊ increased attention during relevance judgment.
Classification
© Jacek Gwizdka 62
� Two binary classification models ◦ first visits to relevant pages and visits to relevant pages during which
participants judged relevance ◦ visits to irrelevant pages and visits to relevant pages during which
participants judged relevance ◦ Flexible discriminant analysis (FDA) ◦ Maximize ROC
Classification Results
© Jacek Gwizdka 63
Classification accuracy in experiment 2 with static text stimuli was 0.65-0.72
Variable Model 1 Model 2Fixation duration 29.38 29.66Saccade duration 81.27 59.94Saccade length 46.70 0.00Saccade angle 0.00 54.94
Relative pupil dilation 100.00 100.00
Model Accuracy Sensitivity SpecificityModel 1 0.57 0.57 0.57
Model 2 0.61 0.57 0.62
On-going Projects
64
� EEG and Eye-tracking to infer relevant words (PCDE) ◦ PCDE: Personalized Complex Data ExploraKon (with Lockheed MarKn and U Maryland) ◦ Hypothesis: reading of relevant vs. irrelevant words can be detected by EEG in combinaKon with eye-‐tracking: EFRPs – Eye-‐fixaKon related potenKals
� Learning about Search as Learning (LaSaL) � Detecting Periods of Mindless Information Seeking (DeMIS) ◦ employing eye-tracking methodology
© Jacek Gwizdka
In Closing…
© Jacek Gwizdka 65
Challenges in using NP methods
© Jacek Gwizdka 66
� Lack of standardized metrics � Lack of standardized tasks � Lack of baseline NP data
� Some NP methods intrusive ◦ unnatural way to perform tasks, if possible at all (e.g., fMRI)
� Highly specialized expertise required � Equipment settings, algorithms
� But … it is still worth doing !
Mapping Search Variables on NP Responses
© Jacek Gwizdka 67
Mostafa J., & Gwizdka J., (in press) “Deepening the role of the User: Neuro-Physiological Evidence as a basis for Studying and Improving Search” to appear in CHIIR 2016
68
Acknowledgements Funding: IMLS National Leadership Research CAREER Award
Google Faculty Award Rutgers University (fMRI) Lockheed Martin Corporation iSchool at UT Austin - fellowship
Collaboration: Profs. Paul Kantor and Steve Hanson with Dr. Catherine Hanson (fMRI – RUBIC/RU) Dr. Shouyi Wang (UT Arlington) (EEG data analysis) PhD candidate Michael Cole (now Dr.) Masters student Yinglong Zhang (now PhD student)
© Jacek Gwizdka
Thank You
Questions?
More info & contact: [email protected] www.gwizdka.com/research
www.neuroinfoscience.org | www.neuroir.org