Upload
shuguang-han
View
270
Download
0
Embed Size (px)
Citation preview
Contextual Support for Collaborative
Information RetrievalShuguang Han, Daqing He, Zhen
Yue and Jiepu Jiang
1
Shuguang Han, Daqing He, Zhen Yue, and Jiepu Jiang. 2016. Contextual Support for Collaborative Information Retrieval. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR '16). ACM, New York, NY, USA, 33-42. DOI=http://dx.doi.org/10.1145/2854946.2854963
MotivationCollaborative Information Retrieval (COL) VS. Individual Information Retrieval
(IND)Studies observed an increasing trend of COL, in which people search together for the same taskCOL provides unique & interesting contextual factors for search support
team project
ski trip
students
family
Communication & Coordination
Shared Search History
Search
Search2
Research Questions
RQ1: Are IND contextual support directly applicable in COL without handling complex COL interactions?
3
RQ2: Can partners’ search histories be employed for contextual search support in COL?
RQ3: Can team members’ chat content be applied for contextual search support in COL?
Contextual Support for COLSearch contexts
Five types (since chat is highly interactive and is not isolated, we merge self- and partner-chat)
4
Contextual search support modelsDocument re-ranking based on search contexts
Experiment setup for contextual search supportData about user search contextsGround-truth
Obtaining Data CollectionUser studyA self-developed collaborative web search system
Search Shared
Workspace
Shared Query HistoryCommunication &
Coordination5
Obtaining Data CollectionSearch Tasks
T1: Academic TaskWriting a report to summarize the development of SNS
T2: Leisure TaskPlanning a trip to Helsinki
User Study ProcedureConditions: Collaborative search (COL) & Individual search (IND)Within subject design
Each team (in COL) / individual(in IND) works 30 mins for each taskPost-task questionnaire for relevance rating of each saved document
User Study Data18 pairs (for COL) and 18 individuals (for IND)970 queries, 1,384 clicks and 909 saved documents
6
Ground-truth● For each team {U, V}, pooling document relevance
○ Aggregating document relevance by averaging relevance judgement from other users
● Personalize ground-truth for each query q○ Pooled relevant documents already-saved relevant documents
7
Utilizing Self Search Histories in COL RQ1: Are IND contextual support directly
applicable in COL?
8
ND
CG
@N
MA
P
G: pure google results (baseline)G + H_QUS: re-rank Google results using self query historyG + H_CLS: re-rank Google results using self click history
Take-away messages● Self search history is also useful in COL● QU > CL (different from [Shen et al. 2004])
○ may because of the task nature○ may because of that self search
behaviors in COL are influenced by collaboration
Utilizing Self Search Histories in IND
Academic Leisure
#chat messages
21.56 (20.98) 38.86 (27.24)
#queries 12.97 (6.98) 9.25 (5.30)
#click-through 24.00 (13.23)
14.44 (7.25)
search-intensive task collaboration-intensive task 9
ND
CG
@N
MA
P
Take-away messages● IND is consistent with COL for Academic task● IND is inconsistent with COL for Leisure task
○ Hypothesis: Strong collaboration effect in leisure task
G: pure google results (baseline)G + H_QUS: re-rank Google results using self query historyG + H_CLS: re-rank Google results using self click history
Utilizing Partners’ Search Histories in COL
ND
CG
@N
RQ2: Can partners’ search histories be employed for contextual search support in COL?
MA
P
10
G + H_QU: re-rank Google by both (self + partner) query historyG + H_CL: re-rank Google by both (self + partner) click history
Take-away messages● Incorporating search histories from
partners can provide a better contextual support
Utilizing Chat Histories in COL
Take-away messages● Chat-based approach performs the best in
the leisure task● Chat-based approach outperforms click-
based method in the academic task
● This finding goes beyond our expectation○ Chat contains a lot noise (our previous study
find that 30% of chats are irrlevant to tasks)
ND
CG
@N
RQ3: Can team members’ chat content be applied for contextual search support in COL?
G + H_CH: re-rank Google results by chat history
MA
P
11
Category Description
Task social (TS) Chats concerning attitude to obtained resources
Task coordination (TC) Chats about the coordination of a task
Task content (TT) Chats related to the content of the search task
Non-task (NT) Chats that are not related to the search task
G + H_X: re-rank Google with {TS, TC, TT, NT} chat history
ND
CG
@N
MA
PTake-away messages
● Any type of chat (even for NT) helps improve search
● Involving all chat messages achieves the best
Categorization schema for manual chat message coding [2]
12
Utilizing Chat Histories in COL
Analyzing Non-Task Chat in COL
Task NT (×10-
3)Others (×10-
3)
T1 p(X|C)
0.520 1.936
p(X|R)
0.509 2.422
T2 p(X|C)
0.333 2.120
p(X|R)
0.346 3.139
Why NT works?● They contain useful information. Each chat message refers to all the content a user typed in chat
box before she hits the submit button, which may cover multiple types of chat content. ● Noise in NT (e.g., lol, haha, okay) does not hurt result ranking
13
● p(NT|R) ≈ p(NT|C)● p(Others|R) > p(Others|
C)
p(X|C)
p(X|R)
corpus
relevant docs
C
R
Take-away MessagesContextual support method developed in IND can be directly applied in COL
Task may affect the utility of different search contexts
Partners’ search histories can help for contextual support in COLCombination of self and partners’ search histories provides better utility
Chat messages can help in for contextual support in COLNoise in chat messages may not affect the utility of chat-based context
14
Q & A
15
Task Socialwell, hello there
yeah! we are going to Helsinki!
everything looks great so far!
Task Coordinationyou do stats and I’ll do impacts on students and professionals
have you done impact yet?
Task Contentok so outdoor activities will be hard
in December they set up tons of markets and stuff in the streets
Non-taskCan we eat after this?
I wish there was a notification every time we saved a page
16
Examples of Different Chat Types