Upload
others
View
8
Download
0
Embed Size (px)
Citation preview
Introduction to Information Retrieval
Ricardo Campos
Evaluation
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
AGENDA
What is this talk about?
Batch/Lab
Evaluation2
Relevance
1
User-Study
Evaluation3
Metrics
5
Online
Evaluation4
Q&A
6
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
•
•
What is Information Retrieval?
R
R
R
R
R
R
What is Information Retrieval?
R
R
R
R
R
What is Information Retrieval?
R
R
R
R
R
What is Information Retrieval?
R
R
R
R
R
What is Information Retrieval?
R
R
R
R
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
•
•
•
What is Information Retrieval?
Batch evaluation / laboratory;•
•
•
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
•
•
•
What is Information Retrieval?
•
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
•
•
•
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
User-study evaluation;
•
•
•
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
•
•
What is Information Retrieval?
•
•
Online evaluation;•
What is Information Retrieval?
•
•
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =# 𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑡 𝐷𝑜𝑐𝑠 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑
# 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑 𝐷𝑜𝑐𝑠
𝑅𝑒𝑐𝑎𝑙𝑙 =# 𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑡 𝐷𝑜𝑐𝑠 𝑅𝑒𝑡𝑟𝑖𝑒𝑣𝑒𝑑
# 𝑅𝑒𝑙𝑒𝑣𝑎𝑛𝑡 𝐷𝑜𝑐𝑠
What is Information Retrieval?
Gold Standard
Condittion
Positive
Condition
Negative
Test
Ou
tco
me Test Outcome Positive TP FP (Type I
Error)
Test Outcome
Negative
FN (Type II
Error)
TN
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =𝑇𝑃
𝑇𝑃 + 𝐹𝑃
𝑅𝑒𝑐𝑎𝑙𝑙 =𝑇𝑃
𝑇𝑃 + 𝐹𝑁
What is Information Retrieval?
•
•
What is Information Retrieval?
RetrievedRelevant
Relevant Retrieved Precision
1.0
Recall
1.0
What is Information Retrieval?
RetrievedRelevant
Relevant Retrieved Precision
1.0
Recall
1.0
What is Information Retrieval?
R
R
R
R
R
R
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?
What is Information Retrieval?