Upload
rchbeir
View
125
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
Prepared by:
Presented to:
Latent Semantic Indexing
Bayonne/2013
HATOUM Saria
DONGO Irvin
Prof. CHBEIR Richard
2
• Introduction– Information Retrieval• Vector Space Model
• Problems
• Latent Semantic Indexing– Algorithm
– Example
– Advantages
– Disadvantages
Overview
3
• Many documents available.
• The need to extract information.
• Sorted and classified information.
• Queries the information.
Introduction
4
Information Retrieval
• Before LSI: – Literally Matching Text corpus with many documents.• Given a query, find relevant documents.
– Some terms in a user's query will literally match terms in irrelevant documents .
5
• Set-Theoretic– Fuzzy Set
• Algebraic– Vector Space• Generalised Vector Space• Latent Semantic Indexing
• Probabilistic– Binary Interdependence
Some Methods for IR
6
• An algebraic model for representing text documents.
• Documents and Queries are both vectors
dj =(w1,j , w2,j , …, wt,j)
qj =(w1,q , w2,q , …, wt,q)
Vector Space Model
7
vector space method– Term (rows) by document (columns) matrix, based on occurrence– one vector will be associate for each document– Cosine to measure distance between vectors (documents)• small angle = large cosine = similar• large angle = small cosine = dissimilar
8
• Sim(di, dj) = 1, if di = dj.
• Sim(di, dj) = 0, if di and dj are different.
Cosine Similarity Meausure
9
Document vector space
Query
Word 1
Word 2
10
Problem Introduction• Traditional term-matching method doesn’t work well in information
retrieval
• We want to capture the concepts instead of words. Concepts are reflected in the words. However,– One term may have multiple meaning
– Different terms may have the same meaning.
11
The Problems• Two problems that arose using the vector space model:– synonymy: many ways to express a given concept e.g.
“automobile” when querying on “car”• leads to poor recall “the percentage of all relevant documents are
retrieved”– polysemy: words have multiple meanings e.g. “surfing”• leads to poor precision “the percentage of the retrieved documents
are relevant”
• The context of the documents.
12
Polysemy and Context• Document similarity on single word level: polysemy and context
carcompany
•••dodgeford
meaning 2
ringjupiter
•••space
meaning 1…
saturn...
…planet
...
contribution to similarity, if used in 1st meaning, but not if in 2nd
13
Problematic• Allow users to retrieve information on the basis of a conceptual topic or
meaning of a document.
14
Latent Semantic Indexing• Overcome these problems of lexical matching :– Using a statistical information retrieval method that is capable of retrieving
text based on the concepts it contains, not just by matching specific keywords.
15
Characteristics of LSI • Documents are represented as "bags of words", where the order of the
words in a document is not important, only how many times each word appears in a document.
• Is a technique that projects queries and documents into a space with “latent” semantic dimensions.
• Convert high-dimensional space to lower-dimensional space
16
•Concepts are represented as patterns of words that usually appear together in documents.
– For example “jaguar", “car", and “speed" might usually appear in documents about sports cars, whereas “jaguar”, “animal”, “hunting” might refer to the concept of jaguar the animal.
•LSI is based on the principle that words that are used in the same contexts tend to have similar meanings.
•LSI uses Singular Value Decomposition for the mapping of terms to concepts.
Characteristics of LSI
17
• Number of words is huge
• throw out noise ‘and’, ‘is’, ‘at’, ‘the’, .etc.
• Select and use a smaller set of words that are of interest
• Stemming which means remove endings e.g. learning , learned , learn
Generate matrix
18
“Semantic” Space
House
Home
Domicile
Kumquat
Apple
Orange
Pear
19
Information Retrieval• Represent each document as a word vector
• Represent corpus as term-document matrix (T-D matrix) using a linear analysis method called SVD
• A classical method:– Create new vector from query terms
– Find documents with highest cosine similarity
20
• We decompose the term-document matrix into three matrices.
Singular Value Decomposition(SVD)
21
• d1: Shipment of gold damaged in a fire.
• d2: Delivery of silver arrived in a silver truck.
• d3: Shipment of gold arrived in a truck.
• q: Gold silver truck
Example
22
Example
23
Example
24
Example
25
New vectors
• d1 = [-0.4945, 0.6492]
• d2 = [-0.6458, -0.7194]
• d3 = [-0.5817, 0.2469]
Example
26
Example
27
sim(q, di) = CosΘ
• sim(q,d1) = -0.0541
• sim(q,d2) = 0.9910
• sim(q,d3) = 0.4478
Example
28
Advantages• LSI overcomes two of the most problematic constraints of queries:– Synonymy
– Polysemy
• True (latent) dimensions: the new dimensions are a better representation of documents and queries.
• Term Dependence: The traditional vector space model assumes term independence but LSI has strong associations between terms like the language.
29
Disadvantages• Storage– Many documents have more than 150 unique terms so the sparce.
• Efficiency– With LSI, the query must be compared to every document in the collection.
• Static Matrix– If we have new documents, we need to do a new SVD in the main matrix.
30
References• [Furnas et al., 1988] Furnas, G. W., Deerwester, S., Dumais, S. T., Landauer,
T. K., Harshman, R. A., Streeter, L. A., and Lochbaum, K. E. (1988). Information retrieval using a singular value decomposition model of latent semantic structure. In Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '88, pages 465-480, New York, NY, USA. ACM.
• [Hull, 1994] Hull, D. (1994). Improving text retrieval for the routing problem using latent semantic indexing. In Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '94, pages 282-291, New York, NY, USA. Springer-Verlag New York, Inc.
31
• [Atreya and Elkan, 2011] Atreya, A. and Elkan, C. (2011). Latent semantic indexing (lsi) fails for trec collections. SIGKDD Explor. Newsl., 12(2):5-10.
• [Deerwester et al., 1990] Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., and Harshman, R. (1990). Indexing by latent semantic analysis. JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE, 41(6):391-407.
• [Littman et al., 1998] Littman, M., Dumais, S. T., and Landauer, T. K. (1998). Automatic cross-language information retrieval using latent semantic indexing. In Cross-Language Information Retrieval, chapter 5, pages 51{62. Kluwer Academic Publishers.
References
32
Thank you for your Attention!!!
MILESKER ANITZ
33