Upload
hasan-h-topcu
View
363
Download
1
Embed Size (px)
Citation preview
ì
Learning To Rank: From Pairwise Approach to Listwise Approach
Zhe Cao, Tao Qin, Tie-‐Yan Liu, Ming-‐Feng Tsai, and Hang Li
Hasan Hüseyin Topcu
Learning To Rank
Outline
ì Related Work
ì Learning System
ì Learning to Rank
ì Pairwise vs. Listwise Approach
ì Experiments
ì Conclusion
Related Work
ì Pairwise Approach : Learning task is formalized as classificaNon of object pairs into two categories ( correctly ranked and incorrectly ranked)
ì The methods of classificaNon: ì Ranking SVM (Herbrich et al., 1999) and Joachims(2002) applied
RankingSVM to InformaNon Retrieval ì RankBoost ( Freund et al. 1998) ì RankNet (Burges et al. 2005):
Learning System
Learning System
Training Data, Data Preprocessing, …
How objects are idenNfied?
How instances are modeled?
SVM, ANN, BoosNng
Evaluate with test data
Adapted from Paaern Classificaton(Duda, Hart, Stork)
Ranking
Learning to Rank
Learning to Rank
ì A number of queries are provided
ì Each query is associated with perfect ranking list of documents (Ground-‐Truth)
ì A Ranking funcNon is created using the training data such that the model can precisely predict the ranking list.
ì Try to opNmize a Loss funcNon for learning. Note that the loss funcNon for ranking is slightly different in the sense that it makes use of sorNng.
Training Process
Data Labeling
ì Explicit Human Judgment (Perfect, Excellent, Good, Fair, Bad)
ì Implicit Relevance Judgment : Derived from click data (Search log data)
ì Ordered Pairs between documents (A > B)
ì List of judgments(scores)
Features
Pairwise Approach
ì Training data instances are document pairs in learning
Pairwise Approach
ì Collects document pairs from the ranking list and for each document pairs it assigns a label.
ì Data labels +1 if score of A > B and -‐1 if A < B
ì Formalizes the problem of learning to rank as binary classificaNon
ì RankingSVM, RankBoost and RankNet
Pairwise Approach Drawbacks
ì ObjecNve of learning is formalized as minimizing errors in classificaNon of document pairs rather than minimizing errors in ranking of documents.
ì Training process is computaNonally costly, as the documents of pairs is very large.
Pairwise Approach Drawbacks
ì Equally treats document pairs across different grades (labels) (Ex.1)
ì The number of generated document pairs varies largely from query to query, which will result in training a model biased toward queries with more document pairs. (Ex.2)
Listwise Approach
ì Training data instances are document list
ì The objecNve of learning is formalized as minimizaNon of the total loses with respect to the training data.
ì Listwise Loss FuncNon uses probability models: Permuta(on Probability and Top One Probability
Learning to Rank: From Pairwise Approach to Listwise Approach
we describe in details the listwise approach. In follow-ing descriptions, we use superscript to indicate the indexof queries and subscript to indicate the index of documentsfor a specific query.
In training, a set of queries Q = {q(1), q(2), · · · , q(m)} isgiven. Each query q(i) is associated with a list of docu-ments d(i) =
⇣
d(i)1 , d
(i)2 , · · · , d
(i)n(i)
⌘
, where d(i)j denotes the j-th
document and n(i) denotes the sizes of d(i). Furthermore,each list of documents d(i) is associated with a list of judg-ments (scores) y(i) =
⇣
y(i)1 , y
(i)2 , · · · , y
(i)n(i)
⌘
where y(i)j denotes
the judgment on document d(i)j with respect to query q(i).
The judgment y(i)j represents the relevance degree of d(i)
j toq(i), and can be a score explicitly or implicitly given by hu-mans. For example, y(i)
j can be number of clicks on d(i)j
when d(i)j is retrieved and returned for query q(i) at a search
engine (Joachims, 2002). The assumption is that the higherclick-on rate is observed for d(i)
j and q(i) the stronger rele-vance exists between them.
A feature vector x(i)j = (q(i), d(i)
j ) is created fromeach query-document pair (q(i), d(i)
j ), i = 1, 2, · · · ,m; j =
1, 2, · · · , n(i). Each list of features x(i) =⇣
x(i)1 , · · · , x
(i)n(i)
⌘
and the corresponding list of scores y(i) =⇣
y(i)1 , · · · , y
(i)n(i)
⌘
then form an ‘instance’. The training set can be denoted asT =
n
(x(i), y(i))om
i=1.
We then create a ranking function f ; for each feature vec-tor x(i)
j (corresponding to document d(i)j ) it outputs a score
f (x(i)j ). For the list of feature vectors x(i) we obtain a list of
scores z(i) =⇣
f (x(i)1 ), · · · , f (x(i)
n(i) )⌘
. The objective of learn-ing is formalized as minimization of the total losses withrespect to the training data.
mX
i=1
L(y(i), z(i)) (1)
where L is a listwise loss function.
In ranking, when a new query q(i0) and its associated docu-ments d(i0) are given, we construct feature vectors x(i0) fromthem and use the trained ranking function to assign scoresto the documents d(i0). Finally we rank the documents d(i0)
in descending order of the scores. We call the learningproblem described above as the listwise approach to learn-ing to rank.
By contrast, in the pairwise approach, a new training dataset T 0 is created from T , in which each feature vector pairx(i)
j and x(i)k forms a new instance where j , k, and +1 is
assigned to the pair if y(i)j is larger than y(i)
k otherwise �1.It turns out that the training data T 0 is a data set of bi-nary classification. A classification model like SVM canbe created. As explained in Section 1, although the pair-
wise approach has advantages, it also su↵ers from draw-backs. The listwise approach can naturally deal with theproblems, which will be made clearer in Section 6.
4. Probability ModelsWe propose using probability models to calculate the list-wise loss function in Eq. (1). Specifically we map a list ofscores to a probability distribution using one of the prob-ability models described in this section and then take anymetric between two probability distributions as a loss func-tion . The two models are referred to as permutation prob-ability and top one probability.
4.1. Permutation Probability
Suppose that the set of objects to be ranked are identifiedwith the numbers 1, 2, ..., n . A permutation ⇡ on the objectsis defined as a bijection from {1, 2, ..., n} to itself. We writethe permutation as ⇡ = h⇡(1), ⇡(2), ..., ⇡(n)i. Here, ⇡( j) de-notes the object at position j in the permutation. The set ofall possible permutations of n objects is denoted as ⌦n.
Suppose that there is a ranking function which assignsscores to the n objects. We use s to denote the list of scoress = (s1, s2, ..., sn), where s j is the score of the j-th object.Hereafter we sometimes make interchangeable the rankingfunction and the list of scores given by the ranking func-tion.
We assume that there is uncertainty in the prediction ofranking lists (permutations) using the ranking function. Inother words, any permutation is assumed to be possible, butdi↵erent permutations may have di↵erent likelihood calcu-lated based on the ranking function. We define the per-mutation probability, so that it has desirable properties forrepresenting the likelihood of a permutation (ranking list),given the ranking function.
Definition 1 Suppose that ⇡ is a permutation on the n ob-jects, and �(.) is an increasing and strictly positive func-tion. Then, the probability of permutation ⇡ given the listof scores s is defined as
Ps(⇡) =nY
j=1
�(s⇡( j))Pn
k= j �(s⇡(k))
where s⇡( j) is the score of object at position j of permutation⇡.
Let us consider an example with three objects {1, 2, 3} hav-ing scores s = (s1, s2, s3). The probabilities of permuta-tions ⇡ = h1, 2, 3i and ⇡0 = h3, 2, 1i are calculated as fol-lows:
Ps(⇡) =�(s1)
�(s1) + �(s2) + �(s3)· �(s2)�(s2) + �(s3)
· �(s3)�(s3)
.
Learning to Rank: From Pairwise Approach to Listwise Approach
we describe in details the listwise approach. In follow-ing descriptions, we use superscript to indicate the indexof queries and subscript to indicate the index of documentsfor a specific query.
In training, a set of queries Q = {q(1), q(2), · · · , q(m)} isgiven. Each query q(i) is associated with a list of docu-ments d(i) =
⇣
d(i)1 , d
(i)2 , · · · , d
(i)n(i)
⌘
, where d(i)j denotes the j-th
document and n(i) denotes the sizes of d(i). Furthermore,each list of documents d(i) is associated with a list of judg-ments (scores) y(i) =
⇣
y(i)1 , y
(i)2 , · · · , y
(i)n(i)
⌘
where y(i)j denotes
the judgment on document d(i)j with respect to query q(i).
The judgment y(i)j represents the relevance degree of d(i)
j toq(i), and can be a score explicitly or implicitly given by hu-mans. For example, y(i)
j can be number of clicks on d(i)j
when d(i)j is retrieved and returned for query q(i) at a search
engine (Joachims, 2002). The assumption is that the higherclick-on rate is observed for d(i)
j and q(i) the stronger rele-vance exists between them.
A feature vector x(i)j = (q(i), d(i)
j ) is created fromeach query-document pair (q(i), d(i)
j ), i = 1, 2, · · · ,m; j =
1, 2, · · · , n(i). Each list of features x(i) =⇣
x(i)1 , · · · , x
(i)n(i)
⌘
and the corresponding list of scores y(i) =⇣
y(i)1 , · · · , y
(i)n(i)
⌘
then form an ‘instance’. The training set can be denoted asT =
n
(x(i), y(i))om
i=1.
We then create a ranking function f ; for each feature vec-tor x(i)
j (corresponding to document d(i)j ) it outputs a score
f (x(i)j ). For the list of feature vectors x(i) we obtain a list of
scores z(i) =⇣
f (x(i)1 ), · · · , f (x(i)
n(i) )⌘
. The objective of learn-ing is formalized as minimization of the total losses withrespect to the training data.
mX
i=1
L(y(i), z(i)) (1)
where L is a listwise loss function.
In ranking, when a new query q(i0) and its associated docu-ments d(i0) are given, we construct feature vectors x(i0) fromthem and use the trained ranking function to assign scoresto the documents d(i0). Finally we rank the documents d(i0)
in descending order of the scores. We call the learningproblem described above as the listwise approach to learn-ing to rank.
By contrast, in the pairwise approach, a new training dataset T 0 is created from T , in which each feature vector pairx(i)
j and x(i)k forms a new instance where j , k, and +1 is
assigned to the pair if y(i)j is larger than y(i)
k otherwise �1.It turns out that the training data T 0 is a data set of bi-nary classification. A classification model like SVM canbe created. As explained in Section 1, although the pair-
wise approach has advantages, it also su↵ers from draw-backs. The listwise approach can naturally deal with theproblems, which will be made clearer in Section 6.
4. Probability ModelsWe propose using probability models to calculate the list-wise loss function in Eq. (1). Specifically we map a list ofscores to a probability distribution using one of the prob-ability models described in this section and then take anymetric between two probability distributions as a loss func-tion . The two models are referred to as permutation prob-ability and top one probability.
4.1. Permutation Probability
Suppose that the set of objects to be ranked are identifiedwith the numbers 1, 2, ..., n . A permutation ⇡ on the objectsis defined as a bijection from {1, 2, ..., n} to itself. We writethe permutation as ⇡ = h⇡(1), ⇡(2), ..., ⇡(n)i. Here, ⇡( j) de-notes the object at position j in the permutation. The set ofall possible permutations of n objects is denoted as ⌦n.
Suppose that there is a ranking function which assignsscores to the n objects. We use s to denote the list of scoress = (s1, s2, ..., sn), where s j is the score of the j-th object.Hereafter we sometimes make interchangeable the rankingfunction and the list of scores given by the ranking func-tion.
We assume that there is uncertainty in the prediction ofranking lists (permutations) using the ranking function. Inother words, any permutation is assumed to be possible, butdi↵erent permutations may have di↵erent likelihood calcu-lated based on the ranking function. We define the per-mutation probability, so that it has desirable properties forrepresenting the likelihood of a permutation (ranking list),given the ranking function.
Definition 1 Suppose that ⇡ is a permutation on the n ob-jects, and �(.) is an increasing and strictly positive func-tion. Then, the probability of permutation ⇡ given the listof scores s is defined as
Ps(⇡) =nY
j=1
�(s⇡( j))Pn
k= j �(s⇡(k))
where s⇡( j) is the score of object at position j of permutation⇡.
Let us consider an example with three objects {1, 2, 3} hav-ing scores s = (s1, s2, s3). The probabilities of permuta-tions ⇡ = h1, 2, 3i and ⇡0 = h3, 2, 1i are calculated as fol-lows:
Ps(⇡) =�(s1)
�(s1) + �(s2) + �(s3)· �(s2)�(s2) + �(s3)
· �(s3)�(s3)
.
Learning to Rank: From Pairwise Approach to Listwise Approach
we describe in details the listwise approach. In follow-ing descriptions, we use superscript to indicate the indexof queries and subscript to indicate the index of documentsfor a specific query.
In training, a set of queries Q = {q(1), q(2), · · · , q(m)} isgiven. Each query q(i) is associated with a list of docu-ments d(i) =
⇣
d(i)1 , d
(i)2 , · · · , d
(i)n(i)
⌘
, where d(i)j denotes the j-th
document and n(i) denotes the sizes of d(i). Furthermore,each list of documents d(i) is associated with a list of judg-ments (scores) y(i) =
⇣
y(i)1 , y
(i)2 , · · · , y
(i)n(i)
⌘
where y(i)j denotes
the judgment on document d(i)j with respect to query q(i).
The judgment y(i)j represents the relevance degree of d(i)
j toq(i), and can be a score explicitly or implicitly given by hu-mans. For example, y(i)
j can be number of clicks on d(i)j
when d(i)j is retrieved and returned for query q(i) at a search
engine (Joachims, 2002). The assumption is that the higherclick-on rate is observed for d(i)
j and q(i) the stronger rele-vance exists between them.
A feature vector x(i)j = (q(i), d(i)
j ) is created fromeach query-document pair (q(i), d(i)
j ), i = 1, 2, · · · ,m; j =
1, 2, · · · , n(i). Each list of features x(i) =⇣
x(i)1 , · · · , x
(i)n(i)
⌘
and the corresponding list of scores y(i) =⇣
y(i)1 , · · · , y
(i)n(i)
⌘
then form an ‘instance’. The training set can be denoted asT =
n
(x(i), y(i))om
i=1.
We then create a ranking function f ; for each feature vec-tor x(i)
j (corresponding to document d(i)j ) it outputs a score
f (x(i)j ). For the list of feature vectors x(i) we obtain a list of
scores z(i) =⇣
f (x(i)1 ), · · · , f (x(i)
n(i) )⌘
. The objective of learn-ing is formalized as minimization of the total losses withrespect to the training data.
mX
i=1
L(y(i), z(i)) (1)
where L is a listwise loss function.
In ranking, when a new query q(i0) and its associated docu-ments d(i0) are given, we construct feature vectors x(i0) fromthem and use the trained ranking function to assign scoresto the documents d(i0). Finally we rank the documents d(i0)
in descending order of the scores. We call the learningproblem described above as the listwise approach to learn-ing to rank.
By contrast, in the pairwise approach, a new training dataset T 0 is created from T , in which each feature vector pairx(i)
j and x(i)k forms a new instance where j , k, and +1 is
assigned to the pair if y(i)j is larger than y(i)
k otherwise �1.It turns out that the training data T 0 is a data set of bi-nary classification. A classification model like SVM canbe created. As explained in Section 1, although the pair-
wise approach has advantages, it also su↵ers from draw-backs. The listwise approach can naturally deal with theproblems, which will be made clearer in Section 6.
4. Probability ModelsWe propose using probability models to calculate the list-wise loss function in Eq. (1). Specifically we map a list ofscores to a probability distribution using one of the prob-ability models described in this section and then take anymetric between two probability distributions as a loss func-tion . The two models are referred to as permutation prob-ability and top one probability.
4.1. Permutation Probability
Suppose that the set of objects to be ranked are identifiedwith the numbers 1, 2, ..., n . A permutation ⇡ on the objectsis defined as a bijection from {1, 2, ..., n} to itself. We writethe permutation as ⇡ = h⇡(1), ⇡(2), ..., ⇡(n)i. Here, ⇡( j) de-notes the object at position j in the permutation. The set ofall possible permutations of n objects is denoted as ⌦n.
Suppose that there is a ranking function which assignsscores to the n objects. We use s to denote the list of scoress = (s1, s2, ..., sn), where s j is the score of the j-th object.Hereafter we sometimes make interchangeable the rankingfunction and the list of scores given by the ranking func-tion.
We assume that there is uncertainty in the prediction ofranking lists (permutations) using the ranking function. Inother words, any permutation is assumed to be possible, butdi↵erent permutations may have di↵erent likelihood calcu-lated based on the ranking function. We define the per-mutation probability, so that it has desirable properties forrepresenting the likelihood of a permutation (ranking list),given the ranking function.
Definition 1 Suppose that ⇡ is a permutation on the n ob-jects, and �(.) is an increasing and strictly positive func-tion. Then, the probability of permutation ⇡ given the listof scores s is defined as
Ps(⇡) =nY
j=1
�(s⇡( j))Pn
k= j �(s⇡(k))
where s⇡( j) is the score of object at position j of permutation⇡.
Let us consider an example with three objects {1, 2, 3} hav-ing scores s = (s1, s2, s3). The probabilities of permuta-tions ⇡ = h1, 2, 3i and ⇡0 = h3, 2, 1i are calculated as fol-lows:
Ps(⇡) =�(s1)
�(s1) + �(s2) + �(s3)· �(s2)�(s2) + �(s3)
· �(s3)�(s3)
.
Permutation Probability
ì Objects : {A,B,C} and PermutaNons: ABC, ACB, BAC, BCA, CAB, CBA
ì Suppose Ranking funcNon that assigns scores to objects sA, sB and sC
ì Permuta5on Probabilty: Likelihood of a permutaNon
ì P(ABC) > P(CBA) if sA > sB > sC
Top One Probability
ì Objects : {A,B,C} and PermutaNons: ABC, ACB, BAC, BCA, CAB, CBA
ì Suppose Ranking funcNon that assigns scores to objects sA, sB and sC
ì Top one probability of an object represents the probability of its being ranked on the top, given the scores of all the objects
ì P(A) = P(ABC) + P(ACB)
ì NoNce that in order to calculate n top one probabiliNes, we sNll need to calculate n! permutaNon probabiliNes. ì P(A) = P(ABC) + P(ACB) ì P(B) = P(BAC) + P(BCA) ì P(C) = P(CBA) + P(CAB)
Listwise Loss Function
ì With the use of top one probability, given two lists of scores we can use any metric to represent the distance between two score lists.
ì For example when we use Cross Entropy as metric, the listwise loss funcNon becomes
ì Ground Truth: ABCD vs. Ranking Output: ACBD or ABDC
ListNet
ì Learning Method: ListNet
ì OpNmize Listwise Loss funcNon based on top one probability with Neural Network and Gradient Descent as opNmizaNon algorithm.
ì Linear Network Model is used for simplicity: y = wTx + b
ListNet
Ranking Accuracy
ì ListNet vs. RankNet, RankingSVM, RankBoost
ì 3 Datasets: TREC 2003, OHSUMED and CSearch ì TREC 2003: Relevance Judgments (Relevant and Irrelevant), 20 features
extracted ì OHSUMED: Relevance Judgments (Definitely Relevant, PosiNvely
Relevant and Irrelevant), 30 features ì CSearch: Relevance Judgments from 4(‘Perfect Match’) to 0 (‘Bad
Match’), 600 features
ì EvaluaNon Measures: Normalized Discounted CumulaNve Gain (NDCG) and Mean Average Precision(MAP)
Experiments
ì NDCG@n on TREC
Experiments
ì NDCG@n on OHSUMED
Experiments
ì NDCG@n on CSearch
Conclusion
ì Discussed ì Learning to Rank ì Pairwise approach and its drawbacks ì Listwise Approach outperforms the exisNng Pairwise Approaches
ì EvaluaNon of the Paper ì Linear Neural Network model is used. What about Non-‐Linear
model? ì Listwise Loss FuncNon is the key issue.(Probability models)
References
ì Zhe Cao, Tao Qin, Tie-‐Yan Liu, Ming-‐Feng Tsai, and Hang Li. 2007. Learning to rank: from pairwise approach to listwise approach. In Proceedings of the 24th interna(onal conference on Machine learning (ICML '07), Zoubin Ghahramani (Ed.). ACM, New York, NY, USA, 129-‐136. DOI=10.1145/1273496.1273513 hap://doi.acm.org/10.1145/1273496.1273513
ì Hang Li: A Short Introduc5on to Learning to Rank. IEICE TransacNons 94-‐D(10): 1854-‐1862 (2011)
ì Learning to Rank. Hang Li. Microsow Research Asia. ACL-‐IJCNLP 2009 Tutorial. Aug. 2, 2009. Singapore