Upload
shauna-jennings
View
222
Download
0
Tags:
Embed Size (px)
Citation preview
Improving web image search results using query-relative classifiers
Josip Krapacy Moray Allanyy Jakob Verbeeky Fr´ed´eric Jurieyy
OutlineIntroductionQuery-relative featuresExperimental evaluationConclusion
OutlineIntroductionQuery-relative featuresExperimental evaluationConclusion
IntroductionGoogle’s image search engine
have a precision of only 39%[16]Recently research improve image
search performance by visual information and not only text
Similar outlier detection, current setting the majority of retrieved image may be outliers, and inliers can be diverse
IntroductionRecently methods have the same
drawback : ◦a separate image re-ranking model is
learned for each and every query – large number of possible queries make these approach wasted computational time
IntroductionKey contribution :
◦Propose an image re-ranking method, based on textual and visual feature
◦Does not require learning a separate model for every query
◦The model parameters are shared across queries and learned once
IntroductionImage re-ranking approach :
Our image re-ranking approach :
OutlineIntroductionQuery-relative featuresExperimental evaluationConclusion
Query-relative featuresQuery-relative text feature
◦Binary features◦Contextual features
Visual feature
Query-relative visual feature
Query-relative text featureOur base query-relative text
feature follow [6,16]◦ContexR◦Context10◦Filedir◦Filename◦Imagealt◦Imagetitle◦Websitetitle
Binary featureNine binary features indicate the
presence or absence of query terms :◦Surrounding text◦Image’s alternative text◦Web page’s title◦Image file’s URL’s hostname, directory
and filename◦Web page’s hostname, directory and
filenameWhich is active if some of the query
terms, but not all, are present in the field
Contextual featuresCan be understood as a form of
pseudo-relevance feedback
Divide the image’s text annotation in three parts :◦Text surrounding the image◦Image’s alternative text◦Words in the web page’s title
Contextual featuresDefine contextual features by
computing word histograms using all the image in the query set
Histogram of word counts : Image : iWord indexed : k
Contextual featuresUse (1) to define a set of
additional context featuresThe kth binary feature represents
the presence or absence of kth most common word
We trim these features down to the first N element, so we have 9+9+3N binary feature
Visual featuresOur image representation is based
on local appearance and position histograms
Local appearance◦Hierarchical k-means clustering ◦11-levels of quantisation, and k = 2
Position quantisation ◦Quad-tree with three level
The image is represented by appearance-position histogram
Query-relative visual featuresNo direct correspondence
between query terms and image appearance
We can find which visual words are strongly associated with query set by contextual text features
Define a set of visual features to represent their presence or absence in a given image
Query-relative visual featuresOrder the visual features :
◦A : query set◦T : training set◦ : average visual word
histogram
The kth feature relates to the visual word kth most related to this query
Query-relative visual featuresWe compared three ways of
representing each visual word’s presence or absence◦The visual word’s normalised count
for this image ◦The ratio ◦Binary version of this ratio, threshold
at 1:
OutlineIntroductionQuery-relative featuresExperimental evaluationConclusion
Experimental evaluationNew data setModel trainingEvaluationRanking images by textual
featuresRanking images by visual
featuresCombining textual and visual
featuresPerformance on Fergus data set
New data setPrevious data set contain image
for only a few classes, and at most case without their corresponding meta-data
In our data set, we provide the top-ranked images with their associated meta-data
Our data set of 353 image search queries and in total there are 71478 images
Model trainingTrain a binary logistic
discriminant classifierQuery-relative features of
relevant images are used as positive examples
Query-relative features of irrelevant images are used as negative examples
Rank images for the query by the probability
Only need to be learnt once
Evaluation Used mean average precisionLow Precision(LP): 25 queries where
the search engine performs worstHigh Precision(HP): 25 queries where
the search engine performs bestSearch Engine Poor(SEP): 25 queries
where the search engine least over random ordering of query set
Search Engine Good(SEG): 25 queries where the search engine most over random ordering of query set
Ranking images by textual features
Diminishing gain per additional feature
Ranking images by visual features
Ranking images by visual features
Adding more visual features increases the overall performance, but with diminishing gain
Combining textual and visual features
◦a = visual features, 50~400◦b = additional context features,
20~100
10%
Performance on Fergus data set
Our method better than Google[4],[7] perform better, but they
require time-consuming training for every new query
Results
OutlineIntroductionQuery-relative featuresExperimental evaluationConclusion
ConclusionConstruct query-relative features
that can be used to train generic classifiers
Rank images for previously unseen search queries without additional model training
The feature combined textual and visual information
Presence a new public data set
Thank you!!! &Happy New Year!!!!