Multimodal Social Book Search

Zürcher Fachhochschule

Melanie Imhof, Ismail Badache, Mohand Boughanem

Université de Neuchâtel, Neuchâtel, Switzerland

Zurich University of Applied Sciences, Winterthur, Switzerland

IRIT - Paul Sabatier University, Toulouse, France


Motivation

• Only the first few “recommendations” are considered

• Many modalities

• Goals

– Fuse textual baseline with non-textual and social

modalities

• Ratings, number of tags, book price and number of pages

– Include user preferences

2


Retrieval Models

3

Learning to Rank

Textual Models

Social Signals-Based Model


Textual Models

• Single text field that contains all textual fields

• Query expansion

– Blind relevance feedback (RF)

– Example books with positive and neutral sentiment

• Filter books already read by the topic creator (user

catalog & examples)

4



• Social prior probability

• 𝑃 𝐷 = 𝑎𝑖 ∈𝐴log 1+ 𝐷𝑎𝑖 + 𝜇 ∙𝑃 𝑎𝑖 𝐶

log 1+ 𝐷𝑎 + 𝜇

• 𝐷𝑎𝑖 is the number of actions of type 𝑎𝑖, e.g. number of tags.

• 𝐷𝑎 is the number of all actions on document.

• 𝑃 𝑎𝑖 𝐶 probability of 𝑎𝑖 in the collection

• 𝜇 smoothing parameter 5

More popular higher probability to be relevant

Assumption



• Social prior probability for ratings

• 𝑃𝐵𝐴 𝐷 =1+log(1+𝐵𝐴 𝐷 )

1+log(1+ 𝐷′∈ 𝐶

𝐵𝐴(𝐷′))

• 𝐵𝐴 𝐷 =𝑎𝑣𝑔 𝐷𝑟 + 𝐷𝑟 + 𝐷′∈ 𝐶 𝑎𝑣𝑔 𝐷𝑟

′ ∙|𝐷𝑟′|

𝐷𝑟 + 𝐷′∈ 𝐶 |𝐷𝑟′|

6

More and higher ratings higher probability to be relevant

Assumption


Learning to Rank (Random Forests)

• Learn how to combine textual and non-textual

modalities into a single ranked list

– Price

– Number of pages

– Ratings

• User preference

– Estimated by the average values in the user’s catalog

– Use difference of document value to user preference

7

~ 190 pages~30 €


Experimental Evaluation: Runs I

• Textual Model

– Run1: Textual baseline using BM25 with example based

relevance feedback using 35 terms and read book filtering.

• Social Signal-Based Models

– Run3: Run1 using language model combined with

Bayesian average re-ranking based on ratings.

– Run4: Run1 using language model combined with re-

ranking based on the tags.

– Run5: Run1 combined with re-ranking based on the tags

and Bayesian average of ratings.8


Experimental Evaluation: Runs II

• Random Forests

– Run2: Random forests trained with 10 trees based

on six textual runs and three non-textual

modalities

– Run6: Random forests trained with 100 trees

based on six textual runs and three non-textual

modalities combined with re-ranking based on the

tags and Bayesian average of ratings

9


Results

• Training exceeds no training

• Non-textual modalities contain relevant information

• Examples RF and filtering improve textual baseline

• Social signal prior improves textual baseline

10


Conclusion

• Superiority of social approach compared to textual

model (baseline).

• Test learning approach with completely separated

training and test datasets.

• Find methods that do not rely on learning (cold start

problem).

11

Internet

Multimodal Social Book Search