Booking.com
Recommend Now, Not in the Past
Lucas Bernardi, Melanie Mueller Data scientists @ Booking.com
Leveraging Contextual User Profilesfor Destination Recommendations
Booking.com
Introduction: Recommending travel destinations
Part I: Ranking destinations
Part II: Contextual recommendations
Conclusions
Outline
Booking.com
Travel agents
Booking.com
Online travel agents
Booking.com
Destination Finder
Booking.com
Destination Finder
Booking.com
Destination Finder
Booking.com
Which destinations to recommend?
Destination recommendations
1) Match activities
2) Recommend best matching destinations
Booking.com
• 5 million reviews
Endorsement data
• 256 activities mined from reviews (LDA)
• Ask users to ‘endorse’ a destination after their stay,e.g. ‘Beaches’, ‘Temples’
Endorsement #given #destinations Shopping 876,726 11,708 Food 525,111 20,538 Beach 505,192 11,422 … … … Mythology 1,065 406 Heliskiing 354 165
Booking.com
Endorsement data
Booking.com
Endorsement data• Standard recommender: user gives rating for item
• Here: multi-criteria, negative opinions are hidden
Booking.com
Ranking for ‘beach’
• Naive Bayes
P(Miami | beach) = (# beach endorsements for Miami)
(# beach endorsements)
• Keep it simple!
Booking.com
Destination Finder
Booking.com
Evaluation
• Naive Bayes
• Random
• Popularity
How to test?
Booking.com
A/B testing• Version A • Version B • Version C
Booking.com
A/B testing
Ranker #users Random 10079 Popularity 9838 Naive Bayes 9895
• Which metric?
User engagement → clicks
Booking.com
A/B testing
Ranker #users #clickers conversion g-test Random 10079 2465 24.46% Popularity 9838 2509 25.50% 90.8% Naive Bayes 9895 2645 26.73% 99.9%
• Which metric?
User engagement → clicks
Booking.com
Introduction: Recommending travel destinations
Part I: Ranking destinations
Part II: Contextual recommendations
Conclusions
Outline
Booking.com
History is history
Traditional recommending systems use past user ratings to predict unknown ratings
User History is short: User Cold start problem. Users have different personas: History becomes less
relevant User Interests are volatile: History becomes less relevant
Continuous User Cold start
Booking.com
Context
DefinitionSet of features that inform about the current situation of the user.
ExampleLocation, device, weather conditions, season, day of the week,
hour of the day.
HypothesisUsers in similar contexts have similar interests.
Booking.com
Context
Traditional Collaborative Filtering RecommendationsU x I → R
Destination FinderU x I x C → <r0, r1, r2, …, rn>
Booking.com
Framework
Booking.com
Framework
Booking.com
Discovering contextual profiles
Data points: Reviews Features:
Endorsements Contextual features All features one-hot-encoded
Example:<Ubuntu, Firefox, Tuesday, beach, temples><0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0...,0,1,0...0>
Booking.com
Discovering contextual profiles
k-means Clustering Clean up clusters Final output
Q binary n-dimensional centroids Q Contextual Profiles
Booking.com
Discovering contextual profiles
Contextual Profiles
i - 1 i i + 1 i + 1
… iPhone.OS.7.Chrome Windows.Phone …
iPhone.OS.5.Chrome Windows.Vista
iPhone.OS.6.Chrome Friday
Android.2.2 Sunday
Android.2.2.Tablet
Android.3.1.Tablet
Android.4.0.Tablet
Android.4.4.Tablet
Android.2.1.Tablet
Android.3.0.Tablet
Android.4.1
Android.4.3.Tablet
Booking.com
Applying contextual profiles
Booking.com
Applying contextual profiles
Contextual Profiles are simply a binary vector Find the most similar Contextual Profile for each review Train a ranker for each Contextual Profile
Booking.com
Computing recommendations
Booking.com
Computing recommendations
For a given user, compute a feature vector using contextual features
Contextualize: Find closest Contextual Profile Compute recommendations using the ranker trained on the
selected Contextual Profile
Booking.com
Framework
Booking.com
Results
Ranker Users Conversion CTR
Baseline 13,306 21.7 ± 0.7% 18.5 ± 0.4%
Contextual 13,562 21.3 ± 0.7% 22.2 ± 0.4%
Improved CTR by 22.5%
Booking.com
Conclusions• Multi-criteria destinations Recommender System• Simple rankers increase user engagement• Context matters
•Improves user engagement•Attacks Continuous cold start problem
• Reusable Contextual Profiles• On-line evaluation
Booking.com
Julia Kiseleva, PhD student at Eindhoven University,Research intern at Booking.com Nov 204 - Mar 2015
Booking.com: Chad Davis, Ivan KovacekMats Stafseng Einarsen
Academia: Djoerd Hiemstra, Jaap KampsMykola Pechenizkiy, Alexander Tuzhilin
Thanks
Booking.com
We're hiring