Embed Size (px)
DESCRIPTION
The talk I gave at the Big Data Innovation Summit.
Citation preview
Recruiting Solutions Recruiting Solutions Recruiting Solutions
Recommending Jobs You May Be Interested In
Anuj Goyal Recommendations at LinkedIn
Anuj
§ About LinkedIn § Search vs. Recommendations § Recommendation Opportunities § Evaluating Job Recommendations § Job Recommendation Algorithm § Challenges § Summary
2
Overview
270+ M
Company Pages
>3M *
Professional searches in 2012
~5.7B
90% Fortune 100 Companies use LinkedIn to hire
*
*as of March 31, 2014
New Members joining
~2/sec
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
3
World’s largest professional network Over 65% of members are now international
4
Recommendation Versus Search
Explicit Implicit
The Recommendations Opportunity
5
6
50% 7
Value of Recommendations
Job Recommendations
8
9
Jobs You May be Interested In
Evaluation
§ Upside metrics – Are users getting relevant jobs?
§ Downside metrics – Are users getting offending jobs?
10
Evaluation - Upside
§ Total Job Views (Clicks) § Total Applications § Total Viewers § Total Applicants
11
Evaluation - Downside
§ Applications per Click § Clicks per Impression § Applications per Impression § Expert Judgments
12
User Features & Recommendation Algorithm
14
Positions Education
Summary
Experience
Skills
User Features
Corpus Stats
Candidate Jobs
User Base
title geo company
industry description functional area
…
Candidate
General expertise specialties education headline geo experience
Current Position title summary tenure length industry functional area …
Similarity (candidate expertise, job description)
0.56 Similarity
(candidate specialties, job description)
0.2 Transition probability
(candidate industry, job industry)
0.43
Title Similarity
0.8
Similarity (headline, title)
0.7 . . .
derived
Matching Binary Exact matches: geo, industry, … Soft transition probabilities, similarity, … Text
Recommendation Algorithm
Transition probabilities Connectivity yrs of experience to reach title education needed for this title …
15
Job Collection
Challenges & Feature Engineering
Challenge: Entity Resolution
17
‘IBM’ has 13000+ variations - ibm – ireland - ibm research - T J Watson Labs - International Bus. Machines
Are All Companies The Same?
18
- Software Engineer - Technical Yahoo - Member Technical Staff - Software Development Engineer - SDE
Are All Titles The Same?
19
Same company with different name
Same name but different companies
Name Variations for IBM? “Orion” refers to 20 diff. companies
large scale: 100M+ members, 2M+ company entities
IBM: Intl Brotherhood of Magicians ~ 13000
Challenges – Entity Resolution
20
§ Binary classifier (LR), not ranker § P({position, company
entity} is a match) § Features
§ Content § Social § Behavior
§ Company candidate set leveraged from Social graph and cosine similarity 97% Precision
at 50% Coverage Asonam’11, KDD’11
Challenges – Entity Resolution
21
Prec
isio
n Coverage
Challenges – Geo Location
22
§ Zip code mapped to Regions § How sticky are those locations?
Feature Engineering – Sticky locations
23
§ Open to relocation ? § Region similarity based on profiles or network § Region transition probability
§ Predict individuals propensity to migrate and most likely migration target
Feature Engineering – Sticky locations
24
Feature Engineering – The Network effect
25
Hybrid Recommendation Title : Research Engineer Company : Yahoo! Location : CA,USA Skills : Stats, ML, Java
Title : Data Scientist Company : Samsung Location : PA,USA Skills : Stats, R
Title : Analyst Company : Microsoft Location : CA, USA Skills : R, ML
Title : Research Engineer <1>, Data Scientist <1>, Analyst <1> Company : Yahoo<1>, Samsung<1>, Microsoft<1>
Location : CA,USA <2>, PA,USA<1> Skills : Stats<2>, ML<2>, R<2>, Java<1>
Applicant Features
Distribution
Data Scientist / Senior Data Scientist San Jose
26
Information Gain
Pick Top K overrepresented features from the applicants distribution
A representative projection of the job in the member feature space
27
Hybrid Recommendation
§ Why Jobs Recommendations are Different § Recommendation Algorithm § Challenges
– Entity Resolution – Location Resolution
28
Summary
