Hadoop’s Impact on Recruit Company
Recruit Technologies Co., Ltd
Big Data product development group
Group manager
Nobuyuki Ishikawa
2Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Business Models of Recruit
Recruit operates various matching business in Japan.
Matching Business
HR
Bridal
GroupBuying
UsedCars
TravelReal
Estate
Beauty Gourmet
Social Games
E-Commerce
Ad Network
New Business
Consumers Enterprise
3Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Business Domain of Recruit
Provide information services that support “choices”
Life event area Lifestyle Area
Travel
IT/ TrendLifestyle
Health & Beauty
Job Hunt
Marriage
Job Change
Home Purchase
Car Purchase
Child Birth
Education
4Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Company Structure of Recruit
4
HD Recruit Career
Recruit Sumai Company
Recruit Lifestyle
Recruit Jobs
Recruit Marketing Partners
Recruit Technologies
Recruit Administration
Recruit Staffing
Staff service Holdings
Recruit Communications
Media & Solution
SBU
Global temporary employment
SBU
Global Online HR SBU
Administration Division
SBU management department
Infrastructure/Security
Project Management
UXD/SEOInternet
Marketing
Big Data Solutions
Technology R&D
Systems Development
Provide
IT solutions.
5Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
History of Hadoop in Recruit
2011
Start using Hadoop
2013Next‐generation
environmental construction
6Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
History of Hadoop in Recruit
2014
Expansion of Hadoop Eco-system’s use
2015
Further data utility promotion
7Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Hadoop utility circumstance in Recruit
Server Database
Release server:174Development server:41
1915 TB
8Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
We use Hadoop !!
We use Hadoop in various ways.
Matching Logic Optimization Search Logic Optimization
Cross customer referrals Content personalization
9Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
2016年6月7日 日経産業新聞
Our Client provides a service that make recommendation of applicants to Employers,based on both the data of talents who have passed document screening round, and job seeker’s browsing history.
This matching logic creation is stored by Hadoop.
Optimize Matching Logic
10Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Articles Personalized
Zexy delivers to customers appropriate marriage-related articles personalized among integrated information published on media.
20 million people, everyday, 15 articlesPersonalized on Hadoop
Cold startReflect user’s phases
Adjust priority of unread articles
Adjust priority of new articles
Customer’s preference
11Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Optimize Search Logic
Solr Hadoop HBase
SegmentBatch Processing
User Segment
User SegmentAPI
User Segment
Web
API Search
SegmentScore
3times/Day run
SolrFeed Batch
SolrIndex
Data fetchingData fetching
API RequestSearch Query
Search Action
Based on data that have been submitted in the past, area, station, occupation, feature information, calculate script potential score in the logistic regression.
12Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Cross Use referrals
GenesisAPI
Hadoop
HBase
JSP or JavaScript
Activity Log
Monitoring API
Activity Log (Accumulate
d)
DWH(Exadata)Hadoop Cluster
Business Data
Business Data
Referral Data
Batch
Log
accumulation
Batch
Business Data
Display API
Referral
API
Referral Data
Log API
Mall API(Item information
acquisition)
PointBuild foundation for intersite
cross use promotion in HDP
■ Each business by the API development can be introduced referral frame at low cost.
■ By using HDP, all businesses of Recruit achieve at high speed and low cost.
13Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
System use has expanded since Hadoop ecosystem
2010 2011-2014 2015 2016
14Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Unstructured data can also be used...
HDFS
・store any data regardless of native format
・perform schema design after storing data
is considered to be promoted in unstructured data use.
15Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Types of Data
transaction master view
text image location operating data
Apply History
Purchase History
Reserve History
etc
Company master
Area master
Customer attribute
etc
Browse History
etc
Review data
Script data
Job offer
etc
Shop image
Item image
etc
Indoor Location
GPS
etc
Sale activity history
Examine, review
Voice data
etc
16Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Use-range expanded
Customer Client
Attracting customers Select Action Effect
Informationpublished
Targetclient
Matching
cross tabulation
Recommendation
Image search
Advertising expenses optimization
Ad Targeting
Manuscript reviewer
Auto review
Article creation support
Sales support
Competitive analysis
17Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Diversified data analysis
✓In addition to recommendation and report, there are more and more new genre data analysis solutions of "human work alternative"
Profit contribution
Indicator/Purpose
● CVR maximization
● CPA optimization
Indicator/Purpose
● Optimization
● Next year’s strategy making
Indicator/Purpose
● Reduction of waste
● Reduction of man hours
● Having people be more creative
Cost reduction
Recommendation Report Work alternative(AI domain)
18Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
New Data Utility
Recommendation
Banner targeting
OCR Image Analysis
Manuscript Suggestion
Text proofreading
Text summarization
Text Classification
Voice-to-text
Indoor location positioning
Developing a variety of data analysis solutions API called A3RT in Recruit Group
19Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
3 sessions
Case study of DevOps for Hadoop in Recruit.26th October, 2016, 4:10PM - 4:50PM
Room: Yamato Nishi (3rd Floor)
A3RT - the details and actual use cases of "Analytics & Artificial intelligence API via Recruit technologies".
26th October, 2016, 3:00PM - 3:40PM
Room: Katsura & Kasuga
Struggle against cross domain data complexity in Recruit group27th October, 2016, 11:10AM - 11:50AM
Room: Katsura & Kasuga
20Copyright © Recruit Technologies Co., Ltd. All Rights Reserved.
Context represented by Hadoop
✓Store data
✓Store and use unstructured data
✓Provide various data and operation methods to data analyst
✓Combine and trigger data analysis and planning
Hadoop is not only a system, it can
Hadoop made culture, and plays as key technology for development of data analysis.