Upload
data2b
View
204
Download
0
Embed Size (px)
Citation preview
72K JOBS PER DAYW E L C O M E T O C R I T E O F A C T O R YH U G R E N N E S – S E P T E M B E R 3 0 2 0 1 5
Copyright © 2015 Criteo
2About me
Nicolas Helleringer
DevOps by nature
Engineering Manager
Site Reliability Engineering
@nhelleringer
3
Criteo ?Performance Display
Advertisement
27 offices
in the worldMulti
channels
1 billion
Internet users
reach
1600
employees
A behavior driven
recommendation algorithm
is the core of the company Founded
by 3
engineers
Technology is a huge driver
for the R&D and also for all the
company
C# basis SQL Server
Best relational DB on the market
Replication
Ramp up
on proven
technologies
› Good efficiency
› Known scaling
Ease of use
Microsoft
stack
Quick deployment
Homogeneous data layers
Online, offline and BI
Scaled for quite a long time
Investments in infrastructure
6
Technical insightsOf Machines and Men
540 engineers
1/3 of the
company
15k servers
3B display a day
1 million http
requests/sec
Try, failLearn, Repeat
Try at scale !
Be prepared to dig
into details
Test your use case
not a random
benchmark
HDFS raw
39 PiB
51 PiB
2 big clusters
2x 15k Cores
Hadoop at scale
1200 servers
700 servers
4 local clusters
240/320 Cores
1 pre prod cluster
1800 Cores
1 dev cluster
1200 Cores
Slider ?
Resiliency
Compute power
vs
SLAs
All on the clusterRoles outside the
cluster are hard to
maintain
Thank you [email protected]