Big data summit

Proprietary amp Confidential Copyright copy 2014

Hadoop Operations Rocket Fuel

Wersquore Hiringrocketfuelcomcareers

Kishore Kumar Yellamraju

The Web Is Monetized By Advertising

Delivery Methods

raquoDisplayraquoVideoraquoMobileraquoSocial

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Some Exchange Partners

Ad Exchange

Optimize

Rocket Fuel Platform

Real-time BidderAutomated Decisions

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

5 RocketfuelWinning Ad

$238965$06782$17234

$009$178964$16782$17234$0809$242125

$211$126

$2178$2056$0809$242125

$211$126$278$156

$1809$242125

$211$126$278$056$242125

$211$126$278

$0756$0809$242125

$211$126$278

$1256$1809$242125

$211$126$278

$0586$2009

125$211$126$278$156

[ + ][ + ]

SitePageGeoWeatherTime of DayBrand AffinityUser

Always buying the best impressions amp serving the best ad

Real Time Bidding and Serving

GoalLeadsamp sales

GoalCoupondownloads

GoalBrandawareness

SitePageGeoWeatherTime of DayBrand AffinityDemo

Impression ScorecardDemoBrand AffinityTime of DayGeoWeatherSitePageAd PositionIn-marketBehaviorResponse

Impression ScorecardDemoBrand AffinityTime of DayGeoWeatherSitePageAd PositionIn-MarketBehaviorResponse X

Impression ScorecardDemoBrand AffinityTime of DayGeoWeatherSitePageAd PositionIn-MarketBehaviorResponse

+100+40-20+20+15+10+40+35

+40-70-20+10+15-25-40-18

+10-10-20+20+10-35-25+10

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

Facebook likes

Searches on Google

Bid Requests Considered by Rocketfuel

Requests per day

Throughput

Blink of an eye

SF to Tokyo network round trip

One beat of a hummindbirds wing

Look up in Blackbird

Time (ms)

Latency

Architecture and Scale

raquoDatacentersraquoScaleraquoGrowthraquoArchitecture

Data Center Expansion

raquoabc

Data Center Design

bull Racks custom built at Rocket Fuelbull Leased spacebandwidth in colocation facilities

Hadoop Server24 2U servers (85kW)

Bidders40 2-U Twin 2 servers (17kW)

Rocket Fuel Scale

raquo34474 CPU processor coresndash2655 serversndash1874 Teraflops of computing

raquo188 Terabytes of memoryndash13X the memory of IBM computer Watson that

played Jeopardy

raquo42PB Petabytes of storagendash106X the data volume of the entire Library of

Congress

Hadoop at Rocket Fuel

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

raquo 12K daily MR jobs

200 Servers 1400 Servers

1 Year

41 PB8x

Growth

Data Architecture 30

Batch and Real Time Pipelines

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

raquo 6x2TB Disksraquo 2x6 coreraquo 196 GB RAMraquo 2x1G NIC

raquo 12x3TB Disksraquo 2x6 coreraquo 64 GB RAMraquo 10G NIC

raquo same as DNrsquosraquo Dedicated disk

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

raquo Maintenanceraquo Performance Tuningraquo Monitoringraquo BCPraquo YARN

Puppet+

Infradb

Automation is key

Maintenance is Not Easy

Puppet and Infradb

raquo Automate as much as you canraquo Adding a slave node to Hadoop cluster lt 120 secondsraquo Bringing up a new Hadoop cluster lt 500 secondsraquo MR slots are automatically determined based on hardware config

Isnrsquot it cool

Just define once

No issues when cluster is small Problems starts when it grows

Performance Tuning

dfsdatanodehandlercount dfsnamenodehandlercount

dfsdatanodemaxtransferthreads dfsimagetransfertimeout

mapredreduceparallelcopies

mapredjobtrackerhandlercount

iosortmbiosortfactor

maxClientCnxns ZK

IMP MAPREDUCE-2026

-XX+UseConcMarkSweepGC

-XXCMSFullGCsBeforeCompaction=1

-XXCMSInitiatingOccupancyFraction=60

ha-timeoutms

Performance Tuning

mapreducereduceshuffleparallelcopies

Operations

Monitoring

Wall of Ops

Monitoring

hadoopnamenodeCallQueueLength hadoopjobtrackerjvmmemheapusedm

Donrsquot fly blind you will crash

MR Workload Monitoring

Network Monitoring

Donrsquot blame network instead monitor it Network Mesh can be mess

Alerting

Monitoring is not enough need better Alerting

Alerts

httphostnameportjmx

qry=Hadoopservice=NameNodename=NameNodeInfo

gtgt Checking whether NN and JT are up is a no brainer gtgt Reduce alert noise by having summaryaggregate alerts gtgt We heavily rely on custom scripts that query jmx for NN and JT

qry=hadoopservice=JobTrackername=JobTrackerInfo

NameDirStatuses DeadNodes NumberOfMissingBlocks

qry=Hadoopservice=NameNodename=FSNamesystemState

FSState CapacityRemaining NumDeadDataNodes UnderReplicatedBlocks

Blacklisted TTrsquos jobs slots_used ThreadCount

qry=javalangtype=Memory

Used jvm free jvm etc

MR Workload Alerting

raquo Monitoring MR workload and alertndash In-house tool that use ldquohoudahrdquo ruby gem monitorsndash Long running jobs jobs with more map tasks blacklisted TTrsquos

with more failure counts etchellip

raquo Collect details and auto-restart blacklisted TTrsquosraquo Parse the JT logfile for rouge jobsraquo Parse the JT log and collects all Job related inforaquo White-elephant or hraven could helpraquo Parse the scheduler html page or use metrics page httpltJT-hostnamegt50030scheduleradvanced httpltJT-hostnamegt50030metrics

Modeling

Ad-hoc

Multi Tenancy

raquo create separate Queuesraquo Enable ACLrsquos for queuesraquo limit no of jobs per user and per queueraquo set pre-emption timeouts based on priorityraquo set weight based on priority

No Scheduler is perfect unless you understand and tune it properly

Scheduling

Operations

raquo BCP Business Continuity Planraquo Near real time reporting over 15+ TB of daily dataraquo Freshness of models trained over petabytes of data

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

USEUHK Serving Clusters

Research

Queries

Processed Data

JobTracker

raquo Resource Manager - Global resource scheduler - Hierarchical queues - Application management

raquo Node Manager - Per-machine agent - Manages life cycle of container - Container resource monitoring

raquo Application Master - Per-application - Manages application scheduling and task execution

YARN at Rocket FueI

raquo Yarn in production raquo 1000+ nodesraquo 51TB RAM 123K disks 123K cores raquo Primary use case Map-Reduceraquo HBase on Yarnraquo Tez Spark Storm are in race

We Are Hiring

THANKS

kishorerocketfuelcom

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Performance Tuning (2)

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Delivery Methods

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

$238965$06782$17234

$009$178964$16782$17234$0809$242125

$211$126

$2178$2056$0809$242125

$211$126$278$156

$1809$242125

$211$126$278$056$242125

$211$126$278

$0756$0809$242125

$211$126$278

$1256$1809$242125

$211$126$278

$0586$2009

125$211$126$278$156

[ + ][ + ]

GoalLeadsamp sales

GoalCoupondownloads

GoalBrandawareness

+100+40-20+20+15+10+40+35

+40-70-20+10+15-25-40-18

+10-10-20+20+10-35-25+10

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

Facebook likes

Searches on Google

Requests per day

Throughput

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Delivery Methods

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

$238965$06782$17234

$009$178964$16782$17234$0809$242125

$211$126

$2178$2056$0809$242125

$211$126$278$156

$1809$242125

$211$126$278$056$242125

$211$126$278

$0756$0809$242125

$211$126$278

$1256$1809$242125

$211$126$278

$0586$2009

125$211$126$278$156

[ + ][ + ]

GoalLeadsamp sales

GoalCoupondownloads

GoalBrandawareness

+100+40-20+20+15+10+40+35

+40-70-20+10+15-25-40-18

+10-10-20+20+10-35-25+10

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

Facebook likes

Searches on Google

Requests per day

Throughput

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

$238965$06782$17234

$009$178964$16782$17234$0809$242125

$211$126

$2178$2056$0809$242125

$211$126$278$156

$1809$242125

$211$126$278$056$242125

$211$126$278

$0756$0809$242125

$211$126$278

$1256$1809$242125

$211$126$278

$0586$2009

125$211$126$278$156

[ + ][ + ]

GoalLeadsamp sales

GoalCoupondownloads

GoalBrandawareness

+100+40-20+20+15+10+40+35

+40-70-20+10+15-25-40-18

+10-10-20+20+10-35-25+10

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

Facebook likes

Searches on Google

Requests per day

Throughput

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

$238965$06782$17234

$009$178964$16782$17234$0809$242125

$211$126

$2178$2056$0809$242125

$211$126$278$156

$1809$242125

$211$126$278$056$242125

$211$126$278

$0756$0809$242125

$211$126$278

$1256$1809$242125

$211$126$278

$0586$2009

125$211$126$278$156

[ + ][ + ]

GoalLeadsamp sales

GoalCoupondownloads

GoalBrandawareness

+100+40-20+20+15+10+40+35

+40-70-20+10+15-25-40-18

+10-10-20+20+10-35-25+10

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

Facebook likes

Searches on Google

Requests per day

Throughput

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

GoalLeadsamp sales

GoalCoupondownloads

GoalBrandawareness

+100+40-20+20+15+10+40+35

+40-70-20+10+15-25-40-18

+10-10-20+20+10-35-25+10

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

Facebook likes

Searches on Google

Requests per day

Throughput

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

6 Ad Served

User Segments

3 Bid Reques

Overview

Publishers

2 Ad Request

1 Page Request

4 Bid amp Ad

User Engagemen

Data Partners

Advertisers

Browser

Ad Exchange

Optimize

Models

Refresh learning

Data Store

Ads ampBudget

ModelScores

Events

Facebook likes

Searches on Google

Requests per day

Throughput

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Facebook likes

Searches on Google

Requests per day

Throughput

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Blink of an eye

Time (ms)

Latency

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

raquoabc

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Data Center Design

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Rocket Fuel Scale

played Jeopardy

Congress

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

raquo 1400 servers

raquo 15K Disks

raquo 15K Cores

raquo 90 TB

raquo 30K MR slots

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

1 Year

41 PB8x

Growth

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Webservers

scribe

Zookeeper

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Hadoop Setup

QJM ZK Quorum

to ZK or JN

Standby NN

ZKFCZKFC

Active NN

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Operations

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Puppet+

Infradb

Automation is key

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Puppet and Infradb

Isnrsquot it cool

Just define once

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Performance Tuning

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

maxClientCnxns ZK

IMP MAPREDUCE-2026

ha-timeoutms

Performance Tuning

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Operations

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Monitoring

Wall of Ops

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Monitoring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Network Monitoring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Alerting

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Alerts

httphostnameportjmx

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Modeling

Ad-hoc

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Multi Tenancy

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Scheduling

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Operations

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Data BCP Cluster

INW Data Cluster

US Serving Clusters

EU Serving Clusters

HK Serving Clusters

Modeling

Reporting

User Quer

Amazon Backup

LSV Data Clust

Research

Queries

Processed Data

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

JobTracker

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

THANKS

Delivery Methods

Overview

Overview (2)

Throughput

Latency

Data Center Design

Rocket Fuel Scale

Growth

Hadoop Setup

Operations

Puppet and Infradb

Performance Tuning

Operations (2)

Monitoring

Monitoring (2)

Network Monitoring

Alerting

Alerts

Multi Tenancy

Multi Tenancy (2)

Scheduling

Operations (3)

Data BCP Cluster

YARN at Rocket FueI

We Are Hiring

Engineering

Big data summit