P ris, France 16 & 17 October, 2014
Copyright © Institut Lean France 2014
Lean IT Summit ®®
Enabling Lean IT with AWS
Carlos Conde – @caarlco
« The most radical and transformative of
inventions are often those that empower
others to unleash their creativity – to pursue
their dreams. »
Jeff Bezos
1. EXPERIMENT & INNOVATE
2. MEASURE EVERYTHING
3. EMBRACE FAILURE
4. ITERATE & MOVE FAST
5. FOCUS ON YOUR BUSINESS
1. EXPERIMENT & INNOVATE
2. MEASURE EVERYTHING
3. EMBRACE FAILURE
4. ITERATE & MOVE FAST
5. FOCUS ON YOUR BUSINESS
TO INCREASE INNOVATION
LOWER THE COST OF FAILURE
FAIL FAST
FAIL LOW-COST
+ ELASTIC CAPACITY
+ NO CAPEX
+ PAY AS YOU GO
+ AVAILABLE ON-DEMAND
= NO CONSTRAINTS
CUSTOMER TEST ENVIRONMENTS SYSTEM INTEGRATORS
& CONSULTANCY FIRMS
WWW.123DAPP.COM
GPU INSTANCES
G2
CG1
1x NVIDIA Kepler GK104
8 vCPU (Intel Xeon E5-2670)
2x NVIDIA Fermi M2050
16 vCPU (Intel Xeon X5570)
$0.65/h
$2.10/h
3 HOURS FOR $4828.85/hr
…INSTEAD OF
$20+ MILLIONS IN INFRASTRUCTURE
1. EXPERIMENT & INNOVATE
2. MEASURE EVERYTHING
3. EMBRACE FAILURE
4. ITERATE & MOVE FAST
5. FOCUS ON YOUR BUSINESS
IF YOU DON’T MEASURE IT, IT DIDN’T HAPPEN
COST AS A TECHNICAL METRIC
1. EXPERIMENT & INNOVATE
2. MEASURE EVERYTHING
3. EMBRACE FAILURE
4. ITERATE & MOVE FAST
5. FOCUS ON YOUR BUSINESS
FAIL FAST
FAIL LOW-COST
SUCCESS MAKE PEOPLE FEEL GOOD
FAILURES MAKES PEOPLE BETTER
GOOD WEATHER DOESN’T MAKE GOOD SAILORS
GAME DAYS SIMULATE CRISIS SITUATIONS IN A
CONTROLED ENVIRONMENT
SUPERVISORS & PLAYERS
MAKE A PLAN ROLE PLAY & SCOPE DEFINITION
PLANNING KEY TEAM MEMBERS “FAILURE”
PRE-MORTEM
SET A DATE (DEADLINE)
MAKE IT FUN
CREATE THE SIMULATION ENVIRONMENT
GAMEDAY ENVIRONMENT
AWS ACCOUNT A
PRODUCTION ENVIRONMENT
AWS ACCOUNT B
CLOUDFORMATION
TEMPLATE
SIMULATE ACTIVITY
PRE-RECORDED NETWORK ACTIVITY
(THROTTLED) LOAD TEST USE SPOT INSTANCES + TOOLS
USING AMAZON EC2 TO SIMULATE
2.4 MILLION PLAYERS
COMMUNICATION CHANNELS IRC, SHARED DOCS, VIBER, …
SIMULATE FAILURES
TERMINATE RESOURCES CHANGE SECURITY GROUPS CHANGE IAM ROLES DISABLE IAM USER CHANGE /ETC/HOSTS FILE AMAZON RDS FAIL-OVER TEST
PLAY ! COMPARED TO REALITY, YOU’RE NOT
SADISTIC ENOUGH
ACTIVATE AWS CLOUDTRAIL GAMEPLAY RECORDING & AUDITING
DEBRIEF WHERE THE MAGIC HAPPENS
HOW LONG DOES IT TAKE TO DETECT AN EVENT ?
COMMUNICATION CHANEL & CHAIN OF COMMAND
VALIDATE ASSUMPTIONS PROVE YOUR ARCHITECTURE KNOW YOUR PROCEDURES
1. EXPERIMENT & INNOVATE
2. MEASURE EVERYTHING
3. EMBRACE FAILURE
4. ITERATE & MOVE FAST
5. FOCUS ON YOUR BUSINESS
11.6s
Mean time between
deployments
(weekday)
1,079
Max number of
deployments in a
single hour
10,000
Mean # of hosts
simultaneously
receiving a
deployment
30,000
Max # of hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT
AMAZON.COM
DEPLOY TO PROD EVERY 20 MINUTES
2 PIZZA TEAMS
INSTAGRAM “2 backend engineers can scale a
system to 30+ million users”
ANY GIVEN NIGHT 300,000 PEOPLE ARE HOSTED
1000s OF SERVERS, TBs OF STORAGE
1. EXPERIMENT & INNOVATE
2. MEASURE EVERYTHING
3. EMBRACE FAILURE
4. ITERATE & MOVE FAST
5. FOCUS ON YOUR BUSINESS
MAKE A TO-DON’T LIST
REMOVE WASTE Waste is anything that does not lead to
direct value for the customer
Your
Business
70%
ON-PREMISE
INFRASTRUCTURE
30%
Managing All of the
“Undifferentiated Heavy Lifting”
AWS
CLOUD-BASED
INFRASTRUCTURE
Your
Business
More time and resources
to focus on your business
Configuring
cloud assets
70%
30% 70%
ON-PREMISE
INFRASTRUCTURE
30%
Managing All of the
“Undifferentiated Heavy Lifting”
1. EXPERIMENT & INNOVATE
2. MEASURE EVERYTHING
3. EMBRACE FAILURE
4. ITERATE & MOVE FAST
5. FOCUS ON YOUR BUSINESS
« Don’t be worried about our competitors because
they’re never going to send us any money anyway.
Let’s be worried about our customers and stay
heads-down focused. »
– Jeff Bezos
@caarlco
EXTRA
Dropcam is the biggest inbound video service on the Web
• More data uploaded per minute than YouTube
• Petabytes of data processed every month
• Billions of motion events detected