6
1 Web Scale Computing: Compete on Ideas, Not Resources Werner Vogels CTO – Amazon.com 2 Fast Forward to 33 minutes 3 4 5 What ifLaunching a new business on the web was simple? You only had to focus on the business”? You could manage growth more easily? 6 What ifLaunching a new business on the web was simple? You only had to focus on the business”? You could manage growth more easily? What if you only had to compete on ideas?

Werner Vogels @ FOWA Feb 07

Embed Size (px)

Citation preview

Page 1: Werner Vogels @ FOWA Feb 07

1

Web Scale Computing:Compete on Ideas, Not Resources

Werner Vogels

CTO – Amazon.com

2

Fast Forward to 33 minutes

3 4

5

What if…

• Launching a new business on the web was simple?

• You only had to focus on “the business”?

• You could manage growth more easily?

6

What if…

• Launching a new business on the web was simple?• You only had to focus on “the business”?• You could manage growth more easily?

What if you only had to compete on ideas?

Page 2: Werner Vogels @ FOWA Feb 07

2

7 8

PUSH VS PULL

• Demand is anticipated

• Top down design

• Centralized design

• Procedural

• Tightly coupled

• Resource centric

• Restricted participation

• Limited re-engineering

• Demand is uncertain

• Emergent design

• Decentralized

• Modular

• Loosely coupled

• People centric

• Open participation

• Rapid incremental innovation

9

Forces Driving Alternative Resource Models

• Increasing Uncertainty

• Growing Abundance

• Intensifying Competition

• Growing Power of Customers

• Greater Focus on Learning and Improvisation

10

DESIGN

DEPLOY

EXECUTE& MONITOR

REFINE

Push Model

11

FIND

CONNECT

INNOVATE

REFLECT

Pull Model

12

Resource Management in an Uncertain World

• Acquire resources on demand

• Release resources when no longer needed

• Pay for what you use

• Leverage other’s core competencies

Page 3: Werner Vogels @ FOWA Feb 07

3

13

A New Web Business using the Push Model

Your IdeaSuccessful

ProductUndifferentiated “Heavy Lifting”

Hardware CostsSoftware CostsMaintenance

Expertise

Load Balancing

Scaling and Physical Growth

Costs to run idle serversBandwidth ManagementServer Hosting

14

The 70 / 30 Switch

30 % of time, energy, and dollars on differentiated value creation

70 % of time, energy, and dollars on undifferentiated heavy lifting

15 16

How can you ever build an application that needs to

• scale to 100% - 1000% increase popularity?

• provide 4 nines uptime?

• survive a complete datacenter failure?

• survive a network partition

• While keeping cost low at the same time?

17

Chapter 4: Priorities – Scale Later – It is too hard to get right

18

The Reliability of Hard Disks

Failure Trends in a Large Disk Drive PopulationEduardo Pinheiro, Wolf-Dietrich Weber and Luiz André Barroso

Page 4: Werner Vogels @ FOWA Feb 07

4

19

Challenges to just get the infrastructure right

• Networks, hardware, operating systems, databases, middleware all fail all the time

• How to beat the CAP theorem

• Datacenters also fail (tornados, heat waves)

• To use scale as a cost-effective advantage you need really large numbers

• Operations is just plain hard, large & small

• Invest intellectual bandwidth to build the infrastructure

• Invest funds for something that may not happen (yet)

20

Resources in the Pull Model: Web-Scale Computing

Scalable Infrastructure that allow your applications to meet infinite demand, cheaply and reliably

• Turn huge fixed costs into variable

• Scale up and down as your business does

• Pay as you go

• Leverage other’s core competencies

• Focus on your Idea

21

Amazon S3, EC2 & SQS

Scalable – Increase or decrease capacity in minutes

Cost-Effective – Low rate, pay-as-you-go

Reliable – Runs on Amazon’s proven infrastructure

Simple – SOAP- and REST-based Web Service Calls

Compatible – Use Amazon EC2 and S3 together to receive free data transmission between services, decreased latency, and consistency.

22

Amazon Simple Storage Service (S3)

• Storage for the internet via web service calls

• Private and public storage options

• $.15 per GB/ per Month to store data

• $.20 per GB/ per Month to transfer data

• Use Cases: Unlimited Data Storage, Media Sharing/ Distribution, Archiving, Server Back-ups

23 24

020406080

100120140

2002 2003 2004 2005 2006

millions ofphotos

Smugmug’s growth

Page 5: Werner Vogels @ FOWA Feb 07

5

25

Smugmug & Amazon S3

• Saved $474,000 in first 7 months of operation

• Pre-S3:– Buying $40K/month of new hardware– Without S3 this would now be $80K/month

• Post-S3:– Selling excess hardware on eBay– Costs even lower than projected

• Expecting savings of $1M to $2M in 200726

27 28

29

S3 Explorer

filicio.us

Jungle Disk

S3 Firefox Organizer MyOwnDB

30

Amazon Elastic Compute Cloud (EC2)

Virtual Computing EnvironmentScalable Capacity – Pay as You Go

$.10 per Server Hour$.20 per GB of data transfer

Use cases: Load testing, Time or Traffic-based Scaling, Simulation and Analysis, Rendering, SaaSPlatform

Page 6: Werner Vogels @ FOWA Feb 07

6

31 32

33 34

Infrastructure:

Amazon Simple Queue Service

Amazon Simple Storage Service

Amazon Elastic Compute Cloud

E-Commerce:

Amazon E-Commerce Service

Amazon Historical Pricing

AWS Product Family

Web Search:

Alexa Web Search

Alexa Web Information Service

Alexa Top Sites

Alexa Site Thumbnail

Workforce / Workflow:

Amazon Mechanical Turk

35

http: / /aws.amazon.com