End-to-End Evaluation of Amazon’s S3 and EC2

Simson L. Garfinkel

1

Amazon’s Web Services

Originally: E-Commerce, Alexa,

Summer 2006: Commodity Computing

S3 — Simple Storage Service

EC2 — Elastic Compute Cloud

SQS — Simple Queue Service

BusinessWeek: “Jeff Bezo’s Risky Bet”

2

Why use these services?

Backed by one of the world’s largest clusters.

Highly redundant, highly scalable.

Cheaper than building it yourself.

Isolates from the “slashdot effect.”

3

S3: “Unlimited” Storage

10 ¢/GB/month for storage

15 ¢/GB to upload or download

SOAP or “REST”

http://s3.amazonaws.com/bucket/object

http://bucket.s3.amazonaws.com/object

Object size 1-5GB (2GB); 4KB Metadata

4

EC2: “Unlimited” Compute

10 ¢/CPU-HOUR

Xen-based Virtual machine with:

1.7 Ghz x86 processor

1.75 GB RAM

160 GB hard drive

250 MB/s of network bandwidth

5

SQS: Easy Queues

10 ¢/10,000 messages

0-256K message size with metadata

Unlimited Queue Size

Reliable Delivery; Random Order

PUT/READ-LOCK/DELETE functionality

SOAP & REST API

6

Who is the customer for these services?

Amazon won’t say. “They’re for everybody!”

Large customers that need reliable off-site backup.

E-businesses that need scalable infrastructure.

7

My Analysis: Summary

The services work... most of the time.

The services scale well.

Key problem points:

Availability

Security model

Lack of an SLA

8

Experimental Setup

S3EC21, 1k, 1M, 16M, 100M

PUT & GET

Data Repository

25,066 probes between Nov. 3 and Jan 3rd.

9

Experimental Setup 2EC2

Data Repository

MIT

Harvard

LAISP

PITISP

MIT

Harvard

94,615 probes between Nov. 3 and Jan 3rd.

S3

FiOS

10

Average Performance (EC2)

Size GET PUT

1B 40 TPS 19 TPS

1M 1 MBS/sec 4 MB/sec

100M 9 MB/sec 6 MB/sec

11

Peak (top 1%) Performance (EC2)

Size GET PUT

1B 76 TPS 42 TPS

1M 1 MBS/sec 8 MB/sec

100M 19 MB/sec 14 MB/sec

12

GET performance: 21 MBytes/sec

PUT performance: 17 MBytes/sec

Success rate: <5%

Best performance: when routers failed

13

Performance from outside Amazon (KB/sec)

Host avg GET avg PUTEC2 1024 3760

Harvard 1 741 1360Harvard 2 638 847

MIT 1 726 1730MIT 2 796 2077

ISP PIT 599 1428ISP LA 930 933FiOS 548 199

14

Average daily read throughput (1MB GETs)

Oct

29

Nov

05

Nov

12

Nov

19

Nov

26

Dec

03

Dec

10

Dec

17

Dec

24

Dec

31

Jan

07 0 KB/s

200 KB/s

400 KB/s

600 KB/s

800 KB/s

1000 KB/s

1200 KB/s

1400 KB/sAverage daily read throughput (1MB GETs) from EC2

15

Average hourly throughput

0 5 10 15 20200 KB/s

400 KB/s

600 KB/s

800 KB/s

1000 KB/s

1200 KB/s

1400 KB/sAverage hourly throughput

16

GET Distribution

0 10 20 30 40 50 60 70 80 90Bytes/s

0

200

400

600

800

1000

N

Histogram 1B GET observed throughput (5579 samples)

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4Mbytes/s

0

100

200

300

400

500

600

700

N

Histogram 1M GET observed throughput (5564 samples)

0 5 10 15 20 25Mbytes/s

0

50

100

150

200

250

N

Histogram 16M GET observed throughput (4177 samples)

17

GET Distribution

0 10 20 30 40 50 60 70 80 90Bytes/s

0

200

400

600

800

1000

N

Histogram 1B GET observed throughput (5579 samples)

0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4Mbytes/s

0

100

200

300

400

500

600

700

N

Histogram 1M GET observed throughput (5564 samples)

3

0 5 10 15 20 25Mbytes/s

0

50

100

150

200

250

N

Histogram 16M GET observed throughput (4177 samples)

17

Security Issues 1: Authentication

18

Security Issues 1: Authentication

Per-Object & Per-Bucket access control lists

18

Security Issues 1: Authentication

Per-Object & Per-Bucket access control lists

Authentication with HMAC & Secret Key

18

Security Issues 1: Authentication

Per-Object & Per-Bucket access control lists

Authentication with HMAC & Secret Key

Secret Key downloaded from AWS website

18

Security Issues 1: Authentication

Per-Object & Per-Bucket access control lists

Authentication with HMAC & Secret Key

Secret Key downloaded from AWS website

Password reset with email challenge/response

18

Security Issues 1: Authentication

Per-Object & Per-Bucket access control lists

Authentication with HMAC & Secret Key

Secret Key downloaded from AWS website

Password reset with email challenge/response

Control of email = control of all S3 data

18

Security Issues 2: availability

S3 has experienced repeated service interruptions.

Amazon’s long-term commitment to the service is unknown.

19

Security Issues 3: Amazon Web Services License Agreement

Allows termination at any time for any reason.

Prohibits content that is “obscene, libelous, defamatory or otherwise malicious or harmful to any person or entity”

Prohibits service “in any way that is otherwise illegal or promotes illegal activities, including without limitation in any manner that might be discriminatory based on race, sex, religion, nationality, disability , sexual orientation or age.”

20