Amazon’s Cloud Amazon’s Cloud Computing Efforts: S3 and Computing Efforts: S3 and

EC2EC2By: Chris SosaBy: Chris Sosa

Materials Provided ByMaterials Provided By• ““An Evaluation of Amazon’s Grid An Evaluation of Amazon’s Grid

Computing Services: EC2, S3 and SQS” Computing Services: EC2, S3 and SQS” by Garfinkelby Garfinkel– When I refer to tests, I mean from this When I refer to tests, I mean from this

paperpaper• BlogsBlogs• Miscellaneous articlesMiscellaneous articles• Amazon’s Web Services websiteAmazon’s Web Services website

MotivationMotivation• Producing a large Producing a large

accessible computing accessible computing and storage and storage infrastructure is infrastructure is expensiveexpensive– Physical Space, cooling, Physical Space, cooling,

power, managementpower, management– MoneyMoney– ExperienceExperience

• Large maintenance costsLarge maintenance costs– Never sure how much we Never sure how much we

actually needactually need

Utility ComputingUtility Computing• Basically outsourcing infrastructure to the Basically outsourcing infrastructure to the

bigbig guysguys• It’s all about economy of scaleIt’s all about economy of scale• Goal to make web-scale computing easierGoal to make web-scale computing easier

… … how many buzz words does it take to fill a how many buzz words does it take to fill a gallon jar?gallon jar?

Amazon’s Cloud Computing Amazon’s Cloud Computing OverviewOverview

• Amazon’s Elastic Compute Amazon’s Elastic Compute Cloud (EC2)Cloud (EC2)– Provides resizable / on-Provides resizable / on-

demand computing capacitydemand computing capacity• Amazon’s Simple Storage Amazon’s Simple Storage

Service (S3)Service (S3)– Storage for the InternetStorage for the Internet– Provides web services Provides web services

interface to store and interface to store and retrieve any amount of data retrieve any amount of data at any time and from at any time and from anywhere (for a price)anywhere (for a price)

Amazon’s Web Service Amazon’s Web Service ModelModel• InterfaceInterface

– Web-based dashboardWeb-based dashboard– REST API – standard HTTP(/s) GET, PUT, DELETE REST API – standard HTTP(/s) GET, PUT, DELETE

commands commands – Remote procedure calls through SOAPRemote procedure calls through SOAP– Variety of tools that perform basic operationsVariety of tools that perform basic operations

• SecuritySecurity– Each request is signedEach request is signed

• 40-character secret key with HMAC 40-character secret key with HMAC • X.509 certificates either given to or created by AmazonX.509 certificates either given to or created by Amazon

– Signed timestamps to prevent replay attacksSigned timestamps to prevent replay attacks– Can use TLS / SSLCan use TLS / SSL

First Up – EC2First Up – EC2• Amazon uses Linux-based Virtual Machines Amazon uses Linux-based Virtual Machines

running on Xenrunning on Xen• Actual resources are virtualizedActual resources are virtualized

– So much so that they are not even backed by So much so that they are not even backed by real hardware real hardware

– ““Predictably provided”Predictably provided”• Two security mechanismsTwo security mechanisms

– Write ssh public key into the VM imageWrite ssh public key into the VM image– Firewall at the server sideFirewall at the server side

EC2 Cost ModelEC2 Cost Model• InstancesInstances

– $0.10 - Small Instance (per hour)$0.10 - Small Instance (per hour)• 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core - 1.7 GHz 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core - 1.7 GHz

processor), 160 GB of instance storage, 32-bit platform processor), 160 GB of instance storage, 32-bit platform – $0.40 - Large Instance (per hour)$0.40 - Large Instance (per hour)

• 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform Compute Units each), 850 GB of instance storage, 64-bit platform

– $0.80 - Extra Large Instance (per hour)$0.80 - Extra Large Instance (per hour)• 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2

Compute Units each), 1690 GB of instance storage, 64-bit platformCompute Units each), 1690 GB of instance storage, 64-bit platform• Data Transfer Data Transfer

– $0.10 per GB - all data transfer in $0.10 per GB - all data transfer in – $0.18 per GB - first 10 TB / month data transfer out$0.18 per GB - first 10 TB / month data transfer out– $0.16 per GB - next 40 TB / month data transfer out$0.16 per GB - next 40 TB / month data transfer out– $0.13 per GB - data transfer out / month over 50 TB$0.13 per GB - data transfer out / month over 50 TB

The GoodThe Good• Only pay for what you needOnly pay for what you need• ““Guaranteed” to be 99.9% reliable Guaranteed” to be 99.9% reliable

(good enough?) (targets 99.99%)(good enough?) (targets 99.99%)• Fast, responsive and reliableFast, responsive and reliable• Only takes a couple minutes to start a Only takes a couple minutes to start a

new instancenew instance• Data transfer free between S3 and EC2Data transfer free between S3 and EC2

The BadThe Bad• Still pay for computing you aren’t using but still runningStill pay for computing you aren’t using but still running• No persistent storageNo persistent storage

– Can use S3Can use S3– Writing to S3 slower and literally more expensiveWriting to S3 slower and literally more expensive

• Poor termination clausePoor termination clause– Only given 60 day noticeOnly given 60 day notice

Second – S3Second – S3• 250 Mb/second network link250 Mb/second network link• Objects stored in bucketsObjects stored in buckets

– Buckets have own namespaceBuckets have own namespace– Up to 100 buckets per account Up to 100 buckets per account – Unlimited objects per bucketUnlimited objects per bucket– 5 GB limit on size of objects5 GB limit on size of objects

• Computes MD5 checksum’s to check if transfers are Computes MD5 checksum’s to check if transfers are correctcorrect

• Up to application to retryUp to application to retry• ACL’s by accounts / certificates allowed per object ACL’s by accounts / certificates allowed per object

and per bucket (two-level)and per bucket (two-level)• Basic interface to read/write/create/delete/chmod etcBasic interface to read/write/create/delete/chmod etc

S3 PricingS3 Pricing• StorageStorage

– $0.15 per GB-Month of storage used $0.15 per GB-Month of storage used • Data TransferData Transfer

– $0.10 per GB - all data transfer in$0.10 per GB - all data transfer in– $0.18 per GB - first 10 TB / month data transfer out$0.18 per GB - first 10 TB / month data transfer out– $0.16 per GB - next 40 TB / month data transfer out$0.16 per GB - next 40 TB / month data transfer out– $0.13 per GB - data transfer out / month over 50 TB$0.13 per GB - data transfer out / month over 50 TB

• RequestsRequests– $0.01 per 1,000 PUT or LIST requests$0.01 per 1,000 PUT or LIST requests– $0.01 per 10,000 GET and all other requests*$0.01 per 10,000 GET and all other requests*

* No charge for delete requests * No charge for delete requests

The PrettyThe Pretty• Available during 100% of testsAvailable during 100% of tests• PerformancePerformance

– Works well for large file sizesWorks well for large file sizes– Approaches maximum of 30 Mb/s with Approaches maximum of 30 Mb/s with

multiple threads (6) for test in papermultiple threads (6) for test in paper– Performs at about 10 MB /s with one Performs at about 10 MB /s with one

thread doing one large data pushthread doing one large data push

The Pretty UglyThe Pretty Ugly• No guarantee of persistence in No guarantee of persistence in

datadata– Will real companies buy into this?Will real companies buy into this?

• Claimed throughput only Claimed throughput only achievable with data transfers achievable with data transfers of 16MB or largerof 16MB or larger– High transaction overheadHigh transaction overhead– Amazon has declined to explain Amazon has declined to explain

why!why!• Same Service Level Agreement Same Service Level Agreement

terms as EC2terms as EC2

ConclusionConclusion• EC2 and S3 provide applications with the EC2 and S3 provide applications with the

ability to allocate computing and storage ability to allocate computing and storage resources on demandresources on demand– This IS usefulThis IS useful– Cheaper than the alternative for reasonably sized Cheaper than the alternative for reasonably sized

companiescompanies• Most claims of service have been validatedMost claims of service have been validated

– Issues with bandwidth claims for S3Issues with bandwidth claims for S3• Issues with its Service Level Terms may plague Issues with its Service Level Terms may plague

EC2 and S3 from becoming more utilizedEC2 and S3 from becoming more utilized

QuestionsQuestions