Big Data Trends: Cloud Cost Benchmark Study

DAVID FEINLEIBTHE BIG DATA GROUP, LLC

BIG DATA TRENDS:PUBLIC CLOUD COST BENCHMARK STUDY

250 COMPANIES250,000 INSTANCES2.5PB OF STORAGE

THIRD QUARTER 2012

SPONSORED BY

thebigdatagroup.com Copyright © 2012 The Big Data Group, LLC bigdatalandscape.com

http://www.cloudyn.com

http://www.cloudyn.com

THE BIG DATA GROUP 2

Web Browser Market Share Study Goals

• Benchmark industry public cloud costs and utilization

• Enable buyers to improve operational efficiency based on cloud utilization best practices

• Leverage previously unavailable sources of Big Data to provide market insights

THE BIG DATA GROUP


Web Browser Market Share Study Data

• 250 companies

• 250,000 Amazon EC2 instances

• 7,543,955 instance hours

• 2.5PB of storage

• Measured Oct 1 - 31, 2012

THE BIG DATA GROUP


Web Browser Market Share

Public cloud buyers can achieve

significant savings by optimizing usage

Key Findings

• 33% of instances are running as reserved instances but some 94% should be

• 17% of storage is unattached (unused)

• Elastic Cloud compute resources account for more spend (at 62% of total spend) than any other resource type

THE BIG DATA GROUP

Many instances are underutilized. Significant storage goes unattached. Many buyers are reluctant to purchase reserved instances, which can provide cost savings when used for at least three months.


Web Browser Market Share Instance Definitions

THE BIG DATA GROUP

Type Instances

Small t1.micro, m1.small, c1.medium, m1.medium

Medium m1.large, m2.xlarge

Large m1.xlarge, c1.xlarge, m2.2xlarge, m2.4xlarge

Huge cc1.4xlarge, cc2.8xlarge, hi1,4xlarge, cg1.4xlarge

For the purposes of this study, instances were grouped into Small, Medium, Large, and Huge instance types. The instances included in each type are shown below.



Small instances account for about 42% of launched instances, medium for about 40%, large for just over 17%, and huge instances for under 1%.

The majority of launched instances

are small and mid sized.

Launched Instances By Type

0.44%17%

40%

42%

Small Medium Large Huge


Web Browser Market ShareSmall instances account for 53% of instance runtime hours while medium instances account for 29%.

Instance Runtime By Type

Huge1%Large

18%

Medium29%

Small53%


Web Browser Market ShareSmall instances account for 53% of instance runtime hours compared with 42% of launched instances. Medium instances account for 29% of instance runtime hours compared with 40% of launched instances.

Smaller instances account for more

hours run compared with the number that

are launched

Launched And Runtime Instances

Huge0.64%Large

18%

Medium29%

Small53%

Runtime InstancesThird Quarter 2012

Huge0.44%

Large17%

Medium40%

Small42%

Launched InstancesThird Quarter 2012


Web Browser Market ShareMedium instances are only about 12% utilized while small instances are just under 17% utilized. The low utilization rates of medium instances combined with medium instances accounting for some 40% of launched instances means that most customers use medium instances when they should use small instances instead.

Most customers use medium instances when they should

use small instances

Average Instance Utilization Rates

Small Medium Large Huge

3.9%

12.8%11.9%

16.7%


Web Browser Market ShareAmazon Web Services customers in the sample group spend more on EC2 instances than on any other type of resource.

EC2 Instances account for the

majority of spend

Top Cloud Spend Categories

S3 Estimated26%

RDS Databases3%

EBS Volumes7%

EC2 Instances64%

Note: S3 Data is based on a smaller set of randomly selected customers


Web Browser Market ShareSome 44% of customers operate 1 - 10 instances per month while 45% operate 11 - 100 instances per month.

The majority of public cloud customers

operate between 1 and 100 instances

per month

Deployment Size Breakdown

1%10%

45%

44%

1-10 instances/mo11-100 instances/mo101-500 instances/mo501+ instances/mo


Web Browser Market ShareOnly 33% of instances are reserved today, while some 94% of instances, if converted to reserved instances, would result in expected cost savings.

Reserved and Unreserved InstancesThird Quarter 2012

Reserved And Unreserved Instances

Unreserved67%

Reserved33%

Actual Recommended

94%

33%

Reserved Instances - Actual and RecommendedThird Quarter 2012

Most customers underutilize reserved instances, losing out

on significant cost savings



Many reasons buyers cite for not

purchasing reserved instances prove

financially unsound

Top Reasons Customers Avoid Reserved Instances

• Expect project to run only for a short time

• Undecided about proper instance size

• Lack of time to purchase

• Fear of commitment to reserved instances

THE BIG DATA GROUP

Many buyers are reluctant to purchase reserved instances, which provide significant cost savings when used for between three and seven months. Projects typically last twice as long as buyers expect and buyers tend to stay with the same instance size for at least half a year, making the purchase of reserved instances worthwhile.


Web Browser Market Share Emerging Trends

• Amazon RDS has not gained widespread adoption, likely due to the difficulty of moving to and from RDS databases and the inability to sync only specific database tables

• Amazon Glacier, which is 10X cheaper than Simple Storage Service (S3) but significantly slower, is expected to replace a lot of S3 storage for digital archiving

• The Reserved Instance Marketplace will need changes to gain widespread adoption

The marketplace lacks significant volume today because it is relatively new and because a company needs to have a US bank account to use it. In the future, without changes, we do not expect the marketplace to gain significant volume because prices tend to go down over the reservation term, making the resale of reserved instances at originally committed prices uncompetitive with lower prices at the time of resale

THE BIG DATA GROUP


Web Browser Market ShareCustomers can

achieve significant cost savings through

proper public cloud management

Recommendations For Buyers

• Evaluate

• Applicability of reserved instances

• Any unattached storage

• Instance sizes relative to utilization

• Consider

• Smaller instances

• Reserved instances

• Retiring unattached storage

• Ensure total response time stays within acceptable range (1.5 - 2.5 seconds)

THE BIG DATA GROUP


Web Browser Market Share For More Information

David [email protected]

THE BIG DATA GROUP

mailto:[email protected]

mailto:[email protected]

Technology

Big Data Trends: Cloud Cost Benchmark Study