Embed Size (px)
Citation preview
©2017,AmazonWebServices,Inc.oritsAffiliates.Allrightsreserved.
HeatherMatson,SeniorAccountExecutiveEducation
RandyRidgley,SolutionsArchitect
February2017
CostOptimizationatScale
WhattoexpectWewillintroduceourframeworkforoptimizingyourAWSbillbyfocusingonpayingonlyforwhatyouneed.
Sharelessonslearnedbasedonexamplesfromsomeofourmostsuccessfulcustomers.
Andleaveyouwithactionableideas totakebacktotheofficetoimplementimmediately.
ChapterI:BuildWithCostInMind(WellArchitected)
BestPracticeFrameworkforArchitectingonAWS
PerformanceEnsureasystemdeliversmaximumperformanceforasetofresources.
CostOptimizationAchievethelowestpriceforaworkloadtakingintoaccountfluctuatingneeds.
ReliabilityEnsuringagivensystemisarchitectedto
meetoperationalthresholdsduringaspecificperiodoftime.
SecurityReviewdefinitionsandcompliancebest
practices.
HowdocustomerslowertheirTCO withAWS?
1
Source: IDC Whitepaper, sponsored by Amazon, “The Business Value of Amazon Web Services Accelerates
Over Time.” December 2013
“Average of 400 servers replaced per
customer”
Replaceup-frontcapitalexpensewithlower“payforwhatyouuse”variablecost
model
3
59 Price Reductions since 2006
EconomiesofscaleallowAWSto
continuallylowercosts
2
Pricingmodelchoicetosupportvariable&stable
workloads
On-DemandReserved
SpotDedicated
LoweringTCOthroughcloudoptimization
LiftandShift Optimized fortheCloud Designedforthecloud
Micro-services&Serverless
ManagedServicesInfrastructureServices
TCO:upto30%* TCO:upto58%* TCO:upto75%*
*https://aws.amazon.com/economics/learn-more/
Ex:AmazonEC2,AmazonEBS,AmazonEFS,AmazonVPC,VMImport/export…
Ex:AmazonRDS,AmazonAurora,AmazonElasticache,AmazonRedshift,AmazonOpsWorks,AmazonElasticSearch,AmazonEMR
Ex:AmazonS3,AmazonSQS,AWSLambda,AmazonKinesis,AmazonDynamoDB,AmazonCognito,AWSDMS,AWSWAF,AWSIOT
Where do you start?
TheFivePillarsofCostOptimization
Right-sizingyourinstances
Picktherightpricingmodel
Increaseelasticity
Measuring&monitoring
Matchusagetostorageclass
Pillar1:Right-SizingRight-sizing• Selectingthecheapestinstanceavailablewhile
meetingperformancerequirements• LookingatCPU,RAM,storage,andnetwork
utilizationtoidentifypotentialinstancesthatcanbedownsized
• LeveragingAmazonCloudWatchmetricsandsettingupcustomRAMmetrics
Ruleofthumb:Rightsize,thenreserve.(Butifyou’reinapinch,reservefirst.)
Pillar2:IncreaseElasticityTurnoffnonproductioninstances• Lookfordev/test,nonproductioninstancesthatare
runningalways-onandturnthemoff.
• Lambda+CloudWatch =AutomatedScheduling*
Autoscaleproduction• UseAutoScalingtoscaleupanddownbasedon
demandandusage(forexample,spikes).
*https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/
Pillar3:LeveragingtheRightPricingModel
On-DemandInstancesReservedInstancesSpotInstances
On-DemandInstances
Short-term, spiky and unpredictable
Low cost and flexible
Develop and test
ReservedInstancesforAlways-On Instances
Commitmentlevel• 1year• 3year
AWSservicesofferingReservedInstances• AmazonEC2• AmazonRDS• AmazonDynamoDB• AmazonRedshift• AmazonElastiCache
*DependentonspecificAWSservice,size/type,andregion
EC2InstanceGrowth
2007 2008 201120102009 20162013 2014 2015
3
60+
IncreasingcustomerchoicethroughacceleratingInstanceinnovation
InstancesLaunchedin2016:
• AcceleratedComputing:p2• MemoryOptimized:x1.16xl• Generalpurpose:m4.16xl
57
11 12
23
42
52
…ConvertYourReservedInstancesWithaConvertible ReservedInstance,youcanmodify yourexistingreservationacross:
Instancefamilies
Instancesizes
Operatingsystems
Tenancy
ConvertibleReservedInstancedetails
Commitmentlevel• 3year
AWSservicesofferingReservedInstances• AmazonEC2
*DependentonspecificAWSservice,size/type,andregion
ReservedInstancesStep1:ReservedInstanceCoverage• Coveralways-onresourceswithstandardor
convertibleReservedInstances
Step2:IncreaseReservedInstanceUtilization• Knownarchitectures:LeverageStandardReserved
Instanceflexibilitytoincreaseutilization.• Growingorchangingarchitectures: Leverage
ConvertibleReservedInstancesacrossfamilies,sizes,andOS.
• RegionalBenefit: Consolidatedbilling,reservationnotcritical
SpotInstances
Userswithurgentcomputingneedsorlargeamountsofadditionalcapacity
Timeorinstanceflexible
Experimentand/orbuildcost-sensitivebusinesses
ConsiderSpot for Elastic Workloads
Options• SpotFleettomaintaininstanceavailability
• SpotBlockdurations(1-6hours)forworkloadsthatmustruncontinuously
Commitmentlevel• None
*ComparedtoOnDemandpricebasedonspecificEC2instancetype,region,andAvailabilityZone
SpotRules
50%ofOD
75%ofOD
25%ofOD
You pay the market price87% discount!
You’llNeverPayMoreThanYourBid
WhichEC2purchasingmodelisrightforme?
StrikeaBalance
Findingbalancebetweenpricingoptions
ReservedInstances
On-Demand Spot
“Butmyapplicationsaretoosmall
forAutoScaling!”
AmazonEC2Container Service
• EasilymanageDockercontainers• Flexiblecontainer placement• DesignedforusewithotherAWSservices• Extensible• PerformanceatScale• Secure
10%
15%
7%
12%
20%
9%
ConsolidatewithAmazonECS
App1 App2
App3 App4
App5 App6
6
12 345
AmazonECScluster
GetridofidletimewithAWSLambda• Automaticscaling• Automaticprovisioning• Noneedtomanageinfrastructure• Justbringyourcode• $0.20permillionrequests,1Mfree• 100mspaymentgranularity• Neverpayforidle
Lessthan40%utilization?ConsiderusingAWSLambdainstead!
Usecase:Serverlesswebapps
• AmazonS3forservingstaticcontent• AWSLambdafordynamiccontent• AmazonAPIGatewayforhttpsaccess• AmazonDynamoDBforNoSQLdatastorage
DynamiccontentinAWSLambda
DatastoredinAmazonDynamoDBAPIGatewayStaticcontentin
AmazonS3 Browser
Pillar4:LeveragingtheRightStorageClass
BlockStorageObjectStorage
AWSImport/ExportSnowball• AcceleratePBswithAWS-provided
appliances• 80TBmodel,globalavailability
AWSStorageGateway• Instanthybridcloud• Upto120MB/sclouduploadrate
(4ximprovement),and
DataingestionintoAWSstorageservices
AmazonKinesisFirehose• Ingestdevicestreamsdirectlyinto
AWSdatastores
AWSDirectConnect• COLOtoAWS
ISVConnectors• CommVault• Veritas• etcetera
AmazonS3TransferAcceleration• Movedataupto300%faster
usingAWS’sprivatenetwork
AWSImport/ExportSnowball• AcceleratePBswithAWS-provided
appliances• 80TBmodel,globalavailability
AWSStorageGateway• Instanthybridcloud• Upto120MB/sclouduploadrate
(4ximprovement),and
DataingestionintoAWSstorageservices
AmazonKinesisFirehose• Ingestdevicestreamsdirectlyinto
AWSdatastores
AWSDirectConnect• COLOtoAWS
ISVConnectors• CommVault• Veritas• etcetera
AmazonS3TransferAcceleration• Movedataupto300%faster
usingAWS’sprivatenetwork
Butareyouoptimizing yourobjectstorage?
Activedata ArchivedataInfrequentlyaccesseddata
Milliseconds MultipleRetrievalRatesMilliseconds
$0.023/GB/mo* $0.004/GB/mo$0.0125/GB/mo
ObjectStorageClassesonAmazonS3
*First50TB,Next450TB$0.022/GB,Over500TB$0.021/GB
Standard
Hot
Standard- InfrequentAccess
Warm
AmazonGlacier
Cold
RunningtheNumbers:S3orS3-IAComparing1PBofobjectstorage*
1PBMonthly
S3 S3-IA Savings%
$24,117 $14,116 41%
Ruleofthumb:Breakeven=105% RetrievedperMonth
ContentAccessedperMonth
10%
$24,117 $18,350 24%50%1PBMonthly
$24,117 $23,593 2%100%1PBMonthly
*BasedonUS-EastPrices
CaseStudy:PuttingitTogether
ATechnologyCompany
•Inthreemonths…
ATechnologyCompany
•Doubled CPUandtrafficusedbyitsWebservers
ATechnologyCompany
•WhilereducingEC2spendby33%
$72ksavingpermonth!
Howdidtheydoit?
Solvingthegrowthchallenge
Step1:Right-sizeandupdateinstancesm1ondemand$0.07perECU
c4ondemand$0.02perECU
Theimpactofright-sizing
70%reductioninunitcost
Step2:Reserve
Theimpactofreservations
30%reductionInunitcost
Puttingittogether
85%reductioninunitcost!
Soundsprettyeasy,right?Notreally.
Inreality,itisverycomplex.• Scale• Behavioralchange• Visibility• Ownership
ChapterII:Implement&Maintain(WellOperated)
Howdoyouimplementthosebestpracticesatscale?(SpoilerAlert:Rememberthe5th PillarofCostOptimization?)
Pillar5:Measure&Monitor
Automation.
1. Identify always on instances.2. Identify instances to downsize.3. Identify warm / cold storage.4. Recommend Reserved
Instances to purchase.5. Dashboard our status.6. Report on savings.
EnablingCostTransparency
EnablingCostTransparencyMust have tagging:• Cost center• Application or workload • User• Expiration date• Automation support
ToolsforMeasuring&Monitoring
Option1:DIYBuildyourowncustomdashboard
We’vehelpedcustomerssetuptheirowndashboardsandtools.
Metrictrackingsavings
Spend&metricsbyaccount
EC2usagebypurchaseoption
Option2:UseaPartner
Andpartnershavebeensuccessfulhelpingcustomersmonitor&managecost
Option3:AWSToolsTrustedAdvisor
AWSTrustedAdvisorHelping customers automate best practices (checks) across cost optimization, security, fault tolerance, and performance improvement
Red (action recommended)Orange (investigation recommended)Green (no problem detected)
IdleTestInstancesRunninginus-west-1
AutomatingTrustedAdvisorwithAWSLambda
AWSTrustedAdvisor
AWSLambda
ActionsonAWSresources
AmazonCloudWatchevents
Notifications
http://docs.aws.amazon.com/awssupport/latest/user/cloudwatch-events-ta.html
AWSBillingalerts
AWSBillingconsole
Leveragemanagedservices
• UseAmazonRDS,DynamoDB,ElastiCacheforRedisorAmazonRedshift
• Insteadof runningyourowndatabase
• AmazonEMR,Elasticsearch Service• Insteadof runningyourowncluster
• AmazonSQS• AmazonKinesis,
AmazonKinesisFirehose,Analytics,AmazonSNS,andmore…
AWS has experts for each service
RDSAmazonRedshift
AmazonElasticsearch
Service
AmazonKinesisSQS
EstablishClearTargets&Metrics
MetricsandTargets
• %instancesturnedoffdaily• %ofinstancesright-sized• %always-onresourcescoveredbyReservedInstances
• %ReservedInstanceutilization
✔✔✔✔✘
✘
✘
✘
Definesuccessbyestablishingmetricsandtrackprogress
Cost-Based
Cycleofcostoptimization✔✔✔✔✘
✘
✘
✘
$$$$$
