Auto Scaling x Spot Instances によるスケーラビリティと コストカット
Preview:
Citation preview
- 1. Auto Scaling x Spot Instances 2015-06-03 @akitsukada ()
- 2. @akitsukada AWS Lambda #iOS #AWS #Rails #MySQL #Redis CAC()
(CTO) About me ()
- 3. (^q^)
- 4. Purpose of today 1. Auto ScalingMin-Max Spot&Reserved
Instances 2.
- 5. Purpose of today 1. Auto ScalingSpot Instances : [AWS] &
http://www.slideshare.net/AmazonWebServicesJapan/20131023-
aws-meisterregeneraterispotpublic 2. () 3.
- 6. 1. / 2. () 3. 4. 5. 6. 1. / 2. () 3. 4. 5. 6. Agenda
- 7.
- 8. AutoScaling, Spot Instances, Reseved Instances 1. 2. (
3.
- 9. AutoScaling, Spot Instances, Reseved Instances 1. 2. ( 3. 1.
4.5 2. 4.5 3. 1
- 10. ()
- 11. () ()
- 12.
- 13. Twitter(BtoC) =100 =3050% =110%/ (313) : /
- 14. Ruby on Rails (Apache+Passenger) HTTP(S) Worker m3.2xlarge
8@1a,1c Rails+Resque**, Rails+Resque+ResqueScheduler***
WebAPIResqueSchedulerJob c4.2xlarge 10@1a,1c WebAPI on EC2 Worker
on EC2* / *Worker2014/08, AWS Startup Tech Meetup
http://j.mp/20140817_aws_tsukada **ResqueGitHub Redis
***ResqueSchedulercronResqueJobKick
- 15. WorkerWebAPI AWS cloud Amazon RDS Amazon SES ElastiCache
(Redis for Resque) Internet users Elastic LoadBalancing users
Availability Zone instances Availability Zone instances /
- 16. WorkerWebAPI AWS cloud Amazon RDS Amazon SES ElastiCache
(Redis for Resque) Internet users Elastic LoadBalancing users
Availability Zone instances Availability Zone instances / 1.AB
@mention 2.B
- 17. WebAPI AWS cloud Amazon RDS Amazon SES ElastiCache (Redis
for Resque) Internet Elastic LoadBalancing Availability Zone
instances Availability Zone instances User B User A Worker 1 Job
mention / 1.AB @mention 2.B
- 18. WebAPI AWS cloud Amazon RDS Amazon SES ElastiCache (Redis
for Resque) Internet Elastic LoadBalancing Availability Zone
instances Availability Zone instances User B User A Worker 1.AB
@mention 2.B 1 Job /
- 19. / WebAPI Sum Requests (ELB)
- 20. / (24,000req/min) (600req/min) WebAPI Sum Requests (ELB) /
(40)
- 21. / (40) () (600req/min) (10,000req/min) WebAPI Sum Requests
(ELB) / (24,000req/min)
- 22. / (40) () WebAPI Sum Requests (ELB) / (600req/min)
(10,000req/min) (24,000req/min) 8 ! 1 8 7 6 5 4 3 2 1
- 23.
- 24.
- 25. Auto Scaling EC2 (^o^) $9,195113/ *201505EC2
- 26. needed
- 27. ()
- 28. Auto Scaling Spot Instances JobAuto Scaling Spot
Instances() SpotAuto Scaling Group SpotReserved1 WorkerWebAPI Spot
Instances Reserved InstancesAuto Scaling m3.2xlarge
(8vCPU/26ECU/30GiB) m3.large x 4 c4.2xlarge (8vCPU/31ECU/15GiB)
c4.large x 4 (Push)Scheduled Action ()
- 29. AutoScaling Spot Instances WorkerWebAPI Spot Instances
Reserved InstancesAutoScaling m3.2xlarge (8vCPU/26ECU/30GiB)
m3.large x 4 c4.2xlarge (8vCPU/31ECU/15GiB) c4.large x 4
(Push)Scheduled Action () JobAutoScaling Spot Instances() SpotAuto
Scaling Group SpotReserved1
- 30. /Reserved Spot Instance SpotSpot Auto Scaling Group Spot
Instance 1 Reserved (Max20) SpotStuckJob Job ... Spot()
- 31. /Reserved Spot Instance SpotSpot Spot Instance 1 Reserved
(Max20) SpotStuckJob Job ... Spot() 1
- 32. WorkerWebAPI () ELB ELB* Spot /Reserved ELBHealth Check
*http://bit.ly/1KxDZBx - ELB[Qiita] (25) AMI chef** threshold Spot
Request*** **http://amzn.to/1Eniqlm - AutoScale
***http://aws.amazon.com/jp/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2
- 33. Auto Scaling Reserved Instances
- 34. Auto Scaling/WebAPI
- 35. Name: WEB-API-ASG ELB: web-api-balancer Min: 4 Max: 36 Auto
Scaling Group Name: WEB-API-LC-v1 AMI: ami-XXXXXXX Instance Type:
m3.large Launch Configuration Scale In Alarm Comparison Operator:
LessThanThreshold Evaluation Periods: 3 Period: 300
Statistic:Maximum Metric Name:CPUUtilization Threshold: 30
Unit:Percent Cloud Watch Alarm Scale Out Alarm Comparison Operator:
GreaterThanThreshold Evaluation Periods: 1 Period: 300
Statistic:Maximum Metric Name:CPUUtilization Threshold: 65
Unit:Percent Adjustment Type: ChangeInCapacity Scaling Adjustment:
2 Cooldown: 300 Scaling Policy Health Check Grace Period: 300
Availability Zones: ap-northeast-1a, ap-northeast-1c /Auto
Scaling/WebAPI WebAPI
- 36. Name:event-push-start Min: 22 Max: 36 Reccurence: 45 1 * *
1 Name:event-push-stop Min: 4 Max: 36 Reccurence: 30 2 * * 1
Scheduled Actions # 10:45(JST) # Min/Max(yamlDRY) # 11:30(JST)
config yaml API config WebAPI /Auto Scaling/WebAPI
- 37. Auto Scaling WebAPI Sum Requests (ELB) /Auto Scaling/WebAPI
Req
- 38. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts
(ELB) /Auto Scaling/WebAPI
- 39. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts
(ELB) Req /Auto Scaling/WebAPI
- 40. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts
(ELB) Req Scheduled- Action /Auto Scaling/WebAPI
- 41. Auto Scaling WebAPI Sum Requests (ELB) WebAPI Healthy Hosts
(ELB) Req Scheduled- Action HostsAZ Average 1a&1c 4362x2(AZ)=4
18x2(AZ)=36 /Auto Scaling/WebAPI
- 42. Auto Scaling/ Worker&Worker-Spot
- 43. Scale In Alarm Comparison Operator: LessThanThreshold
Evaluation Periods: 1 Period: 300 Statistic:Average Metric
Name:CPUUtilization Threshold: 35 Unit:Percent Scale Out Alarm
Comparison Operator: GreaterThanThreshold Evaluation Periods: 1
Period: 300 Statistic:Average Metric Name:CPUUtilization Threshold:
70 Unit:Percent Name: WORKER-ASG ELB: - Min: 1 Max: 26 Health Check
Grace Period: 300 Availability Zones ap-northeast-1a,
ap-northeast-1c Name: WORKER-LC-v1 AMI: ami-XXXXXXX Instance Type:
c4.large Spot Price: - Adjustment Type:ChangeInCapacity Scaling
Adjustment: 2 Cooldown: 300 /Auto Scaling/Worker&Worker-Spot
WorkerAuto Scaling Group Launch Configuration Cloud Watch Alarm
Scaling Policy
- 44. Scale In Alarm Comparison Operator: LessThanThreshold
Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric
Name:CPUUtilization Threshold: 25 Unit:Percent Scale Out Alarm
Comparison Operator: GreaterThanThreshold Evaluation Periods: 1
Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold:
60 Unit:Percent Name: WORKER-ASG-SPOT ELB: - Min: 1 Max: 26 Name:
WORKER-SPOT-LC-v1 AMI: ami-XXXXXXX Instance Type: c4.large Spot
Price:0.147 Adjustment Type:ChangeInCapacity Scaling Adjustment: 2
Cooldown: 300 /Auto Scaling/Worker&Worker-Spot Worker-SpotAuto
Scaling Group Launch Configuration Cloud Watch Alarm Scaling Policy
Health Check Grace Period: 300 Availability Zones ap-northeast-1a,
ap-northeast-1c
- 45. Scale In Alarm Comparison Operator: LessThanThreshold
Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric
Name:CPUUtilization Threshold: 25 Unit:Percent Scale Out Alarm
Comparison Operator: GreaterThanThreshold Evaluation Periods: 1
Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold:
60 Unit:Percent Name: WORKER-ASG-SPOT ELB: - Min: 1 Max: 26 Name:
WORKER-SPOT-LC-v1 AMI: ami-XXXXXXX Instance Type: c4.large Spot
Price:0.147 Adjustment Type:ChangeInCapacity Scaling Adjustment: 2
Cooldown: 300 /Auto Scaling/Worker&Worker-Spot Worker-SpotAuto
Scaling Group Launch Configuration Cloud Watch Alarm Scaling Policy
Health Check Grace Period: 300 Availability Zones ap-northeast-1a,
ap-northeast-1c
http://www.slideshare.net/AmazonWebServicesJapan/20131023-aws-meisterregeneraterispotpublic/72
- 46. Scale In Alarm Comparison Operator: LessThanThreshold
Evaluation Periods: 1 Period: 300 Statistic:Maximum Metric
Name:CPUUtilization Threshold: 25 Unit:Percent Scale Out Alarm
Comparison Operator: GreaterThanThreshold Evaluation Periods: 1
Period: 300 Statistic:Maximum Metric Name:CPUUtilization Threshold:
60 Unit:Percent Name: WORKER-ASG-SPOT ELB: - Min: 1 Max: 26 Name:
WORKER-SPOT-LC-v1 AMI: ami-XXXXXXX Instance Type: c4.large Spot
Price:0.147 Adjustment Type:ChangeInCapacity Scaling Adjustment: 2
Cooldown: 300 /Auto Scaling/Worker&Worker-Spot Worker-SpotAuto
Scaling Group Launch Configuration Cloud Watch Alarm Scaling Policy
Health Check Grace Period: 300 Availability Zones ap-northeast-1a,
ap-northeast-1c WorkerWorker-Spot
http://www.slideshare.net/AmazonWebServicesJapan/20131023-aws-meisterregeneraterispotpublic/72
- 47. 1. Max= 2. Min=Max*(Req/Req)+ 3. Threshold(CPU Utilization)
Top=CPU Bottom=1Max CPU 4. Period, Evaluation Periods, Cooldown * *
/Auto Scaling/
- 48. Reserved Instances /Reserved Instances
- 49. /Reserved Instances Reserved Instances AutoScaling Billing
Infomation * * http://calculator.s3.amazonaws.com/index.html
- 50. /Reserved Instances Reserved Instances
- 51. /Reserved Instances Reserved Instances Min 4 / n%xm% y
xReserved
- 52. /Reserved Instances Reserved Instances 1080% All
Upfront
- 53. http://j.mp/summit2015_tsukada_sheet /
- 54. http://j.mp/summit2015_tsukada_sheet
- 55. http://j.mp/summit2015_tsukada_sheet (2) -858792!!
- 56. http://j.mp/summit2015_tsukada_sheet
- 57. http://j.mp/summit2015_tsukada_sheet 3
- 58. http://j.mp/summit2015_tsukada_sheet 12 +8389656 3
- 59.
- 60. RDS(Reserved Instances), ElastiCache(Reserved Cache Nodes),
Cloudfront(Reserved Capacity), DynamoDB(Reserved Capacity) AWS
Black Belt Tech &
http://www.slideshare.net/AmazonWebServicesJapan/20150325-aws-
blackbeltrispotpublic Spot Instances c1.medium Worker Lambda
Lambda/Job SpotSpotFleet API () SQLRDS
- 61.
- 62. !!!!!!!!!! Lambda HTTP/2 Scale ...?
- 63. Notes
- 64. THE END 2015-06-03 @akitsukada ()