© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Chetan Dandekar, Senior Product Manager, AWS

May 20, 2015

Deep Dive: Infrastructure as Code

You are on-board …

needs to experiment, innovate, reduce riskBusiness

of services and applicationsContinuous

Delivery

culture, automation, measurement, sharingDevOps

infrastructure-as-codeCloud

AWS CloudFormation

AWS CloudFormation

Create templates of the infrastructure and applications you want to run on AWS

Have the CloudFormation service automatically provision the required AWS resources and their relationships from the templates

Easily version control, replicate or update the infrastructure and applications using the templates

Integrates with other development, CI/CD, and management tools.

Basic workflow

Design

Create Infrastructure

Templates

Write Application

Code

Create Stacks

Iterate

depends on

Design -

Imagine building a food ordering service

Food Catalog Website

Ordering Website

Customer DB Service

Inventory Service

Recommendations Service

Analytics Service Fulfillment

Service

Payment

Service

Create template –

For example, for the food catalog website

security group

Auto Scaling group

EC2

instance

Elastic Load

Balancing

Customer DB Service

Inventory Service

Recommendations Service

ElastiCache

memcached cluster

Software pkgs,

config, & dataCloudWatch

alarms

Create template – Resources

security group

Auto Scaling group

EC2

instance

Elastic Load

Balancing

ElastiCache

memcached cluster

Software pkgs,

config, & dataCloudWatch

alarms

"Resources" : {"SecurityGroup" : {},"WebServerGroup" : {

"Type" : "AWS::AutoScaling::AutoScalingGroup","Properties" : {

"MinSize" : "1","MaxSize" : "3","LoadBalancerNames" : [ { "Ref" :

"LoadBalancer" } ],...

}},"LoadBalancer" : {},"CacheCluster" : {},"Alarm" : {}

},

CloudFormation Template

Create template – Parameters

Auto Scaling group

EC2

instance

Recommendations Service

Inventory Service

Customer DB Service

Info to Customize Stack at Creation

Examples: Instance Type, App Pkg Version

"Parameters" : {"CustomerDBServiceEndPoint" : {

"Description" : "URL of the Customer DB Service","Type" : "String"

},"CustomerDBServiceKey" : {

"Description" : "API key for the Customer DB Service",

"Type" : "String","NoEcho" : "true"

},"InstanceType" : {

"Description" : "WebServer EC2 instance type","Type" : "String","Default" : "m3.medium","AllowedValues" :

["m3.medium","m3.large","m3.xlarge"],"ConstraintDescription" : "Must be a valid

instance type"

CloudFormation Template

Create template – Outputs

Elastic Load

Balancing

"Resources" : {

"LoadBalancer" : {},

...

},

"Outputs" : {

"WebsiteDNSName" : {

"Description" : "The DNS name of the website",

"Value" : {

"Fn::GetAtt" : [ "LoadBalancer", "DNSName" ]

}

}

}

CloudFormation Template

Create template – Deploy and configure software

Auto Scaling group

EC2

instance

Software pkgs,

config, & data

"AWS::CloudFormation::Init": {

"webapp-config": {

"packages" : {}, "sources" : {}, "files" : {},

"groups" : {}, "users" : {},

"commands" : {}, "services" : {}

},

"chef-config" : {}

}

CloudFormation Template

Declarative

Debug-able

Updatable

Highly Secure

BIOT™ Bring In

Other Tools

Create template – Language features

Create stack

Operate stack

Use a wide range of AWS services

Auto Scaling

Amazon CloudFront

AWS CloudTrail

Amazon CloudWatch

Amazon DynamoDB

Amazon EC2

AWS Elastic Beanstalk

Amazon ElastiCache

Elastic Load Balancing

Amazon Kinesis

IAM

AWS OpsWorks

Amazon RDS

Amazon Redshift

Amazon Route 53

Amazon S3

Amazon SimpleDB

Amazon SNS

Amazon SQS

Amazon VPC

and more …

Basic workflow

Design

Create Infrastructure

Templates

Write Application

Code

Create Stacks

Iterate

Infrastructure-as-code workflow

Code templates

Version control

Code review

Integrate

“It’s all software”

“It’s all software” – organize like it’s software

Frontend Services

• Consumer Website, Seller Website, Mobile Backend

Backend Services

• Search, Payments, Reviews, Recommendations

Shared Services

• CRM DBs, Common Monitoring, Alarms, Subnets, Security Groups

Base Network

• VPCs, Internet Gateways, VPNs, NATs

Identity • IAM Users, Groups, Roles

“It’s all software” – build and operate like it’s

software

Application software

Source code

Package

Loader/Interpreter

Desired application state in memory

Infrastructure software

JSON templates / JSON template generators

JSON templates

AWS CloudFormation

Desired infrastructure in the cloud

Iterate on infrastructure

Update stack

In-place Blue-Green

Faster

Cost-efficient

Simpler state and data

migration

Working stack not

touched

Extending AWS CloudFormation

Extend with Custom Resources

security group

Auto Scaling group

EC2

instance

Elastic Load

Balancing

ElastiCache

memcached cluster

Software pkgs,

config, & dataCloudWatch

alarmsWeb Analytics

ServiceAWS

CloudFormation

Provision

AWS Resources

"Resources" : {

"WebAnalyticsTrackingID" : {

"Type" : "Custom::WebAnalyticsService::TrackingID",

"Properties" : {

"ServiceToken" : "arn:aws:sns:...",

"Target" : {"Fn::GetAtt" : ["LoadBalancer", "DNSName"]},

"Plan" : "Gold"

}

},

...

“Success” + Metadata

“Create, Update, Rollback, or Delete”

+ Metadata

Lambda-backed custom resources

security group

Auto Scaling group

EC2

instance

Elastic Load

Balancing

ElastiCache

memcached cluster

Software pkgs,

config, & dataCloudWatch

alarms

Your AWS CloudFormation stack

// Implement custom logic here

Look up an AMI ID

Your AWS Lambda functions

Look up VPC ID and Subnet ID

Reverse an IP address

Lambda-powered

custom resources

Application-deployment-as-code

Infrastructure Provisioning

EC2

SQS, SNS, Amazon Kinesis, etc.

Databases

VPC

IAM

Application Deployment

Download Packages, Install Software, Configure Apps,

Bootstrap Apps, Update Software, Restart Apps,

etc.

CloudFormation

• Templatize

• Replicate

• Automate

Application-deployment-as-code

inside a CloudFormation template

Amazon Machine Images

CloudFormation::Init

Chef, Puppet, CodeDeploy, …

OpsWorks

Chef

Metadata

AWS::CloudFormation::Init

AWS::CloudFormation::Init

Declarative

Reusable

Grouping & Ordering

Debug-able

Updatable

Highly Secure

BIOT™ (Bring In Other Tools)

ow.ly/DiNCm

"AWS::CloudFormation::Init": {

"webapp-config": {

"packages" : {}, "sources" : {}, "files" : {},

"groups" : {}, "users" : {},

"commands" : {}, "services" : {}

AWS::CloudFormation::Init

Declarative

AWS::CloudFormation::Init

Debug-able

AWS::CloudFormation::Init

Supports updates

"packages" : {}, "sources" : {}, "files" : {}, "groups" : {}, "users" : {},"commands" : {}, "services" : {}

AWS::CloudFormation::Init

"install_chef" : {},

"install_wordpress" : {

"commands" : {

"01_get_cookbook" : {}, ...,

"05_configure_node_run_list" : {

"command" : "knife node run_list add -z `knife node list -z` recipe[wordpress]",

"cwd" : "/var/chef/chef-repo",

"env" : { "HOME" : "/var/chef" }

Flexibility to bring in other tools such as AWS CodeDeploy and Chef

ow.ly/DiNkz

AWS::CloudFormation::Init

"YourInstance": {

"Metadata": {

"AWS::CloudFormation::Authentication": {

"S3AccessCreds": {

"type": "S3",

"roleName": { "Ref" : "InstanceRole"},

"buckets" : ["your-bucket"]

}

},

"AWS::CloudFormation::Init": {}

Supports role-based auth

Securely download

Choose auth type.

IAM Role is

recommended

ow.ly/DqkrB

Use AWS::CloudFormation::Init

"UserData": {

"# Get the latest CloudFormation helper scripts package\n","yum update -y aws-cfn-bootstrap\n",

"# Trigger CloudFormation::Init configuration \n","/opt/aws/bin/cfn-init --stack ", {"Ref": "AWS::StackId"},

" --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n",

"# Signal completion\n","/opt/aws/bin/cfn-signal –e $? --stack ", {"Ref": "AWS::StackId"},

" --resource WebServerInstance ", " --region ", {"Ref": "AWS::Region"}, "\n"

Use CloudWatch Logs for debugging

"install_logs": {

"packages" : { ... "awslogs" ... },

"services" : { ... "awslogs" ... }

"files": {

"/tmp/cwlogs/cfn-logs.conf": {}

file = /var/log/cfn-init.loglog_stream_name = {instance_id}/cfn-init.log

file = /var/log/cfn-hup.loglog_stream_name = {instance_id}/cfn-hup.log

ow.ly/E0zO3

Use CloudWatch Logs for debugging

ow.ly/E0zO3

Bake AMIs for faster booting

Bake AMIs for maintaining golden images

Dev/test stacks Bake AMI Staging/prod stacks

Tracking

Using CloudFormation and

OpsWorks together

Infrastructure Provisioning

EC2

SQS, SNS, Amazon Kinesis, etc.

Databases

VPC

IAM

Application Deployment

Download Packages, Install Software, Configure Apps,

Bootstrap Apps, Update Software, Restart Apps,

etc.

CloudFormation

• Templatize

• Replicate

• Automate

OpsWorks

• Built-in Application Lifecycle

• Interactive Application Console

OpsWorks & CloudFormation “side-by-side”

OpsWorks

• Built-in Application Lifecycle

• Interactive Application Console

Infrastructure Provisioning

EC2

SQS, SNS, Amazon Kinesis, etc.

Databases

VPC

IAM

Application Deployment

Download Packages, Install Software, Configure Apps,

Bootstrap Apps, Update Software, Restart Apps,

etc.

CloudFormation

• Templatize

• Replicate

• Automate

OpsWorks “inside” CloudFormation

Infrastructure-as-code in a

CI/CD pipeline

CloudFormation in a CI/CD pipeline

AWS CloudFormationIssue Tracker

App Developers

DevOps Engineers,Infrastructure Developers,

Systems Engineers

Dev Env Code Repo

App Pkgs, CloudFormation Templates, Etc.

CI Server

Test

Staging

ProdCode Review

"Infra-as-Code"

App Code & Templates

Templatize existing resources

CloudFormer: Templatize existing resources

1. Launch a CloudFormer

application stack

2. Walkthrough the

CloudFormer UI & select

resources to templatize

4. Customize

Example: parameterize

resource properties

5. Create a new stack

Practitioners of infrastructure-as-code

• Developers/DevOps teams value CloudFormation for its ability to treat infrastructure as code, allowing them to apply software engineering principles, such as SOA, revision control, code reviews, integration testing to infrastructure.

• IT Admins and MSPs value CloudFormation as a platform to enable standardization, managed consumption, and role-specialization.

• ISVs value CloudFormation for its ability to support scaling out of multi-tenant SaaS products by quickly replicating or updating stacks. ISVs also value CloudFormation as a way to package and deploy their software in their customer accounts on AWS.

AWS Summit – Chicago: An exciting, free cloud conference designed to educate and inform new

customers about the AWS platform, best practices and new cloud services.

Details• July 1, 2015

• Chicago, Illinois

• @ McCormick Place

Featuring• New product launches

• 36+ sessions, labs, and bootcamps

• Executive and partner networking

Registration is now open• Come and see what AWS and the cloud can do for you.

CTA Script

- If you are interested in learning more about how to navigate the cloud to grow

your business - then attend the AWS Summit Chicago, July 1st.

- Register today to learn from technical sessions led by AWS engineers, hear best

practices from AWS customers and partners, and participate in some of the 30+

paid sessions and labs.

- Simply go to

https://aws.amazon.com/summits/chicago/?trkcampaign=summit_chicago_bootc

amps&trk=Webinar_slide

to register today.

- Registration is FREE.

TRACKING CODE:

- Listed above.

Thank you!