Integrating Puppet with Cloud Infrastructures-Remco Overdijk

Remco OverdijkLEAD OPERATIONS ENGINEER

Automating the Cloud

Integrating Puppet with Cloud Infrastructures

@MaxServ @RemzJay

Automating the Cloud

3AUTOMATING THE CLOUD

Traditional Website Hosting

MySQL

AUTOMATING THE CLOUD

Single Server Infrastructure Issues

PROBLEM

Limited headroom

Service issues affect other services

One outage means downtime

Maintenance during deployment

4

Image credit: https://commons.wikimedia.org/wiki/File:Grumpy-Cat.jpg - Rjommabolli (CC 4.0)

https://commons.wikimedia.org/wiki/File:Grumpy-Cat.jpg


Single Server Infrastructure Issues6

Scalability

Service Isolation

Redundancy

Zero-Downtime Deployments

SOLUTIONPROBLEM

Limited headroom

Affected services

One outage means downtime

Maintenance during deployment

• AWS specific, but applies to most (if not any) Cloud platforms.


Scope7

• LAMP stack, but works for most stacks.

• Mix & match for best results.

• The method that works best depends on your own setup.

• Based on Puppet Open Source.

• Things may be different in Puppet Enterprise (Orchestrator).

• Ready-to-go AMI’s may be faster to launch, but harder to maintain.


AWS Infrastructure Principles

Read more

https://media.amazonwebservices.com/AWS_Cloud_Best_Practices.pdf

8

• Infrastructure is failure-prone; Service robustness is achieved through redundancy.

• EC2 instances should be considered volatile.

• Scaling should be both vertical and horizontal.

• Legacy applications aren’t magically cloud-ready.

• Loose coupling improves scalability.

• Isolation increases security and decreases dependencies.


AWS Infra: A lot of moving parts9

ProvisioningInfrastructure

• IAM Server Certificates

• IAM Instance Profiles

• IAM Roles

• IAM Policies

• IAM Role Policies

• CodeDeploy Apps

• CodeDeploy

Deployment Groups

• EC2 Instances

• EC2 Elastic IP’s

• CloudWatch Metrics

• CloudWatch Alarms

• EIP Attachments

• ElastiCache Subnet

Groups

Provisioning Infrastructure

AWS Resources

• ElastiCache Clusters

• ElastiCache Parameter

Groups

• EC2 Elastic Load Balancers

• ELB Health Checks

• ELB App Cookie Stickiness

Policies

• Key Pairs

• RDS Subnet Groups

• RDS Parameter Groups

• RDS DB Instances

• Route53 Zones

• Route53 Records

• S3 Buckets

• S3 Policies

• S3 CORS Configuration

• Security Groups

• SNS Topics

• SNS Topic

Subscriptions

• VPC’s

• VPC Subnets

• VPC Internet Gateways

• VPC (S3) Endpoints

• VPC Route Tables

• VPC Customer

Gateways

• VPC VPN Gateways

• VPC DHCP Option Sets

• VPC VPN Routes

11


That’s a lot of clicking13

• Manual changes are extremely error-prone.

• Manual changes result in an inconsistent platform.

• Collaboration is difficult; People are scared to break things.

• Changes are scattered throughout the AWS console.


Provisioning14

• Uses the same DSL as your ”regular” Puppet code.


Puppetlabs-AWS15

• Most benefits from ”regular” Puppet;like relationships, defined types and the graph.

• At the moment this module only supports a few of the resources in the AWS API.

• Does NOT run using agents / puppet masters, but manually fromyour workstation using `puppet apply`.

• Not as idempotent as you’d like at times.

Read more

https://github.com/puppetlabs/puppetlabs-aws

https://forge.puppetlabs.com/puppetlabs/aws

`AWS_PROFILE=my-aws-account AWS_REGION=eu-west-1 puppet apply aws-stack.pp --templatedir ./templates/`


Hashicorp Terraform

• Uses HCL, which looks a lot like Puppet DSL.

• Maintains a dependency graph, just like Puppet.

• Runs from your workstation.

• Is not AWS API feature complete, but covers most services.

• Allows flexible scaling and destruction.

17

Read more

https://github.com/hashicorp/terraform

https://www.terraform.io/


Hashicorp Terraform18

View & try the full source

https://github.com/MaxServ/Terraform-Puppet-Demo/tree/master/Terraform


Puppet?20

• Automated Infrastructure is awesome.

• We like Puppet too.

• Empty EC2 instances don’t serve websites.

• How do we connect Puppet to the infrastructure we’ve just provisioned?

BootstrappingPuppet

Bootstrapping Puppet

Machine Configuration States22

SLIDE CREDIT: Tim Bell, CERN – PuppetConf 2012

https://speakerdeck.com/puppetlabs/cern-accelerating-science-with-puppet

https://youtu.be/-Ykb2j2ojYU?t=19m33s

Image credit: Geert Orye, https://flic.kr/p/7zDqXY (CC BY 2.0)

https://flic.kr/p/7zDqXY


So, Cattle.

• Configuration should be at type level. Not node level.

• Nodes should be replaceable.

• Data on the nodes should be considered volatile.

• Only versioned and cached content should be present.

25

• Provisioning should be automated; No manual intervention should be required.


Puppet: Autosigning

• Automates indoctrination for new nodes.

• Multiple mechanisms available:

Naïve, Basic and Policy-based Autosigning.

26

View & try

https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Puppet/puppet.conf

Read more

https://docs.puppetlabs.com/puppet/latest/reference/ssl_autosign.html

Be very careful with naïve autosigning.

Don’t do it in production without strict firewalls.


Puppet: Autosigning27


Node Manifests using Regex28

Read more

https://docs.puppetlabs.com/puppet/latest/reference/lang_node_definitions.html#regular-expression-names


Hiera

• Configuration at `type` level instead of `clientcert`.

• Use node-level overrides when required.

• Combine with host `%H` mount points for master/slave.

29

View & try

https://github.com/MaxServ/Terraform-Puppet-Demo/tree/master/Puppet/hieradata

Read more

https://docs.puppetlabs.com/hiera/latest/

https://docs.puppetlabs.com/guides/file_serving.html#file-server-configuration

https://docs.puppetlabs.com/hiera/latest/


Hiera30


ENC’s in Foreman31

Read more

https://docs.puppetlabs.com/guides/external_nodes.html

http://theforeman.org/manuals/1.10/index.html#2.Quickstart


Foreman: Default Host Groups

• The same idea as Autosigning in Puppet.

• Uses a Foreman Plugin.

• Use facts for assigning host groups.

32

Read more

https://github.com/theforeman/foreman_default_hostgroup


Foreman: Default Host Groups33


Assigning Configuration: Facter34

View & try

https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Terraform/templates/userdata.tpl

https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Puppet/Vagrantfile

Read more

https://docs.puppetlabs.com/facter/2.4/custom_facts.html#structured-data-facts


AWS: User data

• User data allows you to pass configuration data to an instance during launch.

• User data can be used to provide cloud-init configuration.

35


AWS: User data36


Cloud-init37

• Handles early initialization of cloud instances.

• Supports EC2, CloudStack, OpenStack, OnApp, OpenNebula, RHEVm, vSphere & more.

• Can install packages, do basic config and more.

• Is able to bootstrap Puppet.

Read more

http://cloudinit.readthedocs.org/en/latest/index.html

http://cloudinit.readthedocs.org/en/latest/topics/examples.html#setup-and-run-puppet


Cloud-init with Puppetlabs-Aws38

Read more

https://docs.puppetlabs.com/puppet/latest/reference/lang_template_erb.html


Cloud-init with Terraform39

View & try

https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Terraform/ec2.tf

Read more

https://docs.puppetlabs.com/puppet/latest/reference/lang_template_erb.html


Don’t forget to clean up!

• Revoke Puppet-CA certificates for decommissioned nodes.

• Clean up Salt keys.

• Remove old reports, exported resources and catalogs from PuppetDB.

• Clean connected resources like Load Balancers.

40

Sloppiness will catch up with you when it hurts the most.

Demo

Demo

Terraform & Puppet

THIS DEMONSTRATION INCLUDES:

VIEW & TRY THE FULL SOURCE OF THIS DEMO

https://github.com/MaxServ/Terraform-Puppet-Demo

A

B

C

Docker container running a Puppetmaster.

AWS Stack with 4 webservers using Terraform.

Webserver provisioning using Puppet based on type.

42

Demo

Virtual Private Cloud43

44

0416 - 30 10 00

Remco OverdijkLEAD OPERATIONS ENGINEER

[email protected]

Questions?

@MaxServ @RemzJay

mailto:[email protected]

Technology

Integrating Puppet with Cloud Infrastructures-Remco Overdijk