Learn you some Ansible for great good!

OpenStack Vancouver Summit

Learn you some Ansible for Great Good!

Juergen Brendel (@brendelconsult) , David Lapsley (@devlaps)

May 21, 2015

Unified test and deployment environments

Dev, Test, Deploy

Dev, Test, Deploy

Why didn't we

catch this bug

in testing?

Dev, Test, Deploy

Why didn't we

catch this bug

in testing?

We can't

reproduce the

issues.

Dev, Test, Deploy

Why didn't we

catch this bug

in testing?

We can't

reproduce the

issues.

I don't have

access to our

test server.

Dev, Test, Deploy

Why didn't we

catch this bug

in testing?

How do I setup

my

development

environment?

We can't

reproduce the

issues.

I don't have

access to our

test server.

Dev, Test, Deploy

Why didn't we

catch this bug

in testing?

How do I setup

my

development

environment?

We can't

reproduce the

issues.

I don't have

access to our

test server.

“It works for me...”

(shrug)

Dev, Test, Deploy

Why didn't we

catch this bug

in testing?

How do I setup

my

development

environment?

We can't

reproduce the

issues.

I don't have

access to our

test server.

“It works for me...”

(shrug)Wouldn't this be nice instead?

Single command: Dev environment created

Single command: Test environment created

Summary

• Configuration management background

• Ansible introduction

• Rise of APIs

• Unified test and deployment environments

• Demonstration

Configuration Management Tools

Overview

Configuring servers

How do you configure a server?

Arcane

magic

Configuring servers


Arcane

magic

Configuring servers


Manual

instructions

Arcane

magic

Configuring servers


Manual

instructions

Scripts

Arcane

magic

Configuring servers


Manual

instructions

Scripts

CM tools

Arcane

magic

Configuring servers


Manual

instructions

Scripts

CM tools

Automation!

CM Tools

Describe the desired state

Ensure all

system packages

are updated.

CM Tools


Ensure all

system packages

are updated.

CM Tools


Ensure package

“apache” is

installed.

Ensure all

system packages

are updated.

CM Tools


Ensure that user

“xyz” exists.

Ensure package

“apache” is

installed.

Ensure all

system packages

are updated.

CM Tools


Ensure that user

“xyz” exists.

Ensure package

“apache” is

installed.Have latest

sources: Clone

repo, update if it

exists already.

Ensure all

system packages

are updated.

CM Tools


Ensure that user

“xyz” exists.

Ensure package

“apache” is


sources: Clone

repo, update if it

exists already.

Ensure package

“postgres” v9.1 is

installed.

Ensure all

system packages

are updated.

CM Tools


Ensure that user

“xyz” exists.

Ensure package

“apache” is


sources: Clone

repo, update if it

exists already.

Ensure package

“postgres” v9.1 is

installed. Ensure DB

“app_data” exists

with password

“****”.

Configuration Management Tools

Varieties

Puppet (2005)“powerful, feature-rich, enterprise-y”

Chef (2009)

CM Tools variety


Chef (2009)

Salt (2011)“simple, fast, good for most things”

Ansible (2012)

CM Tools variety


Chef (2009)

Salt (2011)“simple, fast, good for most things”

Ansible (2012)

Fabric (2011)“not really CMS tools”

Scripts

CM Tools variety

Ansible Overview

• “Orchestration engine” for CM and deployment

• Written in Python

• Uses YAML

• “Playbooks”

• Config specs or explicit commands

Ansible overview

• Key Points:• No central configuration server

• No key management

• No agent to install on target machine

• Explicit order

Ansible simplicity

• Key Points:• No central configuration server

• No key management

• No agent to install on target machine

• Explicit order

• Requirements:• Need SSH access (with key or password)

• Need Python installed on target machine

Ansible simplicity

Ansible architecture

Server 1

Server 2Server 3

Server 4

Server 5

Your laptop

Ansible Overview

Modules

Modules

Hundreds of them. They know how to do stuff…

Command

Shell

Script

Modules


Command

Shell

Script

Copy

Sync

Templates

Line ops

Modules


Command

Shell

Script

Copy

Sync

Templates

Line ops

Install packages

Users and groups

Networking

Services

Modules


Command

Shell

Script

Copy

Sync

Templates

Line ops

Install packages

Users and groups

Networking

ServicesRepositories

Message queues

Monitoring

Notification

Modules


Command

Shell

Script

Copy

Sync

Templates

Line ops

Install packages

Users and groups

Networking

ServicesRepositories

Message queues

Monitoring

Notification

Web servers

Database servers

Cloud infra

Ansible Overview

How does it

work?

How does it work?

Laptop

How does it work?

Python Module

“Install Apache”

Laptop

run playbook

How does it work?

Python Module


Run ModuleLaptop

ssh

How does it work?

Python Module


Run Module

(then delete)

Run ModuleLaptop

ssh

How does it work?

Python Module


Run Module

(then delete)

Run ModuleLaptop

return results

Ansible Overview

Details

Inventory and groups

Define hosts, organized in groups

Inventory and groups

Define hosts, organized in groups

by function

by location

by hosting provider

...

[europe]

server1.somehoster.co.uk

server2.otherhoster.de

[north-america]

host-a.serverhost.com

host-b.serverhost.com

[frontend]



[backend]

server2.otherhoster.de

host-a.serverhost.com

Adhoc commands

Single commands, applied to groups

Adhoc commands

Single commands, applied to groups

$ ansible -i hosts europe –a “uname -a”

$ ansible -i hosts frontend -a “/sbin/reboot” -f 3

• Tell Ansible what to do

Playbooks

---

- hosts: frontend

sudo: yes

tasks:

- name: Update the system

apt: pkg=nginx state=latest

- name: Create the user account

user: name=appuser shell=/bin/bash state=present

- name: Copy files to remote user's home

copy: >

src=files/names.txt dst=/home/appuser

owner=appuser mode=0644

• Provide input to Ansible templates

Variables

---

- hosts: all

sudo: yes

vars:

username: appuser

tasks:

- name: Create the user account

user: >

name={{ username }}

shell=/bin/bash

state=present

• Simple layout for arranging Ansible playbooks, variables, templates, metadata, etc.

Simple Project Layout

/

my_hosts

group_vars/

all

frontend

backend

europe

north-america

site.yml

• Best practices layout for arranging Ansible playbooks, variables, templates, metadata, etc.

• Better suited for larger projects

• More extensible

Best Practice Project Layout

/

ansible.cfg

deploy_hosts

staging_hosts

group_vars/

all

frontend

backend

europe

north-america

host_vars/



site.yml

roles/

common/

tasks/

main.yml

handlers/

main.yml

templates/

sshd_config.j2

files/

my_script.sh

vars/

main.yml

web/

...

db/

...

Playbooks with roles

---

- hosts: frontend

sudo: yes

roles:

- common

- web

The rise and rise of APIs


APIs


APIs

Local


APIs

Local Infrastructure


APIs

Local Infrastructure Services


APIs

Local Infrastructure Services

Ansible 'cloud' modules

Public cloud

OpenStack

Amazon AWS

Google Compute

Azure

Digital Ocean

Linode

Private cloud

• OpenStack

• Eucalyptus

• Vsphere

• Docker

• libvirt

Example: AWS Modules

EC2 / infrastructure

• Instances

• Images

• VPCs

• Load balancers

Services

• S3

• Route 53

• Databases

• Cache

• Create instances via AWS and OpenStack

Example: Create instances

- name: Booting EC2 guests

ec2:

key_name: my-key

group: my-security-group

instance_type: t2.micro

image: ami-120abc90

region: us-east-1

count: 1

register: ec2results

- name: Booting OpenStack guests

nova_compute:

state: present

login_username: "{{ openstack_username }}"

login_password: "{{ openstack_password }}"

login_tenant_name: "{{ openstack_tenantname }}"

name: "{{ cluster_id }}-{{ item }}"

image_id: "{{ openstack_image_id }}"

key_name: "{{ openstack_keyname }}"

wait_for: 60

flavor_id: "{{ openstack_flavor_id }}"

nics:

- net-id: "{{ openstack_internal_net_id }}”

register: openstack_guests

• Add hosts to inventory

Example: Create instances

- local_action:

module: ec2

key_name: my-key

group: my-security-group

instance_type: t2.micro

image: ami-120abc90

region: us-east-1

count: 3

register: ec2results

- local_action:

module: add_host

hostname: {{ item.public_ip }}

groupname: my-server-group

with_items: ec2results.instances


Vagrant

Use Vagrant to spin-up VMs local (VirtualBox, VMware, etc.)

cloud (EC2)

Use Ansible as 'provisioner'

Make an inventory file with just your VM

Point at same playbook as before

Vagrant

• Tells vagrant which VMs to construct

• How to construct them:• RAM

• Virtual CPUs

• Network interfaces (public, private, static, nat’d)

Vagrant config: Vagrantfile

Vagrant.configure(2) do |config|

config.vm.box = "saucy64"

config.vm.box_url = "http://cloud-

images.ubuntu.com/vagrant/..."

config.vm.host_name = "myapp-test"

config.vm.network "private_network", ip: "192.168.1.2”

config.vm.provision "ansible" do |ansible|

ansible.playbook = "site.yml"

#ansible.verbose = "vvvv"

ansible.inventory_path = "vagrant_hosts"

ansible.host_key_checking= false

end

end

• Specifies which VMs/Groups Ansible should run against

Inventory: Vagrant Hosts

[vagrant]

vagrant_host

ansible_ssh_host=192.168.1.2

[frontend-hosts]

vagrant_host

[applayer-hosts]

vagrant_host

[backend-hosts]

vagrant_host

[db-access:children]

applayer-hosts

backend-hosts

[appserver-access:children]

frontend-hosts

applayer-hosts

Vars: group_vars/vagrant

Variables that only apply to Vagrant instances

---

ansible_ssh_user: vagrant

Create and configure VMs

$ vagrant up

...

$ vagrant provision


Cattle, not

pets!

Desired development/deployment workflow

- Local unit tests

- Local provisioning with Vagrant + Ansible

- Integration tests

Local dev

and testing


- Create/update cloud

staging servers with

Ansible

- Provision servers with

Ansible

Local dev

and testing

Cloud

testing


- Create/update cloud production

servers with Ansible

- Provision servers with Ansible

Local dev

and testing

Cloud

testing

Cloud deployment

Demo

Dev Environment

Cacher (apt/pip)

MCP MHV1 MHV2

Ansible

Git cache

• Same Ansible playbooks can be used to provision application locally or in the cloud

Key Takeaways

• Same Ansible playbooks can be used to provision application locally or in the cloud

• With cloud APIs and Ansible modules (OpenStack, AWS, Rackspace, …) playbooks can also be used to provision infrastructure

Key Takeaways

References

• Questions: [email protected], [email protected]

• Ansible playbooks: http://bit.ly/devstack-ansible

• Ansible docs: http://docs.ansible.com/

• Ansible source: https://github.com/ansible/ansible

• Vagrant: http://www.vagrantup.com/

• Example project: http://bit.ly/ansible-devstack

mailto:[email protected]

mailto:[email protected]

http://bit.ly/devstack-ansible

http://docs.ansible.com/

https://github.com/ansible/ansible

http://www.vagrantup.com/

http://bit.ly/ansible-devstack

@brendelconsult, @devlaps

Thank You

Internet

Learn you some Ansible for great good!