1#Dynatrace QCon London 2015 Martin Etmajer, Technology Strategist @ Dynatrace Deploying On-Prem as SaaS Why we go with Ansible

1 #Dynatrace

QCon London 2015

Martin Etmajer, Technology Strategist @ Dynatrace

Deploying On-Prem as SaaSWhy we go with Ansible

2 #Dynatrace

Martin EtmajerTechnology Strategist @Dynatrace

[email protected]

@metmajer

http://blog.dynatrace.com

3 #Dynatrace

Agenda» Dynatrace On-Prem

» Dynatrace SaaS Architecture

» Ansible as Orchestration Engine

» Test-Driven Infrastructure with Test Kitchen

4 #Dynatrace

Dynatrace

5 #Dynatrace

Dynatrace On-Prem Environment

6 #Dynatrace


7 #Dynatrace


8 #Dynatrace


9 #Dynatrace


10 #Dynatrace

Dynatrace. So What?

Presentation TierBusiness Logic Tier

11 #Dynatrace

Dynatrace. So What?

# Requests

Health

Timings

12 #Dynatrace

Dynatrace. So What?

Frustrated Users

13 #Dynatrace

Dynatrace. So What?

What happened?

14 #Dynatrace

Dynatrace. So What?

Erroneous Call Username

SQL Statements

Exception

15 #Dynatrace

Reasons for Going SaaS

16 #Dynatrace


#1: Operational RiskMoves from Customer to Service Provider

17 #Dynatrace


#2: Customers Want SolutionsWithout Buying Extra Infrastructure

18 #Dynatrace


#3: Customers Already in the Cloud

19 #Dynatrace

Dynatrace SaaS Architecture

20 #Dynatrace

On-Prem Dynatrace Environment

21 #Dynatrace

Dynatrace SaaS Environment

22 #Dynatrace


23 #Dynatrace


24 #Dynatrace

for Dynatrace SaaS

Elastic Compute Cloud (EC2)

25 #Dynatrace

for Dynatrace SaaS

Elastic Block Store (EBS)

26 #Dynatrace

for Dynatrace SaaS

Relational Database Service (RDS)

27 #Dynatrace

for Dynatrace SaaS

Elastic IP (EIP)

28 #Dynatrace

for Dynatrace SaaS

Route 53 (DNS)

29 #Dynatrace

for Dynatrace SaaS

Simple Storage Service (S3)

30 #Dynatrace

for Dynatrace SaaS

CloudWatch

31 #Dynatrace

Design Decisions

32 #Dynatrace

#1 – Dedicated Customer Resources

Clean Separation of Customers & Data

33 #Dynatrace


Customizable SecurityIP-Based Access Rules

34 #Dynatrace


Grow with the CustomerResize Resources on Demand

35 #Dynatrace

#2 – Data Persistence

Prevent Data Loss When Resources Go Down

36 #Dynatrace

#3 – Regular Backups

Enable < Daily Data RecoveryAll Configuration and Data

37 #Dynatrace

Orchestration Platform

38 #Dynatrace


39 #Dynatrace


40 #Dynatrace


41 #Dynatrace

How We Did ItLessions Learned

42 #Dynatrace

We Coded Our Own Solution

#1: Lifecycle Management of Resources

43 #Dynatrace


#2: Use-Cases

44 #Dynatrace


#3: Monitoring

45 #Dynatrace


Extensive Testing

Too much Manual Work Involved

Large Codebase, Tied to Single Vendor

46 #Dynatrace


47 #Dynatrace

Hey, Ansible!

48 #Dynatrace

Ansible is a radically simple IT automation engine for

» environment and infrastructure provisioning

» configuration management

» application deployment

» etc.

What is Ansible?

http://www.ansible.com/

49 #Dynatrace

“I wrote Ansible because none of the existing tools fit my brain. I wanted a tool that I could not use for 6 months, come back later, and still remember how it worked.”Michael DeHaan, Ansible Founder

“We need to do a rolling deployment of changes that have certain dependencies (including external services).With Ansible this becomes trivial.Puppet on the other hand feels like the Wild West.”User IUseRhetoric on reddit.com

Why Yet Another Tool?

http://www.reddit.com/r/ansible/comments/2f5t0m/reasons_for_moving_from_chef_puppet_to_ansible/ck6k61g

50 #Dynatrace

» No Agents ✓

» No Scripting ✓

» Simple and Powerful ✓

Ansible Design Principles

51 #Dynatrace

Agent-Based Architecture

52 #Dynatrace


53 #Dynatrace


54 #Dynatrace

Ansible’s Agentless Architecture

55 #Dynatrace


56 #Dynatrace


57 #Dynatrace


58 #Dynatrace

Ansible is an Orchestration Engine. So What?

59 #Dynatrace


60 #Dynatrace


61 #Dynatrace


62 #Dynatrace


63 #Dynatrace


64 #Dynatrace


65 #Dynatrace

Ansible for the Cloud

66 #Dynatrace

Ansible Cloud Modules

67 #Dynatrace

Ansible Cloud Modules

68 #Dynatrace

Ansible Cloud Modules for Amazon

69 #Dynatrace

Ansible Concepts: Inventories

70 #Dynatrace

» Ansible provisions groups of servers at once

» Groups and hosts are defined in inventories

» Use inventories for staging, production, etc.

Static vs. Dynamic Inventories

» Static: text files expressed in an INI-like format

» Dynamic: Python scripts for dynamic environments (cloud)

» Static + Dynamic: combine multiple inventories (hybrid cloud)

Ansible Inventories

71 #Dynatrace

Example: Static Inventory# file: production

[balancers]www.example.com

[webservers]www[0-9].example.com

[dbservers]db[a:f].example.com

[monitoring]dynatrace.example.com

GroupHost

Numeric Range

Alphabetic Range

72 #Dynatrace

Python scripts that get data from dynamic sources such as:

» Cloud: Amazon, DigitalOcean, Google, OpenShift, OpenStack, etc.

» Distributed Information Services: LDAP, etcd, etc.

Dynamic Inventories

73 #Dynatrace

Ansible Concepts: Playbooks

74 #Dynatrace

Ansible Playbooks$> ansible-playbook [–i <inventory>] <playbook.yml>

What is a Playbook?

» Describes policies your managed machines shall enforce

» Consist of vars, tasks, handlers, files, templates and roles

» Expressed in the YAML format (dictionaries, lists and scalars)

75 #Dynatrace

Example: Ansible Playbook--- # file: webservers.yml- hosts: webservers handlers: - name: reload apache2 service: name=apache2 state=reloaded tasks: - name: Install Apache HTTP Server apt: name=apache2 update_cache=yes - name: Install Apache Modules apache2_module: name={{ item }} state=present with_items: - proxy - proxy_httpd notify: reload apache2 remote_user: deploy sudo: yes

Play

Module Arguments

Variable

Notify Handler

76 #Dynatrace

Example: Ansible Playbook--- # file: playbook.yml- include: balancers.yml- include: webservers.yml- include: dbservers.yml- include: monitoring.yml

77 #Dynatrace

Example: Ansible Playbook$> ansible-playbook –i production webservers.yml

PLAY [webservers]*******************************************************TASK: [Install Apache HTTP Server]*******************************************************changed: [www0.example.com]changed: [www1.example.com]...

PLAY RECAP *******************************************************web0.example.com: ok=3 changed=3 unreachable=0 failed=0 web1.example.com: ok=3 changed=3 unreachable=0 failed=0...

78 #Dynatrace

Ansible Concepts: Roles

79 #Dynatrace

Ansible Roles» Are the best way to organize a playbook!

» Structure content into related vars, tasks, files, handlers, etc.

» File structure for automated inclusion of role-specific content

» Roles can be shared and pulled from Ansible Galaxy, GitHub, etc.

80 #Dynatrace

Ansible Roles: File Structureansible.cfg

production

staging

webservers.yml

roles/

common/

defaults/

files/

handlers/

meta/

tasks/

templates/

vars/

apache2/

...

81 #Dynatrace

Example: Ansible Playbook--- # file: webservers.yml- hosts: webservers roles: - { role: common } - { role: apache2 } remote_user: deploy sudo: yes

82 #Dynatrace

Ansible: Our Use-Cases

83 #Dynatrace

Use-Case #1Create Dynatrace AMI

84 #Dynatrace

#1 – Create Dynatrace AMI

#1: Remove Existing Dynatrace AMI

85 #Dynatrace


#2: Launch Builder Instance

86 #Dynatrace


#3: Add Persistent Volumes

87 #Dynatrace


#4: Install Dynatrace

88 #Dynatrace


#5: Bundle Dynatrace AMI

89 #Dynatrace


#6: Register Dynatrace AMI

90 #Dynatrace


#7: Clean Up Temporary Resources

91 #Dynatrace

Use-Case #2Create Dynatrace Instance

92 #Dynatrace

#2 – Launch Dynatrace Instance

#1: Get Dynatrace AMI ID

93 #Dynatrace


#2: Launch Dynatrace Instance

94 #Dynatrace


#3: Associate Static IP

95 #Dynatrace


#4: Associate Domain Name

96 #Dynatrace


#5: Launch Database Instance

97 #Dynatrace

Use-Case #3Deploy Dynatrace Agents

98 #Dynatrace


99 #Dynatrace

Test-Driven Infrastructurewith Test KitchenBecause your Infrastructure deserves tests, too!

100 #Dynatrace

What is Test Kitchen?Test Kitchen allows you to test your infrastructure as code

» on multiple platforms in isolation

» supporting a wide range of drivers

» with support for various provisioners

» by using a variety of testing frameworks

» in a pluggable architecture

CentOS, Debian, Fedora, etc.

Vagrant, Docker, EC2, etc.

Ansible, Chef, Puppet

bash, Cucumber, Serverspec, etc.

extend as desired!

http://kitchen.ci/

101 #Dynatrace

Test Kitchen: Configure--- # file: .kitchen.ymlprovisioner: name: ansible_playbook require_ansible_repo: true

driver: name: vagrant customize: cpus: 2 memory: 2048 cpuexecutioncap: 50

platforms: # Opscode vagrant boxes: http://www.vagrantbox.es/ - name: centos-6.5 - name: ubuntu-12.04

suites: - name: dynatrace-agents - name: dynatrace-server

your test suites here!

102 #Dynatrace

Test Kitchen: Ansible Playbook# file: test/integration/dynatrace-server/default.yml

---- hosts: all roles: - role: Dynatrace-Server foo: bar remote_user: vagrant

define a particular role under test

103 #Dynatrace

Test Kitchen: Serverspec# file: test/integration/dynatrace-server/serverspec/spec.rb

require 'serverspec’...

describe user('dynatrace') do it { should exist } it { should belong_to_group 'dynatrace' }end

describe service('dynaTraceServer') do it { should be_enabled } it { should be_running }end

describe port(2021) do it { should be_listening }end

declare your assumptions

104 #Dynatrace

Test Kitchen: List Instances$> kitchen list

Instance Driver Provisionerdynatrace-agents-centos-65 Vagrant AnsiblePlaybookdynatrace-agents-ubuntu-1204 Vagrant AnsiblePlaybookdynatrace-server-centos-65 Vagrant AnsiblePlaybookdynatrace-server-ubuntu-1204 Vagrant AnsiblePlaybook...

let’s test dynatrace-server* !

105 #Dynatrace

Test Kitchen: Test Instances$> kitchen test dynatrace-server

-----> Starting Kitchen (v1.3.1)-----> Cleaning up any prior instances of <dynatrace-server-centos-65> ...-----> Creating <dynatrace-server-centos-65>... Bringing machine 'default' up with 'virtualbox' provider... ... Finished in 0.6908 seconds (files took 0.74047 seconds to load) 24 examples, 0 failures

-----> Starting Kitchen (v1.3.1)-----> Cleaning up any prior instances of <dynatrace-server-ubuntu-1204> ...-----> Creating <dynatrace-server-ubuntu-1204>... Bringing machine 'default' up with 'virtualbox' provider... ... Finished in 0.6947 seconds (files took 0.82016 seconds to load) 24 examples, 0 failures

106 #Dynatrace

Questions?

107 #Dynatrace

108 #Dynatrace

Documents

1#Dynatrace QCon London 2015 Martin Etmajer, Technology Strategist @ Dynatrace Deploying On-Prem as SaaS Why we go with Ansible