DevOps  in  prac-ce  –  How  to  keep  focus  on  your  product?  

André  Christ  Startup  Camp  Berlin,  March    2015  

Hint:  The  former  capital  ;-­‐)  

2  Photo:  Michael  Sondermann/Bundesstadt  Bonn  

About  

3  

André  Christ  •  Co-­‐Founder  &  Co-­‐CEO  •  Responsible  for  IT  &  Finance  •  Before:  DHL,  HTGF,  Startups  

LeanIX  GmbH  •  SaaS-­‐Provider  for  IT  Architecture  Mgmt  •  Self-­‐Funded,  Founded  2012  in  Bonn  •  >  20  Customers  in  US,  Europe  and  Australia    

What’s  your  level  of  experience  with  these  tools?  

4  

“Build,  ship,  and  run  distributed  applicaZons”  

“Building/tesZng  so\ware  projects  conZnuously”  

“IT  automaZon  engine  that  automates  applicaZon  

deployment,  configuraZon  management,  ….”  

Ansible  and  Docker  clearly  connected  to  the  DevOps  “Hype”  …  

5  Source:  Google  Trends  

DevOps  Docker  Ansible  

…  while  Jenkins  is  really,  really  mature  and  widely  adopted  by  all  company  sizes  

6  Source:  Google  Trends  

DevOps  Docker  Ansible  Jenkins  

Agenda  

7  

Docker  

Ansible  

Jenkins  

Mo-va-on  –  How  to  keep  focus  on  your  product?  

Summary  /  Learnings  

Challenge  1:  Serve  high-­‐profile  customers  worldwide  with  very  limited  resources  

8  

Challenge  2:  Scale  the  plahorm  with  the  right  tools  for  the  job  

2012   2014  2013  

“MVP”  SSO  &  Micro  services  

Professional  Build  Process  

Service-­‐  &  Quality  Ensurance  

2015  

Single  Page  JS  App  

Challenge  3:  B2B-­‐SaaS  requires  flexible  deployment  models  –  Cloud  and  On-­‐Premise    

10  

Different  cloud  environments  

•  Amazon  Webservices  (US  &  Australia)  •  Strato  (Germany)  •  Further  providers  planned  (e.g.  Azure)  •  …  

Individual  plaPorms  per  company  

•  Ubuntu  Linux  •  Redhat  •  Windows  •  ….    

11  

Write  proper  documenta-on  and  manuals?!  

Does  not  solve  our  issues*  

•  High  effort  to  test  •  Not  repeatable  •  Outdated  very  soon  •  Prohibits  scale  •  Not  customer  friendly  (e.g.  on  premise)  

*)  Does  not  mean  we  don’t  write  proper  documentaZon  where  needed  J  

ProducZon  Servers  US  ProducZon  Servers  US  

Our  solu-on:  Highly  automated  build,  test  and  deploy  tool  chain  

12  

Develop   Build   Test   Deploy  

Staging  Server  

ProducZon  Servers  Europe  

Developer  Machines  

Real-­‐life  example:  LeanIX  Synclog  micro  service  

13  

Track  synchroniza-on  results:  Success,  errors  etc.  of  all  LeanIX  3rd  party  integraZons  

Synclog  Microservice  

Synclog  Architecture:  RESTful  micro  service  with  database  and  Single  Page  App  GUI  

DB  

Synclog  REST-­‐API  

Synclog  GUI   Angular.JS  Nginx  Webserver  

Postgres  

Java  Dropwizard  

API  

Agenda  

14  

Docker  

Ansible  

Jenkins  

MoZvaZon  –  How  to  keep  focus  on  your  product?  

Summary  /  Learnings  

How  does  Docker  compare  to  virtual  machines?  

15  

Server  

Host  OS  

Hypervisor  

Server  

Host  OS  

Docker  Engine  

Guest  OS  

Bins/Libs  

App  A  

Guest  OS  

Bins/Libs  

App  B  

Svc  1   Svc  2  

Bins/Libs  

Virtualisa-on   Docker   Advantages  

•  Significantly  faster  deployment  

•  Much  less  overhead  •  Easier  migraZon  

•  Faster  restart  &  cleanup  •  Fits  perfect  to  micro  services  architectures  

Topics  to  be  adressed  

•  IsolaZon  /  Security  •  Requires  Linux  Kernel  (Workarounds  via  boot2docker)  

•  Maturity  of  tools  

VM   VM  

Container   Container  

Svc  3  

Container  

Bins/Libs  

What  is  a  Docker  container  and  image?  

16  

Docker  Client  

Host   Docker  Registry  

Docker  Deamon  

Container  1  

Container  2  

Container  3  

Image  1  

Container  N  

Image  2  

Image  3  

Image  4  

Docker  API  

Private  running   stopped  

Simple  example:  Run  a  Database  

17  

Docker  Client  

Host   Docker  Registry  

Docker  Deamon  

mydb  

leanix-­‐synclog  

postgres:9.4  

nginx  

leanix-­‐synclog  

// Start a Postgres DB $ docker pull postgres:9.4 $ docker run –name mydb postgres:9.4 $ docker stop mydb // Create a new image $ docker build –-tag=“leanix/leanix-synclog” . $ docker push leanix/leanix-synclog

Docker  API  

…  

Private  running   stopped  

Docker  Registry  (Docker  Hub)  

18  

•  100  „Official“  images  •  >  45.000  public  images  •  SubscripZon  model  for  private  images  (similar  to  GitHub)  

Our  example  is  cut  into  3  containers  

19  

DB  

Synclog  REST-­‐API  

Synclog  GUI   Angular.JS  Nginx  Webserver  

Postgres  

Java  Dropwizard  

Example   Docker  Images  

postgres:9.4  

leanix-­‐synclog  

leanix-­‐synclog-­‐client  

Synclog  service  in  context  of  enZre  leanIX  architecture  

20  

Reverse  Proxy  (nginx)  

eam    

(Core  App)  

mtm    

(User  Mgmt)   synclog    

expo

rt  

web

hooks  

noZfi

caZo

ns  

images  

….  

Requests  

•  Auth  between  Services:  oAuth2  •  Single  Sign  on  (with  customer)  SAML  2.0  

Create  custom  docker  image  based  on  exisZng  images  

21  Source:  Docker  

FROM dockerfile/java:oracle-java7 # Copy JAR and config into container COPY target/leanix-synclog-0.1.0 / COPY config.yml / COPY run.sh / RUN chmod +x /run.sh # Expose port 9100 and run java process EXPOSE 9100 WORKDIR / # Run command CMD ["./run.sh"]

leanix-­‐synclog/Dockerfile  

$ docker build –-tag=“leanix/leanix-synclog” . $ docker push leanix/leanix-synclog

How  to  Ze  everything  together?  docker-­‐compose  (was:  fig)  

22  

api: build: . links: - db ports: - "9100:9100” db: image: postgres

docker-­‐compose.yml  

$ docker-compose up $ docker-compose stop

#!/bin/bash # Replace address sed -i "s/localhost:5432/ $DB_1_PORT_5432_TCP_ADDR:5432/" config.yml java -jar leanix-synclog-0.1.0.jar db migrate config.yml java -jar leanix-synclog-0.1.0.jar server config.yml

leanix-­‐synclog/run.sh  

Docker  UI  –  Get  visual  overview  and  controls  for  Docker  (made  with  Angular)  

23  

$ docker run -d -p 9999:9000 --name dockerui --privileged -v /var/run/docker.sock:/var/run/docker.sock dockerui/dockerui

Keep  an  eye  on:  Giant  swarm  Docker  orchestraZon  made  easy  

24  

{          "app_name":  "leanix-­‐synclog",          "services":  [{                          "service_name":  "leanix-­‐synclog",                          "components":  [                                {                                          "component_name":  "leanix-­‐synclog",                                          "image":  ”leanix/leanix-­‐synclog",                                          "ports":  [  ”80/tcp"  ],                                          "domains":  {  "synclog-­‐andrechrist.giganZc.io":      "80"  },                                          "dependencies":  [  {                                                          "name":  "db_1",                                                          "port":  5432}]                                  },                                  {                                          "component_name":  "db_1",                                          "image"  :  "postgres:9.4",                                          "ports":  ["5432/tcp"]                                  }                          ]                  }          ]  }

$ swarm login $ swarm up $ swarm logs <ID> $ swarm stop $ swarm scaleup <COMP>

swarm.json  

Agenda  

25  

Docker  

Ansible  

Jenkins  

MoZvaZon  –  How  to  keep  focus  on  your  product?  

Summary  /  Learnings  

Jenkins  Overview  

26  

Simple  

Extensible  

Scalable  

Flexible  

Open  Source  

Community  

Good  Web  GUI  

ConZnuous  IntegraZon  (CI)  Server  

Thousands  of  plug-­‐ins  

Distributed  BUILD  System  

ConfiguraZon  of  Jobs  

MIT  License  

Very  acZve  (since  2011)  

Jenkins  Job:  leanix-­‐synclog  

27  Image  Source:  robots.com  

Trigger:  GitHub  Push    1.  Builds,  e.g.  using  

maven  (Java)  2.  Executes  Unit  Tests  3.  Creates  Docker  image  4.  Pushes  Docker  image  

to  Docker  Hub  5.  Deploys  to  staging  

system  (via  Ansible)  

Agenda  

28  

Docker  

Ansible  

Jenkins  

MoZvaZon  –  How  to  keep  focus  on  your  product?  

Summary  /  Learnings  

Ansible  Overview  

29  

YAML  Syntax  

Library  of  modules  

Switch  from  ssh  push  to  pull  

EncapsulaZon  by  roles  

MIT  License  

Strong  Growth  (since  2012)  

OrchestraZon  of  Admin-­‐Tasks  

Simple  

Extensible  

Scalable  

Flexible  

Open  Source  

Community  

Automate  repeatable  admin  tasks,  e.g.  install  packages,  set  configs,  ...  

Image  Source:  robots.com  

Ansible  Architecture  

30  

Tasks  

Install  Package  

Copy  file  

Apply  template  

...  

Deploy  synclog  Apply  template  

Run  docker  

Roles  

init  

...  

docker  

Playbooks  

Provision  Server  

leanix-­‐synclog  

...  

postgres  

Hosts  

SVR-­‐DE-­‐TEST  

SVR-­‐DE-­‐DEV  

SVR-­‐DE-­‐WEB-­‐1  

SVR-­‐US-­‐WEB-­‐1  

via  ssh  

...  

Ansible  Example:  Provision  servers  

31  

# Provisions the frontend servers --- - hosts: frontend sudo: true roles: - {role: 'init'} - {role: 'docker'} - {role: 'updates'}

provision_server.yml  

$ ansible_playbook provision_server.yml –I hosts/prod -v

hosts/prod  

[frontend] srv-de-web-1.leanix.net srv-de-web-2.leanix.net srv-us-web-1.leanix.net srv-us-web-2.leanix.net [backend] srv-de-app-1.leanix.net srv-de-app-2.leanix.net srv-us-app-1.leanix.net srv-us-app-2.leanix.net

Ansible  Example:  Configure  system  

32  

[…] - name: Install System Packages apt: pkg={{ item }} state=latest with_items: ["curl", "wget", "python-software-properties", "software-properties-common", "daemon", "supervisor"] - name: German kb command: loadkeys de changed_when: false - name: Set hostname on boot to short name from inventory list template: src=hostname.j2 dest=/etc/hostname owner=root group=root mode=0644 register: hostname_file […]

roles/init/tasks/main.yml  

Ansible  Example:  Deploy  service  

33  

[…] - name: Start leanix synclog Docker container shell: docker run -d

--name {{ item.1.name }} -p {{ ansible_eth1.ipv4.address }}:{{ item.1.synclog_port }}:{{SERVICE_PORT }} -e SERVICE=synclog -e PROXY_SERVICE={{ PROXY_SERVICE }} -e SWAGGER_BASEPATH={{ SWAGGER_BASEPATH }}

leanix/leanix-synclog with_indexed_items: SYNCLOG_SERVICES when: synclog_running.results[{{ item.0 }}].rc != 0 […]

roles/init/tasks/main.yml  

Agenda  

34  

Docker  

Ansible  

Jenkins  

MoZvaZon  –  How  to  keep  focus  on  your  product?  

Summary  /  Learnings  

Key  Issues  &  SoluZon  Approaches  

35  

•  Docker:  Usage  of  boot2docker  on  non  Linux  (OS  X,  Win)  

•  Docker:  Container  orchestraZon  on  producZon  systems  with  distributed  docker  deamons  

•  Ansible:  v1  not  running  on  Win  

•  Ansible:  For  some  tasks,  special  python  libs  need  to  be  on  the  target  host  

Contribute  to  helper  tools  ;-­‐)  e.g.  b2d-­‐sync  (node.js  tool)  

Create  custom  soluZons  based  on  Ambassador  pa~ern  and  etcd  orchestrated  by  Ansible  

Use  Vagrant  or  get  rid  of  Windows  ;-­‐))    Check  in  tasks,  if  libraries  are  already  installed  

Summary  

36  

20  days  

3  hours  

<  30  min  

Effort  for  setup,  learning  and  trouble  shooZng  for  Ansible  &  Jenkins  

Ramp-­‐Up  of  a  new  employee  unZl  first  commit  

Time  unZl  a  fresh  Server  is  setup  with  all  required  services  

14  days   For  changing  our  Vagrant  based  environment  to  docker  &  docker-­‐compose  

37  

THANKS!  Follow  us  @leanix_net  

Download  Ansible  Example  Code  github.com/leanix/codetalks_2014_demo  

We  are  hiring!!!  •  Frontend  Developers  •  Backend  Developers  •  MarkeZng  &  Sales  

www.leanix.net  

Backup  

38  

Growth  and  DigitalizaZon  leads  to  increasing  IT  complexity  

39  

-me  

Growth  /  Digitaliza-on  

20-­‐30  ApplicaZons  

50-­‐80  ApplicaZons  

>  150  ApplicaZons  

Lacking  transparency  of  the  IT  landscape  is  a  strong  barrier  for  innovaZon  

40  

35  b€  overspent  due  to  redundant  applicaZons    

Source:  Detecon  /  McKinsey,  15%  of  applicaZons  can  be  consolidated  

Inefficient  processes  due  to  head  monopoly  

leanIX  is  a  smart  plahorm  which  helps  companies  to  opZmize  their  IT  landscape  

41  

Fact  Sheets  &  Tagging  

Context-­‐based  Search  

API,  Import  &  Export  

Comments  &  Threads  

IT  Inventory   Collabora-on  PlaPorm   Interac-ve  Repor-ng  

AcZvity  Stream  &  NoZficaZons  

SubscripZon  

Print  &  Export  (PDF)  

Best  PracZce  Reports  

InteracZve  AdapZon  

leanIX  connects  the  dots  in  the  ecosystem  of  modern  IT  management  

42  

Collabora-on-­‐  &  Document  Mgmt  

IT  Service  Management  &  Help  Desk  

Requirements-­‐  &  Business  Process  Mgmt  

Project  PorPolio-­‐  and  Ressource-­‐Management  

Selected  products  to  illustrate  

Out-­‐of-­‐the-­‐box  IntegraZonen  

REST  API  &  SDKs  

developer.leanix.net  

Partnering  /  IntegraZon  

43  

Sync