Embed Size (px)
Citation preview
You don’t need DTAPDevOps and Continuous Delivery at Backbase
DevOps Amsterdam meetup, 17-12-2015
Backbase, Amsterdam
Pavel Chunyayev
@PavelChunyayev
Amsterdam
Levi9 HQAmsterdam – 2005
25 people
Novi Sad
SerbiaNovi Sad – 2005
320+ people
Zrenjanin
SerbiaZrenjanin– 2014
30+ people
Iasi
RomaniaIasi – 200780+ people
Kiev
UkraineKiev – 2008130+ people
@PavelChunyayev
Electronic
Retail
Digital
Marketing
Traffic and
Transport
Software
Services
@PavelChunyayev
Customer satisfaction 2015
• 4th year great results in Outsourcing Performance study
• 2015: 100% customer recommendation scoreExcellent trust score
@PavelChunyayev
Forester Wave: Omni channel Banking Solutions Q3‘15
@PavelChunyayev
Gartner Magic Quadrant for Horizontal Portals 2014
@PavelChunyayev
Top 10 companies in NL with best work-life balance
@PavelChunyayev
About me
• 11 years of IT experience
• Lived and worked in Ukraine and Estonia
• Moved a year ago to the Netherlands
• Learning Dutch
• Love cycling
• De Holandse 100• https://www.dehollandse100.nl/actie/pavel-chunyayev
Continuous Delivery
Safely, rapidly and predictably deliver new features to production
Quality > Speed
DTAP
@PavelChunyayev
Journey across environments
Development Testing Acceptance Production
@PavelChunyayev
DTAP is history
• It was hard to create infrastructure -> we had to reuse it
• Manually provisioned environments• Took a lot of time
• Ticket to system administrator
• Nightmare for both parties• End up with golden images
• Fixed set of environments• Name them
• Configure once
• Maybe even automate deployment
@PavelChunyayev
DTAP is evil
• Configuration drift• Environments have nothing to do with production
• With time they move more and more apart
• Dirty environments• Undocumented, forgotten hacks
• No one can recreate environment as no one knows how to configure them
• Testing is complicated• Different bugs on different environments
Downwards spiral
You don’t need DTAP
Immutable infrastructure
Immutable infrastructureImmutable delivery
Easy on-demand environments
@PavelChunyayev
Change in paradigm
• Change in paradigm of the way software is built, shipped and managed
• No more static infrastructure
• Provision and create infrastructure just in time when it’s needed
• And then throw it away
Create • Use • Dispose
@PavelChunyayev
Never change again
• Never touch or change the running system. • No configuration changes
• No complicated update processed
• Any change = new deploy
• Create infrastructure right before deployment
• You always start from the scratch• Identical process
• No more dirty environments
• Does it matter how environment is called if the deployment process is identical?
@PavelChunyayev
Variability
• Variability is enemy
• Manufacturing: Six sigma
• How to deliver environments at scale?
@PavelChunyayev
Prerequisites
• Cloud • Manage infrastructure lifecycle as code / through API
• Docker containers
• Automate the setup and deployment of your application
@PavelChunyayev
Continuous Delivery
• Immutable infrastructure enables CD
Build • Test • Deploy
• When you want to change - go through the whole pipeline
@PavelChunyayev
Infrastructure is integral part of application
• Application is delivered as a set of immutable virtual machines or containers.
• Any change in the application is actually a change of infrastructure
• Exactly the same process to deploy any version of the application
• MTTR is predictable • It’s time required for whole pipeline execution
• Gives incentive to optimize the process in order to speed it up
@PavelChunyayev
Immutable infrastructure applies to
• Stateless applications• State is stored outside (DB)
• Even more relevant for microservices
• Pushing of responsibility to someone else
• Databases
• Separate DBMS from data
• There are solutions for this (e.g. Flocker)
@PavelChunyayev
Deliverables
• Server images (AMIs) or containers should be built using automated CI/CD process• Base image + application = deliverable
• Or the configuration process and deployment scripts are applied over and over again
@PavelChunyayev
Deliverables
• Create your own images, don't reuse someone else's
• Images must be versioned and all history must be saved
• Implement automated testing of infrastructure
• Create building blocks
• Test deployments without any fear• Deployments are automated
• Identical deployment procedures to all environments
• Green/blue, canary and rolling deployments
@PavelChunyayev
Immutable infrastructure
• Get infrastructure exactly when it’s needed
• Dispose immediately after use
• Never regret
Continuous Delivery at BackbaseAnd immutable infrastructure
@PavelChunyayev
Backbase
• Research and Development
• Professional Services
Customer Experience Platform
Launchpad Mobile SDK
Digital Banking Services
Digital Banking Platform
@PavelChunyayev
Publishing
• Multiserver configuration is needed to test publishing
Editorial Staging Live
@PavelChunyayev
Dual deployment model
• Cloud Native first
• But bank are not ready
• Dual deployment model• Containers/cloud – modern
• Enterprise style
@PavelChunyayev
Teams structure
• Squads• Team is divided into squads
• Each squad is responsible for one or more components
• Components• Components = microservices
• >10 BE components
• Dozens of FE components
• Continuous Delivery squad is a Platform team
@PavelChunyayev
Platform team
• Now consists of 4 people, 5th starting in January
• You can be part of this team
• Several generations of platform• We are in the middle of introduction 4th generation
@PavelChunyayev
Generations
1. Infrastructure as a service MVP
2. All configurations for CXP 5.6 and testing at scale
3. Docker and microservices support
4. Unified way of delivering infrastructure (being rolled out)
@PavelChunyayev
Development environments
• Docker containers = building blocks• Docker Machine
• Docker Compose
• No need to use virtual machines
• No need to use jetty and h2
• No need to start whole environment
@PavelChunyayev
Building blocks
• Ansible roles
• Packer
• Infrastructure testing• Before – Starting full environments
• Now – Testing roles in isolation• Test-kitchen
• Serverspec
@PavelChunyayev
Delivery pipelines
git Build Version Test Release
Component 1
git Build Version Test Release
Component 2
git Build Version Test Release
Component 3
Assemble CXP
Test Release
342
128
473
921
@PavelChunyayev
Pipelines as code
• Pipelines as code• Changes -> same process as code changes
• First stage is enforcing specifications
• GoCD + Gomatic (updated)• https://github.com/Backbase/gomatic
@PavelChunyayev
Platform
• Services to manage environments lifecycle
• The same way for everyone
• Autoconfig• AWS
• Autodocker• Docker
@PavelChunyayev
Autodocker
• Swarm cluster• Interface for docker environments provisioning
• Limitations• One host for networking
• Memory checks
• Destruction after TTL
• Configuration is a docker compose file
• Start exact version of your component and released version of all other components
@PavelChunyayev
Autoconfig
• AWS on-demand multi-server environments• Any application version• Any configuration option
• JSON (Form data) with parameters{
"server": "jboss",
"database": "oracle",
"universal_collection": "true",
"htts": "true"
}
@PavelChunyayev
Autoconfig APIs
• GET /api/stacks - List stacks available for provisioning
• GET /api/environments - List all currently provisioned environments
• POST /api/stacks/<stack_name> - Provision specified stack
• DELETE /api/environments/<environment_id> - Destroy environment with specified id
• DELETE /api/environments/all - Destroy all environments
@PavelChunyayev
Autodocker APIs
• POST /api/docker - create new docker environment
• GET /api/docker/<env-name> - get info about specified environment
• GET /api/docker - get info about docker environments provisioned by the user Optional parameters:• all_users = True • uptime (value in hours) • owner (AD username)
• DELETE /api/docker/<env-name> - delete specified docker environment
• DELETE /api/docker - remove docker environments provisioned by the user Optional parameters:• all_users = True • uptime (value in hours)• owner (AD username)
@PavelChunyayev
Quality
• Build the quality in the process
• Move tests to the left
• Testing pyramid
• No more manual testing
• Ideal testing framework • Manage parallelism
• Create environments
M
E2E
Contract
Component
Integration
Unit
@PavelChunyayev
Releasing
• Banks don’t want frequent releases
• 3 stage release process• Rolling
• In the effort to not release too infrequently
• Technical previews• At the end of every sprint
• Marketing releases• When product manager decides to release
@PavelChunyayev
Culture
• No blaming, no fingerpointing
• Failing the build is a learning opportunity
• We failed a lot• Several iterations of JS deployments
• Visibility
• Shared responsibility• Squad – for component delivery
• Tribe – for product delivery
@PavelChunyayev
Key takeaways
• Forget about DTAP
• Believe in immutable infrastructure
• Create repeatable and reliable process for releasing software
• Build quality in
• Improve continuously
Any questions?
