Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Jean-Paul van Deursen
Wiebe de Roos
ABN-AMRO
Dockerizing the enterprise –fast and secure
Wiebe de Roos
CI/CD Consultant / IT Wizard
• Studied Communication & Multimedia Design and Master of Management & ICT
• 12+ years of IT expertise • CI/CD Consultant / Engineer implementing
Jenkins Enterprise in AWS at ABN AMRO• Lots of expertise about Docker (security) topics• Speaker at ABN AMRO and other industry
conferences
Who are we?
Jean-Paul van Deursen
IT Wizard
• Studied Electrotechnical Engineering @TU Delft
• 20+ years of experience in IT in various roles• Currently active as Wizard in the Center of
Expertise Software Development and Control• Mission: make ABN AMRO the leading digital
bank
What’s on the menu?
• Docker Use Cases @ ABN AMRO• ABN-AMRO – current status of CI/CD• The existing CI platform• Challenges and limitations• Vision of the future• The new & improved CI platform• Docker containers everywhere• Pipelines to fit all use cases• Security• What’s next?• Questions and answers / discussion
Docker Use Cases @ABN AMRO
• PR like Dev provisioning – Shift Left• Mocking dependencies• Encapsulate technical debt• Checkpointing and versioning• CICD Pipeline components (masters/agents)
Use cases for CI/CD
5
Produce automated builds and detect errors as soon as possible, by integrating and testing all changes on a regular (daily) basis.
High frequency delivery of a tested functional piece of software that can be deployed to production rapidly.
Fully automated process including deployment to production without human interaction.
Continuous Integration
Continuous Deployment
Continuous Delivery
Many manual handovers and approvals
Long lead time for software delivery
Software quality issues found at a late stage
Code merging happening at a late stage
Inefficient cooperation between DEV and OPS
Big non-frequent releases to Production
It is not only about tooling but mainly mindset & behavior, a changed Way of Working and process improvements.
• Increase maturity of teams • Set up the conditions (tooling, pipelines, generic building blocks) for the teams to get
working.• Train the blocks on applying the right mindset, knowledge and appropriate tooling
We know other large companies which need 3 - 8 years, and changed their approach along the way.
Therefore we keep the overall stages in mind, but plan for the coming three months. Focus on learning and improving instead of long term planning.
CI/CD pipeline orchestration midrange
7
Dependency scan
Check out project from
SCM
Developer triggers build
Build project and execute unit
tests
Code quality scan
Secure coding scan
Publish Deployable
artifact
N
Y
ABN AMRO has introduced a set of quality gates and build breakers. The principle is that the Jenkins build is broken once therequired quality or security is not met and the developer needs to fix the defect in order to proceed. The developer has access to software quality in his IDE so defects can be detected and fixed in an early stage
Standard CI pipelines and buildbreakers
Existing CI platform – Jenkins on VMs• Statistics:
• +/-1500 users• 350+ projects• 10000+ Jenkins jobs
• 1 Jenkins Operation Centre• 10 Jenkins Masters
• 30+ Linux build slaves• 30+ Windows build slaves• 4 OSX build slaves• 25+ HP-fortify (secure coding) slaves
70+ (!!!) VMs in on-prem datacentre…and GROWING…
Challenges and limitations – how to…?• Hard to handle growth of DEV teams.• A lot of static VMs, constantly upscaling needed. • Hard to maintain all the servers.• Server configuration out of sync.• No Docker container support.• No true team autonomy.• A mix of tools and versions on
each build slave.• Innovation is slow.
Five major improvements
1. Empower the CI/CD teams: decentralized maintenance.
2. Docker containers instead of static VMs
3. Support flexibility of tech stacks and configuration.
4. Infrastructure as Code & Configuration as Code.
5. Cloudbees Jenkins Enterprise is critical to the CI/CD program
The new and improved CI platform
AWS
CMS CI
Master Slave
CD
CI
The new and improved CI platform - architecture
Teams can create their own Jenkins master and run their own pipelines.This solution prevents interference of teams with each other. Reduces conflicts.
Context of containers in Jenkins Enterprise
1. Platform2. Running Jenkins jobs3. Build containers4. Application containers
Specific
Generic
Use case: Jenkins Build agents (containers)
Have a proper
solution for the configuration
difficulties
A never ending story…
Pipelines - overview• Q1 2017: Birth of the standard pipelines (STPLs)• Lots of benefits but also challenges• Q1 2018: Birth of the new (Dockerized) pipelines:
• A pipeline for Docker images• A Dockerized pipeline for Java applications
• Easy to use, easy to implement & extend• Security is build-in• A reference for other technologies
Docker image pipeline – main building blocks
16
Smoke test
Jenkinsfile + Dockerfile from
SCM
Developer triggers Docker
image build
Build Docker image
Docker lint syntax check
Docker container dependencies
check
Sign + Publish Docker image in trusted registry
N
YDocker container configuration
check
Apply security profiles
Pipelines – Docker image pipeline
• A pipeline which creates Docker images• That are secure• That are versioned and tested• Which are “official” and “approved”• Ready to re-use by DEV teams
Pipelines – Java pipeline Dockerized
• A pipeline which uses Docker images (building blocks from previous pipeline)
• Create Java artefact• Package in Docker image• Security stages in place• Push to registry• Ready to deploy to (Xlrelease/Xldeploy, AWS, Kubernetes)
Docker Security topics on all levels
Security is needed on every level
Security – why all this?
To avoid compromised containers where-ever they are used: secure business continuity
Security (1): Syntax check
v1.6.2-6-gcfb547a: Pulling from hadolint/hadolintStatus: Downloaded newer image for hadolint/hadolint:v1.6.2-6-gcfb547a/dev/stdin:3 DL3005 Do not use apt-get upgrade or dist-upgrade/dev/stdin:3 DL3009 Delete the apt-get lists after installing something/dev/stdin:4 DL3008 Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`/dev/stdin:4 DL3015 Avoid additional packages by specifying `--no-install-recommends`Docker lint syntax check (just like SonarQube
Security (2): Anchore
Security (3): Sonatype - Nexus Lifecycle
Security (4): Docker benchmark (OSS)
What’s next - roadmap
CJE to PR
Finish Dockerizedpipelines
Onboard 50 teams this year
Docker runtime scanning
Choose a container runtime on AWS
PoC for a small number of innovative teams
Enterprise based solution for all DEV teams
Questions and Answers
Thank you!
Wiebe de Roos – [email protected] Paul van Deursen – [email protected]