The road to continuous deployment (V6-16x9-DevOpsMoscow) · continuous delivery @michieltcs. dev...

THE ROAD TO CONTINUOUS DEPLOYMENT - A CASE STUDY

MICHIEL ROOK

@michieltcs

▸ Developer, consultant, trainer, speaker

▸ @michieltcs

SAY MICROSERVICES

ONE MORE TIME

WORKED FINE IN DEV

OPS PROBLEM NOW

DEVOPS

DEVSECOPS

DEVSECQAOPS

DEVSECQABIZOPS

DEVSECQABIZNETOPS

DEVSECQABIZSYSNETOPS

DEVTESTSECQABIZSYSNETOPS

STORY TIME

@michieltcs

@michieltcs

Large aging monolith

@michieltcs

Large aging monolith

Generates income

@michieltcs

Large aging monolith

Slow & complex

Generates income

@michieltcs

Large aging monolith

Slow & complex

Generates incomeTechnical debt

@michieltcs

@michieltcs

Manual releases

@michieltcs

Manual releases

Fragile tests

@michieltcs

Manual releases

Frequent issues

Fragile tests

@michieltcs

Manual releases

Frequent issues

Fragile testsLow velocity

GOALS

@michieltcs

GOALS

IMPROVE QUALITY

1

@michieltcs

GOALS

IMPROVE QUALITY

REDUCE TIME TO MARKET

1 2

@michieltcs

REFACTOR? REBUILD?

APPROACH

▸ API first

▸ Services per domain object (job, jobseeker, ...)

▸ Migrate individual pages

@michieltcs

ORIGINAL MONOLITH

DB

@michieltcs

ORIGINAL MONOLITH

PROXY

SERVICEORIGINAL MONOLITH

DBDB

DB

@michieltcs

ORIGINAL MONOLITH

PROXY

SERVICEORIGINAL MONOLITH

ORIGINAL MONOLITH

SERVICE SERVICE

SERVICE

PROXY

DBDB

DBDB

DB DB

@michieltcs

ARCHITECTURE

FRONTENDS ARE SERVICES

@michieltcs

ARCHITECTURE

FRONTENDS ARE SERVICES

SERVICES BEHIND LOAD BALANCERS

@michieltcs

ARCHITECTURE

FRONTENDS ARE SERVICES

SERVICES BEHIND LOAD BALANCERS

SERVICES IN CONTAINERS

@michieltcs

IMPROVING

CONTINUOUS EVERYTHING

DEV BUILD / TEST

CONTINUOUS INTEGRATION

@michieltcs

DEV BUILD / TEST ACCEPTANCE PRODUCTION

CONTINUOUS DELIVERY

@michieltcs

DEV BUILD / TEST STAGING / ACCEPTANCE PRODUCTION

CONTINUOUS DEPLOYMENT

@michieltcs

EVERY COMMIT GOES TO PRODUCTION

IF IT HURTS DO IT MORE OFTEN

SMALL STEPS

@michieltcs

SMALL STEPS EARLY FEEDBACK

@michieltcs

SMALL STEPS EARLY FEEDBACK

REDUCE TIME TO RECOVER

@michieltcs

SMALL STEPS

EXPERIMENTS!

EARLY FEEDBACK

REDUCE TIME TO RECOVER

@michieltcs

ONLY COMMIT TO MASTER

NO BRANCHES

NO BRANCHESREALLY.

DELAYING INTEGRATION

CONFLICTS

FEATURE TOGGLES

DECOUPLE DEPLOYMENTSFROM RELEASES

@michieltcs

FEATURE TOGGLE DEBT

CODE REVIEWS?

PAIR PROGRAMMING

CONTINUOUSINLINE CODE REVIEW

BUILD PIPELINE

CONNECT THE DOTS

AUTOMATE REPEATABLE THINGS

CONTINUOUS TESTING

CONFIDENCE

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTS

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCE

UI TESTS

@Test public void jobCannotBeFound() { when(jobRepository.getById(EXPECTED_JOB_ID)) .thenReturn(null); JobService jobService = new JobService(jobRepository); assertNull(jobService.getById(EXPECTED_JOB_ID)); verify(jobRepository).getById(EXPECTED_JOB_ID); }

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTS

@Test public void shouldFindJob() { expectedJob = loadFixture('active_job.yml'); actualJob = repository.getById(expectedJob.getId()); assertThat(actualJob, isA(Job.class)); assertEquals(expectedJob.getId(), actualJob.getId()); }

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTSScenario: Link to related job Given a job exists And there are related jobs available When that job is viewed Then a list of related jobs is shown And each related job links to the detail page of the related job

@michieltcs

DEFENSE IN DEPTH

UNIT TESTS

INTEGRATIONTESTS

ACCEPTANCETESTS

UI TESTS

@michieltcs

UNIT TESTS

INTEGRATION TESTS

ACCEPTANCE TESTS

UI TESTS

SMOKETESTS

Cost Speed

Exploratorytesting Monitoring

@michieltcs

GOOD ENOUGH

PIPELINE SPEED = KEY

DEPLOYMENT

ROLLING UPDATE

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.0 SERVICE V1.0SERVICE V1.0

@michieltcs

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.0 SERVICE V1.0SERVICE V1.0

SERVICE V1.1

@michieltcs

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.0 SERVICE V1.1SERVICE V1.0

SERVICE V1.0

@michieltcs

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.0 SERVICE V1.1SERVICE V1.0

SERVICE V1.1

@michieltcs

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.0 SERVICE V1.1SERVICE V1.1

SERVICE V1.0

@michieltcs

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.0 SERVICE V1.1SERVICE V1.1

SERVICE V1.1

@michieltcs

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.1 SERVICE V1.1SERVICE V1.1

SERVICE V1.0

@michieltcs

LOAD BALANCER

ROLLING UPDATE

SERVICE V1.1 SERVICE V1.1SERVICE V1.1

@michieltcs

PIPELINE AS CODEnode { stage('Run tests') { sh "phpunit" sh "behat" } stage('Build docker image') { sh "docker build -t jobservice:${env.BUILD_NUMBER} ." sh "docker push jobservice:${env.BUILD_NUMBER}" } stage('Deploy staging') { sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER} -i staging deploy.yml" } stage('Deploy production') { sh "ansible-playbook -e BUILD=${env.BUILD_NUMBER} -i prod deploy.yml" } }

@michieltcs

@michieltcs

FEEDBACK!

RESULTS

@michieltcs

Team acceptance

Feature toggle cap

@michieltcs

Team acceptance

Feature toggle cap

Business alignment & agility

@michieltcs

Team acceptance

Feature toggle cap

Business alignment & agility

Focus on replacing legacy

@michieltcs

Team acceptance

Reduced number of

issues

1

@michieltcs

Reduced number of

issues

1

Build time< 10 min.(50+ per

day)

2 4

@michieltcs

Reduced number of

issues

1

Build time< 10 min.(50+ per

day)

Improved metrics & audience statistics

2 3

@michieltcs

Reduced number of

issues

1

Build time< 10 min.(50+ per

day)

Improved metrics & audience statistics

Increased confidence, velocity &

fun

2 3 4

@michieltcs

THANK YOU! @michieltcs / mrook@fourscouts.nl

www.fourscouts.nlwww.michielrook.nl