Upload
techwellpresentations
View
160
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Continuous delivery is a practice that enables teams to release code at any time, based on changing business requirements. However, continuous delivery requires a substantial investment in infrastructure and possibly fundamental architectural changes to support the process. Anti-patterns that would render a continuous delivery pipeline a burden rather than a beneficial tool for continuous delivery must be avoided. Seng Lin Shee describes how Ancestry.com has transitioned from small scale web deployments to rapidly deploying new features without compromising product quality—all while not adding more complicated processes in the development lifecycle. A disciplined team is necessary to successfully use a delivery pipeline to its utmost potential. Seng Lin shares his team’s experience in the development of pipeline testing processes, introducing different sets of environments for different testing purposes, isolation testing, and test case management in a continuous delivery environment. Take away key ideas for implementing continuous delivery in your organization.
Citation preview
BW11 Session 6/5/2013 3:45 PM
"Continuous Delivery at Ancestry.com"
Presented by:
Seng Shee Ancestry.com
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Seng Lin Shee Ancestry.com
Leading the testing effort for the API team of Ancestry.com, Seng Lin Shee educates the team and defines test strategies and direction of the testing effort. He has introduced the team to behavior-driven design and test-driven design methodologies, and contributed to the design of the continuous delivery pipeline within the company. Seng Lin’s experience ranges from software development and testing, to router/switch manufacturing, to microprocessor architecture research. Seng Lin was formerly a software development engineer in test at Microsoft where he worked on Windows Server 2008 R2, Active Directory Federation Services, and Access Control Services.
Continuous Delivery at Ancestry.comSeng Lin Shee
Take Away from this Presentation
• Understanding of Continuous Delivery
• Role of Continuous Delivery in web services
I h A• Impact on the teams at Ancestry.com
• Strategies for an effective Continuous Delivery pipeline
• Changes in responsibilities and expectations
2
Agenda
1. What is Continuous Delivery
2. Continuous Delivery Architecture
3 S f Lif l i C i D li3. Software Lifecycle via Continuous Delivery
4. Testing Methodology
5. Roles and Responsibilities (Mindsets)
6. Learnings / Pitfalls
3
The Issue…
• Traditional deployment strategy of web services:– Involvement of entire engineering team– Manual monitoring– Manual test execution– Tests executed within unstable environment
4
Value Stream
• Flow of materials and information required to bring a product or service to a consumer
• Value stream map: value add and wait timep– Product concept– Product discovery– Product planning– Development– Testing
5
– Release
• We need to be more Agile!
Philosophy
• “The essence of my philosophy to software delivery is to build software so that it is always in a state where it could be put into production. We call this Continuous Delivery because we are continuously runningDelivery because we are continuously running a deployment pipeline that tests if this software is in a state to be delivered.” (Martin Fowler)
6
What is Continuous Delivery?
Continuous Deployment
Continuous Delivery
Continuous Integration
7
Cycle Time
• How to optimize cycle time– Teach developers how to write production‐ready code– Running Continuous Integration and tests on production‐like systems
– Implement an Agile development process – Instituting cross‐functional teams– Automating the build, test and release process– Implement Continuous Delivery using a Deployment Pipeline
8
Learning Release Management
• Continuous Delivery is about keeping your software in a state where it could be released into production at any time AND being able to actually release it when the business wants to.business wants to.
9
Release Management Principles and Practices
• Release small changes frequently rather than large changes infrequently
• Automated Deploymentp y
• Automated Configuration Management– OS, services, applications, everything– No more manual configuration
• Automated processes are self documenting
• Automated change tracking to production resources
• Automated deployment smoke tests (test the deploy)
• Automation, automation, automation…10
Tooling
• Provides the following services– Distributed build systemDistributed build system– Continuous integration– Workflow engine (deployment pipeline)– Job control system– Dashboard (web application)
• Hub spoke design multi platform• Hub‐spoke design, multi‐platform– Single Go server– Multiple Go agents (one or more per machine)
11
Continuous Delivery Architecture
Traditional Deployment Environments
DEV Environment STAGE Environment LIVE Environment
13
Version Control System
Towards Continuous Delivery
Tests critical functions
Tests remaining functionalities
Tests all functionalities
Build, Unit Tests & Packaging
14
Acceptance Preproduction LIVEBuild
Then & Now
NOW• All environments
are provided with isolated boxes for testing
• Weekly rolls take no more than 15 minutes
THEN• DEV & STAGE
environments are easily broken
• Weekly rolls take hours to complete testing
15
Maturity of Pipeline
Rolls are Continuous Continuous Automatic roll
MATURITY LEVEL
No
pipe
line
Rolls are performed in a concerted effort every fortnight
Basi
c te
sts
are
in
plac
ed in
pip
elin
e
Continuous Delivery pipeline rolls binaries every fortnight
ke t
ests
and
mor
e ro
bust
tes
ts
Continuous Delivery pipeline rolls with tight supervision
depl
oym
ent
issu
es
afte
r 1
mon
th
Automatic roll during business hours
p
Smok
No
d
16
Software Lifecycle via Continuous Delivery
Kanban Principals
• Very much suited for our release cycles
• Favored by Project Managers
A i DONE h f i DEPLOYED• A story is DONE when a feature is DEPLOYED to a particular ENVIRONMENT
BACKLOG IN DEVELOPMENT DONE
18
Testing Methodology
Smoke Tests during Deployment
Internal smoke tests
Dependency Dependency smoke tests
20
Profile of Tests at each Quality Gate
Tests critical functions
• Acceptance Tests
• Regression Tests
P f T• Performance Tests
• Tests are categorized into– READY– QUARANTINE– REGRESSION
Acceptance
21
Acceptance
GO Pipeline FAQ
• READYThese are tests that are in the Acceptance Test project which are ready, but with stories not completed. These tests will not block the pipeline We should be observing this group of testblock the pipeline. We should be observing this group of test
• QUARANTINEThese are tests which are known to fail, due to a certain bug (which will take time to fix). We do not mark them as ignored because these are still valid tests.
• REGRESSIONThese are normal test that should pass 100% of the time. If they fail, please retry the respective block of test.
22
Tests Parallelism
• Pipelines should be made as wide and as short as possible
• GOAL– Reduce time for testing
• CHALLENGES– Tests should not interfere with one another– Tests should not depend on sequence of execution– Non‐performance tests should not overload server– Non‐performance tests should not overload server
• SOLUTION– Parallelism via namespaces / test categories / test classes
23
Roles and Responsibilities (Mindset)
Continuous Integration Key Principles and Practices
• Maintain a Single Source Repository
• Everyone Commits To the Mainline Every Day
A FAST lf i b ild ( f h M i li• Automate a FAST, self‐testing build (of the Mainline check‐ins) on an integration machine
• Test in a Clone of the Production Environment
• Make it Easy for Anyone to Get the Latest Executable
E h ' h i• Everyone can see what's happening
25
Continuous Delivery Key Principles
• Single source code branch (e.g. trunk/mainline)
• Single build (no DEV and PROD build definition)
D l h i• Deploy the same way to every environment
• Configuration management
• Infrastructure as code
# of tests
Execution time
26
Pipeline progress
# of tests
Continuous Delivery Values and Practices
• Automated testing and deployment
• Workflow handles check‐in to release (deployment pipeline)p p )– Pipeline execution instances are versioned– Pipeline output becomes an artifact in repository
• Teams own their build and workflow– Use Ant/NAnt script to define build – Use Continuous Integration tool to design workflow– Use Continuous Integration tool to design workflow– Elements: pipeline group, pipeline, stage, job and task
27
Continuous Delivery Anti‐Patterns
• Multiple branches
• Feature branches that are long‐lived (> 1 day)
C i ki l h 10 i• Commit tests taking longer than 10 minutes
• Team culture allows ignoringwhat happens after check‐in
• Multiple builds in a pipeline
28
Developers Own the Releases
• Developers are testers
• Responsible to address checked‐in builds that fail in any pipeline stagesp p g
• File bugs, mark tests are quarantined and move on with development
29
Broken Pipelines happen…stuff happens…
• Run local builds with unit tests first if your check‐ins are constantly causing build breaks (pipeline breaks)
• Breaks are transparent; fix within 10 minutes, or roll backp
• “Always in a state where it could be put into production”
30
Learnings and Pitfalls
Warm up time of services
• A .NET Web Service (IIS hosted services) cannot handle requests immediately upon deployment.
• Loading and compiling of .NET assemblies on first g p grequest.
• The DEPLOY job has been configured to performed smoke tests which exercises critical areas of code, a cluster at a time.
• Product code has to be engineered for:Product code has to be engineered for:– Quick boot up time– Robust against frequent recycling of application pools
32
Extreme Long Release Latency
• A deployment pipeline has the following goal– Increase throughput via simultaneous execution of stages– Early fault detection
• Longer delay between CheckIn’s and Releases due to– Increase in the number of quality gates– Repetition of the exact tests in differing quality gates– Longer sequential tests
• This impacts the Time To Resolve when a quick fix is• This impacts the Time‐To‐Resolve when a quick fix is needed
33
Backward and Forward Compatibility
• Deploy at anytime
• Release features independently
B ki h f il d CD Obj i• Breaking change ‐> failed CD Objective
• Invoke test suites of dependent components
34
Conclusion
Conclusion
• Continuous Delivery ensures that code is always ready for release when the business wants to.
• Quality is owned by the entire team.y y
• Pipelines should be made as wide and as short as possible
• Quickly detect faults
• Teams confidently, rapidly rolly p y
36
QUESTION TIME