Upload
aaron-eden
View
506
Download
1
Tags:
Embed Size (px)
Citation preview
CONTINUOUS INTEGRATION
INTRODUCTIONS
Brendan Jennings
Alexandre Morot
Intuit
Any opinions are our own
OVERVIEW OF PRESENTATION
What Is Continuous Integration Previous System Pain Points Build Server Unit Tests Automated Deployment Verdict Questions
WHAT IS CONTINUOUS INTEGRATION
PREVIOUS SYSTEM PAIN POINTS
SOURCE CONTROL/BROKEN BUILDS
SOURCE CONTROL/BROKEN BUILDS
Issues Encountered (It works on my machine) Incorrect DLL references
DLL is in a different location to hard coded path DLL does not exist on machine DLL reference is to output directory
SOURCE CONTROL/BROKEN BUILDS
Issues Encountered (It works on my machine) Incorrect DLL references Incomplete check-ins
Files without project Project without new files
SOURCE CONTROL/BROKEN BUILDS
Issues Encountered (It works on my machine) Incorrect DLL references Incomplete check-ins Overwritten changes Check in timing (end of day check-ins) Staying on old version to avoid build errors
Fairly Frequent Downtime for Entire Team Resolving Build Errors
MANUAL TESTING OF CHANGES
MANUAL TESTING OF CHANGES
Time Intensive No Full Regression Test Testing Time Reduced When Running
Late
MANUAL PUSH TO PRODUCTION
MANUAL PUSH TO PRODUCTION
Requires Observer to Reduce Risk of Errors
STEPS INVOLVED
STEPS INVOLVED
Build
STEPS INVOLVED
Zip
STEPS INVOLVED
Push to Datacenter
STEPS INVOLVED
Pull to Pre-Production
STEPS INVOLVED
Extract and Replace Build
STEPS INVOLVED
QA
STEPS INVOLVED
QA Restart if Failed
STEPS INVOLVED
For Each Server
STEPS INVOLVED
For Each Server Pull from Rotation
STEPS INVOLVED
For Each Server Extract and Replace Build
STEPS INVOLVED
For Each Server Warm Up Server
STEPS INVOLVED
For Each Server Put Back In Rotation
STEPS INVOLVED
Pray
STEPS INVOLVED
Release Complete Typically 2 Hours Later
SERVERS NOT ALL PUSHED TOGETHER
Leaves Site in Inconsistent State
SERVERS NOT ALL PUSHED TOGETHER
Leaves Site in Inconsistent State If inconsistency is severe, site needs to be
taken down during push
BUILD SERVER
BUILD SERVER
Continuous Build of Every Check-In
BUILD SERVER
Continuous Build of Every Check-In Used Hudson, later switched to Jenkins
BUILD SERVER
Continuous Build of Every Check-In Immediate Notification of Build Errors
Names and shames guilty party Notifications come through to phone Encourages pride in build quality Only check-in working code Immediate action must be taken for build errors Notification sent for every check-in that is still broken Final notification sent when problem resolved
UNIT TESTS
UNIT TESTS
Gradually Adding Unit Tests for Existing Code Focus on coverage for most critical areas
first
UNIT TESTS
Gradually Adding Unit Tests for Existing Code
Default to Unit Tests for New/Updated Code
Add Test Execution to Build Server Failed unit tests also send notifications to
team History maintained; can track fixes to tests
UNIT TESTS
Gradually Adding Unit Tests for Existing Code
Default to Unit Tests for New/Updated Code
Add Test Execution to Build Server All Tests Must Pass
If the test has found a bug, the code must be fixed
If the test has failed incorrectly, the test must be fixed
DEPLOYMENT
DEPLOYMENT
Latest Build Added server to run latest development
version All successful builds pushed automatically
DEPLOYMENT
Latest Build Data Center
Use source control labels Precise control of what is included in each build History of all prior builds; can recreate prior
versions of site
DEPLOYMENT
SCRIPT TO AUTOMATE PUSHES
SCRIPT TO AUTOMATE PUSHES
Push to Different Environments
SCRIPT PROCESSING
Take One Server Out Of Rotation
SCRIPT PROCESSING
Updates Server with Differential Push
SCRIPT PROCESSING
Checks For Deployment Errors
SCRIPT PROCESSING
Manual Final Verification Step
SCRIPT PROCESSING
Takes Half of the Servers Out of Rotation
SCRIPT PROCESSING
Pushes New Code to these Servers in Parallel with Differential Pushes
SCRIPT PROCESSING
Warms Up Servers
SCRIPT PROCESSING
Checks for Deployment Errors
SCRIPT PROCESSING
Swaps All Servers In/Out of Rotation Site is now effectively upgraded
SCRIPT PROCESSING
Updates Remaining Servers with Differential Pushes
SCRIPT PROCESSING
Warms Up Servers
SCRIPT PROCESSING
Check for Deployment Errors
SCRIPT PROCESSING
Puts Servers Back into Rotation
SCRIPT PROCESSING
Final Verification that All Servers are In Rotation
TECHNOLOGIES USED
MICROSOFT WEB DEPLOY
Web Deploy simplifies the migration, management and
deployment of IIS Web servers, Web applications and Web sites.
CYGWIN BASH AND SSH SERVER
Linux-like environment for Windows making it possible to port software running on POSIX systems (such as Linux, BSD, and Unix systems) to
Windows.
JENKINS
Jenkins is an open-source continuous integration server
with 300+ plugins to support all kinds of software
PLUGINS FOR TFS AND MSBUILD
TFS plugin customized to handle labeling strategy
CustomizedTFS Plugin
MSBUILD Plugin
Deployment ScriptCalled from Jenkins
VERDICT
QUALITY
QUALITY
Build Issues Typically Identified Rapidly No longer have the ‘Monday morning blues’ where
nobody can build
QUALITY
Build Issues Typically Identified Rapidly Automated Regression Tests provide Higher
Levels of Confidence Issues are uncovered that we believe we would have
missed in the past Analytics bugs were extremely unlikely to be seen
until someone complained
DEPLOYMENT
DEPLOYMENT
Can Be Deployed by One Person No Downtime Required in Most Updates Time to Deploy Reduced to 6 Minutes
Team more productive Enables rapid deployment of small
changes Multiple deployments per day 2 hour window can be hard to find between
meetings
DEPLOYMENT
Can Be Deployed by One Person No Downtime Required in Most Updates Time to Deploy Reduced to 6 Minutes Can Rollback Quickly if Necessary Deployment can be Unattended
If something fails, deployment aborts automatically
Failure notification sent including failure reason
DEPLOYMENT
Can Be Deployed by One Person No Downtime Required in Most Updates Time to Deploy Reduced to 6 Minutes Can Rollback Quickly if Necessary Deployment can be Unattended Team Do Not Need Production Server
Login Removes temptation to perform fix directly
on server
QUESTIONS