Embed Size (px)
Citation preview
Continuous Integration and Delivery at NSIDC
Julia Collins
National Snow and Ice Data CenterSupporting Cryospheric Research Since 1976
National Snow and Ice Data CenterCooperative Institute for Research in Environmental Sciences, University of Colorado, Boulder, CO
UCAR SEA 20147 April 2014
UCAR SEA 20147 April 2014
The Goal
2
Continuous integration and
Continuous delivery
?
UCAR SEA 20147 April 2014
Definitions
3
Continuous integration: Merge changes frequently into shared code base (and run unit/integration tests) to catch conflicts early.
Requires: Code repositoryAutomated build (build all commits)Automated testsIf fail, fail fast
UCAR SEA 20147 April 2014
Definitions
4
Continuous delivery: A software development discipline where you build software in such a way that the software can be released to production at any time.
http://martinfowler.com/bliki/ContinuousDelivery.html
UCAR SEA 20147 April 2014
Definitions
5
Continuous deployment: Passing builds are automatically deployed to the production environment
http://continuousdelivery.com/2010/08/continuous-delivery-vs-continuous-deployment/
UCAR SEA 20147 April 2014
Release
Development Context
6
Time
Release
Iterative development with frequent feature releases
UCAR SEA 20147 April 2014
Jenkins (was Hudson)
7
UCAR SEA 20147 April 2014
Jenkins
8
Open source (MIT license)Core installation great for running scriptsCan run on a schedule, poll SCM for changesCan be extended via plugins. Many (many many) plugins already exist.
Cons: Plugin developers not necessarily writing with other (unrelated) plugin behaviors in mind.
UCAR SEA 20147 April 2014
Jenkins Build
9
ABuild Job
Build/unit test
Build Job
Acceptance test
Build Job
Deploy
UCAR SEA 20147 April 2014
Another definition
10
a running virtual machine
“Artifact”
tar file
jar file
SCM tag or revision number
The by-product of the build; the testable entity
UCAR SEA 20147 April 2014
Artifacts
11
We had no artifact repository
Solution: Roll our own
Result: Overhead added to job configurations to support the artifact handling
UCAR SEA 20147 April 2014
One day you look around . . .
12http://xkcd.com/1319/
UCAR SEA 20147 April 2014
But really, automation is good
13
Consistent steps, no matter who’s running the build
History and reproducibility
Build configuration conveys information about application environment and structure
UCAR SEA 20147 April 2014
Meanwhile . . .
14
Moving from svn towards git
Bitbucket repositories
Take advantage of git workflows
UCAR SEA 20147 April 2014
Branching Strategies
15
Feature AFeature B
Master (trunk): Always in deployable state
Merge master into Feature A
UCAR SEA 20147 April 2014
Many jobs, one build server
16
Applications using different versions of same language or libraries
Conflict between projects needing upgrades vs. those frozen at a particular version
Would like option to build, test and run applications on different OS than default
UCAR SEA 20147 April 2014
The Wish List
17
Feature workflow:The ability to build/test/approve a feature branch.
The ability to merge master into a branch prior to build/test.
The ability to merge all successfully integrated (built/tested) feature branches into master for review within the QA environment.
The ability to push the local master (with all approved feature changes) to origin/master on the remote repository, along with tags indicating released code.
UCAR SEA 20147 April 2014
The Wish List
18
Build and deployment:The ability to create multiple build environments to satisfy different application needs for language versions and supporting libraries.
A continuous build environment that produces/manages deployment-ready artifacts, including managing the state of approval (or not).
Deployment: Ops must be able to identify the approved artifact(s) and run the required steps for deployment.
Deployment: Ops must be able to easily roll back to a previous release.
UCAR SEA 20147 April 2014
Road Map
19
Stick with Jenkins, OR
Transition to a different build server
UCAR SEA 20147 April 2014
Jenkins in the future
20
Use existing plugins to provide needed functionality (artifact handling, master/slave implementation for multiple build agents)
Contribute to plugin development
Move in-line shell commands to application task runners (remove redundancies)
UCAR SEA 20147 April 2014
Bamboo (Atlassian)
21
Stage 1
Stage 2
Stage 3
Job 1
Job 2
Job 3
Run
in o
rder
Jobs run in parallel (different agents)
task 1
task 2
task 3
task 1
task 2
task 3
task 1
Ru
n i
n o
rde
r
Build Plan
UCAR SEA 20147 April 2014
Bamboo
22
UCAR SEA 20147 April 2014
Bamboo
23
UCAR SEA 20147 April 2014
The Critical Facts
24
Jenkins: “free”
TeamCityFree edition: 3 build agents, 20 build configurations$299: 1additional build agent, 10 additional configs$1999+: Enterprise options
BambooHosted vs. local options$10: No remote agents, 10 jobs$800: 1 remote agent, unlimited jobs$2200: 4 remote agents, unlimited jobs
UCAR SEA 20147 April 2014
Lessons Learned
Build configurations tend to proliferate.
With some effort, Jenkins can support many requirements.
It’s not just using the right tool, it’s how you use it.
25
UCAR SEA 20147 April 2014
References
• Jenkins: http://jenkins-ci.org
• TeamCity: http://www.jetbrains.com/teamcity/
• Bamboo: https://www.atlassian.com/software/bamboo
• CI: http://martinfowler.com/bliki/ContinuousDelivery.html
• CD: http://continuousdelivery.com/2010/08/continuous-delivery-vs-continuous-deployment/
26
Julia [email protected]
