Upload
kohsuke-kawaguchi
View
36.859
Download
3
Tags:
Embed Size (px)
DESCRIPTION
My JavaOne 2012 talk about choreography/workflow related improvements/plugins in Jenkins.
Citation preview
©2010 CloudBees, Inc. All Rights Reserved
©2010 CloudBees, Inc. All Rights Reserved
Large-Scale Automation with JenkinsKohsuke Kawaguchi / [email protected] / @kohsukekawaArchitect, CloudBees, Inc.
©2010 CloudBees, Inc. All Rights Reserved
2©2010 CloudBees, Inc. All Rights Reserved
Have you met Jenkins?
• #1 OSS CI server–Written in Java– 47K+ installations
• About 7 years old• Two key emphasis– Easy to install/use– Extensible via 600+ plugins
http://jenkins-ci.org/
©2010 CloudBees, Inc. All Rights Reserved
3©2010 CloudBees, Inc. All Rights Reserved
Context
• First steps in Jenkins usage– Build your project– Run tests on your project–Maybe run code analysis tools
Where to go from here?
©2010 CloudBees, Inc. All Rights Reserved
4©2010 CloudBees, Inc. All Rights Reserved
Workflow!
• By which I mean interconnected jobs
• Why?– Lots of manual process– Dividing by the responsibility boundary– Single job isn’t meant for complex
things– Reusing pieces
©2010 CloudBees, Inc. All Rights Reserved
5©2010 CloudBees, Inc. All Rights Reserved
Parameterized Builds
• Plain jobs can be thought of like a procedure without any input
• Ability to pass parameters make it more useful
void buildAcmeLibrary() { … }
void buildAcmeLibrary(targetPlatform) { … }
©2010 CloudBees, Inc. All Rights Reserved
6©2010 CloudBees, Inc. All Rights Reserved
Three Things You Need To Do
• #1: Define Parameters
Lots of different parameter types
©2010 CloudBees, Inc. All Rights Reserved
7©2010 CloudBees, Inc. All Rights Reserved
Three Things You Need To Do
• #2: Refer to parameter values– As variable expansions: ${Browser}– As environment variables from your
builds– Some parameter types expose data in
different ways• File parameter
©2010 CloudBees, Inc. All Rights Reserved
8©2010 CloudBees, Inc. All Rights Reserved
Three Things You Need To Do
• #3: Specify actual values when you run
©2010 CloudBees, Inc. All Rights Reserved
9©2010 CloudBees, Inc. All Rights Reserved
Parameterized Trigger Plugin
• Call other jobs (with parameters)–Wait for their completions (optional)
Foo
Sub1 Sub2 Sub3
Foo
Sub1
Sub2
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin
©2010 CloudBees, Inc. All Rights Reserved
10©2010 CloudBees, Inc. All Rights Reserved
Multi-Configuration Project
• You often do the same thing with slight variations– Compile C++ code for different
platforms– Test with different browsers– Or more generally, think of it as
for (x in [a,b,c]) { for (y in [d,e,f]) { doSomethingWith(x,y,...); }}
©2010 CloudBees, Inc. All Rights Reserved
11©2010 CloudBees, Inc. All Rights Reserved
Model
• Define axes– One axis ≈ one for loop
• Choose from pre-defined types of Axis– Generic axis: arbitrary values exposed as
environment variables– Slave axis: pick slaves by their names or
their labels• e.g., linux, solaris, and windows
– JDK axis
©2010 CloudBees, Inc. All Rights Reserved
12©2010 CloudBees, Inc. All Rights Reserved
Multi-Configuration Project Gimmicks• Filtering– Otherwise combinations increase
exponentially– Not all combinations make sense– Use boolean expression to trim down the size
– Or tell Jenkins to cut the workload to N%• Jenkins will thin out the combinations by itself
(label==“windows”).implies(browser==“iexplore”) &&(label==“mac”).implies(browser==“safari”)
©2010 CloudBees, Inc. All Rights Reserved
©2010 CloudBees, Inc. All Rights Reserved
Demo
©2010 CloudBees, Inc. All Rights Reserved
14©2010 CloudBees, Inc. All Rights Reserved
Other Simple Choreography Tools
• Join Pluginhttps://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin
©2010 CloudBees, Inc. All Rights Reserved
15©2010 CloudBees, Inc. All Rights Reserved
When Jobs Start Working Together…
• Copy Artifact Plugin– Copy artifacts into a workspace– By using various criteria
https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
acme-build #15
foo.jar
acme-test #24
foo.jar
©2010 CloudBees, Inc. All Rights Reserved
16©2010 CloudBees, Inc. All Rights Reserved
acme-build #10acme-build #11
acme-build #12acme-build #13
acme-build #14
Copy Artifact vs External Repository
• Almost as if artifacts are versioned
acme-build #15
foo.jar
Last stable build
Last saved build
©2010 CloudBees, Inc. All Rights Reserved
17©2010 CloudBees, Inc. All Rights Reserved
Labeling Builds Is Useful
• Especially when labels have semantics
acme-build #10acme-build #11
acme-build #12acme-build #13
acme-build #14acme-build #15
foo.jar
Last stable build
Signed off to QA
Deployed to production
©2010 CloudBees, Inc. All Rights Reserved
18©2010 CloudBees, Inc. All Rights Reserved
Labeling Builds Is Useful
• More so when you automate them• Take “Signed off to QA” label for
example
acme-build #12acme-devtest #15
acme-findbugs #22
acme-devtest #15
acme-findbugs #22
Signed off to QA
foo.jar
©2010 CloudBees, Inc. All Rights Reserved
19©2010 CloudBees, Inc. All Rights Reserved
Introducing Promoted Builds Plugin
• Promotion = act of giving a build a label
• You specify:– Promotion criteria– what happens after promotion
• Label is a nice hand-off between teams– It’s like sausage making process
https://wiki.jenkins-ci.org/display/JENKINS/Promoted+Builds+Plugin
©2010 CloudBees, Inc. All Rights Reserved
©2010 CloudBees, Inc. All Rights Reserved
Demo
©2010 CloudBees, Inc. All Rights Reserved
21©2010 CloudBees, Inc. All Rights Reserved
Maven Repository Plugin
• Virtual Maven repository– Expose artifacts from specific build– And its upstream builds
https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Maven+Repository+Server
©2010 CloudBees, Inc. All Rights Reserved
22©2010 CloudBees, Inc. All Rights Reserved
Challenge: visualization
http://www.flickr.com/photos/andresmusta/7990193487
©2010 CloudBees, Inc. All Rights Reserved
23©2010 CloudBees, Inc. All Rights Reserved
Challenge: Visualization
• Edge traversal breaks down on large workflow
©2010 CloudBees, Inc. All Rights Reserved
24©2010 CloudBees, Inc. All Rights Reserved
Dependency Graph
• Uses GraphViz to draw a graph of relations
• Supports copy-artifact plugin
https://wiki.jenkins-ci.org/display/JENKINS/Dependency+Graph+View+Plugin
©2010 CloudBees, Inc. All Rights Reserved
25©2010 CloudBees, Inc. All Rights Reserved
Build Pipeline Pluginhttps://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin
• It shows how far each change has gone
©2010 CloudBees, Inc. All Rights Reserved
26©2010 CloudBees, Inc. All Rights Reserved
Fingerprint
• Yet another angle to look at data• Think of it as a lifelog for a blob
Air travel Jenkins
Airline check-in/TSA/immigration Jobs and workflow
My going through immigration Build #13
My travel experience Fingerprint view
©2010 CloudBees, Inc. All Rights Reserved
27©2010 CloudBees, Inc. All Rights Reserved
Fingerprint: Mechanism
• MD5 checksum of a file– Recorded against builds that it
appeared– (And actions that were taken)
acme-build #15
MD5=604e7e07
acme-test #13
MD5=604e7e07
©2010 CloudBees, Inc. All Rights Reserved
28©2010 CloudBees, Inc. All Rights Reserved
Fingerprint: Why?
• Track down where it came from–My component integrates to product
XYZ, and a bug was reported against XYZ 3.0.5. Which build of the component did it contain?
• Cross-correlate jobs that aren’t directly related
©2010 CloudBees, Inc. All Rights Reserved
29©2010 CloudBees, Inc. All Rights Reserved
Next Step in Workflow
Aggregation of results
Aggregation of definitions
✓
?
©2010 CloudBees, Inc. All Rights Reserved
30©2010 CloudBees, Inc. All Rights Reserved
Build Flow Plugin
• Groovy DSL for kicking builds– High-level primitives– Ability to define abstractions
b = build(“acme-build”)guard {
parallel ({ build(“acme-test1”,
param1:b.number) },{ build(“acme-test2”,
param1:b.number) })
} rescue {build(“acme-teardown”)
}
https://wiki.jenkins-ci.org/display/JENKINS/Build+Flow+Plugin
©2010 CloudBees, Inc. All Rights Reserved
©2010 CloudBees, Inc. All Rights Reserved
Demo
©2010 CloudBees, Inc. All Rights Reserved
32©2010 CloudBees, Inc. All Rights Reserved
Jenkow Plugin
• Embed BPMN workflow engine in Jenkins– Timeout, fork, join, …
• Workflow is version controlled in Git– Push to Jenkins to
load them up
https://wiki.jenkins-ci.org/display/JENKINS/Jenkow+Plugin
©2010 CloudBees, Inc. All Rights Reserved
33©2010 CloudBees, Inc. All Rights Reserved
Next Step in Workflow
Choreography defined in one place
Everything defined in one place
✓
?
©2010 CloudBees, Inc. All Rights Reserved
34©2010 CloudBees, Inc. All Rights Reserved
Job DSL Plugin
• Groovy DSL for defining jobsdef project = "jenkinsci/jenkins"def branchApi = new URL("https://api.github.com/repos/${project}/branches")def branches = new JsonSlurper().parse(branchApi.newReader())branches.each { b -> job { name "${project}-${b.name}".replaceAll('/','-') scm { git("git://github.com/${project}.git", b.name) } steps { maven("install") } }}
https://wiki.jenkins-ci.org/display/JENKINS/Job+DSL+Plugin
©2010 CloudBees, Inc. All Rights Reserved
35©2010 CloudBees, Inc. All Rights Reserved
Or More Likely…
• Take Existing Job, Make Adjustmentsdef project = "jenkinsci/jenkins"def branchApi = new URL("https://api.github.com/repos/${project}/branches")def branches = new JsonSlurper().parse(branchApi.newReader())branches.each { b -> job { using "jenkins-build" name "${project}-${b.name}".replaceAll('/','-') scm { git("git://github.com/${project}.git", b.name) } }}
©2010 CloudBees, Inc. All Rights Reserved
36©2010 CloudBees, Inc. All Rights Reserved
Job DSL Plugin
• You can go down to XML definitions
• The program itself executes as Jenkins job– Control over when it executes– Store definitions in VCS
©2010 CloudBees, Inc. All Rights Reserved
37©2010 CloudBees, Inc. All Rights Reserved
Or just a bit of Perl/Python/Ruby scripts• Programmatically CRUD jobs
$ ssh jenkins get-job foo \| sed -e 's/old.gitserver.com/new.gitserver.com/g’ \| ssh jenkins update-job foo
©2010 CloudBees, Inc. All Rights Reserved
38©2010 CloudBees, Inc. All Rights Reserved
Templates (in Jenkins Enterprise by CloudBees)
• Share some traits with Job DSL– Define job once, generate many
variations– Update definition, and reflect it
everywhere
• But different– Templates are defined in GUI, not in a
program– Individual variations are manually
updated by users
http://www.cloudbees.com/jenkins-enterprise-by-cloudbees-overview.cb
©2010 CloudBees, Inc. All Rights Reserved
39©2010 CloudBees, Inc. All Rights Reserved
Conclusion
• Lots of useful building blocks for automating even more– That means many people are doing
this
• Take your automation to the next level