Jenkins State of union 2013

Preview:

DESCRIPTION

Current state of the Jenkins project

Citation preview

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Jenkins: State of Union

#jenkinsci

Kohsuke KawaguchiJenkins Creator

kk@kohsuke.org / @kohsukekawa

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

In Average Week of Jenkins

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

1 new release comes out

Gets downloaded 48,000 times

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

That fills up 513 DVDs

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

3.5 Brand-new Plugins Released

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

25 New Versions of Plugins Released

Gets downloaded 500,000 times

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

80 Tickets Filed

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

50 Tickets Solved

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

145 Commits Are Made

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

400 Jenkins Installations Pops Up

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

1200 New Slaves Are Added

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

19,000 New Jobs Are Added

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

# of plugins

Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-12 Jan-130

100

200

300

400

500

600

700

800

767

# of unique plugins

releases/weeknew plugins/week

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Ticket Activities

Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-12 Jan-130

2000

4000

6000

8000

10000

12000

14000

16000

18000

17500

Total tickets

Resolved tickets

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Tracked Installations (conservative)

Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-12 Jan-130

10000

20000

30000

40000

50000

60000

64000

# of masters

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Recently Introduced Featureshttp://www.flickr.com/photos/randar/3879926596/

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Lazy loading of build records

• Jenkins slow to start up?– It was loading build records upon boot

• This is finally fixed, including fallouts– Loaded on demand– In weak references

• Now on LTS: 1.509.1

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Master/slave performance

• Better sustained data transfer rate– Copy large files, test reports, etc– Now on par with scp

• Classloading improvements– Local cache & prefetching– Will help ephemeral/elastic slaves & maven jobs

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Performance/Scalability Improvements

• Lock strength reduction• Fingerprint scalability improvements• Better caching of images & scripts in UI• The “people” page• Handling large test result

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Testing Improvements

• Previous testing scheme– Embedded Jetty in Maven– Code from ~/.m2/repository and resources from

war– The setup was different from how you run it

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

New Blackbox Acceptance Test

• Tests the final deliverables– jenkins.war, .deb, .rpm, etc– With Tomcat, JBoss, or “java -jar jenkins.war”

• In different OSes– In VirtualBox (via vagrant)– Hopefully with EC2 (upcoming)

• Tests via Selenium– Capybara + Cucumber– Sauce OnDemand to test different browsers

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Database Plugin

• For other plugins, not for users– But large data set in RDBMS should result in tangible

benefits for users• Provides unified database connectivity– UI for admin to select the backend– Expose DataSource for other plugins– JPA support, per-job database

• Looking for plugins to start using this

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Plugins that generate jobs

• Job DSL plugin– Groovy DSL for describing job definitions– DSL can be evaluated from within Jenkins– E.g., automatically generate new jobs for new

branches• Job Generator plugin– Create a new ste of jobs from “template jobs” by

eagerly evaluating some build parameters

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Build Flow Plugin

• Groovy DSL for kicking builds– High-level primitives– Ability to define abstractions

b = build(“acme-build”)guard {

parallel ({ build(“acme-test1”,

param1:b.number) },{ build(“acme-test2”,

param1:b.number) })

} rescue {build(“acme-teardown”)

}

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Jenkow Plugin

• Embed BPMN workflow engine in Jenkins– Timeout, fork, join, …

• Workflow is version controlled in Git– Push to Jenkins to load them up

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Recipe Plugin

• Sharing a set of jobs and other settings in Jenkins with the community

• Convenient for…– Sample setup / quick start for plugins– Show cases

• Simplify the first “getting started” phase

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Parallel Test Executor Plugin

• Split tests N-way, run them in parallel• Create even splits by using timing information

Test Group #1 Test Group #2 Test Group #3

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Demo

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Upcoming Features

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Future Direction: Which Way?

http://www.flickr.com/photos/quinnanya/3500158343/

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Make Complex Things Doable

• Running big, complex deployment should be easier– Master-to-master communication channel– Per-job database storage, open for plugins

• More automatic slave managements– Better diagnostics

• Access control for running builds

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Make Complex Things Doable

• Better Workflow/pipeline support– Already a lot going on in plugins!– Slice up build histories– Shared context between related builds

• Modeling external systems– Databases, test environments, …– Remembering who’s using what– Remember what was done to them

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Make Simple Things Simpler

• More spying, less asking– E.g., “where are unit test reports?”

• Simpler project creation ala BuildHive– Only needs to be 70% accurate– Sharable archetypes, repository sniffing

• Encourage sharing of recipes

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Help You Find Plugins You Need

• Revisit bundled plugins– Why Git plugin isn’t in there?

• Plugin packs (aka iTunes playlist)– Follow the lead of the Jenkins PHP website

• Recommendation based on what you have• Reviews, feedbacks

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

More UI Improvements

• FOSDEM-driven on-going effort

• TODO in this round:– Hunt on stale data in HTML– Performance on loading config page– More compact UI controls

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Improving Plugin Development Experience

• More extension points• Better Plugin Development Environment– More Wiki articles/samples?– Training course?

• Better development mode– Skeleton generation– Checking common pitfalls in the dev mode

• Tell us what you need!

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Conclusions

• A lot has been done, not just in code– And a lot more to come!

• Please help the project• Please enjoy the day!

Jenkins User Conference Shefayim, Jun 2013 #jenkinsci

Recommended