Upload
kohsuke-kawaguchi
View
3.265
Download
2
Tags:
Embed Size (px)
DESCRIPTION
My JUC 2012 keynote talk.
Citation preview
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Jenkins: State of Union
Kohsuke KawaguchiJenkins Creator@kohsukekawa
#jenkinsconf
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Thank You To Our Sponsors
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
In Average Week of Jenkins
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
1 new release comes out
Gets downloaded 48,000 times
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
That fills up 513 DVDs
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
3.5 Brand-new Plugins Released
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
25 New Versions of Plugins Released
Gets downloaded 500,000 times
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
80 Tickets Filed
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
50 Tickets Solved
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
145 Commits Are Made
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
400 Jenkins Installations Pops Up
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
1200 New Slaves Are Added
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
19,000 New Jobs Are Added
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
# of plugins
Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-120
100
200
300
400
500
600
606
# of unique plugins
releases/weeknew plugins/week
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Ticket Activities
Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-120
2000
4000
6000
8000
10000
12000
14000
16000
15200
Total tickets
Resolved tickets
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Tracked Installations (conservative)
Jan-07 Jan-08 Jan-09 Jan-10 Jan-11 Jan-120
5000
10000
15000
20000
25000
30000
35000
40000
45000 47000
# of masters
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Recently Introduced Featureshttp://www.flickr.com/photos/randar/3879926596/
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Demo
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Not Every Frontend Change Is Visible
• Modernization– Dependency bump-ups: Prototype 1.7, YUI 3– Native JSON/CSS selectors where available
• Page loading performance– Liberal use of gzip compression– Client-side caching for assets in plugins
• Reuse common dependencies as plugins: jQuery
• Modularization of core JavaScript code• No ajax updates when pages aren’t visible
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Multi-configuration Project Improvement
• Concurrent parent builds (between #N, #N+1, …)• More flexible workspace sharing• Sparse and see-through child builds• Axis whose values change dynamically• SCM checkout behavior is now extensible• Execution order / filtering is
now extensible
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
More Backend Stuff
• REST API improvements– API token– Update center / plugin manager / executors / slaves– Efficient data filtering by the “tree” parameter– Nestor is a good client example
• CLI improvements– Transport encryption– User authentication with SSH keys– ssh(1) as your client
ssh -p 2222 jenkins.acme.com build foo -s -v
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Fresh Off The Oven
• Does your Jenkins take 5 minutes to start?– Lazy-loading to the rescue– Unloading, too– More changes to come
• More self-descriptive XML
<scm class=“hudson.plugins.git.GitSCM” plugin=“[email protected]”> <branches> …
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
And Long List of Improvements…
• Subversion 1.7• Localization to 40 languages• Enforce naming convention to jobs• Core spins off more plugins• Arbitrary build step in Maven project type• Hash support in Cron• Slaves automatically comes back when disk is
available• Symlink support on Windows Vista/2003+
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Improvements for Plugin Developers
• Cut off JDK5 compatibility to support JDK7• Dependency injection via Guice• JUnit4 test harness• Groovy for view templates– Debuggable– Your IDE auto-completes expressions
• More extension points (111 total)• Gradle, JRebel support
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Survey
• http://jenkins-ci.org/survey• Tell us what you care– If you can, please give us testimonials– We’ll be discussing this later today
• Prizes– 2 Dr Dre headphones– Kindle– 2 $100 Visa Gift Cards– 1 Lightsaber, Sphero– T-shirts, giftbags, conference passes, JRebel license,…
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Java Requirement
• Proposal: start requiring Java6– For running masters and slaves– For running Maven project type
• Data points– About 1.5% people run their Jenkins on Java5– Java5 reached end-of-life in Oct 2009– Widespread consensus among developers
• If you have a problem, tell us now!
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
With My Day Job Hat On…
http://www.flickr.com/photos/sushicam/7211787664/
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Demo
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
BuildHive.cloudbees.com
• Free for anyone• Love to get your feedback
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Jenkins Enterprise by CloudBees
• Enterprise plugins + Support– Active/standby high-availability– Validated merge (aka “Unbreakable builds”)– Templates, RBAC, custom update center, …
• CloudBees Free Enterprise Plugins– Folder– Backup
http://www.cloudbees.com/jenkins-enterprise-by-cloudbees-overview.cb
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Beyond Coding
http://www.flickr.com/photos/acmace/4410650624/
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
JUCs Around the World
• Last year, we went to 5 places
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
… and more meet-ups
• FOSDEM, SCALE 10x• Copenhagen user event
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
… and more meet-ups / hackathon
• Austin, Munich, 3 more places in Japan
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Upcoming Events
• FOSDEM and JUC 2013 in SF for next 12 months• Shift to cheaper & more open events– Half-day, to avoid catering– Cheaper venue - university, large companies– Free/minimal admission
• We want to work with local folks– “Start your own Jenkins meetup in 10 easy steps”
http://bit.ly/QtZMdK– Asia, where are you?
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Jenkins CIA Program
• Speaking about Jenkins in a meet-up?– We’ll send you a T-shirt
and stickers• We are building a world
map with push pins
• Join the CIA to spread words!
https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+CIA+Program
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Security Advisories
• When vulnerabilities are discovered, we prepare a fix and issue a security advisory
• Please subscribe via e-mail or RSS– http://jenkins-ci.org/advisories
• Especially if you run Jenkins on the internet
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
On Timely Resolution of Vulnerabilities
• File issues to the SECURITY project– http://issues.jenkins-ci.org/browse/SECURITY/
• [email protected]– Closed-door discussion group for devs
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Wiki Static Cache
ConfluenceNginx
HTML
in cache?
fallback
generate upon changes &
periodically
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Usage Statistics
• Installation count growth in plugin page• Also available in JSON
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Stats.jenkins-ci.org
• Understand how Jenkins & plugins are used– Platform– # of slaves– Types of jobs
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Blackbox e2e Testing
• Motivations– Test the final deliverables– Test in the same setup you use
• Results thus far– Ruby + Capybara + Cucumber– Test Jenkins via Selenium– Abstraction to start/stop Jenkins– Plugin to assist assertions & selections
https://github.com/jenkinsci/selenium-tests
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Better Source Code Access
• Mother of all repositories– Search usage of code across all the plugins– http://git.jenkins-ci.org/all.git
• List of all the repositories– https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Repositories– Along with groupId:artifactId
• List of all the extension points– https://wiki.jenkins-ci.org/display/JENKINS/Extension+points– Along with their usage
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Other Infra Improvements
• Server monitoring + Pager Duty• 9 mirrors around the world• Puppet-managed DNS servers• More offloading– repo.jenkins-ci.org, thanks to JFrog– jenkins.ci.cloudbees.com,
thanks to CloudBees
http://nagios.jenkins-ci.org/
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Office Hours
• Bi-weekly WebEx conference call– Wednesday, complements project meetings
• Originally intended as virtual meet-up– Mini presentation, Q&A, show&tell
• Currently more like “Ask Kohsuke” session for plugin/core developers
https://wiki.jenkins-ci.org/display/JENKINS/Office+Hours
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Upcoming Features
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Future Direction: Which Way?
http://www.flickr.com/photos/quinnanya/3500158343/
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Make Complex Things Easier
• Running big, complex deployment should be easier– Master-to-master communication channel– Follow-up work for lazy loading– Per-job database storage, open for plugins
• More automatic slave managements
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Make Complex Things Easier
• Better Workflow/pipeline support– Already a lot going on in plugins!– Slice up build histories– Shared context between related builds
• Modeling external systems– Databases, test environments, …– Remembering who’s using what– Remember what was done to them
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Make Simple Things Simpler
• More spying, less asking– E.g., “where are unit test reports?”
• Simpler project creation ala BuildHive– Only needs to be 70% accurate– Sharable archetypes, repository sniffing
• Encourage sharing of recipes
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Help You Find Plugins You Need
• Revisit bundled plugins– Why Git plugin isn’t in there?
• Plugin packs (aka iTunes playlist)– Follow the lead of the Jenkins PHP website
• Recommendation based on what you have• Reviews, feedbacks
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Improving Plugin Development Experience
• More extension points• Better Plugin Development Environment– More Wiki articles/samples?– Training course?
• Better development mode– Skeleton generation– Checking common pitfalls in the dev mode
• Tell us what you need!
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Conclusions
• A lot has been done, not just in code– And a lot more to come!
• Please help the project• Please enjoy the day!
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
Fighting Spams (AKA we are popular)
• Wiki spams have been a problem– Manual attack?– A scanner was developed to flag & delete them
• JIRA spam has been spotted• E-mail spams have been a problem
• If you find them, please report them• If you are willing to help policing them, let us know
https://github.com/jenkinsci/backend-confluence-spam-remover
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
More Number Crunching Needed
• Correlation• Recommendation
• We need volunteers– Should be fun!
Jenkins User Conference San Francisco, Sept 30 2012 #jenkinsconf
And Longer Term…
• If we have 100 machines per developer, what can Jenkins do?
• Distributed parallel test execution• Job that spans multiple processes– Load test, system test, …
• What visualizations do we need?