7 Ways to Optimize Hudson in Production

7 Ways To Optimize Hudson/Jenkins for Production Installation

Kohsuke Kawaguchi Creator of the Hudson project

©2011 Cloud Bees, Inc. All Rights Reserved

2 ©2011 Cloud Bees, Inc. All Rights Reserved

About CloudBees Our Mission

Strategy

Become the leading Java™ Pla3orm as a Service (PaaS)

Why We’re Different

CloudBees services the complete lifecycle of Cloud applica>on development and deployment. No Servers. No Virtual Machines. No IT.

§  DEV@cloud – Cloud Services for Developers

§  RUN@cloud – Fric@onless run@me PaaS for Java apps

3

Benefits of DEV@cloud Hudson Service: •  Scale your Hudson environment with the power of the Cloud

•  Ease your Hudson management overhead •  Speed your Hudson builds •  Save money with on-‐demand Hudson Service.

©2011 Cloud Bees, Inc. All Rights Reserved

Continuous Integration - Hudson

0

5,000

10,000

15,000

20,000

25,000

Hudson Adoption, by Builds

Source: Hudson Labs

Nectar – On-‐Premise Enterprise Hudson •  Support from the Hudson experts. •  VMware scale your Hudson environment. •  Enterprise Features extend Hudson for large environments.

•  Integrate with the Cloud integra@on with DEV@Cloud and RUN@Cloud coming

4 ©2011 CloudBees, Inc. All Rights Reserved

•  7 relatively easy techniques you can apply today

•  It helps to spend some time setting it up “right” – Even if Hudson is easy enough to deploy – Especially for serious production use

•  It’s not too late even if you are already running Hudson

Idea Behind This Webinar

©2011 CloudBees, Inc. All Rights Reserved


•  Better late than never •  Useful for

– Disaster recovery – Accidental configuration changes

#1: Backup


•  Everything is under $HUDSON_HOME – Check system config to see where it is – No need to backup slaves

•  Live vs offline – Live backup is OK – Configuration writes are atomic

#1: Backup Planning


•  /war (exploded war) •  /cache (downloaded tools) •  /tools (extracted tools) •  /jobs/*

– builds (build records) – builds/*/archive (archived artifacts) – workspace

#1: Subsetting backup


•  File system-level snapshot for maximum consistency – LVM – Other storage-level snapshots

•  Don’t forget fsfreeze – ZFS, if you are on Solaris

•  Incremental, too

•  Nectar can help you

#1: More on backup


•  Just copy it somewhere and “java -jar hudson.war --httpPort=9999” – HUDSON_HOME is relocateable

#1: Testing restore


•  Prepare for disk usage growth – Especially when you start to host jobs from

people who aren’t close to you – Make sure you can throw more disks at the

problem later

•  No need to waste money on 15000rpm SCSI disks – But bigger disk is nice

#2: Disk


•  “Spanned volume” on Windows •  LVM on Linux

– This needs the most upfront planning •  ZFS

– Shame there’s no viable free Solaris flavor – $HUDSON_HOME should be its own FS

•  If it’s too late, symlink is your friend

#2: Expandable volume


•  Comes in .deb/.rpm – Much easier to install/upgrade – Reproducible –  init script & daemon user

•  Configuration in –  /etc/default/hudson –  /etc/sysconfig/hudson

•  Windows --- work in progress •  Existing $HUDSON_HOME can be

migrated

#3: Native packages


•  You will grow beyond a single system – Load is not the only factor

•  Better isolation between builds •  Diversity in the test/build platform

– Cheaper to scale horizontally

•  Let Hudson use multiple computers

#4: Distributed builds

Demo


•  Let master launch slaves (if you can) – Via SSH or DCOM – Easier to keep the cluster up & running –  It’s amazing how easily a computer gets sick

•  Set up SSH public key authentication – Login to systems without typing password – Must-have! Inexcusable! – Only takes 5 minutes or so to set up

#4: Slave reconnectivity


•  Treat build machines like livestock, not like pets – Don’t tie builds to specific build machine – Build machines should be interchangeable

•  Labels, instead of names – Tag slaves by their capabilities/environments – Jobs can be tied to boolean expression of

labels •  Better resource utilization, manageability

#5: Labels

Demo


•  If your users can’t see Hudson, much of the benefit is lost – Make the URL easier to remember

#6: Invest in good URL

hVp://sca14-‐3530.sca.cloudbees.com:8080/hudson/

hVp://hudson.cloudbees.com/


•  Get the host alias, instead of using primary machine name –  If your IT isn’t helping, use dynamic DNS – This also makes your service relocatable

#6: Use service name


•  Apache reverse proxy – Let you run Hudson in non-root, too

•  For Windows –  IIS7+URL Rewrite+Application Request

Routing

#6: Share port 80 with other apps

Apache Hudson browser


•  Use virtual host to distinguish multiple apps, not context path

#6: No “/hudson”

Apache / IIS

Hudson

browser

Redmine hVp://hudson.cloudbees.com/ hVp://redmine.cloudbees.com/


•  Discard old build records if you can – Helps start-up time, memory usage – Often feasible, for example CI builds

•  Setting is per-project basis

•  The point is to avoid unbounded consumption. Don’t go overboard

#7: Keeping build records under control

Demo


•  7 tips – Backup – Disk planning – Native packages – Distributed builds – Labels – Reverse proxy – Discard build records

Conclusion


•  CloudBees –  http://www.cloudbees.com/

•  Nectar –  http://nectar.cloudbees.com/

•  Try Dev@Cloud –  https://grandcentral.cloudbees.com/account/signup

•  Register for news from CloudBees –  http://www.cloudbees.com/company.cb

•  CloudBees on YouTube: new videos soon –  http://www.youtube.com/user/CloudBeesTV

CloudBees Resources


•  Instructor-led •  More comprehensive, all-round •  London, New York, and Tokyo

•  To receive updates: – http://www.cloudbees.com/company.cb

Coming Soon: 1 Day Hudson Training

Q&A

©2011 CloudBees, Inc. All Rights Reserved

Technology

7 Ways to Optimize Hudson in Production