Jenkins Evolutions

Anton ArhipovZeroTurnaroundJRebel

@antonarhipovhttp://arhipov.blogspot.com

Why Evolutions?

Why Evolutions?

test automation

Why Evolutions?

continuousintegration

Why Evolutions?

continuousdeployment

Why Evolutions?

cron on steroids

Agenda

Background

Stories• Functional tests• Servlet tests• Scaling

Random stuff

Background - 2008

First installation

Clicked “Run” when wanted to test stuff

Background 2012

54 215 324 jobs2 5 8 dedicated machinesAmazon ~7000 instance-hour per month3 in house plugin developedCustom dashboard (50 x 300 grid)

https://github.com/ jenkinsci/ jenkins-scripts

Custom Dashboard

story of functional tests

JRebel Functional JDK Tests

JRebel Functional JDK Tests

JRebel Functional JDK Tests

Per JDK• Per functional test• Compile couple of classes• Run some methods and assert• Recompile couple of classes• Run some methods and assert• …

JRebel Functional JDK Tests

Per JDK• Per functional test• Compile couple of classes• Run some methods and assert• Recompile couple of classes• Run some methods and assert• …

JRebel Functional JDK Tests

Per JDK• Per functional test• Compile couple of classes• Run some methods and assert• Recompile couple of classes• Run some methods and assert• …

JRebel Functional JDK Tests

Per JDK• Per functional test• Compile couple of classes• Run some methods and assert• Recompile couple of classes• Run some methods and assert• …

JRebel Functional JDK Tests

Per JDK …Which Jenkins feature is the best here?

JRebel Functional JDK Tests

Configuration matrix FTW!

JRebel Functional JDK Tests

Executed with a Groovy script

JRebel Functional JDK Tests

JRebel Functional JDK Tests

How to get results? Jenkins feature…

JRebel Functional JDK Tests

Output JUnit XML!

JRebel Functional JDK Tests

JRebel Functional JDK Tests

Creates ~ 10 000 new filesCompile ~ 1700 java classesInvoke the JVM ~ 550 times

JRebel Functional JDK Tests

Sloooooooow!

JRebel Functional JDK Tests

I/OCPUMemory

JRebel Functional JDK Tests

JRebel Functional JDK Tests

Bare metalCloudBare metal (+ cloud)

JRebel Functional JDK Tests

Cloud• 1 small instance per JDK• Manual update of AMIs• Slow• Scales!

JRebel Functional JDK Tests

Which Jenkins plugin to use?

JRebel Functional JDK Tests

Amazon EC plugin

JRebel Functional JDK Tests

Bare metal• 8 core & 16GB RAM gaming

machines• Recycled developer machines• Marketing demo machines (iMac)• Developer machines, ad-hoc (?)

JenkinsMaster

JenkinsMaster

WindowsWindowsWindows

*nix*nix*nix*nix*nix

JenkinsMaster

WindowsWindowsWindows

*nix*nix*nix*nix*nixAWS

JenkinsMaster

WindowsWindowsWindows

*nix*nix*nix*nix*nixAWS

My Laptop

iMac

Adding new machines

How to manage the configurations?How to automate the setup?

Adding new machines

Chef, Puppet

Taming I/O

SSD: ~30% performance gain

story ofservlet tests

Servlet Tests

Start a container version• WebSphere, Weblogic, JBoss, etc

Deploy applications

Test results, reload changes, test results, …

Custom Dashboard

Servlet Tests

How to start a server?

Servlet Tests

Duh! With a startup script!

Servlet Tests

Problem: 1 app server, 2 jobs. Which plugin to use?

Servlet Tests

Throttle Concurrent Builds plugin

Servlet Tests

Servlet Tests

How do you deploy an application?

Servlet Tests

Start TomcatCopy WAR files to webapps/GET /some-urlAssertUpdate applicationGET /some-urlAssert

Servlet Tests

Sloooooooow!

Servlet Tests

I/OCPUMemory

Testing Multiple Containers

Cloud• 1 small instance per container• Need to update AMIs• Slow• Isolation

Testing Multiple Containers

Single machine• Port hell• Memory limits• Fast(er)!

story ofDVCS

Story of DVCS

Problem• Many servers• Many containers• Many tests• Many JDKs

Slow feedback cycle

Story of DVCS

DEVN tests on N servers

(30 minutes)

STABLEM tests on M servers

(3 hours)

Tests green?Push to branch/repository

Story of DVCS

Story of DVCS

scalability

Story of Scaling

Problem• Exponential growths of tests

• (containers) X (configurations) X (tests) + const

• Fight for executors

Story of Scaling

Add more machines?Spin up more cloud instances?Attach developer machines at night?Cut tests scope?

Story of Scaling

Jenkins helps with elasticity!

Story of Scaling

Jenkins helps with elasticity!

WIN!

random stuff

Random Stuff

There’s an app plugin for that!

Some Cool Plugins

Mercurial pluginThrottle Concurrent Builds pluginMatrix Configuration pluginMatrix Reloaded pluginSSH slaves pluginPublish over SSH pluginCopy Artifact pluginCopy to slave plugin

Some Cool Plugins: Continuous Deployment

Build Pipeline pluginParametrized Trigger pluginDownstream Buildview pluginFingerprints

http://zeroturnaround.com/blog/how-to-use-jenkins-for-job-chaining-and-visualizations/

Some Cool Plugins

Chuck Norris plugin

BUILDSUCCESSFUL

Some Cool Plugins

Chuck Norris plugin

BUILD FAILED

Build-it-all!

JRebel

Build-it-all!

JRebel

Build-it-all!IntelliJ

NetBeans

Eclipse

Installer

JRebel

Build-it-all!IntelliJ

NetBeans

Eclipse

Installer

JRebel

Build-it-all!IntelliJ

NetBeans

Eclipse

Installer

JRebel

Tests

UI

Questions?