Jenkins User Conference San Francisco #jenkinsconf
Ruby CI with Github,
Jenkins, and
EC2 Spot Instances
John Russell (@jjrussell)
Director of Tools at Tapjoy
http://tech.tapjoy.com/engineering-blog
October 23, 2014
#jenkinsconf
Jenkins User Conference San Francisco #jenkinsconf
Who am I?
●John Russell
●Director of Tools at Tapjoy
●We make tools that make it so that
product engineers can do the right thing
Jenkins User Conference San Francisco #jenkinsconf
This is a story about…
●Ruby and Rspec?
●Unit testing?
Jenkins User Conference San Francisco #jenkinsconf
This is a story about…
●Ruby and Rspec?
●Unit testing?
●Spot instances on AWS?
Jenkins User Conference San Francisco #jenkinsconf
This is a story about…
●Ruby and Rspec?
●Unit testing?
●Spot instances on AWS?
●Parallelization?
Jenkins User Conference San Francisco #jenkinsconf
This is a story about…
●Ruby and Rspec?
●Unit testing?
●Spot instances on AWS?
●Parallelization?
●Money?
Jenkins User Conference San Francisco #jenkinsconf
This is a story about…
●Ruby and Rspec?
●Unit testing?
●Spot instances on AWS?
●Parallelization?
●Money?
●Jenkins?
Jenkins User Conference San Francisco #jenkinsconf
This is a story about…
●Ruby and Rspec?
●Unit testing?
●Spot instances on AWS?
●Parallelization?
●Money?
●Jenkins?
●Intrigue?
Jenkins User Conference San Francisco #jenkinsconf
This is a story about…
●Ruby and Rspec?
●Unit testing?
●Spot instances on AWS?
●Parallelization?
●Money?
●Jenkins?
●Intrigue?
●Betrayal?
Jenkins User Conference San Francisco #jenkinsconf
We can parallelize this!https://github.com/grosser/parallel_tests
Jenkins User Conference San Francisco #jenkinsconf
Thousands of legacy tests can’t run at the
same time on the same DB
because…reasons.
Jenkins User Conference San Francisco #jenkinsconf
Thousands of legacy tests can’t run at the
same time on the same DB
because…reasons.
But this could have been any
parallelization problem…
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub
gh
prb
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub
gh
prbWebhooks
● Permissions
● Webhook events (PR not just commit)
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub Webhooks
gh
prb
My PR is
awesome!
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub Webhooks
gh
prb
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub Webhooks
gh
prb
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub Webhooks
gh
prb
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub Webhooks
gh
prb
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
Jenkins spec_buildflowspec files
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
Jenkins spec_buildflowall spec files
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
Jenkins spec_buildflowall spec files
some spec files
spec_runner
some spec files
spec_runner
some spec files
spec_runner25 times
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
some spec files
spec_runner
Jenkins
some spec files
spec_runner
some spec files
spec_runner25 times
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
some spec files
spec_runner
Jenkins
Wow, that’s a lot of
jobs...
some spec files
spec_runner
some spec files
spec_runner25 times
Jenkins User Conference San Francisco #jenkinsconf
EC2 Plugin - Bring in the spots
some spec files
spec_runner
Jenkins
AWS
ec2
Jenkins User Conference San Francisco #jenkinsconf
EC2 Plugin - Bring in the spots
some spec files
spec_runner
Jenkins
AWSHey, I can run
these jobs on a
spot instance for
really cheap!
ec2
Jenkins User Conference San Francisco #jenkinsconf
EC2 Plugin - Bring in the spots
some spec files
spec_runner
Jenkins
AWS
ec2
create new node and
request spot instance
Jenkins User Conference San Francisco #jenkinsconf
EC2 Plugin - Bring in the spots
some spec files
spec_runner
Jenkins
AWS
ec2
create new node and
request spot instance
grant spot instance
Jenkins User Conference San Francisco #jenkinsconf
EC2 Plugin - Bring in the spots
some spec files
spec_runner
Jenkins
AWS
ec2
create new node and
request spot instance
grant spot instance
start job on new spot instance
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
some spec files
spec_runner
Jenkins
some spec files
spec_runner
some spec files
spec_runner25 times
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
Jenkins spec_buildflowall spec files
some spec files
spec_runner
some spec files
spec_runner
some spec files
spec_runner25 times
test results
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
Jenkins spec_buildflowall spec files
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
Jenkins spec_buildflowall spec files
Murder some puppies in order to
aggregate test results.
Jenkins User Conference San Francisco #jenkinsconf
Jenkins Build Flow Plugin
Jenkins spec_buildflowtest results
Jenkins User Conference San Francisco #jenkinsconf
Github Pull Request Builder Plugin
JenkinsGithub
gh
prb
Update PR status
Jenkins User Conference San Francisco #jenkinsconf
Phenomenal Time Vortices
● Spot instance AMI
○ Reverses Jenkins node model
○ Automation can crash spot market
Jenkins User Conference San Francisco #jenkinsconf
Phenomenal Time Vortices
● Spot instance AMI
○ Reverses Jenkins node model
○ Automation can crash spot market
● Github
○ Webhooks config
○ Permissions
Jenkins User Conference San Francisco #jenkinsconf
Phenomenal Time Vortices
● Spot instance AMI
○ Reverses Jenkins node model
○ Automation can crash spot market
● Github
○ Webhooks config
○ Permissions
● The craziest fscking but I’ve seen in a while
○ https://github.com/jenkinsci/ec2-plugin/pull/106
Jenkins User Conference San Francisco #jenkinsconf
The Magic Pixie Dust
● Jenkins plugin authors
● Build Flow Plugin groovy glue
Jenkins User Conference San Francisco #jenkinsconf
The Magic Pixie Dust
● Jenkins plugin authors
● Build Flow Plugin groovy glue
● Scriptler
Jenkins User Conference San Francisco #jenkinsconf
The Magic Pixie Dust
● Jenkins plugin authors
● Build Flow Plugin groovy glue
● Scriptler
● Spot instances
○ Pick the right AWS Region Spot market
Jenkins User Conference San Francisco #jenkinsconf
@jjrussell
http://tech.tapjoy.com/engi
neering-blog