Embed Size (px)
DESCRIPTION
This presentation on Continuous Delivery is from the November 2013 Automated Testing San Francisco meetup that took place at Constant Contact. The author/presenter is Matt Wilson, CTO of Lab Zero. Matt has advised clients at various industries including consumer brands, non-profits, start-ups, and financial services on Agile development, web application development, and other technology leadership challenges. This overview on Continuous Delivery highlights some of the best practices that Lab Zero has distilled, based on their many client engagements. --- About Matt Wilson: Matt is an enthused agile developer, architect, and consultant. He enjoys building elegant web services in Ruby. He believes that high-fives are underrated and measures the success of his day by how many he's seen. Prior to joining Lab Zero, Matt's work history includes: Co-founder/Architect at Earfl.com, Architect at Kodak Gallery, Developer at Westwave Communications, Engineer at Motorola, and Developer at Coldwell Banker. About Lab Zero: Lab Zero Innovations, Inc. provides web application development and technology leadership consulting. Our client relationships include staff augmentation, pure software development, project management, system integration, advisor/leadership roles. Contact us about your next project.
Citation preview
LAB ZERO INNOVATIONS, INC.
PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111
T:415.839.6861 F:415.839.6901 [email protected] HTTP://LABZERO.COM
Don’t Hate, Automate.Lessons learned from implementing Continuous Delivery.
Matt Wilson, CTO Lab Zero
Who we are
Our evolution toward Continuous D*
What we’ve learned along the way
Questions & Answers
Overview
LAB ZERO INNOVATIONS, INC.
Product development, design, and strategy team based in SF.
Early-stage entrepreneurs and startup founders: earfl.com, meez.com, gofish.com, echosign.com, veodia.com.
Held senior product design & dev roles at Apple, Google, Yahoo, Kodak, Accenture.
Our clients are often startups and Fortune 500s.
Founded in 2008
Many of our clients hire us just to build out their Continuous D* rigs.
Who We Are
LAB ZERO INNOVATIONS, INC.
Our path to Continuous D* and what we’ve seen
Just a build…
We had code and compiled it. We even had some unit tests for some areas of the code.
The official build ran nightly but we certainly weren’t using it for fast feedback.
Then we started deploying with automation. “cap production deploy” Those were some fancy pants.
Then we started running our tests on every check in.
Then we needed to automate the validation of our app running in different environments
Then we started automating the creation of our environments. Just can’t have enough.
In the beginning there was
LAB ZERO INNOVATIONS, INC.
We have to start new projects very often.
The teams we collaborate with want to see progress often.
Projects started without a clear delivery process were often slower to deliver over time.
At some point manually running our deployment scripts every time we wanted to share our work seemed like too much hassle. </first_world_problem>
We need to know that the feature works in the QA environment.
We figured out how to get Continuous D* running on our projects in less than a day.
Necessity struck
LAB ZERO INNOVATIONS, INC.
Our Webapp “Build”
Cucumber:
We write our ATDD suites in Cucumber and use Capybara to actually drive browsers.
Rspec:
Unit tests and generates code coverage reports.
Jasmine + PhantomJS:
Javascript unit testing.
Brakeman:
Static security analyzer to help us find obvious security vulnerabilities.
Capistrano:
Automated deployments of RoR, PHP, Java... applications.
LAB ZERO INNOVATIONS, INC.
Our Mobile “Build”
Cucumber:
Using Calabash to actually drive the native devices and emulators.
OCUnit, JUnit:
iOS and Android unit testing frameworks.
Xcodebuild, Ant+javac:
iOS and Android command line tools.
TestFlight:
Deploy every passing build to TestFlight for easy distribution to team devices.
LAB ZERO INNOVATIONS, INC.
The story starts at your laptop and ends in production.
LAB ZERO INNOVATIONS, INC.
TDDthe first “d” is for “driven”
Build Failed
“When the light is green, the trap is clean.” --Ghostbusters
Build your culture of automation awareness.
There is a correlation between writing tests and running them. Those that write them tend to run them more often.
Developers and QA both need to run tests before committing.
If developers aren’t running the tests because they take too long, it’s worth the investment to speed them up.
CD can fall apart quickly if the team doesn’t react early to failed builds.
It’s hard to keep the light green or meaningful if tests are written afterwards.
LAB ZERO INNOVATIONS, INC.
Speed will matterwhen you’re doing it right
How long does it take to deliver a feature to QA?
LAB ZERO INNOVATIONS, INC.
We want to run as many tests as we can but once we’ve invested in a fair amount of browser and emulator testing they get really slow.
When they get slow we become reluctant to run them.
Tests that don’t run, can’t find bugs.
The longer you wait to run them all, the less likely they are to still pass.
Tests that are broken and remain unfixed become dead weight.
Releasing under-tested products often means we’re asking too much of our QA teams and usually won’t find bugs until they are in production.
Ouch, my tests!
LAB ZERO INNOVATIONS, INC.
Heavy use of fixtures and db cleaning (factory-girl and database-cleaner)Consider using mocks/stubs instead of db’s when possible.
Android emulatorGet a real device and plug it into your CI server.
Assets compiled on each server at deploy timePre-compile your static assets on the build server and deploy archives instead.
Usual suspects for slowness in the pipe
LAB ZERO INNOVATIONS, INC.
Selenium tests are just too slow to not do thisRemote browser tests will need you to run multiple tests at once. Especially when you start considering running your tests in more than one browser/OS combo.
To do this well you should lean on the PaaSs out there likeSolano CI (beaucoup parallelization-fu), Sauce Labs (browsers and mobile emulators, can be used from Solano), Travis CI, etc...
Unit tests will need it tooIf you’re doing TDD, you likely already have this problem.
Parallulala, parallaluh, parallelize them
LAB ZERO INNOVATIONS, INC.
Continuous D* wants to use your test automationacross many environments
CI Server Run tests here first to find anything that should prevent us from deploying the app. Unit tests, static analysis,
DevInteg First deployed spot for the app, the environment will be volatile and will serve little use other than for CI. May also serve as QA server if the numbers of commits are low.
QA1, 2, 3..You may need one of these for each product team.
StagingRuns in production and is often used as a place to validate the app in production before going live.
ProductionLive.
You’ll need a few environments to pull this off.
LAB ZERO INNOVATIONS, INC.
We want to get the features out, but they have to be verified in QA and Staging before we go live.
LAB ZERO INNOVATIONS, INC.
Co-own yer ATDDwith QA
Share the same ATDD codeDevs and QA should collaborate on their functional testing investment. Many organizations separate these groups activities which leads to redundant investments.
Devs need to lead in the spirit of TDDRelying only on QA for functional automation causes flapping red->red->green->red->green builds.
It works better when you work togetherDevs can do the heavy lifting on writing reusable steps and tests. QA can add more cases if they see fit. Devs should expect to train and support QA on the tools.
Allow QA to auditWith our QA hats on, let’s have an open conversation about what should be tested and what is missing. This is how we can focus our beams to make the light brighter.
Force multiplier
LAB ZERO INNOVATIONS, INC.
Running tests just one thing your CI server can do
Empower folks with a magic buttonThere are likely things that your team is always asking someone else to do for them. De-spof it by making a job that folks can click whenever needed, even from their phones. Zomg seriously.
Database RefreshesNearly every webapp project needs to be able to reset/refresh the dev/qa/staging databases easily.
Production deploy jobsThis is your parent’s liquor cabinet: make sure the kids don’t get into it, but make it easy to open when needed.
Robots, ACL’s and users, what could go wrong?
LAB ZERO INNOVATIONS, INC.
Thank You!Questions?
[email protected]/aim/yim/skype: w1150n
LAB ZERO INNOVATIONS, INC.
PIER 9, THE EMBARCADERO SAN FRANCISCO, CA 94111
T:415.839.6861 F:415.839.6901 [email protected] WWW.LABZERO.COM
