Upload
dimakovalenko
View
1.945
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Talk at 2013 Selenium Conference on the topic of stabilizing a build and making it green over all
Citation preview
Love My Job
• Love testing• Love hacking• Love talking about above
Partner In Crime
Darko MarinovAssociate ProfessorUniversity of Illinois
Stability is a Journey not destination
Culture
Culture Changes
• Convince Management that flakey tests waste a lot of money
• Discourage developers from relying on tests• Prevent red build from merging into
master/trunk• Red master is NOT OK!
Fix Ajax Waiting
• jQuery– evaluate_script("jQuery.active") == 0– evaluate_script('$(":animated").length') == 0
Flakey Tests
Self contained tests
• Separate Databases• Self reliant tests– Setup/Teardown
Faster Feedback Loop
• Fight to get build to 10 mins• Notify of failures before build is finished– Rspec/Cucumber Early Notification Formatters– Blog post to follow– engineering.groupon.com
Flakey Detector
• Check Jenkins or Git diff• If a test was modified, kick of flake detector
build– Run changed test 5k times in 10 mins– In random order– They should always be green!
• engineering.groupon.com
Fix Environment Problems!!
• Tests get blamed if environment fails• Standardize test boxes– Chef, Puppet, w/e
Flakey Environment
Windows Tips
• Don’t use IE• Move mouse out of browser in IE• Maximize test window• Don’t use RPD/RDC– Use VNC
Linux Tips
• Run Firefox in separate Desktop
Speed up tests
• Parallel more• Does it have to be Selenium?– Can this be a unit test?– Can it be headless?– Can you run Jasmine instead?
Black Hole Proxy
• Capture any external requests, return 200• Speeds tests up• Reliability goes up• em-proxy– https://gist.github.com/dimacus/5757573
Gem Cacher
• Cache any external dependencies locally• Don’t rely on rubygems.org to always be up!– github.com/dimacus/gem-cacher
• Local Maven repo
Credit on Grid idea
• David Burns• @AutomatedTester
Selenium Grid Extras
• github.com/groupon/Selenium-Grid-Extras
Basic Idea
• Help maintain the Environment to produce stable tests
• Turn Selenium Grid Nodes into dumb browsers
Control OS from tests
• Clean up old sessions• Kill browsers• Take full OS Screenshots• Move mouse• Reboot windows
Future plan
• Record videos• Share files with grid nodes• Spin up/down VMs on the fly• Image comparison• Auto update Selenium WebDriver• Black-hole proxy integration• Improve Grid API• Restart SE Node• Idle machine utilization
Need your help
• https://github.com/groupon/Selenium-Grid-Extras
Questions?
42!