NoVaTAIG

April 7, 2010Automated In-Browser Testing with

Cucumber and Watir

Rudy Regner Mike Perz Erik Scheirer Rasik Pandey

Our Situation

• Writing regression tests for an app in production– Only required to support IE

• Rails v1.2.3– Precludes using Webrat latest version– ?? Could possibly have used earlier version ??

• why regression tests for earmarks?– Bug made it through manual testing – Encapsulate critical functionality in automated

regression test scripts to decrease likelihood of this failure occuring again.

Our Learning Curve• Started with Watir

– Easy to get up and running• Added Cucumber (RSpec) • Replaced Watir with Webrat and Selenium

– Sought flexibility gain through re-use of Webrat to drive Selenium, Mechanize or straight Rails

• Productivity dropped with Selenium– Script debugging difficult without irb

• You can attach to an browser window* (with Watir)– Watir can manipulate the DOM as objects. (you can iterate

through the cells of an html table)– With Selenium you would have to find each cell individually.

• Converted existing Selenium code to Watir

Once we switched back to Watir

• At first our tests were dependent on each other– They needed to be run in a particular order. – We used Rake to accomplish this.

• Open 4 browsers with 4 different users (Watir::IE.new_process vs. Watir::IE.new)

– Now we can attach Watir to whichever browser session we need

• Watir::IE.attach

– We use SQLite to keep track of which user is associated with which browser session

Code Samples• Cucumber

– feature file– Background– Scenarios– Scenario Outline (tables)

• Webrat• Watircuke• Selenium• Step Definitions

Interesting side benefits to writing automated tests

• Writing automated tests helped/made us understand the app better.

• Found application bugs that were missed earlier

Other useful tools

• Watircuke• IE Developer Toolbar• Firebug• FireWatir recorder:

www.itest2.com/downloads

Cucumber as the common artifact

Ideal BDD Process:

• 1. Describe behaviour in plain text• 2. Write a step definition in cucumber• 3. Run and watch it fail• 4. Write code to make the step pass• 5. Run again and see the step pass• 6. Repeat 2-5 for all your functionality

Performance:

• Since it takes 3 hours to run a test suite we should investigate using spork and --drb and testjour to distribute the test load if possible to speed up test runs.

Test Web apps in any language: Drive a full or headless browser using one of these

• *Webrat – Ruby acceptance testing for web applications • *Capybara – Acceptance testing framework with a webrat-like API and support for multiple

backends, including RackTest, Selenium, Celerity and Culerity • *Steam – Drives a fast headless browser with Javascript support. Support for the normal

webrat step definitions, see Setting up Steam • *Testing PHP app using Webrat • *WebDriver – Drives IE, Firefox, Chrome • *Watir – Drives IE (Windows only), see: Setting up FireWatir • *FireWatir – Drives Firefox on Windows, OSX and GNU/Linux • *SafariWatir – Drives Safari (OS X only) • *ChromeWatir – Drives Google Chrome • *Celerity – Drives a fast headless browser with Javascript support. Examples here • *Culerity – For when you can’t run your app under jRuby • *Selenium – Runs any browser (any OS), see: Setting up Selenium • *Mechanize – Runs a headless browser (any OS)

Examples of Cucumber being used in the real world across

various languages/platforms to test all sorts of things.• http://wiki.github.com/aslakhellesoy/cucumb

er/tutorials-and-related-blog-posts