View
738
Download
0
Category
Tags:
Preview:
DESCRIPTION
This was a full-length talk presented by Aman King in vodQA-3 : A QA Meet held in ThoughtWorks, Pune.
Citation preview
Saturday, 26Saturday, 26thth March, 2011 March, 2011
Reducing build timewhen patience is not a virtue
Aman KingApplication Developer
ThoughtWorks
Recognize these?
Rank these
Did you choose this?
1 32
Or did you choose this?
1 23
Patience is not always a virtue!
Fail fast!quick feedback
Rank these
High functional coverage
Short build time
Did you choose this?
①High functional coverage
②Short build time
Or did you choose this?
①Short build time
②High functional coverage
You can choose both!
①Short build time
①High functional coverage
But before we see how…
Project background
Content driven, community oriented
website
Ruby on Rails
Cucumber+
Capybara+
Selenium
4 : 1 ratio of Dev : QA
Everyone writes functional tests in a common automation
suite
Acceptance Test Driven Development (ATDD)
Our problems
Long build time~ 55 minutes
Non-deterministic failures
Manual reruns needed
Our solution:Parallelization
Basic idea
Reduced build time~ 5 minutes
Build time chart
What we did
Parallelization on a single multicore machine
used Ruby library parallel_tests
Report generation in parallelwrote custom report formatter
Isolating databaseseach process needs its own database
database.ymltest: &test adapter: mysql2 encoding: utf8 reconnect: false database: myapp_test<%= ENV['TEST_ENV_NUMBER'] %> pool: 5 username: root password:
Isolating external dependencies (Solr)
each process needs its own Solr instance
sunspot.ymltest: solr: hostname: localhost port: <%= 8982 - ENV['TEST_ENV_NUMBER'].to_i %> log_level: INFO #WARNING log_file: <%= File.join(::Rails.root, 'log', "solr_#{ENV['TEST_ENV_NUMBER'].to_i}.log") %> data_path: <%= File.join(::Rails.root, 'solr', 'data’, "#{ENV['TEST_ENV_NUMBER'].to_i}") %> pid_path: <%= File.join(::Rails.root, 'solr', 'pids', "#{ENV['TEST_ENV_NUMBER'].to_i}") %>
Handling multiple Firefox instances
and Selenium’s use of
shared ephemeral ports
Monitoring Memory and CPU usage
adjust number of parallel process accordingly
Auto rerunning failed testsnon-deterministic failures are likelier with
the CPU under stress
Beyond parallelization…
Stubbing external callsreduce dependency-related delays where
avoidable
Consolidating scenarioscombine similar test scenarios into single
runs instead of separate tests
Maintaining conventions when writing automated tests
avoid time-based wait statements, use test framework APIs that take lesser time, etc
Conclusion
Resourceshttp://cukes.info
http://rubygems.org/gems/capybara
http://seleniumhq.org/projects/remote-control
http://rubygems.org/gems/parallel_tests
http://selenium-grid.seleniumhq.org
http://test-load-balancer.github.com
http://github.com/bblimke/webmock
http://testing.thoughtworks.com
Thank you
king@thoughtworks.comhttp://www.wikyblog.com/AmanKing
Recommended