Upload
techwellpresentations
View
91
Download
3
Embed Size (px)
DESCRIPTION
Test automation scripts are in a constant state of obsolescence. New features are added, changes are made, and testers learn about these changes long after they've been implemented. Marcus Merrell helped design a system in which a "model" is created each time a developer changes code that affects the UI. That model is checked against the suite of automated tests for validity. Changes that break the tests are apparent to the developer before his code is even checked in. Then, when features are added, the model is regenerated and automation can immediately address brand-new areas of the UI. Marcus describes fundamental test design and architecture best practices, applicable to any project. Then he demonstrates this new approach: parsing an application's presentation layer to generate an addressable model for testing. Marcus shows several case studies and successful implementations, as well as an open-source project that can have you prototyping your own model before you leave for home.
Citation preview
A Walk in the FieldsWhy Test Automation Is Really, Really Difficult, and Why It Usually Sucks
The Problem The product is ever-changing Developers do not communicate
changes proactively (or at all) 40% of our time was spent accounting
for these changes
We should be guided by features, not failures
The Solution Move testing further UP the chain, not
down Embed automation testers with regular
developers Use automation to GATE the release Apply some Principles of Real Software
Design to our little corner of the process
What’s in it for me? Managers – Increase your confidence
day-to-day and release-to-release Developers – no more whiny testers,
now they’re whiny PARTNERS Testers – worry about developing NEW
tests, not fixing broken ones
Add a User
Let’s Look at Some Code(Don’t worry about trying to read this, I’m trying to convey quantity, not quality)
Server-side input validation
Velocity Definition This is just for that one button!
DAO Code Hooking the fields to the database
Some SQL
What Was all That??! Recursive Velocity templates Java (with Hibernate annotations) In-depth Velocity macro definitions More Java SQL We didn’t even look at the CSS or
JavaScript
What Does QA Typically Know About This Stuff?
Some Numbers Some analysis and estimation around the
form: 150 lines of Velocity 600 lines of Java 200 lines of SQL 6,000+ lines of shared JavaScript libraries 1,000 lines of shared CSS
Whenever any of this changes, it can affect the functional, presentational, or performance aspects of this form
More Numbers This is one of 8 different forms
surrounding the creation/maintenance of users
User is one of ~200 top-level entities in the system
800 pages Thousands of forms, buttons, and
dynamic screen elements What happens if one thing changes?
Facts are facts Little of the code that does show up in
the browser was written, or would be easily understood, by the developer
It’s either auto-generated by the templates, or was designed by the UX team
What Makes it Complicated for Testing?
Why So Serious? 6 big fat lozenges of code that you
didn’t write Then there’s the “robot” you use to
control it, which you also didn’t write QTP SilkTest Watir Selenium/WebDriver
High Maintenance Tests Everything is in the test, in one big layer
Baby Steps Lower maintenance:
Utility or “Helper” methods Locators stored in xml or properties files
or something
Adolescence Even lower maintenance:
Page Object Model—tests are driven by the functionality of the site
Strongly-typed WebElements Database abstractions—an “API” for
testing Continuous Integration Screen shots upon failures
Adulthood No longer guided by failure:
User-modeled actions—tests are driven by use cases more than pure functionality
Domain-specific language, coupled with strong Page Objects
EventFiringWebDriver to give you pinpoint logging and error handling
TestNG TestListeners coupled with the above for fine-grained failure handling
Nirvana Virtually maintenance-free:
All of the above, plus… Page Object generation, created/updated
at build time whenever the developer makes an update
A Velocity Template
The generated code
Managers: You Want a Strategy? TEST AUTOMATION DEVELOPMENT IS
SOFTWARE DEVELOPMENT Treat it like any other project, but one in
which you *can* control the variables You already know 90% of what you need to
know Just make the connection and make the
commitment Insist on automation involvement from the
earliest point
Engineers: Now what? Make a connection with your automation
resources Find areas where your presentation
layer could be more consistent Prototype on a simple page or single
element type Figure out how you can communicate
change via continuous build instead of hallway conversations
Testers Now you don’t have to chase down
changes You understand much better how the
software gets built You can provide metrics and perspective
you couldn’t before Your code base is tied to theirs—sounds
dubious, but turns out to be awesome
Pros Maintenance goes way down Automation code is now tied to
developer code Including version-specific tests
QA now has skin in the development game
Development now has skin in the automation game
Cons Heavy up-front development time
But you should be quick-to-fail Need a higher level skill set than the
traditional QA role Difficult to implement on non-templated
systems Velocity, JSF, JSP, ASP, wicket OK PHP, JavaScript, Java-based HTML not so
easy
The Risk? “If they plugged up the leaks that allow
us to hack FireFox, this whole thing would go away.”
“Developers are a huge risk to test automation. They shouldn’t be allowed near the codebase.”
—Simon Stewart
Other ideas Could you somehow track the changes,
to get an idea of “churn rate” in your presentation layer?
Compile to some kind of meta-step, so the target, rather than being pure Java, could interface with anything?
Standard for “Object Repository”, maybe as an addition to Selenium?
Questions?