Sustainable Automation Frameworks by Kelsey Shannahan

Sustainable Automation Frameworks

© 2016 CoverMyMeds LLC. All Rights Reserved.

Time to Get Serious


• The ultimate goal of testing is to allow our products to reach production faster and safer.

• We can’t do this if we’re sinking time into maintaining our test suite.• This helps no one.

• Our automated test framework should be lightweight and easy to resolve problems.• This allows us to get back to the more important task of

actually ensuring the quality of our applications.

How do we get there?


• Page-object model• Step definition reusability• Data management• Other topics as my abstract promised

The Problem: Hardcoding


• There’s a lot of STUFF in the THING you’re testing• Lots of pages and objects on a web app• Lots of services in an API

• Straight out of the box Google, most frameworks don’t have a good way to manage this STUFF

• Then, when something changes, there is no one central repository to update and we wind up having to make fixes EVERYWHERE

The Solution: Page-Object Model


• So we call it page-object… but really it’s applying object orientation

• For each logical group we’re testing, we have a class that describes it

• This works for web apps• One page, one class

• Or for APIs• One service, one class

What does this do for us?


• Makes our code reusable• We can even package it into a gem and share with other

areas• Makes our code easy to update

• One change can fix every failing test• Makes our code easy to understand

• The code now reflects the system that is under test in a logical manner

The Problem: Too Many Step Definitions


• Someone creates some step definitions that do the thing.• Someone else creates another step definitions to do a thing.• Nobody checks to see if there’s an existing step that can do

the thing.• Everyone just keeps adding more step definitions until

there’s too many for a reasonable human being to search through.

Why does this happen?


• Step definitions aren’t written to do what they say.• Sure, there’s a step that says it does that… but it only

partially does it. Or does some other thing. And it’s not what you actually need.

• Step definitions are unclear.• You could spend an hour figuring out what that thing is

supposed to be doing… or you could spend ten minutes writing your own step.

• No one polices the step definitions.• Because we all really hate maintaining our test

framework.

The Solution: No One Quick Fix


• Standards• Have a method of organization for your library• Define at what level you want your steps to be written• Everyone has to follow these!

• Step definitions ARE NOT CODE• Structuring step definitions to represent coding logic

results in many incomprehensible steps• Write step definitions to represent higher level business

logic

The Solution: No One Quick Fix


• Review your step definition library regularly• This should be a quick check that can be done as part of

the normal process of writing tests• If you’re having to make time or use special tools to

manage your library… your library is probably too big• Get it under control and keep it under control

• Abstract your steps• Don’t make your steps dependent on data• Instead, let data drive the test

The Problem: Data Management


• If your framework consists of objects and your steps are flexible and business-oriented… data becomes the driver for testing

• Data is no longer locked in as part of the step definition, so we can load different data for each test

• So how do we manage large amounts of data?

The Solution


• Have a consistent, easy-to-use way of loading data into your test• Helper methods?• Dump it into a fixture in a hook?

• Use descriptive naming for your data sets• Reuse data where possible

• Make default data for stuff you don’t care about• Creating a new test becomes as simple as changing the

data around

How to Keep it Pretty


• Be disciplined• Don’t put off tomorrow what you should really be doing

today• Have a hierarchy of changes

• Change the data• Change the gherkin• Change the step definition• Change the page-object

How to Keep it Pretty


• Know what’s in your test suite and trust it to do it’s job• Use all the resources available

• Don’t write unit tests with a GUI-driven test suite• Not everything needs an automated test

Technology

Sustainable Automation Frameworks by Kelsey Shannahan