Test automation day 2015: ask the right questions

Ask the right questions!

Asking only“What is the best tool?”

is asking for trouble

Martin GijsenIndependent test automation architect

Test automation day 2015

© 2015 Martin Gijsen

Overview

The PuPPET factors Applying the PuPPET factors

© 2015 Martin Gijsen

The PuPPET factorsTest automation business goals

Test automation approach

People

Processes

Policies

Environment

Technology

Continuity

© 2015 Martin Gijsen

People

A test tool is no apple tree Test automation = testing + automation Multiple skills, but always people Easy to use = effective? What about the maintenance phase?

© 2015 Martin Gijsen

ProcessesAs the head of salesI want large orders to receive free shippingSo that clients will buy more per order

Order size Domestic? Total amount Free shipping?

1 Yes 99 No

9 Yes 99 No

10 Yes 99 Yes

1 Yes 100 Yes

10 No 100 No

This 'specification by example' can serve as: Acceptance criteria Automated acceptance test Automated regression test Living documentation Helpdesk tool

© 2015 Martin Gijsen

Policies

Open source? Real support? Vendor lock-in?

Order size Domestic? Total amount Free shipping?

1 Yes 99 No

9 Yes 99 No

10 Yes 99 Yes

1 Yes 100 Yes

10 No 100 No

Interface tool

Engine

Logic

Test object

Facade / API

© 2015 Martin Gijsen

Environment: smart stubs / mocks

Test object

Test case

Stub 1 Stub 2

Automation

© 2015 Martin Gijsen

Environment: test resources

Add order for client JohnDoe.comAdd order for client Jane Doe Inc.

Resources:X : clientY : clientX.city = Y.city

Add order for XAdd order for Y

Resources:Client X, client YX and Y are located in the same city

Add order for XAdd order for Y

Environment 1

JohnDoe.com

Jane Doe Inc.

Environment 2

JohnDoe.comSomething else

© 2015 Martin Gijsen

System X

Technology

Test caseSystem X UI

System X WS

System X DB

Logic

UI module

Tool WS module

DB module

System YLogic UI moduleTool

Test case

Test caseOrch

estratio

ntool

© 2015 Martin Gijsen

Continuity

Maintenance: Structure & technical quality Death to the hero! Use consultants, but train employees Reuse tools & testware in projects No prose version of automated test case

© 2015 Martin Gijsen

How?

Consider ROI only for large investments Small steps = low risk + quick result What hurts the most? Expect a productivity dip

It depends. Always. So:

1) Ask the right questions(use the whole PuPPET)

2) Find the right answers

3) Achieve lasting success

Conclusion

© 2015 Martin Gijsen

Thank you!