Beyond the GUI

7/28/2019 Beyond the GUI

1/31

Testing Beyond the GUI with Rational

Visual Test

Daniel Rybowski

IT ManagerHitext


2/31

Based on real world experiences...

Explain the limits of the Capture/Playback test

automation

Examine an alternative approach to GUI test

automation

Outline the implementation of a test

automation tool with Rational Visual Test

Objectives of this session


3/31

Presentation outline (1/4)

The context

An application under test: architecture

and technology

A test strategy: scope of the tests, test

methodology


4/31


Initial approach: test automation withSQA Robot

Selecting the test tool

Our experience with Capture/Playback

Why we failed


5/31


A new approach: the Programmed model

What is it?

Comparison with Capture/Playback

A new test tool: Rational Visual Test


6/31


The Programmed model in practice: the test

driver Pillory

Global design

Using Rational Visual Test features

HiTest: the future of Pillory


7/31

The context (1/5)

An application under test: architecture and

technology

Client/Server software

Client-side applications for Windows 3.1

and 95

Multi-lingual Client applications


8/31

The context (2/5)

An application under test: architecture and

technology

Server-side applications for Windows NT

Client and Server databases

GUI based on client-customizable viewsand custom controls


9/31

The context (3/5)


methodology

Scope of the tests: conformity of the

software with its specification

No performance, usability,... testing

Behavioral (black box) tests only


10/31

The context (4/5)


methodology

Test design based on the specified

business rules

RAD software development

methodology with incremental builds


11/31

The context (5/5)


methodology

Decision: the tests must be automated


12/31

Initial approach: test automation with SQA Robot (1/4)

Selecting the test tool

The tests cannot be automated in

advance:

custom controls

specifications subject to change

Decision: use a Capture/PlaybackCAST tool


13/31


Our experience with Capture/Playback

Creation of the test cases

Verification of the test results

Maintenance of the test cases

Verification of the fixed defects


14/31


Why we failed

Functionality changes of the application

under test

GUI changes of the application under

test

Custom controls handling


15/31


Why we failed

Sensitivity to environmental changes

Sensitivity to change of locale

Atomicity of the recorded test cases

Test engineers need a wide expertise


16/31

A new approach: the Programmed model (1/9)

What is it?

A new model of GUI test automation

Separates the test case logic from the

interaction with the application

Test Designer and Test AutomationEngineer are distinct roles


17/31


What is it?

Three-tier model:

test cases description in high level

script language

generic test driver, interpreting the

test scripts

low-level primitives to interact with aspecific application under test


18/31


What is it?

Test scripts in high level language

GUI access primitives

Test driver


19/31


What is it?

The driver collects and logs test results

The test scripts are independent of any

GUI changes

The GUI access primitives areindependent of any functional changes


20/31



The Programmed model has up-front

costs:

The test driver (once)

The primitives (once per application

under test)


21/31



The Capture/Playback model has higher

implementation costs

The Capture/Playback model has higher

maintenance costs


22/31



The Capture/Playback model can lead to

failure:

for functional tests

if the application is big

if the functionality is complex

if the development cycle isincremental


23/31



The Capture/Playback model can lead to

failure:

if the functionality changes often

if the GUI changes often

if there are custom controls

if the application is localized


24/31


A new test tool: Rational Visual Test

Strong programming environment

Powerful language primitives

Interfaces with a version control system

Inexpensive

The Programmed model in practice: the test driver


25/31


Pillory (1/5)

Global design

Master/Slaves architecture to allow for

Client/Server testing

Single test script for all test stations, run

by the Master

Test scripts in Microsoft Excel sheets



26/31


Pillory (2/5)

Global design

Low-level primitives programmed in

Visual Test

Handles test suites as trees of test

cases



27/31


Pillory (3/5)

Using Rational Visual Test features

Master/Slaves dialog with the Talk

primitives

Access to the Excel sheets with ActiveX

automation and the Ole primitives

Interaction with the Win32 API



28/31


Pillory (4/5)


Cleanroom implementation of Pillory in

Java

Uses Rational Visual Test DLL

Processes asynchronous events



29/31


Pillory (5/5)


Testing of Web pages

Testing of localized software

Public release


30/31

Thank you!


31/31

Any questions?

Documents

Beyond the GUI