Test Automation Approach to Real Life

8/8/2019 Test Automation Approach to Real Life

1/19

Yury Makedonov p. 1 of 19

International Quality Conference, Toronto, Ontario, Canada October 1, 2003

October 01, 2003 Test Automation Presentation for QAI Conference 1

Test Automation Approaches to Real Life

Copyright 2003 CGI Group Inc. October 1, 2003October 1, 2003

The International Quality Conference, Toronto, Ontario, CanadaThe International Quality Conference, Toronto, Ontario, Canada

Yury Makedonov, CGIYury Makedonov, CGI

Golam Mustofa, CGIGolam Mustofa, CGI


Agenda

Why Automate?

Reality of GUI Test Automation today

What are the challenges?

Test Automation Approaches in CGI

Stand Alone Automation Approach

Fully Integrated Automation approach Summary

Q&A


2/19




Pr imary Uses Of GUI Test Aut omat ion

Perform regression tests

Volume tests

Reduce human resource requirements


Why Autom ate?

Save Time

Reduce Total $$ (COQ)

Repeatability/More Consistent Coverage

Reliability

Reusability

Portability


3/19




Real i ty Of GUI Test Aut om at ion Today

By many accounts:

More than 50% of GUI Test Automation

projects fail

Test scripts & tools become shelfware


What Are The Chal lenges?

GUIs are dynamically changing

Application is not stable enough (last minutes changes,

scope creep)

Not enough time to update scripts (poorly designed)

Lack of in-house expertise

Budget constraints

Immature Test Automation methodologies


4/19




Test Aut om at ion Approaches in CGITest Aut om at ion Approaches in CGI

Test Automation Approaches

Two Approaches Used In CGI

Stand Alone

WinRunnerFully Integrated

WinRunner TestDirector


Standalone WinRunner

Yury Makedonov


5/19




Stand alone WinRunner

The fundamental principles will be covered in this section

Test automation from point of view of:

Portability of test scripts

Maintainability of test scripts

Test Automation team supporting:

Multiple versions

Of multiple applications

In different environments

In different locations


Application

GUI Map physical object description:

OK{class: push_button,label: OK,vb_name: cmdOK

}

What is a GUI Map?

WinRunner Script logical object name:

button_press ("OK");

OK


6/19




St ep One Tools eva luat ion

Can a GUI testing tool handle our application?

How time consuming is test automation (Does

it make sense to automate)?

Do we care about a proper organization of test

scripts?


Step Tw o Fi rs t vers ion o f the app l ica t ion :

load("C:\\WR_Scripts\\Lib_UT_Functions");

GUI_load("C:\\WR_Scripts\\ UTGuiMap.gui");

web_browser_invoke(IE,"http://confut.xyz.ont.ca");

UT_Select_Language("English");

UT_Login("a327012", "12345");


7/19




St ep Three Second vers ion:

New version of theapplication delivered

Cloning of scripts

load("Lib_UT_Functions);

GUI_load(UTGuiMap.gui);

No full path for a GUI map or ascript


St ep Four Several Test ers :

Several testers working with WinRunner Scripts moved onto shared Z: drive

load("Lib_UT_Functions");

GUI_load("CoolGuiMap.gui");

UT_Open_Site( );

UT_Select_Language("English");

UT_Login("a327012", "12345");UT_Select_Portfolio("KEY");

call "Invoice_Create"( );


8/19




St ep Four Several Test ers :

All modifications are performed on a local copy of ascript.

After debugging script will be copied onto a shareddrive.

Each script has an owner. Owner stores mastercopy of the script on his local workstation.

Simultaneous execution/modification of the same

script/library on different workstations.


St ep Five Several App l ica t ions:

Test automation forseveral applications

Application (version)specific libraries

General libraries (usedby all projects)

Major rework of librariesof functions


9/19




St ep Six Severa l Env i ronment s,Brow sers, User Roles:

C:\TestData\UTEnvironment.xls file:

URL B rows er U se rID Pa ss word

http://prodaceman1.configurator.com/ut IE a327842 123

We need to execute the same script concurrently:

against different environments, using different browsers,

using different user roles/login IDs.

Store setup parameters in a local file.

Site_Open(flURL, flBrowser, fluserID, flPassword);


Rec omm endat ions for WinRunner pro jec t so f any complex i t y :

Implement the complete hierarchy of folders for WinRunner scripts fromday one: Folder for general libraries

Folders for different Application

Subfolders for different Versions of the application

All scripts for a specific version of an application arestored on the same level of hierarchy

Use path relative to a current test to load an object (Nospecification of a full path, starting with a drive letter, allowed)

Implement a standard script header containing description of ascript + all initialization steps: GUI Load,

Libraries load,

Reading data file, etc.

Implement Script Ownership concept: master copy of anyscript belongs to a single person

MAKE IT A STANDARD!!!


10/19




Fully Integrated Test Automation - WinRunner &TestDirector

Golam Mustofa


St ep One Know The Tools

Functional Testing Tool

Develops the scripts based on detail Test Cases

Needs to create GUI file and Data Spreadsheet

Needs to connect to the TestDirector

Test Management Tool

WinRunner scripts have to be integrated withTestDirector

TD schedules execution of WinRunner scripts

Manages Defects


11/19




St ep Tw o Underst and The Overa l lProcess

WinRunner activities

Learn GUIs of the application under testing

Develop / Record scripts

Debug scripts

Import or Save scripts into the TestDirector

TestDirector activities

Locate the Test Data Spreadsheet into a shared drive or attachto the TestDirector

Build the Test Sets

Schedule execution of the Test Sets

Analyse Test Result and create Test Report

Report or Manage defects


St ep Three Create An Ef fec t iveFramework

Create a root folder into the Test Plan Tab of

TD

Design an Initial / Startup scripts

Test Environment setup

Load GUI and function library

Declare variables, data spreadsheet location, etc

It itself can have a data spreadsheet as well

Example

Create sub folders based on components ofthe AUT

Make a folder for compiled module or

functions library Function library for generic functions

Function library for application specificfunctions


12/19




St ep Three Create An Ef fect iveFramework

Build a td_files folder into the

shared drive Make sub folders named

datatables and gui_filesunder the td_files folder

Have sub folders based oncomponents of the AUT underthe datatables folder

Keep all the GUI files into thegui_files folder orTestDirectors project


Step Four Ut i l ize Data-Dr iven Test Techniques

Single scripts will handle negative and positive boundarytest reading data from the data spreadsheet

Bel l CanadaBel l Canada

Bel l CanadaBel l Canada

Blank dataBlank data

Erro r MessageEr ro r Message

Special charactersSpecial characters

Valid dataValid data

AcceptedAccepted

$!% #$!% #

$!% #$!% #

Erro r MessageEr ro r Message

Have control on the spreadsheet

Skip some rows

Stop in row

Bel l Mobi l i tyBel l Mobi l i t y

Bel l Bel l ExpressVuExpressVu

BCEBCE Nexx ia Nexx ia

BCEBCEAct iMediaAc t iMedia

BCE Medi aBCE Media

Bel l Bel l Emerg is Emerg is

Sympat i coSympat i co

Bel lZ inc.caBel lZ inc.ca

XX

XX

XX

StopStop

Write important data or results into the spreadsheetafter the transactions for future analysis

PassedPassed

PassedPassed

PassedPassed

PassedPassed

PassedPassed

Fai ledFai led

PassedPassed

PassedPassed

PassedPassed

$!% #$!% #

PassedPassed


13/19




Step Five Locat e Data Table (Spreadsheet )

Attach the data spreadsheet withthe Test Case

Download the spreadsheet on thefly during scripts execution

Read the spreadsheet as input file

Or keep the spreadsheet into theshared drive

TestDirector doesnt allow to save the Data spreadsheet into theProjects database


Step S ix Bui ld Test Sets

Identify all Testing scripts which needs to be into a bundle basedon applications functionality

Create a Test Set

Attach all identified Testing scripts into the Test set


14/19




Step Seven Sc hedule And Exe cut e Test Sets

From Execution Flow tab click on clock icon

Set the time and date youd like to the Test Set or Individual Test

Press on Run Test Set or Run button

TestDirector will invoke WinRunner and perform all the tests


Step E ight Manage Your Defect s

Analyse the Test Result

Manage any defect through the Defect Tab of TestDirector


15/19




Summary

Test Automation, when used to perform Regression &Volume testing can reduce cost, human resources andtime

Implement Stand Alone or Fully Integrated methodbased on your environment, goals/priorities, budgetand expertise of resources

Test Automation is worth the investment and effort, ifits implemented properly!!!!


Questions??????


16/19


17/19


18/19




Appendix B - Design ing In i t ia l / St ar tupsc r ip ts


Appendix B - Sk ip some row s o r in a rowon the dat a spreadsheet


19/19


I t ti l Q lit C f T t O t i C d O


Appendix B - Loca te Dat a Tab le(Spreadsheet)


Contac t In format ion

Senior Consultant,

Centre for Testing & Quality

CGI Services to BCE

595 Bay Street, Floor 10-B

Toronto, Ontario,

Canada M5G 2C2

(416) 215-3593

[email protected]

Senior Consultant,

Centre for Testing & Quality

CGI Services to BCE

595 Bay Street, Floor 10-B

Toronto, Ontario,

Canada M5G 2C2

(416) 215-7534

[email protected]

Golam MustofaYury Makedonov

Documents

Test Automation Approach to Real Life