AutomatedQA Keyword Driven Testing Documentation This document explains how to use the STAR Keyword Driven Testing framework for automated test case management. Version 1.2

AutomatedQA

2

Table of Contents Narrative ....................................................................................................................................................... 3

Keyword Driven Testing Architecture ........................................................................................................... 4

Keyword Driven Testing Setup ...................................................................................................................... 5

Using the Keyword Driven Testing Framework ............................................................................................ 6

Software Planner Automation Folders ..................................................................................................... 6

Application Areas ...................................................................................................................................... 7

Application Objects ................................................................................................................................... 8

Automated Test Cases ............................................................................................................................ 10

Test Sets .................................................................................................................................................. 11

Analyzing your Automated Test Runs ..................................................................................................... 13

Setting up Multiple Hosts ........................................................................................................................... 14

Scheduling Automated Test Runs ............................................................................................................... 15

Launching TestComplete Test Cases ........................................................................................................... 16

TestComplete Test Cases Organized by Unit .......................................................................................... 16

TestComplete Test Cases Organized by Subroutine ............................................................................... 17

TestComplete Test Cases Organized by Keyword ................................................................................... 17

Training ....................................................................................................................................................... 18

Learn More .................................................................................................................................................. 19

AutomatedQA

3

Narrative Automated Testing has many benefits including:

Quicker Releases – By having your regression test cases run automatically, your software quality

team can concentrate on testing new features of your software and less time regressing existing

features.

Higher quality releases – Your software releases will have fewer bugs and require less customer

support because they will be of higher quality.

Happier Customers – Your customers will be happier and more willing to serve as testimonials

for future prospects.

For companies that have experimented with automated testing in the past, they may have encountered

these issues:

Automated Tool Skills – Testers must learn how to use an automated tool.

Scripting Skills – Record and playback does not normally suffice for sophisticated automated

test cases, so testers must learn scripting languages (like Visual Basic, JavaScript, etc) to create

automated test cases.

Reusability – When using record and playback and scripting, test cases are not as reusable as

they could be, so many times tester will waste valuable time recreating the same tests again and

again. Also, as features in your software change, each affected test case must be changed to

accommodate new screen controls/fields/etc.

Reporting – Most automated testing tools do not allow you to see how your automated testing

effort is trending and do not provide the reporting needed to evaluate its effectiveness.

Multi-User Issues – Most automated testing tools do not allow testing groups to share in the

automated testing process because of sharing limitations of the tool and scripts.

Understanding the huge benefits of automated testing and the issues that companies often face,

AutomatedQA and STAR (http://www.star-qa.com) teamed to create a keyword driven testing

framework that removes the issues surrounding automated testing. This document describes the

keyword driven testing framework and how to use it.

AutomatedQA

4

Keyword Driven Testing Architecture If you have a software application you wish to setup automated testing for and would like to use the

keyword driven testing framework, you will need the following components:

Automated Testing Tool – If you do not have an automated testing tool, we highly recommend

AutomatedQA’s TestComplete™ (http://www.testcomplete.com). Simply put, it is the most full-

featured yet economically priced tool on the market. If you already have an existing tool,

Software Planner also supports TestComplete™, HP Quick Test Pro™, HP WinRunner™, Rational

Robot™, Rational Functional Tester™, Borland Silk™, and Compuware Test Partner™.

Automated Testing Bridge – Software Planner offers an automated testing bridge for your

automated testing tool (from the list mentioned above). The cost of the bridge is a one-time

cost of $400.

Keyword Driven Testing (KDT) Framework (optional) – This allows you to create automated test

cases without any scripting and allows you to reuse those test cases and schedule to run test

sets at specific intervals. The cost of the KDT framework is a one-time cost of $400.

Software Planner – Software Planner is used to track your requirements, test cases (both

manual and automated) and defects. Its rich dashboards provide metrics and run history.

Your Software

Application (Windows/Web/etc)

Results of each automated test is

recorded into Software Planner

where you can inspect the run results

on dashboards and reports.

Your tester (s)

Your testers use Software Planner to

define automated test cases

Software Planner

Your testers execute automated test

cases from Software Planner. It calls the

automated testing tool to drive your

software application testing.

Automated Testing Tool

The automated tool drives your

software application.

AutomatedQA

5

Keyword Driven Testing Setup When you purchase the Keyword Driven Testing, AutomatedQA will setup your Software Planner

account to use the Software Planner web service and the Automated Testing Bridge. You will also

receive an installation routine that can be installed on one or more PCs or quality assurance servers.

You can run your automated test cases directly from Software Planner or you can schedule your

automated test cases to run at specific intervals. The integration also supports multiple hosts, so if you

wish to run automated test cases on different machines (hosts), you are free to schedule them to run on

different hosts.

AutomatedQA

6

Using the Keyword Driven Testing Framework Once your keyword driven testing framework is setup, you can then begin to create your automated test

cases without any knowledge of programming languages or scripting.

Software Planner Automation Folders The first step in automating your test cases is to setup folder in the Test Cases area that will house your

automation. To do this, log into Software Planner and click Test Cases / Automation Lab / Setup

Automation Folders:

To add a new automation folder, click Add New. It will prompt you for:

Automation Type - This is the automated tool you are using.

Short Description - Can be anything you like.

Script UNC – This is the location and script name that runs the automation (Automated QA will

provide you the value for this upon installation of the KDT).

Default Test Case Status – This is the status you wish to set when creating new test cases that

flow into Software Planner (this is the test case status inside Software Planner).

Once an automation folder is created, you can navigate to the Test Cases area of Software Planner;

notice it automatically creates the folder on your behalf:

AutomatedQA

7

Application Areas Next, you will define your Application Areas. Application Areas are simply the areas of your software.

For example, you may have an application area for your login process and other functional areas of your

software. To define application areas, log into Software Planner and click Test Cases / Automation Lab

/ Keyword Driven Testing / Application Objects / Application Areas. Below is an example:

To add a new application area, click Add New. It will prompt you for:

AutomatedQA

8

Application Area - This is the name of the application area (Login, etc).

Parent Window – If your application is a web application, this will be the URL of the application.

You can use an asterisk (or any regular expression) as wildcards. If your application is a

windows application, it will be the parent window for the windows application.

Child Window – If your application is a web application, this will be the URL of the application.

You can use an asterisk (or any regular expression) as wildcards. If your application is a

windows application, it will be the child window for the windows application.

Application Objects Next, you will define your Application Objects. Application Objects are controls that are on the screen.

For example, you may have may a login screen similar to this:

It has 3 controls on the screen (User Name, Password and the Login button). You will define an

application object for each of these).

AutomatedQA

9

To create an application object, log into Software Planner and click Test Cases / Automation Lab /

Keyword Driven Testing / Application Objects. Below is an example:

To add a new application area, click Add New. It will prompt you for:

Application Area – Choose this from an existing application area (Stock Trader Login, etc).

Object Name – Describe the object on the screen. In the example above, we are describing the

UserName edit box. This can be any name you wish, just ensure it describes what the object is.

Object Type – This is the type of object, it will be one of these values:

Object Type Description

CheckBox If the control is a checkbox, use this type. TextArea If the control is a text area that contains different choices. DataGrid If the control is a data grid (or HTML table), use this type. Dropdown If the control is a combo box, use this type. DropdownGroup If the control is a combo box that is using a grouping, use this type. EditBox If the control is a data entry text box, use this type. Environment If the control is a URL or another common line option to be launched. ErrorMsg Used to verify that a specific error message is generated. Link If the control is a hyperlink, use this type. Menu If the control is a menu item or button, use this type. PushButton If the control is a push button (like a submit button), use this type. Tab If the control is a tab (like tabs on forms separate data), use this type. TreeView If the control is a explorer-style tree view, use this type.

Object Definition – This is the object name. It must begin with Set sObject =

sWindow.document… Automated Tools (like TestComplete) have an object browser that allows

AutomatedQA

10

you to navigate to the form, open the object browser and drag the object finder onto the form

element. This will then tell you the object definition (you can copy/paste it from your

automated tool).

Automated Test Cases Next, you will define your Automated Test Cases. For example, lets’ assume you wish to create an

automated test case that logs into the Stock Trader application (see screen shot in Application Objects

section).

To create a test case, log into Software Planner and click Test Cases / Automation Lab / Keyword Driven

Testing. Below is an example:

To add a new test case, click Add New. It will prompt you to clone from an existing test case or create

one from scratch. Once you enter a test case name, you can then set the steps to execute:

Seq – This is the order in which the step will be run. If you wish to reorganize test steps, you

can overtype the sequence number and press Save to re-sequence them. Test Steps are

automatically renumbered in increments of 100 to allow you to easily insert steps in between

others. To add a new step, click the green plus sign. To delete a step, click the delete checkbox.

Application Object – Choose the application object (defined in Application Objects section).

Action – This is the action to take on the object. Below are the valid values:

Action Description

Click This will click the object. Used for buttons/etc. DblClick This will double click the object. Used for buttons/etc. Enter This will enter data. Used for text fields/etc. Launch This will launch a website or program. Used for URLs/etc. Select This will select an item from a combo box. SelectReplace This will select an item from a combo box and replacing it with a

AutomatedQA

11

another value. Verify This will Verify that the expected value appears in a text box.

Value – This is the data to enter into the control. If the action is Click, the value should be set to

Click.

Expected Status – This is either PASS or FAIL. In most cases, set this to PASS. This means that

you would expect the step to work correctly. If you set it to FAIL, you would expect the step to

fail.

Test Sets Once your Automated Test Cases are defined, you can then create a Test Set that contains one or

automated more test cases. For example, you may create a test set of all your regression test cases and

schedule it to run nightly.

To define a Test Set, log into Software Planner and click Test Cases / Automation Lab, then click Add

New. Below is an example of some existing test sets (regression and smoke tests):

In the example above, notice we have 44 test cases in the Full Regression Set:

AutomatedQA

12

Notice that it shows the linked test cases at the top (and the sequence in which they will be run). If you

wish to add more test cases, click the INCLUDE checkbox and set the sequence. Once the test set is

saved, it will automatically renumber the sequences in increments of 100 for easy insertion. If you wish

to re-arrange a sequence, overtype the sequence and save the record.

If you wish to run the Test now, return to the Automation Lab and click Execute Now. Once you launch

the automation, it allows you to begin executing the tests (or specific test cases within the test set):

AutomatedQA

13

Analyzing your Automated Test Runs Software Planner provides reports and dashboards that show the results of each automation run. Below

is a trending report that shows the daily runs:

AutomatedQA

14

Setting up Multiple Hosts If your team has multiple PCs or Servers that you wish to launch automated test runs on, you can run

them on different hosts. The host must have TestComplete or TestExecute, but this approach does

allow a company to test on different hardware and spread the test load to different machines. To

create a new host, click Test Cases / Automation Lab / Manage Hosts, and then click Add New. Below

is an example of the Manage Hosts screen:

Note: If you wish to setup a host to launch TestExecute instead of TestComplete, simply append [TE] to

the test host name ( e.g. in example above, the Test Host name would be GP-XPS [TE] ).

AutomatedQA

15

Scheduling Automated Test Runs Once a test host has been setup, you can then set schedules to allow unattended scheduling of test runs.

To schedule a test run, click Test Cases / Automation Lab, and then choose the Test Host from the

combo box at the top of the page, then find the test set to schedule and click Set Schedule:

From here you can set the date to begin running the automation. If you wish to stop running the

automation on a specific date, enter the end date (this is optional). Next, choose the time of day to run

the automation and the days of the week for it to run.

AutomatedQA

16

Launching TestComplete Test Cases If your team already has TestComplete test cases defined in TestComplete units, subroutines and/or

TestComplete’s keyword engine, you can also automate those test cases without using our Keyword

Driven Testing feature while still enjoying the test set, multiple host and scheduling features.

To manage these test cases, log into Software Planner and click Test Cases / Show Test Cases. Then add

a folder to contain those test cases (you can use one or more folders if desired). Once done, add a

Software Planner test case for each test case you created in TestComplete, set the Software Planner

status to Not Automated (this will trigger Software Planner to prompt you for the TestComplete project

information that links the Software Planner test case to the TestComplete test case).

TestComplete Test Cases Organized by Unit If you organize your TestComplete test cases by Unit, you will add those test cases to Software Planner

(described above) then create a Test Set in Software Planner by clicking Test Cases / Automation Lab /

Add New. Once a test set is created that contains your test cases, you will then link the Software

Planner Test Cases to the Test Complete Units. To do this, click Test Cases / Automation Lab / Execute

for the test set you created. It will launch the Automation Bridge:

Notice that each test case has a Software Planner icon next to it (see red arrow above). If you click that

link, it will then prompt you for where your TestComplete unit resides:

AutomatedQA

17

From here, choose your TestComplete project, folder and unit.

TestComplete Test Cases Organized by Subroutine If you organize your TestComplete test cases by subroutines, you will follow the same instructions as in

the prior section, but when linking your test cases, you will also enter the Routine name on the above

screen.

TestComplete Test Cases Organized by Keyword TestComplete 7.0 has its own keyword engine that allows test engineers to create automated test cases

using keywords instead of scripting. If you organize your TestComplete test cases by keywords, you will

follow the same instructions as in the prior section, but when linking your test cases, you will not enter a

unit or routine, instead, you will enter the project, project folder and keyword test item name. However,

to have it report the results to Software Planner, you will need to tell TestComplete’s keyword tool to

provide logging (this is automatic if organized by Unit or Subroutine but not with the keyword tool). To

create logging, open up the keyword item in TestComplete and click the Logging area. Then drag and

drop the Software Planner icon to the bottom of your keyword test then save your keyword test.

AutomatedQA

18

Training If your team plans to implement keyword driven testing, we highly recommend that you purchase our

training classes. The training classes are specifically designed to help you get up and running quickly to

maximize your investment. Below are the suggested training classes:

STAR 101 – STAR Add-ins Overview (http://www.softwareplanner.com/ServicesTraining_STAR_101.asp)

STAR 102 – STAR Add-ins Workshop (http://www.softwareplanner.com/ServicesTraining_STAR_102.asp)

STAR 103 – Keyword Driven Framework (http://www.softwareplanner.com/ServicesTraining_STAR_103.asp)

Other training classes can be found at http://www.SoftwarePlanner.com/training.

AutomatedQA

19

Learn More If you wish to learn more, here are some helpful links:

Software Planner

If you wish to learn more about Software Planner, request a free a trial, or receive a personalized demo

of the product, contact AutomatedQA at +1 303-768-7480. You can also learn more at

http://www.SoftwarePlanner.com.

TestComplete

If you are interested in an automated testing tool, we highly recommend AutomatedQA’s TestComplete.

If you wish to learn more about TestComplete, request a free a trial, or receive a personalized demo of

the product, contact AutomatedQA at +1 978-236-7900. You can also learn more at

http://www.TestComplete.com.

STAR (Automation Experts)

If you are interested in help with your automation, we recommend STAR. They have many years of

automation experience and can help you get up and running quickly with automation. You can learn

more about the STAR at http://www.Star-QA.com or at +1 303-997-1555.