© Affiliated Computer Services, Inc. (ACS) 2007, 2008

© Affiliated Computer Services, Inc. (ACS) 2007, 2008

Facilitating the Move from Predominantly Manual Testing to an Automated Test Environment Tim Leach, Systems Consulting Manager

© Affiliated Computer Services, Inc. (ACS) 2007Slide 2

Planning the move to Automation

First Step; Understand Resources: Personnel

1. Skills with coding background2. Availability

The right tools Enough equipment

Next choose the Project: Project that has the best fit Ready to move and step up Project with trained resources

Last Step; Plan and Execute: Align resources Train everyone from management to developers Plan each step Last step execute the plan


Understand the Resources

Personnel Select resources that will make the move successful Make sure they are available, then assign them to the project

Tools Test tools from HP, Segue, IBM or other tool possibilities

http://www.softwareqatest.com/qatweb1.html

Equipment PCs for development and execution Align enough test environments to accomplish tasks Gathering the correct tools and developing the infrastructure


Select Project

Project that has the best fitImportant step to find a project that is willing take on

additional workBe aware that they might be under the microscope Be able to add resources both equipment and personnel

Ready to move and step upCan start early in the develop cycleAdd development resources to develop testsA motivated team

Project with trained resourcesHave resources on the team that understands test methods

an test development


Pull together resources: Align equipment Align personnel Prep Project team

Build Plan: Develop plan step by step Analyze and document current

testing state Review plan and get approval Set realistic expectations

Document ROI and Sell: Gather ROI information and report Sell the Plan

Execute Plan: Be ready to change

ExecutionBusiness

CaseDevelopment

Concept Analysis

ConceptCreation

Plan Development


Execute: Execute plan Stay on plan Resist scope creep

Monitor: Monitor progress of

development and test coverage Status of meeting timeframe

objective

Adjust: If problems occur modify plan

and report change Reset expectations early

Report: Report status using standard

reports This enables you to show

progress

ReportAdjustMonitor Execute

Plan Execution


Some keys to success

Make sure that the correct expectations are set with team and management. Make sure that the plan to automation is not in the critical path of the project. Make sure that the effort to automate does not negatively affect the projects

goals. Make sure everyone is informed on the status of the process at each step. Make sure the plan is not over ambitious. Best to plan several mini goals. Make sure the automation architecture is reusable. This will win over

management and prove a good ROI.


Questions


Test Automation – Definition & Quotes

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process. - Wikipedia

In today’s environment of plummeting cycle times, test automation becomes an increasingly critical and strategic necessity. Assuming the level of testing in the past was sufficient (which is rarely the case), how do we possibly keep up with this new explosive pace of web-enabled deployment while retaining satisfactory test coverage and reducing risk? The answer is either more people for manual testing, or a greater level of test automation. After all, a reduction in project cycle times generally correlates to a reduction of time for test. - Carl J. Nagle


Test Automation – Definition & Quotes Continued

Test automation raises our hopes yet often frustrates and disappoints us. Although automation promises to deliver us from a tough situation, implementing automated tests can create as many problems as it solves. - Bret Pettichord

Over the past several years, tools that help programmers quickly create applications with graphical user interfaces have dramatically improved programmer productivity. This has increased the pressure on testers, who are often perceived as bottlenecks to the delivery of software products. Testers are being asked to test more and more code in less and less time. They need to dramatically improve their own productivity. Test automation is one way to do this. - Bret Pettichord


Automation Myths

1. Test automation is record and playback.2. Test automation does not need specialized skills.3. Tools do all the work.4. Purchasing a tool gains you automation.5. Once you have automation you can remove manual testing.6. Automation will find all new bugs in test cycles.7. Automation will reduce testing resources.8. Once you have automation developed, you’re done!


Automation Problems

Test automation is seen as a spare time project Lack of clear goals and objectives Lack of expertise High resource turnover To much focus on tools and technology Lack of long term planning and goals in automation Lack of understanding Unrealistic expectations Over simplification of development Limited resources


Automation Planning

1. Test Automation development Before you start, take account of your current situation

Develop a complete detailed metric baseline of your current state, great for comparison laterHow long does your testing effort take?How many resources are you using?

Resource planning and utilizationAutomation will increase the need for resources, we need to manage them

closely, schedule usage, assign by specialty and skill levels


Automation Planning

Then develop a strategy and goal (long and short term)Plan should complement the afore mentioned metrics - track

your progress, report statusDefine Requirements for testingProve the automation concept Champion product testability Design for sustainability Plan for deployment and movement through the SDLCFace the Challenges of Success Improve the Testing Process


Automation Planning

Then develop an architecture and design for test developmentPlan for reliable and easy maintenanceDevelop reusable scripts for usage on other projectsPlan validation goals and objectiveDevelop a standard detailed reporting status Metrics will help show success and make it easy to compare

incremental success

Test Automation is a development processNeed to have change control, release management, and

testing


Automation Planning

Test data managementNeed to have plan for collecting, developing and resetting

test data


Report Success

Compare the difference to overall test coverage Report differences to test response time Report consistent and reproducible results

Quicker turnaround time for test cycles Add reports on retesting, regression, integration and bug

verification


Testing Tools

Benefits of a testing development and management tool Test execution management and results reporting

Can run tests and collect results in toolReport of execution status using management reporting

functionalityManage traceability and test coverageManage development of scripts

Single point of testing artifacts Help in developing tests


Tool Planning

What tools are needed? What is the automation framework that best fits the project? What skills do we have? What is the time frame to finish?

What architecture will we use? Keyword scripting, test development scripting, and manual to automation?

What approach will we use: focused requirements base, scenario testing, or regression testing?


Simple Place to Start

Keyword Scripts Gives you both manual and automation scripting Focus resources on areas of expertise Easy maintenance and reuse Works with or without testing tools

Regression testing Start automation from manual tests scripts that are proven

Defect Validation Great use of documented tests and a method of verifying progress


Keyword Scripting

A method that can speed up automation and maintain both manual and automation code in one format.

Use a keyword driver to tie the UI with the actions - this is the one area that actual code is needed and maintained.

Run – Tests 1-3

Run – Tests 1-3 Application

under test

Application under test

Driver Script - QTP

Driver Script - QTP 3 KeyWord

Script - Excel

3 KeyWord Script - Excel






Keyword Script explained

Keyword Script Layout The file layout for this can be done using various formats: Excel files, database files,

and if you’re using Quality Center you can develop to the Test Plan script tables. The approach I have taken is to use Excel - most people have it and are familiar with

using it so training is less (and it can be used outside of any tool) To run the test you’re basically executing the driver which loads the keyword script and

executes the steps. The driver ties the application via the Applications UI to the Keyword actions.


Keyword Script explained

Functional_Area used to group all the actions and Test_Steps to a page or screen

These groups will be selectable from a dropdown list of available actions The dropdown can be controlled to keep only the item in the list to be used or

you can open it up for any keywords Skip_Step allows Pseudo script development. (script development that is

done before the application is working in the application or keyword driver)


Keyword Driver Code

Automation Layout The code snipit below explains how the process works for tying the UI of an application to test

actions in the keyword script. Driver scripting

This is the heart of the keyword functionality. The driver is the tie to the application and the scripts. The driver basically uses a Case statement to handle the action in the script.

Note the first Case below, “Case ‘Enter From Date’“, that ties to the action in the Excel script “Enter From Date”. This ties the UI in the application to the action.


Keyword Driver and Script Compare

ExampleScript.xlsDriver-zip.zip


Questions

Documents

© Affiliated Computer Services, Inc. (ACS) 2007, 2008