Upload
quontrasolutionsppts
View
35
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Learn through Experience -- We differentiate our training and development program by delivering Role-Based training instead of Product-based training. Ultimately, our goal is to deliver the best IT Training to our clients. Attendees also learn: 1. Resume Preparation Guidelines and Tips 2. Mock Interviews and Interview Preparation Tips For more visit http://www.quontrasolutions.co.uk/selenium-training-classes/ - PowerPoint PPT Presentation
Citation preview
Automated Software Testing Framework
Presented By www.QuontraSolutions.co.uk
IT Courses Online Training Call Us: 20-3734-1498 Email: [email protected]
1
TopicsAutomated software testing.
What? Why?Automated vs. ManualCommon types of automated
testing methodsAutomated software testing
frameworkSummary
2
What is automated software testing?
A process which covers:the use of software to control the execution of teststhe comparison of actual outcomes to predicted outcomesthe setting up of test preconditionsother test control and test reporting functions.
3
Why automated testing?A time savings translates directly into
cost savings Improves testing productivity Improves accuracy Increases test coverageDoes what manual testing cannot do
5
Effort and Cost Let’s assume 6 test cases Effort required to run all 6 manually => 10 min Effort required to write unit tests for all 6 cases => 10 min Effort required to run unit tests for all 6 cases => < 1 min Number of testing iterations => 5 Total manual testing time => 50 min Total unit testing time => 10 min
6
Release Manual Test Auto TestManual TestCumulative
1 10 10 10
2 10 0 20
3 10 0 30
4 10 0 40
5 10 0 50
Automated vs. Manual testing
Not all tests can be automated and most times is difficult to decide what to automate and what to manually test.
7
Advantages
8
Automated Testing Manual Testing• Need to run a set of tests repeatedly
• Test cases have to be run a small number of times
• Helps performing "compatibility testing" (on different configurations and platforms)
• Allows the tester to perform more specific tests
•Long term costs are reduced • Short term testing costs are reduced
• Possible to run regressions on a code that is continuously changing and in shorter time
• The more time tester spends testing a module the grater chance to find real bugs
Disadvantages & other factors
9
Automated Testing Manual Testing• It's more expensive to automate (bigger initial investments)
• Manual tests can be very time consuming
• You cannot automate everything, some tests still have to be done manually
• For every release you must rerun the same set of tests which can be tiresome
Other Factors• The performance of your test tools• The knowledge level of your testing team• The continuous growth of software to be tested
Common types of automated testing methods
Monkey testing
Capture / playback
Code-based (Unit) testing
Intelligent test automation
10
Monkey testing Randomly selecting inputs from a large
range of values and monitoring if exceptions are thrown.
For example, a monkey test can enter random strings into text boxes to ensure handling of all possible user input.
It applies not only for GUI or WEB testing, but also for Unit testing.
11
Tools example
Monkey – Android testing tools. The simplest way to use the
monkey is with the following command, which will launch the application and send 500 pseudo-random events to it.
$ adb shell monkey -v -p your.package.name 500
12
Capture / playback It’s a set of software programs that capture
user inputs and stores it into a script for later replay.
+ Repeated testing can be performed quickly.+ Does not require programming skills.- When the GUI changes, input sequences
previously recorded may no longer be valid.- Difficult to determine location of bugs.
13
Code-based (Unit) testing Individual units of source code are tested
to determine if they fit for use. Ideally, each test case is independent from
the others+ Bugs can be found in early development
stage+ Easy to test boundary cases- Not effective for the integrated system
testing
15
Random Input values generated randomlyRun with generated inputsUser checks output values
(pass/fail) and saves them for later automatic checking (Oracle).
17
GeneticThis methods approach is to combine JML (JAVA model language) and genetic algorithms. JUnit is used as a test execution platform.
18
Tools examplesParasoft Jtest is an automated Java testing and static code analysis product
19
TestGen is a collection of open-source tools that automatically generates unit test cases.
Intelligent test automation A model-based method for generating tests from
a description of an application’s behavior (UML)+ Models are simple to modify, so model-based
tests are economical to maintain over the life of a project.
- Most of these methods are researched only theoretically.
20
OCL + UML 21
Inputs generator
UML + OCL
Tested software
Oracle
Report
Inputs Constraints
ConstraintsOutputs
pass / fail
Automated testing frameworkA framework is an integrated system that sets the rules of automation of a specific product.
This system integrates the function libraries, test data sources, object details and various reusable modules.
The framework provides the basis of test automation and simplifies the automation effort.
22
Data-driven testingData-focused automationUser defines just data sets to run tests withData is defined in external data source and
de-coupled from scriptGood fit for a single application testing,
featuring multi-environment, big datasets, and rarely changing Test Cases
24
+ -
Good usability and re-usability
Quality and coverage strongly depends on the automation skills of implementing person
Good test flow coverage Continuous maintenance issues due to a big amount of the code
Multi-environment support
Manual validation is required
Keyword-driven testing Good fit for a multi-application testing featuring
multi-environment, big datasets, and a big number of short and straight Test Cases
Users de-compose test cases into reusable action keywords
Non-coders are able to author automated test cases with action keywords
26
+ -
Test development doesn’t require programming skills
Due to keywords limit doesn’t allow covering very complicated Test Cases
Good test flow coverage Meta-language specific training is required for staff
Compact code Manual validation is required
Hybrid testingCombines the best of data-driven and
keyword-driven testing: User defines data sets to run tests with User also defines flow control through action
keywords Data is separated from script and stored in external
data source with action keywords
28
+ -
Test Plan / Test Scenario creation doesn’t require programming or training on the Testing Tool
Requires initial investment for design and implementation of the framework
Compact code and scalable architecture
Built-in strict validation
Framework components Test generation (data and script generators).Tools
might create specialized data such as randomized email messages, or populate databases etc.
System configuration. Tools might preserve or reproduce system parameters.
Test execution. Tools might operate the software itself, either simulating a user working through the GUI and using an alternative testable interface.
Oracles. An oracle is any mechanism by which we detect failure or success. Tools might automatically detect certain kinds of error conditions in a product.
Activity recording & coverage analysis. Tools might watch testing as it happens and retrospectively report what was and was not tested.
Test management. Tools might record test results, organize test ideas or metrics.
29
Common functional automated test toolsTool + -
IBM / Rational Functional Tester (RFT)
•Built as Eclipse plug-in with full IDE, Java support and source management
•Supports Web-based applications, .Net, Java, Siebel, SAP®, PowerBuilder, AJAX, Adobe Flex, Adobe PDF documents
•Full GUI Object Map repository
•License cost
HP / Mercury Quick Test Pro (QTP)
•Supports Web 2.0, GWT, Dojo, YUI, Silverlight, Flex, Ajax, JAVA, .NET applications
•Full GUI Object Map repository•Seamless integration with
QualityCenter
•No IDE (changing in release 10)
•License cost•VisualBasic
scripting is limited
Selenium RC & IDE
•Good browser support•Good language support (JAVA,
Ruby, C#)•Can be easily extended as JUnit
suite•Open-source (no license cost)
•No GUI Object repository
•Only web-based application support
•Tool support is limited
31
SummaryAutomated testing improves productivity,
accuracy, increases test coverage, saves time and cost
Need to decide what to automate and what to manually test
The framework provides the basis of test automation and simplifies the automation effort.
33
More information http://opuntia.cs.utep.edu/utjml/utest.html http://www.ibm.com/developerworks/rational/library/
591.html http://www.satisfice.com/presentations/agileauto.pdf http://automation-beyond.com/2009/05/30/test-automation-
practices-recordplayback/ http://en.wikipedia.org/wiki/Test_automation
34