Upload
softwarecentral
View
1.527
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
1
Software testing toolsand the selection process
Geoff Quentin
QBIT
TestExpo 17th March 2005
2
Contents
• Introduction
• Specific points about processes
• Specific types of tools
• Implementation
• The way forward
3
Testing - why do it?
• Testing is to add confidence
• Testing is to show compliance
• Testing is to identify where to reduce risk
• Identify defects and then have them fixed
• Face problems if not done
4
FUNCTIONALSPECIFICATION
The simple V Model
CODE
UNITTEST
SYSTEMTEST
ACCEPTANCETEST
TECHNICALSPECIFICATION
USERREQUIREMENTS
UNITTEST
MATERIAL
SYS TESTMATERIAL
UAT TESTMATERIAL
PROGRAMSPECIFICATION
INT TESTMATERIAL
INTEGRATIONTEST
TestExpo © QBIT 2002
5
The project plan showing earlytest planning and creation
Specification ____
Design (user) ____
Design (tech) ______
Design (prog) _____
Code ____
Plan unit test _____QC
Unit test ____QC
Plan integration test _______QC
Integration test _____QC
Plan acceptance _________QC
Acceptance test ____QC
Installation _QC
6
Or for more detail use ISO 12207
• Lots more activities
• Specification paired with design
• System treated separately from software
7
The fit with 12207
Softwaredetaileddesign
Softwarecoding
SoftwareComponent
testing
Softwareintegration
Softwarequalification
testing
Systemintegration
Systemqualification
testing
Systemrequirements
analysis
Systemarchitectural
design
Softwarerequirements
analysis
Softwarearchitectural
design
Acceptance& completion
Specifybusiness
requirements
Software developmentteam (includes software
test team)
Softwarequalification
test team
Systemqualification
test team
The users (includes useracceptance test team)
The acquirerThe supplier
Softwareconfiguration
team
Systemdesignteam
Softwaredesignteam
TestExpo © QBIT 2002
8
Systemrequirements
analysis
Systemarchitectural
design
Acceptance& completion
Specifybusiness
requirements
Useracceptance test
team
Test item transmittalreport
The acceptance tester’s view and ISO 12207
9
Systemqualification
testing
Systemrequirements
analysis
Systemarchitectural
design
Specifybusiness
requirements
Test item transmittalreport
Systemqualification
test team
The system tester’s view and ISO 12207
10
Three documents for you
• This presentationwww.qbit.co.uk/testexpo
• The Generic Test Processwww.qbit.co.uk/gtp
• The Consolidated Test Processwww.qbit.co.uk/ctp
11
Why automate?
• So what is the objective?
• Return on investment
• Achieve a difficult goal
• Improve an existing process
• Implement a new process
• Scale up the process for bigger projects
• Ensure compliance
12
Process focus
• The basic process should be good
• The process upon which the tool is basedshould be good
• Conformity
• Repeatability
• Now seeing tools based on good processes
13
The test process
• Record (set up reporting)
• Create strategy
• Plan tests
• Analyse
• Design
• Schedule
• Execute
14
The testprocessmodel
LOW LEVELTEST PLAN
TESTOBJECTIVES
LIST
TEST EXECUTION
TESTITEM
TESTSUMMARY
REPORT
HIGH LEVELTEST PLAN
LOW LEVELTEST PLAN
LOW LEVELTEST PLAN
LOW LEVELTEST PLAN
TESTSCRIPT
TESTINCIDENTREPORT
TESTSTRATEGY
PROJECTDOC
SPEC
TEST ITEMTRANSMITTAL
REPORT
TESTLOG
TEST ITEMTRANSMITTAL
REPORT
TESTSCHEDULE
4. TEST ANALYSIS
5. TEST DESIGN
6. TESTSCHEDULING
7. TEST EXECUTION
2. TEST STRATEGY
3. TEST PLAN(various)
1. TEST STATUS REPORTING
Copyright © QBIT 2002
15
So which level to apply theprocess?
• Acceptance tests
• System tests
• Integration tests
• Unit tests
So who will use the chosen tool(s)?
16
Roles and responsibilities
• Manager
• Test team leader
• Tool user (the test analyst)
• Tools specialist (to implement)
• Process specialist (to improve the process)
17
Resources required
• Tools specialist
• Trainer
• Sponsor
18
Time table
• Critical to introduce tools early
• Time to familiarise
• Time to tune
• Time to achieve proficiency
• Now to use in earnest
19
So what might be bought?
• Tools to support existing processes
• Tools to implement a new process
• Tools to speed up existing processes
• Tools to enhance, make possible, process
• Tools to make economical a process
20
The tools on offer
• Management of testing
• Capture of tests
• Playback of tests
• Simulation, modelling and prototyping
• Static analysis
• Dynamic analysis
• General purpose tools
21
Tools to support thereview process
• Verification– checking the test object
• Validation– checking context
• Certification
• Defect recording and tracking
• Metrics
22
Management of testing
• Traceability
• Identification of tests
• Storage and retrieval of tests
• Reporting progress of tests
• Repeatability of tests
• Reusability of tests
• Long term support for the testing effort
For the test manager
and team leader
23
System design phase testing
• Capture of tests
• Selection and replay of tests
• Prototyping
• Simulation
• Mathematical modelling
• Change and configuration management
For the team leader
and test analystsystem test and
UAT
24
Software design phase
• Performance prediction
• Performance measurement
• Change and configuration management
For thesystem tester
and the integration tester
25
Programming
• Static analysis
• Dynamic analysis
• Unit test execution
• Change and configuration management
For the programmer
26
Research for tools
• Need to identify where processes need tools
• Need to gather metrics
• Evaluate the return on investment
• Tools to fit the processes
• Process based tools to fit the problem
27
Deployment of the chosen tool(s)
• Small project
• Limit the scope for failure
• Representative
• Support
• Training
28
Support once in use
• Supplier
• Internal
• User groups
• Independent consultants
• Independent trainers
29
Process improvement
• The tool must add value– improve efficiency
– improve effectiveness
– improve coverage
– reduce costs
– reduce duration
– simplify the work
– remove/reduce the chance of errors
30
So what is on offer at thisQBIT tools fair?
• Tools and consultancy
31
Visit• Cresta• Empirix• Vizuri• Compuware• Embarcadero• Princeton Softech• Facilita• Testhouse• Transition• Focus Frame• Propel Systems
• T-Plan
• BCS/iseb
• Sim/SQS
• Contemporary
• QBIT
• Serpico
• ITR
• Original Software
• Books and journals– Artech House
– Better Software
– Professional Tester
32
Summary
• Identify the objective(s)
• Check the processes
• Check the functionality
• Check the overhead of deployment
33
This presentation www.qbit.co.uk/testexpoThe Generic Test Process www.qbit.co.uk/gtpThe Consolidated Test Process www.qbit.co.uk/ctp
Thank you
34