Upload
ramp
View
219
Download
0
Embed Size (px)
Citation preview
8/14/2019 Practitioner Flashcards Fronts
1/15
A Technical Review (also known asa peer review), is considered to be aformal review type, even though noManagers are expected to attend. It
involves a structured encounter, inwhich a peer/s analyse the workwith a view to improve the quality ofthe original work.
Ideally led by the Moderator
Attended by peers /technical experts
Documentation is required
No Management presence
Decision making
Solving technical problems
A walkthrough is a set of proceduresand techniques designed for a peergroup, lead by the author to reviewsoftware code. It is considered to be
a fairly informal type of review. Thewalkthrough takes the form ameeting, normally between one andtwo hours in length.
Led by the Author
Attended by a peer group
Varying level of formality
Knowledge gathering
Defect finding
An inspection is a formal type ofreview. It requires preparation onthe part the review team membersbefore the inspection meeting takes
place. A follow-up stage is also arequirement of the inspection. Thisensures that any re-working iscarried out correctly.
Led by a Moderator
Attended by specified roles
Metrics are included
Formal process
Entry and Exit Criteria
Defect finding
An informal review is an extremelypopular choice early on in thedevelopment lifecycle of bothsoftware and documentation. The
review is commonly performed bypeer or someone with relevantexperience, and should be informaland brief.
Low cost
No formal process
No documentation required
Widely used review
Software Validation and Verification
can involve analysis, reviewing,
demonstrating or testing of all
software developments. This will
include the development process
and the development product itself.
Verification and Validation is
normally carried out at the end of
the development lifecycle (after all
software developing is complete).
But it can also be performed much
earlier on in the development
lifecycle by simply using reviews.
Validation involves the actual
testing. This should take place after
verificat ion phase has been
completed.
Validation: confirmation by
examination and provision of
objective evidence that the
particular requirements for a specific
intended use have been fulfilled.
Validation: Are we building the right
product?
Verification would normally involve
meetings and reviews and to
evaluate the documents, plans,
requirements and specifications.
This can be achieved by using
reviews and meetings etc.
Verification:confirmation byexamination and provision of
objective evidence that specified
requirements have been fulfilled.
Verification: Are we building the
product right?
The Waterfall model is also known
as the Sequential model. Each
stage follows on from the previous
one. The testing is performed in
block as the last stage.
Planning or Test creation is notconsidered until the actual softwarecode has been written. This canresult in problems being found muchlater in the project lifecycle than isdesirable.
Walkthrough ReviewTechnical Review Inspection Review Informal Review
V & V Validation Verification Waterfall Model
8/14/2019 Practitioner Flashcards Fronts
2/15
The V-Model is an industry standard
framework that shows clearly the
software development lifecycle in
relation to testing. It also highlightsthe fact that the testing is just as
important as the software
development itself. The relationships
between development and testing
are clearly defined.
The V-Model improves the presenceof the testing activities to display amore balanced approach.
The Spiral model is an incrementaltesting approach to bothDevelopment and Testing. This isused most effectively when the
users do not know all of therequirements. From what is known,initial requirements can be defined.Then from these the code and testcases are created. As time goes on,more details of the requirements areknown and implemented in furtheriterations of design, coding andtesting phases. The system isconsidered to be complete, whenenough of the iterations have takenplace.
RAD represents Rapid ApplicationDevelopment, and is a softwaredevelopment process that wasdeveloped in the mid 1980s. It was
developed to overcome the rigidityof such processes as The WaterfallModel.
Elements:
Prototyping
Iterative Development
Time-boxing
Team Members
Management Approach
RAD Tools
DSDM (Dynamic Systems
Development Methodology) is
basically a high level framework of
already proven RAD techniques, andalso management controls that are
used to increase the chances of
successful RAD projects.
The high level framework allows for
a process that can be easi ly
modified for individual projects
specific needs. But, this quite simple
framework also results in poor
implementation due to lack of detail.
As a Tester, your focus will be fixed
on the test process. But we must
consider other processes that exist,
and also their interaction with the
test process.
Project Management
Change Management
Configuration Management
Software Development
Technical Writing
Technical Support
Component testing is also known as
Unit, Module, or Program Testing. In
simple terms, this type of testing
focuses simply on testing of the
individual components themselves.
It is common for component testing
to be carried out by the Developer of
the software. This however has a
very low rating of testing
independence.
This type of Integration testing isconcerned with ensuring theinteractions between the softwarecomponents at the module levelbehave as expected. ComponentIntegration Testing is also often
referred to as Integration Testing inthe Small.
It is commonly performed after anyComponent Testing has completed,and the behaviour tested may coverboth functional and non-functionalaspects of the integrated system.
Requirements-based Testing issimply testing the functionality of thesoftware/system based on therequirements. The tests themselvesshould be derived from thedocumented requirements and not
based on the software code itself.This method of functional testingensures that the users will be gettingwhat they want, as the requirementsdocument basically specifies whatthe user has asked for.
Spiral ModelV - Model RAD DDSM
Process Interfaces Component TestingComponent Integration
TestingRequirements BasedFunctional Testing
8/14/2019 Practitioner Flashcards Fronts
3/15
Different types of users may use thedeveloped software in different
ways. These ways are analysed andbusiness scenarios are thencreated. User profiles are often usedin Business Process FunctionalTesting. Remember that all of thefunctionality should be tested for,not just the most commonly usedareas.
Testing the ability of the system tobe able to bear loads. An examplewould be testing that a system could
process a specified amount oftransactions within a specified timeperiod. So you are effectivelyloading the system up to a highlevel, then ensuring it can stillfunction correctly whilst under thisheavy load.
A program/system may haverequirements to meet certain levelsof performance. For a program, this
could be the speed of which it canprocess a given task. For anetworking device, it could mean thethroughput of network traffic rate.Often, Performance Testing isdesigned to be negative, i.e. provethat the system does not meet itsrequired level of performance.
Stress Testing simply means puttingthe system under stress. The testingis not normally carried out over a
long period, as this would effectivelybe a form of duration testing.Imagine a system was designed toprocess a maximum of 1000transactions in an hour. A stress testwould be seeing if the systems couldactually cope with that manytransactions in a given time period. Auseful test in this case would be tosee how the system copes whenasked to process more than 1000.
A major requirement in todayssoftware/systems is security,particularly with the internetrevolution. Security testing isfocused at finding loopholes in theprograms security checks. Acommon approach is to create test
cases based on known problemsfrom a similar program, and testthese against the program undertest.
This is where consideration is takeninto account of how the user will usethe product. It is common forconsiderable resources to be spenton defining exactly what thecustomer requires and simple it is touse the program to achieve there
aims. For example; test cases couldbe created based on the GraphicalUser Interface, to see how easy itwould be to use in relation to atypical customer scenario.
This type of testing may focus onthe actual memory used by aprogram or system under certainconditions. Also disk space used bythe program/system could also be afactor. These factors may actuallycome from a requirement, and
should be approached from anegative testing point of view.
Volume Testing is a form of SystemsTesting. It primary focus is toconcentrate on testing the systemswhile subjected to heavy volumes ofdata. Testing should be approachedfrom a negative point of view toshow that the program/system
cannot operate correctly when usingthe volume of data specified in therequirements.
Load TestingBusiness ProcessFunctional Testing
Performance Testing Stress Testing
Security Testing Useability Testing Storage Testing Volume Testing
8/14/2019 Practitioner Flashcards Fronts
4/15
A complicated program may alsohave a complicated installationprocess. Consideration should bemade as to whether the program willbe installed by a customer or aninstallation engineer. Customerinstallations commonly use somekind of automated installationprogram. This would obviously haveto under go significant testing initself, as an incorrect installationprocedure could render the targetmachine/system useless.
Documentation in todaysenvironment can take several forms,as the documentation could be aprinted document, an integral help
file or even a web page. Dependingof the documentation media type,some example areas to focus oncould be, spelling, usability,technical accuracy etc.
Recovery Testing is normally carriedout by using test cases based onspecific requirements. A system maybe designed to fail under a given
scenario, for example if attacked bya malicious user; theprogram/system may have beendesigned to shut down. Recoverytesting should focus on how thesystem handles the failure and howit handles the recovery process.
This type of Integration Testing isconcerned with ensuring theinteractions between systemsbehave as expected. It is commonly
performed after any Systems Testinghas completed. Typically not allsystems referenced in the testing arecontrolled by the developingorganization. Some systems maybecontrolled by other organizations, butinterface directly with the systemunder test.
User Acceptance Testing or UAT iscommonly the last testing performedon the software product before itsactual release. It is common for thecustomer to perform this type oftesting, or at least be partiallyinvolved. Often, the test ingenvironment used to perform UserAcceptance Testing is based on a
model of the customer senvironment. This is done to try andsimulate as closely as possible theway in which the software productwil l actually be used by thecustomer.
This type of Acceptance Testing isaimed at ensuring the acceptancecriteria within the original contracthave indeed been met by thedeveloped software. Normally anyacceptance criteria is defined whenthe contract is agreed. RegulationAcceptance Testing is performed
when there exists specificregulations that must be adhered to,for example, there may be safetyregulations, or legal regulations.
This form of acceptance testing iscommonly performed by a SystemAdministrator and would normally beconcerned with ensuring thatfunctionality such as;backup/restore, maintenance, andsecurity functionality is present andbehaves as expected.
Alpha Testing should be performedat the developers si te, andpredominantly performed by internaltesters only. Often, other companydepartment personnel can act astesters. The marketing or salesdepartments are often chosen forthis purpose.
Documentation TestingInstallability Testing Recovery TestingSystem Integration
Testing
UAT Contract & RegulationAcceptance Testing
Operational AcceptanceTesting
Alpha Testing
8/14/2019 Practitioner Flashcards Fronts
5/15
Beta Testing is commonly performedat the customers site, and normallycarried out by the customersthemselves. Potential customers areoften eager to trial a new product ornew software version. This allowsthe customer to see anyimprovements at first hand andascertain whether or not it satisfiestheir requirements. On the flip side,it gives invaluable feedback to thedeveloper, often at little or no cost.
It is imperative that when a fault isfixed it is re-tested to ensure thefault has indeed been correctlyfixed.
Re-test:Whenever a fault is detected andfixed then the software should bere-tested to ensure that the originalfault has been successful lyremoved.
When checking a fixed fault, youcan also consider checking thatother existing functionality has notbeen adversely affected by the fix.
This is called Regression Testing.
Regression Test:Regression testing attempts toverify that modifications have notcaused unintended adverse sideeffects in the unchanged software(regression faults) and that themodified system still meets itsrequirements.
A Test Plan should be a singledocument that basically containswhat is going to be tested, why it isgoing to be tested, and how it is
going to be tested. I t is alsoimportant to clarify what is not goingto be tested in the software producttoo. With regards to using a standardTest Plan layout, then we can look tothe advice given by theIEEE(Institute of Electrical andElectronic Engineers) located in theInternational Standard IEEE Std929-1998.
A standard test process that iscommonly used exists within theBS7925-2 Standard for SoftwareComponent Testing:
Test Planning
Test Specification
Test Execution
Test Checking & Recording
Checking for TestCompletion
This should apply to both new
projects and maintenance work.
Normally fairly short in length, the
test policy should be a high-level
document, and should contain the
following items:
Definition of testing
The testing process
Evaluation of testing
Quality levels
Improvement approach
Based on the test policy, the test
strategy is designed to give an
overview of the test requirements for
a programme or even organization.
Information relating to risks should
be documented here, specifically
the risks that will be addressed by
the testing, and the specific tests
that will be used against each risk.
Exactly how the test strategy for a
particular project will be
implemented is displayed in the
project plan. The project test plan will
normally be referenced from the
overall project plan. In relation to the
test strategy, the project plan shoulddetail items from the test strategy
that it is complying with, and also
items it is not complying with.
Re-TestBeta Testing Regression TestingTest PlanDocument
Generic TestProcess
Test Policy Test Strategy Project Plan
8/14/2019 Practitioner Flashcards Fronts
6/15
The specific details of the approach
taken by the testing for a specific
test phase is shown in this
document. It can be thought of asbeing based on the project plan, but
with greater amounts of detail
included, such as testing activities
based on day to day plan, or
expected amounts of man hours to
complete individual tasks.
Risk management comprises of thefollowing three components:
Risk Identification
Risk Analysis
Risk Mitigation
Risk management should be aconsideration for everyone involvedin the project.
The following techniques can all beused to identify risks associated withproducts and projects. The list is byno means rigid, as many
organisations will have there owntechniques.
Expert Interviews
Independent Assessment
Risk Templates
Lessons Learned
Risk Workshops
Brainstorming andChecklists
So what does the term Risk
Analysis actually mean? It is simply
studying the identified risks. A
simple formula can be used tocalculate risk:
Frequency (likelihood) X Severity
(impact)
By using the above formula we can
produce a figure, otherwise known
as the exposure.
Risk mitigation is simply the
response to the analysed risks. A
choice must be made as what action
should be carried out once a risk
has been identified. Some possible
choices could be:
Do nothing
Take preventative action
(test it)
Contingency plan (what we
should do if the predicted
fault actually occurs)
What this method allows you to dois effectively partition the possibleprogram inputs. For each of theabove input fields, it should notmatter which values are entered aslong as they are within the correctrange and of the correct type.
So the point of equivalenceportioning is to reduce the amountof testing by choosing a smallselection of the possible values tobe tested, as the program willhandle them in the same way.
By the use of equivalencepartitioning, a tester can performeffective testing without testingevery possible value. This methodcan be enhanced further by anothermethod called Boundary ValueAnalysis. After time, an experiencedTester will be often realise that
problems can occur at theboundaries of the input and outputspaces. When testing only a smallamount of possible values, theminimum and maximum possiblevalues should be amongst the firstitems to be tested.
The classification tree method is alsoknown as a decision tree methodand the terms can be usedinterchangeably as they mean thesame thing. A decision tree can belearned by splitting the source setinto subsets based on an attributevalue test. This process is repeated
on each subset in a recursively. Therecursion is completed when splittingis either not possible, or a singleclassification can be applied to eachelement of the subset.
Risk ManagementPhase Test Plan Risk Identification Risk Analysis
Risk Mitigation EquivalencePartitioning
Boundary ValueAnalysis
Classification TreeMethod
8/14/2019 Practitioner Flashcards Fronts
7/15
This type of Black-box testing isbased on the concept of states andfinite-states, and is based on thetester being able to view thesoftwares states, transition betweenstates, and what will trigger a statechange. Test cases can then bedesigned to execute the statechanges.
This testing method involves usinga model of the source code whichidentifies statements. Thesestatements are the categorized as
being either executable or non-executable. In order to use thismethod, the input to eachcomponent must be identified. Also,each test case must be able toidentify each individual statement.Lastly, the expected outcome ofeach test case must be clearlydefined
This test method uses a model ofthe source code which identifiesindividual decisions, and theiroutcomes. A decision is defined as
being an executable statementcontaining its own logic.
This logic may also have thecapability to transfer control toanother statement. Each test case isdesigned to exercise the decisionoutcomes. In order to use thismethod, the input to eachcomponent must be identified.
Branch Condition Testing uses a
model of the source code, and
identifies decisions based on
individual Boolean operands withineach decision condition. A decision
is defined as being an executable
statement containing its own logic.
An example of a decision would be a
loop in a program.
Branch Condition Combination
Testing uses a model of the source
code, and identifies decisions based
on combinations of Boolean
operands within decision conditions.
This logic may also have the
capability to transfer control to
another statement. The decision
condition is a Boolean expressionwhich is evaluated to determine the
outcome of the decision.
Requirements-based Testing issimply testing the functionality of thesoftware/system based on therequirements. The tests themselvesshould be derived from thedocumented requirements and notbased on the software code itself.This method of functional testingensures that the users will begetting what they want, as therequirements document basicallyspecifies what the user has askedfor.
This is where consideration is takeninto account of how the user will usethe product. It is common forconsiderable resources to be spenton defining exactly what thecustomer requires and how simple itis to use the program to achievethere aims. For example; test casescould be created based on the
Graphical User Interface, to seehow easy it would be to use inrelation to a typical customerscenario.
Volume Testing is a form of SystemsTesting. It primary focus is toconcentrate on testing the systemswhile subjected to heavy volumes ofdata. Testing should be approachedfrom a negative point of view toshow that the program/systemcannot operate correctly when usingthe volume of data specified in therequirements.
Statement TestingState Transition
TestingBranch Decision
TestingBranch Condition
Testing
Branch ConditionCombination Testing Requirements BasedFunctional TestingUseability Testing Volume Testing
8/14/2019 Practitioner Flashcards Fronts
8/15
A program/system may haverequirements to meet certain levelsof performance. For a program, thiscould be the speed of which it canprocess a given task. For anetworking device, it could mean thethroughput of network traffic rate.Often, Performance Testing isdesigned to be negative, i.e. provethat the system does not meet itsrequired level of performance.
Stress Testing simply means puttingthe system under stress. The testingis not normally carried out over along period, as this would effectively
be a form of duration testing.Imagine a system was designed toprocess a maximum of 1000transactions in an hour. A stress testwould be seeing if the systemscould actually cope with that manytransactions in a given time period.A useful test in this case would be tosee how the system copes whenasked to process more than 1000.
Dynamic analysis is a testingmethod that can provide informationon the state of software. It canachieve this dynamically i.e. it
provides information when thesoftware is actually running. It iscommonly used to exercise parts ofthe program that use memoryresources e.g.:
Memory allocation
Memory usage
Memory de-allocation
Memory leaks
Unassigned pointers
Static Analysis is a set of methodsdesigned to analyse software codein an effort to establish it is correct,prior to actually running the software.
As we already know, the earlier wefind a fault the cheaper it is to fix. Soby using Static Analysis, we caneffectively test the program evenbefore it has been written. Thiswould obviously only find a limitednumber of problems, but at least it issomething that can be done veryearly on in the development lifecycle.
Control flow graphs display the logicstructure of software. The flow oflogic through the program ischarted. It is normally used only byDevelopers as it is a very low levelform testing, often used inComponent Testing.
It can be used to determine thenumber of test cases required totest the programs logic. It can alsoprovide confidence that the detail ofthe logic in the code has beenchecked.
Cyclomatic Complexity is a softwaremetric that is used to measure thecomplexity of a software program.Once we know now how complexthe program is, we then know howeasy it will be to test.
C = E N + P
C = Cyclomatic Complexity
E = number of edges
N = number of nodes
P = number of components
The most basic form of a complexitymetric is the Lines of Code metric,or LOC metric. Its purpose likeother complexity metrics is toestimate the amount of effort thatwill be required not only to developsuch a program, but also assist inestimating how much effort will be
required to test it.
In its simplest form we could use theLOC metric by literally counting thenumber of lines of code in theprogram.
The idea behind Data-flow Analysisis to work-out the dependenciesbetween items of data that are usedby a program. When a program isran, it rarely runs in a sequentialorder i.e. starting at line 1 andfinishing at line 100. What usuallyhappens is that the dependencies ofthe data within the program will
determine the order. Data-flowAnalysis can be used to f inddefinitions that have no interveninguse. Data-flow analysis is also usedto detect variables that are usedafter it has effectively been killed.
Stress TestingPerformance Testing Dynamic Analysis Static Analysis
Control FlowGraphing
Cyclomatic Complexity Lines of Code Data Flow Analysis
8/14/2019 Practitioner Flashcards Fronts
9/15
Why can one Tester find more errorsthan another Tester in the samepiece of software? More often thannot this is down to a techniquecalled Error Guessing. To besuccessful at Error Guessing, acertain level of knowledge andexperience is required. A Tester canthen make an educated guess atwhere potential problems may arise.This could be based on the Testersexperience with a previous iterationof the software, or just a level ofknowledge in that area of technology.
This type of testing is normallygoverned by time. It consists ofusing tests based on a test chapterthat contains test objectives. It is
most effective when there are littleor no specifications available. Itshould only really be used to assistwith, or compliment a more formalapproach. It can basically ensurethat major functionality is working asexpected without fully testing it.
This type of testing is considered tobe the most informal, and by many itis considered to be the leasteffective. Ad-hoc testing is simply
making up the tests as you goalong. Often, it is used when there isonly a very small amount of time totest something. A common mistaketo make with Ad-hoc testing is notdocumenting the tests performedand the test results. Even if thisinformation is included, more oftenthan not additional information is notlogged such as, software versions,dates, test environment details etc.
A Tester normally selects test inputdata from what is termed an inputdomain. Random Testing is simplywhen the Tester selects data from
the input domain randomly. As youcan tell, there is little structureinvolved in Random Testing. Inorder to avoid dealing with the abovequestions, a more structured Black-box Test Design could beimplemented instead. However,using a random approach could savevaluable time and resources if usedin the right circumstances.
A Quality Assurance (QA) standardsimply specifies that testing shouldbe performed.
Example: ISO 9000
An industry specific standard willdetail exactly what level of testing isto be performed.
Examples:
Railway Signalling standard
DO-178B
Nuclear Industry standard
MISRA guidelines for motorvehicle software
Pharmaceutical standards
Testing standards will detail how toperform the testing. Ideally, atest ing standard should bereferenced from a QA or Industryspecific standard.
Example: BS7925-1, BS7925-2
Review: A process or meeting duringwhich a work product, or set of workproducts, is presented to projectpersonnel, managers, users or otherinterested parties for comment orapproval. [IEEE]
A review should be performed whenall of the supporting documentationis available. This can include designdocuments, requi rementsdocuments, standards documents,basically any documentation that haseither been inf luential or isapplicable to the document to bereviewed.
Exploratory TestingError Guessing Ad-hoc Testing Random Testing
Quality AssuranceStandards
Industry SpecificStandards
Testing Standards Review Definition
8/14/2019 Practitioner Flashcards Fronts
10/15
Organisations will commonly havedifferent named roles than thoselisted below, but this will give you anidea of a commonly used set of
roles used throughout the world.
Manager
Moderator
Author
Reviewer
Scribe
An example of a typical reviewprocess is below. This is probablythe most documented reviewprocess you will find in the software
development world, and is open tointerpretation:
Planning
Kick-off
Preparation
Meeting
Rework
Follow-up
Exit Criteria
We term an incident; any significant,unplanned event that occurs duringtesting that requires subsequentinvestigation and/or correction. The
incident should be raised when theactual result differs from theexpected result. After the inevitableinvestigation of the incident, theremay be a reason other than asoftware fault, for example:
Test environment incorrectlyset up
Incorrect Test Data used
Incorrect Test Specification
This standard aims to provide astandard approach to classificationof anomalies found in software. Itincludes descript ions of the
processes involved in a software lifecycle, including details on howanomalies should be recorded andsubsequently processed. It consistsof four sequential steps;Recognition, Investigation, Action,Disposition. Each of those steps hasthree administrative activities whichare; Recording, Classifying,Identifying Impact.
A maturity model is basically a
collection of elements that are
structured in such a way that they
can describe characteristics of
processes and their effectiveness. A
maturity model can provide:
A starting point A shared vision
A structure for organising
actions
Use of previous experience
value of improvements
The Capability Maturity Model,
simply put, is a baseline of practices
that should be implemented in order
to develop or maintain a product.
The product can be completely
software, or just partially software.
The SW-CMM focuses on thesoftware practices whereas with the
CMMI, you may find both software
and systems practices.
The CMM defines five maturitylevels which form the top-levelstructure of the CMM itself. Eachlevel is basically a foundation thatcan be built upon to improve theprocess in sequence. Starting withbasic management practices andprogressing through successiveproven levels.
Initial
Managed
Defined
Quantitatively Managed
Optimising
The software process capabilitydefines what can be achieved byundertaking a specific softwareprocess. It achieves this bydescribing the range of expectedresults. There are six capabilitylevels.
Incomplete
Performed
Managed
Defined
Quantitatively Managed
Optimising
Review ProcessStructure
Review Roles Incident Management IEEE Std. 1044-1993
Maturity ModelDefinition
SEI Capability MaturityModel (CMMI)
CMM Maturity Levels CMM Capability Levels
8/14/2019 Practitioner Flashcards Fronts
11/15
The ISO/IEC 15504 is also knownas SPICE. Spice stands forSoftware Process Improvement andCapability dEtermination. It isessentially a framework forassessing software processes.Rather than concerning itself withspecific standards, ISO/ISEC 15504concerns itself with is thecapabili ties provided by anorganisations structure. Thesestructures include its managementstructure and its process definitionstructure.
The Illinois Institute of Technology(IIT) developed the Testing MaturityModel (TMM) in 1996. The mainreason for developing the TMM was
that existing maturity models didntproperly address real testing issues.It was designed to complement theexisting CMM. The main purpose ofthe TMM is to support assessmentand improvement drives within anorganisation. The model comprisesof a Maturity Model and anAssessment Model.
The maturity levels are basicallydefined levels of maturity that canbe achieved by showing thatspecific practices have been carried
out. The TMM has five differentachievable levels:
Initial
Phase Definition
Integration
Management/Measurement
Optimisation/defectPrevention and QualityControl
Developed by Koomen and Pol in1997, the Test Process ImprovementModel or TPI was created with thegoal of simplifying the sometimesover-complicated testing process.
The TPI model itself identifies thegood and bad parts of a testingprocess. The maturity of the processcan also be assessed by using theTPI. The TPI consists of thefollowing four components:
A Maturity Model
A Test Maturity Matrix
A Checklist
Improvement Suggestions
The TPI model consists of threematurity levels and fourteen scales.The individual levels contain severaldifferent scales. The scalesthemselves provide indication ofwhich key areas requireimprovement.
Scales 1 to 5 focus on bringthe testing process undercontrol
Scales 6 to 10 focus onestablishing test processefficiency
Scales 11 to 14 focus ontest process optimisation
The TPI takes into account the
different aspects of a test process,
including design techniques, test
tool usage and reporting. Structured
evaluation of various key areas,
highlights the test processes
strengths and weaknesses. The
state of a key area is determined byassigning a level to it, commonly A
to B to C etc. The levels are
increased based on time, cost and
quality.
This type of tool is designed toassist with verification and validationof requirements, for example;consistency checking.
By examining the code instead ofrunning test cases through the code,this type of tool can provideinformation on the actual quality ofthe software. Cyclomatic complexityis one such characteristic that canbe obtained by using this type oftool.
TMM DefinitionISO/IEC 15504 (SPICE)
DefinitionTMM Maturity Levels TPI Definition
TPI Model TPITest Maturity Matrix
RequirementsTesting Tools
Static AnalysisTools
8/14/2019 Practitioner Flashcards Fronts
12/15
This type of tool can generate testcases from specifications, which arenormally stored in a CASE toolrepository. Some variations of this
type of tool can also generate testcases from analysing the code itself.
Data can be selected from existingtest specific databases by using thistype of tool. Advanced types of thistool can utilise a range of database
and file formats.
These are an extremely populartype of tool. They provide captureand replay facilities for WIMPinterface based applications. The
tools can simulate mousemovement, mouse clicks andkeyboard inputs. The tools can evenrecognize windows and buttons,thus making them extremelyversatile. The test procedures arenormally written in a specificscripting language. This tool isanother popular choice forregression testing.
If the software under test does not
have a user interface, then test
harnesses and drivers can be used
to execute the software. These typesof tools can be bought off the shelf,
but more commonly they are built for
a specific purpose.
Creates actual test scripts based oninformation held within a testspecification. Simulators arecommonly used where it isimpracticable to use them, forexample software to control a spaceprobes trajectory.
This type of tool comprises of two
components; Load Generation and
Test Transaction Measurement.,
Load Generation is commonly
performed by running the
application using its interface or by
using drivers. The number of
transactions performed this way arethen logged. Performance test tools
will commonly be able to display
reports and graphs of load against
response time.
Run-time information on the state of
the executing software is achieved
by using Dynamic Analysis Tools.
These tools are ideally suited for
monitoring the use and allocation of
memory. Faults such as memory
leaks, unassigned pointers can be
found, which would otherwise be
difficult to find manually.
Debugging tools are often used by
programmers to try and reproduce
code related errors in order to
investigate a problem. The debugger
allows the program to be run line by
line. This enables halted the program
on demand to examine and set
program variables.
Test Input DataPreparation Tool
Test Design Tools Test Running Tools Test Harnesses
Test Script GeneratorsPerformance Test
Tools
Dynamic AnalysisTools
Debugging Tools
8/14/2019 Practitioner Flashcards Fronts
13/15
This type of tool is used to highlightdifferences between actual resultsand expected results. Off the shelfComparison Tools can normally dealwith a range of file and database
formats. This type of tool often hasfilter capabilities to allow ignoring ofrows or columns of data or evenareas on a screen
Test Management Tools commonly
have multiple features. Test
Management is mainly concerned
with the management, creation and
control of test documentation. Moreadvanced tools have additional
capabilities such as test
management features, for example;
result logging and test scheduling.
This type of tool provides objective
measures of structural test coverage
when the actual tests are executed.
Before the programs are compiled,
they are first instrumented. Oncethis has been completed they can
then be tested. The instrumentation
process allows the coverage data to
be logged whilst the program is
running. Once testing is complete,
the logs can provide statistics on the
details of the tests covered.
These tools are simply used tocheck that no broken hyperlinks existon a web site.
These tools are typically used fortesting e-commerce and e-businessapplications. The main purpose ofthis tool is to check web sites toensure that they are available tocustomers and also to producewarnings if problems are detected.
These tools are commonly used fortesting e-commerce and e-businessapplications, and sometimes websites. A security testing tool willcheck for any parts of a web basedsystem that could cause potentialsecurity risks if attacked.
A Test Oracle is used toautomatically generate expectedresults. They are commonly used insituations where an old system isupgraded with a new system withthe same functionality, so the oldsystem can be used as an Oracle.
A suggested tool selection andevaluation process is:
Determine the actual problem orrequirement
Ensure that there are no obviousalternative solutions
Prepare a business case
Identify any constraints
Identify any specific required toolfeatures or characteristics
Prepare a short-list of possiblesuitable tools
Perform a detailed evaluation
Perform a competitive trial, ifneeded
Test ManagementTools
Comparison ToolsCoverage Measurement
ToolsHyperlink Testing
Tools
Monitoring Tools Security TestingTools Test Oracles Tool SelectionProcess
8/14/2019 Practitioner Flashcards Fronts
14/15
The last thing we want is tointroduce a tool into theorganisation, only to find a fewweeks down the line it fails resultingin potentially disastrous scenarios.
In order to avoid this situation, wecan implement a pilot project. Thebenefits of using a pilot project are;
Gaining experience usingthe tool
Identify any test processchanges
Identify any shortcomingssuitability of the tool
An implementation team can beformed to evaluate a new toolconsisting of:
A Champion A Change Agent
A Tool Custodian
A User is someone who has hadexperience actually using thesoftware under test, or similar typesof software. This knowledge can be
useful to determine the type of faultsthat a typical user may comeacross, which to most developmentswould also have the most impact.The User would probably not havesufficient knowledge to test thesoftware to extreme depths though.
A developers background may haveprovided them with experience withcode, design or requirementsanalysis. This knowledge can be
extremely useful, as the developerwould probably have some ideawhen looking at the software of howit was developed, and so wouldprobably know where to look forweaknesses.
Previous knowledge of testing hasits obvious advantages. They shouldbe able to analyse a specification,design test cases, execute testcases, and produce results andreports. An individual with previoustesting experience would also havethe right mindset for testing, as theywould already know the reasoning
behind why testing is performed.
Shapers: Challenging, dynamic,thrives on pressure. The drive andcourage to overcome obstacles.Prone to provocation. Offendspeople's feelingsImplementer: Disciplined, reliable,conservative and eff icient.Somewhat inflexible. Slow torespond to new possibilities
Completer Finisher: Painstaking,conscientious, anxious. Searchesout errors and omissions. Deliverson time. Inclined to worry unduly.Reluctant to delegate.
Coordinator: Mature, confident, agood chairperson. Clarifies goals,promotes decision-making,delegates well. Can often be seenas manipulative.Team Worker: Co-operative, mild,perceptive and diplomatic. Listens,builds, averts friction. Indecisive incrunch situations.Resource Investigator: Extrovert,enthusiastic, communicative.Explores opportunities. Developscontacts. Over - optimistic. Losesinterest after short period.
Plant: Creative, imaginative,unorthodox. Solves diff icultproblems. Ignores incidentals. Toopre-occupied to communicateeffectively.Monitor Evaluator: Sober,strategic and discerning. Sees alloptions. Judges accurately. Lacksdrive and ability to inspire others.Specialist: Single-minded, self-
starting, dedicated. Providesknowledge and skills in rare supply.Contributes only on a narrow front.Dwells on technicalities.
Test ToolImplementation Team
Pilot Projects User Skills Developer Skills
Tester Skills Belbins ActionOriented Roles
Belbins PeopleOriented Roles
Belbins ThoughtOriented Roles
8/14/2019 Practitioner Flashcards Fronts
15/15
The Test Leader will commonlycome from a testing backgroundand have a full understanding ofhow testing is performed. They willalso possess good managerial
expertise. They are also responsiblefor ensuring that test coverage issufficient and will be required toproduce reports.
The Tester obviously provides theskills necessary to perform theTesting itself. This role can includetest design and test execution.
Automated testing skills are also apossible requirement of this role.
Preparation of test data
Execute tests
Review other peoples tests
Review Test Plan
Involvement in automationof tests
Create Test Specifications
The client is effectively the projectsponsor, and will provide the budgetfor the project. The Client can alsobe the business owner.
Management skills are provided bythe Project Manager. The ProjectManager will be actively involvedthroughout the project and will
provide feedback to the client.
A Developer will provide the skills towrite the actual software code andperform Unit Testing. They may alsobe called upon at a later stage toprovide bug fixes and technicaladvice.
The Business Analyst will provideknowledge of the business andanalysis skills. The Business Analystwill also be responsible for creatingUser Requirements based on talkswith the Users.
Systems design will be provided bythe Systems Analyst. The SystemsAnalyst will also be responsible fordeveloping the FunctionalSpecif icat ion from the UserRequirements.
Technical detail and support to thesystem design is the responsibility ofthe Technical Designer. This rolemay include databaseadministration.
The TesterThe Test Leader The Client The Project Manager
The Developer The Business Analyst The Systems Analyst The Technical Designer