Upload
horace-ross
View
227
Download
0
Tags:
Embed Size (px)
Citation preview
Theory and Practice ofCo-verification Process: UniTesK Story
RedVerst group of ISP RAShttp://www.ispras.ru/groups/rv/rv.html
Alexander K. Petrenko ([email protected])Victor V. Kuliamin ([email protected])
Overview
• Introduction : Why co-verification?slides 3-13
• Main part : What is UniTesK?Solving Engineering Problemsslides 14-55
• Case studiesslides 56-64
What do these numbers mean?
180 109 $ 21 109 $
total revenue of US software development
companies
loss caused by inadequate testing
infrastructure for US economy1
1. The Economic Impacts of Inadequate Infrastructure for Software Testing,NIST Report, May 2002
9 : 1
Waterflow Process Model
Requirements
Design
Implementation
Testing
Deployment
Iterative Process Model
Requirements
Design
Implementation
Testing
Deployment
Inception Elaboration Construction Transition
Race for Quality : CMM Certified Organizations
0 5 10 15 20 25 30 35
1998
1999
2000
2001
2002
CMM Level 5
CMM Level 4
CMM Level 3
CMM Level 2
According to Compiled List of Organizations Publicly Announced their Maturity Levels,http://seir.sei.cmu.edu/pml/
Race for Flexibility : Agile Development Methods
According to “The Decision is in: Agile versus Heavy Methodologies” by Robert Charette,Cutter IT Journal, vol. 2, No. 19http://www.cutter.com/freestuff/apmupdate.html
Percent of organizations using modern development processes
0%
10%
20%
30%
40%
50%
60%
Agile Development Methods
RUP
CMM
ISO 9000-based Methods
Need for Quick Change Response
0%
10%
20%
30%
40%
50%
60%
2001
2002
2003
According to “The Decision is in: Agile versus Heavy Methodologies” by Robert Charette,Cutter IT Journal, vol. 2, No. 19http://www.cutter.com/freestuff/apmupdate.html
Percent of organizations recognizing more than 50% of projects as agile
Inadequate Quality of Software
Loss Potential Cost Reduction
Total Sales
Software Vendors
21.2109 $ 10.6109 $ 180109 $
Software Users
38.3109 $ 11.7109 $
Total 59.5109 $ 22.2109 $
The Economic Impacts of Inadequate Infrastructure for Software Testing,NIST Report, May 2002
Evolution of Testing
• Localization of errors• Demonstration of errors
– Testing is the process of executing a program or system with the intent of finding errors [Myers, 1979]
– The purpose of testing is to show that a program has bugs [Hetzel, 1988]
• Evaluation of quality– Testing is the process of operating a system or component
under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component [IEEE 90]
Co-verification
Verification confirms that activity products properly reflect the requirements specified for them at the beginning of the activity.
Co-verification process
• Perform verification of activity before proceeding to the dependent activities
• Prepare all the artifacts needed for verification concurrently with main activity products
Traditional Development Process
Business Modeling
Requirements
Architecture Design
Component Design
Implementation
Integration
Deployment
Co-verification Development Process
Business Modeling
Requirements
Architecture Design
Component Design
Implementation
Integration
Deployment
Main Part Overview
• Traditional approaches to verification
• UniTesK approach
• Example
Traditional Software Development Process
• Requirements Analysis• Design• Implementation and
debugging• Requirements
Elicitation• Test Case Design• Test Implementation• Test Execution• Test Result Analysis
Design and development Verification
Co-verification Process
• Requirements Analysis
• Design• Implementation and
debugging
• Requirements Elicitation
• Test Case Design• Test Implementation• Test Execution• Test Result Analysis
Design and development Verification
Uniform Test
Architecture
UniTesK Approach
Uniform Specification
Extension
Integration with Development Environments
CT
esK
J@T
.N@
T
VD
M+
+T
esK
Fou
ndat
ions
Too
ls
UniTesK
Model Based Testing
Requirements Formalization
Requirements FormalSpecifications
Co-verification Support
FormalSpecificationsRequirements
Ambiguity? Incompleteness?
Inconsistency?
Engineering Problems (1)
• Specification technique should support– Easy transformation of requirements into
specifications– Easy automation of further test development– Functional test coverage definition– High reusability of specifications
• Specification notation should not require special education and skills
Specification Techniques
• ExecutableImperative state based specifications
• ConstraintsState based data type constraints, pre- and postconditions, internal invariants
• AxiomaticAction based axioms
Comparison of Specification Techniques
0 1 2 3 4 5 6
Reusability
Ease of oracle generation
Source for coverage models
Closeness to requirements
Decoupling with implementation
Ease of use in the industry
Constraints
Executable
Algebraic
Comparison Results
0 1 2 3 4
Average
Constraints
Executable
Algebraic
Specification Notation
• Specification language– Suitable for capture abstract properties– Has formal semantics– Requires complex mediator layer for implementation– Requires special education, mastering is enduring
• Extension of programming language– Abstract concepts can be added by libraries– Ambiguous parts of language can be excluded– Complex mediators are not required– Mastering can be made more effective
– Facilitates co-verification
UniTesK Specification Technique
Uniform Test ArchitectureUniform
Specification Extension
Integration with Development Environments
CT
esK
J@T
.N@
T
VD
M+
+T
esK
UniTesK
Model Based Testing
Uniform Specification Extension• Constraint Specifications
– Preconditions and postconditions of operations
– Data type constraints
• Functional coverage description based on specification structure
UniTesK Specification Technique
• Constraint Specifications– Preconditions and postconditions of operations
– Data type constraints
specification Operation()pre block, returning Boolean value
post block, returning Boolean value use @ to refer to pre-value of expressions
invariant Inv() block, returning Boolean value
Uniform Test ArchitectureUniform
Specification Extension
Integration with Development Environments
CT
esK
J@T
.N@
T
VD
M+
+T
esK
UniTesK
Model Based Testing
J@va : Specification Extension of Java
specification package pqueue;
public class PQueueSpecification{ specification public void enq(Object obj, int prty) reads obj, prty updates items.?, priorities.? { pre { return obj != null; } post { int i = 0; for(i = 0; i < items.size() && priorities.elementAt(i) > prty; i++ ); ...
Coverage Goals Definition
Requirements
Formal Specificationspre ---------------------------post --------------------------- -------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- --------------------------------------------- -------------------------------------------------- ---- -------------------------------------
Test Case 1--------------------------------------
Formal Specificationspre ---------------------------post ---------------------------
-------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- ---------------------------------------------- --------------------------------------------------
---- -------------------------------------
123
Co-verification Support
Formal Specificationspre ---------------------------post --------------------------- -------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- --------------------------------------------- -------------------------------------------------- ---- -------------------------------------
Testable Specifications
pre ---------------------------post ---------------------------
-------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- ---------------------------------------------- --------------------------------------------------
---- -------------------------------------
Software --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- ---------------------------------------------------
UniTesK supports this transformation
with techniques and tools
Engineering Problems (2)
• Automatic extraction of coverage goals from the specification structure
• More than one coverage metric is needed
• Test designer should be able to introduce additional goals
Several Levels of Coverage Metrics
Formal Specifications
pre ---------------------------post ---------------------------
-------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- ---------------------------------------------- --------------------------------------------------
---- -------------------------------------
123
1.11.21.3
2.12.2
3.13.2
1.2.11.2.2
1.3.11.3.21.3.3
3.1.13.1.23.1.3
Definition of Coverage Goals : Functionality Branches
• Functional coverage description based on specification structurepost if(a || b) ... branch “Case 1”; ... else if(!c && d) ... branch “Case 2”; ... else ... branch “Case 3”; ...
Branches Disjuncts
Case 1a
!a b
Case 2 !a !b !c d
Case 3!a !b c
!a !b !c !d
Definition of Additional Coverage Goals : Marked Paths
post if(a || b || c) {
Branches Marked Paths Disjuncts
Case 1
“a holds”; “Case 1” a
“Case 1”!a b
!a !b c
branch “Case 1”; ... }
if(a) mark “a holds”;
Test Implementation
Test Case 1--------------------------------------
Formal Specificationspre ---------------------------post ---------------------------
-------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- ---------------------------------------------- --------------------------------------------------
---- -------------------------------------
Test Program
Test Scenario
Co-verification Support
Formal Specifications
pre ---------------------------post --------------------------- -------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- --------------------------------------------- -------------------------------------------------- ---- -------------------------------------
Testable Specifications
pre ---------------------------post ---------------------------
-------------------------------------------------- ---------------------------------------------- ------------ ------------------------------------- ---------------------------------------------- ---------------------------------------------- --------------------------------------------------
---- -------------------------------------
Software --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- --------------------------------------------------- ---------------------------------------------------
UniTesK supports this transformations with techniques and tools
Test Scenario
Engineering Problems (3)
• Test construction technique should ensure coverage goals achievement
• Test designer should be able to introduce additional tests
• Tests should be decoupled with implementation
Overview of UniTesK Approach to Test Implementation
• Test construction technique : traversal of FSM– FSM is constructed in so far that its traversal ensures
coverage
• FSM represented implicitly as test scenario– Implicit representation saves a lot of work
– Test scenarios can be• Generated on the base of templates and specifications
• Developed manually
• Developed in mixed way
• Mediators are used to decouple test scenarios and implementation– The same three ways to develop mediators
Uniform Test ArchitectureUniform
Specification Extension
Integration with Development Environments
CT
esK
J@T
.N@
T
VD
M+
+T
esK
UniTesK
Model Based Testing
UniTesK Test Architecture
• From specification we can generate oracle to check the conformance of system behavior
• The entire test is constructed as a test sequence intended to achieve some coverage– Test sequence required is produced on-the-fly
during test execution as a transition tour of FSM model
Test sequence construction
UniTesK Test Architecture
Oracle
Target system
Test Engine
Test Action Iterator
Specification
Test Sequence Construction
Test Engine
Test Action Iterator
Describes
Executes
Engineering Problems: FSM Construction (4)
• Implicit specifications are hard to resolve• Nondeterminism• Huge numbers of states and transitions
Factorization Based on Coverage Goals
I. B. Bourdonov, A. S. Kossatchev, V. V. Kuliamin. Using Finite State Machines in Program Testing. Programming and Computer Software, Vol. 26, No. 2, 2000, pp. 61-73
Helps to cope with state explosion and nondeterminism
Implicit State Machine Description
Allows not to resolve implicit constraintsMakes factorization easier
State Machine Construction Based on Coverage Goals
states
parametersoperation domain
defined by precondition
1
2
3coverage goals
resulting implicit FSM description
Test Sequence Construction for Implicit FSM
Test Engine
Test Action Iterator
Executes and obtains the complete structure
Test Scenario
Test sequence construction
Oracle
Target system
Test Engine
Test Action Iterator
Specification
Test Scenario
Using Nondeterministic FSMs : Bounded Queue
addEmpty
Single element
Intermediate
Almost full
Fullremove
Empty
Single element
Intermediate
Almost full
Full
fill
empty
Testing Concurrency
• Fair Concurrency Axiom :Concurrent execution of several operations is correct if and only if it is equivalent to somehow ordered one
• So, to test the concurrent execution of calls we should check that it behaves like some sequence of the same calls– The system can be modeled as an ordinary FSM– Test engine generates pairs, triples, and so on, of
concurrent actions
• System not satisfying Fair Concurrency Axiom should be modeled as asynchronous FSM
Possible behavior:input: abaouput: yyx xxyxyyyx xyyyxyall possibleoutputs: { x*yx*yy*x, x*yx*yy*xx*yy*x, x*yy*xx*yy*x }
Asynchronous FSM
• Some transitions are fired by a stimulus• Some transitions are fired along with a reaction• Some transitions are empty, modeling internal operation
a/
/x /y
b/a/
/y/x
b/
Co-verification Support : Changes
Formal Specifications
pre ---------------------post --------------------- ----------------------------------------- ------------------------------------- ------------ ---------------------------- ------------------------------------
Testable Specifications
pre ---------------------post --------------------- ----------------------------------------- ------------------------------------- ------------ ---------------------------- ------------------------------------- ----------------------------------
Prototype ------------------------------------------- ------------------------------------------- ------------------------------------------- -----------------------------------------
Test Scenario
Software ------------------------------------------- ------------------------------------------- ------------------------------------------- -----------------------------------------
Interface changed
???
Engineering Problems (5)
• Tests and implementation should be decoupled– Tests can be reused many times– Easy regression testing
• Specification should be as abstract as possible
• Specification should be even more reusable than tests
Test sequence construction
Oracle
Mediator
Mediator decouples specification and implementation
Mediator
Target system
Mediator Construction
Mediator
Target system
Oracle
Mediatorin Extended Language
Specification
Test Engine
Test Action Iterator
Test Scenario
Open State Testing
• Mediator constructs new model state only on the base of new implementation state
• Requires: Implementation state should be accessible by public fields or reliable observers
• Implies: Current model state actually corresponds to implementation one– Detected failure demonstrates a fault in the last operation– Transition tour is sufficient to ensure reliability
MediatorTarget system
call callresponse
get state data
construct new model state
state data
Hidden State Testing
• Mediator constructs new model state on the base of old one and implementation call results
• Implies: Current model state is actually a hypothesis– Detected failure can be manifestation of a fault in any
previously called operation
– Transition tour is insufficient to ensure reliability
MediatorTarget system
call callresponse
construct new model state
UniTesK Practice Overview
• UniTesK tools
• UniTesK history
• Training courses
• QUTILUI project
• Other case studies
UniTesK Tools
• J@T
• CTesK
• VDM++TesK
• .N@T
Uniform Test ArchitectureUniform
Specification Extension
Integration with Development Environments
CT
esK
J@T
.N@
T
VD
M+
+T
esK
UniTesK
Model Based Testing
UniTesK History
pre-UniTesK• 1994 – 1996
ISP RAS – Nortel Networks contract onfunctional test suite development for Switch Operating System kernel– Hundreds of bugs found in the OS kernel, which had been
10 years in use
• About 600K lines of Nortel code tested by 2000But failed to be introduced in Nortel processes
UniTesK Tools History
• 2000– Conception
• 2001– J@T Prototype– CTesK Lite
• 2002– VDM++ TesK– J@T Product
• 2003– J@T 1.2– CTesK Full– .N@T
postcondition
class SqrtSpecification{ specification static double sqrt(double x) reads x { pre { return x >= 0; } post { if(x == 0) { branch "Zero argument"; return sqrt == 0; } else { branch "Positive argument"; return sqrt >= 0 && Math.abs((sqrt*sqrt-x)/x < epsilon; } } }}
specification double SQRT(double x) reads (double)x{ pre { return x >= 0.; } coverage ZP { if(x == 0.) return (ZERO, "Zero argument"); else return (POS, "Positive argument"); } post { if(coverage(ZP, ZERO)) return SQRT == 0.; else return SQRT >= 0. && abs((SQRT*SQRT - x)/x) < epsilon; } }}
operation signature declaration
preconditionaccess constraints
Sqrt Specification in J@va and Extended C
functional branches definition
Training Courses
CTesK training was conducted for Tercom (Russia)
J@T training was conducted for Systematic Software Engineering (Denmark),Saarland University (Germany)
QUTILUI Project
• GoalRedesign of SOS Queue Manipulation Utilities Subsystem
• Activities– Requirements elicitation
– Specification development
– Specification review
– Architecture design
– Component design
– Implementation
– Testing and debugging
QUTILUI Project Results
• Bugs removed– No bugs found after the project end!
• 25% decrease of implementation size
• Design documentation appeared– And it became actual!
• Tests demonstrated conformance with requirements
--------------------
- - - - - - -- - - - - - -- - - - - - -- - - - - - -
Other Examples of UniTesK Usage
• IPv6 implementations– Microsoft Research– Mobile IPv6 (in Windows CE 4.1)
• mpC Workshop– Debug API– mpC expression static and dynamic semantics
• Optimization units in Intel compilers• UniTesK tools
– J@T test system runtime support– Components of J@va translator– CTesK abstract types library
• Lanit-Tercom– IPv6 implementation– DSP software
Appendices
• References slide 66• Detailed comparison of specification techniques
slide 67-69• Priority Queue Example
– Description slide 70– Specifications slides 71-79– Functional branches slides 80-82– Additional coverage goals slides 83-86– FSM construction slides 87-91– Test scenario slides 92-96– Mediator and its usage slides 97-101
References
1. V. Kuliamin, A. Petrenko, I. Bourdonov, A. Kossatchev. UniTesK Test Suite Architecture // Proceedings of FME’2002 conference, Copenhagen, Denmark, LNCS, No. 2391, 2002, pp. 77-88.
2. V. Kuliamin, A. Petrenko, I. Burdonov, A. Demakov, A. Jarov, A. Kossatchev, S. Zelenov. J@va : extension of Java for real-life specification and testing // Proc. of Andrei Ershov Fourth International Conference PCI’01, Novosibirsk, LNCS, Vol. 2244, 2001, pp.301-308.
3. A. Petrenko, V. Kuliamin, I. Bourdonov, A. Kossatchev. Experiences in using testing tools and technology in real-life applications // Proceedings of SETT’01, India, Pune, 2001
4. A. Petrenko. Specification Based Testing: Towards Practice // Proceedings of PSI’01, Novosibirsk, LNCS 2244, 2001, pp.157-162.
5. A. Petrenko, A. Vorobiev. Industrial Experience in Using Formal Methods for Software Development in Nortel Networks // Proceedings of Testing Computer Software Conference TCS 2000, Washington, June, 2000.
6. I. Bourdonov, A. Kossatchev, V. Kuliamin. Using Finite State Machines in Program Testing // Programming and Computer Software, Vol. 26, No. 2, 2000, pp. 61-73.
7. I. Bourdonov, A. Kossatchev, A. Petrenko, and D. Galter. KVEST: Automated Generation of Test Suites from Formal Specifications // Proceedings of World Congress of Formal Methods, Toulouse, France, LNCS, No. 1708, 1999, pp. 608-621
Executable Specifications
• Are very close to some implementation – Are easy to use in the industry– Can be transformed into prototypes
• Are not close to requirements( √¯ = e½ln = lim(xn+1 = ½(xn +x/xn)) )– Unsuitable for test coverage measurement– Can cause problems with conformance checking
How to compare the results?
• Are highly reusable as executable code• But are low reusable as criteria of correctness
Constraint Specifications
• Have the structure similar with implementation– Are easy to use in the industry
• But have different form• Are close to requirements in most cases
– Are easy to construct from requirements
– Suitable for test coverage measurement
– Counterexample: memory management subsystem
• Can be directly used in conformance checking• Special constructs enabling reuse can be added
Axiomatic Specifications
• Are far from common implementations and have greatly different structure– Can hardly be introduced in the industry
• Are usually far from requirements – Are hard to develop in real-life projects– Can hardly be used for coverage measurement– But sometimes are the only appropriate form
• Can be used for conformance checking• But sharpen error localization problems• Reusability is a problem
• void enq (Object obj, int priority)priority [Min_Priority, Max_Priority]
• Object deq ()
• int size ()
Priority Queue Example
5
deq () enq ()
size ()
5 4 3 1 1
enq ()enq ()
05 2
Model State Definition
public class PQueueSpecification
{
// List of elements of the queue
public Vector items = new Vector();
// Accompanying list of their priorities
public IntList priorities = new IntList();
}
Data Integrity Constraints : Lists are not Null
public class PQueueSpecification{ public List items = new List(); public IntList priorities = new IntList();
invariant ListsAreNotNull() { return items != null && priorities != null; }}
Data Integrity Constraints : Lists’ Sizes are Equal
public class PQueueSpecification{ invariant ListsSizesAreEqual() { return items.size() == priorities.size(); }}
Data Integrity Constraints : Priorities Lie in the Range
public class PQueueSpecification{ public static int Min_Priority; public static int Max_Priority;
invariant PrioritiesLieInTheRange() { for(int i = 0; i < priorities.size(); i++) { if( priorities.get(i) < Min_Priority || priorities.get(i) > Min_Priority ) return false; } return true; }}
Data Integrity Constraints : Priorities do not Increase
public class PQueueSpecification{ invariant PrioritiesDoNotIncrease() { for(int i = 1; i < priorities.size(); i++) { if( priorities.get(i-1) < priorities.get(i) ) return false; }
return true; }}
Operation Specification : size() Method
public class PQueueSpecification
{
specification public int size ()
reads this.?
{
pre { return true; }
post
{
return size == items.size();
}
}
}
Operation Specification : enq() Method
public class PQueueSpecification{ specification public void enq (Object obj, int prty) reads obj, prty updates this.? { pre { return obj != null; } post { int i = 0; for(i = 0; i < items.size() && priorities.get(i) >= prty; i++ );
PQueueSpecification new_state = (PQueueSpecification)@clone(); new_state.items.add(i, obj); new_state.priorities.add(i, prty); return this.equals(new_state); } }}
Auxiliary Methods
public class PQueueSpecification{ public boolean equals (PQueueSpecification other) { return items.equals(other.items) && priorities.equals(other.priorities); } public Object clone () { PQueueSpecification result = new PQueueSpecification(); result.items = (List)items.clone(); result.priorities = (IntList)priorities.clone(); return result; }}
Operation Specification : deq() Method
public class PQueueSpecification
{
specification public Object deq ()
updates this.?
{
post
{
if(items.isEmpty())
return deq == null && items.isEmpty();
else
{
PQueueSpecification new_state = (PQueueSpecification)@clone();
Object result = new_state.items.first();
new_state.items.removeFirst();
new_state.priorities.removeFirst();
return this.equals(new_state) && deq == result;
}
}
}
}
public class PQueueSpecification
{
specification public int size ()
reads this.?
{
pre { return true; }
post
{
branch “Single branch”;
return size == items.size();
}
}
}
Functionality Branches in size() Method
specification public void enq (Object obj, int prty) reads obj, prty updates this.?{ pre { return obj != null; } post { int i = 0; for(i = 0; i < items.size() && priorities.get(i) >= prty; i++ );
branch “Single branch”;
PQueueSpecification new_state = (PQueueSpecification)@clone(); new_state.items.add(i, obj); new_state.priorities.add(i, prty); return this.equals(new_state); }}
Functionality Branches in enq() Method
specification public Object deq ()
updates this.?
{
post
{
if(items.isEmpty())
{
branch “Empty queue”;
return deq == null && items.isEmpty();
}
else
{
branch “Nonempty queue”;
PQueueSpecification new_state = (PQueueSpecification)@clone();
Object result = new_state.items.firstElement();
new_state.items.removeFirstElement();
new_state.priorities.removeFirstElement();
return this.equals(new_state) && deq == result;
}
}
}
Functionality Branches in deq() Method
Additional Coverage Tasks : enq()
5 5 4 3 1 1
05 27 4 1
1 1
1
1
Additional Coverage Tasks : deq()
5 5 4 3 1 1
5 4 3 1
Marked Paths in enq() Method
post{ int i = 0; for(i = 0; i < items.size() && priorities.get(i) >= prty; i++ );
if( items.isEmpty() ) mark "Insertion in the empty queue"; else if( prty > priorities.maximum() ) mark "Insertion in the head"; else if( prty == priorities.maximum() && prty == priorities.minimum() ) mark "Insertion with single existing priority"; else if( prty == priorities.maximum() ) mark "Insertion with maximum priority"; else if( prty < priorities.minimum() ) mark "Insertion in the tail with new priority"; else if( prty == priorities.minimum() ) mark "Insertion with minimum priority"; else if( !priorities.contains(prty) ) mark "Insertion in the middle with new priority"; else mark "Insertion in the middle with existing priority";
branch “Single branch”; ...}
post
{
if(items.isEmpty())
{
branch “Empty queue”;
...
}
else
{
if( items.size() == 1 )
mark "Single element in the queue";
else if( !(priorities.maximum() == priorities.get(1)) )
mark "Single element with maximum priority, several elements in the queue";
else if( priorities.toSet().size() > 1 )
mark "Several elements with maximum priority, there are other priorities";
else
mark "Several elements in the queue with the same priority";
branch “Nonempty queue”;
...
}
}
Marked Paths in deq() Method
Priority Queue Example : enq() Coverage Goals
if( items.isEmpty() ) mark "Insertion in the empty queue";else if( prty > priorities.maximum() ) mark "Insertion in the head";else if( prty == priorities.maximum() && prty == priorities.minimum() ) mark "Insertion with single existing priority";else if( prty == priorities.maximum() ) mark "Insertion with maximum priority";else if( prty < priorities.minimum() ) mark "Insertion in the tail with new priority";else if( prty == priorities.minimum() ) mark "Insertion with minimum priority";else if( !priorities.contains(prty) ) mark "Insertion in the middle with new priority";else mark "Insertion in the middle with existing priority";
States:
Arguments:
Empty queueSingle priority is usedAt least two priorities are usedAt least three priorities are used
At least three different priorities should be provided
Priority Queue Example : deq() Coverage Goals
if(items.isEmpty())
branch “Empty queue”;
else
if( items.size() == 1 )
mark "Single element in the queue";
else if( !(priorities.maximum() == priorities.get(1)) )
mark "Single element with maximum priority, several elements in the queue";
else if( priorities.toSet().size() > 1 )
mark "Several elements with maximum priority, there are other priorities";
else
mark "Several elements in the queue with the same priority";
States: Empty queueSingle element in the queueSingle element with maximum priority, several prioritiesSeveral elements with maximum priority, several priorities
Several elements with the single used priority
Priority Queue Example : Analysis of Determinism
0 : Empty queue
1 : Single element in the queue
2 : Several elements with single used priority
0 1 2
deq()
To make the behavior deterministic we should split this state according to the number of elements
Priority Queue Example : Analysis of Determinism
0,1,2, … : Number of elements with maximum priority
A : Single element with maximum priority, several priorities
0 1 2 3
deq()
To make the behavior deterministic we should split this state according to the number of elements with the next priority
A
…
And so on, for all priorities (by induction)
Priority Queue Example : Resulting State
Resulting state S : int → int ismap from integers to integers, wherekey is priorityvalue is the number of queue elements having such the priority
Reference to Specification Object
scenario public class PQueueTestScenario
{
// Reference to the object under test
public PQueueSpecification queue;
}
State Construction Method
scenario public class PQueueTestScenario{ public AbstractGenState getGenState() { IntToIntMapGenState state = new IntToIntMapGenState(); for(int p = PQueueSpecification.Min_Priority; p < queue.priorities.toSet().size(); p++ ) { int n = queue.priorities.numberOf(p);
if( n != 0 ) state.put(p, n); } return state; }}
Definition of Test Actions : size() method
scenario public class PQueueTestScenario{ scenario public boolean size() { queue.size(); return true; }}
Definition of Test Actions : enq() method
scenario public class PQueueTestScenario
{
scenario public boolean enq()
{
Object arg = new Object();
iterate( int priority = PQueueSpecification.Min_Priority;
priority <= PQueueSpecification.Max_Priority;
priority++;
queue.priorities.toSet().size() <= 3
|| queue.priorities.contains(priority)
&& queue.priorities.numberOf(priority) < 2
)
{
queue.enq(param, priority);
}
return true;
}
}
Definition of Test Actions : deq() method
scenario public class PQueueTestScenario{ scenario public boolean deq() { queue.deq(); return true; }}
Mediator for Priority Queue
mediator public class PQueueMediator // Extends specification class extends PQueueSpecification{ // Reference to an implementation object implementation public PriorityQueue target;}
Mediator Methods (Open State Testing)
mediator public class PQueueMediator{ public int size() { return target.size(); } public void enq(Object obj, int prty) { target.enq(obj, prty); } public Object deq() { return target.deq(); }}
Model State Synchronization Method (Open State Testing)
mediator public class PQueueMediator{ public void mapStateUp() { items.clear(); priorities.clear(); if( target != null ) { for(int i = target._clusters.size()- 1; i >= 0; i--) { Vector v = (Vector)target._clusters.elementAt(i); for(int j = 0; j < v.size(); j++) { items.add(v.elementAt(j)); priorities.add(i); } } } }}
Mediator Methods (Hidden State Testing)
mediator public class PQueueMediator{ public int size() { return target.size(); } public void enq(Object obj, int prty) { target.enq(obj, prty);
items.add(obj); priorities.add(prty); } public Object deq() { Object result = target.deq(); items.removeFirst(); priorities.removeFirst();
return result; }}
Test Scenario Initialization
scenario public class PQueueTestScenario
{
public PQueueTestScenario()
{
setTestEngine ( new DeterministicFSM_TestEngine() );
PQueueMediator.initClassState();
queue = PQueueMediator.create(new PriorityQueue());
queue.attachOracle();
}
}