Upload
phillip-maximillian-pearson
View
223
Download
0
Embed Size (px)
Citation preview
VDMUnit Testing, Combinatorial Testing, Model Quality and Code Generation
Professor Peter Gorm LarsenDepartment of Engineering, Aarhus University
([email protected]) and
PhD student Peter Jørgensen ([email protected])
2
Agenda
The VDMUnit Testing Classes
• The Combinatorial Testing Principles
• Internal Consistency
• External Consistency
• Code Generation
• Project Proposals
3
The VDMUnit Framework
4
The TestSuite Class
class TestSuite is subclass of Test
instance variables tests : seq of Test := [];
operations public Run: () ==> () Run () == (dcl ntr : TestResult := new TestResult(); Run(ntr); ntr.Show()); public Run: TestResult ==> () Run (result) == for test in tests do test.Run(result);
public AddTest: Test ==> () AddTest(test) == tests := tests ^ [test];
end TestSuite
The TestCase Class
public Run: TestResult ==> ()Run (ptr) == trap <FAILURE> with ptr.AddFailure(self) in (SetUp(); RunTest(); TearDown()); protected SetUp: () ==> ()SetUp () == is subclass responsibility;protected RunTest: () ==> ()RunTest () == is subclass responsibility;protected TearDown: () ==> ()TearDown () == is subclass responsibility
end TestCase
class TestCase is subclass of Test
instance variables name : seq of char
operations
public TestCase: seq of char ==> TestCaseTestCase(nm) == name := nm;
public GetName: () ==> seq of charGetName () == return name;
protected AssertTrue: bool ==> ()AssertTrue (pb) == if not pb then exit <FAILURE>;
protected AssertFalse: bool ==> ()AssertFalse (pb) == if pb then exit <FAILURE>;
6
The TestResult Class
class TestResult
instance variables failures : seq of TestCase := [] operations public AddFailure: TestCase ==> () AddFailure (ptst) == failures := failures ^ [ptst];
public Print: seq of char ==> () Print (pstr) == def - = new IO().echo(pstr ^ "\n") in skip; public Show: () ==> () Show () == if failures = [] then Print ("No failures detected") else for failure in failures do Print (failure.GetName() ^ " failed") end TestResult
7
Enigma Unit Tests
8
Agenda
The VDMUnit Testing Classes
The Combinatorial Testing Principles
• Internal Consistency
• External Consistency
• Code Generation
• Project Proposals
9 9
Combinatorial Testing Perspective
Regular expression
Overview of results
Detailed test case and results
10
Combinatorial Testing Overview
VDM source Parser
ASTPattern
tree
Fully expanded
tests
Shape reduced
tests
Filtered test
results
T: let x in set {1, ..., 100} in obj.op(x){1,3};
11
Test case execution
Test cases Test engine
Check
if
filt
ere
d b
y o
ther
test
case
s
Failed
Inconclusive
Passed
Filtered
Test results
Filter out test cases with a prefix resulting in a test failure
12
operationspublic insert : int ==> ()insert(val)== skippre val > 1;
tracesT1 : let x in set {1,2} in insert(x);
13
class Avaluesobj : A = new A();
operationspublic op : nat ==> natop (x) == return x;
tracesT2: let x,y in set {1, ..., 10}in (obj.op(x);obj.op(y));
end A
14
Regular Expressions in Traces
• Let definitions (local naming)• Let be such that definitions (all selection)• Repeat traces (fixed or variable number of times)• Concurrency traces• Application expression (calling operations)
15
Agenda
The VDMUnit Testing Classes
The Combinatorial Testing Principles
Internal Consistency
• External Consistency
• Code Generation
• Project Proposals
16
Introduction
• What is the value of the models you have produced?
• How do we assess the quality of a model?• Internal consistency:
• Does the model describe something?• Syntax, type checking and proof obligations• No potential run-time errors
• External consistency: • Does the model describe the right thing?• Validation with domain expert• Does the model have desirable properties?
17
POP3: Protection of Partial Operatorsclass POP3Server...instance variables maildrop : MailDrop; ...
types
public MailDrop = map POP3Types`UserName to MailBox;public MailBox :: msgs : seq of POP3Message locked : bool;
operations
GetUserMessages: POP3Types`UserName ==> seq of POP3MessageGetUserMessages(user) == return GetUserMail(user).msgspre UserKnown(user);
end POP3Server
18
Booking of Flights: Invariant Preservationclass Trip
typesFlight :: departure : seq of char destination : seq of char
instance variables journey: seq of Flight;
inv forall i in set {1,...,len journey -1} & journey(i).destination = journey(i+1).departure
operations
AddFlight: Flight ==> ()AddFlight(f) == journey := journey ^ [f]pre journey(len journey).destination = f.departure
end Trip
journey <> [] =>
19
Robot Routes: Satisfiability 1
class Routeinstance variables
points: set of Point;
inv forall p1, p2 in set points & p1.GetCoord() = p2.GetCoord() => p1 = p2 and forall p in set points & p.GetIndex() <> card points => GetNext(p).GetCoord() in set {n.GetCoord() | n in set p.Neighbour()}…end Route
A pre/post condition specification is satisfiable, if, for all states and inputs satisfying the precondition, there exists some output and “after” state satisfying the postcondition and any relevant invariants.
20
Robot Routes: Satisfiability 2
class Routefunctions
staticpublic AvoidanceRoutes( obstacles: set of (nat * nat), currentPosition: Point, nextWaypoint: Point) routes: set of Routepost forall r in set routes & r.GetFirst().GetCoord() = currentPosition.GetCoord() and r.GetLast().GetCoord() = nextWaypoint.GetCoord() and r.GetCoords() inter obstacles = {};
end Route
21
Robot Routes: Satisfiability 3
• For implicit definitions there must exist at least one potential result for each input satisfying the pre-condition
Proof Obligation (or integrity constraint):
forall obstacles: set of (nat * nat), currentPosition: Point, nextWaypoint: Point & exists routes: set of Route & post-AvoidanceRoutes(obstances,currentPosition, nextWaypoint,routes)
Can in principle be proved formally
22
Agenda
The VDMUnit Testing Classes
The Combinatorial Testing Principles
Internal Consistency
External Consistency
• Code Generation
• Project Proposals
23
Dialogue with Domain Experts
• Typically domain experts know little about IT• Understanding their intended usage may be a
challenge• Creating a model will create further questions to
experts• Model should seldom be shown directly• Scenarios to be used for test purposes can
typically be discussed• A Java-based API can be used to ”demonstrate”
ideas to domain experts/end users
24
The External Interfaces of Overture
• External aid is needed to support communication.
• Overture supports two different types of external interfacing• External Call Interface
• Only calls from the VDM model to an external interface
• Remote Control Interface• Allows for external calls into a
VDM Model
25
External Call Interface• Enables call to native Java code from inside a model• A type of delegate scheme is created between the
model and an external jar• In the model an operation is created which contains
an "is not yet specified" statement
public receivedMessage: int ==> () receivedMessage(vecID) == is not yet specified;
• When a non-specified statement is encountered, the interpreter will considered it an delegate, an attempt to look it up in Java CLASSPATH.
26
External Call Interface• The Java class must have the same name as the module
or class. • Package separators are denoted with underscores in the
model.VDM:class gui_GraphicsJava:package gui;class Graphics
• If a Java class and method is found, a delegate object is created an bound to the delegate function.
• MATH, IO and VDMUtil standard library functions in Overture are implemented using the same delegate scheme
27
External Call Interface Example : VDM
class gui_Graphics public receivedMessage : int ==> () receivedMessage(vecID) == is not yet specified;end gui_Graphics
28
External Call Interface Example : Javapackage gui;
import org.overture.interpreter.runtime.ValueException;import org.overture.interpreter.values.Value;import org.overture.interpreter.values.VoidValue;
public class Graphics implements Serializable { public Value receivedMessage(Value vecID) throws ValueException { model.receivedMessage(vecID.intValue(null)); return new VoidValue(); }}
29
External Call Interface Example : Java IDE• In order to use VDM Types the Java program must
have the Overture java library in its build path.• ast.jar, parser.jar and interpreter.jar must be
available on the built path.• These jar files are provided with Overture
• <Overturedir>\Plugins\• Once the Java program is finished, it must be exported to a
Jar file and placed in the lib directory in the Overture/VDM projects directory, in order for Overture to find it.
30
External Call Interface Example
31
Remote Control Interface• Allows for a VDM model to be controlled by an
external Java programs• Essentially a console like access to the Overture
interpreter which can be bootstrapped through the Overture Debugger
• A RemoteControl interface is defined, which must be implemented by the external programs.public interface RemoteControl{
public void run(RemoteInterpreter interpreter) throws Exception;
}
32
RemoteInterpreter• RemoteInterpreter is passed from Overture to the
external programs • It has four central methods
• Void Init()• Re-initializes the interpreter
• Void Create(String, String)• Takes a variable name as the first argument
and expression as the second. • String Execute(String)
• Takes expression as string argument, returns a string value.
• Value ExecuteValue(String)• Takes expression as string argument, returns a
generic VDM Value represented in Java
33
Example model to use
class A
operations
public op: int ==> intop(n) ==
return n + 1pre n > 0
end A
34
Use of the RemoteInterpreter
interpreter.init()
interpreter.create(("a", "new A()"); //create a := new A();
Value result = interpreter.valueExecute("a.op(5)");
System.out.println(result.intValue(null)); //prints 6
35
Use of the Remote Control Interface• Using the Remote Control Interface to create an
interactive GUI for a model
• Connecting a GUI to the KLV /CSLaM model• Create a GUI• Implement RemoteControl interface• Bind GUI with the RemoteInterpreter through the
RemoteControl.
36
Remote Control Interface Example (RemoteControl)import org.overture.interpreter.debug.RemoteControl;import org.overture.interpreter.debug.RemoteInterpreter
public class KlvRemote implements RemoteControl{@Overridepublic void run(RemoteInterpreter intrprtr){
Thread remoteThread = new Thread(new Runnable(){ public void run(){ KLVgui gui = new KLVgui(); KLVgui.vdmklv = new KlvOvertureComm(intrprtr, gui); gui.initialising(); }); remoteThread.setDaemon(true); remoteThread.start(); intrprtr.processRemoteCalls();}
37
Remote Control Interface Example (KlvOvertureComm)public int getMaxSpeed() { try{
Value g = execute("klv.getMaxSpeed()");return new Long(g.intValue(null)).intValue();
} catch (Exception err) {printError("API error: " + err.getMessage());}
return 0;}
private Value execute(String arguments) throws Exception{System.out.println("Calling Overture with: " + cmd);Value result = interpreter.valueExecute(cmd);
return result; }
38
Setting up the Remote Control Interface• External Java class must be configured in the Overture
Debug Configurations
Validating KLV using the API
40
Agenda
The VDMUnit Testing Classes
The Combinatorial Testing Principles
Internal Consistency
External Consistency
Code Generation
• Project Proposals
41
Overture Code Generation• A VDM++ to Java Code Generator • Available in Overture releases 2.0.6 onwards• Early work: VDM++ to C++ Code Generator
42
Code generation challenges (Java)
VDM:public op : () ==> natop () == ( dcl v1 : Vector2D := mk_Vector2D(1,2); dcl v2 : Vector2D := v1; v1.x := 2; return v2.x;)
Java:public Number op() { Vector2D v1 = new Vector2D(1L, 2L); Vector2D v2 = v1; v1.x = 2L; return v2.x;}
• Code generating VDM++ models to Java• Preserving the semantics in the generated code• Code generation across paradigms
• How do we generate: {x | x in set S & pred(x)}• Or Lambdas, e.g. (lambda x : int & (lambda y : int & x + y))
Java (correct):public Number op() { Vector2D v1 = new Vector2D(1L, 2L); Vector2D v2 = v1.clone(); v1.x = 2L; return v2.x;}
43
Example: Set comprehensions
public f : () -> set of natf () ==let a = {x | x in set S & pred(x)}in g(a,a);
public op : () ==> set of natop () ==( dcl setCompResult : set of nat := {}; for all x in set S do if pred(x) then setCompResult := setCompResult union {x}; (dcl a : set of nat := setCompResult; return g(a,a)));
44
Code Generation Platform
45
Agenda
The VDMUnit Testing Classes
The Combinatorial Testing Principles
Internal Consistency
External Consistency
Code Generation
Project Proposals
46
Project Proposals• R&D projects (5 ECTS) or MSc thesis (30 ECTS)• Validating a code generator for embedded systems
• Code generate for an embedded platform• NXT, Raspberry Pi etc.• Identify and address issues
• Validating the code generation platform architecture• Adding (partial) support for a new language• Identify changes/extensions to the IR• Feedback for the platform architecture
• Code Generating VDM-SL models• Converting a VDM-SL model into a VDM++ model• Without changing the code generator• Implementation of the conversion rules
47
Project Proposals• Code Generating Union Types
• How can union types be represented in (say) Java• Comparing different approaches• Implementing support for union types
• Code Generating VDM-RT• In VDM-RT objects can be deployed on different CPUs• CPUs communicate via buses• Objects message exchange results in bus communication
• Code generation of VDM concurrency concepts• Other projects:
http://wiki.overturetool.org/index.php/Overture_projects#Additional_Potential_Projects_for_.22Technical_IT.22_students_at_Aarhus_University
48
Summary• What have I presented today?
• Introduced VDMUnit for testing• Introduced combinatorial testing principles• Assessing model quality• Internal consistency• External consistency• Code Generation• Project Proposals
• What do you need to do now?• Read chapter 13• Assess your own mini-project VDM model’s consistency• Carry out either the testing exercise or the exercise with a
graphical front-end using java
49
Quote of the day
Bertrand Meyer
Formal specifications may become for software engineers what, say, differential equations are
for engineers of other fields