The State of Hybrid Model-Based Testing Michiel van Osch 21-06-2005

The State of Hybrid Model-Based Testing

Michiel van Osch

21-06-2005

2 2

Index

• Introduction• Testing a Water Tank Controller• Hybrid Model-Based Testing

– Conformance Relation– Test-case generation

• Work in Progress• Questions

3 3

Model-Based Testing

DiscreteModel

Test-tool DiscreteSystem

pass

!“red”!“red”

?“stop traffic” ?“stop traffic”

4 4

Hybrid Systems

“brake”

Hydrolicpressure

“1”, “2”, “1”

Coffee withMilk and Sugar

Room Temperature

Desired Temperature: 25o

5 5

Hybrid Model-based Testing

Hybrid-ModelTest-tool

Hybrid System

Heater OFF Heater ON

SP £ S

SP > ST

T

?Set_setpoint(sp)

!Print(“warning”)

?Set_setpoint(sp)

pass

Lenstemperature

Lenstemperature

!”openCoolingValve”

!”openCoolingValve”

6 6

Challenges

• Theory: – Develop Conformance Concepts– Develop Test-case Generation Algorithms– Prove Correctness and Applicability of Theory

• Practice: – Implement Tooling– Exactly Model Relevant Aspects– Connect Models, Tools and Systems

7 7

Index




8 8

The water Tank Controller

Tank Controller

• Discrete input: Set point

• Discrete output: communication

• Continuous input: Temperature

(from sensor)

• Continuous output: Heat

(from Heater)

WaterTank

Tank Controller

TemperatureSensor

Heater

set point

messages

9 9

Specification (1)

S

Set_setpoint(sp)

T

Tank Controller

H

Print(“Warning”)

Hybrid I/O automaton

Tank Controller

• Discrete input: Set_setpoint(sp)

• Discrete output: Print(“warning”)

• Continuous input: ST

• Continuous output: H

10 10

Specification (2)



SP £ S

SP > ST

T

?Set_setpoint(sp)


?Set_setpoint(sp)

H

H

ST

ST

input output input output

11 11

Specification Test-tool Implementation

S

Set_setpoint(sp)

T

Tank Controller

H


Test Setup

Tank Controller

TemperatureSensor Heater

Set_Setpoint(sp) Set-point

ST Temp. Sens.

Print(..) Message

HeatH

12 12


SP £ S

SP > ST

T

?Set_setpoint(sp)


?Set_setpoint(sp)

Specification

Test-tool

Implementation

S

Set_setpoint(sp)

T

Tank Controller

H


Testing...

Tank Controller


ST

H Heat

Temp. Sens.

Specification Implementation

13 13

Testing…


SP £ S

SP > ST

T

?Set_setpoint(sp)


?Set_setpoint(sp)

Test-tool

Set_setpoint(SP) Set-point

Specification

S

Set_setpoint(sp)

T

Tank Controller

H


Implementation

Tank Controller


ST

H Heat

Temp. Sens.


14 14

ST

H Heat

Temp. Sens.

Testing…

Test-tool


SP £ S

SP > ST

T

?Set_setpoint(sp)


?Set_setpoint(sp)

Specification

S

Set_setpoint(sp)

T

Tank Controller

H


Implementation

Tank Controller



15 15

Testing


SP £ S

SP > ST

T

?Set_setpoint(sp)


?Set_setpoint(sp)

Test-toolPrint(“warning”) “error”

Specification

S

Set_setpoint(sp)

T

Tank Controller

H


Implementation

Tank Controller


ST

H Heat

Temp. Sens.


fail

16 16

Testing…

Test-tool

Specification

S

Set_setpoint(sp)

T

Tank Controller

H


Implementation

Tank Controller


fail


SP £ S

SP > ST

T

?Set_setpoint(sp)


?Set_setpoint(sp)

ST

H Heat

Temp. Sens.


17 17

Index




18 18

Conformance

A hybrid implementation is input-output conform a hybrid specification if for all observable traces of the specification:

The set of possible output actions and trajectories,

after the execution of a trace on the implementation,

is a subset of

the set of possible output actions or trajectories,

after the execution of a trace on the specification.

19 19

Some Decisions

• Defined on Hybrid I/O Automata– Input Action Enabled– Input Trajectory Enabled– Non-zeno

• State Based: possible output actions and trajectories in a certain state

• Input and output trajectories are treated together• There is no quiescence observation (as of yet)

20 20

Conformance relation

We Formally Define:

• A hybrid trace = 0 a1 1 a2 … as a sequence of discrete actions and trajectories of continuous variables (alternating)

• A after : the set of states in which A can be after executing trace

• out(q): the set of output actions or trajectories possible in state q• Out(Q): the set of output actions or trajectories possible in the set

of states Q

Then implementation HIOA i is hioco specification HIOA s if:

i hioco s if traces(s): out(i after ) out(s after )

21 21

Examples

!a

!b

!a

!b

!cCorrect!

implementationspecification

implementation

qq

specification

Correct!

22 22

Examples

!a

!b

!a

!b

Incorrect!

implementationspecification

!c

implementation

q

specification

Incorrect! q

23 23

Test-Case generation & Execution

• Input actions: Select and stimulate• Output actions: Observe and validate• Trajectories:

1. Select a time up to which to apply input 2. Select an input trajectory to use for stimulation3. Stimulate and observe output variables :

– Until the end of the trajectory, without observing a mismatch between observed and specified output trajectories, in this case continue

– Until an output action is observed, in this case validate the output action and if correct, then continue

– Until a mismatch between observed and specified output variables occurs

• As long as verdict “fail” is given, the verdict “pass” can be given

24 24

Test-case generation

fail

failpass

pass fail

?Set_setpoint(SP)

!Print(“warning”) !Print(“error”)

{SP > ST}

25 25

Index




26 26

Work in Progress (1)

• Does a continuous variable always have a value?

• State Changes During a Trajectory

“1”,”2”

Coffee

temperature

When no coffee comes

out of the machine, temperature

Is not continuously 0!

While applying an input trajectory

and observing the output,

States change

q q’

27 27

Work in Progress (2)

• Quiescence– Is quiescence a property on the discrete actions of the system

or also on continuous behavior of the system?

• Notation

What does this mean?

28 28

Current & Future Work

• Further Develop these Concepts (Formalization, Theorems, Proofs)

• Make Concepts Workable in Practice– Sampling of continuous behavior– Selecting Realistic Input Trajectories– Deal with Uncountable Many Possible Input and Output

Trajectories and States– Deal with Inaccuracy of Observed Output

• Implement Proof of Concept

29 29

Questions

Documents

The State of Hybrid Model-Based Testing Michiel van Osch 21-06-2005