Embed Size (px)
Citation preview
Simulated Time for Host-Based Testing with TTCN-3
Stefan Blom (Innsbruck Univ.), Thomas Deiss (Nokia Research Center, NRC),
Natalia Ioustinova (CWI), Ari Kontio (NRC),Jaco van de Pol (CWI), Axel Rennoch (FOKUS),
Natalia Sidorova (TU/e)
RoadmapRoadmap
Motivation for host-based testing General structure of a TTCN-3 test system Simulated time for host-based testing Solution for host-based testing in simulated time
with TTCN-3 Applications (transportation, telecom) Conclusion
Systems around us ...Systems around us ...
Embedded Systems
Distributed Systems
Many components/processes/services working in parallel Asynchronous communication Complex behavior driven by data and time limitations Open: in continuous interaction with the outside world Scalability etc.
are complex and often unpredictableare complex and often unpredictable
V&VVerification: are we building the product right? Validation: are we building the right product? (testing) (peer reviews)
Why host-based testing?Why host-based testing?
Target environment: the environment where software is supposed to work
Host environment: the environment used for developing the software
Many of components do not exist until the latest phase of development => they are not always available for integration testing=> testing in the target environment can be expensive and even
dangerous
Find and fix as many errors as possible prior to testing in
target environment => host-based testinghost-based testing
Host-based testing Host-based testing Host environment makes use of
simulated environments (1)(represent missing software/hardware components)
emulation of target OS services (2) monitoring (3) instrumentation (4)
Time constraints are imposed on the system and its environment
Host-based testing Host-based testing Test results are adequate iff the effect of 1-3 on
timed behavior of the system is negligible Simulators/emulators/monitoring/instrumentation
with negligible effect are often expensive and system-specific
In practice: probe effects are often “not negligible” performance constraints (time limits on system) [target] behavioral constraints (time limits on environment)
[host]
Here we focus on host-based testing of behavioral time-dependant features
TTCN-3 Test SystemTTCN-3 Test SystemC
ompo
nent
Han
dlin
g (
CH
)
Test Management (TM)
TRI
Test Logging (TL)
TTCN-3 executable (TE)
Main Test Component
Test Component
Cod
ing
and
Dec
odin
g(C
D)
Platform Adapter(PA)
System Adapter(SA)
TCI-TM TCI-TL
TC
I-C
H
TC
I-C
D
Timers in TTCN-3Timers in TTCN-3
Timer states: deactivated or active Active timer keeps information on time left until its
expiration Timer operations: set, reset, timeout Time progression decreases time left until expiration
of active timers When time left until expiration becomes 0, the timer
expires Expiration results in producing a timeout (message) Timeout is enqueued at the component using the
timer
Simulated TimeSimulated Time Time is modeled by a discrete logical clock We work with systems where external delays are
significantly larger compared to duration of computations within the system
To obtain test results independent of probe effects we assume instrumentation/monitoring are done at zero time message passing, scheduling/synchronization and
other emulated OS services are provided at zero time Actions are instantaneous => time progression has the least priority Time progresses by action tick decreasing the time left
until expiration of active timers by one Periods between two ticks are time slice
IdlenessIdleness
We focus on closed systems (SUT+Test System) consisting of multiple components communicating with each other
Local idleness: a component is idle iff it cannot proceed by performing computations, receiving messages or processing timeouts
Global idleness: a closed system is idle iff all components are idle and there are no messages or timeouts pending
ChallengeChallenge Time semantics in TTCN-3 has been intentionally left
open to enable use of TTCN-3 with different time semantics
Focus has been on using TTCN-3 for real time testing => existing TCI and TRI interfaces provide excellent support
for real time but lack operations necessary for implementing simulated time.
In simulated time all entities of TTCN-3 test system and SUT should agree on simulated time
Time may progress only if the system is idle We need a mechanism that
detects global idleness of the closed system progresses time if idleness is detected
Idleness of Closed SystemIdleness of Closed System
TE1 TEn
PA1 SA1 PAn SAn
CH
TS1 TSn
SUT
All components are idle
No messages pending
Global Idleness of TTCN-3 test system Global Idleness of TTCN-3 test system
ni
EnqdSUTSAiSent..1
All components are idle
)()]()()(:..1[ idleSUTidleSAiidlePAiidleTEini
No messages pending
ni ni
TCIEnqdTEiTCISentTEi..1 ..1
TRIEnqdTEiPAiTRISentSAini
PAiTRIEnqdSAiTRISentTEini
:..1
:..1
from TS to SUT
ni
EnqdSAiSentSUT..1
from SUT to TS
from TSi to TSj
from TEi to PAi & SAi
from PAi & SAi to TEi
Distributed Termination DetectionDistributed Termination Detection
COMPONENT COMPONENT
COMPONENT COMPONENT
IDLENESSHANDLER
IDLENESSHANDLER
IDLENESSHANDLER
IDLENESSHANDLER
TIMEMANAGER
We extend DTD algorithm of Dijkstra-Safra to detect idleness and to progress time
Time ManagerTime Manager
Initiates global idleness detection by sending a token consisting of a global message counter and a global flag along the ring.
Initially, global message counter is 0 and the global flag has value IDLE_TAG.
If time manager receives the token back with the global message counter equal to 0 and the global flag equal to IDLE_TAG, it detects global idleness.
Otherwise, time manager repeats idleness detection. If global idleness is detected, time manager reactivates
components in the new time slice, then progresses time by sending the token with flag TICK_TAG and restarts global idleness detection in the next time slice
idle
idle
idle
idle
flagflag
flagflag
0 0
0 0
Time Manager 0idleInitially
msg0
idle
-1 1
4
3
1
2
1 sends a message to 4
4 and 1 update their message counters
idle
idle
idle
idle
flagflag
flagflag0 0
Time Manager
1 and 2 become idle 0
idle
-1 1
4
3
1
2
idle
1 updates the token 1
running
1 sends the token to the next
running1
1 updates local flag
flagidle
idle
idle idle
flag
flagflag0 0
Time Manager
-1 1
4
3
1
2
idle
2 becomes idle and updates token
running1
flag
2 sends token to the next and updates local flag
idlerunning1 flag
idle
idle idle
flag
flagflag0 0
Time Manager
-1 1
4
3
1
2
idle
3 becomes idle and updates token
flag
3 sends token to the next and updates local flag
idlerunning1 flag
idle
idle
running1
flag
idle
idle idle
flag
flagflag0 0
Time Manager
-1 1
4
3
1
2
idleflag
4 sends token to the next and updates local flag
idle
running0
flag
4 updates token
idleflag
idle
running1
flag
0flag
Maybe one of the components is active, repeat detection
4 becomes idle
flag
idle
idle idle
flag
flagflag0 0
Time Manager
-1 1
4
3
1
2
idleflag
idle
idle0
flag
idleflag
idleflag
flag
The system is idle, progress time!
idleflag
idle1
idle1
idle0 idle
0
idle01
idle
idle idle
flag
flagflag0 0
Time Manager
-1 1
4
3
1
2
idleflag
idleflag
idleflag
idleflag
flag
Reactivate components in the new time slice
idleflag
restart
restartflag
restartflagrestart flag
0restart flag
0
restart
idle
idle idle
flag
flagflag0 0
Time Manager
0 0
4
3
1
2
idleflag
idleflag
idleflag
idleflag
flag
Progress time
idleflag
tick
tickflag
tickflagtick flag
0tick flag
0
tick
Start new idleness detection round
TTCN-3 Test SystemTTCN-3 Test SystemC
ompo
nent
Han
dlin
g (
CH
)
Test Management (TM)
TRI
Test Logging (TL)
TTCN-3 executable (TE)
Main Test Component
Test Component
Cod
ing
and
Dec
odin
g(C
D)
Platform Adapter(PA)
System Adapter(SA)
TCI-TM TCI-TL
TC
I-C
H
TC
I-C
D
Simulated Time InterfaceSimulated Time Interface
IdlePAIdle(int TRISentPA, int TRIEnqdPA)SAIdle(int TRISentSA, int TRIEnqdSA, int SASentSUT, int SUTEnqdSA)TEIdle(int TRISentTE, int TRIEnqdTE, int TCISentTE, int TCIEnqdTE)
Active PAActive()SAActive()TEActive()
Idleness Handler for TSi Idleness Handler for TSi
Counts number of messages exchanged by the TSi via TCI Counts number of messages exchanged by the TSi via TRI Counts number of messages exchanged by the TSi (actually
SAi) with SUT Keeps idle variables for TEi, SAi and PAi Keeps flag variables for TEi and SAi If idleness token present, SAi/PAi/TEi are idle and there are
no messages pending at TRI, then
the idleness handler of a TSi detects local idleness of the TSi updates the idleness token
and sends it to the next handler
Testing Railway Interlockings (FATES’05)Provide a framework for testing railway interlockings with
TTCN-3
INTERLOCKING LAYER(guarantees safety of railway control system)
INFRASTRUCTURE(railway tracks, signals, level crossings, points etc.)
LOGISTIC LAYER(human-experts interface) Railway Control
System
Apply the framework for testing interlocking software for Hoorn-
Kersenboogerd station
Testing layer 3 of a 2G/3GTesting layer 3 of a 2G/3G
RR
GSM
RRC
WCDMA
phone application
SUT
L1
L3
L2 L2/GSM L2/WCDMA
L1/GSM L1/WCDMA
CTRL
L1_GSM L1_WCDMA
MM/CC
ConclusionConclusion
We proposed host based testing with simulated time in TTCN-3 Possible to implement on the level of TTCN-3 code
(FATES’05)
Pro: does not require new interfaces
Con: test suite is suitable for testing with simulated time only Proposed a solution on the level of adapters for distributed
host-based testing
Con: require new interface (submitted as change request to ETSI by Nokia)
Pro: test suites are suitable for testing both in real and in simulated time
