Applying Model Based Testing in Different Contexts

Applying Model Based Testing inDifferent Contexts

Alexander PetrenkoVictor KuliaminISP RAS, Moscow

Introduction

MBT application success depends on the context of the project Staff Organization Development culture, paradigms, and stereotypes

Can we modify technical aspects of MBT method used to fit the context more?

Yes! (at least sometimes…)

Outline

Introduction Examples

Software component testing – UniTesK Formal language processor testing – OTK

The Origin

1994 – 1996ISP RAS – Nortel Networks project onfunctional test suite development for Switch Operating System kernel A lot of bugs found in the OS kernel, which had

been in use for 10 yearsSeveral of them cause cold restart

About 600K lines of code tested by 2000

Proposed Solutions

Software contract specifications for functionality Pre- and postconditions of operationsInvariants of data types

Test adequacy criteria based on coverage of specifications

Transformation of pre- and postconditions into (IO)FSM, traversal of which guarantees the required coverage level

On-the-fly test generation – construction of a transition tour of the (IO)FSM or other paths

Ch@se Tool

The Whole Picture

System under TestBehavior Model Testing Model

Coverage Model

Test Input GenerationSingle Input Checking

Tools

J@T (with C++ Link)Java (C++) / NetBeans, Eclipse

CTesKC / Visual Studio 6.0, gcc

Ch@seC# / Visual Studio .NET 7.1

Applications

IPv6 implementations - 2001-2003 Microsoft Research Mobile IPv6 (in Windows CE 4.1) Oktet

Web-based banking client management system Enterprise application development framework Billing system Components of TinyOS

http://www.unitesk.com

Tester

Application Success Prerequisites

Knowledge source on actual functionality Documents Domain experts Developers, architects

Direct access to the interface under test SUT

Outline

Introduction Examples

Software component testing – UniTesK Formal language processor testing – OTK

The Origin

2001 – 2003ISP RAS – Intel project ontesting correctness of set of optimizers Several dozens of bugs found

Restrictions

No access to actual functionality – only general processing algorithm descriptions

No access to unit interface –only external compiler interface

Tester SUT

Proposed Solutions

Program structure (abstracted of irrelevant elements) considered as test data model

Structured set of data generators Checking optimizer correctness by comparison

with nonoptimized program Coverage goals

Processing algorithm is taken into account by model Heuristics

Example: Simple Expressions

Test

Expression

BinaryExpr

op : {+,-,*,/}

Variable

id : string

Constant

value : int

stmts1..*

left 1

right 1

Statement

var 1

expr1

xi = Expr

xi 17 Expr1 + Expr2

OTK Tool

Test Data Generation

Processing Algorithm

Basic Blocks and Connectors

Test Data (Programs)

Combination Iterator

Basic Blocks Mapping

if(…)…else …

for(…;…;…)

Language

Optimizer Correctness Checking

Test Program

CompilerOptimizer

Compiler

==?

Further Work

The whole compiler testingSyntax checkerStatic semantics checkerBack-end

Formal semantics description needed

Tools integration

In work

?

References

1. V. Kuliamin, A. Petrenko, I. Bourdonov, and A. Kossatchev. UniTesK Test Suite Architecture. Proc. of FME 2002. LNCS 2391, pp. 77-88, Springer-Verlag, 2002.

2. V. Kuliamin, A. Petrenko, N. Pakoulin, I. Bourdonov, and A. Kossatchev. Integration of Functional and Timed Testing of Real-time and Concurrent Systems. Proc. of PSI 2003. LNCS 2890, pp. 450-461, Springer-Verlag, 2003.

3. A. Kossatchev, A. Petrenko, S. Zelenov, S. Zelenova. Using Model-Based Approach for Automated Testing of Optimizing Compilers. Proc. Intl. Workshop on Program Undestanding, Gorno-Altaisk, 2003.

4. V. Kuliamin, A. Petrenko, A. Kossatchev, and I. Burdonov. The UniTesK Approach to Designing Test Suites. Programming and Computer Software, Vol. 29, No. 6 , 2003, pp. 310-322. (Translation from Russian)

5. S. Zelenov, S. Zelenova, A. Kossatchev, A. Petrenko. Test Generation for Compilers and Other Formal Text Processors. Programming and Computer Software, Vol. 29, No. 2 , 2003, pp. 104-111. (Translation from Russian)

Contacts

Alexander K. Petrenko Victor V. Kuliamin

petrenko@ispras.ru kuliamin@ispras.ru

109004, B. Kommunisticheskaya, 25

Moscow, Russia

Web: http://www.ispras.ru/groups/rv/rv.html

Phone: +7-095-9125317

Fax: +7-095-9121524