www.eu-etics.org

ETICS All Hands meeting ETICS All Hands meeting BBolognaologna, , OctoberOctober 2 233--2525, 2006, 2006

WP4 StatusWP4 Status

Eva TAKACSEva TAKACS

2

Overview

• Objectives of WP4Objectives of WP4• Events of WP4Events of WP4• Activities of WP4Activities of WP4• Deliverable listDeliverable list• Present statusPresent status• Future workFuture work• Discussion pointsDiscussion points

3

Objectives of WP4

The objectives of WP4 are:

• to collect requirements for unit, functional and regression testing

• to deploy, adapt and maintain the software testing tools in addition to collect and adapt the related testing methodology

• to identify quality metrics, and evaluate the different methods and tools based on them, provide measurement collectors for the reporting engine

4

Deliverable list of WP4

• D4.1 Requirements and specifications for unit, functional and regression testing – (PM06 4D Soft)

• D4.2 Distributed test execution system (prototype)– (PM12 4D Soft)

• D4.3 Distributed test execution system (final release) Coherent documentation of the selected software testing tools and methodology– (PM22 UoW)

5

Work has been done

• a comprehensive a comprehensive studystudy of the relevant testing methods of the relevant testing methods has been done has been done

• a selected descriptiona selected description of well-known or state-of-art of well-known or state-of-art testing toolstesting tools has been done. Distributed environment has been done. Distributed environment was taken into account. was taken into account.

• based on the existing methods and tools requirements based on the existing methods and tools requirements for both the testing methods and toolsfor both the testing methods and tools has been done has been done

OutputOutput: : D4.1 Requirements and specifications for unit, functional and regression testing

6

Events of WP4

• WP4 Internal meeting WP4 Internal meeting – 1st october 2006 at CERN Geneva1st october 2006 at CERN Geneva

• Scope of the meetingScope of the meeting– to discuss on detail the requirements relating testing tools and to discuss on detail the requirements relating testing tools and

methodsmethods– connections with other WPs and their tasksconnections with other WPs and their tasks

• Outcome of the meeting Outcome of the meeting – Central part of the testing toolkit to be integrated in ETICS Central part of the testing toolkit to be integrated in ETICS

should be a Test and Metrics Plug-in Framework and the testing should be a Test and Metrics Plug-in Framework and the testing tools and metrics calculation utilities will be plugged heretools and metrics calculation utilities will be plugged here

7

Why a plugin framework?

Making an application extensible using the Making an application extensible using the plugin system has a strong advantage over plugin system has a strong advantage over other techniques:other techniques:

• It allows third-party developers to easily enhance It allows third-party developers to easily enhance software in a way that is loosely coupled: only the software in a way that is loosely coupled: only the plugin API is required to remain stable.plugin API is required to remain stable.

8

Why a Plugin Framework in ETICS?

The rationale behind using „Eclipse style” plug-in architecture The rationale behind using „Eclipse style” plug-in architecture isis

• that mthat most of ost of present and/or future ETICSpresent and/or future ETICS users will users will come come with some tests and testing tools already in placewith some tests and testing tools already in place and of and of course they would like to use them work with ETICScourse they would like to use them work with ETICS

For exampleFor example: A specific : A specific user would like to use his charting system when user would like to use his charting system when creating reports after running some functional testcases with creating reports after running some functional testcases with TestManagerTestManager

9

Why a Plugin Framework in ETICS?

On the other hand

• they should be provided with „built-in” (default) testing tools when they do not have their own but would like to use something

• (Assuming – we plugged the best tools beforehand)

For exampleFor example: : Users would like to use a coverage tool to measure the Users would like to use a coverage tool to measure the effectiveness of their Junit testcases but they do not have one at hand. effectiveness of their Junit testcases but they do not have one at hand. Then he can use EMMA provided by the framework.Then he can use EMMA provided by the framework.

10

Activities of WP4

Two main areas:

• NMI – as a testing tool• Test and Metrics Plugin Framework

– Architecture of the framework– Test tools to be included as plugins– Test tools and methodologies cross reference

documentation– Metrics

11

Plugin Framework – First steps

It was decided that as a first stepIt was decided that as a first step

– TestManager a testdriver developed in CERN and used for gLite TestManager a testdriver developed in CERN and used for gLite functional testingfunctional testing

– Two coverage tools Two coverage tools – EMMA, an open source tool requested by future ETICS usersEMMA, an open source tool requested by future ETICS users– JavaCov, a commercial tool providing a very strong MC/DC JavaCov, a commercial tool providing a very strong MC/DC

coverage criterion coverage criterion

will be taken into account. will be taken into account.

12

Next steps

• Testing coverage tools for ETICS code (gLite) in a real environment

• Having TestManager in a modularised way • Architectural specification and prototype for the Plug-in

Framework

13

Future work

• 20062006• NMI- as a testing tool – prototypeNMI- as a testing tool – prototype• Test and Metrics Plug-in Framework – prototype, Test and Metrics Plug-in Framework – prototype,

architectural specificationarchitectural specification

• 20072007• NMI- as a testing tool – final release, documentationNMI- as a testing tool – final release, documentation• Test and Metrics Plug-in Framework – final release, Test and Metrics Plug-in Framework – final release,

documentationdocumentation

14

Other topics

• Some of the user requirements:Some of the user requirements:

• Dependency analysis Dependency analysis For example: in case of gLite services helping the testers to identify For example: in case of gLite services helping the testers to identify

the „calling chain” of servicesthe „calling chain” of services

• Workflow system for designing system level testsWorkflow system for designing system level tests