EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

ARC tools for revision and nightly functional tests

Jozef Cernak, Marek Kocan, Eva Cernakova

(P. J. Safarik University in Kosice, Kosice, Slovak Republic)

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Testing strategy in ARC• ARC-EMI testing tools

– Build testing• Nightly (every night)• Revision (after each commit of developers)

– Automatic functional testing of nightly builds• Future plans

– Increasing number of automatic functional tests– Automatic performance testing

• Conclusion

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 2

Outline

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Several roles in project: developers, testers and ARC release manager • Adopted tests:

– Unit tests(developers)– Build tests (testers)

• Revision (Kosice) • Nightly (Copenhagen)• EMI-RC (ETICS)

– Functional tests (testers, developers)• Manual • Automatic (Kosice)

– Regression testing (testers, developers)– Performance testing (testers)

• Manual• Automatic in development phase

– Large scale testing (in preparation phase)Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 3

Testing practice in ARC

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611 ARC-EMI testing tools

Relation between ARC and EMI testing tools

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 4

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Block diagram of automatic revision and Functional toolsSimple database server with PHP and Python scripts

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 5

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Infrastructure (local cluster)– 2 SUN 2200M2 servers (2x4GB RAM, 250GB

disk, AMD 64 bits)– 2 desktops (2x 1GB RAM, 500GB disk 32 bits)– 1 MAC mini (2GB RAM, 160GB, 32 bits)

• Software– MySQL, PHP, Python– Code is free

• Distributed data processing – SVN

Technical details

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 6

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Features– Supported are several platforms of OS:

• CentOS (32,64), • Debian(32,64), • and MAX OS (64)

– Detailed identification of errors during building process• Autogen• Configuration• make• make dist• Build

Revision automatic testing

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 7

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

– User friendly interface to define search criteria• Daily• Weakly• Monthly • Custom

– Revision number, etc (see next examples)

• Direct connection with SVN• Warning messages for developers if code is broken

– CCCC metric – Quick mechanisms to identify breaking event

…continued.

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 8

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Example of www interface for automatic testing toolsHome page: http://arc-emi.grid.upjs.sk/ (Revision database is slow)

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 9

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Several criteria to search results of revision testing

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 10

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611 Example of the results of daily revisions

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 11

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Features– Automatic download code from SVN based on several criteria:

• trunk, • revision code,• nightly builds

– Automatic build– Automatic deployment– User friendly interface for:

• submitting proposals of test cases• to search the results of functional tests

– Tests are grouped into two main groups:• functional tests of server,• functional tests of client.

Automatic functional tests

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 12

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611 Interface to enter search criteria

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 13

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611 The results of automatic functional tests

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 14

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• client_arcsub_to_arex_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured A-REX service.

• client_arcsub_to_grid-manager_gridmap_simplejob_jsdl– Tries to submit a simple JSDL job to a grid-map secured ARC0 CE.

• client_arcsub_to_GIIS_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service.

• client_arcsub_to_ISIS_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud.

• client_arcsub_random_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using random broker.

• client_arcsub_random_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to an GIIS cloud using random broker.

• client_arcsub_fastestqueue_broker_isis_simplejob_jsdl– Submission of a simple JSDL job to an ISIS cloud using fastest queue broker.

• client_arcsub_fastestqueue_broker_giis_simplejob_jsdl– Submission of a simple JSDL job to a GIIS service using fastest queue broker.

• Failed– There was some error before we managed to start testing. The error could have been in

code retrieval, installation or somewhere else.

Functional test scenaria

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 15

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Requirements:– several servers and clients– Monitoring of several distributed processes:

• resources: – Memory– CPU– Network usage

• Performance– Scalability– Number of concurrent requests

• Reliability of services:– Ratio between successful tasks and total tasks.

Examples of tasks: job submission, file transfer and etc.

Automatic performance tests

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 16

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Proposed infrastructure:– Grid servers and clients, we plan to use

existing infrastructure– Communication layer • XML-RPC client-server architecture

– Central database server– Common Python library on each server and

client (for example start stop service)– Central control program

… continued.

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 17

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Automatic testing covers:– Build process (ARC, ETICS),– Functional,– Performance tests (in progress).

• Utilization of automatic test tools increases probability to find weak part of the software. On the other hand, manual tests are important.

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 18

Conclusions

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

• Testing in ARC– http://wiki.nordugrid.org/index.php/Testing

• Revision tests– http://download.nordugrid.org/builds/– http://arc-emi.grid.upjs.sk/revisionTests.php

• Functional tests– http://arc-emi.grid.upjs.sk/functionalTests.php

• Code:– http://svn.nordugrid.org/trac/workarea/browser/

ARCTestScriptshttp://svn.nordugrid.org/trac/workarea/browser/ARCTestScripts

References

Use Insert Header & Footer to set this field

Use Insert Header & Footer to set this field 19

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Thank you

Use Insert Header & Footer to set this field

20Use Insert Header & Footer to set this field

EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611