Network integration testing: concepts, test specifications and tools for automatic telecommunication services verification

Network integration testing: concepts, test speci®cations andtools for automatic telecommunication services veri®cation

Giulio Maggiore, Giulio Brusasco, Marco Vecchiato

CSELT Centro Studi e Laboratori Telecomunicazioni, via Reiss Romoli 274, 10148 Turin, Italy

Abstract

The purpose of this tutorial is to provide concepts and historical background of the ``network integration testing''

(NIT) methodology. NIT is a ``grey box'' testing technique that is aimed at verifying the correct behaviour of inter-

connected networks (operated by di�erent operators) in provisioning services to end users, or the behaviour of a

complex network operated by a unique operator. The main technical concepts behind this technique are presented along

with the history of some International projects that have contributed to its early de®nition and application. European

Institute for Research and Strategic Studies in Telecommunication (EURESCOM) has actually been very active, with

many projects, in de®ning the NIT basic methodology and providing actual NIT speci®cations (for narrow-band and

broad-band services, covering both voice and data). EURESCOM has also been acting as a focal point in the area, e.g.,

encouraging the Industry in developing commercial tools supporting NIT. In particular, the EURESCOM P412 project

(1994±1996) ®rst explicitly de®ned the NIT methodology (the methodological aspects include test notation, test im-

plementation, test processes, distributed testing and related co-ordination aspects). P412 applied the methodology to

ISDN whilst another project, P410, applied NIT to data services. The P613 project (1997±1999) extended the basic NIT

methodology to the broad band and GSM. More into details, the areas covered currently by NIT test speci®cations

developed by EURESCOM projects include N-ISDN, N-ISUP, POTS, B-ISDN, B-ISUP, IP over ATM, ATM/FR,

GSM, focusing also on their `ìnter-working'' cases (e.g., ISDN/ISDN, ISDN/GSM, etc.). ETSI, the European Tele-

communication Standards Institute, also contributed to NIT development (e.g., the de®nition of the TSP1+ protocol,

used for the functional co-ordination and timing synchronisation of all tools involved in a distributed testing session).

The paper also discusses NIT in relation to the recent major changes (processes) within the telecommunication (TLC)

community. Beyond the new needs coming from the pure technical aspects (integration of voice and data, ®xed mobile

convergence, etc.) the full deregulation of the TLC sector has already generated new processes and new testing needs

(e.g., Interconnection Testing) that had a signi®cant in¯uence on the methodology. NIT is likely to continue to develop

further in the future according to the needs of telecom operators, authorities, userÕs associations and suppliers. Ó 2000

Elsevier Science B.V. All rights reserved.

Keywords: Network Integration Testing; End-to-end; Node-to-node; GSM; ISDN; FR; ATM; Testing

ABR: 3PTY, three party; ATM, asynchronous transfer mode; ATS, abstract test suite; BC, bearer capability; BRA, basic rate access;

BSC, base station controller; BSS, base station system; B-ISDN, broad-band ISDN; HLC, high layer compatibility; HLI, high layer

information; HLR, home location register; CFB, call forwarding on busy; CFU, call forwarding unconditionally; CD, call de¯ection;

CLIP, calling line identity presentation; CLIR, calling line identity restrictions; COLP, connected line presentation; COLR,

connected line restrictions; CS, capability set; CUG, closed user group; DDI, direct dialling in; DSS1, digital signalling system No 1;

Computer Networks 34 (2000) 799±819

www.elsevier.com/locate/comnet

E-mail addresses: [email protected] (G. Maggiore), [email protected] (G. Brusasco), [email protected]

(M. Vecchiato).

1389-1286/00/$ - see front matter Ó 2000 Elsevier Science B.V. All rights reserved.

PII: S 1 3 8 9 - 1 2 8 6 ( 0 0 ) 0 0 1 2 9 - 8

ECT, explicit call transfer; ETS, executable test suite; ETSI, European Telecommunication Standard Institute; EURESCOM,

European Institute for Research and Strategic Studies in Telecommunication; FE, front end; FR, frame relay; FPH, freephone;

GSM, global system for mobile communication; ICS, implementation conformance statement; IP, internet protocol; ITU,

International Telecommunication Union; IUT, implementation under test; ISDN, Integrated Services Digital Network; ISO,

International Standard Organisation; ISUP, ISDN user part; IXIT, implementation extra information for testing; IWF, inter

working function; LLC, low layer compatibility; LLI, low layer information; MCID, malicious call identity; MSC, mobile switching

centre; MSN, multiple subscriber number; MTP, message transfer part; NE, network element; NIT, network integration testing;

NNI, network network interface; PDU, protocol data unit; PMI, pilot machine interface; PNO, public network operator; POTS,

plain ordinary telephone set; PRA, primary rate access; PSTN, public switched telephone network; PVC, permanent virtual circuit;

QoS, quality of service; SS, system supervisor; SS7, common channel signalling No. 7; SUB, subaddressing; SVC, switched virtual

connection; TLC, telecommunications; TSP, test synchronisation protocol; TTCN, tree and tabular combined notation; TUP,

telephone user part; TP, terminal portability; UUS1, user to user service 1; Um, radio interface; UNI, user network interface; WAN,

wide area network

1. Introduction

The new technical possibilities, the deregulationof telecom markets and increasing customer de-mands create new challenges for telecom serviceproviders and operators. In particular, due toshorter and shorter time to market, telecom op-erators cannot often verify thoroughly the indi-vidual network elementsÕ (NE) behaviour beforestarting their roll out and subsequent deploymentin their networks.

To face such new challenges, telecom operatorshave been devising new methodology and tools inorder to re-engineer the traditional ``type ap-proval ± acceptance testing'' phases. Services,performance, integrity, etc., are no longer deeplytested in a standalone manner but veri®cationsoccur mainly when two or more NEs are inter-connected to each other, i.e., ``testing takes placein a network context''. Network integration test-ing (NIT) approaches typically apply to such``network-oriented'' situations, when complexnetwork inter-workings may occur (inter-workingrelated to provisioning, operations, protocols,services).

European Institute for Research and StrategicStudies in Telecommunication (EURESCOM) hasbeen developing substantial experience in devel-oping NIT abstract test suite test speci®cations(NIT ATSs). After a pioneering P104 project in1990, the subsequent P410, P412 and P613 projectshave been working actively on setting and re®ningthe NIT methodology and producing NIT ATSs.The results of the mentioned projects have been

previously published as EURESCOM delivera-bles. 1

It may be worth noting that P613 devoted asigni®cant e�ort in the GSM area, de®ning an`ènd-to-end'' ATS that included also this type ofnetwork access. Actually the P613 end-to-end ATSapplies both to the Um (radio) and A interfaces asfar as the GSM sides are concerned. The P613 testcases are at call control level, therefore, they runon top and are independent of the transport layers,which, for the GSM sides, are Um or SS7/MTP. Inthe GSM case, an attempt was made to cover notonly the functional aspects but also to specify testsfor load and overload conditions, fault injection,etc. 2

When discussing NIT, a recurrent issue is ``whoshould be the main user of NIT, who shouldbene®t from NIT, who should pay for NIT'',whether the suppliers or the operators. The simpleanswer is that NIT is mainly a technique for theoperator, since it is up to the operator to verifyand guarantee that his network is working globallyin spite of its complexity. Other types of testing(conformance, standalone testing), which are morefocused on the behaviour/performance of a single

1 For each deliverable belonging to PXYZ please refer to

http://www.eurescom.de/Public/Projects/pX00-series/pXYZ.

htm (e.g., for P613 refer to http://www.eurescom.de/Public/

Projects/p600-series/P613/P613.HTM).2 Such extensions of the basic NIT methodology can only be

applied on dedicated test plants (o�-line facilities) because such

tests may in¯uence and impair the normal operation of a

network.

800 G. Maggiore et al. / Computer Networks 34 (2000) 799±819

system or sub-system (e.g., a software block) arede®nitely in the domain of the supplier. However,suppliers may also be willing to use NIT tools inorder to be able to verify globally their o�er anddemonstrate to operators (their clients), the qual-ity of their products, and allowing operators tofocus testing only on multi-supplierÕs network ± orNEs ± integration (NIT applied in multi-vendorcon®gurations).

As already mentioned, the pressure of regula-tion is pushing towards the full opening of publicnetworks in order to enable or increase competi-tion in the market. Interconnection of networks ofdi�erent operators must take place. Even if in thiscase such interconnection is more an obligation (atleast for the dominant operators) rather that acommercial choice, still there is a ®nal need for allparties involved to make sure that the networksworks properly. Also, regulators and other publicauthorities are interested to ensure and determinewhether `ìt works'', `ìs secure'', in order to pre-serves the networksÕ integrity, etc.

Therefore, the deployment of full competition(regulated competition) must be mentioned in as-sociation to the NIT concept, because it is anotherstrong driver for the further development of spe-ci®c technical bases in the domain of networktesting. NIT ± when applied to the regulatory as-pects ± can be referred to as ``network intercon-nection testing''. The basic skills, and many testsuites (ATSs and ETSs) or commercial testingequipment can be used ± sometimes with limitedadaptation ± also for network interconnectiontesting.

2. The initial step: the P104 project

At the beginning of 1990s, several Europeanadministrations and network operators announcedthe availability of ISDN in their respective net-works. Deregulation and full competition had notyet touched most markets, and in most Europeancountries, only one public operator was entitled too�er voice/data services. In nearly all countries, thecommercial use of ISDN (Euro-ISDN) was in-tended to start after a pilot project, with theapplication of a memorandum of understanding

based on the application of the relevant ETSIstandards.

A number of services and protocols were beingde®ned or speci®ed in that period: networks werebecoming more and more complex: di�erentkinds of users, di�erent services ± bearer and/orsupplementary services ± and di�erent transporttechnologies ± circuit/packet switching ± had tointer-operate in the establishment of telecommu-nication services. Operation and maintenance ac-tions on NEs, somewhere in the network, mightalso in¯uence the end-to-end services quality. Forexample when a new supplementary service orwhen a new version of a signalling protocol (e.g.,ISUP) started to run in a part of the global net-work (i.e., in a given country) some problemsmight occur. The services should however be of-fered in a reliable and homogeneous way despitethe many networks (with their di�erent suppliers),the national variants and the standards optionsand interpretations. The `Èuro-Network'' behav-iour had to be tested and monitored using ``not-only domestic'' approaches and techniques.

In this scenario, the EURESCOM project P104was launched in 1990 with the purpose of pro-viding a common methodology and harmonisedtest procedures to public network operators(PNO), who intended to establish or maintain aninternational ISDN service (switched end-to-endconnections) between them, for commercial rea-sons. A common speci®cation for the veri®cationof the end-to-end ISDN service was believed tospeed up the initial ``negotiation'' and the commonunderstanding of the test cases proposed by eachoperator.

2.1. The P104 methodology

EURESCOM project P104 `ìnternationalISDN end-to-end testing'', was the ®rst to adoptISO 9646 methodology [1±7], adapting it beyondits original scope, which was pure conformancetesting. The implementation under test (IUT) usedin the P104 NIT case, is shown in Fig. 1. SeveralNEs (access exchanges, trunk exchanges, andinternational gateways) compose the IUT. Theinterfaces between the testers and the IUT (at theexternal borders of the IUT) vary according to

G. Maggiore et al. / Computer Networks 34 (2000) 799±819 801

the type of accesses (ISDN BRA, ISDN PRA, andPOTS). The NIT ATS included all their possiblecombinations (BRA±BRA, BRA±PRA, BRA±POTS, POTS±POTS, etc.), but had a special focuson ISDN (ISDN±ISDN end-to-end).

For the abstract test speci®cation (ATS) of theISDN layer 3 (DSS1), the tree and tabular com-bined notation (TTCN) [3,8] was used. Each testcall was described with two test cases, re¯ecting

the A or B side of the call respectively, using twodi�erent and independent TTCN tables, as shownin Figs. 2 and 3. The ``logical link'' between the Aand B sides was not explicitly described in theformal TTCN test speci®cation, and the fact thatthe two test cases are actually correlated was basedon naming conventions (A_110101 and B_110101`àre the same call'' because both end with_110101).

Fig. 2. Example of test case for DSS1, A side.

Fig. 1. P104 implementation under test.


2.2. The P104 results

The main P104 results [23], are the ISDN end-to-end ATS (functional/protocol aspects) and thetransmission performance measurement speci®ca-tions.

The ATS coverage of the ISDN service (func-tional/protocol aspects) was actually limited sincethe ``rule'' of the project was to cover just once theveri®cation of the transport of the di�erent infor-mation elements of the DSS1 PDUs thorough theinternational network. In other words, the com-bination of all the rate adaptations in the BC andLLC information elements for the various speedswere missing, as well as an accurate supplementaryservices coverage.

The transmission performance measurements,were introduced because the future of ISDN wasthought strongly dependent on the transmissionquality that could have been achieved (in countrieswhere the transmission characteristics of existinganalogue circuits were considered poor, ISDNcould have been one more reason to attract newsubscribers). The procedures for the measurementsof the in¯uence of transmission errors on the end-to-end communication are various. In particular,those described were:

· Bit error rate (BER) for 64 kbit/s channels inboth basic and primary rate accesses using thevalues de®ned in ETSI and ITU recommenda-tions [19±21].

· Connection processing delays de®ned in Rec.I.352 [22].

The application of the P104 results and the ex-perience coming from the real word testingactivities for Euro-ISDN showed that the end-to-end phase was typically the last bilateral testingactivity between two countries/operators beforeopening commercially an international ISDNservice between the two parties. Before end-to-end, a so-called node-to-node phase was sched-uled to test the compatibility at the signallinglevel between the two interconnected internationalgateways. Node-to-node (also referred to as``point-to-point'') was focused on SS7 ISUPcompatibility (the underlying MTP functions wereseldom re-tested because these were normally al-ready operational on both gateways for the sup-port of TUP or other old fashioned SS7application parts). The need for a reviewedoverall methodology and the coverage limitationsin end-to-end P104, indicated the need for furtherstudies in ETSI and EURESCOM in order toprovide:

Fig. 3. Example of test case for DSS1, B side.


· A more suitable testing methodology.· TTCN ATSs suitable of being compiled by

TTCN compilers in order to produce automati-cally ETSs without implicit assumptions, alsobecause TTCN speci®cations were not appreci-ated too much by O&M sta�.

· An improved reference testing architecture, in-cluding a suitable synchronisation mechanismin order to be able to run the A- and B-side testcases `àutomatically''.

3. ETSI contribution to de®ne the NIT concept

In parallel and in co-ordination with the EU-RESCOM P412 project (presented in the nextsection), also in ETSI (speci®cally in the MTS``methods for testing and speci®cation'' technicalcommittee) an activity was performed in order toget to more accurate de®nitions of NIT, was de-®ned in ETR 193 [9], as ``the set of all the checkingnecessary to verify that a given network works as itis expected, and to verify the compatibility of thesingle network components (NEs). Conformancetesting of each network component is assumed as apre-requisite''.

In Fig. 4, a very generic reference model isshown. A and B represent interfaces, where testerscan be attached (using DSS1 or ISUP, for exam-ple). M represents an optional monitoring point(more of them might exist along the call path, allof them measuring the same parameters andevents, according to the ATS). The network mayor may not exists in real NIT executions, 3 but if itexists, its complexity might vary, possibly includ-ing many inter-working of protocols and/or na-tional networks.

The ``system'' under test (SUT) is actuallycomposed by all the network components that areplaced between the interfaces where the testers areto be connected.

From a practical point of view, NIT is con-ceived to be executed in two di�erent situations:

· In a controlled situation, i.e., in a local or dis-tributed test plant, before the new functionsand services are deployed into a real, operation-al network; this could be the case, for example,of testing the compatibility of a few well-identi-®ed SS7 nodes.

· In real situations, when the new functions andservices have already been deployed in the realNEs, which are therefore in operational states;this con®guration implies that the test suiteand the related procedures should be designedso as not to disturb the normal network behav-iour. In this case, full control over all NEs is notassured, also because the routing of test callsmight vary according to tra�c conditions. 4

From the point of view of the SUT, a main dis-tinction within NIT is between end-to-end testingand node-to-node testing:· End-to-end testing: the network is tested as it is

seen from the user's terminal equipment (e.g.,A�B� ISDN basic access protocol as shownin Fig. 4), i.e., taking the user-network interfacesas PCOs. M is an optional, generic monitorpoint that might be used to check the internalbehaviour of the network (e.g., for troubleshooting) but it would not normally be speci®edin a ``pure'' end-to-end ATS.

· Node-to-node testing: the network is tested as itis seen from other network components (e.g.,A�B� ISUP protocol as shown in Fig. 4),i.e., taking as PCOs the external network-net-work interfaces. M is the monitor point that isnormally used to check the internal behaviourof the network, and the set of measures and

3 For example, in the case of ISUP Compatibility Testing

between two International Gateways directly connected via an

ISUP link, the M monitoring point drops signals directly from

the SS7 link and no additional SS7 signaling points exist.

4 ``Tricks'' have sometimes been devised and used to avoid

this situation, such as the usage of ``dummy'', temporary area

codes.

Fig. 4. The system under test (SUT) for NIT.


observations is normally part of a pure node-to-node ATS.

As described in ETR 193 [9], the chosen formalmethod for NIT is multi-party test method(MPTM). In the case of end-to-end testing, anapplicable method is MPTM without upper tester(UT). In the case of the node-to-node testing, itmay be necessary to in¯uence the SUT creatingevents/changes that are relevant to the test pur-poses (e.g., block/unblocking bearer circuits). Inthis case, an applicable formal method is MPTMwith an UT (see Fig. 5).

4. The P412 approach and results

In 1994±1996, the EURESCOM project P412``methods and tools for ISDN NIT'', [13±16,24,25], building on the P104 experience, furtherdeveloped the concept and started applying sys-tematically the new methodology to the Euro-ISDN. Another goal was to explicitly de®ne themethodology (in co-operation with ETSI). In thesame time frame, the EURESCOM P410 [33]

project covered with similar aims the testing ofasynchronous transfer mode (ATM)-based net-works. The two projects worked in liaison. Inpractice the P412 objective was to provide a meansof verifying that a given narrowband (ISDN) ser-vice:· worked properly;· was o�ered consistently to all users;· could hide to the user the fact of being possibly

made up of separate components, developed bydi�erent manufacturers and operated by di�er-ent PNOs.

A P412 suitable area of application was identi-®ed again in supporting the Euro-ISDN service(not completely covered by P104). P412 devel-oped e�cient and reliable methods and proce-dures for detecting failures and localisingpossible faults.

A complementary technique to P412 NIT wastra�c route testing, where the aspects of servicequality (high availability as well as good perfor-mance) and their correlation with the networkgeographical distribution and with routing datacorrectness are emphasised.

Fig. 5. The MPTM applied to end-to-end and node-to-node.


4.1. The objective of the project

The overall objective of the project P412(methodology and tools for ISDN network inte-gration and tra�c route testing) was to enhancethe speci®cation and ISDN end-to-end test suites,hence to improve the QoS of ISDN internationallinks. The concept of validation of test speci®ca-tions via a test execution experiment was intro-duced (that project planned since the beginning tocarry out test trials (test executions) to validate thecorrectness and usefulness of the envisaged testcases). Test execution guidelines were written, tosupport and facilitate PNO±PNO executions.

4.2. The P412 results

The results produced by P412 are a set of vali-dated speci®cations for testing the Euro-ISDNservice:· A set of test execution guidelines to assist PNOs

in preparing and executing end-to-end andnode-to-node bilateral test campaigns [13].

· An end-to-end ATS, developed using a ``non-concurrent'' TTCN, for testing the InternationalISDN network as seen from the point of view ofaccess interfaces located within di�erent PNOs.The ATS is focused on correct delivery of proto-col information elements on the D-channel,although veri®cation of B-channel is also inthe scope [14].

· A node-to-node ATS in concurrent TTCN fortesting the International ISUP connection be-tween two PNOs, focused on compatibilitychecking of the link, where particular emphasisis given in co-ordinating the test components(emulators/testers and monitors) so that the testis performed in a synchronised manner [15].

· A test synchronisation protocol (TSP) for co-ordinating a distributed test system, where athree-level structure has been speci®ed togetherwith a protocol for communication betweenmembers of this structure [16].

· A test execution experiment inside the project,chosen as the means of validating the aboveresults [24].

· A speci®cation for a tra�c route testing systemfor measuring the QoS, performance and reli-

ability of the network by generating randomtra�c similar to the tra�c of the subscribersand checking the accessibility and quality ofthe transmissions. The results of the checks tobe reported to a central computer. This wouldgenerate statistical data from the results thatcould be used in long-term network planning.The speci®cation has been developed by the pro-ject, although no validation of it was possible inP412 [25].

In the following paragraphs, the main projectresults are described, indicating the added valuecompared to P104.

4.2.1. Test execution guidelinesThis document was written to assist PNOs to

prepare and execute end-to-end and node-to-nodetest campaigns. The guidelines have been usedduring the P412 ATSÕs execution trials and thenimproved according to the remarks generated.Starting from the scheduling aspects of the testcampaigns, these guidelines help the involved or-ganisations in taking agreements on the followingaspects:· Schedule (where/when the test campaign is to be

carried out, with an estimation of its expectedduration).

· Information on who, when and how to contactto exchange all the necessary preliminary infor-mation.

· People involved in each organisation in per-forming the tests.

· Types of co-ordination procedures, i.e., possibleadvanced mechanisms, if any, available/pre-ferred in each site for the synchronisation/auto-mation of the execution.

· Documentation to be used and exchanged.According to the guidelines, each organisationparticipating in the test campaign must have aclear view of the test cases to be run at each side.The guidelines say how to interpret the rules/variables/parameters described in the TTCNATS. Those rules are actually based on the in-formation provided in ICS and IXIT question-naires for the NIT testing, where the neededinformation is requested independently from bothparties. Before selecting the test cases, both PNOsneed to ®ll and exchange the ICS and IXIT


documents (this normally ends the preparationphase).

After the preparation phase, in which both testoperators have agreed on the test cases to run onand how to co-ordinate the execution, the realexecution phase may start with the means of test-ing preparation. An important aspect is how thetest synchronisation/co-ordination will be handledin each side. For example, which method has beenimplemented in the ETS to handle the initialisa-tion, test execution control and trace requestmessages.

The synchronisation can be performed usingdi�erent methods/levels:· Automatic synchronisation: testers are managed

and test procedures are processed with an auto-matic and fully synchronised method (a NITsynchronisation protocol for full automation,named TSP1 [10,11], was de®ned by P412).

· Semi-automatic synchronisation: management oftesters (con®guration, transfer of trace ®les) isperformed manually and test execution is per-formed automatically (a subset of TSP1 is used).

· Manual synchronisation: test procedures are notautomated (all co-ordination and tester manage-ment is performed manually by an operator).

· With respect to the generic information for end-to-end given in P104, the guidelines cover alsothe node-to-node testing phase and the meansof testing preparation (either in an automaticway or in a semiautomatic/manual way).

4.2.2. End-to-end and node-to-node speci®cationsThese are actually compound documents that

contain the test speci®cations for internationalend-to-end and node-to-node support of ISDNservices. They include the test methodology, testsuite structure and test purposes list, ICS and IXITquestionnaires and the TTCN ATS.

The methodology used inside P412 was basedon the MPTM and the test speci®cations havebeen produced in concurrent TTCN. But due tothe lack (at that time) of concurrent TTCN com-pilers, the ISDN end-to-end ATS has been pro-duced also in non-concurrent TTCN, because oneof the projectÕs general objectives was to producespeci®cations that would be acceptable by existingTTCN compilers to get ÔautomaticallyÕ the exe-

cutable test suites (ETSs). For the node-to-node,due to the strong synchronisation requirementsbetween the main test component and each par-allel test component (A-side, B-side, and a moni-toring point), only the concurrent notation wasactually used [3,8].

4.2.2.1. End-to-end ATS. The end-to-end testspeci®cation covers the support of the basic calland the supplementary services. More into details,the following aspects are covered:· ISDN basic services: ISDN±ISDN call establish-

ment and clearing, end-to-end transport ofcause information and B-channel connectivity.

· Teleservices: end-to-end transport of LLC andHLC information elements related to the com-patibility checking of terminals, covering mostof the standard teleservices.

· Rate adaptation in BC, LLC, and in BC andLLC at the same time for the most commonspeeds (2.4, 4.8, 9.6 kbit/s, for synchronousand asynchronous rates).

· Supplementary services: CLIP, CLIR, COLP,COLR, SUB, TP, CUG, UUS, MCID, FPH,CFU, CFB, CFNR, CD, ECT, HOLD, CW,CONF and 3PTY, i.e., those supported by inter-national ISUP signalling links.

· Inter-working with PSTN (ISDN ® PSTN andPSTN ® ISDN calls).

4.2.2.2. Node-to-node ATS. The aim of the node-to-node tests is to verify ISUP (Q.767) betweentwo network operators. It covers protocol testingof the basic call and also of the supplementaryservices UUS1 implicit, CUG, CLIP, CLIR,COLP and COLR; DDI, MSN, SUB and TP aretested in parallel to the testing of the basic callprocedure (Q.767).

The node-to-node ATS is applicable in twonetwork con®gurations:· The two operators (A, B) interconnected by

direct voice circuits (i.e., adjacent gateways),· The two operators (A, B) interconnected

through long distance carriers (at least one tran-sit operator C).

The P412 node-to-node ATS was actually designedfor the ®rst scenario; with minor adaptations it canbe used in the second case as well.


Direct circuits. In this scenario, the two gate-ways A, B are adjacent signalling points. Thismeans that there are no other operators serving as``voice gateways'' among A and B. This scenario isdepicted in Fig. 6.

Indirect circuits. In this scenario, there are nocircuits directly interconnecting the gateways ofthe two operators. This situation occurs frequentlyin the international interconnections when twooperators A, B use the services of at least a thirdoperator C to interconnect themselves end-to-end.This scenario is depicted in Fig. 7.

4.2.3. Validation of test speci®cationsThe level of complexity of modern services re-

quires a corresponding level of power and techni-cal quality also in the testing methods and testingtools themselves, if reliable results are to beachieved.

One way of achieving such high technicalquality is to de®ne the test speci®cations usinginternationally available notations of good repu-tation and well known, such as the TTCN nota-tion. But using TTCN is often not enough, andthere is also often awareness that TTCN test suitesmay be produced that contain so many errors (e.g.,syntax errors) that their exploitation may behampered or at least discouraged.

This awareness was present since the initialdiscussions that gave birth to the P412 project. Inorder to ensure the technical level of the end-to-end and node-to-node TTCN test speci®cationsreleased, the project agreed to ``validate'' all itsown outputs.

4.2.3.1. Objectives of test speci®cations' validation.The main validation objectives were the following:1. To check the intrinsic technical validity of each

test speci®cation. In particular:1.1 Absence of technical errors:� Syntax errors (misuse of TTCN).� Static-semantics errors (misuse of

TTCN).� Misinterpretations of the ISDN proto-

cols speci®cations (ISDN Signalling).1.2. Good testing coverage, completeness.

2. To check the real usability of each speci®cation:� No unnecessary complexity.� Understandability by human experts.� Implementability into ETSs by automated

tools (e.g., TTCN compilers).� Implementability into ETSs also by human

experts (for the case when compilers are notavailable).

3. To demonstrate and communicate ``to the mar-ket'' the technical quality achieved.

4. To document possible remaining de®ciencies.5. To validate the validation approach itself, i.e.,

to see if concrete validation tasks could be per-formed satisfactory in EURESCOM Projectsaimed at producing testing speci®cations.

Almost all the previous ``validation levels'' havebeen applied to the P412 products: guidelines,ATSs (end-to-end and node-to-node) and TSP1,which was one of the important results producedby P412.

4.2.4. Test synchronisation protocols (TSP1/TSP1+)4.2.4.1. TSP1 ± test synchronisation protocol 1.TSP1 is a co-ordination protocol that can be usedto automate NIT campaigns. Automation mayresult in considerable time saving. Moreover TSP1is a generic command language to control severaltesters independently and without any need toknow/manage details related to their actual

Fig. 6. The direct circuit con®guration for node-to-node.

Fig. 7. The indirect circuit con®guration for node-to-node.


physical nature. For example, there is no need toknow what commercial tools are actually presentat the far ends, as long as such equipment is de-clared capable to execute the relevant ATS testcases (those agreed).

The test synchronisation protocol 1 provides aset of ``services'', divided into:· Pre-testing phase: initialisation of physical test

tools.· Testing-phase: running of test cases in a co-ordi-

nated manner (timing and functional co-ordina-tion).

· Post-testing phase: collecting test results andclosing the testing session.

TSP1 is based on a three-level architecture (a tree)as described below and illustrated in Fig. 8:· System Supervisor (SS) is the functional entity

at the higher level. It interfaces with the test-ing operator using any implementation depen-dent man-machine language but communicateswith the second-level functional entities (frontends, FEs) using only the abstract languageTSP1;

· FEs, which are the second level, are mediationdevices. Each FE is a functional entity that re-ceives the TSP1 protocol data units (PDUs)from the SS and translates them to the con-trolled third-level equipment (Testers) usingthe appropriate proprietary language, and vice-versa. The control language for each tester con-trolled by each FE is normally referenced

conventionally with the name ``TSP2''. ActuallyTSP2s are a family of proprietary languages,each of which is likely to evolve independentlyfrom the others, according to the technicaland/or commercial decisions of each testerÕsmanufacturer.

· Testers (Txx). Testers are at the third level, theyare the concrete testing machines that receivetheir own TSP2 messages ``from their FE'' andperform the requested actions; they may sendacknowledgements and other signals to theirFEs (always using their TSP2).

The TSP1 is speci®ed as an application layerprotocol, so any kind of transport network(s), withtheir own protocol stacks, can be used to transportTSP1 PDUs (i.e., TSP1 messages) between a SSand its FEs.

4.2.4.2. TSP1+ as a standard. The TSP1 speci®-cation published by EURESCOM (P412) has alsobeen adopted by ETSI as ETR 303. In the sametime frame, a European project named `ÌNTOOLopen testing environment (OTE)'' had been laun-ched by the European Commission with the ob-jective to open ± as much as possible ± the testingtool market. Starting from TSP1 speci®cationpublished in ETR 303, INTOOL produced anenhancement of the protocol named PMI [17,18],whose purpose was to extend TSP1 to the testde®nition/speci®cation phases, and to improveTSP1 e�ciency, during execution.

After the publication of PMI, the originalTSP1 speci®cation was further improved (insideETSI TC MTS ± methods for testing and speci-®cation) importing some feature and conceptsfrom PMI, and simplifying/clarifying some otherparts of the previous TSP1. The new ETSI workitem was for a new ®nal protocol, named TSP1+,to be a test synchronisation ETSI standard. Atthe time this document is being written, TSP1+ isunder going the approval procedure in ETSI TCMTS.

The goals of TSP1+ are:· Automate the NIT execution.· To support the concept of ``virtual tester'' for

the connection of new testers to a given testingplatform. In the TSP1+ speci®cation, all kindsof Testers have been taken into account: fromFig. 8. The TSP1 architecture.


those with only the very basic hardware/®rm-ware on board to those fully programmableand most complex. TSP1+ is now a powerfulbut ¯exible protocol.

· To facilitate merging of TTCN and TSP1 forautomatic concurrent TTCN translators usablein geographic environments (i.e., for NIT pur-poses).

As already said, the SS and the FEs are functionalentities. In particular, a FE can be implementedand embedded directly in physical Testers. Fig. 9illustrates the di�erent possibilities for allocatingthe FE functions. Obviously the appearance ofTSP1+ native testers would result into a greatcontribution to the opening of the testing toolmarket.

4.2.4.3. An example of linked usage of both TTCNand TSP1+. A simple example of TSP1+ use inautomatic test execution is shown in Fig. 10. Let usimagine having an ATS in concurrent TTCN, andto have produced from that an ETS for a distrib-uted test execution. The main test component runsin the SS while the parallel test components areexecuted in the protocol testers.

After the preliminary initialisation phase, asimple test case execution takes place in the fol-lowing manner: in running the main test compo-

nent the SS executes the TTCN CREATEkeyword (associated to an identi®er of the testcomponent to be actually run, in our example,B_110101). The e�ect is that the SS sends a TSP1+CREATE message to the FE, which translates itinto the proprietary set of appropriate instructionsfor the speci®c protocol tester to execute the se-lected parallel test component. It can be imaginedthat the SS also asks in parallel another PT toexecute a corresponding di�erent test component,e.g., A_110101. When the test case ends, with otherproprietary means the tester communicates thisevent to its FE, which in turn generates a standardTSP1+ VERDICT message towards the SS (whichcloses the main test component with the TTCNkeyword DONE).

4.2.4.4. Early NIT tools available on the market.The availability of validated test speci®cationswith an international context gave a push to thetesting toolmakers to produce ETSs according tothose speci®cations [31]. Such speci®cations, act-ing de facto as a reference, contributed tothe creation of a market (for testing equipment).The ®rst commercial NIT ETS coming from themarket, as far as the authors of this paperare aware, were the ISDN end-to-end, producedby SIGOSâ, Tektronixâ, and Clemmessyâ. In

Fig. 9. TSP1+ as an open testing environment solution.


addition, a TSP1 implementation following ETR303 was made available by Clemmessy, and otherswere made available by Alcatel TITN and AlcatelSTR (Wandell&Golterman) following the PMIspeci®cation; an implementation of the TSP1+speci®cation is being considered by Tektronixâ.

5. The NIT extensions to broad-band and GSM: the

P613 results

Between 1996 and 1999, the EURESCOMproject P613 developed NIT test suites, spanningthe areas of narrow-band and broad-band testing[32]. On the broad-band side, P613 extended theP410 NIT results providing an integrated ``pack-age'' of test speci®cations for the veri®cation of theEuropean ATM network, both switched (involv-ing signalling) and non-switched. On the narrow-band side, new narrow-band NIT test suites addedgreater coverage therefore supporting the intero-perability of services delivered by ®xed (ISDN orPSTN) and mobile (GSM) interconnected net-works.

In particular, P613 provided:· Node-to-node test speci®cations for broad-band

NNI protocol (B-ISUP) and for inter-workingbetween broad-band and narrow-band NNIprotocols (B-ISUP against N-ISUP) [27].

· End-to-end test speci®cations for broad-bandUNI protocol (B-ISDN), for inter-workingbetween broad-band and narrow-band UNIprotocols (B-ISDN against N-ISDN) and forIP over ATM [26,28,29].

· End-to-end test speci®cations for inter-workingbetween N-ISDN and GSM UNI protocols [30].

· Maintenance of the P412 test speci®cationsbased on comments received and evolutioninside standardisation bodies [30].

· Test speci®cations for inter-working betweenATM and FR protocols [26].

5.1. B-ISDN end-to-end ATS

The main objective of the end-to-end B-ISDNtest speci®cation [28] is to assure the end-to-end compatibility of the protocol mappings andcall control in a user-to-user B-ISDN/B-ISDN

Fig. 10. Example of TTCN and TSP1+ use in automatic test case execution.


environment and in a B-ISDN/N-ISDN con®gu-ration using the appropriate ISUP in between. TheATS can therefore be used for testing Interna-tional B-ISDN between network operators;moreover the ATS includes the inter-working testsbetween broad-band and narrow-band ISDNs asmentioned before.

The test speci®cation covers the B-ISDN accessdescribed in Q.2931 by ITU as well as the UNI 3.1described by ATM Forum.

The proposed test speci®cation for B-ISDN andN-ISDN assumes that each B-ISDN access im-plementation corresponds to Q.2931 or UNI 3.1,the N-ISDN access implementation correspondsto Q.931; the B-ISUP is modeled according toQ.2761±Q.2764 and the N-ISUP follows Q.761±Q.764.

The following test subjects are covered:· Support of basic services (CS1 and partly CS2.1

functionality): normal call/connection includingbearer services, HLI/LLI-transport and LLI-ne-gotiation, unsuccessful call set-up, normal callrelease, deterministic bit rate, statistical bit rate,point-to-multipoint and bandwidth negotiation.The support of bandwidth modi®cation andavailable bit rate are foreseen to be covered inthe project extension.

· Support of supplementary services: CLIP/R,COLP/R, SUB, UUS, CUG.

· Support of inter-working between B-ISDN andN-ISDN: basic call including normal connec-tion, unsuccessful call set-up, normal call re-lease.

5.2. B-ISUP node-to-node ATS

The main purpose of the node-to-node testspeci®cation was to be used for testing interna-tional B-ISUP links between European or inter-national network operators; moreover the testspeci®cations include the inter-working betweenbroad-band and narrow-band ISUP [27].

The node-to-node test speci®cation provides aset of tests for testing B-ISUP compatibility and itsinter-working with N-ISUP referring to basic andsupplementary services, checking basically thenode-to-node characteristics. The following testsubjects are covered:

· Support of basic services (CS1 and CS2.1 func-tionality): call control (normal connection,unsuccessful connection set-up, normal connec-tion release, propagation delay determination),maintenance control (reset, blocking and un-blocking, consistency check procedures), com-patibility (message and parameter), timers,bandwidth negotiation, bandwidth modi®ca-tion, statistical bit rate, point-to-multipoint,ATM end system address, frame relay (FR), net-work generated session identify.

· Support of supplementary services: operation in anational or International environment (UUS,CLIP/R, SUB, COLP/R, CUG).

· Support of inter-working between B-ISUP andN-ISUP: basic call (normal connection, unsuc-cessful connection set-up, normal connectionrelease, propagation delay determination, sus-pend and resume) and supplementary services(CLIP/R, COLP/R, SUB, CFB-CFNR-CFU-CD, CW, CONF-3PTY, UUS).

5.3. Testing of data services (ATM±FR)

P613 produced NIT speci®cations for data ser-vices for the case of the ATM and FR technologies[26,29]. But before presenting such results, it maybe worth recalling some characteristics of suchservices.

ATM and FR are two leading technologies forWAN applications. ATM was sometimes identi-®ed as an all-in-one technology, incorporatingbackbone, LAN and WAN into one network ar-chitecture. FR sometimes has been considered asan interim network solution toward full imple-mentation of ATM. Actually the two technologiesare complementary and not alternative: FR iswidely deployed for providing WAN services,while ATM is a mature technology for the WANbackbone.

Each network technology has its own functionsand ways to provide network services: therefore,inter-working between the two technologies isneeded, and NIT approaches may apply when itcomes to testing.

Inter-working between ATM and FRmeans using an inter-working function (IWF),which is a set of de®nitions standardising how a


device connecting two networks (i.e., FR andATM) performs the translation from one serviceto another. Translation deals with a certain num-ber of issues, such as address translation, frameformatting and delimiting, tra�c parametersmapping, congestion indication translation, upperlayer protocol encapsulation, connection set-upand permanent virtual circuit (PVC) managementprocedures translation, etc.

The inter-working between FR and ATM canbe mainly realised in two ways: network inter-working and service inter-working. In each case,the inter-working can be done by PVC or byswitched virtual connection (SVC).

5.3.1. Case 1 ± FR access and ATM network inter-working

Network inter-working applies when a FR ser-vice user inter-works with a second FR serviceuser, using an ATM bearer service as a transportbackbone.

The FR users perform no ATM speci®c func-tions. All inter-working is done by the IWF, which

is responsible for mapping WAN tra�c into theATM transport service. Two equal IWFs, one oneach side of the ATM network, allow FR users toaccess the network and send their data withoutimplementing ATM speci®c functions. The refer-ence scenario for network inter-working, as de-®ned by FR Forum in [34], is shown in Fig. 11.

5.3.2. Case 2 ± ATM and FR service inter-workingService inter-working applies when a FR user

inter-works with an ATM service user. The FRservice user performs no ATM speci®c functionsand the ATM service user performs no FR speci®cfunctions. All inter-working is performed by theIWF. The IWF is responsible for mapping FRtra�c into ATM tra�c. Fig. 12 shows the scenariofor service inter-working between FR and ATMservices.

In the case of service inter-working the ATMservice user has no knowledge that the distant useris a FR user, and vice-versa: this means that theinterconnected network is service transparent.In some cases (e.g., LAN), higher layer tra�c

Fig. 11. Network inter-working scenario between ATM and FR.


encapsulation may not be the same on both net-works. Inter-working must also translate betweendi�erent higher layer encapsulation schemes andheaders.

5.3.3. ATM and FR (PVC) service inter-workingtest speci®cation

The test speci®cation produced by P613 for thiscase is an end-to-end ATS, this time covering dataservices. TTCN was not used for such ATS. Themain purpose of the ATM/FR inter-working ATSis the veri®cation of the PVC service inter-workingbetween FR and ATM technologies.

The ATM/FR PVC service inter-working ATSprovides a set of tests for testing interoperabilityof an ATM service user with a FR service user.The SUT is the IWF, as introduced in the pre-vious paragraph. The test speci®cation applies toPVC services. The following test subjects arecovered:· Support of tra�c mapping.· Support of discard eligibility to cell loss priority

mapping.· Support of congestion management mapping.· Support of PVC management procedures inter-

working.

· Support of upper layers user±protocol encapsu-lation mapping.

The ATS does not cover capacity and performanceaspects, i.e., load testing to determine performancewith various con®gurations. As well, it does notapply to inter-working by SVC.

5.4. GSM ISDN PSTN ATS

The P613 project released also a validated end-to-end ATS to cover mobility (GSM) [30]. Theend-to-end actually covered GSM services also ininter-working situations with ISDN or POTSusers. The end-to-end ATS covers NIT betweenGSM±GSM, GSM±ISDN, ISDN±GSM, GSM±PSTN, and PSTN±GSM. The narrow-band (mo-bility + ®xed) ATS, written in concurrent TTCN,actually covers:· Basic services: call establishment and clearing

both from an GSM calling user to a ISDN userand from a ISDN calling user to an GSM userand GSM±GSM calls; transport of ``cause''information in all such cases.

· Supplementary services: those supported by in-ternational ISUP (Q.767) signaling links.

Fig. 12. Service inter-working scenarios between ATM and FR.


· Combinations of supplementary services.· Non-symmetrical tests.In particular, the call establishment between aGSM calling user and an ISDN user is checkedfor many bearer services: ``speech'', ``3.1 KHz exPLMN'', `ÙDI'', ``facsimile G3'', ``speech fol-lowed by data'', `àlternate speech/fax'', `àlternatespeech/data'', `èmergency calls''.

On the mobile side, the test suite is applicableeither to the radio interface or to the A interface(the interface between a BSC and an MSC in theGSM architecture reference model. This approachallows the mobile operator to use the same ATSwith and without a BSS, i.e., allowing separationof BSS and MSC problems).

5.4.1. The reference network under testWith the test speci®cation provided, the refer-

ence network under test can take into account notonly the ®xed network but also the GSM NE asshown in Fig. 13.

5.4.2. The methodology usedA peculiarity of the ATS produced is its ap-

plicability to the end-to-end and to the inter-working methodology (end-to-end applies whentesters are located at the used accesses, and inter-working applies when at least one the tester, amonitoring point, is located `ìnside'' the net-work). Most often, for the same protocol is notpossible to apply the same ATS in both cases

because access and network protocol are toodi�erent.

In case of GSM, as shown in Fig. 14, two dif-ferent interfaces have been identi®ed using thesame protocols. In fact, in the radio and A pro-tocol stack, there are the common layers CM andMM, so using a MM SAP, exchanging the CMPDUs, it is possible to apply the same ATS onboth interfaces.

This feature allows the use of the same ATSwith di�erent testing tools, depending on theavailability of them or the actual needs and goalsof testing. In fact, the same test can be executed forGSM user from the end point, addressing the userpoint of view, and from the A interface, allowingto the operator to separately take into account theBSC part from the MSC in order to ®nd out spe-ci®c inter-working problems.

Simpli®ed descriptions of possible interactionsamong real testing equipment involved in a P613end-to-end ISDN to GSM test call, co-ordinatedusing TSP1+, is shown in Fig. 15.

5.4.3. NIT methodology extensionThe NIT approach, as de®ned in EURESCOM

P412 and ETSI [9,13], is used to verify complexinter-working of protocols as far as the real net-work performance is concerned. NIT basic meth-odology focuses on the signalling aspects.

But the load of each single node generally af-fects the performance of the whole network: some

Fig. 13. P613 GSM ISDN PSTN implementation under test.


Fig. 14. Methodological approach the GSM end-to-end testing.

Fig. 15. An ISDN to GSM end-to-end test call, using TSP1+ for the test execution co-ordination.


tra�c condition can stress an exchange, whichbecomes ``the bottleneck'' of the whole networkand reduces the quality of service o�ered to cus-tomers. NIT suggests evaluating the network per-formance under signi®cant tra�c stimuli.

To get performance information it is not nec-essary to use new NIT test cases; it is often enoughto indicate conditions under which the functionaltests have to be run. This could mean that a testoperator has to de®ne/create the tra�c load, whicheach node has to carry out.

So, integration between functional and loadtesting is a natural compromise between accuracyof veri®cation and costs and an easy means toevaluate network performance over load condi-tion. End-to-end executions take place overbackground tra�c (an appropriate mix of di�erentcalls or call attempts), which must reproduce realtra�c conditions.

This testing may temporarily a�ect the integrityof some nodes. For this reason, this NIT extensionrequires tests execution to take place in test plants(may be in a network of test plants). Statisticalapproaches may be used to evaluate the resultsof this type of testing, and to emit a verdict (i.e.,result OK, or acceptable, if less than 10E)6 callsare badly served in a 24 h test run).

5.5. Validation of P613 test speci®cations

The same validation concepts used in EURES-COM P412 were applied to P613. The results ofthe validation activities have been the following:· Node-to-node B-ISUP: This has been only for-

mally validated, as there was no test tool avail-able implementing the ATS; the validation hasbeen limited to checking the formal correctnessof the TTCN (using TTCN editors and compil-ers);

· End-to-end B-ISDN: the validation level reachedfor this ATS was limited due to the lack of ap-propriate systems under test supporting all theCS2.1 capabilities, supplementary services andinter-working to N-ISDN facilities;

· End-to-end GSM±ISDN±PSTN: The ATS wasused in many real life NIT executions and itcan be considered of very high quality (verygood validation level).

· For the test speci®cations for inter-working be-tween ATM and FR protocols, only intra-LANsvalidation tests were performed. The ATM Fo-rum UNI 3.1 Signaling was the ATM signalingprotocol used during the implementation ofthe LANs test cases.

During P613, ATSsÕ validation was performedwith the cooperation of toolmakers (implemen-tations of the test speci®cations). Tools fromTektronixâ and HPâ were provided for thebroad-band area. SIGOSâ covered GSM±ISDN±PSTN, and Tektronixâ GSM±ISDN and ISDN±ISDN.

6. From NIT to network interconnection testing

As we have seen, the term NIT is often used todenote network related testing activities, testsuites, administrative procedures, etc., that areperformed and used by an operator for improvedperformance of its NEs, and of its own network asa whole [12].

This operator, for example, may wish:· to ensure that the di�erent NEs within its own

infrastructure are interoperating correctly; or· that its infrastructure is interoperating correctly

with the infrastructure of another non-competi-tive operator which is interconnected on a com-mercial basis for the global provision of some``global'' telecommunication services (e.g.,Euro-ISDN);

· the term Network Interconnection Testing de-notes the testing activities, test suites, adminis-trative procedures, etc., that are performed andused by an operator which must interconnectits network to the network or equipment ofsome other licensed operator or service provid-er, because of some obligation coming from reg-ulation.Both operators may share some interest to

make such interconnection tests:· to assess that the other ``to be interconnected''

network is not `à menace'' for one another net-work, integrity;

· to assess that the interconnection tra�c iswell documented (for detailed accountingreasons);


· to ensure that a minimum level of quality (of theTLC services) exists at the beginning and is like-ly to be maintained.

In any case, the di�erent types of testing activitiesintroduced so far (integration, interconnection),which are generated according to di�erent moti-vations, share quite a few ``components''. Theseare the ATSs, tools, etc., that are needed in orderto test `ìf and how'' two interconnected networksor di�erent NEs integrated in a given networkperform together.

7. NIT as a suppliers products system-integration

methodology

Another use of the NIT test speci®cation staysinside the ``system-integration'' processes of sup-pliers. In fact, the use of NIT test speci®cations,originally designed for the point of view of oper-ators, can help the manufactures in increasing thequality of their products from the viewpoint oftheir customers (the operators).

In the industrial environment, each product isnormally delivered as a stand-alone NE and onlywhen put in operation does it really interact withthe rest of the network. This fact may produceundesirable impacts on the overall service quality(sometimes even on the availability of the serviceitself), e.g., each time a new supplier release isdeployed.

The NIT methodology and tools can thereforehelp improve the ``quality'' of any new product tobe delivered, and become an asset also for manu-facturers. At the minimum these can use NIT inassessing the capability of the new product to in-ter-operate correctly with early versions or releases(even in a single-vendor situation). In this way,they can insure that the performance of the net-work will be not be impaired (at single least withinthe NEs of that vendor in question).

Acknowledgements

A particular acknowledgement is due to all theparticipants in the EURESCOM P104, P412,

P410, P613 projects that contributed to the de-velopment of methodology, test speci®cations andtools for NIT.

References

[1] ISO/IEC 9646-1 Information Technology ± Open Systems

Interconnection ± Conformance Testing Methodology and

Framework ± Part 1: General Concepts, 1994.



Framework ± Part 2: Abstract Test Suite Speci®cation,

1994.



Framework ± Part 3: Tree and Tabular Combined Nota-

tion (TTCN), 1994.



Framework ± Part 4: Test Realisation, 1994.



Framework ± Part 5: Requirements on Test Laboratories

and Clients for the Conformance Assessment Process,

1994.



Framework ± Part 6: Protocol Pro®le Test Speci®cation,

1994.



Framework ± Part 7: Implementation Conformance State-

ments.

[8] ETSI TR 100 000 Information Technology ± Open Systems


Framework ± Part 3: Tree and Tabular Combined Nota-

tion (TTCN), 1999.

[9] ETR 193 Methods for Testing and Speci®cation (MTS);

Network Integration Testing (NIT); Methodology Aspects;

Test Co-ordination Procedure (TCP) Style Guide, 1995.

[10] DES/MTS-00051 Method for Testing and Speci®cation

(MTS) ± Test Synchronization Protocol 1+ (TSP1+)

Speci®cation, 1999.

[11] ETR 303 Method for Testing and Speci®cation (MTS) ±

Test Synchronization; Architectural Reference; Test Syn-

chronization Protocol 1 (TSP1) Speci®cation, 1999.

[12] TR 101 667 Method for Testing and Speci®cation ±

Network Integration/Interconnection Testing, Reasons

and Goals for a Global Testing Approach.

[13] EURESCOM P.412 Deliverable 3: Guidelines for NIT

Session Management: Volume 2: Guidelines and Profor-

mas.


Session Management: Volume 3: End-to-end Test Speci®-

cations.



Session Management: Volume 4: Node-to-Node Test

Speci®cation.


Session Management: Volume 5: Test Synchronisation

Protocol 1 (TSP1) Speci®cation.

[17] INTOOL/OTE EC007 OTE Architecture.

[18] INTOOL/OTE EC026 OTE Piloting Protocol Design

Speci®cation.

[19] ETSI ETS 300 011.

[20] ETSI ETS 300 012.

[21] CCITT G.821.

[22] CCITT I.352.

[23] EURESCOM P.104 Final Deliverable Volume 1: General

Testing Information.

[24] EURESCOM P.412 Deliverable 2: Report on Experience

Gained in Executing P412 NIT Tests for ISDN.

[25] EURESCOM P.412 Deliverable 1: Speci®cation of a

Tra�c Route Test System.

[26] EURESCOM P.613 Deliverable 1: Speci®cation for B-

ISDN NIT: Test Speci®cation for ATM/Frame Relay

Inter-working.

[27] EURESCOM P.613 Deliverable 2 Volume 1: Node-

to-Node Test Speci®cation Speci®cation for B-ISDN NIT.

[28] EURESCOM P.613 Deliverable 2 Volume 2: End-to-End

Test Speci®cation for B-ISDN NIT.

[29] EURESCOM P613 Deliverable 2 Volume 3: IP over ATM

Test Speci®cation.

[30] EURESCOM P613 Deliverable 3: Test Speci®cation for

Narrow-Band Services (GSM, ISDN, PSTN).

[31] EURESCOM P613 Deliverable 4: Evaluation of Test

Execution Experiments.

[32] EURESCOM P613 Deliverable 5: Executive Summary and

Main Findings.

[33] EURESCOM P.410 Deliverable 3: Practical Guidelines for

End-to-End Test Campaigns over ATM networks.

[34] FRF.5- Frame Relay/ATM PVC Network Inter-working

Implementation Agreement, 20/12/95.

Giulio Maggiore was born in Brindisiin 1966. He received the B.S. degree inelectronic engineering from the Po-litecnico di Torino in 1991, and theMaster in Information Technologyand Telecommunication from COREPin 1992. He joined CSELT, theTELECOM ITALIA Group's Com-pany for the study, research, experi-mentation and quali®cation inTelecommunications and InformationTechnology, in August 1992 inSwitching and Network services/Net-work Quali®cation/Functional Quali-

®cation group. After a ®rst experience in testing of ISDNSupplementary Services he was in charge of `Ènd-to-End''

testing phase for opening EURO ISDN links with foreigncountries. He gained experience in network signalling, dealingwith ISDN and GSM network testing. He gained experiencealso in Testing methodology, languages and tools. He has beeninvolved into relevant EURESCOM projects (P412, P613) inthe area of Network Integration Testing for ISDN and GSM,with the role of task leader. He promoted the development andthe adoption of TSP1 (Test Synchronisation Protocol 1) insidethe ``Network Integration Testing'' methodology. Since 1993 heis member of the ETSI TC MTS ``Methods for Testing andSpeci®cation'', where he has been responsible since 1995 for thestandardisation of TSP1+, which has been approved by TCMTS as ETSI standard in march 2000. Since January 1999 hecontributed to create TSG (Test Sub Group) SS (SwitchingSystems), a sub group of CME 20 Ericsson user group. Hebecame the TSG SS project leader on behalf of TIM for the R8release, whose purpose is to introduce GPRS service, which willbe launched in 2000.

Giulio Brusasco was born in Turin in1947. He graduated in electronic engi-neering in 1972 and joined CSELT ±the Telecom Italia Group's Companyfor study, research, experimentationand testing. At CSELT, he gainedexperience on di�erent evolutionaryaspects of the telecommunicationnetwork, including new architecturesand innovative services. He focused, inrecent years, on the related testing as-pects. From 1994 to 1997, he chairedthe ETSI Technical Committee MTS``Methods for Testing and Speci®ca-

tion''. Currently responsible of the CSELT Project `Ìntegration& Engineering of the Switching Platform''.

Marco Vecchiato was born in Rome in1962. After graduating in computersciences he joined CSELT in Decem-ber 1988, where he was involved in the®eld of testing of advanced telecom-munication services. He was involvedin test campaigns for quali®cations ofISDN, DECT, GSM, CCSS#7 signal-ing protocols; research and prototyp-ing work in the ®eld of advanced testsystems (architectural studies, auto-matic test generation from formallanguages); chairman of ETSI techni-cal subcommittee SPS5/Working

Group 3 (Signaling Protocols and Switching/testing of accesssignaling; project leader of the EURESCOM P412 (Methodsand Tools for ISDN Network Integration Testing) and P613(Methods and Tools for B-ISDN and N-ISDN Network Inte-gration Testing) projects. Currently he is responsible for theSwitching System testing activities as a part of the quali®cationof the GSM service, performed by CSELT on behalf of Tele-com Italia Mobile (TIM). From March 2000 he is also re-sponsible for interconnection testing activity between TelecomItalia and Italian OLOs. His current job title is ``TechnicalLeader''.


Documents

Network integration testing: concepts, test specifications and tools for automatic telecommunication services verification