TESTING OF SDI COMPONENTS – A FUNDAMENTAL INTEROPERABILITY

ELEMENT WITHIN INSPIRE AND NATIONAL SDI’S

T. Kliment, M. Tuchyňa, M. Kliment

This presentation has been transmitted remotely as TELE lecture from Pallanza (IT) thanks to “BizBiz-Tool”

Presentation outlines

• Introduction• Basic description of involved organizations within testing

o Slovak Environmental Agency (SEA)o Slovak University of Technology (SUT)o Slovak University of Agriculture (SUA)o Proposed tasks related to SDI components testing

• SDI component to be provided by SEA to INSPIRE&NSDI• Testing methodology• Testing environment proposal• Testing tools • Pilot testing of the SEA SDI components • Testing results summary• Conclusions and future work

Introduction

• European and National SDI should provide data discovery, evaluation and consumption via central point – Geoportal

• Network services are engines to serve data and metadata from repository to central SDI Geoportal

• Network service behaviour is defined by standards& specifications (ISO, OGC) and specified by regulations (INSPIRE)

• Before the NS connection to geoportal testing has to be against above requirements has to be done to ensure interoperability

• Voluntary collaboration established under the auspices of SDI PTB for such purposes

• Similar activities are nowadays problematic due to missing such experience in mandated organization

• Therefore such type of collaboration should increase among all organizations involved in SDI establishment

Basic description of involved organization

• Slovak Environmental Agencyo Public sector bodyo Coordinating SDI development within environmental domain o Contributing to national and international SDI development

implementation via:Metadata system for spatial and non spatial data Network services (discovery, view, download,

transformation, spatial data services Consolidated spatial data repository (Central Geografical

System) Client applications (Environmental geoportal, domain

specific web map clients)o Providing technical support for INSPIRE transposition and NSDI

implementationo LMO & expert representation in IOC TF within INSPIRE

Basic description of involved organization

• Slovak University of Technology - Geodetic departmentso Academic sector organizationo Has a study program geoinformatics in master degree levelo Has defended 2 PhD thesis and has 4 in processing phase that

are related to GIS/SDI domain - data quality, data modelling, web services, GIS usage in specific domains (archaeology, floods, geodesy)

o Performs education and research within GIS/SDIo Implements web map clients, web map services and testing

toolso Performed initial testing of discovery and view services against

INSPIRE requirements presented at conferences (GIS Ostrava, EnviroIForum,

GI2010) SDIC & expert representation in IOC TF within INSPIRE

Basic description of involved organization

• Slovak University of Agriculture – Dept. of Landscape Planning and Ground Designo Academic sector organizationo Has a study program Ground design and GIS in master degree

levelo Dept. is a producer of huge amount of data about land as a

results from ground design projectso Currently the pilot project is related to design and

implementation of IS about hydro physical properties of soils in Slovakia based on SDI principles:o Metadata and discovery services o View and download serviceso Processing services for spatial analyses

SDI components provided by SEA to INSPIRE/NSDI

• SEA covers data themes from all 3 INSPIRE annexes• Annex I – Hydrography, Protected sites• Annex II – Land Cover • Annex III – Biogeo regions, Habitates&Biotopes, Species

distribution

Network service type

Annex I Annex II Annex III

Discovery service YES

(terra catalog CSW 2.0.2)

YES(terra catalog

CSW 2.0.2)NOT YET

View serviceYES

(ArcGIS Server WMS 1.3)

YES(ArcGIS Server

WMS 1.3)

YES(ArcGIS Server

WMS 1.3)

Download service YES

(ArcGIS Server WFS 1.1)

YES(ArcGIS Server

WFS 1.1)

YES(ArcGIS Server

WFS 1.1)

Maps served by view service (WMS) Data served by download service (WFS)

Metadata for datasets&services served by discovery service (CSW)

Testing methodology

Testing Coverage

Testing scope

Testing temporalextent

Testing scenarios

Testingperformance

Testing reporting

INSPIRE, ISO, OGC

Complex/partial testing model

Short/long term

Conceptual/applicationdesign, testing tool

Service interface/quality/content

Report template

Testing environment proposals act SUTSEAT_testing

SDI components

tester SDI components

prov ider

testingEnv ironment

publishNetworkServ ice

displayResults

Communicate

SetupTestingScenarios

deployTests

reportResults

SelectTestingScenario

sendReportdownloadReport

«invokes»

«invokes»

«invokes»

•Users – SDI component tester, SDI component provider

•Use cases – PublishNS, Define/ConfigureTestingScenario,

SelectTestingScenario, PerformTest, ReportResults, ViewResults

SendResults, SaveResults, Communicate,

Testing tools

• Webtest tool - web application for web service testingo Developed by testers at SUT with Java and JSP technologieso Web GUIo Single, multiple, simultaneous GET and POST requestso Measures times: Between RQ and the first byte of the RS download Between RQ and the last byte of the RS download

o Counts presence of predefined stringo Configuration based on XML files Service endpoint definition Testing scenario definition (request to service)

o Provides results in the tabular formo Does not provide statistics for long term testing yeto Online accessible on: http://geo.vm.stuba.sk:8080/webtest/

Testing tools - webtest

testcase.xml results

Testing tools

• MDValidator - desktop application for batch metadata validation

o Developed by testers at SUT as Java application

o Desktop GUI

o Performs batch validation of MD as XML files from local dir

o Invokes online REST Web INSPIRE validator service

o Provides results in XML/HTML form

o Does not validate against ISO gmd schema yet

o Useful after batch MD transformation (xslt)`

Testing realization - discovery service

• Testing coverage - INSPIRE requirementso Search criteria - 25 queryableso Operations - 4 operations with predefined parameterso Quality of Service - 3 parameters

• Testing scopeo Tested 3 operations (DiscoverMD, GetDSMD, PublishMD) and

23 queryableso Estimated 2 quality parameters (Performance, Capacity), MD

• Temporal extento Short term testing - all predefined testing scenarios launched

few times• Testing scenarios + tools

o One for queryables+performance+DiscoverMD operationo One for capacity as combination of operationso One for Publish MD operation+performanceo webtest+MDValidator

Testing realization - view services

• Testing coverage - INSPIRE requirementso Operations - 3 operations with predefined parameterso Quality of Service - 3 parameters

• Testing extento Tested 2 operations (GetMap, GetVSMD)o Estimated 2 quality parameters (Performance, Capacity)

• Temporal extento Short term testing - all predefined testing scenarios launched

once• Testing scenarios + tool

o One for GetMap operation+performance estimationo One for capacity as combination of operationso webtesto

Testing realization - download services

• Testing coverage - INSPIRE requirementso Operations - 4 operations + 2 more for spatial object accesso Quality of Service - 3 parameters

• Testing extento Tested 3 operations (GetSDS, GetDSMD, DescribeSDS)o Tested 2 parameters of QoS

• Temporal extento Short term testing - all predefined testing scenarios launched

once• Testing scenarios + tool

o One for GetSpatialDataSet+performance&capacity estimationo One for DescribeSpatialDS and GetDSMD+performanceo webtest

Testing results summaryReport template structure proposals 1/2

Tested service Discovery

service View

serviceDownload

service

Interface (operations&paramters)

Results

GetDSMetadata SupportedParameters no tested

SupportedParameters no tested

SupportedParameters no tested

DiscoverMetadata SupportedParameters no tested

PublishMetadata SupportedParameters no tested

LinkService No tested No tested No tested

GetMap SupportedParameters no tested

GetSpatialDataSet SupportedParameters no tested

DescribeSpatiaDataSet SupportedParameters no tested

GetSpatialObjectNo tested

DescribeSpatialObjectNo tested

Testing results summaryReport template structure proposals 2/2

Tested service Discovery

service View

serviceDownload

service

Quality of service Results

PerformanceSatisfied

(115 requests sent

115 responses < 3s)

Satisfied on 90%(30 requests sent 27

responses < 5s)

Satisfied(30/10 requests sent

30/10 responses <

10/30s)

Capacity Satisfied(30 simultaneousrequests sent 30

responses < 3s)

Satisfied on 70%(20 simultaneous requests sent 14

responses < 5s)

Satisfied(10 simultaneous requests sent 10

responses < 30s)

Availability No tested No tested No tested

Other criteria Results

Search criteria Supported23/25 criteria tested

Search Criteria for the Get Spatial Object Operation

No tested

Content of the service Results

Metadata modelsNo tested

Data modelsNo tested

Conclusions

• Initial short-term testing has brought first and quite positive results• Collaboration between public and academic sector is important• Testing methodology needs high knowledge and understanding of

all requirements defined in related regulations• Testing scenarios preparation is important and consumes much

time• Testing results should be easily interpreted and understandable• Where possible and appropriate various levels of compliancy can

be introduced

Future work

• Discussions, proposals, suggestions on Testing methodology• Testing scenarios extensions

o For all required operations (Link NS)o RQ + RS parameters validation against INSPIRE specific

constraints (e.g. NS INSPIRE MD within GetNSMD response, Language parameter, Layer MD, Spatial Dataset MD)

o Long term testing scenarios for accurate service quality parameters estimation

o documentation of individual scenarios (conceptual level)• Testing report template

o Discussions and decisions on the form and content of the reports - (e.g. tabular form with information as Date, Test description, Test execution, Results, Pass/Fail definition, comments ....)

Future work

• Testing environment extensionso Functions for reporting exports, statistics calculations, plotso Testing series implementation (e.g. INSPIRE discovery service

testing scenario, ...)o Results storage in form of database to avoid loss of results in long term

• Testing of local spatial data and services compliancy against INSPIRE Annex II+III data specificationso Feasibility testing (Annex III)o Fitness for purpose testing

• Efforts to promote testing & validation (awareness rising)• Efforts to formalise testing & validation (proposal for

establishemnt of common testing platform allowing sharing tools, materials, methodologies, experience and expertice related to SDI components testing)

Thank you very much!

Contact info:SUT in Bratislava, Dept. of Theoretical GeodesySEA in Banská Bystrica, Dept. of Environmental informaticsSUA in Nitra, Dept. of Landscape Planning and Ground Design

tomas.kliment@gmail.commartin.tuchyna@gmail.com

marcel.kliment@uniag.sk