PROCESSINGDSLSPECIFICATION

‐BASELINEVERSIONDeliverableD2.4.1

Circulation: PU:PublicLeadpartner: FraunhoferContributingpartners: Authors: MichelKrämer,IvoSennerQualityControllers: HeinoRudolf,NormanKießlichVersion: 1.0Date: 30.04.2014

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

1

©Copyright2014:TheIQmulusConsortium

Consistingof

SINTEF STIFTELSENSINTEF,DepartmentofAppliedMathematics,Oslo,NorwayFraunhofer FraunhoferInstituteforComputerGraphicsResearch,Darmstadt,GermanyCNR‐IMATI‐GE InstituteforAppliedMathematicsandInformationTechnologiesofthe

NationalResearchCouncil(CNR‐IMATI),Genova,ItalyMOSS M.O.S.S.ComputerGrafikSystemeGmbH(MOSS),Munich,GermanyHRW HRWallingfordLtd(HRW),Wallingford,UKFOMI HungarianNationalMappingandCadastralAgency(FOMI),Instituteof

Geodesy,CartographyandRemoteSensing,Budapest,HungaryUCL UniversityCollegeLondon(UCL),ResearchcentreforPhotogrammetry,3D

ImagingandMetrology,London,UKTUDelft DelftUniversityofTechnology(TUDelft),DepartmentofEarthandClimate

Sciences&Man‐MachineInteractionGroup,Delft,TheNetherlandsIGN InstitutNationaldel’InformationGéographiqueetForestière(IGN),Paris,

FranceUBO UniversitédeBretagneOccidentale(UBO),EuropeanInstituteforMarine

Studies,Brest,FranceIfremer L’InstitutFrancaisdeRecherchepourl´ExploitationdelaMer(Ifremer),

Brest,FranceLiguria RegioneLiguria,Genova,Italy

Thisdocumentmaynotbecopied,reproduced,ormodifiedinwholeorinpartforanypurposewithout written permission from the IQmulus Consortium. In addition to such writtenpermissiontocopy,reproduce,ormodifythisdocumentinwholeorpart,anacknowledgementoftheauthorsofthedocumentandallapplicableportionsofthecopyrightnoticemustbeclearlyreferenced.

Allrightsreserved.

Thisdocumentmaychangewithoutnotice.

DOCUMENTHISTORY

Version1 IssueDate Stage ContentandChanges0.1 2014‐04‐26 Draft Earlydraftsentoutforreview0.2 2014‐04‐27 Draft Updateddraftforreview(urbanshowcase

complete,marineshowcasecomplete)0.3 2014‐04‐28 Draft Finaldraftforreview(landshowcasecomplete,

documentisnowconsistent)0.4 2014‐04‐29 Draft ReviewbyHeinoandNorman1.0 2014‐04‐30 Final Incorporatedcommentsfromreview,submitted

finalversion

1Integerscorrespondtosubmittedversions

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

2

EXECUTIVE SUMMARY

Thisdocumentcontainsthebaselinespecificationofadomain‐specificlanguageusedtodefinegeospatialprocessingworkflows.Westartwithadescriptionofourmethodtocreatedomain‐specificlanguages(DSLs).AfterthatwegothrougheachofthethreeIQmulusshowcases(urban,marine,land)andperformtheactualDSLmodelling.Wecreateaprototypicalgrammarthatcanbeused inthecontrolcomponentsdescribed indeliverableD3.2ControlComponents–verticalprototyperelease, inparticular theworkfloweditor andDSLparser.Thisdocument concludeswithadditionalnotesontherationaleforthegrammar/syntaxwechoseforourDSL.

‘BigDatahandling’considerations:

TheDSLcontainshigh‐levelandlow‐levelstatements.

Usinghigh‐levelstatementsuserscanspecifyprocessingworkflowsinawaythatisveryeasytoreadand learn.Thesekindsofstatementsaretargetedtodecisionmakersornon‐GISexperts,butexpertusersmayusethemaswellifthey,forexample,needaquickwaytoperformqueries.Thehigh‐levelstatementsfocusmoreontheendresultsinsteadofontheprocess.Usersdonotneedtoknowspecificsoftheinfrastructuretheworkflowisexecutedon.Inaddition,withhigh‐level statements users do not have to knowwhich data is available in the system and whatprocessingalgorithmshavetobeusedtoproduceacertainresult.Thisinformation/knowledgeisgenerated/deducedbythesystemitselfbasedonrulesspecifyingdeclarativeandproceduralknowledge(seedeliverableD3.2Controlcomponents–verticalprototyperelease).

Thelow‐levelstatementscanbeusedbyexpertswhowanttospecifyexactlythealgorithmsthatshouldbeexecuted,inwhichordertheyshouldrun,aswellasonwhichdatathesealgorithmsshould operate. Still these expert users do not need to know specifics of the infrastructure(number of cloud nodes, operating system, data location, process service location, etc.). Thisknowledgeisencodedinproceduralrules(seedeliverableD3.2).

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

3

TABLE OF CONTENTS

Executivesummary...........................................................................................................................................................2 

1  Introduction................................................................................................................................................................4 

2  DSLModelling............................................................................................................................................................4 

2.1  Relatedapproach............................................................................................................................................5 

3  ProcessingDSL‐Baselinespecification..........................................................................................................6 

3.1  Urbanshowcase..............................................................................................................................................6 

3.1.1  Vocabulary...............................................................................................................................................6 

3.1.2  DomainModel........................................................................................................................................7 

3.1.3  ProcessRelations..................................................................................................................................7 

3.1.4  PrototypeDSLworkflow....................................................................................................................9 

3.2  Marineshowcase.........................................................................................................................................12 

3.2.1  Vocabulary............................................................................................................................................12 

3.2.2  DomainModel.....................................................................................................................................13 

3.2.3  ProcessRelations...............................................................................................................................13 

3.2.4  PrototypeDSLworkflow.................................................................................................................14 

3.3  Landshowcase..............................................................................................................................................15 

3.3.1  Vocabulary............................................................................................................................................15 

3.3.2  DomainModel.....................................................................................................................................16 

3.3.3  ProcessRelations...............................................................................................................................16 

3.3.4  PrototypeDSLworkflow.................................................................................................................18 

4  RationaleforthechosenDSLSyntax.............................................................................................................19 

5  References................................................................................................................................................................20 

6  Appendix:ParsingExpressionGrammarfortheDSL............................................................................20 

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

4

1 INTRODUCTION

Deliverable D3.2 ControlComponents – verticalprototype release addresses a tool chain thatevaluatesgeospatialprocessingworkflowsspecifiedbyexpertusers.ThatdeliverabledescribesaworkfloweditorthatmakesuseofaDomain‐SpecificLanguage(DSL)toallowuserstospecifyworkflowsataveryhighlevelwithoutrequiringthemtohavedeepknowledgeofthespecificsoftheinfrastructuretheworkflowwillbeexecutedon.

InthisdocumentwespecifythesyntaxandgrammarfortheDSLusedintheworkfloweditor.WefirstdescribeamethodforDSLmodellingandthenapplyittothethreeIQmulusshowcases(urban,marine,land).

Pleasenote that theDSLspecificationpresentedhere isabaselineversion.Thismeans that itmaybesubjecttochangeasitisbasedonthecurrentdescriptionoftheIQmulusshowcases(asofApril2014).Anupdatedfinalversionofthisdeliverablewillbereleasedlater(seeD2.4.2DSLspecification–finalversiondueinApril2015).

Alsonotethatthisdocumentonlycontainsthelanguagespecificationbutdoesnotgivedetailson how the language is actually parsed or interpreted. This information can be found indeliverableD3.2Controlcomponents–verticalprototyperelease.

2 DSL MODELLING

Domain‐specificlanguages(DSLs)arelanguageswiththefollowingproperties(Krämer&Stein,2014):

Theyaretailoredtoaspecificapplicationdomainoreventoasingleusecase; Thelanguage’svocabularycontainswordswellknowntothedomainexpert; Thelanguage’sexpressivenessisratherlimited.

The lattermeans the language cannot be used for general purposes—that iswhy it is indeedcalledadomain‐specificlanguage—butinsteaditisaloteasiertounderstandandtousefornon‐ITpersonnel.

The IQmulus workflow editor allows users to specify geospatial processing workflows via adomain‐specific language. The DSLs used in IQmulus have the following properties (seedeliverableD2.3.1IQmulusarchitecture–baselineversion):

They are domain‐specific in the sense that they are tailored to describing geospatialworkflows.

Theyaredomain‐specificinrespecttotheusecaseorscenariotheyareusedin(i.e.thethreeIQmulusshowcasesforurban,marineandlandapplications).

In order to create an IQmulus processing DSL that meets these requirements, we use thefollowingmodellingmethod.We start with a typical approach from object‐oriented softwareengineering, consisting of text analysis and domainmodelling. From thiswe then derive DSLsamplescriptsandfinallycreateasyntaxandgrammar.Indetail,ourmethodisasfollows.

1. Analysetheapplicationdomain.2. Createscenarios/storyboards(showcasedescriptions).3. Analysestoryboardsand look for relevant subjects,objects, adjectives,andverbs.This

analysisprovidesthebasisforthedomainvocabulary.

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

5

4. Createadomainmodelusingsubjectsandobjectsfoundinthestoryboardsasclasses.5. Analyse process relations to identify relevant verbswhichwill become actions in the

DSL.6. BuildsampleDSLscriptsbasedonthemodelleddomain.7. DeriveformalizedgrammarandsyntaxfromthesampleDSLscripts.8. Reviewandreiterateifneeded.

Thefirststep(domainanalysis)hasalreadybeendoneintheIQmulusproject.Theresultsaresummarized in the deliverables D1.2.1 InitialUserRequirements and D1.2.2 ConsolidatedUserRequirements.

Itiscrucialthatlanguagemodellingisperformedinstrongcollaborationwithdomainusers,inorder to make sure the final language contains the vocabulary that is actually used in thetargeteddomainandcaninfactbeunderstoodbythedomainexperts.IntheIQmulusprojectwethereforecreatedashowcasetaskforcewiththeresponsibilitytocreateadetaileddescriptionofthethreeshowcasesurban,landandmarine(step2ofourprocess).Theresultofthiseffortwasadocumentthatweanalysedindepth(step3).

2.1 RELATEDAPPROACH

Krämer and Stein present another modelling method that helps create domain‐specificlanguageswhichareeasytounderstandanduse(Krämer&Stein,2014).Themethodconsistsofthefollowingsteps(citedfromtheoriginalpaper):

1. Analysetheapplicationdomain.2. Createscenarios/storyboards.3. Analysestoryboardsand look for subjectsandobjects.Createanontologyanduse the

subjectsandobjectsasconcepts.4. Look for verbs.Use them in the ontology as relations to connect subjects and objects.

Freeverbsthatarenotrelatedtoconceptsbecomeactionsinyourlanguage.5. BuildsampleDSLscriptsthatusethecreatedontologyandthefreeverbs.6. Reviewandreiterateifneeded.

Krämer and Stein use domain ontologies to identify the vocabulary of their DSLs. Such anontologyisasetofconcepts(thingsthatexistinthatdomain)aswellastheirclassificationandrelations(Nicola,Missikoff,&Navigli,2009).

Thedefinitionofaformalontologyisconsideredanessentialstepfordomain‐specificlanguagedesignoreven foranysoftwareproject(Gaševic,Djuric,&Devedžic,2009).Ontologiescanbevery useful for the definition of domain‐specific languages where they act as the basis fromwhichthevocabularyandpartsofthegrammararederived.

Analysing the IQmulus showcase descriptions results in a large number of concepts andrelations.CreatingontologiesforallshowcasesdoesnothelpmodeltheIQmulusprocessingDSLas ontologies are typically rather generic and cannot be translated to actual software code aseasyasdomainmodelswhicharebasicallytailoredforthatpurpose.ThemismatchbetweenthegenericityofontologiesandthespecificityofdomainmodelshasalsobeenrecognizedbySpynsetal.(Spyns,Meersman,&Jarrar,2002).Theybasicallyclaimthat, inordertobevaluableandreusable,ontologiesshouldbe“asmuchgenericandtask‐independentaspossible”,whichmeanstheyshouldbeshareable,portableandinteroperable.OntologiesarealsooftenusedintheareaoftheSemanticWebwheretheyactasthebasisforSemanticReasoningandrelatedtechniques.Thisiswherethegenericityandportabilitycomesinveryhandy.Domainmodelsontheother

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

6

handareratherspecific toausecaseorapplication.For the IQmulusprocessingDSL,domainmodels aremore appropriate than ontologies as it is not planned to reuse the concepts andrelations inanyothersoftware.Furthermore, thespecificityofdomainmodelsallowsadirectmappingofconcepts(i.e.classes)toactualcode(i.e.elementsintheDSL).

3 PROCESSING DSL ‐BASELINE SPECIFICATION

This section describes the process of creating the DSL and the workflows related to theshowcases.TheDSL containshigh‐level and low‐level statements.Usinghigh‐level statementsuserscanspecifyprocessingworkflowsinawaythatisveryeasytoreadandlearn.Thesekindsof statements are targeted to decisionmakers or non‐GIS experts, but expert users may usethemaswell if they, forexample,needaquickwaytoperformqueries.Comparedto the low‐level statements, thehigh‐level ones focusmore on the end results insteadof on theprocess.Users do not need to know specifics of the infrastructure the workflow is executed on. Inaddition,withhigh‐level statementsusersdonothave toknowwhichdata is available in thesystem and what processing algorithms have to be used to produce a certain result. Thisinformation/knowledge is generated/deduced by the system itself based on rules specifyingdeclarative and procedural knowledge (see deliverable D3.2 Control components – verticalprototyperelease).

Thelow‐levelstatementscanbeusedbyexpertswhowanttospecifyexactlythealgorithmsthatshouldbeexecuted,inwhichordertheyshouldrun,aswellasonwhichdatathesealgorithmsshouldoperate.Stillusersdonotneedtoknowspecificsoftheinfrastructure(numberofcloudnodes,operatingsystem,datalocation,processservicelocation,etc.).Thisknowledgeisencodedinproceduralrules(seedeliverableD3.2).

Wedecidedtousethesamedomainspecific language(DSL)forbothkindsofstatements.Thisshouldhelptoincreasetheuserexperienceandreducethelearningtime.

As described in section 2,we start the DSLmodelling by first examining the showcase. Afterextracting the vocabulary which is used, we build a domain model containing all relevantconceptsandhavealookattheprocessingstepsandtheirrelations.Usingthisinformationwecanstarttobuildtwoworkflowsusingthehigh‐levelandlow‐levelstatements,respectively.Inthefollowingwewillusethetermshigh‐levelworkflowandlow‐levelworkflowtodifferentiatethem.When applying the same process to the other showcases,we keep the already createdworkflows in mind to extend the functionality rather than building everything from scratchagain.

3.1 URBANSHOWCASE

Wewillanalysetheurbanshowcasefirstsinceit involvesacoupleoftechniqueswhichcanbereusedlateron.Toexcludemovingobjects(e.g.cars)andidentifydifferentkindofobjects(e.g.treesand facades) featureextractionandclassificationneeds tobeused.Furthermorechangedetectionisusedtodifferentiatebetweenadded,removedandespeciallydeformedobjects(e.g.facades).

3.1.1 Vocabulary

Startingwiththeurbanshowcaseweneedtoextractallinformationwhichhelpsustodescribethe domain and the workflow, respectively. In particular, these are all subjects and objects(concepts),verbsandadjectiveswhicharedirectlyrelatedtotheprocessingscenario.

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

7

Concepts

3DCatalogue Chimney 3DCityModelUrbanTopographicObject FacadeElement PointCloudCar Roof LMMSDataRubbishBin Antenna ImageNon‐StaticObject ChargePoint ColorBike TopographicObject InventoryPeople CableNetwork DistanceStaticObject StreetEdge IntersectionTree UrbanFurniture DatasetBusStop TrafficLight FacadePhoneBooth TopographicMap ChangeDetectionChange Addition RemovalDeformation Verbs

update takeout identifyremove compare determineinclude store co‐registerexclude give colorizemonitor save applyforesee visualize selectAdjectives

certain uncertain cleanstochastic recent addeddeformed

3.1.2 DomainModel

Usingtheabove‐createdlistofconceptsallowsustomodelthedomainwithallitsentitiesandrelations(seeFigure1,nextpage).Theresultallowsustohaveaclearinsightforallelementswhichmightbeinvolvedintheprocessingworkflow.

3.1.3 ProcessRelations

The examination of the urban showcase reveals two main processing issues, namely featureextractionand changedetection.The first one isused to findand classify topographicobjectsfrom the source data, while change detection uses the result and a formerly detected set ofobjectsfromthesameareatoproducealistofthechangesthatoccurred.

FIGURE1DOMAINMODELFORTHEURBANSHOWCASE

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

9

3.1.4 PrototypeDSLworkflow

Startingwiththehigh‐levelworkflow,wecanderiveacoupleoftermsfromthevocabularyandthedomainmodel,whichhasbeencreatedearlier.Actually,therearethreecategoriesofwordssofar.Thetermconceptisusedtodescribeagroupofentitiesorevenaparticularentityofthedomain likeTopographicObjectorBike,whileanadjective isdirectlyrelatedtosuchanobjectanditsproperties,respectively.Obviouslyaverbdescribesanactionwhichshouldbeappliedtoa concept. At this point we introduce a fourth category of words containing keywords of thelanguage itself. These keywords are used to make the language more readable and provideadditional,domain‐independentfunctionality.

Tocreatethefirstworkflowwhichaimstodescribetheprocessingoftheurbanshowcase,weneedtoselectafewconcepts,adjectivesandverbs,whicharelistedinthefollowingtable.

Concept Adjective VerbData recent excludeCityModel added selectNonStaticObjects deformed addtoTrees visualizeFacadeElements Antennas

Furthermorewe introduce the followingkeywords to increaseusabilityand readabilityof theworkflow.

Keyword Descriptionwith[…][…] Usethefirsttermandapplyittothesecond.do[…]end Ablockofstatements,where allofthemareexecutedinthesame

context.Acontextmightcontainvalueswhichwillbeappliedtoallstatementssequentially.

[…]and[…] A simple enumeration of elements. If there are more than twoelements,“and” istypicallyusedin frontofthelastelementonly,whileallpriorelementsareseparatedusingacomma.

Whilewithandandaresimplytermstoenhancethestructureofthelanguage,andhencewrite‐and readability, do […] end is providing more complex functionality. Every block statementcreated by these keywords relates to a context which might envelope a value, i.e. if used incombinationwiththekeywordwith.Whentheblockisexecuted,thevaluestoredinthecontextwillbeappliedtothefirststatementifitexpectsaparameterofthesametype.Ifthisstatementreturnsaresultthecontentofthecontextwillbeoverwrittenbyit.Allsubsequentstatementswillbeprocessedinthesamewayusingtheupdatedcontext.

Usingthesetermsweareabletocreateasimpleworkflow.

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

10

with recent Data do exclude NonStaticObjects select added Trees and deformed FacadeElements add to recent CityModel end with recent CityModel do exclude Antennas visualize end 

Inthisandallfurtherlistings,thetermsarecolorizedaccordingtotheircategory:Lightblueisusedforkeywords,purpleforconcepts,orangeforadjectivesandredforverbs.

ThefirstblockhasbeendefinedwiththekeywordwithtousethemostrecentDataasanimplicitvalue.ThereforethefirststatementexcludesallNonStaticObjectsfromthatdataandreplacestheimplicitvaluestoredinthecontextwiththeresult.Thereforeallsubsequentstatementswillusethemanipulatedone.TheselectstatementacceptsalistofconceptsortobepreciseTopographicObjectsandselectsonlythosefromthedataset,inthiscasegivenbytheimplicitvalue.Thelaststatementwilladdtheresultofthepriorstatement,i.e.select,tothemostrecentCityModel.

Thesecondblockworkssimilartothefirstone.ThefirststatementexcludesallAntennasfromthemostrecentCityModel,whilethesecondvisualizestheresult.

Since a workflow should describe a process which should be applied not only once on aparticular dataset, there has to be a way to design it more flexibly. In order to reapply aworkflow to a number of different datasets, there has to be a dynamic way to address it. Asimpleway todoso is to introduceplaceholders.Theseplaceholdersneed tobereplaced justbeforetheworkflowisexecuted.Hencetheseplaceholderscanbeconsideredasparametersfortheworkflowitself.

Applyingthistotheformerexample,weendupwithaworkflowsimilartothefollowing.

with [Area] do exclude NonStaticObjects select added Trees and deformed FacadeElements add to [CityModel] end with [CityModel] do exclude Antennas visualize end 

TheplaceholderAreawillbeusedtolettheuserspecifyanareaofinterestbeforeexecutingthisparticularworkflow.Asaresult,theJobManagerfromtheprocessingchain(seedeliverableD3.2Control components – vertical prototype release) has to select all data which is needed torepresent this area prior to execution. Analogously the placeholderCityModel can be used tospecifyacitymodelwhichisupdatedwiththeaddedtreesanddeformedfacadeelements.

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

11

Tospecifyalow‐levelworkflowthereisanothercategoryoftermsrequiredwhichmapstothelistofservices.Therefore thiscategorydoesnothaveaspecificnumberof terms,butasmanytermsasservicesavailableintheIQmulusinfrastructure.Thesetermsarecoloredingreen.

Keyword Description[…]with[…] Thesame likewith[…][…] but itapplies thesecondstatement to

thefirst.Ifthereisanapplicableimplicitvalueitwillbecombinedwiththeresultofthesecondstatement.

[…]using[…] Apply key‐value pairs using the form key: value defined by thesecond statement to the first one. It is even possible to use theform {key1, key2, …: value1, value2, …} which might be moreconvenientinsomesituations.

[…]as[…] The result of the first statementwill be stored and accessible inthefutureusingthespecifiedterm.

[…]of[…] Createa subset ofthesecond statement limitedbythefirstone.

apply Intersection3D with recent DataSets using {x, y, z: [X], [Y], [Z]} and {notX, notY, notZ: [NX], [NY], [NZ]} as data with [Area] of data do apply MultiObjectClassification exclude certain NonStaticObjects select certain Trees as trees select certain FacadeElements as facadeElements join trees and facadeElements as treesAndFacades apply StochasticChangeDetection with [CityModel] and treesAndFacades using cv: [CriticalValue] as changes select added Trees with changes as newTrees select deformed FacadeElements with changes as newFacadeElements apply Intersection3D with newTrees using {x, y, z: [X], [Y], [Z]} and {notX, notY, notZ: [NX], [NY], [NZ]} apply Intersection3D with newFacadeElements using {x, y, z: [X], [Y], [Z]} and {notX, notY, notZ: [NX], [NY], [NZ]} store end with [CityModel] do apply MultiObjectClassification exclude all Antennas visualize end 

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

12

Therearejustafewadditionaladjectivesandverbsneededforthisworkflow:certainisusedtolimitasetofclassifiedobjects,whileallallowsittoselecteventheobjectswhicharesuspectedtobeofacertainkind.Theverbsstore,joinandapplyareusedtostoreaparticulardatasetbacktothedistributedfilesystem,jointwosetsandcallaparticularservice,respectively.

Concept Adjective Verb certain store all join apply

The resulting low‐level workflow actually provides the same functionality as the high‐levelworkflowintroducedearlier.Anexpertuser,however,getsalotmorecontrolontheprocessing.

3.2 MARINE SHOWCASE

The purpose of this showcase is to generate a digital terrainmodel of the sea ground,whiledealing with a wide variety of different data sources. An important processing step in thisscenarioisdeconflictiontomergethedifferentdatasetsproperly.

3.2.1 Vocabulary

Toget therelevantconcepts,verbsandadjectivesused todescribe themarineshowcase,wehavetoapplythesameprocesslikewedidfortheurbanshowcase.

Concept

SeaBottom GeologicalProcess AntrophicImpactWaterflow SandDune SurfaceHumanIntervention Wreck DigitalTerrainModel(DTM)TimeSource SpaceSource ChangesovertimeFeatures SeaCharts SurveyQualitySurveyNumber Deconfliction SurveySurveyCoverage 3DData DigitalElevationModelInteractiveDeconfliction Changedetection 3DDataAcquisitionPointCloud ShapeRepresentation DataSetSeaBottomDataSet ComparisonResult LR‐SplineSurfaceVisualization Triangulation ShapeSpline Rock PointSetOutcrops Crater StructureGlacier Tile ThresholdSurfaceTolerance PolynomialCellWidth ToleranceUncertainty Pattern FeatureDetectionSandBank Stone LR‐SplineApproximationPipeline ClusterofPoints IcebergDragMarks Outline LiDARDataSonarData Verb

Track Detect GatherApply Visualize Perform

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

13

Validate Compare RestrictReproduce Produce IdentifySelect Adjective

various old singlemultiple multi‐temporal smoothhumanmade rocky minimalwavelike waviness belowlong narrow recent

3.2.2 DomainModel

Whenmodelling thedomainof themarine showcasewekept thedomainmodelof theurbanshowcase inmind:Not to influence theentitiesand theirrelations,but toreuseobjectsof theformermodelifapplicable.Theresultactuallylooksquitesimilartotheurbanmodelwhichisreasonable as both showcases use datasetswhich contain information about an environmentandtheobjectswhichcanbefoundinit.ThesourceofthedatamightbeconsideredasaspecificentityrelatedtothegeneralconceptDataset.

FIGURE2DOMAINMODELFORTHEMARINESHOWCASE

3.2.3 ProcessRelations

The processing steps in the marine showcase, apart from visualization, can be described asdeconflictionandsurfacegeneration.Thereforetheoverallworkflowshouldusethesourcedatafor deconfliction first,which results in a set of deconflicted data.Hereupon, the result canbeusedtogenerateanLR‐SplineSurface.

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

14

3.2.4 PrototypeDSLworkflow

Since the domainmodel of themarine showcase looks quite similar to the one of the urbanshowcase, we are even able to create a similar workflow by using the already introducedlanguage elements. There are still a few different requirements compared to the urbanworkflow.

Workinginamarinescenariousuallyrequirestakingmoredatasourcesintoaccount.WhileanurbanscenarioismainlybasedonLiDARDatacapturedbycarorairplane,thedatawhichisusedinmarinescenarioscanresultamongstothersfromSonarScansorSeaCharts.Asaconsequenceofthedifferentnatureofdata,therearedatasetswhichmightbemoreup‐to‐date,whileothersaremoreaccurate.

Theworkflowhas to consider this diversity, in particularwhen it comes to deconfliction andsurface generation, respectively. Using the keyword of in the high‐level workflow, which hasbeen introduced in Section 3.1.4 for the low‐level workflow, allows the user to choose aparticularkindofdataforthefurtherprocess.Thereareactuallytwowaystospecifythedata.Adjectives like recent orhigh‐qualitymay be used to select datawhich fulfil these propertieswhilenottakingintoaccountthesource.OntheotherhandtheuseofconceptslikeSeaChartsorSonarDataletstheuserselectdatabysourceortype.

with [Area] of recent SeaCharts do generate Surface using threshold: 42 and target: [SurfaceTarget] visualize end 

Theworkflowshouldproceedwiththesurfacegenerationandallstepswhicharenecessaryforthat. For this purpose the verb generate is introduced. generate is used to get a particularconcept from a set of input data,where the processing exceeds a simple transformation. Thekeywordusing,whichhasbeendefinedinsection3.1.4,canbereusedinthehigh‐levelworkflowtoappendparameterstothegenerationprocess.Inthiscaseathresholdneedstobesetinordertododeconfliction,andatargetdatasethastobespecifiediftheresultistobestored.

The following table lists concepts, adjectives andverbswhichareused tobuild thehigh‐levelmarineworkflowsofar.

Concept Adjective VerbSeaCharts recent generateSurface visualize

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

15

Thelow‐levelworkflowwhichdescribesthesameprocessingstepsmightlooklikethefollowing.Thereisnoneedtointroduceadditionaltermsorcategoriesbutweintroduceanewparameterfor the verb store which allows specifying the target which is used in this case to store thesurface.

with recent SeaCharts do apply Deconfliction using threshold: [Threshold] apply Intersection3D using {x, y, z: [X], [Y], [Z]} and {notX, notY, notZ: [NX], [NY], [NZ]} as data apply SurfaceGeneration store using target: [SurfaceTarget] visualize end 

3.3 LAND SHOWCASE

Thethirdshowcasedescribesascenarioinwhichsatelliteimagesareprocessedtosupportfloodand waterlogging analysis and prediction. After the images have been merged and pre‐processed,waterloggingclassificationisappliedtofindandvisualizethefloodedareas.

3.3.1 Vocabulary

We extracted all needed terms for the urban and marine showcase earlier. The sameinvestigationhastobedoneforthelandshowcase.

Concept

Event Landslide ModelLandslideEvent Flood SimulationFloodingEvent Precipitation ReferenceMeasurementCriticalEvent Images CloudmaskSlopedeformation OpticalImages TopoftheatmosphereSpectralindices SatelliteImages ToAreflectanceNDVI Cloudshadowing CloudNDSI NaturalWaterMask RiverNDWI Lake LandParcelIdentification

System(LPIS)NaturalWaters Waterlog SoilSeriouslyAffected ModeratelyAffected WeaklyAffectedVegetationinwaterlog Dryareasclouds NosupportedareasThreshold SurfaceRepresentation InterpolationVerb

analyze predict monitorcompare obtain producecompute preprocess transformcalibrate filter reprojectcalculate process classify

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

16

Adjective

High‐quality landslide floodingmechanical Hydrological geometriccalibrated Moderatelyaffected

3.3.2 DomainModel

Again, using the concepts from the description of the land scenario allows us to create thedomain model which is built considering the two models already created for the urban andmarine showcase (see Figure ). The entities of Dataset and Topographic Object, theirdescendants and their relation hardly changes from the former ones, except for the domainspecificentities,namelythelandtopographicobjectanditschildren.Ontheotherhand,thenewentity TopographicObjectCategory is introduced which can be used to group different LandTopographicObjects. This is used in particular to create a ThematicMap in this showcase.FurthermoreaTopographicObjectCategorymightbeassociatedwithanEventwhichitselfcanbeassociatedtoasetofchanges.

3.3.3 ProcessRelations

The landshowcasebasicallycontains threemajorprocessingsteps.First, theselectedsatelliteimageneeds tobepre‐processed,which includes topof atmospherereflectioncalibrationandcloud masking. The result can be used to calculate spectral indices and consequently thetopographicobjectcategoriescanbefoundandclassified.

FIGURE3DOMAINMODELFORTHELANDSHOWCASE

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

18

3.3.4 PrototypeDSLworkflow

Creating a simple high‐levelworkflow from the showcase includes a lot of termswhich havebeenintroducedearlier.Thedatasourceisselectedinthesamewayasintheformerworkflowsandtheverbgenerate,whichhasbeenintroducedinsection3.2.4,willbeusedtogeneratetheThematicMap.Anewverbwhichneedstobespecifiedhereiscolorize.Thisoneallowsapplyingauserdefinedcolourtoanentityofthedomain.InthisscenarioallobjectsassociatedwiththecategoryWaterlogwillbecolorized.

with [Area] of recent SatelliteImage do generate ThematicMap colorize Waterlog [WaterlogColor] visualize end 

Allterms,apartfromthelanguagespecificones,usedinthisworkflowarelistedinthefollowingtable.

Concept Adjective VerbSatelliteImage recent generateThematicMap colorizeWaterlog visualize

Aswehaveseeninsection3.1.4andsection3.2.4,a low‐levelworkflowismainlybuilt fromachain of services,with the proper selection of parameters,which are called consecutively. Allneeded terms and keywords for this purpose have been introduced in the former sectionsalready.

with [Area] of recent SatelliteImage do apply RasterDataPreprocessing apply SpectralIndicesComputation using si: [SpectralIndices] and wl: [WaveLengthInfo] as indices apply ThematicClassification using indices: indices apply Coloring using category: Waterlog and color: [WaterlogColor] visualize end 

Basically, thisworkflowselectsthemostrecentsatellite imagewhichcanbe foundforagivenarea and uses the serviceRasterDataPreprocessing to do pre‐processing. After calculating thespectral indices and using them to create the thematic map the categoryWaterlog will becolorizedinthesamewaylikeinthehigh‐levelworkflow.

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

19

4 RATIONALE FOR THE CHOSEN DSL SYNTAX

IntheprevioussectionswepresentedsampleDSLworkflowsbasedonthevocabularyfromourdomain models. Even though the three showcases use different domain models and hencedifferent terms in the DSL workflows, the syntax of all sample scripts looks quite similar.Identifyingthiscommonsyntaxwasanadditional taskthatweperformedinordertocreateaDSLthatlookssimilarinallshowcasesandthathenceiseasiertolearn.

Wetestedvarioussyntaxes,startingwiththefollowing:

with DataSet exclude NonStaticObjects from it select added Trees and deformed FacadeElements from it add it to CityModel 

Inthissamplescriptweusedthekeyword‘it’torefertoanobjectfromthepreviouslineortorefertotheresultoftheprocessperformedinthepreviousline.Thiscontext‐sensitiveapproachrequires an intelligent parser that is able to clearly identifywhat ‘it’means in the respectivecontext. However,we realized that even human users have problems understandingwhat ‘it’actuallymeans,sowedecidedtoremovethiskeywordfromourlanguage.

Nextwecreatedthefollowingsyntax:

exclude NonStaticObjects from DataSet and select added Trees and deformed FacadeElements and add to CityModel 

In this case individual processing steps are connected with the ‘and’ keyword. While thisapproachleadstounambiguousworkflows,thescriptscanveryquicklybecomehardlyreadable.The longer the workflows get, the harder it is to understand them as the sentences becomelonger and longer. We hence decided to introduce blocks using the ‘with … do’ and ‘end’keywords,tomakeclearwhichprocessingstepsaffectwhichdataset.

with recent Data do exclude NonStaticObjects select added Trees and deformed FacadeElements add to recent CityModel end 

Thissyntaxisveryreadableandcanbeappliedtoallthreeshowcasesasshownintheprevioussections.

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

20

5 REFERENCES

Gaševic, D., Djuric, D., & Devedžic, V. (2009). Model Driven Engineering and Ontology Development 

(2nd ed.). Springer. 

Krämer, M., & Stein, A. (2014). Automated urban management processes: integrating a graphical 

editor for modular domain‐specific languages into a 3D GIS. Proceedings of the 19th 

international conference on urban planning and regional development in the information 

society GeoMultimedia.  

De Nicola A., Missikoff M., & Navigli R. (2009): A software engineering approach to ontology building. 

Information Systems, Volume 34 Issue 2 (pp. 258‐275). 

Spyns, P., Meersman, R., & Jarrar, M. (2002). Data modelling versus ontology engineering. ACM 

SIGMOD Record, Volume 31 Issue 4 (pp. 12‐17). New York, NY, USA: ACM. 

6 APPENDIX: PARSING EXPRESSION GRAMMAR FOR THE DSL

start = SP* statements SP* statements = statement ( SP+ statement )* statement = block / process ( SP+ "with" SP+ dataset_expression ( SP+ "and" SP+ dataset_expression )* )? ( SP+ "using" SP+ params )? ( SP+ "as" SP+ ID )? block = with SP+ statements SP+ "end" with = "with" SP+ dataset_expression SP+ "do" dataset_expression = dataset ( SP+ "of" SP+ dataset )? dataset = "recent" SP+ ID / "latest" SP+ ID / placeholder / ID params = param ( SP+ "and" SP+ param )*

D2.4.1 IQmulus(FP7‐ICT‐2011‐318787)

21

param = "{" SP* tuple_ids SP* ":" SP* tuple_expression SP* "}" / ID SP* ":" SP* expression tuple_expression = expression ( "," SP* expression )* expression = placeholder / INTEGER / string / ID tuple_ids = ID ( "," SP* ID )* placeholder = "[" ID "]" string = '"' string_character* '"' / "'" string_character* "'" string_character = !["\\\r\n] . / "\\" ESCAPE_CHARACTER INTEGER = [0-9]+ ESCAPE_CHARACTER = ['"\\bfnrtv] ID = [_a-zA-Z][_a-zA-Z0-9]* SP = [ \t\n\r] process = "apply" SP+ ID / "store" / "visualize" / urban_process / marine_process / land_process urban_process = "exclude" ( SP+ urban_dataset_param )? SP+ ID / "join" SP+ ID ( SP+ "and" SP+ ID )* / "select" SP+ urban_dataset_param SP+ ID ( SP+ "and" SP+ urban_dataset_param SP+ ID )* / "add" SP+ "to" SP+ dataset

IQmulus(FP7‐ICT‐2011‐318787) D2.4.1

22

urban_dataset_param = "added" / "all" / "certain" / "deformed" marine_process = "generate" SP+ ID land_process = "colorize" SP+ ID SP+ dataset