Pywps a tutorial for beginners and developers

Embed Size (px)

Citation preview

PyWPS a tutorial for beginners and developers

Jorge de Jesus (Plymouth Marine Laboratory)Luca Casagrande (Universit degli Studi di Perugia) Jachym epicky (Help Service Remote Sensing Company)

Before starting

Please put your OSGEO livecd or usb-stick inside your laptop and start your machine.

Program

Introduction partWPS Standard and PyWPSNewbie partInstallation, setup and first processDevelopers partPyWPS in detail, mod_python,jython, GRASSGalleryExamples of applications using PyWPS

Introduction Section

Install the tutorial

Open Firefox and download the script from the pyWPS main page:http://pywps.wald.intevation.org/Make it executable (password is user):sudo chmod +x install_pywps_svn.shStart the script:sudo ./install_pywps_svn.shYou can edit files using nano, but remember to always use sudo (password is user).

Definitions

Web Processing Service (WPS) is an OGC standard protocol to make GIS calculation available to the internet

Open Geospatial Consortium (OGC) is a non-profit, international, voluntary consensus standards organization that is leading the development of standard for geospatial and location based service.

WPS standard

... provides rules for standardizing how inputs and outputs (requests and responses) for geospatial processing services, such as polygon overlay. The standard also defines how a client can request the execution of a process, and how the output from the process is handled..

Invoking WPS server

HTTP GET method with key-value-pairs encoded request (KVP)

HTTP POST method, using XML format of the request

SOAP

For this tutorial we will use just the HTTP GET method

Key Value Pairs request

http://localhost/cgi-bin/wps.py?service=WPS&request=GetCapabilities

http://localhost/cgi-bin/wps.py Is the server addressThe ? sign indicates, that the request parameters will startservice=WPS&request=GetCapabilitiesThe KVP-encoded request. We send two request parameters to the server:service - which we set to WPSrequest - which is set to GetCapabilities

XML Request

In this case, the request is encoded in XML form and send to the server directly via HTTP POST (the WPS server will read the file from standard input directly).

1.0.0

WPS operations

GetCapabilities: returns service-level metadata

DescribeProcess: returns a description a process including its inputs and outputs

Execute: returns the output(s) of a processDespite the specification that requests should be case insensitive, it is recommended to use the upper camel case standard in all sorts of WPS operation requests

GetCapabilities

Mandatory parameters:service,requestThe server returns basic Capabilities document. Among others it contains:ServiceIdentification

Service provider (Name, Organization, Address, ...)

ProcessOfferings - List of available processes

http://apps.esdi-humboldt.cz/pywps/?service=WPS&request=GetCapabilities

DescribeProcess

Mandatory parameters:version, name of the process or all keywordThe ProcessDescriptions document contains detailed description of selected processes. Each process is identified by:Title

Identifier

DataInputs

DataOutputs

http://apps.esdi-humboldt.cz/pywps/?service=WPS&version=1.0.0&request=DescribeProcess&identifier=all

Execute

Mandatory parameters:No mandatory parameters

http://apps.esdi-humboldt.cz/pywps/?service=WPS&version=1.0.0&request=Execute&identifier=literalprocess&datainputs=[int=1;float=3.2;zeroset=0;string=spam]

KVP inputs encoding

A semicolon (;) shall be used to separate one input from the next

An equal sign (=) shall be used to separate an input name from its value and attributes, and an attribute name from its value

An at symbol (@) shall be used to separate an input value from its attributes and one attribute from another.

All field values and attribute values shall be encoded using the standard Internet practice for encoding URLs

PyWPS also supports the use of [ ] to group the datainputs as follows: datainputs=[int=1;float=3.2]

Description of Data Inputs and Outputs

Three types of inputs and outputs are defined in the OGC standard. LiteralData, ComplexData and BoundingBox data.

LiteralData

LiteralData can be any character string, float,date, etc normally described as Primitive datatype in the W3C XML

WPS standard also allows the use of UOM (Unit of Measures), default values and AllowedValues.

ComplexData

Complex Data data type is used for pasting complex - Vector- Raster- or other data to the server or obtain it as result of the process. There are two ways, how this complex data are handled:Either you send them directly as part of the request to the server or you obtain them as part of the XML response from the server.

Or you send or obtain just reference to the data URL to the file or service, where the data can be downloaded.

Bounding Box Data

It is used to describe some sort of bounding box area . The input description must state: the default coordinate reference system (CRS) used

other CRS supported

&bboxInput=71.63,41.75,-70.78,42.90,urn:ogc:def:crs:EPSG:6.6:4326,2

PyWPS

pyWPS is:A framework for the implementation of WPS 1.0.0

Totally written in python 2.6

Organized into packages and classes for easy maintenance

Open to new developments and functionalities that can be integrated in WPS 1.0.0

Is run from bash or as a cgi process

pyWPS is not:Is not an analytical tool or engine. It does not perform any type of geospatial calculation

Is not a XML parser or generator. It does not validate the GMLs against given schemas (yet), it does not build GML from Python objects

Can't be used to build general web applications like CherryPy framework

PyWPS

PyWPS history

2006First pyWPS 1.0.0 release, a project was born

First presentation held at FOSS4G 2006 Lausanne: "GRASS goes web: pyWPS"

2007pyWPS 2.0.0 gets release, supporting WPS 0.4.0

PyWPS history

2008pyWPS 3.0.0 gets release:Support for WPS 1.0.0

New simple configuration files

Support for multiple WPS servers with one pyWPS Installation

Support for internationalization

Simple code structure

New examples of processes

PyWPS history

2009pyWPS 3.1.0 gets released with new featuresUp-to-date examples

New generic WPS JavaScript library

Multiple fixes in both, source code and templates

New style In- and Outputs Complex object

Tons of bugs fixed

PyWPS history

2010pyWPS is recomended as THE WPS tool in GIGAS project ( GEOSS, INSPIRE and GMES an Action in Support). As explained in the "GIGAS Technology Watch Report WPS"

"PyWPS Web Processing Service: is a Python program which implements the OGC WPS 1.0.0 standard (with a few omissions). PyWPS was chosen as it is up to date with the WPS standard and has a low footprint, making it easy to install on most Linux systems.

Newbies Section

Installation

Configuration file

PyWPS instance / Wrapper Script

First process

Installation Requirements:

python currently 2.5, 2.6python xml packagehtmltmpl engine older versions

For cool stuff you may need:

GRASS-GIS

Install files can be found in:

http://pywps.wald.intevation.org/download/SVN access to the latest code:

svn checkout https://svn.wald.intevation.org/svn/pywps/trunkLatest package:

http://wald.intevation.org/frs/download.php/589/pywps-3.1.0.tar.gz

Clean install:

> tar -xvzf /tmp/pywps-VERSION.tar.gz> cd pywsp-VERSION> python setup.py installThere's DEB and RPM packages. Badly maintain :(

Testing the script by running the wps.py (/usr/bin) script

> /usr/bin/wps.pyIf everything is ok....

PyWPS NoApplicableCode: Locator: None; Value: No query string found.Content-type: text/xml

No query string found.

Configuration file for PyWPS can be located on several places.

There are global and local PyWPS configuration files.

Local files overwrite the global one

Global:

/etc/pywps.cfg/usr/local/pywps-VERSION/etc/pywps.cfgAny path defined in thePYWPS_CFG environment variableLocal:

pywps.cfg is a Key = Value text file

4 sections are present in the file:

[wps][provide][server][grass]Remember: The file is case-sensitive

[wps]titleversionabstractfeeskeywordslang

Baisic meta information necessary to populate the WPS doc.

Other variables aren't shown in this example

Server configuration options, path locations and URL translation and service limits

[server]maxoperationsmaxinputparamlengthmaxfilesizeoutputUrloutputPathprocessesPath

Other variables aren't shown in this example

Most important: outputURL, outputPath, processPath

outputURL:

URL that will be used to point to the WPS outputs

outputPath:

Folder where PyWPS will drop the outputs (server accessible)

http://localhost/wpsoutputhttp://rsg.pml.ac.uk/wps/wpsoutput/var/www/html/wpsoutput/usr/local/apache/htdocs/wps/wpsoutput/var/www/html/wpsoutput/var/www/html/wpsoutput

processPath:

Folder path with stored processes

/usr/local/pywps/processes/usr/local/pywps/processes/usr/local/pywps/processes/usr/local/pywps/processes/usr/local/pywps/processes/home/user/processesIt's important that these 3 parameters are properly configured

PyWPS can be installed once in a server,

but it may be configured to run several WPS services (instances).

WPS instanceProcess folderpywps.cfg file

1) Setup a process folder

2) copy configuration file-template and edit it to desired configuration

> mkdir -p /usr/local/wps/processes> cp pywps-VERSION/pywps/default.cfg /usr/local/wps/pywps.cfg> nano /usr/local/wps/pywps.cfg3) We need to populate the process directory

> cp pywps-VERSION/examples/ultimatequestionprocess.py /usr/local/wps/processes/

4) Every process in the process folder needs to be registered in a file called __init__.py

> cd /usr/local/wps/processes/> echo "__all__=['ultimatequestionprocess']" > __init__.py__all__ it's a python array will the processe list

We've done 50% of an instance :)

A WPS instance is just a script that alters some parameters before calling wps.py

#!/bin/sh # Author: Jachym Cepicky# Purpose: CGI script for wrapping PyWPS script# Licence: GNU/GPL# Usage: Put this script to your web server cgi-bin directory, e.g.# /usr/lib/cgi-bin/ and make it executable (chmod 755 pywps.cgi) # NOTE: tested on linux/apache export PYWPS_CFG=/usr/local/wps/pywps.cfgexport PYWPS_PROCESSES=/usr/local/wps/processes/ /usr/local/pywps-VERSION/wps.py $1wps.cgi file

We need to configure PYWPS_CFG and PYWPS_CFG to specify the instance

We can copy the wrapper script to Apache's cgi-bin folder

> cp wps.cgi /usr/lib/cgi-binAssuming that Apache is configure to support script execution...

http://localhost/cgi-bin/pywps.cgi?request=DescribeProcess&service=WPS&version=1.0.0&process=ultimatequestionprocess

ultimatequestionprocess Answer to Life, the Universe and Everything .... answer The numerical answer to Life, Universe and Everything integer

POSTGETSOAP

GetCapabilitiesDescribeProcessExecute12

1Load ProcessCheckPropertiesWPS output

2Load ProcessgetInputrunsetOuputWPS output

PyWPS's assemble factory approach

User's process

Process as an extended class of WPSProcess with method run() that will execute the code

WPSProcess classProcess1Process2ProcessN

All processes have the following skeleton:

from pywps.Process.Process import WPSProcess class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self, < Process's information like: identifier, title, status >) < Inclusion of inputs and outputs to process class > def execute(self):< code >WPSProcess class provides extra functionalities like:- Command line util: self.cmd()- Status setting: self.status.set(message,percentage)

Process's attributes:

class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self,identifier=firstprocess, #same file name title=foo, abstract=bacon and eggs,version = "0.1", storeSupported = "true", statusSupported = "true", )The only mandatory attribute is: identifier

LiteralDataComplexDataBBOX3 types of Input/Output defined in WPS:

Each Input/Output is a method of WPSProcess class

Each Input/Output is created when class in initiated

LiteralDataComplexDataBBOXself.addLiteralInput(...)self.addLiteralOutput(...)self.addComplexInput(...)self.addComplexOutput(...)self.addBBoxInput(...)self.addBBoxOutput(...)

Each self.add*() defines/creates an input. The class constructor accepts the WPS parameters:

self.Input1 = self.addLiteralInput(identifier = "input1",title = "Input1 number",abstract=foo,minOccurs=1, type=types.IntType default="100")Only identifier and title are mandatory

A more Complex example

self.dataIn = self.addComplexInput(identifier="data", title="Input vector data",abstract=foo formats = [{'mimeType':'text/xml'}])

Only identifier and title are mandatory

What about outputs

Identical syntax and procedure :)

self.dataOut = self.addComplexOutput(identifier="output", title="Output vector data", formats = [{'mimeType':'text/xml'}])

self.Output1 = self.addLiteralOutput(identifier="output1", title="foo")

The add*Input and add*Output are set in the beginning of the class (__init__ method):

from pywps.Process.Process import WPSProcess class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self, < Process's information like: identifier, title, status >) self.Input1=self.addLiteralInput(identifier=input1) self.dataOut =self.addComplexInput(identifier=outputs) def execute(self):< code >

OK we have WPS inputs and output, how can I get them ?!?!?!

Using the JAVA get and set philosophy :)

Each Input has a getValue() method

Each Output has a setValue() method

This is done inside the execute method()

from pywps.Process.Process import WPSProcess class Process(WPSProcess): def __init__(self): # init process WPSProcess.__init__(self, < Process's information like: identifier, title, status >) self.Input1=self.addLiteralInput(identifier=input1) self.dataOut =self.addComplexInput(identifier=outputs) def execute(self):input1=self.Input1.getValue()XMLdata=foo self.dataOut.setValue(XMLData) #input or file objectPlease check wiki !!!!

http://pywps.wikispaces.com/InputOutput

Fist Process, a returner process

from pywps.Process import WPSProcess class Process(WPSProcess): def __init__(self): ## # Process initialization WPSProcess.__init__(self, identifier = "returner", title="Return process", abstract="""This is demonstration process of PyWPS, returns the same file, it gets on input, as the output.""", version = "1.0", storeSupported = "true", statusSupported = "true")Process class initiation, identifier, WPS status and storeExecuteResponse definition

## # Adding process inputs self.dataIn = self.addComplexInput(identifier="data", title="Input vector data", formats = [{'mimeType':'text/xml'}])

self.textIn = self.addLiteralInput(identifier="text", title = "Some width")Attention to indent !!! It's python !!!!

## # Adding process outputs

self.dataOut = self.addComplexOutput(identifier="output", title="Output vector data", formats = [{'mimeType':'text/xml'}])

self.textOut = self.addLiteralOutput(identifier = "text", title="Output literal data") ## # Adding process inputs self.dataIn = self.addComplexInput(identifier="data", title="Input vector data", formats = [{'mimeType':'text/xml'}])

self.textIn = self.addLiteralInput(identifier="text", title = "Some width")

def execute(self):

# just copy the input values to output values self.dataOut.setValue( self.dataIn.getValue() ) self.textOut.setValue( self.textIn.getValue() )

returnHopefully this will work:

http://localhost/python/wps.py?request=Execute&service=WPS&identifier=returner&Version=1.0.0&datainputs=[text=abc;data=foo]

Developers Section

Hacking ( wps.py code for dummies)

Logging (and wood)

GRASS (after the wood)

Mod_Python (Pythons and Horses)

Tomcat server (Pythons and Cats)

Mapserver support (More OGC stuff)

OpenLayers (Let there be layers.....)

SOAP/WSDL (Beatiful soap... so they say...)

PyCallGraph (The all enchilada!!!)

PyWPS's wps.py has the following pseudo-code structure:

Start wps.py:1. Determine request_method (GET or POST)2. if no input: raise Exception and exit3. try: initiate PyWPS class according to request_method parse Request do Request get Response and make proper reply4. exception: reply Error response

Some Hacking experiments:

Exit if request is POST (around line 95):

if method==pywps.METHOD_POST: sys.exit(1)Encript a WPS response:

if response: pywps.response.response(response, sys.stdout,wps.parser.soapVersion,wps.parser.isSoap, wps.request.contentType)if response: response=xor_crypt_string(response,key="FOSS4G") pywps.response.response(response,.....

In PyWPS you can use the logging module anywhere in the code.

import logginglogging.debug("Something has been debugged")pywps.cfg file contains the path to the log file

debug=true # deprecated since 3.2, use logLevel insteadlogFile=/etc/httpd/logs/pywps.loglogLevel=DEBUGThen the file log will contain a line, like this:

PyWPS [2010-08-25 17:39:00,499] DEBUG: Something has been debugged

Eclipse IDE is the default debugging platform using PyDEV tools

Code to be debugged should contain a path to the PyDEV tools:

sys.path.append("..plugins/org.python.pydev.debug_1.6.0.2010071813/pysrc/")After this path append, it is possible to import pydev module

import pydevdNow the code with pydevd is enabled for debbuging

Next step is to activate the debug server that will listen to the script.

Now everytime that the python interperter finds:

pydevd.settrace('localhost', port=5678, stdoutToServer=True, stderrToServer=True)It will stop and send the variable to debug server

From eclipse it will be possible to continue or stop the script

The code was launched from Apache using a GetCapabilities request

PyWPS doesn't come with out-of-the-box tools

PyPWS is Python, so connect, connect, connect !!!!! To GRASS GIS

You may work with a predefined grassLocation or a temporary one

WPSProcess.__init__(self, identifier = "foo", ... grassLocation = True)Temporary grassLocation

XY coordinate system

WPSProcess.__init__(self, identifier = "foo", ... grassLocation = spearfish60)Permanent grassLocation

Previous defined coordinate system

gisdbase base path specified in configuration file (pywps.cfg)

Absolute path in grassLocation

OR

Commands passed as an array with command + arguments:

self.cmd(["r.los","in=elevation.dem","out=los","coord=1000,1000"]) self.cmd(["d.mon","start=PNG"],stdout=False)Inside the execute method()self.cmd() is a WPSProcess class method

It's not a simple Python bash command util

A total Pythonic way to connect to GRASS!!!!!!

from grass.script import core as grassdef execute(self): ret=grass.run_command("d.his", h_map="drap_map", i_map="relief_map", brighten=0) return

Command line strategy

grass.mapcalc("MASK=if(($cloudResampName < 0.01000),1,null())", cloudResampName = cloudResampName)

Package access strategy

"Mod_python is an Apache module that embeds the Python interpreter within the server.

In: http://www.modpython.org/

version 3.2 provides a wps.py script designed to be integrated into mod_python

https://svn.wald.intevation.org/svn/pywps/trunk/webservices/mod_python/wps.pySo what is the advantage ?! SPEED !!!!!

50x faster on request processing

Integration with Apache's API

Ability to handle request phases, filters and connections

Default httpd.conf for PyWPS:

AddHandler mod_python .py Options Indexes FollowSymLinks AllowOverride None Order allow,deny Allow from all PythonHandler wps PythonDebug On PythonAutoReload On PythonOption PYWPS_PROCESSES /../python2.6/site-packages/pywps/processes PythonOption PYWPS_CFG /etc/pywps.cfg

Inform Apache that wps is the default handler of any request

Pass env variables PYWPS_PROCESSES and PYWPS_CFG

Mod_python can be used to restrict access to the WPS:

: PythonHandler wps PythonAuthenHandler wps : AuthType Basic AuthName "Restricted Area" require valid-user

PythonAuthenHandler wps

def authenhandler(req):

pw=req.get_basic_auth_pw() user=req.user

if (user=="bacon" and pw=="eggs"): return apache.OK else: return apache.HTTP_UNAUTHORIZED

Function added to wps.py

apache.OK continue wps.pyUNAUTHORIZED Stop wps.py

mod_python can apply filters on HTTP request/response

The filter needs to be register to Apache and mod_python

PythonOutputFilter wps ENCRPYTAddOutputFilter ENCRYPT .pydef outputfilter(filter): req=filter.req #getting requirement s = filter.read() #You have always to read and then write the filter (no matter what) if req.status == apache.HTTP_OK: s_crpyt=xor_crypt_string(s, "FOSS4G") filter.write(s_crpyt) else: filter.write(s) filter.close() #Always close the stream otherwise it will write twiceThe filter is applied to any WPS output, encrypting the response

WPS client/server 'secured' interaction

The server provides getCapabilities and describeProcess to anyone.

The execute is permitted only to authorized users

Note: the base authentication credentials are used to allow the geo web service to receive delegation (downloading a proxy certificate) from another web service .The authentication/authorization of the execute is managed through X.509 certificates which are handled through the GridSite module for Apache (http://www.gridsite.org/).

Who is using mod_python and PyWPS ?!

GENESI-DR, (Ground European Network for Earth Science Interoperations - Digital Repositories),www.genesi-dr.eu & www.genesi-dec.eu

INFRA-2007-1.2.1 : Scientific Digital Repositories

Python Code

Jython Compiler

Java ByteCode

TomCat Instance

No Voodoo , Just computer science !!!!

PywpsServlet.pyInstead of wps.py we have a new script called PywpsServelet.py

JAVA Servelet structure

def doGet(self,request,response):: def doPost(self,request,response)::def doWPS(self,request,response)

class PywpsServlet(HttpServlet):

https://svn.wald.intevation.org/svn/pywps/trunk/webservices/tomcat/WEB-INFPywpsServlet.py

Configuration file used by TomCat

The twin brother or wps.py

- All PyWPS code needs to be copied to the Tomcat folder running the instance:

> cd apache-tomcat-6.0.29/wps> mkdir wps> mkdir wps/WEB-INF> mkdir wps/lib> mkdir wps/WEB-INF> mkdir wps/lib> cp $PYWPS_SOURCE/webservices/tomcat/web.xml wps/WEB-INF/> cp -r $PYWPS_SOURCE/pywps wps/

Now we just need the Jython Library :)

> cp jythonlib.jar $CATALINA_HOME/webapps/wps/lib/

And we have just the last piece of the puzzle missing...

> cp $PYWPS_SOURCE/webservices/tomcat/PywpsServlet.py

What about PYWPS_PROCESSES ?!

import osos.environ["PYWPS_PROCESSES"] = "/path/to/processes"Inside PywpsServlet.py

- After start/stop of tomcat we should be able to make a request:

http://localhost:8080/wps/PywpsServlet?service=wps&request=getcapabilities

Still in the SVN tree, highly experimental !!!!!

Who's using it ?! ...or will be using.....

Its intended to be the default WPS service for Conceptual Schema Transformer

Yes, PyWPS even supports Mapserver :)

Still experimental in the SVN.....

ComplexData output@asReference=TrueComplexData reference link outputed as a OGC service

http://foo/bar/cg-bin/mapserv?map=/outputs/output-123.map&service=WCS&request=GetCoverage&....

According to the data type the link will point to a WMS, WFS or WCS service.

So how is it set ?!

self.imageOut=self.addComplexOutput (identifier="tiffOut", title="GeotTiff result", useMapscript=True formats = [ {"mimeType":"image/tiff"}, {"mimeType":"image/png"}])

userMapscript=True

mimeType

image/tiff == WCSimage/png == WMS

text/xml == WFS

In the SVN tree we have a WPS client specific for Openlayers

https://svn.wald.intevation.org/svn/pywps/trunk/webclient/WPS.jsJust append the file to the HTML's script tags

Now a OpenLayers.WPS class should be available

WPS+describeProcess()+getCapabilities()+execute()+onDescrivedProcess: callback+onGotCapabilities: callback+onExecuted: callbackWe have 2 major classes, WPS and process

WPS API will make all the requests,

parse the result and when finish will run a call back function

A simple example:

wps = new OpenLayers.WPS(url, {onGotCapabilities: onGetCapabilities});wps.getCapabilities();function onGetCapabilities() { : var capabilities = ""+wps.title+""; capabilities += "Abstract"+wps.abstract;: document.getElementById("wps-result").innerHTML = capabilities; };Please check wiki for an extensive explanation !!!!

http://pywps.wikispaces.com/OpenLayers

SOAP == Simple Object Access Protocol

WSDL == Web Services Description Language

OGC defines that WPS 1.0.0 should support these standards

PyWPS has some support for SOAP

PyWPS generates a simple WSDL file

SOAP is a messaging framework, meaning, a structured way to pass, explain and process a message.

XML content (message)SOAP functionalities (authentification, actors etc)

XML wrapper defining SOAP

Example:

1.0.0

- Currently PyWPS will accept SOAP XML requests

- BUT it will not process any header content or special Execute tags

- WSDL is a XML document describes a Web service.

-Considering a WPS process, then a WSDL would some something like:

WSDL Doc == GetCapabilites+ DescribeProcess+ Execute+ OGC WPS standard definition (schema)

Redundancy !!!

Data type Message defined by dataType Operations and messages SOAP, HTTP POST etc "http://localhost/wps.py"

http://pywps.wikispaces.com/WSDL

WSDL file is served as follows:

http://foo/wps.py?WSDL

File location:pywps.cfg

http://foo/wps.py/ultimatequestionprocess?WSDLNo specific process WSDL file request or support :(

SVN branch pywps-3.2-SOAP for WSDL and SOAP development

Next PyWPS release will have better SOAP/WSDL support

WPS 2.0.0 to have better SOAP/WSDL support

Million dolar question ?! Why do we need SOAP/WSDL

- Orchestration and interaction with other web services

- Ability to use BPEL (Bussines Procedure Language) to orchestrate services

-PyCallGraph is used to generate a graphic representation of code being run

- Useful to check bottlenecks and code problems

> python wps.py "request=GetCapabilities&service=WPS"

- Major time consumption in initProcess() method.

- More processes == Slower output

A detailed analysis using pyCallGraphic has shown:

NO MAJOR BOTTLE NECKSExecute/DescribeProcess spent most of time in process handling

If PyWPS is slow, blame the process code

Minimum overhead when callingUpdate and Exception reports

Gallery / Examples

NETMAR Open Service Network for Marine Environmental Data

NETMAR aims to develop a pilot European Marine Information System (EUMIS) for searching, downloading and integrating satellite, in situ and model data from ocean and coastal areas.

Provide a user-configurable system offering flexible service discovery, access and chaining facilities using OGC, OPeNDAP and W3C standards.

Lots of WPS processes!!!Use a semantic framework coupled with ontologies for identifying and accessing distributed data, such as near-real time, forecast and historical data.

http://rsg.pml.ac.uk/wps/wps.cgi?request=Execute&service=wps&version=1.0.0&identifier=histogramprocess&datainputs=[imageInput=http://rsg.pml.ac.uk/wps/sat_image.tif]&responsedocument=histogramOutput=@asreference=true

http://rsg.pml.ac.uk/wps/wps.cgi?request=Execute&service=wps&version=1.0.0&identifier=histogramprocess&datainputs=[imageInput=http://rsg.pml.ac.uk/wps/srtm.tif]&responsedocument=histogramOutput=@asreference=true

http://rsg.pml.ac.uk/wps/wps.cgi?request=Execute&service=wps&version=1.0.0&identifier=reducer&datainputs=[reductionFactor=0.1;imageInput=http://rsg.pml.ac.uk/wps/srtm.tif]&responsedocument=imageOutput=@asreference=true&status=true&storeExecuteResponse=true

60 megas

700kb

Acknowledgments:

Simone Gentilini (JRC). GENESI-DR Project funded by FP7 program under (INFRA-2007-1.2.1) Scientific Digital Repositories www.genesi-dr.eu & www.genesi-dec.eu

Plymouth Marine Laboratory Remote Sensing Groupwww.pml.ac.uk & http://rsg.pml.ac.uk

Netmar project. Project partially funded by FP7 program under (ICT-2009.6.4) Information & Communication Technologies.http://netmar.nersc.no/

HS-RS Help Service Remote Sensing http://www.bnhelp.cz/