Embed Size (px)
Citation preview
Priority enabled Web Processing Service
Thorsten DeelmannMartin Wilden
52°North Student Innovation Prize 2009
Agenda
• Motivation• Conceptual work
o Extensions GetJobs PauseJob ResumeJob CancelJob
o Exceptions• Implementational work
o Challengeso uDig-Plugin
• Questions
Motivation: Overview
• SDIs focus on allocation of geospatial data
• Geoprocessing in SDIs to gain information on-demand
• WPS as Geoprocessing element in SDIs
Motivation: WPS-Introduction
• Web Processing Serviceo perform GIS calculation as web service
• 3 Methods:o GetCapabilities OWS Common compliant service description
o DescribeProcess Extended algorithm description
o Execute Perform the requested algorithm
Motivation: Scenario
Motivation: What's missing?
Give calculations precedence
Control Processes (Pause, Resume, Cancel)
See running and queued processes
Motivation: What's missing?
Give calculations precedence
Control Processes (Pause, Resume, Cancel)
See running and queued processes
WPS becomes more flexible and user friendly
Potential for new scenarios and business models
Conceptual Work
Four new operations to add the job control ability
GetJobs
PauseJob
ResumeJob
CancelJob
Standard operations unchanged“Packaging” is different
User is required to invoke the operations via SOAP
SOAP & Security
• SOAP describes a message exchange mechanism• Every SOAP Message consists of two parts: SOAP-Header and
SOAP-Body.o Header for Message related Informationo Body for the real Message (Payload)
• Regarding to the security two main questions occure:o Who submitted the Job?o What is the priority of the job?
• Solution: Use PriorityToken in SOAP-Header
Security - PriorityToken
SOAP-Message
Security - PriorityToken
SOAP-Message
SOAP-Header
SOAP-Body: The playload
Security - PriorityToken
SOAP-Message
SOAP-Header
PriorityToken
SOAP-Body: The playload
Security - PriorityToken
SOAP-Message
SOAP-Header
PriorityToken
SOAP-Body: The playload
User Credentials
Priority
Security - PriorityToken
• PriorityToken is issued by a Security Token Service.o username and password authentication
• PriorityToken extends SAML-Tokeno Containing elements: Username Priority (optional)
• All security components are WS-* compliant
Component Diagram
GetJobs
Overview of:• currently running jobs• queued jobs• recently finished jobs
Input parameter:• None
Output parameter:• List of jobs• Every job contains the following elements:
o JobID, ProcessIdentifier, JobOwner, JobPriority, JobSubmissionTime, JobStatus
Sample Request/Response
Sample Request/Response
PauseJob
Enables you to pause a job, which was once executed by yourself.
Input parameter:• JobID• PriorityToken
o includes user credentials (ID)
Output parameter: • Description (human readable)• OperationResult (machine readable URI) • Job-ID
Sample Request/Response
Sample Request/Response
ResumeJob
Enables you to resume a job, which was once executed and paused by yourself.
Input parameter:• JobID• PriorityToken
o includes user credentials (ID)
Output parameter: • Desription (human readable)• OperationResult (machine readable URI) • Job-ID
Sample Request/Response
Sample Request/Response
CancelJob
Enables you to cancel a job, which was once executed by yourself.
Input parameter:• JobID• PriorityToken
o includes user credentials (ID)
Output parameter: • Desription (human readable)• OperationResult (machine readable URI) • Job-ID
Sample Request/Response
Sample Request/Response
Exceptions
Information in responses given in a human- and machine-readable way
No such JobID• JobID in request does not exist
No Authority• Authenticated user did not start the requested job
JobID empty• No JobID is set in the request
Job already cancelled• Job has been canceled before
Job not running• Job is paused or cancelled
Job not paused • Job is already resumed or still running
Challenges
• Main problem: Persist jobs
o Once a job it paused it makes no sence to hold intermediary results in the RAM
o Goal: Store results after a job is paused on the hard drive
o But how to implement it?
Challenges
Considered Solution: Serialization
• Every Algorithm has to be serializable
• Advantage: Intermediary results can be stored on hard drive and the job can be resumed after deserialization
• Disadvantage: WPS-P becomes less generic
Challenges
• Problem: Job information
o Owner, priority and submission time are sent in SOAP-Header
o Information has to be assigned to correspondent WPS process
Challenges
• Solution:Metadata population through SOAP-Header-Processor
o Header information are extracted at server side
o Storing of metadata in a "JobMetadata"-Object
o Building of correspondent request with appropriate metadata information
Challenges
• Problem: Job organization
o Incoming Jobs have to be arranged to have access to them
o Every job has to get an unique ID
Challenges
• Solution: PriorityRegistry
o Jobs are stored as PriorityRegistryEntries in a concurrent hash map
o Entries have additional attributes "status" and "job finished time"
o Unique ID is generated via a hash-method over the metadata
Internal Workflow
Operation Dispatcher
• Forwards request tocorrespondenthandler
SOAP-Header Processor
• Extractsinformationfrom SOAP –Header
MetadataPopulator
• Gains additional metadata
PriorityRequestExecutor
• Executes thejob andmodifies theregistry
PriorityRequestHandler
• Adds/Modifiesjob in PriorityRegistry
uDig-Plugin
• 52North developed Plugin for uDig• Extended Plugin supports WPS-P
o also supports "normal" WPS• Plugin is able to detect if the Web Processing Service is priority
enabledo PriorityEnabled Element in Capabilities Document
uDig-Plugin
uDig-Plugin
uDig-Plugin
uDig-Plugin
uDig-Plugin
Future Work
Implementation: • Enhance uDig abilities (support all WPS-P operations)• Persistance
Conceptual:• Currently every autherized user can choose the priority itself
o Use licenses to manage job priorities
Thank you
