View
41
Download
0
Category
Preview:
DESCRIPTION
Summary of distributed tools of potential use for JRA3. Dugan Witherick HPC Programmer for the Miracle Consortium University College London JRA3 Workshop @ UCL: 22-24 April 2010. Talk Outline. Summarise the Goals of JRA3 Tasks 3 and 4 - PowerPoint PPT Presentation
Citation preview
http://europlanet-ri.eu
Summary of distributed tools of potential use for JRA3
Dugan WitherickHPC Programmer for the Miracle Consortium
University College LondonJRA3 Workshop @ UCL: 22-24 April 2010
Talk Outline
2JRA3 Workshop @ UCL: 22-24/04/2010
Summarise the Goals of JRA3 Tasks 3 and 4 Task 3: Distributed Numerical Modelling Laboratory (DNML) Task 4: Distributed Data Analysis Laboratory (DDAL)
What is IDIS? What are other people doing (two examples)? A possible solution from the Virtual Observatory
Who’s using Virtual Observatory Technology The Universal Worker Service
3JRA3 Workshop @ UCL: 22-24/04/2010
Summary of Tasks 3 and 4 Goals (a reminder)
4JRA3 Workshop @ UCL: 22-24/04/2010
Task 3: The main goal of this task is to bring together the numerical simulation efforts
currently underway in Europe and to create a coordinated European multi-disciplinary modelling facility for Planetary Science: a Distributed Numerical Modelling Laboratory (DNML).
….feeding into the catalogue part of SA IDIS. Task 4:
The main goal of this task is to bring together expert teams in data analysis with their present research infrastructures and create a coordinated European interdisciplinary data analysis facility for Planetary Science: a Distributed Data Analysis Laboratory (DDAL).
….feeding into the catalogue part of SA IDIS. The main technical requirements for these tasks could be satisfied with the
same solution
5JRA3 Workshop @ UCL: 22-24/04/2010
Integrated and Distributed Information Service(IDIS)
6JRA3 Workshop @ UCL: 22-24/04/2010
What is the purpose of IDIS? The task of IDIS is to provide an easy-to-use web-based platform to give access to
available data, initiate new research activities needed to interpret observations or to solve open questions, locate teams and laboratories with special knowledge needed, and exploit synergies between space-based missions and capabilities of ground-based observatories.
A set of tools for describing, accessing and combining information and data from different sources will be developed, with the objective to offer a Virtual Observatory like access to many data essential for planetary research from European and non-European sources.
Any solution we come up with must fit in with the existing plans of IDIS
7JRA3 Workshop @ UCL: 22-24/04/2010
What are other people doing?(two examples)
8JRA3 Workshop @ UCL: 22-24/04/2010
Automated Multi Dataset Analysis (AMDA) A generic Webtool for Space Physics data
o automated event search and characterisationo catalogue generation and exploitationo automated database conditional extractiono access to remote Data Centers
Web portal located at http://cdpp-amda.cesr.fr/ The service can be reached and used from other sites according to the Virtual
Observatory paradigm. Not so much a distributed data analysis tool but a single tool linked into the
Virtual Observatory
9JRA3 Workshop @ UCL: 22-24/04/2010
10JRA3 Workshop @ UCL: 22-24/04/2010
Community Coordinated Modelling Centre CCMC http://ccmc.gsfc.nasa.gov/ Focus on Space Weather
Models are run through an automated request system: Instant Run – results are provided immediately or Run on Request – user is informed by email when run has completed
o These longer running models are added to a queue to run on their Beowulf Cluster
Capable of running more than one model/analysis tool Not really distributed
Appears that all work is being completed at one site
11JRA3 Workshop @ UCL: 22-24/04/2010
12JRA3 Workshop @ UCL: 22-24/04/2010
13JRA3 Workshop @ UCL: 22-24/04/2010
Remember IDIS (Distributed is a keyword)
14JRA3 Workshop @ UCL: 22-24/04/2010
15JRA3 Workshop @ UCL: 22-24/04/2010
A Possible Solution from the Virtual Observatory
16JRA3 Workshop @ UCL: 22-24/04/2010
Common Execution Controller (CEC) Web service for running various asynchronous applications. Part of the Common Execution Architecture (CEA) suite of software from Astrogrid. Designed to wrap existing applications so that they are presented as IVOA standard
web services.o Commandline applicationso Existing web form based HTTP applicationso Custom written Java applications
Suitable for applications that:o have no graphical interfaceo need no user interaction once launch
Supports the following standards: UWS, VOSI, General Security, Registry, Delegation, VOSpace
JRA3 Workshop @ UCL: 22-24/04/2010 17
The Universal Worker Service (UWS) An International Virtual Observatory Specification http://www.ivoa.net/Documents/UWS/ The Universal Worker Service (UWS) pattern defines how to manage asynchronous
execution of jobs on a service Has features over the previous standards of
o Control for job lifetimeo Access to user sandbox (in order to inspect intermediate files
There are UWS deployments that do not use the CEAo Not necessary to use CEA
18JRA3 Workshop @ UCL: 22-24/04/2010
Virtual Observatory Registry Essentially the “Yellow Pages” or Telephone Directory for services User client (web portal or client tool) is used to search the registry
o Information about the resource is returned Brief summary Contact Name Inputs for the model/tool Location for the model/tool
Two types of registryo Local registry which lists the services installed locallyo Full registry harvests the local registries to find out what they have installedo Users search the full registry
JRA3 Workshop @ UCL: 22-24/04/2010 19
JRA3 Workshop @ UCL: 22-24/04/2010 20
JRA3 Workshop @ UCL: 22-24/04/2010 21
Advantages to this plan Allows us to take advantage of the extensive effort made in developing standards
o Saves us a lot of effort We can use/modify existing VO compliant tools
o Tool for OPerations on Catalogues And Tableso AladinVO – VO compliant image analysis toolo VOSpec – spectral analysis tool
Possible access to workflow tools Provides us with VO access to other data/tools Some IDIS data sources are already published to the VO
Disadvantages to this plan Unclear whether this would be supported by IDIS The VO has had its own problems which we may end up getting caught in This may not be appropriate for fulfilling our deliverables
JRA3 Workshop @ UCL: 22-24/04/2010 22
Recommended