Embed Size (px)
DESCRIPTION
GRIP: Interoperability between UNICORE and Globus. D. Erwin, M. Rambadt, Ph. Wieder Zentralinstitut für Angewandte Mathematik Forschungszentrum Jülich Terena 2002, Limerick, Ireland June 5, 2002 - Session 7A. Contents. GRIP Partners UNICORE and Globus GRIP Objectives - PowerPoint PPT Presentation
Citation preview
GRIP: Interoperability between GRIP: Interoperability between UNICORE and GlobusUNICORE and Globus
D. Erwin, M. Rambadt, Ph. Wieder
Zentralinstitut für Angewandte Mathematik
Forschungszentrum Jülich
Terena 2002, Limerick, Ireland
June 5, 2002 - Session 7A
June 5, 2002 Terena 2002 2
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
ContentsContents
June 5, 2002 Terena 2002 3
GRIP: GRid Interoperability Project
Funded in part by EU grant IST 2001-32257
Duration: January 2002 – December 2003
Budget: 1.9 Mio €
URL: www.interoperability-project.org
June 5, 2002 Terena 2002 4
• Forschungszentrum Jülich, D (Coordinator)
• Pallas GmbH, D
• University of Manchester, UK
• University of Warsaw, PL
• Deutscher Wetterdienst, D
• University Southampton, UK
• Fecit, UK
• Argonne National Laboratory, US
Foils created by partners of the GRIP project
PartnersPartners
June 5, 2002 Terena 2002 5
GRIP Partners
UNICORE and Globus UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002 Terena 2002 6
UNICORE:
UNiform Interface to COmputing Resources
• conceived prior to the ‘invention’ of the GRID
• UNICORE has both a modest and an ambitious goal:
• Create– Seamless– Secure– Intuitive
access to distributed German HPC resources• Create a system that can be used in production at the
German HPC centers
UNICORE GoalsUNICORE Goals
June 5, 2002 Terena 2002 7
LRZ: SR 8000, VPPRUS: Hitachi, SX-5FZJ: 2xT3E, T90, SV1ZIB: T3E, SGIDWD: T3E, IBM SPRUKA: IBM SPPC²: hpcLineTUD: SGIPallas: Sunfecit: VPP, Sun
FZJFZJ
PC²PC²ZIBZIB
TUDTUD
LRZLRZ
RUSRUS
DWDDWD
PallasPallas
RUKaRUKa
June 5, 2002 Terena 2002 8
• Creation of system-independent jobs
• Execution at different sites
• Creation of complex interdependent multi-system and multi-site jobs
• Monitoring and control of jobs
• Dependencies
• Workflow
UNICORE FunctionsUNICORE Functions
June 5, 2002 Terena 2002 9
• Transfer of data to and from the workstation
• Automatic data transfer between systems and sites
• Access to data in Unix file systems and archives
• Application support
• No changes to third party applications
• Support for legacy job scripts
• Support for Metacomputing
• Performance analysis using VAMPIR
UNICORE FunctionsUNICORE Functions
June 5, 2002 Terena 2002 10
• Single sign-on
• No changes to local user management and naming conventions
• Secure communication over the Internet
• Support for all platforms, operating systems, and batch systems at partner sites
• Retaining of administrative autonomy and security policies of participating sites
UNICORE FunctionsUNICORE Functions
June 5, 2002 Terena 2002 11
• Continued development and support after the end of the project
• Basis for additional national and international projects
• Production quality prototype
Objects will be achieved by year end 2002
URL: www.unicore.de
UNICORE FunctionsUNICORE Functions
June 5, 2002 Terena 2002 12
• European Grid Testbed
• Based on UNICORE technology
• Development of additional UNICORE functions
• Funded in part by EU grant IST-1999-29247
• Duration: November 2000 - October 2003
URL: www.eurogrid.org
EUROGRIDEUROGRID
June 5, 2002 Terena 2002 13
GeographyGeography
June 5, 2002 Terena 2002 14
– Forschungszentrum Jülich, D– Pallas GmbH, D (Project Coordinator)
– Parallab - University of Bergen, N– CNRS - IDRIS, F– Warsaw University - ICM, PL– Victoria University of Manchester, UK– Deutscher Wetterdienst, D– GIE EADS CCR, F– ETH Zürich (CSCS Manno), CH– Fecit, UK– T-Systems, D
EUROGRID PartnerEUROGRID Partner
June 5, 2002 Terena 2002 15
• Development by Argonne National Laboratory and other US partners
• Defines Protocols and APIs for GRIDs
• Toolbox to enable development of GRID aware applications
• Basis for many international Grid projects
GlobusGlobus
June 5, 2002 Terena 2002 16
Application
Layer
Collective
Layer
Resource
Layer
Fabric
Layer
Grid ArchitectureGrid Architecture
Applications, Portals, Environments
Higher Services, APIs, Protocols
APIs, Protocols, SDKs
Computers, Networks, Devices
June 5, 2002 Terena 2002 17
Application
Layer
Collective
Layer
Resource
Layer
Fabric
Layer
Applications, Portals, Environments
Higher Services, APIs, Protocols
APIs, Protocols, SDKs
Computers, Networks, DevicesGlobus
Grid ArchitectureGrid Architecture
June 5, 2002 Terena 2002 18
Application
Layer
Collective
Layer
Resource
Layer
Fabric
Layer
Applications, Portals, Environments
Higher Services, APIs, Protocols
APIs, Protocols, SDKs
Computers, Networks, Devices
Legion
Grid ArchitectureGrid Architecture
June 5, 2002 Terena 2002 19
Application
Layer
Collective
Layer
Resource
Layer
Fabric
Layer
Applications, Portals, Environments
Higher Services, APIs, Protocols
APIs, Protocols, SDKs
Computers, Networks, Devices
UNICORE
Grid ArchitectureGrid Architecture
June 5, 2002 Terena 2002 20
Globus
Application
Layer
Collective
Layer
Resource
Layer
Fabric
Layer
Applications, Portals, Environments
Higher Services, APIs, Protocols
APIs, Protocols, SDKs
Computers, Networks, Devices
UNICORE
Grid ArchitectureGrid Architecture
June 5, 2002 Terena 2002 21
GRIP Partners
UNICORE and Globus
GRIP ObjectivesGRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002 Terena 2002 22
• Develop software to facilitate interoperation between UNICORE and Globus combining the unique strength of each system (Pallas, fecit, UoM, FZJ, ANL)
• Build and demonstrate biomolecular (MD & QC codes) and meteorological (RLM) inter-grid applications (ICM, DWD, SOTON)
• Create wrappers to support commercial applications in a Grid (no code changes)
• Contribute to and influence international Grid standards through the Global Grid Forum
GRIP ObjectivesGRIP Objectives
June 5, 2002 Terena 2002 23
• Run a UNICORE job on Globus resources
• Run Jobs using Globus and UNICORE resources together
• Prototype proposed standards in future versions of UNICORE
GRIP ObjectivesGRIP Objectives
June 5, 2002 Terena 2002 24
Key UNICORE Technologies
• Seamless Computing Model– Job abstraction– Incarnation– File staging and transfer support
• Security Model– X509 certificate based security model– Consigner plus endorser model– Several CA policies– Fully flexible firewall support
• Generic Client– No changes to application needed– Handy GUI environment
• Portable Server– Java + Perl
June 5, 2002 Terena 2002 25
Parallels and Contrasts
• Resource Description– UNICORE: Same model for discovery and request– Globus: Different models for discovery and request
• Jobs vs Applications– UNICORE: Workflow environment– Globus: Application APIs and toolkit
• Security– UNICORE: End-to-End security model– Globus: Requires transitive trust
• Incarnation and Grounding– UNICORE: Incarnation of abstractions at server– Globus: Client side grounding (substitution to ‘normal form’)?
• Protocol Complexities– UNICORE: Polling (request/response)– Globus: Call back
June 5, 2002 Terena 2002 26
Missing Technologies
• Missing Models in UNICORE– Distributed directory services– Multi-language Implementations– Application level message passing– Event management– Synchronized job start
• Implementation to be completed in UNICORE – Resource reservation– Interactive processing – Dynamic support for new Vsites on the fly
June 5, 2002 Terena 2002 27
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP ArchitectureThe GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002 Terena 2002 28
User Workstation
UNICORE GUI
UNICORE Server
Gateway
UNICORE Site 1
UNICORE Server
Gateway
UNICORE Site n
SSL
TCP/IP TCP/IP
Site ListHTTP
SSL
BatchsystemBatchsystemTarget System Interface
Batch SubSystem
BatchsystemBatchsystemTarget System Interface
Batch SubSystem
Network Job SupervisorNetwork Job SupervisorNetwork Job SupervisorNetwork Job SupervisorNetwork Job SupervisorNetwork Job Supervisor
UNICORE ArchitectureUNICORE Architecture
June 5, 2002 Terena 2002 29
Server@Usite
X.509User CertificateJob Preparation Agent
(JPA)Job Monitor Controller
(JMC)
User@WorkstationUNICORE Client
hostsin a Vsite
system1TS Interface
batch subsystema
systemnTS Interface
batch subsystemx
batch jobs, status requests, data
Abstract jobs, status requests
AuthenticationUNICORE Login
GatewayUser validationSite-specific
authentication
optional firewall
optional firewall
Localuser DB
Localuser DB
Network Job Supervisor (NJS) (Incarnation/Scheduling)Network Job Supervisora (NJS) (Incarnation/Scheduling)
Incarnation DBIncarnation DB
UNICORE ArchitectureUNICORE Architecture
June 5, 2002 Terena 2002 30
Proposed Architecture
Clientgrid-proxy-init
plugin
(Logical) Globus UsiteGateway
Vsite ANJS
Vsite BNJS
GenericGlobus IDB
Globus JMHost A
Globus JMHost B
Globus TSI Globus TSI
GlobusGIIS
IDBMaker
IDBHost A
IDBHost B
GRIS GRIS
June 5, 2002 Terena 2002 31
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototypeExperience with an early prototype
GRIP and OSGA
June 5, 2002 Terena 2002 32
Enhanced Target System Interface
Creation
Request
GRAM job request
Callback
Data
Batch Job
Data
June 5, 2002 Terena 2002 33
Simplified Architecture implemented by M. Rambadt as part of his Master thesis:
• No changes to UNICORE or Globus
• Fixed Globus resource as a Visite
• Extended TSI interfaces UNICORE and Globus:
– To create proxy certificates
– To map AJO to RSL
– To acts as Globus client
– To return results and status from Globus to NJS and Client
Proof of concept
ExperienceExperience
June 5, 2002 Terena 2002 34
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGAGRIP and OSGA
June 5, 2002 Terena 2002 35
• First GRIP deliverable (end 2002) will interface UNICORE and Globus 2.0
• Project partners work with GGF on future developments
• OGSA is an important topic
• OSGA will be considered by GRIP project for 2003 (requires agreement by EU)
OutlookOutlook
June 5, 2002 Terena 2002 36
• Architecturally OGSA and UNICORE are very compatible
• WDSL service types for AJO classes
– Semi-automatic translation for Web Services
– More work for OGSA extension
– Demonstrated by Dave Snelling in Manchester
• Authorization and Security
– Not yet fully defined in OGSA
– UNICORE expects X.509 to Unix login mapping
OutlookOutlook
June 5, 2002 Terena 2002 37
• UNICORE will continue to contribute to Grid developments
• Grid development will benefit from experience gained at UNICORE Test and Production sites
• Accepted standards with proven implementations are essential
OutlookOutlook
June 5, 2002 Terena 2002 38
Web addresses:
• http://www.unicore.de Information about the project
• http://www.fz-juelich.de/unicore-testFree UNICORE Client Software
• http://www.unicore.orgDownload for Software + Sources
• http://www.eurogrid.org Information about EUROGRID
• http://www.grid-interoperability.orgInformation about GRIP
June 5, 2002 Terena 2002 39
?http://www.unicore.de
http://www.fz-juelich.de/unicore-test
http://www.eurogrid.org
http://www.grid-interoperability.org
![InterGrid: A Case for Internetworking Islands of Grids · 2016. 8. 30. · Globus [4], Legion [5], UNICORE [6] and gLite [7]; schedulers like Application Level Schedulers (AppLeS)](https://img.pdfslide.net/doc/110x75/602e7c78301bf378255be197/intergrid-a-case-for-internetworking-islands-of-2016-8-30-globus-4-legion.jpg)
