EMI is partially funded by the European Commission under Grant Agreement RI-261611

Discovering Infrastructures with EMI Registry (EMIR)

Emidio Giorgio – INFN - IGIOn behalf of Shiraz Memon, Jülich Supercomputing Center

ISGC 2012, 29.02.2012

EMI I

NFS

O-R

I-261

611

• Motivation• Overview• High level architecture

– Registry Hierarchies– Global level replication– Information Model

• Integration with EMI Services – UNICORE, ARC• Conclusions

Outline

19/04/23 ISGC 2012 2

EMI I

NFS

O-R

I-261

611

• EMI Registry (EMIR)

Motivation

19/04/23 ISGC 2012 3

• Lack of a common service index; within EMI– UNICORE (Global) Registry– ARC ISIS– Site / Top Level BDII

• Middleware specific Discovery • Various Information Models• Not Federation Aware

Unfortunately, nothing to do with oil

motivationmotivation

EMI I

NFS

O-R

I-261

611

• Cross Middleware Service Discovery• Supports Hierarchies (Theoretically infinite)• Common Access Point and Unified Method

to service registrations and discovery• Lightweight REST-full API• Standards based GLUE 2.0 Information

Model

EMIR: A Unified Service Registry

19/04/23 ISGC 2012 4motivationmotivation

EMI I

NFS

O-R

I-261

611

• Domain Service Registry (DSR)– A registry node at any level of the hierarchy– Registration and Rich Query Interface– PKI based Authentication– XACML or simplified ACL for Authorization– Synchronization with a parent registry node– Filtered Registrations

• Global Service Registry (GSR)– Similar to DSR– Replication with other GSR using P2P techniques– Bootstrap from publicly available Global List

Main Components

19/04/23 ISGC 2012 5overviewoverview

EMI I

NFS

O-R

I-261

611

High Level Architecture

19/04/23 ISGC 2012 6High level architectureHigh level architecture

EMI I

NFS

O-R

I-261

611

Communication of DSRs (1/2)

19/04/23 ISGC 2012 7

<<Child>>

DSR

<<Parent>>

DSR

Service

publish

<<Global>>GSR

Domain

Computing Center

State/Region

Country

Publish

Publish

Liveness at the GSR ∞ Level of Hierarchy

High level architectureHigh level architecture

EMI I

NFS

O-R

I-261

611

• Predefined Parent ref. at the Child DSR• Synchronization is done through events• Active Sync.

– Parent notified as soon as the changes occurred in the child DSR

• Passive Sync.– Parent notified as soon as it joins the hierarchy

(fresh startup or recovered from a failure)– Temporary caching

Communication among DSRs (2/2)

19/04/23 ISGC 2012 8High level architectureHigh level architecture

EMI I

NFS

O-R

I-261

611

Global Level Replication

19/04/23 ISGC 2012 9

• Replication of Registries based on (structured) P2P

• Data Synchronization at the Global level

• Why? • Federation level Service

Discovery• Failover• Robustness• Scalability• Load Balancing

• Same query and registration interface• Where to find the GSRs?

• Publicly known list

<<Global>>

GSR

<<Global>>

GSR

<<Global>>

GSR

https://gsr1,https://gsr2,https://

gsr3 Bootstrap

<<Global>>

GSR

GSR List

High level architectureHigh level architecture

EMI I

NFS

O-R

I-261

611

• Schema free rendering• Adopts GLUE 2.0’s generic service entities• Supports standard XML as a response on

query functions• JSON for message exchange (query, de-

registration) and service record• Expiration time for every service record• Constraining service URL as “unique”

identifier

Information Model

19/04/23 ISGC 2012 10High level architectureHigh level architecture

EMI I

NFS

O-R

I-261

611

• Service Name• Service Type• Endpoint URL• Service Capabilities• Downtime Info.

JSON Rendering

19/04/23 ISGC 2012 11

[{ "Service_Name": “EMI Execution Service", "Service_CreationTime": {"$date": "2011-10-16T11:47:24Z"}, "Service_Type": “eu.emi.es", "Service_Capability": [“activity submission", “activity creation"], "Service_QualityLevel": "production", "Service_Endpoint_URL": "http://eu-emi.eu/emi-es", "Service_Endpoint_Technology": “SOAP over HTTP", "Service_Endpoint_InterfaceName": “ActivityManagement", "Service_Endpoint_InterfaceVersion": ["1.0“ ], "Service_Endpoint_WSDL": "http://eu-emi.eu/emi-es ?wsdl", "Service_Endpoint_HealthState": "ok", "Service_Endpoint_HealthStateInfo": “it is OK, I can see it", "Service_Endpoint_ServingState": "production", "Service_Endpoint_StartTime": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_DowntimeAnnounce": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_DowntimeStart": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_DowntimeEnd": {"$date": "2011-07-21T11:47:24Z"}, "Service_Endpoint_QualityLevel": "production", "Service_ExpireOn": {"$date": "2011-07-21T11:47:24Z"}, "Service_UpdateOn": {"$date": "2011-07-21T11:47:24Z"}, "Service_OwnerDN": “CN=Shiraz Memon,O=FZJ,OU=JSC” }]

High level architectureHigh level architecture

EMI I

NFS

O-R

I-261

611

Integration with EMI Services

19/04/23 ISGC 2012 12

Services•TargetSystemFactory •BESFactory•EMI-ES•StorageFactory

UNICORE

Services•AREX Compute•gridftpd•Chelonia Storage

ARC

Register

Register

ARC Client

UNICORE Command Line Client (UCC)

Lookup & Discover EMI Services

Integration with EMI

EMI I

NFS

O-R

I-261

611

• Integration with all the EMI Services• Automatic (de)commissioning of a Registry

node in the hierarchical network• Handling synchronization and failures while

replicating the large number of service entries

Future Work

19/04/23 ISGC 2012 13Conclusions

EMI I

NFS

O-R

I-261

611

• Common Service Index for EMI• Support for federation• Architecture• Information Model• Integration Scenario

Summary

19/04/23 ISGC 2012 14Conclusions

EMI I

NFS

O-R

I-261

611

• Documentation • (REST API (WADL Document)

• Wiki• Code• Packages• Contacts => EMI Forum

Links

19/04/23 ISGC 2012 15Conclusions

EMI I

NFS

O-R

I-261

611

Questions ?

19/04/23 ISGC 2012 16Conclusions