13
Data Management Expert Panel - WP2

Data Management Expert Panel - WP2. WP2 Overview

Embed Size (px)

Citation preview

Data Management Expert Panel - WP2

WP2 Overview

Talk Outline

Introduction to EDG Work Package 2

WP2 Services: Design and Interactions Spitfire Replication Services Grid Simulation Security

Status

Grid middleware architecture hourglass

Current Grid architectural functional blocks:

OS, Storage & Network services

Basic Grid Services

High Level Grid Services

Grid Application Services (LCG)Common application layer

CMS ATLAS CMS LHCbSpecific application layer

GLOBUS 2.2

EU DataGrid

WP2

EU DataGrid WP2Data Management Work Package

Responsible for

Transparent data location and secure access

Wide-area replication

Data access optimization

Metadata access

NOT responsible for (but partially relying on other WPs for)

Data storage

Proper Relational Database bindings

Remote I/O

Security infrastructure

WP2 Service Paradigms

Choice of technology: Web Services (servers implemented in Java)

Tomcat, Oracle 9iAS

Interface definitions are exposed in WSDL

Client stubs for many languages (Java, C, C++) Axis, gSOAP (auto-generated)

Persistent service data in Relational Databases MySQL, Oracle

Modularity Modular service design for pluggability and extensibility

No vendor specific lock-ins

Evolvable Easy adaptation to OGSA foreseen, based on the same technology

Largely independent of underlying OS, RDBMS

Spitfire: Grid-enabling RDBMS

Capabilities: Simple Grid enabled front-end to any type of local or remote

RDBMS through secure web services

Sample generic RDBMS methods may easily be customized with little additional development

Web browser integration

GSI authentication

Hooks in place for local authorization

Status: current release version 2.1 Used by EU DataGrid Earth Observation and Biomedical

applications.

Not currently suitable for the retrieval of LARGE result sets

StorageElement

Replication Services: Basic Functionality

Replica ManagerReplica Location

Service

Replica Metadata Catalog

StorageElement

Files have replicas stored at many Grid sites on Storage Elements.

Each file has a unique Grid ID (GUID). Replica Location Service maps the GUID to the multiple physical locations of that file.

Users may assign aliases to the GUIDs. These are kept in the Replica Metadata Catalog.

The Replica Manager provides atomicity for file operations, assuring consistency of SE and catalog contents.

StorageElement

Higher Level Replication Services

Replica ManagerReplica Location

Service

Replica Optimization Service

Replica Metadata Catalog

SEMonitor

Network MonitorStorageElement

The Replica Manager calls the Replica Optimization service to find the best replica based upon network and SE monitoring information.

StorageElement

Interactions with other Grid components

Replica ManagerReplica Location

Service

Replica Optimization Service

Replica Metadata Catalog

SEMonitor

Network Monitor

Information Service

Resource Broker

User Interface orWorker Node

StorageElement

Virtual OrganizationMembership Service

Applications and users will manage data only through the Replica Manager - either directly or via the Resource Broker. Management calls should never go directly to the SE.

Grid Simulation (OptorSim)

Standalone data-centric Grid simulation used to develop and evaluate replication strategies

- Grid2003: Simulation e.g. of CMS spring 2002 testbed.- 1000’s of jobs, ~100 GB files (50 GB capacity SEs).- Access patterns based on measured CDF analysis jobs.- To add in measured background traffic on network links

Security: Infrastructure for Java-based Web Services Trust Manager

Mutual client-server authentication using GSI (ie PKI X509 certificates) for all WP2 services

Supports everything transported over SSL

Authorization Manager Supports coarse grained authorization:

Mapping user DN -> role -> attribute

Fine grained authorization through policies, role and attribute maps

Web-based Admin interface for managing the authorization policies and tables

Status: Fully implemented, authentication is enabled on the service level

Delegation implementation currently being developed

Authorization (using VOMS) currently being integrated with WP2 services.

WP2 Status

Current Status All components are available now

Initial tests show that expected performance can be met

Need proper testing in a ‘real user environment’ – EDG2; LCG1

Good results from OptorSim. Work continuing.

Work-plan for next release Full integration of the authorization module.

Replica Location Index.

See James’ talk.