Embed Size (px)
Citation preview
Grid Resources for Industrial Applications
Steve Taylor
IT Innovation Centre
19 October 2004
Introduction
• GRIA is an infrastructure providing business process management of computational resources
• Aimed at business users – e.g. HPC providers, SME clients
• GRIA enables:– HPC providers to rent out their spare CPU cycles– Clients to outsource HPC application codes
• Application focused– HPC providers connect GRIA to application codes &
advertise this as a service
Functionality and Positioning
• GRIA = A Grid for Today– secure "off the shelf" Web Services technology– supports well-defined B2B processes– designed to support today’s legacy applications– available immediately
• Key features– file storage and processing infrastructure– accounting: billing and billability (trust)– QoS: managing service load
Business Process OverviewSERVICE PROVIDER ORGANISATIONCLIENT ORGANISATION
BudgetHolder
ServiceProviderSystems
Account
Interface
JobS
ubmission
Interface
Data U
pload /D
ownload
InterfaceA
ccou
nt S
ervi
ceJo
b R
unni
ngS
ervi
ce
AccountDetails
Credit and Payment Details
(Pricing Terms)
(Pricing Requirements)
Specify jobs, data storage,QoS needs
QoS Offer
Input data Stager, Output Data Stager
Output data
Input Data
Check job progress
MainUser
Account ID and URI
Res
ourc
eA
lloca
tion
Ser
vice
Dat
a S
tagi
ngS
ervi
ce
Allocation
Interface
Start
Business Model Features
• Hierarchical business model• Contractual terms are agreed per account
– the relationship between the client and service provider
– no need to re-specify for each job
• Users can request multi-job, long term Resource Allocations within an account
• Jobs and data storage are set up within the context of a resource allocation– all users need to do is run the job
Business Model Features• Jobs and Data are handled by separate
services– data may be uploaded to a data stager, and used
in a different jobs
• Delegation is used extensively– account holders can delegate access to the
account– resource allocation owners can delegate access to
the allocation– data stager owners can delegate access to that
data stager
GRIA Business Process
AccountAccount
(Trust)(Trust)
ServiceService
App requirementsApp requirements
CC NumberCC Number
Account RefAccount Ref
(ContextID)(ContextID)
GRIA Business Process
AccountAccount
(Trust)(Trust)
ServiceService
Trust(AccountRef,Trust(AccountRef,
User Credentials)User Credentials)
ConfirmUser ConfirmUser
User CredentialsUser Credentials
GRIA Business Process
AccountAccount
(Trust)(Trust)
ServiceService
InitResourceAlloc(ResourceService,InitResourceAlloc(ResourceService,
AccountRef)AccountRef)
ResourceRefResourceRef
(ContextID)(ContextID)
ResourceResource
AllocationAllocation
ServiceService
GetAlloc(ResourceRef,GetAlloc(ResourceRef,
Requirements)Requirements)
Allocation OfferAllocation Offer
GRIA Business Process
ResourceResource
AllocationAllocation
ServiceService
InitJob(JobService,InitJob(JobService,
ResourceRef)ResourceRef)
JobRefJobRef
(ContextID)(ContextID)
JobJob
ServiceService
StartJob(JobRef,StartJob(JobRef,
InputRef, OutputRef)InputRef, OutputRef)
AcknowedgementAcknowedgement
GRIA Business Process
JobJob
ServiceService
Check(JobRef)Check(JobRef)
StatusStatus
DataData
ServiceService
Read(OutputRef)Read(OutputRef)
DataData
GRIA Business Process
DataData
ServiceService
AllowRead(DataRef,AllowRead(DataRef,
CustomerCredential)CustomerCredential)
ConfirmUserConfirmUser
Read(DataRef)Read(DataRef)
DataData
Customer CredentialsCustomer Credentials
GRIA Business Process
AccountAccount
(Trust)(Trust)
ServiceServiceStatement Statement
Check(AccountRef)Check(AccountRef)
out of band settlementout of band settlement(payment logging)(payment logging)
Accounting Model
• An account is given a credit limit & incurs liability– liability must always be less than credit limit!
• Credit limit determined when account is opened• Liability is increased when a resource allocation is
confirmed– the financial cost of the entire resource allocation
• Liability is reduced by payments
Account Service
• Provides a means for clients to open & maintain accounts– request opening of account– get statements– close accounts
• Financial operations handled out of band– payments, credit checks, etc– internal operations to log payments, update liability, etc
Resource Allocation Service• Used within the context of an account• User can request a resource allocation
– service responds with an offer or a failure
• User may accept or reject an offer– acceptance transforms the offer into an allocation– rejection removes the pending offer
• User may initialise job and data services in an allocation
• User may request an extension to an existing allocation– e.g. time, disk space
Job Service
• Used within the context of a resource allocation• Used to run computational jobs
– retrieve input (subject to authorisation)– submit job to resource manager– monitor job– write output (subject to authorisation)
• All inputs and outputs are URIs & context IDs of data stagers
Data Service• Used within the context of a resource allocation• Manages data staging areas
– each data staging area is identified by URI of data service and context ID
• Users can upload and download data and check the status of a staging area
• Users can delegate access to others (e.g. remote job services or other users) for access to the data in the staging area
High Level Architecture
• Secure– X.509 compliant public key encryption– Certification policy for user authentication – Services accessed via Apache HTTP server and
Mod_SSL: popular and well-patched
• Modular– Distinct roles for software modules– Code reuse with other projects– GRIA v2 a distribution of modules, some from outside
GRIA
Client Architecture
ConversationObjects
Clie
ntA
PI
NetworkSecurity
Service
Proxies
MessageTransport
SimpleWS-Security
PluginHelpers
Conversation/Service Registry
SecurityContext Manager
Clie
ntA
pplic
atio
n
Client SecurityClient Infrastructure
App
licat
ion
Wor
kloa
dE
stim
ator
IT Innov
IAM
NTUA
Users
Dolphin GEMSS
UK e-Science
Third Party
HT
TP
S
Service Provider Architecture
SCI Management
Message Router
PBACAuthorisation
HT
TP
S(A
pach
e/M
odS
SL
TrustRelationship
Service
ResourceAllocationService
DataStorageService
JobSubmission
Service
FileStore
ResourceModel
Service Framework Basic ServicesNetwork Security Resources
Cluster
Internal Admin
Message Transport
SimpleWS-Security
Plugin
Service Implementation Factory
Used By AllServices
Benchmarks
IT Innov
IAM
NTUA
Users
Dolphin GEMSS
UK e-Science
Third Party
Client Installation
Client machine
CRL
Keystore
Private key
CA certificate
GRIA Client
Java
CA signed cert
Public key
GRIA software
Pre-requisites
Service Provider Installation
GRIA Host(SUSE 9.0
Linux)
Resource Cluster
Shared FileSystem
GRIA Services
Postgres
Apache
Tomcat
PBS SubmitNode
CRL
Applications(INDIA, AIR, etc)
ApplicationWrappers
PBS Client
Java
GRIAsoftware
Pre-requisitesbundled with SUSE 9.0
Third partyapplications
Keystore
Private key
CA certificate
CA signed cert
Public key
PBS Master
Status and Future Plans• GRIA project status
– project duration: 12/2001 – 11/2004– GRIA v3 software currently being evaluated by partners
• GRIA used & further developed in ongoing research
• GRIA v3 will be released open source under LGPL– freely available – [email protected]– commercial software support/customisation from IT
Innovation– selected application services from Dolphin
