Upload
angelina-buchanan
View
214
Download
0
Embed Size (px)
Citation preview
Grid Architectures
Steve Fisher/RAL
GridPP, Edinburgh, November 2001
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 2
GGF - GPA• I will be talking about the GGF Grid Protocol
Architecture Working Group (http://www-itg.lbl.gov/GPA/)– It is THE body to address standardisation of Grid
Architectures– Chaired by Bill Johnston, Ian Foster and Reagan
Moore – Two meetings so far
• Some of my slides derived from:– Bill Johnston’s summary of GGF3 – and talks given there
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 3
Revised Proposed Charter• The role of the Grid Protocol Architecture Working
Group is to provide a conceptual framework for discussing the interrelationships, completeness, and minimality of the protocol approach to Grid services that is coming out of GGF
• The GPA will initially have five work areas:– produce an initial overview architecture to provide a starting
point– explore the issue of different “views” of Grids– establish a methodology for representing the architecture– document the relationship of the the GGF architecture to
other, existing, Grid-like architectures from outside GGF– evolve the initial overview architecture
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 4
Issues• How do we answer the question “what is a minimal
set of Grid functions/protocols” and how do we represent their inter-relationships?– characterise existing Gird protocols and interfaces and
where GF WGs are currently working– look at existing architecture representations– Is UML suitable?
• Important to make sure that we are considering a sufficient set of services– examine what is currently bring done, or trying to be done,
with Grids as the first approach
• There are different views of Grids– will these all admit to a common architecture?– If not, how do they differ?
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 5
Is UML suitable?• What is architecture?
– divisions into sub-systems– and the control model
• Often represented by simple box and line diagrams– Just show the sub-systems and main communication– Easy to understand
• UML Component and Deployment diagrams– More notation
• More explicit
• Harder for the casual reader
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 6
UML Component Diagram
Component
Dependency
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 7
UML Deployment Diagram
Shows software deployed in hardware
Node
Connection
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 8
More UML• UML also offers class
diagrams to expose an API• And Sequence and
Collaboration diagrams to show possible interactions of objects
• Oxford have EPSRC funding to work with DataGrid – they will be looking at the applicability of UML for architecture
• Will facilitate more UK input to GGF
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 9
Are all Grids the same?• Grids originated for “supercomputers”• Knowledge Grids?• Grid and the web• DataGrid(s)
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 10
Knowledge Based Data Grids• The knowledge based Data Grid (Moore)
grew out of the need to manage very complex data and inter-relationships– Data objects from many sources
• e.g. different imaging modalities and anatomical data about the brain
– Information about the data objects (attributes)– Knowledge about the data objects (relationships)
• e.g. how do the observations from the different types of imaging - e.g. compositional and metabolic – relate to anatomical structure
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 11
Use Cases• NIH Biomedical Informatics Research
Network– Federation of multiple existing digital libraries– Support information discovery, data access, data
movement, and data analysis on distributed resources
• NARA Persistent Archive– Build a data collection that maintains authenticity
of digital data while technology evolves– Support information discovery, data access, and
migration to new data encoding standards
AttributesSemantics
Knowledge
Information
Data
Ingest Services
Management AccessServices
(Model-based Access)
(Data Handling System)
MC
AT
/HD
F
Gri
ds
XM
L D
TD
SD
LIP
XT
M D
TD
Rul
es -
KQ
L
InformationRepository
Attribute- based Query
Feature-basedQuery
Knowledge orTopic-Based Query / Browse
KnowledgeRepository for Rules
RelationshipsBetweenConcepts
FieldsContainersFolders
Storage(Replicas,Persistent IDs)
National Partnership for Advanced Computational Infrastructure
Knowledge-Based Data Grids
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 13
Queries across data sources from a common interface
@SENSELAB: X1 := select output from parallel fiber ;@MEDIATOR: X2 := “hang off” X1 from Domain Map;
@MEDIATOR: X3 := subregion-closure(X2);
@NCMIR: X4 := select PROT-data(X3, Ryanodine Receptors);
@MEDIATOR: X5 := compute aggregate(X4);
"How does the parallel fiber output relate to the distribution of Ryanodine Receptors?"
Sources: NCMIR UCSD / Yale Senselab
KIND Mediator
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 14
Web Services Grids• Web based Grid services represent the
intersection of the commercial Web services (e.g. IBM’s WebSphere) and scientific and engineering oriented Grid services
• Also driven by the desire to use commercial tools to facilitate the construction of “portals” – application / domain specific frameworks that present “packaged” Grid services toGrid end-users (e.g. scientists and engineers)
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 15
So are they different?• I have seen nothing so far to distinguish
these other types of grids– They may have some distinguishing services– But no fundamental differences
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 16
Layering• A general layering approach based on
increasing levels of abstraction (abstract machines) is useful for conceptualizing the services
=Globus service
Problem SolvingEnvironment
Applicationsand Supporting
Tools
Tools to implement the human interfaces, and the mechanisms to express, organize, and manage the workflow of solving a problem
ApplicationDevelopment
Support
GridCommonServices
LocalResources
Ap
plic
atio
nC
od
es
Vis
ual
izat
ion
To
olk
its
Co
llab
ora
tio
nT
oo
lkit
s
Inst
rum
ent
Man
agem
ent
To
olk
its
Dat
a P
ub
licat
ion
and
Su
bsc
rip
tio
nT
oo
lkit
s
Grid Enabled Libraries
Glo
bu
s M
PI
CO
RB
A
Co
nd
or
Java
/Jin
i
PR
E/C
OR
BA
OL
E/D
CO
M
Gri
d
Info
rmat
ion
S
ervi
ce
Un
ifo
rmR
eso
urc
eA
cces
s
Bro
keri
ng
Glo
bal
Q
ueu
ing
Glo
bal
Eve
nt
Ser
vice
s
Co
-S
ched
uli
ng
Dat
a C
atal
og
uin
g
Un
ifo
rm D
ata
Acc
ess
Co
llab
ora
tio
n
and
Rem
ote
In
stru
men
t S
ervi
ces
Co
mm
un
icat
ion
S
ervi
ces
Au
then
tic
atio
n
Au
tho
riza
tio
n
Sec
uri
ty
Ser
vice
s
Au
dit
ing
/
Acc
ou
nti
ng
Fau
lt
Man
ag
emen
t
Mo
nit
ori
ng
Communication Services
ResourceManager
CPUs
ResourceManagerTertiary Storage
ResourceManagerOn-Line Storage
ResourceManagerScientific
Instruments
ResourceManagerNetwork Monitors
ResourceManager
Highspeed Data
Transport
ResourceManager
QoS
collective
resource
fabric
layers of increasing abstraction - taxonomy #1 (“Johnston”)
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 18
EDG Architecture (as circulated initially to GPA)
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 19
Comments on layering
• “things” in each layer use only those in same layer and below– BUT not only immediately below– i.e not an abstract machine model
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 20
EDG Architectureanother view
Collective ServicesCollective Services
Information & Monitoring
Information & Monitoring
Replica ManagerReplica Manager Grid SchedulerGrid Scheduler
Grid Application LayerGrid Application Layer
Job ManagementJob Management
Local ApplicationLocal Application Local DatabaseLocal Database
Fabric servicesFabric services
ConfigurationManagement
ConfigurationManagement
Node Installation &Management
Node Installation &Management
Monitoringand
Fault Tolerance
Monitoringand
Fault Tolerance
Resource Management
Resource Management
Fabric StorageManagement
Fabric StorageManagement
Grid
Fabric
Local Computing
Grid
Data ManagementData Management Metadata Management
Metadata Management
Object to File Mapper
Object to File Mapper
Underlying Grid ServicesUnderlying Grid Services
Computing Element Services
Computing Element Services
Authorisation, Authentication
and Accounting
Authorisation, Authentication
and Accounting
Replica CatalogReplica Catalog
Storage Element Services
Storage Element Services
SQL Database Service
SQL Database Service
Service Index
Service Index
Slightly simplified version of a WP2 diagram
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 21
Comments on layering• This diagram has layers referencing BOTH
ways• When you look at each service you rarely find
a clean layering – especially as the services get more complex, they are implemented using many other services both “above” and “below”
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 22
Layering and client server• It was a principle EDG defined early on that all
services should be available – only restricted by policies.– Good
• anyone can compose new services easily
– Bad • policies are needed to keep things under control – e.g.
modifying replica catalog
• Should we: – just say that these constraints are policy issues and not
represent them in the architecture?– Or try to come up with a simple representation of constraints
for the conceptual representation of the Grid architecture?
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 23
Set of Services• EDG Architecture has been criticised as “just
a set of services”• This may be good however
– Provided that we demonstrate that they are sufficient
• e.g. sequence or collaboration diagrams (UML)
• For each EDG service we define upon what services it depends– IS IT ARCHITECTURE OR IMPLEMENTATION?
• Architecture from EDG middleware standpoint• But implementation from a user standpoint
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 24
Service and Protocols• To a client the protocol is uninteresting:
– only the API– service could be local
• Need to allow for migration to and/or integration of new services
• A service could respond to >1 protocol• A service could support >1 security
mechanism• Makes it easy to introduce new services
Edinburgh, November 2001Grid architectures - Steve Fisher/RAL 25
Need for openness• We should have lots of overlapping services and use
the ones we want – typically a VO would choose– Though a lot is dictated by the RB
• Do we want InterGrid or OpenGrid?• We don’t want to tie ourselves down in a fast moving
world• Need to watch industry
– IBM– SUN– Microsoft
SOAP, WSDL
JINI, UDDI
.NET, Passport etc.