Programming the Grid with Components

Extreme! Computing Lab, Dept. of Computer Science, Indiana University 1

Programming the Grid with Components

Madhu Govindaraju Aleksander Slominski

Dennis GannonSriram Krishnan


Outline• Software Components• XCAT

– Component and Services model– Web Services– Programming model– OGSI


Software Components• Analogy with hardware• Standard design paradigm for app

development• Goal: to build applications by

composition of well tested and well behaved subsystems (components).– simplify application design process– promote code reusability – plug and play


Component Architecture• A Component Architecture consists of

two parts:– Components

• software objects that implement a set of required behaviors

– Frameworks• A runtime environment• A set of services used by components


Industry Standards• COM/DCOM

– interoperability for Microsoft applications

• Java Beans, Enterprise Java Beans (EJB) – Java based desktop and enterprise

applications

• OMG CORBA Component Model (CCM)– language neutral superset of EJB


Academic Research• SciRun from Utah

– scalable parallel applications and viz

• Webflow from Syracuse– graphical composition palette

• CCAT and XCAT from Indiana University– framework for Grid based applications


CCA: Common Component Architecture

CCA: Initiative to define minimal specification for scientific components Department of Energy (National Labs) Few universities

Aim: build components for high performance computing

Draws ideas from CCM and other models as a baseline

Targeting Parallel Distributed


The Common Component Architecture

• Key Idea: dynamic composition by linking “ports” – Provides ports: interface of “services”

provided– Uses ports: a call-site for a service to be

provided by another component.

Uses port-

ProvidesPort


What is new about CCA?• Minimal specification• Envision connections as dynamic

– Can add, remove and connect ports at run-time

• Tailored to build Problem Solving Environments– End user manipulates connections

• Not specific to any underlying distributed object model

• Notion of collective ports– logically one connection– implementation: multiple network connections


Components on the Grid: Issues

• Discovering static information about components

• Launching components• Discovering references for running instances• Communication protocols• Messages and Events• Authentication and Authorization• Encapsulating legacy applications• Efficient scheduling• Run-time environment


Grid Programming Model: Needs• Need a set of APIs, protocols, libraries

and tools that allow access to Grid resources

• Examples: Globus, Condor– provide infrastructure – do not provide programming model

• What is missing?– How can we make it easier to “program”

the Grid?


XCAT

• Implementation of the CCA specification

• Designed for distributed applications• Allows creation of Grid application

from components• Wraps Grid Services as components• Test-bed for CCA

– Is there an alternative to OMG IDL?


Component Communication

• How do components communicate?– Use Remote Procedure Call (RPC)

Mechanism• XCAT uses SOAP 1.1• XCAT ports can serve as web services

– Events/Messages• Objects encoded as XML documents


XCAT Services Architecture

• Default services for all components• XCAT services

– Directory• locate components based on port types and other attributes

– Registry• locate running instances of components

– Creation• create running instance of a component

– Connection• connect ports of two running instances

– Events• publish/subscribe framework for messages


IDS: Information Directory Service• Store Component descriptions in XML• Used to search databases and remote

repositories

InformationDirectoryService

Component

MySQL Database

File system

LDAP

RDBSRequest component info

Return XML specs.


X

Creation Service• Creates a running instance of another

component• Encapsulates authentication issues• Supports GRAM, SSH and Local-Exec

CreationService

Component

Launch an instance ofcomponent X on resource Y

Returns: remote referenceto new component instance

Globus resource Y


Registry Service Stores instance information (runtime

configurations) Used to advertise and search other

instances Tradeoff: distributed vs. centralized

Find a running instanceof component X

Returns: remote referenceto component instance

RegistryService

Component


Connection Service• A component that can be used to connect

a “uses” port of one component to the “provides” port of another

• Can export ports of other components as one’s own

Y

ConnectionService

Component

X

Connect port A of component Xto port B of component Y

A

B


Event Service• Channel for publish-subscribe event

model• Creation and connection events• Application users can use it for

asynchronous information transfer

B

Event ChannelService

Component

C

A

Subscribe tolisten for eventsof type X

x x

x

x

Notification of publicationof events of type X


Features of Events

• Event/Notification Services are an essential part of all distributed systems

• Application Events– “I am done.” “I just wrote to a file,” “here is

a result.”• Event Publishers & Listeners

– Some Listeners subscribe, others poll – Must have persistent event channels

• record events for later analysis• application history logs• allow for retrieval in chunks


XCAT: Other Features

• Security:– Every remote method call is intercepted– port, method and parameter level– authentication based on SSL– authorization based on simple ACL

• Component Handle:– in XML and can be converted to WSDL – GSH /GSR

• Exceptions– Supports throw and catch across

components


XCAT Processes

CreationService

OtherServices

ConnectionService

FrameworkImpl.

Scripts,Front-end

Tools

ApplicationCode

FrameworkImpl.

ApplicationCode

Scripts,Front-end,

Tools

CreationService

ConnectionService

OtherServices

PortWrapper Code


XCAT-Web Services Stack

Framework Layer: Creation, Connection,…

Service Discovery Layer: LDAP

Service Description Layer: Subset of WSDL

Service Messaging Layer: XSOAP

Service Transport Layer: HTTP


XCAT Programming: Examples• Builder: A tool used to select and link

components together• Builder Options

– Swing based GUI– Jython scripting – Portal

• XCAT– Jython scripting


CCAT Gui image


Scripting XCAT Applications

import xcat

stringDump = xcat.createComponent(‘StringDump’)printer = xcat.createComponent(‘Printer’)

xcat.setCreationMechanism(stringDump, ‘gram’)xcat.setCreationMechanism(printer, ‘ssh’)

xcat.setMachineName(stringDump, “exodus”)xcat.setMachineName(printer, “rainier”)


Scripting XCAT Applications: contd

printer.put(“exec-fqn”, “samples.printer.Printer”)

stringDump.put(“exec-fqn”, “samples.stringDump.StringDump)

xcat.createInstance(printer)xcat.createInstance(stringDump)

xcat.connectPorts(stringDump, ‘outputString’, printer, ‘inputString’)


Encapsulating Legacy Apps

• Common Case– Legacy App that reads

files and writes files– Use a “scripted

component”• Component runs a

python script

– The App Script• Stages files• Launches and monitors

application• Writes output files• publishes event streams

application

XCAT comp

inputfiles

output files

App Script

EventStream

Control


Scripting Grid Computations

• Grid Scripts are executable documents that– Orchestrate Computations

• Select resources • Supply application parameters and launch remote

jobs• Subscribe to event streams • Create components• Connect components


Composition Using Components

• Two ways of composing applications– Composition in space:

• one component/service directly invokes the services of another

– Composition in time:• A workflow engine schedules tasks that involve

accessing remote services and responding to events


XCAT, OGSI and Web Services

GS

Component A

GS

Component B StandardGrid Service

Grid Service Client


XCAT and OGSI compliance

• Add a GridService port to each component

• Merge component handle with GSH/GSR• OGSI messaging is non-reliable push

model– XCAT has a reliable, persistent model

• OGSI has factory portType for instantiating new services– XCAT has an extended distributed factory

model

Documents

Programming the Grid with Components