Michael Sobolewski Texas Tech sobol@cs.ttu.edu Grid and Service-Oriented Computing: The Intergrid...

Preview:

Citation preview

Michael SobolewskiTexas Tech sobol@cs.ttu.edu

Grid and Service-Oriented Computing:The Intergrid Perspective

Part II

Presentation Agenda – Part II• Grid Federated S2S Environment • GISO Programming• Monitoring Execution of SO Programs• ServiceUI• Provisioning and Autonomic Computing• Design Issues – UML Class Diagrams• Code Mobility in SORCER• File Store Service• SGrid and Intergrid• Surrogate Architecture for Mobility• Web/Grid Services in SORCER• DB Context Providers• A Grid Killer Application• SORCER Research Projects• Summary

Applying OO Techniques

• Service activity is a special object of type: Exertion

• Exertions are executed by network objects/service providers of type: Servicer

• Service providers form P2P environment• Service is requested by calling the method: service(Exertion)

• Service providers are identified by a type with methods:

public ServiceContext selector(ServiceContext)

public interface Exertion {// Apply this exertion method to the specified contextpublic Exertion exert() throws RemoteException, ExertionException;

…}

Exertion Interface

• All service activities implement this interface:

public interface Servicer {// Put into action the specified exertion

public Exertion service(Exertion exertion) throws RemoteException, ExertionException;

// Monitoring methods…

}

Service Peer Interface: Servicer

• All services implement this interface:

Contexts and Context Methods

Job Task Service Context Service Method

…CC

TC

TC – Task Context, CC – Control Context

ServiceMethodScriptMethodXYZMethod

DAS

GEOMMAT

Shank

MAT_RENE5

Mat_Rene5.dat

PP

STRESS

Post_Stress.dat

LC

Gas_Loads

ExtGLoads.dat

BC

Cyclic

BCCyclic.dat

DiskAirfoil

UIF

Airfoil.uif

Mesh

Disk_PRT

AS_PRT

AS_Shank.dat

AS_Shank.prt

AS_PRT

Disk.prt

Mesh

Stress_Tet

Stress_Tet

AS_Shank.datAS_Shank.dat

AS_Shank.datModal_Hex

CC

ContextMethod attributes: service type, selector, group, provider name, method typeMethod type: preprocess, process, postprocess, append

Service BindingJob as a Megaapplication

Job

Task

Context

Method

Method type: preprocess, process, postprocess, append

Federation Member

Federation of Services

Job

Task

Context

Method

Method type: preprocess, process, postprocess, append

Job as Runtime Environment

GISO Programming

Domain ServicesDomain Services

Service-Oriented,Service-Oriented,Grid-ProgrammingGrid-Programming

andandDevelopment ToolsDevelopment Tools

Service-Oriented ProgramsService-Oriented Programs

Generic End-User ClientGeneric End-User Client

Service-Oriented Infrastructure Services Service-Oriented Infrastructure Services (SORCER.core)(SORCER.core)

Grid Interactive Service-Oriented Programming

Job Structure

Airfoil

Platform

Shank

Dovetail

FiperJob

FiperContext

FiperTask

FiperMethod

FiperContext

Exertion

FiperMethod

FiperContext

FiperTask

FiperMethod

FiperContext

Exertion

FiperMethod

.

.

ControlContext

FiperMethod

Turbine Blade Solid Geometry

Mechnical Analysis of TB

Turbine Blade FEM

DiscretizeSolid Geometry Into FEM(ICEM, Mesh)

in

out

DescretizationStrategy

Turbine Blade FEM w/ BCs

Apply Boundary Conditions to FEM(SIESTA, ApplyBCs)

in

out

Boundary Condition (BC) Model

Turbine Blade FEM w/ BCs & Materials

Apply Materials to FEM(SIESTA, ApplyMaterails)

in

out

Materials Model

Turbine Blade FEM Element &Nodal Stresses

Solve FEM for Mechanical Stresses(ANSYS, Mechanical Analysis)

in

out

Mechanical Solution Strategy

AutoShank Solid Geometry Part

Generate Solid Shank (AutoShank, Autoshank)

in

out

AutoShank Seed Geometry Part,AutoShank Parameters

in

in

in

in

Turbine Blade FEM

DiscretizeSolid Geometry Into FEM(ICEM, Mesh)

in

out

DescretizationStrategy

Turbine Blade FEM

DiscretizeSolid Geometry Into FEM(ICEM, Mesh)

in

out

DescretizationStrategy

Turbine Blade FEM w/ BCs

Apply Boundary Conditions to FEM(SIESTA, ApplyBCs)

in

out

Boundary Condition (BC) Model

Turbine Blade FEM w/ BCs

Apply Boundary Conditions to FEM(SIESTA, ApplyBCs)

in

out

Boundary Condition (BC) Model

Turbine Blade FEM w/ BCs & Materials

Apply Materials to FEM(SIESTA, ApplyMaterails)

in

out

Materials Model

Turbine Blade FEM w/ BCs & Materials

Apply Materials to FEM(SIESTA, ApplyMaterails)

in

out

Materials Model

Turbine Blade FEM Element &Nodal Stresses

Solve FEM for Mechanical Stresses(ANSYS, Mechanical Analysis)

in

out

Mechanical Solution Strategy

Turbine Blade FEM Element &Nodal Stresses

Solve FEM for Mechanical Stresses(ANSYS, Mechanical Analysis)

in

out

Mechanical Solution Strategy

AutoShank Solid Geometry Part

Generate Solid Shank (AutoShank, Autoshank)

in

out

AutoShank Seed Geometry Part,AutoShank Parameters

AutoShank Solid Geometry Part

Generate Solid Shank (AutoShank, Autoshank)

in

out

AutoShank Seed Geometry Part,AutoShank Parameters

inin

inin

inin

inin

Actions Data

Process Turbine Analysis

Web Based User Agent

}FiperContextFIPERTaskFIPERJob

Context EditorContext NameDomain Subdomain

Context Node

Data Node

Task Editor

Task Editor

FiperMethod Definition

Context forTask

Task Name

Created Task

FiperMethod

FiperContext

Task Name

Job Browser

Job Editor OperationsDomain Subdomain Job Name

JobTasks

Job EditorFeatures

Create a new Job

and add to Job

Create a new Task and add to Job

{

Task EditorFeatures

Browse & Add Job to Job Browse & Add Task to Job

Control Context

Job Context

Job Task 0

Context0

Task1

Context 1

Drag and Drop shows that thisInput will come from task0, AutoShank Output Node

Adding Context to Task

Selecting a Task for a Job

Monitoring Conceptual View

Monitorable Interface

All ServiceProviders are Monitorable

to monitor, control execution and debug SO programs

Execution States of Exertions

DeletedInitial

Error

Running

Done FailedSuspended

run stop

suspend

resumestep

resumestep

• ServiceProvider – Exertion Status Map (ESM)

• Jobber – Dispatcher Thread Map (DTM)

• Exception/Errors returned in ServiceContexts

Monitoring Framework

•State of running SO program updated via DB•Clients can monitor the SO program via web portal•Clients can stop/resume/suspend•Monitoring requests are handled by Service Broker

WebPortal

Client1

Client2 Persister

Jobber

p1 p1 p1

mon

itor(

rex)

service(ex)service(job)

service(job)

monitor(rJob)

DB

monitor(r

Job)

Job Browser

Control Panel

Launcher

Job List

Exertion List

Control Context

Review = Suspend

Job Monitor

Job Monitor

Full Job Edit Capability

Job ExecutionControls

MO - Complete Framework

Monitorable ServiceBrokerMonitorable

Monitorable

register(RemoteEventListenerduration,timeout SOP) : RuntimeExertion

1

execute ( sop-x)

update(..)stop(..)error(..)

3

4For (every sop-x) x=0,1,2,… Init (cookie, monitorable, duration, timeout)

2

session sessionsession

Monitor ServiceUI

ServiceRequestor

SOPFederation

session Persistent session

session Transient session

SOP = { sop1, sop2, …..}Or = sop-x , x=1,2,……

MonitorManager

SessionManager

session

UIManager

notify (Monitor Event)

5

Result or Failure Notification

Monitorable ServiceBrokerMonitorable

Monitorableexecute ( sop-x)

update(..)stop(..)

notify (Monitor Event)

7

8

Service Provider

9

MonitorManager

session sessionsessionsession

Suspend/stop/resume a Job

MonitorableServicer

suspend/stop/resume job(……)

MonitorableServicer

MonitorableServicerMonitorable

Servicer

suspend/stop/resume job(……)

Service Broker

notify (Monitor Event)

1

2

3Monitorable

Servicer

MonitorableServicer

MonitorManager

•register (leaseDuration, job,

listener)

MonitorSession•init (monitorable, duration,

timeout)•getState()•update(context)•done(context)•failed(context)•getLease()

invokes

MonitorableServicer

•stop(Ueid)•suspend(Ueid)•resume(Ueid)•step(Ueid)

Service Provider

Jobber

MonitorableDispatcher

Servicer.

Registers and gets backRuntime Job

xxxServiceProvider

DispatchesExertion withSession Objects

IssuesMonitorablecalls

Remote EventListener

MonitorableProvider

SessionManager

•init(cookie, monitorable, duration, timeout)•update(cookie, context)•done(cookie, context)•failed(cookie, context)

MonitorUIManager

•getSOPInfo(Principal)•getSOP(referenceID,

Principal)

ServiceJob

ServiceTask

1

0..*

Dispatches exertions in job

Providers use sessionto update informationabout runtime exertions

1

1

11

Monitoring Framework

Service UI

A "direct-use" client talks to a service through its object interface

Service UI

A user interacts with a service via a UI object

EntryServiceTask

Context/Method/Task/Job

RemoteServiceMethod

ServiceContext

ServiceMethod

ArithmeticMethod

ServiceTask

0..*

Hasdata

Entry

EntryServiceJob

ServiceJob

ClientSite

ServerSite

1..*

Defines behavior

Defines remotebehavior

Exertion

Serializable

EntryExertion

Remote1..*

EntryServiceTask

RemoteServiceMethod

Task Execution

SorcerJoiner

ServiceContext

ArithmeticMethod

ServiceProvider

Servicer

ArithmeticProviderImpl Publishes

Proxy

ServiceTask

Executes

UnicastRemoteObject

0..* Hasdata

ArithmeticRequestor

RequestorRunner

Provider ArithmeticInterface

ProviderWorkerJavaSpace

Uses

Runs

Invokes

ArithmeticRemote

Remote

RequestorDrops task

Submits task

Submits task

Job Execution

Uses

RequestorRunner

<<HTTP>>

SorcerLauncher

Cataloger

ServiceCatalogImpl

ProxyProtocol

JobberImpl DisapatcherFactory

TaskDispatcherFactory

Creates

Invokes

ServiceTaskDispatcher

CatalogTaskDispatcher

TaskDispatcher

Uses

SpaceTaskDispatcher

ServiceServlet Jobber

Provider Requestor

ServletProtocol

ArithmeticRequestor

Service access: Direct, Catalog, Space

ProviderAccessor

Uses

JobBrowser JavaSpace

Submitsjob

Drops job

ServiceProvider

SORCER Smart ProxyBL on Client and Server

UnicastRemoteObject

ProviderProviderWorker

JavaSpace

Uses

Uses

ServiceBean

ServiceProvider

ServiceBean

Adapter

ProviderDelegate

Delegates to

ServiceProxy

Remote

ArithmeticProxy

ServiceJoiner

ArithmeticBean

Exports Provider

ArithmeticService

Service

ArithmeticProvider

Provisioner

Deploys by QoS

ArithmeticRemote

Remote

ArithmeticInterface

Servicer

Publishes

provider’s proxy

Uses

Mobility

• Mobility stems from desire to move either toward resources or away from scarcity

• Mobility of computing entities– Physical mobility

• Computers– Logical mobility

• Running user application (active process)– Migrate within a local cluster of computers– Enable load distribution and fault resilience

• A mobile agent– Migrate anywhere on the Internet– Act on the user behalf to pursue specific goals

• Mobile code– Applets– MIDlets– RMI/Jini/Rio/SORCER

Mobile Agents

• A mobile agent – a network application that migrates in a network (grid) – Active– Autonomous– Goal-driven– Acting on behalf of a user

• Stationary agents– Intelligent agents– Multi-agent systems

Agents and Mobile Code

• Mobile code solves problems– Heterogeneity (platform independence)– Dynamic class loading– Security and safety– Multi threading support– Object serialization– Reflection– Universal availability of JVM– A large number of docking stations

Mobile Code

1. Proxies

2. Exertions

3. Task Methods

4. Agents

5. SORCER Beans (JSBs)

6. Service UIs

SORCER Code Mobility has many forms

Agents in SORCER

SelfAgent

ServiceTask ServiceJob

ServiceContext

Uses

EntryTask EntryJob

RemoteAgent

Remote

ObjectLoggerStores

Restores

1..*

1..*

EntryExertion

AgentBootstrapBootstraps

JDK1.1 surrogateLimited objects

Agents

Dropper

Provider

doTaskstopTaskdoJobstopJobsuspendJobresumeJobstepJob

Servicer

service

Uses

Uses

ServiceProvider

Jobber

Entry

Requestor

getTaskgetJob

RequestorRunner

Uses

Uses

Uses

ServiceMethod

EntryMethod

Uses

Uses

1..*1..*

Uses

Agent

act

Exertionpreprocessprocesspostprocess

Self

think

SORCER ProvidersJDK 1.4 objects

TaskAgent JobAgent1..*

Self-Aware Grid

• A grid federation knows what is doing

• Learn from experience and adapt to surprises

• Is aware of its behavior and explain itself

• Is able to anticipate different scenarios and predict and plan for novel futures

• It would learn, not crash, when faced with a new situation

• Self-testing, self-debugging, and self-explaining within a federation

What is Rio?

• A Dynamic Service Delivery architecture based on the capability to provision services through the network using Qualitative and Quantitative QoS attributes

• Addresses essential issues for the development of a dynamic self healing services environment

RIO & Autonomous Computing

Dynamic, Distributed systems- Capability to adapt to unforeseen changes on the network- Driven by increasing componentization, distribution &

heterogeneity- Scale to orders of magnitude

Dynamic Policy driven systems- Injecting rules & policies into the service fabric allowing

greater automation, scalability and controlled behavior

Telemetry- Being able to monitor, meter, gauge and observe stimulus

through the system- Not just machine based, but network-wide

Rio Architecture Overview

Development & AssemblyDevelopment & Assembly

Dynamic ProvisioningDynamic Provisioning

SubstratesSubstrates

Jini Service BeansJini Service Beans

ToolsTools

FederationFederation

ResourcesResources

JavaTM 2, JiniTMJavaTM 2, JiniTM

DynamicContainer

AssimilationIntegration &

Interoperability

AssimilationIntegration &

Interoperability

Jini Service Bean Basics

• Jini Service Beans (JSBs) are the fundamental domain specific computational entities on the network

• JSBs are Java objects

• JSBs can encapsulate access to NDI (legacy) components

• Provides an easy to use programming model while maintaining access to low-level APIs

• Are provisionable based on their QoS attribute

Cybernode

• Jini Service Beans are instantiated by Cybernodes– Cybernodes run on computational resources– Cybernodes can contain multiple service beans

• Address lifecycle issues, including development, deployment, and runtime

• Provides the basic infrastructure to load, instantiate and destroy Jini Service Beans

CybernodeCybernode

Cybernode

Jini Jini Lookup Lookup ServiceService

Code Code ServerServer

Service Bean Attributes

Service Bean Attributes

Download JSB resources

Instantiate JSBs

JSBs register with LUS

Platform Attributes

Platform Attributes

Operational String

• Aggregated collection of application and/or infrastructure software assets that when put together provide a specific service on the network

• An Operational String is an object graph composed of objects that provide context on how to provision and instantiate services

• Can be created by referencing an LDAP repository, serialized objects, or from structuredXML documents (there can be many other sources as well)

Operational String

ServiceElement

ServiceProvisionManagement

ServiceBeanAttributes1

1

0..*

Quality of Service

• Approach– Software components need to run on most

appropriate compute resource based on definable criteria

– Compute resources have capabilities• CPU, Disk, Connectivity, Bandwidth, …

• Rio provides an extensible model allowing the declarative association of QoS

Provision Manager

• Provide a provisioning model to dynamically deploy, monitor & manage service components as described in an Operational String

• Implement a Fault Direction and Recovery strategy for service components

• Provide pluggablepluggable load distribution and Resource Cost analysis mechanisms to effectively take use of resources on the network

Dynamic Provisioning

Jini Lookup Service

discover

Start a monitor for each service in an Operational String

notify

ProvisionManager

Discover/join

Register/update for provision notifications with QoS attribute

Load an Operational String from disk or via remote method invocation

Notify ServiceInstantiation resources with a ServiceProvisionEvent based on the assignability of the JSBs QoS to the ServiceInstantiation resources QoS

Code Server

As needed download JSB class files

Instantiate & initialize JSB

ServiceBean

Instantiator

Service Instantiator

Fire notification events to event registrants for added, removed, updated Operational String actions as well as failed Provision attempts

Lincoln

• Provides the capability to enable dynamic discovery of Jini Lookup Services across networks that are out of multicast range, or do not forward multicast packets

JLS JLSLincoln Lincoln

Multicast Announcement

Group BGroup A

svca svcb

Multicast Announcement

RIO Provisioner

Lincoln Example

• Coast to coast discovery

• No extra service configuration

• Key to collaboration

Fullerton, CA

Minneapolis, MNTewksbury, MA

Portsmouth, RI

Developer Facilities

• Events– Built-in registration

and lease handling of user defined events

– Simple semantic for specifying and discovering events

• Resource Pools– Threads– Objects– Database

Connections

• Watches– Every component is

‘watchable’– Add data points to a

collectible set and graph accumulated results

Service Beans Service Beans

• Jini™ technology-based Service Beans (“JSBs”) are the fundamental domain specific computational entities on the network

• Are provisionable based on their QoS attribute

• Jini technology-based Service Beans are instantiated by Cybernodes– Cybernodes run on computational resources– Cybernodes can contain multiple service

beans

Summary

Provider Bootstrapping

Bootstrapping Type Server Type NDS Technology

java SorcerJoiner

-sProviderClass

RMI server

(JRMP/IIOP)

JNDI/RMI Reg

JNDI/LDAP

RMI/CORBA

java SorcerJoiner

-pProviderClass

Service provider

(Jini)

LUS Jini

java SorcerJoiner

-pProviderClass:ProxyClass

Service provider

with smart proxy

(Jini)

LUS Jini

Provisioning (Rio) JSB LUS Rio/Jini

Command Design Pattern

File System

File SystemRDBMS

API Classes

Utility Classes

AC

LM

anag

er

JDB

C

Cmd 1

Cmd 2

Cmd 3

Cmd 4

Cmd 5

Cm

d M

anag

er

File

Sto

reP

rovi

der

RemoteInputStreamServer

RemoteOutputStreamServer

1

2

34

5

6

2

1 34

56

1,1 - Requestor requests FSS provider for file upload/download based on DocumentDescriptor

2,2 - FSS forwards it through the command execution engine

3,3 - Database operations are done

4,4 - FSS spawns a RemoteOutputStream/RemoteInputStream server based on-if client wants to upload/download and returns back the InputStram/OutputStream adapter inside DocumentDescriptor

5,6 - Client communicates to server via OutputStreamProxy and uploads file.

5,6 - Client communicates to server via InputStreamAdapter and downloads file.

Standalone

JVMClients

SORCERDistributed File System

File System (UNIX)

Servlet EngineFileStorer

SORCER Service

FileStorerSORCER Service

File System (Linux)

FileStorerSORCER Service

File System (Win 2003)

Browser

JVM

Program

JVM

FileStore Clients Interactions

Client request is passed on from one server to another until one of the File Store service has the file actually present in its File System and the handle of file is passed onto client.

DBMS

ORACLE

DBMS

mySQL DBMS

MS SQL

Document Descriptor

DocumentDescriptordocumentIDdocumentNamefolderPathoutputStreaminputStream

InputStreamAdapter

InputStream

Remote

RemoteInputStreamServer RemoteOutputStreamServerFileStorer

getInputDescriptor(DocumentDescriptor)getOutputDescriptor(DocumentDescriptor)

OutputStreamProxy

OutputStream

contains

containscontains

contains

Document Descriptor: Details

DocumentDescriptor

setDocumentID(String)setFolderPath(String)setDocumentName(Name)setPrincipal(GAppPricipal)

FileStorer

getInputDescriptor()getOutputDescriptor()

InputStreamAdapter

read( ): intread( byte[] ): intread( byte[], int, int): int

InputStream

read( ): intread( byte[] ): intread( byte[], int, int): int

write( int )write( byte[] )write( byte[], int, int)

RemoteOutputStream

Remote

write( int )write( byte[] )write( byte[], int, int)

OutputStreamProxy

OutputStream

write( int )write( byte[] )write( byte[], int, int)

read( int ): byte [ ]

RemoteInputStream

containscontains

containscontains

ApplicationServlet

UploadServlet

RemoteInputStreamServer

RemoteOutputStreamServer

GAppUses

Uses

Uses

Servlet Engine

Servlet Engine FS Provider

Upload

Download

HTTP

HTTP

JRMP

JRMP

vs. FileStore Provider

...

CRD Internal Network

GRC HTTP Proxy Server

Web Browser

FIPER Intraportal

1 2

CRD Perimeter Network Controlled by Firewall

3 4 GRC SSL FIPER

ExtraPortal

Web Browser

FIPER Intraportal

7 8

Parker Perimeter Network controlled by Firewall

Parker HTTPProxy Server5 6 Parker SSL

FIPER

Extraportal

GRC Firewall

Parker Internal Network

Parker Firewall

B2B Setup

Nozzle Combustor CAD/IO B2B

IKS

4. Perform modal analysis

2. Request for nozzle validation

1. Update combustor PCS

3. Check for nozzle insertion

5. Perform CFD blow analysis

(UG) (ProE)

(Blow Analysis)

Vertical iGrid Grids

Cybernodes – iGrid.meshCybernodes – iGrid.mesh

Service Providers – iGrid.gridService Providers – iGrid.grid

Exertions – iGrid.spaceExertions – iGrid.space

Computing Devices – iGrid.netComputing Devices – iGrid.net

SORCER.coreSORCER.core

SORCER.gridSORCER.grid

iGrid.grid – service providers including services from technology (horizontal) gridsSORCER.core – SORCER infrastructure service providersSORCER.grid – SORCER domain specific service providers

SS Beans – iGrid.fieldSS Beans – iGrid.field

iGrid Layered Architecture

Domain iGrid ServicesDomain iGrid Services

Service-Oriented,Service-Oriented,Grid-ProgrammingGrid-Programming

andandDevelopment ToolsDevelopment Tools

Service-Oriented ProgramsService-Oriented ProgramsiGridApplication

Generic End-User Client/ServiceUIGeneric End-User Client/ServiceUI

Technology-Oriented (Horizontal) GridsTechnology-Oriented (Horizontal) Grids

iGridMiddleware

Service-OrientedService-OrientedComputing Computing

EnviRonment (SORCER.core)EnviRonment (SORCER.core)

Horizontal iGrid GridsWrapper + Native + SORCER

JXTAOGSA

SORCER.ogsa

SORCER.grid

Native ProviderTechnology SORCER wrapper-grids: Jini, JXTA, CORBA, Web Services, Grid Services, .NETSORCER.gridTechnology native grids

Service RequestorService BrokerSORCER Service ProviderSORCER Wrapper Provider

SORCER.jxta

SORCER.grid

SORCER.core

SGrid Dispatcher UI

• Choose the Application to run (For example Proth)

• Specify the Job Size for the jobs

• Set the Arguments, Attributes and Executables for the application

Arguments

• Specify Arguments, Input Files, Output Files for the Application

• Can be added above or below the selected option

• Can be reordered according to user’s requirement (Up, Down, Delete buttons)

• Specify which caller to use for execution, depending upon its

- Location (HPCC, SORCER)- Host Name (129.118.56.61)- Operating System- Node Name (Amber)

• ‘*’ is the default option which indicates that the application can be run by any caller in the Federation

Executable Attributes

Executables - Windows

• Specify Windows Executables and Library Files

• The files can be dynamically downloaded from File Store

Grid Provider

• A Grid Provider is responsible for SGrid Publishing ServiceUI Accepting user inputs and creating a Job Submitting the Job to Jobber Receiving results, notifying client, getting

inputs and storing results in the File Store

• Advantages of such a design• User friendly with respect to the Job Editor UI• Zero install with a Jini service browser and a

web-based File Store user agent

Caller Provider

• A Caller receives a caller service context that abstracts any system call across different platforms (Solaris/Linux/Windows)

• Callers are capable of Downloading platform specific executable

binaries and libraries Downloading source and compile it on-the-fly Make system calls with arguments specified in

the service context

• Callers download/upload files from/to the SORCER File Store Provider

Caller Service Context

String

src/bin/bytecode/callx

array attribute<attribute name>[ ]

URI

caller

call

cmd dir

envp[ ]in[ ] out[ ]

program

compiler

win/linux/unix

src[ ] bin[ ]

exec type

x

lib[ ]

parameters

arg[ ]load lib[ ]

isOveritable

SORCER URIs• A generic way to represent any resource in SORCER.grid

URLssorcer://<authority>/<prv interface>/<prv name>?<query>

- authority is a network host- query is a list of additional provider’s published attributes

URNssorcer:<prv interface>/<prv name>?<query>sorcer:serviceID?<query>

SORCER File Store URNs are in the format:sorcer:sorcer.core.FileStorer/<FS name>?folder=<folder name>&file=<file-name>orsorcer:FileStorer?folder=<folder name>&file=<file-name>orsorcer:FileStorer/HPCC?folder=<folder name>&file=<file-name>

- with no interface qualification used it is assumed that the interface is from sorcer.core.*

Proth’s Theorem

Proth's Theorem (1878):

Let N = h.2k+1 with 2k > h.  

If there is an integer a such that

a (N-1)/2 = -1 (mod N),

then N is prime.

GridDispatcher

Jobber

SORCER Proth Interactions

Download ServiceUI

Upload inputs

Submit Proth Job

SORCER.space

File StoreProvider

Caller

Caller

File StorePortal

Caller

Caller

Caller

Caller

SORCER.grid

Download outputs

Upload inputs

Get results

Jini Service Browser – download Proth ServiceUIMail Reader – get notifications

Web Browser – download/upload filesMail Reader – get notifications

Submit JobGet results

Get inputsStore outputs

Running Proth

FSS Web User Agent

Distributed BLAST in SORCERIntroduction

• BLAST (Basic Local Alignment Search Tool) is a sophisticated software package used in Bioinformatics and Sequence Analysis for rapid searching of nucleotide and protein databases

• BLAST is powerful and has been optimized for performance; however the public sequence databases are big and growing rapidly . Thus, it takes a huge amount of computer resources to search sequence queries in a large database

• As a solution to this problem we provide an implementation of BLAST in a distributed environment using SORCER

Goals

• To increase performance by making the BLAST Process Distributed and sharing idle CPU and other resources

• Focus on high throughput of large numbers of submissions instead of high performance on any single job

• Provide a working system that is easy to Install and Use

• Heterogeneous network- provide a portable system that can be installed on different Compute devices

• The Business Logic is separate from the whole framework. All Libraries and Business Logic Code is Downloaded.

• Self-healing environment, can overcome crashes i.e. fault detection and recovery

• Enable large batch BLAST processes distributed over regular WAN or LAN

BlastProvider

Jobber

S-BLAST Interactions

1 - Download S-BLAST service UI2 - Submit S-BLAST data3 - Read Input files4 - Submit job5 - Write tasks6 - Read task7 - Write task8 - Read tasks9 - Return job10 - Store results11 - Download outputs

Upload inputs

SORCER.space

File StoreProvider

Tasker

Tasker

File StorePortal

Tasker

TaskerTasker

SORCER.grid

Download outputs

Upload inputs

Jini Service Browser – download S-BLAST service UIMail Reader – get notifications

Web Browser – download/upload filesMail Reader – get notifications

Jobber

Jobber

BlastProvider

4

9

12

8

5 7’

6’

6’’

11

7’’’

7’’

6’’’6’’’’7’’’’

6’’’’’

7’’’’’

10

3

Accessing SORCER.grid

SORCER.grid

LUS

Registers services using service proxy

Gateway

SORCER service

Service proxy

Service oriented program

Jobber

Mobileclient 1

Request service

Downloads service proxy

3

Transfers results

6

Discover service

2

4Gets service

5Renders service

Mobile Devices Support

HotSpot CVM KVM Card VM

Java 2Enterprise

EditionJava 2

StandardEdition

J2ME CDC

Handheld

ProfileMID

Profile

SmartCard

ProfileJ2ME CLDC

TVProfile

FoundationProfile

RMIProfile

TVProfile

PersonalProfile

TVProfile

CarProfile

Auto ProfileAuto

Profile

Vertical Grids of iGrid

Cybernodes – iGrid.meshCybernodes – iGrid.mesh

Service Providers – iGrid.gridService Providers – iGrid.grid

Exertions – iGrid.spaceExertions – iGrid.space

Computing Devices – iGrid.netComputing Devices – iGrid.net

SORCER.core

SORCER.grid

iGrid.grid – service providers including services from technology (horizontal) grids

SORCER.core – SORCER infrastructure service providers

SORCER.grid – SORCER domain specific service providers

xmlExertions–iGrid.xmlxmlExertions–iGrid.xml

Application Servers – iGrid.srvApplication Servers – iGrid.srv

Web Services – iGrid.wsWeb Services – iGrid.ws

Surrogate Services

Jini Capable Machine

Surrogate Host

HTTP/HTTPS

Private Protocol

Exported ServiceOther

Exported Service

Inter-connect Specific Code

SORCER.grid

SORCERservice

Jobber

Service Oriented Program

Deploy Calendar Service

Request

Request

Dep

loy

Cal

enda

r Ser

vice

Deployed SORCER SUROGATE Service

Mr. X

Get me C

alendar of M

r. X

Service UI

Request

SORCER Calendar Service Created

SORCER.grid

Surrogate Client

Provide Service

Mr X service

Interaction Using Private Protocol

Web Service to SORCER

UDDIUDDI

SORCER-Web Service GatewaySORCER-Web Service Gateway

JobberExertion Space

Exertion Space

Service CatalogerService Cataloger

Application ServersiGrid.srv

1

2

3’

3’’’

3’’ SORCER.grid

SORCER to Web Service

SORCER – Web Service GatewaySORCER – Web Service Gateway

JobberExertion Space

Exertion Space

Service CatalogerService Cataloger

Application ServersiGrid.srv

UDDIUDDI

2

1

3

5

SORCER.grid

4

Web Service

SO Context Providers

Miner

EDSEDS

EDS

CDS

SORCER.grid FSS

EDS

EDS

CDS

ReconciledContext

ReconciledContext

MAPM

AP

EDS – Elementary Data Store Provider, CDS – Compound Data Store Provider

Durability of Competitive Advantage

Improve engineeringproductivity 10%-20%

Reduce product development

cycle time by 30%

Create a“Category Killer”

Product

1

2

3

4

5

Years

Innovate or Die

Electric Power

Technology Evolution

Mass Adoption

Lab

Public Recognition

Early Adopters

Grid Killer Application

• Generator - electric bulb

• Power grid – lighting and electric appliances

• Microprocessor - PC (spreadsheets, editors)

• Client/server – web browser (documents)

• Grid – service browser (services)

Next Medium?

Time Technology Medium

1950 TV Broadcast

1980 Time sharing E-mail

1990 Client/Server WWW

2000 Grid Computing ???

Service Browsers

• Continuous discover of services

• List of actually available services

• Service properties

• Service UI

• Admin Service UI

• Available for mobile devices

• Integrated with Web Browser?

Inca X Service Browser

MIDP Service Browser

Blitz Dashboard

SORCER Research Domains• Service-Oriented ProgrammingMission: To address research issues in utilizing network objects to execute a network-centric and reliable control strategy of a grid-based program.

• Service-Oriented Computing EnvironmentMission: To develop architectural innovations by distributing service and context providers with grid infrastructure providers to enable execution of service-oriented programs.

• Service-Oriented Programming Development ToolsMission: To improve service-oriented programming and software deployment by improving the methods used to create, test, debug and monitor execution of such programs.

• Service-Federated  Assurance and SecurityMission: To study the cryptography and information security to secure service-oriented grid environments.

• Self-Aware Service FederationsMission: To develop technology that facilitates self-awareness of intelligent service federations. To build and investigate high-performance, practical self-healing software systems for service-federated environments.

• Autonomic Service FederationsMission: To conduct research that combines computer science and biology and facilitates self-management of very complex service-federated environments.

• Service Federated GridsMission: To lead the object-oriented federated services to its full potential by developing common protocols integrated with grids that promote its evolution and ensure its operability.

Theses and 4-blockers

1. Autonomic Management and Monitoring of SO Programs Sekhar Soorianarayanan (Dr. Mike Sobolewski (chair), Dr. Phil Smith, Dr. Noe Lopez Benitez) 

2. Mobile Computing Environment with SORCER Surrogate ServicesRavi Malladi (Dr. Mike Sobolewski (chair), Dr. Phil Smith, Dr. Per Anderson) 

3. A Surrogate Framework for Personal Profile Devices Mukundan Desikan (Dr. Mike Sobolewski (chair), Dr. Hector Hernandez, Dr. Jean Strahlendorf) 

4. A Framework for Integrating SORCER with Web Services Pathangi Rama Krishna Rao (Dr. Mike Sobolewski (chair), Dr. Phil Smith, Dr. Hector Hernandez) 

5. Integrating SORCER with RPC Style Web ServicesKiran Masapari (Dr. Mike Sobolewski (chair), Dr. Hector Hernandez, Dr. Yu Zhuang) 

6. Service-Oriented Data Mining in SORCERTimmayya Kalappa Ame (Dr. Mike Sobolewski (chair), Dr. Susan Mengel, Dr. Hector Hernandez) 

7. Agent-based MetamodelingSandhya Madireddy 

8. SILENUS - SORCER Integrated Local New User's StorageMax Berger

Michael Sobolewskisobol@cs.ttu.com

Recommended