28
05/16/22 05/16/22 uri="gxos://gridform/ggf4/talk/fox" uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" email="[email protected]" 1 Integrating Peer to Peer Networks and Web Services with the Grid PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http://grids.ucs.indiana.edu/ptliupages/presentations/ggf4 feb02 [email protected] Part of GGF4 Update Track Toronto February 18 2002 Grid Programming with Web Services, Software Components and Portal Development Tools

Integrating Peer to Peer Networks and Web Services with the Grid

Embed Size (px)

DESCRIPTION

Part of GGF4 Update Track Toronto February 18 2002 Grid Programming with Web Services, Software Components and Portal Development Tools. Integrating Peer to Peer Networks and Web Services with the Grid. PTLIU Laboratory for Community Grids - PowerPoint PPT Presentation

Citation preview

Page 1: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email=uri="gxos://gridform/ggf4/talk/fox" email="[email protected]""[email protected]"

11

Integrating Peer to Peer Networks and Web Services with the Grid

PTLIU Laboratory for Community GridsGeoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn

Computer Science, Informatics, PhysicsIndiana University, Bloomington IN 47404

http://grids.ucs.indiana.edu/ptliupages/presentations/ggf4feb02

[email protected]

Part of GGF4 Update Track Toronto February 18 2002 Grid Programming with Web Services, Software Components and

Portal Development Tools

Page 2: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 22

Gateway and Web Services We will use the Gateway Computing Portal as an

example (http://www.gatewayportal.org)• It is largely built using CORBA with a Java Server Pages

front end• http://community.ucs.indiana.edu:8004/GCWS/BatchScriptGen/Main.jsp

Several capabilities have been interfaced using WSDL• Job Submission (11 Methods including execute local and

remote command, copy files etc. as well as Submit Job)• Manage WebFlow Session (67 Methods)• Generate Batch Script (just 1 method but two

implementations developed – one at SDSC and one at Indiana – with UDDI to manage)

• Each is one service – could have used finer grain services• Sample files are at

http://grids.ucs.indiana.edu/ptliupages/presentations/ggf4feb02

Page 3: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 33

Gateway Architecture As needed DoD approved Kerberos/SecureID security,

does not usually link to standard Grid technology (Globus)

backend

Globusor submit to localqueue system

Corba middle tierSECIOP withJava modules

JSP SSL

backend

SOAP wrapperJSP Servlet

CORBA

WSDL

Will replacehistorical CORBABy EJB

Add CastorJava to XML

Pre WSDL Post WSDL

Page 4: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 44

Page 5: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 55

Page 6: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 66

Page 7: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 77

WSDL Message Example<message name="submitRequest"> <part name="xmljob" type="xsd:string"/></message><message name="submitResponse"> <part name="response" type="xsd:string"/></message>

For the batch script service, we pass the XML description of the job as a string and get back the script as a string.In general, any XML primitive or complex types can be used in messages.

We could improve our service by defining a BatchScript complex type.

Page 8: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 88

WSDL portTypes Example <portType name="BatchScriptServicePortType"> <operation name="batchGen"> <output message="tns:submitResponse"

name="submitResponse"/> <input message="tns:submitRequest"

name="submitRequest"/> </operation> </portType>

A portType corresponds to a Java class, so if we compile this WSDL to make client stubs, we will generate a BatchScriptServiceBinding.java class.

Page 9: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 99

WSDL SOAP Binding Example<binding name="BatchBinding" type="tns:BatchScriptServicePortType"> <soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> <operation name="batchGen"> <soap:operation soapAction=""/> <input> <soap:body use="encoded“ namespace="urn:BatchScriptService" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/> </input> <output> <soap:body use="encoded" namespace="urn:BatchScriptService“

encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/> </output> </operation></binding>

note binding’s “type” attribute points back

to the portType tag by name

Page 10: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1010

Page 11: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1111

WSDL Ports and Services<service name="BatchScriptService"> <documentation>BS stands for Batch Script </documentation> <port binding="BatchBinding”

name="BatchPort"> <soap:address location=

"http://yourserver/soap/servlet/rpcrouter/"/> </port> </service></definitions>

ports are concrete implementations of portTypes and point back to a particular binding (by name). They also point to the specific location of a server that implements the service.A service is a collection of one or more ports.

Page 12: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1212

What is a Virtual XML Interface We can specify interfaces in XML but we are not required to

implement in XML. Example 1:We aren’t likely to change syntax of mail

Reply-to:Geoffrey Fox <[email protected]>To: Geoffrey Fox <[email protected]>Subject: A Test for Tutorial

A simple mail messageGeoffrey Fox [email protected] FAX 8128567972Phones Cell 315-254-6387 Home 812323919

But we could specify and indeed store in XML with transport done using conventional SMTP.

So conventional mail is easy to give a virtual XML specification for withname:value becoming<name>value</name> but this is not only way

Page 13: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1313

Mail in XML <mailasxml uri=“gxos://mail/users/gcf/sent/2002/february/290” >

<smtpheaders><reply-to email=“[email protected]” >Geoffrey Fox</reply-to><to email=“[email protected]” >Geoffrey Fox</to> <subject>A Test for Tutorial</subject>

</smtpheaders><smtpbody> <message whitespace="collapse"> A simple mail message</message> <signature personuri=“ssn://123/45/6789” whitespace=“preserve” > Geoffrey Fox [email protected] FAX 8128567972 Phones Cell 315-254-6387 Home 812323919 </signature> </smtpbody>

</mailasxml> Such an interface could be used by “User Messaging as a Web service” which

could perhaps integrate News Groups, e-mail, text chat, instant messenger, voicemail

Page 14: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1414

Virtual XML for MPI I In days gone by, we used to study:

MPI_SEND(buf, count, datatype, dest, tag, comm) [ IN buf] initial address of send buffer (choice) [ IN count] number of elements in send buffer (nonnegative integer) [ IN datatype] datatype of each send buffer element (handle) [ IN dest] rank of destination (integer) [ IN tag] message tag (integer) [ IN comm] communicator (handle)

Now our SPMD code is a Web service running in some large scale parallel machine – perhaps integrated with an Apache server running on node• Need to support distributed and shared memory

We want to define all interfaces in WSDL – at least all of them which could be “external” ports

Page 15: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1515

Virtual XML for MPI II datatype becomes an XML Schema specifying structure of

transmitted data – could either be a URI for Schema or its realization• There would be a built-in schema suitable for handling standard

Fortran/C datatypes

buf is a handle to an XML instance of Schema datatype count is still an integer (xs:integer where xs is Schema

namespace xmlns:xs=http://www.w3.org/2001/XMLSchema)• For some bindings count is implied by buf

dest becomes a URI with special notation such as legacympi://procnum/local/nnn for processor numbers

tag becomes an instance of (new) mpitag Schema comm becomes an instance of (new) mpicommunicator Schema

Page 16: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1616

Virtual XML for MPI III So one defines mpi.wsdl with all 100 or so methods

defined as (abstract) operations• We will define several bindings including the classic HPCC

method calls for Fortran C C++ and Java with either distributed memory or shared memory mode

• For C++ and Java we would implements bindings for either “object” or “packed consecutive byte” interpretation of buf

Note one can use both static and run-time compilation to map XML specified services into particular implementations

MPI_INIT() might be implemented with JXTA (peer-to-peer advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH

This brings Web Services to MPI but also MPI to web services

Page 17: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1717

CollabWS: Collaboration as a Web Service This includes Audio-Video conferencing, Instant

Messengers, shared applications and tools There are basic standards H323 and SIP There are peer to peer collaboration approaches like

JXTA from Sun There are various commercial products – Groove

Networks effectively defines some shared application interfaces

CollabWS defines a superset of collaboration properties and subservices in XML/WSDL

Then you provide several bindings of collabws.wsdl to new transports and old ones (binary H323)

Page 18: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1818

H323 SIP

JXTA

Page 19: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 1919

SIPH323 JXTA

H323 JXTA

SIP

Page 20: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2020

A Typical SIP Message REGISTER sip:registrar.biloxi.com

Via: SIP/2.0/UDP 10.4.1.4:5060To: Bob (sip:[email protected])From: Bob (sip:[email protected]);tag=456248Call-ID: [email protected]: 1826 REGISTERContact: (sip:[email protected])Expires: 7200Contact-Length: 0

Initially build a wrapper that accepts such messages and converts to …..

Page 21: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2121

Collaboration as a Web service - now

Page 22: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2222

SIP as XML <sipasxml>

<register> sip:registrar.biloxi.com</register><via> SIP/2.0/UDP 10.4.1.4:5060</via><to> Bob (sip:[email protected]) </to><from tag="456248" > Bob (sip:[email protected]) </from><call-id> [email protected]</call-id><cseq> 1826 REGISTER </cseq><contact> (sip:[email protected]) </contact><expires> 7200 </expires><contact-length> 0 </contact-length></sipasxml>

So this is either an internal format to which we map SIP messages or it is an external format used for Collaboration Web Service

Page 23: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2323

Quick SIP as XML (SOAP based WS) <kwiksipasxmal>

REGISTER sip:registrar.biloxi.comVia: SIP/2.0/UDP 10.4.1.4:5060To: Bob (sip:[email protected])From: Bob (sip:[email protected]);tag=456248Call-ID: [email protected]: 1826 REGISTERContact: (sip:[email protected])Expires: 7200Contact-Length: 0</kwiksipasxml>

This requires a SIP parser

Page 24: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2424

Future Collaboration Web Service

Page 25: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2525

Messaging/Events as a Web Service We can implement messaging subsystem (between

WSDL resources) with either direct messages or by a queued system where you publish messages to queues and subscribe as receiver to particular queues

There are many different publish/subscribe models• JMS is a cluster of central servers• JXTA is a very dynamic Peer to Peer model where pipes are

queues and topics (metadata) are service advertisements Implement JMS API with JXTA protocol – different

WSDL bindings here have different fault tolerance/reliability semantics• Could use JMS as long distance “carrier” between JXTA peers• JXTA provides higher performance than JMS for nearby

recipients

Page 26: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2626

Event Web Service

Web Service 1

(Virtual)Queue

Web Service 2

Destination Source Matching FilterRouting workflow

WSDLPorts

WSDLPorts

Page 27: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2727

Broker Network

Database

Resource

Broker

Broker

Broker

Broker

Broker

Broker

Software multicastP2P: Brokers are clients

Page 28: Integrating Peer to Peer Networks and Web Services with the Grid

04/19/2304/19/23 uri="gxos://gridform/ggf4/talk/fox" email="[email protected]"uri="gxos://gridform/ggf4/talk/fox" email="[email protected]" 2828

Narada JXTA/JMS Integration http://grids.ucs.indiana.edu/ptliupages/projects/narada/ Is a network of event brokers which can reliably deliver XML

specified events Using openJMS selection module, becomes a distributed or

conventional Java Message Service Linking special JXTA

proxies, it can link JXTA communities

Think of JXTA JMS andNarada as differentbindings toevent/messagingweb service