A Transport Framework for Distributed Brokering Systems

Shrideep Pallickara, Geoffrey Fox, John Yin, Gurhan Gunduz, Hongbin Liu, Ahmet Uyar, Mustafa Varank

spallick, gcf@indiana.eduCommunity Grid Computing Laboratory,

Pervasive Technology LabsIndiana University.

http://www.naradabrokering.org

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

2

Talk Outline• Motivation

• Issues that need to be addressed

• Overview of NaradaBrokering

• Discussion of elements in the framework

• Experimental results

• Conclusions

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

3

Motivation• Services need to interact with entities with

disparate networking capabilities– In networking environments.

• Different transports suitable for different tasks• TCP for reliable, Multicast in high concentrations.

• Services work on higher level abstraction of links

• Manage communications to achieve highest possible performance and reliability.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

4

Issues that need to be addressed• Framework Design

– Abstract communication requirements of service layers.– General enough, no details pertaining to specific transports.– Should be able to incorporate support for new transports

easily.

• Ability to deploy special links for specific applications e.g. UDP for audio/video traffic.

• Performance Monitoring• Migration Support • Negotiate best transports

– Eg. Multicast in clusters instead of point-to-point UDP.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

5

NaradaBrokering: Features

• Based on a network of cooperating broker nodes– Cluster based architecture allows system to scale.

• Originally designed as a distributed event service.• Now has four major core functions

– Message transport (based on performance) in multi-link fashion

– General publish-subscribe including JMS, JXTA and Gnutella (started)

– Support for RTP-based audio/video conferencing.

– Federation of multiple instances (just starting) of Grid services

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

6

Why such a Framework for NB• An event traversing over multiple hops could

traverse multiple protocols.• Protocol layers do not need to deal with transport

details– Deals with the abstraction notion of connection link.– Delegate determination of failures to transport layer.

• Broker network should be able to reorganize connections to disseminate data optimally– Transport interfaces need to provide information to

manage this.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

7

Link• Abstracts communications between 2 end points.

– Implementations of Link interface can incorporate transport specific handshaking protocols.

• Failure Detection – Check status of underlying protocols and report

failures.

• Garbage collection of alternate links– Certain links may be short-lived.

• Transport protocol migrations• Security Information

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

8

Link Factory• Manages Links of a specific communications

type.

• Facilitate creation of inbound and outbound communication links.

• Manage migration of communications from a different underlying communication protocol.

• Enable or disable measurement of performance monitoring over a link or its entire set of managed links.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

9

Administrative Links• Used to exchange information between two

nodes regarding the available communications.– Depending on firewall, proxy and NAT boundaries

communications will be over a small set of protocols.

• This information is then used to negotiate best possible transport.

• Communication over the Administrative Link should be HTTP based.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

10

Transport Handler• Manages all registered LinkFactorys

• Maintains reference to Monitoring Service

• Serves as an intermediary between the transport and protocol layers.

• Individual links report availability of data streams to Transport Handler.

• Manage creation of alternate links– Special Links deployed to handle communication

requirements of specific applications.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

11

Performance Monitoring• Every link measures and exposes a set of metrics

– Average delays, jitters, loss rates, throughput.

• Individual links can disable measurements for individual or the entire set of metrics.

• Measurement intervals can also be varied

• Monitoring Service, returns measured metrics to Performance Aggregator.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

12

Performance Monitoring Snapshot

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

13

NaradaBrokering Communications

Transport InterfacesLink

PerformanceData

TransportHandler Link

Factory

LinkFactory

LinksSpecific to a transport

Link Monitors

Data accumulated byMonitoring Service

Brokernode Administrative Link (HTTP)

Optimal Transport

Alternate Link

TransportInterfaces

(Application andContent Dependent)

Negotiated with infoexchanged over

Administrative Link

Brokernode

MonitoringService

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

14

Other advantages• Possible to implement transport interfaces to deal

with legacy clients.– For example we implemented RTP based interfaces

to deal with legacy clients.

• Can also be used to deal with device specific transcoding and transformations.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

15

0

1

2

3

4

5

6

7

8

9

100 1000

Tra

nsi

t D

ela

y (

Mill

iseco

nds)

Message Payload Size (Bytes)

Mean transit delay for message samples in NaradaBrokering: Different communication hops

hop-2 hop-3 hop-5 hop-7

Pentium-3, 1GHz, 256 MB RAM100 Mbps LAN

JRE 1.3 Linux

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

16

0

10

20

30

40

50

60

0 200 400 600 800 1000 1200 1400 1600 1800 2000

De

lay

(Mill

ise

con

ds)

Packet Number

Average delays/packet for 12 (of the 50 total) video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms

NaradaBrokering-RTP JMF-RTP

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

17

0

1

2

3

4

5

6

7

8

0 200 400 600 800 1000 1200 1400 1600 1800 2000

Jitt

er

(M

illis

eco

nd

s)

Packet Number

Average jitter/packet for 12 (of the 50 total) video clients. NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms

NaradaBrokering-RTP JMF-RTP

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

18

Conclusions and Future work• We presented a transport framework that is

appropriate for distributed brokering systems.

• Some issues need to be investigated further– Specification of migration constraints.– Determine feasibility of constraint evaluations in

real-time settings.– Inclusion of codec transformations in link interfaces.

June 25thPDPTA 2003

http://www.naradabrokering.orgspallick,gcf@indiana.edu

19

Related Works • JXTA

• CORBA

• JMS

• NWS