FIRMS: Federation & Implementation of Reliable Messaging Specifications for

Web Services

Geoffrey Fox (managerial contact)Shrideep Pallickara (technical contact)

October 21 2004 Southampton

Staffing and Contract Update

• Contract still waiting negotiation of terms and conditions– e.g. is the State of Indiana part of British Empire and

subject to its laws

• Have interviewed two good software engineers– Shrideep gave them each a one day exam

FIRMS and FINS• Are built on NaradaBrokering Software with a different

leaner deployment – Can use original deployment if need additional features

Broker Broker

Broker Broker

Publisher

Subscriber

Subscriber

Subscriber

Original

FIRMS

Service NB as Appl. Logic Handler

NB as ServiceHandler Appl. Logic

Multiple protocol transport supportIn publish-subscribeParadigm with differentProtocols on each link

Transport protocols supported include TCP,  Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS.Communications through authenticating proxies/firewalls & NATs. Network QoS based RoutingAllows Highest performance transport

Subscription Formats Subscription can be Strings, Integers, XPath queries, Regular Expressions, SQL and tag=value pairs.

Reliable delivery Robust and exactly-once delivery in presence of failures

Ordered delivery Producer Order and Total Order over a message type. Time Ordered delivery using Grid-wide NTP based absolute time

Recovery and Replay Recovery from failures and disconnects.Replay of events/messages at any time. Buffering services.

Security Message-level WS-Security compatible security

Message Payload options

Compression and Decompression of payloadsFragmentation and Coalescing of payloads

Messaging Related Compliance

Java Message Service (JMS) 1.0.2b compliant Support for routing P2P JXTA interactions.

Grid Feature Support NaradaBrokering enhanced Grid-FTP. Bridge toGlobus GT3.

Web Service reliability Prototype implementation of WS-ReliableMessaging

Current NaradaBrokering Features

Forthcoming Features

• Production implementations of WS-Eventing, WS-Notification, WS-RM and WS-Reliability.

• Time Differential Services: Preserve time spacing between events, that are time-stamped using high-resolution timers.

• Active replay support: Pause and Replay live streams.

• Replicated storage support for fault tolerance and resiliency to storage failures.

WS-Reliable Messaging

• Specification from IBM, and Microsoft• Leverages the WS-Addressing and WS-Policy

specifications.• Provides support for both positive and negative

acknowledgements.• Provides operations for explicit creation and

termination of sequences.• Delivery assurance mode supported: at-least-

once, at-most-once, exactly-once, and ordered delivery

WS-Reliability

• Specification from Fujitsu, Oracle, and Sun• Provides support for positive acknowledgements.• Provides support for a variety of message-exchange

patterns.– Request/Response, One-way, Polling

• Delivery assurance mode supported– Unreliable, at-least-once, ordered-and-exactly-once

• Under consideration by the OASIS to be a standard

Mechanisms for Reliable Messaging I

• There are essentially sequence numbers on each message• Unreliable transmission detected by non-arrival of a

message with a particular sequence number• This is “just some TCP reliability” built at application level

(Service level Internet)• One can either use ACK’s – Receiver (service B) positively

acknowledges messages when received– Service A fully responsible for reliability

• Or NAK’s – Service B is partially responsible and tracks message numbers – sends a NAK if sequence number missing

Service B Service A

M(n+1)M(n)

Mechanisms for Reliable Messaging II• Each message has a retransmission time; messages are

retransmitted if ACK’s not received in time– Uses some increasing time delay if retransmit fails

• Note need to be informed (eventually) that OK to throw away messages at sender; pure NAK insufficient

• Note this is reliability from final end-point to beginning end-point: TCP reliability is for each link and has different grain size and less flexible reliability mechanisms

• There are several efficiency issues– Divide messages into groups and sequence within groups– Do not ACK each message but rather sequences of messages

• NAK based system attractive if high latency (some mobile devices) on messaging from receiver back to sender

Custom Message Reliability

NaradaBroker

Filter 1

Filter 2

WS-RM

WirelessOptimizedWS-RM

WS-Reliability

2 second PDA reply latency!

Different endpoints may well need different reliability schemes. Another reason to use application layer.Need to define easy touse “standard reliabilityprofiles

Handlers and Filters in-memory Processing

Container

Service

Filter pipeline

Filter

Service method

ContainerComponent

Container Component

BytePackets

Support Classes

SOAP Message

Component Classes

Client-side

Application Logic

Filter pipeline

Filter

SupportClasses

Support Classes

Byte processor

Type mappings

SOAP object

SOAPString

ProcessWS-RM

Built-inHandlers

Deployment Issues for “System Services”• “System Services” are ones that act before the real

application logic of a service• They gobble up part of the SOAP header identified

by the namespace they care about and possibly part or all of the SOAP body– e.g. the XML elements in header from the WS-RM

namespace

• They return a modified SOAP header and body to next handler in chain

WS-RMHandler

WS-……..Handler

Header

Body

e.g. ……. Could be WS-Eventing WS-Transfer ….

Proxy Distributed WS-RM Processing• A handler is like an in memory “service” so one

can build handlers that can alternatively be deployed “outside” application service and look like a WS-RM service.

• Comments on handlers as services paradigm?– Will capture this as a deployment memo – Handlers could be just part of application logic – more

natural for WS-Eventing than WS-RM

WS-RM enabledSOAP

WS-RMService

LegacyService

WS-RM removedfrom SOAP Header

WSRM Implementation: Overview

SOAP Parser

WSRMProcessor

ProtocolImplementation

Sequence Policy Information

ProtocolConstants

StableStorage

SOAP Parser

WSRMProcessor

Stable Storage• This is where messages are stored until receiver

indicates that message has been successfully processed– In memory, Flat Files, MySQL supported– In memory (default?) or Flat File is sufficient for

WS-RM messages that do not require sophisticated search

• Can store messages at one or more distributed NaradaBrokering sites– Could keep messages for a long time!

• All “new” communication will be done using SOAP and WSDL defined ports

Complicated NaradaBrokering

• NaradaBrokering has more capabilities than OMII needs– We will make them optional to reduce code bloat

• NaradaBrokering could implement Handler chains for protocols and WSDL bindings not supported by AXIS– UDP plus WS-RM (or equivalent) has been shown

to outperform traditional TCP over high bandwidth high latency links

– Also supports parallel TCP and GridFTP

WS-Reliability WS-ReliableMessaging

Defines Defines elements and attributes in the header block of a SOAP envelope.

An XML based schema for elements that are needed for reliable messaging.

Related Specifications

SOAP, WS-Security WS_addressing, WS-Policy, WS-Security

Companies Involved Sun, Oracle, Fujitsu IBM, Microsoft, BEA

Submission Status Under consideration by OASIS to be a standard

Not submitted yet.

Delivery modes supported

Unreliable, at-least-once, ordered-and-exactly-once

At most once, at least once, ordered and exactly-once.

WS-Reliability WS-ReliableMessaging

Groups of messages

Identified by GroupId information associated with every message in sequence. Individual messages in the group also have a SequenceNumber which increases monotonically.

Grouped together using Sequence element, which has a unique identifier, and a message number which increments sequentially.

Support for creation and termination of message groups

NO YES

Ordering Is guaranteed only within a group of messages.

Is guaranteed only within a group of messages

Ordering and Duplication dependence

Order is always tied to Guaranteed delivery and cannot be separately specified.

Order is not necessarily tied to guaranteed delivery

WS-Reliability WS-ReliableMessaging

Exchange and Specification of protocol constants

Through an abstract concept referred to as Agreement. The spec does not recommend one.

WS-Policy.

Numbering info associated with first element in a group of messages.

SequenceNumber starts at 0 for the first message in a group.

MessageNumber starts at 1 for the first message in a group.

Defines Message-Exchange patterns

Request/Response, One-way and Polling

Not defined

Support for negative acknowledgements

NO YES. This enables receiver-initiated error corrections.

Support for acknowledging range of messages

YES YES

Requesting acknowledgements

The AckRequested element is REQUIRED in every message for which reliable delivery needs to be ensured.

AckRequested is used to request the receiving entity to acknowledge the message received.

WS-Reliability WS-ReliableMessaging

Time based expiry

removerAfter: Receiver Maintains value of GroupId until the end of the sequence is received or after the expiry of a specified time. Plays a role in order/duplicate detection.ExpiryTime: Defines the expiration time associated with an individual message.

Expires: Provides an indication of the expiry time for the sequence specified in the Sequence element.There is also an Inactivity timeout associated with sequences. This is specified in milliseconds.

Retransmissions

Triggered after receipt of a set of acknowledgements. In the event an acknowledgement is not received, the message is retransmitted until a specified number of resend attempts have been made.

Triggered by either positive or negative acknowledgments. Specification of a Retransmission Interval for a sequence. This effects every message within a sequence of messages. The interval can also be adjusted based on the exponential backoff algorithm.

WS-Reliability WS-ReliableMessaging

Security Relies on WS-Security and assorted specifications

Relies on WS-Security and assorted specifications

Errors Are notified through SOAP faults.

Are notified through SOAP faults. Fault processing is more sophisticated since one can leverage WS-Addressing’s message information headers.

WSRM/WSR Similarities

• Messages are part of a sequence/group of messages.• Addresses issues pertaining to ordering and duplicate

detection.• Quality of service constraints are applied to groups of

messages.• Recommends message-level security: specifically

WS-Security for secure delivery of messages.• Provides framework for reporting faults/errors in

processing between endpoints involved in reliable delivery

• Provide support for acknowledging multiple range of messages received within a group/sequence.

WSRM/WSR Differences• WSR has no support for negative acknowledgements.

Retransmissions are always initiated by the source of messages. WSRM supports negative acknowledgements.

• WSRM has an explicit operation for the creation of sequence and associated sequence identifier. WSR has no such operation, a new group begins when a receiver has encountered a new groupID.– disadvantage: difficult to resolve collisions in group id namespace

• WSRM uses WS-Policy for specifying and exchanging featured info. WSR does not advocate any specification though it alludes to an abstract concept of agreements.

• Order is always tied to duplication elimination in WSR. WSRM allows order and duplication detection to exist independent of each other

• WSR incorporates a HTTP binding for its specification. WSRM currently does not have one; though one can simply use SOAP’s HTTP binding.

• WSRM has explicit termination of sequences. WSR groups cannot be terminated. They are first closed and then removed.

Federation, Why?

• WSRM being supported by powerful group: IBM/Microsoft.

• WSR being actively considered for recommendation as a standard by OASIS

• It is quite possible that these specification may continue to co-exist for a while.

• Federation would allow end-points belonging to different specifications to communicate with each other.

Federation, How?• Mapping of physical (XML) elements and semantics associated

with these specifications.– Mapping of sequence numbering. WSRM starts numbering at 1, WSR starts at

0.– NAKs in WSRM messages will simply be ignored, since WSR does not support

it.• Mapping of policy elements and rules associated with

where/when and the combination in which multiple policy/agreement elements may occur.

• Enforcing constraints on delivery. WSR provides a subset of the delivery modes available in WSRM

• Mapping of faults/error reporting• Creation/Termination of sequence in WSRM have no equivalents

in WSR.– So terminate-sequence in WSRM will trigger multiple requests/retransmission

to ensure WSR has received everything. Group expiry time then needs to be updated at the WSR side.

FIRMS Schedule

FINS: Federation & Implementation of Notification Specifications for Web

Services

Geoffrey Fox (managerial contact)

Shrideep Pallickara (technical contact)

WS-Eventing

• Delivery Modes: Push– Pull and Trap (UDP Notifications) through WS-Management

• Subscription Manager can be a separate Web service or part of handler bundled with Source Web Service when it gives as in OGSI special ports for each Source service

Source SinkSubscription

Manager

Subscribe

Unsubscribe

Renew

getStatus

Subscription End

Notifications

WS-Notification

• WS-BaseNotification, WS-BrokeredNotification, WS-Topics, WS-ResourceProperties and WS-ResourceLifetime

ProducerSubscription

Manager Subscriber Consumer

Subscribe

Get current message

Pause subscription

Resume subscription

Notify

NotificationBroker

Publisher Subscriber

NotifyRegisterPublisher

Subscribe

WS-Eventing/Notification Issues• IBM (Sun ..) joined Microsoft (BEA. Tibco) in new August

2004 specification • WS-Eventing will presumably replace WS-BaseNotification

as core of WS-BrokeredNotification• We made WS-Eventing as our highest priority• Eventually support WS-Eventing, WS-Notification and JMS

(Java Message Service) in a federated model• WS-Metadata needed to query WS-Eventing sources

– Not in current OMII plans but clear how to add• Note FINS will use FIRMS in all messages if desired

FINS Schedule

WS-Notification WS-Eventing

Delegated Management of subscriptions

Yes. Through the subscription manager interface.

Yes. Through the subscription manager interface.

Support for replay like features

One can get last message to a topic. A sink can also retrieve messages issued between the pausing and resumption of a subscription.

No.

Subscription operations

Subscribe, Pause and Resume. (There is NO exchange to unsubscribe).

Subscribe, Renew, Unsubscribe and Subscription End.

Support for filters to narrow notifications

YES YES

Subscription lifetimes

Defined using the WS-Resource Lifetime specification.

Contained within the Subscribe and Renew exchanges.

WS-Notification WS-Eventing

Notification filters and topic expressions supported

Topic Expressions supported: QName, “/” separated Strings, and XPath path expressions.

Filter supported is XPath.User defined filters allowed

Hierarchical topics and Wildcards support

Yes. Supports * and // wildcards for selection of topic descendants in a topic tree.

No.

Topic space management

Defined using WS-Topics. The topic space will also support exchanges as defined by the WS-ResourceProperties specification.

No formal recommendation regarding topic management.

Advertisement of supported topics

Yes. The NotificationProducer interface allows inspection of available topics.

No.

WS-Notification WS-Eventing

On demand publishing

YES. This is supported through the WS-Brokered Notification specification. This allows a publisher to publish ONLY if there is a consumer interested in receipt of notifications.

No.

Notification messages

Provides support for both a Notify message as well as “raw” application specific message,

Does not define any special Notification message type.Application can define special ports for different notifications distinct from “real” messages

Suggested Security

WS-Security and assorted specifications.

WS-Security & assorted specifications.

WS-Eventing and WS-NotificationWS-Notification WS-Eventing

Related Specifications

SOAP, WS-Addressing, WS-BaseNotification, WS-Brokered Notification, WS-Topics, WS-Resource Properties and WS-ResourceLifetime

SOAP, WS-Addressing

Support for loosely coupled notifications. (Producers need not know consumers)

Yes. The intermediary called Notification Broker and the exchanges that need to be supported are defined in the WS-Brokered Notification specification.

No.

Delivery modes supported

Push PushPull and Trap (UDP transmissions) defined in WS-Management