Overview of the OMG Data Distribution Service Douglas C. Schmidt & Jeff Parsons {schmidt,parsons}@dre.vanderbilt.edu schmidt

Overview of the OMGData Distribution Service

Douglas C. Schmidt & Jeff Parsons {schmidt,parsons}@dre.vanderbilt.eduhttp://www.dre.vanderbilt.edu/~schmidt/

Professor of EECS Vanderbilt University Nashville, Tennessee

Tutorial on DDS Douglas C. Schmidt

2

Past R&D Successes: Platform-centric Systems

Legacy DoD systems are designed to be:• Stovepiped• Proprietary • Brittle & non-adaptive• Expensive to develop & evolve• Vulnerable

From this design paradigm…

AirFrame

AP

Nav WTS

GPS IFF

FLIR

Cyclic Exec

Problem: Small changes can break nearly anything & everything


3

…and this operation paradigm…

Real-time QoS requirements for platform-centric DoD systems:• Ensure end-to-end QoS, e.g.,

• Minimize latency, jitter, & footprint• Bound priority inversions

• Allocate & manage resources statically

Uti

lity

Resources

Utility “Curve”

“Broken” “Works”

“Harder” Requirements

Problem: Lack of any resource can break nearly anything & everything

Past R&D Successes: Platform-centric Systems


4

…to this design paradigm…Past R&D Successes: Network-centric Systems

Event Channel

ReplicationService

GPS IFF FLIR

Object Request Broker

AirFrame

AP Nav WTS

Today’s leading-edge DoD systems are designed to be:• Layered & componentized• More standard & COTS • Robust to expected failures & adaptive

for non-critical tasks• Expensive to evolve & retarget• Vulnerable

Problem: Unanticipated changes can break many things


5

…and this operational paradigm…

Past R&D Successes: Network-centric Systems

Endsystem

ApplicationsApplications

Endsystem

MiddlewareMiddleware MiddlewareMiddleware

ApplicationsApplications

Mechanism & PropertyManagers

Sys Cond Sys Cond Sys CondInterceptor Interceptor

LocalResourceManagers

Sys Cond

{}QoS Contracts QoS Contracts

Network latency & bandwidth

Workload & Replicas

CPU & memory

Connections & priority bands

Network latency & bandwidth

Workload & Replicas

CPU & memory

Connections & priority bands

LocalResourceManagers

Adaptive Real-time Middleware


6

…and this operational paradigm…

Past R&D Successes: Network-centric Systems

Problem: Network-centricity is an afterthought in today’s systems

Resources

Uti

lity

Desired Utility Curve

“Working Range”

“Softer” Requirements


8

Coordination Of Multiple UAVs

Dynamic MissionReplanning

Feedback &Control

Image Processing & Tracking

Case Study: QoS-enabled Publish/Subscribe Technologies for Tactical Information Management

DARPA PCES Capstone demo, 4/14/05, White Sands Missile Range


9

Coordination Of Multiple UAVs

Dynamic MissionReplanning

Feedback &Control

Image Processing & Tracking

Case Study: QoS-enabled Publish/Subscribe Technologies for Tactical Information Management

Synchronization

MemoryManagement

PhysicalMemoryAccess

AsynchronousEvent Handling

Scheduling

AsynchronousTransfer of

Control

Modeling Tools Model CheckingReal-time JVMs Real-time ORBs Aspect Languages


10

Limitations with Demo’d PCES Information Management Technologies

Real-time Event Service

Tactical Network & RTOS

Object RequestBroker

Real-time Info to Cockpit

• Shooter information management was based on platform-centric Real-time CORBA & Real-time CORBA Event Service• Well-suited for point-to-point & static pub/sub

command processing over wireline networks• e.g., statically provisioned QoS policies

• Poorly suited for dynamic pub/sub info dissemination to multiple nodes in MANETs

• e.g., too many layers, excessive time/space overhead, inflexible QoS policies & pub/sub model, etc.

Problem: Net-centricity is afterthought in platform-centric technologies


11

Limitations with Demo’d PCES Information Management Technologies

• C2 & C4ISR information management based on J2EE & Java Messaging Service (JMS)• Best suited for operational

enterprise environments• e.g., large data centers

connect via wireline networks• Poorly suited for tactical

environments • e.g., lack of QoS policies &

RTOS integration, extremely high time/space overhead

Java MessagingService

Enterprise Network & OS

J2EEMiddleware

TrackProcessing

Problem: Enterprise technologies are ill suited for tactical systems


12

Our R&D Goal: Evaluate QoS-enabled Info Brokering for Tactical Systems

• Solutions must function properly where• Communication bandwidth is limited/variable• Connectivity is intermittent• Connections are noisy• Processing & storage capacity are limited• Power & weight limits affect usage patterns• Unanticipated workflows are common• Dynamic network topology & membership

changes are frequent• Ideally, solutions should be COTS, standard, &

integrate with heterogeneous legacy systems

Goal is “better than best-effort,” subject to resource constraints…


13

Data Reader

R

Data Writer

R

Publisher Subscriber

Topic

R

Overview of the Data Distribution Service (DDS)

Tactical Network & RTOS

DDS Pub/SubInfrastructure

RT Info to Cockpit & Track Processing

• DDS is an highly efficient OMG pub/sub standard

• e.g., fewer layers, less overhead


14

Data Reader

R

Data Writer

R


Topic

R

NEW TOPIC

NEW

SUBSCRIBER

Overview of the Data Distribution Service (DDS)• DDS is an highly efficient OMG

pub/sub standard• e.g., fewer layers, less

overhead • DDS provides meta-events for

detecting dynamic changes

NEW

PUBLISHER


15



• DDS provides meta-events for detecting dynamic changes

• DDS provides policies for specifying many QoS requirements of tactical information management systems, e.g.,

• Establish contracts that precisely specify a wide variety of QoS policies at multiple system layers

Data Reader

R

Data Writer

R


S1

S2

S3

S4

S5

S6

S7

S6 S5 S4 S3 S2 S1

Topic

R

S7 S7X

HISTORY

RELIABILITYCOHERENCY

RESOURCE LIMITS

LATENCY



16



• DDS provides meta-events for detecting dynamic changes

• DDS provides policies for specifying many QoS requirements of tactical information management systems, e.g.,

• Establish contracts that precisely specify a wide variety of QoS policies at multiple system layers

• Move processing closer to data

Data Reader

R

Data Writer

R


S1

S2

S3

S4

S5

S6

S7

Topic

R

SOURCE

FILTER

DESTINATION

FILTER


TIME-BASED

FILTER


18

DDS Architectural Elements

Data-Centric Publish-Subscribe (DCPS)

– The lower layer APIs apps can use to exchange topic data with other DDS-enabled apps according to designated QoS policies

Data Local Reconstruction Layer (DLRL)

– The upper layer APIs that define how to build a local object cache so apps can access topic data as if it were local

• DDS spec only defines policies & interfaces between application & service

• Doesn’t address protocols & techniques for different actors implementing the service

• Doesn’t address management of internal DDS resources


19

DDS Application Architecture

DCPS

Application

DLRL

Application

DLRL

Application

DLRL

Application

DLRL

Communication

The Application{


20

DDS Domains & Domain Participants

1

2

31

2

3

1

1

DomainParticipant

Node

Domain 1

Domain 2 Domain 3

Node

NodeNodeNode

Node

• The domain is the basic construct used to bind individual applications together for communication

• Like a VPN


21

DCPS EntitiesDCPS Entities include

– Topics

• Typed data

– Publishers

• Contain DataWriters

– Subscribers

• Contain DataReaders

– DomainParticipants

• Entry points

• Data can be accessed in two ways

– Wait-based (synchronous calls)

– Listener-based (asynchronous callbacks)

• Sophisticated support for filtering

– e.g., Topic, Content-FilteredTopic, or MultiTopic

• Configurable via (many) QoS policies

Topic Topic Topic

Data Reader

Data Writer

Data Writer

Data Reader

Data Reader

Data Writer

Subscriber PublisherPublisher Subscriber

Data Domain

Domain Participant


22

QoS Policies Supported by DDS• DCPS entities (i.e., topics, data readers/writers) configurable via QoS policies

• QoS tailored to data distribution in tactical information systems

• Request/offered compatibility checked by DDS

– DEADLINE

• Establishes contract regarding rate at which periodic data is refreshed

– LATENCY_BUDGET

• Establishes guidelines for acceptable end-to-end delays

– TIME_BASED_FILTER

• Mediates exchanges between slow consumers & fast producers

– RESOURCE_LIMITS

• Controls resources utilized by service

– RELIABILITY (BEST_EFFORT, RELIABLE)

• Enables use of real-time transports for data

– HISTORY (KEEP_LAST, KEEP_ALL)

• Controls which (of multiple) data values are delivered

– DURABILITY (VOLATILE, TRANSIENT, PERSISTENT)

• Determines if data outlives time when they are written

– … and many more …


23

Best Effort Reliability QoS in DDS

Publisher

Subscriber

Subscriber

Subscriber

QoS Reliability = BEST_EFFORT

Notification of new data objects

no notification notification time

time-based filter

deadline timeout

• Very predictable

• Data is sent without reply

• Publishers and subscribers match and obey QoS Deadline policy settings

• Time-based Filter QoS policy gives bandwidth control


24

Reliable QoS in DDSQoS Reliability = RELIABLE

Topic

R

Data Reader

R

Data Writer

R


history

S1

S2

S3

S4

S5

S6

S7

S7 S6 S5 S4 S3 S2 S1

Ordered instance delivery is guaranteed


25

Tradeoff Between Reliability & Determinism

X

XX

QoS Reliability = RELIABLE

QoS Reliability = BEST_EFFORT• Can’t have reliability and determinism.

• Best effort mode for streaming data.

• No retries of dropped packets.

• Minimizes latency.

• Reliable mode for commands & events.

• Retry dropped packets until timeout.

• Every packet received in order.

• Speculative cache mechanism.

• DDS reliability is tunable.

• Can be adjusted per subscription.


26

All QoS Policies in DDS• Deadline

• Destination Order

• Durability

• Entity Factory

• Group Data

• History

• Latency Budget

• Lifespan

• Liveliness

• Ownership

• Ownership Strength

• Partition

• Presentation

• Reader Data Lifecycle

• Reliability

• Resource Limits

• Time-Based Filter

• Topic Data

• Transport Priority

• User Data

• Writer Data Lifecycle


27

Single vs. Multiple Domain Systems


28

Data Writers & Publishers• Data Writers are the primary access

point for an application to publish data into a DDS data domain

• The Publisher entity is a container to group together individual Data Writers

• User applications

– Associate Data Writers with Topics

– Provide data to Data Writers

– Data is typically defined using OMG IDL

• It can therefore be as strongly or weakly typed as you desire


29

Data Readers & Subscribers• A Data Reader is the primary access

point for an application to access data that has been received by a Subscriber

• Subscriber is used to group together Data Readers

• Subscribers & Data Readers can have their own QoS policies

• User applications

– Associate Data Readers with Topics

– Receive data from Data Readers using Listeners (async) or Wait-Sets (sync)


30

Types & Keys• A DDS Type is represented by a collection of data items.

– e.g. “IDL struct” in the CORBA PSM

struct AnalogSensor {

string sensor_id; // key

float value; // other sensor data

};• A subset of the collection is designated as the Key.

– The Key can be indicated by IDL annotation in CORBA PSM, e.g.,

#pragma DDS_KEY AnalogSensor::sensor_id• It’s manipulated by means of automatically-generated typed interfaces.

– IDL compiler may be used in CORBA PSM implementation• A Type is associated with generated code:

–AnalogSensorDataWriter // write values–AnalogSensorDataReader // read values–AnalogSensorType // can register itself with Domain


31

Topics• A DDS Topic is the connection

between publishers & subscribers.

• A Topic is comprised of a Name and a Type.

–Name must be unique in the Domain.

–Many Topics can have the same Type.

• Provision is made for content-based subscriptions.

–MultiTopics correspond to SQL join

–Content-Filtered Topics correspond to SQL select.

Domain ID 35

Topic

name “MySensor”

Type

string sensor_id [ key ]

float value

“AnalogSensor” name


32

Topic-Based Publish/Subscribe

• DataWriter is bound (at creation time) to a single Topic

• DataReader is bound (at creation time) with one or more topics (Topic, ContentFilteredTopic, or MultiTopic)

– ContentFilteredTopic & MultiTopic provide means for content-based subscriptions & “joins”, respectively

PressureTemperature


33

Subscription = Topic + DataReader

Topic 1

application

Topic 2 Topic n

DataReader

DataReader

DataReader

subscriber subscriber

QoS

QoS

QoS


34

Content-based Subscriptions

•ContentFilteredTopic (like a DB-selection)

–Enables subscriber to only receive data-updates whose value verifies a condition.

–e.g. subscribe to “Pressure” of type AnalogData

–where “value > 200”

•MultiTopic (like a DB-join operation)

–Enables subscription to multiple topics & re-arrangement of the data-format

–e.g. combine subscription to “Pressure” & “Temperature” & re-arrange the data into a new type:

struct { float pres; float temp; };


35

DDS Content-Filtered Topics

Content-Filtered Topic

Topic

Filter Expression: Value > 260

Expression Params

Topic Instances in Domain

Instance 1

Instance 2

Instance 3

Instance 4

Instance 5

Instance 6

Instance 7

Value = 249

Value = 230

Value = 275

Value = 262

Value = 258

Value = 261

Value = 259

Filter Expression and Expression Params determine which Topic instances the Subscriber receives.


36

DDS MultiTopic SubscriptionsTopic Topic Topic Topic

MultiTopic

Data Reader

Data Reader

Data Reader

Data Reader

Subscriber Subscriber Subscriber

MultiTopics can combine, filter, and rearrange data from multiple Topics.

Domain Participant Domain Participant


37

Example: Create Domain Participant

// used to identify the participantDomainId_t domain_id = ...;

// get the singleton factory instanceDomainParticipantFactory_var dpf = DomainParticipantFactory::get_instance ();

// create domain participant from factoryDomainParticipant_var dp = dpf->create_participant (domain_id, PARTICIPANT_QOS_DEFAULT, NULL);

• DomainParticipant object acts as factory for Publisher, Subscriber, Topic and MultiTopic entity objects


38

Example: Create Topic

……// register the data type associated with the topicFooDataType foo_dt;foo_dt.register_type (dp, “Foo”);

// create a topicTopic_var foo_topic = dp->create_topic (“Foo_topic”, //topic name “Foo”, // type name TOPIC_QOS_DEFAULT, // Qos policy NULL); // topic listener


39

Example: Create Subscriber and DataReader

……// create a subscriber from the domain participantSubscriberQos sQos;dp->get_default_subscriber_qos (sQos);Subscriber_var s = dp->create_subscriber (sQos, NULL);// create a data reader from the subscriber // and associate it with the created topicDataReader_var reader = s->create_datareader (foo_topic.in (), DATAREADER_QOS_DEFAULT, NULL); FooDataReader_var foo_reader = FooDataReader::_narrow (reader.in ());


40

Example: Create Publisher and DataWriter……// create a publisher from the domain participantPublisherQos pQos;

dp->get_default_publisher_qos (pQos);Publisher_var p = dp->create_publisher (pQos, NULL);

// create a data writer from the publisher// and associate it with the created topicDataWriter_var writer = p->create_datawriter (foo_topic.in (), DATAWRITER_QOS_DEFAULT, NULL);

// narrow down to specific data writerFooDataWriter_var foo_writer = FooDataWriter::_narrow (writer);

// publish user-defined dataFoo foo_data;foo_writer->write (foo_data);


41

How to Get Data (Async Listener-based)

Listener_var subscriber_listener = new MyListener();foo_reader->set_listener(subscriber_listener);

MyListener::on_data_available(DataReader reader){ FooSeq_var received_data; SampleInfoSeq_var sample_info;

reader->take(received_data.out (), sample_info.out (),

ANY_SAMPLE_STATE, ANY_LIFECYCLE_STATE);

// Use received_data ……}


42

How to Get Data (Sync Wait-based)

Condition_var foo_condition = reader->create_readcondition(ANY_SAMPLE_STATE, ANY_LIFECYCLE_STATE);WaitSet waitset;waitset->attach_condition(foo_condition);

ConditionSeq_var active_conditions;Duration_t timeout = {3,0};waitset->wait(active_conditions.out (), timeout);...FooSeq_var received_data;SampleInfoSeq_var sample_info;reader->take_w_condition(received_data.out (), sample_info.out (),

foo_condition);// Use received_data


43

Benchmark Scenario

• Two processes perform IPC in which a client initiates a request to transmit a number of bytes to the server along with a seq_num (pubmessage), & the server simply replies with the same seq_num (ackmessage).

–The invocation is essentially a two-way call, i.e., the client/server waits for the request to be completed.

• The client & server are collocated.

• DDS & JMS provides topic-based pub/sub model.• Notification Service uses push model.

• SOAP uses p2p schema-based model.


44

Testbed Configuration• Hostname

blade14.isislab.vanderbilt.edu• OS version (uname -a)

Linux version 2.6.14-1.1637_FC4smp ([email protected])

• GCC Version g++ (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-47.fc4)

• CPU info Intel(R) Xeon(TM) CPU 2.80GHz w/ 1GB ram


45

Test Parameters

• Average round-trip latency & dispersion

• Message types:– sequence of bytes– sequence of complex type

• Lengths in powers of 2• Ack message of 4 bytes• 100 primer iterations• 10,000 stats iterations

// Complex Sequence Type

struct Inner {

string info;

long index;

};

typedef sequence<Inner> InnerSeq;

struct Outer {

long length;

InnerSeq nested_member;

};

typedef sequence<Outer> ComplexSeq;


46

Roundtrip Latency Results (1/2)DDS/GSOAP/JMS/Notification Service Comparison - Latency

10

100

1000

10000

100000

4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384

Message Size (bytes)

Avg

. Lat

ency

(use

cs)

DDS1 DDS2

DDS3 GSOAP

JMS Notification Service


47

Roundtrip Latency Results (2/2)


48

Roundtrip Latency Results Analysis

• From the results we can see that DDS has significantly better performance than other SOA & pub/sub services.

• Although there is a wide variation in the performance of the DDS implementations, they are all at least twice as fast as other pub/sub services.


49

Encoding/Decoding Benchmarks

• Measured overhead and dispersion of– encoding C++ data types for transmission– decoding C++ data types from transmission

• DDS3 and GSOAP implementations compared

• Same data types, platform, compiler and test parameters as for roundtrip latency benchmarks


50

Encoding/Decoding Results (1/2)

DDS/GSOAP Comparison - Encoding Overhead

0.01

0.1

1

10

100

1000

10000

100000

1000000

4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384

Sequence Length

Tim

e (u

secs

)

DDS3 Byte Sequence

DDS3 Complex Sequence

GSOAP Byte Sequence

GSOAP Complex Sequence


51

Encoding/Decoding Results (2/2)

DDS/GSOAP Comparison - Decoding Overhead

1

10

100

1000

10000

100000

1000000

4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384

Sequence Length

Tim

e (u

secs

)

DDS3 Byte Sequence

DDS3 Complex Sequence

GSOAP Byte Sequence

GSOAP Complex Sequence


52

Encoding/Decoding Results Analysis

• Slowest DDS implementation is compared with GSOAP.• DDS is faster.

– Almost always by a factor of 10 or more.– GSOAP is encoding XML strings.

• Difference is larger for byte sequences.– DDS implementation has optimization for byte seq.

• Encodes sequence as a single block – no iteration.

– GSOAP always iterates to encode sequences.

• Jitter discontinuities occur at consistent payload sizes.


53

sub

scri

bers p

ub

lisher

sSummary of DCPS features

• Efficient Publish/Subscribe interfaces

• QoS suitable for real-time systems

– deadlines, levels of reliability, latency, resource usage, time-based filter

• Listener & wait-based data access suits different application styles

• Support for content-based subscriptions

• Support for data-ownership

• Support for history & persistence of data-modifications

DDS

Information consumer subscribe to information of interestInformation producer publish informationDDS matches & routes relevant info to interested subscribers


54

Data Local Reconstruction Layer (DLRL)

TrackTrack 3D_Track

Track_Topic 3D -Track

DLRL

DCPS

Cache


55

Goals of DLRL

• Data Local Reconstruction Layer (DLRL) model is local to an application

• “Object-oriented” access to data

• Application developers can

–describe classes with their methods, data fields, & relations

–attach some of those data fields to DCPS entities

–manipulate these objects (i.e., create, read, write, delete) using native language constructs

–activate attached DCPS entities to update objects

–have these objects managed in a cache

• Like a mapping or binding (intuition only)


56

DLRL Objects

• DLRL objects are (native) language objects with:

–data members and methods

• Only the data members are relevant to data distribution; they can be:

–attributes, i.e., values

– relations, i.e., reference other objects

–plain local data members (i.e., not known or involved in data distribution) are also supported

• DLRL classes can be organised by inheritance

• DLRL objects maps to one or more DCPS Topics


57

DLRL Object Examples

Track

x : real

y : real

comments [*] : string

w : integer

Track3D

z : real

Radar

x : real

y : real

comments [*] : string

z : real

tracks a_radar

* 0..1

w : integer


58

Oid2

z300

T3D_TOPIC

Oid11

RADAR_TOPIC

Oid1

2

x100

102

y200

201

TRACK_TOPIC

ClassTrack

Track3D

radar11

11

COMMENTS_TOPIC

Oid1

1

commentsa comment

another comment

ClassTrack

Track

index0

1

R_Oid11

11

RADAR_TRACKS_TOPIC

T_Oid1

2

T_ClassTrack

Track3D

index0

1

DLRL Radar ExampleTrack

x : realy : realcomments [*] : stringw : integer

Track3D

z : real

Radar

x : realy : realcomments [*] : string

z : real

tracks a_radar

* 0..1

w : integer


59

Evaluating DDSPros• Low overhead & efficient use of transport bandwidth

– e.g., less features/overhead than CORBA in the main request path• Dynamically scalable & highly flexible

– e.g., can receive notification about all sorts of meta-events, such as new topics, publishers, subscribers, etc.

• Supports one-to-one, one-to-many, many-to-one, & many-to-many communication

• Large number of configuration parameters & QoS policies that give developers extensive control of message transmission & reception

Cons• DDS is not well suited to request-reply services, file transfer, & transaction

processing• The data-distribution paradigm is not optimized for sending a request &

waiting for a reply• Implementations don’t yet cover the entire spec• Lack of interoperability between different vendor’s implementations


60

Comparing CORBA with DDS

Distributed object• Client/server• Remote method calls• Reliable transportBest for• Remote command processing• File transfer• Synchronous transactions

Distributed data• Publish/subscribe• Multicast data• Configurable QoSBest for• Quick dissemination to many nodes• Dynamic nets• Flexible delivery requirements

DDS & CORBA address different needs

Complex systems often need both…

Documents

Overview of the OMG Data Distribution Service Douglas C. Schmidt & Jeff Parsons {schmidt,parsons}@dre.vanderbilt.edu schmidt