Interoperability for Intelligence Applications using Data-Centric Middleware

Your systems. Working as one.

Prerequisites of network centric intelligence:

Data Distribution Bus

Intelligence Workshop, Rome, May 2012Gerardo Pardo‐Castellote, Ph.D. [[email protected]] CTO, Real‐Time Innovations, Inc. [www.rti.com]Co‐author of DDS specification Co‐chair of the OMG Data‐Distribution SIG

Enhance interoperability, reduce system costs and increase

capability via data‐centric system integration

© 2012 RTI • ALL RIGHTS RESERVED 2

DATA ‐> INFORMATION ‐> KNOWLEDGE + INTELLIGENCE

RTI: Global leader in DDS

• Over 70% worldwide embedded messaging middleware market share

• First with…– DDS API (2004)– RTPS interoperability protocol (2007)

• Active in OMG standardization– Board of Directors member

– Co‐chair DDS SIG– Chair DDS standard revision committees

• Most mature solution– 15+ years of commercial availability

– Diverse range of industries: defense, finance, medical, industrial control, power

generation, communications

– 500+ commercial customers, 100+ research projects

– 350,000+ licensed copies

3© 2012 RTI • ALL RIGHTS RESERVED

4

Challenge: More Data, More Speed, More

SourcesTRENDS:• Growing Information Volume• Lowering Decision Latency• Increasing System Availability• Accelerating technology insertion

and deployment

Next‐generation systems

needs:

• Scalability• Robustness & Availability• Performance• Security• Integration & Evolution• Interoperability


Integration time & cost

High

Low

Small LargeSystem Scale

The Key Challenge: Integration

Interoperability

• Operational– The ability of systems, units, or forces to provide

services to and accept services from other systems, units, or forces, and to use the services so exchanged

to enable them to operate effectively together. (DoD Joint Publication 1‐02)

• Software System– The ability of software systems to exchange

information without loss or change, and to use the exchanged information to produce a useful result.


Why Do We Care?• Interoperability is a force multiplier

– System of Systems capability provide

greatly increased

effectiveness

beyond the "sum of the parts" (individual

systems and technologies)• Operator’s perspective:

– Interoperability allows use of costly

systems at their full

potential• Taxpayer perspective:

– Interoperability allows us to pay once for a capability, vs. many times, and opens the market for multiple

component providers.– Interoperability significantly reduces the largest portion of

total ownership cost ‐

operations and support.

© 2012 RTI • ALL RIGHTS RESERVED

State Of Practice

• Recent studies have shown a growth in interoperability policy issuance in DoD

– Thousands of pages of directives, instructions, and mandates

– Numerous standards and architecture bodies in the DoD• Weak Correlation between Increased Interoperability

and Standards– Standards are necessary, but not sufficient for

interoperability• Conventional means of developing platform and

systems are complex, manpower intensive, and time consuming.

– Achieving Interoperability can increase complexity

© 2012 RTI • ALL RIGHTS RESERVED

Approaches to Software Integration

AppApp

AppApp

AppAppAppApp

AppApp

AppApp Point‐to‐pointPoint‐to‐point


AppApp

AppApp

AppAppAppApp

AppApp


AppApp

AppApp

AppAppAppApp

AppApp

AppApp Server/Broker/ESB

Server/Broker/ESB


DDS Data‐Centric BusDDS Data‐Centric Bus

AppApp AppApp AppApp AppApp AppApp AppApp

AppApp

AppApp

AppAppAppApp

AppApp


AppApp

AppApp

AppAppAppApp

AppApp

AppApp Server/Broker/ESB

Server/Broker/ESB

Levels of Conceptual Interoperability (LCIM)

Level 0No Interoperability

Level 6Conceptual Interoperability

Level 5Dynamic Interoperability

Level 4Pragmatic Interoperability

Level 3Semantic Interoperability

Level 2Syntactic Interoperability

Level 1Technical Interoperability

Stand alone systems that have no interoperability

Full assumptions and constraints of meaningful abstraction of

reality. Fully specified but independent model

Maintains state changes between systems during run time.

Includes assumptions and constraints that effect data interchange

Systems are aware of methods & procedures of other systems.

Context is understood by all participating systems

Meaning of data is exchanged through use of a common

information model. The meaning of information is shared and

unambiguously defined.

Common structure or common data format for exchanging

information. The format of the information exchange is

unambiguously defined

Communication protocol for exchanging data. Bits & Bytes are

exchanged in an unambiguous manner

Tradition

alMiddlew

are

Data‐Ce

ntric

Middlew

are

Copyright © 2010 RTI ‐

All rights Reserved. . 13

Data‐Centric Middleware allows applications to be integrated to the Information Model

APP APP APP APP

DDS Global Data Space

DataModel

Standard Mapping(*)

Standard API

No custom mappings / code necessaryDirect support for data‐centric actions: create, dispose, read/take

Everyday Example: Calendaring

Alternative Process #1 (message‐centric):

1.

Email: “Meeting Monday at 10:00.”

2.

Email: “Here’s dial‐in info for meeting…”

3.

Email: “Meeting moved to Tuesday”

4.

You: “Where do I have to be? When?”

5.

You: (sifting through email messages…)


Example: Calendaring

Alternative Process #2:1.

Calendar: (add meeting Monday at 10:00)

2.

Calendar: (add dial‐in info)3.

Calendar: (move meeting to Tuesday)

4.

You: “Where do I have to be? When?”5.

You: (check calendar. Contains

consolidated‐state)

The difference is state! The infrastructure consolidates changes and maintains it


DDS: Standards‐based

Data‐Centric Integration

Streaming

DataSensors Events

Real‐TimeApplications

EnterpriseApplications

Actuators


Web‐EnabledDDS

2012

Family of Specifications

17

DDSImplementation

Network / TCP / UDP / IP

App

DDSImplementation

App

DDSImplementation

DDS Spec

2004

DDSInteroperablity

2006

UML Profilefor DDS

2008

DDS forLw

CCM

2009

DDS X‐Types

2010

DDSSecurity

20122010

DDS‐STD‐C++DDS‐JAVA5

App


DDS mandated by key DoD

Programs

• UK Generic Vehicle Architecture– Mandates DDS for vehicle comm.

– Mandates DDS‐RTPS for interop.

• DISR– Mandates DDS for Pub‐Sub API– Mandates DDS‐RTPS for Interop

• Army, OSD– UCS, Unmanned Vehicle Control

• US Navy Open Architecture– Mandates DDS for Pub‐Sub

• SPAWAR NESI– Mandates DDS for Pub‐Sub SOA


RTI Connext

DDS Application Examples

Aegis Weapon SystemLockheed MartinRadar, weapons, displays, C2

B‐1B BomberBoeingC2, communications, weapons

Common Link Integration

Processing (CLIP)

Northrop GrummanStandards‐compliant interface

to legacy and new tactical

data links

Air Force, Navy, B‐1B and B‐52

ScanEagle

UAVBoeing

Sensors, ground station

Advanced Cockpit Ground Control

Station

Predator and SkyWarrior

UASGeneral Atomics

Telemetry data, multiple

workstations

RoboScoutBase10

Internal data bus and link to

communications center


RTI Connext


Full‐immersion simulationNational Highway

Transportation Safety

Authority

Migrated from CORBA,

DCOM for performance

Air‐Traffic ManagementINDRA.Deployed inUK, Germany, SpainStandards, Performance,

Scalability

Industrial ControlSchneider ElectricVxWorks‐based PLCscommunicate via RTI‐DDS

Signal ProcessingPLATH GMBH

RTI supports modular

programming across

product line

Large TelescopesEuropean Southern

Observatory

Performance &

Scalability

1000 mirrors, 1sec loop

Radar SystemsAWACS upgrade

Evolvability,

Mainteinability, and

supportability


RTI Connext


Multi‐ship simulator

FORCE Technology

Controls, simulation

display

Mobile asset tracking

Wi‐Tronix

GPS, operational status

over wireless links

Highway traffic

monitoring

City of Tokyo

Roadway sensors, roadside

kiosks, control center

Driver safety

Volkswagen

vision systems, analysis, driver

information systems

Medical imaging

NMR and MRI

Sensors, RF generators, user

interface, control

computers

Automated trading

Automated Trading Desk (ATD,

now Citigroup)

Market data feed handlers,

pricing engines, algorithmic

trading applications


Data‐Centric Qos‐Aware Pub‐Sub Model

Persistence

ServiceRecording

Service

Virtual, decentralized global data space

CRUD operations22© 2012 RTI • ALL RIGHTS RESERVED

Source (key) Latitude Longitude Altitude

UAV1 37.4 -122.0 500.0

UAV2 40.7 -74.0 250.0

UAV3 50.2 -0.7 2000.0

Quality of Service (QoS)

• Aside from the actual data to be delivered, users often need to specify HOW to send it …

…

reliably (or “send and forget”)

… how much data (all data , last 5 samples, every 2 secs)… how long before data is regarded as ‘stale’

and is discarded

… how many publishers of the same data is allowed… how to ‘failover’

if an existing publisher stops sending data

… how to detect “dead”

applications

… …

• These options are controlled by formally‐defined Quality of Service (QoS)

Real‐Time Quality of Service (QoS)

QoS PolicyDURABILITY

HISTORY

READER DATA LIFECYCLE

WRITER DATA LIFECYCLE

LIFESPAN

ENTITY FACTORY

RESOURCE LIMITS

RELIABILITY

TIME BASED FILTER

DEADLINE

CONTENT FILTERS

Volatility

User Q

oS

Delivery

PresentationRedundancy

Infrastructure

Transport

QoS Policy

USER DATA

TOPIC DATA

GROUP DATA

PARTITION

PRESENTATION

DESTINATION ORDER

OWNERSHIP

OWNERSHIP STRENGTH

LIVELINESS

LATENCY BUDGET

TRANSPORT PRIORITY

Operational robustness and performance

© 2012 RTI • ALL RIGHTS RESERVED 25

Architecture for the next‐generation systems

• Existing technologies are reaching robustness/performance/scalability limits

• DDS provides a fundamentally new DataBus

architecture and approach– Powerful data‐centric model

– Ultra‐scalable and robust– Fully decentralized, peer‐to‐peer, “no bottlenecks”

architecture

– Superior Wire Protocol

– Standards‐based, multi‐platform

Single‐lane trafficNo prioritization

Brokers aschoke‐points Connext

DDS Approach


Real‐Time Performance: U.S. Navy Analysis

NESI part 5 v3.0 pg 70

Performance

• Reliable multicast• Fully meshed, reliable

Number of Subscribers

Orders of

magnitude faster

than IT solutions

Fastest DDS solution


Scalability

1 ~1000 subscribers, < 15% throughput decrease

600,000

500,000

400,000

300,000

200,000

100,000

0

Messages pe

r Second

Per Subscriber (2

00 Bytes)

0 200 400 600 800

1,000

Number of Subscribers

• Scalable Performance!• Millions of data

elements• .5m updates/sec

(batched)• 10s µs latency• 1000s consumers

per update• Orders of magnitude

more scalable

than IT solutions


Comparison with other technologies

DDS/GSOAP/JMS/Notification Service Comparison - Latency

0

500

1000

1500

2000

2500

4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384

Message Size (bytes)

DDS JMS Notification Service

Message Length (samples)

Adapted from Vanderbilt presentation at July 2006 OMG Workshop on RT Systems30© 2012 RTI • ALL RIGHTS RESERVED

Joint Battle Command (Blue Force Tracker): Poor Performance, Lack of Maintainability

Mission:• Track positions of friendly and

hostile forces on the battlefield

• Design goal: 100K tracked

updates/sec

Legacy Capability:• 500K lines of code• 8 yrs to develop• 21 servers• Achieved: 20K tracked updates/sec,

reliability and uptime challenges

“This would not have been possible

with any other known technology.”

—Network Ops Center Technical Lead

Next‐Gen Capability:• 50K lines of code—order of

magnitude less

• 1 yr to develop—8x less

• 1 laptop—20x less

• Achieved: 250K+ tracked updates/sec,

no single point of failure


Conclusions• DDS is a mature international Standard from OMG

– Platform Neutral: Operating systems and Programming

Languages

– Deployed worldwide in Military systems and other

Demanding real‐time applications

• DDS Is mandated by DoD

for Publish‐Subscribe and

data‐distribution applications

• DDS is an ideal integration platform for Intelligent

Systems– Highly Tunable via Quality of Service (QoS)– Rich services (persistence, filtering, high‐availability)

• RTI is the Leading provider of DDS technology & Services


Find out more…

www.rti.com

community.rti.com

demo.rti.com

www.youtube.com/realtimeinnovations

blogs.rti.com

www.twitter.com/RealTimeInnov

www.facebook.com/RTIsoftware

www.slideshare.net/GerardoPardo

dds.omg.org

www.omg.org

©2012 RTI • ALL RIGHTS RESERVED 33

http://www.rti.com

http://dds.omg.org

http://www.omg.org

Thank You!
