Data Distribution Service DDS Tutorial Building Internet of Things Systems with Ease

AngeloCorsaro,PhDCTO,ADLINKTech.Inc.Co-Chair,OMGDDS-SIGangelo.corsaro@adlinktech.com

The DDS TutorialBuildingIoTSystemswithEase

Internet of Things

The Industrial Internet Consortium (IIC) and the Industrie 4.0 (I4.0)

have defined reference models and architectures for IIoT systems.

As IIoT requirements are a superset of CIoT’s we’ll

investigate the need of the former

CIoT and IIoT

Interoperability. Machines, devices, sensors, and people can freely communicate with each other

Information Transparency. A virtual representation of the physical world is made available by enriching digital plant models with sensor data

Technical assistance. Leverage information to make more informed decisions and solving urgent problems on short notice. Physically support humans by conducting a range of tasks that are unpleasant, too exhausting, or unsafe for humans.

Decentralised Decisions. Autonomous decisions are the norm. Higher level delegation happens only in presence of interferences or conflicting goals

Industrie 4.0Design Principles

The Industrie 4.0 Reference Architecture (RAMI) is three dimensional and organises

the life-cycle/value streams and the manufacturing

hierarchy levels across the six layers of the IT

representation of Industrie 4.0

RAMI 4.0

Imagefrom:“ReferenceArchitectureModelIndustrie4.0”

The ability to virtualise physical entities and make

information available is key to RAMI4.0 and captured

as part of the Virtual Representation layer

RAMI 4.0

Imagefrom:“ReferenceArchitectureModelIndustrie4.0”

The IIC Reference architecture is defined on a three tier model and

across four different view points

The key requirements for each viewpoint and tier are identified by

the IIRA

IIRA

Imagefrom:“IndustrialInternetConsortiumReferenceImplementationv1.7”

IIRA connectivity foresees the use of a Connectivity

Core Standard (such as DDS) and then Gateways

to integrate other connectivity technologies

IIRA Connectivity

Imagefrom:“IndustrialInternetReferenceArchitecturev1.7”

IIRA/I4.0 relationship

Connectivity, syntactic interoperability and ubiquitously available virtual representation of physical entities are common needs of both the IIRA and RAMI4.0

DDS, OPC-UA, MQTT, AMQP are some of the key the standards that have been identified as supporting some or all the connectivity and data sharing needs of IIRA/RAMI4.0

IIRA /RAMI4.ocommonalities

Desirable Properties of a Large Scale Distributed System

Performance

Performance requirements in IIoT

can vary from real-time low latency to

soft real-time and interactive Rea

l-Time

SoftRe

al-Time

Interact

ive

Scalability

The architecture should be designed to promote

scale-out and scale-down as opposed to

scale up

Loose Coupling

Functionalities should be completely isolated

from each other in time and space

Unconstrained

Innovation should not be constrained with respect to

supporting devices, new target platforms, new visualisation

engines, new control or optimisation strategies, etc.

Functionalities should be implemented in the most

effective programming language

Innovation

Incremental

The platform should be as modular as possible to facilitate the individual

functionalities updates and upgrade

Additionally, adding new features should be as much as

possible transparent for the currently running system

Change

Fault-Tolerance

The system should be self-healing and

autonomous

DDS: The Data Distribution Service

DDS. Describes the semantics of the information sharing

abstraction supported by DDS. Defines a nominal type system for

describing DDS information models and ensuring syntactical

interoperability.

DDSI-RTPS. Defines a protocol for interoperable wire implementation

of the DDS semantics.

Standard Structure

DDS-XTypes. Extends the DDS type system with support for structural typing as well as a

dynamic type definition.

DDS-Security. Introduces data centric security in DDS for data in

movement as as well as data at rest.

Standard Structure

DDS-RPC. Extends DDS with support for Remote Procedure Calls.

DDS-PSM-*. Defines highly ergonomic and optimised API

mapping for specific programming languages instead of deriving those

for the DDS-PSM-IDL

DLRL. Defines a language independent Object/Relational

Mapping for DDS

Standard Structure

DDS functionalities span from the Session to the Application level in the

ISO OSI Stack

DDS & ISO OSI

UDP TCP

DDSI-RTPS

DDS

User App.

7. ApplicationHigh-level APIs, including resource sharing, remote file access, directory services and virtual terminals

6. Presentation

Translation of data between a networking service and an application; including character encoding, data compression and encryption/decryption

5. Session

Managing communication sessions, i.e. continuous exchange of information in the form of multiple back-and-forth transmissions between two nodes

4. Transport

Reliable transmission of data segments between points on a network, including segmentation, acknowledgement and multiplexing

3. NetworkStructuring and managing a multi-node network, including addressing, routing and traffic control

2. Data LinkReliable transmission of data frames between two nodes connected by a physical layer

1. PhysicalTransmission and reception of raw bit streams over a physical medium

IP

Programming language, Operating System,

and HW architecture Independent

PlatformIndependent

DDS Abstractions

Applications can autonomously and

asynchronously read and write data enjoying

spatial and temporal decoupling

Virtualised Data Space

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Virtualised Data Space

Virtualised Data Space

Data Writer

Virtualised Data Space

Data Writer

Virtualised Data Space

Virtualised Data Space

Data Reader

DDS’s virtualised data space is key in enabling loose coupling essential to deal with the fault-tolerance, scale and the heterogeneity needs of IIoT systems

Virtualised Data Space

DDS’s Data Space is eventually consistent with

respect to writes

That means that readers of some kind of data will

eventually see a write, but they may not observe it at

the “same time”

CONSISTENCYMODEL

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

A Topic defines a domain-wide information’s class by a

<name, type, qos> triple

DDS Topics allow to express functional and non-

functional properties of a system information model

Topic

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicType

Name

QoS

Topic types can be expressed using

different syntaxes, including IDL and

ProtoBuf

Topic Type

struct TempSensor { long sid; float temp; float hum; float precision; }; #pragma keylist CarDynamics cid

IDL

Built-in dynamic discovery isolates applications from

network topology and connectivity details

Dynamic

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Discovery

locationtransparency

Cloud Computing

Fog Computing

Device-to-Cloud Communication

Device-to-Device Communication

Fog-to-Cloud Communication

Cloud-to-Cloud Communication

Device-to-Device Communication

Collect | Store | Analyse | Share

Fog Computing

Fog Computing

No single point of failure or bottleneck

Decentralised

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicDQoS

TopicDQoS

TopicAQoS

Data-Space

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

Connectivity is dynamically adapted

to choose the most effective way of

sharing data

Adaptive

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

TopicDQoS

TopicDQoS

TopicAQoS

ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbepeer-to-peerexploitingUDP/IP(UnicastandMulticast)orTCP/IP

ThecommunicationbetweentheDataWriterandmatchingDataReaderscanbe“brokered”butstillexploitingUDP/IP(UnicastandMulticast)orTCP/IP

Connectivity

QoS policies allow the expression and control over data’s temporal

and availability constraints

QoS Enabled

DDS Global Data Space

...

Data Writer

Data Writer

Data Writer

Data Reader

Data Reader

Data Reader

Data Reader

Data Writer

TopicAQoS

TopicBQoS

TopicCQoS

TopicDQoS

QoS Policies controlling end-to-end

properties follow a Request vs. Offered

QoS Domain

Participant

DURABILITY

OWENERSHIP

DEADLINE

LATENCY BUDGET

LIVELINESS

RELIABILITY

DEST. ORDER

Publisher

DataWriter

PARTITION

DataReader

Subscriber

DomainParticipant

offered QoS

Topicwrites reads

Domain Idjoins joins

produces-in consumes-from

RxO QoS Policies

requested QoS

DDS support for peer-to-peer communication along with its rich set of QoS enable extremely high and controlled performances

Performance

DDS has applicability across the 6 IT levels

Concerning the SCADA and the life-cycle layers, DDS

applicability depends on the constraints of the device

DDS-XRCE will bring connectivity to extremely small devices, i.e. at

most 100KB of RAM

Applicability

DDS

DDSD

DS

Due to its performance and scalability

characteristics DDS is applicable across the

three theirs identified by the IIRA

Applicability

DD

S

Data Modeling Idioms

A state that is periodically updated

Examples are the reading of a sensor (e.g. Temperature

Sensor), the position of a vehicle, etc.

Soft State

Reliability=>BestEffortDurability=>VolatileHistory=>KeepLast(n)Deadline=>updatePeriodLatencyBudget=>updatePeriod/3DestinationOrder=>SourceTimestamp

A state that is sporadically updated and that often has

temporal persistence requirements

Examples are system configuration, a price

estimate, etc.

Hard State

Reliability=>ReliableDurability=>Transient|PersistentHistory=>KeepLast(n)DestinationOrder=>SourceTimestamp

The occurrence of something noteworthy for

our system

Examples are a collision alert, the temperature

beyond a given threshold, etc.

Events

Reliability=>ReliableDurability=>anyHistory=>KeepAllDestinationOrder=>SourceTimestamp

Data-centric design leverage the same principle of Feedback-

control loops to assert a state

In other terms, the desired state is asserted by writing a topic and

the actual state is monitored.

A control action is taken when the desired and the actual state

differ

Control-Loop

Data-centric design leverage the same principle of Feedback-

control loops to assert a state

In other terms, the desired state is asserted by writing a topic and

the actual state is monitored.

A control action is taken when the desired and the actual state

differ

Control-LoopHard State

microservice

Soft State

DDS. Microservices. IoT.

microservice“The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms.” [2]

Architecturalstyle

Server Tier

ProcessFunctionality

Microservice

Individual functionalities become unit of

deployment and run in their own process

Architectures

Microservice Architectures

Microservices communicate through

some lightweight mechanism

Server Tier

ProcessFunctionality

Server TierBenefits

Scaling microservice applications is easier as

we can scale out individual functionalities

Benefits

Unconstrained Innovation as we

choose the technologies stack that makes the

most sense for implementing a given

feature

Server Tier

ProcessFunctionality

Incremental Change is facilitated allowing

incremental deployment of new functionalities.

Potentially different version of the same micro service

could be running at the same time!

Benefits

Server Tier

ProcessFunctionality

Loose Coupling and High Cohesion are

easier to achieved to preserve as the

“barriers” between functionalities are very

thick

Server Tier

Process Boundary / Share Nothing

Benefits

Performance of microservice

architectures may be degraded by the higher

cost of communication if the right technology is

not used

Server Tier

Challenges

Monolithic Implementation

Microservice Implementation

In-Process Communication

Inter-Process/Host Communication

Deployment and Operation of systems

based on the microservice

architectures is inherently more complex,

especially at scale and when the right

technologies are not used

Server Tier

Challenges

Monolithic Implementation

Microservice Implementation

Putting it All Together

The architectural style promoted by DDS is ideal for Microservices

architectures

Microservices are one of the emerging paradigms to design

IIoT systems

Additionally DDS mitigates some of the challenges posed by

Microservices

DDS + Microsvcs

A DDS-based Microservice Framework

First class support for Data-Centric modelling idioms, i.e.

Soft/Hard State and Events

Currently supporting only JVM as a target

AgentV

AgentV provides a set of microsvcs that facilitate

the development and deployment of IoT

applications with the Intel Edison IoT board

Agentv and Edison IOT

Thanks to Vortex AgentV micro services can be

seamless deployed on anything, devices, fog

nodes and cloud

Agentv and Vortex

Coding Lab

Your First AgentV Microsvc

IIRA and RAMI4.0 define the key architectural attributes and concerns that need to be addressed in IoT systems

DDS provides a very good match for providing the connectivity and syntactical interoperability

DDS-based Microservices provide a very elegant way of composing IoT systems

Summing Up

AgentV