Solving the Industry 4.0. challenges on the logistics domain using Apache Mesos

© COPYRIGHT IKERLAN 2016 www.ikerlan.es

Solving the Industry 4.0. challenges on the logistics domain using Apache Mesos

Technological Centre

Angel Conde, Cristobal Arellano, Santi Charramendieta (IK4-Ikerlan)

Oscar Ocariz (ULMA Handling Systems)


I. Introduction

II. Components

III. Architecture

IV. Conclusions & Future Work

© COPYRIGHT IKERLAN 2016 www.ikerlan.es© COPYRIGHT IKERLAN 2016 3

ULMA Handling Systems

Provides all-round logistics systems (e.g., automatic warehouses)

Custom turnkey solutions

Design, development, assembly & maintenance

World Wide presence


An ULMA Warehouse


COMPONENTS

ConveyorStacker Crane

Elevator

A system that is physically distributed composed by independent devices that carry out automatic tasks

© COPYRIGHT IKERLAN 2016 www.ikerlan.es© COPYRIGHT IKERLAN 2016

COMMON FAILURES

• Physical failures &Deterioration

• Logical failuresSW errorsSW updatesLogical/Physical mismatch

“Automatic elements fail sooner or later”

6


ULMA & the Industry 4.0.

Reduce down time

Integration with 3rd systems

Lower maintenance costs

Predictive maintenance

Smart Warehouse

Industry 4.0

BIG DATA

CIBERSECURITY

IIOT

CLOUD

MODELING & SIMULATION

M2M


Supervisor

Message Bus

DDSAdapter

DDSAdapter

DDSAdapter

ULMA - Supervisor

Message Bus based on DDS

Operational Data

“The Supervisor system gathers information about operational data and malfunctions”

Distributed: monitoring system natureScalable: different size of warehouseExtendible: useful with different devicesReliable: without errors


The Supervisor & Industry 4.0.Useful for local maintenanceLocal data valuable for later analytics

However….

No remote monitoringManual data collectionOperational data can be lost

Industry 4.0

BIG DATA

CIBERSECURITY

IIOT

CLOUD


M2M


The Cloud Supervisor

Live Mirror of each Real Supervisor

Responsible of storing the operational data

Can define its own rules

Aggregates must be supported

Supervisor1

Supervisor2

Real Supervisors

Cloud Superviso

r 1

Cloud Superviso

r 1 & 2

Cloud

Operational Data


I. Introduction

II. Components

III. Architecture



PLATFORM GOALS

Public Cloud, Private Cloud, Hybrid

Avoid Vendor Locking

Resiliency

Elasticity

Efficient resource usage

?


CLOUD AGNOSTIC PLATFORMWhy ?

From Static Resource Partitioning to Elastic Sharing

REST APICassandraSpark

Static Partitioning

Elastic Sharing

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiS9bDCiefPAhUC6RQKHQfwBDgQjRwIBw&url=http://tusimagenesde.com/imagenes-de-netflix-logo/&psig=AFQjCNHkG73uupEIjohw9FGX7N8WRWLPjw&ust=1476973588415263


MESOS OVERVIEW

HTTPJVM

Python

Ruby

distributed resources: CPU, GPU, RAM, I/O, FS, rack locality, etc.

distributed file system

Cluster

DFS

Frameworks

KernelC++

Apps & Services

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwiGiLyGxOTPAhXGSRoKHQZ9DMcQjRwIBw&url=https://twitter.github.io/heron/&bvm=bv.135974163,d.d2s&psig=AFQjCNHasF5ie7A6pTtd--Vk1tRDpfCZpg&ust=1476886219363673

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjx5NuexOTPAhVEcBoKHaaaD_oQjRwIBw&url=https://hortonworks.com/apache/zeppelin/&bvm=bv.135974163,d.d2s&psig=AFQjCNHUXXL2I4HYhw0aVH0pdLIry-oGsg&ust=1476886271290869


MESOS FEATURES

Battle tested on Twitter

Up to 10,000 nodes

Launch any task using or cgroups

Resource isolation

Commercial support via


INFRASTRUCTURE

Launch platform on any cloud provider

Provisioning machines

Automate updates/upgrades

Configuration

“Infrastructure as a code”

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwj4opPuwOTPAhXMuhQKHeweAYsQjRwIBw&url=http://wccftech.com/dev-confirms-power-microsofts-azure-cloud-explains-developers-reclutant/&psig=AFQjCNGfEFjSHaQQNYLyoZtKGkMHCQfbUg&ust=1476885363279236

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjQnazGweTPAhVLthoKHTA4BsoQjRwIBw&url=http://www.averesystems.com/google-cloud-platform&psig=AFQjCNHcPILcJiJU5kLK9MOaLXHxthVxCw&ust=1476885549288635


BUILDING BLOCKS

Data Ingestion & Message Bus (Data Sources)

RT/Batch Data Analytics (Machine learning, failure prediction)

Data Storage (Horizontal scaling)

Advanced Visualization and User Experience

Clou

d In

frast

ruct

ure

17

Supervisor

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=&url=http://datasciencedojo.com/about/services/&bvm=bv.135974163,d.d2s&psig=AFQjCNEs9YBsl_2jd2H2YjxcJW7vu_Bevg&ust=1476884570042397

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwitwO6hvuTPAhWJXhoKHQa8AlwQjRwIBw&url=https://spring.io/&psig=AFQjCNHixzrmdLTtv0KeXvzZqo8wjkwlLA&ust=1476884666937814

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwj4opPuwOTPAhXMuhQKHeweAYsQjRwIBw&url=http://wccftech.com/dev-confirms-power-microsofts-azure-cloud-explains-developers-reclutant/&psig=AFQjCNGfEFjSHaQQNYLyoZtKGkMHCQfbUg&ust=1476885363279236

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwj-nZ-bweTPAhUB2BoKHX2zAhUQjRwIBw&url=https://mesosphere.com/why-mesos/&psig=AFQjCNHD3HmINIe1sql6ByNn2kaGPOlvFw&ust=1476885458658003

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjQnazGweTPAhVLthoKHTA4BsoQjRwIBw&url=http://www.averesystems.com/google-cloud-platform&psig=AFQjCNHcPILcJiJU5kLK9MOaLXHxthVxCw&ust=1476885549288635


REVERSE PROXY& LB

Multiple backends (Mesos, Consul, Docker…)

Dynamic Watchers for backends

Open Source

HTTPS SNI

Deployed on the “edge node”


MESSAGE BUSDecouple the components

De Facto Standard in Big Data Architectures

Publish / Subscriber model

HA enabled

High Performance

Deployed using the Mesos Framework


STORAGE

HDFS deployed in HA (Mesos Master Nodes)

Files stored in Columnar format (Parquet)

Not deployed as Mesos framework

Horizontal scalability

“Small Files problem”


ANALYTICS

Big Data Analytics in Memory

Standard SQL Support

Deployed using the Mesos Spark Scheduler

Machine Learning

Real Time Processing


SERVICE DISCOVERY & MONITORING

Service Discovery using DNS records

Port discovery via DNS SVR records

Service Health-monitoring

Circuit Breakers

Node/Service Monitoring

Task registration via Marathon/Consul


PLATFORM OVERVIEW

Edge Node

MESOS Agent Nodes

Consul Client

Consul Client

Agent

Proxy

MESOS Master NodeNameNode

Consul Server

Master

External traffic

(HTTPS)

Secured Cloud Infrastructure

DataNode

Consul Client

Agent

DataNode

HTTP

ZooKeeper

Mesos coordinatorsHDFS NamenodesConsul for Service DiscoveryTraffic Ingestion

HTTPS SNI

Mesos Agent Consul ClientHDFS DatanodesTasks (Supervisors)


I. Introduction

II. Components

III. Architecture


© COPYRIGHT IKERLAN 2016 www.ikerlan.es 25

Message BusIngestion/ Analytics

ULMA Cloud

Storage

Storage

Storage

Cloud Supervisor

Cloud Supervisor

Cloud Supervisor

Ingestion/ Analytics

Ingestion/ Analytics

1,000 FT OVERVIEW

Supervisor1

Supervisor2

Supervisor3

Oper

atio

nal D

ata


JVM web based app

Dynamic port binding via Marathon

Cgroups used as containerizer

Entry points assigned via labels

Look for:- Random number generation exhaustion - JVM DNS Caching

26

PLATFORM COMPONENTSThe Cloud Supervisor

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwitwO6hvuTPAhWJXhoKHQa8AlwQjRwIBw&url=https://spring.io/&psig=AFQjCNHixzrmdLTtv0KeXvzZqo8wjkwlLA&ust=1476884666937814


Kafka producer module

Sends data to Kafka instead of the local system

Local queue used in case of connection problems

Each supervisor sends data to its own topic (aggregates)

27

PLATFORM COMPONENTSThe Save Interface for the (cloud) Supervisor


Deployed using Spark Mesos Scheduler

Lambda architecture (RT / Batch)

Data Ingestion & Storage from Kafka

AVRO schemas simulating IEC61850.

Ingestion tested up to 100,000 events/sec

28

The Spark ServerPLATFORM COMPONENTS


The Spark Server (II)PLATFORM COMPONENTS

Cloud Superviso

r1

Cloud Superviso

r2.

Operational DataStreaming

Retains recent data on memoryPersists operational data to HDFSLambda (RT/Batch) queries via SQL endpointREST API

SQL

Decouples the systemEach supervisor has its own topic


The Spark Server (III)PLATFORM COMPONENTS

Operational Data

StreamingSQL

• Real-Time Data“Recent Data” in memory tableRT algorithmsData is saved to the “staging” HDFS directoryUncompacted Partitioned Parquet (date/supervisor)

• Batch DataExposes “batch” data via “Old Data” Table Reload compacted data from HDFS when the

compaction is done


HDFS must: Avoid small files!!!!How to solve it?

Scheduled task executed by Chronos

Kite project is used for the task

“Staging” “Compacted”

31

PLATFORM COMPONENTSThe Compactor


I. Introduction

II. Components

III. Architecture



CONCLUSIONS

Remote Monitoring already valuable for clients

Global data collection available 24x7 Platform easily deployable on any provider

Efficient resource usage

Analytics on recent and batch data using standard SQL

Industry 4.0

BIG DATA

CIBERSECURITY

IIOT

CLOUD


M2M


FUTURE WORKMove to

Real-time analytics

Evaluate Cassandra as backend storage

Structured streaming, dynamic allocation on Spark 2.0.+

Run more ULMA software on the platform

Overlay networks, aka “IP per container”


www.ikerlan.es

Questions: [email protected]

Software

Solving the Industry 4.0. challenges on the logistics domain using Apache Mesos