Why Cloud, IoT, Machine Learning and Cybersecurity Alldownload.1105media.com/tdwi/Remote-assets/Events/2017/Savann… · Real‐Time Streaming Integration • Approach – Collect,

© 2017 Striim, Inc. All rights reserved.

Steve WilkesStriim Co‐Founder and CTO

Why Cloud, IoT, Machine Learning and CybersecurityAll Need Streaming Integration & Analytics


Enterprise, Cloud and IoT Are Not Islands

Enterprise Cloud IoT


They Are Part of A Connected Eco‐System

Enterprise

Cloud IoT


Part of a Digital Transformation that Includes AI

Enterprise

Cloud IoT

Predictive maintenance/ part management

Cybersecurity Analytics CGM predictive monitoring

Automating customer engagement through smart botsAI driven

adaptive AML

NLP Call Center/Sentiment AnalysisRetail Banking

Machine Learning


Every Industry Is Undergoing Digital Transformation

Financial Services Healthcare Manufacturing

Retail Communication Transportation/Logistics

ITInsurance Public Sector


Data Generation Rates Are Growing Exponentially

0

20

40

60

80

100

120

140

160

180

2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025

Data Generated and Stored Annually (Zettabytes) *

Other Data Realtime Data Available Storage

Today We Generate Around 16ZB Data Annually

By 2025 This Will Leap to 160ZB

By 2025 25% Of All Data Will be

Real-Time

Only A Small Percentage Of This Data Can Be Stored

About 5% Of This Is Real-Time Data

* Data Age 2025: The Evolution of Data to Life-Critical. An IDC White Paper, Sponsored by Seagate

95% Of Real-Time Data Will Be

Generated By IoT


If You Can’t Store All Data – What Can You Do?

0

20

40

60

80

100

120

140

160

180

2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025

Data Generated and Stored Annually (Zettabytes) *

Other Data Realtime Data Available Storage

* Data Age 2025: The Evolution of Data to Life-Critical. An IDC White Paper, Sponsored by Seagate


Not Just IoT Data is Massive – Cybersecurity For Example

SERVERS &SERVICES

NETWORKDEVICES

SECURITYDEVICES

How do you correlate all events for immediate insights and proactive responses?

How do you avoid losing or ignoring valuable data, while still storing only the minimum?

How do you act promptly to better serve customers, protect reputation, and beat competitors?


All Data Arrives in Streams Not Batches

databasehumans

eventsdevices

logsmachines

… stream processing hasemerged as a majorinfrastructure requirement

streaming


Data Modernization

Moving to a “streaming first” data architecture, cloud, streaming analytics, and IoT.

Bridge the old and new worlds of data.


Customer Comments on Data Modernization

Our Legacy Systems Can’t Keep Up…““ ””

Our Legacy Systems Can’t Keep Up…“ ”We Need to Prepare

For a Huge Increase In Data Volumes““

””We Need to Prepare

For a Huge Increase In Data Volumes“

”

We Need to Manage a Massive Up‐Tick in Integration & Analytics Requirements

““””

We Need to Manage a Massive Up‐Tick in Integration & Analytics Requirements

“”

…We Can’t Just Rip and Replace Systems““ ””

…We Can’t Just Rip and Replace Systems“ ”

New and Old Systems Need to Co‐Exist Until The Old Systems can be Retired. With Failover…

““””

New and Old Systems Need to Co‐Exist Until The Old Systems can be Retired. With Failover…

“”

Our Current Latency is Way Too High…““ ””

Our Current Latency is Way Too High…“ ”


Part of Overall Data Architecture

Multiple data sources

(Existing) ETL Jobs

Streaming Integration & Analytics

Bat

ch /

Hig

h-La

tenc

yR

eal t

ime

/Lo

w-L

aten

cy ODS/ EDW

Real-Time Applications

Legacy Applications

HadoopSpark Hive

Big DataApplications

Users

MachineLearning

MLAnalytics


Drag and Drop UI+ Command Line Interface

Distributed High Speed Message Infrastructure

Distributed In‐Memory Data Grid for Metadata / Control

Distributed In‐Memory Data Grid for Context Data

Distributed Persistent High Speed Message Infrastructure

Distributed Results Storage

…

Real‐Time Streaming Dashboardsto Surface In‐Memory Analytics

TQL / JDBC / ODBC / REST / WS APIsTQL / JDBC / ODBC / REST / WS APIs

SQL‐Based ProcessingAnd Analytics

CLUSTER

ContinuousData Collection

Databases (CDC)FilesMessagingCloudBig DataDevices

ContinuousData Delivery

DatabasesFiles

MessagingCloud

Big Data

Kafka

Scalability, Distribution, Clustering & Failover

Reliability, Recovery & E1P

Role‐Based Security & Encryption

Managem

ent & M

onitoring

Sources Targets

Streaming Analytics Platforms Require Lots of Pieces


IoT Requires a Smart Data Architecture


That Bridges the Edge, On‐Premise and Cloud

Cloud IoT Apps

On-Prem ServerEdge Server

Devices Edge

Cloud Server

Build & Deploy

Visualize&

Monitor

UI

Feedback

Scale / Speed Intelligence / Depth


Integrate Machine Learning

Collect Data to Train Model and Use Model in a Streaming Fashion to Perform Real-Time Anomaly Detection or Predictions• Source Data From Databases, Files, Kafka, etc.• Process and Prepare the Data and Write to Disk to Train Machine Learning• Export Trained Model and Use in Streaming Analytics Real-Time Scoring• Present Results on Dashboard and Alert on Anomalies

ServerReal-Time

Dashboard

Training Files

Process & Prepare Data

MachineLearning

ExportedModel

WrapperFunction

Streaming Analytics

CQ Real‐TimeScoring


Different Categories of Business Use Cases

Real-Time Data Integration

Data Lake Integration Offloading Reporting Operational DS/ DW

Cloud MigrationData Distribution

IoT Edge Processing

Detecting Patterns & Anomalies

Fraud DetectionCyber Security

Anti-Money Laundering Location-based Offers

Predictive MaintenanceIoT Analytics

Monitoring Real-Time Metrics and KPIs

Call Center Quality Network QualityPOS MonitoringSLA Monitoring

Replication MonitoringDevice Monitoring

BusinessUse Case

Technology Solution


Example Use Cases

Hybrid‐Cloud IntegrationHybrid‐Cloud Integration

Real‐Time Streaming IntegrationReal‐Time Streaming Integration

Cyber SecurityCyber Security

Health Care Device MonitoringHealth Care Device Monitoring

Location TrackingLocation Tracking


Hybrid‐Cloud Integration

• Approach– Use Initial Load

+ CDC to Move Data• True Real-Time Integration

– CDC pushes new data real-time– Process as necessary– Monitor and alert on issues

• Benefits– Streaming not Batch– Cloud Always Up to Date– Not Limited to Single Target


Real‐Time Streaming Integration

• Approach– Collect, Prepare and Enrich

Streaming Data for Delivery to Multiple Targets

• Simple SQL-Based Processing– Filter, Transform, Aggregate &

Enrich Streaming Data– Many Targets in one flow

• Benefits– Easy to Collect Real-Time Data– SQL Enables Non-Developers– Simple Deployment / Monitoring


Cyber Security

• Approach– Collect and Correlate Data From

Network, VPN, Firewall, Devices, Motion Sensors, etc.

• Identifies Multi-Phase Attacks– Port Scans + External Access– Operationalize AI– Unusual User & Machine Behavior

• Benefits– Instant Insights– Proactive vs Reactive– Not Limited to Single Solution


Health Care Device Monitoring

• Approach– Collect, Analyze, Aggregate

Device Data. Join with Patient Data. Obfuscate for Cloud

• Real-Time Patient Monitoring– Multiple Medical Measurements– Use ML on Anonymous Data– Look for Anomalies / Issues

• Benefits– Doctors Have Real-Time Insights– React Immediately– Large Scale Data for Trends


Location Tracking

• Approach– Collect and Analyze Location Data

Enriched With Contextual Information And Zones

• Real-Time Tracking– 1000s Real-Time Locations– Multiple Active Zones– Identify Entry / Exit / Wait / etc.

• Benefits– Spot Unusual Activity– Integrate With Existing Context– Real-Time Insights & Alerts

+


Key Takeaways

• Customer demand now requires immediate insight and action for operational excellence

• Growing data volumes require pre‐processing in‐flight before storing data

IT’S THE RIGHT TIME FOR STREAMING FIRSTIT’S THE RIGHT TIME FOR STREAMING FIRST

• Streaming data architecture addresses both concerns • Add an enterprise‐grade streaming data platform to existing infrastructure with small use cases and expand gradually

• No need to rip and replace batch solutions

YOU NEED A FULL END‐TO‐END PLATFORMYOU NEED A FULL END‐TO‐END PLATFORM


About Striim


Striim Is a Complete End‐to‐End Platform

ContinuousData Collection

DBs (thru CDC), files, HDFS,system logs, message

queues, sensors

Stream Processing

Real‐Time Filtering, Transformation,

Aggregation, Enrichment

StreamingAnalytics

Correlation, CEP, Statistical, ML, Alerts and Visualization,Trigger External Systems

Continuous Results Delivery

Enterprise & CloudDBs, files, Big Data, Blob Storage, Kafka, etc.

Enterprise Grade Streaming First ArchitectureClustered, Distributed, Scalable, Reliable and Secure

Streaming Integration & Analytics PlatformSupporting Enterprise, Cloud and IoT

Flexible Architecture With Deployment On‐Premise / At The Edge / In The Cloud

Integration With Existing Enterprise Software


Integration and Analytics Through Data Flows


Visualization Through Streaming Dashboards


Striim’s Key Differentiation

Striim is unique in the market by providing all 4 of the following in a single platform.

End-to-End Easy to Use

Enterprise Grade

Easy to Integrate

• Log-based Change Data Capture• Deep integration with Kafka• Integrates with other technologies

easily to collect data and distribute• Top 3 Cloud Platforms• Top 3 Big Data Platforms• Major Enterprise Databases• Multiple Open Source Solutions

• Single Platform for Collection, Processing, Analysis, Delivery and Visualization of Streaming Data

• Supports wide variety of data sources, targets, and data types

• Converged In-Memory Platform• Consistent end-to-end UI

• Low configuration installation• Fast to build and deploy

apps in days• Easy to iterate using

SQL-like language• Continuous ingestion and

processing• Multi-stream correlation• Time series/windowing

• Secure with built-in authentication, protectionand encryption

• High performance and highly scalable with distributed architecture

• Reliable with fault-tolerant architecture and “exactly once” processing


Get Started Today

@StriimTeam facebook.com/Striim

Resources

www.striim.com/resources/

Product Page

www.striim.com/product/

Download the Striim Platform

www.striim.com/download-striim/

Share

Documents

Why Cloud, IoT, Machine Learning and Cybersecurity Alldownload.1105media.com/tdwi/Remote-assets/Events/2017/Savann… · Real‐Time Streaming Integration • Approach – Collect,