Streaming iPhone sensor data to SAS® Event Stream Processing · 2017-11-22 · • Text analytics • Streaming geofencing • Reference historic data –Lambda architecture *SAS

Copyright © SAS Inst itute Inc. A l l r ights reserved.

SAS® USER FORUMFINLAND 2017

Streaming iPhone sensor data to SAS® Event Stream ProcessingPasi Helenius – Senior Advisor



SAS® Event Stream Processing3 KEY CHARACTERISTICS

Technology Process steams of data events, on the move, prior to storage, when events happen

SpeedProcess huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisecond)

Actionable Intelligence

Filters/aggregates/correlates stream(s) to focus and detect specific events, patterns or characteristics, that help the business



New era of information processing

USERS NEED IMMEDIATE Answers

Processing streaming data is about getting immediate answers to reduce time to decisionTime to decisionMicro-seconds Days

Move analysis to event source

Analyze before data is stored

Keep what is relevant

Batch

Streaming

Streaming

Analytics

Micro-Batch



nternet

OF

hings

IT

Healthcare

Connected Car/ Transportation

Communications

Energy

Connected CustomerSmart Cities and Homes

Surveillance

Building Management

Agriculture

Retail

ManufacturingInsurance



The IOT Promise

AnalyticsDATAB I G

IOT High Velocity

Complex

Large

New Business Models

Quality of Life

Early Warnings

Efficiencies

New Value

Act

Understand ActSense



Traditional Analytics Lifecycle

DeployETL

Data Data Storage

f

Access – Store - Analyze

Alerts / Reports



Streaming Analytics LifecycleStream – Understand – Act

DeployETL

Data Data Storage

Alerts - Reports Decisioning

Streaming Data Streaming Model Execution

Dep

loy

Train

Train

Score

Sto

re

Enri

ch

Supervise

Score

Copyr i g ht © 2015, SAS Ins t i tu t e Inc . A l l r ights reser ve d .

SAS® Event Stream Processing

ENGINEERED FOR FAST AND ADAPTIVE ACTION

SAS-generated Insights

Event Actions

SAS In-Memory

SAS® Event Stream Processing Model

Continuous QueryP

ub

lish

Sub

scri

be

Streaming Events

Enrichment Data

Analytic Models

Business Rules

Pattern detection at event stream source

Offline, data at rest identifies emerging trends

Feedback new insights back into event streams

Dynamically update queries in-stream

Viya-enabled



SAS® Event Stream ProcessingEngineered for AGILITY

Lightweight embedding

technology

Cloud ready

OS native application

Clustering

Dynamic model update

Low footprint OS native application

From lightweight embedded technology to cloud

distributed architecture

Fulfill new IoT architecture needs

Edge Small Large Cluster Cloud



Edge-to-Enterprise IoT Analytics Platform Cisco and SAS

http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-c-series-rack-servers/aag_Cisco_SAS_Edge_to_Enterprise_IoT_Analytics_Platform.pdf

http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-c-series-rack-servers/aag_Cisco_SAS_Edge_to_Enterprise_IoT_Analytics_Platform.pdf



SAS® Event Stream ProcessingIOT Scenario

SAS ESP Analytics

SAS ESP Studio

SAS ESP Server ESP for Edge Computing

SAS ESP Server

SAS ESP AnalyticsSAS Streamviewer

SAS Event Stream Manager


Demo



Demo



ESP Model showing in ESP modelSetting retention to get previous N readings

Calculate the average acceleration of X,Y,Z axials

Join the moving average with current reading


SAS® Event Stream ProcessingAdvanced Analytics


SAS® USER FORUMSWEDEN 2017

Streaming Analytics LifecycleStream – Understand – Act

DeployETL

Data Data Storage

Alerts - Reports Decisioning

Streaming Data Streaming Model Execution

Dep

loy

Train

Train

Score

Sto

re

Enri

ch

Supervise

Score



SAS® Event Stream ProcessingLearning Models and Feedback Loop

– Machine Learning streaming algorithm support• DBSCAN (density-based

clustering)• K-Means

– ESP Studio• Model authoring assistant

– More algorithms and a monitoring windows to be added in future releases



SAS® Event Stream ProcessingHigh End Streaming Analytics

In-Stream TrainingStreaming K-MeansStreaming DBSCANStreaming Linear Regression*Streaming Support Vector Machines*Streaming Logistic Regression*

In-Stream analytic models processing• SAS® Model Manager 9.4 Integration• SAS® Datastep, SAS® DS2, Python, C• SAS® ASTORE Scoring support

• Temporal pattern detection and analysis• Business rules data quality and policy

definitions• Filter, aggregate and correlate events• Text analytics• Streaming geofencing• Reference historic data – Lambda

architecture

*SAS Event Stream Processing 5.1

Out-of-Stream TrainingRandom ForestGradient Boosting TreeFactorization MachineSupport Vector MachineSupport Vector Data Description

Streaming AlgorithmsStreaming Summary - Univariate StatisticsStreaming Pearson’s CorrelationStreaming Segmented CorrelationWeibull Distribution FittingShort Time Fourier TransformStreaming Text Tokenization



New ESP Window TypesMachine learning and high frequency analytics support

• Calculate Window• Streaming Univariate Statistics and Correlation

• Train Window/Score Window• Streaming K-Means and DBSCAN, ASTORE Support,

Summary, Text, Fourier Transforms

• Model Supervisor Window• Coordination with Model Reader and Train window

to control runtime model deployment

• Queue-based model management

• Model Reader Window• Inject models to Score window, e.g., ASTORE

config

data

score

config

data

model model

data

score

signal

model

modelModel Supervisor

Calculate

Train Score

config

modelModel Reader



SAS® ESP: Text Analytics ScoringIn-stream unstructured text analysis

• Detect interesting events in unstructured text data streams• Define taxonomies/models within SAS® Text Analytics suite• Process streams with dedicated ESP windows

– Extract concepts and categorize content• Text Context & Text Category windows

– Analyze sentiment• Text Sentiment window

• Future: SAS ESP 4.3: Release 17w21– New Text Topics window– Text Topic window runs Text Mine actions on events



SAS® Event Stream Processing3 KEY CHARACTERISTICS

Technology Process steams of data events, on the move, prior to storage, when events happen

SpeedProcess huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisecond)

Actionable Intelligence

Filters/aggregates/correlates stream(s) to focus and detect specific events, patterns or characteristics, that help the business



Thank You!

Documents

Streaming iPhone sensor data to SAS® Event Stream Processing · 2017-11-22 · • Text analytics • Streaming geofencing • Reference historic data –Lambda architecture *SAS