View
4
Download
0
Category
Preview:
Citation preview
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® USER FORUMFINLAND 2017
Streaming iPhone sensor data to SAS® Event Stream ProcessingPasi Helenius – Senior Advisor
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® Event Stream Processing3 KEY CHARACTERISTICS
Technology Process steams of data events, on the move, prior to storage, when events happen
SpeedProcess huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisecond)
Actionable Intelligence
Filters/aggregates/correlates stream(s) to focus and detect specific events, patterns or characteristics, that help the business
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
New era of information processing
USERS NEED IMMEDIATE Answers
Processing streaming data is about getting immediate answers to reduce time to decisionTime to decisionMicro-seconds Days
Move analysis to event source
Analyze before data is stored
Keep what is relevant
Batch
Streaming
Streaming
Analytics
Micro-Batch
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
nternet
OF
hings
IT
Healthcare
Connected Car/ Transportation
Communications
Energy
Connected CustomerSmart Cities and Homes
Surveillance
Building Management
Agriculture
Retail
ManufacturingInsurance
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
The IOT Promise
AnalyticsDATAB I G
IOT High Velocity
Complex
Large
New Business Models
Quality of Life
Early Warnings
Efficiencies
New Value
Act
Understand ActSense
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Traditional Analytics Lifecycle
DeployETL
Data Data Storage
f
Access – Store - Analyze
Alerts / Reports
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Streaming Analytics LifecycleStream – Understand – Act
DeployETL
Data Data Storage
Alerts - Reports Decisioning
Streaming Data Streaming Model Execution
Dep
loy
Train
Train
Score
Sto
re
Enri
ch
Supervise
Score
Copyr i g ht © 2015, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SAS® Event Stream Processing
ENGINEERED FOR FAST AND ADAPTIVE ACTION
SAS-generated Insights
Event Actions
SAS In-Memory
SAS® Event Stream Processing Model
Continuous QueryP
ub
lish
Sub
scri
be
Streaming Events
Enrichment Data
Analytic Models
Business Rules
Pattern detection at event stream source
Offline, data at rest identifies emerging trends
Feedback new insights back into event streams
Dynamically update queries in-stream
Viya-enabled
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® Event Stream ProcessingEngineered for AGILITY
Lightweight embedding
technology
Cloud ready
OS native application
Clustering
Dynamic model update
Low footprint OS native application
From lightweight embedded technology to cloud
distributed architecture
Fulfill new IoT architecture needs
Edge Small Large Cluster Cloud
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Edge-to-Enterprise IoT Analytics Platform Cisco and SAS
http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-c-series-rack-servers/aag_Cisco_SAS_Edge_to_Enterprise_IoT_Analytics_Platform.pdf
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® Event Stream ProcessingIOT Scenario
SAS ESP Analytics
SAS ESP Studio
SAS ESP Server ESP for Edge Computing
SAS ESP Server
SAS ESP AnalyticsSAS Streamviewer
SAS Event Stream Manager
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Demo
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Demo
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
ESP Model showing in ESP modelSetting retention to get previous N readings
Calculate the average acceleration of X,Y,Z axials
Join the moving average with current reading
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® Event Stream ProcessingAdvanced Analytics
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® USER FORUMSWEDEN 2017
Streaming Analytics LifecycleStream – Understand – Act
DeployETL
Data Data Storage
Alerts - Reports Decisioning
Streaming Data Streaming Model Execution
Dep
loy
Train
Train
Score
Sto
re
Enri
ch
Supervise
Score
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® Event Stream ProcessingLearning Models and Feedback Loop
– Machine Learning streaming algorithm support• DBSCAN (density-based
clustering)• K-Means
– ESP Studio• Model authoring assistant
– More algorithms and a monitoring windows to be added in future releases
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® Event Stream ProcessingHigh End Streaming Analytics
In-Stream TrainingStreaming K-MeansStreaming DBSCANStreaming Linear Regression*Streaming Support Vector Machines*Streaming Logistic Regression*
In-Stream analytic models processing• SAS® Model Manager 9.4 Integration• SAS® Datastep, SAS® DS2, Python, C• SAS® ASTORE Scoring support
• Temporal pattern detection and analysis• Business rules data quality and policy
definitions• Filter, aggregate and correlate events• Text analytics• Streaming geofencing• Reference historic data – Lambda
architecture
*SAS Event Stream Processing 5.1
Out-of-Stream TrainingRandom ForestGradient Boosting TreeFactorization MachineSupport Vector MachineSupport Vector Data Description
Streaming AlgorithmsStreaming Summary - Univariate StatisticsStreaming Pearson’s CorrelationStreaming Segmented CorrelationWeibull Distribution FittingShort Time Fourier TransformStreaming Text Tokenization
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
New ESP Window TypesMachine learning and high frequency analytics support
• Calculate Window• Streaming Univariate Statistics and Correlation
• Train Window/Score Window• Streaming K-Means and DBSCAN, ASTORE Support,
Summary, Text, Fourier Transforms
• Model Supervisor Window• Coordination with Model Reader and Train window
to control runtime model deployment
• Queue-based model management
• Model Reader Window• Inject models to Score window, e.g., ASTORE
config
data
score
config
data
model model
data
score
signal
model
modelModel Supervisor
Calculate
Train Score
config
modelModel Reader
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® ESP: Text Analytics ScoringIn-stream unstructured text analysis
• Detect interesting events in unstructured text data streams• Define taxonomies/models within SAS® Text Analytics suite• Process streams with dedicated ESP windows
– Extract concepts and categorize content• Text Context & Text Category windows
– Analyze sentiment• Text Sentiment window
• Future: SAS ESP 4.3: Release 17w21– New Text Topics window– Text Topic window runs Text Mine actions on events
SAS® USER FORUMFINLAND 2017
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® Event Stream Processing3 KEY CHARACTERISTICS
Technology Process steams of data events, on the move, prior to storage, when events happen
SpeedProcess huge volumes of streaming data flowing at very high rates (Millions of events/sec) with very low latency (<1 millisecond)
Actionable Intelligence
Filters/aggregates/correlates stream(s) to focus and detect specific events, patterns or characteristics, that help the business
Copyright © SAS Inst itute Inc. A l l r ights reserved.
SAS® USER FORUMFINLAND 2017
Thank You!
Recommended