Embed Size (px)
Citation preview
www.ovum.com
© Copyright Ovum 2014. All rights reserved.
Fast Data: The Rebirth of Streaming Analytics
Tony Baer
Ovum
Teradata Partners, October 21, 2015
© Copyright Ovum 2014. All rights reserved.
What is Streaming Analytics?
The Rebirth
Technology Landscape
Takeaways
Agenda
© Copyright Ovum 2014. All rights reserved.
What is Streaming Analytics?Analyzing & Acting on data in motion
Incoming data
In Motion
Filtered extract
Streaming Analytics
Conventional Analytics
Sense, Transform/Filter,
Analyze Data Respond
Analyze Respond
Event processor
Ingest, Persist Data
Data store
Data with perishable value
Data with historical value
Incoming data
© Copyright Ovum 2014. All rights reserved.
Streaming Analytics is not simply responding to
alarms or outliers
© Copyright Ovum 2014. All rights reserved.
Streaming Analytics examples
Telcos – process CDRs for mediation, revenue assurance, fraud detection, churn prevention
FS – process trades for fraud detection & anomalous activity, refine trading strategies
Utilities – process smart meter data for demand-side management programs
Healthcare – patient monitoring for alerts (e.g., sepsis outbreaks) & offline clinical research
© Copyright Ovum 2014. All rights reserved.
The roots of Streaming Analytics
© Copyright Ovum 2014. All rights reserved.
Streaming Analytics roots
Complex Event Processing (CEP)
Define Event & relationships to other events
Define Event/state Transition
Define Pattern matching rules
Define Response rules
Event Stream Processing (ESP)
Sliding time windows for correlation &
aggregation of events
© Copyright Ovum 2014. All rights reserved.
The showstopper?
Complexity
Costly hardware
Limited bandwidth
Proprietary software
Narrow market appealLimited skills base
No standards
CEP
© Copyright Ovum 2014. All rights reserved.
The rebirth of
Streaming Analytics
© Copyright Ovum 2014. All rights reserved.
What’s changed?
Use Cases – Driven by explosion of Mobile & IoT data
Commodity Infrastructure – Scale-out clusters, multi-core CPUs, gigabit networks, affordable DRAM & Flash storage
Open Source – lowering barriers to entry for developers, data scientists, enterprises, and vendors
Machine Learning provides more flexible, adaptive alternative to rules
© Copyright Ovum 2014. All rights reserved.
Mobile data growth
Source: Ovum
© Copyright Ovum 2014. All rights reserved.
IoT growth
Source: Cisco
2014 2019
67%
40%
By 2019, most IP traffic will come from non-PC devices
By 2019 Global IP traffic will grow 3x to 2 zettabytes/yr.
By 2016, most IP traffic to come from wireless devices
© Copyright Ovum 2014. All rights reserved.
Emerging use cases
Retail – real-time customer engagement via smartphone interaction
Manufacturing – prescriptive maintenance
Telco – Real-time message routing optimization & bottleneck prevention
Local govt. – Real-time Smart City applications
Cybersecurity – Real-time detection & thwarting of intrusions/attacks
© Copyright Ovum 2014. All rights reserved.
Streaming Analytics Technology Landscape –Then
Tibco Streambase
Software AG Apama
SAP Complex Event Processing
Oracle Event Processing
Informatica Rulepoint
IBM InfoSphere Streams
© Copyright Ovum 2014. All rights reserved.
Streaming Analytics Technology Landscape –Now
Veterans Community Open Source New Players
Tibco
Software AG
SAP
Oracle
Informatica
IBM
Spark Streaming
Flink
Kafka
Storm
Samza
DataTorrent
Msft. Azure Stream Analytics
Amazon Kinesis
Teradata Listener
Tigon
Heron
SAS
© Copyright Ovum 2014. All rights reserved.
Streaming Analytics Technology Landscape –Contrasts
Veterans Community Open Source New Players
• CEP/ESP rebranded & leveraging modern commodity infrastructure
• Mature enterprise software
• Mix of proprietary & vendor-lead open source
• Cloud prominence
• Expanding the practitioner base
• Leveraging ML instead or in addition to rules
• Manual coding
Tibco
Software AG
SAP
Oracle
Informatica
IBM
Spark Streaming
Flink
Kafka
Storm
Samza
DataTorrent
Msft. Azure Stream Analytics
Amazon Kinesis
Teradata Listener
Tigon
Heron
SAS
© Copyright Ovum 2014. All rights reserved.
Takeaways
Streaming Analytics… is back!
It’s not only for Wall St. anymore
Mobile & IoT driving compelling real-time use cases outside traditional FS/capital markets niche
Machine Learning provides more adaptive, flexible alternative (or addition) to rules
Commodity infrastructure & open source makes Streaming Analytics affordable, scalable & performant
Open source erodes barriers to entry – but the software is still raw
Don’t rule out mature commercial products – but they must exploit modern commodity, scale-out distributed architectures!
www.ovum.com
© Copyright Ovum 2014. All rights reserved.
Thank you
Tony Baer
Ovum
(646) 546-5330
[email protected] Twitter: @TonyBaer
