Upload
hadoop-summit
View
345
Download
0
Embed Size (px)
Citation preview
Copyright © 2016 Splunk Inc.
Log I am your father:search your data, you know it to be true
James Hodge,Principal Product Manager
Spelunking:
Splunking:
to explore underground caves
to explore machine data
Explore your IoT data
4
The data we know and use
The available data we don’t know or use
5
Nice, structured, tidy data
It began with logs…
7
Machine data time series, in motion, unstructured
It can be big data…
…and it is slightly crazy “data”…
10
… and it is messy and lazy
SECURITYIT OPERATIONS
CUSTOMER EXPERIENCERISK & COMPLIANCE
PRODUCT PERFORMANCEIOT
But it has many uses
and lots of untapped value
There’s gold in that there data
13
Make machine data accessible,usable and valuable to everyone.
13
[2014-09-04-14.45.54.608000] proc_source="B24A", tmst_target="2013-09-04-14.45.54.724000", serv_id="ISS", proc_input="MAST", proc_target="B24H", interface_acq="BNET_1", interface_iss="02008", cod_msg="XJYZ", oper_rrn="XJYZ", card_id="52xxxxxxxxxxx", oper_amount="000000000050", oper_currency="978", oper_country="380", term_id="0059XXXX", circuito="", sett_merc="4722", bin_acq="XXXX", id_merc="32xxxxxxxxxx", prcode="XYZ", action_code="XXX", ......auth_rout_id="HISO_AUTH", msg_subst="", ndg="00000xxxxxxxx", station_acq="STA-BNET-MI1", acceptor="A COOL SHOP", tmst_ins="2013-09-04-14.48.56.277466", ...
Mastercard Circuit
Client IDMerchant name
AmountCard ID (masked)
Merchant category IDMerchant ID
There’s gold in that there data
Turning Machine Data Into Business Value
15
Platform for Machine Data
ApplicationDelivery
ITOperations
Security,Complianceand Fraud
BusinessAnalytics Industrial
Data andInternet of
Things
Security,Complianceand Fraud
BusinessAnalytics Industrial
Data andInternet of
Things
HA/DR Admin Data Security Apps SDKs/APIsScale
CollectData
IndexData
Enrich Data
Search & Explore
Analyze& Predict
Report &Visualize
Alert & Action
16
Fully Integrated Enterprise Platform
Technology Telecommunications Travel and Leisure
Education
Healthcare
Energy and Utilities
Manufacturing
Financial Services and Insurance
Media
Proven at 11,000+ Customers in 100 CountriesMore Than 80 of the Fortune 100
Retail
Cloud and Online Services
Government
Splunk Capabilities for IoT
Data Ingest at Scale
PartnerEcosystem
Advanced Analytics and Visualization
Sense and Respond
IoT and Industrial Machine Data
DevelopVisualize PredictAlertSearch
Engineers Data Analysts
Security Analysts
Business Users
Native InputsTCP, UDP, Logs, Scripts, Wire, Mobile
SDKs and APIsJava, JS, C#, Python, Ruby, PHP
Modular InputsMQTT, AMQP, COAP, REST, JMS
HTTP Event CollectorToken Authenticated Events
Real-time
Technology PartnershipsKepware, ThingWorx, Cisco, Palo Alto
MaintenanceInfo
AssetInfo
DataStores
External Lookups/Enrichment
OT
Industrial Assets
IT
Consumer and Mobile Devices
HTTP Event CollectorSupports DevOps and IoT data analysis needs at scale
20
• Standard API and logging libraries send events directly to Splunk• Libraries integrated with popular platforms and services
Scales to Millions of Events/Second
EVENT COLLECTOR API
Advanced Analytics
Splunk ML Extensions• New/Enhanced Commands: Data Sampling,
Fit, Apply, Summary, Predict• Access to Python Data Science Library• Model storage and export to production
Splunk ML App• Step-wise guidance to create, test and deploy
custom ML models • Purpose-built visualizations• Sample data and best practices
Optimizing enterprise operations with predictive ML analytics
ADVANCED VISUALIZATIONS
How VW Visualizes Connected Car Data
VW Data LabsConnected Car program
Post-sales big data visualization
Customer loyalty & retention
IoT analysis & prediction of customer needs
Sense and RespondUse Splunk Alerts and Custom Alert Actions to trigger & automate workflows
● Allows packaged integration with third-party applications
● Simple admin/user configuration● Developers can build, package, and
publish alert actions within an app● Growing list of integrations available
24
Splunk’s IoT and Industrial Partner Ecosystem
25
SDKs UI
Ingest and Platforms
IoT and ICS SecurityAdvanced Analytics and ML Custom User Interfaces
Services and Delivery
Customer Success
Use Cases
27
Monitoring,
Diagnostics
Security, Safety
& Compliance
Preventative Maintenance
28
Zebra Technologies CorporationFounded in 1969, Zebra offers customers a complete end-to-end solution – from mobile computers and scanners to specialty printers, RFID, software and services – for identifying, tracking and managing critical assets, people and transactions.
Headquarters: Lincolnshire, ILOffices: 122 offices across 81 countriesEmployees: 7000+ globally
Market Capitalization: $5,640M1
Revenue: $2,275M 1
Profit (EBITDA): $439M 1
Mobile ComputingPrintersData Capture (barcode scanners)
RFIDLocation solutionsWireless LAN
About
Global presence
Financials
Products
2,000+ NFL players tagged and tracked since 2014, generating more that 68+ billion bytes of player position data
1 Zebra corporate fact sheet, summer 2015
29
Challenge for Zebra Printer DivisionLimited data analysis capabilities . . .
• Data not being collected for long-term use• Reports not being generated• Data not shareable across departments• Changes to traditional database took a long time• Minimal analytics capabilities (e.g., Excel)• Unable to scale to increasing data volumes
Reactive mindset: Teams were reacting to issues after they occurred
Little continuous improvement: Company was unable to leverage data to improve• Product design• Manufacturing• Repair and service
Resulted in . . .
Goal is to design a system that:• Captures complete device and test history• Is flexible and can be easily modified• Is capable of robust analytics, trending, & alerting capabilities• Can easily share data to enable better business decisions
30
Solution developed by Zebra
Test HistoryHistory of the test performed by
the test utility on the device
Test DataAny data deemed necessary to
log and store
Device (Printer) DataRaw data output of device
(4 data formats)
1 manufacturing center8 service centers3 reconfig centers30+ applications600+ computers globally Allows for real-time view of production data to address low yields
and drive quality improvements
Communicates product performance over time, leveraging statistical methods to baseline product performance
Anomaly detection analytics utilized to capture known defect patterns that prevent product shipment
Dashboard
Reports
Analytics
31
Value realized by Zebra
Quick access to test history and device data allows for enhanced customer experience
Faster customer resolution
Enhanced product development
Reduced return rate
Improved business decisions
Reduced cost
Data being captured early in development drives improvements in design
Anomaly detection analytics used to capture known defect patterns that prevent product shipment
Data being easily shared across Quality, Product, Technical Support, and Services enables data-driven decision making process
Quantified cost reduction of $90,000 per year through media reduction alone
1
2
3
4
5
Planes, Trains and Automobiles(and Coke, Zombies, Floods, Buildings, Crops and Medicine)
Data-driven RefreshmentAggregate machine data from
Coca-Cola freestyle® machines
Insights into customer interactions and decisions
Reduced Downtime andIncreased Consumer Satisfaction
Vending machine performance and diagnostics
More people buy Coke just before The Walking
Dead
35
How Gatwick Airport Ensures Better Passenger Experience With Splunk Cloud
On-time efficiency & dramatic queue reduction with 925 flights per day
Real-time, predictive airfield analytics - mobile app & CEO’s apple watch
Data from airport gates, board pass scans, x-ray, travel, passenger flow
Saving The US Rail Industry A Billion Dollars And 250 Million Acres Of Trees in CO2
Train sensor dataFuel savingsBetter trained drivers
24 Hour DB Hackathon
Highlight defect impact & rail construction issues
Predictive maintenance & reduced disruption
Transport, infrastructure, environment & journey data
How Deutsche Bahn Analyzed Tracks in 24 Hours
Top ten types of notifications about issues that have occurred in the transportation infrastructure
What kinds of defects occur and quantify how big the deviations are
Correlated with materials of the track sleepers: concrete (“Beton”) vs. wood (“Holz”)
Track deviations with indication notifications. Blue bars denote “no known issues”
Width of Sankey bar shows amount of track deviations between different destinationsFulda and Frankfurt has high track deviations (it is the widest bar)
Indicates the need for upcoming repair, maintenance and possible renewal
Water Level Sensor
Water Level Sensor Data in
Monitors
Real-time
Data
In Advance of Future Impending Flood
Situations
Alert Mobile Devices
Robot Analytics to Reduce Costs in the Supply Chain
4% Increased Throughput per Distribution Center
Aggregate machine data from robots
Failure pattern detection and reporting
Preventative maintenance scheduling
46
Remote Freight TrainMonitoring
Energy Efficiency Calculations
Driving Strategy Recommendations
Over $1 Billion
Saved
Sensor Data Delivering Millions of Dollars in Energy Savings
+Content browsed, purchased and
watched. All tracked by time and MAC address
Customerbehavior analytics
Customerprofile and MAC address / device
assignments
Understanding Customer Behavior
IoT WORKLOADS
Blurring the Lines Between Digital & Physical
Security Ops
Center
Business Ops
Center
IT Ops Center
CLOUD WORKLOADS ENTERPRISE IT WORKLOADS
ADVANCED ANALYTICSRAPID SOLUTION DEVELOPMENT
DATA INGEST AT SCALE
Thank You