50
Copyright © 2016 Splun Inc. Log I am your father: search your data, you know it to be true James Hodge, Principal Product Manager

Log I am your father

Embed Size (px)

Citation preview

Page 1: Log I am your father

Copyright © 2016 Splunk Inc.

Log I am your father:search your data, you know it to be true

James Hodge,Principal Product Manager

Page 2: Log I am your father

Spelunking:

Splunking:

to explore underground caves

to explore machine data

Page 3: Log I am your father

Explore your IoT data

Page 4: Log I am your father

4

The data we know and use

The available data we don’t know or use

Page 5: Log I am your father

5

Nice, structured, tidy data

Page 6: Log I am your father

It began with logs…

Page 7: Log I am your father

7

Machine data time series, in motion, unstructured

Page 8: Log I am your father

It can be big data…

Page 9: Log I am your father

…and it is slightly crazy “data”…

Page 10: Log I am your father

10

… and it is messy and lazy

Page 11: Log I am your father

SECURITYIT OPERATIONS

CUSTOMER EXPERIENCERISK & COMPLIANCE

PRODUCT PERFORMANCEIOT

But it has many uses

and lots of untapped value

Page 12: Log I am your father

There’s gold in that there data

Page 13: Log I am your father

13

Make machine data accessible,usable and valuable to everyone.

13

Page 14: Log I am your father

[2014-09-04-14.45.54.608000] proc_source="B24A", tmst_target="2013-09-04-14.45.54.724000", serv_id="ISS", proc_input="MAST", proc_target="B24H", interface_acq="BNET_1", interface_iss="02008", cod_msg="XJYZ", oper_rrn="XJYZ", card_id="52xxxxxxxxxxx", oper_amount="000000000050", oper_currency="978", oper_country="380", term_id="0059XXXX", circuito="", sett_merc="4722", bin_acq="XXXX", id_merc="32xxxxxxxxxx", prcode="XYZ", action_code="XXX", ......auth_rout_id="HISO_AUTH", msg_subst="", ndg="00000xxxxxxxx", station_acq="STA-BNET-MI1", acceptor="A COOL SHOP", tmst_ins="2013-09-04-14.48.56.277466", ...

Mastercard Circuit

Client IDMerchant name

AmountCard ID (masked)

Merchant category IDMerchant ID

There’s gold in that there data

Page 15: Log I am your father

Turning Machine Data Into Business Value

15

Platform for Machine Data

ApplicationDelivery

ITOperations

Security,Complianceand Fraud

BusinessAnalytics Industrial

Data andInternet of

Things

Security,Complianceand Fraud

BusinessAnalytics Industrial

Data andInternet of

Things

Page 16: Log I am your father

HA/DR Admin Data Security Apps SDKs/APIsScale

CollectData

IndexData

Enrich Data

Search & Explore

Analyze& Predict

Report &Visualize

Alert & Action

16

Fully Integrated Enterprise Platform

Page 17: Log I am your father

Technology Telecommunications Travel and Leisure

Education

Healthcare

Energy and Utilities

Manufacturing

Financial Services and Insurance

Media

Proven at 11,000+ Customers in 100 CountriesMore Than 80 of the Fortune 100

Retail

Cloud and Online Services

Government

Page 18: Log I am your father

Splunk Capabilities for IoT

Data Ingest at Scale

PartnerEcosystem

Advanced Analytics and Visualization

Sense and Respond

Page 19: Log I am your father

IoT and Industrial Machine Data

DevelopVisualize PredictAlertSearch

Engineers Data Analysts

Security Analysts

Business Users

Native InputsTCP, UDP, Logs, Scripts, Wire, Mobile

SDKs and APIsJava, JS, C#, Python, Ruby, PHP

Modular InputsMQTT, AMQP, COAP, REST, JMS

HTTP Event CollectorToken Authenticated Events

Real-time

Technology PartnershipsKepware, ThingWorx, Cisco, Palo Alto

MaintenanceInfo

AssetInfo

DataStores

External Lookups/Enrichment

OT

Industrial Assets

IT

Consumer and Mobile Devices

Page 20: Log I am your father

HTTP Event CollectorSupports DevOps and IoT data analysis needs at scale

20

• Standard API and logging libraries send events directly to Splunk• Libraries integrated with popular platforms and services

Scales to Millions of Events/Second

EVENT COLLECTOR API

Page 21: Log I am your father

Advanced Analytics

Splunk ML Extensions• New/Enhanced Commands: Data Sampling,

Fit, Apply, Summary, Predict• Access to Python Data Science Library• Model storage and export to production

Splunk ML App• Step-wise guidance to create, test and deploy

custom ML models • Purpose-built visualizations• Sample data and best practices

Optimizing enterprise operations with predictive ML analytics

Page 22: Log I am your father

ADVANCED VISUALIZATIONS

Page 23: Log I am your father

How VW Visualizes Connected Car Data

VW Data LabsConnected Car program

Post-sales big data visualization

Customer loyalty & retention

IoT analysis & prediction of customer needs

Page 24: Log I am your father

Sense and RespondUse Splunk Alerts and Custom Alert Actions to trigger & automate workflows

● Allows packaged integration with third-party applications

● Simple admin/user configuration● Developers can build, package, and

publish alert actions within an app● Growing list of integrations available

24

Page 25: Log I am your father

Splunk’s IoT and Industrial Partner Ecosystem

25

SDKs UI

Ingest and Platforms

IoT and ICS SecurityAdvanced Analytics and ML Custom User Interfaces

Services and Delivery

Page 26: Log I am your father

Customer Success

Page 27: Log I am your father

Use Cases

27

Monitoring,

Diagnostics

Security, Safety

& Compliance

Preventative Maintenance

Page 28: Log I am your father

28

Zebra Technologies CorporationFounded in 1969, Zebra offers customers a complete end-to-end solution – from mobile computers and scanners to specialty printers, RFID, software and services – for identifying, tracking and managing critical assets, people and transactions.

Headquarters: Lincolnshire, ILOffices: 122 offices across 81 countriesEmployees: 7000+ globally

Market Capitalization: $5,640M1

Revenue: $2,275M 1

Profit (EBITDA): $439M 1

Mobile ComputingPrintersData Capture (barcode scanners)

RFIDLocation solutionsWireless LAN

About

Global presence

Financials

Products

2,000+ NFL players tagged and tracked since 2014, generating more that 68+ billion bytes of player position data

1 Zebra corporate fact sheet, summer 2015

Page 29: Log I am your father

29

Challenge for Zebra Printer DivisionLimited data analysis capabilities . . .

• Data not being collected for long-term use• Reports not being generated• Data not shareable across departments• Changes to traditional database took a long time• Minimal analytics capabilities (e.g., Excel)• Unable to scale to increasing data volumes

Reactive mindset: Teams were reacting to issues after they occurred

Little continuous improvement: Company was unable to leverage data to improve• Product design• Manufacturing• Repair and service

Resulted in . . .

Goal is to design a system that:• Captures complete device and test history• Is flexible and can be easily modified• Is capable of robust analytics, trending, & alerting capabilities• Can easily share data to enable better business decisions

Page 30: Log I am your father

30

Solution developed by Zebra

Test HistoryHistory of the test performed by

the test utility on the device

Test DataAny data deemed necessary to

log and store

Device (Printer) DataRaw data output of device

(4 data formats)

1 manufacturing center8 service centers3 reconfig centers30+ applications600+ computers globally Allows for real-time view of production data to address low yields

and drive quality improvements

Communicates product performance over time, leveraging statistical methods to baseline product performance

Anomaly detection analytics utilized to capture known defect patterns that prevent product shipment

Dashboard

Reports

Analytics

Page 31: Log I am your father

31

Value realized by Zebra

Quick access to test history and device data allows for enhanced customer experience

Faster customer resolution

Enhanced product development

Reduced return rate

Improved business decisions

Reduced cost

Data being captured early in development drives improvements in design

Anomaly detection analytics used to capture known defect patterns that prevent product shipment

Data being easily shared across Quality, Product, Technical Support, and Services enables data-driven decision making process

Quantified cost reduction of $90,000 per year through media reduction alone

1

2

3

4

5

Page 32: Log I am your father

Planes, Trains and Automobiles(and Coke, Zombies, Floods, Buildings, Crops and Medicine)

Page 33: Log I am your father

Data-driven RefreshmentAggregate machine data from

Coca-Cola freestyle® machines

Insights into customer interactions and decisions

Reduced Downtime andIncreased Consumer Satisfaction

Vending machine performance and diagnostics

Page 34: Log I am your father

More people buy Coke just before The Walking

Dead

Page 35: Log I am your father

35

How Gatwick Airport Ensures Better Passenger Experience With Splunk Cloud

On-time efficiency & dramatic queue reduction with 925 flights per day

Real-time, predictive airfield analytics - mobile app & CEO’s apple watch

Data from airport gates, board pass scans, x-ray, travel, passenger flow

Page 36: Log I am your father
Page 37: Log I am your father
Page 38: Log I am your father

Saving The US Rail Industry A Billion Dollars And 250 Million Acres Of Trees in CO2

Train sensor dataFuel savingsBetter trained drivers

Page 39: Log I am your father

24 Hour DB Hackathon

Highlight defect impact & rail construction issues

Predictive maintenance & reduced disruption

Transport, infrastructure, environment & journey data

How Deutsche Bahn Analyzed Tracks in 24 Hours

Page 40: Log I am your father
Page 41: Log I am your father

Top ten types of notifications about issues that have occurred in the transportation infrastructure

Page 42: Log I am your father

What kinds of defects occur and quantify how big the deviations are

Correlated with materials of the track sleepers: concrete (“Beton”) vs. wood (“Holz”)

Track deviations with indication notifications. Blue bars denote “no known issues”

Page 43: Log I am your father

Width of Sankey bar shows amount of track deviations between different destinationsFulda and Frankfurt has high track deviations (it is the widest bar)

Indicates the need for upcoming repair, maintenance and possible renewal

Page 44: Log I am your father

Water Level Sensor

Water Level Sensor Data in

Monitors

Real-time

Data

In Advance of Future Impending Flood

Situations

Alert Mobile Devices

Page 45: Log I am your father

Robot Analytics to Reduce Costs in the Supply Chain

4% Increased Throughput per Distribution Center

Aggregate machine data from robots

Failure pattern detection and reporting

Preventative maintenance scheduling

Page 46: Log I am your father

46

Remote Freight TrainMonitoring

Energy Efficiency Calculations

Driving Strategy Recommendations

Over $1 Billion

Saved

Page 47: Log I am your father

Sensor Data Delivering Millions of Dollars in Energy Savings

Page 48: Log I am your father

+Content browsed, purchased and

watched. All tracked by time and MAC address

Customerbehavior analytics

Customerprofile and MAC address / device

assignments

Understanding Customer Behavior

Page 49: Log I am your father

IoT WORKLOADS

Blurring the Lines Between Digital & Physical

Security Ops

Center

Business Ops

Center

IT Ops Center

CLOUD WORKLOADS ENTERPRISE IT WORKLOADS

ADVANCED ANALYTICSRAPID SOLUTION DEVELOPMENT

DATA INGEST AT SCALE

Page 50: Log I am your father

Thank You