Embed Size (px)
Citation preview
Copyright © 2015 Splunk Inc.
The Splunk App for Stream
Kai-‐Ping Seidenschnur Sr. Sales Engineer
Agenda • Splunk Enterprise • IntroducDon to Wire Data • The Splunk App for Stream Overview • What’s New • Important Features • Architecture and Deployment • Demo • Customer Success Examples • FAQ and Summary
2
Industry Leading PlaPorm For Machine Data Machine Data: Any Loca0on, Type, Volume
Online Services Web
Services
Servers Security GPS
LocaDon
Storage Desktops
Networks
Packaged ApplicaDons
Custom ApplicaDons Messaging
Telecoms Online
Shopping Cart
Web Clickstreams
Databases
Energy Meters
Call Detail Records
Smartphones and Devices
RFID
On-‐ Premises
Private Cloud
Public Cloud
Pla9orm Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Ques0on
Developer Pla9orm
Report and analyze
Custom dashboards
Monitor and alert
Ad hoc search
3
Industry Leading PlaPorm For Machine Data Machine Data: Any Loca0on, Type, Volume
Online Services Web
Services
Servers Security GPS
LocaDon
Storage Desktops
Networks
Packaged ApplicaDons
Custom ApplicaDons Messaging
Telecoms Online
Shopping Cart
Web Clickstreams
Databases
Energy Meters
Call Detail Records
Smartphones and Devices
RFID
On-‐ Premises
Private Cloud
Public Cloud
Pla9orm Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Ques0on
Developer Pla9orm
Report and analyze
Custom dashboards
Monitor and alert
Ad hoc search
Any amount, any locaDon, any source
Schema-‐on-‐the-‐fly
Universal indexing
No back-‐end RDBMS
No need to filter data
4
Copyright © 2015 Splunk Inc.
IntroducDon to Wire Data
What’s Wire Data?
" Machine data " Poly-‐structured data " AuthoritaDve record of real-‐Dme and historical communicaDon between machines and applicaDons
6
tcpdump -‐qns 0 -‐A -‐r blah.pcap 20:57:47.368107 IP 205.188.159.57.25 > 67.23.28.65.42385: tcp 480 0x0000: 4500 0214 834c 4000 3306 f649 cdbc 9f39 [email protected] 0x0010: 4317 1c41 0019 a591 50fe 18ca 9da0 4681 C..A....P.....F. 0x0020: 8018 05a8 848f 0000 0101 080a ffd4 9bb0 ................ 0x0030: 2e43 6bb9 3232 302d 726c 792d 6461 3033 .Ck.220-‐rly-‐da03 0x0040: 2e6d 782e 616f 6c2e 636f 6d20 4553 4d54 .mx.aol.com.ESMT 0x0050: 5020 6d61 696c 5f72 656c 6179 5f69 6e2d P.mail_relay_in-‐ 0x0060: 6461 3033 2e34 3b20 5468 752c 2030 3920 da03.4;.Thu,.09. 0x0070: 4a75 6c20 3230 3039 2031 363a 3537 3a34 Jul.2009.16:57:4 0x0080: 3720 2d30 3430 300d 0a32 3230 2d41 6d65 7.-‐0400..220-‐Ame 0x0090: 7269 6361 204f 6e6c 696e 6520 2841 4f4c rica.Online.(AOL 0x00a0: 2920 616e 6420 6974 7320 6166 6669 6c69 ).and.its.affili 0x00b0: 6174 6564 2063 6f6d 7061 6e69 6573 2064 ated.companies.d
Ad hoc Analysis on Wire Data Is Challenging
Volume, velocity and variety
make it difficult to collect, explore, analyze and visualize wire data.
Distributed infrastructures
introduce challenges in accessing wire data from public and hybrid clouds.
Complex network environments
make installaDon and management of probes and appliances laborious.
7
8
Why Wire Data?
Deep Insights Across Use Cases IT, security and business data transmit over the wire
Non-‐Intrusive and Passive No impact to workloads
No need for instrumentaDon and tagging of applicaDons
Holis0c and Comprehensive Real-‐Dme communicaDon across various protocols
Correlate with logs, events and metrics for comprehensive analyDcs
The Splunk App for Stream Overview
See Everything With the Splunk App for Stream
Enables real-‐0me insights into private, public and hybrid
cloud infrastructures
Delivers rapid deployment, easy
scale out and efficient wire data capture
Capture and analyze cri0cal events not
found in logs or with other collec0on
methods
1 2 3
Enhance Opera0onal Intelligence With Wire Data Capture
Examples of What’s Available From the Wire
11
Performance Metrics
Round Trip Time
Client Request Time
Server Reply Time
Server Send Time
Total Time Taken
Base HTML Load Time
Page Content Load Time
Total Page Load Time
Applica0on Data
POST Content
AJAX Data
SecDon
Sub-‐SecDon
Page Title
Session Cookie
Proxied IP Address
Error Message
Business Data
Product ID
Customer ID
Shopping Cart ID
Cart Items
Cart Values
Discounts
Order ID
Abandoned?
12
Enable New OperaDonal Insights • Add informaDon about applicaDon, infrastructure, security and business acDvity, without needing instrumentaDon
• Support new and extends exisDng Splunk use cases across IT, security and the business with wire data capture
Enhanced Opera0onal Intelligence
Efficient, Cloud-‐Ready Wire Data Collec0on
Fast Time to Value
• Gain visibility into any public, private or hybrid cloud infrastructures with a sopware soluDon
• Control data collecDon volumes with fine-‐grained protocol and aqribute filtering
• Deploy quickly from interface-‐driven install
• Enable rapid incident response
• Easily scale out with centralized management
Beqer Insights for IT OperaDons
• Get real-‐Dme granular insights to reduce MTTR without costly appliances
• Analyze all applicaDons and user behavior, measure applicaDon response Dmes and trace transacDon paths
• IdenDfy infrastructure performance issues, capacity constraints, changes and establish baselines
Value
ApplicaDon logs, infrastructure (storage, network, server) logs, performance metrics, events
13
SQL queries, DNS records, IP conversaDons, transacDon traces, ICA latency, response Dmes
+
Contextual Data
Wire Data
Beqer Insights for App Management
Protocol conversaDons on database performance, DNS lookups, client data, business transacDon paths… Measure applicaDon response
Dmes, deeper insights for root-‐cause diagnosDcs, trace transacDons paths, establish baselines, etc.
Enriched View
ApplicaDon logs, monitoring data, metrics, events
14
+ Contextual
Data
Wire Data
Beqer Insights for Security
• Real-‐Dme DPI with analyDcs enables easier forensics analyses and quicker incident response
• Analyze user and applicaDons behavior
• Respond Dmely to threats with cost-‐efficient real-‐Dme header and payload field extracDon
• Baseline network traffic and understand anomalies associated with APTs and insider threats
• Quick install at endpoints, on-‐premises and cloud infrastructures without expensive appliances
Value
+ Contextual
Data Firewall logs, applicaDon logs, IDS logs, network logs, perf. metrics, events
15
User and applicaDon traffic, protocol idenDficaDon (TCP, DNS, HTTP, etc.), protocol headers & payload extracDon, SSL decrypDon
Wire Data
Beqer Insights for Digital MarkeDng
Browser-‐level customer interacDons
Customer Experience – analyze website and applicaDon boqlenecks to improve customer experience and online revenues Customer Support (online, call center) – faster root-‐cause analysis and resoluDon of customer issues with website or apps
Enriched View
Website log acDvity, clickstream data, metrics
16
+ Contextual
Data
Wire Data
What’s New
Distributed Forwarder Management " More deployment flexibility " Per-‐forwarder protocol control
to increases management efficiency
" Tailored data collecDon by assigning different sets of protocols to groups of forwarders
18
TNS MySQL
HTTP DNS TCP
SIP Diameter UDP
Protocol SelecDon,
ConfiguraDon & DistribuDon
Important Features
20
Custom Content ExtracDon Enables Efficient Real-‐Time Insights
• Easily and selecDvely analyze web traffic for security risks • IdenDfy data exfiltraDon, including PII or exposed assets • Prevent data loss, perform forensics and reduce
troubleshooDng Dme
Improved Security Posture
Efficient Real-‐Time Business Analyses
Efficient IT Ops and Applica0ons Visibility
• Real-‐Dme granular insights into key business indicators from web traffic
• SelecDve on-‐the-‐fly visibility into shopping carts, user interacDons, etc.
• Monitor web services performance on-‐the-‐fly for quick troubleshooDng and performance analysis
• Enable real-‐Dme custom protocol monitoring
21
Stream Stats Dashboard Enables Granular Analysis of Traffic and Indexing Volume
• ProacDvely plan Stream deployment with per-‐protocol visibility into applicaDons traffic bandwidth and Splunk indexing stats
• EsDmate per-‐protocol Splunk indexing volume, incoming, outgoing or total traffic bandwidth
Supported Protocols and PlaPorms • UDP
• TCP • HTTP
• IMAP • MySQL (login/cmd/
query)
• Oracle (TNS) • PostgreSQL
• Sybase/SQL Server (TDS)
• FTP • SMB • NFS
• POP3 • SMTP
• LDAP/AD • SIP • XMPP
• AMQP • MAPI
• IRC
Supports Windows 7 (64-‐bit), Windows 2008 R2 (64 bit), Linux (32-‐bit/64-‐bit) and Mac OSX (64-‐bit)
• DNS • DHCP • RADIUS
• Diameter • BitTorrent
• SMPP
22
Improved performance requiring less compute/memory power!
Architecture and Deployment
Stream Forwarder Architecture
Protocol Decoder
(Deep Packet Inspec0on)
Events Decryp0on Request/Response
Network Interface (eth1)
Standard Out (To Splunk Forwarder) Packets
Streams
Request/Response
Request/Response
Protocol Decoder
(Deep Packet Inspec0on)
Events Decryp0on Standard Out (To Splunk Forwarder)
Protocol Decoder
(Deep Packet Inspec0on)
Events Decryp0on Standard Out (To Splunk Forwarder)
Network Interface (ethN)
Packets
…
Threads
24
Architecture: Dedicated Server
25
End Users TAP or SPAN
Firewall
Search Head Linux Forwarder Splunk_TA_Stream
Servers Internet
Splunk Indexers
Architecture: Run on Servers
26
Splunk Indexers
Search Head
Physical or Virtual Servers Universal Forwarder Splunk_TA_stream
Physical Datacenter, Public or Private Cloud
End Users
Firewall Internet
Demo
Customer Success Stories
Cross-‐Der Visibility Helps Break the Silos
Kris Laxdal, IT Manager & Security Analyst
“You cannot show up with tradi0onal packet captures tool in the boardroom. Stream and Splunk help us understand issues at the high level and if exec team wants to see the details we can drill down easily. That is what's great about Stream!”
IT Opera0ons • High level view with contextual drill-‐down ability
• Easy access and visibility into producDon MySQL environment helps app developers troubleshoot issues and roll out releases quicker
• Improved collaboraDon between teams: IT operaDons, QA (pre-‐producDon tesDng),security and development
• Improved customer response Dmes due to real-‐Dme visibility into app issues
Security • CorrelaDon against indicators of compromise helps invesDgate and
miDgate APTs, potenDal data exfiltraDon & other risks
Key Customer Benefits
• Granular applicaDon and network visibility drives easy remediaDon
• ProacDve applicaDons and network traffic monitoring enables beqer capacity reporDng and planning
• Powerful analyDcal engine enables data analyses by novice users
• Quick host-‐based deployment at criDcal network segments
– Ability to observe both client and server traffic
Key Customer Benefits
ApplicaDons Visibility for Capacity Planning Helps with Datacenter MigraDon
AVP of Networks and Communica0ons, Large Na0onal Bank
“I enjoyed using the Splunk App for Stream as it's giving us a bunch of different perspecDves on our traffic and beqer granularity compared to some of the other tools we used.”
ApplicaDons Visibility Drives Beqer Digital Asset Management
Systems Engineer, Major Media Company
“With Splunk and Stream, we have this rich data pla9orm that is bridging all the different data silos. Our MTTR went from days to minutes while the granularity and insight improved. We went from having very liqle visibility into operaDonal and security issues to full insight.”
Key Customer Benefits • IT Opera0ons: improved operaDonal insight into digital asset management and streamlined lengthy processes
• DevOps/app delivery: faster app releases due to visibility into app performance
– Real-‐Dme insight into database queries and latencies
– Cross-‐correlaDon with system-‐level performance and user access
• Security: visibility into user behavior throughout enDre asset management system helps protect digital assets
Real-‐Time Insights into Database AcDvity
32
IT Infrastructure Manager, Leading Taiwanese Telco
“With Stream, we are are able to roll out applicaDons faster and perform quicker invesDgaDons into operaDonal issues. The Splunk plaPorm is a single interface to all the data for our IT ops and security teams.”
Key Customer Benefits • Gain deep operaDonal Oracle database access monitoring • Audit assistance: who, when, how performed database access
• Client-‐side visibility
• IdenDficaDon of abnormal connecDons
• Resolve issues faster with cross-‐correlaDon of applicaDon logs with database-‐access monitoring
• Get lightweight monitoring without impact on server performance
Wire Data Intelligence Improves Security
Security Analyst, Payment Processing Company
“The thing that makes Stream beqer than any other packet analysis soluDon out there is the staDsDcal analysis from Splunk Enterprise. You can apply it freely to all of the wire data, which enables me to analyze this data in ways not possible before. This visibility help us prevents external infiltraDon and avoid malicious aqacks.”
Key Customer Benefits • Real-‐Dme security intelligence to prevent aqacks and infiltraDons
• Baselining, trending and applying analyDcs to detect anomalies in traffic (mySQL, postgres, etc.)
• Centralized management of all wire data results in operaDonal cost savings
• Efficient monitoring of user authenDcaDons for audit and security
• Non-‐intrusive and easy monitoring of server communicaDon
• Flexible and easy integraDon with Splunk security dashboards
Streaming AnalyDcs Helps Speed Up Forensics InvesDgaDon
34
Key Customer Benefits • 90% reducDon in incident triage and invesDgaDon Dme • Deeper, quicker and easier understanding of traffic and user acDvity
• Immediate insights and improved data collecDon
– EliminaDon of moving pcap files around between several tools
• Flexible and easy deployment on key network locaDons Security Engineer,
Financial Services Ins0tu0on
“The biggest value of Stream is how fast we can resolve and close security cases. Before Stream, I had to collect data from mulDple systems and it would take me an hour. With Stream, informaDon is already there and I can get answers within 5 minutes. “
FAQ and Summary
FAQ • Yes. The app enables capture of only the relevant wire data for analyDcs, through filters and aggregaDon rules
• Select or deselect protocols and associated aqributes with fine-‐grained precision within the app interface
Can I limit the amount of data collected with Stream?
• Data volume can vary based upon the number of selected protocols, aqributes and the amount of network traffic. UDlize Stream Stats to understand the licensing impact
How can I es0mate my indexing volume?
• The Stream Examples App contains searches, examples and instrucDons, enabling use cases such as network security scenarios, funnel analysis, shopping cart revenue, SIP conversaDons, and applicaDon and database latencies
How can I explore the data collected with Stream?
36
Enables real-‐0me insights into private, public and hybrid
cloud infrastructures
Delivers rapid deployment, easy
scale out and efficient wire data capture
Capture and analyze cri0cal events not
found in logs or with other collec0on
methods
1 2 3
37
See Everything with Splunk App for Stream Enhance Opera0onal Intelligence With Wire Data Capture
Thank You
