Upload
greg-hanchin
View
371
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Splunk for VMWARE Application
Citation preview
Copyright © 2013 Splunk, Inc.
Splunk Overview & Splunk for VMware
Presenter NamePresenter Title
Agenda
What Is Splunk?
Splunk Value in Virtualized Datacenters
Splunk App for VMware Intro
What Customers Are Saying
Demo/Screenshots
2
Make machine data accessible, usable and valuable to everyone.
3
Customer Facing Data
Outside the Datacenter
ApplicationsWeb logsLog4J, JMS, JMX.NET eventsCode and scripts
NetworkingConfigurationssyslogSNMPnetflow
DatabasesConfigurationsAudit/query logsTablesSchemas
Virtualization & Cloud
HypervisorGuest OS, AppsCloud
Linux/UnixConfigurationssyslogFile systemps, iostat, top
WindowsRegistryEvent logsFile systemsysinternals
Logfiles Configs Messages Traps Alerts
Metrics Scripts TicketsChanges
Click-stream dataShopping cart dataOnline transaction data
Manufacturing, logistics…CDRs & IPDRsPower consumptionRFID dataGPS data
Splunk Collects and Indexes Any Machine Data
4
Splunk Collects and Indexes Any Machine Data
Customer Facing Data
Outside the Datacenter
ApplicationsWeb logsLog4J, JMS, JMX.NET eventsCode and scripts
NetworkingConfigurationssyslogSNMPnetflow
DatabasesConfigurationsAudit/query logsTablesSchemas
Virtualization & Cloud
HypervisorGuest OS, AppsCloud
Linux/UnixConfigurationssyslogFile systemps, iostat, top
WindowsRegistryEvent logsFile systemsysinternals
Logfiles Configs Messages Traps Alerts
Metrics Scripts TicketsChanges
Click-stream dataShopping cart dataOnline transaction data
Manufacturing, logistics…CDRs & IPDRsPower consumptionRFID dataGPS data
• Any amount, any location, any sourceNo upfront schemaNo custom connectorsNo RDBMSNo need to filter/forward
5
Splunk Turns Machine Data into Operational Intelligence
Search and Investigate
Proactive Monitoring
Operational Visibility
Real-time Business Insights
Proactive
ReactiveMachine Data
Universe
6
A Single Solution for Operational Intelligence
Real-time Visibility• Live dashboards• Event correlation• Monitoring and alerting• Performance issues• Transaction levels• SLA tracking
Three Primary CapabilitiesHistorical Analytics• Baseline and thresholds• Trending• Operational insights• Historical patterns• Compliance reports
Single Data Store Single UI Across Use Cases
Search / Navigation
• Data drilldown• “Needle in a haystack”• Root cause analysis /
troubleshooting• Incident investigations
7
Splunk Delivers Value Across the Enterprise
8
Web Intelligence
ApplicationMgmt
Security&
Compliance
IT Operations
Business Analytics
9
Education
Healthcare
Technology
Energy and Utilities
Manufacturing
Telecommunications
Cloud and Online Services
Government
Retail
Financial Services and Insurance
Media
Travel and Leisure
Proven at 5,200+ Customers in 90+ CountriesOver Half the Fortune 100
Splunk In Virtualized Datacenters
The Virtualized Datacenter Problem
End user devices
End user devices
End user devices
End user devices
Virtualized Servers
Storage
Networking
Connection Broker
Active Directory
/LDAP Virtual
Machines
Networking
Security
Shared Resources = Shared Problems
No ability to link user or application level issues with hardware or
hypervisor problems
No way to go back in time and recreate environment state
How Does Splunk Help?
End user devices
End user devices
End user devices
End user devices
Virtualized Servers
Storage
Networking
Connection Broker
Active Directory
/LDAP Virtual
Machines
Networking
Security
Talks to every technology in your stack
Correlates data across the different tiers – find causal links
Built for Big Data - Visualize, analyze, trend all your data at large scale
Splunk App for VMware
Splunk App for VMwareComplete Operational Visibility Into VMware Environments
• Collects and persists logs and performance metrics directly from ESX/i hosts , avoiding the VC bottleneck
• Integrates data with VC topology information
• Collects and persists tasks & events from VC to get complete visibility into actions
• Pre-built dashboards and views to showcase initial use cases
APP
OSVM
VMware vSphere
VM
Physical Layer
ServersStorage Network Devices
APP
OS
VMware vCenter Server(VC)
Why Use the Splunk App for VMware?
Harness virtualization layer data– Special forwarders for ESX/ESXi and
VC– Comprehensive data collection at
scale– Initial visualizations to navigate the
data
+The Power of Splunk– Mash up data with all other
technology tiers– Correlate, analyze, visualize– Monitor and alert in real time– Adaptive reporting and dash
boarding
Unlock the Value of your Virtualization Data
ITOps.
Security & Compliance
Web Intelligence
App Dev &
App Mgmt.
Developer Platform (REST API, SDKs)
Business Analytics
Industrial Internet and Internet of
Things
Small Data. Big Data. Huge Data.
How It Works
i
i
Splunk App for VMware
>
Splunk Add-on for vCenter
>
Splunk UF/LF
vCenter server
Provides: Dashboards, Views,
Field Extractions
From ESX/i Host:Host Inventory / Hierarchy, Time, Performance*, and Log Data
From VC:VC Logs, Time Data
Splunk virtual FA for VMware
>
VMwarePerl SDK
From VC:VC Inventory / Hierarchy,
Tasks, and Events Data
* Performance data at 20 s granularity
Data Volume
18
1 Forwarder Appliance is required per 20-30 physical ESX hosts –we don’t charge for the app or the appliancesEach host typically generates 400MB-1 GB of data per day– Includes logs, tasks & events, inventory & hierarchy , “aggregated” metrics
data– “Per instance” metrics collection is an additional ~1 GB – this is turned off by
default– Configuration choices for data types available
What Differentiates The Splunk App for VMware
19
End to End Visibility
20
Splunk used to correlate the business data (Users , Usage) with the IT/Infrastructure data Understand resource/usage and cost per customer Monitor the entire environment from server, storage, network, hypervisors, custom cloud back-end for possible SLA issues, trouble spots and more
We have deep visibility and correlation across all tiers of our cloud infrastructure – giving us not only ongoing monitoring of key datacenter statistics, but also giving us business visibility into customer experience and usage ”
“
Elad Gotfrid,Manager of IT
One Splunk – Many Uses
21
Using Splunk for VMware gets us our data in one place, for many uses: capacity planning, event monitoring, performance analysis, security monitoring and more..
“
”Helps retain a definitive record of what happened in our environment
Analyze and trend performance as well as user activities very easily
Useful for both operational monitoring, capacity usage, performance metrics and for security monitoring
Peter ColeTechnical Lead, ITS Operations
22
Detailed History For Analysis & Troubleshooting
I love that I can track virtual machines in my environment as they move from host to host.I can now identify the root cause of issues or errors.
Matthew CluverNetwork Operations Analyst
“
” Splunk already used for operating system and applications event monitoring & analysisFor the first time, they have insight into granular virtualization layer data – helps solve problems immediately
Benefits of the Splunk Approach1. Collects all the data without interfering with the normal operation of VC – Data how
you want it, when you want it
“We really wanted to get our business units off of VC for reporting, this is a great way to serve up the data they need”
-Melbourne IT
2. Persists the data at scale, hard to do with any other tool
“Its comforting to know the data is in Splunk for me to go back and resolve any issue” -Nancy Kafer, Homesteader’s Life Insurance
3. Enables correlation with application data, network data, storage data “Splunk helps me correlate application performance issues with hypervisor level sharing
issues”. -Major Mobile Technology Provider
Why Splunk Over Everyone Else?You don’t know what data you will need till you need it– Every other tool goes through VC - only has access to 5 min summaries of data, Splunk collects
20 second granularity – Most don’t even incorporate log data
24
“Splunk already paid for itself, it helped us find an issue with SCSI resets on storage that we wouldn’t have found otherwise”
-Commerzbank
Splunk isn't JUST for virtualization – it is for everything– Can be used for many use cases– capacity, configuration monitoring, security, compliance etc– Can be used with any other technology in the stack
“We initially wanted something to aggregate the logs, but now we use the detailed data from the VMs and the hosts to optimize resource allocation in our environment”
-Major Energy Company
What Customers Are Saying
25
“Thank you for bypassing VC – it resolves a huge bottleneck in our environment”.
“VC shows me information for now, but I cant see what yesterday looked like or what this time last month looked like”
“I really like the changed events tracking, let’s me see exactly what people are doing in the environment”
Why Use the Splunk App for VMwareIsn’t this the data I can see in VC?
NO• VC has only 5 minute summaries of performance data which hides problems
rather than surface them• VC summarizes this data further after 12 hours, you cannot go back to the level
of detail• Even the data it does retain is hard to report on• Try constructing a dashboard in VC that has VM CPU metrics, tasks associated
with the VM, host storage metrics, tasks associated with the host, metrics reported inside the VM
Why Use the Splunk App for VMwareVCOps tells me what to do and “predicts” the health of my
environment
VCOps is virtualization only – we complement it for everything elseThe VCOps sku that lets you integrate in data from non-VMware sources is 4 times more expensive – integrating other data sources is default in SplunkVCOps predictive algorithms are black box – you can’t teach it what you already know and it takes a while to learnDoes not have reportingLimited scale
Screenshots
29
Track any metric historically as the VM moves from host to host
30
View any tasks performed/changes made to the host or vms
31
Detailed host and VM metrics including disk latencies, IO, storage queue depths..
Configurable Capacity Reporting
32
Cluster level/Host level/VM level
Log Browsing
33
Datastore Capacity Reporting
34
How much storage is being taken up by snapshots?
How much are my datastores over-provisioned by?
Thank You
Backup Slides
What is vSphere?Virtualization Layer –Abstracts and Pools Server, Storage, Networking
Installs on servers, managed by one or many vCenter Servers
Applications running inside virtual machines have no knowledge of who else they are sharing compute, storage or networking capacity with
Not just CPU/Memory/Storage virtualization :– Resource aggregation– Automated load balancing– High Availability
APP
OSVM
VMware vSphere
VM
Physical Layer
ServersStorage Network Devices
APP
OS
VMware vCenter Server(VC)
What Could Possibly Go Wrong?Nothing. We run 1 VM per host, attached to local disk. No Problems.
CPU Memory Network Storage
HA agent crashes, behaves
badly
DRS causes
too many
vMotions
vMotion fail to
execute
Contention among VMs for resources manifests as application performance problems
Unauthorized changes by unaware people
Hypervisor functionality failures
Reality