Splunk Webinar: Maschinendaten anreichern mit Informationen

  • View
    897

  • Download
    0

Embed Size (px)

Text of Splunk Webinar: Maschinendaten anreichern mit Informationen

Slide 1

Maschinendaten anreichern mit InformationenPhilipp DriegerSales Engineer

Copyright 2014 Splunk Inc.

#The Splunk Enterprise Technical Overview1

Splunk WebinarMaschinendaten anreichern mit Information

Ihr Ansprechpartner:Philipp DriegerSales Engineer

philipp@splunk.com

#DisclaimerDuring the course of this presentation, we may make forward looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in the this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make.

In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release.

#AgendaSplunk fr MaschinendatenMethoden zur AnreicherungLive: CSV | KVStore | Script | DBX | ODBCQ&A

#This presentation covers 4 key areas about our technology and how it is typically used.

4

Make machine data accessible, usable and valuable to everyone.

#As a company our mission is to make machine data accessible, usable and valuable to everyone. This overarching mission is what drives our company and product priorities.

5

Das beschleunigte Wachstum von DatenVolume | Velocity | Variety | VariabilityGPS,RFID,Hypervisor,Web Servers,Email, Messaging,Clickstreams, Mobile, Telephony, IVR, Databases,Sensors, Telematics, Storage,Servers, Security Devices, Desktops

Machinendaten umfassen den am schellsten wachsenden, komplexesten und wertvollsten Bereich von Big Data6

#Data is growing and embodies new characteristics not found in traditional structured data: Volume, Velocity, Variety, Variability.

Machine data is one of the fastest, growing, most complex and most valuable segments of big data. "Big data" is a term applied to these expanding data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time.

All the webservers, applications, network devices all of the technology infrastructure running an enterprise or organization generates massive streams of data, in an array of unpredictable formats that are difficult to process and analyze by traditional methods or in a timely manner.

Why is this machine data valuable? Because it contains a trace - a categorical record - of user behavior, cyber-security risks, application behavior, service levels, fraudulent activity and customer experience.6

Machinendaten enthalten wertvolle Informationen

Order ID

Customers Tweet

Time Waiting On Hold

Product ID

Companys Twitter ID

Order ID

Customer ID

Twitter ID

Customer ID

Customer ID

SourcesOrder ProcessingTwitterCare IVRMiddleware Error

#To frame our discussion, lets use this example of purchasing a product from your tablet or smartphone: the purchase transaction fails, you call the call center and then tweet about your experience. All these events are captured - as they occur - in the machine data.Each of the underlying systems hast the potential to generate millions of machine data events daily. Here we see small excerpts from just some of them.

When we look more closely at the data we see that it contains valuable information right down to what was tweeted.Whats important, is first of all, the ability to actually see across all these data sources, but then also to correlate related events and provide meaningful insight.

If you can correlate and visualize the data, you can build a picture of activity, behavior and experience. And what if you can do all of this in real-time? You can respond more quickly to events that matter.

This example ties into your scenario but you can also extrapolate this example to a wide range of use cases security and fraud, transaction monitoring and analysis, web analytics, IT operations and so on. 7

Industry Leading Platform For Machine Data Machine Data: Any Location, Type, VolumeOnline ServicesWeb ServicesServersSecurityGPS LocationStorageDesktopsNetworksPackaged ApplicationsCustomApplicationsMessagingTelecomsOnline Shopping CartWeb ClickstreamsDatabasesEnergy MetersCall Detail RecordsSmartphones and DevicesRFID

On-Premises

Private Cloud

Public Cloud

Platform Support (Apps / API / SDKs)Enterprise ScalabilityUniversal IndexingAnswer Any QuestionDeveloperPlatformReport and analyzeCustom dashboardsMonitor and alertAd hoc search

Universal Machine Data Platform

#One of of the key differentiators of Splunk is the ability to digest all machine data and allow users to quickly analyze it for insight. We call this the universal machine data platform. Well look at this in more detail in a bit, but for now, understand that the platform was designed around the premise of being able to consume any machine data even if the format changes; something a relational database cannot do.

(Splunk Cloud is only available in the U.S. and Canada.)8

Industry Leading Platform For Machine Data Machine Data: Any Location, Type, VolumeOnline ServicesWeb ServicesServersSecurityGPS LocationStorageDesktopsNetworksPackaged ApplicationsCustomApplicationsMessagingTelecomsOnline Shopping CartWeb ClickstreamsDatabasesEnergy MetersCall Detail RecordsSmartphones and DevicesRFID

On-Premises

Private Cloud

Public Cloud

Platform Support (Apps / API / SDKs)Enterprise ScalabilityUniversal IndexingAnswer Any QuestionDeveloperPlatformReport and analyzeCustom dashboardsMonitor and alertAd hoc search

Any amount, any location, any sourceSchema-on-the-flyUniversal indexingNo back-end RDBMSNo need to filter data

Schema on the Fly

#Splunk is able to do this because theres no requirement to understand the data upfront this is one of our key differentiators that we call schema on the fly.

Simply point Splunk at the data or deploy Splunk forwarders to stream data from remote systems. Splunk immediately starts collecting and indexing, so users can start searching and analyzing. No more armies of consultants, backend database or DBA to make it work. Once youve Splunked your data, it is time-stamped and easily searchable. Because we dont have to do all the up front work to be able to look at the data we can load it all and make it all relevant. Theres no need to limit what you load and what you dont.

9

Methoden zur Anreicherung

#Now that we understand the high level question of What is Splunk Enterprise, lets talk about how the technology can be deployed and integrated into your existing environment. 10

Anreicherung von Events in SplunkErweiterung der raw events um zustzliche Felder, welche aus externen Datenquellen stammen. LDAP, AD

WatchLists

CRM/ERP

CMDB

Externe Datenquellen

Insight OUTData IN

#The data for example may have a userid but you want to search on a name. Splunks lookup capability can enrich the raw data by adding additional fields at search time. Some common use cases including event and error code description fields. Think about how much easier it would be if you could seePage not Found instead of the error code 404 in the search results. Enriching your data can lead to entirely new insight.

In the example shown, Splunk took the userid and looked up the name and role of the user from an HR database. Similarly, it determined the location of the failed log in attempt by correlating the IP address. Even though these fields dont exist in the raw data, Splunk allows you to search or pivot on them at any time.

You can also mask data. For example, you may want social security numbers to be replaced with all Xs for regular users but not masked for others. Removing data can also be useful, such as filtering PII, before writing it to an index in Splunk.

11

Anreicherung mit LookupsDurch Lookups knnen in Splunk Maschinendaten mit zustzlichen Informationen angereichert werden.

Es wird dabei ein Mapping von Feldwerten in Events auf Feldwerte einer externen Datenquelle realisiert und neue Werte den Eventdaten zugefgt.

Beispiel: Lookup von HTTP Status Codes in einem CSV File mit der entsprechenden Beschreibung des Codes.

#

12

Mehrwerte durch LookupsDarstellung von Maschinendaten in der Sprache der FachabteilungenDifferenziertere Analysen und Aufteilungen von Auswertungen z.B. Monitoring von Manager User Accounts, HR, Finance, ITVerlinkung von Maschinendaten zu geschftsrelevanten Prozessen. Z.B. Anreicherung von Bestelldaten mit Artikellisten inklusive Beschreibung, Verfgbarkeit, Preis etc.

Integration von SAP BestandsdatenCRM DatenProduktinfosPreislistenWHOISGeolocationZip codes

#bersicht: Methoden fr Lookups in Splunk

ODBC driver (MS Excel, Tableau, )

CSV FileLookupScript(Python, Perl, shell, )

DB Connect(DB2, Oracle, MySQL, )

KVStore(Key Value St