Copyright©2016Splunk Inc.

JonWebsterCompetitiveIntelligenceManager,Splunk

Splunk&OpenSource:Buildvs.BuyWorkshop

Disclaimer

2

Duringthecourseofthispresentation,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.Wecautionyouthatsuchstatementsreflectourcurrentexpectationsandestimatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthose

containedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentationarebeingmadeasofthetimeanddateofitslivepresentation.Ifreviewedafteritslivepresentation,thispresentationmaynotcontaincurrentoraccurateinformation.Wedonotassumeanyobligationtoupdateanyforwardlookingstatementswemaymake.Inaddition,anyinformationaboutourroadmapoutlinesourgeneralproductdirectionandissubjecttochangeatanytimewithoutnotice.Itisforinformationalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.Splunkundertakesnoobligationeithertodevelopthefeaturesor

functionalitydescribedortoincludeanysuchfeatureorfunctionalityinafuturerelease.

Agenda

ADecisionFrameworkforChoosingtherighttoolforthejobOpenSourceisGreat!OpenSourceCustomerInterviewsOpenSourceisChallenging!TotalCostofOwnershipComponentsBuildingyourTCOModelCustomerExamplesQ&A

3

JonWebster5yearsatSplunkFormerly:

SalesEngineerClientArchitect

jon@splunk.com

Hasthiseverhappenedtoyou?

5

IthinkyoushoulduseOSS.IthasthemostRAM!

GofigureoutwhethertouseSplunkorOSS.

We’reusingOSSforXYZ.CanweuseitinsteadofSplunk?

Howdoyoudecide?

6

Requirements:deliverables,projectlifecycle,timeline,valueResources:staffing,end-users,training,infrastructure,time,moneyTechnology:on-prem/cloud,java/C++,hadoop/SQL,web/appProjectrisk:skills,complexity,codematurity,supportBusinessrisks:Opportunitycost?Whatiftheprojectisdelayed?Failstodeliver?Personalrisk:Whatdoesitmeantomeiftheprojectfails?Politics(sigh)

Howdoyoudecide?

7

Stipulatetherequiredfeatures&servicesEstimatethecosts&impactoftopoptionsRanktheoptionsbycost/impactBuildTCO/ROImodelcomparingtopoptionsProposebestoption,referringtoTCO/ROIcomparison

SampleWorksheet

8

WhyTryOpenSource?

9

Its“free”– freeFreeFREE!Muah-hahahaha!– Splunkseemscost-prohibitive– Don’twanttoorcan’tgetbudgetforSplunk– OpenSourceseemsgoodenough

“OpenSourceFirst”Orientation– Organizational“OpenSourceInitiative”forcostsavings– Open-sourceorbuildcultureValidDevelopmentusecases– Sub-secondresponsetimeforapplicationstack;web,document,or

productsearch

WhyDeveloperslikeOpenSource

10

Complexendlessprojects=JobsecurityNewtraining&skillsResumebuilding– SamSmithSr.Developer Sr.DataScientistBuildreputationinOSSforfuturejobs/consulting– StackOverflow,GitHub

WhyManagerslikeOpenSource• They’reseenasreducingcosts/addingvalue– it’sfree!• Solvetheproblemwithoutmanagementcycles• ShiftCapex(license)toOpex(salaries)• Nobudgetforsoftware,havedevelopersonhand• “Buildit”mentalityorOpenSourcereligion• Morestaff&infrastructure=biggerbudget&jobpromotion

11

Who’sMostLikelytoUseOpenSource?

12

Developmentteams,DevOps teams,SaaSprovidersTeams/Managerswhodon’tpayforinfrastructureTeams/Managerswhohavelotsofdevelopers/sysadmins andcanabsorbthestaffingcosts

OpenSourceCustomerInterviewsInterviewingCompetitors’HappyProductionCustomers

13

UserConferenceInterviews• 17Presenters:

– 4ITOps– 1SecOps– 8CustomAppDev– 4WebSearch

• 100Attendees– 50%AppDev/WebSearch– 50%DevOps/ITOpsLogging– Largest:35GB/day10Nodes

ProductionInterviews• 9Time-SeriesUseCases:

– 7ITOperationsLogging– 2SecurityOperations

• 4Non-Time-SeriesUseCases:– 1CustomApplicationDevelopment– 1WebsiteSearchEngine– 1MediaDocumentSearchEngine– 1Multi-DatabaseSearchCache

OpenSourceCustomerInterviews• Almostallwereunder25GB/dayper8core,50GB/dayper16core• OSSneeds5-10serverstomatchasingleSplunkserver,plusnodesforparsing,visualization,clustermasters,clientnodes,kafka,zookeeper,reverseproxy,alerting,jobscheduling,monitoring,andmaybeaHadoopclusterformulti-sitereplicationanddatapersistence

• OSSneedsmanytimesthediskspaceofSplunk– Yestherearewaystooptimizestorage,but…– Optimizingforinfrastructuresavingsreducesfunctionality

14

OpenSourceCustomerInterviews• 1TB/dayandlargertakes6-18monthstodevelop&deploy• Multipleclustersneededforlargeusecases– additionaltooling• Additionalpersistentdatastoreusuallyrequired(hadoop)• Ingestionisabottleneck– timeconsumingandfragile(maintenance!)• Visualizationislimited– manydeploymentsbuildtheirownUI• 90%oflargedeploymentsimplementmessagebus(kafka,redis,MQ)• End-userrequestscreatedevbacklog

15

WhysoMuchStorage?JSONformat,indexeveryfield,redundant“message”,“_source”,&“_all”fields.

ELK:1910chars,56indexes,1TBraw=4.8TBondisk(includingGeoIP &Identitydata)

Splunk:297chars,1index,1TBraw =½TBondisk150.128.102.148 - - [07/Aug/2014:00:59:52 +0000] \"GET /images/web/2009/banner.png HTTP/1.1\" 200 52315 \"http://www.semicomplete.com/blog/articles/week-of-unix-tools/day-1-sed.html\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36\

SplunkDataisenrichedatsearchtimesonoextradataisstoredorindexed!

WanttoenrichELKdata?Green: OriginalsyslogeventOrange: Identitydataadded

Red: GeoIPdataadded

{ "_index": "logstash-2014.08.07", "_type": "logs", "_id": "AUzqaoFTJX0-Q5nESGxf", "_score": null, "_source": { "message": "150.128.102.148 -- [07/Aug/2014:00:59:52 +0000] \"GET /images/web/2009/banner.pngHTTP/1.1\" 200 52315 \"http://www.semicomplete.com/blog/articles/week-of-unix-tools/day-1-sed.html\" \"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36\"", "@version": "1", "@timestamp": "2014-08-07T00:59:52.000Z", "host": "ctest08.sv.splunk.com", "clientip": "150.128.102.148", "ident": "-", "auth": "-", "timestamp": "07/Aug/2014:00:59:52 +0000", "verb": "GET", "request": "/images/web/2009/banner.png",

"httpversion": "1.1", "response": 200, "bytes": 52315, "referrer": "\"http://www.semicomplete.com/blog/articles/week-of-unix-tools/day-1-sed.html\"", "agent": "\"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36\"", "useragent": { "name": "Chrome", "os": "Windows 7", "os_name": "Windows 7", "device": "Other", "major": "32", "minor": "0", "patch": "1700" } }, "fields": { "@timestamp": [ 1407373192000 ] }, "sort": [ 1407373192000 ] },”

identity” {"personalTitle”: "Technical Manager","displayName” : ”First Lastname”,"givenName": "First Lastname”,

"sn": ”123-45-6789”,"suffix": “”,"mail": ”flastname@organization.org”,"telephoneNumber": ”123.456.7894”,"mobile": ”123.456.7894”,"manager": ”Another Manager”,"priority": ”3”,"department": "Technical Department”,"category": "Technical Manager”,"watchlist": ”whatever”,"whenCreated": [ 1407373192000 ] ,"endDate": [ 1407373192000 ] },

"geoip": { "ip": "150.128.102.148", "country_code2": "ES", "country_code3": "ESP", ”country_name": "Spain", "continent_code": "EU", "latitude": 40, "longitude": -4, "location": [ -4, 40 ] }

WhysoMuchStorage?Storageoptimization– atwhatcost?

17

Whichmeans:• AffectsCompliance&DebugUses• NoFull-TextSearchCapabilities• Notpracticalfordeploymentswith

100s– 1000sofdatasources• Moreinfrastructurerequiredto

maintainperformance• DisablesupdateAPI,onthefly

highlighting,&reindexAPI

Recommendations:• Deletetheoriginal”message”field• Disablethe“_all”field• Disablethe”_source”field• Setoptimalindex/analyzeoptions

inschemaforeachdatasource• Usebest_compression optionto

reducediskspace

WhysomanyServers?Memoryrequirementsdriveserverexplosion

Expertspointedustothesehostingservicesforbestpractices:• ObjectRocket provisions0.125GBmemoryforeachGBofdisk

– http://objectrocket.com/elasticsearch

• Compose.io (anIBMcompany)provisions0.1GBmemoryforeachGBofdisk– https://www.compose.io/articles/elasticsearch-at-compose-how-it-fits

• Bonsaiprovisions0.1GBmemoryforeachGBondisk– https://bonsai.io/pricing

• Qbox provisions0.05GBmemoryforeachGBofdisk– https://qbox.io/pricing

• Elastic.co’s ElasticCloudprovisions0.043GBmemoryforeachGBofdisk– https://www.elastic.co/cloud/pricing

18

WhysomanyServers?1TB/dayfor90days– 635Servers?!

Elastic.co Qbox Bonsai Compose.io (IBM) ObjectRocket

TotalDisk 945,000 945,000 945,000 945,000 945,000

Ratio 0.043 0.05 0.1 0.1 0.125

GBMemory 40,635 47,250 94,500 94,500 118,125

TotalServers@64GB/node 635 738 1,476 1,476 1,845

19

Expertspointedustothesehostingservicesforbestpractices:1TB/day,90daysretention,350%raw/diskratio,3totalcopiesofdata=945,000GBtotaldisk

USAAPresentationat2016UserConferenceFromVendorWebsite

OurDimensionsfor1TB/day,30daysretention:• Sevenclustersforeventfeeds (groupedbyfeedtype)• 60+Linuxvirtualservers:12core,96GB,6TBDisk,plus:

– 192TBSAN– 1.6PBoflonger-termsnapshotstorage

• 16servers(4Shippers&12Parsers)• 4KafkaServers(96partitions),plus3ZookeeperServersTotal:83Servers,192TBSAN,1.6PBAdd’l Storage

20

USAAPresentationat2016UserConference

21

ElasticInfrastructurealonealmostequalsSplunk’sTCO

Pricesdisplayedarelistprice

VerizonPresentationat2015UserConferenceFromVendorWebsite

ELKfor2.7TB/day,50daysretention:• 128Servers:8core,64GB,6TBDisk768• 50HadoopServers:24core,256GB,20TBDisk

– RetainrawdatainHDFSincaseofdatalossinelasticsearch

• NomentionofadditionalLogstash,MessageBus&otherServersTotal:Atleast178Servers,1768TBDisk

22

VerizonPresentationat2015UserConference

23

ElasticInfrastructurealonealmostequalsSplunk’sTCO

Pricesdisplayedarelistprice

Acustomermeeting,wherewe:• Sharewhatwe’velearnedfromdozensofOpenSourceProductionDeployments

• Discussthecustomer’sactualOpenSourceexperienceandmetrics• Translatethecustomer’smetricsintorealcosts• PrepareaBuildvs.BuyTotalCostofOwnershipModel• HavetheCustomervalidateandowntheModel• DeliveraCFO-ReadyBusinessCase

WhatistheSplunkBuildvs.BuyWorkshop?

AdditionalCommonCustomerDeliverables:• CFO-ReadyBusinessCases

• ValueRealizationStudies• DataSource&UseCaseAnalysis• CustomerandIndustryBenchmarks

• EnterpriseAdoptionRoadmaps• Skills&StaffingReadiness

BusinessValueConsultingServices

BusinessValueConsultingServicescustomizeyourvalueassessmentbyincludingtheservicesthatapply

UncoverkeygroupsthatwillbenefitfromSplunk

3hoursonsitewithstakeholders

DemandMatrixclickfordetails

AssessTCOforCloudvs.On-PremisesorSplunkvs.ELK

1 hourwithSplunkAdmin

TCOAnalysisclickfordetails

Planadeploymentbasedonvalueanddatasources

60minuteswithSplunkAdmin

Multi-YearRoadmapclickfordetails

Assesskeyroles,responsibilitiesandskills

60minuteswithSplunkAdmin

CenterofExcellenceclickfordetails

AlignSplunkcapabilitieswithkeyobjectivesandpainpoints

60minuteswithstakeholders

ValueStackclickfordetails

Quantifycurrentand/orfuturevaluebyusecase

60minutespervaluecenter

ValueQuantificationclickfordetails

Document2-3reallifevaluestoriesfromyourdeployment

45minutesperstory

SuccessStoriesclickfordetails

Uncoverusecasestodrivemorevaluefromyourdata

30minutesperteam

DataSourceAnalysisclickfordetails

Appendix:Buildvs.BuyWorkshopExecutive-ReadyBusinessCase

Splunkvs.OpenSource:3Considerations1. TimetoMarket

– Valueisachievedfasterwithaplatformvs.thetimerequiredtobuildit

2. BenefitRealization– Asolution’sabilitytoproduceprovencustomersuccess

increaseslikelihoodthatbenefitswillberealized– Aplatformbuiltfrom10,000+customerswillyieldmore

valuethanasolutionbuiltentirelyfromscratch

3. TotalCostofOwnership– Opensourcesoftwareisnotfree– Productiondeploymentscaneasilyexceed4-10xSplunkcost

28

Consideration1: TimetoMarket• Valueisachievedfasterwithapurpose-builtplatformvs.thetimerequiredtobuildit(evenbasicfunctions)

• Pre-builtapps speedsdeployment(SplunkBase has1000+apps)• Time impactshowmuchvaluewillberealized• EXAMPLE: Applyingthisconsideration

– Assuming$1.2M/yearofprojectedbenefitsfromadeployment– IfSplunktakes2monthstodeploy,itdelivers$1Mofvalueinyear1– IfOpenSourcetakes10monthstodeploy,itdelivers$200kofvalueinyear1– Assumingthesameendresult,Splunkdelivers$800kMOREvalueinyear1– TCOwouldshow$800kas“lostopportunitycost”intheOpenSourcecalculation

29

RealExample:Splunkvs.OpenSourceFromaFortune50TelecommunicationsCompany

Project: Executivedashboardfornearreal-timeTVProgrammingAnalytics

30

OpenSourceBuild “Buy”w/Splunk

Took6people6months’effort

Modificationsaresmalldevelopmentprojects

Took1person2weeks’effort

Modificationsaremadebyusers

onthefly

VS

Splunkdeliveredin92%lesscalendartimewith99%lesseffort

Multipleopensourcesolutionsmanuallystitchedtogether

Consideration2: BenefitRealizationOpenSourceSplunk

• 12,000+productioncustomers• Vibrantusercommunity• 1000+Splunkapps• Provencustomersuccess• Documentedbenefitbenchmarks

• Unknown#ofproductioncustomers• Vibrantdevelopmentcommunity• Nopre-builtappstore• Nopublishedbenchmarks

31

EXAMPLE: Applyingthisconsideration• AnITOperationsprojectisexpectedtoreduceincidentinvestigationtime• Splunk’sdocumentedbenchmarksshowthecustomerwillachieve70-90%reduction• SinceallfunctionalitymustbebuiltforElasticStack,itmaynotachievethesamebenefitlevel• IndoingaTCOanalysisthismustbeconsidered.Itwouldbeaddedasa“lostopportunitycost”totheOpen

Sourcecalculation

Consideration3: TotalCostofOwnership• Considerallthecomponents ofcost

– It’smorethanjustlicensefees

• Evaluateproduction-gradedeployments– Smallsideprojectsmayhidetruecosts

• Scalabilityandefficiencyimpactinfrastructureandadmincosts– Hardware,people,etc.

• Differentskillsets arerequiredtobuildvs.configure– Highlycompensatedandscarceopensourcedevelopersvs.general

adminsmorewidelyavailableandaffordable

32

ThereareManyComponents ofTCO

• Server,network,workstationhardware

• Softwarelicense• Installationandintegration• Purchasingresearch• Warrantiesandlicenses• Licensetracking– compliance• Migrationexpenses• Risks– vulnerabilities,

upgrades,patches,failure

• Facilityandpower• Testingcosts• Downtime,outageandfailure

expenses• Diminishedperformance

(usershavingtowait,etc.)• Security(breaches,lossof

reputation,recoveryandprevention)

• Backupandrecovery process

33

• Technologytraining• Audit(internalandexternal)• Insurance• Technologystaff• Managementtime• Replacement• Futureupgradeorscalability

expenses• Decommissioning• …

Licensecostsareonlyoneofthem…

RealitiesofProductionGradeDeploymentsConsiderationsforplatformselection– Infrastructure,people,andtime

• Singleplatformandsolution

• Rich,powerfulquerylanguage

• Lowercost,availablelevel1or2resources

• Architectureoptimizedforscale

• Communityofpre-built‘apps’

• Rapidtimetovalue

34

Multipleseparate,opensourceproducts

Limitedquerycapabilities

Highlypaid,scarce,level3or4resourcesrequired

Infrastructurecostsat5-10xSplunk

Significantdevelopmenteffortrequired

Lostopportunitycostduetoslowtimetomarket

OpenSource

or

Splunkvs.OpenSourceTCOModelFulldetailedcomparisonofSplunkvs.OpenSourcecostsbasedonCustomer’snumbers

• Hardwareacquisitionandmaintenance– Servers,storage,loadbalancers,datacentercosts

• Softwarelicensingandmaintenance– Perpetual,subscription,includingrenewals

• Professionalservices– Implementation,configuration

• Splunktraining/education– Includesongoingrecommendations

• Ongoingadministrationsupport– Sysadmin,architect,developer,poweruser,Splunkadmin

• OpportunityCost

35

SampleTCOSummaries

36

$-

$5,000,000

$10,000,000

$15,000,000

$20,000,000

$25,000,000

$30,000,000

200GB 1TB 5TB 10TB

TCOfor3Years30dayretention

Splunk

OSS

$-

$5,000,000

$10,000,000

$15,000,000

$20,000,000

$25,000,000

$30,000,000

200GB 1TB 5TB 10TB

TCOfor3Years60dayretention

Splunk

OSS

37

YearlySchedule

Thischartrepresentsthe3yearbenefitsforSplunkvsELK.

38

CumulativeResults

Thischartrepresentsthecumulativeresultsover5yearsforOn-Premesis,SplunkCloudandAWS.

SecurityMatters

Opensourceiscommunitydriven;sourcecodeispublicLackoftrueproductmanagement,softwaredevelopmentandtest/QAopensrealvulnerabilities

39

threatpost

“HackershavetakenaninterestinElasticsearch…”

Splunkvs.OpenSourceSummaryofthe3considerations

OpenSourceSplunk• Timetovalue

– Realizedinlessthanthreemonths• Benefitrealization

– Documentedbenchmarksandprovencustomersuccess

• TCO:$2,860,251

• Timetovalue– Realized6to12+months

• Benefitrealization– Nopublishedbenchmarksorproven

customersuccess• TCO:$5,577,184

40

THANKYOU