1 13763 ArcSight6-2011 LogMgt Survey

8/4/2019 1 13763 ArcSight6-2011 LogMgt Survey

1/18

Sponsored by ArcSight

SANS Seventh Annual

Log Management Survey ReportA SANS Whitepaper April 2011

Written by Jerry Shenk

Survey Sample

Why Companies Collect Log

Data

Users Want Better Log Data

(and More of It!)

Top Challenges to Eective LogManagement

Advisors:

Dave Shackleford and Barbara Filkins


2/18

SANS Analyst Program 1 SANS Seventh Annual Log Management Survey Report

Everyspringsince2005,theSANSLogManagementsurveyhastrackedthegrowthandmaturityof

thelogmanagementindustry.Thissurveyhasconsistentlyidentiedareasinwhichorganizationsare

focusingtheir logmanagement initiatives andcontinuesto providea roadmaptothe industry for

futureimprovement.Overtheyears,thesesurveyshaveshowngrowthinthecollectionanduseoflogs

forsecurityandcompliance.Mostrecently,inthepasttwoyears,thesesurveyshaveshownthatorga-

nizationsareseekingmoreusesfromtheirlogs,buttheyhaveproblemsgettingthevaluetheywant

fromthoselogs.

Whenthissurveystartedsevenyearsago,logcollectionwasonlybeingdoneby43percentofrespon-

dents,comparedwith89percentwhoindicatedtheycollectedlogsthisyear,whichisconsistentwith

lastyearssurvey.So,logcollectionisnolongerasmuchofaproblemasitwasinthepast.Now,theyre

alsocollectinglogsformuchmorethandetectingsuspiciousbehaviorandtroubleshooting,asinthe

recentpast.Overthepasttwoyears,morerespondentsarealsocollectinglogsforuseinforensicanaly-

sisandcorrelationandtomeet/proveregulatorycompliance.Infact,thesethreeusesforlogsrankclose

enoughinimportancethatitisfairtosaythatforalogmanagementsolutiontobeeffectivetoday,it

mustsupportallthree.

Inadditiontotheabovetopthreeuses,organizationsarecollectingmoredata fromphysicalplant/

operationssystems(e.g.,HVAC,SCADA),mobileplatforms,andpoint-of-sale(PoS)devices.Thismeans

morelogtypestocollectandanalyzeeachwiththeirowndataformatsthatcanvarywidely.Even

whentheselogdataformatdifferencesareslight(suchasonedateformatbeingMMDDYYYYand

anotherbeingMM-DD-YYYY),theymustbeadjustedinordertoaccuratelycorrelateandreportonthe

data.Thishasbeenanongoingproblemforusersoflogmanagementtechnologies,particularlyastheystarttousetheirlogsformorepurposes.

Inadditiontonormalization,respondentsarealsostrugglingwithsearching,correlatingandreporting

functionalities.Figure1illustratestheaspectsoflogmanagementthatrespondentsconsideredmost

challengingormoderatelychallenging.

Executive Summary


3/18


Figure 1. Log Management Challenges

Themechanicsofcollecting,storingandarchivingthelogdataarenolongerthechallengeintodays

worldofalmostunlimiteddatastorage.Thechallengenowisextractingtheneededinformationfor

monitoring,management,complianceanddecision-making(ofteninnearreal-time)fromwhatrespon-

dentssayisupwardsof100,000eventsrecordedperday.

Thisyear,respondentswereaskedspecicallyaboutwhatwasandwasnotusefulintermsofsearch-

ingandreportingcapabilities.Theyselectedreal-timealertsastheirmostusefulfeature.However,they

werelessenthusiasticabouttheirlogmanagementsystemsabilitytointerfacewiththird-partytools

orlargerSIEMenvironments.Usersalsocitedproblemswithcorrelation,searchingandinterfacingwith

heterogeneoussystems,anddifcultieslocatinginformationwithinlogs.

Inparticular,Windowssystemsarestilldifculttodrawandnormalizelogsfrom.Thisisaprimaryprob-

lemfororganizationsthisyear,asinyearspast,accordingtoresponses.Windows,pervasivethroughout

mostindustries,iswidelycriticizedforitsunfriendlinesstologanalysis.However,allvendorsoflog

managementapplicationsaremakingtheirsystemsinteractbetterwithmultiplesourcesoflogdata,

includingfromWindowssystems.However,asonecommenterwrote,allvendorsstillneedtogetbetter

atgeneratingusefulevents.

Despiteshortcomingsrespondentsreport,organizationsareincreasinglydependentonlogmanage-

menttosupportcorebusinessfunctionsincludingcostmanagement,servicelevelandline-of-business

applicationmonitoring,aswellasmoretraditionalIT-andsecurity-focusedactivities,accordingto

responses.Therestofthis reportdetailswhatorganizationsaredoingwiththeir logs

todayandwhattheystillwantfromtheirlogsinordertoachievethehighestvalue

fortheirbusiness,securityandcomplianceoperations.


4/18


Atotalof747 organizationsstarted this years survey, with 571 completingthe surveyall the way

throughtotheend.Organizationsrepresentedinthisyearssurvey(seeFigure2)encompassedawide

rangeofindustriesandsizes.Thelargestindustryverticalsrepresentedwerenancial(19percent)and

government(18percent).Healthcareandeducationwerewellrepresentedaswell.Theadditional23

percentthatrepliedotherincludedgoodrepresentationfromsoftwarecompanies,entertainment,

managedservicesandconsultantsworkingamongtheseverticals.

Figure 2. Industries Represented in This Years Survey

Survey Sample


5/18


Respondentswerenearlyequallybalancedbetweenlargeorganizations(over2000employees)and

mid-sizedandsmallorganizations,asshowninFigure3.

Figure 3. Size of Organizations Based on Responses

Thevastmajorityofrespondentsheldstaffpositions(ratherthanbeingconsultants).Thisyear,ahigher

percentageof respondentsheldasecurity-orientedrolein theirorganizations,asopposedtoa net-

work-orientedrole,whichthereweremoreoflastyear.Of the747respondentstoanswerthisques-tion,73percenthadsecuritytitles,whereas35percenthadnetworkingtitles.Somerespondents,seven

percent,alsohadcomplianceofcerroles.Thetotalexceeds100percentbecausesomerespondents

dutiesoverlapamongtheareasofnetworking,securityandcompliance.


6/18


Inthisyearssurvey(asinthe2009and2010surveys),detectingincidents,determiningwhathappened

(forensicsandanalysis),andmeetingcompliancerequirementswerethetopthreereasonsforcollect-

inglogs.Onceagainthisyear,themostimportantreasonforcollectinglogdatawastoDetect/track

suspiciousbehaviorandpreventincidents,asillustratedinFigure4.SecondplacewenttoSupport

forensics analysis and correlation, and third wasMeet/prove compliance with regulatory require-

ments.

Figure 4. Why Respondents Collect Logs

Whilemaybenotcritical,supportingotherIToperationsrankedhighinlevelofimportance,andmore

than50percentoforganizationsthinkthatlogscanbeimportantinreducingcostsandsupporting

otherprocessesbesidessecurityandcomplianceoperations.Theseoptionswerenotprovidedinlast

yearssurvey,butsurveyrespondentslastyear(andthisyear)indicatedanincreasingdesiretoderive

morebusinessvaluefromtheirlogs.

Why Companies Collect Log Data


7/18


Most Useful Features

Oncetheycollecttheirlogs,respondentssaythemostusefulfeatureoflogmanagementsystemsis

real-timealerts,with68percentindicatingtheyareveryusefuland25percentindicatingtheyare

somewhatuseful.ThesecondandthirdmostusefulfeatureswereIntuitiveuserinterfaceforsearch

andUniedinterfaceforalllog-relatedactivities.Tobeprecise,thereisnosuchthingasareal-timealert,duetodelaysinlogeventanalysisandnotications.Whatsimportantisthatmanyrespondents

aregettingusefulalertsfromtheirlogmanagementsystemsinatimelyenoughmanner.

ThefourthmostusefulfeaturewasGoodperformanceforalllog-relatedactivities,whetherindivid-

ualorsimultaneous.Inthepast,logmanagementsystemperformancereceivedlowmarksbysurvey

respondents.Itisgoodtoseethat55percentofrespondentsgavethisthehighestmark,while37per-

centgaveitamid-rangemark.Combined,thatsmorethana90percentapprovalrating.Integration

withlargerSIEMenvironmentrankedninthonthe listofusefulness.Somecommentsindicatethat

respondentsareintheprocessofinstallingSIEMsystems,sotherewilllikelybestrongerresponsesto

thisquestionnextyear.Figure5showstheoverallratingsforVeryandSomewhatusefulfeaturesbased

onresponses.

Figure 5. Features Deemed Most Useful by Respondents


8/18


Flippingthequestionaround,itsalsointerestingtonotethattheleastusefulfeaturesoflogmanage-

mentpointtootherintegrationproblems.Thequestionwas,Howusefuldoyouratethefollowingfea-

turesinsupportofyourloganalysisandreportingactivities?Thechoiceswere,VeryUseful,Somewhat

Useful,andNotUseful.NotUsefulwaschosenmostforInterfacewiththird-partyreportingtools,with

27percentofrespondentschoosingthisoption.Sharingthebottomofthelistwaswitha21percent

negativevotewasIntegrationwith largerSIEMenvironment.Figure6 showsthefeaturesdeemed

leastusefulbyrespondents.Overall,thesearerelativelylownegativescores,whichsuggeststhatthe

usefulnessoflogmanagementsystemsisimproving.

Figure 6. What Respondents Find Least Useful About Their Log Management Systems


9/18


Users Want Better Log Data (and More of It!)

Thenumbersofsourcesfromwhichorganizationsarecollectinglogscontinuestoexpand.Thisyears

surveyshowsthat59percentofrespondentsarecollectinglogdatafromtheirlineofbusinessapplica-

tions,and14percentofrespondentsarecollectinglogdatafromtheirphysicalplantcontrolsystems,

suchasHVAC.Thesewerenotconsideredamajorsourceforlogdatainpreviousyears.Othernew

sourcesincludedinthisyearssurveyarelogcollectionfrommobiledevices(15percent)andcloud

services(14percent).Point-of-sale(PoS)deviceswerenotonthelistbutwerereferencedincomments.

Accordingtothisyearssurvey,mostorganizationsarecollectinglogsfrommorethan50devices,with

only30percentcollectingfromfewerthan50devices.Thevastmajorityofsurveyrespondentsindicate

theyarecollectinglogsforcompliancepurposes,leadingwithPCIDSS.Figure7showswhatcompli-

ancemandatesaredrivingtheirlogmanagementprograms.

Figure 7. PCI DSS is the Leading Compliance Driver for Log Collection


10/18


Thetypesof loginformationrespondentsconsidertobethe mostvaluableareSource/destination

IPaddressandTime/datestamp.ThesewerenearlytiedwithEventinformation(name,category,

type),followedbySource/destinationTCP/UDPportandUserinformation.Thislevelofdetailedlog

data,correlatedasneededandinreal-time,helpsoperatorsndeventsonthenetworkwithminimal

manualsearchingandbetteraccuracy.Thisquestionalsohadanothercategory,inwhichrespondents

indicatedtheywantedevenmoreinformationfromtheirlogmanagementsystems,includingdetailed

networkconnectionlogs,completeURLstrings,fullpacketcapture,andpayload.A logmanager

mightnotbethebestplaceforsomeofthatdata.Instead,IPS,continuousmonitoringorSIEMmight

collectthesedatatypesmoreeffectively.However,thecommentshighlightthepointthatmanyana-

lystswantmoreinformationcorrelatedagainstmorethreat-monitoringdevicestohelpthemmake

decisionsaboutpossibleevents.

Vendorsneedtogetbetteratgeneratingeventsthatareusefulbecauseitdoesntmatterhowgood

yourlogmanagementsolutionisiftheeventscomingintoitaregarbage,wroteonecommenter,Jim

Murray,aninformationsecurityarchitectintheinsurancesector.Vendorsofhardwareandsoftwarethat

generatelogsshoulddifferentiatethemselvesfromtheircompetitionbystandardizingtheirlogdataanditssyntaxandimprovingthelevelofloginformationtheymakeavailable.


11/18


Top Challenges to Eective Log Management

Yearoveryear,trendsuncoveredinthissurveyhavedirectlyreectedthematuringoftheindustry.

Initially,thetopproblemreportedwassimplycollectinglogs.Afewyearsago,collectinglogsdropped

tothe leastproblematicissue,andnowrespondentsexpresstroublesin theareasof normalization,

categorization,searchingandreporting.Seegure8.

Figure 8. Top Challenges Reported by Log Management Users

Normalizingandcategorizinginformationwasthetopissuethisyear(42percentclaimedthisastheir

mostchallengingproblem,and37percentconsidereditaproblem).Thesecondmostnotedissuewas

searching(32percentconsideredthistheirmostchallengingproblem,and48percentconsideredit

aproblem).Usinglogsforreportingandanalysiscameinthird(18percentconsideredthistheirtop

challenge,with50percentconsideringthisaproblem).Nearlyashighapercentage(49percent)con-

sideredusinglogsforoperationsandmaintenancetobeaproblem,with18percentconsideringittheir

topchallenge.Thesechallengestiecloselytoresultsfromarelatedquestionaboutthetophindrances

insearchingandanalyzinglogs.Inorder,thesetopproblemswereinabilitytosearchacrossdifferent

logmanagementsystems, lackofcorrelationcapabilities, interfacingwithotherIT groups,and

locatingneededinformationwithinthelogscollected.


12/18


13/18


Integrationwithmultiplelogmanagementtoolsisbecomingafactorbecauserespondentsthisyear,

aswellasinrecentyearspast,reportusingamixofhomegrownandthirdpartytools.Manyreport

usingmultiplethirdpartylogmanagementtools.Responsesalsoindicatemultiplehomegrowntools

insingleenvironments,withaverysmallnumberusinglogmanagementasaservice.

Surveyresponsesalsopointtotheneedforstrongergraphicalanddatarepresentation,withonly32

percentofrespondentsrankingthesefeaturesasVeryusefulintheirlogmanagementsystems.Awell-

designedgraphorchartcanconveyalotofinformationquicklyandcanevensupportnon-technical

managerswhennecessary.Onecommenterpointedoutthatfromabusinessperspective,sometimes

includinggraphicsisanexpectedpartofapresentation,evenifthegraphicsvalueislimited.Responses

indicatethatpeoplehaveworkedwiththeirlogmanagersgraphicoptionsandwouldliketoinclude

graphics,buttheyarentabletogetwhattheywouldlikeoutofthepresentationcapabilitiesofcurrent

logmanagementsystems.Thisisanotherareaofgrowthforvendors.

Theabilitytoscriptroutinetaskswasalsobroughtupbyonerespondent.Anyseriousloganalystknows

thattheabilitytosetupscriptstorunrepetitivetaskscanbeahugetimesaver.Scriptsoftenmakeit

possibletotrackeventsandstatistics,allowingreviewthatwouldnotbeavailableanyotherway.Many

loganalystssetupprocessestorunintheearlymorningtogivethemsomequickbaselinestoreview

whentheygetintowork.Othersrunscriptsperiodicallytodetectsuspiciousorovertlyhostileactivity

(thesinglefeatureratedmostuseful).Inordertocollectandconsolidateinformationthatdoesntneatly

tintoareport,theabilitytorunlow-levelscriptsisoftennecessary.Manylogmanagementsystems

havesomecapabilitytoscriptandrunsomereportsonascheduleanddeliverthemovere-mail,via

web,pagerorsmartphone;however,basedonresponses,theyneedevenmorescriptabilitythanthey

alreadyoffer.

Managing Windows Logs

This isthesecondyearthesurvey includedquestionsspecicallyaddressingWindowslogmanage-

ment.Theresultsareessentiallythesameforbothyears:Windows,themostheavilyusedoperating

systemthroughouttheworld,stillgetsabadgradeforits loggingenvironment.Asonerespondent

statedsimply,Windowsmakesitdifculttocollectlogs.

CollectionandstorageofWindowslogsreceiveda40percentapprovalscore,withabout10percent

reportingthey wereVerySatised andabout 30percentreporting theywereSatised. Allother

categorieshelddismalsatisfactionratings:FivetosevenpercentreportedbeingVerySatisedand

between18and24percentwereSatisedwiththeirWindowslogmanagementcapabili-

ties.Thatleavesapproximately50to60percentofrespondentsbeingonlySome-whatSatisedorDissatised(seeFigure9).


14/18


Figure 9. Windows Log Management Still Gets Low Scores from Respondents

AnalysisisthetopproblemthatorganizationshavewithWindowslogmanagement,closelyfollowed

byreporting.ThereareanumberoffactorsthatmakeWindowslogmanagementmoredifcultthan

othersoftware(UNIX/Linux)andhardwareplatforms,suchasrouters,rewallsandswitches.Windows

doesnotnativelysupportsysloginanyavorforlogcollection.Yet,accordingtothesurvey,UDPSyslog

isstillthemostpopularlogcollectionmethod.TCPSyslogismoreresilientandcanscalebetter,and50

percentofrespondentsalsosupportTCPSyslog.NeitherversionofSyslogissupportedbyWindows.

ItwouldbehelpfulifMicrosoftwouldincorporatesomechangesintheiroperatingsystemstomakeiteasiertocollect,normalize,parseandanalyzeeventscomingfromWindowssystemsandsubsystems.

Usersofteninstallthird-partyadd-onapplicationstogetthisfunctionality.Thoseleavingcomments

listedtheSnareagentasthemostpopularwaytosendeventlogdatafromaWindowsservertoasys-

logserver,buttherearealsootheroptions.SomelogmanagementsystemspulllogdatafromWindows

servers,aswell.Today,theburdenofanalysisrestsmostlyonthelogmanagementsoftwaretopulland

normalizeWindowseventsintousableinformation.

SatisfactionwithWindowslogmanagementhasdecreasedinsomecategoriessincelastyear(monitor-

ing,performanceandcollection)withnoimprovementsinreportingandonlyminorimprove-

mentsinanalysisandstorage(seeFigure10).So,vendorshavealongwaytogoto

satisfyWindowsusers.


15/18


Figure 10. Windows Log Management Scores Worse This Year in Some Areas

Where to Start? A Primer for Windows Log Management

Dr.AntonChuvakin,leadauthoroftheSANSLogManagementcourse,says,Oneoftherstthingsthat

peopleshoulddotostartgettingvaluefromtheirWindowseventlogsistoactuallystartcentrallycol-

lectingthemfromalltheWindowssystems.Beforeyoucandoanalyticsandalerts,itmakessenseto

buildaworkinglogrepository.Itwillhugelyhelpyouduringincidentresponse.

OnepopularwaytodothisisusingtheSnare 2agent,althoughthereareotheroptions.Itisalsopossible

topulltheinformationfromtheeventlogsusingLASSO3oroneoftheotheragentsthatareavailable.

ForafullWindowsshop,thelogservercouldrunonaWindowscomputer.TheKiwiSyslogServer4

isapopularoption.TherearealsofreelogserversthatrunonLinux,andthereareanumberofcommercial

logservers.Oncethesyslogserverisrunning,youcansearchthroughtheeventsforeventsofinterest.

Dr.Chuvakinalsorecommendslearningthenormallogpatternsrightaftercollection.Storedlogsare

useful(suchasforincidentresponse),buttouselogsforincidentdetection,youneedtoknowwhatis

abnormalandthatbeginswithknowingwhatisnormal!

OnthewebpagefortheSANScourseoncomplianceformanagers,5thereisalsoalinktothecourses

PDF,whichcontainsachecklistforsecurityincidents.Inthelowerleftcornerofthatleisalistofafew

ofthemostcriticalWindowsevents.Thesecanbeagoodstartingpoint.

Whenexaminingthelogs,youllneeda placeto lookupeventIDstogetmoreinformationonthem.

SearchingforthespeciceventID(e.g.,eventid528)ontheMicrosoftTechNetSupportwebsite 6can

behelpful.Thesite,eventid.net,isalsoaquick,handyresourceforinformationaboutspecicWindows

eventIDs.RandyFranklinswebsite7hasanextensivelistofWindowseventIDs.

2www.intersectalliance.com/projects/SnareWindows/3http://sourceforge.net/projects/lassolog/4www.kiwisyslog.com/kiwi-syslog-server-features-and-benets5www.sans.org/security-training/log-management-in-depth-compliance-security-forensics-troubleshooting-1217-mid6http://technet.microsoft.com/en-us/ms772425.aspx7www.ultimatewindowssecurity.com/securitylog/encyclopedia/Default.aspx


16/18


Organizationsareincreasinglymeasuringtheirsecurityeffectivenessbasedontheirabilitytoimprove

incidentremediation,reduceincidentsandmeetcompliance,accordingtothisyearssurvey.Theyare

alsomeasuringeffectivenessbyhowmuchtheyreduceoverallsecurityandmaintenancecosts,aswell

asimproveoverallsystemperformance.

Measuringeffectiveness andmakingimprovementsdepends, in largepart, upon logs. Log analysts

wantbetterlogdatafrommoredevices,andtheyarelookingforbetterqualitylogdatatobegleaned

fromtheirmonitoreddevices.Thetopreasonsorganizationscollectlogsaretodetect,trackandanalyze

securityincidentsand tomeetregulatorycompliancerequirements.Thedevicestheywantlogdata

fromareextendingbeyondthetraditionalsources(e.g.,servers,rewallsandrouters)tothephysical

plant(e.g.,HVAC,SCADA)andremotelyattacheddevices,withasmallpercentagealreadycollecting

logsfromphonesandPoSterminals.ITdepartmentsarealsolookingforlogmanagementsystemsthat

providequick,accurateandcorrelatedresponsestoqueries.Theyalsowanttobeabletoturnthose

queriesintoreportswithvisualsandgraphics,whilebeingabletoeasilycustomizequeriestosupport

industry-specicapplicationsanddevicesinusewithintheirorganizations.

Whilesatisfactionisimprovingoverall,respondentsarehavingproblemswithanalysisandreporting.

TheirbiggestproblemismanaginglogsfromWindowssystemsaprettybigproblembecauseWin-

dowsoperatingsystemsaresopervasive.Inboththe2010and2011surveys,userspointtoWindows

logcollectionproblemsandmessagesthataredifculttoanalyze.It wouldbenicetoseeMicrosoft

includenativesyslogcapabilitiesfortheiroperatingsystemsandsoftware.Logmanagementvendors

needtocontinueworkingtosolvetheproblem,andmanyarealreadymakingheadway.ITdepartments

alsoneedtodevelopinternalresourcestostudylogdataandlearnwhateventsmean.Thiswilltakecommitment,buttherewardswillbeincreasedproductivity,complianceandsecurity.

Summary


17/18


Jerry ShenkcurrentlyservesasasenioranalystfortheSANSInstituteandisseniorsecurityanalyst

forWindstreamCommunicationsinEphrata,PA.Since1984,hehasconsultedwithcompaniesand

nancialandeducationalinstitutionsonissuesofnetworkdesign,security,forensicanalysisandpen-

etrationtesting.Hisexperiencespanssmallhome-ofcesystemstoglobalnetworks.Alongwithsome

vendor-speciccertications,JerryholdssixGIACcertications,allcompletedwithhonors:GCIA,GCIH,

GCFW,GSNA,GPENandGCFA.FiveofhiscerticationsareGOLDcertications.

About the Author


18/18


SANS would like to thank its sponsor:

Documents

1 13763 ArcSight6-2011 LogMgt Survey