Nagios Conference 2013 - Mike Weber - SNMP Extensions

  • Upload
    nagios

  • View
    1.083

  • Download
    4

Embed Size (px)

DESCRIPTION

Mike Weber's presentation on Mike Weber. The presentation was given during the Nagios World Conference North America held Sept 20-Oct 2nd, 2013 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna

Citation preview

2. 2013 2 SNMP: 9000 Puzzle Pieces 3. 2013 3 Building the SNMP Puzzle Opening the Box * dumping the pieces * stand up box to see the goal Random Puzzle Pieces * examining the connections * searching for straight pieces * organizing the colors Fitting the Puzzle Pieces Together * clicking the pieces together * making sense of the mess 4. 2013 4 SNMP: 9000 Puzzle Pieces 5. 2013 5 SNMP: The Challenging Puzzle Actual Quote From Amazon.com I have to be truthful. I have not finished the 9000 piece Ravensburger Puzzle entitled Underwater Paradise. Despite my valiant attempt at this immense and challenging endeavor, I simply don't think that I'm smart enough or patient enough to do so. I have been bested by a puzzle and I will readily admit it to anyone that asks. 6. 2013 6 SNMP extend Scripts extend dfcheck /bin/dfh extend http_event"/bin/sh/usr/local/nagios/libexec/http_event" extendprovidesmultilineoutputfromthecommandandisindicatedbyanamenota number Creating New Options with SNMP Edit /etc/snmp/snmpd.conf 7. 2013 7 SNMP extend Scripts: Windows It is possible to edit the Windows registry to allow SNMP Extensions. http://support.microsoft.com/kb/128729 Possible but not probable. 8. 2013 8 SNMP Monitoring: Limitations 9. 2013 9 SNMP Data Collection: Static Data snmpwalk -v2c -c public 192.168.5.190 hrSWRunName.9909 HOST-RESOURCES-MIB::hrSWRunName.9909 = STRING: "httpd" Dynamic Data snmpget -v2c -c public 192.168.5.190 sysUpTimeInstance DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (574870) 1:35:48.70 10. 2013 10 Solution: NRPE or NRDS 11. 2013 11 SNMP Monitoring: No Limitations 12. 2013 12 SNMP Customized Data Collection: Static Data snmpwalk -v2c -c public 192.168.5.190 hrSWRunName.9909 HOST-RESOURCES-MIB::hrSWRunName.9909 = STRING: "httpd" Dynamic Data snmpget -v2c -c public 192.168.5.190 sysUpTimeInstance DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (574870) 1:35:48.70 Customized Data: Executing Commands/Scripts extenddfcheck/bin/dfh Creatingcommandsandarguments Leverage Nagios Plugins Reactions to Events extendhttp_event/bin/sh /usr/local/nagios/libexec/http_event 13. 2013 13 Why Make the Effort? Complete Flexibility for Monitoring capture data capture data created leverage Nagios plugins and scripts Ability to Create Responses to Server Situations * event handlers * security responses Total SNMP Server Monitoring * avoid other methods of monitoring Using check_snmp a Compiled Plugin * saves on resources 14. 2013 14 Compiled Plugin Savings!!! Compiled NSCA NSClient++ SSH Perl 0 2 4 6 8 10 12 RAM 15. 2013 15 SNMP Extensions: C Modules SNMP agent for PostgresSQL dlmod Modules mod-apache-snmp, HP Insight Management Agents, Software RAID AgentX VMWare ESX Server SNMP Agent, Net-SNMP Subagent for NFSv3 Perl sub-agent MySQL Monitoring,Bind9 Statistics, APC UPS Monitoring pass Scripts Bacula Subagent exec/extend Scripts Bind9 Statistics, Linux Disk IO, Extending Net-SNMP Proxy Squid SNMP Support 16. 2013 16 Standard MIBs AGENTXMIB.txtIPV6TC.txtSNMPNOTIFICATIONMIB.txt DISMANEVENTMIB.txtIPV6UDPMIB.txtSNMPPROXYMIB.txt DISMANSCHEDULEMIB.txtLMSENSORSMIB.txtSNMPTARGETMIB.txt DISMANSCRIPTMIB.txtMTAMIB.txtSNMPUSERBASEDSMMIB.txt EtherLikeMIB.txtNETSNMPAGENTMIB.txtSNMPUSMAESMIB.txt HCNUMTC.txtNETSNMPEXAMPLESMIB.txtSNMPUSMDHOBJECTSMIB.txt HOSTRESOURCESMIB.txtNETSNMPEXTENDMIB.txtSNMPv2CONF.txt HOSTRESOURCESTYPES.txtNETSNMPMIB.txtSNMPv2MIB.txt IANAADDRESSFAMILYNUMBERSMIB.txtNETSNMPTC.txtSNMPv2SMI.txt IANAifTypeMIB.txtNETWORKSERVICESMIB.txtSNMPv2TC.txt IANALANGUAGEMIB.txtNOTIFICATIONLOGMIB.txtSNMPv2TM.txt IANARTPROTOMIB.txtRFC1155SMI.txtSNMPVIEWBASEDACMMIB.txt IFINVERTEDSTACKMIB.txtRFC1213MIB.txtTCPMIB.txt IFMIB.txtRFC1215.txtTRANSPORTADDRESSMIB.txt INETADDRESSMIB.txtRMONMIB.txtUCDDEMOMIB.txt IPFORWARDMIB.txtSCTPMIB.txtUCDDISKIOMIB.txt IPMIB.txtSMUXMIB.txtUCDDLMODMIB.txt IPV6ICMPMIB.txtSNMPCOMMUNITYMIB.txtUCDIPFWACCMIB.txt IPV6MIB.txtSNMPFRAMEWORKMIB.txtUCDSNMPMIB.txt IPV6TCPMIB.txtSNMPMPDMIB.txtUDPMIB.txt 17. 2013 17 netSnmpExtendMIB HereisthetreeofthenetSnmpExtendMib 1.3.6.1.4.1.8072.1.3.1 1 = iso 3 = org 6 = dod 1 = internet 4 = private 1 = enterprises 8072 = netSNMP 1 = nsExtensions 3 = nsSnmpExtendMIB 18. 2013 18 Extend MIB netSnmpExtendMIB 1.3.6.1.4.1.8072.1.3.1 nsExtendObjects 1.3.6.1.4.1.8072.1.3.2 nsExtendNumEntries 1.3.6.1.4.1.8072.1.3.2.1 nsExtendConfigTable 1.3.6.1.4.1.8072.1.3.2.2 nsExtendConfigEntry 1.3.6.1.4.1.8072.1.3.2.2.1 nsExtendToken 1.3.6.1.4.1.8072.1.3.2.2.1.1 nsExtendCommand 1.3.6.1.4.1.8072.1.3.2.2.1.2 nsExtendStorage 1.3.6.1.4.1.8072.1.3.2.2.1.20 nsExtendStatus 1.3.6.1.4.1.8072.1.3.2.2.1.21 nsExtendArgs 1.3.6.1.4.1.8072.1.3.2.2.1.3 nsExtendInput 1.3.6.1.4.1.8072.1.3.2.2.1.4 nsExtendCacheTime 1.3.6.1.4.1.8072.1.3.2.2.1.5 nsExtendExecType 1.3.6.1.4.1.8072.1.3.2.2.1.6 nsExtendRunType 1.3.6.1.4.1.8072.1.3.2.2.1.7 nsExtendOutput1Table 1.3.6.1.4.1.8072.1.3.2.3 nsExtendOutput1Entry 1.3.6.1.4.1.8072.1.3.2.3.1 nsExtendOutput1Line 1.3.6.1.4.1.8072.1.3.2.3.1.1 nsExtendOutputFull 1.3.6.1.4.1.8072.1.3.2.3.1.2 nsExtendOutNumLines 1.3.6.1.4.1.8072.1.3.2.3.1.3 nsExtendResult 1.3.6.1.4.1.8072.1.3.2.3.1.4 nsExtendOutput2Table 1.3.6.1.4.1.8072.1.3.2.4 nsExtendOutput2Entry 1.3.6.1.4.1.8072.1.3.2.4.1 nsExtendLineIndex 1.3.6.1.4.1.8072.1.3.2.4.1.1 nsExtendOutLine 1.3.6.1.4.1.8072.1.3.2.4.1.2 nsExtendGroups 1.3.6.1.4.1.8072.1.3.3 nsExtendConfigGroup 1.3.6.1.4.1.8072.1.3.3.1 nsExtendOutputGroup 1.3.6.1.4.1.8072.1.3.3.2 19. 2013 19 Extend MIB snmpwalkv1cpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2 NETSNMPEXTENDMIB::nsExtendNumEntries.0=INTEGER:2 NETSNMPEXTENDMIB::nsExtendCommand."dfcheck"=STRING:/bin/df NETSNMPEXTENDMIB::nsExtendCommand."http_event"=STRING:/bin/sh/usr/local/nagios/libexec/http_event NETSNMPEXTENDMIB::nsExtendArgs."dfcheck"=STRING:h NETSNMPEXTENDMIB::nsExtendArgs."http_event"=STRING: NETSNMPEXTENDMIB::nsExtendInput."dfcheck"=STRING: NETSNMPEXTENDMIB::nsExtendInput."http_event"=STRING: NETSNMPEXTENDMIB::nsExtendCacheTime."dfcheck"=INTEGER:5 NETSNMPEXTENDMIB::nsExtendCacheTime."http_event"=INTEGER:5 NETSNMPEXTENDMIB::nsExtendExecType."dfcheck"=INTEGER:exec(1) NETSNMPEXTENDMIB::nsExtendExecType."http_event"=INTEGER:exec(1) NETSNMPEXTENDMIB::nsExtendRunType."dfcheck"=INTEGER:runonread(1) NETSNMPEXTENDMIB::nsExtendRunType."http_event"=INTEGER:runonread(1) NETSNMPEXTENDMIB::nsExtendStorage."dfcheck"=INTEGER:permanent(4) NETSNMPEXTENDMIB::nsExtendStorage."http_event"=INTEGER:permanent(4) NETSNMPEXTENDMIB::nsExtendStatus."dfcheck"=INTEGER:active(1) NETSNMPEXTENDMIB::nsExtendStatus."http_event"=INTEGER:active(1) NETSNMPEXTENDMIB::nsExtendOutput1Line."dfcheck"=STRING:FilesystemSizeUsedAvailUse%Mountedon NETSNMPEXTENDMIB::nsExtendOutput1Line."http_event"=STRING:Stoppinghttpd:[OK] NETSNMPEXTENDMIB::nsExtendOutputFull."dfcheck"=STRING:FilesystemSizeUsedAvailUse%Mountedon /dev/simfs4.0G3.1G844M79%/ none303M4.0K303M1%/dev NETSNMPEXTENDMIB::nsExtendOutputFull."http_event"=STRING:Stoppinghttpd:[OK] Startinghttpd:httpd:Couldnotreliablydeterminetheserver'sfullyqualifieddomainname,using192.168.5.190for ServerName [OK] NETSNMPEXTENDMIB::nsExtendOutNumLines."dfcheck"=INTEGER:3 NETSNMPEXTENDMIB::nsExtendOutNumLines."http_event"=INTEGER:3 NETSNMPEXTENDMIB::nsExtendResult."dfcheck"=INTEGER:0 NETSNMPEXTENDMIB::nsExtendResult."http_event"=INTEGER:0 NETSNMPEXTENDMIB::nsExtendOutLine."dfcheck".1=STRING:FilesystemSizeUsedAvailUse%Mountedon NETSNMPEXTENDMIB::nsExtendOutLine."dfcheck".2=STRING:/dev/simfs4.0G3.1G844M79%/ NETSNMPEXTENDMIB::nsExtendOutLine."dfcheck".3=STRING:none303M4.0K303M1%/dev NETSNMPEXTENDMIB::nsExtendOutLine."http_event".1=STRING:Stoppinghttpd:[OK] NETSNMPEXTENDMIB::nsExtendOutLine."http_event".2=STRING:Startinghttpd:httpd:Couldnotreliablydeterminethe server'sfullyqualifieddomainname,using192.168.5.190forServerName NETSNMPEXTENDMIB::nsExtendOutLine."http_event".3=STRING:[OK] 20. 2013 20 Convert to Numerical Values snmpwalkv1cpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2On .1.3.6.1.4.1.8072.1.3.2.1.0=INTEGER:2 .1.3.6.1.4.1.8072.1.3.2.2.1.2.7.100.102.99.104.101.99.107=STRING:/bin/df .1.3.6.1.4.1.8072.1.3.2.2.1.2.10.104.116.116.112.95.101.118.101.110.116=STRING:/bin/sh /usr/local/nagios/libexec/http_event .1.3.6.1.4.1.8072.1.3.2.2.1.3.7.100.102.99.104.101.99.107=STRING:h .1.3.6.1.4.1.8072.1.3.2.2.1.3.10.104.116.116.112.95.101.118.101.110.116=STRING: .1.3.6.1.4.1.8072.1.3.2.2.1.4.7.100.102.99.104.101.99.107=STRING: .1.3.6.1.4.1.8072.1.3.2.2.1.4.10.104.116.116.112.95.101.118.101.110.116=STRING: .1.3.6.1.4.1.8072.1.3.2.2.1.5.7.100.102.99.104.101.99.107=INTEGER:5 .1.3.6.1.4.1.8072.1.3.2.2.1.5.10.104.116.116.112.95.101.118.101.110.116=INTEGER:5 .1.3.6.1.4.1.8072.1.3.2.2.1.6.7.100.102.99.104.101.99.107=INTEGER:exec(1) .1.3.6.1.4.1.8072.1.3.2.2.1.6.10.104.116.116.112.95.101.118.101.110.116=INTEGER:exec(1) .1.3.6.1.4.1.8072.1.3.2.2.1.7.7.100.102.99.104.101.99.107=INTEGER:runonread(1) .1.3.6.1.4.1.8072.1.3.2.2.1.7.10.104.116.116.112.95.101.118.101.110.116=INTEGER:runonread(1) .1.3.6.1.4.1.8072.1.3.2.2.1.20.7.100.102.99.104.101.99.107=INTEGER:permanent(4) .1.3.6.1.4.1.8072.1.3.2.2.1.20.10.104.116.116.112.95.101.118.101.110.116=INTEGER:permanent(4) .1.3.6.1.4.1.8072.1.3.2.2.1.21.7.100.102.99.104.101.99.107=INTEGER:active(1) .1.3.6.1.4.1.8072.1.3.2.2.1.21.10.104.116.116.112.95.101.118.101.110.116=INTEGER:active(1) .1.3.6.1.4.1.8072.1.3.2.3.1.1.7.100.102.99.104.101.99.107=STRING:FilesystemSizeUsedAvailUse%Mountedon .1.3.6.1.4.1.8072.1.3.2.3.1.1.10.104.116.116.112.95.101.118.101.110.116=STRING:Stoppinghttpd:[OK] .1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107=STRING:FilesystemSizeUsedAvailUse%Mountedon /dev/simfs4.0G3.1G843M79%/ none303M4.0K303M1%/dev .1.3.6.1.4.1.8072.1.3.2.3.1.2.10.104.116.116.112.95.101.118.101.110.116=STRING:Stoppinghttpd:[OK] Startinghttpd:httpd:Couldnotreliablydeterminetheserver'sfullyqualifieddomainname,using192.168.5.190for ServerName [OK] .1.3.6.1.4.1.8072.1.3.2.3.1.3.7.100.102.99.104.101.99.107=INTEGER:3 .1.3.6.1.4.1.8072.1.3.2.3.1.3.10.104.116.116.112.95.101.118.101.110.116=INTEGER:3 .1.3.6.1.4.1.8072.1.3.2.3.1.4.7.100.102.99.104.101.99.107=INTEGER:0 .1.3.6.1.4.1.8072.1.3.2.3.1.4.10.104.116.116.112.95.101.118.101.110.116=INTEGER:0 .1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.1=STRING:FilesystemSizeUsedAvailUse%Mountedon .1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.2=STRING:/dev/simfs4.0G3.1G843M79%/ .1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.3=STRING:none303M4.0K303M1%/dev .1.3.6.1.4.1.8072.1.3.2.4.1.2.10.104.116.116.112.95.101.118.101.110.116.1=STRING:Stoppinghttpd:[OK] .1.3.6.1.4.1.8072.1.3.2.4.1.2.10.104.116.116.112.95.101.118.101.110.116.2=STRING:Startinghttpd:httpd:Couldnot reliablydeterminetheserver'sfullyqualifieddomainname,using192.168.5.190forServerName .1.3.6.1.4.1.8072.1.3.2.4.1.2.10.104.116.116.112.95.101.118.101.110.116.3=STRING:[OK] 21. 2013 21 SNMP Extend: Puzzle Pieces 22. 2013 22 nsExtendNumEntries nsExtendNumEntries 1.3.6.1.4.1.8072.1.3.2.1 ThiswillbeanintegerindicatingthenumberofrowsinthensExtendConfigTable.Thisis thenumberofextendcommandsthatareavailable.Withthreeextendcommandsyouwillgeta returnof"3". snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.1 NETSNMPEXTENDMIB::nsExtendNumEntries.0=INTEGER:3 Hereiscurrentlistofextendscriptsfoundinthe/etc/snmp/snmpd.conf. extendhttp_event"/bin/sh/usr/local/nagios/libexec/http_event" extenddfcheck/bin/dfh extendwho/usr/bin/who 23. 2013 23 nsExtendCommand nsExtendCommand1.3.6.1.4.1.8072.1.3.2.2.1.2 Thefullpathofthecommandtorun. snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.2 NETSNMPEXTENDMIB::nsExtendCommand."who"=STRING:/usr/bin/who NETSNMPEXTENDMIB::nsExtendCommand."dfcheck"=STRING:/bin/df NETSNMPEXTENDMIB::nsExtendCommand."http_event"=STRING:/bin/sh /usr/local/nagios/libexec/http_event /etc/snmp/snmpd.conf extendhttp_event"/bin/sh/usr/local/nagios/libexec/http_event" extenddfcheck/bin/dfh extendwho/usr/bin/who 24. 2013 24 nsExtendStatus nsExtendStatus 1.3.6.1.4.1.8072.1.3.2.2.1.21 snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.21 NETSNMPEXTENDMIB::nsExtendStatus."who"=INTEGER:active(1) NETSNMPEXTENDMIB::nsExtendStatus."dfcheck"=INTEGER:active(1) NETSNMPEXTENDMIB::nsExtendStatus."http_event"=INTEGER:active(1) 25. 2013 25 nsExtendArgs nsExtendArgs 1.3.6.1.4.1.8072.1.3.2.2.1.3 Commandlineargumentsforthecommand. snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.3 NETSNMPEXTENDMIB::nsExtendArgs."who"=STRING: NETSNMPEXTENDMIB::nsExtendArgs."dfcheck"=STRING:h NETSNMPEXTENDMIB::nsExtendArgs."http_event"=STRING: /etc/snmp/snmpd.conf extendhttp_event"/bin/sh/usr/local/nagios/libexec/http_event" extenddfcheck/bin/dfh extendwho/usr/bin/who 26. 2013 26 nsExtendCacheTime nsExtendCacheTime 1.3.6.1.4.1.8072.1.3.2.2.1.5 Thelengthoftimetheoutputforthecommandwillbecached.Ifanadditionalcommandis receivedduringthecachetime,thecommandwillnotbereruntoobtainthevalue.Ifthe valueof1isusedthecachewillnotbeused. snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.5 NETSNMPEXTENDMIB::nsExtendCacheTime."who"=INTEGER:5 NETSNMPEXTENDMIB::nsExtendCacheTime."dfcheck"=INTEGER:5 NETSNMPEXTENDMIB::nsExtendCacheTime."http_event"=INTEGER:5 27. 2013 27 nsExtendCacheTime nsExtendCacheTime 1.3.6.1.4.1.8072.1.3.2.2.1.5 Extendcachetime snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.5 NETSNMPEXTENDMIB::nsExtendCacheTime."who"=INTEGER:5 NETSNMPEXTENDMIB::nsExtendCacheTime."dfcheck"=INTEGER:5 NETSNMPEXTENDMIB::nsExtendCacheTime."http_event"=INTEGER:5 snmpsetv2ccnagios192.168.5.190'NETSNMPEXTENDMIB::nsExtendCacheTime."dfcheck"'i20 snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.5 NETSNMPEXTENDMIB::nsExtendCacheTime."who"=INTEGER:5 NETSNMPEXTENDMIB::nsExtendCacheTime."dfcheck"=INTEGER:20 NETSNMPEXTENDMIB::nsExtendCacheTime."http_event"=INTEGER:5 Thevalueisresettothedefaultwhensnmpdisrestarted. 28. 2013 28 nsExtendExecType nsExtendExecType1.3.6.1.4.1.8072.1.3.2.2.1.6 Therearetwooptions;execandshell.Thisisthemechanismusedtoinvokethecommand. snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.6 NETSNMPEXTENDMIB::nsExtendExecType."who"=INTEGER:exec(1) NETSNMPEXTENDMIB::nsExtendExecType."dfcheck"=INTEGER:exec(1) NETSNMPEXTENDMIB::nsExtendExecType."http_event"=INTEGER:exec(1) 29. 2013 29 nsExtendRunType nsExtendRunType 1.3.6.1.4.1.8072.1.3.2.2.1.7 Therearetwooptionshere;runonreadwhichimmediatelyinvokesthecommandorrunonset whichwillonlybeinvokedonthereceiptofaSETassignment. snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.2.1.7 NETSNMPEXTENDMIB::nsExtendRunType."who"=INTEGER:runonread(1) NETSNMPEXTENDMIB::nsExtendRunType."dfcheck"=INTEGER:runonread(1) NETSNMPEXTENDMIB::nsExtendRunType."http_event"=INTEGER:runonread(1) 30. 2013 30 nsExtendOutputFull nsExtendOutputFull 1.3.6.1.4.1.8072.1.3.2.3.1.2 Fulloutputofcommandasastring. snmpwalkv2ccpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.3 NETSNMPEXTENDMIB::nsExtendOutput1Line."who"=STRING:rootpts/020130831 23:19(192.168.5.103) NETSNMPEXTENDMIB::nsExtendOutput1Line."dfcheck"=STRING:FilesystemSize UsedAvailUse%Mountedon NETSNMPEXTENDMIB::nsExtendOutput1Line."http_event"=STRING:Stoppinghttpd:[OK] NETSNMPEXTENDMIB::nsExtendOutputFull."who"=STRING:rootpts/020130831 23:19(192.168.5.103) NETSNMPEXTENDMIB::nsExtendOutputFull."dfcheck"=STRING:FilesystemSizeUsed AvailUse%Mountedon /dev/simfs4.0G3.4G526M87%/ none303M4.0K303M1%/dev NETSNMPEXTENDMIB::nsExtendOutputFull."http_event"=STRING:Stoppinghttpd:[OK] Startinghttpd:httpd:Couldnotreliablydeterminetheserver'sfullyqualifieddomain name,using192.168.5.190forServerName [OK] NETSNMPEXTENDMIB::nsExtendOutNumLines."who"=INTEGER:1 NETSNMPEXTENDMIB::nsExtendOutNumLines."dfcheck"=INTEGER:3 NETSNMPEXTENDMIB::nsExtendOutNumLines."http_event"=INTEGER:3 NETSNMPEXTENDMIB::nsExtendResult."who"=INTEGER:0 NETSNMPEXTENDMIB::nsExtendResult."dfcheck"=INTEGER:0 NETSNMPEXTENDMIB::nsExtendResult."http_event"=INTEGER:0 31. 2013 31 Recognizing Command OIDs: 107 Hereisanextendscriptfoundinthe/etc/snmp/snmpd.conf. extenddfcheck/bin/dfh .1.3.6.1.4.1.8072.1.3.2.2.1.2.7.100.102.99.104.101.99.107=STRING:/bin/df .1.3.6.1.4.1.8072.1.3.2.2.1.3.7.100.102.99.104.101.99.107=STRING:h .1.3.6.1.4.1.8072.1.3.2.2.1.4.7.100.102.99.104.101.99.107=STRING: .1.3.6.1.4.1.8072.1.3.2.2.1.5.7.100.102.99.104.101.99.107=INTEGER:5 .1.3.6.1.4.1.8072.1.3.2.2.1.6.7.100.102.99.104.101.99.107=INTEGER:exec(1) .1.3.6.1.4.1.8072.1.3.2.2.1.7.7.100.102.99.104.101.99.107=INTEGER:runonread(1) .1.3.6.1.4.1.8072.1.3.2.2.1.20.7.100.102.99.104.101.99.107=INTEGER:permanent(4) .1.3.6.1.4.1.8072.1.3.2.2.1.21.7.100.102.99.104.101.99.107=INTEGER:active(1) .1.3.6.1.4.1.8072.1.3.2.3.1.1.7.100.102.99.104.101.99.107=STRING:FilesystemSizeUsedAvailUse%Mountedon .1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107=STRING:FilesystemSizeUsedAvailUse%Mountedon /dev/simfs4.0G3.1G843M79%/ none303M4.0K303M1%/dev .1.3.6.1.4.1.8072.1.3.2.3.1.3.7.100.102.99.104.101.99.107=INTEGER:3 .1.3.6.1.4.1.8072.1.3.2.3.1.4.7.100.102.99.104.101.99.107=INTEGER:0 .1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.1=STRING:FilesystemSizeUsedAvailUse%Mountedon .1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.2=STRING:/dev/simfs4.0G3.1G843M79%/ .1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.3=STRING:none303M4.0K303M1%/dev 32. 2013 32 Recognizing Command OIDs: 116 Hereisanextendscriptfoundinthe/etc/snmp/snmpd.conf. extendhttp_event"/bin/sh/usr/local/nagios/libexec/http_event" .1.3.6.1.4.1.8072.1.3.2.2.1.2.10.104.116.116.112.95.101.118.101.110.116=STRING:/bin/sh /usr/local/nagios/libexec/http_event .1.3.6.1.4.1.8072.1.3.2.2.1.3.10.104.116.116.112.95.101.118.101.110.116=STRING: .1.3.6.1.4.1.8072.1.3.2.2.1.4.10.104.116.116.112.95.101.118.101.110.116=STRING: .1.3.6.1.4.1.8072.1.3.2.2.1.5.10.104.116.116.112.95.101.118.101.110.116=INTEGER:5 .1.3.6.1.4.1.8072.1.3.2.2.1.6.10.104.116.116.112.95.101.118.101.110.116=INTEGER:exec(1) .1.3.6.1.4.1.8072.1.3.2.2.1.7.10.104.116.116.112.95.101.118.101.110.116=INTEGER:runonread(1) .1.3.6.1.4.1.8072.1.3.2.2.1.20.10.104.116.116.112.95.101.118.101.110.116=INTEGER:permanent(4) .1.3.6.1.4.1.8072.1.3.2.2.1.21.10.104.116.116.112.95.101.118.101.110.116=INTEGER:active(1) .1.3.6.1.4.1.8072.1.3.2.3.1.1.10.104.116.116.112.95.101.118.101.110.116=STRING:Stoppinghttpd:[OK] .1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107=STRING:FilesystemSizeUsedAvailUse%Mountedon /dev/simfs4.0G3.1G843M79%/ none303M4.0K303M1%/dev .1.3.6.1.4.1.8072.1.3.2.3.1.2.10.104.116.116.112.95.101.118.101.110.116=STRING:Stoppinghttpd:[OK] Startinghttpd:httpd:Couldnotreliablydeterminetheserver'sfullyqualifieddomainname,using192.168.5.190for ServerName [OK] .1.3.6.1.4.1.8072.1.3.2.3.1.3.10.104.116.116.112.95.101.118.101.110.116=INTEGER:3 .1.3.6.1.4.1.8072.1.3.2.3.1.4.10.104.116.116.112.95.101.118.101.110.116=INTEGER:0 .1.3.6.1.4.1.8072.1.3.2.4.1.2.10.104.116.116.112.95.101.118.101.110.116.1=STRING:Stoppinghttpd:[OK] .1.3.6.1.4.1.8072.1.3.2.4.1.2.10.104.116.116.112.95.101.118.101.110.116.2=STRING:Startinghttpd:httpd:Couldnot reliablydeterminetheserver'sfullyqualifieddomainname,using192.168.5.190forServerName .1.3.6.1.4.1.8072.1.3.2.4.1.2.10.104.116.116.112.95.101.118.101.110.116.3=STRING:[OK] 33. 2013 33 Multiple Systems: OIDs the Same ItisimportantthattheOIDsarethesame,ifthecommandisexactlythesame,on multiplemachines. CentOS6 .1.3.6.1.4.1.8072.1.3.2.2.1.2.7.100.102.99.104.101.99.107=STRING:/bin/df .1.3.6.1.4.1.8072.1.3.2.2.1.2.10.104.116.116.112.95.101.118.101.110.116=STRING: /bin/sh/usr/local/nagios/libexec/http_event Ubuntu12.04 .1.3.6.1.4.1.8072.1.3.2.2.1.2.7.100.102.99.104.101.99.107=STRING:/bin/df .1.3.6.1.4.1.8072.1.3.2.2.1.2.10.104.116.116.112.95.101.118.101.110.116=STRING: /bin/sh/usr/local/nagios/libexec/http_event 34. 2013 34 OIDs May Produce Different Outcome InthisexampletheOIDisthesame,howevertheoutputisdifferentbecausethe filesystemisdifferent. CentOS6.3 .1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107=STRING:Filesystem SizeUsedAvailUse%Mountedon /dev/simfs4.0G3.1G843M79%/ none303M4.0K303M1%/dev Ubuntu12.04 .1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107=STRING:Filesystem SizeUsedAvailUse%Mountedon /dev/vda20G1.3G18G7%/ none243M148K243M1%/dev none247M0247M0%/dev/shm none247M84K247M1%/var/run none247M0247M0%/var/lock none247M0247M0%/lib/init/rw none20G1.3G18G7%/var/lib/ureadahead/debugfs 35. Fitting the Pieces TogetherFitting the Pieces Together 36. 2013 36 Edit /etc/snmp/snmpd.conf extend dfcheck /bin/dfh extend http_event"/bin/sh/usr/local/nagios/libexec/http_event" 37. 2013 37 Example: extend dfcheck /bin/df -h ProblemDevelops: LimitingtoOIDThatListsPartitions snmpwalkv1cpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107 NETSNMPEXTENDMIB::nsExtendOutputFull."dfcheck"=STRING:FilesystemSizeUsedAvail Use%Mountedon /dev/simfs4.0G3.1G843M79%/ none303M4.0K303M1%/dev Limitwithgrepfor/Partition snmpwalkv1cpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107| grep"/$" /dev/simfs4.0G3.1G843M79%/ Problem:TestaPlugin check_snmpH192.168.5.190Cpublico.1.3.6.1.4.1.8072.1.3.2.3.1.2.7.100.102.99.104.101.99.107 SNMPOK"FilesystemSizeUsedAvailUse%Mountedon/dev/simfs4.0G3.1G 843M79%/ none303M4.0K303M1%/dev" 38. 2013 38 Creating the Service Check ScanthetreeforanOIDfortheCommand snmpwalkv1cpublic192.168.5.190.1.3.6.1.4.1.8072.1.3.2 .1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.2=STRING:/dev/simfs4.0G 3.1G843M79%/ TextStringforServiceCheck Cpublico.1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.2 39. 2013 39 XI Service Check 40. 2013 40 Core Service Check definecommand{ command_namecheck_snmp command_line$USER1$/check_snmpH$HOSTADDRESS$$ARG1$ } defineservice{ usegenericservice host_namecentos service_descriptionextend:/Partition check_commandcheck_snmp!Cpublico. 1.3.6.1.4.1.8072.1.3.2.4.1.2.7.100.102.99.104.101.99.107.2 } 41. 2013 41 XI Service Check: Results 42. 2013 42 SNMP: Event Handler 43. 2013 43 sudo: Event Handler extendhttp_event/bin/sh/usr/local/nagios/libexec/http_event sudoPermissionsfornagiosUser User_AliasNAGIOS=nagios,nagiocmd Cmnd_AliasNAGIOSCOM=/sbin/service,/etc/rc.d/init.d/httpd,/usr/sbin/httpd Defaults:NAGIOS!requiretty NAGIOSALL=(ALL)NOPASSWD:NAGIOSCOM Testsudocommands sunagios sudo/sbin/servicehttpdrestart 44. 2013 44 ACLs: Logs UsingACLs getfacl/var/log/messages #file:var/log/messages #owner:root #group:root user::rw group:: other:: setfaclmu:nagios:r/var/log/messages #file:var/log/messages #owner:root #group:root user::rw user:nagios:r group:: mask::r Other:: rwr+1rootroot464963Oct206:51/var/log/messages 45. 2013 45 Example: Event Handler ListtheOIDtoExecute snmpwalkv2c192.168.5.190cpublic1.3.6.1.4.1.8072.1.3.2.3.1.4 NETSNMPEXTENDMIB::nsExtendResult."http_event"=INTEGER:0 RetrievetheNumericalValues snmpwalkv2c192.168.5.190cpublic1.3.6.1.4.1.8072.1.3.2.3.1.4On .1.3.6.1.4.1.8072.1.3.2.3.1.4.10.104.116.116.112.95.101.118.101.110.116=INTEGER:0 SNMPExecutesthescriptwhentheextendname(http_event)iscalled 46. 2013 46 Example: Event Handler CreateaScriptonNagiostoMonitorandExecute #!/bin/sh host=$1 string="NAGIOSTESTPAGE" output=$(/usr/local/nagios/libexec/check_http$hosts$string|grep"HTTPOKHTTP/1.1 200OK"|wcl) if[$outputeq1] then echo"WebPageisOK" exit0 else echo"WebPageisBeingRestarted" snmpwalkv2c$hostcpublic. 1.3.6.1.4.1.8072.1.3.2.3.1.4.10.104.116.116.112.95.101.118.101.110.116 exit2 fi 47. 2013 47 Event Handler: Service Check 48. 2013 48 Event Handler: Service Check 49. Questions?Questions?