2
Where I Want to be.....
+
3
What I'll Settle For.....
4
Requirements
Rock solid stability Automated node addition (discovery) Scales horizontally Service dependency models Easy to write plugins Promotes sane workflows Unified front end view Flexible configuration
5
Tool Stack
6
What????but #monitoringsucks and #ihatenagios
How could you?
7
In defense of Nagios
Been around since 1996 Has Service dependencies Easy to write plugins Easy-ish to troubleshoot ROCK SOLID
8
Valid attacks on Nagios
No automated discovery It's complicated to setup Text files – really? Front end won't win any beauty contests Development is slow Stats collection is a PITA
9
Solutions Use Icinga! Use Puppet to auto configure Stats – leave it to graphite. It's really good at that Big boys and girls learn their tools
10
Icinga Fork of Nagios Configurations are compatible More solid architecture ( core, API, Web, IDODB ) Nice front end, nice mobile front end Can use NRPE
11
High Level View
12
Configure Icinga Servers using Puppet Standard Types
13
Things to configure with Standard Types
icinga.cfg (file) => icinga main config file Apache icinga.conf (file) => http access to each server cgiauth.cfg (file) => cgi access cgi.cfg (file) => options, users templates.cfg (file) got lazy => use for basic classes idomod.cfg (template) => template for hostname to DB
14
Configure Icinga using Nagios Types
15
Puppet Nagios Types
nagios_command nagios_contact nagios_contactgroup nagios_host nagios_hostdependency nagios_hostescalation nagios_hostextinfo
nagios_hostgroup nagios_service nagios_servicedependency nagios_serviceescalation nagios_serviceextinfo nagios_servicegroup nagios_timeperiod
16
Configuring Hosts
17
Overview
18
Detailed Overview
19
Store Configs
Store puppet info in a DB Retrieve information from
DB Share info across nodes Use thin_storeconfigs Set up on puppet master
20
Exporting Nagios_host Resources
Export = Save to DB Use facter for dynamic data PRO TIP: use ENC PRO TIP: use targets PRO TIP: hostgroups PRO TIP: use tags
21
PRO TIP: Use your ENC
22
PRO TIP: use targets
Use cfg_dir in icinga.cfg Create a unique file per host
or service Addition and removal are
now super easy Also default dirs are in a
horrible place /etc/nagios
23
PRO TIP: hostgroups Add machines to a
hostgroup Add services to a hostgroup New machines inherit all of
the services associated with a hostgroup
24
PRO TIP: use tags
Tags allow you to filter resources so that you only realize those resources that you need
25
Configuring Services/Commands
26
Icinga Services OR 'Stuff I want to monitor' Associate with a hostgroup Use a target
27
Icinga Commands
OR 'What actually gets run'
Use Macros to set paths in resource.cfg
28
Dependencies
29
PRO TIP: Dependencies
Unreliable services Cut down on the number of alerts Tell me what's really wrong Route alerts accordingly
30
Nagios_servicedependencies
31
NRPE
32
NRPE
Runs on client Secured via SSL Has ACLs Runs as nobody Can run commands Useful for other things...
33
Configuring NRPE
34
NRPE Checks
35
Plugins
36
exchange.nagios.org
37
Writing Plugins
Write in any language Output 1 line to stdout NRPE/Icinga/Nagios all use exit
codes to determine status Run by hand to check
38
Workflows
39
Watching Monitoring
40
Scheduling Downtime
41
Filtering
42
Alerting
43
#monitoringisawesome
REMOVE unreliable checks Just MONITOR – don't bolt on - especially stats TIER your monitoring Use timeperiods for sanity Delegate responses Use dependencies to pin down problems quickly Work smart
44
45
Resources Icinga
http://icinga.org
Puppet
http://docs.puppetlabs.com/references/latest/type.html#nagioscommand
NRPE
http://nagios.sourceforge.net/docs/3_0/addons.html
IRC
##infra-talk, #icinga, #puppet
Contact
[email protected], @maguec, #gaijin (freenode), http://blog.mague.com
Thanks
Yvonne Kong, Michael Catlin, Juan Ortega, Anthony Kong, Puppet Labs, Icinga Team
Recommended
