Friday, May 4, 2012

Monitoring tool - setup icinga-web with pnp4nagios

As introducing how to install Icinga and Icinga-web before, I am trying to install pnp4nagios and setup icinga-web with pnp4nagios. As Nagios and Icinga themselves have no graphing function, some may use Cacti or Munin for graphing performance data.  There's an addon called PNP4nagios released in Nagios Addon Project, which analyzes performance data provided by nagios-plugins and stores them automatically into RRD-databases.




Then, let's see how to install pnp4nagios and integrate it into Icinga-web. 

install pnp4nagios

  • install rrdtool, perl-rrdtool
# rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
# rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
# sed -i 's|enabled\s\?=\s\?1|enabled = 0|' /etc/yum.repos.d/rpmforge.repo
# yum -y install pango.x86_64 pango-devel.x86_64 glib2.x86_64 glib2-devel.x86_64
# yum -y --enablerepo=rpmforge install rrdtool perl-rrdtool  
  • install pnp4nagios
$ curl http://downloads.sourceforge.net/project/pnp4nagios/PNP-0.6/pnp4nagios-0.6.16.tar.gz | tar zx
$ cd pnp4nagios-0.6.16
./configure  \
--prefix=/usr/local/pnp4nagios-0.6.16  \
--with-nagios-user=icinga \
--with-nagios-group=icinga \
--with-rrdtool \
--with-httpd-conf=/usr/local/httpd/conf/extra \
--with-init-dir=/etc/init.d \
--with-perfdata-logfile=/var/log/icinga
$ make all
# make install install-webconf install-config install-init
# ln -s /usr/local/pnp4nagios-0.6.16 /usr/local/pnp4nagios


Before setting up the configuration, I'll explain the art of collecting data of pnp4nagios. Pnp4nagios has 5 kinds of ways to collect data, synchronous-mode, bulk mode, bulk mode with NPCD, bulk mode with npcdmod, and gearman mode. The detail about each way is shown, here. I adopt bulk mode with npcdmod, but I am also showing how to setup bulk mode with NPCD.

setup bulk mode with NPCD

  • setup configuration files
# cd /usr/local/pnp4nagios/etc/
# mv npcd.cfg-sample npcd.cfg
# mv process_perfdata.cfg-sample process_perfdata.cfg
# mv rra.cfg-sample rra.cfg
  • setup icinga.cfg
# cat >> /usr/local/icinga/etc/icinga.cfg << EOF
## Bulk / NPCD mode
process_performance_data=1
# service performance data
service_perfdata_file=/usr/local/pnp4nagios/var/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::\$TIMET\$\tHOSTNAME::\$HOSTNAME\$\tSERVICEDESC::\$SERVICEDESC\$\tSERVICEPERFDATA::\$SERVICEPERFDATA
\$\tSERVICECHECKCOMMAND::\$SERVICECHECKCOMMAND\$\tHOSTSTATE::\$HOSTSTATE\$\tHOSTSTATETYPE::\$HOSTSTATETYPE\$\tSERVICESTATE::\$SERVICESTATE\$\tSERVICESTATETYPE::\$SERVI
CESTATETYPE\$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=process-service-perfdata-file

# host performance data starting with Nagios 3.0
host_perfdata_file=/usr/local/pnp4nagios/var/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::\$TIMET\$\tHOSTNAME::\$HOSTNAME\$\tHOSTPERFDATA::\$HOSTPERFDATA\$\tHOSTCHECKCOMMAND::\$HOSTCHECKCOMMAND\$\
tHOSTSTATE::\$HOSTSTATE\$\tHOSTSTATETYPE::\$HOSTSTATETYPE\$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=process-host-perfdata-file
EOF
  • setup commands.cfg
# cat >> /usr/local/icinga/etc/objects/commands.cfg << EOF
## Bulk with NPCD mode
define command{
     command_name    process-service-perfdata-file
     command_line    /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.\$TIMET\$
}

define command{
     command_name    process-host-perfdata-file
     command_line    /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.\$TIMET\$
}
EOF
  • start apache, npcd, and icinga daemon
# /etc/init.d/httpd start
# /etc/init.d/npcd start
# /etc/init.d/icinga restart
  • check the configuration settings and performance data
    It passes without critical error.
# wget http://verify.pnp4nagios.org/verify_pnp_config
# perl verify_pnp_config -m bulk+npcd -c /usr/local/icinga/etc/icinga.cfg -p /usr/local/pnp4nagios/etc/
[INFO]  ========== Starting Environment Checks ============
[INFO]  My version is: verify_pnp_config-0.6.17-R.33
[INFO]  Reading /usr/local/icinga/etc/icinga.cfg
[OK  ]  Running product is 'icinga'
[OK  ]  object_cache_file is defined
[OK  ]  object_cache_file=/usr/local/icinga/var/objects.cache
[INFO]  Reading /usr/local/icinga/var/objects.cache
[OK  ]  resource_file is defined
[OK  ]  resource_file=/usr/local/icinga/etc/resource.cfg
[INFO]  Reading /usr/local/icinga/etc/resource.cfg
[INFO]  Reading /usr/local/pnp4nagios/etc//process_perfdata.cfg
[OK  ]  No pnp4nagios_release file found. This might be an older version of PNP4Nagios
[OK  ]  Effective User is 'icinga'
[OK  ]  User icinga exists with ID '1002'
[OK  ]  Effective group is 'icinga'
[OK  ]  Group icinga exists with ID '1002'
[INFO]  ========== Checking Bulk Mode + NPCD Config  ============
[OK  ]  process_performance_data is 1 compared with '/1/'
[OK  ]  service_perfdata_file is defined
[OK  ]  service_perfdata_file=/usr/local/pnp4nagios/var/service-perfdata
[OK  ]  service_perfdata_file_template is defined
[OK  ]  service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$
[OK  ]  PERFDATA template looks good
[OK  ]  service_perfdata_file_mode is defined
[OK  ]  service_perfdata_file_mode=a
[OK  ]  service_perfdata_file_processing_interval is defined
[OK  ]  service_perfdata_file_processing_interval=15
[OK  ]  service_perfdata_file_processing_command is defined
[OK  ]  service_perfdata_file_processing_command=process-service-perfdata-file
[OK  ]  host_perfdata_file is defined
[OK  ]  host_perfdata_file=/usr/local/pnp4nagios/var/host-perfdata
[OK  ]  host_perfdata_file_template is defined
[OK  ]  host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$
[OK  ]  PERFDATA template looks good
[OK  ]  host_perfdata_file_mode is defined
[OK  ]  host_perfdata_file_mode=a
[OK  ]  host_perfdata_file_processing_interval is defined
[OK  ]  host_perfdata_file_processing_interval=15
[OK  ]  host_perfdata_file_processing_command is defined
[OK  ]  host_perfdata_file_processing_command=process-host-perfdata-file
[INFO]  Icinga config looks good so far
[INFO]  ========== Checking config values ============
[OK  ]  npcd daemon is running
[OK  ]  /usr/local/pnp4nagios/etc/npcd.cfg is used by npcd and readable
[INFO]  Reading /usr/local/pnp4nagios/etc/npcd.cfg
[OK  ]  perfdata_spool_dir is defined
[OK  ]  perfdata_spool_dir=/usr/local/pnp4nagios/var/spool/
[OK  ]  Command process-service-perfdata-file is defined
[OK  ]  '/bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$'
[OK  ]  Command looks good
[OK  ]  Command process-host-perfdata-file is defined
[OK  ]  '/bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$'
[OK  ]  Command looks good
[OK  ]  Script /usr/local/pnp4nagios/libexec/process_perfdata.pl is executable
[INFO]  ========== Starting global checks ============
[OK  ]  status_file is defined
[OK  ]  status_file=/usr/local/icinga/var/status.dat
[INFO]  Reading /usr/local/icinga/var/status.dat
[INFO]  ==== Starting rrdtool checks ====
[OK  ]  RRDTOOL is defined
[OK  ]  RRDTOOL=/usr/bin/rrdtool
[OK  ]  /usr/bin/rrdtool is executable
[OK  ]  RRDtool 1.4.7  Copyright 1997-2012 by Tobias Oetiker <tobi@oetiker.ch>
[OK  ]  USE_RRDs is defined
[OK  ]  USE_RRDs=1
[WARN]  Perl RRDs modules are not loadable
[INFO]  ==== Starting directory checks ====
[OK  ]  RRDPATH is defined
[OK  ]  RRDPATH=/usr/local/pnp4nagios/var/perfdata
[OK  ]  Perfdata directory '/usr/local/pnp4nagios/var/perfdata' exists
[WARN]  7 hosts/services are not providing performance data
[WARN]  'process_perf_data 1' is set for 8 hosts/services which are not providing performance data!
[OK  ]  'process_perf_data 1' is set for 34 of your hosts/services
[INFO]  ==== System sizing ====
[OK  ]  33 hosts/service objects defined
[INFO]  ==== Check statistics ====
[WARN]  Warning: 3, Critical: 0
[WARN]  Checks finished...

setup bulk mode with npcdmod

  • setup configuration files 
# cd /usr/local/pnp4nagios/etc/
# mv npcd.cfg-sample npcd.cfg
# mv process_perfdata.cfg-sample process_perfdata.cfg
# mv rra.cfg-sample rra.cfg
  • setup icinga.cfg
# cat >> /usr/local/icinga/icinga.cfg << EOF
process_performance_data=1
broker_module=/usr/local/pnp4nagios/lib/npcdmod.o config_file=/usr/local/pnp4nagios/etc/npcd.cfg
EOF
  • start apache, npcd, and icinga
# /etc/init.d/httpd start
# /etc/init.d/npcd start
# /etc/init.d/icinga start
  • icinga.log after enabling the ndomod
# tail -f /var/log/icinga/icinga.log
[1335878074] Caught SIGTERM, shutting down...
[1335878074] Successfully shutdown... (PID=21017)
[1335878074] idomod: Shutdown complete.
[1335878074] Event broker module '/usr/local/icinga/bin/idomod.o' deinitialized successfully.
[1335878074] npcdmod: If you don't like me, I will go out! Bye.
[1335878074] Event broker module '/usr/local/pnp4nagios/lib/npcdmod.o' deinitialized successfully.
[1335878075] Icinga 1.6.1 starting... (PID=21253)
[1335878075] Local time is Tue May 01 22:14:35 JST 2012
[1335878075] LOG VERSION: 2.0
[1335878075] idomod: IDOMOD 1.6.1 (12-02-2011) Copyright(c) 2005-2008 Ethan Galstad, Copyright(c) 2009-2011 Icinga Development Team (https://www.icinga.org)
[1335878075] idomod: Successfully connected to data sink.  0 queued items to flush.
[1335878075] Event broker module '/usr/local/icinga/bin/idomod.o' initialized successfully.
[1335878075] npcdmod: Copyright (c) 2008-2009 Hendrik Baecker (andurin@process-zero.de) - http://www.pnp4nagios.org
[1335878075] npcdmod: /usr/local/pnp4nagios/etc/npcd.cfg initialized
[1335878075] npcdmod: spool_dir = '/usr/local/pnp4nagios/var/spool/'.
[1335878075] npcdmod: perfdata file '/usr/local/pnp4nagios/var/perfdata.dump'.
[1335878075] npcdmod: Ready to run to have some fun!
[1335878075] Event broker module '/usr/local/pnp4nagios/lib/npcdmod.o' initialized successfully.
[1335878075] Finished daemonizing... (New PID=21256)
[1335878076] Event loop started...
  • check the configuration settings and performance data
# wget http://verify.pnp4nagios.org/verify_pnp_config
# perl verify_pnp_config -m npcdmod -c /usr/local/icinga/etc/icinga.cfg -p /usr/local/pnp4nagios/etc/
[INFO]  ========== Starting Environment Checks ============
[INFO]  My version is: verify_pnp_config-0.6.17-R.33
[INFO]  Reading /usr/local/icinga/etc/icinga.cfg
[OK  ]  Running product is 'icinga'
[OK  ]  object_cache_file is defined
[OK  ]  object_cache_file=/usr/local/icinga/var/objects.cache
[INFO]  Reading /usr/local/icinga/var/objects.cache
[OK  ]  resource_file is defined
[OK  ]  resource_file=/usr/local/icinga/etc/resource.cfg
[INFO]  Reading /usr/local/icinga/etc/resource.cfg
[INFO]  Reading /usr/local/pnp4nagios/etc//process_perfdata.cfg
[OK  ]  No pnp4nagios_release file found. This might be an older version of PNP4Nagios
[OK  ]  Effective User is 'icinga'
[OK  ]  User icinga exists with ID '1002'
[OK  ]  Effective group is 'icinga'
[OK  ]  Group icinga exists with ID '1002'
[INFO]  ========== Checking npcdmod Mode Config  ============
[OK  ]  process_performance_data is 1 compared with '/1/'
[OK  ]  event_broker_options is defined
[OK  ]  event_broker_options=-1
[OK  ]  event_broker_option bits 2 and 3 enabled (12)
[OK  ]  broker_module is defined
[OK  ]  broker_module=/usr/local/pnp4nagios/lib/npcdmod.o config_file=/usr/local/pnp4nagios/etc/npcd.cfg
[OK  ]  npcdmod.o config file is /usr/local/pnp4nagios/etc/npcd.cfg
[OK  ]  /usr/local/pnp4nagios/etc/npcd.cfg used by npcdmod.o is readable
[OK  ]  npcd daemon is running
[OK  ]  /usr/local/pnp4nagios/etc/npcd.cfg is used by npcd and readable
[OK  ]  npcd and npcdmod.o are using the same config file (/usr/local/pnp4nagios/etc/npcd.cfg)
[INFO]  Icinga config looks good so far
[INFO]  ========== Checking config values ============
[INFO]  Reading /usr/local/pnp4nagios/etc/npcd.cfg
[OK  ]  Script /usr/local/pnp4nagios/libexec/process_perfdata.pl is executable
[INFO]  ========== Starting global checks ============
[OK  ]  status_file is defined
[OK  ]  status_file=/usr/local/icinga/var/status.dat
[INFO]  Reading /usr/local/icinga/var/status.dat
[INFO]  ==== Starting rrdtool checks ====
[OK  ]  RRDTOOL is defined
[OK  ]  RRDTOOL=/usr/bin/rrdtool
[OK  ]  /usr/bin/rrdtool is executable
[OK  ]  RRDtool 1.4.7  Copyright 1997-2012 by Tobias Oetiker <tobi@oetiker.ch>
[OK  ]  USE_RRDs is defined
[OK  ]  USE_RRDs=1
[WARN]  Perl RRDs modules are not loadable
[INFO]  ==== Starting directory checks ====
[OK  ]  RRDPATH is defined
[OK  ]  RRDPATH=/usr/local/pnp4nagios/var/perfdata
[OK  ]  Perfdata directory '/usr/local/pnp4nagios/var/perfdata' exists
[WARN]  7 hosts/services are not providing performance data
[WARN]  'process_perf_data 1' is set for 8 hosts/services which are not providing performance data!
[OK  ]  'process_perf_data 1' is set for 34 of your hosts/services
[INFO]  ==== System sizing ====
[OK  ]  33 hosts/service objects defined
[INFO]  ==== Check statistics ====
[WARN]  Warning: 3, Critical: 0
[WARN]  Checks finished...

Integrate pnp4nagios into icinga-web

  • integrate XML extension
# cp /usr/local/src/icinga-web-1.6.2/contrib/PNP_Integration/templateExtensions/* \
/usr/local/icinga-web/app/modules/Cronks/data/xml/extensions/
  • clear cache
# /usr/local/icinga-web/bin/clearcache.sh
Basedir: /usr/local/icinga-web-1.6.2 Cachedir: /usr/local/icinga-web-1.6.2/app/cache
Deleting cache from config (70 files) ... ok
Deleting cache from Squished (4 files) ... ok
  • restart icinga
# /etc/init.d/icinga restart

Verify pnp4nagios and icinga-web with pnp4nagios

_images/pnp4nagios_env.png


  • remove install.php if test passes
# rm -f /usr/local/pnp4nagios-0.6.16/share/install.php
_images/pnp4nagios_overview.png
  • icinga-web with pnp4nagios(pop-up performance graph)
_images/icinga-web_pnp4nagios_01.png
  • performance graph in detail
_images/icinga-web_pnp4nagios_02.png


I'm going to introduce about configurations, especially init scripts.
Posted by at
Labels: , , , , , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

iJAWS@Doorkeeper

iJAWS #6 @GDI Communications