Icinga Check Scripts

Various check scripts for use in icinga.

All scripts support option --help, so call ./scriptname --help for more information.

load_per_cpu.sh

Icinga plugin to check load average per CPU. Shows 1min, 5min 15min average per CPU. Different to check_load, the load is divided by the number of CPU units and therefore normalized. So the same warning and critical levels fit for any server.

If a server's load is often high, you can find tipps in Isolating Linux High System Load, in short words:

• see the load average in uptime
• check system logs with dmesg
• see vmstat:
  • high WA (wait) column: CPU is often waiting, you probably have high disk load
    • check swap → si and so, if they are often much above 0, you are out of memory
      • check memory usage of processes with ps awwlx --sort=vsz
      • solution: add more memory
  • if cpu → ID (idle) is around 0, your CPU is overloaded
    • if SY (system) is high, there could be large directories, e.g. mail spam
    • if SY (system) is high, it could be the firewall iptables
    • if US (user-space) is high, check with top for CPU consuming processes
  • if io → bi and bo (in/out) are high, check with iostat or sudo iotop

mountpoint.sh

Icinga plugin to check whether a given mountpoint is available. Detects problems, e.g. with glusterfs: In case of a problem, there is an error message such as "socket not connected". Called with path to check.

diskio.sh

Icinga plugin to check whether a given path is writable and checks the performance of the given path. Detects problems with the filesystem or specific io performance. Writes a test file using dd and reports write speed. It is recommended to use a subdirectory tmp for writing test files.

gluster-geo-replication.sh

Icinga plugin to check gluster geo replication. Returns OK, if no connection is Faulty and one connection is Active. Returns CRITICAL, if one connection is Faulty- Returns WARNING, if no connection is Active.

Installation

Install

Checkout to a path, e.g. /opt/icinga-checks:

cd /opt
sudo git co https://mrw.sh/admin-scripts/icinga-checks.git

Update

cd /opt/icinga-checks
sudo git pull

Configure

Go to Icinga Director, in Commands add one:

• Command type: Plugin Check Command
• Command name: load per cpu
• Command: check_by_ssh

After adding the command, add the following parameters:

-C /opt/icinga-checks/load_per_cpu.sh
-H $host.name$
-oStrictHostKeyChecking=no

The last argument -oStrictHostKeyChecking=no is a key only ith no value.

To check mountpoint of /var/volumes use argument name -C with value /opt/icinga-checks/mountpount.sh /var/volumes.

Then create a new Service Template, give it a name. e.g. load per cpu and assign the check command just created.

Finally add a service, again you can use the same name and import the template just created, then assign the hosts to check. I use a host template for all linux servers, so the command is run in each of them.

Don't forget to deploy the activity log in Icinga Director.