If a server's load is often high, you can find tipps in [Isolating Linux High System Load](https://www.tummy.com/articles/isolating-heavy-load/), in short words:
- see the load average in `uptime`
- check system logs with `dmesg`
- see `vmstat`:
- high `WA` (wait) column: CPU is often waiting, you probably have high disk load
- check `swap` → `si` and `so`, if they are often much above 0, you are out of memory
- check memory usage of processes with `ps awwlx --sort=vsz`
- solution: add more memory
- if `cpu` → `ID` (idle) is around 0, your CPU is overloaded
- if `SY` (system) is high, there could be large directories, e.g. mail spam
- if `SY` (system) is high, it could be the firewall iptables
- if `US` (user-space) is high, check with `top` for CPU consuming processes
- if `io` → `bi` and `bo` (in/out) are high, check with `iostat` or `sudo iotop`
Go to [Icinga Director](https://mrw.sh/docker/icingaweb2), in `Commands` add one:
- Command type: `Plugin Check Command`
- Command name: `load per cpu`
- Command: `check_by_ssh`
After adding the command, add the following parameters:
-C /opt/icinga-checks/load_per_cpu.sh
-H $host.name$
-oStrictHostKeyChecking=no
The last argument `-oStrictHostKeyChecking=no` is a key only ith no value.
To check mountpoint of `/var/volumes` use argument name `-C` with value `/opt/icinga-checks/mountpount.sh /var/volumes`.
Then create a new Service Template, give it a name. e.g. `load per cpu` and assign the check command just created.
Finally add a service, again you can use the same name and import the template just created, then assign the hosts to check. I use a host template for all linux servers, so the command is run in each of them.
Don't forget to deploy the activity log in Icinga Director.