backup-generic/scripts/backup-generic.texi.in

\input texinfo   @c -*-texinfo-*-
@c %**start of header
@setfilename backup-generic.info
@documentencoding UTF-8
@include version.texi
@include variables.texi
@set AUTHOR Marc Wäckerlin
@set TITLE @value{PACKAGE} - differential rsync backups
@settitle @value{TITLE} @value{VERSION}
@c %**end of header

@copying
Copyright @copyright{} 2012 @verbatiminclude @top_srcdir@/AUTHORS
@end copying

@titlepage
@title @value{TITLE}
@author @value{AUTHOR}
@page
@vskip 0pt plus 1filll
@insertcopying
@end titlepage

@c So the toc is printed at the start.
@contents

@ifnottex
@node Top
@top @value{TITLE}

This manual is for @value{PACKAGE}, version @value{VERSION}.
@end ifnottex

@menu
* Introduction::     Introducion, what's all about?
* Installation::     Installation
* Naming::           Naming used in this manual
* Configuration::    Configuration of the backup server
* Setup Host::       Setup a host to be backed up
* Monitoring::       Monitoring the backup server
* Internals::        What happens behind the scenes

* Copying::          Your rights and freedoms.
* Index::            Index
@end menu



@node Introduction
@chapter Introduction
@include @top_srcdir@/README



@node Installation
@chapter Installation Of Package @value{PACKAGE}
@cindex installation

@section Install From Ubuntu/Debian Repository
@cindex ubuntu repository
@cindex debian repository
@cindex debian package
@cindex dpkg

If you use Ubuntu or Debian, installation can be done directly out of
a repository. Configure apt to use the author's repository:

@verbatim
wget -O- http://dev.marc.waeckerlin.org/repo/PublicKey | sudo apt-key add -
sudo apt-get install python-software-properties
sudo apt-add-repository http://dev.marc.waeckerlin.org/repo
sudo apt-get update
@end verbatim

Then install the package from the repository:

@verbatim
sudo apt-get install generic-backup
@end verbatim

@section Install From Sources
@cindex configure
@cindex make
@cindex build
@cindex sources

Download the latest archive, untar, configure, build and install:

@example
wget http://dev.marc.waeckerlin.org/repo/sources/@value{PACKAGE}-@value{VERSION}.tar.bz2
tar xjf @value{PACKAGE}-@value{VERSION}.tar.bz2
cd @value{PACKAGE}-@value{VERSION}
./bootstrap.sh
./configure
make
sudo make install
@end example


@node Naming
@chapter Naming Used In This Manual
@cindex naming conventions
@cindex setup
@cindex installation

In this manual, we use the following naming:

@table @dfn
@item server
is the @dfn{backup server} where these backup scripts run
@cindex server, backup
@cindex backup server
@item host
is a @dfn{host} in the network that will be backuped onto the backup server
@cindex host
@end table

Moreover, in all examples, the example hostname will be
@samp{host.company-name.com} and your username is @samp{user}. You
will have to replace these placeholders with the name of your hosts
and with your username.



@node Configuration
@chapter Configuration
@cindex configuration files

@section The Configuration Files

The backup scripts can be configured globally, per user and per host
in the following way:

@table @file

@item /etc/backup.conf
@cindex /etc/backup.conf

This is the global configuration file and the default for all
hosts. This file should be prefered for your configuration.

@item ~/.backup
@cindex ~/.backup

This is the local configuration file for all hosts of the user that
runs the backup script (normally @samp{root}). It overwrites
configurations from the global configuratin file.

@item /etc/backup-host.company-name.com.conf
@cindex /etc/backup-@samp{host.company-name.com}.conf

This is a global configuration file that is only evaluated when
backing up host @samp{host.company-name.com}.

@item ~/.backup-host.company-name.com
@cindex ~/.backup-@samp{host.company-name.com}

This is a local user's configuration file that is only evaluated when
backing up host @samp{host.company-name.com}.

@end table

Configuration files on top of this list are evaluated first that
means, the files below overwrite variables of the files above. So
these files can be combined, any or none of the files can be
given. There are also good default values in case no configuration is
given at all.

@section Variables In The Configuration files

@subsection Common Variables

These are the variables you probably want to change:

@table @var

@item DISABLE

A path to a file to temporarily stop execution of backups (examply used when you want to start restauration). If the file exists, no new backup will be started. Create the file to stop all backups, delete it to continue with backups. Default: @file{/var/run/nobackup}

@item EMAIL

The email address to send error reports to. If empty, no emails are
sent. It requires a command line @command{mail} client. Default: empty

@item NUM

Number of backups to keep per type. If NUM is e.g. 5, then 5 daily backups will bee kept, 5 weekly backups and 5 monthly backups. Default: @code{5}

@item TARGET

The path where backups should go to. Make shure this is on a huge harddisk. Default: @file{/var/backup}

@item 

@end table

@subsection Special Variables

These are the variables that come with good default values and should
normally not be changed:

@table @var

@item HOST

Name of the host to backup. The special keyword @code{generic} stands for the local host (without @command{ssh} network access). This is where the name @command{backup-generic} comes from. Default: detect automatically from file name

@item SOURCES

The source paths to backup. This is hosts specific. Since normally the backup uses @command{rsync} option @code{--one-file-system} (or @code{-x}), @var{SOURCES} must contain a list of all mounted filesystems. These are automaticall detected by using command @command{mount} on the host @var{HOST} and filtering the mountpoint of the backup target @var{TARGET}. Default: setect automatically by calling @command{mount}

@item MORE_ARGS

Additional arguments you want to pass to @command{rsync}, such as @option{--exclude=/some/path}. Use @var{MORE_ARGS} instead of @var{ARGS}. Default: empty

@item ARGS

The standard arguments that are passed to @command{rsync}. Dont't overwrite unless you know exactly what you do. Use @var{MORE_ARGS} instead of @var{ARGS} to add additional arguments. Default:  @option{-aqx --delete}

@item LOGFILE

The file where details of the backups are logged to. Default: @file{/var/log/backup.log}

@item LOCK

Lockfile to prevent two backups of the same host at the same time. Default: @code{$@{TARGET@}/$@{HOST@}.lock}

@item TMPFILE

Temporary file to store log information of the just running backup. This file will be removed after backup. If there is any error while backing up, this file is sent per email to help finding the problem. Default: @code{/tmp/$@{0##*/@}.$$}

@item RSYNC_CMD

The @code{rsync} command. Default: @code{nice -n 19 ionice -c 2 -n 7 rsync}

@item LOCAL_RSYNC_CMD

The local @code{rsync} command. use thios variable, if it is different from the remote @code{rsync} command. Default: @code{$@{RSYNC_CMD@}}

@item REMOTE_RSYNC_CMD

The remote @code{rsync} command. use thios variable, if it is different from the local @code{rsync} command. Default: @code{$@{RSYNC_CMD@}}

@end table

@section Configuration File Syntax
@cindex configuration file syntax
@cindex syntax, configuration file

The configuration files are in @command{bash} format and declare
variables that are used during the backup. Any @command{bash} syntax
is possible in the configuration files. So normally, a configuration
file is just a list of variable definitions, but it can also contain
more complex statements:

@subsection Example Configuration File

@verbatim
EMAIL=me@somewhere.com
NUM=12
TARGET="/var/backup"
MORE_ARGS="--exclude /media --exclude /var/backup"
SOURCES=$(ssh $HOST mount \
          | awk '$2=="on" {print $3}' \
          | sed '/\/\(run\|proc\|sys\|dev\|var\|lib\)\|'"${TARGET//\//\\/}"'$/d')
@end verbatim



@node Setup Host
@chapter Setup Host

@section Setup Host Key Exchange

@cindex ssh
@cindex key exchange
@cindex setup password less login
@cindex preparations

Before you can backup a host, you must make sure that the @code{root}
user of the backup server has SSH login access to the @code{root} of
the host. This outside of the scope of this backup toolset. It can be
done using the tools from OpenSSH client package, namely
@command{ssh-keygen} and @command{ssh-copyid}. This means, before you
can setup a backup for a specific host, you need to exchange the SSH
login keys for root manually to gain password less @command{ssh}
login. To backup for example a host named
@samp{host.company-name.com}, follow these steps:

@enumerate

@item First create a SSH key for root (if not already done, to be done
only once):

@cindex ssh-keygen
@example
sudo -H ssh-keygen
@end example

This generates the two files @file{/root/.ssh/id_rsa} and
@file{/root/.ssh/id_rsa.pub}, which are then the backup server's
private and public key.

@item For each host that should be backed up, repeat the following step:

@enumerate

@item Append the backup server's public key @file{/root/.ssh/id_rsa.pub}
to the the host's @file{/root/.ssh/authorized_keys}. The easiest way
to achieve this is to use @command{ssh-copyid}. In this example, we
want to backup host @samp{host.company-name.com}, replace it with the
name of the host you want to backup:

@cindex ssh-copyid
@verbatim
sudo -H ssh-copy-id root@host.company-name.com
@end verbatim

@item After you have done this, you should test, if you can login to
@samp{host.company-name.com} without being asked for a password (with
no user interaction):

@verbatim
sudo -H ssh root@host.company-name.com
@end verbatim

If you get to the console of @samp{host.company-name.com} without
having to type a password, then your backup is prepared and should
work.

You could be asked for the @command{sudo} password, but you must not be prompted for the host's password:

@itemize
@item This is ok (in this example, your login name is @samp{user}):
@verbatim
[sudo] password for user:
@end verbatim
@item This should not happen:
@verbatim
root@host.company-name.com's password:
@end verbatim
@end itemize

@end enumerate
@end enumerate

@section Setup Hosts to Backup

@cindex add host to backup
@cindex backup host
When you have setup the password less login to the host as specified
in the previous section, it is then easy to setup the backup mechanism
for your hosts. Simply call the script @command{setup-backup} with a
list of hosts to backup as arguments, e.g. to backup host
@samp{host.company-name.com}, call:

@verbatim
sudo setup-backup host.company-name.com
@end verbatim



@node Monitoring
@chapter Icinga Monitoring
@cindex icinga
@cindex nagios
@cindex monitoring

@section Setup NRPE On Backup Server
@cindex nrpe

Icinga (former Nagios) can be used to monitor the backup activities. For this purpose, simply install the @code{nagios-nrpe-server} package of your distribution on the backup server. Then enable passing arguments to nrpe and add your Icinga server's IP adress to the list of allowed hosts. Provided the IP address of your Nagios server is @samp{172.17.0.1}, change the following two lines in @file{/etc/nagios/nrpe.cfg}:

@verbatim
allowed_hosts=127.0.0.1,172.17.0.1
dont_blame_nrpe=1
@end verbatim

Of course, yu could setup Icinga differentely, i.e. without nrpe or
without having to specify arguments. But this is out of scope of these
instructions.

@section Setup Backup Checks On Nagios Server
@cindex nagios server

@image{icinga-monitoring,12cm}
On the server where you run Nagios, add the following configuration, e.g. in @file{/etc/icinga/objects/backups.cfg}:
@verbatim
define service{
 use                 generic-service
 host_name           host.company-name.com
 service_description Daily Backup host.company-name.com
 check_command       check_nrpe!check_backup!host.company-name.com daily
}

define service{
 use                 generic-service
 host_name           host.company-name.com
 service_description Weekly Backup host.company-name.com
 check_command       check_nrpe!check_backup!host.company-name.com weekly
}

define service{
 use                 generic-service
 host_name           host.company-name.com
 service_description Monthly Backup host.company-name.com
 check_command       check_nrpe!check_backup!host.company-name.com monthly
}
@end verbatim



@node Internals
@chapter Behind The Scenes
@cindex cron

@section Backup Host Configuration
@cindex setup-backup
@cindex backup-generic
@cindex /etc/cron.daily/backup-host_company-name_com
@cindex /etc/cron.weekly/backup-host_company-name_com
@cindex /etc/cron.monthly/backup-host_company-name_com
@cindex /etc/cron.daily
@cindex /etc/cron.weekly
@cindex /etc/cron.monthly

The script @command{setup-backup} links for all hosts and all backup
timings to the script @command{backup-generic}. Dots in the hostname
are replaced by underscores, because there must not be dots in cron
files (this is a limitation of @command{run-parts}). That means, for
each host, which is e.g. @samp{host.company-name.com}, it creates
three links to @command{backup-generic}:
@itemize
@item @file{/etc/cron.daily/backup-host_company-name_com}
@item @file{/etc/cron.weekly/backup-host_company-name_com}
@item @file{/etc/cron.monthly/backup-host_company-name_com}
@end itemize

The script @command{backup-generic} is then called as
@command{/etc/cron.daily/backup-host_company-name_com} once every
day. From the filename, the script detects the period (here
@samp{daily}) and the hostname to backup (here
@samp{host.company-name.com}).

If any older backup exists, either from the same host, or from another
host, @command{backup-generic} tries to hardlink as many files as
possible to a previous backup using the @command{rsync}'s
@option{--link-dest} feature. This way, only files that don't already
and files that have been changed are copied over the network. This
saves time, bandwidth and disk storage.

Removing a host from the backup is then done by removing all three
@file{backup-host_company-name_com} links in @file{/etc/cron.daily},
@file{/etc/cron.weekly} and @file{/etc/cron.hourly}. If you purge the
package (@command{sudo apt-get purge backup-generic}), then all links
are cleaned up.

@section Backup Escalation
@cindex /etc/cron.hourly
@cindex setup-backup

The script @command{setup-backup} also links @command{escalate-backup}
to @file{/etc/cron.hourly}. This script looks for @file{backup-*}
files in @file{/etc/cron.hourly}, @file{/etc/cron.daily},
@file{/etc/cron.weekly} and @file{/etc/cron.monthly}. For all backups
that have been setup, it looks in the backup target, whether there is
an actual backup or not. This is done by calling the Icinga check
script @command{check_backup} with hostname and backup period als
parameter. In case of an error, the backup is executed immediately.



@node Copying
@chapter Copying
@cindex copying
@cindex license
@verbatiminclude @top_srcdir@/COPYING
@node Index
@unnumbered Index

@printindex cp

@bye