Pull to refresh

Holland — бекапы MySQL/PostgreSQL без головной боли

Reading time8 min
Views19K
Holland LogoВ один из дней мне надоело использовать самописные скрипты для создания резервных копий баз данных. Не важно, разработаны они были мной или найдены где-то на просторах интернета. Исходя из принципа, что время является самым дорогим ресурсом системного администратора (инженера, архитектора), было найдено решение, отвечающее следующим требованиям: простая установка, быстрая настройка и, как сумма предыдущих требований, быстрое введение в эксплуатацию.

Согласно официальному сайту, Holland — фреймворк с открытым исходным кодом для создания резервных копий, разработанный Rackspace и написанный на языке Python. Проект преследует цель создания бекапов с большой гибкостью настройки, логичной структурой и простотой использования. В данный момент Holland работает с MySQL и PostgreSQL, однако в будущем будет включать большее разнообразие баз данных, и даже приложения, никак не относящимся к базам данных. Благодаря модульной структуре Holland может быть использован для создания резервных копий чего угодно, как угодно.

Представим себе, что наш сценарий предусматривает ежедневный бекап одной базы MySQL (утилитой mysqldump) с ротацией семи копий.
И для начала сабж надо скачать и установить на сервер. В пакетных дистрибутивах это не должно вызвать трудностей. Также представим, что у нас CentOS.

Скачать


Holland существует в репозиториях у:
  • Debian 6/7
  • Centos 5/6
  • RHEL 4/5/6
  • Ubuntu 10.04/11.10/12.04/12.10/13.04

Также присутствует на github.com

Предполагаю, при таком выборе не должно возникнуть проблем с установкой даже на «старые» сервера. Пакетный менеджер дистрибутива, с которым придется работать, поможет в этом деле. Следуя поставленной задаче, вводим в консоль, при необходимости добавив повысив себе привилегии:
yum install -y holland holland-mysqldump


Следующим этапом, что логично, будет настройка.

Настроить


Структура /etc/holland проста и понятна. В директории с конфигурацией программы помимо конфигурационного файла с основными параметрами работы есть две директории:
providers содержит шаблоны с настройками для работы с утилитами типа mysqldump или xtrabackup
backupsets содержит конкретные планы резервного копирования с параметрами типа: кол-во копий, метод и степень сжатия и т.п.

Я счел нужным представить примеры конфигураций в нетронутом виде для быстрого ознакомления читателем.

/etc/holland.conf
## Root holland config file
[holland]

## Paths where holland plugins may be found.
## Can be comma separated
plugin_dirs = /usr/share/holland/plugins

## Top level directory where backups are held
backup_directory = /var/spool/holland

## List of enabled backup sets. Can be comma separated. 
## Read from <config_dir>/backupsets/<name>.conf
# backupsets = example, traditional, parallel_backups, non_transactional
backupsets = default 

# Define a umask for file generated by holland
umask = 0007

# Define a path for holland and its spawned processes
path = /usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin

[logging]
## where to write the log
filename = /var/log/holland/holland.log

## debug, info, warning, error, critical (case insensitive)
level = info

providers/mysqldump.conf
## Global settings for the mysqldump provider - Requires holland-mysqldump
##
## Unless overwritten, all backup-sets implementing this provider will use
## the following settings.

[mysqldump]

## Override the path where we can find mysql command line utilities
#mysql-binpath       = /usr/bin/mysqldump

## One of: flush-lock, lock-tables, single-transaction, auto-detect, none
##
## flush-lock will place a global lock on all tables involved in the backup
## regardless of whether or not they are in the backup-set. If 
## file-per-database is enabled, then flush-lock will lock all tables 
## for every database being backed up. In other words, this option may not
## make much sense when using file-per-database.
##
## lock-tables will lock all tables involved in the backup. If
## file-per-database is enabled, then lock-tables will only lock all the
## tables associated with that database.
##
## single-transaction will force running a backup within a transaction.
## This allows backing up of transactional tables without imposing a lock
## howerver will NOT properly backup non-transactional tables.
##
## Auto-detect will choose single-transaction unless Holland finds
## non-transactional tables in the backup-set.
##
## None will completely disable locking. This is generally only viable
## on a MySQL slave and only after traffic has been diverted, or slave
## services suspended.
lock-method         = auto-detect

## comma-delimited glob patterns for matching databases
## only databases matching these patterns will be backed up
## default: include everything
#databases           = "*"

## comma-delimited glob patterns to exclude particular 
## databases
#exclude-databases   = 

## only include the specified tables
#tables              = "*"

## exclude specific tables
#exclude-tables      = ""

## Whether to dump routines explicitly
## (routines are implicitly included in the mysql database)
dump-routines       = no

## Whether to dump events explicitly.
## Note that this feature requires MySQL 5.1 or later.
dump-events			= no

## Whether to stop the slave before commencing with the backup
stop-slave          = no

## Whether to record the binary log name and position at the time of the
## backup.
bin-log-position    = no

## Whether or not to run FLUSH LOGS in MySQL with the backup. When FLUSH
## LOGS is actually executed depends on which if database filtering is being
## used and whether or not file-per-database is enabled. Generally speaking,
## it does not make sense to use flush-logs with file-per-database since the 
## binary logs will not be consistent with the backup.
flush-logs			= no

## Whether to run a separate mysqldump for each database. Note that while
## this may initially sound like a good idea, it is far simpler to backup
## all databases in one file, although that makes the restore process
## more difficult when only certain data needs to be restored.
file-per-database   = no

## any additional options to the 'mysqldump' command-line utility
## these should show up exactly as they are on the command line
## e.g.: --flush-privileges --reset-master
additional-options  = ""

## Compression Settings
[compression]

## compress method: gzip, gzip-rsyncable, bzip2, pbzip2, or lzop
## Which compression method to use, which can be either gzip, bzip2, or lzop.
## Note that lzop is not often installed by default on many Linux 
## distributions and may need to be installed separately.
method              = gzip

## Whether to compress data as it is provided from 'mysqldump', or to
## compress after a dump has finished. In general, it is often better to use
## inline compression. The overhead, particularly when using a lower 
## compression level, is often minial since the entire process is often I/O
## bound (as opposed to being CPU bound).
inline              = yes

## What compression level to use. Lower numbers mean faster compression, 
## though also generally a worse compression ratio. Generally, levels 1-3
## are considered fairly fast and still offer good compression for textual
## data. Levels above 7 can often cause a larger impact on the system due to
## needing much more CPU resources. Setting the level to 0 effectively 
## disables compresion.
level               = 1

## If the path to the compression program is in a non-standard location,
## or not in the system-path, you can provide it here.
##
## FIXME: Currently not implemented, compression binary is looked up by
## which.
##
#bin-path           = /usr/bin/gzip

## MySQL connection settings. Note that Holland will try ot read from
## the provided files defined in the 'defaults-extra-file', although 
## explicitly defining the connection inforamtion here will take precedence.
[mysql:client]
defaults-extra-file  = /root/.my.cnf,~/.my.cnf,
#user                = hollandbackup
#password            = "hollandpw"
#socket              = /tmp/mysqld.sock
#host                = localhost
#port                = 3306

backupsets/mysqldump.conf
## Holland mysqldump Example Backup-Set
##
## This implements a vanilla backup-set using the mysqldump provider which,
## in turn, uses the 'mysqldump' utility.
##
## Many of these options have global defaults which can be found in the
## configuration file for the provider (which can be found, by default
## in /etc/holland/providers).

[holland:backup]
plugin = mysqldump
backups-to-keep = 1
auto-purge-failures = yes
purge-policy = after-backup
estimated-size-factor = 1.0

# This section defines the configuration options specific to the backup
# plugin. In other words, the name of this section should match the name
# of the plugin defined above.
[mysqldump]

## Override the path where we can find mysql command line utilities
#mysql-binpath       = /usr/bin/mysqldump

## One of: flush-lock, lock-tables, single-transaction, auto-detect, none
##
## flush-lock will run a FLUSH TABLES WITH READ LOCK prior to the backup
##
## lock-tables will instruct 'mysqldump' to lock all tables involved
## in the backup.
##
## single-transaction will force running a backup within a transaction.
## This allows backing up of transactional tables without imposing a lock
## howerver will NOT properly backup non-transacitonal tables.
##
## Auto-detect will choose single-transaction unless Holland finds
## non-transactional tables in the backup-set.
##
## None will completely disable locking. This is generally only viable
## on a MySQL slave and only after traffic has been diverted, or slave
## services suspended.
lock-method         = auto-detect

## comma-delimited glob patterns for matching databases
## only databases matching these patterns will be backed up
## default: include everything
databases           = "*"

## comma-delimited glob patterns to exclude particular 
## databases
#exclude-databases   = 

## only include the specified tables
tables              = "*"

## exclude specific tables
#exclude-tables      = ""

## Whether to dump routines explicitly
## (routines are implicitly included in the mysql database)
dump-routines       = no

## Whether to dump events explicitly.
## Note that this feature requires MySQL 5.1 or later.
dump-events			= no

## Whether to stop the slave before commencing with the backup
stop-slave          = no

## Whether to record the binary log name and position at the time of the
## backup.
bin-log-position    = no

## Whether or not to run FLUSH LOGS in MySQL with the backup. When FLUSH
## LOGS is actually executed depends on which if database filtering is being
## used and whether or not file-per-database is enabled. Generally speaking,
## it does not make sense to use flush-logs with file-per-database since the 
## binary logs will not be consistent with the backup.
flush-logs			= no

## Whether to run a separate mysqldump for each database. Note that while
## this may initially sound like a good idea, it is far simpler to backup
## all databases in one file, although that makes the restore process
## more difficult when only certain data needs to be restored.
file-per-database   = no

## any additional options to the 'mysqldump' command-line utility
## these should show up exactly as they are on the command line
## e.g.: --flush-privileges --reset-master
additional-options  = ""

## Compression Settings
[compression]

## compress method: gzip, gzip-rsyncable, bzip2, pbzip2, lzop, or xz
## Which compression method to use, which can be either gzip, bzip2, or lzop.
## Note that pbzip2 and lzop are not often installed by default on many Linux 
## distributions and may need to be installed separately.
method              = gzip

## Whether to compress data as it is provided from 'mysqldump', or to
## compress after a dump has finished. In general, it is often better to use
## inline compression. The overhead, particularly when using a lower 
## compression level, is often minial since the entire process is often I/O
## bound (as opposed to being CPU bound).
inline              = yes

## What compression level to use. Lower numbers mean faster compression, 
## though also generally a worse compression ratio. Generally, levels 1-3
## are considered fairly fast and still offer good compression for textual
## data. Levels above 7 can often cause a larger impact on the system due to
## needing much more CPU resources. Setting the level to 0 effectively 
## disables compresion.
level               = 1

## If the path to the compression program is in a non-standard location,
## or not in the system-path, you can provide it here.
#bin-path           = /usr/bin/gzip

## MySQL connection settings. Note that these can be inherited from the
## provider itself allowing for global defaults. Providing connection 
## information for a backup-set can often be helpful when, for instance
## a backup-set is backing up a remote MySQL server.
#[mysql:client]
#user                = hollandbackup
#password            = "hollandpw"
#socket              = /tmp/mysqld.sock
#host                = localhost
#port                = 3306


И, несмотря на многообразие переменных, которыми изобилуют конфиги, для осуществления задумки нам достаточно лишь:
  1. Указать имя сценария в holland.conf
    backupsets = mysqldump
    

  2. Скопировать сценарий из /usr/share/doc/holland-*/examples/mysqldump.conf в /etc/holland/backupsets
    cp /usr/share/doc/holland-*/examples/mysqldump.conf /etc/holland/backupsets/
    

  3. Указать в сценарии mysqldump.conf количество копий, нужные базы, и доступ с достаточными правами
    backups-to-keep = 7
    databases = «somedb»
    user = hollandbackup
    password = «hollandpw»
    socket = /tmp/mysqld.sock
    

  4. Добавить в планировщик (например, cron) запись о ежедневном выполнении команды
    holland backup
    



Прочее

Конфигурация для PostgreSQL будет отличаться лишь другим установленным плагином (holland-postgresql) и другим скопированным примером. Впрочем, файлы примеров меня заинтересовали даже просто своим названием, взгляните:
  • maatkit.conf
  • mysqldump.conf
  • mysqldump-lvm.conf
  • mysqlhotcopy.conf
  • mysql-lvm.conf
  • random.conf
  • sqlite.conf
  • xtrabackup.conf

Не забудьте зайти на сервер через несколько дней и проверить, выполняется ли план резервного копирования, его успешность.

Надеюсь, эти несколько шагов помогут вам сэкономить время и силы на столь непопулярном занятии, как резервное копирование.
Tags:
Hubs:
+30
Comments15

Articles