Borgmatic Backup¶
+Introduction¶
+Borgmatic is a great way to run backups on your Mailcow setup as it securely encrypts your data and is extremely easy to +set up.
+Due to it's deduplication capabilities you can store a great number of backups without wasting large amounts of disk +space. This allows you to run backups in very short intervals to ensure minimal data loss when the need arises to +recover data from a backup.
+This document guides you through the process to enable continuous backups for mailcow with borgmatic. The borgmatic
+functionality is provided by the borgmatic Docker image by b3vis. Check out
+the README in that repository to find out about the other options (such as push notifications) that are available.
+This guide only covers the basics.
Setting up borgmatic¶
+Create or amend docker-compose.override.yml¶
+In the mailcow-dockerized root folder create or edit docker-compose.override.yml and insert the following
+configuration:
version: '2.1'
+services:
+ borgmatic-mailcow:
+ image: b3vis/borgmatic
+ restart: always
+ dns: ${IPV4_NETWORK:-172.22.1}.254
+ volumes:
+ - vmail-vol-1:/mnt/source/vmail:ro
+ - mysql-socket-vol-1:/var/run/mysqld/:z
+ - ./data/conf/borgmatic/etc:/etc/borgmatic.d:Z
+ - ./data/conf/borgmatic/state:/root/.config/borg:Z
+ - ./data/conf/borgmatic/ssh:/root/.ssh:Z
+ environment:
+ - TZ=${TZ}
+ - BORG_PASSPHRASE=YouBetterPutSomethingRealGoodHere
+ networks:
+ mailcow-network:
+ aliases:
+ - borgmatic
+Ensure that you change the BORG_PASSPHRASE to a secure passphrase of your choosing.
For security reasons we mount the maildir as read-only. If you later want to restore data you will need to remove
+the ro flag prior to restoring the data. This is described in the section on restoring backups.
Create data/conf/borgmatic/etc/config.yaml¶
+Next, we need to create the borgmatic configuration.
+source mailcow.conf
+cat <<EOF > data/conf/borgmatic/etc/config.yaml
+location:
+ source_directories:
+ - /mnt/source
+ repositories:
+ - user@rsync.net:mailcow
+ remote_path: borg1
+
+retention:
+ keep_hourly: 24
+ keep_daily: 7
+ keep_weekly: 4
+ keep_monthly: 6
+
+hooks:
+ mysql_databases:
+ - name: ${DBNAME}
+ username: ${DBUSER}
+ password: ${DBPASS}
+ options: --default-character-set=utf8mb4
+EOF
+Creating the file in this way ensures the correct MySQL credentials are pulled in from mailcow.conf.
This file is a minimal example for using borgmatic with an account user on the cloud storage provider rsync.net for
+a repository called mailcow (see repositories setting). It will backup both the maildir and MySQL database, which is
+all you should need to restore your mailcow setup after an incident. The retention settings will keep one archive for
+each hour of the past 24 hours, one per day of the week, one per week of the month and one per month of the past half
+year.
Check the borgmatic documentation on how to use other types of repositories or
+configuration options. If you choose to use a local filesystem as a backup destination make sure to mount it into the
+container. The container defines a volume called /mnt/borg-repository for this purpose.
Note
+If you do not use rsync.net you can most likely drop the remote_path element from your config.
Create a crontab¶
+Create a new text file in data/conf/borgmatic/etc/crontab.txt with the following content:
14 * * * * PATH=$PATH:/usr/bin /usr/bin/borgmatic --stats -v 0 2>&1
+This file expects crontab syntax. The example shown here will trigger the backup to run every hour at 14 minutes past +the hour and log some nice stats at the end.
+Place SSH keys in folder¶
+Place the SSH keys you intend to use for remote repository connections in data/conf/borgmatic/ssh. OpenSSH expects the
+usual id_rsa, id_ed25519 or similar to be in this directory. Ensure the file is chmod 600 and not world readable
+or OpenSSH will refuse to use the SSH key.
Bring up the container¶
+For the next step we need the container to be up and running in a configured state. To do that run:
+docker-compose up -d
+Initialize the repository¶
+By now your borgmatic container is up and running, but the backups will currently fail due to the repository not being +initialized.
+To initialize the repository run:
+docker-compose exec borgmatic-mailcow borgmatic init --encryption repokey-blake2
+You will be asked you to authenticate the SSH host key of your remote repository server. See if it matches and confirm
+the prompt by entering yes. The repository will be initialized with the passphrase you set in the BORG_PASSPHRASE
+environment variable earlier.
When using any of the repokey encryption methods the encryption key will be stored in the repository itself and not on
+the client, so there is no further action required in this regard. If you decide to use a keyfile instead of
+a repokey make sure you export the key and back it up separately. Check the Exporting Keys section
+for how to retrieve the key.
Restart container¶
+Now that we finished configuring and initializing the repository restart the container to ensure it is in a defined +state:
+docker-compose restart borgmatic-mailcow
+Restoring from a backup¶
+Restoring a backup assumes you are starting off with a fresh installation of mailcow, and you currently do not have +any custom data in your maildir or your mailcow database.
+Restore maildir¶
+Warning
+Doing this will overwrite files in your maildir! Do not run this unless you actually intend to recover mail +files from a backup.
+If you use SELinux in Enforcing mode
+If you are using mailcow on a host with SELinux in Enforcing mode you will have to temporarily disable it during +extraction of the archive as the mailcow setup labels the vmail volume as private, belonging to the dovecot container +exclusively. SELinux will (rightfully) prevent any other container, such as the borgmatic container, from writing to +this volume.
+Before running a restore you must make the vmail volume writeable in docker-compose.override.yml by removing
+the ro flag from the volume.
+Then you can use the following command to restore the maildir from a backup:
docker-compose exec borgmatic-mailcow borgmatic extract --path mnt/source --archive latest
+Alternatively you can specify any archive name from the list of archives (see +Listing all available archives)
+Restore MySQL¶
+Warning
+Running this command will delete and recreate the mailcow database! Do not run this unless you actually +intend to recover the mailcow database from a backup.
+To restore the MySQL database from the latest archive use this command:
+docker-compose exec borgmatic-mailcow borgmatic restore --archive latest
+Alternatively you can specify any archive name from the list of archives (see +Listing all available archives)
+After restoring¶
+After restoring you need to restart mailcow. If you disabled SELinux enforcing mode now would be a good time to +re-enable it.
+To restart mailcow use the follwing command:
+docker-compose down && docker-compose up -d
+If you use SELinux this will also trigger the re-labeling of all files in your vmail volume. Be patient, as this may +take a while if you have lots of files.
+Useful commands¶
+Manual archiving run (with debugging output)¶
+docker-compose exec borgmatic-mailcow borgmatic -v 2
+Listing all available archives¶
+docker-compose exec borgmatic-mailcow borgmatic list
+Break lock¶
+When borg is interrupted during an archiving run it will leave behind a stale lock that needs to be cleared before any +new operations can be performed:
+docker-compose exec borgmatic-mailcow borg break-lock user@rsync.net:mailcow
+Where user@rsync.net:mailcow is the URI to your repository.
Now would be a good time to do a manual archiving run to ensure it can be successfully performed.
+Exporting keys¶
+When using any of the keyfile methods for encryption you MUST take care of backing up the key files yourself. The
+key files are generated when you initialize the repository. The repokey methods store the key file within the
+repository, so a manual backup isn't as essential.
Note that in either case you also must have the passphrase to decrypt any archives.
+To fetch the keyfile run:
+docker-compose exec borgmatic-mailcow borg key export --paper user@rsync.net:mailcow
+Where user@rsync.net:mailcow is the URI to your repository.