TIL #1 - Using volumes in docker for backup (and other) purposes

2022-11-09

docker , TIL , backup

A little history

I’ve been using docker on my Synology NAS for over two years now. The process of moving to containers was a slow one at the beginning but after I learned a lot about the inner workings of docker and the limitations Synology has in its implementation (not too problematic and can be solved) it was a game changer for me.

So, I started off by using Synology’s own implemantetion for the docker GUI which is quite useful for someone with no prior knowledge of the topic but after a while I wanted more control, flexibiliy, possibilities, etc.

Enter docker-compose and the `.yml` files.

I’ve quickly moved over to putting the configuration of my self-hosted applications in individual docker-compose.yml files and even using a git repo for storing then (self-hosted as well, of course).

The flexibility of using the .yml files is really great and after I configured my NAS’s docker context on my daily driver machine I was content for a while.

Backup

Still no word about docker volumes, but trust me, we are getting there.

As the number and complexity of my self-hosted applications grew I was starting to feel uncomfortable with my current backup solution. It was a Hyperbackup (Synology’s own backup software) backup of the docker folder: /volume1/docker

This could work of course for containers without databases, or without files getting written constantly, but it’s far from ideal and I was concerned with losing data.

Scripting the whole thing

First I came up with a bash script which took the backup target list of containers from a .txt file, then going over the list one-by-one, shutting down the container, compressing all its mounted directories into a time-stamped tar.gz file stored in a backup folder and finally starting the container again.

This was working fine but felt a little bit unsophisticated, so I started looking into alternatives.

Enter offen/docker-volume-backup

I was aware of the concept of docker volumes since some of my already running containers used them but it was not very interesting for me because using bind mounts was enough for me, basically for the reason that all of the files were readily accessible in the filesystem.

After finding a recommendation for volume based backups on r/selfhosted I checked out the github page of Offen and started to convert my containers to use volumes because the solution appealed to me and wanted to use it. During this process I realized that going the volumes route solves a recurring problem of access rights with the container files, since a few of them does not provide any settings to set proper UID and GID, hence the created files are only accessible for the root user. Also, I still can access the files if I wanted in the volumes directory as root (Synology puts these here: /volume1/@docker/volumes)

The offen configuration was a little bit confusing at first, but I started to look for examples[1][2] on their github page and after that came up with the following to test the solution on my linkding installation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35


services:
  linkding:
    container_name: linkding
    image: sissbruecker/linkding:latest
    network_mode: "bridge"
    restart: unless-stopped
    volumes:
      - data:/etc/linkding/data
    ports:
      - 9090:9090
    labels:
      - flame.type=application
      - flame.name=Linkding
      - flame.url=<omitted>
      - flame.icon=link-variant
      - docker-volume-backup.stop-during-backup=true

  backup:
    image: offen/docker-volume-backup:v2
    container_name: linkding_backup
    restart: always
    network_mode: "bridge"
    environment:
      BACKUP_FILENAME: backup_linkding-%Y-%m-%dT%H-%M-%S.tar.gz
      BACKUP_LATEST_SYMLINK: backup_linkding-latest.tar.gz
    volumes:
      # Volumes to backup
      - data:/backup/data:ro
      # Docker socket needed for communicating with the docker daemon
      - /var/run/docker.sock:/var/run/docker.sock
      # Backup target directory
      - /volume1/docker/backup:/archive      

volumes:
  data:

After restarting the container I was able to run a backup manually with:

docker exec linkding_backup backup

This seems to work, so in the long run I’ll adapt this to all of my containers I want to backup.

Future

I want to have some kind of strategy in place for retention and backup rotation (Offen has a solution for this) and I’d like to have a notification system in place which should work similar to my already working Discord channels. Offen has a notification system as well, so I don’t think this will be a big issue.

Overall, I’m satisfied with this solution.