Backing up FreeBSD with Bacula via ZFS snapshot

Leave a Comment / Bacula, Open Source, snapshots, ZFS / By

Tonight, while watching an episode of The Great British Bake Off, I configured a new Bacula backup jobs to cover some datasets which were not already backed up. I already have a backup jobs to backup all the jails, but only for datasets which sit right under zroot/jails (for example). This new solution takes a list of datasets, snapshots them, backs them up, then destroys those snapshots.

Why backup a snapshot? Consistency. During a backup of a live filesystem, changes occur. In a snapshot, no changes can occur; it represents a specific instance in time.

In this post:

  • FreeBSD 14.0
  • Bacula 13.0.3 (server)
  • Bacula 9.6.7 (client)

HOWEVER, this approach fails for jailed ZFS datasets. See FreeBSD jailed ZFS datasets – how do I find the .zfs/snapshot directory? for an overview of the problem I’m still attempting to resolve.

The backup job

This is the backup job:

Job {
  Name            = "r730-01 snapshots"
  JobDefs         = "DefaultJob"
  Client          = r730-01-fd
  FileSet         = "r730-01 snapshots"

  RunScript {
    RunsWhen       = Before
    FailJobOnError = Yes
    Command        = "/usr/local/sbin/snapshots-for-backup.sh create"
  }

  RunScript {
    RunsWhen       = After
    FailJobOnError = No
    Command        = "/usr/local/sbin/snapshots-for-backup.sh destroy"
  }
}

The file set

FileSet {
  Name = "r730-01 snapshots"

  Ignore FileSet Changes = yes

  Include {
    Options {
      signature = MD5 
    }

    Exclude Dir Containing = .NOBACKUP

    File = "\\|/usr/local/sbin/snapshots-for-backup.sh list"
  }
}

The script

[1:49 r730-01 dvl ~] % cat /usr/local/sbin/snapshots-for-backup.sh
#!/bin/sh

ACTION=$1

SNAPSHOTDIRECTORY='/.zfs/snapshot/'

SNAPNAME="snapshot-for-backup"

DATASETS="
data02/freshports/dev-ingress01/dan-src
data02/freshports/dev-ingress01/freshports/message-queues/archive
data02/freshports/dev-ingress01/modules
data02/freshports/dev-ingress01/scripts
data02/freshports/dev-nginx01/www/freshports
data02/freshports/dev-nginx01/www/freshsource
data02/freshports/jailed/dev-ingress01/data/latest_commits
data02/freshports/jailed/stage-ingress01/data/latest_commits
data02/freshports/jailed/test-ingress01/data/latest_commits
"

for dataset in $DATASETS
do
    MOUNTPOINT=`/sbin/zfs get -H -o value mountpoint ${dataset}`
    if [ "${MOUNTPOINT}" = "none" ]
    then
       continue
    fi
  
    # the snapshot name is of the form: main_tank/iocage/jails/fedex@snapshot-for-backup
    SNAPSHOTFORBACKUP="${dataset}@${SNAPNAME}"

    # the backup dir is of the form: /usr/local/jails/fedex/.zfs/snapshot/snapshot-for-backup
    BACKUPDIR="${MOUNTPOINT}${SNAPSHOTDIRECTORY}${SNAPNAME}"

    case ${ACTION} in
      "create")
         zfs snapshot ${SNAPSHOTFORBACKUP}
         ;;

      "list")
         # echo back out the directory for backup...
         echo ${BACKUPDIR}
         ;;

      "destroy")
         zfs destroy ${SNAPSHOTFORBACKUP}
         ;;
    esac    
done

Here’s me running it at the command line:

[1:53 r730-01 dvl ~] % /usr/local/sbin/snapshots-for-backup.sh list                            
/jails/dev-ingress01/usr/home/dan/src/.zfs/snapshot/snapshot-for-backup
/jails/dev-ingress01/var/db/freshports/message-queues/archive/.zfs/snapshot/snapshot-for-backup
/jails/dev-ingress01/usr/local/lib/perl5/site_perl/FreshPorts/.zfs/snapshot/snapshot-for-backup
/jails/dev-ingress01/usr/local/libexec/freshports/.zfs/snapshot/snapshot-for-backup
/jails/dev-nginx01/usr/local/www/freshports/.zfs/snapshot/snapshot-for-backup
/jails/dev-nginx01/usr/local/www/freshsource/.zfs/snapshot/snapshot-for-backup
/var/db/ingress/latest_commits/.zfs/snapshot/snapshot-for-backup
/var/db/ingress/latest_commits/.zfs/snapshot/snapshot-for-backup
/var/db/ingress/latest_commits/.zfs/snapshot/snapshot-for-backup
[1:53 r730-01 dvl ~] %

It just works

This is the first job, a full:

24-Dec 04:34 bacula-sd-04 JobId 362069: Sending spooled attrs to the Director. Despooling 282 bytes ...
24-Dec 04:34 bacula-dir JobId 362069: Bacula bacula-dir 13.0.3 (02May23):
  Build OS:               amd64-portbld-freebsd14.0 freebsd 14.0-RELEASE-p1
  JobId:                  362069
  Job:                    BackupCatalog.2023-12-24_03.05.00_20
  Backup Level:           Differential, since=2023-12-03 12:47:15
  Client:                 "dbclone-fd" 9.6.7 (10Dec20) amd64-portbld-freebsd14.0,freebsd,14.0-RELEASE-p1
  FileSet:                "Catalog" 2015-10-08 12:14:19
  Pool:                   "FullFileNoNextPool-04" (From Run DiffPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "bacula-sd-04-FullFileNoNextPool" (From Pool resource)
  Scheduled time:         24-Dec-2023 03:05:00
  Start time:             24-Dec-2023 04:16:27
  End time:               24-Dec-2023 04:34:36
  Elapsed time:           18 mins 9 secs
  Priority:               30
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       182,678,106,072 (182.6 GB)
  SD Bytes Written:       182,678,106,236 (182.6 GB)
  Rate:                   167748.5 KB/s
  Software Compression:   None
  Comm Line Compression:  71.3% 3.5:1
  Snapshot/VSS:           no
  Encryption:             no
  Accurate:               no
  Volume name(s):         FullAutoNoNextPool-04-17883|FullAutoNoNextPool-04-17884|FullAutoNoNextPool-04-17558|FullAutoNoNextPool-04-17557|FullAutoNoNextPool-04-17556|FullAutoNoNextPool-04-17554|FullAutoNoNextPool-04-17553|FullAutoNoNextPool-04-17555|FullAutoNoNextPool-04-17551
  Volume Session Id:      25
  Volume Session Time:    1703358455
  Last Volume Bytes:      11,015,014,848 (11.01 GB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

24-Dec 04:34 bacula-dir JobId 362069: Begin pruning Jobs older than 3 years .
24-Dec 04:34 bacula-dir JobId 362069: No Jobs found to prune.
24-Dec 04:34 bacula-dir JobId 362069: Begin pruning Files.
24-Dec 04:34 bacula-dir JobId 362069: No Files found to prune.
24-Dec 04:34 bacula-dir JobId 362069: End auto prune.

This is the next job, after I did a touch on one of the files.

*m
  Build OS:               amd64-portbld-freebsd14.0 freebsd 14.0-RELEASE-p1
  JobId:                  362072
  Job:                    r730-01_snapshots.2023-12-25_01.50.59_58
  Backup Level:           Incremental, since=2023-12-25 01:40:13
  Client:                 "r730-01-fd" 9.6.7 (10Dec20) amd64-portbld-freebsd14.0,freebsd,14.0-RELEASE-p1
  FileSet:                "r730-01 snapshots" 2023-12-25 01:30:49
  Pool:                   "IncrFile-04" (From Job IncPool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "bacula-sd-04-IncrFile" (From Pool resource)
  Scheduled time:         25-Dec-2023 01:50:57
  Start time:             25-Dec-2023 01:51:02
  End time:               25-Dec-2023 01:51:15
  Elapsed time:           13 secs
  Priority:               10
  FD Files Written:       1
  SD Files Written:       1
  FD Bytes Written:       3,899 (3.899 KB)
  SD Bytes Written:       4,092 (4.092 KB)
  Rate:                   0.3 KB/s
  Software Compression:   None
  Comm Line Compression:  31.8% 1.5:1
  Snapshot/VSS:           no
  Encryption:             no
  Accurate:               no
  Volume name(s):         IncrAuto-04-14732
  Volume Session Id:      28
  Volume Session Time:    1703358455
  Last Volume Bytes:      4,841 (4.841 KB)
  Non-fatal FD errors:    3
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK -- with warnings

25-Dec 01:51 bacula-dir JobId 362072: Begin pruning Jobs older than 3 years .
25-Dec 01:51 bacula-dir JobId 362072: No Jobs found to prune.
25-Dec 01:51 bacula-dir JobId 362072: Begin pruning Files.
25-Dec 01:51 bacula-dir JobId 362072: No Files found to prune.
25-Dec 01:51 bacula-dir JobId 362072: End auto prune.

Hope this helps. Merry Christmas.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

Leave a Comment

Scroll to Top