Nov 112019
 

I’m going to implement zfstools on all my ZFS-based hosts today. I first started using this tool in July 2019.

In this post:

  1. FreeBSD 12.0 and 12.1
  2. zfstools 0.3.6_1
  3. sanoid-2.0.1_2

Local snapshots only

I will be using zfstool only for creating local snapshots. If I wanted snapshots for sending to other hosts, I would probably use a tool such as sysutils/sanoid, which is policy-driven solution for snapshot management and replication.

For now, there is no replication in my plans.

But let’s look at sanoid first

Hmm, as I type this, I want to check out sanoid.

Let’s build it and install it on a server, and see if it does the one thing I’m afraid I’ll miss from zfstools: deletion of zero-size snapshots.

The test sandoid configuration

My test configuration looks like this:

[system/usr/home]
        use_template = production

[template_production]
        frequent_period = 1
        frequently = 5
        hourly     = 36
        daily      = 30
        monthly    = 3
        yearly     = 0
        autosnap   = yes
        autoprune  = yes

This is just for testing. That’s 5 frequent snapshot, every minute or so.

When setting values, it is useful to know the defaults.

Let’s run it.

Some test runs

For these tests, I ran this from the command line:

$ sudo /usr/local/bin/sanoid --cron
$ zfs list -r -t snapshot system/usr/home
NAME                                                      USED  AVAIL  REFER  MOUNTPOINT
system/usr/home@autosnap_2019-11-11_18:20:17_daily           0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:20:17_monthly         0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:20:17_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:20:17_hourly          0      -  22.3G  -

That seems to make sense. One of everything.

Then I decided to delete all of them:

$ sudo zfs destroy system/usr/home@autosnap_2019-11-11_18:20:17_daily
$ sudo zfs destroy system/usr/home@autosnap_2019-11-11_18:20:17_monthly
$ sudo zfs destroy system/usr/home@autosnap_2019-11-11_18:20:17_frequently
$ sudo zfs destroy system/usr/home@autosnap_2019-11-11_18:20:17_hourly

Run it again:

$ sudo /usr/local/bin/sanoid --cron
$ zfs list -r -t snapshot system/usr/home
NAME                                                      USED  AVAIL  REFER  MOUNTPOINT
system/usr/home@autosnap_2019-11-11_18:25:12_frequently      0      -  22.3G  -

OK, just one. Why?

One more time.

$ sudo /usr/local/bin/sanoid --cron
$ zfs list -r -t snapshot system/usr/home
NAME                                                      USED  AVAIL  REFER  MOUNTPOINT
system/usr/home@autosnap_2019-11-11_18:25:12_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_daily           0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_hourly          0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_monthly         0      -  22.3G  -

That seems a little better. Let’s try another.

$ sudo /usr/local/bin/sanoid --cron
$ zfs list -r -t snapshot system/usr/home
NAME                                                      USED  AVAIL  REFER  MOUNTPOINT
system/usr/home@autosnap_2019-11-11_18:25:12_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_daily           0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_hourly          0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_monthly         0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:31:18_frequently      0      -  22.3G  -

Well, one more frequently, of zero size.

Enough testing

After lunch, a bit of tv, and some more test runs. I have this:

$ sudo /usr/local/bin/sanoid --cron
$ zfs list -r -t snapshot system/usr/home
NAME                                                      USED  AVAIL  REFER  MOUNTPOINT
system/usr/home@autosnap_2019-11-11_18:27:32_daily           0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_hourly          0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:27:32_monthly         0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:31:18_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:32:41_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_18:50:38_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_19:00:40_hourly          0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_19:00:40_frequently      0      -  22.3G  -
system/usr/home@autosnap_2019-11-11_19:22:22_frequently      0      -  22.3G  -

This seems to work as advertised.

The running configuration

This is the configuration I am using now:

[dan@tallboy:/usr/home] $ cat /usr/local/etc/sanoid/sanoid.conf
[system]
	use_template = production
	recursive    = yes

[template_production]
	frequently = 5
	hourly     = 48
	daily      = 90
	monthly    = 6
	yearly     = 2
	autosnap   = yes
	autoprune  = yes

[template_ignore]
	autoprune = no
	autosnap  = no
	monitor   = no
[dan@tallboy:/usr/home] $ 

Monitoring

sanoid is the first tool I have enountered with some with built-in monitoring for Nagios.

$ /usr/local/bin/sanoid --monitor-capacity
OK ZPOOL system : 30%

$ /usr/local/bin/sanoid --monitor-health
OK ZPOOL system : ONLINE {Size:452G Free:315G Cap:30%} 

$ /usr/local/bin/sanoid --monitor-snapshots
OK: all monitored datasets (system, system/bootenv, system/bootenv/11.1-RELEASE, system/bootenv/default, system/iocage, system/iocage/.defaults, system/iocage/base, system/iocage/base/10.2-RELEASE, system/iocage/base/10.2-RELEASE/root, system/iocage/base/10.2-RELEASE/root/bin, system/iocage/base/10.2-RELEASE/root/boot, system/iocage/base/10.2-RELEASE/root/lib, system/iocage/base/10.2-RELEASE/root/libexec, system/iocage/base/10.2-RELEASE/root/rescue, system/iocage/base/10.2-RELEASE/root/sbin, system/iocage/base/10.2-RELEASE/root/usr, system/iocage/base/10.2-RELEASE/root/usr/bin, system/iocage/base/10.2-RELEASE/root/usr/include, system/iocage/base/10.2-RELEASE/root/usr/lib, system/iocage/base/10.2-RELEASE/root/usr/lib32, system/iocage/base/10.2-RELEASE/root/usr/libdata, system/iocage/base/10.2-RELEASE/root/usr/libexec, system/iocage/base/10.2-RELEASE/root/usr/sbin, system/iocage/base/10.2-RELEASE/root/usr/share, system/iocage/base/10.2-RELEASE/root/usr/src, system/iocage/base/10.3-RELEASE, system/iocage/base/10.3-RELEASE/root, system/iocage/base/10.3-RELEASE/root/bin, system/iocage/base/10.3-RELEASE/root/boot, system/iocage/base/10.3-RELEASE/root/lib, system/iocage/base/10.3-RELEASE/root/libexec, system/iocage/base/10.3-RELEASE/root/rescue, system/iocage/base/10.3-RELEASE/root/sbin, system/iocage/base/10.3-RELEASE/root/usr, system/iocage/base/10.3-RELEASE/root/usr/bin, system/iocage/base/10.3-RELEASE/root/usr/include, system/iocage/base/10.3-RELEASE/root/usr/lib, system/iocage/base/10.3-RELEASE/root/usr/lib32, system/iocage/base/10.3-RELEASE/root/usr/libdata, system/iocage/base/10.3-RELEASE/root/usr/libexec, system/iocage/base/10.3-RELEASE/root/usr/sbin, system/iocage/base/10.3-RELEASE/root/usr/share, system/iocage/base/10.3-RELEASE/root/usr/src, system/iocage/download, system/iocage/download/10.1-RELEASE, system/iocage/download/10.2-RELEASE, system/iocage/download/10.3-RELEASE, system/iocage/download/11.1-RELEASE, system/iocage/images, system/iocage/jails, system/iocage/jails/clavin2, system/iocage/jails/clavin2/root, system/iocage/jails/mailjail2, system/iocage/jails/mailjail2/root, system/iocage/jails/mx-ingress02.freshports.org, system/iocage/jails/mx-ingress02.freshports.org/root, system/iocage/jails/ns1, system/iocage/jails/ns1/root, system/iocage/jails/ns2.unixathome.org, system/iocage/jails/ns2.unixathome.org/root, system/iocage/jails/tallboy-mqtt, system/iocage/jails/tallboy-mqtt/root, system/iocage/jails/tallboy-mqtt/root.cloned.DELETE.ME.after.2020.01.01, system/iocage/jails/wikis, system/iocage/jails/wikis/root, system/iocage/log, system/iocage/releases, system/iocage/releases/10.1-RELEASE, system/iocage/releases/10.1-RELEASE/root, system/iocage/releases/10.2-RELEASE, system/iocage/releases/10.2-RELEASE/root, system/iocage/releases/10.3-RELEASE, system/iocage/releases/10.3-RELEASE/root, system/iocage/releases/11.1-RELEASE, system/iocage/templates, system/jails.DELETEME.AFTER.2020.01.01, system/jails.DELETEME.AFTER.2020.01.01/releases, system/tmp, system/usr, system/usr/home, system/usr/local, system/usr/obj, system/usr/ports, system/usr/src, system/var, system/var/crash, system/var/db, system/var/db/pkg, system/var/empty, system/var/log, system/var/mail, system/var/run, system/var/tmp) have fresh snapshots
[dan@tallboy:/usr/home] $ 

All tools should be following this example.

Running from cron

If I was going to implement sanoid, I’d follow on from the above with this crontab entry:

# mail any output to `dan', no matter whose crontab this is
MAILTO=dan@example.org

PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
 
*/5 * * * * root /usr/local/bin/sanoid --cron

I would place the above into this file: /usr/local/etc/cron.d/sanoid

Why there?

  • Simplicity.
  • When upgrading the host, I don’t have to merge anything with respect to /etc/crontab.
  • man 8 cron says that directory is for third-party package provided crontab files.

zfstools

I used zfstools on r710-01. Ironically, that host is powered off now. I pulled up the crontab for that host via Bacula. It contained this:

[dan@r720-01:~] $ sudo cat /tmp/bacula-restores/usr/local/etc/cron.d/zfstools
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin

15,30,45 * * * * root /usr/local/sbin/zfs-auto-snapshot frequent  4
0        * * * * root /usr/local/sbin/zfs-auto-snapshot hourly   24
7        0 * * * root /usr/local/sbin/zfs-auto-snapshot daily     7
14       0 * * 7 root /usr/local/sbin/zfs-auto-snapshot weekly    4
28       0 1 * * root /usr/local/sbin/zfs-auto-snapshot monthly  12
[dan@r720-01:~] $ 

As I said earlier, one feature of zfstools I would like to see on other applications is the auto-deletion of zero-size snapshots. For example, in the above hourly snapshots, the 24 snapshots might be spread over a longer than 24 hour period if data is not written in every hour.

For example, in the following, you can see that my 24 hourly snapshots start on Aug 8 and run through to Sep 26. This is a dataset which does not update on a frequent basis. I argue that this approach is more useful to me than 24 zero-sized snapshots.

tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-08-31-06h00    13.8M      -  9.80G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-01-13h00    1.91M      -  9.79G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-01-14h00    1.93M      -  9.74G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-02-18h00    53.3M      -  9.78G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-02-19h00    53.3M      -  9.78G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-03-05h00    63.3M      -  9.79G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-05-14h00    2.08M      -  9.69G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-05-19h00    1.94M      -  9.58G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-05-20h00    2.58M      -  9.58G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-05-21h00    1.94M      -  9.58G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-06-15h00    1.96M      -  9.58G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-06-16h00     468K      -  9.65G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-06-18h00     468K      -  9.65G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-07-04h00    4.18M      -  9.50G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-08-04h00    53.4M      -  9.50G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-15-05h00    3.28M      -  9.52G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-17-05h00    42.2M      -  9.81G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-19-05h00     648K      -  9.70G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-19-11h00     296K      -  9.70G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-19-13h00    2.05M      -  9.70G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-20-11h00    2.08M      -  9.70G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-20-13h00    2.06M      -  9.70G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-22-11h00     240K      -  9.73G  -
tank_fast/poudriere/data/packages@zfs-auto-snap_hourly-2019-09-26-14h00        0      -  9.73G  -

This is very impressive feature and I’d like to see it as an option in all such tools.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive