I have a jail on r730-01 called dvl-ingress01.
Here are some filesystems related to that jail:
[11:50 r730-01 dvl ~] % zfs list | grep dvl-ingress01 | grep repos NAME USED AVAIL REFER MOUNTPOINT data02/freshports/dvl-ingress01/ingress/repos 8.65G 1011G 112K /jails/dvl-ingress01/var/db/ingress/repos data02/freshports/dvl-ingress01/ingress/repos/doc 706M 1011G 678M /jails/dvl-ingress01/var/db/ingress/repos/doc data02/freshports/dvl-ingress01/ingress/repos/ports 3.53G 1011G 2.30G /jails/dvl-ingress01/var/db/ingress/repos/ports data02/freshports/dvl-ingress01/ingress/repos/src 4.44G 1011G 2.73G /jails/dvl-ingress01/var/db/ingress/repos/src data02/freshports/dvl-ingress01/repos 120K 1011G 120K /jails/dvl-ingress01/var/db/ingress/repos
That is not a complete list, but it illustrates the filesystems I’m working with for this blog post.
I’m sure many of you have done this before. You need some data separate from the jail operating system. You want it to persist between jail upgrades. One of the easiest ways to do that is mounting the data like the above.
Case in point: I have two filesystems mounted at /jails/dvl-ingress01/var/db/ingress/repos:
- data02/freshports/dvl-ingress01/ingress/repos
- data02/freshports/dvl-ingress01/repos
It is the second one which I want to delete. How do I know? It has no data (only 120K USED). I also recall relocating the repos from that dataset up to the others. This is a lingering crufty filesystem which can be deleted. If I wanted to be extra safe, I could first rename it.
But wait, I can’t!
[11:38 r730-01 dvl ~] % sudo zfs umount data02/freshports/dvl-ingress01/repos cannot unmount '/jails/dvl-ingress01/var/db/ingress/repos': pool or dataset is busy
Notice that the mount point is mentioned. To me, that signals that something else is mounted at that same location and that other mount is blocking this umount.
Let’s umount the other stuff:
[11:47 r730-01 dvl ~] % sudo zfs umount data02/freshports/dvl-ingress01/ingress/repos/doc [11:47 r730-01 dvl ~] % sudo zfs umount data02/freshports/dvl-ingress01/ingress/repos/ports [11:47 r730-01 dvl ~] % sudo zfs umount data02/freshports/dvl-ingress01/ingress/repos/src [11:47 r730-01 dvl ~] %
And now I can do what I wanted to do:
[11:47 r730-01 dvl ~] % sudo zfs umount data02/freshports/dvl-ingress01/repos [11:57 r730-01 dvl ~] % sudo zfs set canmount=noauto data02/freshports/dvl-ingress01/repos [11:57 r730-01 dvl ~] % sudo zfs set mountpoint=none data02/freshports/dvl-ingress01/repos [11:58 r730-01 dvl ~] % sudo zfs rename data02/freshports/dvl-ingress01/repos data02/freshports/dvl-ingress01/repos.DELETE.ME.after.2025.09.30 [11:58 r730-01 dvl ~] % zfs list | grep /jails/dvl-ingress01/var/db/ingress/repos data02/freshports/dvl-ingress01/ingress/repos 8.65G 1012G 112K /jails/dvl-ingress01/var/db/ingress/repos data02/freshports/dvl-ingress01/ingress/repos/doc 706M 1012G 678M /jails/dvl-ingress01/var/db/ingress/repos/doc data02/freshports/dvl-ingress01/ingress/repos/ports 3.53G 1012G 2.30G /jails/dvl-ingress01/var/db/ingress/repos/ports data02/freshports/dvl-ingress01/ingress/repos/src 4.44G 1012G 2.73G /jails/dvl-ingress01/var/db/ingress/repos/src [11:58 r730-01 dvl ~] %
There, that’s better.
Let’s automate
This situation, where a mounted filesytem prevents me from doing what I want to do, occurs infrequently yet is annoying enough for me to automate it. These are the entries I add to the jail configuration file for this particular jail:
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos/doc";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos/ports";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos/src";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos/doc";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos/ports";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos/src";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos";
Taking that one step farther, I decide to turn off automount on those datasets:
[12:06 r730-01 dvl ~] % sudo zfs set canmount=noauto data02/freshports/dvl-ingress01/ingress/repos/doc [12:06 r730-01 dvl ~] % sudo zfs set canmount=noauto data02/freshports/dvl-ingress01/ingress/repos/src [12:06 r730-01 dvl ~] % sudo zfs set canmount=noauto data02/freshports/dvl-ingress01/ingress/repos/ports [12:06 r730-01 dvl ~] % zfs get -t filesystem -r canmount data02/freshports/dvl-ingress01/ingress/repos NAME PROPERTY VALUE SOURCE data02/freshports/dvl-ingress01/ingress/repos canmount noauto local data02/freshports/dvl-ingress01/ingress/repos/doc canmount noauto local data02/freshports/dvl-ingress01/ingress/repos/ports canmount noauto local data02/freshports/dvl-ingress01/ingress/repos/src canmount noauto local
I kept repeating that until I had them all.
Now the jail configuration looks like:
exec.created+="zfs set jailed=on data02/freshports/jailed/dvl-ingress01";
exec.created+="zfs jail $name data02/freshports/jailed/dvl-ingress01";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ports";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos/doc";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos/ports";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/repos/src";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/dvl-src";
# the html dataset is fullfs mounted by dvl-nginx01 - we need to keep it
# mounted.
# exec.poststart+="zfs mount data02/freshports/dvl-ingress01/freshports/cache";
# exec.poststart+="zfs mount data02/freshports/dvl-ingress01/freshports/cache/html";
# exec.poststart+="zfs mount data02/freshports/dvl-ingress01/freshports/cache/spooling";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/freshports/message-queues";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/freshports/message-queues/archive";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/latest_commits";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/ingress/message-queues";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/modules";
exec.poststart+="zfs mount data02/freshports/dvl-ingress01/scripts";
# doing this allows me to do this after stopping the jail:
# [19:09 r730-01 dvl ~] % sudo zfs rename data02/freshports/jailed/dvl-ingress01/jails/freshports-14.3 data02/freshports/jailed/dvl-ingress01/jails/freshports
#
exec.poststop+="zfs set jailed=off data02/freshports/jailed/dvl-ingress01";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ports";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos/doc";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos/ports";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos/src";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/repos";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/dvl-src";
# the html dataset is fullfs mounted by dvl-nginx01 - we need to keep it
# mounted.
# exec.poststop+="zfs umount data02/freshports/dvl-ingress01/freshports/cache";
# exec.poststop+="zfs umount data02/freshports/dvl-ingress01/freshports/cache/html";
# exec.poststop+="zfs umount data02/freshports/dvl-ingress01/freshports/cache/spooling";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/freshports/message-queues";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/freshports/message-queues/archive";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/latest_commits";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/ingress/message-queues";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/modules";
exec.poststop+="zfs umount data02/freshports/dvl-ingress01/scripts";
Mostly for convenience.
Thanks for coming to my TED talk.
