Feb 282021

Tonight I got blocked by iocage and handling a ZFS filesystem from within an iocage jail.

These are the steps I followed to convert that jail from iocage to a vanilla jail.

The variable

To make this solution easier to use, at least for me, on future jail migrations, I have these variables:

$ export MYJAIL=devgit-nginx01      # the name of your jail
$ export IOCAGE_ZPOOL=system/iocage/jails # the zfs filesystem where your iocage jails reside
$ export VANILLA_ZPOOL=system/jails # the zfs filesystem where the vanilla jails will reside

For example, my devgit-nginx01 jail is at:

[dan@knew:~] $ zfs list system/iocage/jails/mysql56
NAME                          USED  AVAIL  REFER  MOUNTPOINT
system/iocage/jails/mysql56  11.5G  20.6T   210K  /iocage/jails/mysql56

In my case, I have to create the new destination filesystem first.

$ sudo zfs create -o compression=on -o atime=off -o mountpoint=/jails $VANILLA_ZPOOL

Copy the jail

First, I turn off the jail, just to be clean.

EDIT: On 2021-03-20 I amended this post to use a variable MYJAIL. This will make it easier for me to copy & paste the commands as I go along.

$ sudo iocage stop $MYJAIL
* Stopping devgit-nginx01
  + Executing prestop OK
  + Stopping services OK
  + Removing devfs_ruleset: 1021 OK
  + Removing jail process OK
  + Executing poststop OK

Copying the jail is not required, but it is cleaner. It leaves you a fallback position.

I snapshot the jail and send it away:

$ sudo zfs snapshot $IOCAGE_ZPOOL/$MYJAIL/root@vanilla
$ sudo zfs send $IOCAGE_ZPOOL/$MYJAIL/root@vanilla | sudo zfs recv $VANILLA_ZPOOL/$MYJAIL

This is what we wind up with:

NAME                          USED  AVAIL  REFER  MOUNTPOINT
system/jails/devgit-nginx01  8.09G  12.6T  8.09G  /jails/devgit-nginx01

$ cd /jails/$MYJAIL
$ ls -l
total 202
-r--r--r--   1 root    wheel  6177 Dec 13 12:51 COPYRIGHT
drwxr-xr-x   2 root    wheel    46 Feb 24 14:25 bin
drwxr-xr-x  10 root    wheel    63 Feb 24 14:25 boot
drwxr-xr-x   7 root    wheel     7 Jul  6  2020 cache
dr-xr-xr-x   2 root    wheel     2 Nov  1  2019 dev
drwxr-xr-x  25 root    wheel   109 Feb 24 14:25 etc
drwxr-xr-x   5 root    wheel    62 Feb 24 14:25 lib
drwxr-xr-x   3 root    wheel     5 Dec 13 12:50 libexec
drwxr-xr-x   2 root    wheel     2 Nov  1  2019 media
drwxr-xr-x   2 root    wheel     2 Nov  1  2019 mnt
drwxr-xr-x   2 root    wheel     2 Nov  1  2019 net
dr-xr-xr-x   2 root    wheel     2 Nov  1  2019 proc
drwxr-xr-x   2 root    wheel   150 Feb 24 14:25 rescue
drwxr-xr-x   3 root    wheel    11 Feb 12 00:44 root
drwxr-xr-x   2 root    wheel   137 Feb 24 14:25 sbin
drwxr-xr-x   2 root    wheel     2 Jul  6  2020 signals
lrwxr-xr-x   1 root    wheel    11 May  7  2020 sys -> usr/src/sys
drwxrwxrwt  23 root    wheel   119 Feb 28 18:39 tmp
drwxr-xr-x  15 root    wheel    15 Jun 21  2020 usr
drwxr-xr-x  28 nagios  wheel    28 Feb 28 18:39 var

Disable the iocage jail

$ sudo iocage set boot=off $MYJAIL
boot: 1 -> 0

Create an entry for jail.conf

Add a new entry to /etc/jail.conf, based on an existing entry, stagegit-nginx01:

devgit-nginx01 {
    host.hostname = "devgit-nginx01.int.unixathome.org";
    ip4.addr = "ix2|";

    # You don't need the rest of this you're doing zfs management
    # or mounting stuff in your jail


    # because want to mount zfs, we do that before the rc start up
    # not sure if we MUST do it in that order.
    exec.start="zfs mount -a";
    exec.start+="/bin/sh /etc/rc";

    exec.created+="zfs set jailed=on nvd/freshports/devgit-nginx01/var/db/freshports/cache";
    exec.created+="zfs jail $name    nvd/freshports/devgit-nginx01/var/db/freshports/cache";

The fstab file

You possibly won’t need an fstab file, but I did.

$ cd /etc/
$ sudo cp -i fstab.stagegit-nginx01 fstab.$MYJAIL

I updated that file to look like this, mostly mount points etc.

$ cat /etc/fstab.$MYJAIL
# The ingress jail generates and maintains some static HTML for use by the nginx jail
/iocage/jails/devgit-ingress01/root/var/db/freshports/cache/html /jails/devgit-nginx01/var/db/freshports/cache/html   nullfs  ro,nosuid,noexec  0   0

You probably don’t need that.

Start the jail

$ sudo service jail start $MYJAIL
Starting jails: devgit-nginx01.

ssh into the jail

Let’s go see

$ zfs list
NAME                                                               USED  AVAIL  REFER  MOUNTPOINT
nvd                                                                185G  29.6G    23K  /nvd
nvd/freshports                                                     185G  29.6G    23K  none
nvd/freshports/devgit-nginx01                                      336K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var                                  312K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db                               288K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports                    264K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache              240K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/categories    24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/commits       24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/daily         24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/general       24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/news          24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/packages      24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/pages         24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/ports         24K  29.6G    24K  none
nvd/freshports/devgit-nginx01/var/db/freshports/cache/spooling      24K  29.6G    24K  none


Changes for backups

This change of location requires a change to the snapshot backup script. Fortunately, it was just a rearrangement and an addition of a for loop.


You might want to go back and delete the snapshots.

sudo zfs destroy $IOCAGE_ZPOOL/$MYJAIL/root@vanilla
sudo zfs destroy $VANILLA_ZPOOL/$MYJAIL@vanilla
Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive