Be warned, this failed. I’m stalled and I have not completed this.

I’m going to do jails within a jail. I already do that with poudriere in a jail but here I want to test an older version of iocage before upgrading my current jail hosts to a newer version.

In this post:

• FreeBSD 12.1
• py36-iocage-1.2_3
• py36-iocage-1.2_4

This post includes my errors and mistakes. Perhaps you should proceed carefully and read it all first.

My first attempt

After installing iocage within the empty jail (to be clear, that is the jail name, not a description), I attempted this:

[dan@empty:~] $ sudo iocage fetch
No zpools found! Please create one before using iocage.
[dan@empty:~] $ zfs list
no datasets available

Oh yes, I’m going to need some jailed ZFS datasets.

For that, I need to get onto the host.

Creating the filesystem

[dan@knew:~] $ sudo zfs create system/data/empty-iocage

The jailed attribute must be set on, that ensure that mountpoint is relative to inside the jail.

The jailed setting does much more than just that. Please read up on it in the man page.

Next, I need to set that mountpoint:

[dan@knew:~] $ sudo zfs set mountpoint=/iocage/jails system/data/empty-iocage
cannot set property for 'system/data/empty-iocage': 'mountpoint' cannot be set on dataset in a non-global zone
[dan@knew:~] $ 

Oh yeah, I have to do this from within the jail now that I’ve set it.

Or I could do this:

# turn this off, so I can work with it from the host

[dan@knew:~] $ sudo zfs set jailed=off system/data/empty-iocage

# verify it is not mounted

[dan@knew:~] $ zfs get mounted system/data/empty-iocage
NAME                      PROPERTY  VALUE    SOURCE
system/data/empty-iocage  mounted   no       -

# I'm afraid it will automount when I set the mountpoint, and I'm on the host and I
# don't want this to interfere with my host

[dan@knew:~] $ sudo zfs set canmount=noauto system/data/empty-iocage

# there, set and jailed again

[dan@knew:~] $ sudo zfs set mountpoint=/iocage/jails system/data/empty-iocage
[dan@knew:~] $ sudo zfs set jailed=on system/data/empty-iocageage
[dan@knew:~] $ 

Associate the filesystem with the jail

PLEASE NOTE: data/empty-iocage is not the full name of the dataset. As mentioned in man iocage: “Takes the ZFS filesystem name without pool name”.

[dan@knew:~] $ sudo iocage set jail_zfs_dataset=data/empty-iocage empty
empty is running.
Please stop it first!
[dan@knew:~] $ 

Oh, OK then:

[dan@knew:~] $ sudo iocage stop empty
* Stopping empty
  + Executing prestop OK
  + Stopping services OK
  + Removing devfs_ruleset: 25 OK
  + Removing jail process OK
  + Executing poststop OK
[dan@knew:~] $ sudo iocage set jail_zfs_dataset=data/empty-iocage empty
jail_zfs_dataset: iocage/jails/empty/data -> data/empty-iocage
[dan@knew:~] $ sudo iocage start empty
* Ruleset 25 does not exist, using defaults
* Starting empty
  + Started OK
  + Using devfs_ruleset: 25
  + Using IP options: ip4.addr=10.55.0.21,ix0|172.16.10.10,ix0|172.16.10.11,ix0|172.16.10.12 ip4.saddrsel=1 ip4=new ip6.saddrsel=1 ip6=new
  + Starting services OK
  + Executing poststart OK
[dan@knew:~] $ 

But when I logged into that jail, I did not see my filesystem and I did not have ZFS capabilities:

[dan@empty:~] $ zfs list
no datasets available

Fixing up the jail

Going back to my poudriere in a jail blog post, I extracted and modified the settings. I used this:

sudo iocage set children_max=100 \
            allow_mount=true \
            allow_mount_zfs=true \
            allow_mount_nullfs=true \
            allow_raw_sockets=true \
            allow_socket_af=true \
            enforce_statfs=1 \
            jail_zfs=1 \
            empty

I’m saving the output of the above command here in case I ever need to know the original values:

\
[dan@knew:~] $ sudo iocage ge stop empty
* Stopping empty
  + Executing prestop OK
  + Stopping services OK
  + Removing devfs_ruleset: 25 OK
  + Removing jail process OK
  + Executing poststop OK
[dan@knew:~] $ sudo iocage set children_max=100 \
>             allow_mount=true \
>             allow_mount_zfs=true \
>             allow_mount_nullfs=true \
>             allow_raw_sockets=true \
>             allow_socket_af=true \
>             enforce_statfs=1 \
>             jail_zfs=1 \
>             empty
children_max: 0 -> 100
allow_mount: 0 -> 1
allow_mount_zfs: 1 -> 1
allow_mount_nullfs: 0 -> 1
allow_raw_sockets: 0 -> 1
allow_socket_af: 0 -> 1
enforce_statfs: 2 -> 1
jail_zfs: 0 -> 1
[dan@knew:~] $ sudo iocage ge start empty
* Ruleset 25 does not exist, using defaults
* Starting empty
  + Started OK
  + Using devfs_ruleset: 25
  + Using IP options: ip4.addr=10.55.0.21,ix0|172.16.10.10,ix0|172.16.10.11,ix0|172.16.10.12 ip4.saddrsel=1 ip4=new ip6.saddrsel=1 ip6=new
  + Starting services OK
  + Executing poststart OK
[dan@knew:~] $ 

Now when I ssh into the jail I see:

[dan@empty:~] $ zfs list
NAME                       USED  AVAIL  REFER  MOUNTPOINT
system                    38.1T  28.7T   320K  none
system/data               37.9T  28.7T   329K  none
system/data/empty-iocage   201K  28.7T   201K  /iocage/jails
[dan@empty:~] $ 

Score!

Back to iocage!

Let’s get this party started!

[dan@empty:~] $ sudo iocage fetch
Creating system/iocage
permission denied

Oh. It turns out, iocage wants the iocage filesystem at the top of the zpool.

I’ve raised an issue to confirm this.