Getting started with iocage for jails on FreeBSD

All of my previous jail hosts have used ezjail. I have recently played with both bsdploy and iocage.

The primary purpose of this post is showing you how to install FreeBSD 10.1, which has already been installed as a zfs root layout. I have also configured that host to use Ansible and sudo-auth via ssh-agent, but neither of those steps are necessary for what I’m about to show you.

What to expect

You will see the following death defying feats of sysadmin:

1. install iocage
2. configure iocage to create a jail
3. create a 10.1 jail
4. start that jail
5. profit

Installing iocage

I installed iocage with this command:

pkg install iocage

Configure iocage to create a jail

In this step, we tell iocage to download an image of FreeBSD. I accept the default selection.

# iocage fetch
  please select a pool for iocage jails [system]:

In my case, I have one zpool on my server, system. Yours may be called zroot, tank, or something else. NOTE this is a zpool name, not a filesystem name (e.g. system/usr/local).

I selected the default, and the command continued:

Supported releases are: 
   10.1-RELEASE
   10.0-RELEASE
    9.3-RELEASE
    9.2-RELEASE
    9.1-RELEASE
Please select a release [10.1-RELEASE]: 
base.txz                                      100% of   63 MB   10 MBps 00m06s
doc.txz                                       100% of 1395 kB 7014 kBps 00m01s
lib32.txz                                     100% of   15 MB   10 MBps 00m01s
Exctracting: base.txz
Exctracting: doc.txz
Exctracting: lib32.txz
* Updating base jail..
Looking up update.FreeBSD.org mirrors... 5 mirrors found.
Fetching metadata signature for 10.1-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

The following files will be added as part of updating to 10.1-RELEASE-p6:
/usr/src/crypto/openssl/util/mkbuildinf.pl

The following files will be updated as part of updating to 10.1-RELEASE-p6:
/bin/freebsd-version
/lib/libc.so.7
/lib/libcrypto.so.7
/rescue/[
/rescue/atmconfig
/rescue/badsect
...
...
/usr/share/openssl/man/man3/x509.3.gz
/var/db/mergemaster.mtree
     Installing updates...   install: /iocage/releases/10.1-RELEASE/root//usr/src/crypto/openssl/util/mkbuildinf.pl: No such file or directory
 done.
#

What just happened?

You will notice that the distribution files are fetched and then freebsd-update is invoked to get the latest patches for that release.

Look at your filesystems!

On my server, this is what I have installed after the above:

system/iocage                              509M   436G    96K  /iocage
system/iocage/download                    80.7M   436G    96K  /iocage/download
system/iocage/download/10.1-RELEASE       80.6M   436G  80.6M  /iocage/download/10.1-RELEASE
system/iocage/jails                         96K   436G    96K  /iocage/jails
system/iocage/releases                     428M   436G    96K  /iocage/releases
system/iocage/releases/10.1-RELEASE        428M   436G    96K  /iocage/releases/10.1-RELEASE
system/iocage/releases/10.1-RELEASE/root   428M   436G   428M  /iocage/releases/10.1-RELEASE/root

Here is what you can find in those locations.

This is what was fetched:

# ls /iocage/download/10.1-RELEASE
base.txz	doc.txz		lib32.txz

This is what was extracted from that fetch:

# ls /iocage/releases/10.1-RELEASE/root
.cshrc		boot		libexec		rescue		tmp
.profile	dev		media		root		usr
COPYRIGHT	etc		mnt		sbin		var
bin		lib		proc		sys

Creating the first jail

Let’s create a jail. It will have one IP address which will be added as an alias by iocage when the jail is started, and removed when the jail is stopped. The alias will be added to the em0 device.

The tag allows you to refer to the newly created jail as wiki.bsdan.org.

iocage create tag=wiki.bsdcan.org ip4_addr="em0|10.1.2.195"
Configuring jail..
** interfaces=vnet0:bridge0,vnet1:bridge1
** vnet=off
** host_hostname=fc6024ec-c4ff-11e4-9bc4-00259094119e
** hostname=fc6024ec-c4ff-11e4-9bc4-00259094119e
** ip4_addr=em0|10.1.2.195
** ip4_saddrsel=1
** ip4=new
** ip6_addr=none
** ip6_saddrsel=1
** ip6=new
** defaultrouter=none
** exec_fib=0
** devfs_ruleset=4
** mount_devfs=1
** exec_start=/bin/sh /etc/rc
** exec_stop=/bin/sh /etc/rc.shutdown
** exec_prestart=/usr/bin/true
** exec_prestop=/usr/bin/true
** exec_poststop=/usr/bin/true
** exec_poststart=/usr/bin/true
** exec_clean=1
** exec_timeout=60
** stop_timeout=30
** exec_jail_user=root
** exec_system_jail_user=0
** exec_system_user=root
** mount_fdescfs=1
** mount_procfs=0
** enforce_statfs=2
** children_max=0
** login_flags=-f root
** securelevel=2
** allow_set_hostname=1
** allow_sysvipc=0
** allow_raw_sockets=0
** allow_chflags=0
** allow_mount=0
** allow_mount_devfs=0
** allow_mount_nullfs=0
** allow_mount_procfs=0
** allow_mount_tmpfs=0
** allow_mount_zfs=0
** allow_quotas=0
** allow_socket_af=0
** host_hostuuid=fc6024ec-c4ff-11e4-9bc4-00259094119e
** memoryuse=8G:log
** memorylocked=off
** vmemoryuse=off
** maxproc=off
** cputime=off
** pcpu=off
** datasize=off
** stacksize=off
** coredumpsize=off
** openfiles=off
** pseudoterminals=off
** swapuse=off
** nthr=off
** msgqqueued=off
** msgqsize=off
** nmsgq=off
** nsemop=off
** nshm=off
** shmsize=off
** wallclock=off
** tag=wiki.bsdcan.org
** template=no
** rlimits=off
** boot=off
** notes=none
** owner=root
** priority=99
** last_started=none
** type=jail
** hostid=49434d53-0200-9094-2500-949025009e11
** cpuset=off
** jail_zfs=off
** jail_zfs_dataset=iocage/jails/fc6024ec-c4ff-11e4-9bc4-00259094119e/root/data
** release=10.1-RELEASE
** sync_target=none
** sync_tgt_zpool=none

Now what do you have?

Now you have the following filesystems:

system/iocage/jails                                                  428M   435G    96K  /iocage/jails
system/iocage/jails/dda0b520-c500-11e4-9bc4-00259094119e             428M  14.6G    96K  /iocage/jails/dda0b520-c500-11e4-9bc4-00259094119e
system/iocage/jails/dda0b520-c500-11e4-9bc4-00259094119e/root        428M  14.6G   428M  /iocage/jails/dda0b520-c500-11e4-9bc4-00259094119e/root
system/iocage/jails/dda0b520-c500-11e4-9bc4-00259094119e/root/data    96K  14.6G    96K  none

You will notice that /iocage/jails has been created.

And yes, the ‘name’ of the new jail is a UUID (dda0b520-c500-11e4-9bc4-00259094119e).

Don’t be put off by this. Give it time. You’ll learn it’s not as important as you think it is and it lends itself to much more flexibility than are apparent upon first impression.

List your jails

# iocage list
UUID                                  BOOT  STATE  TAG
dda0b520-c500-11e4-9bc4-00259094119e  off   down   wiki.bsdcan.org

You can see that the boot flag is off. This means the jail will not be started at boot time. See the next section for more information.

Starting jails automatically

The following is taken from How to turn on auto boot.

To allow iocage to start your jails at boot time, add this to /etc/rc.conf:

iocage_enable="YES"

Individual jails can be marked for starting at by issuing this command:

iocage set boot=on www.bsdcan.org

You can specify the order for jails to start by setting their priority:

iocage set priority=20 wiki.bsdan.org

Lower value means higher boot priority (i.e. boot sooner).

Jails have other properties

As you might guess from the output of jail creation and the previous section, jails have many properties. The hostname defaults to the UUID, but you can specify something more appropriate if you want:

# iocage set hostname=wiki.bsdcan.org dda0b520

When a UUID is required, you only have to supply enough to identify a given UUID. Thus, only dda0b520 is shown above. It’s easy to copy/paste.

You can view the hostname with this command. Assigning the same value to tag and hostname is useful

# iocage get hostname wiki.bsdcan.org
wiki.bsdcan.org

chroot is your friend

You can chroot into your jail without running the jail:

# iocage chroot wiki.bsdcan.org
root@tallboy:/ # 

This can be useful for editing various files. I used this to edit /etc/rc.conf and set the hostname in that file.

To get out of the chroot, press CONTROL-D or type exit.

Starting a jail

# iocage start wiki.bsdcan.org
* Starting dda0b520-c500-11e4-9bc4-00259094119e (wiki.bsdcan.org)
  + Started (shared IP mode) OK
  + Starting services        OK

If you list your jails now, this one will be marked as STATE = up:

# iocage list
UUID                                  BOOT  STATE  TAG
dda0b520-c500-11e4-9bc4-00259094119e  off   up     wiki.bsdcan.org

Accessing a jail console

This command allows you access to the jail without logging into it:

# iocage console wiki.bsdcan.org
FreeBSD 10.1-RELEASE-p5 (GENERIC) #0: Tue Jan 27 08:55:07 UTC 2015

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
root@wiki:~ # 

You are now in the jail as if you had logged in at the console. For example, I can now start sshd manually. Watch as it creates missing keys automatically:

# service sshd onestart
Generating RSA1 host key.
2048 9d:49:74:76:31:54:0f:8d:5f:0b:76:d9:ab:e1:3b:c6  root@wiki.bsdcan.org (RSA1)
Generating RSA host key.
2048 e3:39:60:9b:f9:9f:de:cf:60:f3:8b:6c:13:a2:ee:75  root@wiki.bsdcan.org (RSA)
Generating DSA host key.
1024 f4:5b:ae:b0:28:0b:35:2d:77:c2:99:c1:6b:7f:2d:6c  root@wiki.bsdcan.org (DSA)
Generating ECDSA host key.
256 a1:e0:1d:c4:2c:10:3a:2c:06:7f:c9:ef:d1:cc:20:b0  root@wiki.bsdcan.org (ECDSA)
Generating ED25519 host key.
256 9e:53:5a:9f:d5:05:f5:d4:71:88:f1:d6:ab:36:75:0a  root@wiki.bsdcan.org (ED25519)
Performing sanity check on sshd configuration.
Starting sshd.
root@wiki:~ # 

Stopping a jail

Stopping a jail is easy:

# iocage stop wiki.bsdcan.org
* Stopping dda0b520-c500-11e4-9bc4-00259094119e (wiki.bsdcan.org)
  + Running pre-stop         OK
  + Stopping services        OK
  + Removing jail process    OK
  + Running post-stop        OK

Destroying a jail

If you wish to remove a jail from a system, you destroy it:

# iocage destroy wiki.bsdcan.org
 
  WARNING: this will destroy jail dda0b520-c500-11e4-9bc4-00259094119e
  Dataset: system/iocage/jails/dda0b520-c500-11e4-9bc4-00259094119e
 
  Are you sure ? Y[n]: Y
  Destroying: dda0b520-c500-11e4-9bc4-00259094119e

There is no trace of the jail left. You must type an upper-case ‘Y’ for this command to succeed. Everything else will fail.

Loads of potential

I see a lot of potential here for iocage. I am going to use it on this host and see how it goes. I like the direction the project is headed. The devs are active and responsive. The project is on github, and that makes it easy for you to contribute code/docs.

I want to investigate:

• the record command and the template property. Both sound like a great way to create jails which will be used as templates for cloning other jails.
• the update command to update jail to latest patch level.
• the upgrade command for upgrading a jail to specified RELEASE.
• the pkglist property, which specifies a text file containing one package per line. These packages will be auto installed when a jail is created.

I’m also working on upgrading my Bacula script to cater for iocage.

I urge you to give it a try.