In my work on FreshPorts, I’ve learned more than the average non-committer about the FreeBSD ports tree and its oddities. Just about everything learns about a port it gets from ‘make -V’. Even this method can be fraught with difficulty. In an attempt to get the fairest information possible, FreshPorts uses a chroot solution, first proposed by Ryan Steinmetz
Over the past few months, I’ve been using FreeBSD jails to host most things I do, including my FreshPorts development environment. The website is in one jail, my development environment is in another. Another jail hosts my svn server, and the PostgreSQL server is in a host system. I’ve quite taken this separation of tasks and environments.
Tonight this fought back. But I think I’ve found a solution.
I was upgrading my servers (via freebsd-update) to pick up the latest security-related fixes. My main server didn’t come back.
The problem was these entries in /etc/fstab:
$ grep jester /etc/fstab /usr/jails/jester/usr/local/FreshPorts/PORTS-SVN /usr/jails/jester/usr/local/FreshPorts/ports-jail/usr/ports nullfs ro,nosuid,noexec 0 0 /usr/jails/basejail/usr/share/mk /usr/jails/jester/usr/local/FreshPorts/ports-jail/usr/share/mk nullfs ro,nosuid,noexec 0 0 /usr/jails/basejail/usr/sbin /usr/jails/jester/usr/local/FreshPorts/ports-jail/usr/sbin nullfs ro,nosuid 0 0 /usr/jails/basejail/usr/bin /usr/jails/jester/usr/local/FreshPorts/ports-jail/usr/bin nullfs ro,nosuid 0 0 /usr/jails/basejail/usr/libexec /usr/jails/jester/usr/local/FreshPorts/ports-jail/usr/libexec nullfs ro,nosuid 0 0 /usr/jails/basejail/libexec /usr/jails/jester/usr/local/FreshPorts/ports-jail/libexec nullfs ro,nosuid 0 0 /usr/jails/basejail/usr/lib /usr/jails/jester/usr/local/FreshPorts/ports-jail/usr/lib nullfs ro,nosuid 0 0 /usr/jails/basejail/sbin /usr/jails/jester/usr/local/FreshPorts/ports-jail/sbin nullfs ro,nosuid 0 0 /usr/jails/basejail/lib /usr/jails/jester/usr/local/FreshPorts/ports-jail/lib nullfs ro,nosuid 0 0 /usr/jails/basejail/bin /usr/jails/jester/usr/local/FreshPorts/ports-jail/bin nullfs ro,nosuid 0 0 none /usr/jails/jester/usr/local/FreshPorts/ports-jail/dev devfs rw 0 0
These mount points will correspond to those outlined in a recent blog post, but they won’t work long term.
I think I have a solution, found by searching for ‘ezjail mounting other mount’.
I will be moving the entries in /etc/fstab to /etc/fstab.jester, which will look like this:
/usr/local/jails/basejail /usr/local/jails/jester.unixathome.org/basejail nullfs ro 0 0 # for my dev jail in a jail /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/PORTS-SVN /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/usr/ports nullfs ro,nosuid,noexec 0 0 /usr/local/jails/basejail/usr/share/mk /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/usr/share/mk nullfs ro,nosuid,noexec 0 0 /usr/local/jails/basejail/usr/sbin /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/usr/sbin nullfs ro,nosuid 0 0 /usr/local/jails/basejail/usr/bin /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/usr/bin nullfs ro,nosuid 0 0 /usr/local/jails/basejail/usr/libexec //usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/usr/libexec nullfs ro,nosuid 0 0 /usr/local/jails/basejail/libexec /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/libexec nullfs ro,nosuid 0 0 /usr/local/jails/basejail/usr/lib /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/usr/lib nullfs ro,nosuid 0 0 /usr/local/jails/basejail/sbin /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/sbin nullfs ro,nosuid 0 0 /usr/local/jails/basejail/lib /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/lib nullfs ro,nosuid 0 0 /usr/local/jails/basejail/bin /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/bin nullfs ro,nosuid 0 0 none /usr/local/jails/jester.unixathome.org/usr/local/FreshPorts/ports-jail/dev devfs rw 0 0
This works perfectly for my needs.
ADDENDA: 2014 Aug 23
When I ran a recent ezjail-admin config, my chroot stopped working. I started seeing errors like this:
This command (FreshPorts code 1): /usr/local/bin/sudo /usr/sbin/chroot -u dan /usr/local/FreshPorts/ports-jail /make-port.sh /usr/local/PORTS-head biology/consed 2>/tmp/FreshPorts.biology.consed.make-error.2014.8.23.18.41.36.74756 produced this error: Error message is: chroot: /make-port.sh: No such file or directory
I tried a chroot into the jail:
$ /usr/local/bin/sudo /usr/sbin/chroot -u dan /usr/local/FreshPorts/ports-jail Password: chroot: /bin/sh: No such file or directory
Oh. I looked up this blog post. I had already looked at /etc/fstab.jester and noticed it had just one line, as created by ezjail-admin.
My next step, restore that file from backup (as done by Bacula). Then I adjusted the paths to reflect the new locations and restarted the jail. All fixed.
