I’m in the process of moving a system from one server to another. Today, I’m going to try moving a jail using ezjail-admin archive.
The two systems
I am copying jails between these two systems:
- source: FreeBSD 8.4-RELEASE
- destination: FreeBSD 9.1-RELEASE-p4
I really am interested to see if this works.
Creating the archive
Best practice is to stop the jail. You can archive a running jail, but I content that you will get best results from a stopped jail.
# ezjail-admin stop building.example.org Stopping jails: building.example.org. # ezjail-admin archive building.example.org
Copying the archive
The archive is automatically placed in the ezjail_archivedir directory (see /usr/local/etc/ezjail.conf):
# cd /usr/jails/ezjail_archives # ls -lh total 687079 -rw-r--r-- 1 root wheel 670M Jul 26 12:26 building_example_org-201307261222.01.tar.gz #
I copied that file to the new server:
$ scp building_example_org-201307261222.01.tar.gz knew:
Using ZFS? Create a new filesystem
Since I am using ZFS, it’s a good idea to create a new filesystem for the jails. Why? Because you can. Snapshots. Control. It’s all good.
# zfs create system/usr/jails # zfs set compression=lzjb system/usr/jails # zfs get compression system/usr/jails NAME PROPERTY VALUE SOURCE system/usr/jails compression lzjb local
I also set compression on.
And then, I took a snapshot, so I can rollback, should I mess up.
# zfs snapshot system/usr/jails@BeforeInstall
Setting up ezjail
Please ignore this step if you already have ezjail installed and configured. I hadn’t run an install yet, and did not have /usr/jails create, nor any basejails or jail flavors.
Because I am using ZFS, I set these items in /usr/local/etc/ezjail.conf:
ezjail_use_zfs="YES" ezjail_use_zfs_for_jails="YES" ezjail_jailzfs="system/usr/jails"
The value for ezjail_jailzfs relates to the ‘zfs create’ I did above.
I already had ezjail installed. I had not created any jails, but I don’t think that’s relevant.
I am running ZFS, so I decided to take a snapshot before I proceeded:
# zfs snapshot system/usr@beforeFirstJail
The filesystem name is specific to my configuration. Yours may vary, and it probably will.
With that snapshot taken, I’m ready to install my first jail, from the archive I just copied over.
First attempt fails
Here is what happened when I first tried. The -p option will also fetch a ports tree and install it into the base jail.
$ sudo ezjail-admin install -p Password: Your system is 9.1-RELEASE-p4. Normally FTP-servers don't provide non-RELEASE-builds. Querying your ftp-server... The ftp server you specified (ftp.freebsd.org) seems to provide the following builds: lrwxr-xr-x 1 520 100 14 Feb 12 08:32 8.3-RELEASE -> ../8.3-RELEASE drwxrwxr-x 2 980 100 512 May 22 12:59 9.1-RELEASE drwxrwxr-x 2 1237 100 512 Jul 21 21:47 9.2-BETA1 drwxrwxr-x 5 980 100 512 Jul 21 23:19 ISO-IMAGES Release to fetch [ 9.1-RELEASE-p4 ]: fetch: ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/9.1-RELEASE-p4/base.txz: File unavailable (e.g., file not found, no access) fetch: ftp://ftp.freebsd.org/pub/FreeBSD/snapshot/amd64/amd64/9.1-RELEASE-p4/base.txz: File unavailable (e.g., file not found, no access) fetch: ftp://ftp.freebsd.org/pub/FreeBSD/amd64/amd64/9.1-RELEASE-p4/base.txz: File unavailable (e.g., file not found, no access) fetch: ftp://ftp.freebsd.org/releases/amd64/amd64/9.1-RELEASE-p4/base.txz: File unavailable (e.g., file not found, no access) fetch: ftp://ftp.freebsd.org/snapshots/amd64/amd64/9.1-RELEASE-p4/base.txz: File unavailable (e.g., file not found, no access) fetch: ftp://ftp.freebsd.org/pub/FreeBSD-Archive/old-releases/amd64/amd64/9.1-RELEASE-p4/base.txz: File unavailable (e.g., file not found, no access) Could not fetch base from ftp://ftp.freebsd.org. Maybe your release (9.1-RELEASE-p4) is specified incorrectly or the host ftp.freebsd.org does not provide that release build. Use the -r option to specify an existing release or the -h option to specify an alternative ftp server.
My second attempt was more successful. This time, I specified the release on the command line. I could have entered it at the prompt, as shown in my first attempt, where I just hit ENTER).
$ sudo ezjail-admin install -p -r 9.1-RELEASE Password: base.txz 100% of 57 MB 4002 kBps 00m00s lib32.txz 100% of 9515 kB 734 kBps 00m00s /usr/jails/basejail/bin /usr/jails/basejail/bin/stty /usr/jails/basejail/bin/domainname /usr/jails/basejail/bin/mv /usr/jails/basejail/bin/kenv .... /usr/jails/basejail/usr/ports/x11/yalias/ /usr/jails/basejail/usr/ports/x11/yeahconsole/ /usr/jails/basejail/usr/ports/x11/yelp/ /usr/jails/basejail/usr/ports/x11/zenity/ Building new INDEX files... done. $
That succeeded. The build was fetched, and the ports tree installed.
What are the implications of running 9.1-RELEASE jails on a 9.1-RELEASE-p4 system? Not much I bet. I will also try upgrading the jails after this initial install. Complicating the issue, I’m about to install a jail from an 8.4 system.
Using the archive
ezjail-admin install will look for the archive file in the archives directory. The default location for this is /usr/jails/ezjail_archives. Since I do not have that directory, let me set that up, and move the archive I copied earlier.
# zfs create /usr/jails/ezjail_archives # mv ~dan/building_example_org-201307261222.01.tar.gz /usr/jails/ezjail_archives/
If you don’t already have an IP address for this about-to-be-created jail, let’s create one now. I’m going to use the loopback device, and an address of 127.0.0.10. This jail still exists on the other server (but not running). I will eventually assign the real IP address to this jail.
# ifconfig lo0 alias 127.0.0.10 netmask 255.255.255.255 # ifconfig lo0 lo0: flags=8049
metric 0 mtu 16384 options=600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa inet 127.0.0.1 netmask 0xff000000 inet 127.0.0.10 netmask 0xffffffff nd6 options=21
Now we create the jail:
$ sudo ezjail-admin create -a building_example_org-201307261222.01.tar.gz building.example.org 127.0.0.10 Error: No archive found at building_example_org-201307261222.01.tar.gz.
Oh. So on the archive create, it puts the jail in the ezjail_archives directory, but on the install, it wants a full path. OK fair enough.
$ sudo ezjail-admin create -a /usr/jails/ezjail_archives/building_example_org-201307261222.01.tar.gz building.example.org 127.0.0.10 pax: ./tmp/ssh-kGzw5MTnF3/agent.55026 skipped. Sockets cannot be copied or extracted Warning: Some services already seem to be listening on IP 127.0.0.10 This may cause some confusion, here they are: root ntpd 1335 28 udp4 127.0.0.10:123 *:* Warning: Some services already seem to be listening on all IP, (including 127.0.0.10) This may cause some confusion, here they are: root sshd 1451 3 tcp6 *:22 *:* root sshd 1451 4 tcp4 *:22 *:* root ntpd 1335 20 udp4 *:123 *:* root ntpd 1335 21 udp6 *:123 *:*
Yes, I should fix up sshd. ntpd cannot be changed from what I’ve read in the past. It always listens on all interfaces. The actual fix is outside scope. Short answer: set ListenAddress in /etc/ssh/sshd_config and restart sshd.
Start the jail
Let’s start that jail!
$ sudo ezjail-admin start building.example.org Configuring jails:. Starting jails: building.example.org. $ jls JID IP Address Hostname Path 1 127.0.0.10 building.example.org /usr/jails/building.example.org $
Clearly, it is running. But can I connect to it?
Connect to the jail
From the host server, I attempted to ssh into the newly created jail:
$ ssh -A 127.0.0.10 The authenticity of host '127.0.0.10 (127.0.0.10)' can't be established. ECDSA key fingerprint is 67:7a:66:99:fc:00:be:6d:06:75:f4:be:e6:0b:1d:74. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '127.0.0.10' (ECDSA) to the list of known hosts. FreeBSD 9.1-RELEASE-p4 (GENERIC) #0: Mon Jun 17 11:42:37 UTC 2013 Welcome to FreeBSD! Before seeking technical support, please use the following resources: o Security advisories and updated errata information for all releases are at http://www.FreeBSD.org/releases/ - always consult the ERRATA section for your release first as it's updated frequently. o The Handbook and FAQ documents are at http://www.FreeBSD.org/ and, along with the mailing lists, can be searched by going to http://www.FreeBSD.org/search/. If the doc distribution has been installed, they're also available formatted in /usr/share/doc. If you still have a question or problem, please take the output of `uname -a', along with any relevant error messages, and email it as a question to the questions@FreeBSD.org mailing list. If you are unfamiliar with FreeBSD's directory layout, please refer to the hier(7) manual page. If you are not familiar with manual pages, type `man man'. You may also use sysinstall(8) to re-enter the installation and configuration utility. Edit /etc/motd to change this login announcement. $ ps auwx USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 66205 0.0 0.0 12052 1248 ?? IsJ 1:43PM 0:00.00 /usr/sbin/syslogd -ss nagios 66324 0.0 0.0 19144 2564 ?? SsJ 1:43PM 0:00.01 /usr/local/sbin/nrpe2 -c /usr/local/etc/nrpe.cfg -d root 66358 0.0 0.0 46744 3464 ?? IsJ 1:43PM 0:00.00 /usr/sbin/sshd root 66369 0.0 0.0 14128 1436 ?? IsJ 1:43PM 0:00.00 /usr/sbin/cron -J 15 -s root 66424 0.0 0.0 67884 4132 ?? IsJ 1:43PM 0:00.02 sshd: dan [priv] (sshd) dan 66426 0.0 0.0 67884 4160 ?? SJ 1:43PM 0:00.00 sshd: dan@pts/3 (sshd) dan 66427 0.0 0.0 14504 1860 3 SsJ 1:43PM 0:00.00 -sh (sh) dan 66457 0.0 0.0 14188 1348 3 R+J 1:44PM 0:00.00 ps auwx $ psql -l psql: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"? $
Yes, I’m in. Some things are running, but not everything, such as PostgreSQL. But I am not shocked. Everything in this jail should be recomplied. That is, all the ports should be refreshed.
I will run ‘portupgrade -fa’ on the jail, this will/should fix all the apps.