My goal is to make an easy way to upgrade with beadm by using the idea found in this post. I had a problem with my first attempt which I solved with the details in this post.
boot mfsBSD
This tool has very quickly become one of my favorites. I like how easy it is to pop in a USB drive and boot a live version of FreeBSD to do stuff.
The import
I imported my zpool without mounting it.
zpool import -N system
What I found (slightly redacted for brevity):
root@mfsbsd:~ # zfs list NAME USED AVAIL REFER MOUNTPOINT system 4.52T 5.97T 288K / system/backups 671K 5.97T 304K /backups system/backups/knew 288K 5.97T 288K /backups/knew system/bootenv 17.5G 5.97T 1.31M legacy system/bootenv/10.1-RELEASE 1.05M 5.97T 3.77G / system/bootenv/10.1-RELEASE/tmp 224K 5.97T 531M /tmp system/bootenv/10.1-RELEASE/usr 272K 5.97T 336K /usr system/bootenv/10.1-RELEASE/usr/local 16.0K 5.97T 948M /usr/local system/bootenv/10.1-RELEASE/usr/obj 16.0K 5.97T 408K /usr/obj system/bootenv/10.1-RELEASE/usr/src 16.0K 5.97T 818M /usr/src system/bootenv/10.1-RELEASE/var 304K 5.97T 352K /var system/bootenv/10.1-RELEASE/var/audit 16.0K 5.97T 320K /var/audit system/bootenv/10.1-RELEASE/var/empty 16.0K 5.97T 288K /var/empty system/bootenv/10.1-RELEASE/var/log 16.0K 5.97T 7.84M /var/log system/bootenv/10.1-RELEASE/var/tmp 16.0K 5.97T 1.57M /var/tmp system/bootenv/default 17.5G 5.97T 4.46G / system/bootenv/default/tmp 546M 5.97T 532M /tmp system/bootenv/default/usr 8.89G 5.97T 3.94G /usr system/bootenv/default/usr/local 2.82G 5.97T 948M /usr/local system/bootenv/default/usr/obj 127M 5.97T 408K /usr/obj system/bootenv/default/usr/src 2.01G 5.97T 818M /usr/src system/bootenv/default/var 48.4M 5.97T 288K /var system/bootenv/default/var/audit 416K 5.97T 320K /var/audit system/bootenv/default/var/empty 288K 5.97T 288K /var/empty system/bootenv/default/var/log 45.0M 5.97T 7.77M /var/log system/bootenv/default/var/tmp 2.44M 5.97T 1.57M /var/tmp system/data 4.50T 5.97T 943K /data system/data/homes 1.76T 5.97T 288K /usr/home system/data/homes/dan 1.76T 5.97T 649G /usr/home/dan system/data/homes/dan/mailjail 6.42G 5.97T 304K /usr/home/dan/mailjail system/data/homes/dan/mailjail/DansMaildir 6.42G 5.97T 5.49G /usr/home/dan/mailjail/DansMaildir system/data/jails 1.85T 5.97T 288K /usr/jails ... system/data/pgsql 864G 5.97T 480G /usr/local/pgsql system/data/ports 6.97G 5.97T 1.90G /usr/ports system/data/ports/distfiles 3.68G 5.97T 3.68G /usr/ports/distfiles system/data/poudriere 31.1G 5.97T 336K /usr/local/poudriere ... system/data/root 40.9M 5.97T 39.7M /root system/zzz-DELETE-ME.poudriere 1.57G 5.97T 288K legacy system/zzz-DELETE-ME.poudriere/data 1.57G 5.97T 400K legacy system/zzz-DELETE-ME.poudriere/data/.m 591K 5.97T 591K legacy system/zzz-DELETE-ME.poudriere/data/cache 99.3M 5.97T 99.3M legacy system/zzz-DELETE-ME.poudriere/data/logs 88.3M 5.97T 88.3M legacy system/zzz-DELETE-ME.poudriere/data/packages 1.38G 5.97T 1.38G legacy system/zzz-DELETE-ME.poudriere/data/wrkdirs 288K 5.97T 288K legacy
The rename & copy
Rename the dataset first:
# zfs rename system/bootenv/default/usr system/zzz.DELETE.ME.bootenv.default.usr
Create my new one:
# zfs create system/bootenv/default/usr
Create a mount point & mount it:
# mkdir /NEWUSER # mount -t zfs system/bootenv/default/usr /NEWUSER/ mount: system/bootenv/default/usr: Device busy
What?
# zfs mount system/bootenv/default /mnt system/bootenv/default/usr /usr
Oh. When I created, it auto-mounted. I could have avoided that by specifying zfs create -u.
# zfs umount system/bootenv/default/usr # mount -t zfs system/bootenv/default/usr /NEWUSER/
How much data will I be copying over?
# du -ch -d 1 /mnt/usr/ 306K /mnt/usr/libdata 93M /mnt/usr/bin 18M /mnt/usr/libexec 627K /mnt/usr/jails 17K /mnt/usr/src 17K /mnt/usr/ports 17K /mnt/usr/obj 17K /mnt/usr/games 79M /mnt/usr/lib 37M /mnt/usr/sbin 93M /mnt/usr/share 99K /mnt/usr/local 36M /mnt/usr/include 176K /mnt/usr/home 66K /mnt/usr/lib32 357M /mnt/usr/ 357M total
Seems reasonable.
But first, let’s snapshot:
root@mfsbsd:~ # zfs snapshot -r system@BeforeTarOfUsr
Really, I should have done this at the start of this post.
The copy
This copies from /mnt/usr to /NEWUSER and is superior to cp because it caters for all types of files.
root@mfsbsd:~ # tar clf - -C /mnt/usr . | tar xpvf - -C /NEWUSER x ./ x ./libdata/ x ./bin/ x ./libexec/ x ./jails/ x ./src/ x ./ports/ x ./obj/ x ./games/ ... ... x ./libdata/ldscripts/elf_x86_64_fbsd.xu x ./libdata/ldscripts/elf_x86_64_fbsd.x x ./libdata/ldscripts/elf_x86_64_fbsd.xr
Check what we copied
Now, diff it:
root@mfsbsd:~ # diff -ruN /mnt/usr /NEWUSER diff: /mnt/usr/bin/perl: No such file or directory diff: /NEWUSER/bin/perl: No such file or directory diff: /mnt/usr/bin/perl5: No such file or directory diff: /NEWUSER/bin/perl5: No such file or directory diff: /mnt/usr/lib/libcrypto.so: No such file or directory diff: /NEWUSER/lib/libcrypto.so: No such file or directory diff: /mnt/usr/lib/libkvm.so: No such file or directory diff: /NEWUSER/lib/libkvm.so: No such file or directory diff: /mnt/usr/lib/libmd.so: No such file or directory diff: /NEWUSER/lib/libmd.so: No such file or directory
That’s OK. They are in pairs, and are symlinks elsewhere:
root@mfsbsd:~ # ls -l /mnt/usr/bin/perl lrwxr-xr-x 1 root wheel 25 Dec 27 20:35 /mnt/usr/bin/perl -> /usr/local/bin/perl5.16.3 root@mfsbsd:~ # ls -l /mnt/usr/bin/perl5 lrwxr-xr-x 1 root wheel 25 Dec 27 20:35 /mnt/usr/bin/perl5 -> /usr/local/bin/perl5.16.3 root@mfsbsd:~ # ls -l /mnt/usr/lib/libcrypto.so lrwxr-xr-x 1 root wheel 19 Dec 4 2012 /mnt/usr/lib/libcrypto.so -> /lib/libcrypto.so.6 root@mfsbsd:~ # ls -l /NEWUSER/lib/libkvm.so lrwxr-xr-x 1 root wheel 16 Dec 4 2012 /NEWUSER/lib/libkvm.so -> /lib/libkvm.so.5 root@mfsbsd:~ # ls -l /mnt/usr/lib/libmd.so lrwxr-xr-x 1 root wheel 15 Dec 4 2012 /mnt/usr/lib/libmd.so -> /lib/libmd.so.5
And…
One more look:
root@mfsbsd:~ # zfs list NAME USED AVAIL REFER MOUNTPOINT system 4.52T 5.97T 288K / system/backups 671K 5.97T 304K /backups system/backups/knew 288K 5.97T 288K /backups/knew system/bootenv 8.96G 5.97T 1.31M legacy system/bootenv/10.1-RELEASE 1.05M 5.97T 3.77G / system/bootenv/10.1-RELEASE/tmp 224K 5.97T 531M /tmp system/bootenv/10.1-RELEASE/usr 272K 5.97T 336K /usr system/bootenv/10.1-RELEASE/usr/local 16.0K 5.97T 948M /usr/local system/bootenv/10.1-RELEASE/usr/obj 16.0K 5.97T 408K /usr/obj system/bootenv/10.1-RELEASE/usr/src 16.0K 5.97T 818M /usr/src system/bootenv/10.1-RELEASE/var 304K 5.97T 352K /var system/bootenv/10.1-RELEASE/var/audit 16.0K 5.97T 320K /var/audit system/bootenv/10.1-RELEASE/var/empty 16.0K 5.97T 288K /var/empty system/bootenv/10.1-RELEASE/var/log 16.0K 5.97T 7.84M /var/log system/bootenv/10.1-RELEASE/var/tmp 16.0K 5.97T 1.57M /var/tmp system/bootenv/default 8.96G 5.97T 4.46G / system/bootenv/default/tmp 546M 5.97T 532M /tmp system/bootenv/default/usr 358M 5.97T 358M /usr system/bootenv/default/var 48.4M 5.97T 288K /var system/bootenv/default/var/audit 416K 5.97T 320K /var/audit system/bootenv/default/var/empty 288K 5.97T 288K /var/empty system/bootenv/default/var/log 45.0M 5.97T 7.77M /var/log system/bootenv/default/var/tmp 2.44M 5.97T 1.57M /var/tmp system/data 4.50T 5.97T 943K /data system/data/homes 1.76T 5.97T 288K /usr/home system/data/homes/dan 1.76T 5.97T 649G /usr/home/dan system/data/homes/dan/mailjail 6.42G 5.97T 304K /usr/home/dan/mailjail system/data/homes/dan/mailjail/DansMaildir 6.42G 5.97T 5.49G /usr/home/dan/mailjail/DansMaildir system/data/jails 1.85T 5.97T 288K /usr/jails ... system/data/pgsql 864G 5.97T 480G /usr/local/pgsql system/data/ports 6.97G 5.97T 1.90G /usr/ports system/data/ports/distfiles 3.68G 5.97T 3.68G /usr/ports/distfiles system/data/poudriere 31.1G 5.97T 336K /usr/local/poudriere ... system/data/root 40.9M 5.97T 39.7M /root system/zzz-DELETE-ME.poudriere 1.57G 5.97T 288K legacy system/zzz-DELETE-ME.poudriere/data 1.57G 5.97T 400K legacy system/zzz-DELETE-ME.poudriere/data/.m 591K 5.97T 591K legacy system/zzz-DELETE-ME.poudriere/data/cache 99.3M 5.97T 99.3M legacy system/zzz-DELETE-ME.poudriere/data/logs 88.3M 5.97T 88.3M legacy system/zzz-DELETE-ME.poudriere/data/packages 1.38G 5.97T 1.38G legacy system/zzz-DELETE-ME.poudriere/data/wrkdirs 288K 5.97T 288K legacy system/zzz.DELETE.ME.bootenv.default.usr 8.89G 5.97T 3.94G /zzz.DELETE.ME.bootenv.default.usr system/zzz.DELETE.ME.bootenv.default.usr/local 2.82G 5.97T 948M /zzz.DELETE.ME.bootenv.default.usr/local system/zzz.DELETE.ME.bootenv.default.usr/obj 127M 5.97T 408K /zzz.DELETE.ME.bootenv.default.usr/obj system/zzz.DELETE.ME.bootenv.default.usr/src 2.01G 5.97T 818M /zzz.DELETE.ME.bootenv.default.usr/src
Look over that carefully, and see if you can anticipate the problem I encountered after rebooting.
After rebooting
After rebooting, I ssh’d in and found this:
$ bash bash: not found
OK, I have no /usr/local. Well, I do, but it is not complete:
$ ls -l /usr/local total 50 drwxr-xr-x 3 root wheel 3 Sep 10 2013 jails drwxr-xr-x 4 pgsql pgsql 8 Feb 10 17:26 pgsql drwxr-xr-x 5 root wheel 5 Jun 11 2014 poudriere $
What’s here?
$ zfs list -r system/bootenv/default NAME USED AVAIL REFER MOUNTPOINT system/bootenv/default 8.96G 6.13T 4.46G / system/bootenv/default/tmp 547M 6.13T 532M /tmp system/bootenv/default/usr 358M 6.13T 358M /usr system/bootenv/default/var 48.7M 6.13T 288K /var system/bootenv/default/var/audit 432K 6.13T 320K /var/audit system/bootenv/default/var/empty 288K 6.13T 288K /var/empty system/bootenv/default/var/log 45.2M 6.13T 7.77M /var/log system/bootenv/default/var/tmp 2.46M 6.13T 1.57M /var/tmp
OK, but let me check….
$ zfs mount | grep default system/bootenv/default / system/bootenv/default/tmp /tmp system/bootenv/default/usr /usr system/data/poudriere/ports/default /usr/local/poudriere/ports/default system/bootenv/default/var/audit /var/audit system/bootenv/default/var/empty /var/empty system/bootenv/default/var/log /var/log system/bootenv/default/var/tmp /var/tmp system/zzz.DELETE.ME.bootenv.default.usr/local /zzz.DELETE.ME.bootenv.default.usr/local system/zzz.DELETE.ME.bootenv.default.usr/obj /zzz.DELETE.ME.bootenv.default.usr/obj system/zzz.DELETE.ME.bootenv.default.usr/src /zzz.DELETE.ME.bootenv.default.usr/src
Ahhh, there, mounted in the wrong place, because they took their names from their parent. OK, I will move those back.
$ su Password:
Unmount them:
# zfs umount system/zzz.DELETE.ME.bootenv.default.usr/local # zfs umount system/zzz.DELETE.ME.bootenv.default.usr/obj # zfs umount system/zzz.DELETE.ME.bootenv.default.usr/src
Rename them
# zfs rename system/zzz.DELETE.ME.bootenv.default.usr/local system/bootenv/default/usr/local # zfs rename system/zzz.DELETE.ME.bootenv.default.usr/obj system/bootenv/default/usr/obj # zfs rename system/zzz.DELETE.ME.bootenv.default.usr/src system/bootenv/default/usr/src
List them:
# zfs list -r system/bootenv/default NAME USED AVAIL REFER MOUNTPOINT system/bootenv/default 13.9G 6.13T 4.46G / system/bootenv/default/tmp 547M 6.13T 532M /tmp system/bootenv/default/usr 5.30G 6.13T 358M /usr system/bootenv/default/usr/local 2.82G 6.13T 948M /usr/local system/bootenv/default/usr/obj 127M 6.13T 408K /usr/obj system/bootenv/default/usr/src 2.01G 6.13T 818M /usr/src system/bootenv/default/var 48.7M 6.13T 288K /var system/bootenv/default/var/audit 432K 6.13T 320K /var/audit system/bootenv/default/var/empty 288K 6.13T 288K /var/empty system/bootenv/default/var/log 45.2M 6.13T 7.77M /var/log system/bootenv/default/var/tmp 2.46M 6.13T 1.57M /var/tmp
Mount them
# zfs mount system/bootenv/default/usr/local # zfs mount system/bootenv/default/usr/obj # zfs mount system/bootenv/default/usr/src # bash bash: Command not found.
Oh, try this:
# rehash # bash Shared object "libintl.so.8" not found, required by "bash"
So, now the command is found, but it’s not working.
# cd /usr/local/lib/ # ls -l /usr/local/lib/libintl.so.8 lrwxr-xr-x 1 root wheel 16 Feb 10 16:24 /usr/local/lib/libintl.so.8 -> libintl.so.8.1.3 # ls -l libintl.so.8.1.3 -rw-r--r-- 1 root wheel 56191 Feb 10 16:24 libintl.so.8.1.3 # bash Shared object "libintl.so.8" not found, required by "bash" # rehash # hash # bash Shared object "libintl.so.8" not found, required by "bash" #
I have no idea why that won’t run now. So….
Let’s reboot.
And I did.
All good then.
HTH
Hope this helps. I posted this mostly for recording what I did.
HEADS UP: I think I have to do the same thing for /var now:
# beadm create 10.1-RELEASE Created successfully # beadm mount 10.1-RELEASE /mnt Mounted successfully on '/mnt' # mount -t devfs devfs /mnt/dev # chroot /mnt # chroot /mnt root@slocum:/ # bash Shared object "libintl.so.8" not found, required by "bash"
OK, so bash still doesn’t work in the chroot but that is not my biggest issue:
root@slocum:/ # freebsd-update -r 10.1-RELEASE upgrade freebsd-update: Directory does not exist or is not writable: /var/db/freebsd-update root@slocum:/ # cd /var root@slocum:/var # ls -l total 66 drwxr-xr-x 4 root wheel 4 Nov 8 2013 audit drwxr-xr-x 2 root wheel 2 Apr 14 2014 empty drwxr-xr-x 3 root wheel 149 Mar 12 14:00 log drwxrwxrwt 6 root wheel 6 Jan 16 22:07 tmp
Perhaps I need to do to /var what I just did for /usr. But that will be for another night.
See that list above? See how closely it matches this list?
$ zfs list -r system/bootenv/10.1-RELEASE/var NAME USED AVAIL REFER MOUNTPOINT system/bootenv/10.1-RELEASE/var 304K 6.13T 352K /var system/bootenv/10.1-RELEASE/var/audit 16.0K 6.13T 320K /var/audit system/bootenv/10.1-RELEASE/var/empty 16.0K 6.13T 288K /var/empty system/bootenv/10.1-RELEASE/var/log 16.0K 6.13T 7.80M /var/log system/bootenv/10.1-RELEASE/var/tmp 16.0K 6.13T 1.57M /var/tmp
That’s not a coincidence. I’m moving it all over to the parent. I know why those directories are separate; it’s what pjd did.
