As mentioned in several previous posts, I’m moving a server from one chassis to another. None of the hardware is coming over. Just the jails and data.
Recently, I’ve been trying out two new 12TB drives. I’m looking forward to the reduced noise level, as well as the reduced power consumption and heat.
In this post:
- FreeBSD 13.2-RELEASE-p2
The testing zpool
I created a testing zpool and started copying data into it.
[17:29 r730-03 dvl ~] % zfs list -r testing01 NAME USED AVAIL REFER MOUNTPOINT testing01 7.29T 3.49T 136K /testing01 testing01/dbclone.backups.rsyncer.copy01 1.34T 3.49T 195G /testing01/dbclone.backups.rsyncer.copy01 testing01/dbclone.backups.rsyncer.copy02 1.49T 3.49T 195G /testing01/dbclone.backups.rsyncer.copy02 testing01/dbclone.backups.rsyncer.copy03 1.49T 3.49T 195G /testing01/dbclone.backups.rsyncer.copy03 testing01/dbclone.backups.rsyncer.copy04 1.49T 3.49T 195G /testing01/dbclone.backups.rsyncer.copy04 testing01/dbclone.backups.rsyncer.copy05 1.49T 3.49T 195G /testing01/dbclone.backups.rsyncer.copy05
Time for it to go, so I can add those drives to the production zpool.
[17:30 r730-03 dvl ~] % sudo zpool destroy testing01
Add it into the existing zpool
The existing zpool is a mirror of two drives.
[17:33 r730-03 dvl ~] % zpool list -v data01
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data01 10.9T 2.32T 8.58T - - 1% 21% 1.00x ONLINE -
mirror-0 10.9T 2.32T 8.58T - - 1% 21.3% - ONLINE
gpt/HGST_8CJVT8YE 10.9T - - - - - - - ONLINE
gpt/SG_ZHZ16KEX 10.9T - - - - - - - ONLINE
Here we go:
[17:34 r730-03 dvl ~] % sudo zpool add data01 mirror /dev/gpt/SG_ZHZ03BAT /dev/gpt/HGST_8CJW1G4E
And here is the result
[17:34 r730-03 dvl ~] % zpool list -v data01
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data01 21.8T 2.32T 19.5T - - 0% 10% 1.00x ONLINE -
mirror-0 10.9T 2.32T 8.58T - - 1% 21.3% - ONLINE
gpt/HGST_8CJVT8YE 10.9T - - - - - - - ONLINE
gpt/SG_ZHZ16KEX 10.9T - - - - - - - ONLINE
mirror-1 10.9T 0 10.9T - - 0% 0.00% - ONLINE
gpt/SG_ZHZ03BAT 10.9T - - - - - - - ONLINE
gpt/HGST_8CJW1G4E 10.9T - - - - - - - ONLINE
Notice how mirror-0 has 2.32T allocated.
The newly added mirror (mirror-1) has 0 allocated to it.
This is expected.
I’ll watch how that changes over time.
13 minutes later
Just a short time later, mirror-1 contains a whole 15.7M. Whoa, slow down there….
[17:47 r730-03 dvl ~] % zpool list -v data01
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data01 21.8T 2.32T 19.5T - - 0% 10% 1.00x ONLINE -
mirror-0 10.9T 2.32T 8.58T - - 1% 21.3% - ONLINE
gpt/HGST_8CJVT8YE 10.9T - - - - - - - ONLINE
gpt/SG_ZHZ16KEX 10.9T - - - - - - - ONLINE
mirror-1 10.9T 15.7M 10.9T - - 0% 0.00% - ONLINE
gpt/SG_ZHZ03BAT 10.9T - - - - - - - ONLINE
gpt/HGST_8CJW1G4E 10.9T - - - - - - - ONLINE
[18:45 r730-03 dvl ~] % date
The next day
The next morning, I found this:
[13:55 r730-03 dvl ~] % zpool list -v data01
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data01 21.8T 2.51T 19.3T - - 0% 11% 1.00x ONLINE -
mirror-0 10.9T 2.30T 8.61T - - 1% 21.1% - ONLINE
gpt/HGST_8CJVT8YE 10.9T - - - - - - - ONLINE
gpt/SG_ZHZ16KEX 10.9T - - - - - - - ONLINE
mirror-1 10.9T 213G 10.7T - - 0% 1.91% - ONLINE
gpt/SG_ZHZ03BAT 10.9T - - - - - - - ONLINE
gpt/HGST_8CJW1G4E 10.9T - - - - - - - ONLINE
[13:56 r730-03 dvl ~] %
Now mirror-1 has 216G. That’s after last night’s backups.
2023-08-29 – two days later
Today’s reading:
[16:07 r730-03 dvl ~] % zpool list -v data01
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data01 21.8T 2.68T 19.1T - - 0% 12% 1.00x ONLINE -
mirror-0 10.9T 2.25T 8.66T - - 1% 20.6% - ONLINE
gpt/HGST_8CJVT8YE 10.9T - - - - - - - ONLINE
gpt/SG_ZHZ16KEX 10.9T - - - - - - - ONLINE
mirror-1 10.9T 442G 10.5T - - 0% 3.95% - ONLINE
gpt/SG_ZHZ03BAT 10.9T - - - - - - - ONLINE
gpt/HGST_8CJW1G4E 10.9T - - - - - - - ONLINE
[16:07 r730-03 dvl ~] %
2023-09-03 – 7 days later
Here we go a week afterwards:
[0:00 r730-03 dvl ~] % zpool list -v data01
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data01 21.8T 3.77T 18.0T - - 0% 17% 1.00x ONLINE -
mirror-0 10.9T 2.69T 8.22T - - 1% 24.6% - ONLINE
gpt/HGST_8CJVT8YE 10.9T - - - - - - - ONLINE
gpt/SG_ZHZ16KEX 10.9T - - - - - - - ONLINE
mirror-1 10.9T 1.08T 9.82T - - 0% 9.93% - ONLINE
gpt/SG_ZHZ03BAT 10.9T - - - - - - - ONLINE
gpt/HGST_8CJW1G4E 10.9T - - - - - - - ONLINE
[0:00 r730-03 dvl ~] %
The new drive is catching up.
2023-09-09
Another update:
[13:13 r730-03 dvl ~] % zpool list -v data01
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
data01 21.8T 4.85T 17.0T - - 0% 22% 1.00x ONLINE -
mirror-0 10.9T 3.15T 7.75T - - 1% 28.9% - ONLINE
gpt/HGST_8CJVT8YE 10.9T - - - - - - - ONLINE
gpt/SG_ZHZ16KEX 10.9T - - - - - - - ONLINE
mirror-1 10.9T 1.70T 9.21T - - 0% 15.6% - ONLINE
gpt/SG_ZHZ03BAT 10.9T - - - - - - - ONLINE
gpt/HGST_8CJW1G4E 10.9T - - - - - - - ONLINE
