This post has been replaced by a newer post.
This is the previous post for this system configuration.
For future reference, this is the knew server … oh wait, I think it’s this server which is was mounted in the 3U chassis mentioned in this post (perviously erroneously referred to as a 4U). It runs a few jails, including Bacula regression testing services.
It is now mounted in a SuperChassis 846E16-R1200B
This recent post outlines the modifications so it boots off a mirror of SSDs.
Of note, I went from three SAS2008 to one SAS2308 with this new system, because it contained a SAS expander.
File systems
[dan@knew:~] $ zfs list NAME USED AVAIL REFER MOUNTPOINT system 34.6T 32.2T 320K none system/backups-by-snapshot 9.37G 32.2T 384K /usr/home/ansel/snapshots system/backups-by-snapshot/mailjail 9.37G 32.2T 9.37G /usr/home/ansel/snapshots/mailjail system/backups-by-snapshot/mounted 329K 32.2T 329K /usr/home/ansel/snapshots/mounted system/backups-by-snapshot/testing 329K 32.2T 329K /usr/home/ansel/snapshots/testing system/bootenv 329K 32.2T 329K none system/data 34.0T 32.2T 329K none system/data/bacula 1.30M 32.2T 219K /usr/jails/bacula-sd-01/usr/local/bacula system/data/bacula-volumes 31.5T 32.2T 219K none system/data/bacula-volumes/DailyBackups 336G 32.2T 166G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/DailyBackups system/data/bacula-volumes/DiffFile 715G 32.2T 376G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/DiffFile system/data/bacula-volumes/FullAutoNoNextPool 681G 32.2T 681G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/FullAutoNoNextPool system/data/bacula-volumes/FullFile 27.9T 32.2T 26.3T /usr/jails/bacula-sd-01/usr/local/bacula/volumes/FullFile system/data/bacula-volumes/IncrFile 1.48T 32.2T 720G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/IncrFile system/data/bacula-volumes/IncrFileNoNextPool 193G 32.2T 193G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/IncrFileNoNextPool system/data/bacula-volumes/MonthlyBackups 79.7G 32.2T 68.7G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/MonthlyBackups system/data/bacula-volumes/TwoHourlyBackups 27.1G 32.2T 12.8G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/TwoHourlyBackups system/data/bacula-volumes/WeeklyBackups 92.0G 32.2T 46.2G /usr/jails/bacula-sd-01/usr/local/bacula/volumes/WeeklyBackups system/data/bacula/spooling 201K 32.2T 201K /usr/jails/bacula-sd-01/usr/local/bacula/spooling system/data/bacula/working 914K 32.2T 283K /usr/jails/bacula-sd-01/usr/local/bacula/working system/data/time_capsule 2.48T 32.2T 201K /data/time_capsule system/data/time_capsule/dvl-air01 739G 30.0G 570G /data/time_capsule/dvl-air01 system/data/time_capsule/dvl-dent 504G 246G 401G /data/time_capsule/dvl-dent system/data/time_capsule/dvl-dent-sparse 361G 153G 167G /data/time_capsule/dvl-dent-sparse system/data/time_capsule/dvl-pro02 938G 39.4G 561G /data/time_capsule/dvl-pro02 system/root 612K 32.2T 612K none system/rootfs 15.6G 32.2T 15.6G none system/tank_data 4.39G 32.2T 4.39G none system/tank_data/data 402K 32.2T 201K none system/tank_data/data/bacula-volumes 201K 32.2T 201K none system/tmp 2.95M 32.2T 2.95M none system/usr 574G 32.2T 329K none system/usr/home 2.67G 32.2T 329K /usr/home system/usr/home/dan 2.67G 32.2T 2.67G /usr/home/dan system/usr/jails 565G 32.2T 35.3M /usr/jails system/usr/jails/ansible 3.58G 32.2T 2.88G /usr/jails/ansible system/usr/jails/bacula-sd-01 3.27G 32.2T 2.48G /usr/jails/bacula-sd-01 system/usr/jails/basejail 2.60G 32.2T 2.23G /usr/jails/basejail system/usr/jails/cliff 2.41G 32.2T 2.03G /usr/jails/cliff system/usr/jails/crey 1.70G 32.2T 1.70G /usr/jails/crey system/usr/jails/dbclone 132G 32.2T 120G /usr/jails/dbclone system/usr/jails/dbclone.backups.rsyncer 203G 32.2T 97.6G /usr/jails/dbclone/usr/home/rsyncer/backups system/usr/jails/empty 7.06G 32.2T 6.61G /usr/jails/empty system/usr/jails/ezjail_archives 256K 32.2T 256K /usr/jails/ezjail_archives system/usr/jails/fruity-ext 1.76G 32.2T 1.27G /usr/jails/fruity-ext system/usr/jails/fruity-int 1.86G 32.2T 1.37G /usr/jails/fruity-int system/usr/jails/metrics2 177G 32.2T 177G /usr/jails/metrics2 system/usr/jails/mysql51 7.12M 32.2T 7.12M /usr/jails/mysql51 system/usr/jails/mysql55 3.97G 32.2T 3.59G /usr/jails/mysql55 system/usr/jails/mysql56 3.39G 32.2T 3.02G /usr/jails/mysql56 system/usr/jails/newjail 6.34M 32.2T 6.34M /usr/jails/newjail system/usr/jails/pg10 3.51G 32.2T 3.14G /usr/jails/pg10 system/usr/jails/pg93 3.56G 32.2T 3.20G /usr/jails/pg93 system/usr/jails/pg94 2.99G 32.2T 2.62G /usr/jails/pg94 system/usr/jails/pg95 2.37G 32.2T 2.01G /usr/jails/pg95 system/usr/jails/pg96 2.10G 32.2T 1.74G /usr/jails/pg96 system/usr/jails/snapshots 3.65G 32.2T 2.38G /usr/jails/snapshots system/usr/jails/toiler 2.67G 32.2T 2.26G /usr/jails/toiler system/usr/local 883M 32.2T 883M none system/usr/obj 1.74G 32.2T 1.74G none system/usr/ports 1.80G 32.2T 1.80G none system/usr/ports/distfiles 256K 32.2T 256K none system/usr/src 1.82G 32.2T 1.82G none system/var 717M 32.2T 329K none system/var/audit 366K 32.2T 366K none system/var/empty 329K 32.2T 329K none system/var/log 716M 32.2T 716M none system/var/tmp 457K 32.2T 457K none zroot 14.5G 4.79G 23K /zroot zroot/ROOT 9.77G 4.79G 23K none zroot/ROOT/10.3-RELEASE 1K 4.79G 6.97G / zroot/ROOT/11.1-RELEASE 1K 4.79G 8.05G / zroot/ROOT/default 9.77G 4.79G 8.33G / zroot/tmp 45K 2.00G 27K /tmp zroot/usr 848M 4.79G 23K /usr zroot/usr/local 848M 4.79G 490M /usr/local zroot/var 3.85G 4.79G 25K /var zroot/var/audit 23K 4.79G 23K /var/audit zroot/var/crash 1.50G 4.79G 1.50G /var/crash zroot/var/empty 23K 4.79G 23K /var/empty zroot/var/log 2.35G 1.15G 656K /var/log zroot/var/mail 44K 456K 27K /var/mail zroot/var/tmp 352K 2.00G 322K /var/tmp [dan@knew:~] $
Paritions
[dan@knew:~] $ gpart show
=> 40 468862048 ada0 GPT (224G)
40 2008 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 41943040 3 freebsd-zfs (20G)
50335744 418526344 - free - (200G)
=> 40 468862048 ada1 GPT (224G)
40 2008 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 41943040 3 freebsd-zfs (20G)
50335744 418526344 - free - (200G)
=> 34 9767541101 da0 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
=> 34 9767541101 da1 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
=> 34 9767541101 da2 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da3 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da4 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
=> 34 9767541101 da5 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
=> 40 9767541088 da6 GPT (4.5T)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541088 - free - (752M)
=> 34 9767541101 da7 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da8 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da9 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da10 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da11 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da12 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
=> 34 9767541101 da13 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da14 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da15 GPT (4.5T)
34 2014 - free - (1.0M)
2048 1024 1 freebsd-boot (512K)
3072 1024 - free - (512K)
4096 8388608 2 freebsd-swap (4.0G)
8392704 9759129600 3 freebsd-zfs (4.5T)
9767522304 18831 - free - (9.2M)
=> 34 9767541101 da16 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
=> 34 9767541101 da17 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
=> 40 9767541088 da18 GPT (4.5T)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541088 - free - (752M)
=> 34 9767541101 da19 GPT (4.5T)
34 6 - free - (3.0K)
40 9766000000 1 freebsd-zfs (4.5T)
9766000040 1541095 - free - (752M)
[dan@knew:~] $
There two other SSDs mounted inside the case, but not yet hooked up. The cables for that arrive today.
zpools
[dan@knew:~] $ zpool status pool: system state: ONLINE scan: scrub repaired 0 in 17h46m with 0 errors on Fri Nov 16 21:10:49 2018 config: NAME STATE READ WRITE CKSUM system ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 gpt/X643KHBFF57D.r5.c4 ONLINE 0 0 0 gpt/4728K24SF57D.r3.c2 ONLINE 0 0 0 gpt/37KVK1JRF57D.r2.c1 ONLINE 0 0 0 gpt/37D4KBJPF57D.r5.c3 ONLINE 0 0 0 gpt/5782KL6VF57D.r2.c2 ONLINE 0 0 0 gpt/6525K2DGFS9A.r2.c4 ONLINE 0 0 0 gpt/579HKDZYF57D.r3.c3 ONLINE 0 0 0 gpt/579HKDZXF57D.r2.c3 ONLINE 0 0 0 gpt/5782KL6MF57D.r3.c1 ONLINE 0 0 0 gpt/X6IEKELNF57D.r4.c4 ONLINE 0 0 0 raidz2-1 ONLINE 0 0 0 gpt/653BK12JFS9A.r4.c2 ONLINE 0 0 0 gpt/579IK5RMF57D.r4.c3 ONLINE 0 0 0 gpt/653EK93PFS9A.r1.c4 ONLINE 0 0 0 gpt/653DK7WPFS9A.r3.c4 ONLINE 0 0 0 gpt/653DK7WCFS9A.r4.c1 ONLINE 0 0 0 gpt/653EK93QFS9A.r5.c2 ONLINE 0 0 0 gpt/653AK2MXFS9A.r1.c2 ONLINE 0 0 0 gpt/6539K3OJFS9A.r1.c1 ONLINE 0 0 0 gpt/653IK1IBFS9A.r5.c1 ONLINE 0 0 0 gpt/57NGK1ZGF57D.r1.c3 ONLINE 0 0 0 errors: No known data errors pool: zroot state: ONLINE scan: scrub repaired 0 in 0h0m with 0 errors on Fri Nov 16 03:25:00 2018 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada1p3 ONLINE 0 0 0 ada0p3 ONLINE 0 0 0 errors: No known data errors [dan@knew:~] $
Those labels should be updated to reflect the new location (e.g. r2.c3) because I am sure not everything went into the same place.
/var/run/dmesg.boot
[dan@knew:~] $ cat /var/run/dmesg.boot
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.2-RELEASE-p4 #0: Thu Sep 27 08:16:24 UTC 2018
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)
VT(vga): resolution 640x480
CPU: Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz (3691.53-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x306f2 Family=0x6 Model=0x3f Stepping=2
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x21<LAHF,ABM>
Structured Extended Features=0x37ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,NFPUSG>
XSAVE Features=0x1<XSAVEOPT>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
TSC: P-state invariant, performance statistics
real memory = 68719476736 (65536 MB)
avail memory = 66537959424 (63455 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
SMP: AP CPU #1 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #2 Launched!
Timecounter "TSC-low" frequency 1845766764 Hz quality 1000
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff80ff4580, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
nexus0
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <SUPERM SMCI--MB> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71,0x74-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> on acpi0
pcib0: _OSC returned error 0x10
pci0: <ACPI PCI bus> on pcib0
pci0: <dasp, performance counters> at device 11.1 (no driver attached)
pci0: <dasp, performance counters> at device 11.2 (no driver attached)
pci0: <dasp, performance counters> at device 16.1 (no driver attached)
pci0: <dasp, performance counters> at device 16.6 (no driver attached)
pci0: <dasp, performance counters> at device 18.1 (no driver attached)
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> on acpi0
acpi_syscontainer2: <System Container> on acpi0
acpi_syscontainer3: <System Container> on acpi0
pcib1: <ACPI Host-PCI bridge> port 0xcf8-0xcff numa-domain 0 on acpi0
pcib1: _OSC returned error 0x10
pci1: <ACPI PCI bus> numa-domain 0 on pcib1
pcib2: <ACPI PCI-PCI bridge> irq 26 at device 1.0 numa-domain 0 on pci1
pci2: <ACPI PCI bus> numa-domain 0 on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 32 at device 2.0 numa-domain 0 on pci1
pci3: <ACPI PCI bus> numa-domain 0 on pcib3
ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0xe000-0xe01f mem 0xfb740000-0xfb75ffff,0xfb700000-0xfb73ffff,0xfb760000-0xfb763fff irq 32 at device 0.0 numa-domain 0 on pci3
ix0: Using MSI-X interrupts with 9 vectors
ix0: Ethernet address: 00:1b:21:39:a9:a1
ix0: PCI Express Bus: Speed 2.5GT/s Width x8
ix0: netmap queues/slots: TX 8/2048, RX 8/2048
pcib4: <ACPI PCI-PCI bridge> irq 32 at device 2.2 numa-domain 0 on pci1
pci4: <ACPI PCI bus> numa-domain 0 on pcib4
pcib5: <ACPI PCI-PCI bridge> irq 40 at device 3.0 numa-domain 0 on pci1
pci5: <ACPI PCI bus> numa-domain 0 on pcib5
mps0: <Avago Technologies (LSI) SAS2308> port 0xd000-0xd0ff mem 0xfb240000-0xfb24ffff,0xfb200000-0xfb23ffff irq 40 at device 0.0 numa-domain 0 on pci5
mps0: Firmware: 20.00.07.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities: 5a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc>
pci1: <unknown> at device 17.0 (no driver attached)
ahci0: <Intel Wellsburg AHCI SATA controller> port 0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0f7,0xf0e0-0xf0e3,0xf020-0xf03f mem 0xfb81c000-0xfb81c7ff irq 16 at device 17.4 numa-domain 0 on pci1
ahci0: AHCI v1.30 with 4 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
xhci0: <Intel Wellsburg USB 3.0 controller> mem 0xfb800000-0xfb80ffff irq 19 at device 20.0 numa-domain 0 on pci1
xhci0: 32 bytes context size, 64-bit DMA
usbus0 numa-domain 0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ehci0: <Intel Wellsburg USB 2.0 controller> mem 0xfb818000-0xfb8183ff irq 18 at device 26.0 numa-domain 0 on pci1
usbus1: EHCI version 1.0
usbus1 numa-domain 0 on ehci0
usbus1: 480Mbps High Speed USB v2.0
hdac0: <Intel Wellsburg HDA Controller> mem 0xfb810000-0xfb813fff irq 22 at device 27.0 numa-domain 0 on pci1
pcib6: <ACPI PCI-PCI bridge> irq 16 at device 28.0 numa-domain 0 on pci1
pci6: <ACPI PCI bus> numa-domain 0 on pcib6
xhci1: <ASMedia ASM1042A USB 3.0 controller> mem 0xfb600000-0xfb607fff irq 16 at device 0.0 numa-domain 0 on pci6
xhci1: 32 bytes context size, 64-bit DMA
xhci1: Unable to map MSI-X table
usbus2 numa-domain 0 on xhci1
usbus2: 5.0Gbps Super Speed USB v3.0
pcib7: <ACPI PCI-PCI bridge> irq 18 at device 28.2 numa-domain 0 on pci1
pci7: <ACPI PCI bus> numa-domain 0 on pcib7
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xc000-0xc01f mem 0xfb500000-0xfb57ffff,0xfb580000-0xfb583fff irq 18 at device 0.0 numa-domain 0 on pci7
igb0: Using MSIX interrupts with 5 vectors
igb0: Ethernet address: 0c:c4:7a:4c:b1:40
igb0: Bound queue 0 to cpu 0
igb0: Bound queue 1 to cpu 1
igb0: Bound queue 2 to cpu 2
igb0: Bound queue 3 to cpu 3
igb0: netmap queues/slots: TX 4/1024, RX 4/1024
pcib8: <ACPI PCI-PCI bridge> irq 19 at device 28.3 numa-domain 0 on pci1
pci8: <ACPI PCI bus> numa-domain 0 on pcib8
igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0xb000-0xb01f mem 0xfb400000-0xfb47ffff,0xfb480000-0xfb483fff irq 19 at device 0.0 numa-domain 0 on pci8
igb1: Using MSIX interrupts with 5 vectors
igb1: Ethernet address: 0c:c4:7a:4c:b1:41
igb1: Bound queue 0 to cpu 4
igb1: Bound queue 1 to cpu 5
igb1: Bound queue 2 to cpu 6
igb1: Bound queue 3 to cpu 7
igb1: netmap queues/slots: TX 4/1024, RX 4/1024
pcib9: <ACPI PCI-PCI bridge> irq 16 at device 28.4 numa-domain 0 on pci1
pci9: <ACPI PCI bus> numa-domain 0 on pcib9
xhci2: <ASMedia ASM1042A USB 3.0 controller> mem 0xfb300000-0xfb307fff irq 16 at device 0.0 numa-domain 0 on pci9
xhci2: 32 bytes context size, 64-bit DMA
xhci2: Unable to map MSI-X table
usbus3 numa-domain 0 on xhci2
usbus3: 5.0Gbps Super Speed USB v3.0
pcib10: <ACPI PCI-PCI bridge> irq 18 at device 28.6 numa-domain 0 on pci1
pci10: <ACPI PCI bus> numa-domain 0 on pcib10
pcib11: <ACPI PCI-PCI bridge> at device 0.0 numa-domain 0 on pci10
pci11: <ACPI PCI bus> numa-domain 0 on pcib11
vgapci0: <VGA-compatible display> port 0xa000-0xa07f mem 0xfa000000-0xfaffffff,0xfb000000-0xfb01ffff irq 18 at device 0.0 numa-domain 0 on pci11
vgapci0: Boot video device
pcib12: <ACPI PCI-PCI bridge> irq 19 at device 28.7 numa-domain 0 on pci1
pci12: <ACPI PCI bus> numa-domain 0 on pcib12
pcib13: <PCI-PCI bridge> irq 16 at device 0.0 numa-domain 0 on pci12
pci13: <PCI bus> numa-domain 0 on pcib13
pcib14: <PCI-PCI bridge> irq 19 at device 3.0 numa-domain 0 on pci13
pci14: <PCI bus> numa-domain 0 on pcib14
pcib15: <PCI-PCI bridge> irq 19 at device 7.0 numa-domain 0 on pci13
pci15: <PCI bus> numa-domain 0 on pcib15
ehci1: <Intel Wellsburg USB 2.0 controller> mem 0xfb817000-0xfb8173ff irq 18 at device 29.0 numa-domain 0 on pci1
usbus4: EHCI version 1.0
usbus4 numa-domain 0 on ehci1
usbus4: 480Mbps High Speed USB v2.0
isab0: <PCI-ISA bridge> at device 31.0 numa-domain 0 on pci1
isa0: <ISA bus> numa-domain 0 on isab0
ahci1: <Intel Wellsburg AHCI SATA controller> port 0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f mem 0xfb816000-0xfb8167ff irq 16 at device 31.2 numa-domain 0 on pci1
ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich4: <AHCI channel> at channel 0 on ahci1
ahcich5: <AHCI channel> at channel 1 on ahci1
ahcich6: <AHCI channel> at channel 2 on ahci1
ahcich7: <AHCI channel> at channel 3 on ahci1
ahcich8: <AHCI channel> at channel 4 on ahci1
ahcich9: <AHCI channel> at channel 5 on ahci1
ahciem1: <AHCI enclosure management bridge> on ahci1
acpi_button0: <Power Button> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est0 attach returned 6
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est1 attach returned 6
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est2 attach returned 6
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est3 attach returned 6
coretemp4: <CPU On-Die Thermal Sensors> on cpu4
est4: <Enhanced SpeedStep Frequency Control> on cpu4
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est4 attach returned 6
coretemp5: <CPU On-Die Thermal Sensors> on cpu5
est5: <Enhanced SpeedStep Frequency Control> on cpu5
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est5 attach returned 6
coretemp6: <CPU On-Die Thermal Sensors> on cpu6
est6: <Enhanced SpeedStep Frequency Control> on cpu6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est6 attach returned 6
coretemp7: <CPU On-Die Thermal Sensors> on cpu7
est7: <Enhanced SpeedStep Frequency Control> on cpu7
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 230500002500
device_attach: est7 attach returned 6
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
ugen4.1: <Intel EHCI root HUB> at usbus4
ugen2.1: <0x1b21 XHCI root HUB> at usbus2
ugen0.1: <0x8086 XHCI root HUB> at usbus0
ugen3.1: <0x1b21 XHCI root HUB> at usbus3
ugen1.1: <Intel EHCI root HUB> at usbus1
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
uhub1: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub2: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub4: <0x1b21 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus3
hdacc0: mps0: <Realtek ALC1150 HDA CODEC>SAS Address for SATA device = d5374c18adac7087
at cad 0 on hdac0
mps0: SAS Address for SATA device = d53d5a19adac7082
hdaa0: <Realtek ALC1150 Audio Function Group>mps0: at nid 1 on hdacc0
SAS Address for SATA device = 45293c139580ada8
mps0: SAS Address for SATA device = 4544280e97809a84
pcm0: <Realtek ALC1150 (Rear Analog 5.1/2.0)>mps0: at nid SAS Address for SATA device = d5395220adac7077
20,22,21 and 24,26 on hdaa0
mps0: SAS Address for SATA device = d53f3627adac7083
mps0: SAS Address for SATA device = d5373520adac7080
mps0: SAS Address for SATA device = 452a26149680948a
mps0: SAS Address for SATA device = 4544281797809a84
pcm1: <Realtek ALC1150 (Front Analog)> at nid mps0: 27 and 25 on hdaa0
SAS Address from SATA device = d5374c18adac7087
mps0: SAS Address from SATA device = d53d5a19adac7082
mps0: SAS Address from SATA device = 45293c139580ada8
mps0: SAS Address from SATA device = 4544280e97809a84
pcm2: <Realtek ALC1150 (Rear Digital)> at nid mps0: 30 on hdaa0
SAS Address from SATA device = d5395220adac7077
mps0: SAS Address from SATA device = d53f3627adac7083
uhub3: <0x1b21 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2
mps0: SAS Address from SATA device = d5373520adac7080
mps0: SAS Address from SATA device = 452a26149680948a
mps0: SAS Address from SATA device = 4544281797809a84
mps0: SAS Address for SATA device = d538502eadac707f
mps0: SAS Address for SATA device = 453a3c119580a686
mps0: SAS Address for SATA device = 452d440e97809b9b
mps0: SAS Address for SATA device = 453c4c1a97809b9a
mps0: SAS Address for SATA device = 453c4c1997809b9a
mps0: SAS Address for SATA device = 45294c089780b099
mps0: SAS Address for SATA device = 45403407ba7f9685
mps0: SAS Address for SATA device = 453d3e0fba7fab97
mps0: SAS Address for SATA device = d53d5a26adac7082
mps0: SAS Address for SATA device = d538471dadac6f73
mps0: SAS Address for SATA device = d53f3626adac7083
mps0: SAS Address from SATA device = d538502eadac707f
mps0: SAS Address from SATA device = 453a3c119580a686
mps0: SAS Address from SATA device = 452d440e97809b9b
mps0: SAS Address from SATA device = 453c4c1a97809b9a
mps0: SAS Address from SATA device = 453c4c1997809b9a
mps0: SAS Address from SATA device = 45294c089780b099
mps0: SAS Address from SATA device = 45403407ba7f9685
mps0: SAS Address from SATA device = 453d3e0fba7fab97
mps0: SAS Address from SATA device = d53d5a26adac7082
mps0: SAS Address from SATA device = d538471dadac6f73
mps0: SAS Address from SATA device = d53f3626adac7083
uhub4: 4 ports with 4 removable, self powered
uhub3: 4 ports with 4 removable, self powered
ses0 at mps0 bus 0 scbus0 target 8 lun 0
ses0: <LSI SAS2X36 0e12> Fixed Enclosure Services SPC-3 SCSI device
ses0: Serial Number
ses0: 600.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 ENC Device
ses1 at ahciem0 bus 0 scbus5 target 0 lun 0
ses1: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses1: SEMB SES Device
ses2 at ahciem1 bus 0 scbus12 target 0 lun 0
ses2: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses2: SEMB SES Device
ses0: (none): Element descriptor: 'Slot 01'
ada0 at ahcich4 bus 0 scbus6 target 0 lun 0
ses0: (none): SAS Device Slot Element: 1 Phys at Slot 0
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e788c
ada0: <INTEL SSDSC2BP240G4 L2010420> ATA8-ACS SATA 3.x device
ada0: Serial Number BTJR5090037J240AGN
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 228936MB (468862128 512 byte sectors)
ada1 at ahcich6 bus 0 scbus8 target 0 lun 0
ada1: <INTEL SSDSC2BP240G4 L2010420> ATA8-ACS SATA 3.x device
ada1: Serial Number BTJR509003N7240AGN
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada1: Command Queueing enabled
ada1: 228936MB (468862128 512 byte sectors)
ses0: da4: Element descriptor: 'Slot 02'
ses0: da4: SAS Device Slot Element: 1 Phys at Slot 1
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e788d
ses0: da9,pass10: Element descriptor: 'Slot 03'
ses0: da9,pass10: SAS Device Slot Element: 1 Phys at Slot 2
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e788e
ses0: da8,pass9: Element descriptor: 'Slot 04'
ses0: da8,pass9: SAS Device Slot Element: 1 Phys at Slot 3
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e788f
ses0: da16,pass20: Element descriptor: 'Slot 05'
da14 at mps0 bus 0 scbus0 target 24 lun 0
da1 at mps0 bus 0 scbus0 target 11 lun 0
ses0: da16,pass20: SAS Device Slot Element: 1 Phys at Slot 4
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7890
da1: da14: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
<ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number 653EK93QFS9A
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 4769307MB (9767541168 512 byte sectors)
da7 at mps0 bus 0 scbus0 target 17 lun 0
da14: Serial Number 579HKDZXF57D
da14: 600.000MB/s transfers
da14: Command Queueing enabled
da14: 4769307MB (9767541168 512 byte sectors)
uhub0: da7: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da7: Serial Number X6IEKELNF57D
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 4769307MB (9767541168 512 byte sectors)
da13 at mps0 bus 0 scbus0 target 23 lun 0
da2 at mps0 bus 0 scbus0 target 12 lun 0
2 ports with 2 removable, self powered
uhub2: da13: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da13: Serial Number 5782KL6VF57D
da13: 600.000MB/s transfers
da13: Command Queueing enabled
da13: 4769307MB (9767541168 512 byte sectors)
da2: 2 ports with 2 removable, self powered
<ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da9 at mps0 bus 0 scbus0 target 19 lun 0
da9: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da9: Serial Number 37KVK1JRF57D
da2: Serial Number 37D4KBJPF57D
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 4769307MB (9767541168 512 byte sectors)
da10 at mps0 bus 0 scbus0 target 20 lun 0
da9: 600.000MB/s transfers
da10: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da10: Serial Number 4728K24SF57D
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 4769307MB (9767541168 512 byte sectors)
da9: Command Queueing enabled
da9: 4769307MB (9767541168 512 byte sectors)
da11 at mps0 bus 0 scbus0 target 21 lun 0
da11: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da11: Serial Number 579HKDZYF57D
da18 at mps0 bus 0 scbus0 target 28 lun 0
da18: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da18: Serial Number 57NGK1ZGF57D
da18: 600.000MB/s transfers
da18: Command Queueing enabled
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 4769307MB (9767541168 512 byte sectors)
da3 at mps0 bus 0 scbus0 target 13 lun 0
da3: da18: 4769307MB (9767541168 512 byte sectors)
<ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
uhub1: da3: Serial Number X643KHBFF57D
da3: 600.000MB/s transfers21 ports with 21 removable, self powered
da6 at mps0 bus 0 scbus0 target 16 lun 0
ses0: da1,pass2: Element descriptor: 'Slot 07'
da6: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da3: Command Queueing enabled
da3: 4769307MB (9767541168 512 byte sectors)
da15 at mps0 bus 0 scbus0 target 25 lun 0
ses0: da1,pass2: SAS Device Slot Element: 1 Phys at Slot 6
ses0: phy 0: SATA device
da6: Serial Number 579IK5RMF57D
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da15: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da6: 4769307MB (9767541168 512 byte sectors)
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7892
da15: Serial Number 6525K2DGFS9A
da8 at mps0 bus 0 scbus0 target 18 lun 0
da8: <ATA TOSHIBA HDWE150 FP2A> Fixed Direct Access SPC-4 SCSI device
da8: Serial Number 5782KL6MF57D
da8: 600.000MB/s transfersda15: 600.000MB/s transfers
da15: Command Queueing enabled
da15: 4769307MB (9767541168 512 byte sectors)
da5 at mps0 bus 0 scbus0 target 15 lun 0
da8: Command Queueing enabled
da8: 4769307MB (9767541168 512 byte sectors)
da4 at mps0 bus 0 scbus0 target 14 lun 0
da4: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da4: Serial Number 653DK7WCFS9A
da5: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da5: Serial Number 653BK12JFS9A
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 4769307MB (9767541168 512 byte sectors)
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da19 at mps0 bus 0 scbus0 target 29 lun 0
da19: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da19: Serial Number 653EK93PFS9A
da19: 600.000MB/s transfers
da19: Command Queueing enabled
da4: 4769307MB (9767541168 512 byte sectors)
da0 at mps0 bus 0 scbus0 target 10 lun 0
da19: 4769307MB (9767541168 512 byte sectors)
da17 at mps0 bus 0 scbus0 target 27 lun 0
ses0: da5,pass6: Element descriptor: 'Slot 08'
da17: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da0: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number 653IK1IBFS9A
da0: 600.000MB/s transfers
da0: Command Queueing enabled
ses0: da5,pass6: SAS Device Slot Element: 1 Phys at Slot 7
da17: Serial Number 653AK2MXFS9A
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7893
da0: 4769307MB (9767541168 512 byte sectors)
da16 at mps0 bus 0 scbus0 target 26 lun 0
da17: 600.000MB/s transfers
da17: Command Queueing enabled
da17: 4769307MB (9767541168 512 byte sectors)
da16: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da16: Serial Number 6539K3OJFS9A
da16: 600.000MB/s transfers
da16: Command Queueing enabled
da16: 4769307MB (9767541168 512 byte sectors)
da12 at mps0 bus 0 scbus0 target 22 lun 0
da12: <ATA TOSHIBA MD04ACA5 FP2A> Fixed Direct Access SPC-4 SCSI device
da12: Serial Number 653DK7WPFS9A
da12: 600.000MB/s transfers
da12: Command Queueing enabled
da12: 4769307MB (9767541168 512 byte sectors)
ses0: da10,pass12: Element descriptor: 'Slot 09'
ses0: da10,pass12: SAS Device Slot Element: 1 Phys at Slot 8
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7894
ses0: da13,pass15: Element descriptor: 'Slot 10'
ses0: da13,pass15: SAS Device Slot Element: 1 Phys at Slot 9
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7895
ses0: da17,pass21: Element descriptor: 'Slot 11'
ses0: da17,pass21: SAS Device Slot Element: 1 Phys at Slot 10
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7896
ses0: da2,pass3: Element descriptor: 'Slot 13'
ugen4.2: <vendor 0x8087 product 0x8002> at usbus4
ses0: da2,pass3: SAS Device Slot Element: 1 Phys at Slot 12
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7898
ses0: da6,pass7: Element descriptor: 'Slot 14'
uhub5ses0: da6,pass7: SAS Device Slot Element: 1 Phys at Slot 13
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e7899
ses0: da11,pass13: Element descriptor: 'Slot 15'
numa-domain 0 on uhub0
uhub5: <vendor 0x8087 product 0x8002, class 9/0, rev 2.00/0.05, addr 2> on usbus4
ugen1.2: <vendor 0x8087 product 0x800a> at usbus1
ses0: da11,pass13: SAS Device Slot Element: 1 Phys at Slot 14
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e789a
ses0: da14,pass17: Element descriptor: 'Slot 16'
uhub6ses0: da14,pass17: SAS Device Slot Element: 1 Phys at Slot 15
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e789b
ses0: da18,pass22: Element descriptor: 'Slot 17'
numa-domain 0ses0: da18,pass22: SAS Device Slot Element: 1 Phys at Slot 16
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e789c
ses0: da3,pass4: Element descriptor: 'Slot 19'
on uhub2
ses0: da3,pass4: SAS Device Slot Element: 1 Phys at Slot 18
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e789e
ses0: da7,pass8: Element descriptor: 'Slot 20'
uhub6: <vendor 0x8087 product 0x800a, class 9/0, rev 2.00/0.05, addr 2> on usbus1
ugen0.2: <vendor 0x0557 product 0x7000> at usbus0
ses0: da7,pass8: SAS Device Slot Element: 1 Phys at Slot 19
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e789f
ses0: da12,pass14: Element descriptor: 'Slot 21'
uhub7ses0: da12,pass14: SAS Device Slot Element: 1 Phys at Slot 20
numa-domain 0 on uhub1
uhub7: <vendor 0x0557 product 0x7000, class 9/0, rev 2.00/0.00, addr 1> on usbus0
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e78a0
ses0: da15,pass19: Element descriptor: 'Slot 22'
ses0: da15,pass19: SAS Device Slot Element: 1 Phys at Slot 21
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e78a1
ses0: da19,pass23: Element descriptor: 'Slot 24'
ses0: da19,pass23: SAS Device Slot Element: 1 Phys at Slot 23
ses0: phy 0: SATA device
ses0: phy 0: parent 50030480014e78bf addr 50030480014e78a3
Trying to mount root from zfs:zroot/ROOT/default []...
uhub6: 6 ports with 6 removable, self powered
uhub7: 4 ports with 3 removable, self powered
uhub5: 8 ports with 8 removable, self powered
random: unblocking device.
ugen0.3: <vendor 0x0557 product 0x2419> at usbus0
ukbd0 numa-domain 0 on uhub7
ukbd0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
kbd2 at ukbd0
GEOM_MIRROR: Force device swap start due to timeout.
GEOM_MIRROR: Device mirror/swap launched (2/10).
ix0: link state changed to UP
ums0 numa-domain 0 on uhub7
ums0: <vendor 0x0557 product 0x2419, class 0/0, rev 1.10/1.00, addr 2> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
[dan@knew:~] $
