Aug 162013
 

Today I created the new gateway machine. It is a ZFS-only system with 3x3TB drives in a mirror configuration.

The hardware was my old ZFS workhorse, now replaced by something with more storage.

I had a moment of panic, thinking this server had just one NIC, but I was wrong. em0 will be the local 1G LAN. re0 will have to be my upstream connection. That’s pretty sad, given it’s only a 10Mb NIC. I am sure I have some fxp(4) NICs around here. I should use that instead. I know they were 100Mb NICs.

$ ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,VLAN_HWTSO>
	ether 00:1b:21:51:ab:2d
	inet 10.105.0.12 netmask 0xffffff00 broadcast 10.105.0.255
	inet6 fe80::21b:21ff:fe51:ab2d%em0 prefixlen 64 scopeid 0x1 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
re0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
	ether e0:cb:4e:42:f0:ff
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect (10baseT/UTP <half-duplex>)
	status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
# zfs list
NAME                         USED  AVAIL  REFER  MOUNTPOINT
system                       664M  2.68T   144K  /
system/root                  144K  2.68T   144K  /root
system/rootfs                661M  2.68T   661M  legacy
system/tmp                   176K  2.68T   176K  /tmp
system/usr                  1.16M  2.68T   144K  /usr
system/usr/home              320K  2.68T   144K  /usr/home
system/usr/home/dan          176K  2.68T   176K  /usr/home/dan
system/usr/local             144K  2.68T   144K  /usr/local
system/usr/obj               144K  2.68T   144K  /usr/obj
system/usr/ports             296K  2.68T   152K  /usr/ports
system/usr/ports/distfiles   144K  2.68T   144K  /usr/ports/distfiles
system/usr/src               144K  2.68T   144K  /usr/src
system/var                   636K  2.68T   144K  /var
system/var/audit             144K  2.68T   144K  /var/audit
system/var/log               196K  2.68T   196K  /var/log
system/var/tmp               152K  2.68T   152K  /var/tmp


# zpool status
  pool: system
 state: ONLINE
  scan: none requested
config:

	NAME           STATE     READ WRITE CKSUM
	system         ONLINE       0     0     0
	  mirror-0     ONLINE       0     0     0
	    gpt/disk0  ONLINE       0     0     0
	    gpt/disk1  ONLINE       0     0     0
	    gpt/disk2  ONLINE       0     0     0

errors: No known data errors


# zpool list
NAME     SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
system  2.72T   664M  2.72T     0%  1.00x  ONLINE  -


# gpart show
=>        34  5860533101  ada0  GPT  (2.7T)
          34          94     1  freebsd-boot  (47k)
         128     4194304     2  freebsd-swap  (2.0G)
     4194432  5851054080     3  freebsd-zfs  (2.7T)
  5855248512     5284623        - free -  (2.5G)

=>        34  5860533101  ada1  GPT  (2.7T)
          34          94     1  freebsd-boot  (47k)
         128     4194304     2  freebsd-swap  (2.0G)
     4194432  5851054080     3  freebsd-zfs  (2.7T)
  5855248512     5284623        - free -  (2.5G)

=>        34  5860533101  ada2  GPT  (2.7T)
          34          94     1  freebsd-boot  (47k)
         128     4194304     2  freebsd-swap  (2.0G)
     4194432  5851054080     3  freebsd-zfs  (2.7T)
  5855248512     5284623        - free -  (2.5G)

# 

A simple test of speed:

# time portsnap fetch extract
Looking up portsnap.FreeBSD.org mirrors... 7 mirrors found.
Fetching public key from your-org.portsnap.freebsd.org... done.
Fetching snapshot tag from your-org.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Fetching snapshot generated at Fri Aug 16 00:05:33 UTC 2013:
521ce25aa9b1fb3a14c289c669f22023b4e6d12fcfb2c4100% of   69 MB 2657 kBps 00m00s
Extracting snapshot... 
...
/usr/ports/x11/yalias/
/usr/ports/x11/yeahconsole/
/usr/ports/x11/yelp/
/usr/ports/x11/zenity/
Building new INDEX files... done.
52.774u 95.826s 2:40.30 92.6%	78+1512k 380+148972io 351pf+0w
# 

I have 26 packages installed. I have no plans for any more. Unless I missed something… Nothing is configured yet. Just installed.

$ pkg_info
autoconf-2.69       Automatically configure source code on many Un*x platforms
autoconf-wrapper-20130530 Wrapper script for GNU autoconf
automake-1.14       GNU Standards-compliant Makefile generator
automake-wrapper-20130530 Wrapper script for GNU automake
bacula-client-5.2.12_3 Network backup solution (client)
bash-4.2.45         The GNU Project's Bourne Again SHell
bison-2.7.1,1       A parser generator from FSF, (mostly) compatible with Yacc
bsdstats-5.5_3      Monthly script for reporting anonymous statistics about you
db41-4.1.25_4       The Berkeley DB package, revision 4.1
dialog4ports-0.1.5_1 Console Interface to configure ports
easy-rsa-2.2.0.m    Small RSA key management package based on openssl
gettext-0.18.3      GNU gettext package
gmake-3.82_1        GNU version of 'make' utility
help2man-1.43.3     Automatically generating simple manual pages from program o
libexecinfo-1.1_3   A library for inspecting program's backtrace
libffi-3.0.13       Foreign Function Interface
libiconv-1.14_1     A character set conversion library
libtool-2.4.2       Generic shared library support script
libyaml-0.1.4_2     A YAML 1.1 parser and emitter written in C
logcheck-1.3.15_2   Auditing tool for system logs on Unix boxes
lzo2-2.06           Portable speedy, lossless data compression library
m4-1.4.16_1,1       GNU m4
mime-construct-1.11 Construct and optionally send MIME messages from command li
nagios-plugins-1.4.16_2,1 Plugins for Nagios
noip-2.1.9_2        No-IP.com's dynamic DNS update client
nrpe-2.13_2         Nagios Remote Plugin Executor
openvpn-2.3.2       Secure IP/Ethernet tunnel daemon
p5-IPC-Signal-1.00  IPC::Signal - Utility functions dealing with signals
p5-Locale-gettext-1.05_3 Message handling functions
p5-MIME-Base64-3.14 Perl5 module for Base64 and Quoted-Printable encodings
p5-MIME-Types-1.38  Perl extension for determining MIME types
p5-Proc-WaitStat-1.00 Interpret and act on wait() status values
pcre-8.33           Perl Compatible Regular Expressions library
perl-5.14.4         Practical Extraction and Report Language
pkgconf-0.9.2_1     Utility to help to configure compiler and linker flags
portaudit-0.6.1     Checks installed ports against a list of security vulnerabi
portupgrade-2.4.11,2 FreeBSD ports/packages administration and management tool s
postfix-2.10.0,1    Secure alternative to widely-used Sendmail
procmail-3.22_7     A local mail delivery agent
ruby-1.8.7.371,1    An object-oriented interpreted scripting language
ruby-1.9.3.448,1    An object-oriented interpreted scripting language
ruby19-bdb-0.6.6_1  Ruby interface to Oracle Berkeley DB revision 2 or later
ruby19-date2-4.0.19 An alternative date class for Ruby

The dmesg is below. I’m only slightly concerned by the messages on lines 8-13.

cat /var/run/dmesg.boot
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-RELEASE #0 r243825: Tue Dec  4 09:23:10 UTC 2012
    root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
can't re-use a leaf (geom_label)!
can't re-use a leaf (geom_part_gpt)!
module_register: module g_label already exists!
Module g_label failed to register: 17
module_register: module g_part_gpt already exists!
Module g_part_gpt failed to register: 17
CPU: AMD Phenom(tm) II X4 945 Processor (3010.22-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100f42  Family = 10  Model = 4  Stepping = 2
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
  TSC: P-state invariant
real memory  = 8589934592 (8192 MB)
avail memory = 8241164288 (7859 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <111909 APIC1708>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
ACPI Warning: Optional field Pm2ControlBlock has zero address or length: 0x0000000000000000/0x1 (20110527/tbfadt-586)
ioapic0 <Version 2.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
GEOM_PART: scheme GPT is already registered!
cryptosoft0: <software crypto> on motherboard
acpi0: <111909 RSDT1708> on motherboard
acpi0: Power Button (fixed)
acpi0: reservation of fee00000, 1000 (3) failed
acpi0: reservation of ffb80000, 80000 (3) failed
acpi0: reservation of fec10000, 20 (3) failed
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, dfe00000 (3) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 450
Event timer "HPET1" frequency 14318180 Hz quality 450
Event timer "HPET2" frequency 14318180 Hz quality 450
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 18 at device 2.0 on pci0
pci3: <ACPI PCI bus> on pcib1
em0: <Intel(R) PRO/1000 Network Connection 7.3.2> port 0xec00-0xec1f mem 0xfbfe0000-0xfbffffff,0xfbf00000-0xfbf7ffff,0xfbfdc000-0xfbfdffff irq 18 at device 0.0 on pci3
em0: Using MSIX interrupts with 3 vectors
em0: Ethernet address: 00:1b:21:51:ab:2d
pcib2: <ACPI PCI-PCI bridge> irq 18 at device 6.0 on pci0
pci2: <ACPI PCI bus> on pcib2
re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F PCIe Gigabit Ethernet> port 0xd800-0xd8ff mem 0xfbeff000-0xfbefffff irq 18 at device 0.0 on pci2
re0: Using 1 MSI message
re0: Chip rev. 0x38000000
re0: MAC rev. 0x00000000
miibus0: <MII bus> on re0
rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
rgephy0:  none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
re0: Ethernet address: e0:cb:4e:42:f0:ff
ahci0: <ATI IXP700 AHCI SATA controller> port 0xc000-0xc007,0xb000-0xb003,0xa000-0xa007,0x9000-0x9003,0x8000-0x800f mem 0xfb7fe400-0xfb7fe7ff irq 22 at device 17.0 on pci0
ahci0: AHCI v1.10 with 4 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ohci0: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfb7fa000-0xfb7fafff irq 16 at device 18.0 on pci0
usbus0 on ohci0
ohci1: <OHCI (generic) USB controller> mem 0xfb7fb000-0xfb7fbfff irq 16 at device 18.1 on pci0
usbus1 on ohci1
ehci0: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfb7fe800-0xfb7fe8ff irq 17 at device 18.2 on pci0
ehci0: AMD SB600/700 quirk applied
usbus2: EHCI version 1.0
usbus2 on ehci0
ohci2: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfb7fc000-0xfb7fcfff irq 18 at device 19.0 on pci0
usbus3 on ohci2
ohci3: <OHCI (generic) USB controller> mem 0xfb7fd000-0xfb7fdfff irq 18 at device 19.1 on pci0
usbus4 on ohci3
ehci1: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfb7fec00-0xfb7fecff irq 19 at device 19.2 on pci0
ehci1: AMD SB600/700 quirk applied
usbus5: EHCI version 1.0
usbus5 on ehci1
pci0: <serial bus, SMBus> at device 20.0 (no driver attached)
atapci0: <ATI IXP700/800 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
pcib3: <ACPI PCI-PCI bridge> at device 20.4 on pci0
pci1: <ACPI PCI bus> on pcib3
vgapci0: <VGA-compatible display> mem 0xfb800000-0xfbbfffff,0xfbdf0000-0xfbdfffff irq 21 at device 6.0 on pci1
ohci4: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfb7ff000-0xfb7fffff irq 18 at device 20.5 on pci0
usbus6 on ohci4
acpi_button0: <Power Button> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
fdc0: <floppy drive controller (FDE)> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
ctl: CAM Target Layer loaded
acpi_throttle0: <ACPI CPU Throttling> on cpu0
hwpstate0: <Cool`n'Quiet 2.0> on cpu0
ZFS filesystem version 5
ZFS storage pool version 28
Timecounters tick every 1.000 msec
The GEOM class LABEL is already loaded.
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 480Mbps High Speed USB v2.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 480Mbps High Speed USB v2.0
usbus6: 12Mbps Full Speed USB v1.0
ugen0.1: <ATI> at usbus0
uhub0: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <ATI> at usbus1
uhub1: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <ATI> at usbus2
uhub2: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
ugen3.1: <ATI> at usbus3
uhub3: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <ATI> at usbus4
uhub4: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <ATI> at usbus5
uhub5: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus5
ugen6.1: <ATI> at usbus6
uhub6: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
uhub6: 2 ports with 2 removable, self powered
uhub0: 3 ports with 3 removable, self powered
uhub1: 3 ports with 3 removable, self powered
uhub3: 3 ports with 3 removable, self powered
uhub4: 3 ports with 3 removable, self powered
(aprobe0:ahcich0:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe0:ahcich0:0:15:0): CAM status: Command timeout
(aprobe0:ahcich0:0:15:0): Error 5, Retries exhausted
(aprobe1:ahcich1:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe1:ahcich1:0:15:0): CAM status: Command timeout
(aprobe1:ahcich1:0:15:0): Error 5, Retries exhausted
(aprobe2:ahcich2:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe2:ahcich2:0:15:0): CAM status: Command timeout
(aprobe2:ahcich2:0:15:0): Error 5, Retries exhausted
uhub2: 6 ports with 6 removable, self powered
uhub5: 6 ports with 6 removable, self powered
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <TOSHIBA DT01ACA300 MX6OABB0> ATA-8 SATA 3.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <TOSHIBA DT01ACA300 MX6OABB0> ATA-8 SATA 3.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
ada2: <TOSHIBA DT01ACA300 MX6OABB0> ATA-8 SATA 3.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad8
SMP: AP CPU #2 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
Timecounter "TSC-low" frequency 11758654 Hz quality 800
GEOM_MIRROR: Device mirror/swap launched (3/3).
Trying to mount root from zfs:system/rootfs []...
GEOM_ELI: Device mirror/swap.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:     Crypto: software
Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive