Why didn’t I choose the bigger CPUs? – Dan Langille's Other Diary

I have four Dell R730 servers in the basement:

r730-01 – main development server
r730-02 – unused
r730-03 – main storage server
r730-04 – unused

So what’s on each one?

If you look at each of the links, you’ll find this information on CPU and RAM. The information is collected from /var/run/dmesg.boot, but not presented as a direct copy/paste. It has been rearranged for ease of reading.

r730-01

This host is used for package building, FreshPorts (dev, test, and stage), database servers, and other things. FYI, a full build of all 1104 packages takes about 3:10:55.

For reference, the link to the chip specifications is provided.

“Thermal Design Power (TDP) represents the average power, in watts, the processor dissipates when operating at Base Frequency with all cores active under an Intel-defined, high-complexity workload. Refer to Datasheet for thermal solution requirements.”

Which means the W value shown is the maximum usage, not usage at idle. These hosts are largely idle, with daily peak periods.

real memory  = 481032142848 (458748 MB)
avail memory = 468404133888 (446704 MB)
CPU: Intel(R) Xeon(R) CPU E5-2650L v3 @ 1.80GHz (1800.09-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 48 CPUs
FreeBSD/SMP: 2 package(s) x 12 core(s) x 2 hardware threads

ark.intel.com – TDP 65 W – r730-01

r730-02

real memory  = 68717379584 (65534 MB)
avail memory = 66623410176 (63537 MB)
CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz (2300.13-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 72 CPUs
FreeBSD/SMP: 2 package(s) x 18 core(s) x 2 hardware threads

ark.intel.com – TDP 145 W – r730-02

r730-03

real memory  = 68717379584 (65534 MB)
avail memory = 66705068032 (63614 MB)
CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz (2300.12-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 72 CPUs
FreeBSD/SMP: 2 package(s) x 18 core(s) x 2 hardware threads

ark.intel.com – TDP 145 W – r730-03

r730-04

real memory  = 137434759168 (131068 MB)
avail memory = 133590097920 (127401 MB)
CPU: Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (3200.16-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 hardware threads

ark.intel.com – TDP 135 W – r730-04

So why those choices?

You might noticed that r730-02 has 72 CPUs and r730-04 has 132 GB of RAM.

Compare that to my primary server (r730-01) which has 48 CPUs and 458 GB of RAM. That host can get busy. Here it is building packages. I think it would benefit from additional CPUs.

last pid: 57029;  load averages: 106.93, 116.37, 128.24                                                                                                             up 16+02:03:24  14:50:20
37 processes:  1 running, 36 sleeping
CPU: 65.8% user, 28.7% nice,  5.4% system,  0.0% interrupt,  0.0% idle
Mem: 43G Active, 75G Inact, 70G Wired, 247G Free
ARC: 31G Total, 18G MFU, 10G MRU, 55M Anon, 811M Header, 1979M Other
     26G Compressed, 30G Uncompressed, 1.16:1 Ratio
Swap: 20G Total, 20G Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
36042 root         14  20    0  8251M  7279M kqread  43  48.5H  34.65% bhyve: hass (bhyve)
52106 dvl           1  20    0    14M  3988K CPU5     5   0:00   0.32% top
 4635 root          1  20    0    47M    28M select  11  20:42   0.17% /usr/local/sbin/snmpd -p /var/run/net_snmpd.pid -c /usr/local/etc/snmpd.conf -a -r
 4160 nut           1  20    0    17M  5868K nanslp  14  13:03   0.10% /usr/local/libexec/nut/dummy-ups -a ups02
 4176 nut           1  20    0    17M  5776K nanslp  35   0:29   0.01% /usr/local/sbin/upsmon localhost
 4358 ntpd          1  20    0    21M  6636K select  29   1:52   0.01% /usr/sbin/ntpd -L -p /var/db/ntp/ntpd.pid -c /etc/ntp.conf -f /var/db/ntp/ntpd.drift
 4164 nut           1  20    0   408M  5732K select  13   0:42   0.01% /usr/local/sbin/upsd
 3896 root          1  20    0    13M  2508K kqread   5   0:58   0.01% /usr/sbin/rtsold -i -m bridge0
22205 dvl           1  20    0    21M    10M select  13   0:00   0.01% sshd: dvl@pts/3 (sshd)
 3904 root          1  20    0    13M  2588K select   5   0:45   0.01% rtsold: system.syslog (rtsold)
 4162 nut           1  20    0    16M  5588K nanslp  40   0:35   0.01% /usr/local/libexec/nut/dummy-ups -a heartbeat
 3894 _pflogd       1  20    0    13M  3052K bpf     31   0:31   0.00% pflogd: [running] -s 116 -i pflog0 -f /var/log/pflog (pflogd)

So why did I choose the hosts I did?

Drive slots.

r730-01 has 2.5″ drive bays, so I choose it first.

Later, when I needed HDD, I chose r730-03 because it has 3.5″ drive bays.