My HDD are full. Time for a new server.

14 Comments / General / By

My disks are full.

[dan@knew:~] $ zpool list
NAME     SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
system    27T  21.4T  5.57T    79%  1.00x  ONLINE  -
[dan@knew:~] $ zpool status
  pool: system
 state: ONLINE
  scan: scrub in progress since Tue Jun  9 03:26:26 2015
        12.9T scanned out of 21.5T at 98.1M/s, 25h32m to go
        0 repaired, 59.92% done
config:

	NAME           STATE     READ WRITE CKSUM
	system         ONLINE       0     0     0
	  raidz2-0     ONLINE       0     0     0
	    gpt/disk0  ONLINE       0     0     0
	    gpt/disk1  ONLINE       0     0     0
	    gpt/disk2  ONLINE       0     0     0
	    gpt/disk3  ONLINE       0     0     0
	    gpt/disk4  ONLINE       0     0     0
	    gpt/disk5  ONLINE       0     0     0
	    gpt/disk6  ONLINE       0     0     0
	    gpt/disk7  ONLINE       0     0     0
	    gpt/disk8  ONLINE       0     0     0
	    gpt/disk9  ONLINE       0     0     0

errors: No known data errors
[dan@knew:~] $

It’s time to build a new server. What better place to seek opinions than BSDCan1?

Constraints

  1. This server will sit in my spare bedroom in my apartment. It must be reasonably quiet. This eliminates most rackmount cases.
  2. Unless reasonably priced, I’m buying consumer, not enterprise products.
  3. My goal is ZFS raidz3 storage, so I’m considering getting 11 HDD.
  4. I think 6TB drives are too expensive just now, so my preference is 5TB HDD.
  5. The above means SATA (as opposed to SAS).
  6. I will get a power supply which is quite over powered for what I’m using, because skimping on a PSU is rarely a recipe for success.
  7. The system will run FreeBSD 10.x, with ZFS as root-file system, with a beadm layout.

The starting point

The case: LIAN LI PC-A75 Black Aluminum ATX Full Tower $175

THE BIGGER CASE: LIAN LI PC-D8000 and I see it at PROVANTAGE for $340.

The power supply: Antec TruePower Classic series TP-750C 750W $90

The main board: TYAN S5535AG2NR-HE Micro ATX Server Motherboard LGA 1150 Intel C226 DDR3 1600/1333/1066 $180

The CPU: Intel Core i3-4350 Haswell Dual-Core 3.6GHz LGA 1150 54W BX80646I34350 $150

The RAM (x4): Kingston 8GB 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1600 (PC3 12800) 4 x $68 = $272

The storage

This SAS RAID controller was mentioned. It’s an IBM – SERVERAID M1015 8CHANNEL PCI-E X8 SAS/SATA. 2 x $100 = $200

The harddrives I’m thinking of are the TOSHIBA PH3500U-1I72 5TB 7200 RPM 128MB Cache Serial ATA 3.0 11 * $160 = $1760

From here?

Total cost (not including tax, possible shipping, various cables): $2827

From here, I’m not sure where we’ll go. I might order this tomorrow. I might wait and see. Recommendations and comments are most welcome.

1Full disclosure: I am the founder of BSDCan.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

14 thoughts on “My HDD are full. Time for a new server.”

  1. Ken

    Why build out a new server vs simply swap out the drives?

    Other than that the above looks like a nice setup. I would consider using NAS4Free unless your needed the tools of a full FreeBSD OS.

    1. Dan Langille

      I have thought of just swapping drives. That would leave me with 10x 3TB unused, which I could then put into another machine.

      The knew server (opteron, spec in that link) seems ok, however it is not as snappy as slocum (xeon, specs are in the link). Both have 32GB RAM and the same brand/model of HDD. But slocum is much faster.

      My current thinking (sort of outlined on this FreeBSD Forums post) is to get a bigger case, and put the existing slocum server into a bigger box and add another zpool of HDD. I think this would result in some things being faster (e.g. dumping a database from one zpool to disk in another zpool).

      The concept of merging both knew and slocum into single server and adding additional HDD to *that* server seems rather appealing at present.

    2. Dan Langille

      Thinking lightly of swapping the HDD, yes, I could give knew, the server running short of space, 10x5TB HDD instead of the present 10x3TB HDD. This would increase available space by 66%. That’s quite a boost.

      Hmmm, I now realize the wapping the HDD would involve removing 1 HDD at a time, and allowing time for re-silvering. That would take about 10 days I’m guessing. One HDD a day: power off, remove HDD, insert new HDD, power up, add HDD into zpool.

      I could save time by doing the gpart on another system. That would also reduce the chance of error.

      After all the replacements are done, I could take the existing 3TB drives (at present in use by knew) and create the new zpool.

      Comments? Ideas? Suggestions?

  4. Ken

    29 drives, wow that’s a lot of storage, a lot of power consumption

    At that scale what is the better bet mirrored drives aka RaidZ or mirrored servers via HAST & CARP?

      1. Ken

        I probably should have stated earlier my questions were more for my own learning.

        My use case is absolutely puny in comparison! I have a NAS4Free box with 3 X 2TB WD Greens where only 45 % of the space is used.

        I always find large storage systems fascinating and appreciate the dialogue, thank you

        1. Dan Langille

          HAST & CARP is more for redundancy and maintaining uptime. I don’t need that. I need storage. If this system dies, nobody gets hurt.

          This isn’t for production. It’s for my ‘home’ backups. I like to keep several years worth of backups.

  5. Dan Langille

    NOTE: to self: seasons for a faster server:

    * more PCI lanes to run multiple HBA (for the large number of disk drives) and a SCSI card (for external tape library)
    * more RAM (current is 32GB) to run Elasticsearch
    * right now, 1368 MB free out of 2047 MB SWAP indicates to me, more RAM would be better
    * merging jails from another server into this one (they run the regression testing).

Leave a Comment

Scroll to Top