NOTES:
- All prices exclude taxes and shipping, unless otherwise specified.
- When a price indicates delivered, it includes delivery and taxes.
- All prices are in USD.
Yes, it’s time to build a new server because my current backup server is running out of space. This server will be used for more than just backups. What else, I’m not sure, but the servers it will replace also does these tasks (sorted by my estimate order of total system load):
- Database backup regression testing (loads the daily database backup to ensure the backup is valid)
- Bacula regression testing for various combinations of Bacula 5.x & Bacula 7.x, and MySQL 5.1, 5.5, & 5.6, and PostgreSQL 9.0, 9.1, 9.2, 9.3, & 9.4.
- Runs an instance of ElasticSearch gathering various trivial statistics
- the outgoing MTA for my home network
- DNS
- My Ansible instance
The decision process
I can make most coding and design decisions quickly. Hardware is … harder. I posted on https://www.reddit.com/r/homelab and to https://www.reddit.com/r/buildapc. I did a lot of asking on IRC and emailed a few people. Many suggestions were made. Finally, the selection is up to me.
The parts list
The case I have decided upon is a huge monster of a box. The Lian Li PC-D8000 holds 20 HDD and has plenty of ventilation capacity. I found it on eBay for $351.15 delivered (it should arrive this coming Friday).
The PSU is a EVGA SuperNOVA 850 G2 80+ GOLD, 850W ECO Mode Fully Modular for $139.99 delivered.
The motherboard is a Supermicro LGA2011/Intel C612 ATX Server X10SRA-F-O (NOTE: I was told by Supermicro that the -O suffix indicates a retail box). $318.00
The CPU is an Intel Xeon E5-1630 v3 Quad-Core Haswell Processor 3.7GHz 0GT for $369.99
For the OS, I bought two SSD in the form 2 x Intel 730 240 GB SSD for $144.89 each, total of $289.78 delivered.
I bought 64GB of RAM in 4 sticks of Samsung DDR4-2133 16GB/2Gx72 ECC/REG CL15 Server Memory at $153.99 per stick, total of $615.96
For cooling, I bought 7 x Cooler Master Blade Master 120 – Sleeve Bearing 120mm fans at $11.70 each, total of $81.90 delivered.
I have ordered two fan filters, which attach to the outside of the case. The frame filter uses an adhesive backing. The filter then attaches via a magnetic frame. I ordered the Standard Triple 120mm Radiator Dust Filter (by DEMCiflex) from OutletPC (link to their product page). Cost was $19.98 per filter, for a total cost of $47.91 including shipping.
Other parts
I have some Toshiba 3TB HDD which might form the basis of the storage in this system. I have been using about 18 of the Toshiba DT01ACA300 for over 2 years, in constant 24×7 use. They have been used for database dumps and loads, following by Bacula regression testing etc. They do not have a leisurely life.
I have not bought CPU cooling yet. This is intentional. I want to have all the parts and then measure to see what will fit.
When the time comes to buy more disk controllers, I will buy the M1015 which is an LSI card inside. They seem to be a staple for FreeNAS systems.
What’s next?
There will be photos of the assembly as I go along. This won’t be a quick build. It will take weeks to get everything collected and assembled. I plan to have superior cables, for both data and power.
If you have any suggestions for fan filters, data or power cables for SATA, please comment away.
Hmm… Since you bought PWM fans (4-pin) it’s going to be a bit trickier… Did you made a typo? 8 instead of 7? (3 on each side and two in the back?)
I’d recommend you to make your own 6/8-pin PCIe (VGA) to 4-Molex cables, since you don’t have that many SATA/4-pin Molex connectors to begin with and to save the clutter….
http://www.evga.com/products/images/gallery/220-G2-0850-XR_XL_5.jpg
So, 8-pin PCIe (VGA) –> 4-pin Molex –> 4-Pin Molex to 3-pin fan splitter (very limited selection due to PWM fans (4-pin)) –> Fan resistors –> Fans
3 * Your own cable (preferably one for each side of the case)
3 * http://www.amazon.com/Phobya-4-Pin-3-Pin-Splitter-Connection/dp/B007S3T4O0 – Splitter (preferably one for each side)
3 * http://www.amazon.com/gp/offer-listing/B0015DA0YM/ref=olp_tab_all?ie=UTF8&qid=1436425938&sr=1-3 – Fan resistor (very simple to do it yourself)
As for the HDDs I’d probably measure since you probably need extensions (additionally to splitters) on one side since the PSU will be mounted on one side.
I mentioned it earlier but this is most likely your best bet if you want to keep the noise down and still have good cooling performance given the space around the socket.
http://www.rakuten.com/prod/scythe-scmgd-1000-mugen-intel-lga-1150-max-glidestream-140mm-pwm/271094790.html?listingId=360781889
As for the SFF cables (M1015), LSI/Avago and Adaptec have good cables that are usually pretty cheap…
http://www.superbiiz.com/detail.php?p=CBLSFF10M
http://www.superbiiz.com/detail.php?name=CBLSFFCF05
(Make sure you get “forward” cables)
> Did you made a typo? 8 instead of 7?
There is no typo. I bought 7 fans.
What about using: http://www.amazon.com/StarTech-com-12-Inch-Power-Splitter-Cable/dp/B005G3010U
Or, I could use a fan controller.
That will take up a 5.5″ slot, and reduce the number of HDD I can use.
This might be useful: http://www.amazon.com/Phobya-4-Pin-Splitter-Power-Eight/dp/B00OD7MO6E
8 x 4-Pin PWM Fan Splitter PCB (Power Eight Fans from One Power Board!)
The M/B has 5 fan connections, all of which are PWM aware. They are rated at 2.5A. My fans are 0.36A max. I was planning to run the fans off there.
The BIOS has these setting:
Fan Speed Control Mode
The Fan Speed Control Mode sets the speed of the system fans to either
Full Speed (100% of PWM cycle), Heavy I/O or Standard speed (50%
of PWM cycle, temperature controlled). Select Heavy I/O mode when IO
devices are under heavy load.
I think I like this connector better: http://www.amazon.com/dp/B00IF6R4C8/
I could build. That would give me the length I want. Or I could buy: http://www.ebay.com/itm/4-Pin-Molex-to-6-Pin-PCI-Express-PCIE-Video-Card-Power-Converter-Adapter-Cable-/111684933316
Here are SATA power extension cables which might be useful:
* http://www.moddiy.com/products/Premium-Single-Braid-Sleeved-5%252din%252d1-SATA-Extension-Cable-%28Black%7B47%7DRed%29.html
* http://www.moddiy.com/products/Premium-Silver-Wire-SATA-Power-Cable-%2840CM%2C-1x-Molex-to-5x-SATA%29.html
* http://www.moddiy.com/products/EVGA-Premium-Single-Sleeved-Power-Supply-Modular-SATA-x-4-Cable-%286%252dPin%29.html
If you haven’t already ordered the parts, or can return them without expense, I would recommend going with the Supermicro X10SRH-CF as the LSI 3008 on it can be flashed to IT mode, thus enabling both JBOD and SATA Passthrough which are necessary for your zpool).
Does your reply indicate the X10SRA-F-O cannot do passthrough? I’m not sure if that’s your point, or if it’s that the X10SRH-CF has an LSI 3008.
No, the X10SRA-F-O does support SATA Passthrough through the C612 controller, but you mentioned buying the M1015 later, so I thought I’d recommend the X10SRH-CF since it comes with essentially the same hardware already attached to the motherboard.
I’m going to need at least one M1015 anyway since the goal is to fill this with HDD. The M/B won’t run them all.
Ah, I see. Yeah, even with the motherboard I suggested you’d be two ports short if you end up filling your entire case.
The case can hold 20 HDD. 28 if I want to start putting them into the 5.5″ slots.
Be very careful with LSI 3*** controllers, ppl have been reporting unreliable results with those in FreeBSD and on Linux (there’s a forum thread about it on forums.freebsd.org if you want to read more). You want to use AHCI if possible not passthrough and the IBM cards just works ™.
//Danne
PCI-E to molex:
* http://www.frozencpu.com/products/16472/cab-988/ModRight_Black-Out_Series_2x_4-Pin_to_8-Pin_PCI-E_Adapter_Cable_-_12.html?tl=g2c413s427#blank
Wrong way, you want a female connector to molex
I think you are saying the molex connector should be female.
Gah, tired…
http://img208.imageshack.us/img208/9451/6pinmolexpcieconnectors.jpg
You want a male PCIe connection “block” to a female molex
My plan was to plug this cable directly into my PSU but now I understand what you just said.
Useful information for anyone using M1015 on FreeBSD: http://roycebits.blogspot.com/2015/01/freebsd-lsi-sas9211-8i-hba-firmware.html