Thoughts on new workstation/server using Haswell CPUs

[atcghelix]

Does anyone have opinions on the suitability of the new 8-core Haswell 5960X processor from Intel for a bioinformatics workstation/server? We have about $4,000 to spend, and we need to order everything early this week. I'm considering the setup below, but my experience is limited to building single-processor PCs. I've never built a dual-processor Xeon system, so I'm not really sure how the below setup would stack up against a Xeon setup for the same cost (if it's even possible to build a similar Xeon setup for under $4,000).

The system will mainly be used for things like read mapping, reduced-complexity de novo assemblies (not mega RAM-intensive, see http://ibest.github.io/ARC/), and population genomic analyses (often using ANGSD).


Intel Core i7-5960X Haswell 8-Core 3.0GHz LGA 2011 140W Desktop Processor BX80648I75960X - $1050
EVGA X99 FTW (150-HE-E997-KR) LGA 2011-3 Intel X99 SATA 6Gb/s USB 3.0 Extended ATX Intel Motherboard - $310
2X Crucial 32GB (4 x 8GB) 288-Pin DDR4 SDRAM DDR4 2133 (PC4-17000) Desktop Memory Model CT4K8G4DFD8213 - $818
SAMSUNG 840 EVO MZ-7TE1T0BW 2.5" 1TB SATA III MLC (SSD) - $470
4X Seagate Desktop HDD.15 ST4000DM000 4TB 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive Bare Drive - $600
Cooler Master HAF 932 Advanced Blue Edition - High Air Flow Full Tower Computer Case with USB 3.0 and All-Black Interior - $140
CORSAIR HX Series HX850 850W ATX12V 2.3 / EPS12V 2.91 SLI CrossFire 80 PLUS GOLD Certified Modular Active PFC Power Supply New 4th Gen CPU Certified Haswell Ready - $150
ASUS DRW-24B1ST/BLK/B/AS Black SATA 24X DVD Burner - $20
Cooler Master Hyper 212 EVO CPU cooler- $35

==$3,941 including tax and shipping==

Please let me know if you have any thoughts. Thanks so much!

[GenoMax]

Is there a reason to get the extreme edition processor (are you going to overclock it)? Why not get the 6-core i7 5820K that is a step down and use the money to bump the RAM up to 64 GB (you will need another ~$200 after the price difference)? I would rather have enough head room on the RAM than the fastest CPU available.

[atcghelix]

Sorry it's not too clear above--that setup is for two 32GB RAM quad-channel sets for 64GB total. The only reason for the extreme edition is it's the only 8-core processor in that lineup.

We currently have a machine with a 3930k 6-core and 64GB of RAM, and it seems like it's usually enough for what we're doing. I think we're usually CPU and I/O bottlenecked in the stuff we do.

[lh3]

I would push RAM to 128GB at least. You are going to use this machine for several years. The sequencing throughput will be much higher. You may also work on new applications or data types, or use better but more memory hungry software which might assume 128GB is the norm by then. It is safer to bet on more RAM than on faster CPU.

[Brian Bushnell]

For aggregate multithreaded throughput in a 1S system, you probably can't beat the 5960X.

However, the 4930K (6 cores X 3.4GHz = 20.4) or, as GenoMax mentioned, 5820K (6 x 3.3 = 19.8) would offer only slightly lower performance than the 5960X (8 x 3.0 = 24) at half the price or less. Also, the 4930K would let you use DDR3, which is currently cheaper than DDR4 and does not seem to hinder performance in tests. The useful Xeons on Newegg, such as E5-2690 v2 (10 cores x 3.0GHz = 30), are probably out of your price range. So, the 3 above are probably the best choices.

Still, for single-socket Xeon you could consider this (6 x 3.7) and for dual-socket you could consider this (2 x 6 x 2.6). Both would allow you to use DDR3.

While mapping scales near-perfectly with core count, assembly typically doesn't, and many other common bioinformatics tasks don't, either. So I would lean toward 6 faster cores instead of 8 slower cores, and save the extra money. Also, I believe that Haswell's extra performance over prior generations largely comes from new floating point / vector instructions that won't help most branchy integer-based bioinformatics programs.

And I agree that 128GB RAM would probably be wise, especially if you could fit it in the budget by buying a cheaper CPU.

[woodydon]

Can't be more admiring...

[atcghelix]

Thanks everyone--I really appreciate the feedback. If it really makes more sense to push things to at least 128GB then I agree we should probably do that even if it's going to cost us some more. If I'm not mistaken, that pushes us squarely into the Xeon server motherboards for the extra RAM slots and availability of 16gb ECC DIMMs. I think I could put together a dual E5-2650 v2 (2cpu * 8core * 2.6GHz) system for around $6k, which sounds like it'd be significantly more powerful and useful than a 5960X system.

So at the moment I'm thinking of something along these lines:

2 Intel Xeon E5-2650 v2 Ivy Bridge-EP 2.6GHz LGA 2011 95W Server Processors $2,339.98
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
ASRock EP2C602-4L/D16 SSI EEB Server Motherboard Dual LGA 2011 Intel C602 DDR3 $299.99
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
2 sets of Kingston 64GB (4 x 16GB) 240-Pin DDR3 SDRAM DDR3 1600 ECC Registered Server Memory DR x4 Model KVR16R11D4K4/64 $1,387.98
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
SAMSUNG 840 EVO MZ-7TE1T0BW 2.5" 1TB SATA III MLC Internal Solid State Drive (SSD) $499.99
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
Phanteks “ Enthoo Pro Series “ PH-ES614P_BK Black Steel / Plastic ATX Full Tower Computer Case $99.99
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
EVGA 02G-P4-3753-KR G-SYNC Support GeForce GTX 750 Ti Superclocked 2GB 128-Bit GDDR5 PCI Express 3.0 Video Card $139.99
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
CORSAIR HX1000i (CP-9020074) 1000W ATX12V / EPS12V 80 PLUS PLATINUM Certified Full Modular Power Supply $229.99
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
4 Seagate Desktop HDD.15 ST4000DM000 4TB 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drives Bare Drive $599.96
http://www.newegg.com/Product/Product.aspx?Item=N82E168...
ASUS DRW-24B1ST/BLK/B/AS Black SATA 24X DVD Burner - Bulk - OEM $19.99
http://www.newegg.com/Product/Product.aspx?Item=N82E168...

Total component cost: $5,617.86
Tax: $505.61
Shipping: $28.57

Total cost: $6152.04

[atcghelix]

Alternatively--a dual E5-2630 v2 system would have 12 cores (2 * 6 * 2.6), for a 20% reduction in cost (down to about $5k) and a 25% reduction in power. Hmmm...

[dfhdfh]

Don't forget coolers. You really don't want your machine to overheat.

[GenoMax]

Alternatively--a dual E5-2630 v2 system would have 12 cores (2 * 6 * 2.6), for a 20% reduction in cost (down to about $5k) and a 25% reduction in power. Hmmm...

You are not going to notice an hour or two it would take longer with the slightly slower CPU's when a job runs overnight. That said, if this opportunity is not going to come around for another 4 years, and you do have $6K available, then go for the faster CPU's.

As pointed out by dfhdfh do not forget a good CPU cooling solution (if you can afford a liquid cooled system, all the better). I assume you are not going to co-locate this in machine in a "real" server room. If that is an option then go in for a 1U/2U rack mount chassis.

[atcghelix]

Ah yes, forgot to add those. I was thinking 2 Hyper 212 Evos for cooling.

And no, I don't think we have space in a server room available for this, but I will check to make sure. We'll have it attached to a UPS in a building that is always pretty cool--I think that's the best we can likely do.

Thanks again for your input.

[GenoMax]

Add in a few big/efficient/silent case fans too. More air you move in/out of the box the better.

[Wallysb01]

I think you’ve absolutely made the right choice going with the 2x2650s. A couple of suggestions though:

1) Don’t get seagate drives. I just have 2 out of 5 of them start accumulating bad sectors. After a bit of research it seems seagate is a little worse than Wester Digital and HGST in overall failure rate.

2) Personally, I wouldn’t build my own system. I’ve built PCs before, but as a $6K workstation you need to get your work done on, its is a different animal. I personally wouldn’t like to slip a little and bend some pins on a $1200 CPU. Also, having single point of contact for warranty issues can be nice. If you buy things from newegg or other retailers, you may be left waiting for replacements longer than if you bought the whole machine from a single company, sometimes you may have to go back to the original OEM of the component itself and not the 3rd party retailer even. And as people are bringing up, you have many choices with cooling and may have to cycle through some options to get it as silent as possible. The retailer likely already has this worked out. Plus, you’ll get support help if components start failing and its not immediately obvious what it is. And finally, the mark up for customized workstations isn’t huge.

For example: http://www.thinkmate.com/system/hpx-xs4-2460

If you configure that with 2x2650s, the 256 Samsung 840 Pro, 128GB of RAM it comes to ~$5700. Then you buy the drives for another $600 as you would otherwise. So that $6300 without tax. To me, an extra 800-1000 bucks would be worth the support and lack of building it yourself hassle for a work machine. If you have to, drop to the 2640s. Realistically, you probably won’t notice much difference.

[Wallysb01]

One other quick suggestion, Supermicro motherboards and cases are really nice. That case you have picked out looks pretty slick, but it will probably get cramped with a duel LGA 2011 motherboard and the required heat sinks, etc. Also, that ASUS board has lots of DOA reviews. That thinkmate will come with super micro case/board, which makes up much of the price difference from what you have there. If you add a couple hundred extra for each, plus $50-100 for each heat sink, we’re now talking more like 300-400 dollars extra to give someone else the headache of putting it all together.

For example, I have this case and am really happy with it: http://www.newegg.com/Product/Produc...82E16811152104

[GenoMax]

Since Wallysb01 brought it up ...

@atcghelix: You may want to check out contract pricing on workstations from the favorite vendor at your institution (academic/commercial no matter which). Most of the times the prices will include 3 year comprehensive warranty so please keep that in mind when you do price comparisons. For a "work" server, not having to deal with 7 different vendors on warranty issues is totally worth one's time, not considering the downtime you may encounter as you wait for a replacement part to ship.