how to build a server (hardware requirements)?

[Naren]

Hello,
I have been analysing my NGS(illumina) microbiome data in a i5 4core 1TB Windows desktop using VM (virtual machine) biolinux package and have hit a snag due to limited virtual memory for alignment..

So we have decided to build a better machine suited to NGS analysis. I am not sure about the hardware requirements. Our budget is modest since I am trying this for the first time.

I would appreciate any help with this query.

Thank you
Naren

[Brian Bushnell]

1 TB is actually pretty big, but 4 cores is really small. What kind of budget do you have?

[Markiyan]

I would like to see a core i5 desktop supporting 1TB of RAM....

I think here a user has 4GB - 32GB of RAM and 1TB of HDD/SSD...

[Brian Bushnell]

Originally Posted by Markiyan

I would like to see a core i5 desktop supporting 1TB of RAM....

I think here a user has 4GB - 32GB of RAM and 1TB of HDD/SSD...

Ahh, good point! Also, Naren, it would be helpful if you could mention your sequencing platforms and how many gigabases of data you want to use in a typical assembly. Also, what kind of microbiome you're sequencing. And the actual numbers for physical RAM on your current machine versus hard-drive space; they're completely different for most purposes.

[Naren]

Thank you Markiyan and Brian,
Markiyan is right.. My desktop is 12GB RAM , 1TB hard disk and Quad core processor (i5).

Our budget is only betwwen 1000 to 1500 USD (I know it is kind of cheap, but we can afford only around this range for now)

The microbiome we use is soil microbiome.. i use SILVA data base (~10GB) as reference and my data is not very huge, ive got around 80000 sequences to align against the reference...

in the future we may decide to do more NGS analysis as well..

I appreciate your reply.

Thank you
Naren

[Richard Finney]

Virtual Memory is configurable in linux (but not on the fly, at initial setup).
Often there is only a little virtual memory configured as once the system is using virtual memory, it's usually struggling and it's better to upgrade.
If you're using a Virtual Machine on windows, you might want to see if
you can dual boot instead.

It looks like a 2TB SSD is in your budget. You can install linux on it
and (via your bios) boot from that device. Setup linux to have lots of Virtual Memory, perhaps most of the SSD disk. (You'll need a little bit of disk for the OS and /tmp space).
You'll need to make sure you can read the existing hard disk (formatted in NTFS for windows) by installing and configuring NTFS support.

[Brian Bushnell]

Just to clarify - is this shotgun sequencing or 16S amplicon sequencing?

[Naren]

Brian,
It is 16s amplicon sequencing

Thank you
Naren

[Brian Bushnell]

Originally Posted by Naren

Brian,
It is 16s amplicon sequencing

Thank you
Naren

Oh, perfect! Well, before you go out and buy a new computer, please try this:

1) Download BBMap
2) Run this command:
sendsketch.sh in=reads.fq out=results.txt mode=sequence silva size=1000000

This assumes you've already merged your reads; if not, you should run something like:

bbmerge.sh in1=read1.fq in2=read2.fq out=merged.fq outu=unmerged.fq

... and just use the merged reads.

This uses an alignment-free kmer-based 16S comparison method. I am eager to hear what kind of results you get... in my tests, it works really well. And it only takes a second for a single sequence. I'm not sure how long it takes for 80k ribosomal sequences so it would be great if you could tell me.

[rnablix]

I would recommend at least a 12-core XEON and 1.5 TB SSD. You can also get a high end GPU but not a lot of programs utilize it . Xeons have no integrated GPU so you may have to get a separate GPU anyway if you'll use your computer for other tasks. All this might set you back by around $3000.

[Biocomputronics]

I would suggest learning about cloud computing on a platform such as Amazon Web Services or the like. The days of needing to build hardware infrastructure are over. Just build up your own AMI (Amazon machine image), then spin it up on demand. You can provision whatever hardware configuration you need for any given task. Storing your data on Amazon S3 is probably convenient too for many reasons.

[dericklineker]

Hmm... What about AMD Ryzen 7 + 16 GB RAM + SSD?

[Ola]

Originally Posted by dericklineker

Hmm... What about AMD Ryzen 7 + 16 GB RAM + SSD?

That is what I am using (with 2x16G), excellent performance for the price.

[Brian Bushnell]

Originally Posted by dericklineker

Hmm... What about AMD Ryzen 7 + 16 GB RAM + SSD?

RAM is often the limiting factor in bioinformatics computing. I would not recommend buying a computer that you plan to use for bioinformatics with only 16 GB RAM unless it will be dedicated to some specific program that does not need much memory. SSDs are much less important, typically. If you can't afford more RAM and an SSD, I'd suggest dropping the SSD. Spinning disks are fine and have much more capacity anyway, per unit cost.

[GenoMax]

Unfortunately prices for DDR4 RAM have gone through the roof, so expect to pay top dollar. 2 x 16 GB Desktop sticks are ~ $359 in US (+ tax) as I write this.

[max6307]

Well, I also want to build server for don't know where I have to start. Can anyone help me from starting point what I need and also about cost?