Corona Lite Running Times?

[doxologist]

The manual suggests one day given enough CPUs. Anyone with experience here? What is the speed of your CPUs? How many do you use? How long does it take in your hands?

[westerman]

Unless I have missed something in the new release, each of the corona lite programs runs on a single processor. However like many bioinformatics programs the corona lite programs can be run in 'embarrassingly parallel' mode. I.e., break down your reference sequence by chromosome or other convenient segment and/or the SOLiD file into enough parts to use up your processors.

The matching part of the corona lite pipeline has 6 parts with the 1st and 3rd part being able to be split up. The other 4 parts are solely single processor but also are really just file copies and thus can be fairly fast.

As for overall time it depends, obviously, on the size of your SOLiD data set -- those 14-20 GB files take a while to toss around -- and your reference sequence. The time also go up in a non-linear fashion depending on how many mis-matches you wish to take into consideration.

A big consideration is having enough disk space, both temporary and permanent, to handle the files.

Since I usually work with partially assembled genomes (i.e., lots of contigs) or CDS or EST projects it is quite often the case that I split up the reference into 64 parts and use all 64 CPUs that I have at my disposal. The ultimate speed of the CPUs really doesn't matter that much. Obviously the faster the better. But I would concentrate more on disk speed and physical memory and exactly how many mismatches you want. 1 mismatch is trivial. 3 (the recommend) less so. 6 or more almost impossible on any sizable dataset.

And, yes, I would say 1-2 days of processing given enough CPUs. My recent work on the bee assembly 4 took about 36 hours to go through the matching steps. But I didn't break down the chromosomes nor SOLiD data set and so only used about 1/4 of my CPUs. There are other people on the machine and despite my hoggish nature I did want to play nicely (for once!) SNP calling added time to that process.

[doxologist]

thanks for the extensive reply.

with your experience, how long does a standard run alignment take?
I'm just estimating order of magnitude... hours? days? week?

Is it feasible to match on 1 CPU for a whole run? do I need to have access to servers/clusters with more CPUs?

[westerman]

It should be possible to match using 1 CPU given enough memory (4 GB). Given my experience I would expect running times of about 3 weeks for a non-paired mapping of a SOLiD data set to the reference bee genome. SNP calling would probably take an extra week. But I may be pessimistic.

In any case it will take time and you better hope that your computer stays up and running during the process. Last week I had two instances of the computer or file server crashing on me. They were rare instances that should not occur but irritating never-the-less.

[doxologist]

wow! three weeks! thanks for letting me know... we really need Bowtie to do colorspace soon... hopefully cutting it down to hours.

have you used MAQ for colorspace? Any ideas how long this takes on 1 CPU? Is it also weeks?

[Chipper]

You could try ZOOM!, it does CS alignments and is pretty fast. Have not compared it to Corona but at least it requires much less disk space.

[doxologist]

thanks. any ideas how long it take (order of magnitude) for one SOLID run 15-20G to the human genome (on 1 CPU)?

[Chipper]

No, it depends on how much memory you have, how long sequences, how many mismatrches etc. I am planning to do some tests soon so let me know what you think a good benchmark would be.

[doxologist]

Great. I look forward to the results.

[jlli]

For human genome and 50G reads, it tooks about 36 hours for mapping (1 cpu, 6G memory).

[doxologist]

Thanks for the response.

is this ZOOM or Corona?

[jlli]

corona-lite

[snetmcom]

for future reference,
the size of your reference is a HUGE factor here. from hours to weeks...

dataset and reference

[nilshomer]

You could also try a program I have authored caled BFAST: the Blat-like Fast Accurate Search Tool. You can find download instructions at:

http://genome.ucla.edu/bfast

Nils Homer

[BioWizard]

If your time is precious, try ISAS. Native colorspace and as far as we know its the fastest - if I'm wrong and there is a faster solution please enlighten me !


100 million 25mers on one computer in 30 minutes.
3G human reference, 2 mismatches.
Results identical to corona (just 100 times faster) and same format

See the ISAS thread for more info.