12-01-2013: Faster and much less memory-required LSC 1.alpha is released

[LSC]

12-01-2013: Faster and much less memory-required LSC 1.alpha is released

In the LSC 0.3.0 or 0.3.1, we optimized the setting of bowtie2 and BWA to get much more short read alignment, which improve the the accuracy of error correction a lot/ However, the increase of alignments also requires much more running time (on both alignment and the following error correction step) and memory usage. Therefore, a few users met difficulty of running LSC 0.3.0 or 0.3.1.

In LSC 1.alpha, we apply probabilistic algorithm ("SCD" option) to select ""enough" short read alignment for error correction. LSC 1.alpha does NOT sacrifice the error correction performace (sensitivity and specificity). Please see http://www.stanford.edu/~kinfai/LSC/...l.html#aligner Thus, we save running time and memory usage significantly. The running time is 30-50% of LSC 0.3.1. The peak memory usage decreases to ~10G regardless of the data size.

New features:

- Added probabilistic algorithm ("SCD" option) to pre-select SR alignments results based on LR-SR alignment coverage depth (Significant improvement in running time and memory usage)
- Removed requirement for loading SR dataset in memory to generate LR-SR mapping file (Significant improvement in memory usage)
- Added option "sort_max_mem" in run.cfg to control maximum memory used by unix sort command to avoid unexpected Mem crash

Miscellaneous changes:

- Fixed a bug in generating FASTQ file (it affected some of QualityValue computation results)

If you want to see the manual and tutorial of the old LSC (before 1.alpha), we keep the links of its the Old manual and Old tutorial in the right side bar.

Here is our latest manuscript of how to make use of LSC-corrected PacBio data to identify gene isoforms:

http://www.pnas.org/content/early/2013/11/25/1320101110.full.pdf+html?with-ds=yes

[MakoMori]

Hi LSC,

I'm trying to correct a 11M LR.fa file with 23G SR.fq file.
I used the latest version 1.alpha and set the config as part of following:
-------------
Nthread1 = 1
Nthread2 = 1
sort_max_mem = 1G
-------------

It worked pretty good during the whole process, except that when the "correct.py LR_SR.map.??_tmp" began, the memory increased to more than 40G.

Is that normal?
And could you tell me how to estimate the peak memory of different projects?

Thank you!