View Single Post
Old 12-16-2016, 05:43 PM   #15
chiayi
Member
 
Location: New York

Join Date: Dec 2016
Posts: 22
Default

Hello Brian,

I started to use clumpify and the size was on average reduced ~25% for NextSeq Arabidopsis data. Thanks for the development!

In a recent run for HiSeq maize data, I got an error for some (but not all) of the files. At first the run would stuck at fetching and eventually fail due to not enough memory (set 16G), despite the memory estimate was ~ 2G.

HTML Code:
Clumpify version 36.71
Memory Estimate:        2685 MB
Memory Available:       12836 MB
Set groups to 1
Executing clump.KmerSort [in=input.fastq.bz2, out=clumped.fastq.gz, groups=1, ecco=false, rename=false, shortname=f, unpair=false, repair=false, namesort=false, ow=true, -Xmx16g, reorder=t]

Making comparator.
Made a comparator with k=31, seed=1, border=1, hashes=4
Starting cris 0.
Fetching reads.
Making fetch threads.
Starting threads.
Waiting for threads.
=>> job killed: mem job total 17312912 kb exceeded limit 16777216 kb
When I increased to 48G, the run was killed at making clumps and didn't have a specific reason,

HTML Code:
Starting threads.
Waiting for threads.
Fetch time:     321.985 seconds.
Closing input stream.
Combining thread output.
Combine time:   0.108 seconds.
Sorting.
Sort time:  33.708 seconds.
Making clumps.
/home/cc5544/bin/clumpify.sh: line 180: 45220 Killed
Do you know what may be the cause of this situation? Thank you.
chiayi is offline   Reply With Quote