SRMA freezes after 1/10 run

[a11msp]

Hi everyone,

I'm trying out srma on bowtie and bwa aligned data and while I've
managed to successfully get round two initial problems, I'm stuck on
the third one:
Basically, srma works quite fast until it reaches ~3.5-4M records.
After that it just hangs for hours, reporting nothing - while the
resulting BAM file stays at ~1/10 of the input file size.
Is this normal? Should I expect it to return to life at some point or
I should just kill it? Can I change anything so it doesn't freeze?

Many thanks!

PS As for the two problems that I solved, here are the solutions:
- for bwa data use VALIDATION_STRINGENCY=SILENT
- to prevent bam record order errors, recompile from source using the
latest versions of picard and sam libraries.

PPS I've sent the same message to srma-help, but thought it wouldn't hurt to crosspost here while it awaits moderator approval... I also noticed that the output is 1/10 expected size, not 1/3 as I said there...

[a11msp]

Can I just add that I also compiled a C version and it terminates almost immediately with the following error:


************************************************************
In function "srma_fai_fetch":
Fatal Error: value out of range.
Variable/Value: val.len <= end.
***** Exiting due to errors *****
************************************************************

[nilshomer]

It should keep running and not freeze. Note that the C-version is still in alpha, so feel free to send me bug reports so I can improve it.

For now, try using some more memory ("java -Xmx2G -jar..." etc.), or split the input into XMb genomic regions using the "RANGE" option and then merge the results ("samtools merge"). If you have a test dataset I could download, I would be happy to debug.

[a11msp]

Hi Nils, thanks for your reply!
Turns out, it's hitting the wall on memory - unfortunately, without saying this explicitly.
I've just rerun it with -Xmx40G (sic!) and it takes ~45 mins to finish without any problem. Didn't try to titrate how much memory it minimally needs, but can confirm that it was using all of it

[nilshomer]

Hi Nils, thanks for your reply!
Turns out, it's hitting the wall on memory - unfortunately, without saying this explicitly.
I've just rerun it with -Xmx40G (sic!) and it takes ~45 mins to finish without any problem. Didn't try to titrate how much memory it minimally needs, but can confirm that it was using all of it

The implementation is greedy, so if you give it 40GB, it will use 40GB. For the human genome, I have found between 1.5-3GB is sufficient for 30-100x coverage.

[a11msp]

The implementation is greedy, so if you give it 40GB, it will use 40GB. For the human genome, I have found between 1.5-3GB is sufficient for 30-100x coverage.

Yes, that's what I thought - so I don't assume this is a minimally possible amount. It did freeze with 4Gb, however, so I wanted to stay on the safe side. Note that my dataset was 5-10x coverage of a part of a genome, so containing much fewer reads - and yet 4Gb wasn't enough on my system (assuming it froze because it didn't have enough memory).

[eigenbro]

Hi,
I have a kind of similar problem using SRMA. When I run the programm it only works until it has reached the given memory limit. I tried that with different memory sizes (from 4GB to 40GB) and it still doesn't finish. The number of processed records until it freezes depends on the amount of memory. As parameters I only use VALIDATION_STRINGENCY=SILENT, everything else is set to default.
Is there something I didn't consider when running SRMA?
I'd be glad if someone could help.

[eigenbro]

It turns out if the program freezes for some time I get an out of memory-error as you can see below. I tried using a range file and changed the number of records in memory as well.

[Fri Nov 26 08:25:32 GMT 2010] srma.SRMA INPUT=[/project/solexawork/user/tumor/RecaliBam/090924EAS453_s_7.rc.bam] OUTPUT=[/project/solexawork/user/tumor/RecaliBam/090924EAS453_s_7.SRMA.rc.bam] REFERENCE=/project/solexawork/user/tumor/references/hg19oh.fa RANGES=hg19_chromranges_100kb.txt VALIDATION_STRINGENCY=SILENT MAX_RECORDS_IN_RAM=100000 OFFSET=20 MIN_MAPQ=0 MINIMUM_ALLELE_PROBABILITY=0.1 MINIMUM_ALLELE_COVERAGE=3 MAXIMUM_TOTAL_COVERAGE=100 CORRECT_BASES=false USE_SEQUENCE_QUALITIES=true QUIET_STDERR=false MAX_HEAP_SIZE=8192 MAX_QUEUE_SIZE=65536 NUM_THREADS=1 TMP_DIR=/tmp/eigenbro VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=5 CREATE_INDEX=false CREATE_MD5_FILE=false
Allele coverage cutoffs:
coverage: 1 minimum allele coverage: 0
coverage: 2 minimum allele coverage: 0
coverage: 3 minimum allele coverage: 0
coverage: 4 minimum allele coverage: 1
coverage: 5 minimum allele coverage: 1
coverage: 6 minimum allele coverage: 1
coverage: 7 minimum allele coverage: 2
coverage: 8 minimum allele coverage: 2
coverage: 9 minimum allele coverage: 3
coverage: >9 minimum allele coverage: 3
Records processsed: 851967 (last chr1:181855253-181855328)Exception in thread "Thread-26" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2734)
at java.util.ArrayList.ensureCapacity(ArrayList.java:167)
at java.util.ArrayList.add(ArrayList.java:370)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.AbstractList.add(AbstractList.java:91)
at srma.Graph.addNode(Graph.java:153)
at srma.Graph.addSAMRecord(Graph.java:112)
at srma.SRMA$GraphThread.run(SRMA.java:674)
java.util.ConcurrentModificationException
at java.util.SubList.checkForComodification(AbstractList.java:752)
at java.util.SubList.add(AbstractList.java:632)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.SubList.add(AbstractList.java:633)
at java.util.AbstractList.add(AbstractList.java:91)
at srma.Graph.addNode(Graph.java:153)
at srma.Graph.addSAMRecord(Graph.java:112)
at srma.SRMA$GraphThread.run(SRMA.java:674)