SEQanswers

SEQanswers (http://seqanswers.com/forums/index.php)
-   Bioinformatics (http://seqanswers.com/forums/forumdisplay.php?f=18)
-   -   Introducing BBMap, a new short-read aligner for DNA and RNA (http://seqanswers.com/forums/showthread.php?t=40162)

Brian Bushnell 01-22-2014 02:57 PM

BBMap - a new short read aligner, to be released soon.
 
BBMap will be publicly released soon, pending confirmation with LBL's legal department.

In the meantime feel free to look at these graphs of its performance:

https://drive.google.com/file/d/0B3l...it?usp=sharing

Note that this is a 50MB powerpoint file. It contains graphs of relative performance of BBMap and other short read aligners (bwa, bowtie2, gsnap, smalt) mapping synthetic data.

EDIT:
This thread is now closed; please use this one to post questions.

kopi-o 01-23-2014 12:44 AM

Looks very impressive! Can it beat STAR (speed and accuracy wise) for RNA-seq though? (RNA-seq is listed as one of the use cases towards the end)

Brian Bushnell 01-23-2014 09:24 AM

I have compared it to tophat, which it greatly outperforms in speed and has higher sensitivity on real RNA-seq data. I have not yet compared it to STAR - I tried to but was unable to get STAR to run without core-dumping so I gave up. I may have compiled it wrong; I'll try again eventually.

However, I don't have a really good tool for generating and evaluating synthetic RNA-seq data, so it's harder to quantify. The closest I can get is to generate synthetic DNA reads with very large deletions, which is not quite the same thing since RNA-seq data has other strange artifacts and the introns are not distributed randomly.

dpryan 01-23-2014 10:12 AM

It'd be great if you could get in touch with the authors of this paper and just use their test datasets. That would allow comparisons against most of the popular aligners out there.

GenoMax 01-23-2014 10:28 AM

Quote:

Originally Posted by dpryan (Post 130489)
It'd be great if you could get in touch with the authors of this paper and just use their test datasets. That would allow comparisons against most of the popular aligners out there.

Data is available here:

http://www.ebi.ac.uk/arrayexpress/ex...s/E-MTAB-1728/
http://www.ebi.ac.uk/arrayexpress/ex...-1728/samples/

Brian Bushnell 01-23-2014 10:41 AM

Quote:

Originally Posted by dpryan (Post 130489)
It'd be great if you could get in touch with the authors of this paper and just use their test datasets. That would allow comparisons against most of the popular aligners out there.

Thanks for the suggestion; I'll look into that!

dietmar13 01-23-2014 11:06 AM

Rum
 
why is RUM always neglected by comparing RNA-seq mappers?
In my hands RUM outperforms other pipelines, e.g. tophat, in sensitivity, especially for spliced reads...

RUM: RNA Seq Unified Mapper
https://github.com/itmat/rum/wiki

RUM is rather slow, but using multithreaded servers allows mapping in tolerable time (compared to sample and library generation and data interpretation)

dietmar

Corydoras 06-17-2014 03:17 AM

Hi Brian,

I got a file with cleaned sequence data and I want to assemble this de-novo using velvet. Due to the nature of the sequencing and the library protocol, my kmer coverage is quite variable and I wanted to use BBnorm to normalize the coverage a bit to aid the assembly. Am I correct that BBnorm is the right thing to use for this?

Anyway, currently trying to give it a go and I got this error message:

bbmap$ sh bbnorm.sh in=Fowleri_combined.fastq out=normFowleri.fastq target=15
bbnorm.sh: 104: bbnorm.sh: Bad substitution
bbnorm.sh: 112: bbnorm.sh: [[: not found
bbnorm.sh: 112: bbnorm.sh: [[: not found
bbnorm.sh: 118: bbnorm.sh: source: not found
bbnorm.sh: 119: bbnorm.sh: parseXmx: not found
bbnorm.sh: 120: bbnorm.sh: [[: not found
bbnorm.sh: 123: bbnorm.sh: freeRam: not found
java -ea -Xmxm -cp /home/martin/Downloads/bbmap/current/ jgi.KmerNormalize bits=32 in=Fowleri_combined.fastq
Invalid maximum heap size: -Xmxm
Could not create the Java virtual machine.

Any ideas?

Many thanks,

Sarah

Brian Bushnell 06-17-2014 10:14 AM

Sarah,

Yes, BBNorm is the correct tool.

I'm not sure, but I suspect that your shell is not bash. You could retry the command with "bash" instead of "sh", which may work. But the easier thing is just to skip the shellscript and invoke java manually:

java -ea -Xmx14g -cp /home/martin/Downloads/bbmap/current/ jgi.KmerNormalize bits=32 in=Fowleri_combined.fastq out=normFowleri.fastq target=15

That command would work if you had 16g of RAM. Just set the -Xmx parameter (highlighted in purple) to about 85% of however much RAM is on the machine. If you don't know, you should be able to find out like this on a Linux system:

cat /proc/meminfo

...then look at the first line, "MemTotal".

However, 15x is a fairly low target depth. For velvet I would suggest at least 30x for an optimal assembly, unless you just don't have enough data.

-Brian

Corydoras 06-19-2014 08:17 AM

Hi Brian,

That worked like a charm, thank you! The normalization also greatly improved the assemblies and the kmer-coverage distribution looks much nicer. I was just wondering: by default, bbnorm will use a kmer of 31. But for my assembly I am using 41. The assembly works fine, but is it advisable to normalize the coverage using a kmer of 41?

Thanks,

Sarah

Brian Bushnell 06-19-2014 10:48 AM

Sarah,

It might be better to normalize using a kmer length of 41, but BBNorm only supports a maximum of 31 :) In practice, it should make very little difference, though. Using long kmers is important for assembly, as it helps span short repeats that would otherwise cause contigs to terminate. But normalization is much less sensitive to that issue, and very long kmers can cause problems in the presence of errors. With k=31, a 100bp read with 1 error could yield 31 kmers with a depth of 1, out of a total of 70 kmers - in that case, the median depth would not be impacted. With k=63, there could be 63 of the 70 total kmers spanning the error, thus having a depth of 1, so the median depth of the read would look like 1 instead of its correct value. And BBNorm normalizes based on the median kmer depth of a read.

It's a lot more computationally efficient to use a max kmer length of 31, so that's how I designed it. I've tried shorter kmers down to about k=25 and not noticed an appreciable difference in normalization or error correction.

As for your prior (deleted) post, sorry for not responding - I think the problem was that you were running Java 6 instead of Java 7. Most of the programs in BBTools work fine in Java 6 but it looks like BBNorm requires Java 7 (or higher).

Corydoras 06-20-2014 01:06 AM

Hi Brian,

Thanks so much for that explanation :). I thought I wouldn't be able to go past 31 but it is best to double check.

Sorry as well for just deleting my post (and bombarding you with simple questions, new to the world of NGS!), I played around with updating the Java on our Linux machine and that did the trick :).

Thanks again for your help! And the fantastic and easy to use script!!

Sarah

muol 06-23-2014 04:28 PM

Hi Brian,

Is there an option to set read quality encoding in bbnorm? I had to set qin=33 in bbduk for some Illumina 1.9 paired end libraries, but this option doesn't seem to exist in bbnorm (used BBMap v. 32.32 for Java 7).

Thanks
Olaf

Brian Bushnell 06-23-2014 04:44 PM

Olaf,

It's there, I just forgot to document it; sorry! I'll add that to the shellscript in the next release. I think that all of the programs in the package that read fastq input allow the "qin" flag.

-Brian

muol 06-23-2014 04:54 PM

Indeed, just tried it and it works well with bbnorm.

Thanks
Olaf

muol 06-23-2014 06:07 PM

Brian,

I ran into a smaller issue with bbnorm. When trying to input and output separate files for a PE library like this:

Code:

bbnorm.sh in1=R1.fastq.gz in2=R2.fastq.gz out1=R1.bbnorm.fastq.gz out2=R2.bbnorm.fastq.gz prefilter=t tossbadreads=t ecc=t fixspikes=t qin=33 -Xmx72g target=40
I receive this error during pass 2:

Code:

Exception in thread "main" java.lang.AssertionError: Please do not set 'interleaved=true' with dual input files.
        at stream.ConcurrentGenericReadInputStream.<init>(ConcurrentGenericReadInputStream.java:132)
        at stream.ConcurrentGenericReadInputStream.getReadInputStream(ConcurrentGenericReadInputStream.java:661)
        at stream.ConcurrentGenericReadInputStream.getReadInputStream(ConcurrentGenericReadInputStream.java:641)
        at kmer.KmerCount7MTA.countFastq(KmerCount7MTA.java:355)
        at kmer.KmerCount7MTA.makeKca(KmerCount7MTA.java:222)
        at jgi.KmerNormalize.runPass(KmerNormalize.java:1006)
        at jgi.KmerNormalize.main(KmerNormalize.java:736)

Setting interleaved=false doesn't change that. Outputting to a single, interleaved file (in1=xxx in2=xxx out=xxx) on the other hand works fine. Any ideas?

Olaf

Brian Bushnell 06-23-2014 07:02 PM

Olaf,

Currently, BBNorm uses single interleaved files for temporary storage when using multiple passes. And I have not implemented any way to specify dual files in intermediate stages, since everyone at JGI uses interleaved files for everything.

You have two options.
1) You could set "passes=1", which is faster, but I don't recommend it because it doesn't give as good results as 2-pass normalization.
or
2) You could specify only a single output file, which will get interleaved reads:

bbnorm.sh in1=R1.fastq.gz in2=R2.fastq.gz out=R12.bbnorm.fastq.gz prefilter=t tossbadreads=t ecc=t fixspikes=t qin=33 -Xmx72g target=40

...Then, if you need to, de-interleave it afterward:

reformat.sh in=R12.bbnorm.fastq.gz out1=R1.bbnorm.fastq.gz out2=R2.bbnorm.fastq.gz

Sorry for the inconvenience! I'll try to fix that by the next release, though unlike documenting the "qin" flag, this will take more work so no guarantees. Thanks for bringing it to my attention. FYI, the flag "interleaved" has no effect on output, only input.

-Brian

muol 06-23-2014 08:14 PM

Thanks for the info Brian, it wasn't a big issue.

Olaf

Brian Bushnell 06-27-2014 04:26 PM

Olaf,

This has been fixed in the latest release, 33.04

muol 06-27-2014 05:11 PM

Excellent, just did a test run. This is very useful software!

Olaf


All times are GMT -8. The time now is 10:44 AM.

Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.