SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
short read aligner with 3 mismatch and one gap allowed NicoBxl Bioinformatics 2 11-09-2011 10:26 AM
The best short read aligner Deutsche Bioinformatics 4 04-14-2011 07:12 PM
Short Read Micro re-Aligner Paper nilshomer Literature Watch 0 10-29-2010 09:59 AM
New Short Read Aligner sparks Bioinformatics 48 08-26-2009 08:01 AM
Very Short Read aligner Rupinder Bioinformatics 1 06-02-2009 07:10 PM

Closed Thread
 
Thread Tools
Old 01-22-2014, 01:57 PM   #1
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default BBMap - a new short read aligner, to be released soon.

BBMap will be publicly released soon, pending confirmation with LBL's legal department.

In the meantime feel free to look at these graphs of its performance:

https://drive.google.com/file/d/0B3l...it?usp=sharing

Note that this is a 50MB powerpoint file. It contains graphs of relative performance of BBMap and other short read aligners (bwa, bowtie2, gsnap, smalt) mapping synthetic data.

EDIT:
This thread is now closed; please use this one to post questions.

Last edited by Brian Bushnell; 11-10-2014 at 11:09 AM.
Brian Bushnell is offline  
Old 01-22-2014, 11:44 PM   #2
kopi-o
Senior Member
 
Location: Stockholm, Sweden

Join Date: Feb 2008
Posts: 319
Default

Looks very impressive! Can it beat STAR (speed and accuracy wise) for RNA-seq though? (RNA-seq is listed as one of the use cases towards the end)
kopi-o is offline  
Old 01-23-2014, 08:24 AM   #3
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default

I have compared it to tophat, which it greatly outperforms in speed and has higher sensitivity on real RNA-seq data. I have not yet compared it to STAR - I tried to but was unable to get STAR to run without core-dumping so I gave up. I may have compiled it wrong; I'll try again eventually.

However, I don't have a really good tool for generating and evaluating synthetic RNA-seq data, so it's harder to quantify. The closest I can get is to generate synthetic DNA reads with very large deletions, which is not quite the same thing since RNA-seq data has other strange artifacts and the introns are not distributed randomly.
Brian Bushnell is offline  
Old 01-23-2014, 09:12 AM   #4
dpryan
Devon Ryan
 
Location: Freiburg, Germany

Join Date: Jul 2011
Posts: 3,475
Default

It'd be great if you could get in touch with the authors of this paper and just use their test datasets. That would allow comparisons against most of the popular aligners out there.
dpryan is offline  
Old 01-23-2014, 09:28 AM   #5
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 6,494
Default

Quote:
Originally Posted by dpryan View Post
It'd be great if you could get in touch with the authors of this paper and just use their test datasets. That would allow comparisons against most of the popular aligners out there.
Data is available here:

http://www.ebi.ac.uk/arrayexpress/ex...s/E-MTAB-1728/
http://www.ebi.ac.uk/arrayexpress/ex...-1728/samples/
GenoMax is offline  
Old 01-23-2014, 09:41 AM   #6
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default

Quote:
Originally Posted by dpryan View Post
It'd be great if you could get in touch with the authors of this paper and just use their test datasets. That would allow comparisons against most of the popular aligners out there.
Thanks for the suggestion; I'll look into that!
Brian Bushnell is offline  
Old 01-23-2014, 10:06 AM   #7
dietmar13
Senior Member
 
Location: Vienna

Join Date: Mar 2010
Posts: 107
Default Rum

why is RUM always neglected by comparing RNA-seq mappers?
In my hands RUM outperforms other pipelines, e.g. tophat, in sensitivity, especially for spliced reads...

RUM: RNA Seq Unified Mapper
https://github.com/itmat/rum/wiki

RUM is rather slow, but using multithreaded servers allows mapping in tolerable time (compared to sample and library generation and data interpretation)

dietmar
dietmar13 is offline  
Old 06-17-2014, 02:17 AM   #8
Corydoras
Member
 
Location: Norwich

Join Date: Jan 2014
Posts: 20
Default

Hi Brian,

I got a file with cleaned sequence data and I want to assemble this de-novo using velvet. Due to the nature of the sequencing and the library protocol, my kmer coverage is quite variable and I wanted to use BBnorm to normalize the coverage a bit to aid the assembly. Am I correct that BBnorm is the right thing to use for this?

Anyway, currently trying to give it a go and I got this error message:

bbmap$ sh bbnorm.sh in=Fowleri_combined.fastq out=normFowleri.fastq target=15
bbnorm.sh: 104: bbnorm.sh: Bad substitution
bbnorm.sh: 112: bbnorm.sh: [[: not found
bbnorm.sh: 112: bbnorm.sh: [[: not found
bbnorm.sh: 118: bbnorm.sh: source: not found
bbnorm.sh: 119: bbnorm.sh: parseXmx: not found
bbnorm.sh: 120: bbnorm.sh: [[: not found
bbnorm.sh: 123: bbnorm.sh: freeRam: not found
java -ea -Xmxm -cp /home/martin/Downloads/bbmap/current/ jgi.KmerNormalize bits=32 in=Fowleri_combined.fastq
Invalid maximum heap size: -Xmxm
Could not create the Java virtual machine.

Any ideas?

Many thanks,

Sarah
Corydoras is offline  
Old 06-17-2014, 09:14 AM   #9
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default

Sarah,

Yes, BBNorm is the correct tool.

I'm not sure, but I suspect that your shell is not bash. You could retry the command with "bash" instead of "sh", which may work. But the easier thing is just to skip the shellscript and invoke java manually:

java -ea -Xmx14g -cp /home/martin/Downloads/bbmap/current/ jgi.KmerNormalize bits=32 in=Fowleri_combined.fastq out=normFowleri.fastq target=15

That command would work if you had 16g of RAM. Just set the -Xmx parameter (highlighted in purple) to about 85% of however much RAM is on the machine. If you don't know, you should be able to find out like this on a Linux system:

cat /proc/meminfo

...then look at the first line, "MemTotal".

However, 15x is a fairly low target depth. For velvet I would suggest at least 30x for an optimal assembly, unless you just don't have enough data.

-Brian
Brian Bushnell is offline  
Old 06-19-2014, 07:17 AM   #10
Corydoras
Member
 
Location: Norwich

Join Date: Jan 2014
Posts: 20
Default

Hi Brian,

That worked like a charm, thank you! The normalization also greatly improved the assemblies and the kmer-coverage distribution looks much nicer. I was just wondering: by default, bbnorm will use a kmer of 31. But for my assembly I am using 41. The assembly works fine, but is it advisable to normalize the coverage using a kmer of 41?

Thanks,

Sarah
Corydoras is offline  
Old 06-19-2014, 09:48 AM   #11
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default

Sarah,

It might be better to normalize using a kmer length of 41, but BBNorm only supports a maximum of 31 In practice, it should make very little difference, though. Using long kmers is important for assembly, as it helps span short repeats that would otherwise cause contigs to terminate. But normalization is much less sensitive to that issue, and very long kmers can cause problems in the presence of errors. With k=31, a 100bp read with 1 error could yield 31 kmers with a depth of 1, out of a total of 70 kmers - in that case, the median depth would not be impacted. With k=63, there could be 63 of the 70 total kmers spanning the error, thus having a depth of 1, so the median depth of the read would look like 1 instead of its correct value. And BBNorm normalizes based on the median kmer depth of a read.

It's a lot more computationally efficient to use a max kmer length of 31, so that's how I designed it. I've tried shorter kmers down to about k=25 and not noticed an appreciable difference in normalization or error correction.

As for your prior (deleted) post, sorry for not responding - I think the problem was that you were running Java 6 instead of Java 7. Most of the programs in BBTools work fine in Java 6 but it looks like BBNorm requires Java 7 (or higher).
Brian Bushnell is offline  
Old 06-20-2014, 12:06 AM   #12
Corydoras
Member
 
Location: Norwich

Join Date: Jan 2014
Posts: 20
Default

Hi Brian,

Thanks so much for that explanation . I thought I wouldn't be able to go past 31 but it is best to double check.

Sorry as well for just deleting my post (and bombarding you with simple questions, new to the world of NGS!), I played around with updating the Java on our Linux machine and that did the trick .

Thanks again for your help! And the fantastic and easy to use script!!

Sarah
Corydoras is offline  
Old 06-23-2014, 03:28 PM   #13
muol
Member
 
Location: USA

Join Date: Jun 2012
Posts: 10
Default

Hi Brian,

Is there an option to set read quality encoding in bbnorm? I had to set qin=33 in bbduk for some Illumina 1.9 paired end libraries, but this option doesn't seem to exist in bbnorm (used BBMap v. 32.32 for Java 7).

Thanks
Olaf
muol is offline  
Old 06-23-2014, 03:44 PM   #14
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default

Olaf,

It's there, I just forgot to document it; sorry! I'll add that to the shellscript in the next release. I think that all of the programs in the package that read fastq input allow the "qin" flag.

-Brian
Brian Bushnell is offline  
Old 06-23-2014, 03:54 PM   #15
muol
Member
 
Location: USA

Join Date: Jun 2012
Posts: 10
Default

Indeed, just tried it and it works well with bbnorm.

Thanks
Olaf
muol is offline  
Old 06-23-2014, 05:07 PM   #16
muol
Member
 
Location: USA

Join Date: Jun 2012
Posts: 10
Default

Brian,

I ran into a smaller issue with bbnorm. When trying to input and output separate files for a PE library like this:

Code:
bbnorm.sh in1=R1.fastq.gz in2=R2.fastq.gz out1=R1.bbnorm.fastq.gz out2=R2.bbnorm.fastq.gz prefilter=t tossbadreads=t ecc=t fixspikes=t qin=33 -Xmx72g target=40
I receive this error during pass 2:

Code:
Exception in thread "main" java.lang.AssertionError: Please do not set 'interleaved=true' with dual input files.
	at stream.ConcurrentGenericReadInputStream.<init>(ConcurrentGenericReadInputStream.java:132)
	at stream.ConcurrentGenericReadInputStream.getReadInputStream(ConcurrentGenericReadInputStream.java:661)
	at stream.ConcurrentGenericReadInputStream.getReadInputStream(ConcurrentGenericReadInputStream.java:641)
	at kmer.KmerCount7MTA.countFastq(KmerCount7MTA.java:355)
	at kmer.KmerCount7MTA.makeKca(KmerCount7MTA.java:222)
	at jgi.KmerNormalize.runPass(KmerNormalize.java:1006)
	at jgi.KmerNormalize.main(KmerNormalize.java:736)
Setting interleaved=false doesn't change that. Outputting to a single, interleaved file (in1=xxx in2=xxx out=xxx) on the other hand works fine. Any ideas?

Olaf
muol is offline  
Old 06-23-2014, 06:02 PM   #17
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default

Olaf,

Currently, BBNorm uses single interleaved files for temporary storage when using multiple passes. And I have not implemented any way to specify dual files in intermediate stages, since everyone at JGI uses interleaved files for everything.

You have two options.
1) You could set "passes=1", which is faster, but I don't recommend it because it doesn't give as good results as 2-pass normalization.
or
2) You could specify only a single output file, which will get interleaved reads:

bbnorm.sh in1=R1.fastq.gz in2=R2.fastq.gz out=R12.bbnorm.fastq.gz prefilter=t tossbadreads=t ecc=t fixspikes=t qin=33 -Xmx72g target=40

...Then, if you need to, de-interleave it afterward:

reformat.sh in=R12.bbnorm.fastq.gz out1=R1.bbnorm.fastq.gz out2=R2.bbnorm.fastq.gz

Sorry for the inconvenience! I'll try to fix that by the next release, though unlike documenting the "qin" flag, this will take more work so no guarantees. Thanks for bringing it to my attention. FYI, the flag "interleaved" has no effect on output, only input.

-Brian

Last edited by Brian Bushnell; 06-23-2014 at 06:05 PM.
Brian Bushnell is offline  
Old 06-23-2014, 07:14 PM   #18
muol
Member
 
Location: USA

Join Date: Jun 2012
Posts: 10
Default

Thanks for the info Brian, it wasn't a big issue.

Olaf
muol is offline  
Old 06-27-2014, 03:26 PM   #19
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,695
Default

Olaf,

This has been fixed in the latest release, 33.04
Brian Bushnell is offline  
Old 06-27-2014, 04:11 PM   #20
muol
Member
 
Location: USA

Join Date: Jun 2012
Posts: 10
Default

Excellent, just did a test run. This is very useful software!

Olaf
muol is offline  
Closed Thread

Tags
bbmap, bbnorm, bbtools, short read alignment

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 12:21 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO