SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Frequency distribution of base quality scores from SAM file VeBeKay Bioinformatics 3 02-11-2014 12:48 AM
Bowtie2 SAM output missing base quality scores Comosus Bioinformatics 2 09-09-2012 11:21 PM
Read quality in SAM files from tophat biofreak General 4 07-08-2011 12:31 PM
Extracting base coverage and quality values from BAM files unagaswamy Bioinformatics 9 11-11-2010 06:55 AM
export.txt files/ quality filtering oleg Illumina/Solexa 8 08-19-2010 05:20 AM

Reply
 
Thread Tools
Old 04-08-2014, 01:05 PM   #1
JPreston
Member
 
Location: Eugene, OR

Join Date: Jun 2013
Posts: 17
Default Filtering SAM files by base quality (QUAL)?

Hello,

I need to filter my SAM/BAM files to remove bases below a certain base quality (QUAL) threshold, eg. Q30. Does anybody know if there any tools for this available?

Thanks in advance!
JPreston is offline   Reply With Quote
Old 04-08-2014, 02:51 PM   #2
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,707
Default

BBTools can discard reads from a sam file with average quality below a threshold:

reformat.sh in=reads.sam out=filtered.sam maq=30

However, pairing would get messed up, if the reads are paired. It's easier to do quality-related stuff on the fastq file, then map.
Brian Bushnell is offline   Reply With Quote
Old 04-08-2014, 03:14 PM   #3
JPreston
Member
 
Location: Eugene, OR

Join Date: Jun 2013
Posts: 17
Default

OK thanks a lot! Do you know of any programs that can mask fastq bases by quality, rather than discarding the read?
JPreston is offline   Reply With Quote
Old 04-08-2014, 03:24 PM   #4
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,707
Default

The same tool can trim low-quality bases, rather than discarding reads:

reformat.sh in=reads.fastq out=trimmed.fastq qtrim=rl trimq=30

(include in2 and out2 if you have paired reads in 2 files)

That command would trim the right and left sides of each read with a quality threshold of 30. Masking would be less useful, normally. And by the way, 30 is typically way too high of a threshold for trimming; I recommend around 10 for most uses.
Brian Bushnell is offline   Reply With Quote
Old 04-08-2014, 03:35 PM   #5
JPreston
Member
 
Location: Eugene, OR

Join Date: Jun 2013
Posts: 17
Default

Thanks again. I need a way to keep all of the bases above the quality threshold, rather than trimming the ends. I also want the quality scores to be very high for this application. It should be straightforward to write a script to filter and premask a fastq file, but I was hoping there was already a tool for this available. I wasn't sure how to deal with the quality scores of the masked bases.
JPreston is offline   Reply With Quote
Old 04-08-2014, 03:45 PM   #6
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,707
Default

I'm not sure what you could do with fastq reads that have all bases with quality below some threshold masked... so I have not written anything to do that. What are you trying to accomplish?
Brian Bushnell is offline   Reply With Quote
Old 04-08-2014, 04:01 PM   #7
JPreston
Member
 
Location: Eugene, OR

Join Date: Jun 2013
Posts: 17
Default

I am working on a method to call low-frequency SNPs in heterogeneous populations. The method is based on using overlapping paired-end reads to sequence each DNA molecule twice. This eliminates sequencing errors because an error will only occur in one of the reads. We use a program called SeqPrep to overlap the reads and generate a consensus sequence with very high quality. I have been using a program called LoFreq to call these low frequency SNPs but there is no way to filter the calls based on quality. I want to filter my reads by base quality in order to see how the various methods compare at various quality thresholds.

Last edited by JPreston; 04-08-2014 at 09:04 PM.
JPreston is offline   Reply With Quote
Old 04-08-2014, 04:10 PM   #8
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,707
Default

reformat.sh works with paired reads, it's just that the sam/bam format does not ensure paired reads are together, so it's possible for a read to be filtered out and its mate to be kept, which may or may not matter, depending on the application.

I also have a program called "bbmerge" which will merge overlapped reads to create a consensus:

bbmerge.sh in=read1.fq in2=read2.fq out=merged.fq outu=unmerged.fq

It's very fast and much more accurate than another commonly used tool, Flash (I have not tried SeqPrep, though). The output (merged.fq) is no longer paired, because the merged reads only create 1 consensus sequence, so it seems odd to output two copies of it. So the bam file would be filled with single-ended reads anyway, at least for the reads that were successfully merged.
Brian Bushnell is offline   Reply With Quote
Old 04-08-2014, 04:13 PM   #9
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 6,978
Default

It is odd that you are not doing your quality filtering up front. With SeqPrep you should be able to do that as you overlap the reads.
GenoMax is offline   Reply With Quote
Old 04-08-2014, 09:10 PM   #10
JPreston
Member
 
Location: Eugene, OR

Join Date: Jun 2013
Posts: 17
Default

I do some quality filtering before mapping but I know of no program that will mask the bases rather than trim or remove the reads completely. I don't want to trim reads at such a high quality threshold because I will lose too many bases.
Also, I don't know of an option in seqprep that works to quality filter the output.
JPreston is offline   Reply With Quote
Old 04-08-2014, 10:36 PM   #11
yueluo
Member
 
Location: Guangzhou China

Join Date: Aug 2013
Posts: 82
Default

You can specify your threshold for base-trimming.
Anyway, what do you mean by masking bases?
yueluo is offline   Reply With Quote
Old 04-09-2014, 01:00 PM   #12
JPreston
Member
 
Location: Eugene, OR

Join Date: Jun 2013
Posts: 17
Default

By masking I mean replacing low quality bases with an N.
JPreston is offline   Reply With Quote
Old 04-17-2014, 04:22 PM   #13
JPreston
Member
 
Location: Eugene, OR

Join Date: Jun 2013
Posts: 17
Default

I used BBmap to trim the reads. It worked well. Thanks!
JPreston is offline   Reply With Quote
Old 04-17-2014, 04:34 PM   #14
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,707
Default

Quote:
Originally Posted by JPreston View Post
I used BBmap to trim the reads. It worked well. Thanks!
You're welcome!
Brian Bushnell is offline   Reply With Quote
Old 10-17-2018, 10:03 PM   #15
MNMoller
Junior Member
 
Location: Denmark

Join Date: Oct 2018
Posts: 3
Default

Dear JPreston

Did you find a solution to your problem. I have the same issue and is looking for a tool to filter out bases within a read with low quality value. The tool should be able to do this on the SAM file, or on the FASTQ file, as long as it wont affect the alignment; for example by replacing it with an N like you suggest.
MNMoller is offline   Reply With Quote
Reply

Tags
bam, base quality, qual, quality filterin, sam

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 01:10 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO