SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Process to remove primers, adapters, etc. from Illumina data LizBent Bioinformatics 6 05-14-2012 04:08 AM
Software to process IonTorrent data Marius Ion Torrent 4 02-14-2012 09:49 AM
Calculation of pi(a)/pi(s) for sequence data in pileup format Lisamjb Bioinformatics 1 02-01-2012 12:57 PM
the pileup result from samtools doesn't mach to read data Anney Bioinformatics 3 07-18-2011 04:44 PM
The best to process sample with 2 lanes of data? foxyg Bioinformatics 2 01-06-2011 02:25 PM

Reply
 
Thread Tools
Old 01-04-2012, 03:02 AM   #1
david.tamborero
Member
 
Location: spain

Join Date: Feb 2011
Posts: 60
Default How Varscan process the pileup data?

Hi,

if any of you has used Varscan to find somatic mutations given normal and tumor pileup files, maybe you can give me some light about how it processes the data.

For instance, given the following position in the tumor pileup file:

Code:
chr1	173824	T	21	GG...*..*.G.......GG.	
				##9!!!!;!<#;<<=<<<!#<
Varscan says that 8 tumor reads supports the 'T' and 2 tumor reads supports the 'G'. I do not see how it is fitted!

Another random example:

Code:
chr1	30028	c	9	,tt,,,T,,	
				5::778?.4
Varscan says that in the previous position, there are 3 reads for nucleotide 'T' and 5 for 'C' (why not 6??). There are these kind of discordances in almost all the positions I've checked, so I'm pretty confused.

thanks!
david
david.tamborero is offline   Reply With Quote
Old 01-04-2012, 11:04 AM   #2
dkoboldt
Member
 
Location: St. Louis

Join Date: Mar 2009
Posts: 62
Default

VarScan includes a minimum base quality threshold requirement in order to count reads. By default, this is 15 or 20 depending on the command you use.

In the first example, some bases have a base quality represented by '#', indicating a Phred score of 3. In the second example, one reference base has a quality represented by ".", which is 14. Both of these fall below the minimum base quality threshold in VarScan.
dkoboldt is offline   Reply With Quote
Old 01-04-2012, 05:19 PM   #3
david.tamborero
Member
 
Location: spain

Join Date: Feb 2011
Posts: 60
Default

Thank you for your response, dkoboldt.

I indeed thought in the possibility that some 'default quality filtering' was being applied, but (for instance) in the first example, all G's should be not taken into account since their qualities are either '#' or '!'. However, Varscan stats say that for that position there are two G's. This does not seem to fit.

There is plenty of things like that, and I am not able to find a 'pattern' of quality filtering by checking the pileup positions as compared to the corresponding VArscan entry...

thanks a lot,
cheers
David

Last edited by david.tamborero; 01-04-2012 at 05:24 PM.
david.tamborero is offline   Reply With Quote
Old 01-22-2012, 11:58 AM   #4
david.tamborero
Member
 
Location: spain

Join Date: Feb 2011
Posts: 60
Default

Just for those that can be interested in the varscan filtering according to the base quality: apparently Varscan has a bug that arises when certain characters appear in the field containing the bases (for instance, the '*'). This provokes a shift in the 'base-base quality' correspondance and then the counting of bases supporting each allele becomes wrong.

Therefore, the base quality filtering performed by Varscan fails for those reads containing these characters. I've written to the authors, hope it helps.

cheers,
david
david.tamborero is offline   Reply With Quote
Old 01-23-2012, 12:05 AM   #5
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

Hello David,
I intended to try Varscan with normal and tumoral files, so thanks for mentionning the problem. Please, keep us informed if you got news from the authors. I would like to try the tool when the problem is solved!
Do you know if the problem is also present when using Varsan with only one file?
Jane M is offline   Reply With Quote
Old 01-23-2012, 12:56 AM   #6
david.tamborero
Member
 
Location: spain

Join Date: Feb 2011
Posts: 60
Default

No idea, I only have used the somatic command. You can try the other Varscan commands (as the pileup2snp) with a dummy pileup file and play with the --min-avg-qual parameter to check it out.

By the way, I need that Varscan performs the quality filtering because the samtools mpileup command also does it wrong (the -Q argument does not work). Seems that the base quality filtering is not that important!

cheers,
David
david.tamborero is offline   Reply With Quote
Old 01-23-2012, 01:17 AM   #7
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

Ok, thanks!
Jane M is offline   Reply With Quote
Old 02-02-2012, 07:56 AM   #8
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

I tried VarScan in the somatic mode and I also experienced this problem.
For example, VarScan gives as output the following number of reads:

- for the normal sample: 183 (reference) and 4 (variant)
- for the tumor sample: 8 (reference) and 14 (variant)

This results in a somatic mutation. I checked my bam files in IGV and I found:
- for the normal sample: 185 (reference) and 165 (variant)
- for the tumor sample: 8 (reference) and 359 (variant)

Of course, some reads can be deleted because they have low quality, but I doubt that it could be the cases for so many reads... This variation is rather a LOH rather than a somatic mutation.
And I've got this kind of results several times...

I wonder how this problem affect the SNPs detection and how wrong is my analysis...

Do you know if this problem will be taken into account?
Jane M is offline   Reply With Quote
Old 02-02-2012, 08:26 AM   #9
david.tamborero
Member
 
Location: spain

Join Date: Feb 2011
Posts: 60
Default

I guess than before running varscan you have converted the bam file to pileup file, right?

If so, such 'lost' reads could be explained by the samtools mpileup command. Check its arguments: the mapping_quality filtering works (as far as i remember, and as opposite to the base_quality filtering), therefore some reads can be removed due to this parameter. The other source of reads removal during bam to pileup conversion can be the 'anomalous read pairs': check the '-A' argument of the mpileup command to see if reads counts are more consistent with IGV data (which opens the bam file).
david.tamborero is offline   Reply With Quote
Old 02-06-2012, 07:49 AM   #10
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

Quote:
Originally Posted by david.tamborero View Post
I guess than before running varscan you have converted the bam file to pileup file, right?

If so, such 'lost' reads could be explained by the samtools mpileup command. Check its arguments: the mapping_quality filtering works (as far as i remember, and as opposite to the base_quality filtering), therefore some reads can be removed due to this parameter. The other source of reads removal during bam to pileup conversion can be the 'anomalous read pairs': check the '-A' argument of the mpileup command to see if reads counts are more consistent with IGV data (which opens the bam file).
Exactly, I have converted my bam files into pileup files to use VarScan.

Quote:
samtools mpileup -f ~/fasta/hg19.fasta /data/patient1/s_garma-296_converted_sorted.bam > /data/patient1/s_garma-296_converted_sorted.pileup
The default arguments are:
Quote:
-q INT skip alignments with mapQ smaller than INT [0]
-Q INT skip bases with baseQ/BAQ smaller than INT [13]
I checked on IGV the PHRED of some of the reads at this position and they were good (>30). So the missing reads are probably not due to base quality. I don't know for the mapping quality... How can I check it?

As you suggested me, I re-run samtools mpileup with -A:
Quote:
samtools mpileup -f ~/fasta/hg19.fasta /data/patient1/s_garma-296_converted_sorted.bam > /data/patient1/s_garma-296_converted_sorted.pileup
And I have still the same kind of problems: LOH considered as somatic mutations because of missing reads.
I am not sure what is doing the -A option... Does it add some anomalous reads?

Last edited by Jane M; 02-06-2012 at 07:52 AM.
Jane M is offline   Reply With Quote
Old 03-01-2012, 07:27 AM   #11
dkoboldt
Member
 
Location: St. Louis

Join Date: Mar 2009
Posts: 62
Default

Jane,

It will also help if you provide the pileup output for 1-2 examples of this base dropout - we can quickly look at what's in it to see if VarScan is not counting anything.

Please note that the base quality parsing issue was resolved as of VarScan v2.2.8. If any of you encounter it, please let me know!

Yours,

Dan Koboldt
dkoboldt is offline   Reply With Quote
Old 03-01-2012, 11:29 PM   #12
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

Thank you for your answer Dan,

Here is the output of the pileup file at the position where I have the problem I mentioned:
With VarScan somatic:
- for the normal sample: 183 (reference G) and 4 (variant A)
- for the tumor sample: 8 (reference) and 14 (variant)

With IGV:
- for the normal sample: 185 (reference) and 165 (variant)
- for the tumor sample: 8 (reference) and 359 (variant)

Quote:
[PCJane patient1]$ grep 186380515 s_garma-fibros_converted_sorted.pileup
chr4 186380515 G 350 .$,$a$,$,$a$a$.Aa,aA.aaa,..,a,.....aa,,,a,,AAaaaA.AA,,Aa,,aa...a,..a,,.AAaaa,a...,,a,,,..A,a,a,.,,.Aaaaa,,,.,Aa.,Aa,,Aa,AAa,aa..a,,a..a,aAa,aA.Aaa.aaAAAA.A,a,aA,A,,,..aa,aaa.A.aaa..aa...A,,,aA.AA..Aa,,aa...a,aaaAA.AaAa,a,AAA.,a,AAA....a..,aa,AA,A.a..aA,aaa.,,aA..AAa,aA.,aA,,AA,,,,,AaAAA,A,,,A.aaa..aa,AAa,,,..A,,,,A.a,a,aa,,....aaa,....,,,,..A..,aa......^~A^~, [email protected]!!B([email protected][email protected]:FF75HH55JIJ5GJJ9HH733555H5GII:J3JIJBH4J3J3JIJJJ53333JIJEJ43IJ43JJ43J334F33EJ4JJ3JI3J333J;3J433J333363J4J3F43J6JGIJJ3:J333?3J333JJ33JGJ4JJJ84J33JJ33GJ33JJJ3J:3346J3343J3J>33JJ*J334IJGJ4JJJ34G33F3J4JJ46J433JDJ33IJ333I33JJ33JJ43JIJJJ43334[email protected]CCCC%E
^C
[PCJane patient1]$
I intend to test Varscan in the "simple mode" today to check if I have this issue too.

When using the CASAVA pipeline on my normal sample, I got:

Quote:
186380515 chr4 (G)155 (A) 142
By comparison between, IGV, CASAVA and Varscan in the normal sample, I have:
-for the reference: 185, 155, 183
-for the variant: 165, 142, 4

Moreover, I tried JointSNVMix, which let the direct comparison between normal and tumoral sample. At this specific position, I have nothing. But at several sites, where I see this problem in VarScan ,my results (number of read counts) with JointSNV are closed to the ones in IGV...

Last edited by Jane M; 03-02-2012 at 12:39 AM.
Jane M is offline   Reply With Quote
Old 03-02-2012, 04:11 AM   #13
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

I ran VarScan in the simple mode on the normal sample, using pileup2snp:

Quote:
java -Xmx20g -jar VarScan.v2.2.8.jar pileup2snp /data/patient1/s_garma-fibros_converted_sorted.pileup --min-coverage 10 --min-avg-qual 25 > /data/patient1/output_varscan_snp_garma.snp
At the position where I have a problem in read counts, I got:
Code:
chr4  	186380515	(G)  184	   (A) 5
I have the same problem in both mode. It's very strange... It cannot be related to the version of VarScan since I'm using the latest one, v2.2.8...

I attach here the screenshots from IGV at this specific position.
Attached Images
File Type: png ReadsInNormalSample.png (51.6 KB, 20 views)
File Type: png ReadsInTumoralSample.png (45.2 KB, 14 views)
Jane M is offline   Reply With Quote
Old 03-16-2012, 06:05 AM   #14
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

Hello,

I just wanted to let you know that there are buggs in VarScan, in both default and somatic modes. There are other people experiencing the same problem like me.
To conclude, I give up VarScan but I hope the problem will be taken into account, because appart this issue, this seems to be an interesting tool!
Jane M is offline   Reply With Quote
Old 03-27-2012, 12:17 AM   #15
Jane M
Senior Member
 
Location: Paris

Join Date: Aug 2011
Posts: 237
Default

Hello,

If some users are experiencing the same issue and read this subject, the problem comes from samtools. Add the -B or -E option to disable the BAQ computation.
Jane M is offline   Reply With Quote
Old 04-19-2012, 06:13 AM   #16
SPRA
Junior Member
 
Location: Switzerland

Join Date: Oct 2010
Posts: 2
Default Samtools mpileup with -B or -E falg for Varscan

I just ran varscan on mpileup data with -B or -E flag. And there is quite a difference indeed (see below). Should varscan always be used on mpileup data generated with the -B flag?

samtools mpileup -d 10000 -S -B -C 50 -P Illumina -f hg19.fa normal.bam > normal.mpileup
samtools mpileup -d 10000 -S -B -C 50 -P Illumina -f hg19.fa tumor.bam > tumor.mpileup
java -jar VarScan.v2.2.10.jar somatic normal.mpileup tumor.mpileup tumor_vs_normal
java -jar VarScan.v2.2.10.jar processSomatic tumor_vs_normal.snp
48198 VarScan calls processed
3168 were Somatic (1229 high confidence)
37882 were Germline
6937 were LOH

OR

samtools mpileup -d 10000 -S -E -C 50 -P Illumina -f hg19.fa normal.bam > normal.baq.mpileup
samtools mpileup -d 10000 -S -E -C 50 -P Illumina -f hg19.fa tumor.bam > tumor.baq.mpileup
java -jar VarScan.v2.2.10.jar somatic normal.baq.mpileup tumor.baq.mpileup tumor_vs_normal.baq
java -jar VarScan.v2.2.10.jar processSomatic tumor_vs_normal.baq.snp
7593 VarScan calls processed
517 were Somatic (191 high confidence)
6406 were Germline
636 were LOH
SPRA is offline   Reply With Quote
Old 04-19-2012, 10:14 AM   #17
swbarnes2
Senior Member
 
Location: San Diego

Join Date: May 2008
Posts: 912
Default

Right, it's definately samtools pileup that is causing the problem. If you comapre the .sam file, and the two pileups (with and without -B) in regions where SNPs are being missed, you can see the problem. pileup -B will faithfully represent the letter quality scores found in the .sam file. pileup without -B will sometimes report the quality of those letters as being terrible, which causes SNP calling softwares to ignore them as too poor quality.

Using -B is supposed to reduce the number of false positives. There might be applications where that potential trade off is worth it, but for what it's worth, I'm skeptical. In my work, I'd much rather sift through false positives than miss real mutations.
swbarnes2 is offline   Reply With Quote
Old 05-15-2012, 03:59 AM   #18
bpetersen
Member
 
Location: Germany

Join Date: Mar 2010
Posts: 20
Default

Hello everyone,
I'm currently also experiencing problems with Varscan somatic, but they seem different from what has been described here. I've tried using the -B option for samtools pileup, but still I get completely wrong allele counts for my tumor sample for supposedly somatic mutations with very low p-values.
One example:
Create samtools pileup files with the command:
Code:
samtools pileup -Bf ref.fasta in.bam > out.pileup
run varscan somatic:
Code:
java -Xmx16g -jar /home/sukmb205/software/VarScan.v2.2.11.jar somatic normal.pileup tumor.pileup outfile.var
output varscan:
Code:
chrom	position	ref	var	normal_reads1	normal_reads2	normal_var_freq	normal_gt	tumor_reads1	tumor_reads2	tumor_var_freq	tumor_gt	somatic_status	variant_p_value	somatic_p_value	tumor_reads1_plus	tumor_reads1_minus	tumor_reads2_plus	tumor_reads2_minus
chr5	88171909	A	T	46	0	0%	A	91	95	51.08%	W	Somatic	1.0	6.837592912705086E-13	0	91	0	95
Looks like quite a good somatic mutation concerning the p-value, but when I look into IGV, NOT ONE read actually supports the T in the tumor sample. Instead, there is a single read at this position that has been aligned with a 308bp deletion, could this be the problem??

pileup tumor:
Code:
chr5	88171909	A	63	.,,,,,,,,,,,,,,,,,,,,,,..,,,.$.,.,,,,,,.,,,..,,-308actcctcaaactaccttcccacaaagccatttaagttaaatggtacatttacagactcacctacatgaaggatataacttaaaacatctgcttagacacatacgttctgttcagatataaaaaatgtggcaaaaatttttaaaaatataggaccactatattcttaaaatgtgtgttcttctgtgtgtgtgtgttcattcattcaagagatctttgactgcaattaggtagtcggtcctataaaggcttccttgtgtgacgataatttctaaaagtaaaatgctccagtgaatatttctgctaaataa,,,,,,.,....,,...	/2.*114I,1<(%040":+3/--+?%.-.CE2;/3*0-B8=).0II--7,+,II%[email protected]*CII
pileup normal:
Code:
chr5	88171909	A	71	...$.,,,,,,,,,,,.,,,,,,,,,.,,,,.,,.,,,,,,..,,,.,.,,,..,,,,,,.,...,....^X.^(.	BE,9=2++H8I31.)89,=I).2II=1I,+/8I51I>*2,7+.1I.I%-*-1I/2A-I/[email protected]:I.5I8II(I
I'm seeing similar things at all the high quality somatic mutations that were called and I've tried using mpileup instead of pileup, with option -B, without -B etc. I just don't know what the problem is!
I would really appreciate some comments on this, thanks!
bpetersen is offline   Reply With Quote
Old 05-20-2012, 10:34 PM   #19
bpetersen
Member
 
Location: Germany

Join Date: Mar 2010
Posts: 20
Default

Am I the only one with this issue? I still can't figure out what the problem is, in other datasets Varscan worked fine for me, but if this isn't fixed I'm going to have to use a different tool...
bpetersen is offline   Reply With Quote
Old 06-22-2012, 06:01 AM   #20
david.tamborero
Member
 
Location: spain

Join Date: Feb 2011
Posts: 60
Default

Hi!

I've just landed in this post after a long time without using somatic calling tools.

I am wondering if you solved this, I have no idea why it happened.

Anyway, I've just downloaded the Varscan2, and I've noticed that it incorporates a filter that, among other things, it seems to remove those SNPs which appear close to indels. Could it help to you?
david.tamborero is offline   Reply With Quote
Reply

Tags
somatic mutations, varscan

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 12:04 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO