Allele frequency calculation in SNP calling

[Rachelly]

Hi all,

I was wondering how the allele frequency is calculated by SAMTOOLS (I'm using mpileup and then bcftools).
It seems that my SNPs always have a frequency of either 1 or 0.5, and do not always match the DP4 values.

For example:

chr01   11961   .       G       C       52      .       DP=38;AF1=0.5;CI95=0.5,0.5;DP4=12,16,3,7;MQ=42;FQ=55;PV4=0.71,0.26,0.052,0.41   GT:PL:GQ        0/1:82,0,255:85

Here it seems that the allele frequency should be more 0.3 than 0.5, doesn't it?

Another example:

chr02   667170  .       C       T       14.2    .       DP=135;AF1=1;CI95=1,1;DP4=13,1,79,38;MQ=60;FQ=-71;PV4=0.064,6.2e-46,1,1 GT:PL:GQ        1/1:47,44,0:55

In this case the frequency is closer to 0.9 than to 1...

How is the frequency calculated?

Thanks!
Rachelly.

[clarissaboschi]

I have the same doubt!!!!!!!!

[shawpa]

I am running into the same question with GATK unified genotyper output. Any help? I want the exact frequency using the counts. It seems to always just give a .5 or 1

[lh3]

GATK and samtools assume samples are all diploid. You should look at DP4 or pileup instead.

Also note that samtools does not work with pooled samples.

[shawpa]

GATK and samtools assume samples are all diploid. You should look at DP4 or pileup instead.

Also note that samtools does not work with pooled samples.

Could you clarify what DP4 or pileup is? I wasn't able to find a software package called either of those things. Could you point me in the right direction?

[shawpa]

Could you clarify what DP4 or pileup is? I wasn't able to find a software package called either of those things. Could you point me in the right direction?

Ignore what I just wrote. I realized now I was being very stupid because you were referring to a format. I guess I am just trying to find an easy way to extract those counts so that I can calculate allele frequency from those. I can't do that with GATK.