I am having issues getting Fst values from vcftools that make sense (in range from 0 to 1). I think there is an issue with how alleles are being called (maybe issue with the vcf file?).
I have 2 populations with 4 individuals from each population (8 lines total). I have vcf files of each population separately (4 individuals in each of 2 files) and then a vcf file with all 8 individuals. When I look at allele counts for all 8 individuals it is incorrect (not the sum of allele counts if I add counts from the 2 files that contain only 4 individuals)
Below are allele counts for a single gene. Allele counts for pos 702 look correct to me and so do counts for pos 51 (assuming pop2 is all A), but the other two sites are not correct even though site 12 has variants in both populations.
Does someone have an idea of what may be going wrong?
Each vcf file was generated from mpileup with the exact same command just differing the number and identity of the sorted .bam files
Pop1
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 12 2 8 T:3 G:5
Dpse-Amy3-PA 51 2 8 A:0 G:8
Dpse-Amy3-PA 702 2 8 T:4 C:4
Pop2
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 10 2 8 A:3 C:5
Dpse-Amy3-PA 12 2 8 T:3 G:5
Dpse-Amy3-PA 702 2 8 T:2 C:6
Both Pops
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 10 2 16 A:0 C:16
Dpse-Amy3-PA 12 2 16 T:0 G:16
Dpse-Amy3-PA 51 2 16 A:8 G:8
Dpse-Amy3-PA 702 2 16 T:6 C:10
I have 2 populations with 4 individuals from each population (8 lines total). I have vcf files of each population separately (4 individuals in each of 2 files) and then a vcf file with all 8 individuals. When I look at allele counts for all 8 individuals it is incorrect (not the sum of allele counts if I add counts from the 2 files that contain only 4 individuals)
Below are allele counts for a single gene. Allele counts for pos 702 look correct to me and so do counts for pos 51 (assuming pop2 is all A), but the other two sites are not correct even though site 12 has variants in both populations.
Does someone have an idea of what may be going wrong?
Each vcf file was generated from mpileup with the exact same command just differing the number and identity of the sorted .bam files
Pop1
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 12 2 8 T:3 G:5
Dpse-Amy3-PA 51 2 8 A:0 G:8
Dpse-Amy3-PA 702 2 8 T:4 C:4
Pop2
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 10 2 8 A:3 C:5
Dpse-Amy3-PA 12 2 8 T:3 G:5
Dpse-Amy3-PA 702 2 8 T:2 C:6
Both Pops
CHROM POS N_ALLELES N_CHR {ALLELE:COUNT}
Dpse-Amy3-PA 10 2 16 A:0 C:16
Dpse-Amy3-PA 12 2 16 T:0 G:16
Dpse-Amy3-PA 51 2 16 A:8 G:8
Dpse-Amy3-PA 702 2 16 T:6 C:10
Comment