Slow Pileup After Switching to Stampy

[groundcover]

I'm calling the pileup like this:

samtools pileup -f ref.fa aln_reads.bam -c > out.pileup

I used to use BWA to generate the bam file and the pileup step above was quite fast (under 30 minutes).

Then I switched to using stampy to generate the bam file and this pileup step takes about 24 hours.

Do you guys have any insight into what causes pileup to run slower or faster? Is there anything I can do to my bam file to make it faster? Can I index anything? Would it help to upgrade samtools, I'm on 0.1.7 (r510).

Thanks,

Greg

[colindaven]

Definitely upgrade samtools, the latest is 0.1.18 I believe.

This is a bit strange, as I've used both aligners with Samtools extensively. Is one Stampy aligning many more reads (I doubt it ) ?

[groundcover]

I think Stampy is mapping more reads. So that could be part of the issue. But it's certainly not mapping 100x more reads. Perhaps pileup isn't O(n) though?

I have to be careful upgrading samtools as they're dropped support for pileup.

But perhaps I can try upgrading to the latest version that still has pileup.

[Rocketknight]

The alignments Stampy gets that BWA misses often involve several insertions or deletions, sometimes consecutively. Perhaps alignments that aren't quite as 'clean' are slowing samtools down somehow?

Also, are you aware that pileup has been replaced by mpileup in newer versions? That functionality isn't gone entirely, it's just been renamed. Or do you need some function that's only available with the original pileup?

[groundcover]

I need the 10 column -c (consensus) output of pileup. Sadly the new mpileup command doesn't support that.

I'm now wondering if there's a way to parelize the pileup run? Maybe split it up by chromosomes and then recombine the resulting pileups.