picard markduplicates on huge files

rcorbett · 09-16-2010, 09:47 AM

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

malachig · 09-16-2010, 07:17 PM

Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...

drio · 09-17-2010, 04:39 AM

Quote:

Originally Posted by rcorbett

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

Can you show us how your data looks like and the commands you are using?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Error with MarkDuplicates in Picard	slowsmile	Bioinformatics	6	04-17-2013 07:23 PM
Picard's MarkDuplicates -> OutOfMemoryError	elgor	Bioinformatics	3	06-07-2011 12:15 AM
MarkDuplicates in picard	bair	Bioinformatics	3	12-23-2010 11:00 AM
How to use Picard's MarkDuplicates	cliff	Bioinformatics	11	12-22-2010 04:47 AM
Picard MarkDuplicates	wangzkai	Bioinformatics	2	05-18-2010 09:14 PM

09-16-2010, 09:47 AM	#1
rcorbett Member Location: canada Join Date: Sep 2009 Posts: 29	picard markduplicates on huge files Hey folks, I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done. Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes. thanks!

09-16-2010, 07:17 PM	#2
malachig Senior Member Location: WashU Join Date: Aug 2010 Posts: 114	Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...