picard markduplicates on huge files

rcorbett · 09-16-2010, 10:47 AM

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

malachig · 09-16-2010, 08:17 PM

Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...

drio · 09-17-2010, 05:39 AM

Quote:

Originally Posted by rcorbett

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

Can you show us how your data looks like and the commands you are using?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How to use Picard's MarkDuplicates	cliff	Bioinformatics	12	01-26-2015 11:56 PM
Error with MarkDuplicates in Picard	slowsmile	Bioinformatics	12	09-03-2014 09:32 AM
Picard's MarkDuplicates -> OutOfMemoryError	elgor	Bioinformatics	15	08-05-2013 07:37 AM
MarkDuplicates in picard	bair	Bioinformatics	3	12-23-2010 12:00 PM
Picard MarkDuplicates	wangzkai	Bioinformatics	2	05-18-2010 10:14 PM

09-16-2010, 10:47 AM	#1
rcorbett Member Location: canada Join Date: Sep 2009 Posts: 29	picard markduplicates on huge files Hey folks, I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done. Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes. thanks!

09-16-2010, 08:17 PM	#2
malachig Senior Member Location: WashU Join Date: Aug 2010 Posts: 115	Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...