picard markduplicates on huge files

rcorbett · 09-16-2010, 09:47 AM

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

malachig · 09-16-2010, 07:17 PM

Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...

drio · 09-17-2010, 04:39 AM

Quote:

Originally Posted by rcorbett

Hey folks,
I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

thanks!

Can you show us how your data looks like and the commands you are using?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Error with MarkDuplicates in Picard	slowsmile	Bioinformatics	13	11-01-2015 03:16 AM
How to use Picard's MarkDuplicates	cliff	Bioinformatics	12	01-26-2015 10:56 PM
Picard's MarkDuplicates -> OutOfMemoryError	elgor	Bioinformatics	15	08-05-2013 06:37 AM
MarkDuplicates in picard	bair	Bioinformatics	3	12-23-2010 11:00 AM
Picard MarkDuplicates	wangzkai	Bioinformatics	2	05-18-2010 09:14 PM

09-16-2010, 09:47 AM	#1
rcorbett Member Location: canada Join Date: Sep 2009 Posts: 29	picard markduplicates on huge files Hey folks, I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done. Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes. thanks!

09-16-2010, 07:17 PM	#2
malachig Senior Member Location: WashU Join Date: Aug 2010 Posts: 116	Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...