Unconfigured Ad

Collapse
X
Collapse
 
  • Page of 1
  • Filter
  • Time
  • Show
Clear All
new posts
Previous template Next
  • rcorbett
    Member
    • Sep 2009
    • 29
    #1

    picard markduplicates on huge files

    Hey folks,
    I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

    Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

    thanks!
    Tags: bam, duplicates, picard
  • malachig
    Senior Member
    • Aug 2010
    • 117
    #2
    Is the BAM file sorted already? If not, is it possible that it is during sorting that picard uses excessive memory? If that's the case, you could, presort the BAM file using SAM tools (which allows max memory usage to be specified) and then use picard to mark duplicates using the 'ASSUME_SORTED' option...

    Comment

    • drio
      Senior Member
      • Oct 2008
      • 323
      #3
      Originally posted by rcorbett View Post
      Hey folks,
      I'm trying to run markduplicates on some massive files (merged Solid bams), and often 100Gigs of RAM doesn't get the job done.

      Does anyone have a good suggestion for a workaround. I don't want to split into chromosomes because I lose the ability to mark dups that span multiple chromosomes.

      thanks!
      Can you show us how your data looks like and the commands you are using?
      -drd

      Comment

      Previous template Next

      Latest Articles

      Collapse
      • GATTACAT
        Reply to Nine Things a Sample Prep Scientist Thinks About Before Sequencing
        by GATTACAT
        Love this - good data definitely starts from good input, and poor input can only give relatively poor data. I particularly like the mention of Nanodrop/absorbance based methods for quantification. It's such a toss up if you'll get an accurate reading or what amounts to a randomly generated number, and a lot of library/sequencing related issues can be traced back to poor quant.
        07-01-2026, 11:43 AM
      • SEQadmin2
        Nine Things a Sample Prep Scientist Thinks About Before Sequencing
        by SEQadmin2


        I’m not a sequencing expert. I’m a purification scientist who uses NGS to evaluate workflows my group develops. With this perspective, we think about the sample first and the NGS workflow second. The sequencer is an exceptionally honest reporter, but it can only report on what you give it, so whether you get clean, interpretable data from an NGS workflow is largely determined before you begin.

        Here are nine questions we think about, in roughly the order they matter, before...
        06-18-2026, 07:11 AM
      View All

      ad_right_rmr

      Collapse

      News

      Collapse
      Topics Statistics Last Post
      High-Resolution Sequencing Exposes Hidden Toxoplasma Diversity by SEQadmin2
      Started by SEQadmin2, Yesterday, 11:08 AM
      0 responses
      7 views
      0 reactions
      		 Last Post SEQadmin2
      by SEQadmin2
      Yesterday, 11:08 AM  
      New AI Model Captures Long-Range Genomic Signals to Improve RNA Splice Site Prediction by SEQadmin2
      Started by SEQadmin2, 06-30-2026, 05:37 AM
      0 responses
      12 views
      0 reactions
      		 Last Post SEQadmin2
      by SEQadmin2
      06-30-2026, 05:37 AM  
      Large-Scale Protein Screen Uncovers Hidden Regulators of Alternative Polyadenylation by SEQadmin2
      Started by SEQadmin2, 06-26-2026, 11:10 AM
      0 responses
      20 views
      0 reactions
      		 Last Post SEQadmin2
      by SEQadmin2
      06-26-2026, 11:10 AM  
      Whole-Genome Sequencing Traces Faroe Islands Ancestry to a North Atlantic Founder Population by SEQadmin2
      Started by SEQadmin2, 06-17-2026, 06:09 AM
      0 responses
      54 views
      0 reactions
      		 Last Post SEQadmin2
      by SEQadmin2
      06-17-2026, 06:09 AM  
      View All
      Working...