Go Back   SEQanswers > Bioinformatics > Bioinformatics

Similar Threads
Thread Thread Starter Forum Replies Last Post
merge BAM files from 2 SOLiD PE runs and remove duplicates epigen Bioinformatics 12 03-23-2012 02:55 AM
Picard Mark Duplicates help please hlwright Bioinformatics 7 01-26-2012 10:40 PM
How to differentiate between PCR duplicates and real data? frymor Bioinformatics 5 09-15-2011 01:05 PM
PCR duplicates questions slny Bioinformatics 8 06-07-2011 04:06 AM
Picard MarkDuplicates - How to identify duplicates in generated BAM file makarovv Bioinformatics 6 11-10-2010 09:02 AM

Thread Tools
Old 03-16-2011, 05:57 AM   #1
Junior Member
Location: Columbus, OH

Join Date: Nov 2010
Posts: 6
Default Picard - MakeDuplicates (remove pcr duplicates)

There seems to be only a few options for removing PCR duplicates from Illumina fastq data and/or alignment data. I have use FASTX for (fastx_collapse) for removing duplicates in fastq files but this takes a long time when running against human data of ~10Gb per lane.
The other option I have tried is SAMTOOLS (rmdup) but the documentation admits that it doesn't work well (or at all) with single end data. I have also noticed strange results when using this on my own data sets. (e.g. reads removed that were clearly not duplicates).
The alternative posted on the SAMTOOLS site is to use PICARD. Implementation is straightforward and uses percompiled jar files with well documented options.
I am unfortunately getting a strange Exception thrown when I run the MakeDuplicates jar file. It seems it is recognizing something in my data that wants to call it paired end data. However, my data are single end reads.
Does anyone have any experience with using this command in picard? Here is my call and the error being thrown.

 java -jar /home/bornmand/tools/picard/MarkDuplicates.jar INPUT=101119_first4lanes_bwa.bam OUTPUT=101119_first4lanes_bwaNoD
up.bam METRICS_FILE=101119_first4lanes_duplicateINFO.txt REMOVE_DUPLICATES=true ASSUME_SORTED=true
INFO    2011-03-16 08:09:46     MarkDuplicates  Start of doWork freeMemory: 62375408; totalMemory: 63111168; maxMemory: 935854080
INFO    2011-03-16 08:09:46     MarkDuplicates  Reading input file and constructing read end information.
INFO    2011-03-16 08:09:46     MarkDuplicates  Will retain up to 3713706 data points before spilling to disk.
INFO    2011-03-16 08:09:46     MarkDuplicates  Assuming input is coordinate sorted.
[Wed Mar 16 08:09:46 EDT 2011] net.sf.picard.sam.MarkDuplicates done.
Exception in thread "main" java.lang.RuntimeException: SAM validation error: ERROR: Record 1, Read name 4:20:14143:2730:Y, MRNM should not be set for unpaired read.
        at net.sf.samtools.SAMUtils.processValidationErrors(
        at net.sf.samtools.BAMFileReader$BAMFileIterator.advance(
        at net.sf.samtools.BAMFileReader$BAMFileIterator.<init>(
        at net.sf.samtools.BAMFileReader$BAMFileIterator.<init>(
        at net.sf.samtools.BAMFileReader.getIterator(
        at net.sf.samtools.SAMFileReader.iterator(
        at net.sf.samtools.SAMFileReader.iterator(
        at net.sf.picard.sam.MarkDuplicates.buildSortedReadEndLists(
        at net.sf.picard.sam.MarkDuplicates.doWork(
        at net.sf.picard.cmdline.CommandLineProgram.instanceMain(
        at net.sf.picard.sam.MarkDuplicates.main(
It seems that it runs okay but then encounters a problem.
dmb is offline   Reply With Quote
Old 03-16-2011, 07:50 AM   #2
Junior Member
Location: Dublin

Join Date: Jul 2009
Posts: 5

Hi Dan

Try adding "VALIDATION_STRINGENCY=LENIENT" to the end of your command line and see if this solves the problem.

Often incorrect flags can be set in the output of an aligner which downstream tools can take exception to.

Alternatively you can edit your BAM file to correct the record which is throwing up this error.

Hope this helps.
cormicp is offline   Reply With Quote
Old 03-16-2011, 08:56 AM   #3
Junior Member
Location: Columbus, OH

Join Date: Nov 2010
Posts: 6

That looks like it is working! Thanks a lot.
I did see that flag in the source code but I used "null" instead which was an invalid argument.
My new BAM file is being written to disk now. I will check results and hopefully it makes sense this time.
Thanks again.
dmb is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

All times are GMT -8. The time now is 11:12 AM.

Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2021, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO