Get unique reads when merging FastQ Files from BAM

[rkizen]

Hello,
I have paired end fastq files that were retrieved from two bam files using BamUtil and I would like to merge the respective fastq files. The only issue is that since they come from two bam files that were created using the same original fastq reads, there are many reads which are shared between the two respective files (ie bamfile1_reads_1.fastq has shared reads with bamfile2_reads_1.fastq). Are there any tools that would allow me to merge the files while ensuring that there are no duplicated reads in the merged fastq file?

Thank you!

[Heisman]

Maybe others will have ideas, but if they are exactly the same (ie, identical headers, bases, quality scores), you could convert them to one line, combine the files, sort them, and then use uniq, as shown here kind of: http://thegenomefactory.blogspot.com...-with-ngs.html

So, with file1.fq and file2.fq:

cat file1.fq file2.fq > file_combined.fq
paste - - - - file_combined.fq | sort | uniq > file_combined_unique.fq

[komalsrathi]

New to the forum

how do I create a new thread?

[GenoMax]

how do I create a new thread?

SeqAnswers --> Forums (in site navigation menu) --> Select appropriate forum (e.g. Bioinformatics) --> "New Thread" button in the top left corner.

[rkizen]

Maybe others will have ideas, but if they are exactly the same (ie, identical headers, bases, quality scores), you could convert them to one line, combine the files, sort them, and then use uniq, as shown here kind of: http://thegenomefactory.blogspot.com...-with-ngs.html

So, with file1.fq and file2.fq:

cat file1.fq file2.fq > file_combined.fq
paste - - - - file_combined.fq | sort | uniq > file_combined_unique.fq

Paste is a very cool command. Thank you for the link! Luckily the reads are identical.

Just as a note to anyone else who will be doing this, you may have to change your sort tmp directory when sorting "sort -T dir" if you are sorting a large fastq file and have limited space on your default tmp directory.

Oh and remember to tr "\t" "\n" to get it back in fastq format!