SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Removing duplicates is it really necessary? foxyg Bioinformatics 34 12-02-2016 01:17 PM
RNAseq - removing optical duplicates only BENGwins Bioinformatics 1 11-14-2012 08:37 AM
FastX tool for removing duplicates archie.chauhan Bioinformatics 0 05-15-2012 05:40 AM
removing duplicates in small (si)RNA data Kennels Bioinformatics 1 02-29-2012 09:03 PM
Removing Duplicates Scenario Exome Resequencing Hkins552 Bioinformatics 1 12-05-2011 05:23 PM

Reply
 
Thread Tools
Old 10-28-2014, 04:09 AM   #1
mhadidi2002
Member
 
Location: Germany

Join Date: Jun 2011
Posts: 24
Default Removing duplicates from 16S RNA data

Hello All,

I would like to start a pipeline to analyse 16S RNA data, however I am not sure weather to include the step of removing duplicates or not. The aim of metagenomics is to compare different samples and see how the microbiota is different within samples of different conditions/time points.

After aligning, each sequence will match a sequence from a database, which will be then mapped to a bacteria and assigned taxonomy. the aim is to find how microbial content changes within each sample. If I removed duplicates, each sequence will be represented once, then taxonomy to this organism will be assigned once! so I am losing data here, which is the content of unknown bacteria in a sample with quantity for each one.

I am not sure if I am perceiving this right or not! I appreciate your comments.

Regards,
Bioinfguy
mhadidi2002 is offline   Reply With Quote
Old 10-28-2014, 04:34 AM   #2
kmcarr
Senior Member
 
Location: USA, Midwest

Join Date: May 2008
Posts: 1,178
Default

Quote:
Originally Posted by mhadidi2002 View Post
If I removed duplicates, each sequence will be represented once, then taxonomy to this organism will be assigned once! so I am losing data here, which is the content of unknown bacteria in a sample with quantity for each one.
I think you just answered your own question. No, do not remove duplicates for 16S metagenomics studies.
kmcarr is offline   Reply With Quote
Old 10-28-2014, 05:30 AM   #3
mhadidi2002
Member
 
Location: Germany

Join Date: Jun 2011
Posts: 24
Default

Thanks Kmcarr for the reply. I am asking because in mothur pipeline they do remove duplicates, that's why I am confused. Do you know why they are doing such so?
if you go to the link: http://www.mothur.org/wiki/MiSeq_SOP and searched by "unique.seqs" you will find it!
mhadidi2002 is offline   Reply With Quote
Old 10-28-2014, 05:46 AM   #4
kmcarr
Senior Member
 
Location: USA, Midwest

Join Date: May 2008
Posts: 1,178
Default

Quote:
Originally Posted by mhadidi2002 View Post
Thanks Kmcarr for the reply. I am asking because in mothur pipeline they do remove duplicates, that's why I am confused. Do you know why they are doing such so?
if you go to the link: http://www.mothur.org/wiki/MiSeq_SOP and searched by "unique.seqs" you will find it!
The tutorial explains why they are doing it, for efficiency. They keep a single copy of each unique sequence so that you only have to align that sequence to your reference once. But notice that in this process you also create a file with the counts of each unique sequence in the original data set which is used later in the analysis so you have not lost that abundance data. A more apt name for this is "collapsing duplicates" as opposed to "removing duplicates".
kmcarr is offline   Reply With Quote
Old 10-28-2014, 05:52 AM   #5
mhadidi2002
Member
 
Location: Germany

Join Date: Jun 2011
Posts: 24
Default

Quote:
But notice that in this process you also create a file with the counts of each unique sequence in the original data set which is used later in the analysis so you have not lost that abundance data.
Fastx has a tool called fastq_collaper which output a fasta file like this:

>1-934
CCTACGGGAGGCAGCAGTGAGGAATATTGGTCAATGGGCGGAAGCCTGAA
>2-915
CCTACGGGAGGCAGCAGTGGGGAATATTGCACAATGGGGGAAACCCTGAT

This means that sequence no. 1 has 934 duplicates. when I use this to align against database, the aligner consider 1-934 to be the sequence ID, will not know that it has 934 duplicates. How can I use this in further analysis to keep the abundance?
mhadidi2002 is offline   Reply With Quote
Old 10-28-2014, 05:54 AM   #6
kmcarr
Senior Member
 
Location: USA, Midwest

Join Date: May 2008
Posts: 1,178
Default

Quote:
Originally Posted by mhadidi2002 View Post
Fastx has a tool called fastq_collaper which output a fasta file like this:

>1-934
CCTACGGGAGGCAGCAGTGAGGAATATTGGTCAATGGGCGGAAGCCTGAA
>2-915
CCTACGGGAGGCAGCAGTGGGGAATATTGCACAATGGGGGAAACCCTGAT

This means that sequence no. 1 has 934 duplicates. when I use this to align against database, the aligner consider 1-934 to be the sequence ID, will not know that it has 934 duplicates. How can I use this in further analysis to keep the abundance?
Are you using Mothur or are you using your own custom pipeline? If you are using your own then you will need to figure out a way to add the count information back into your analysis.
kmcarr is offline   Reply With Quote
Old 10-28-2014, 05:59 AM   #7
mhadidi2002
Member
 
Location: Germany

Join Date: Jun 2011
Posts: 24
Default

Quote:
Originally Posted by kmcarr View Post
Are you using Mothur or are you using your own custom pipeline? If you are using your own then you will need to figure out a way to add the count information back into your analysis.
I am using my custom pipeline. so you mean that if I used mothur pipeline, they keep this in consideration to the rest of pipeline, but if I am using my pipeline I have to write a script that do this.. cool. Thnaks for the help!
mhadidi2002 is offline   Reply With Quote
Reply

Tags
metagenomics

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 07:51 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO