Go Back   SEQanswers > Bioinformatics > Bioinformatics

Similar Threads
Thread Thread Starter Forum Replies Last Post
kmeans clustering in R Galgarad Bioinformatics 3 04-08-2014 09:16 PM
GO term clustering cgault General 1 10-28-2013 08:43 PM
Concerns for combining data from HiSeq 2000 and HiSeq 2500 jaaker Illumina/Solexa 1 02-04-2013 02:56 PM
NaOH/clustering jlm Illumina/Solexa 3 05-23-2012 06:28 AM
AVA clustering Xterra 454 Pyrosequencing 0 12-04-2010 05:51 AM

Thread Tools
Old 04-17-2013, 01:21 AM   #1
Location: China

Join Date: Aug 2012
Posts: 11
Default Hiseq clustering

Hi all,

I have a huge dataset from Hiseq (15 million sequences after quality filtering).
I would like to use cd-hit or other clustering software to reduce the redundancy first, and proceed to BLAST against to nr database.

However, for the first step using cd-hit-est (v.4.6.1) with 10 threads, it already took me a week but still not yet finished.

I am new to NGS analysis, anyone could advise to speed up the process?
Your kindly suggestions and help are highly appreciate. Many thanks.
cookiemic is offline   Reply With Quote
Old 04-17-2013, 01:41 AM   #2
Senior Member
Location: uk

Join Date: Mar 2009
Posts: 667
Default Hiseq clustering

15 million reads from a HiSeq these days is not a large dataset.

You don't say what kind of biological experiment you are analysing that would explain why you want to cluster the data first.

In general, for NGS data you want to do either alignment (with a program like Bowtie or BWA, or there are many others) or de novo assembly, if you don't have a reference genome (with programs like Velvet, Soapdenovo, or other assembler).

Have a look at the list of available software in the SeqWiki.
mastal is offline   Reply With Quote
Old 04-17-2013, 07:27 AM   #3
Location: China

Join Date: Aug 2012
Posts: 11

My data is composed of sequences derived from microbes (virus or bacteria) isolated from patients.
And I would like to blastx each sequence to nr database.
But I know blast is too computational intensive, so I would like to reduce the redundancy by clustering first.
I have experience in using bowtie to map the dataset on a particular genome.
But I don't know if bowtie sensitive/accurate to map to nr database?
cookiemic is offline   Reply With Quote
Old 04-17-2013, 05:59 PM   #4
Senior Member
Location: Sydney

Join Date: Feb 2011
Posts: 149

The fastx toolkit has a tool to 'collapse' identical reads in your dataset, and output a header containing how many times the read was detected (e.g >1-xxx , >2-xxx, etc)

I'm sure there are other tools that would do this faster, but you'll have to search.
Also, it won't make much of a difference if you don't have much duplication in your dataset (it is very good for small RNA datasets).
I wouldn't cluster reads 'too much' as this will mask information when you align to a database. However I'm not sure what the end goal of your experiment is, but there might be very high similarity between sequences from different sources in your sample.

You could pull out only virus and bacterial sequences from the nr database, create a bowtie index from it and bowtie against that using perfect matches first. Bowtie2 will give you more stats. It should be relatively fast to align (shouldn't be a week), and you can interrogate the bowtie output rather than sam output which might be easier as a first pass. In fact, I would try this alignment first before doing any 'collapsing' of reads or clustering.
Kennels is offline   Reply With Quote

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

All times are GMT -8. The time now is 07:02 PM.

Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO