Tophat Run time

bassu · 06-21-2010, 06:29 AM

Dear all,

I am currently running tophat to align human genome reference(hg19 bowtie index) to illumina pair-end data of size 17Gb each(17*2). Its almost 24 houurs and its still running. I wonder how much more time it would take to finish the process?

. Concurrently i am also running Maq for 75bp single read of size 15GB with a reference size of 30mb(binary human genome file).

My system configuration is 64Gb Ram with 8 core processor(Which i feel is one among best configuration available in industry. Do i need to update my system configuration for NGS data analysis?If so please provide me the config).

And i also like to know how much processing time it would take if i run the Tophat and Maq separately?

Hopping for a speedy reply asap.

Thanks

Rao · 06-21-2010, 08:03 AM

try with bowtie...
is it RNA-seq data?

lmf_bill · 06-21-2010, 08:09 AM

Using tophat, especial for paired-end reads, it will take long time. In my personal point, your configuration is enough.
BTW, tophat will produce huge tmp file.

john_mu · 06-21-2010, 09:48 AM

Did you run TopHat with multiple threads?

If you are running it with only one thread 17Gb of reads will take several days to run...

What is your read length? Long reads also take much longer than short ones (for the same amount of data).

Running TopHat and Maq at the same time should not cause much problems (Unless you ran TopHat with 8 threads)

Regarding your system configuration, 64Gb should be plenty of RAM for your amount of data.

bassu · 06-22-2010, 01:36 AM

Thanks all,
@Rao: yes i am using RNA-seq data.

@mf_bill: thanks for your valuable comment, I was wondering whether my system configuration was right ? Even though BWT, tophat will produce huge tmp files.. it will get deleted automatically right?

@john_mu: Thanks john, currently i'm running my reads in a single thread. and my read length is 50bp.

DineshCyanam · 10-05-2010, 03:41 PM

@Bassu: So how long did it take for you to finally finish the run? Were u able to reduce the run time in any way? I am having the same problem here...

mrawlins · 10-06-2010, 09:44 AM

Running with n-1 threads on an n-core machine (e.g. 7 threads on a machine with 8 cores) should speed things up. Bowtie has a --shmem option for using shared memory for all threads, so that shouldn't increase the memory footprint by much to use that many threads. I've observed roughly a linear speed up with the number of cores dedicated to bowtie; I suspect similar results for tophat.
I will sometimes run with as many threads as cores, but only if I don't intend to use the computer for anything else while the program runs (i.e. a compute node on our analysis cluster).

crazyhottommy · 09-17-2013, 11:34 AM

I was running Tophat to map a 24G single end RNA-seq fastq to hg19 with the gtf from GENECODE
I run it in a cluster with 1 node, 8 processors, ram=3gb
it took me 45hrs to finish.....

Any way to speed it up? as a regular user of the cluster, the above setting is the max resource I can have.

dpryan · 09-17-2013, 12:17 PM

Quote:

Originally Posted by crazyhottommy

I was running Tophat to map a 24G single end RNA-seq fastq to hg19 with the gtf from GENECODE
I run it in a cluster with 1 node, 8 processors, ram=3gb
it took me 45hrs to finish.....

Any way to speed it up? as a regular user of the cluster, the above setting is the max resource I can have.

Split the fastq file and use multiple nodes. If you had more RAM, you could run STAR.

shi · 09-17-2013, 03:54 PM

You may try Subread, which is >10 times faster.

crazyhottommy · 09-17-2013, 06:47 PM

Quote:

Originally Posted by shi

You may try Subread, which is >10 times faster.

I will give it a shot. Thanks

adamyao · 09-17-2013, 10:59 PM

20.8G bases (104M reads, Pair End) 1 node (16 cores) AMD 2.4GHz 96G ram
- 28 hours
Tophat 2 runs best with 16 cores ( single node) according to our tests otherwise it takes longer.
STAR runs much faster ( less than 40 minutes) but needs a lot more memory (64 cores 128G ram).

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Very long tophat run	seq_newbie	Bioinformatics	3	04-15-2014 08:02 AM
Mi-Seq Run Time?	bwking82	Illumina/Solexa	0	01-09-2012 08:46 AM
How to run Tophat with annotation file	masylichu	Bioinformatics	2	09-06-2011 08:25 PM
TopHat approximate run time & memory usage?	xinchen	Bioinformatics	4	05-18-2010 03:47 AM
SOLiD 4 run time	Bruins	SOLiD	3	03-04-2010 01:34 AM

06-21-2010, 06:29 AM	#1
bassu Junior Member Location: India Join Date: Jun 2010 Posts: 5	Tophat Run time Dear all, I am currently running tophat to align human genome reference(hg19 bowtie index) to illumina pair-end data of size 17Gb each(17*2). Its almost 24 houurs and its still running. I wonder how much more time it would take to finish the process? . Concurrently i am also running Maq for 75bp single read of size 15GB with a reference size of 30mb(binary human genome file). My system configuration is 64Gb Ram with 8 core processor(Which i feel is one among best configuration available in industry. Do i need to update my system configuration for NGS data analysis?If so please provide me the config). And i also like to know how much processing time it would take if i run the Tophat and Maq separately? Hopping for a speedy reply asap. Thanks

06-21-2010, 09:48 AM	#4
john_mu Member Location: Stanford, CA Join Date: May 2010 Posts: 88	Did you run TopHat with multiple threads? If you are running it with only one thread 17Gb of reads will take several days to run... What is your read length? Long reads also take much longer than short ones (for the same amount of data). Running TopHat and Maq at the same time should not cause much problems (Unless you ran TopHat with 8 threads) Regarding your system configuration, 64Gb should be plenty of RAM for your amount of data. __________________ SpliceMap: De novo detection of splice junctions from RNA-seq Download SpliceMap Comment here

06-21-2010, 08:03 AM	#2
Rao Member Location: India Join Date: Oct 2008 Posts: 36	try with bowtie... is it RNA-seq data?

06-21-2010, 08:09 AM	#3
lmf_bill Member Location: New Haven Join Date: Jul 2008 Posts: 36	Using tophat, especial for paired-end reads, it will take long time. In my personal point, your configuration is enough. BTW, tophat will produce huge tmp file.

06-22-2010, 01:36 AM	#5
bassu Junior Member Location: India Join Date: Jun 2010 Posts: 5	Thanks all, @Rao: yes i am using RNA-seq data. @mf_bill: thanks for your valuable comment, I was wondering whether my system configuration was right ? Even though BWT, tophat will produce huge tmp files.. it will get deleted automatically right? @john_mu: Thanks john, currently i'm running my reads in a single thread. and my read length is 50bp.

10-05-2010, 03:41 PM	#6
DineshCyanam Compendia Bio Location: Ann Arbor Join Date: Oct 2010 Posts: 35	@Bassu: So how long did it take for you to finally finish the run? Were u able to reduce the run time in any way? I am having the same problem here...

10-06-2010, 09:44 AM	#7
mrawlins Member Location: Retirement - Not working with bioinformatics anymore. Join Date: Apr 2010 Posts: 63	Running with n-1 threads on an n-core machine (e.g. 7 threads on a machine with 8 cores) should speed things up. Bowtie has a --shmem option for using shared memory for all threads, so that shouldn't increase the memory footprint by much to use that many threads. I've observed roughly a linear speed up with the number of cores dedicated to bowtie; I suspect similar results for tophat. I will sometimes run with as many threads as cores, but only if I don't intend to use the computer for anything else while the program runs (i.e. a compute node on our analysis cluster).

09-17-2013, 11:34 AM	#8
crazyhottommy Senior Member Location: Gainesville Join Date: Apr 2012 Posts: 140	I was running Tophat to map a 24G single end RNA-seq fastq to hg19 with the gtf from GENECODE I run it in a cluster with 1 node, 8 processors, ram=3gb it took me 45hrs to finish..... Any way to speed it up? as a regular user of the cluster, the above setting is the max resource I can have.

09-17-2013, 03:54 PM	#10
shi Wei Shi Location: Australia Join Date: Feb 2010 Posts: 235	You may try Subread, which is >10 times faster.

09-17-2013, 10:59 PM	#12
adamyao Member Location: Taiwan Join Date: Feb 2011 Posts: 19	20.8G bases (104M reads, Pair End) 1 node (16 cores) AMD 2.4GHz 96G ram - 28 hours Tophat 2 runs best with 16 cores ( single node) according to our tests otherwise it takes longer. STAR runs much faster ( less than 40 minutes) but needs a lot more memory (64 cores 128G ram).