I'm using Tophat to align ~33 million 100 bp paired end reads to the mouse genome using -p 8 and and when it gets about 1 hour into "Searching for junctions via segment mapping" memory usage jumps from a few GB to almost 20 GB which exceeds the 16GB on my computer and starts using the swap file. I'm also using the "butterfly-search" which the Tophat manual says will slow slow down the alignment but doesn't say it should use more memory. Is this amount of memory usage normal or did something go wrong?
Seqanswers Leaderboard Ad
Collapse
Announcement
Collapse
No announcement yet.
X
-
Same problem
I'm running Tophat2 on two 389MB paired end reads files, on 8 cores with 2.5G vmem per core (=20GB total) in SGE, using 16 threads.
The alignments go swimmingly (1 hour), then it takes ~19.7G of memory and 5+ hours to run the "Searching for junctions via segment mapping" step (my understanding is that the multiple threads are not used here). Then it makes it to "Mapping left_kept_reads_seg1 against segment_juncs with Bowtie2 (1/4)", but gets booted from the queue, probably due to memory usage.
If I run it with 4 cores, 16 threads, it gets booted at the "Searching for junctions via segment mapping" step (4 cores = only 10GB of memory).
I'm willing to go up to 10-12 cores, but looking for advice first. Any you care to provide would be greatly appreciated.
Thanks,
Robin
-
A slightly different memory issue
I am test driving TopHat 2.0 on a linux cluster.
I prepared 4 sample sets, which contain 13 million to 18 million 50bp paired reads. Tophat was called to process them one by one. I initialized the program with 8GB memory and 8 threads.
The first dataset was processed successfully using almost 2 hr. However, when the second dataset reached the step of "Building Bowtie index from genes.fa", the memory usage went above 18GB, so the system admin terminated my job.
So I re-started my job to only run TopHat 2.0 on dataset 2 using the same system setting. The job finished flawlessly.
I am wondering whether TopHat 2.0 has some memory issues.
Comment
-
Did anyone find a solution to this?
I'm mapping about 30 million 50bp reads to the human genome and during the "searching for junctions via segment mapping" step my memory usage sky rockets. About 2-4GB of memory is used before, then 24-25GB of memory is used and the run finally fails at the "Joining segment hits" step by exceeding my 32GB of memory. This is with the latest tophat2 and bowtie2.
Thanks for any help.
Comment
-
Any more info regarding this? I just realised that my first 14 jobs that use --coverage-search --microexon-search have gone into junction search and they have started gobbling up memory (16-20Gb per sample). If they arnt finished when the next batch of 14 or even worse third batch of 14 comes to the same position I'm not certain that the servers got enough memory...
Comment
-
Originally posted by pettervikman View PostAny more info regarding this? I just realised that my first 14 jobs that use --coverage-search --microexon-search have gone into junction search and they have started gobbling up memory (16-20Gb per sample). If they arnt finished when the next batch of 14 or even worse third batch of 14 comes to the same position I'm not certain that the servers got enough memory...
Comment
Latest Articles
Collapse
-
by seqadmin
Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...-
Channel: Articles
03-22-2024, 06:39 AM -
-
by seqadmin
The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.
Avian Conservation
Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...-
Channel: Articles
03-08-2024, 10:41 AM -
ad_right_rmr
Collapse
News
Collapse
Topics | Statistics | Last Post | ||
---|---|---|---|---|
Started by seqadmin, 03-27-2024, 06:37 PM
|
0 responses
13 views
0 likes
|
Last Post
by seqadmin
03-27-2024, 06:37 PM
|
||
Started by seqadmin, 03-27-2024, 06:07 PM
|
0 responses
11 views
0 likes
|
Last Post
by seqadmin
03-27-2024, 06:07 PM
|
||
Started by seqadmin, 03-22-2024, 10:03 AM
|
0 responses
53 views
0 likes
|
Last Post
by seqadmin
03-22-2024, 10:03 AM
|
||
Started by seqadmin, 03-21-2024, 07:32 AM
|
0 responses
69 views
0 likes
|
Last Post
by seqadmin
03-21-2024, 07:32 AM
|
Comment