I am using Newbler 2.3 on a 8 core workstation with 64 gb memory (running Ubuntu). It seems that during my newbler assemblies, memory usage is limited to ~20 gb. Is there some reason newbler wont use all the memory available (the processors usage is not limiting, they could be running at 10% and still memory usage wont go over ~22 gb)?
Seqanswers Leaderboard Ad
Collapse
Announcement
Collapse
No announcement yet.
X
-
This is during the "reading flowgrams" step when it is generating an output, it is the most time intensive step of my assemblies (these are eukaryote sized assemblies). There is both available CPU and memory, so it should be able to go faster, but something seems to be holding it back. The CPU usage is very low (~10% for each processor). The memory always tops out at ~ 20-21 gigs, even though that is only about a third of the available memory. With Celera assembler, you can change the memory limits during compiling the source, otherwise you are limited in your max memory usage, I didn't know if Newbler had similar limits.
Comment
-
Originally posted by smg283 View PostIs there some reason newbler wont use all the memory available ...?
There is both available CPU and memory, so it should be able to go faster, but something seems to be holding it back.
None of your observations sound unusual or unexpected to me for a program like Newbler.
Comment
-
-
Originally posted by smg283 View PostThis is during the "reading flowgrams" step when it is generating an output, it is the most time intensive step of my assemblies (these are eukaryote sized assemblies). There is both available CPU and memory, so it should be able to go faster, but something seems to be holding it back. The CPU usage is very low (~10% for each processor). The memory always tops out at ~ 20-21 gigs, even though that is only about a third of the available memory. With Celera assembler, you can change the memory limits during compiling the source, otherwise you are limited in your max memory usage, I didn't know if Newbler had similar limits.
Try to force run only on a single CPU core. Is it newbler or gsRunProcessor which which into some of the log files that it will split the memory between the forked threads? I forgot ... And, there is also "-m" commandline switch to force in-memory computation, if my memory serves me right. ;-)
Comment
Latest Articles
Collapse
-
by seqadmin
Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...-
Channel: Articles
04-04-2024, 04:25 PM -
-
by seqadmin
Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...-
Channel: Articles
03-22-2024, 06:39 AM -
ad_right_rmr
Collapse
News
Collapse
Topics | Statistics | Last Post | ||
---|---|---|---|---|
Started by seqadmin, 04-11-2024, 12:08 PM
|
0 responses
30 views
0 likes
|
Last Post
by seqadmin
04-11-2024, 12:08 PM
|
||
Started by seqadmin, 04-10-2024, 10:19 PM
|
0 responses
32 views
0 likes
|
Last Post
by seqadmin
04-10-2024, 10:19 PM
|
||
Started by seqadmin, 04-10-2024, 09:21 AM
|
0 responses
28 views
0 likes
|
Last Post
by seqadmin
04-10-2024, 09:21 AM
|
||
Started by seqadmin, 04-04-2024, 09:00 AM
|
0 responses
53 views
0 likes
|
Last Post
by seqadmin
04-04-2024, 09:00 AM
|
Comment