SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Bowtie2 freezes with 7.5GB RAM sphil Bioinformatics 0 10-15-2013 05:50 AM
Transcriptome assembly using 12GB RAM? ckidner Bioinformatics 7 09-14-2012 03:18 AM
Too many short reads and too little RAM? samanta General 5 09-22-2011 05:48 AM
Hybrid assembly w/ minimal RAM blindtiger454 De novo discovery 2 10-27-2010 08:02 PM
assembly and RAM aurelielaugraud Bioinformatics 0 09-23-2009 06:02 AM

Reply
 
Thread Tools
Old 02-11-2014, 07:39 AM   #1
starbug16
Junior Member
 
Location: London

Join Date: Feb 2014
Posts: 8
Default Trinity RAM issue

Hi,

Having managed to whittle down my Illumina data to 31 million reads, I am trying to run Trinity with 16 CPUs and 16 gigabytes of RAM. Everything works fine until I get to Butterfly which seems to bog down the server so much so that I can't even connect in to check progress and end up having to kill the process.

Guessing that I don't have enough RAM. Does anyone have any suggestions on how to work around this???

Thanks in advance
starbug16 is offline   Reply With Quote
Old 02-11-2014, 07:48 AM   #2
Birdman
Member
 
Location: Montreal

Join Date: Jan 2014
Posts: 21
Default

What is your Trinity command line?
Birdman is offline   Reply With Quote
Old 02-11-2014, 07:55 AM   #3
Wallysb01
Senior Member
 
Location: San Francisco, CA

Join Date: Feb 2011
Posts: 286
Default

For butterfly you’re going to have to drop the CPUs and RAM usage down from defaults or from what you set in other parts of the trinity run. You can control that with:

--bflyHeapSpaceMax
--bflyCPU

You should probably just be running with two CPUs and 8GB of ram set as the bflyHeapSpaceMax
Wallysb01 is offline   Reply With Quote
Old 02-11-2014, 09:36 AM   #4
starbug16
Junior Member
 
Location: London

Join Date: Feb 2014
Posts: 8
Default

Thanks, will definitely give that a try. Probably a how long is a piece of string question but...how long do you think it will take to complete?
starbug16 is offline   Reply With Quote
Old 02-11-2014, 09:43 AM   #5
Birdman
Member
 
Location: Montreal

Join Date: Jan 2014
Posts: 21
Default

I really depends on the amount of reads you have. As an example, in my experience using the latest Trinity version, 1 billion reads took 6-7 days using 64 cores and 256G memory.
Birdman is offline   Reply With Quote
Old 02-11-2014, 01:16 PM   #6
Wallysb01
Senior Member
 
Location: San Francisco, CA

Join Date: Feb 2011
Posts: 286
Default

with only 2 cores running and 31M reads, hopefully run time is not much more than 24 hours.
Wallysb01 is offline   Reply With Quote
Old 02-11-2014, 10:41 PM   #7
starbug16
Junior Member
 
Location: London

Join Date: Feb 2014
Posts: 8
Default

OK, so running on 2 cores and it's run 0.8% of commands overnight. Guessing it may take a while Not sure there is a coffee break long enough for this run
starbug16 is offline   Reply With Quote
Old 02-12-2014, 07:59 AM   #8
Wallysb01
Senior Member
 
Location: San Francisco, CA

Join Date: Feb 2011
Posts: 286
Default

You could drop the RAM usage for each butterfly run more to get 4 cores going instead of 2, but you’ll risk having a lot of jobs to rerun.

Why are you working with only 16GB of RAM but 16 cores? Is this a machine with DDR2 memory? Can you upgrade that RAM? A modern desktop should be able to get you up to 32 GB, and a relatively cheap workstation (i.e. $5000) will give you 16 cores and 128GB of RAM. If you’re going to be doing this much it would be worth it to get something modern or finding a cluster with something better.
Wallysb01 is offline   Reply With Quote
Old 02-12-2014, 09:13 AM   #9
starbug16
Junior Member
 
Location: London

Join Date: Feb 2014
Posts: 8
Default

Ah, just the usual academic institution tug of war between money needing to be spent to be able to do research and them wanting the work in already to justify it. This server was never supposed to be for assembly and they aren't necessarily willing to stump up the cash to add the RAM A month wait for their assembly should do the trick

Thanks for the advice guys, it's much appreciated it's at least running now, just going to take a wee while!
starbug16 is offline   Reply With Quote
Reply

Tags
butterfly, ram, trinity

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 06:04 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO