SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Empty instance in cummeRbund rubbertjes Bioinformatics 1 07-30-2013 06:42 PM
Samtools mpileup multi thread? bioinfun Bioinformatics 4 05-16-2012 07:08 PM
running samtools mpileup with nohup -redirect STDERR? 12jrowley2 Bioinformatics 1 08-09-2011 10:24 AM
Computer for running GS Junior Software Martin Mikkelsen 454 Pyrosequencing 7 01-03-2011 04:37 PM
gigaBayes (MarthLab) is running very slowly sdriscoll Bioinformatics 1 11-03-2009 03:49 PM

Reply
 
Thread Tools
Old 07-18-2013, 12:08 PM   #1
Heisman
Senior Member
 
Location: St. Louis

Join Date: Dec 2010
Posts: 535
Default Should one instance of mpileup use a full thread? Computer running slowly.

Hey all,

I think my computer is running slowly and I have no idea why. I'm also sadly not wonderfully computer savvy so I'm not sure how to troubleshoot this obviously. Are there some basic things to look at (if so please include commands to run if you know them off the top of your head)? In the past I've run mpileup on it and it usually uses a full thread and is fine and right now it's hovering at around 15-20% CPU by looking at it with htop. Thanks for any help.
Heisman is offline   Reply With Quote
Old 07-18-2013, 12:27 PM   #2
Heisman
Senior Member
 
Location: St. Louis

Join Date: Dec 2010
Posts: 535
Default

Figured it out, good learning experience: I was using a bash script and piping the output of mpileup into multiple commands. I think it was splitting up the thread amongst all of the commands. Maybe that's not what was going on but making mpileup separate and then running the others has made mpileup take up a whole thread like I wanted it to.
Heisman is offline   Reply With Quote
Old 07-18-2013, 02:04 PM   #3
Heisman
Senior Member
 
Location: St. Louis

Join Date: Dec 2010
Posts: 535
Default

Actually it's still doing it; driving me crazy. I'm running four instances of mpileup and each is using ~25% of a CPU, this computer has 64 GB RAM and 24 cores. Free -m shows ~50 GB RAM is being used. I have no idea how to make each instance use 100% of a CPU, if possible.
Heisman is offline   Reply With Quote
Old 07-18-2013, 02:22 PM   #4
jimmybee
Senior Member
 
Location: Adelaide, Australia

Join Date: Sep 2010
Posts: 119
Default

Have you tried using each command outside the bash script? or perhaps, use something like parallel (https://en.wikipedia.org/wiki/GNU_parallel)?

I havent used it as such but I know a few people are implementing it in everyday scripts to pretty good effect
jimmybee is offline   Reply With Quote
Old 07-18-2013, 04:57 PM   #5
Heisman
Senior Member
 
Location: St. Louis

Join Date: Dec 2010
Posts: 535
Default

I tried running them stand alone and the same thing happened. One thing holding me back is I don't have any background or general knowledge in how memory works in general (of course I don't know it's a memory problem). I'm rather ignorant in this area. They will spike up to ~100% every now and then but then go back down to 15-40%. Right now if I do free -m I get:

Code:
             total       used       free     shared    buffers     cached
Mem:         64399      54764       9635          0         86      51385
-/+ buffers/cache:       3292      61107
Swap:       123977          0     123977
No idea what's going on, haha.
Heisman is offline   Reply With Quote
Old 07-18-2013, 05:08 PM   #6
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

Your cores are probably waiting on the storage subsystem to deliver data. Unless you are using server grade storage/SSD's in RAID this is probably what is holding your system back.

Check on "these performance metrics"
GenoMax is offline   Reply With Quote
Old 07-19-2013, 04:12 AM   #7
lh3
Senior Member
 
Location: Boston

Join Date: Feb 2008
Posts: 693
Default

In addition to the storage issue mentioned by GenoMax, the commands following mpileup may be the bottleneck if you pipe. Mpileup may be waiting for those commands.
lh3 is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 09:35 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO