SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
BLAST+ creating custom blast database and using blast+ filtering features deniz Bioinformatics 2 10-26-2012 11:04 AM
Max possible threads with Licensed novoalign arkal Bioinformatics 6 07-27-2011 06:14 AM
3k threads and almost 7k members ECO Site Announcements 1 02-04-2010 10:15 PM
FastQ drop (illumina) strob Bioinformatics 2 07-14-2009 10:09 AM
forum threads bioinfosm Site Feedback/Suggestions 2 03-19-2009 11:57 AM

Reply
 
Thread Tools
Old 06-29-2010, 08:25 AM   #1
k-gun12
Member
 
Location: NJ

Join Date: Feb 2010
Posts: 51
Default Blast threads always drop to 1

I've seen this issue with every version of blast+ that I've used recently. When I run jobs on a multi-core machine, I specify -num_threads XX to speed things up. Invariably, no matter how many threads I specify, after a short time it seems as though only 1 thread is active on the machine. I've compiled the blast binaries myself using both gcc and the intel icc compilers. When I start the job, top shows blastn/p/x using, say, 800% of processor if I specify 8 threads. After a few minutes this drops to 100%. The job completes in less time than a single-threaded job, but not by much. Is this normal behavior?

Thanks!
k-gun12 is offline   Reply With Quote
Old 06-29-2010, 09:46 AM   #2
nickloman
Senior Member
 
Location: Birmingham, UK

Join Date: Jul 2009
Posts: 356
Default

One thought that sprung to mind is whether BLAST is I/O limited rather than CPU limited. I guess this might happen if you had a very large database and slow disks. vmstat/iostat might help determine this.
nickloman is offline   Reply With Quote
Old 06-29-2010, 09:49 AM   #3
k-gun12
Member
 
Location: NJ

Join Date: Feb 2010
Posts: 51
Default

Here's the official response from NCBI. Only some of the code is multithreaded..


"BLAST search has three distinctive stages: word matching with database scan, ungapped alignment, gapped alignment with traceback.

As I understand it Only the word match stage is multi-threaded. So what you described make sense and it correct."
k-gun12 is offline   Reply With Quote
Old 06-29-2010, 09:58 AM   #4
nickloman
Senior Member
 
Location: Birmingham, UK

Join Date: Jul 2009
Posts: 356
Default

That does indeed make sense. Sounds like it might be a bottleneck if your search returns a lot of matches that need aligning.
nickloman is offline   Reply With Quote
Old 07-02-2010, 08:50 AM   #5
sjmillerAZ
Junior Member
 
Location: Tucson, AZ

Join Date: Oct 2008
Posts: 3
Default blast+ vs older blastall

I've seen the same behavior for blast+ vs the older blastall. In section 4.5 of the NCBI user manual for blast+ they show a performance improvement over blastall for queries of length 10Kb - 10Mb, but for shorter queries my experience is that blast+ is much, much slower. When I run a blastx with 50 DNA queries of average length 1135 against a protein database of 475000 sequences (161M total letters) using 8 cpus, the blastall 2.2.18 code finishes the run in just under 5 hours with cpu usage 764%. The same blastx with blast+ version 2.2.23 is still running after 16 hours and has only finished 14000 queries, and the cpu usage shows 243% for 8 processors. Needless to say, I won't be encouraging anyone with shorter queries to use blast+ until this problem has been fixed.
sjmillerAZ is offline   Reply With Quote
Old 07-23-2010, 05:20 AM   #6
aleferna
Senior Member
 
Location: sweden

Join Date: Sep 2009
Posts: 121
Default

Well if all fails here's a simple little script to Multithread any application, I use it whenever I have to use blat or maq which don't have multithread support. It counts how many processes have been started and if there are less than the number of threads (24 in the script below) it starts a new one. All you need to do is cut your reads into many pieces.


for x in *.fa
do

while [ $(ps -Af | grep "blast" | wc -l) -gt 24 ]
do
sleep 5
done

blast $x &...
sleep 1

done
aleferna is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 06:29 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO