Tweet
Hello,
I am trying to get PolyPhen2's run_pph.pl to run our linux cluster using PBS/Torque. The README file gives examples of how to run this using Grid Engine and LSF, but not for PBS.
I am attempting to write a script that will automate running this with PBS, but I was wondering if anybody had any experience with the best way to do it.
I've found in the past, when working with genomics tools that attempt to break files up into X pieces to run on X cluster nodes, that the process of breaking up and copying the files can be monumentally slow. For example, I would like to try and run this on our 28 node cluster, each node having 8 processors.
So the question becomes, do I submit 28 8-node jobs and then have each node break up their portion individually and recombine the results (then recombine all 28 at the end) or do I submit 28*8 = 224 individual run_pph.pl scripts and combine all 224 outputs at the end?
I'll probably end up trying both to see which is faster. I'm guessing the former. (Anything you can localize is probably better.)
Thanks.
I am trying to get PolyPhen2's run_pph.pl to run our linux cluster using PBS/Torque. The README file gives examples of how to run this using Grid Engine and LSF, but not for PBS.
I am attempting to write a script that will automate running this with PBS, but I was wondering if anybody had any experience with the best way to do it.
I've found in the past, when working with genomics tools that attempt to break files up into X pieces to run on X cluster nodes, that the process of breaking up and copying the files can be monumentally slow. For example, I would like to try and run this on our 28 node cluster, each node having 8 processors.
So the question becomes, do I submit 28 8-node jobs and then have each node break up their portion individually and recombine the results (then recombine all 28 at the end) or do I submit 28*8 = 224 individual run_pph.pl scripts and combine all 224 outputs at the end?
I'll probably end up trying both to see which is faster. I'm guessing the former. (Anything you can localize is probably better.)
Thanks.