Help is needed with openmpi and Ray on cluster

[yaximik]

Hi,

I run a small Ray job on a cluster to check openmpi and Ray. The current versions on the cluster are openmpi-1.6.4 and Ray is 2.1.

When compiled on my server with openmpi-1.4, Ray2.1 works just fine with large metazoan dataset (about 30 GB) completing assembly in about 20 hours on machine with 2 quad core E5620 (16 threads) and 96GB memory

On the cluster Ray2.1 compiled fine as well, and then I tried to run it with one of datasets (about 36 MB) from the above pile that assembled fine on my server. It failed, generating ouput, which I am not quite understand and would appreciate help in interpretation and fixing the problem. Here is Ray.job file:

#$ -S /bin/bash
#$ -cwd
#
#$ -N RayTest
#$ -pe openmpi 32
#$ -l h_rt=00:10:00,s_rt=00:08:00,h_vmem=3G
#$ -j y
#
#$ -M [email protected]
#$ -m eas
#
# Load the appropriate module files
# Should be loaded already
#$ -V

mpiexec -n $NSLOTS ~/bin/Ray ~/jobs/Ray/Ray.conf

Here is Ray.conf file:

# configuration file
-k 31 # k-mer=31

# default autocalc averageOuterDistance standardDeviation
# all reads trimmed, Q30, FGMGPGIGVA

-s <TestFile.fasta>                    # single read file

-o ~/jobs/Ray/RayOutput         # outputDirectory

-one-color-per-file                # Sets one color per file instead of one per sequence
-amos                           # writes the AMOS file called RayOutput/AMOS.afg
#-show-memory-usage              # Shows memory usage
# -show-memory-allocations        # Shows memory allocation events
#-write-checkpoints ~/SC/Ray/Checkpoints   # Write checkpoint files

The ouput file is a bit loger, so it is attached.

[severin]

make options

Did you specify your MPICXX during compile?

make PREFIX=`pwd` MAXKMERLENGTH=99 MPICXX='/shared/openmpi/intel/bin/mpicxx'

also I use mpirun -np numberofcpu

[yaximik]

[SOLVED] Help is needed with openmpi and Ray on cluster

The cluster guys quickly pointed my error - I compiled Ray with an earlier existing version of openmpi before the latest version was installed on the cluster. I recompuiled it and all works fine now. Thanks to everyone who read my post and responded or was going to respond!