Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Problem with SSPACE, multi-threading and memory restrictions

    Hi,
    I am attempting to use SSPACE version 3 to re-scaffold some mRNA contigs using paired-end RNA-Seq data. While SSPACe was designed for rescaffolding genomes, I see no reason why it should not work in similar fashion for re-scaffolding de novo assembled RNA contigs.

    SSPACE is discussed in seqanswers here:
    Discussion of next-gen sequencing related bioinformatics: resources, algorithms, open source efforts, etc


    The SSPACE webpage can be reached from here:
    Our state-of-the-art bioinformatics and biostatistics solutions guarantee high-quality results and clear answers to your research questions.


    I consistently get the following error message:
    =>date: Mapping reads to contigs with Bowtie
    Thread 8 terminated abnormally: Can't open bwa output -- fatal
    Out of memory!
    Process 'extend/format contigs' failed on date

    with:
    resources_used.mem=47,424,992kb
    resources_used.vmem=64,093,240kb
    resources_used.walltime=00:34:33

    It seems to me that SSPACE is running out of physical memory while trying to open the bwa output.

    The "Thread 8 terminated abnormally" is especially cryptic, as the program was set to only run with 6 threads (-T 6).

    My run parameters are:
    Code:
    #PBS -l nodes=1:ppn=6,mem=47gb,walltime=72:00:00
    SSPACE_FILE=${HOME}/src/SSPACE-STANDARD-3.0_linux-x86_64/SSPACE_Standard_v3.0.pl 
    LIBRARY_FILE=/[I]filepath[/I]/library_file_2.txt
    CONTIG_FILE=/[I]filepath[/I]/A_planci_pcg_transdec_MePath2Renam_echinoHomology.fasta		#-s option, contigs that we are scaffolding
    MIN_LINKS=10		   	# -k 10
    THREADS=6			    # -T, threads
    SKIP=0				    #-S 0=no, -S 1=yes, skip processing of reads
    EXTEND_CONTIGS=1		#-x, extend contigs using sequence data, 0=no, 1=yes, default 0
    VERBOSE=1			    #-v  Runs the scaffolding process in verbose mode (-v 1=yes, -v 0=no, default -v 0, optional)
    $SSPACE_FILE -l $LIBRARY_FILE -s $CONTIG_FILE -k $MIN_LINKS -T $THREADS -S $SKIP -x $EXTEND_CONTIGS -v $VERBOSE
    The supporting library file is attached.

    I previously tried to run SSPACE over multiple nodes, but it seemed to use the cores on only one of the nodes.

    Does anyone know of a way to get SSPACE to run across multiple nodes? Can it be run with mpi multi-threading? If not, does anyone have any suggestions for reducing memory requirements?
    Attached Files
    Last edited by GenoMax; 10-06-2015, 07:50 AM. Reason: added CODE tags to improve display

  • #2
    Hi gwilymh,

    Did you solve your problem?
    I encounter the same error, i would love to beneficit from your help if you found a solution.

    cheers,

    Comment


    • #3
      Did you solve your problem?

      Originally posted by gwilymh View Post
      Hi,
      I am attempting to use SSPACE version 3 to re-scaffold some mRNA contigs using paired-end RNA-Seq data. While SSPACe was designed for rescaffolding genomes, I see no reason why it should not work in similar fashion for re-scaffolding de novo assembled RNA contigs.

      SSPACE is discussed in seqanswers here:
      Discussion of next-gen sequencing related bioinformatics: resources, algorithms, open source efforts, etc


      The SSPACE webpage can be reached from here:
      Our state-of-the-art bioinformatics and biostatistics solutions guarantee high-quality results and clear answers to your research questions.


      I consistently get the following error message:
      =>date: Mapping reads to contigs with Bowtie
      Thread 8 terminated abnormally: Can't open bwa output -- fatal
      Out of memory!
      Process 'extend/format contigs' failed on date

      with:
      resources_used.mem=47,424,992kb
      resources_used.vmem=64,093,240kb
      resources_used.walltime=00:34:33

      It seems to me that SSPACE is running out of physical memory while trying to open the bwa output.

      The "Thread 8 terminated abnormally" is especially cryptic, as the program was set to only run with 6 threads (-T 6).

      My run parameters are:
      Code:
      #PBS -l nodes=1:ppn=6,mem=47gb,walltime=72:00:00
      SSPACE_FILE=${HOME}/src/SSPACE-STANDARD-3.0_linux-x86_64/SSPACE_Standard_v3.0.pl 
      LIBRARY_FILE=/[I]filepath[/I]/library_file_2.txt
      CONTIG_FILE=/[I]filepath[/I]/A_planci_pcg_transdec_MePath2Renam_echinoHomology.fasta		#-s option, contigs that we are scaffolding
      MIN_LINKS=10		   	# -k 10
      THREADS=6			    # -T, threads
      SKIP=0				    #-S 0=no, -S 1=yes, skip processing of reads
      EXTEND_CONTIGS=1		#-x, extend contigs using sequence data, 0=no, 1=yes, default 0
      VERBOSE=1			    #-v  Runs the scaffolding process in verbose mode (-v 1=yes, -v 0=no, default -v 0, optional)
      $SSPACE_FILE -l $LIBRARY_FILE -s $CONTIG_FILE -k $MIN_LINKS -T $THREADS -S $SKIP -x $EXTEND_CONTIGS -v $VERBOSE
      The supporting library file is attached.

      I previously tried to run SSPACE over multiple nodes, but it seemed to use the cores on only one of the nodes.

      Does anyone know of a way to get SSPACE to run across multiple nodes? Can it be run with mpi multi-threading? If not, does anyone have any suggestions for reducing memory requirements?
      I met the same problem. Did you solve your problem?
      If you have, could you share your experiences?
      Thanks

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Recent Advances in Sequencing Analysis Tools
        by seqadmin


        The sequencing world is rapidly changing due to declining costs, enhanced accuracies, and the advent of newer, cutting-edge instruments. Equally important to these developments are improvements in sequencing analysis, a process that converts vast amounts of raw data into a comprehensible and meaningful form. This complex task requires expertise and the right analysis tools. In this article, we highlight the progress and innovation in sequencing analysis by reviewing several of the...
        Yesterday, 07:48 AM
      • seqadmin
        Essential Discoveries and Tools in Epitranscriptomics
        by seqadmin




        The field of epigenetics has traditionally concentrated more on DNA and how changes like methylation and phosphorylation of histones impact gene expression and regulation. However, our increased understanding of RNA modifications and their importance in cellular processes has led to a rise in epitranscriptomics research. “Epitranscriptomics brings together the concepts of epigenetics and gene expression,” explained Adrien Leger, PhD, Principal Research Scientist...
        04-22-2024, 07:01 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, Yesterday, 07:17 AM
      0 responses
      11 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 05-02-2024, 08:06 AM
      0 responses
      19 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-30-2024, 12:17 PM
      0 responses
      20 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 04-29-2024, 10:49 AM
      0 responses
      28 views
      0 likes
      Last Post seqadmin  
      Working...
      X