Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Problem with SSPACE, multi-threading and memory restrictions

    Hi,
    I am attempting to use SSPACE version 3 to re-scaffold some mRNA contigs using paired-end RNA-Seq data. While SSPACe was designed for rescaffolding genomes, I see no reason why it should not work in similar fashion for re-scaffolding de novo assembled RNA contigs.

    SSPACE is discussed in seqanswers here:
    Discussion of next-gen sequencing related bioinformatics: resources, algorithms, open source efforts, etc


    The SSPACE webpage can be reached from here:
    BaseClear offers a wide range of advanced bioinformatics and biostatistics solutions, primarily in the areas of genomics and microbiomics data analysis.


    I consistently get the following error message:
    =>date: Mapping reads to contigs with Bowtie
    Thread 8 terminated abnormally: Can't open bwa output -- fatal
    Out of memory!
    Process 'extend/format contigs' failed on date

    with:
    resources_used.mem=47,424,992kb
    resources_used.vmem=64,093,240kb
    resources_used.walltime=00:34:33

    It seems to me that SSPACE is running out of physical memory while trying to open the bwa output.

    The "Thread 8 terminated abnormally" is especially cryptic, as the program was set to only run with 6 threads (-T 6).

    My run parameters are:
    Code:
    #PBS -l nodes=1:ppn=6,mem=47gb,walltime=72:00:00
    SSPACE_FILE=${HOME}/src/SSPACE-STANDARD-3.0_linux-x86_64/SSPACE_Standard_v3.0.pl 
    LIBRARY_FILE=/[I]filepath[/I]/library_file_2.txt
    CONTIG_FILE=/[I]filepath[/I]/A_planci_pcg_transdec_MePath2Renam_echinoHomology.fasta		#-s option, contigs that we are scaffolding
    MIN_LINKS=10		   	# -k 10
    THREADS=6			    # -T, threads
    SKIP=0				    #-S 0=no, -S 1=yes, skip processing of reads
    EXTEND_CONTIGS=1		#-x, extend contigs using sequence data, 0=no, 1=yes, default 0
    VERBOSE=1			    #-v  Runs the scaffolding process in verbose mode (-v 1=yes, -v 0=no, default -v 0, optional)
    $SSPACE_FILE -l $LIBRARY_FILE -s $CONTIG_FILE -k $MIN_LINKS -T $THREADS -S $SKIP -x $EXTEND_CONTIGS -v $VERBOSE
    The supporting library file is attached.

    I previously tried to run SSPACE over multiple nodes, but it seemed to use the cores on only one of the nodes.

    Does anyone know of a way to get SSPACE to run across multiple nodes? Can it be run with mpi multi-threading? If not, does anyone have any suggestions for reducing memory requirements?
    Attached Files
    Last edited by GenoMax; 10-06-2015, 07:50 AM. Reason: added CODE tags to improve display

  • #2
    Hi gwilymh,

    Did you solve your problem?
    I encounter the same error, i would love to beneficit from your help if you found a solution.

    cheers,

    Comment


    • #3
      Did you solve your problem?

      Originally posted by gwilymh View Post
      Hi,
      I am attempting to use SSPACE version 3 to re-scaffold some mRNA contigs using paired-end RNA-Seq data. While SSPACe was designed for rescaffolding genomes, I see no reason why it should not work in similar fashion for re-scaffolding de novo assembled RNA contigs.

      SSPACE is discussed in seqanswers here:
      Discussion of next-gen sequencing related bioinformatics: resources, algorithms, open source efforts, etc


      The SSPACE webpage can be reached from here:
      BaseClear offers a wide range of advanced bioinformatics and biostatistics solutions, primarily in the areas of genomics and microbiomics data analysis.


      I consistently get the following error message:
      =>date: Mapping reads to contigs with Bowtie
      Thread 8 terminated abnormally: Can't open bwa output -- fatal
      Out of memory!
      Process 'extend/format contigs' failed on date

      with:
      resources_used.mem=47,424,992kb
      resources_used.vmem=64,093,240kb
      resources_used.walltime=00:34:33

      It seems to me that SSPACE is running out of physical memory while trying to open the bwa output.

      The "Thread 8 terminated abnormally" is especially cryptic, as the program was set to only run with 6 threads (-T 6).

      My run parameters are:
      Code:
      #PBS -l nodes=1:ppn=6,mem=47gb,walltime=72:00:00
      SSPACE_FILE=${HOME}/src/SSPACE-STANDARD-3.0_linux-x86_64/SSPACE_Standard_v3.0.pl 
      LIBRARY_FILE=/[I]filepath[/I]/library_file_2.txt
      CONTIG_FILE=/[I]filepath[/I]/A_planci_pcg_transdec_MePath2Renam_echinoHomology.fasta		#-s option, contigs that we are scaffolding
      MIN_LINKS=10		   	# -k 10
      THREADS=6			    # -T, threads
      SKIP=0				    #-S 0=no, -S 1=yes, skip processing of reads
      EXTEND_CONTIGS=1		#-x, extend contigs using sequence data, 0=no, 1=yes, default 0
      VERBOSE=1			    #-v  Runs the scaffolding process in verbose mode (-v 1=yes, -v 0=no, default -v 0, optional)
      $SSPACE_FILE -l $LIBRARY_FILE -s $CONTIG_FILE -k $MIN_LINKS -T $THREADS -S $SKIP -x $EXTEND_CONTIGS -v $VERBOSE
      The supporting library file is attached.

      I previously tried to run SSPACE over multiple nodes, but it seemed to use the cores on only one of the nodes.

      Does anyone know of a way to get SSPACE to run across multiple nodes? Can it be run with mpi multi-threading? If not, does anyone have any suggestions for reducing memory requirements?
      I met the same problem. Did you solve your problem?
      If you have, could you share your experiences?
      Thanks

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Investigating the Gut Microbiome Through Diet and Spatial Biology
        by seqadmin




        The human gut contains trillions of microorganisms that impact digestion, immune functions, and overall health1. Despite major breakthroughs, we’re only beginning to understand the full extent of the microbiome’s influence on health and disease. Advances in next-generation sequencing and spatial biology have opened new windows into this complex environment, yet many questions remain. This article highlights two recent studies exploring how diet influences microbial...
        02-24-2025, 06:31 AM
      • seqadmin
        Quality Control Essentials for Next-Generation Sequencing Workflows
        by seqadmin




        Like all molecular biology applications, next-generation sequencing (NGS) workflows require diligent quality control (QC) measures to ensure accurate and reproducible results. Proper QC begins at nucleic acid extraction and continues all the way through to data analysis. This article outlines the key QC steps in an NGS workflow, along with the commonly used tools and techniques.

        Nucleic Acid Quality Control
        Preparing for NGS starts with isolating the...
        02-10-2025, 01:58 PM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, 03-03-2025, 01:15 PM
      0 responses
      46 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 02-28-2025, 12:58 PM
      0 responses
      167 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 02-24-2025, 02:48 PM
      0 responses
      525 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 02-21-2025, 02:46 PM
      0 responses
      256 views
      0 likes
      Last Post seqadmin  
      Working...
      X