Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Problem with SSPACE, multi-threading and memory restrictions

    Hi,
    I am attempting to use SSPACE version 3 to re-scaffold some mRNA contigs using paired-end RNA-Seq data. While SSPACe was designed for rescaffolding genomes, I see no reason why it should not work in similar fashion for re-scaffolding de novo assembled RNA contigs.

    SSPACE is discussed in seqanswers here:
    Discussion of next-gen sequencing related bioinformatics: resources, algorithms, open source efforts, etc


    The SSPACE webpage can be reached from here:
    Our state-of-the-art bioinformatics and biostatistics solutions guarantee high-quality results and clear answers to your research questions.


    I consistently get the following error message:
    =>date: Mapping reads to contigs with Bowtie
    Thread 8 terminated abnormally: Can't open bwa output -- fatal
    Out of memory!
    Process 'extend/format contigs' failed on date

    with:
    resources_used.mem=47,424,992kb
    resources_used.vmem=64,093,240kb
    resources_used.walltime=00:34:33

    It seems to me that SSPACE is running out of physical memory while trying to open the bwa output.

    The "Thread 8 terminated abnormally" is especially cryptic, as the program was set to only run with 6 threads (-T 6).

    My run parameters are:
    Code:
    #PBS -l nodes=1:ppn=6,mem=47gb,walltime=72:00:00
    SSPACE_FILE=${HOME}/src/SSPACE-STANDARD-3.0_linux-x86_64/SSPACE_Standard_v3.0.pl 
    LIBRARY_FILE=/[I]filepath[/I]/library_file_2.txt
    CONTIG_FILE=/[I]filepath[/I]/A_planci_pcg_transdec_MePath2Renam_echinoHomology.fasta		#-s option, contigs that we are scaffolding
    MIN_LINKS=10		   	# -k 10
    THREADS=6			    # -T, threads
    SKIP=0				    #-S 0=no, -S 1=yes, skip processing of reads
    EXTEND_CONTIGS=1		#-x, extend contigs using sequence data, 0=no, 1=yes, default 0
    VERBOSE=1			    #-v  Runs the scaffolding process in verbose mode (-v 1=yes, -v 0=no, default -v 0, optional)
    $SSPACE_FILE -l $LIBRARY_FILE -s $CONTIG_FILE -k $MIN_LINKS -T $THREADS -S $SKIP -x $EXTEND_CONTIGS -v $VERBOSE
    The supporting library file is attached.

    I previously tried to run SSPACE over multiple nodes, but it seemed to use the cores on only one of the nodes.

    Does anyone know of a way to get SSPACE to run across multiple nodes? Can it be run with mpi multi-threading? If not, does anyone have any suggestions for reducing memory requirements?
    Attached Files
    Last edited by GenoMax; 10-06-2015, 07:50 AM. Reason: added CODE tags to improve display

  • #2
    Hi gwilymh,

    Did you solve your problem?
    I encounter the same error, i would love to beneficit from your help if you found a solution.

    cheers,

    Comment


    • #3
      Did you solve your problem?

      Originally posted by gwilymh View Post
      Hi,
      I am attempting to use SSPACE version 3 to re-scaffold some mRNA contigs using paired-end RNA-Seq data. While SSPACe was designed for rescaffolding genomes, I see no reason why it should not work in similar fashion for re-scaffolding de novo assembled RNA contigs.

      SSPACE is discussed in seqanswers here:
      Discussion of next-gen sequencing related bioinformatics: resources, algorithms, open source efforts, etc


      The SSPACE webpage can be reached from here:
      Our state-of-the-art bioinformatics and biostatistics solutions guarantee high-quality results and clear answers to your research questions.


      I consistently get the following error message:
      =>date: Mapping reads to contigs with Bowtie
      Thread 8 terminated abnormally: Can't open bwa output -- fatal
      Out of memory!
      Process 'extend/format contigs' failed on date

      with:
      resources_used.mem=47,424,992kb
      resources_used.vmem=64,093,240kb
      resources_used.walltime=00:34:33

      It seems to me that SSPACE is running out of physical memory while trying to open the bwa output.

      The "Thread 8 terminated abnormally" is especially cryptic, as the program was set to only run with 6 threads (-T 6).

      My run parameters are:
      Code:
      #PBS -l nodes=1:ppn=6,mem=47gb,walltime=72:00:00
      SSPACE_FILE=${HOME}/src/SSPACE-STANDARD-3.0_linux-x86_64/SSPACE_Standard_v3.0.pl 
      LIBRARY_FILE=/[I]filepath[/I]/library_file_2.txt
      CONTIG_FILE=/[I]filepath[/I]/A_planci_pcg_transdec_MePath2Renam_echinoHomology.fasta		#-s option, contigs that we are scaffolding
      MIN_LINKS=10		   	# -k 10
      THREADS=6			    # -T, threads
      SKIP=0				    #-S 0=no, -S 1=yes, skip processing of reads
      EXTEND_CONTIGS=1		#-x, extend contigs using sequence data, 0=no, 1=yes, default 0
      VERBOSE=1			    #-v  Runs the scaffolding process in verbose mode (-v 1=yes, -v 0=no, default -v 0, optional)
      $SSPACE_FILE -l $LIBRARY_FILE -s $CONTIG_FILE -k $MIN_LINKS -T $THREADS -S $SKIP -x $EXTEND_CONTIGS -v $VERBOSE
      The supporting library file is attached.

      I previously tried to run SSPACE over multiple nodes, but it seemed to use the cores on only one of the nodes.

      Does anyone know of a way to get SSPACE to run across multiple nodes? Can it be run with mpi multi-threading? If not, does anyone have any suggestions for reducing memory requirements?
      I met the same problem. Did you solve your problem?
      If you have, could you share your experiences?
      Thanks

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Strategies for Sequencing Challenging Samples
        by seqadmin


        Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
        03-22-2024, 06:39 AM
      • seqadmin
        Techniques and Challenges in Conservation Genomics
        by seqadmin



        The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

        Avian Conservation
        Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
        03-08-2024, 10:41 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, Yesterday, 06:37 PM
      0 responses
      10 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, Yesterday, 06:07 PM
      0 responses
      9 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-22-2024, 10:03 AM
      0 responses
      49 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-21-2024, 07:32 AM
      0 responses
      67 views
      0 likes
      Last Post seqadmin  
      Working...
      X