Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • SMRTpipe Error after runCaToUnitig

    I am getting strange memory issues after runCaToUnitig stage is compeleted:
    Code:
    [INFO] 2017-01-30 17:39:23,826 Workflow Completion Status 35/56 in (    ...... 62%) tasks completed.
    [INFO] 2017-01-30 17:39:28,836 task P_AssembleUnitig.writeRunCASpec successfully completed (4.00 sec (0.07 min)). freed 1 slots, using 1 slots.
    [INFO] 2017-01-30 17:39:28,837 Workflow Completion Status 36/56 in (    ...... 64%) tasks completed.
    [INFO] 2017-01-30 17:39:33,845 Starting task://Anonymous/P_AssembleUnitig/runCaToUnitig
    [INFO] 2017-01-30 17:39:50,860 task P_PreAssemblerDagcon.preAssemblerJsonReport successfully completed (25.00 sec (0.42 min)). freed 1 slots, using 1 slots.
    [INFO] 2017-01-30 17:39:50,861 Workflow Completion Status 37/56 in (    ...... 66%) tasks completed.
    slurmstepd: Job 4910074 exceeded memory limit (52468956 > 52428800), being killed
    slurmstepd: *** JOB 4910074 ON c5a-s17 CANCELLED AT 2017-01-30T17:47:40 ***
    slurmstepd: Exceeded job memory limit
    slurmstepd: Exceeded step memory limit at some point.
    I am using 50GB memory which should be more than enough for a 10Mb organism. I get stuck with the same error while using data from 1 cell or data from 3 cells.
    All logs seem to exit with code 0:
    Code:
     tail */*.log -n 1
    ==> P_AssembleUnitig/genFrgFile.log <==
    Task genFrgFile with nproc 1 with exit code 0.
    
    ==> P_AssembleUnitig/runCaToUnitig.log <==
    exit ${rcode}
    ==> P_AssembleUnitig/writeRunCASpec.log <==
    Task writeRunCASpec with nproc 1 with exit code 0.
    
    ==> P_Fetch/adapterRpt.log <==
    Task adapterRpt with nproc 1 with exit code 0.
    
    ==> P_Fetch/overviewRpt.log <==
    Task overviewRpt with nproc 1 with exit code 0.
    
    ==> P_Fetch/toFofn.log <==
    Task toFofn with nproc 1 with exit code 0.
    
    ==> P_Filter/filter_001of009.log <==
    Task filter_001of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_002of009.log <==
    Task filter_002of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_003of009.log <==
    Task filter_003of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_004of009.log <==
    Task filter_004of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_005of009.log <==
    Task filter_005of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_006of009.log <==
    Task filter_006of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_007of009.log <==
    Task filter_007of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_008of009.log <==
    Task filter_008of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter_009of009.log <==
    Task filter_009of009 with nproc 2 with exit code 0.
    
    ==> P_Filter/filter.plsFofn.Scatter.log <==
    Task filter.plsFofn.Scatter with nproc 1 with exit code 0.
    
    ==> P_Filter/filter.rgnFofn.Gather.log <==
    Task filter.rgnFofn.Gather with nproc 1 with exit code 0.
    
    ==> P_Filter/filter.summary.Gather.log <==
    Task filter.summary.Gather with nproc 1 with exit code 0.
    
    ==> P_FilterReports/loadingRpt.log <==
    Task loadingRpt with nproc 1 with exit code 0.
    
    ==> P_FilterReports/statsRpt.log <==
    Task statsRpt with nproc 1 with exit code 0.
    
    ==> P_FilterReports/subreadRpt.log <==
    Task subreadRpt with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_001of009.log <==
    Task subreads_001of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_002of009.log <==
    Task subreads_002of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_003of009.log <==
    Task subreads_003of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_004of009.log <==
    Task subreads_004of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_005of009.log <==
    Task subreads_005of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_006of009.log <==
    Task subreads_006of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_007of009.log <==
    Task subreads_007of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_008of009.log <==
    Task subreads_008of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads_009of009.log <==
    Task subreads_009of009 with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads.subreadFastq.Gather.log <==
    Task subreads.subreadFastq.Gather with nproc 1 with exit code 0.
    
    ==> P_Filter/subreads.subreads.Gather.log <==
    Task subreads.subreads.Gather with nproc 1 with exit code 0.
    
    ==> P_Filter/subreadSummary.log <==
    Task subreadSummary with nproc 1 with exit code 0.
    
    ==> P_PreAssemblerDagcon/filterLongReadsByLength.log <==
    Task filterLongReadsByLength with nproc 1 with exit code 0.
    
    ==> P_PreAssemblerDagcon/hgapAlignForCorrection.log <==
    Task hgapAlignForCorrection with nproc 12 with exit code 0.
    
    ==> P_PreAssemblerDagcon/hgapCorrection.log <==
    Task hgapCorrection with nproc 12 with exit code 0.
    
    ==> P_PreAssemblerDagcon/hgapFilterM4.log <==
    Task hgapFilterM4 with nproc 1 with exit code 0.
    
    ==> P_PreAssemblerDagcon/preAssemblerJsonReport.log <==
    Task preAssemblerJsonReport with nproc 1 with exit code 0.
    The log from runCaToUnitig.log is as follows:

    Code:
    #!/bin/bash
    # Setting up SMRTpipe environment
    echo "Setting up ENV on $(uname -n)" for task runCaToUnitig
    
    SEYMOUR_HOME=/apps/pacbio/2.3.0.140936.p5/install/smrtanalysis_2.3.0.140936
    source $SEYMOUR_HOME/etc/setup.sh
    
    # Create the local TMP dir if it doesn't exist
    tmp_dir=$(readlink -m "/apps/pacbio/2.3.0.140936.p5/tmpdir")
    if [ ! -e "$tmp_dir" ]; then
       stat=0
       mkdir -p $tmp_dir || stat=$?
       if [[ $stat -ne 0 ]]; then
           echo "SMRTpipe Unable to create TMP dir '/apps/pacbio/2.3.0.140936.p5/tmpdir' on $(uname -n)" 1>&2
           exit 1
       else
           echo "successfully created or found TMP dir '/apps/pacbio/2.3.0.140936.p5/tmpdir'"
       fi
    elif [[ ! -d "$tmp_dir" ]]; then
       echo "SMRTpipe TMP /apps/pacbio/2.3.0.140936.p5/tmpdir must be a directory on $(uname -n)" 1>&2
       exit 1
    fi
    
    ########### TASK metadata #############
    # Task            : runCaToUnitig
    # Module          : P_AssembleUnitig
    # Module Version  : 2.2.UnknownVersion
    # TaskType        : None
    # URL             : task://Anonymous/P_AssembleUnitig/runCaToUnitig
    # createdAt       : 2017-01-30 11:18:51.164802
    # createdAt (UTC) : 2017-01-30 16:18:51.164828
    # ncmds           : 2
    # LogPath         : /smrtpipe2/matt_review/new/log/P_AssembleUnitig/runCaToUnitig.log
    # Script Path     : /smrtpipe2/matt_review/new/workflow/P_AssembleUnitig/runCaToUnitig.sh
    
    # Input       : /smrtpipe2/matt_review/new/data/corrected.frg
    # Input       : /smrtpipe2/matt_review/new/data/runCA.spec
    # Output      :/smrtpipe2/matt_review/new/data/ca_finished
    #
    ########### END TASK metadata #############
    
    cd /smrtpipe2/matt_review/new/log/P_AssembleUnitig
    # Writing to log file
    cat /smrtpipe2/matt_review/new/workflow/P_AssembleUnitig/runCaToUnitig.sh >> /smrtpip
    e2/matt_review/new/log/P_AssembleUnitig/runCaToUnitig.log;
    
    
    
    echo "Running task://Anonymous/P_AssembleUnitig/runCaToUnitig on $(uname -a)"
    
    echo "Started on $(date -u)"
    echo 'Validating existence of Input Files'
    if [ -e /smrtpipe2/matt_review/new/data/corrected.frg ]
    then
    echo 'Successfully found /smrtpipe2/matt_review/new/data/corrected.frg'
    else
    echo 'WARNING: Unable to find necessary input file, or dir /smrtpipe2/matt_review/new/data/corrected.frg.'
    fi
    if [ -e /smrtpipe2/matt_review/new/data/runCA.spec ]
    then
    echo 'Successfully found /smrtpipe2/matt_review/new/data/runCA.spec'
    else
    echo 'WARNING: Unable to find necessary input file, or dir /smrtpipe2/matt_review/new/data/runCA.spec.'
    fi
    echo 'Successfully validated input files'
    
    # Task runCaToUnitig commands:
    
    
    # Completed writing Task runCaToUnitig commands
    
    
    # Task 1
    runCA -d /smrtpipe2/matt_review/new/data -p celera-assembler -s /smrtpipe2/matt_revie
    w/new/data/runCA.spec /smrtpipe2/matt_review/new/data/corrected.frg || exit $?
    echo "Task 1 completed at $(date)"
    # Task 2
    touch /smrtpipe2/matt_review/new/data/ca_finished || exit $?
    echo "Task 2 completed at $(date)"
    
    
    
    rcode=$?
    echo "Finished on $(date -u)"
    echo "Task runCaToUnitig with nproc 1 with exit code ${rcode}."
    exit ${rcode}
    Can anyone please help?
    Thanks

Latest Articles

Collapse

  • seqadmin
    Strategies for Sequencing Challenging Samples
    by seqadmin


    Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
    03-22-2024, 06:39 AM
  • seqadmin
    Techniques and Challenges in Conservation Genomics
    by seqadmin



    The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

    Avian Conservation
    Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
    03-08-2024, 10:41 AM

ad_right_rmr

Collapse

News

Collapse

Topics Statistics Last Post
Started by seqadmin, Yesterday, 06:37 PM
0 responses
10 views
0 likes
Last Post seqadmin  
Started by seqadmin, Yesterday, 06:07 PM
0 responses
9 views
0 likes
Last Post seqadmin  
Started by seqadmin, 03-22-2024, 10:03 AM
0 responses
49 views
0 likes
Last Post seqadmin  
Started by seqadmin, 03-21-2024, 07:32 AM
0 responses
67 views
0 likes
Last Post seqadmin  
Working...
X