SMRTanalysis 2.3 error during hgapCorrection

[TharMan]

Hi all,

I am trying to denovo assemble a yeast genome of about 15,000,000 bp expected size. I am using data from 2 SMRT cells that were run one after the other in two separate instances on the same library (2 cells 2 videos) for this. I was running the current version of SMRT Portal with the HGAP_v3 protocol with default settings except for the adjusted genome size to 15,000,000 and expected coverage at 30.

I should add that the protocol was running on a single node with 15 CPUs and 180gb of RAM.To me it looks like the error comes up once hgap correction is started. I was able to finish a pre-assembly_v2 run if I used data from only 1 SMRT cell, however it stops the pre-assembly run at the same point if I used both SMRT cells.

Is this due to hardware limitations?

Here is the protocol log I am getting:

###############################################################################
[INFO] 2015-07-09 15:04:09,818 [root main 269] Starting smrtpipe.py : v1.87.139483
[INFO] 2015-07-09 15:04:09,819 [root main 270] Using Python : v2.7.3
[INFO] 2015-07-09 15:04:09,864 [root main 271] Platform info Linux-3.13.0-55-generic-x86_64-with-Ubuntu-14.04-trusty
[INFO] 2015-07-09 15:04:09,965 [root main 272] Running on dge-calc.ad.ciw.edu
[INFO] 2015-07-09 15:04:09,965 [root main 273] Process id 13654
[INFO] 2015-07-09 15:04:09,966 [SMRTpipe.pbpy.io.SecondaryConfig loadFromFile 25] Loading SecondaryConfig from /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/etc/config.xml
[INFO] 2015-07-09 15:04:09,967 [SMRTpipe.SmrtPipeConfig __autoLoad 113] Loading configuration from /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/smrtpipe.rc
[INFO] 2015-07-09 15:04:09,993 [SMRTpipe.SmrtPipeConfig __setAlgorithmParametersPath 212] ALGO_PARAMS key not found in smrtpipe.rc - using latest bundle: /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/algorithm_parameters/2014-09
[INFO] 2015-07-09 15:04:09,996 [SMRTpipe.SmrtPipeMain sanityCheck 796] Found file /srv/smrtanalysis/userdata/inputs_dropbox/SMRT1/Analysis_Results/m150603_081844_42145_c100821092550000001823174411031592_s1_p0.1.bax.h5 3987871673 bytes
[INFO] 2015-07-09 15:04:09,997 [SMRTpipe.SmrtPipeMain _sanityCheck 796] Found file /srv/smrtanalysis/userdata/inputs_dropbox/SMRT1/Analysis_Results/m150603_081844_42145_c100821092550000001823174411031592_s1_p0.2.bax.h5 3851658310 bytes
[INFO] 2015-07-09 15:04:09,997 [SMRTpipe.SmrtPipeMain _sanityCheck 796] Found file /srv/smrtanalysis/userdata/inputs_dropbox/SMRT1/Analysis_Results/m150603_081844_42145_c100821092550000001823174411031592_s1_p0.3.bax.h5 4299284981 bytes
[INFO] 2015-07-09 15:04:09,997 [SMRTpipe.SmrtPipeMain _sanityCheck 796] Found file /srv/smrtanalysis/userdata/inputs_dropbox/SMRT2/Analysis_Results/m150506_193531_42145_c100821702550000001823175011031584_s1_p0.1.bax.h5 3325699888 bytes
[INFO] 2015-07-09 15:04:09,998 [SMRTpipe.SmrtPipeMain _sanityCheck 796] Found file /srv/smrtanalysis/userdata/inputs_dropbox/SMRT2/Analysis_Results/m150506_193531_42145_c100821702550000001823175011031584_s1_p0.2.bax.h5 3462695540 bytes
[INFO] 2015-07-09 15:04:09,998 [SMRTpipe.SmrtPipeMain _sanityCheck 796] Found file /srv/smrtanalysis/userdata/inputs_dropbox/SMRT2/Analysis_Results/m150506_193531_42145_c100821702550000001823175011031584_s1_p0.3.bax.h5 4810546579 bytes
[INFO] 2015-07-09 15:04:09,998 [smrtpipe.status _sanityCheck 804] Successfully found 6 files in xml:/srv/smrtanalysis/userdata/jobs/016/016439/input.xml
[INFO] 2015-07-09 15:04:09,999 [SMRTpipe.pbpy.io.SecondaryConfig _loadFromFile 25] Loading SecondaryConfig from /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/etc/config.xml
[INFO] 2015-07-09 15:04:10,000 [SMRTpipe.SmrtPipeConfig __autoLoad 113] Loading configuration from /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/smrtpipe.rc
[INFO] 2015-07-09 15:04:10,001 [SMRTpipe.SmrtPipeConfig __setAlgorithmParametersPath 212] ALGO_PARAMS key not found in smrtpipe.rc - using latest bundle: /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/algorithm_parameters/2014-09
[INFO] 2015-07-09 15:04:10,001 [SMRTpipe.SmrtPipeContext __init 98] SmrtPipeConfig Summary:
TMP = /srv/smrtanalysis/tmpdir
PROGRESS_URL = http://10.15.7.241:8080/smrtportal/api
CLUSTER_MANAGER = BASH
HOST_PORT = 8080
NCHUNKS = 64
ALGO_PARAMS_PATH = /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/etc/algorithm_parameters/2014-09
MAX_SLOTS = 256
ca_distribute.hgap = useGrid:0, scriptOnGrid:0, frgCorrOnGrid:0, ovlCorrOnGrid:0
CLOUD_LARGE_FILE_CUTOFF = 10MB
SMRTANALYSIS_VERSION = 2.3.0
INPUT_HDF5_EXT = bas.h5
SMRTPIPE_VERSION = 2.3.0.139497
HOST_NAME = 10.15.7.241
DENOVO_READ_SCOPES = small: 0.15, large:15, huge:1700
NPROC = 14
DENOVO_GENOME_SCOPES = small:100, large:1000, huge:40000
NJOBS = 64
CLOUD = False
REFERENCE_SCOPES = small:10, large:1000, veryLarge:100000, huge:10000000
MAX_THREADS = 8
HEARTBEAT_FREQ = 60
VIS_HOST = 10.15.7.241
VIS_PORT = 8080
SHARED_DIR = /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/common/userdata/shared_dir
DEBUG = False
ca_distribute.pb2ca = useGrid:0, scriptOnGrid:0, frgCorrOnGrid:0, ovlCorrOnGrid:0
SETTINGS_PATH = .
EXIT_ON_FAILURE = True
CLOUD_CHUNKS_PER_PROC = 2
READ_SCOPES = small:3.6, large:100, veryLarge:10000, huge:1000000
MAX_CHUNKS = 64
ASSEMBLY_HOME = /srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis
DATA_ROOT = /mnt
VIS_NAMESPACE =
IPY = ipy -X:ColorfulConsole
TOC_HTML_NAME = index.html
[INFO] 2015-07-09 15:04:10,004 [SMRTpipe.SmrtPipeMain run 465] Successfully found or created TMP dir '/srv/smrtanalysis/tmpdir'
[WARNING] 2015-07-09 15:04:10,038 [SMRTpipe.SmrtPipeSettings _importSettings 254] Parameter p_assembleunitig.specInRunCA is missing a value
[WARNING] 2015-07-09 15:04:10,038 [SMRTpipe.SmrtPipeSettings _importSettings 254] Parameter p_referenceuploader.organism is missing a value
[WARNING] 2015-07-09 15:04:10,038 [SMRTpipe.SmrtPipeSettings _importSettings 254] Parameter p_referenceuploader.ploidy is missing a value
[WARNING] 2015-07-09 15:04:10,039 [SMRTpipe.SmrtPipeSettings _importSettings 254] Parameter p_referenceuploader.user is missing a value
[INFO] 2015-07-09 15:04:10,040 [SMRTpipe.SmrtPipeContext initializeContext 235] SmrtPipeSettings Summary:
settings loaded from file /srv/smrtanalysis
p_mapping.maxHits = 10
p_filter.moduleName = P_Filter
p_assembleunitig.merSize = 14
global.version = 3
global.fetch = common/protocols/preprocessing/Fetch.1.xml
p_mapping.maxDivergence = 30
p_referenceuploader.moduleName = P_ReferenceUploader
p_fetch.moduleName = P_Fetch
global.state = active
p_assembleunitig.defaultFrgMinLen = 500
p_mappingreports.moduleName = P_MappingReports
p_preassemblerdagcon.minLongReadLength = 6000
p_assembleunitig.moduleName = P_AssembleUnitig
p_mapping.pbalign_advanced_opts =
global.Control Filtering = common/protocols/control/KeepControlReads.1.xml
global.control = /srv/smrtanalysis/userdata/references/2kb_control
p_preassemblerdagcon.totalBestn = 24
p_preassemblerdagcon.minCorCov = 6
p_assembleunitig.ovlMinLen = 40
p_filter.minSubReadLength = 500
p_assembleunitig.libraryName = pacbioReads
p_assemblypolishing.moduleName = P_AssemblyPolishing
p_mapping.pulseMetrics = DeletionQV,IPD,InsertionQV,PulseWidth,QualityValue,MergeQV,SubstitutionQV,DeletionTag
global.assembly = common/protocols/assembly/PreAssemblerHGAP.3.xml
p_assembleunitig.specTmpl = analysis/etc/celeraAssembler/unitig.spec
global.consensus = common/protocols/consensus/AssemblyPolishing.1.xml
global.description = HGAP version 3. PacBio de novo assembler optimized for speed.
p_referenceuploader.runUploaderUnitig = True
p_filter.description = Filter reads for use in the pre-assembly step of HGAP, the hierarchical genome assembly process.
p_preassemblerdagcon.moduleName = P_PreAssemblerDagcon
p_assembleunitig.description = This module runs Celera Assembler v8.1 to the unitig step, then finishes with our custom unitig consensus caller
p_assemblypolishing.description = Polish a pure-PacBio assembly for maximum accuracy using the Quiver algorithm.
p_referenceuploader.runUploaderHgap = False
p_assembleunitig.genomeSize = 15000000
p_referenceuploader.sawriter = sawriter -blt 8 -welter
p_filter.minLength = 100
p_mapping.pbalign_opts = --seed=1 --minAccuracy=0.75 --minLength=50 --concordant --algorithmOptions="-useQuality"
p_mapping.samBam = True
p_mapping.moduleName = P_Mapping
p_assembleunitig.ovlErrorRate = 0.06
p_preassemblerdagcon.splitBestn = 10
p_preassemblerdagcon.targetChunks = 6
p_assembleunitig.xCoverage = 30
p_mapping.minAnchorSize = 12
global.filtering = common/protocols/filtering/PreAssemblerSFilter.1.xml
p_referenceuploader.runUploader = False
p_preassemblerdagcon.computeLengthCutoff = True
p_filter.readScore = 0.80
p_mapping.description = BLASR maps reads to genomes by finding the highest scoring local alignment or set of local alignments between the read and the genome. The first set of alignments is found by querying an index of the reference genome, and then refining until only high scoring alignments are retained. Additional pulse metrics are loaded into the resulting cmp.h5 file to enable downstream use of the Quiver algorithm.
p_mapping.loadPulsesOpts = bymetric
global.mapping = common/protocols/mapping/BLASR_Resequencing.1.xml
global.otfReference = reference
global.referenceUploader = common/protocols/referenceuploader/ReferenceUploaderUnitig.1.xml
p_mapping.gff2Bed = True
p_mapping.placeRepeatsRandomly = True
global.deferRefCheck = True
global.name = RS_HGAP_Assembly
p_preassemblerdagcon.blasrOpts = -noSplitSubreads -minReadLength 200 -maxScore -1000 -maxLCPLength 16
p_referenceuploader.samIdx = samtools faidx
p_fetch.description = Sets up inputs
p_assembleunitig.maxSlotPerc = 1
p_assemblypolishing.enableMapQVFilter = True
p_preassemblerdagcon.title = Using DAG-based consensus algorithm, pre-assemble long reads as the first step of the Hierarchical Genome Assembly process (HGAP). Version 2 is a stepping stone for scaling to much larger genomes.
p_filterreports.moduleName = P_FilterReports
p_referenceuploader.name = reference
[INFO] 2015-07-09 15:04:10,042 [SMRTpipe.SmrtDataService setupOutputDirectories 132] Creating /srv/smrtanalysis/userdata/jobs/016/016439/data
[INFO] 2015-07-09 15:04:10,112 [SMRTpipe.SmrtDataService setupOutputDirectories 132] Creating /srv/smrtanalysis/userdata/jobs/016/016439/results
[INFO] 2015-07-09 15:04:10,113 [SMRTpipe.SmrtDataService setupOutputDirectories 136] Completed setting up DataServices in /srv/smrtanalysis/userdata/jobs/016/016439
[INFO] 2015-07-09 15:04:10,114 [SMRTpipe.SmrtPipeContext initializeContext 257] Input data is [ file//srv/smrtanalysis/userdata/inputs_dropbox/SMRT1, file//srv/smrtanalysis/userdata/inputs_dropbox/SMRT2 ]
[INFO] 2015-07-09 15:04:10,114 [SMRTpipe.SmrtPipeContext initializeContext 261] Configuration override for PROGRESS_URL: Old: --> New: http://10.15.7.241:8080/smrtportal/api
[INFO] 2015-07-09 15:04:10,114 [SMRTpipe.SmrtPipeContext initializeContext 274] Changing working directory to /srv/smrtanalysis/tmpdir/tmp059Lky
[INFO] 2015-07-09 15:04:10,185 [SMRTpipe.SmrtPipeContext movieFiles 371] Pulling data from tags: *
[INFO] 2015-07-09 15:04:10,185 [SMRTpipe.SmrtPipeContext movieFiles 391] Found /srv/smrtanalysis/userdata/inputs_dropbox/SMRT1/Analysis_Results/m150603_081844_42145_c100821092550000001823174411031592_s1_p0.1.bax.h5 (3987871673 bytes)
[INFO] 2015-07-09 15:04:10,185 [SMRTpipe.SmrtPipeContext movieFiles 391] Found /srv/smrtanalysis/userdata/inputs_dropbox/SMRT1/Analysis_Results/m150603_081844_42145_c100821092550000001823174411031592_s1_p0.2.bax.h5 (3851658310 bytes)
[INFO] 2015-07-09 15:04:10,186 [SMRTpipe.SmrtPipeContext movieFiles 391] Found /srv/smrtanalysis/userdata/inputs_dropbox/SMRT1/Analysis_Results/m150603_081844_42145_c100821092550000001823174411031592_s1_p0.3.bax.h5 (4299284981 bytes)
[INFO] 2015-07-09 15:04:10,186 [SMRTpipe.SmrtPipeContext movieFiles 391] Found /srv/smrtanalysis/userdata/inputs_dropbox/SMRT2/Analysis_Results/m150506_193531_42145_c100821702550000001823175011031584_s1_p0.1.bax.h5 (3325699888 bytes)
[INFO] 2015-07-09 15:04:10,186 [SMRTpipe.SmrtPipeContext movieFiles 391] Found /srv/smrtanalysis/userdata/inputs_dropbox/SMRT2/Analysis_Results/m150506_193531_42145_c100821702550000001823175011031584_s1_p0.2.bax.h5 (3462695540 bytes)
[INFO] 2015-07-09 15:04:10,186 [SMRTpipe.SmrtPipeContext movieFiles 391] Found /srv/smrtanalysis/userdata/inputs_dropbox/SMRT2/Analysis_Results/m150506_193531_42145_c100821702550000001823175011031584_s1_p0.3.bax.h5 (4810546579 bytes)
[INFO] 2015-07-09 15:04:10,197 [SMRTpipe.SmrtPipeMain _getBasVersions 413] Detected basH5 versions set(['-v2.3.0'])
[INFO] 2015-07-09 15:04:11,491 [smrtpipe.status run 503] smrtpipe.py running on dge-calc.ad.ciw.edu
[INFO] 2015-07-09 15:04:11,492 [smrtpipe.status run 519] SMRT Analysis 2.3.0 / SMRTpipe 1.87.139483
[INFO] 2015-07-09 15:04:11,492 [SMRTpipe.SmrtPipeMain run 522] Starting smrtpipe version 1.87.139483
[INFO] 2015-07-09 15:04:12,464 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_Fetch module
[INFO] 2015-07-09 15:04:12,467 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_Fetch v2.4.139483 and validated settings.
[INFO] 2015-07-09 15:04:12,483 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_Filter module
[INFO] 2015-07-09 15:04:12,488 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_Filter v1.7.139483 and validated settings.
[INFO] 2015-07-09 15:04:12,491 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_FilterReports module
[INFO] 2015-07-09 15:04:12,492 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_FilterReports v1.21.139483 and validated settings.
[INFO] 2015-07-09 15:04:12,503 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_PreAssemblerDagcon module
[INFO] 2015-07-09 15:04:12,508 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_PreAssemblerDagcon v2.1.124285 and validated settings.
[INFO] 2015-07-09 15:04:12,527 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_AssembleUnitig module
[INFO] 2015-07-09 15:04:12,532 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_AssembleUnitig v2.2.UnknownVersion and validated settings.
[INFO] 2015-07-09 15:04:12,549 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_ReferenceUploader module
[INFO] 2015-07-09 15:04:12,552 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_ReferenceUploader v1.2.139483 and validated settings.
[INFO] 2015-07-09 15:04:12,560 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_Mapping module
[INFO] 2015-07-09 15:04:12,569 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_Mapping v1.33.141997 and validated settings.
[INFO] 2015-07-09 15:04:12,590 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_MappingReports module
[INFO] 2015-07-09 15:04:12,592 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_MappingReports v1.7.140023 and validated settings.
[INFO] 2015-07-09 15:04:12,596 [SMRTpipe.SmrtPipeMain _dynamicallyLoadModule 846] Dynamically Loaded P_AssemblyPolishing module
[INFO] 2015-07-09 15:04:12,599 [SMRTpipe.SmrtPipeMain _loadModules 212] Loaded module P_AssemblyPolishing v1.36.139483 and validated settings.
[INFO] 2015-07-09 15:04:12,602 [SMRTpipe.SmrtDataService writeTo 424] Writing 6 items to DataStore in {'smrt.data.xmlparam': , 'smrt.output.log': '/srv/smrtanalysis/userdata/jobs/016/016439/log', 'smrt.data.cmdline': >, 'smrt.output.root': '/srv/smrtanalysis/userdata/jobs/016/016439', 'smrt.output.results': '/srv/smrtanalysis/userdata/jobs/016/016439/results', 'smrt.output.data': '/srv/smrtanalysis/userdata/jobs/016/016439/data'}
[INFO] 2015-07-09 15:04:12,726 [SMRTpipe.SmrtPipeMain _runTasks 281] Skipping PreWorkflow as it contains zero tasks
[INFO] 2015-07-09 15:04:12,731 [SMRTpipe.SmrtPipeMain _runTasks 284] Loading 35 tasks into Workflow
[INFO] 2015-07-09 15:04:16,825 [SMRTpipe.SmrtPipeMain _runTasks 298] Executing workflow Workflow
[INFO] 2015-07-09 15:04:16,825 [smrtpipe.status execute 620] Process id 13654
[INFO] 2015-07-09 15:04:24,224 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Fetch/toFofn
[INFO] 2015-07-09 15:04:24,244 [smrtpipe.status refreshTargets 407] task P_Fetch.toFofn successfully completed (0.00 sec (0.00 min)). freed 1 slots, using 0 slots.
[INFO] 2015-07-09 15:04:24,244 [smrtpipe.status refreshTargets 409] Workflow Completion Status 1/50 in ( 2%) tasks completed.
[INFO] 2015-07-09 15:04:29,254 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Fetch/adapterRpt
[INFO] 2015-07-09 15:04:29,254 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads.plsFofn.Scatter
[INFO] 2015-07-09 15:04:29,255 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Fetch/overviewRpt
[INFO] 2015-07-09 15:04:29,277 [smrtpipe.status refreshTargets 407] task P_Filter.subreads.plsFofn.Scatter successfully completed (0.00 sec (0.00 min)). freed 1 slots, using 2 slots.
[INFO] 2015-07-09 15:04:29,278 [smrtpipe.status refreshTargets 409] Workflow Completion Status 2/50 in ( 4%) tasks completed.
[INFO] 2015-07-09 15:04:31,071 [smrtpipe.status refreshTargets 407] task P_Fetch.overviewRpt successfully completed (2.00 sec (0.03 min)). freed 1 slots, using 1 slots.
[INFO] 2015-07-09 15:04:31,072 [smrtpipe.status refreshTargets 409] Workflow Completion Status 3/50 in ( 6%) tasks completed.
[INFO] 2015-07-09 15:04:36,094 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter_005of006
[INFO] 2015-07-09 15:04:36,095 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter_006of006
[INFO] 2015-07-09 15:04:36,095 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter_002of006
[INFO] 2015-07-09 15:04:36,096 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter_001of006
[INFO] 2015-07-09 15:04:36,096 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter_004of006
[INFO] 2015-07-09 15:04:36,097 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter_003of006
[INFO] 2015-07-09 15:05:27,546 [smrtpipe.status refreshTargets 407] task P_Filter.filter_003of006 successfully completed (51.00 sec (0.85 min)). freed 2 slots, using 11 slots.
[INFO] 2015-07-09 15:05:27,547 [smrtpipe.status refreshTargets 409] Workflow Completion Status 4/50 in ( 8%) tasks completed.
[INFO] 2015-07-09 15:05:27,577 [smrtpipe.status refreshTargets 407] task P_Filter.filter_004of006 successfully completed (52.00 sec (0.87 min)). freed 2 slots, using 9 slots.
[INFO] 2015-07-09 15:05:27,577 [smrtpipe.status refreshTargets 409] Workflow Completion Status 5/50 in ( . 10%) tasks completed.
[INFO] 2015-07-09 15:05:27,601 [smrtpipe.status refreshTargets 407] task P_Filter.filter_006of006 successfully completed (52.00 sec (0.87 min)). freed 2 slots, using 7 slots.
[INFO] 2015-07-09 15:05:27,602 [smrtpipe.status refreshTargets 409] Workflow Completion Status 6/50 in ( . 12%) tasks completed.
[INFO] 2015-07-09 15:05:27,625 [smrtpipe.status refreshTargets 407] task P_Filter.filter_001of006 successfully completed (55.00 sec (0.92 min)). freed 2 slots, using 5 slots.
[INFO] 2015-07-09 15:05:27,625 [smrtpipe.status refreshTargets 409] Workflow Completion Status 7/50 in ( . 14%) tasks completed.
[INFO] 2015-07-09 15:05:27,657 [smrtpipe.status refreshTargets 407] task P_Filter.filter_005of006 successfully completed (55.00 sec (0.92 min)). freed 2 slots, using 3 slots.
[INFO] 2015-07-09 15:05:27,657 [smrtpipe.status refreshTargets 409] Workflow Completion Status 8/50 in ( . 16%) tasks completed.
[INFO] 2015-07-09 15:05:27,694 [smrtpipe.status refreshTargets 407] task P_Filter.filter_002of006 successfully completed (56.00 sec (0.93 min)). freed 2 slots, using 1 slots.
[INFO] 2015-07-09 15:05:27,694 [smrtpipe.status refreshTargets 409] Workflow Completion Status 9/50 in ( . 18%) tasks completed.
[INFO] 2015-07-09 15:05:32,737 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads_006of006
[INFO] 2015-07-09 15:05:32,738 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads_005of006
[INFO] 2015-07-09 15:05:32,738 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads_002of006
[INFO] 2015-07-09 15:05:32,739 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads_001of006
[INFO] 2015-07-09 15:05:32,739 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads_004of006
[INFO] 2015-07-09 15:05:32,739 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads_003of006
[INFO] 2015-07-09 15:05:32,739 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter.rgnFofn.Gather
[INFO] 2015-07-09 15:05:32,758 [smrtpipe.status refreshTargets 407] task P_Filter.filter.rgnFofn.Gather successfully completed (1.00 sec (0.02 min)). freed 1 slots, using 7 slots.
[INFO] 2015-07-09 15:05:32,758 [smrtpipe.status refreshTargets 409] Workflow Completion Status 10/50 in ( .. 20%) tasks completed.
[INFO] 2015-07-09 15:05:37,766 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreadSummary
[INFO] 2015-07-09 15:06:01,712 [smrtpipe.status refreshTargets 407] task P_Fetch.adapterRpt successfully completed (88.00 sec (1.47 min)). freed 1 slots, using 7 slots.
[INFO] 2015-07-09 15:06:01,732 [smrtpipe.status refreshTargets 409] Workflow Completion Status 11/50 in ( .. 22%) tasks completed.
[INFO] 2015-07-09 15:06:06,740 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/filter.summary.Gather
[INFO] 2015-07-09 15:06:08,616 [smrtpipe.status refreshTargets 407] task P_Filter.filter.summary.Gather successfully completed (3.00 sec (0.05 min)). freed 1 slots, using 7 slots.
[INFO] 2015-07-09 15:06:08,616 [smrtpipe.status refreshTargets 409] Workflow Completion Status 12/50 in ( .. 24%) tasks completed.
[INFO] 2015-07-09 15:06:13,622 [smrtpipe.status refreshTargets 419] Starting task://016439/P_FilterReports/statsRpt
[INFO] 2015-07-09 15:06:49,997 [smrtpipe.status refreshTargets 407] task P_FilterReports.statsRpt successfully completed (36.00 sec (0.60 min)). freed 1 slots, using 7 slots.
[INFO] 2015-07-09 15:06:49,998 [smrtpipe.status refreshTargets 409] Workflow Completion Status 13/50 in ( .. 26%) tasks completed.
[INFO] 2015-07-09 15:06:55,007 [smrtpipe.status refreshTargets 419] Starting task://016439/P_FilterReports/loadingRpt
[INFO] 2015-07-09 15:07:07,581 [smrtpipe.status refreshTargets 407] task P_FilterReports.loadingRpt successfully completed (12.00 sec (0.20 min)). freed 1 slots, using 7 slots.
[INFO] 2015-07-09 15:07:07,581 [smrtpipe.status refreshTargets 409] Workflow Completion Status 14/50 in ( .. 28%) tasks completed.
[INFO] 2015-07-09 15:07:17,614 [smrtpipe.status refreshTargets 407] task P_Filter.subreads_001of006 successfully completed (108.00 sec (1.80 min)). freed 1 slots, using 6 slots.
[INFO] 2015-07-09 15:07:17,615 [smrtpipe.status refreshTargets 409] Workflow Completion Status 15/50 in ( ... 30%) tasks completed.
[INFO] 2015-07-09 15:07:17,640 [smrtpipe.status refreshTargets 407] task P_Filter.subreads_006of006 successfully completed (109.00 sec (1.82 min)). freed 1 slots, using 5 slots.
[INFO] 2015-07-09 15:07:17,640 [smrtpipe.status refreshTargets 409] Workflow Completion Status 16/50 in ( ... 32%) tasks completed.
[INFO] 2015-07-09 15:07:22,668 [smrtpipe.status refreshTargets 407] task P_Filter.subreads_003of006 successfully completed (113.00 sec (1.88 min)). freed 1 slots, using 4 slots.
[INFO] 2015-07-09 15:07:22,669 [smrtpipe.status refreshTargets 409] Workflow Completion Status 17/50 in ( ... 34%) tasks completed.
[INFO] 2015-07-09 15:07:27,698 [smrtpipe.status refreshTargets 407] task P_Filter.subreads_002of006 successfully completed (117.00 sec (1.95 min)). freed 1 slots, using 3 slots.
[INFO] 2015-07-09 15:07:27,699 [smrtpipe.status refreshTargets 409] Workflow Completion Status 18/50 in ( ... 36%) tasks completed.
[INFO] 2015-07-09 15:07:32,727 [smrtpipe.status refreshTargets 407] task P_Filter.subreads_004of006 successfully completed (120.00 sec (2.00 min)). freed 1 slots, using 2 slots.
[INFO] 2015-07-09 15:07:32,728 [smrtpipe.status refreshTargets 409] Workflow Completion Status 19/50 in ( ... 38%) tasks completed.
[INFO] 2015-07-09 15:07:49,446 [smrtpipe.status refreshTargets 407] task P_Filter.subreads_005of006 successfully completed (140.00 sec (2.33 min)). freed 1 slots, using 1 slots.
[INFO] 2015-07-09 15:07:49,447 [smrtpipe.status refreshTargets 409] Workflow Completion Status 20/50 in ( .... 40%) tasks completed.
[INFO] 2015-07-09 15:07:54,460 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads.subreads.Gather
[INFO] 2015-07-09 15:07:54,461 [smrtpipe.status refreshTargets 419] Starting task://016439/P_Filter/subreads.subreadFastq.Gather
[INFO] 2015-07-09 15:08:13,926 [smrtpipe.status refreshTargets 407] task P_Filter.subreads.subreads.Gather successfully completed (19.00 sec (0.32 min)). freed 1 slots, using 2 slots.
[INFO] 2015-07-09 15:08:13,926 [smrtpipe.status refreshTargets 409] Workflow Completion Status 21/50 in ( .... 42%) tasks completed.
[INFO] 2015-07-09 15:08:18,953 [smrtpipe.status refreshTargets 419] Starting task://016439/P_PreAssemblerDagcon/filterLongReadsByLength
[INFO] 2015-07-09 15:08:25,493 [smrtpipe.status refreshTargets 407] task P_Filter.subreads.subreadFastq.Gather successfully completed (31.00 sec (0.52 min)). freed 1 slots, using 2 slots.
[INFO] 2015-07-09 15:08:25,493 [smrtpipe.status refreshTargets 409] Workflow Completion Status 22/50 in ( .... 44%) tasks completed.
[INFO] 2015-07-09 15:08:57,206 [smrtpipe.status refreshTargets 407] task P_PreAssemblerDagcon.filterLongReadsByLength successfully completed (40.00 sec (0.67 min)). freed 1 slots, using 1 slots.
[INFO] 2015-07-09 15:08:57,207 [smrtpipe.status refreshTargets 409] Workflow Completion Status 23/50 in ( .... 46%) tasks completed.
[INFO] 2015-07-09 15:09:02,215 [smrtpipe.status refreshTargets 419] Starting task://016439/P_PreAssemblerDagcon/hgapAlignForCorrection
[INFO] 2015-07-09 15:09:09,352 [smrtpipe.status refreshTargets 407] task P_Filter.subreadSummary successfully completed (212.00 sec (3.53 min)). freed 1 slots, using 14 slots.
[INFO] 2015-07-09 15:09:09,352 [smrtpipe.status refreshTargets 409] Workflow Completion Status 24/50 in ( .... 48%) tasks completed.
[INFO] 2015-07-09 15:09:14,356 [smrtpipe.status refreshTargets 419] Starting task://016439/P_FilterReports/subreadRpt
[INFO] 2015-07-09 15:09:36,208 [smrtpipe.status refreshTargets 407] task P_FilterReports.subreadRpt successfully completed (20.00 sec (0.33 min)). freed 1 slots, using 14 slots.
[INFO] 2015-07-09 15:09:36,209 [smrtpipe.status refreshTargets 409] Workflow Completion Status 25/50 in ( ..... 50%) tasks completed.
[INFO] 2015-07-09 16:55:55,198 [smrtpipe.status refreshTargets 407] task P_PreAssemblerDagcon.hgapAlignForCorrection successfully completed (6417.00 sec (106.95 min)). freed 14 slots, using 0 slots.
[INFO] 2015-07-09 16:55:55,210 [smrtpipe.status refreshTargets 409] Workflow Completion Status 26/50 in ( ..... 52%) tasks completed.
[INFO] 2015-07-09 16:56:00,218 [smrtpipe.status refreshTargets 419] Starting task://016439/P_PreAssemblerDagcon/hgapFilterM4
[INFO] 2015-07-09 16:56:15,249 [smrtpipe.status refreshTargets 407] task P_PreAssemblerDagcon.hgapFilterM4 successfully completed (16.00 sec (0.27 min)). freed 1 slots, using 0 slots.
[INFO] 2015-07-09 16:56:15,249 [smrtpipe.status refreshTargets 409] Workflow Completion Status 27/50 in ( ..... 54%) tasks completed.
[INFO] 2015-07-09 16:56:20,256 [smrtpipe.status refreshTargets 419] Starting task://016439/P_PreAssemblerDagcon/hgapCorrection
[ERROR] 2015-07-09 17:09:14,125 [smrtpipe.status refreshTargets 413] *** Failed task task://016439/P_PreAssemblerDagcon/hgapCorrection
[INFO] 2015-07-09 17:09:14,128 [smrtpipe.status execute 627] Found 1 failed tasks.
[INFO] 2015-07-09 17:09:14,128 [smrtpipe.status execute 629] task hgapCorrection FAILED
[ERROR] 2015-07-09 17:09:16,265 [SMRTpipe.SmrtPipeMain run 608] SmrtExit task://016439/P_PreAssemblerDagcon/hgapCorrection Failed
Traceback (most recent call last):
File "/srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/redist/python2.7/lib/python2.7/site-packages/SMRTpipe/SmrtPipeMain.py", line 571, in run
self._runTasks(pModules)
File "/srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/redist/python2.7/lib/python2.7/site-packages/SMRTpipe/SmrtPipeMain.py", line 300, in _runTasks
workflow.execute()
File "/srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/redist/python2.7/lib/python2.7/site-packages/SMRTpipe/engine/SmrtPipeWorkflow.py", line 640, in execute
raise SmrtExit(str(e))
SmrtExit: SmrtExit task://016439/P_PreAssemblerDagcon/hgapCorrection Failed
[ERROR] 2015-07-09 17:09:16,266 [SMRTpipe.SmrtPipeMain exit 690] SmrtExit task://016439/P_PreAssemblerDagcon/hgapCorrection Failed
[INFO] 2015-07-09 17:09:16,266 [SMRTpipe.SmrtPipeMain exit 692] Beginning exit process
[INFO] 2015-07-09 17:09:16,266 [SMRTpipe.SmrtPipeMain shutdown 716] beginning Shutdown Process
[INFO] 2015-07-09 17:09:16,267 [SMRTpipe.SmrtPipeMain _generate_final_report 651] Writing final TOC page with error message : SmrtExit task://016439/P_PreAssemblerDagcon/hgapCorrection Failed
[INFO] 2015-07-09 17:09:17,701 [SMRTpipe.SmrtPipeMain shutdown 752] Completed shutdown gracefully
[INFO] 2015-07-09 17:09:17,702 [SMRTpipe.SmrtPipeMain exit 702] Sending message Exiting smrtpipe last error: SmrtExit task://016439/P_PreAssemblerDagcon/hgapCorrection Failed
[INFO] 2015-07-09 17:09:17,702 [smrtpipe.status failed 104] Sending Job 016439 failed message 'Exiting smrtpipe last error: SmrtExit task://016439/P_PreAssemblerDagcon/hgapCorrection Failed' to url http://10.15.7.241:8080/smrtportal/a.../016439/status
[INFO] 2015-07-09 17:09:17,774 [SMRTpipe.SmrtPipeMain run 641] completed SmrtPipeMain.run with exit code -1
[INFO] 2015-07-09 17:09:17,774 [root main 335] Failed smrtpipe version v1.87.139483 with exit code -1 in 7507.78 seconds (125.13 minutes)
###############################################################################

[vcc]

Dear TharMan,
It seems that the chunk number count has to be increased. With lesser chunk number, the size of intermediate chunk file has increased greater than 4gb (as recommended maximum size by smrtanalysis). Try to increase the chunk numbers inside the params.xml file such that chunk file size should reduce to less than 4gb.
Fo example, if your longreads.fasta created during pre-assembly is of size 40gb, then you need to create minimum of 10chunks, to be handled without error by smrtanalysis pipeline.
Also, do upload the log file "job_directory/log/P_PreAssemblerDagcon/hgapalignforCorrection.target.scatter.log", then i'll be able to properly help you out.

[TharMan]

Dear TharMan,
It seems that the chunk number count has to be increased. With lesser chunk number, the size of intermediate chunk file has increased greater than 4gb (as recommended maximum size by smrtanalysis). Try to increase the chunk numbers inside the params.xml file such that chunk file size should reduce to less than 4gb.
Fo example, if your longreads.fasta created during pre-assembly is of size 40gb, then you need to create minimum of 10chunks, to be handled without error by smrtanalysis pipeline.
Also, do upload the log file "job_directory/log/P_PreAssemblerDagcon/hgapalignforCorrection.target.scatter.log", then i'll be able to properly help you out.

Dear vcc,

Thank you for your quick response. I followed your advice and increase the number of chunks from 6 to 15. The pipeline now finishes the hgapalignforCorrection step and writes the seed.m4 but now a new error comes up. The error is now at the HGAPerror correction step.

The HGAP_correction log shows:

###############################################
#!/bin/bash
# Setting up SMRTpipe environment
echo "Setting up ENV on $(uname -n)" for task hgapCorrection

SEYMOUR_HOME=/srv/smrtanalysis/install/smrtanalysis_2.3.0.140936
source $SEYMOUR_HOME/etc/setup.sh

# Create the local TMP dir if it doesn't exist
tmp_dir=$(readlink -m "/srv/smrtanalysis/tmpdir")
if [ ! -e "$tmp_dir" ]; then
stat=0
mkdir -p $tmp_dir || stat=$?
if [[ $stat -ne 0 ]]; then
echo "SMRTpipe Unable to create TMP dir '/srv/smrtanalysis/tmpdir' on $(uname -n)" 1>&2
exit 1
else
echo "successfully created or found TMP dir '/srv/smrtanalysis/tmpdir'"
fi
elif [[ ! -d "$tmp_dir" ]]; then
echo "SMRTpipe TMP /srv/smrtanalysis/tmpdir must be a directory on $(uname -n)" 1>&2
exit 1
fi

########### TASK metadata #############
# Task : hgapCorrection
# Module : P_PreAssemblerDagcon
# Module Version : 2.1.124285
# TaskType : None
# URL : task://016447/P_PreAssemblerDagcon/hgapCorrection
# createdAt : 2015-07-13 14:42:34.067950
# createdAt (UTC) : 2015-07-13 21:42:34.067963
# ncmds : 1
# LogPath : /srv/smrtanalysis/userdata/jobs/016/016447/log/P_PreAssemblerDagcon/hgapCorrection.log
# Script Path : /srv/smrtanalysis/userdata/jobs/016/016447/workflow/P_PreAssemblerDagcon/hgapCorrection.sh

# Input : /srv/smrtanalysis/userdata/jobs/016/016447/.seeds.m4.filter
# Input : /srv/smrtanalysis/userdata/jobs/016/016447/data/filtered_subreads.fasta
# Input : /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4
# Input : /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.fofn
# Output : /srv/smrtanalysis/userdata/jobs/016/016447/data/corrected.fastq
# Output : /srv/smrtanalysis/userdata/jobs/016/016447/data/corrected.fasta
#
########### END TASK metadata #############

cd /srv/smrtanalysis/userdata/jobs/016/016447/log/P_PreAssemblerDagcon
# Writing to log file
cat /srv/smrtanalysis/userdata/jobs/016/016447/workflow/P_PreAssemblerDagcon/hgapCorrection.sh >> /srv/smrtanalysis/userdata/jobs/016/016447/log/P_PreAssemblerDagcon/hgapCorrection.log;



echo "Running task://016447/P_PreAssemblerDagcon/hgapCorrection on $(uname -a)"

echo "Started on $(date -u)"
echo 'Validating existence of Input Files'
if [ -e /srv/smrtanalysis/userdata/jobs/016/016447/.seeds.m4.filter ]
then
echo 'Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/.seeds.m4.filter'
else
echo 'WARNING: Unable to find necessary input file, or dir /srv/smrtanalysis/userdata/jobs/016/016447/.seeds.m4.filter.'
fi
if [ -e /srv/smrtanalysis/userdata/jobs/016/016447/data/filtered_subreads.fasta ]
then
echo 'Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/data/filtered_subreads.fasta'
else
echo 'WARNING: Unable to find necessary input file, or dir /srv/smrtanalysis/userdata/jobs/016/016447/data/filtered_subreads.fasta.'
fi
if [ -e /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4 ]
then
echo 'Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4'
else
echo 'WARNING: Unable to find necessary input file, or dir /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.'
fi
if [ -e /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.fofn ]
then
echo 'Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.fofn'
else
echo 'WARNING: Unable to find necessary input file, or dir /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.fofn.'
fi
echo 'Successfully validated input files'

# Task hgapCorrection commands:


# Completed writing Task hgapCorrection commands


# Task 1
tmp=$(TMPDIR=/srv/smrtanalysis/tmpdir mktemp -d);mym4=/srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4 allm4=/srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.fofn subreads=/srv/smrtanalysis/userdata/jobs/016/016447/data/filtered_subreads.fasta bestn=24 nproc=14 fasta=/srv/smrtanalysis/userdata/jobs/016/016447/data/corrected.fasta fastq=/srv/smrtanalysis/userdata/jobs/016/016447/data/corrected.fastq cov=6 tmp=$tmp pbdagcon_wf.sh; rmdir $tmp || exit $?
echo "Task 1 completed at $(date)"



rcode=$?
echo "Finished on $(date -u)"
echo "Task hgapCorrection with nproc 14 with exit code ${rcode}."
exit ${rcode}# Writing stdout and stderr from Popen:
Setting up ENV on dge-calc.ad.ciw.edu for task hgapCorrection
Running task://016447/P_PreAssemblerDagcon/hgapCorrection on Linux dge-calc.ad.ciw.edu 3.13.0-55-generic #92-Ubuntu SMP Sun Jun 14 18:32:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Started on Tue Jul 14 01:19:42 UTC 2015
Validating existence of Input Files
Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/.seeds.m4.filter
Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/data/filtered_subreads.fasta
Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4
Successfully found /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.fofn
Successfully validated input files
Generating pre-alignments
m4topre.py /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4 /srv/smrtanalysis/userdata/jobs/016/016447/seeds.m4.fofn /srv/smrtanalysis/userdata/jobs/016/016447/data/filtered_subreads.fasta 24 > /srv/smrtanalysis/tmpdir/tmp.AVphUDl414/aln.25315.pre
Traceback (most recent call last):
File "/srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/bin/m4topre.py", line 208, in <module>
sys.exit(main())
File "/srv/smrtanalysis/install/smrtanalysis_2.3.0.140936/analysis/bin/m4topre.py", line 205, in main
rec.tseqlength, str(tst), str(tnd), qseq, tseq])
IOError: [Errno 28] No space left on device
Task 1 completed at Mon Jul 13 18:43:25 PDT 2015
Finished on Tue Jul 14 01:43:25 UTC 2015
Task hgapCorrection with nproc 14 with exit code 0.
###############################################

The errno is 28 no space left on device. I do get a warning when I start smrt portal about space left but there is 4.7 terabyte available for the smrtanalyssis user on the server. Is this physical space, memory or other resources ...to many tmp files?

[vcc]

It seems that there is very less space in /tmp/smrtanalysis or the folder you used while installing this pipeline.
It would be better to reinstall this smrtanalysis toolkit while changing the path of /tmp/smrtanalysis (which is by default mentioned) to some other folder on your harddisk, where there is sufficient space to handle the data. Symlinks will be automatically created on the nodes.
try this out and hope it will work

[TharMan]

Dear vcc

Yes that helped. As it turns out the tmp folder was memory and only 8gb was available. We moved the tmp folder to a hard drive and that fix that step. I though we were finally it done but during the blasr alignment step after pre-assembly a new error occurred.

the align log shows that the pulse metrics information could not be read:
###############################################
/srv/smrtanalysis/userdata/jobs/016/016453/input.fofn
# Input : /srv/smrtanalysis/userdata/jobs/016/016453/data/filtered_regions.fofn
# Output : /srv/smrtanalysis/userdata/jobs/016/016453/data/aligned_reads.cmp.h5
#
########### END TASK metadata #############

cd /srv/smrtanalysis/userdata/jobs/016/016453/log/P_Mapping
# Writing to log file
cat /srv/smrtanalysis/userdata/jobs/016/016453/workflow/P_Mapping/align.sh >> /srv/smrtanalysis/userdata/jobs/016/016453/log/P_Mapping/align.log;



echo "Running task://016453/P_Mapping/align on $(uname -a)"

echo "Started on $(date -u)"
echo 'Validating existence of Input Files'
if [ -e /srv/smrtanalysis/userdata/jobs/016/016453/reference ]
then
echo 'Successfully found /srv/smrtanalysis/userdata/jobs/016/016453/reference'
else
echo 'WARNING: Unable to find necessary input file, or dir /srv/smrtanalysis/userdata/jobs/016/016453/reference.'
fi
if [ -e /srv/smrtanalysis/userdata/jobs/016/016453/input.fofn ]
then
echo 'Successfully found /srv/smrtanalysis/userdata/jobs/016/016453/input.fofn'
else
echo 'WARNING: Unable to find necessary input file, or dir /srv/smrtanalysis/userdata/jobs/016/016453/input.fofn.'
fi
if [ -e /srv/smrtanalysis/userdata/jobs/016/016453/data/filtered_regions.fofn ]
then
echo 'Successfully found /srv/smrtanalysis/userdata/jobs/016/016453/data/filtered_regions.fofn'
else
echo 'WARNING: Unable to find necessary input file, or dir /srv/smrtanalysis/userdata/jobs/016/016453/data/filtered_regions.fofn.'
fi
echo 'Successfully validated input files'

# Task align commands:


# Completed writing Task align commands


# Task 1
pbalign "/srv/smrtanalysis/userdata/jobs/016/016453/input.fofn" "/srv/smrtanalysis/userdata/jobs/016/016453/reference" "/srv/smrtanalysis/userdata/jobs/016/016453/data/aligned_reads.cmp.h5" --seed=1 --minAccuracy=0.75 --minLength=50 --concordant --algorithmOptions="-useQuality" --algorithmOptions=' -minMatch 12 -bestn 10 -minPctIdentity 70.0' --hitPolicy=randombest --tmpDir=/srv/smrtanalysis/tmpdir --nproc=14 --regionTable=/srv/smrtanalysis/userdata/jobs/016/016453/data/filtered_regions.fofn || exit $?
echo "Task 1 completed at $(date)"
# Task 2
echo 'Alignment Complete' || exit $?
echo "Task 2 completed at $(date)"
# Task 3
loadChemistry.py /srv/smrtanalysis/userdata/jobs/016/016453/input.fofn /srv/smrtanalysis/userdata/jobs/016/016453/data/aligned_reads.cmp.h5 && loadPulses /srv/smrtanalysis/userdata/jobs/016/016453/input.fofn /srv/smrtanalysis/userdata/jobs/016/016453/data/aligned_reads.cmp.h5 -metrics DeletionQV,IPD,InsertionQV,PulseWidth,QualityValue,MergeQV,SubstitutionQV,DeletionTag -byread || exit $?
echo "Task 3 completed at $(date)"
# Task 4
echo 'LoadPulses Complete' || exit $?
echo "Task 4 completed at $(date)"



rcode=$?
echo "Finished on $(date -u)"
echo "Task align with nproc 14 with exit code ${rcode}."
exit ${rcode}# Writing stdout and stderr from Popen:
Setting up ENV on dge-calc.ad.ciw.edu for task align
Running task://016453/P_Mapping/align on Linux dge-calc.ad.ciw.edu 3.13.0-55-generic #92-Ubuntu SMP Sun Jun 14 18:32:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Started on Sat Jul 18 02:31:03 UTC 2015
Validating existence of Input Files
Successfully found /srv/smrtanalysis/userdata/jobs/016/016453/reference
Successfully found /srv/smrtanalysis/userdata/jobs/016/016453/input.fofn
Successfully found /srv/smrtanalysis/userdata/jobs/016/016453/data/filtered_regions.fofn

Successfully validated input files
2015-07-17 20:01:53,808
[ERROR] BlasrService returned a non-zero exit status. [INFO] 2015-07-17T19:31:05 [blasr] started.
ERROR, could not read pulse metrics for SMRTSequence m150506_193531_42145_c100821702550000001823175011031584_s1_p0/141051
###############################################

I am not sure why. The folder for both smrt cells are in the normal structure and contain the .h5 files

[TharMan]

Dear vcc,

I just run the HGAPv3 protocol with just the second smrt cell and it did complete the entire run. Does that mean there is some of the pulse metrics data missing in the m150506_193531_42145_c100821702550000001823175011031584_s1_p0 smrt cell?
Is there a way to edit the bas/bax.h5 files and remove the 141051 sequence?

[ymc]

I too had a similar "no space left" error at this step. Then I make sure my tmpdir is on my 6TB hard drive and then all is good. Good luck!