Hi all,
I have a barcoded SMRTcell in bas.h5 format which I had to de-barcode into .fastq files (you can't de-barcode into bas.h5 format). I now want to assemble these fastQ files with HGAP, but I keep getting an error;
The protocol I use is the 'HGAP_Assembly_Advanced.1.xml' with the fastQ file as input. I've also used the 'RS_HGAP_Assembly.1.xml as input, but that did cause an error too.
Does anyone know if I should change a parameter in 'HGAP_Assembly_Advanced.1.xml' protocol in order to use fastQ files as input for HGAP?
Regards and thanks in advance,
Boetsie
I have a barcoded SMRTcell in bas.h5 format which I had to de-barcode into .fastq files (you can't de-barcode into bas.h5 format). I now want to assemble these fastQ files with HGAP, but I keep getting an error;
Code:
[DEBUG] 2013-09-10 11:20:23,899 [pbpy.smrtpipe.SmrtPipeContext initializeContext 240] Initialize Context with dataUrl xml:input.xml [DEBUG] 2013-09-10 11:20:23,899 [pbpy.smrtpipe.SmrtPipeContext initializeContext 241] Initialize Context with settings file HGAP_Assembly_Advanced.1.xml [DEBUG] 2013-09-10 11:20:23,899 [pbpy.smrtpipe.InputData loadXml 197] Loading file input.xml [DEBUG] 2013-09-10 11:20:23,899 [pbpy.io.MetaAnalysisXml load 101] Loading input.xml [DEBUG] 2013-09-10 11:20:23,900 [pbpy.io.MetaAnalysisXml load 116] No header found in input.xml. Unable to load jobId [DEBUG] 2013-09-10 11:20:23,900 [pbpy.io.MetaAnalysisXml _parseRef 584] Successfully parsed run:0000000-0000 to run 0000000-0000 [DEBUG] 2013-09-10 11:20:23,900 [pbpy.smrtpipe.InputData loadXml 214] Skipping assignment of JobId. Unable to find header in input.xml [DEBUG] 2013-09-10 11:20:23,900 [pbpy.smrtpipe.SmrtDataService setupOutputDirectories 113] SmrtDataService (oid 488599504) setting up output directories in /net/fs01/mnt/storage/rawdata/NGS_projects/ [DEBUG] 2013-09-10 11:20:23,900 [pbpy.smrtpipe.SmrtDataService setupOutputDirectories 116] output directory /net/fs01/mnt/storage/rawdata/ has already been created. Skipping [DEBUG] 2013-09-10 11:20:23,900 [pbpy.smrtpipe.SmrtDataService put 191] SmrtDataService (oid 488599504) repository has 1 items [INFO] 2013-09-10 11:20:23,900 [pbpy.smrtpipe.SmrtDataService setupOutputDirectories 132] Creating /net/fs01/mnt/storage/rawdata/data [DEBUG] 2013-09-10 11:20:23,901 [pbpy.smrtpipe.SmrtDataService put 191] SmrtDataService (oid 488599504) repository has 2 items [INFO] 2013-09-10 11:20:23,901 [pbpy.smrtpipe.SmrtDataService setupOutputDirectories 132] Creating /net/fs01/mnt/storage/rawdata/results [DEBUG] 2013-09-10 11:20:23,902 [pbpy.smrtpipe.SmrtDataService put 191] SmrtDataService (oid 488599504) repository has 3 items [DEBUG] 2013-09-10 11:20:23,902 [pbpy.smrtpipe.SmrtDataService setupOutputDirectories 130] Skipping dir creation of log. Found /net/fs01/mnt/storage/rawdata/log [DEBUG] 2013-09-10 11:20:23,902 [pbpy.smrtpipe.SmrtDataService put 191] SmrtDataService (oid 488599504) repository has 4 items [INFO] 2013-09-10 11:20:23,902 [pbpy.smrtpipe.SmrtDataService setupOutputDirectories 136] Completed setting up DataServices in /net/fs01/mnt/storage/rawdata [DEBUG] 2013-09-10 11:20:23,902 [pbpy.smrtpipe.SmrtDataService put 191] SmrtDataService (oid 488599504) repository has 5 items [INFO] 2013-09-10 11:20:23,902 [pbpy.smrtpipe.SmrtPipeContext initializeContext 250] Input data is [ run/0000000-0000 ] [INFO] 2013-09-10 11:20:23,959 [pbpy.smrtpipe.Heartbeat __init__ 41] heartbeat sleepTime set to 60 [DEBUG] 2013-09-10 11:20:23,959 [pbpy.smrtpipe.SmrtPipeContext initializeContext 307] No reference found or given [DEBUG] 2013-09-10 11:20:23,959 [pbpy.smrtpipe.SmrtPipeContext initializeContext 309] Completed initializeContext [DEBUG] 2013-09-10 11:20:23,960 [pbpy.smrtpipe.SmrtPipeContext movieFiles 360] Reading reads data from [ run/0000000-0000 ] [INFO] 2013-09-10 11:20:23,960 [pbpy.smrtpipe.SmrtPipeContext movieFiles 364] Pulling data from tags: * [INFO] 2013-09-10 11:20:23,960 [pbpy.smrtpipe.SmrtPipeMain _getBasVersions 463] Detected basH5 versions set([]) [INFO] 2013-09-10 11:20:23,960 [pbpy.smrtpipe.SmrtPipeMain run 551] smrtpipe.py running on n003.genomics.pssclabs.com n003.genomics.pssclabs.com [INFO] 2013-09-10 11:20:23,960 [pbpy.smrtpipe.SmrtPipeMain run 567] SMRT Analysis v2.0.0 / SMRTpipe 1.58.120456 [INFO] 2013-09-10 11:20:23,960 [pbpy.smrtpipe.SmrtPipeMain run 570] Starting smrtpipe version 1.58.120456 [DEBUG] 2013-09-10 11:20:23,961 [pbpy.smrtpipe.SmrtReportService generateFinalReport 558] jobInfo None dataUrl xml:input.xml paramsFile HGAP_Assembly_Advanced.1.xml [INFO] 2013-09-10 11:20:23,961 [pbpy.smrtpipe.SmrtReportService generateFinalReport 593] Skipping JNLP creation because no job id or no cmp.h5 [DEBUG] 2013-09-10 11:20:23,962 [pbpy.smrtpipe.SmrtReportService _generateIndexHtml 537] Writing Report XML to /net/fs01/mnt/storage/rawdata/toc.xml [DEBUG] 2013-09-10 11:20:23,963 [pbpy.smrtpipe.SmrtReportService _generateIndexHtml 546] Generating html file from xsl /opt/smrtanalysis/analysis/etc/xsl/toc.xsl [DEBUG] 2013-09-10 11:20:23,967 [pbpy.smrtpipe.SmrtWorker backticks 63] Running on n003.genomics.pssclabs.com with cmd saxonb9 -xsl:/opt/smrtanalysis/analysis/etc/xsl/toc.xsl -s:/net/fs01/mnt/storage/rawdata/toc.xml -o:/net/fs01/mnt/storage/rawdata/index.html [DEBUG] 2013-09-10 11:20:24,442 [pbpy.smrtpipe.SmrtWorker backticks 87] Successful output (Return code = 0) in 0.47 sec (0.01 min) of saxonb9 -xsl:/opt/smrtanalysis/analysis/etc/xsl/toc.xsl -s:/net/fs01/mnt/storage/rawdata/toc.xml -o:/net/fs01/mnt/storage/rawdata/index.html [DEBUG] 2013-09-10 11:20:24,442 [pbpy.smrtpipe.SmrtPipeContext exe 449] Output = [] [DEBUG] 2013-09-10 11:20:24,442 [pbpy.smrtpipe.SmrtPipeContext exe 451] Error Code = 0 [DEBUG] 2013-09-10 11:20:24,442 [pbpy.smrtpipe.SmrtPipeContext exe 452] Error Message = [DEBUG] 2013-09-10 11:20:24,443 [pbpy.smrtpipe.SmrtReportService _toRdf 314] Adding Auxiliary Files to RDF. [DEBUG] 2013-09-10 11:20:24,443 [pbpy.smrtpipe.SmrtReportService _toRdf 355] No barcodeFasta to add to ReportService [DEBUG] 2013-09-10 11:20:24,444 [pbpy.smrtpipe.SmrtReportService generateFinalReport 611] Writing Metadata XML to /net/fs01/mnt/storage/rawdata/metadata.rdf [DEBUG] 2013-09-10 11:20:24,446 [pbpy.smrtpipe.engine.SmrtPipeFiles __call__ 53] Adding instance file://jobId/input.fofn with key ('file', 'input.fofn', None) [DEBUG] 2013-09-10 11:20:24,446 [pbpy.smrtpipe.engine.SmrtPipeFiles __call__ 53] Adding instance file://jobId/results/overview.xml with key ('file', 'results/overview.xml', None) [DEBUG] 2013-09-10 11:20:24,446 [pbpy.smrtpipe.engine.SmrtPipeFiles __call__ 53] Adding instance file://jobId/data/chemistry_mapping.xml with key ('file', 'data/chemistry_mapping.xml', None) [DEBUG] 2013-09-10 11:20:24,446 [pbpy.smrtpipe.engine.SmrtPipeTasks task 48] Setting default Task type to LocalTask [DEBUG] 2013-09-10 11:20:24,446 [pbpy.smrtpipe.engine.SmrtPipeTasks task 48] Setting default Task type to LocalTask [DEBUG] 2013-09-10 11:20:24,446 [pbpy.smrtpipe.engine.SmrtPipeTasks task 48] Setting default Task type to LocalTask [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.SmrtPipeMain _dynamicallyLoadModule 895] Loaded pbpy [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.SmrtPipeMain _dynamicallyLoadModule 902] Dynamically loaded module smrtpipe [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.SmrtPipeMain _dynamicallyLoadModule 902] Dynamically loaded module modules [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.SmrtPipeMain _dynamicallyLoadModule 902] Dynamically loaded module P_Fetch [INFO] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.SmrtPipeMain _dynamicallyLoadModule 913] Dynamically Loaded P_Fetch module [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.modules.P_Module setting 76] P_Fetch v1.1.120456 looking for setting p_fetch.getChemistry [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.modules.P_Module setting 83] Found p_fetch.getChemistry with True for P_Fetch v1.1.120456 settings. [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.modules.P_Module setting 76] P_Fetch v1.1.120456 looking for setting p_fetch.overviewRpt [DEBUG] 2013-09-10 11:20:24,447 [pbpy.smrtpipe.modules.P_Module setting 83] Found p_fetch.overviewRpt with True for P_Fetch v1.1.120456 settings. [DEBUG] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.modules.P_Module setting 76] P_Fetch v1.1.120456 looking for setting p_fetch.toFofn [DEBUG] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.modules.P_Module setting 83] Found p_fetch.toFofn with True for P_Fetch v1.1.120456 settings. [DEBUG] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.modules.P_Fetch validateSettings 69] Reordered inputFiles to: [ERROR] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.modules.P_Fetch validateSettings 74] P_Fetch failed with errors No input bas.h5 files found. Analysis cannot be completed. [ERROR] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.SmrtPipeMain _loadModules 198] No input bas.h5 files found. Analysis cannot be completed. [INFO] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.SmrtPipeMain exit 780] Beginning exit process [INFO] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.SmrtPipeMain shutdown 800] beginning Shutdown Process [INFO] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.SmrtPipeMain _generate_final_report 731] Writing final TOC page [DEBUG] 2013-09-10 11:20:24,448 [pbpy.smrtpipe.SmrtReportService merge_from_serialized_datastore 192] Unable to find DataStore /net/fs01/mnt/storage/rawdata/data/data.items.pickle for merging. [DEBUG] 2013-09-10 11:20:24,449 [pbpy.smrtpipe.SmrtReportService generateFinalReport 558] jobInfo None dataUrl xml:input.xml paramsFile HGAP_Assembly_Advanced.1.xml [INFO] 2013-09-10 11:20:24,449 [pbpy.smrtpipe.SmrtReportService generateFinalReport 593] Skipping JNLP creation because no job id or no cmp.h5 [DEBUG] 2013-09-10 11:20:24,450 [pbpy.smrtpipe.SmrtReportService _generateIndexHtml 537] Writing Report XML to /net/fs01/mnt/storage/rawdata/toc.xml [DEBUG] 2013-09-10 11:20:24,450 [pbpy.smrtpipe.SmrtReportService _generateIndexHtml 546] Generating html file from xsl /opt/smrtanalysis/analysis/etc/xsl/toc.xsl [DEBUG] 2013-09-10 11:20:24,454 [pbpy.smrtpipe.SmrtWorker backticks 63] Running on n003.genomics.pssclabs.com with cmd saxonb9 -xsl:/opt/smrtanalysis/analysis/etc/xsl/toc.xsl -s:/net/fs01/mnt/storage/rawdata/toc.xml -o:/net/fs01/mnt/storage/rawdata/index.html [DEBUG] 2013-09-10 11:20:24,929 [pbpy.smrtpipe.SmrtWorker backticks 87] Successful output (Return code = 0) in 0.48 sec (0.01 min) of saxonb9 -xsl:/opt/smrtanalysis/analysis/etc/xsl/toc.xsl -s:/net/fs01/mnt/storage/rawdata/toc.xml -o:/net/fs01/mnt/storage/rawdata/index.html [DEBUG] 2013-09-10 11:20:24,930 [pbpy.smrtpipe.SmrtPipeContext exe 449] Output = [] [DEBUG] 2013-09-10 11:20:24,930 [pbpy.smrtpipe.SmrtPipeContext exe 451] Error Code = 0 [DEBUG] 2013-09-10 11:20:24,930 [pbpy.smrtpipe.SmrtPipeContext exe 452] Error Message = [DEBUG] 2013-09-10 11:20:24,930 [pbpy.smrtpipe.SmrtReportService _toRdf 314] Adding Auxiliary Files to RDF. [DEBUG] 2013-09-10 11:20:24,930 [pbpy.smrtpipe.SmrtReportService _toRdf 355] No barcodeFasta to add to ReportService [DEBUG] 2013-09-10 11:20:24,931 [pbpy.smrtpipe.SmrtReportService generateFinalReport 611] Writing Metadata XML to /net/fs01/mnt/storage/rawdata/metadata.rdf [INFO] 2013-09-10 11:20:24,932 [pbpy.smrtpipe.SmrtPipeMain shutdown 835] Completed shutdown gracefully [CRITICAL] 2013-09-10 11:20:24,932 [pbpy.smrtpipe.SmrtPipeMain exit 789] hard exiting smrtpipe v1.58.120456 with returncode -1 [DEBUG] 2013-09-10 11:20:24,933 [pbpy.smrtpipe.SmrtPipeMain _releaseLock 417] -- Released lock (/net/fs01/mnt/storage/rawdata/smrt.lock) for this output directory. [DEBUG] 2013-09-10 11:20:24,934 [pbpy.smrtpipe.SmrtPipeMain run 688] Shutting down heartbeat service [INFO] 2013-09-10 11:21:23,991 [pbpy.smrtpipe.Heartbeat _startHeartbeat 65] Heartbeat attempting to stop
Does anyone know if I should change a parameter in 'HGAP_Assembly_Advanced.1.xml' protocol in order to use fastQ files as input for HGAP?
Regards and thanks in advance,
Boetsie
Comment