Failing to generate CCS data using SMRT Link 7.0.0.63985

[paxas]

Hi, first of all apologies if this has been posted previously; I'm using SMRT Link 7.0.0.63985 and whereas I could previously use the software to generate CCS data using RSII .XML metadata files as an input (after converting RS to Bam), it doesn't seem possible using sequel .xml files? What happens is that after successfully importing the sequel .xml data file I try to do a CCS analysis which results to the following failed message just after a second:
"FAILED: Task pbcoretools.tasks.filterdataset-0 FAILED in 0.80 sec."

The full error message I get is quite long:
"Pbsmrtpipe job /home/tasos0104/smrtlink_7.0.0.63985/smrtlink/userdata/jobs_root/000/000043 failed with exit code 1. Task pbcoretools.tasks.filterdataset-0 Failed task pbcoretools.tasks.filterdataset exit code 1 in 0.78 sec (See file '/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/userdata/jobs_root/000/000043/tasks/pbcoretools.tasks.filterdataset-0/stderr'.) Extracted from stderr File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/pacbio/pythonpkgs/pbcoretools/lib/python2.7/site-packages/pbcoretools/tasks/filters.py", line 172, in <module> sys.exit(registry_runner(registry, sys.argv[1:])) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcommand/cli/quick.py", line 360, in registry_runner return_code = args.func(args) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcommand/cli/quick.py", line 279, in _w exit_code = func(rtc) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/pacbio/pythonpkgs/pbcoretools/lib/python2.7/site-packages/pbcoretools/tasks/filters.py", line 126, in run_filterDataSet rtc.task.options["pbcoretools.task_options.downsample_factor"]) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/pacbio/pythonpkgs/pbcoretools/lib/python2.7/site-packages/pbcoretools/tasks/filters.py", line 101, in run_filter_dataset dataSet = openDataSet(in_file, skipCounts=True) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcore/io/dataset/DataSetIO.py", line 63, in openDataSet return tbrType(*files, **kwargs) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcore/io/dataset/DataSetIO.py", line 2705, in __init__ super(SubreadSet, self).__init__(*files, **kwargs) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcore/io/dataset/DataSetIO.py", line 1994, in __init__ super(ReadSet, self).__init__(*files, **kwargs) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcore/io/dataset/DataSetIO.py", line 386, in __init__ self._modResources(_fileExists) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcore/io/dataset/DataSetIO.py", line 1252, in _modResources func(item) File "/home/tasos0104/smrtlink_7.0.0.63985/smrtlink/install/smrtlink-release_7.0.0.63985/bundles/smrttools/install/smrttools-release_7.0.0.63823/private/thirdparty/python/python_2.7.9/site-packages/pbcore/io/dataset/DataSetIO.py", line 255, in _fileExists raise InvalidDataSetIOError("Resource {f} not found".format(f=fname)) pbcore.io.dataset.DataSetErrors.InvalidDataSetIOError: Resource /mnt/ruby/PacBio/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrtinub/current/private/pacbio/barcodes/Sequel_RSII_16_barcodes_v2/Sequel_RSII_16_barcodes_v2.barcodeset.xml not found"

Any ideas what's going on?

Cheers,
paxas

[sklages]

Have you already contacted PacBio's techsupport? They are usually very helpful.

[paxas]

I have but on this occasion they couldn't provide much help, given also that we don't have a sequel machine ourselves but instead we are using an external provider to get the data. They did mention though that processing sequel or sequel II data in a machine that doesn't meet the specifications of the head node (i.e. 32 cores with at least 64Gb of RAM) most probably won't work. Our machine doesn't meet those specs but I would expect it to still work albeit at much lower speed?

[rhall]

EDIT.
Is /mnt/ruby/PacBio/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrtinub/current/private/pacbio/barcodes/Sequel_RSII_16_barcodes_v2/Sequel_RSII_16_barcodes_v2.barcodeset.xml your system? It looks like the data was generated from demultiplexing on a different and older SMRT Link system.


If you are not concerned with how long it takes, why not just use the bioconda version of CCS on a single node https://github.com/PacificBiosciences/ccs

[paxas]

Hi, rhall
To answer to your questions before the edit:
I have installed smrt link v7.0 in a couple of different boot drives from scratch and it doesn't seem to make a difference. Also, I can analyse and generate ccs data from RSII using the same SMRT link instance.

I find that highly unlikely as the data came from the provider and they can't be using release 6.0.
I can give it one more go and see if I can get it to work.

No, I am not that concerned, thanks for the link; I'll try it out.
cheers