[rhall]

The raw data is not complete, it should look like this:

./
*.metadata.xml
./Analysis_Results
[the files you have]

[rhall]

Command line AHA including howto to go from filtered subreads and corrected reads:
https://github.com/PacificBioscience...ining/wiki/AHA

[sagarutturkar]

Command line AHA including howto to go from filtered subreads and corrected reads:
https://github.com/PacificBioscience...ining/wiki/AHA

I used the same guidelines and got error as:

[ERROR] 2013-02-19 16:21:52,547 [pbpy.smrtpipe.SmrtPipeMain run 648] invalid literal for int() with base 10:

I tried different input files (fasta, bas.h5) and different data, but same error for each of them. Not yet able to figure out whats going wrong.

About SMRT cells importing - kind of weired things happening.

I downloaded the complete data and imported it again. It showed a message "1 SMRT cell imported". However, in design jobs I do not see any SMRT cells available. I tried to use search option but no luck. Does import of SMRT cells need to be approved by administrator? I might be doing some silly mistake or missing some step

When I tried to re-import from same location, I got message "No New SMRT cells found".

Have you noticed something like this before.

Thanks
Sagar

[rhall]

The [ERROR] is a python exception, could you post the entire log so we can get some more context?
The SMRT cell should show up, it does not need to be approved, my only thought is that maybe you do not have a group assigned to you. If it is just a personal copy of SMRT portal, log in as administrator and check under 'admin' that you are in group 'all' (note you can place a user in multiple groups by 'ctrl' clicking), also check that your role is 'scientist' (although I'm not sure what the different roles actually mean).
SMRT Cells will only import once, any further attempt will give 'No New SMRT cells found', so it sounds like they are there somewhere.

[sagarutturkar]

The [ERROR]
The SMRT cell should show up, it does not need to be approved, my only thought is that maybe you do not have a group assigned to you.

Hi, I am able to import the SMRT cells now. But When I ran the "RS_AHA_scaffolding.1" protocol with my assembly as reference, the job failed.

I have attached log file for the job that failed through SMRTportal (CF80_AHA.zip).

The [ERROR] is a python exception, could you post the entire log so we can get some more context?

The log files for AHA job that failed through command line are also attached (smrtpipe.log.zip).

Many thanks for your help.

-Sagar

[rhall]

For the command line job, the AHA workflow should be ran from a filtered subreads file. You will not get sensible results from a bas.h5 file, although it should work. pls2fasta can be used to convert from bas.h5 to fasta
The SMRT portal job: It looks like the bas.h5 is corrupt, which would also explain the command line job failure. If this is the case, then pls2fasta will also fail, you can use h5dump to look at the contents of a h5 file.
I have seen cases of corrupt bas.h5 files when they are transferred over ftp incorrectly.

[sagarutturkar]

AHA commandline solved

For the command line job, the AHA workflow should be ran from a filtered subreads file. You will not get sensible results from a bas.h5 file, although it should work. pls2fasta can be used to convert from bas.h5 to fasta

I was able to run the AHA using command line and gave me scaffolds for the genome. I used fastX toolkit to convert filtered_subreads.fastq file to fasta format and used existing assembly as reference.

The XML files were created as described here:
https://github.com/PacificBioscience...ining/wiki/AHA

The command used to run smrtpipe was:

/opt/Pacbio/smartanalysis/analysis/bin/smrtpipe.py --params=params.xml xml:input.xml >& smrtpipe.err

Thanks to rhall and GenoMax for all the help. I am still trying to figure out running the SMRTportal and re-downloading the bas.h5 data.

Thanks
Sagar

[sagarutturkar]

I was able to run the AHA using command line and gave me scaffolds for the genome. I used fastX toolkit to convert filtered_subreads.fastq file to fasta format and used existing assembly as reference.


The command used to run smrtpipe was:

/opt/Pacbio/smartanalysis/analysis/bin/smrtpipe.py --params=params.xml xml:input.xml >& smrtpipe.err

However, when I revisit this for some other genome, I am getting errors again.

[ERROR] 2013-05-13 12:57:15,241 [pbpy.smrtpipe.SmrtPipeContext exit 557] Found error: Failed to find any links. Scaffolding failed.
[ERROR] 2013-05-13 12:57:15,241 [pbpy.smrtpipe.SmrtPipeMain run 648] SmrtExit Failed to find any links. Scaffolding failed.
Traceback (most recent call last):
  File "/data2/smart/smartanalysis/analysis/lib/python2.7/pbpy-0.1-py2.7.egg/pbpy/smrtpipe/SmrtPipeMain.py", line 612, in run
    self._runModules(sModules)
  File "/data2/smart/smartanalysis/analysis/lib/python2.7/pbpy-0.1-py2.7.egg/pbpy/smrtpipe/SmrtPipeMain.py", line 346, in _runModules
    instance.run()
  File "/data2/smart/smartanalysis/analysis/lib/python2.7/pbpy-0.1-py2.7.egg/pbpy/smrtpipe/modules/HybridAssembly.py", line 518, in run
    self._context.exit("Failed to find any links. Scaffolding failed.")
  File "/data2/smart/smartanalysis/analysis/lib/python2.7/pbpy-0.1-py2.7.egg/pbpy/smrtpipe/SmrtPipeContext.py", line 558, in exit
    raise SmrtExit(msg)
SmrtExit: SmrtExit Failed to find any links. Scaffolding failed.
[ERROR] 2013-05-13 12:57:15,243 [pbpy.smrtpipe.SmrtPipeMain exit 760] SmrtExit Failed to find any links. Scaffolding failed.

I have attached complete log files. Please help with this.

Thanks
Sagar

[rhall]

The error is simply that no reads have been found that link contigs. You can try changing the parameters:

https://github.com/PacificBiosciences/SMRT-Analysis/wiki/SMRT-Pipe-Reference-Guide-v2.0#parameters-5

I would also double check the reads, by mapping them to the contigs using a standard resequencing protocol.

[GenoMax]

rhall: You have linked the reference guide for smrtanalysis v.2.0. Is that applicable for v.1.4 (in case Sagar is still using that version).

We could never get the v.1.4 to work (smrtpipe.py) on the command line.

We are currently working with tech support to get just test data to work with v.2.0 (HGAP protocol) through SMRTportal. Have not thought about command line yet.

[rhall]

GenoMax:
Unfortunately the reference guide for 1.4 is not as easy to link to (pdf file). That protocol in particular has not changed, and the documentation will be the same.
For 1.4, you had SMRT portal working, but not command line smrtpipe.py? Given that SMRT portal basically generates parameter files then runs smrtpipe.py this would seem like it should have been a simple fix.

[GenoMax]

GenoMax:
For 1.4, you had SMRT portal working, but not command line smrtpipe.py? Given that SMRT portal basically generates parameter files then runs smrtpipe.py this would seem like it should have been a simple fix.

One would think so but the command line usage is disconnected from the SMRTportal in ways we do not fully understand (and have run into for the last 2-3 software revisions).

We use SGE on one cluster. smrtpipe.py when submitted from a qsub script job generates sub-"qsub" jobs on its own .. which SGE does not like.

If you want to know more we can confer off-line as I do not want to hijack this thread.

[rhall]

Some recent test data is available from:

https://github.com/PacificBiosciences/DevNet/wiki/E.-coli-20kb-Size-Selected-Library-with-P4-C2

This is Ecoli data, and has been used to demonstrate HGAP assembly. You can download the data, import it into SMRT Portal and used it as a test case for HGAP, Base Modification and resequencing pipelines.

[sagarutturkar]

PacBio RS|| data

Hi,

We recently got some Data from PacBio RS|| system which looks like this.

10061/
10061/PACBIO_DATA/
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02.ccs_stats.txt
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02.ccs.fastq
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02.1.1.bax.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02_stats.txt
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02.1.3.bax.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02.1.bas.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02_stats.txt
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02.1.1.bax.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02.ccs.fastq
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02.1.2.bax.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02.1.3.bax.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02.1.bas.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02.1.2.bax.h5
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_D02.fastq
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02.ccs_stats.txt
10061/PACBIO_DATA/XSBOR_20130726_RS42150_PL100036291-1_C02.fastq

However, When I copied same folder in inputs_dropbox folder and tried to import - It says no new SMRT cells to import. What might be the problem? After importing I changed the folder name. Also, I do not see the

*.metadata.xml
./Analysis_Results

folders as before. Any suggestions?

[rhall]

It looks like the files have been moved around and renamed, the simplest solution is probably to ask for the files again from the service provider in the original state.

https://github.com/PacificBiosciences/SMRT-Analysis/wiki/Cannot-import-new-SMRT-Cells

As a general rule, raw data from the instrument should not be moved from the generated directory structure and .h5 files should never be renamed.
Richard.