Myrna stops in statistics stage

[Serena Rhie]

Hi,


I'm trying to align on myrna some human RNA seq, paired, illumina with Ensembl 58 reference as on the myrna manual.

This is my command:

$MYRNA_HOME/myrna_local --reference /home/Serena/myrna-1.0.9/reftools/human_ensembl_58 --input /media/SAMSUNG/Data/myrna/paired_8M_preprocess_only/paired_8M_preprocessed --output /media/SAMSUNG/Data/myrna/paired_8M_preprocess_only/output --family gaussian --quality solexa64 --keep-all --cpus 8 --bowtie-args "-m 1 -n 3 -l 10 -q -I 450 -X 550"

I've tried with/without the --family gaussian option; however getting the same result.

And here I'm getting this error message:

Pid 31271 reducing task task-00003 [4 of 32]...
Pid 31268 reducing task task-00000 [1 of 32]...
******
* Aborting master loop because child 31275 failed
* (other children may also have failed)
* Input file or string was:
*   /tmp/myrna/intermediate/27481/stats.reduce.pre/reduce.stasks/stask-31275
* Error message is in file: /tmp/myrna/intermediate/27481/stats.reduce.pre/reduce.err/epart-31275, also printed below
******
* Stats.pl: Family: gaussian
* Stats.pl: # nulls per gene: 0
* Stats.pl: seed (-1 = let R decide): -1
* Stats.pl: Bypass P-value calculation: 0
* Stats.pl: Profiling enabled: 0
* Stats.pl: Add fudge factor: 0
* Stats.pl: Samples are paired: 0
* Get.pm:lsDir: About to parse URL /tmp/myrna/intermediate/27481/globals/multiset/label/
* Get.pm:lsDir: About to handle local
* ls -1 /tmp/myrna/intermediate/27481/globals/multiset/label/
* Result of get_mset_global('label'): 0
* /home/Serena/myrna-1.0.9/R/bin/Rscript --vanilla --default-packages=stats,zoo,grDevices,graphics,utils,methods,lmtest,MASS /home/Serena/myrna-1.0.9/Stats.R --args .tmp.Stats.pl.31279 0 gaussian 0 -1 0 0 0 0
* runAndWait: Child's PID is 31412
* Stats.R [10:55:35]: Called deTest(.tmp.Stats.pl.31279, 0, gaussian, FALSE, 0, FALSE)
* Read 8065 records
* Stats.R [10:55:35]: Processing batch of 8065 alignments
* Stats.R [10:55:35]: Alignment batch has 205 genes
* Stats.R [10:55:35]: Genes: ENSG00000003987G ENSG00000006015G ... ENSG00000243646G ENSG00000244509G
* Stats.R [10:55:35]: Alignment batch has 1 distinct labels
* Stats.R [10:55:35]: Labels: 0
* Stats.R [10:55:35]: Whole dataset has 1 distinct labels
* Stats.R [10:55:35]: Labels: 0
* Stats.R [10:55:35]: Batch has 1 distinct groups
* Stats.R [10:55:35]: Groups: 0
* Stats.R [10:55:35]: Whole dataset has 1 distinct groups
* Stats.R [10:55:35]: Groups: 0
* Stats.R [10:55:35]: Factorizing label column
* Stats.R [10:55:35]: Getting per-sample normalization factors
* Stats.R [10:55:35]: Setting up data matrix
* Stats.R [10:55:35]: Setting up output strings
* Stats.R [10:55:35]: Calculating batch of 205 observed P-values
* reporter:counter:Stats,Observed Pval batches calculated,1 
* reporter:counter:Stats,Observed Pvals calculated,205 
* Error in `contrasts<-`(`*tmp*`, value = "contr.treatment") : 
*   contrasts can be applied only to factors with 2 or more levels
* Calls: deTest ... model.matrix -> model.matrix.default -> contrasts<-
* Execution halted
* Waiting for R (it's been 5 secs)...
* cp: cannot stat `*': No such file or directory
* cp: omitting directory `.'
* cp: omitting directory `..'
* Exitlevel 256 from command /home/Serena/myrna-1.0.9/R/bin/Rscript --vanilla --default-packages=stats,zoo,grDevices,graphics,utils,methods,lmtest,MASS /home/Serena/myrna-1.0.9/Stats.R --args .tmp.Stats.pl.31279 0 gaussian 0 -1 0 0 0 0

The thing is, I'm getting the same error message on my local computer as well as on the web-based interface.

Do I need to provide at least 2 groups in the manifast file?
Maybe, I've totally misunderstood myrna - is the reference only used for aligning? Does the RNA seq not beeing compaired with the reference at all?

Thank you in advance,
Serena Rhie

[Ben Langmead]

Hi Serena,

* Error in `contrasts<-`(`*tmp*`, value = "contr.treatment") : 
*   contrasts can be applied only to factors with 2 or more levels

Looks like your input is all of the same group - so the R code is getting confused when it tries to fit the model as if there are 2 or more groups ("factors"). I will fix this so that it produces a clearer, friendlier error message in a future version of Myrna.

Hope this helps,
Ben

[Serena Rhie]

Hi Ben,

Looks like your input is all of the same group

You are right, we tried it with one group of sequences. I'll try with 2 groups.

One more question - I've noticed that the manifest file about <label> differs from that in distributed myrna-1.0.9/examples/human.

Your manual says:

`<Group ID>-<BioRep ID>-<TechRep ID>`

But the example file looks like this:

# Human reads from the Gilad et al study; note: study also has reads for chimp and rhesus
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2116.fastq.bz2 0 HF-11
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2117.fastq.bz2 0 HF-12
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2118.fastq.bz2 0 HF-21
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2119.fastq.bz2 0 HF-22
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2120.fastq.bz2 0 HF-31
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2121.fastq.bz2 0 HF-32
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2122.fastq.bz2 0 HM-11
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2123.fastq.bz2 0 HM-12
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2124.fastq.bz2 0 HM-21
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2125.fastq.bz2 0 HM-22
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2126.fastq.bz2 0 HM-31
ftp://ftp.ncbi.nlm.nih.gov/sra/stati...2127.fastq.bz2 0 HM-32

Which one is the correct one?
I see myrna is running fairly well on web-based interface with your sample manifest human full.manifest.
But that does not contains the BioRep-ID (if I'm correctly understanding).
If BioRep-ID is qualified, does the BioRep-ID to be different when there are only are only 2 lines of reads?

Our manifest file looks like this:

/media/SAMSUNG/Data/AK3/paired_8M/s_1_1_sequence_8M.txt 0       /media/SAMSUNG/Data/AK3/paired_8M/s_1_2_sequence_8M.txt 0  Normal-Subject1-Lane1
/media/SAMSUNG/Data/AK3/paired_8M/s_2_1_sequence_8M.txt 0       /media/SAMSUNG/Data/AK3/paired_8M/s_2_2_sequence_8M.txt 0  Tumor-Subject1-Lane1

Thanks,
Serena Rhie

[Ben Langmead]

Hi Serena,

Good catch! - we are using an older version of the format string in that manifest file. Our mistake won't affect how Myrna determines each URL's group unless one of the --pool* options is used. If --pool-tech-reps is used, our incorrect labeling will result in the bio reps being pooled instead of the tech reps. I'll fix the manifest file so that it uses the new format.

Your format looks correct.

Hope that helps,
Ben

[Serena Rhie]

Hello Ben,

Thank you for your advice.

Now, I tried with the modified manifest file above:

/path/to/reads/s_1_1_sequence_8M.txt 0 /path/to/reads/s_1_2_sequence_8M.txt 0 Normal-Subject1-Lane1
/path/to/reads/s_2_1_sequence_8M.txt 0 /path/to/reads/s_2_2_sequence_8M.txt 0 Tumor-Subject1-Lane1

But Get stuck in Stage 3 of 6: Normalize and the error message says:

Factored 32 files into 4 non-empty tasks
Pid 4307 sorting task task-00000 [1 of 4]...
Pid 4308 sorting task task-00001 [2 of 4]...
Pid 4309 sorting task task-00002 [3 of 4]...
Pid 4310 sorting task task-00003 [4 of 4]...
Pid 4310 reducing task task-00003 [4 of 4]...
Pid 4309 reducing task task-00002 [3 of 4]...
Pid 4308 reducing task task-00001 [2 of 4]...
Pid 4307 reducing task task-00000 [1 of 4]...
******
* Aborting master loop because child 4308 failed
* (other children may also have failed)
* Input file or string was:
* /tmp/myrna/intermediate/2781/normal.reduce.pre/reduce.stasks/stask-04308
* Error message is in file: /tmp/myrna/intermediate/2781/normal.reduce.pre/reduce.err/epart-04308, also printed below
******
* Normalization: lup
* Output URL: /tmp/myrna/intermediate/2781/count
* Processing label Normal-Subject1-Lane1/2
* stat /tmp/myrna/intermediate/2781/count/Normal-Subject1-Lane1/2.txt >& /dev/null
* stat /tmp/myrna/intermediate/2781/count/Normal-Subject1-Lane1/2.norm >& /dev/null
* stat /tmp/myrna/intermediate/2781/count/Normal-Subject1-Lane1/2.norms >& /dev/null
* mv: cannot move `.tmp.Normal.pl.4316' to `Normal-Subject1-Lane1/2.txt': No such file or directory
* stat /tmp/myrna/intermediate/2781/count/2.txt >& /dev/null
* cp Normal-Subject1-Lane1/2.txt /tmp/myrna/intermediate/2781/count/ >&2
* cp: cannot stat `Normal-Subject1-Lane1/2.txt': No such file or directory
* Can't load Normal-Subject1-Lane1/2.txt to /tmp/myrna/intermediate/2781/count/ (256)
******

Seems like we have an issue during reducing the results.
Could you provide some suggestions?

Our command was:

$MYRNA_HOME/myrna_local --reference $MYRNA_HOME/reftools/human_ensembl_58 --input preprocessed --output output --quality solexa64 --keep-all --cpus 8 --bowtie-args "-m 1 -n 3 -l 10 -q -I 450 -X 550"

And in order to skip the align, overlap process - are there any other options where I could try again from Normalize stage?

Thanks a lot,
Serena Rhie

[Ben Langmead]

Hi Serena,

Sorry for the delay, I will look into this ASAP. Could you send me the complete output from myrna_local? You can email it to me if you like (blangmea at jhsph.edu).

Thanks,
Ben

[Serena Rhie]

Thank you Ben,
I've mailed the output within the /tmp/myrna/intermediate/2781 folder.

Sorry for a bit of delay.
Hope this helps investigating!

[ecwooten]

Any resolution?

I'm having the same issue:

Factored 12 files into 6 non-empty tasks
Pid 30933 sorting task task-00000 [1 of 6]...
Pid 30934 sorting task task-00001 [2 of 6]...
Pid 30935 sorting task task-00002 [3 of 6]...
Pid 30933 reducing task task-00000 [1 of 6]...
Pid 30935 reducing task task-00002 [3 of 6]...
******
* Aborting master loop because child 30935 failed
* (other children may also have failed)
* Input file or string was:
* /private/tmp/myrna/intermediate/15202/normal.reduce.pre/reduce.stasks/stask-30935
* Error message is in file: /private/tmp/myrna/intermediate/15202/normal.reduce.pre/reduce.err/epart-30935, also printed below
******
* Normalization: lup
* Output URL: /private/tmp/myrna/intermediate/15202/count
* Processing label FetalHeart_3
* stat /private/tmp/myrna/intermediate/15202/count/FetalHeart_3.txt >& /dev/null
* stat /private/tmp/myrna/intermediate/15202/count/FetalHeart_3.norm >& /dev/null
* stat /private/tmp/myrna/intermediate/15202/count/FetalHeart_3.norms >& /dev/null
* stat /private/tmp/myrna/intermediate/15202/count/FetalHeart_3.txt >& /dev/null
* cp FetalHeart_3.txt /private/tmp/myrna/intermediate/15202/count/ >&2
* stat /private/tmp/myrna/intermediate/15202/count/FetalHeart_3.norm >& /dev/null
* cp FetalHeart_3.norm /private/tmp/myrna/intermediate/15202/count/ >&2
* stat /private/tmp/myrna/intermediate/15202/count/FetalHeart_3.norms >& /dev/null
* cp FetalHeart_3.norms /private/tmp/myrna/intermediate/15202/count/ >&2
* FetalHeart_3: total 749677
* FetalHeart_3: distinct non-zero counts 10606
* FetalHeart_3: maximum 148998
* FetalHeart_3: upper quartile 20
* FetalHeart_3: median 7
* FetalHeart_3: lower quartile 2
* reporter:counter:Normal,Label FetalHeart_3 total,749677
* reporter:counter:Normal,Label FetalHeart_3 distinct non-zero counts,10606
* reporter:counter:Normal,Label FetalHeart_3 maximum,148998
* reporter:counter:Normal,Label FetalHeart_3 upper quartile,20
* reporter:counter:Normal,Label FetalHeart_3 median,7
* reporter:counter:Normal,Label FetalHeart_3 lower quartile,2
* Processing label HCM-Sample4-Lane1/1
* stat /private/tmp/myrna/intermediate/15202/count/HCM-Sample4-Lane1/1.txt >& /dev/null
* stat /private/tmp/myrna/intermediate/15202/count/HCM-Sample4-Lane1/1.norm >& /dev/null
* stat /private/tmp/myrna/intermediate/15202/count/HCM-Sample4-Lane1/1.norms >& /dev/null
* mv: rename .tmp.Normal.pl.30984 to HCM-Sample4-Lane1/1.txt: No such file or directory
* stat /private/tmp/myrna/intermediate/15202/count/1.txt >& /dev/null
* cp HCM-Sample4-Lane1/1.txt /private/tmp/myrna/intermediate/15202/count/ >&2
* cp: HCM-Sample4-Lane1/1.txt: No such file or directory
* Can't load HCM-Sample4-Lane1/1.txt to /private/tmp/myrna/intermediate/15202/count/ (256)
******
Pid 30934 reducing task task-00001 [2 of 6]...
Fatal error 1.0.9:R330: Aborting because child with PID 30934 exited abnormally

with this command:
./myrna_local --input=$MYRNA_HOME/hcm.manifest --preprocess --reference=$MYRNA_REFS/human_ensembl_58 --output=output_hcm --cpus=3

[Rachelly]

Did anyone manage to solve this problem? I'm having a similar problem with the following error:

******
* Aborting master loop because child 7999 failed
* (other children may also have failed)
* Input file or string was:
*   /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/normal.reduce.pre/reduce.stasks/stask-07999
* Error message is in file: /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/normal.reduce.pre/reduce.err/epart-07999, also printed below
******
* Normal.pl: Normalization: ltot
* Normal.pl: Output URL:    /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/count
* Normal.pl: Counters file: /illumina/runs/110406_SN716_0054_BB0556ABXX/Data/Intensities/BaseCalls/filtered_fastq/FinalMapping/MyrnaForReplicates/myrna_out_counters/counters.txt
* Normal.pl: Retrived 2 counters from previous stages
* Processing label Rep2-sub1-lane/1
* stat /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/count/Rep2-sub1-lane/1.txt >& /dev/null
* stat /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/count/Rep2-sub1-lane/1.norm >& /dev/null
* stat /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/count/Rep2-sub1-lane/1.norms >& /dev/null
* mv: cannot move `.tmp.Normal.pl.8226' to `Rep2-sub1-lane/1.txt': No such file or directory
* stat /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/count/1.txt >& /dev/null
* cp Rep2-sub1-lane/1.txt /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/count/ >&2
* cp: cannot stat `Rep2-sub1-lane/1.txt': No such file or directory
* Can't load Rep2-sub1-lane/1.txt to /tmp/myrna-wgGT7sgmOc/myrna/intermediate/27055/count/ (256)
******
Fatal error 1.1.1:R330: Aborting because child with PID 7999 exited abnormally