Problem with the tol CoNiFer

[vishal.rossi]

Hi,

Does anyone here has ever worked with conifer.
I have this strange kind of error.
I am running 240 samples but it shows some problems with probes on cgr24.
I used grep -v '24' from the probes.txt file as it says to do so but it was of no use.
Before this, I used conifer on ~500 samples and it's working perfectly fine.
Can anyone give me some idea how to fix this issue.
Thanks in advance
Vishal


Error while analysing:
[RUNNING: chr23] Masking 92 probes with median RPKM < 1.000000
[RUNNING: chr23] Calculating ZRPKM scores...
[RUNNING: chr23] SVD decomposition...
[RUNNING: chr23] Saving SVD-ZRPKM values
[RUNNING: chr24] Now on: chrY
[RUNNING: chr24] Found 465 probes; probeID range is [193615-194080]
[RUNNING: chr24] Calculating median RPKM
[RUNNING: chr24] Masking 432 probes with median RPKM < 1.000000
[ERROR] This chromosome has fewer informative probes than there are samples in the analysis! There are probably no mappings on this chromosome. Please remove these probes from the probes.txt file
Closing remaining open files: /vishal/conifer/ige/output/analysis.hd5... done

Error while calling :
[INIT] Initializing caller at threshold = 1.500000
Traceback (most recent call last):
File "conifer.py", line 682, in <module>
args.func(args)
File "conifer.py", line 338, in CF_call
r = cf.rpkm_reader(h5file_in_fn)
File "/sw/conifer_v0.2.2/conifer_functions.py", line 256, in __init__
self.sample_table = self.h5file.root.samples.samples
File "/opt/rrzk/software/python/python-2.7.5/lib/python2.7/site-packages/tables/group.py", line 813, in __getattr__
return self._f_get_child(name)
File "/opt/rrzk/software/python/python-2.7.5/lib/python2.7/site-packages/tables/group.py", line 683, in _f_get_child
self._g_check_has_child(childname)
File "/opt/rrzk/software/python/python-2.7.5/lib/python2.7/site-packages/tables/group.py", line 407, in _g_check_has_child
% (self._v_pathname, name))
tables.exceptions.NoSuchNodeError: group ``/`` does not have a child named ``samples``
Closing remaining open files: /scratch/ccg-ngs/tmp/vishal/conifer/ige/output/analysis.hd5... done

[vishal.rossi]

no one who has worked with conifer?

[Brajbio]

Hi Vishal,

I am facing the similar problem.
Were you able to fix this issue?

Thanks, Braj

[vishal.rossi]

Hi Braj,

I think that there is a bug in the tool, I wrote to the author.

Go to conifer folder and edit conifer.py
Go to line 153, 154 and 155 and comment them (put # in front of them) and then save it and run conifer.

#if num_chr_probes <= len(samples):
#print "[ERROR] This chromosome has fewer informative probes than there are samples in the analysis! There are probably no mappings on this chromosome. Please remove these probes from the probes.txt file"
# sys.exit(0)


Write to me again if it doesn't work.
Regards
Vishal

[Brajbio]

Hi Vishal,

Thanks for your reply.

I tried the way you suggested, but this time I am getting some new error at analysis step. The error which I am getting this time is :

.
.
.
[RUNNING: chr22] Now on: chr22
[RUNNING: chr22] Found 3961 probes; probeID range is [114481-118442]
[RUNNING: chr22] Calculating median RPKM
[RUNNING: chr22] Masking 49 probes with median RPKM < 1.000000
[RUNNING: chr22] Calculating ZRPKM scores...
[RUNNING: chr22] SVD decomposition...
[RUNNING: chr22] Saving SVD-ZRPKM values
[RUNNING: chr23] Now on: chrX
[RUNNING: chr23] Found 6157 probes; probeID range is [179253-185410]
[RUNNING: chr23] Calculating median RPKM
[RUNNING: chr23] Masking 87 probes with median RPKM < 1.000000
[RUNNING: chr23] Calculating ZRPKM scores...
[RUNNING: chr23] SVD decomposition...
[RUNNING: chr23] Saving SVD-ZRPKM values
[RUNNING: chr24] Now on: chrY
[RUNNING: chr24] Found 226 probes; probeID range is [185410-185636]
[RUNNING: chr24] Calculating median RPKM
[RUNNING: chr24] Masking 221 probes with median RPKM < 1.000000
[RUNNING: chr24] Calculating ZRPKM scores...
[RUNNING: chr24] SVD decomposition...
Traceback (most recent call last):
File "../conifer_v0.2.2/conifer.py", line 682, in <module>
args.func(args)
File "../conifer_v0.2.2/conifer.py", line 189, in CF_analyze
rpkm = np.dot(U, np.dot(new_S, Vt))
ValueError: objects are not aligned
Closing remaining open files: analysis.hdf5... done

-----------------------------
I want to give a try to troubleshoot the issue...Let me look at the script, if I could understand.

Please do share in case you got it fixed, it will be a great help.

Thanks & Regards,

Braj

[Brajbio]

Hi Vishal,

I tried to understand this issue what I understand is :

Since you have total 240 samples.

Now if you see the error which you get

.
.
.
[RUNNING: chr24] Found 465 probes; probeID range is [193615-194080]
[RUNNING: chr24] Calculating median RPKM
[RUNNING: chr24] Masking 432 probes with median RPKM < 1.000000
[ERROR] This chromosome has fewer informative probes than there are samples in the analysis! There are probably no mappings on this chromosome. Please remove these probes from the probes.txt file
Closing remaining open files: /vishal/conifer/ige/output/analysis.hd5... done
-----------------------------
In your "Y' chromosome out of 465 probes 432 are masked and remaining is only 33 probes. Since this probe count i.e 33 is lower than the number of total samples i.e 240 hence statistical calculation can't be performed uniformly for this chromosome "Y'.

Therefore no need to comment the lines in conifer.py, it is correct only according to the statistical requirement.

In my probes.txt file , I removed the targets related to chr 'Y' and repeating my analysis. Hope this will be the correct approach according to the error obtained.

Let me know if you don't agree with this explanation.

Thanks & Regards

Braj

[vishal.rossi]

Hi,

I removed the probes and tried to run conifer but it didn't go through. Maybe, split your bam file by chromosomes, remove the Y chromosome alignment and merge the files again and try it.
I didn't do this step, as I run some simulations and found that I am getting the same results even after making comments in the code.


[And did you comment out the 155th line.... # sys.exit(0)]

Regards,
Vishal

[Brajbio]

Hi Vishal,

Once you remove your target coordinates related to 'Y' chromosome from "probes.txt", then rerun the first step i.e RPKM calculation, and follow the next steps as instructed in the tutorial. You don't need to split your bam file, as tool will predict RPKM only for targets in the probes.txt.

No, I didn't commented any line in the script "conifer.py'

I got my result by doing so.


Regards, Braj

[vishnuamaram]

Hey Vishal & Braj,

Looks like you guys are experts in using conifer by now.

I am newbie to bioinfo as well to using NGS & conifer.

Some how, i am learning & working on the stuff.

I was able to succesfully run the commands to the sample data of conifer.
But, when I give commands to my data, errors come.

What is the first step in using conifer analysis.
Hope it's creating RPKM files.
I am not able to that properly.

Here is my commands & error.

python conifer.py rpkm --probes sampledata/probes.txt --input AD1cord-rmdup-realigned.bam --output RPKM/AD1.rpkm.txt

[INIT] Successfully read in 194080 probes from sampledata/probes.txt
[ERROR] Cannot open rpkm file for writing: ['../../odity/IndelTargetCreator/RealignedbamFiles/RPKM/AD1.rpkm.txt']

Help me with proper commands & solving the issue.

[wdemos]

Hi I also am running into problems getting Conifer up and running. I have successfully run through the quick start so I believe all is correctly installed. I also generated 10 rpkm files from bam files utilizing the conifer.py rpkm command. When I run the confier.py analyze command I get the following error:
[RUNNING: chr1] Now on: chr1
[RUNNING: chr1] Found 45973 probes; probeID range is [0-224692]
[RUNNING: chr1] Calculating median RPKM
[RUNNING: chr1] Masking 3125 probes with median RPKM < 1.000000
Traceback (most recent call last):
File "/usr/local/conifer_v0.2.2/conifer.py", line 682, in <module>
args.func(args)
File "/usr/local/conifer_v0.2.2/conifer.py", line 157, in CF_analyze
probeIDs = np.array(map(operator.itemgetter("probeID"),chr_probes))[probe_mask]
ValueError: too many boolean indices
Closing remaining open files:analysis.hdf5...done
My call is: python conifer.py analyze --probes AnnotatedSeqCap_EZ_Exome_v2nochr.bed --rpkm_dir RPKM/ --output analysis.hdf5 --svd 2 --write_svals singular_values.txt

Any suggestions how I can get this off the ground? Thanks

[vishal.rossi]

Hi,

This error might be due to old numpy version. Can you check which numpy version is installed?
Make sure it is numpy1.8 or above.

Vishal

[wdemos]

Thank you Vishal. I am running numpy 1.8. I updated to 1.9 and this still resulted in the same error.
:
[RUNNING: chr1] Now on: chr1
[RUNNING: chr1] Found 45973 probes; probeID range is [0-224692]
[RUNNING: chr1] Calculating median RPKM
[RUNNING: chr1] Masking 3125 probes with median RPKM < 1.000000
Traceback (most recent call last):
File "/usr/local/conifer_v0.2.2/conifer.py", line 682, in <module>
args.func(args)
File "/usr/local/conifer_v0.2.2/conifer.py", line 157, in CF_analyze
probeIDs = np.array(map(operator.itemgetter("probeID"),chr_probes))[probe_mask]
IndexError: index 45973 is out of bounds for axis 1 with size 45973
Closing remaining open files:analysis.hdf5...done

[pepsimax]

Originally Posted by wdemos

Thank you Vishal. I am running numpy 1.8. I updated to 1.9 and this still resulted in the same error.
:
[RUNNING: chr1] Now on: chr1
[RUNNING: chr1] Found 45973 probes; probeID range is [0-224692]
[RUNNING: chr1] Calculating median RPKM
[RUNNING: chr1] Masking 3125 probes with median RPKM < 1.000000
Traceback (most recent call last):
File "/usr/local/conifer_v0.2.2/conifer.py", line 682, in <module>
args.func(args)
File "/usr/local/conifer_v0.2.2/conifer.py", line 157, in CF_analyze
probeIDs = np.array(map(operator.itemgetter("probeID"),chr_probes))[probe_mask]
IndexError: index 45973 is out of bounds for axis 1 with size 45973
Closing remaining open files:analysis.hdf5...done

Was this issue ever solved? I've gotten this error too.

[wdemos]

No, I have not resolved this issue

[aditisk]

Hello,

I am trying to run Conifer on my exome seq data and I am not able to get the step that calculates RPKM values to work. An rpkm.txt file gets created but it is empty. Does anyone have an idea about what is going wrong ?

[Aishwarya Gogate]

Hi,
I am having exactly the same problem. The .txt RPKM files that are generated are empty. I have indexed all my BAM files and I'm giving the complete path for all my files in the command.
What might be going wrong?

[aditisk]

I figured out why my rpkm files were blank and it turns out that my BAM index files are not being recognized even though they are in the same folder as my BAM files.

I don't know why it is unable to find the BAM index files

[ebrown1985]

Originally Posted by aditisk

I figured out why my rpkm files were blank and it turns out that my BAM index files are not being recognized even though they are in the same folder as my BAM files.

I don't know why it is unable to find the BAM index files

Did you generate your bam file using GAKT? The index file should have the extention *.bam.bai no just *.bai as outputted by GATK.

You can just copy the *.bai file to *.bam.bai and it should work.

[shawpa]

I realize this is an outdated thread but I am trying to run CONIFER for the first time and having some issues. I cannot run the very first command to generate the RPKM files. Like someone said above, my files are just blank. When I run it in python I get the error:

“Segmentation fault (core dumped)”

I don't know what the source of the error or what it means. I was wondering if it was a memory issue because the bam files that I am trying to read in are ~20GB. I don't know what size files others in this thread are dealing with. I also read about the issue with the index files not matching but I fixed that cause I ran into the same problem with another CNV detection program. Any help would be greatly appreciated.