Bfast Fatal Error during indexing

[fpruzius]

Hi,

I am trying to index a fastafile with the human genome (build GRCh37, only chromosomes + MT) using bfast. In color space.
I am using version 0.6.4f. As a guildeline I use secion 8.1.2 from the manual.

Converting the reads to fastq and making both the color space and letter space brg files work without any trouble.

However when I want to make the indexes the program grinds to a halt.

The command I use:

bfast index -f human.fasta -m 1111111111111111111111 -w 14 -i 1 -A 1 -T /tmp/

So human.fasta is my reference genome. and the files human.fasta.cs.brg and human.fasta.nt.brg have been generated.

The output:

************************************************************
Checking input parameters supplied by the user ...
Validating fastaFileName human.fasta.
Validating tmpDir path /tmp/.
Input arguments look good!
************************************************************
************************************************************
Printing Program Parameters:
programMode:                            [ExecuteProgram]
fastaFileName:                          human.fasta
space:                                  [Color Space]
mask:                                   1111111111111111111111
depth:                                  0
hashWidth:                              14
indexNumber:                            1
repeatMasker:                           [Not Using]
startContig:                            0
startPos:                               0
endContig:                              2147483647
endPos:                                 2147483647
exonsFileName:                          [Not Using]
numThreads:                             4
tmpDir:                                 /tmp/
timing:                                 [Not Using]
************************************************************
************************************************************
Reading in reference genome from human.fasta.cs.brg.
In total read 25 contigs for a total of 3095693983 bases
************************************************************
Creating the index...
************************************************************
Warning: startContig was less than zero.
Defaulting to contig=1 and position=1.
************************************************************
************************************************************
Warning: endContig was greater than the number of contigs in the reference genome.
Defaulting to reference genome's end contig=25 and position=16571.
************************************************************
Currently on [contig,pos]:
[------25,------16571]
Sorting by thread...                                                                                                                                        
100.000 percent complete************************************************************
In function "RGIndexMergeHelperFromDiskContig_8": Fatal Error[ReadFileError]. Message: Could not read in tmp lower.
The file stream error was:: Bad file descriptor
 ***** Exiting due to errors *****
************************************************************

So it works, but as soon when the index should be written to disk it fails. THis happens after around 24 hours.

I have tried to check if something is wrong with the harddisk, an no fsck dont gives any problems. And I can create a large file with other programs.

The output file human.fasta.cs.1.1.bif is generated, however has zero length.

[nilshomer]

In "configure.ac" in the source directory, try changing the following line

extended_CFLAGS="";# "-m64 -D_FILE_OFFSET_BITS=64";

to

extended_CFLAGS="-m64 -D_FILE_OFFSET_BITS=64";

You then probably need to re-run "bfast fasta2brg".

[fpruzius]

Thanks for the reply.
I changed the configuration file and recompiled the source.

Unoftunately it did not help. still getting the following error:

Sorting...                                                                                                                                              100.000 percent complete************************************************************
In function "RGIndexMergeHelperFromDiskContig_8": Fatal Error[ReadFileError]. Message: Could not read in tmp lower.
The file stream error was:: Bad file descriptor
 ***** Exiting due to errors *****
************************************************************

[nilshomer]

Thanks for the reply.
I changed the configuration file and recompiled the source.

Unoftunately it did not help. still getting the following error:

Sorting...                                                                                                                                              100.000 percent complete************************************************************
In function "RGIndexMergeHelperFromDiskContig_8": Fatal Error[ReadFileError]. Message: Could not read in tmp lower.
The file stream error was:: Bad file descriptor
 ***** Exiting due to errors *****
************************************************************

I can't replicate your error, so unfortunately I cannot debug. Could you try it on a different computer and see if you can replicate?

[brentp]

Also see this thread:

http://sourceforge.net/mailarchive/forum.php?thread_name=05DD23DE980A6347AC2853AAC79105AF79F8819B6F%40Orpheus.NJRC.ORG&forum_name=bfast-help

[cdry7ue]

Nils,
I am getting the exact same error


In function "RGIndexMergeHelperFromDiskContig_8": Fatal Error[ReadFileError]. Message: Could not read in tmp lower.
The file stream error was:: Bad file descriptor
***** Exiting due to errors *****
************************************************************

[1]+ Exit 1 bfast index -m 111111111111111111111111 -w 14 -n 32 -i 1 -f hg19.fa

Were you guys ever able to find out what the deal was with this one? I also have a 64 bit system with 2Tb disk space.
-Ashish

[nilshomer]

I would LOVE to get a test case so that I can reproduce. Until then, I have to keep guessing.

Could you try to apply this change?

diff --git a/bfast/RGIndex.c b/bfast/RGIndex.c
index 3e8f57a..2953596 100644
--- a/bfast/RGIndex.c
+++ b/bfast/RGIndex.c
@@ -1356,8 +1356,15 @@ void RGIndexMergeHelperFromDiskContig_8(RGIndex *index,
        }
 
        /* Move to beginning of the files */
+        /*
        fseek(tmpLowerFP, 0 , SEEK_SET);
        fseek(tmpUpperFP, 0 , SEEK_SET);
+        */
+        fclose(tmpLowerFP);
+        tmpLowerFP = fopen(tmpLowerFileName, "wb+");
+        assert(NULL != tmpLowerFP);
+        tmpUpperFP = fopen(tmpUpperFileName, "wb+");
+        assert(NULL != tmpUpperFP);

[Magnus]

+1, same error for me as well.

Tested both extended_CFLAGS="-m64 -D_FILE_OFFSET_BITS=64" and the patch without any success .

What could I do to help you find a solution Nils?

[nilshomer]

Do you have a test case? Give me a FASTA and detail your environment (OS, machine, bfast version).

Nils