SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
BFAST match abort trap bbachert20 Bioinformatics 1 07-14-2011 07:29 PM
bfast match problem on fedora 14 sammy07 Bioinformatics 5 02-08-2011 09:24 AM
Error during Bfast match step James W. MacDonald Bioinformatics 2 01-05-2011 07:15 AM
BFAST match/localalign help Esther Bioinformatics 3 08-04-2010 06:19 AM
bfast match segmentation fault jmartin Bioinformatics 6 04-14-2010 01:21 AM

Reply
 
Thread Tools
Old 04-09-2010, 04:47 AM   #1
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default BFAST match problem

Hi guys,

I have a problem with BFAST. I am trying to align about 300 Million Solid reads (about 30GB of data) and using multiple indexes. When looking for CAL I unfortunately get a Segmentation Fault that kills the application. At the moment I am testing the software on a 8 cores machine equipped with 32GB of ram running a Linux Debian.
According to the messages sent to the stderr it seems that the software crashes when it is copying (where?) the reads that are not aligned with the primary index in such a way to search them with the secondary indexes.

In the following, an extract of the output messages I get before the software crashes:

Code:
Reads processed: 314999637
Cleaning up index.
Searching index file 1/1 (index #1, bin #1) complete...
Found 14927144 matches.
Found matches for 14927144 reads.
Copying unmatched reads for secondary index search.
Thing is I noticed that while executing BFAST is not using too much RAM (about 1.3GB) but it seems to disk cache quite a lot (eating up all the RAM in this way).

Does anyone have any suggestions on what might be going on?
Any help would be appreciated on how to solve this.
Thanks
blu78 is offline   Reply With Quote
Old 04-09-2010, 09:42 AM   #2
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by blu78 View Post
Hi guys,

I have a problem with BFAST. I am trying to align about 300 Million Solid reads (about 30GB of data) and using multiple indexes. When looking for CAL I unfortunately get a Segmentation Fault that kills the application. At the moment I am testing the software on a 8 cores machine equipped with 32GB of ram running a Linux Debian.
According to the messages sent to the stderr it seems that the software crashes when it is copying (where?) the reads that are not aligned with the primary index in such a way to search them with the secondary indexes.

In the following, an extract of the output messages I get before the software crashes:

Code:
Reads processed: 314999637
Cleaning up index.
Searching index file 1/1 (index #1, bin #1) complete...
Found 14927144 matches.
Found matches for 14927144 reads.
Copying unmatched reads for secondary index search.
Thing is I noticed that while executing BFAST is not using too much RAM (about 1.3GB) but it seems to disk cache quite a lot (eating up all the RAM in this way).

Does anyone have any suggestions on what might be going on?
Any help would be appreciated on how to solve this.
Thanks
Could you give the full command you are using? Also, try avoiding the secondary index search and simply use all your indexes in your primary search. This will improve sensitivity and accuracy, and is the recommended mode indicated in the manual.
nilshomer is offline   Reply With Quote
Old 04-09-2010, 10:33 AM   #3
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

Hi Nils,

Thanks for your reply.
Here is the command I use (on a much smaller set of reads now -- 1 000 000):

Code:
bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 -I 2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb
I get the same results even if I skip secondary indexes and, as you suggest, I use the following command:

Code:
bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 1,2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb
and even if I do not specify the primary indexes with -i (in that case I would use all my 10 indexes instead of just three).

What is kind of wierd is that in this latter case the segmentation fault occurs after searching index 4, while in the first and second case it occurs after searching index 1.

Any clue on what is going on?

Thanks again for your help...
Cheers
blu78 is offline   Reply With Quote
Old 04-09-2010, 10:49 AM   #4
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

For completeness, here is the content of the stderr file obtained by $ cat err_10Mb

First two cases:

Code:
************************************************************
Checking input parameters supplied by the user ...
Validating fastaFileName contigs.fasta.
Validating readsFileName F3_10Mchunk.qfasta.
Validating tmpDir path ./.
**** Input arguments look good!
************************************************************
************************************************************
Printing Program Parameters:
programMode:                            [ExecuteProgram]
fastaFileName:                          contigs.fasta
mainIndexes                             1,2,3
secondaryIndexes                        [Not Using]
readsFileName:                          F3_10Mchunk.qfasta
offsets:                                [Using All]
loadAllIndexes:                         [Not Using]
compression:                            [Not Using]
space:                                  [Color Space]
startReadNum:                           1
endReadNum:                             2147483647
keySize:                                [Not Using]
maxKeyMatches:                          8
maxNumMatches:                          512
whichStrand:                            [Both Strands]
numThreads:                             8
queueLength:                            250000
tmpDir:                                 ./
timing:                                 [Not Using]
************************************************************
Searching for main indexes...
Found 3 index (3 total files).
Not using secondary indexes.
************************************************************
Reading in reference genome from contigs.fasta.cs.brg.
In total read 11 contigs for a total of 776592 bases
************************************************************
Reading F3_10Mchunk.qfasta into a temp file.
Will process 1000000 reads.
************************************************************
Searching index file 1/3 (index #1, bin #1)...
Reading index from contigs.fasta.cs.1.1.bif.
Read index from contigs.fasta.cs.1.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 1/3 (index #1, bin #1) complete...
Found 19503 matches.
************************************************************
Searching index file 2/3 (index #2, bin #1)...
Reading index from contigs.fasta.cs.2.1.bif.

Last case (with 10 indexes):

Code:
************************************************************
Checking input parameters supplied by the user ...
Validating fastaFileName contigs.fasta.
Validating readsFileName F3_10Mchunk.qfasta.
Validating tmpDir path ./.
**** Input arguments look good!
************************************************************
************************************************************
Printing Program Parameters:
programMode:                            [ExecuteProgram]
fastaFileName:                          contigs.fasta
mainIndexes                             [Auto-recognizing]
secondaryIndexes                        [Not Using]
readsFileName:                          F3_10Mchunk.qfasta
offsets:                                [Using All]
loadAllIndexes:                         [Not Using]
compression:                            [Not Using]
space:                                  [Color Space]
startReadNum:                           1
endReadNum:                             2147483647
keySize:                                [Not Using]
maxKeyMatches:                          8
maxNumMatches:                          512
whichStrand:                            [Both Strands]
numThreads:                             8
queueLength:                            250000
tmpDir:                                 ./
timing:                                 [Not Using]
************************************************************
Searching for main indexes...
Found 10 index (10 total files).
Not using secondary indexes.
************************************************************
Reading in reference genome from contigs.fasta.cs.brg.
In total read 11 contigs for a total of 776592 bases
************************************************************
Reading F3_10Mchunk.qfasta into a temp file.
Will process 1000000 reads.
************************************************************
Searching index file 1/10 (index #1, bin #1)...
Reading index from contigs.fasta.cs.1.1.bif.
Read index from contigs.fasta.cs.1.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 1/10 (index #1, bin #1) complete...
Found 19503 matches.
************************************************************
Searching index file 2/10 (index #2, bin #1)...
Reading index from contigs.fasta.cs.2.1.bif.
Read index from contigs.fasta.cs.2.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 2/10 (index #2, bin #1) complete...
Found 18896 matches.
************************************************************
Searching index file 3/10 (index #3, bin #1)...
Reading index from contigs.fasta.cs.3.1.bif.
Read index from contigs.fasta.cs.3.1.bif.
Reads processed: 1000000
Cleaning up index.
Searching index file 3/10 (index #3, bin #1) complete...
Found 14092 matches.
************************************************************
Searching index file 4/10 (index #4, bin #1)...
Reading index from contigs.fasta.cs.4.1.bif.

Thanks
blu78 is offline   Reply With Quote
Old 04-09-2010, 03:02 PM   #5
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by blu78 View Post
Hi Nils,

Thanks for your reply.
Here is the command I use (on a much smaller set of reads now -- 1 000 000):

Code:
bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 -I 2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb
I get the same results even if I skip secondary indexes and, as you suggest, I use the following command:

Code:
bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -i 1 1,2,3 -A 1 -K 8 -M 512 -w 0 -n 8 > matchF3_10Mb 2> err_10Mb
and even if I do not specify the primary indexes with -i (in that case I would use all my 10 indexes instead of just three).

What is kind of wierd is that in this latter case the segmentation fault occurs after searching index 4, while in the first and second case it occurs after searching index 1.

Any clue on what is going on?

Thanks again for your help...
Cheers
Try this:
Code:
bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -A 1 -n 8 > matchF3_10Mb 2> err_10Mb
Make sure that you have built all 10 indexes as recommended in the manual and use them as all as primary indexes. I also removed some options above as I would still recommend using the default parameters

It also looks like your stderr and stdout show that the process is not complete (meaning they don't show crashing on the "copying for next index search").
nilshomer is offline   Reply With Quote
Old 04-10-2010, 02:09 AM   #6
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

I have tried
Code:
bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -A 1 -n 8 > matchF3_10Mb 2> err_10Mb
but I still get the segmentation fault.

Indexes are created with the following commands:
Code:
bfast index -f contigs.fasta -A 1 -m 1111111111111111111111 -i 1 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 111110100111110011111111111 -i 2 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 10111111011001100011111000111111 -i 3 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 1111111100101111000001100011111011 -i 4 -w 12 -n 8

...

bfast index -f contigs.fasta -A 1 -m 1110110001011010011100101111101111 -i 10 -w 12 -n 8
Thanks
blu78 is offline   Reply With Quote
Old 04-10-2010, 09:27 AM   #7
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by blu78 View Post
I have tried
Code:
bfast match -f contigs.fasta -r F3_10Mchunk.qfasta -A 1 -n 8 > matchF3_10Mb 2> err_10Mb
but I still get the segmentation fault.

Indexes are created with the following commands:
Code:
bfast index -f contigs.fasta -A 1 -m 1111111111111111111111 -i 1 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 111110100111110011111111111 -i 2 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 10111111011001100011111000111111 -i 3 -w 12 -n 8
bfast index -f contigs.fasta -A 1 -m 1111111100101111000001100011111011 -i 4 -w 12 -n 8

...

bfast index -f contigs.fasta -A 1 -m 1110110001011010011100101111101111 -i 10 -w 12 -n 8
Thanks
I apologize for your experience so far. The best way to move forward is to email me the reference and reads, and then I can quickly debug myself to see if their is a bug in the program or a configuration issue. What does the "F3_10Mchunk.qfasta" file look like?
nilshomer is offline   Reply With Quote
Old 04-10-2010, 03:54 PM   #8
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

Hi, I think I have a bit more information on this problem.
I tried to rerun the smaller example with 4 cores instead of 8 (i.e. -n 4) and apparently the Segmentation Fault problem is solved.
The box I was running bfast on has eight cores, theoretically should I set -n 8? Does this parameter need to be a power of 2?

At the moment I am trying to run the bigger example with 4 cores. I will fill you in as soon as I will know the result of this computation.

Thanks
blu78 is offline   Reply With Quote
Old 04-10-2010, 08:02 PM   #9
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by blu78 View Post
Hi, I think I have a bit more information on this problem.
I tried to rerun the smaller example with 4 cores instead of 8 (i.e. -n 4) and apparently the Segmentation Fault problem is solved.
The box I was running bfast on has eight cores, theoretically should I set -n 8? Does this parameter need to be a power of 2?

At the moment I am trying to run the bigger example with 4 cores. I will fill you in as soon as I will know the result of this computation.

Thanks
That is very odd. The "index" step needs to be a power of two, but the rest of the program does not. Most users use it successfully with "-n 8" on 8-core machines.
nilshomer is offline   Reply With Quote
Old 04-11-2010, 02:11 AM   #10
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

I also tried the bigger example and that worked too with 4 cores. Oddly with 1 index and 8 cores it works too but with more indexes I cannot use more than 4 cores.

Last edited by blu78; 04-11-2010 at 02:31 AM.
blu78 is offline   Reply With Quote
Old 04-11-2010, 10:25 AM   #11
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by blu78 View Post
I also tried the bigger example and that worked too with 4 cores. Oddly with 1 index and 8 cores it works too but with more indexes I cannot use more than 4 cores.
I can't think of a reasonable explanation for this beyond hardware or OS configuration. What version are you currently using? Could you post the result of your "./configure" set up script?
nilshomer is offline   Reply With Quote
Old 04-11-2010, 10:46 AM   #12
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

Hi,

I am using Linux Debian 64 bit.
Here is the output of configure. Thanks for all your help.
Code:
$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking build system type... x86_64-unknown-linux-gnu
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for a BSD-compatible install... /usr/bin/install -c
./configure: line 3462: git: command not found
checking for BZ2_bzRead in -lbz2... yes
checking for an ANSI C-conforming const... yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible realloc... yes
checking for pow in -lm... yes
checking for gzread in -lz... yes
checking for floor... yes
checking for pow... yes
checking for sqrt... yes
checking for strchr... yes
checking for strdup... yes
checking for strpbrk... yes
checking for strstr... yes
checking for strtok_r... yes
checking for int8_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for uint8_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for short int... yes
checking size of short int... 2
checking for int... yes
checking size of int... 4
checking for long int... yes
checking size of long int... 8
checking for ANSI C header files... (cached) yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking float.h usability... yes
checking float.h presence... yes
checking for float.h... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking bzlib.h usability... yes
checking bzlib.h presence... yes
checking for bzlib.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking for inline... inline
configure: creating ./config.status
config.status: creating Makefile
config.status: creating bfast/Makefile
config.status: creating butil/Makefile
config.status: creating scripts/Makefile
config.status: creating tests/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands

Last edited by blu78; 04-11-2010 at 10:49 AM.
blu78 is offline   Reply With Quote
Old 04-11-2010, 12:35 PM   #13
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by blu78 View Post
Hi,

I am using Linux Debian 64 bit.
Here is the output of configure. Thanks for all your help.
Code:
$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking build system type... x86_64-unknown-linux-gnu
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking for a BSD-compatible install... /usr/bin/install -c
./configure: line 3462: git: command not found
checking for BZ2_bzRead in -lbz2... yes
checking for an ANSI C-conforming const... yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible realloc... yes
checking for pow in -lm... yes
checking for gzread in -lz... yes
checking for floor... yes
checking for pow... yes
checking for sqrt... yes
checking for strchr... yes
checking for strdup... yes
checking for strpbrk... yes
checking for strstr... yes
checking for strtok_r... yes
checking for int8_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for uint8_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for short int... yes
checking size of short int... 2
checking for int... yes
checking size of int... 4
checking for long int... yes
checking size of long int... 8
checking for ANSI C header files... (cached) yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking for stdint.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking float.h usability... yes
checking float.h presence... yes
checking for float.h... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking bzlib.h usability... yes
checking bzlib.h presence... yes
checking for bzlib.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking for inline... inline
configure: creating ./config.status
config.status: creating Makefile
config.status: creating bfast/Makefile
config.status: creating butil/Makefile
config.status: creating scripts/Makefile
config.status: creating tests/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
I just can't figure out why it would not work with 8 threads but only 4 threads. What version of BFAST are you using?

Nils
nilshomer is offline   Reply With Quote
Old 04-11-2010, 12:39 PM   #14
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

Hi,

I am using BFAST 0.6.4a

Thanks for your help
blu78 is offline   Reply With Quote
Old 04-11-2010, 12:49 PM   #15
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by blu78 View Post
Hi,

I am using BFAST 0.6.4a

Thanks for your help
I don't have any new leads. Most users I know run with 8-threads on all parts of BFAST. Maybe there is a memory bus issue that is causing it to crash since there are too many threads trying to access memory. This seems unlikely and is difficult to prove. Stick with your four threads and we'll see if any other users experience the same thing,

Nils
nilshomer is offline   Reply With Quote
Old 04-11-2010, 12:58 PM   #16
blu78
Member
 
Location: italy

Join Date: Apr 2010
Posts: 20
Default

ok thanks again.
I will let you know any news and I will upgrade to the 0.6.4b.

All the best

Last edited by blu78; 04-11-2010 at 01:01 PM.
blu78 is offline   Reply With Quote
Old 07-20-2010, 08:52 AM   #17
fennan
Member
 
Location: Germany

Join Date: Apr 2010
Posts: 19
Default

It seems I am having the same problem...
I am using bfast-0.6.4 and I set the number of threads to 8 I got the "Segmentation fault" error. However when I set it to 4, BFAST works just fine. Was this problem solved?

Thanks!
fennan is offline   Reply With Quote
Old 07-21-2010, 11:12 PM   #18
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Quote:
Originally Posted by fennan View Post
It seems I am having the same problem...
I am using bfast-0.6.4 and I set the number of threads to 8 I got the "Segmentation fault" error. However when I set it to 4, BFAST works just fine. Was this problem solved?

Thanks!
Could you post the results of the "configure" command (during install)?
nilshomer is offline   Reply With Quote
Old 08-02-2010, 08:00 AM   #19
cbarette
Junior Member
 
Location: Paris, France

Join Date: Aug 2010
Posts: 2
Default

Hi,

We have the same problem with debian Lenny 64bits: bfast (0.6.4d but also 0.6.2a and 0.6.3c) crashes when running with 8 CPU but not with 4 (our server has 32 cores so we would really want to use as many resources as possible.

- The configuration phase is the same as blue78 : http://pastebin.com/vW9kXwX9
- It generates the following config.h : http://pastebin.com/gk57XfP9
- and for the compilation: http://pastebin.com/jBLjyL2s (it complains a bit about gzread64 not being declared but nothing wrong during the linking)

We are running the folowwing commands:
Code:
WORK_DIR=/data/ngs/solid/PROJETS/Comp_bowtie_bfast/BFAST
REF_GENOME=/data/ngs/annotations/bfast_indexes/hg19.fa
BFAST_BIN_DIR=/bioinfo/local/build/bfast_0.6.4d/bin
NB_THREADS=8

nohup $BFAST_BIN_DIR/bfast match -f $REF_GENOME -n $NB_THREADS -A 1 -r $WORK_DIR/FastQ/reads100000.200.fastq > $WORK_DIR/Bmf/bmfbfast.matches.file.hg19.200.bmf
The result is
Code:
************************************************************
Checking input parameters supplied by the user ...
Validating fastaFileName /data/ngs/annotations/bfast_indexes/hg19.fa. 
Validating readsFileName /data/ngs/solid/PROJETS/Comp_bowtie_bfast/BFAST/FastQ/reads100000.200.fastq. 
Validating tmpDir path ./. 
**** Input arguments look good!
************************************************************
************************************************************
Printing Program Parameters:
programMode:                            [ExecuteProgram]
fastaFileName:                          /data/ngs/annotations/bfast_indexes/hg19.fa
mainIndexes                             [Auto-recognizing]
secondaryIndexes                        [Not Using]
readsFileName:                          /data/ngs/solid/PROJETS/Comp_bowtie_bfast/BFAST/FastQ/reads100000.200.fastq
offsets:                                [Using All]
loadAllIndexes:                         [Not Using]
compression:                            [Not Using]
space:                                  [Color Space]
startReadNum:                           1
endReadNum:                             2147483647
keySize:                                [Not Using]
maxKeyMatches:                          8
maxNumMatches:                          384
whichStrand:                            [Both Strands]
numThreads:                             8
queueLength:                            250000
tmpDir:                                 ./
timing:                                 [Not Using]
************************************************************
Searching for main indexes...
Found 10 index (10 total files).
Not using secondary indexes.
************************************************************
Reading in reference genome from /data/ngs/annotations/bfast_indexes/hg19.fa.cs.brg.
In total read 25 contigs for a total of 3095693983 bases
************************************************************
Reading /data/ngs/solid/PROJETS/Comp_bowtie_bfast/BFAST/FastQ/reads100000.200.fastq into a temp file.
Will process 100000 reads.
************************************************************
Searching index file 1/10 (index #1, bin #1)...
Reading index from /data/ngs/annotations/bfast_indexes/hg19.fa.cs.1.1.bif.
Read index from /data/ngs/annotations/bfast_indexes/hg19.fa.cs.1.1.bif.
Reads processed: 100000
Cleaning up index.
Searching index file 1/10 (index #1, bin #1) complete...
Found 60065 matches.
************************************************************
Searching index file 2/10 (index #2, bin #1)...
Reading index from /data/ngs/annotations/bfast_indexes/hg19.fa.cs.2.1.bif.
and the it crashes with a segfault.

We generated a core dump but the crash is in a function of libz so we installed the debug version of libz and here is the result: http://pastebin.com/Ae0re589

So if you have any leads, I would be happy to help and test.

Best regards

Last edited by cbarette; 08-02-2010 at 08:31 AM.
cbarette is offline   Reply With Quote
Old 08-02-2010, 09:06 AM   #20
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

Thank-you for providing a good amount of info to help my debug. Googling the compiler warning, it may be the case that your zlib is buggy in Ubuntu (see my links hinting at that). The function "gzopen64" is declared in zlib to provide 64-bit support, but is not being declared correctly. I am guessing it crashes when trying to read in a compressed index (as shown by the last BFAST status message). In "configure.ac", could you change the "extended_CFLAGS" line to the below and see what happens?
extended_CFLAGS="-m64 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE";
The run "sh autogen.sh && ./configure && make".

Links:
http://groups.google.com/a/chromium....517cdd0f29336c
http://lipyrary.blogspot.com/2009/08...it-ubuntu.html
http://bugs.gentoo.org/316377
http://code.google.com/p/chromium/is...etail?id=38073
nilshomer is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 08:45 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2021, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO