BWA building index of full human (ensembl) fails

[inijman]

I need to build a color space index for the complete human genome build GRCh37 including the haplotypic chromosomes.

On the complete set (35 chromosomes) the BWA indexing fails, and seem to hang forever (>20 hrs) on converting the nt PAC to color PAC. I used the 'bwa index -c -a bwtsw 'fasta_file' option.

After playing a bit, removing 1 or 2 chromosomes from this set resolves the issue; actually when the nt.pac file is 1009 MB it works, but adding a chromosome and raising the nt.pac filesize to 1049 MB fails the indexing completely.

So there are no errors or chrashes with the complete build, but it just remains in building the cs version of the PAC file. We've tried this on several systems with 64 GB ram and plenty of disk space. Both version 0.5.1 and 0.5.5 seem to have this issue.

hopefully, this can be fixed.

[lh3]

Use the reference genome here:

ftp://ftp.ncbi.nih.gov/1000genomes/f...cal/reference/

The toplevel fasta contains multiple haplotypes for the same chromosome. You should not use that.

[inijman]

Thanks for your reply Heng. However, I would like to have the phenotypic data included since we have genes covered in this region and SNP combinations clearly affect mapping.

Your suggestion is a work-around but not a solution to the issue. I imagine working with bigger plant genomes will result in problems as well and a real fix would be really welcome.

[lh3]

In any case, you should not include the multiple copies of the entire chromosome 6 and 17. You should identify highly divergent regions first and then do the alignment; otherwise you will get no reads mapped to chr6 and chr17. If you want to minimize misalignment in HLA, you should use the HLA database from EBI.

Bwa will not support genomes longer than 4GB. See the bwa homepage.

EDIT: Alternatively, you may consider the mapper from 1001genomes.org. It aligns reads to multiple genomes simultaneously. I have not tried, though.

[inijman]

Ok, we will split off the variable part and treat it seperately.