BWA indexing mechanism

[Greeeeb]

Hi all,

I am new to bioinformatics. I am parallelizing a code that use BWA for alignment.

Can someone please explain to me in general how the indexing is happening?

I ask because the previous coder index and run alignment by calling BWA. then, the code "reads-in the reference geneome"; the code extract subsequence from the read-in reference by using the starting position that appears in the alignment file. For instance;

assume we have the following read: ACCCA

the reference has two chromosomes:
>chr1
GGGGGTTTTT
>chr2
AAACCCCCCCC

and assume the read is aligned to ACCCC in the second chromosome, where the starting position is: 13--I think BWA will produce 13.

the code reads in the reference and stores every chromosomes in a separate data structure. The problem is when when the code gets the subsequence "ACCCA" from the reference, it uses the starting position (13) to extract the subsequence. The way the reference is stored leads to out of range (13 is more than the 11 bases stored in chr2) and the program crashes. and even if the 13 is in range, we will be reading the wrong subsequence.

I need to know the BWA indexing mechanism so I get the correct subsequence from the read-in reference. I tried to add the length prefix to get the correct subsequence, but it did not work, where there are some dependencies in the code that may led to the problem.

Thanks.

[GenoMax]

If you are a proficient programmer source code for bwa is available on SF. If you need technical help then you could post to bwa mailing list.

[dpryan]

I thought BWA (as ends up being the case with a lot of aligners) ends up concatenating the chromosomes together when indexing (with a bunch of N in between). Anyway, BWA at least used to occasionally produce alignments that extended beyond the chromosome bounds, so if you have an example that makes it crash then post that on github as an issue.

[Greeeeb]

Thanks!

I will go ahead and check the code.

The problem happens when our code is running, not the BWA. Also, I tried to read the indexing step output files. The large ones are not simple text formate. If that was the case, I would try to input small, tailored reference genomes to see how what is the output from the indexing.

As I noted, our program can handle 1 chromosome without problems. So, I merged the chromosomes of a human genome into one; but BWA exits without indexing the whole genome. The largest single chromosome reference BWA indexed successfully was about 160M bytes.

[GenoMax]

Not a direct answer but if you are open to substituting the aligner then BBMap from BBMap suite is an option. It is written in pure Java and author of BBMap participates (Brian Bushnell) here regularly.

[Greeeeb]

Actually it may not be possible to use something else. Since I am not well familiar with BWA, I am afraid I would not be able to preserve the compatibility with the code when I replace BWA commands with BBmap. Figuring out how to read the reference the way BWA does is the safest way.

Thanks.

[Greeeeb]

I found a way to reconstruct the reference subsequence for an aligned read from the alignment information in SAM file. This is the "MD" field in the output. Here is example:

"MD: A string which summarizes the mismatch positions between the aligned read and the reference genome.

Example:
MD:Z:8G61 indicates a single base pair mismatch. Specifically, the aligned read matches the first 8 bases of the reference, after which it fails to match a G in the reference sequence, followed by 61 exact matches to the reference.
"
Source: http://biobits.org/samtools_primer.html

Thanks for all.