Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • How long should paired-end alignment run?

    I'm aligning Illumina paired-end reads using bwa for the first time. With short reads from the same sample (yeast) alignment took 2-3 hours, but now I've been waiting for results for 4 days. It seems to be progressing, but is this normal? Should it take this long? It keeps repeatedly printing a bunch of calculations and then hanging on "align unmapped mate...".

  • #2
    Is it the bwa sampe step which lasts long? I would suggest switching on the -P option (which loads the index into memory). That speeds up the execution quite a lot. It does require 4-5Gb of memory though to run (if I remember correctly on human).

    Comment


    • #3
      bwa c code is notorious for not checking the return call from functions. It's pretty happy to ignore a failure and keep trying. It's a nightmare to figure out what went wrong! Not a big deal though. The upside is it runs really fast.

      It is likely one of several things happened:

      1) you've provided invalid input - make sure stderr from a previous run did not sneak in to your sai file. Make sure a previous run is not "bwa > out.sai 2>&1 ", make sure it's "bwa > out.sai 2> out.err".

      2) you've run out of memory

      3) you're command line arguments are wrong.


      Do a check of your input. Try running on a bigger machine, at least 8GB free memory.

      Comment


      • #4
        I've found bwa can hang eternally when it incorrectly infers an insert size estimate. Unfortunately if you specify a maximum insert size with the -a parameter, BWA will ignore this parameter if its inferred isize function succeeds (in its own mind). I've once specified a maximum insert size of 5000 to have BWA turn around and tell me it's going with its own calculated value of 1483238... as a result, it is likely hanging during the "align unmapped mate" stage.

        To get around this, you can use the -A parameter, which ignores BWA's isize estimates. However, this as a result disables SW alignment for unmapped mates.

        Comment


        • #5
          I also have this problem of infer the insert size and the sampe hang on the align unmapped mate. How can I fix it ?
          Originally posted by dp05yk View Post
          I've found bwa can hang eternally when it incorrectly infers an insert size estimate. Unfortunately if you specify a maximum insert size with the -a parameter, BWA will ignore this parameter if its inferred isize function succeeds (in its own mind). I've once specified a maximum insert size of 5000 to have BWA turn around and tell me it's going with its own calculated value of 1483238... as a result, it is likely hanging during the "align unmapped mate" stage.

          To get around this, you can use the -A parameter, which ignores BWA's isize estimates. However, this as a result disables SW alignment for unmapped mates.

          Comment


          • #6
            Originally posted by elisadouzi View Post
            I also have this problem of infer the insert size and the sampe hang on the align unmapped mate. How can I fix it ?
            I do believe this was already mentioned... see quoted below:

            To get around this, you can use the -A parameter, which ignores BWA's isize estimates. However, this as a result disables SW alignment for unmapped mates.

            Comment


            • #7
              Thanks. I already chose this -a 500 , but it still did not work.
              Originally posted by dp05yk View Post
              I've found bwa can hang eternally when it incorrectly infers an insert size estimate. Unfortunately if you specify a maximum insert size with the -a parameter, BWA will ignore this parameter if its inferred isize function succeeds (in its own mind). I've once specified a maximum insert size of 5000 to have BWA turn around and tell me it's going with its own calculated value of 1483238... as a result, it is likely hanging during the "align unmapped mate" stage.

              To get around this, you can use the -A parameter, which ignores BWA's isize estimates. However, this as a result disables SW alignment for unmapped mates.
              Originally posted by dp05yk View Post
              I do believe this was already mentioned... see quoted below:

              Comment


              • #8
                Hi elisadouzi,

                -a is not the same as -A.

                As mentioned in my post, if you specify an isize with -a, BWA will ignore it if it is able to estimate (in its own mind) a different value. This is why -a 500 isn't working for you - BWA is calculating a value much higher than 500 is overriding your specification with it.

                Use the -A (not a, capitalized A) parameter, and it will ignore its isize estimate (and unmapped mate alignment altogether, unfortunately).

                Comment


                • #9
                  Hello,

                  I am also running into this problem and getting some curious results. Am running resequencing assemblies (mapping with bwa, assembly with Velvet columbus module) on two different 454 assemblies of the same genome, partly to test the quality of assembly. On one assembly, sampe it completes in 2-3 hours, the other, supplied with the exact same parameters etc. has been running 4 days and hanging on the align unmapped mate stage. It is giving me an insert size estimate of around 1Mb (&$&*!%!)...I doubt it's memory, i'm running on 16G and I just checked the input files which are ok, so I am beginning to think it may have something to do with the complexity of the problem/compatibility between the two datasets that is leading it to give crazy insert sizes and hang.

                  I just tried the -A option but bwa complains this is not a valid option...am I missing something or which version are you using/how do you supply this option? Does anyone else have any thoughts/experiences to explain this upo (unidentified phenomena output)?


                  best,

                  Kate

                  Comment


                  • #10
                    Hi,


                    I think -s might help... it disables Smith-Waterman mapping for unmapped mates, and this is likely what is taking so long. If -s doesn't help or doesn't exist as an option, try getting the latest version of BWA (0.5.9) from their sourceforge page.

                    Comment


                    • #11
                      Hi,

                      Thanks...the -a option seems to have solved the problem this time.

                      Comment


                      • #12
                        "-a" is only used when bwa doesn' have enough pairs of reads (<20) to infer insert size. The bug above happens when bwa has enough pairs (>=20) and makes an unrealistic estimate of the insert size (like 1million+ above or ~210K in my example). And then the alignment of unmapped mates takes forever because it has to try SW in a very long window. So "-a" is irrelevant to this bug.

                        What i did is 1. increase the read batch size from 0x40000 (~250K) to 0x60000 (or more);2. increase that crucial 20 to 80 (or more). both changes aim to make sure bwa has enough pairs to make a good inference. here's the code diff:

                        ...:~/script/bio-bwa$ svn diff
                        Index: trunk/bwa/bwape.c
                        ===================================================================
                        --- trunk/bwa/bwape.c (revision 51)
                        +++ trunk/bwa/bwape.c (working copy)
                        @@ -107,7 +107,9 @@
                        if (p[0]->len > max_len) max_len = p[0]->len;
                        if (p[1]->len > max_len) max_len = p[1]->len;
                        }
                        - if (tot < 20) {
                        + // 2011-9-6 yh increase the number below from 20 to 80 to make it less likely that super-big insert sizes would be inferred.
                        + int minNoOfValidPairs = 80;
                        + if (tot < minNoOfValidPairs) {
                        fprintf(stderr, "[infer_isize] fail to infer insert size: too few good pairs\n");
                        free(isizes);
                        return -1;
                        @@ -691,12 +693,14 @@
                        // core loop
                        bwa_print_sam_SQ(bns);
                        bwa_print_sam_PG();
                        - while ((seqs[0] = bwa_read_seq(ks[0], 0x40000, &n_seqs, opt.mode & BWA_MODE_COMPREAD, opt.trim_qual)) != 0) {
                        + // 2011-9-6 increase the number below from 0x40000 to 0x60000 to make it less likely that super-big insert sizes would be inferred.
                        + int n_needed = 0x60000;
                        + while ((seqs[0] = bwa_read_seq(ks[0], n_needed, &n_seqs, opt.mode & BWA_MODE_COMPREAD, opt.trim_qual)) != 0) {
                        int cnt_chg;
                        isize_info_t ii;
                        ubyte_t *pacseq;

                        - seqs[1] = bwa_read_seq(ks[1], 0x40000, &n_seqs, opt.mode & BWA_MODE_COMPREAD, opt.trim_qual);
                        + seqs[1] = bwa_read_seq(ks[1], n_needed, &n_seqs, opt.mode & BWA_MODE_COMPREAD, opt.trim_qual);
                        tot_seqs += n_seqs;
                        t = clock();

                        Comment

                        Latest Articles

                        Collapse

                        • seqadmin
                          Strategies for Sequencing Challenging Samples
                          by seqadmin


                          Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
                          03-22-2024, 06:39 AM
                        • seqadmin
                          Techniques and Challenges in Conservation Genomics
                          by seqadmin



                          The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

                          Avian Conservation
                          Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
                          03-08-2024, 10:41 AM

                        ad_right_rmr

                        Collapse

                        News

                        Collapse

                        Topics Statistics Last Post
                        Started by seqadmin, Yesterday, 06:37 PM
                        0 responses
                        10 views
                        0 likes
                        Last Post seqadmin  
                        Started by seqadmin, Yesterday, 06:07 PM
                        0 responses
                        9 views
                        0 likes
                        Last Post seqadmin  
                        Started by seqadmin, 03-22-2024, 10:03 AM
                        0 responses
                        51 views
                        0 likes
                        Last Post seqadmin  
                        Started by seqadmin, 03-21-2024, 07:32 AM
                        0 responses
                        67 views
                        0 likes
                        Last Post seqadmin  
                        Working...
                        X