Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • WHEN to merge data from single library run on multiple lanes?

    Hi

    Even after reading several posts, I'm still confused about merging data from multiple lanes...

    I've got a single library run on four lanes (each library run with several other libraries in a given lane but the paired-end fastq files are already separated by library). My goal is to generate a single consensus sequence of the mitochondria for each library.

    My overall pipeline for trimmed reads consists of 1) alignment to the reference with BWA mem 2) convert sam to bam 3) sorting with Piccard tools 4) removing duplicates with Piccard tools 5) removing ambiguous reads with samtools and 6) then splitting the bam file into separate nuclear and mitochondrial bams (using samtools).

    I'm specifically wondering if there are any problems with merging the resulting mitochondrial bams from running this pipeline separately for each lane? Or should I be merging the data from the four lanes at an earlier step?

    Also, I'm really confused about the concept of @RGs and @SQ...Is a RG simply the bam version of SQ? I was thinking of using samtools merge with the -r parameter specified...does this replace the original RGs somehow? How does the RG effect things downstream (for me I'm eventually generating an mpileup file and then a consensus sequence...).

    Thanks in advance for the help!

  • #2
    You can pretty much always merge data from a single library run on multiple lanes, since they should have the same biases. If you really wanted, you could align them separately and then give them different PL identifier, though practically speaking I kind of doubt that's normally all that useful (any weird bias that you'll want to filter on should be library, rather than lane, specific, though I'm sure someone has a counter example).

    Unless you have very few reads per sample per lane then it makes little practical difference when you merge things. The only benefit to merging before alignment is that you might get a better estimate of the template length distribution, though unless you really really heavily multiplex I doubt you get much of any difference.

    BTW, BAM files have @SQ lines and SAM files also have @RG lines. The @SQ lines describe reference sequences (names and lengths). These will be the same in all of your samples. The @RG lines describe sample metainformation that you can then keep along with your alignments for post-processing. The nice thing is that you can merge files with different read groups and that information is then preserved (the alignments themselves RG auxiliary tags).

    Yes, the -r flag will replace read groups with something that's likely less useful for you.

    Comment


    • #3
      Originally posted by jullee View Post
      My overall pipeline for trimmed reads consists of 1) alignment to the reference with BWA mem 2) convert sam to bam 3) sorting with Piccard tools 4) removing duplicates with Piccard tools 5) removing ambiguous reads with samtools and 6) then splitting the bam file into separate nuclear and mitochondrial bams (using samtools).

      I'm specifically wondering if there are any problems with merging the resulting mitochondrial bams from running this pipeline separately for each lane? Or should I be merging the data from the four lanes at an earlier step?
      You would definitely want to merge the aligned reads (BAM) before doing duplicate removal. Duplicate removal tools need to know about all reads in your data set to work properly. Also, if you are only interested in the mitochondrial reads it pays to reduce your data set to mitochondria only as soon as is practical, to avoid unnecessary computation. Here is what I would do:

      - Align each lane to reference.
      - Convert output to BAM, sort and index these BAM files.
      - Separate nuclear and mito alignments, keeping only uniquely mapped reads in each case (as you described "removing ambiguous reads").
      - Merge your mito BAM files.
      - Deduplicate your merged, mito BAM file.

      Comment


      • #4
        Thanks for the replies.

        kmcarr does it hurt to do the removal of duplicates twice? I'm thinking the first round takes care of PCR artifacts (e.g. library effects), the second sequencing artifacts (e.g. lane effects)?

        Also do I need to index again after the bam files are merged?

        Thanks!

        Comment


        • #5
          Originally posted by jullee View Post
          Thanks for the replies.

          kmcarr does it hurt to do the removal of duplicates twice? I'm thinking the first round takes care of PCR artifacts (e.g. library effects), the second sequencing artifacts (e.g. lane effects)?
          No, it really doesn't make sense to run de-duplication twice. If by "lane effects" you mean optical duplicates it is true that to detect these you only need the reads from each lane, but these are pretty rare. De-duplication is really all about removing PCR duplicates and PCR duplicates can only be identified if you have all of the reads from a given library together in a single, coordinate sorted BAM file.

          Originally posted by jullee View Post
          Also do I need to index again after the bam files are merged?

          Thanks!
          Yes(*), because your merged BAM is a completely new file and needs a new index.

          (*) I say yes assuming that you will be performing some task further downstream which relies on having a BAM index present, which is just about any task.

          Comment


          • #6
            Thanks kmcarr! Your response was very helpful!

            Comment

            Latest Articles

            Collapse

            • seqadmin
              Strategies for Sequencing Challenging Samples
              by seqadmin


              Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
              03-22-2024, 06:39 AM
            • seqadmin
              Techniques and Challenges in Conservation Genomics
              by seqadmin



              The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

              Avian Conservation
              Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
              03-08-2024, 10:41 AM

            ad_right_rmr

            Collapse

            News

            Collapse

            Topics Statistics Last Post
            Started by seqadmin, Yesterday, 06:37 PM
            0 responses
            11 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, Yesterday, 06:07 PM
            0 responses
            10 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 03-22-2024, 10:03 AM
            0 responses
            51 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 03-21-2024, 07:32 AM
            0 responses
            68 views
            0 likes
            Last Post seqadmin  
            Working...
            X