Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • GATK re aligner step doubles file size?

    Has anyone else seen the GATK IndelRealigner double file size? It happened on a set of 12 genomes I'm processing. After the next step the file sizes returned to their normal range. A quick look at the first reads in the files showed no difference; same reads in the same order with the same amount of metadata, a little of it changed. I'm indexing them now so I can sample a few other regions.

    Thanks!

  • #2
    Sounds odd. The only reason I can think is if GATK LR is returning the BAM unsorted for some reason. A sorted BAM can be smaller than an unsorted BAM since it is easier to compress...

    Comment


    • #3
      Is that the IndelRealigner? I found the same problem with TableRecalibration, but that happens because GATK retains old quality scores for each read. Solved with "--doNotWriteOriginalQuals" options.

      Comment


      • #4
        I had noticed that too... Is there any reason why anyone would want to retain the original quality information (aside from an OCD-esque obsessing with not discarding anything)

        Comment


        • #5
          Hi Dawe --
          While the quality scores are retained in the later files, those files are back down to a reasonable size. The following image shows the progression of file sizes in obsessive detail. There are 12 points per analysis step because I'm running 12 samples in parallel:



          I'm wondering if I could have caused the problem by something atypical I did in preparing the interval file for the realignment: I needed to add read groups to the alignments, so I ran parallel jobs creating the interval file and adding read groups. Then I used the interval file to realign the reads in the file with the newly added read groups (one per file). I reasoned that read groups aren't relevant to realignment when there is one per file, but maybe I tripped over some unexpected consequences.

          Comment


          • #6
            >Is there any reason why anyone would want to retain the original quality information

            swNGS: it's useful if you plan to keep the bams as your sole archive for a sequencing project because if you need to realign in the future (e.g. to a new version of the genome or a new genome entirely because you've been aligning to a related organism since yours isn't sequenced yet), you can recreate fastq from the original quality scores, and not have artifacts in the quality scores from errors in the recalibration based on the first genome.

            But obsessive data retention may play a part too.

            Comment


            • #7
              Manducasexta: hmmm you have a point there!
              We were having a discussion in the lab recently about what data to keep/discard etc. I'm all for keeping the minimum required, and hadnt considered that you could regenerate the FASTQ from the bam.
              What would be the path to achieve this? As I could theoretically discard the original fastqs....

              Comment


              • #8
                swNGS: I haven't done it, so I don't have a method immediately at hand. But having verified that the the information is present in the SAM file, I'm confident that fastq containing the original quality scores could be generated from SAM using perl or (probably) some other tool for parsing SAM format. When using an aligner that can use BAM input, it would be sufficient to replace the recalibrated quality scores with the original scores in a copy of the file.

                Comment

                Latest Articles

                Collapse

                • seqadmin
                  Strategies for Sequencing Challenging Samples
                  by seqadmin


                  Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
                  03-22-2024, 06:39 AM
                • seqadmin
                  Techniques and Challenges in Conservation Genomics
                  by seqadmin



                  The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

                  Avian Conservation
                  Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
                  03-08-2024, 10:41 AM

                ad_right_rmr

                Collapse

                News

                Collapse

                Topics Statistics Last Post
                Started by seqadmin, 03-27-2024, 06:37 PM
                0 responses
                13 views
                0 likes
                Last Post seqadmin  
                Started by seqadmin, 03-27-2024, 06:07 PM
                0 responses
                12 views
                0 likes
                Last Post seqadmin  
                Started by seqadmin, 03-22-2024, 10:03 AM
                0 responses
                53 views
                0 likes
                Last Post seqadmin  
                Started by seqadmin, 03-21-2024, 07:32 AM
                0 responses
                69 views
                0 likes
                Last Post seqadmin  
                Working...
                X