Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • bedtools bamtofastq taking too long

    Hi
    I am running the bedtools bamtofastq for converting bam to splitted fastq (read1 and read2) files for paired end reads with an expected wall time of 24 hours and memory of 4000mb.

    The input bam file is about 140 GB whereas the output file size has reached to merely 0.3 GB. So it seems that the tools is running too slow for some reason. The following is command was used. The input file has been sorted by samtools sort by names.

    bedtools bamtofastq -i X_Sorted.bam -fq X.R1.fq -fq2 X.R2.fq

    Any suggestions appreciated.

  • #2
    Perhaps you have hit memory/wall time limit for this job?

    An alternative may be reformat.sh from BBMap suite. Brian's code tends to be very efficient so you may want to give it a try. Make sure samtools is in your path for something like following to work.

    Code:
    $ reformat.sh in=file.bam out=file.fq.gz
    Are you sure that both reads have been kept in the bam file? If not you will have a problem of singletons and you will need to separate them before de-interleaving the fastq file. Start with this:

    Code:
    $ reformat.sh in=file.fq.gz verifypairing
    $ reformat.sh in=file.fq.gz out1=R1.fq out2=R2.fq

    Comment


    • #3
      Thanks so much. It turned out that the output directory was located in a different file system in a cluster and that somehow causes bedtools output to be written at extremely slow rate. The problem solves if i use the output directory location within the same file system in the cluster. I will post more details in case this happens only with bedtools or is a general I/O issue.

      Regardless- I tried reformat.sh and based on output rate, it turns out to be relatively efficient. Thanks so much for the clue.

      Comment


      • #4
        Use the "bamtofastq" in Biobambam instead. It runs way faster and the code is highly optimized. Bedtools is written by amateurs. That's why it's slow.

        Comment

        Latest Articles

        Collapse

        • seqadmin
          Strategies for Sequencing Challenging Samples
          by seqadmin


          Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
          03-22-2024, 06:39 AM
        • seqadmin
          Techniques and Challenges in Conservation Genomics
          by seqadmin



          The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

          Avian Conservation
          Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
          03-08-2024, 10:41 AM

        ad_right_rmr

        Collapse

        News

        Collapse

        Topics Statistics Last Post
        Started by seqadmin, 03-27-2024, 06:37 PM
        0 responses
        12 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 03-27-2024, 06:07 PM
        0 responses
        11 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 03-22-2024, 10:03 AM
        0 responses
        53 views
        0 likes
        Last Post seqadmin  
        Started by seqadmin, 03-21-2024, 07:32 AM
        0 responses
        68 views
        0 likes
        Last Post seqadmin  
        Working...
        X