Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • shell script for concatenating fastq files

    Hi,

    I have 30 samples, each was sequenced in 3 flow cells in all 8 lanes. So I have three directories: FC1, FC2, and FC3. Inside FC1, there are 30 sub-directories: Sample_1, Sample_2,...,Sample_30. Inside Sample_1, e.g., there are 8 files for forward and 8 files for reverse (pair-end). All are .gz files.

    I wrote a shell script to concatenate all the fastq reads from the same sample so that each sample has two fastq files: Sample_1_R1.fastq and Sample_2_R2.fastq (reverse).

    for i in `seq 1 30`
    do
    gzip -dc Sample_${i}/*_R1_*fastq.gz >>Sample_${i}/Sample_${i}_R1.fastq ## forward
    done

    There are bugs but I can't figure out. Wonder what did I do wrong? Or someone here can show me a btter way to do this?

    thanks
    John

  • #2
    for i in `seq 1 30`
    do
    cat /dev/null > bigSample_${i}.R1.fq # create empty file
    cat /dev/null > bigSample_${i}.R2.fq # create empty file

    # create big R1 file for i-th sample
    zcat FC1/Sample_${i}/*R1*fastq.gz >> bigSample_${i}.R1.fq
    zcat FC2/Sample_${i}/*R1*fastq.gz >> bigSample_${i}.R1.fq
    zcat FC3/Sample_${i}/*R1*fastq.gz >> bigSample_${i}.R1.fq

    # create big R2 file for i-th sample
    zcat FC1/Sample_${i}/*R2*fastq.gz >> bigSample_${i}.R2.fq
    zcat FC2/Sample_${i}/*R2*fastq.gz >> bigSample_${i}.R2.fq
    zcat FC3/Sample_${i}/*R2*fastq.gz >> bigSample_${i}.R2.fq
    done
    Last edited by Richard Finney; 04-21-2016, 11:13 AM.

    Comment


    • #3
      Thanks Richard.
      I got mine to work. But I think your code is much cleaner. I like it.

      John


      Originally posted by Richard Finney View Post
      for i in `seq 1 30`
      do
      cat /dev/null > bigSample_${i}.R1.fq # create empty file
      cat /dev/null > bigSample_${i}.R2.fq # create empty file

      # create big R1 file for i-th sample
      zcat FC1/Sample_${i}/*R1*fastq.gz >> bigSample_${i}.R1.fq
      zcat FC2/Sample_${i}/*R1*fastq.gz >> bigSample_${i}.R1.fq
      zcat FC3/Sample_${i}/*R1*fastq.gz >> bigSample_${i}.R1.fq

      # create big R2 file for i-th sample
      zcat FC1/Sample_${i}/*R2*fastq.gz >> bigSample_${i}.R2.fq
      zcat FC2/Sample_${i}/*R2*fastq.gz >> bigSample_${i}.R2.fq
      zcat FC3/Sample_${i}/*R2*fastq.gz >> bigSample_${i}.R2.fq
      done

      Comment

      Latest Articles

      Collapse

      • seqadmin
        Strategies for Sequencing Challenging Samples
        by seqadmin


        Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
        03-22-2024, 06:39 AM
      • seqadmin
        Techniques and Challenges in Conservation Genomics
        by seqadmin



        The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

        Avian Conservation
        Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
        03-08-2024, 10:41 AM

      ad_right_rmr

      Collapse

      News

      Collapse

      Topics Statistics Last Post
      Started by seqadmin, Yesterday, 06:37 PM
      0 responses
      11 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, Yesterday, 06:07 PM
      0 responses
      10 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-22-2024, 10:03 AM
      0 responses
      51 views
      0 likes
      Last Post seqadmin  
      Started by seqadmin, 03-21-2024, 07:32 AM
      0 responses
      68 views
      0 likes
      Last Post seqadmin  
      Working...
      X