merging fastq files

[shilo]

Hi all,

We have recently got the fastq files from our RNA-seq experiments. For each treatment we got many fastq file (1Gb each). I'm looking for a way to merge these file before running the analysis (Tophat).

Thanks,
Shilo

[upendra_35]

Shilo,

Either you can use 'cat' to merge all fastq files:

cat file1.fq file2.fq file3.fq file4.fq file5.fq > merged.fq

cat file*.fq > merged.fq

OR

Tophat can take them all without merging:

tophat [options] <index> <file1_1, file2_1, etc...> <file1_2,file2_2, etc...>

Hope this helps.....

[shilo]

Thanks, its work.
Just to make sure, by using cat it creats one file that merged the previous files and order the data of one file after the other??

Thanks,
Shilo

[arkal]

Thanks, its work.
Just to make sure, by using cat it creats one file that merged the previous files and order the data of one file after the other??

Thanks,
Shilo

if u do what he said :

Shilo,


cat file1.fq file2.fq file3.fq file4.fq file5.fq > merged.fq

cat file*.fq > merged.fq

then merged.fq will contain all data from file1.fq, file2.fq, file3.fq, file4.fq and file5.fq in that order. Basically you are printing those 5 in that order into file merged.fq instead of onto the terminal.
Hope that answers your question!

-A

[shilo]

Yes, Thanks a lot

[Lovro]

This preserves the original files. Can I rather append the second file to the first, to conserve disk space ?

[Brian Bushnell]

If you want to conserve disk space, I recommend gzipping the files, and deleting the originals.

[Lovro]

I was working on a virtual machine and had only cca 100gb left on the hdd. I have 2 WGS fastq.gz with 50gb each... I solved the problem by allocating extra hdd space to the virtual machine, but I would be nice to know a proven method to append or concatenate fastq.gz-s without temporarily needing extra hdd space.

[Brian Bushnell]

I've wanted something like that too (for time rather than space reasons), but I don't think it's possible.

However, there are a lot of programs that will let you stream from standard in without writing to disk:

cat *.fastq.gz | program