Illumina filtered reads vs unfiltered reads

[rnaeye]

Hi!
I was wondering if someone could tell me why some of my Illumina reads have either 1:N:18:ATCACG or 1:Y:18:ATCACG. The vast majority of reads do not have any of these flags. What does it mean?

My understanding is that some of the reads are filtered and they either pass it or fail, and those are labeled as 1:N:18:ATCACG and 1:Y:18:ATCACG, respectively. Why only some of the reads are filtered? What does filtering do? How about reads they are not flagged with :Y: or :N:? Are those high quality reads.

Can someone help me understand this puzzle. Thank you for your help.

[Wallysb01]

The Y/N is wether or not it failed the pass filtering step. So the better reads are going to have a N. I believe the quality score cut off is 2 however (might be wrong here been a while since I did this). So even reads that passed can be fairly poor in quality. Then the ATCACG is the bar code.

So if some read IDs have this information and other don't you might ask some questions to who ever did the CASAVA pipeline for you, as this doesn't seem like the standard way to go about it. They all should have this info. My guess is that they didn't handle the bar coding of your samples very well.

[kmcarr]

Hi!
I was wondering if someone could tell me why some of my Illumina reads have either 1:N:18:ATCACG or 1:Y:18:ATCACG. The vast majority of reads do not have any of these flags. What does it mean?

I would be very concerned if the majority of the sequence ID lines in your FASTQ files do NOT have this block of information attached; it should be present for every sequence ID. If the majority of your reads do not include this information then the FASTQ file is malformed and you should talk with your sequencing provider. I have attached a PDF which describes in detail the format and meaning of the Illumina FASTQ sequence ID line (as well as the file naming convention) for files generated by CASAVA 1.8 and greater.

My understanding is that some of the reads are filtered and they either pass it or fail, and those are labeled as 1:N:18:ATCACG and 1:Y:18:ATCACG, respectively. Why only some of the reads are filtered? What does filtering do? How about reads they are not flagged with :Y: or :N:? Are those high quality reads.

Filtering is performed by the Illumina Real Time Analysis (RTA) software and happens during the run. The filter algorithm examines the ratio of signal intensities for each cluster at each cycle up to cycle #25. If the intensity ratios exceed a particular threshold more than once during the first 25 cycles the cluster (read) is designated as "failed filtering" and will have a "Y" in its FASTQ header line, otherwise it is passed and will have an "N". Beyond cycle 25 has no impact on whether a read is considered passed or failed.

[rnaeye]

Hi Wallysb01 and kmcarr,

Thanks for the information. I checked the file again. You are right. All the sequences in fastaq file associated with either "...Y..." or "...N..." flags.

I have noticed that it's the SAM (Bowtie output) file does not have flags in most sequences. The sequences without flags are aligned to target genome (has chr number and positions), and sequences tagged with 'Y' or 'N' flag are not perfectly aligned to target genome.