Re-implementing the Chastity Filter algorithm from SCS 2.6 on fastq files?

[sperez317]

We have some older runs we did with SCS 2.6 and CASAVA 1.6. We no longer have the qseq or pretty much any other files, but we do have fastqs. Unfortunately, the fastqs were not filtered for the reads that passed the chastity filter, and that data is not in the fastq file itself.

Does anyone know where I can find the documentation that has the criteria SCS 2.6 used to tell if a read passed filter so that I can write a script to re-evaluate the reads in the fastq files? Or does anyone know of another solution to making these fastq files usable again?

Thanks for the help.

[kmcarr]

Chastity (and Purity) filters are based on relative fluorescent signal intensities of each of the 4 bases at each cycle (well, up to cycle 25) of each cluster. Unless you have the original intensity data there is no way to do Chastity filtering at this point.

Failing that I would suggest you just perform more standard Q-score based trimming/filtering using something like Trimmomatic.

[sperez317]

Ah, good to know. We do not have the original intensity data anymore, so I'll just use Trimmomatic.

Thanks.

[HESmith]

There used to be a flag in the old FASTQs that indicated pass/fail for chastity filtering (I believe it was the last field of the identifier). At one point it was 1/0, then changed to N/Y (although my recollection is that Y meant it FAILED chastity). You should be able to tell by examining reads with all Q-scores = 0 (those invariably fail chastity).

[sperez317]

I don't think there are in this one. An example from the file:

@NA328MB44S:1:1:1036:18513#0/1
CACACCATTAGTTAATGCCACATCTCCCACACCTACAATTTAGAGATCGGAAGAGCGGTTCAGCAGGAACGACGA
+
a\_aaaaaaYXY]OX]UU]UL[YY[_``Y_]]R`]``\_````Q_LYY[T\S\P]]_WSM``\`BBBBBBBBBBB


If I'm reading the documentation correctly, the first line only tells me computer/positioning data, and then multiplexing/mate pair information. I've looked for identifiers based on other projects where we do still have the old information, but I don't think there's any trace of it in these because of the way the person who did these runs did the conversions (These reads are from 2010 and he no longer works here).

[GenoMax]

You could ask the facility that ran the original to see if they kept a backup of the original run data (we do). Then you would be all set.

Edit: Perhaps you *are* in the facility that did the original runs.

[HESmith]

I don't think there are in this one. An example from the file:

@NA328MB44S:1:1:1036:18513#0/1
...

The last character of the identifier (after '/') is the chastity flag.

[sperez317]

If that's the case (I thought it was pair membership), then something went wrong in our initial conversion, since every line has a 1 at the end, even those such as:

@NA328MB44S:1:1:991:17986#0/1
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

which are obviously not good reads.

Trimmomatic is making them usable for our purposes, so it's working out.

[HESmith]

Okay, that's good to know.