Fastq adaptors removal/stripping/cleaning

[Gianza]

Hi guys,
I'm facing such a very dumb problem.
I cannot find a tool which simply strip off adaptors from Fastq Illumina files. I have contamination from library synthesis adaptors (SMART).
-Seqclean only works with fasta.
-Lucy2: libgtk1.2 libraries no longer supported in my linux distro (and I don't even know if it handle fastq)
-fastx_clipper form FastX-toolkit: makes a big mess, cause it doesn't only strip the adaptor but blow away the whole sequence (it's not supposed to behave like this): it results in loss of more than 1/3 of the dataset.

Other solutions are integrated in assembler or aligner, but I need a crude trimmed fastq as output.

Does anybody know something which might be helpful to me?

Thanks in advance!!

Davide

[Adrian_H]

fastx_clipper seemed to work fine for me. Are you sure you're running it with the parameters you want? (I did have some issues with clipping paired-end reads, where fastx_clipper would blow away one of the sequences if it was too short after clipping, leading to unmatched pairs, and had to modify fastx_clipper to leave sequences in even if they were completely clipped)

[maubp]

Personally I use Biopython with some simple adaptor matching.

There ought to be a tool in EMBOSS to do this too...

[raela]

With some of the tools in fastx_toolkit, I've found redirection and flags act differently. Most give the option of using -i infile -o outfile or `cat inline | tool > outfile` - if you're using -i -o, try cat in | > out and see if that works better. I forget which one usually gives me issues with it.

[bioinfosm]

did you try novoaligns adapter trimming? That ought to help

[Gianza]

I'm using this command.
I want to clarify I'm not a bioinformatician.

fastx_clipper -a AAGCAGTGGTATCAACGCAGAGTACGCGGG -i 1M_1.txt -M 20 -n -o output.txt

For Adrian_H: could you please give me your modified version. You just make me remeber the aligner I will use (Mosaik) doesn't accept missing paired-end.

For raela: Sorry I didn't get your point with cat and pipe..could you please write how should I type it? Thanks!

However; I found that I have more than one trouble: while blowing away tons of reads with no reason (other than a regular adaptor match, I guess), it also leaves tons of other adapters in the output sequences (e.g. sequences heavily trimmed, and the little remaining stretch....is an adaptor!! )

If anybody can provide me a working utility, will be my idol.
(I played so well with SeqClean and cln2qual.....why so many format on this world??)

[Gianza]

Novoalign seems to have an internal-only trimming pipeline.
Looking at the manual it doesn't seem it will just return a trimmed fastq, but the final alignment....Am I wrong?

I would like to avoid conversion to fasta+qual, expecially cause I'm dealing with several dataset (need to report clipping to qual).

Mosaik, only accepts perfect matching paired sequences (only a missing one and it will stop, thus I'm also concerned about this issue). Have anybody evere dealed with this kind of issue: keep zero length sequences?

[john_mu]

http://wiki.bioinformatics.ucdavis.edu/index.php/Data_Analysis

There are some perl scripts there, which might help.

[Jose Blanca]

We have build a pipeline, and the first step (the read cleaning) takes care of that. You can take a look at:

http://bioinf.comav.upv.es/ngs_backbone/

[Zigster]

fastx_clipper seemed to work fine for me. Are you sure you're running it with the parameters you want? (I did have some issues with clipping paired-end reads, where fastx_clipper would blow away one of the sequences if it was too short after clipping, leading to unmatched pairs, and had to modify fastx_clipper to leave sequences in even if they were completely clipped)

yes I have written to the fastx guy about orphaned pairs he said the next version might have some solution. At any rate I think the fastx clipping is too aggressive.

[robs]

Did you try a tool called TagCleaner?

http://www.biomedcentral.com/1471-2105/11/341

http://tagcleaner.sourceforge.net/

It's a web-based tool, but I heard you can contact them if your files are large and they will process them offline for you.

[raela]

Not sure if it'll work in your case, but try running it as:
cat 1M_1.txt | fastx_clipper -a AAGCAGTGGTATCAACGCAGAGTACGCGGG -M 20 -n > output.txt

[vgrubor]

You can use Genome Analysis Toolkit (GATK) to do this. http://www.broadinstitute.org/gsa/wi.../Read_Clipping

You can configure it to mask your adapters sequences with Ns so you don't end up with an empty sequence which can cause trouble with aligners when aligning in a paired-end mode.

[Zigster]

oops nevermind I see it

can someone familiar with FASTX explain to me which 14 nt are aligning here? it seems way too aggressive

cat myseq.fq 
@HWI-EASXXX/1
AACGCGATGCCTCCATTGCTGGTGCAACTGAGCCTGGATATCGGCAGTGCGATCCTCATGGACTTGGATCTGGGTT
+HWI-EASXXX/1
`_bb_b_bbYbb^bbbaaXbbb`b_a[S``[[MWO`\``]b_bbJ\^Z\J`Y^a[`^[b_bF^b_BBBBBBBBBBB

>cat myseq.fq | fastx_clipper -a AGATCGGAAGAGCGGTTCAGCAGGAATGCCGAGACCG -M 14
@HWI-EASXXX/1
AACGCGATGCCTCCATTGCTGGTGCAACTGAGCCTGG
+HWI-EASXXX/1
`_bb_b_bbYbb^bbbaaXbbb`b_a[S``[[MWO`\

[Adrian_H]

If you dig into the fastx_clipper source code, you can see what it's doing (I agree with you that I'm not at all sure that it's the right thing to do though!).

if ( alignment_size > 5
&&
alignment_results.target_start == 0
&&
(alignment_results.matches * 100 / alignment_size ) >= 75 ) {
//printf("--2\n");
return alignment_results.query_start ;
}

I think that this is what is aligning:

ATATCGGCAGTGCGAT
: ::::: :: ::: :
AGATCGGAAGAGCGGT


and the it is cutting off everything that follows