Capturing bowtie stats / samtools pipe / file IO?

[rwn]

Hi all,

I've been working on tweaking our NGS analysis pipeline and came across this minor issue for which I could not find a solution.

The current pipeline scripts do some I/O redirection (2>&1) to capture the stats from bowtie alignment (which are printed to stderr): This works fine if you simply issue a command to run bowtie.

bowtie -q -S index fqsequence >> stats.txt 2>&1

stats.txt
# reads processed: 1679207
# reads with at least one reported alignment: 1487157 (88.56%)
# reads that failed to align: 155467 (9.26%)
# reads with alignments suppressed due to -m: 36583 (2.18%)
Reported 1487157 alignments to 1 output stream(s)

As part of the upgrade, I want to pipe the bowtie output directly to SAMtools (instead of creating an intermediate SAM file) to produce the BAM file on the fly. No problem, this also works fine.

bowtie -q -S index fqsequence | samtools view -bS -o testoutput.bam -

[samopen] SAM header is present: 14543 sequences.
# reads processed: 1679207
# reads with at least one reported alignment: 1487157 (88.56%)
# reads that failed to align: 155467 (9.26%)
# reads with alignments suppressed due to -m: 36583 (2.18%)
Reported 1487157 alignments to 1 output stream(s)

I figured it would be easy to capture the stderr output from this pipe and send it to the stats file as before, but appending this command with >> stats.txt 2>&1 only shunts the SAMtools information to the file. The bowtie stats are still printed to the screen. I've played around with the i/o redirect commands and can't get it to work. Any suggestions?

bowtie -q -S index fqsequence | samtools view -bS -o testoutput.bam - >> stats.txt 2>&1

stats.txt
[samopen] SAM header is present: 14543 sequences.

screen output:
# reads processed: 1679207
# reads with at least one reported alignment: 1487157 (88.56%)
# reads that failed to align: 155467 (9.26%)
# reads with alignments suppressed due to -m: 36583 (2.18%)
Reported 1487157 alignments to 1 output stream(s)

[adamdeluca]

demo.pl

#! /usr/bin/perl
print STDERR "error";
print "out";

./demo.pl 2> stderr.txt | cat > stdout.txt

[rwn]

Thanks for the example... hadn't thought of piping the output to cat, but unfortunately, that syntax didn't work either.

[mgogol]

(bowtie genome --sam <(gunzip -c file.fastq.gz) | samtools view -bS -o results.bam - ) 2> bowtie.err

This worked for me (with bash).

There may also be some way to use tee here, but I don't know exactly.

http://stackoverflow.com/questions/692000/how-do-i-write-stderr-to-a-file-while-using-tee-with-a-pipe