Stand Alone Bam to FASTQ

[dcfargo]

Does anyone have a suggested best practice utility for this?

[maubp]

What are you trying to do?
Do you want to pull out the reads as FASTQ records?
Do you care about the strand used for reads which mapped to the reverse stand?
Do you care about how paired end reads are named?

You could try seqret from EMBOSS 6.3.0,

http://lists.open-bio.org/pipermail/emboss/2010-July/003947.html

[dcfargo]

I do care about recovery of all of the information.

I'd like to essentially recover all the initial text information that went into making the BAM file.

[Martin R]

the problem you can run into is, that after alignment the quality values might change.

[dcfargo]

Sorry for my ignorance - why might the quality values change?

[Martin R]

That is no problem. It's also some point that confused me, btw is still confusing me.

The experince I made, ist that the aligned quality values (qv) in the sam files from e.g. bowtie are different from the ones in the original file. I think the values you get after the alignment are the qv from the alignment and not the one from the original file.

[maubp]

I do care about recovery of all of the information.

I'd like to essentially recover all the initial text information that went into making the BAM file.

Assuming I have understood your aim, that is not entirely possible.

e.g. Support you had some paired FASTQ reads like this:

@SRR001666.1/1 071112_SLXA-EAS1_s_7:5:1:817:345 length=36
GGGTGATGGCCGCTGCCGATGGCGTCAAATCCCACC
+
IIIIIIIIIIIIIIIIIIIIIIIIIIIIII9IG9IC
@SRR001666.1/2 071112_SLXA-EAS1_s_7:5:1:817:345 length=36
AAGTTACCCTTAACAACTTAAGGGTTTTCAAATAGA
+
IIIIIIIIIIIIIIIIIIIIDIIIIIII>IIIIII/
...

All that will be stored in SAM/BAM is the pair name without the suffix, here SRR001666.1, the sequence and quality. You lose any description from the FASTQ lines after the ID. Potentially the alignment tool may hard clip the reads so you don't even get the full sequence and quality.

If on converting SAM/BAM back to FASTQ you specify suffixes of /1 and /2, the best you can hope to recover is:

@SRR001666.1/1
GGGTGATGGCCGCTGCCGATGGCGTCAAATCCCACC
+
IIIIIIIIIIIIIIIIIIIIIIIIIIIIII9IG9IC
@SRR001666.1/2
AAGTTACCCTTAACAACTTAAGGGTTTTCAAATAGA
+
IIIIIIIIIIIIIIIIIIIIDIIIIIII>IIIIII/
...

This may or may not suffice for your needs.

[dcfargo]

Thanks so much.

Given some information may be lost and we'll just have to accept that would the best model for conversion be 2 steps such as:

1) SAMtools for BAM -> SAM

2) followed by a home made script for SAM -> FASTQ

[Martin R]

Well you don't have to think that complicated. There are two libraries you can use, and than you have your converter. I e.g. prefer Java and use biojava to read/write FastQ (http://www.biojava.org/wiki/BioJavaownload_1.7.1) and use samtools (http://sourceforge.net/projects/picard/files/) to read BAM/SAM files (it's the same).
Then you only have to transform from a SAM Object to a FastQBuilder:

public FastqBuilder convert(SAMRecord element2) {
FastqBuilder builder = new FastqBuilder();
builder.withDescription(element2.getReadName());
builder.withQuality(element2.getBaseQualityString());
builder.withSequence(element2.getReadString());
return builder;
}

that's the easiest way.

good luck

[maubp]

Thanks so much.

Given some information may be lost and we'll just have to accept that would the best model for conversion be 2 steps such as:

1) SAMtools for BAM -> SAM

2) followed by a home made script for SAM -> FASTQ

Not necessarily.

As mentioned above, EMBOSS 6.3.x can do SAM/BAM direct to FASTQ, although it may not do exactly what you want it to do.

You could also write a script to go from BAM to FASTQ, for example using pysam to access the samtools C API from Python.

Personally I've been doing with SAM/BAM to FASTQ in Biopython (to recover reads to redo a mapping), but this is with an experimental branch and is not ready for general use.

[maubp]

Well you don't have to think that complicated. There are two libraries you can use, and than you have your converter. I e.g. prefer Java and use biojava to read/write FastQ (http://www.biojava.org/wiki/BioJavaownload_1.7.1) and use samtools (http://sourceforge.net/projects/picard/files/) to read BAM/SAM files (it's the same).
Then you only have to transform from a SAM Object to a FastQBuilder:

public FastqBuilder convert(SAMRecord element2) {
FastqBuilder builder = new FastqBuilder();
builder.withDescription(element2.getReadName());
builder.withQuality(element2.getBaseQualityString());
builder.withSequence(element2.getReadString());
return builder;
}

that's the easiest way.

good luck

Plus potentially add code to append /1 and /2 if dealing with paired end data.

Also I would reverse complement any reads mapped to the reverse strand to recover them in their original orientation pre-mapping.

[maubp]

As mentioned above, EMBOSS 6.3.x can do SAM/BAM direct to FASTQ, although it may not do exactly what you want it to do.

Well, EMBOSS 6.3.1 isn't doing what I want it to do

http://lists.open-bio.org/pipermail/emboss-dev/2010-August/000667.html

http://lists.open-bio.org/pipermail/emboss-dev/2010-August/000668.html

This should be resolved in the next patch or point release though

http://lists.open-bio.org/pipermail/emboss-dev/2010-August/000669.html

Peter

[divon]

For the sake of completeness, I will just mention that you can also achieve this with my Genozip program:

genozip file.bam <---- compresses the BAM file
genocat file.bam.genozip --output file.fq.gz <---- converts it to FASTQ


See documentation here: https://genozip.com/sam2fq.html

Paper here: https://www.researchgate.net/publica...ata_Compressor

[divon]

Hi Andrey, which file can't you open?

[divon]

Here's an alternative link: https://genozip.readthedocs.io/sam2fq.html

Does this work?