Assessing quality and accuracy of de novo genome assembly

[rwness]

All,
I am curious whether anyone out there has a method for assessing the quality and accuracy of de novo genome assemblies? I am currently doing in silico simulations of de novo genome assembly from a previously sequenced genome to determine the best assembly parameters (K-mer size, coverage cutoff etc) and optimal dataset (mate pair library size, coverage etc). The ultimate goal will be to use these parameters to assemble the genome of a related species, de novo.

However, the difficulty is that after simulating the data and making a de novo assembly I don't know of any statistics or methods to compare the assembled contigs back to original sequence that they were simulated from. This requires two steps
(1) align assembled contigs to reference genome
(2) assess the fit

People often optimize N50, assembly size, contig number and other length-based measurements - but this only makes for bigger and bigger contigs and there is little information about whether these contigs are accurate. I have been using BLAST to compare the contigs to the reference and asking how well they fit, how long the alignments are and how many mis-assembled contigs there are. If anyone has ideas or methods for assessing the accuracy ( or overall similarity of an assembly and a genome) I would be grateful to hear about it. - Rob

[nickloman]

Good questions - I don't think there is a simple or single answer at this point.

This paper suggests a potential metric

http://genome.cshlp.org/content/20/5/675.full

You might be interested in the recently announced Assemblathon

http://assemblathon.org/

[bckirkup]

To check assemblies...

... you need an external truth. We are using 'optical mapping' from OpGen, for example. Any sort of physical map or some kinds of PCR can be used, however, depending on the size of the genomes. You can't rely on other genomes.

For some microbes, things like skew can help you get a sense of whether your assemblies seem wrong, but they aren't necessarily a solid confirmation one way or the other.

[rwness]

... you need an external truth. We are using 'optical mapping' from OpGen, for example. Any sort of physical map or some kinds of PCR can be used, however, depending on the size of the genomes. You can't rely on other genomes.

For some microbes, things like skew can help you get a sense of whether your assemblies seem wrong, but they aren't necessarily a solid confirmation one way or the other.

Sorry I think I was unclear - what I am doing is simulating short read data from a species with a sequenced genome. Then trying to do de novo assembly of that simulated data. Then I want to compare my assemblies to the original genome to see how well the assemblies performed. It is similar to just testing the quality of short read genome assemblers. However, my ultimate goal is to use what I have learned in this species to apply to de novo assembly of another, related species with NO previously sequenced genome. I am trying to find a good method/metric to assess the quality of those simulated assemblies.
Nickloman's comments are helpful, but I haven't yet been able to read the first paper that was recommended.

[flxlex]

This paper (interesting read otherwise) describes some metrics, I guess there will be other papers with yet more ways of getting quality metrics. Feels like we should develop a consensus...

[azroger]

I don't think there is any one metric for genome assembly quality. Obviously, size matters, but so does representation (how much of the genome sequence is actually covered), mismatch rate, indel rate, and misassembly rate. All these quality metrics are derived relatively easily for a known standard genome, when you use simulated reads extracted from it. Check out www.plantagora.org, for more information on the whole question. That's what it's focus is - simulated read assembly to evaluate a lot of different sequencing and assembly approaches. The project uses a long list of metrics for evaluation of the assemblies (so you can decide which are most important to you).