qRT-PCR... necessary with RNAseq bioreps?

[seq_]

For publication of RNAseq differential expression data, is it strictly necessary to confirm by qRT-PCR? Our data has 3 bio-reps per condition. If we really need to do a qRT-PCR on a small set of genes, we will, but I just don't really see why it would be necessary when using bio-reps for the sequence data... Any pubs that provide justification for not doing the qRT-PCR would be much appreciated.

[chadn737]

Even with replicates, how do you confirm your results? I think its risky to simply assume that your RNA-seq results are correct.

[sjm]

validation of RNASeq results

My two cents' worth is that RNA-seq is in general a vastly superior technique to RT-qPCR, and having published a few papers now involving RNA-seq methods, I"m becoming frustrated with reviewers continually asking for RT-qPCR 'validation' of the data. I don't see reviewers asking for immunoblotting results to be validated using multiple different primary antibodies, or for proteomics+mass spec identification to be validated using immunoblotting. And RT-qPCR methods use oligonucleotide primers that are shorter than the average sequencing read - have all the amplicons generated by RT-qPCR been themselves sequenced to ensure the correct amplicon is being generated?

The logical conclusion of folks requesting validation of RNASeq by RT-qPCR is that ALL differentially expressed mRNAs in a study then ought to validated by RT-qPCR, not just a select handful, and in my opinion that's a waste of resources.

While our group has not performed large-scale RT-qPCR 'validation' of RNA-sequencing (i.e. we haven't done qPCR on hundreds of mRNAs) you could do worse than to take a look at Circ Res (2010) 106:1459, 'Deep mRNA sequencing for in vivo functional analysis of cardiac transcriptional regulators...' and cite the figure we provide comparing the two methods.

[seq_]

Thanks for input. The more we thought about just doing the qPCR rather than argue the point with editor/reviewer, the less it seemed to make sense. We really should be able to trust RNAseq data at this point... It seems like a step backwards to run a qPCR of a few 'randomly selected' genes, show a gel and claim that validates the more sophisticated entire sequencing experiment. As pointed out, the qPCR amplicons aren't being sequenced, so it's not even a guarantee that the correct gene is being assessed.

[dvanic]

have all the amplicons generated by RT-qPCR been themselves sequenced to ensure the correct amplicon is being generated?

Actually, if you start looking at the nitty-gritty of RT-qPCR (i.e. how to actually get it to work and give you true quantitative results), you need to conduct it using the MIQE guidelines, which require (among other things):
- multiple reference (housekeeping) genes for normalization, optimized for your biological system
/I cringe EVERY TIME I see people run qPCR with 1 reference gene./
- consideration of primer amplification efficiencies, and optimization of primers to be efficient
(and, yes, this takes experiments, not just a day with primer3)
- Sanger sequencing validation of all amplicons.
The latter caused me a lot of pain when I moved from far out Russia (where we were poor as, but DID do validations, because no one would trust us otherwise) to Oz, where people thought I was mad to even think that if my gel "showed" a band that was of the correct length, the amplicon could be WAY OFF my gene of interest. I provided them with several such examples, and managed to get a few more, but I am still the ONLY person in my lab, and probably building, who actually uses sequencing validation. And, no, my boss does not love me.

Why? Because people are lazy, and think qPCR is this gold standard method no matter how haphazardly and sloppily it is being used. I DO believe it to be the most accurate, but ONLY if (just like all other methods) done properly.

[sjm]

These are great points on the MIQE guidelines and the necessity of doing qPCR the right way or not at all. Dvanic, I share your pain about the need for multiple reference genes (I've had a very hard time arguing that with multiple colleagues!) and Sanger sequencing (also a needlessly contentious issue).

Do you happen to know of any studies where someone has compared a lot of correctly-performed qPCR to RNASeq for differential gene expression? It would be a great thing to have as a ready reference for manuscript and grant reviewers.

[billstevens]

bump?

If anyone does know of anyone comparing correctly-performed qPCR to RNASeq for gene expression that would be awesome.

But let's talk for a second about the "correct" way of doing the qPCR, if I wanted to do that as well. Most of my "significantly" differentially expressed genes are less than two-fold change difference, so would qPCR even sensitive enough to pull that off? Our lab hasn't been able to really see the difference for anything less than two-fold. I was thinking I might just pick the few genes that did have over two-fold change, and run qPCR on them to show the difference and mention that qPCR is just not as sensitive as RNA-Seq.

If anyone has a reference comparing sensitivity of RNA-seq versus qPCR that would be greatly appreciated.

[dvanic]

In terms of "how to do PCR" - the MIQE guidelines provide a checklist for how to set up a qPCR:
http://www.ncbi.nlm.nih.gov/pubmed/19246619
is the original paper, and
http://www.biomedcentral.com/1471-2199/11/74
has some more useful information.

And in terms of two-fold change: IF you set up qPCR properly, then even a two-fold change should be detectible. qPCR is, technically, the gold standard method for quantitation, but only if it set up properly.
The reason it isn't used down to these sensitivity levels is that people tend to be lazy in designing their assays, and hence can't atribute the difference to biology or technical variability.

[Apis37]

What is meant by statements like "technically qPCR is the gold standard." Where is the evidence that it works better than RNA-Seq? Given all the problems with qPCR I seriously doubt it, but we need data to prove it either way.

In general, how can you validate a higher resolution technique with a lower resolution technique? Even if the qPCR results don't corroborate it doesn't mean anything. Reviewers demanding that you waste your time on stuff like this is very frustrating.

[chadn737]

I think "gold standard" really means that it is what the majority of people are comfortable with, familiar with, and its more easily understood since its PCR. I actually think that qPCR is more error prone.

I don't see it as pointless however. Its good to have multiple lines of validation in any experiment and I say that is true of not just RNA-seq or microarrays. Where I see the real value is validating genes that are of particular interest. I understand that many argue that a correct validation includes a random set of genes expressed at different levels, but I am typically more concerned with genes of interest that make the story.

[dvanic]

I think "gold standard" really means that it is what the majority of people are comfortable with, familiar with, and its more easily understood since its PCR. I actually think that qPCR is more error prone.

It means that the biases of the technology have been better characterized and methods of dealing with them described and validated.

The reason it's more "error prone" is that most people don't do the appropriate controls, validations and optimizations, and so get variable results.

As I mentioned above, qPCR is the gold standard, when used properly (!!!), and has been shown to be in many studies (Steve Bustin's work and references therein are a good place to start reading) in the early to mid 2000s. Some of these studies used spike-ins so the concentrations of the RNA were known beforehand, and with proper analysis these were found to be recovered quite well. The main challenge of conducting this "proper analysis" was found to be identifying the appropriate genes to normalize to, which are somewhat unique to every system.

Given all the problems with qPCR I seriously doubt it, but we need data to prove it either way.

A plethora of studies comparing qPCR and RNA-seq (and microarrays) have been published; some of the key ones include:

http://genome.cshlp.org/content/18/9/1509.full

http://www.sciencemag.org/content/320/5881/1344.full

http://www.nature.com/nmeth/journal/v5/n7/full/nmeth.1226.html

http://nar.oxfordjournals.org/content/39/2/578.full

how can you validate a higher resolution technique with a lower resolution technique?

Sorry, but qPCR is actually a HIGHER RESOLUTION technique for the PARTICULAR locus that you are amplifying. It's like looking at the sky: is the higher resolution (for that locus/region) the one where you see all of the stars, or just focus a super-strong telescope on one star? Which one gives you more information about how real the signal from that locus is/that star is (and what its properties are)?

Reviewers demanding that you waste your time on stuff like this is very frustrating.

These reviewers may be biologists, who have yet to be convinced that we have figured out the statistics of RNA seq data analysis (I'm a bioinformatician, and I am not convinced of this at all - if you are, I'd love to see a paper where this is shown). Using an independent method of validation is always a good thing to do, and has been required by biologists for decades, including in low-throughput "wet lab" experiments. If you think it is a waste of time perhaps theoretical bioinformatics would be a better field to contribute to: you need simulations, but don't need to get your hands dirty and deal with frustrating reviewers who demand you do the basic control in the field (which makes a whole lot of sense when you think and read about it)?...