cannot away with cuffdiff,incredible

[upper]

Hi,all

I have 4(A,B,C,D) sample in 4 times(increasing time),I got diff result in 3 different cuffdiff

1.cuffdiff 3(A,B,C) individual samples,got AB BC AC pairs,AB has 578 diff genes
2.add D to do cuffdiff ,got AB,AC,AD,BC,BD,CD pairs,AB has 78 diff genes.
3. just cuffdiff A,B AB has 692 diff genes.

the result show when cuffdiff with more than two sample,it will consider the third or more one while test diff between one pair.but which one is credible for two sample?
And if I want test diff gene between growth times,how to deal,pair-wise independent with multi cuffdiff test or multiple pairs together with once cuffdiff?

Thanks
Shen

[upper]

Oh,Nobody has same problem

[sdriscoll]

The cuffdiff results don't make sense and I wouldn't know which is more "correct" either. I'd probably look for another tool to run the differential expression tests.

[mbblack]

Hi,all

I have 4(A,B,C,D) sample in 4 times(increasing time),I got diff result in 3 different cuffdiff

1.cuffdiff 3(A,B,C) individual samples,got AB BC AC pairs,AB has 578 diff genes
2.add D to do cuffdiff ,got AB,AC,AD,BC,BD,CD pairs,AB has 78 diff genes.
3. just cuffdiff A,B AB has 692 diff genes.

the result show when cuffdiff with more than two sample,it will consider the third or more one while test diff between one pair.but which one is credible for two sample?
And if I want test diff gene between growth times,how to deal,pair-wise independent with multi cuffdiff test or multiple pairs together with once cuffdiff?

Thanks
Shen

Do you have any biological replicates at all in this study (it sounds like no).

If not, cuffdiff, is unlikely to give reliable nor meaningful statistical results. Even with a single biological replicate, the manual cautions that results may not be reliable if there are in fact many differentially expressed genes. However, in the complete absence of replicates, cuffdiff has no way to estimate dispersion and cannot therefore compute meaningful statistics. It will, I believe, be treating all four of your samples as replicates of a single condition in order to have some type of pooled samples to estimate variance from. But in your case, it would be a false and meaningless estimate of variance, and it would vary with the number of your samples that you use in a particular analysis (in other words, your estimate of variance for statistical testing with A,B,C may be very different from that with A,B,C,D, and neither is in fact an actual meaningful estimate of sample variance in your case).

Without biological replicates, there is no method that can compute reliable or rigorous estimates of statistical significance. This has been discussed a great many times here on these forums.

[sdriscoll]

good point, mbblack. that's very true. i remember a friend of mine here at Salk was getting strange DE results through DESeq and it turned out it was because he had many unrelated samples in in the counts matrix he provided to DESeq. in DESeq's dispersion estimation stage it was using all of those samples to establish the variance estimates which in turn caused the DE test to miss some things and appear to act randomly.

also, yes, the lack of replication makes it so statistics don't work. in fact it's really just for kicks that any DE tool can provide p-values for a 1 vs 1 test. try doing a 1 vs 1 with a t-test...not gonna work. under the assumption that most genes are NOT differentially expressed along with the assumption that genes with similar expression levels will have similar variances the tools can put together an estimated noise/variance background against which the observed changes between single replicate samples can be compared---but that's a lot of assumption. in some cases those assumptions may be valid and that's why these programs don't totally avoid reporting statistics for 1 vs 1 tests although I think they should. with a 1 vs 1 you may just as well go by some arbitrary fold-change cutoff for significance. but don't forget to verify your results ...right mbblack?

[mbblack]

I've said several times on these forums that if you want to do DGE analysis without replicates, forget about statistics altogether. Just rank order your genes by normalized expression and pick those with the greatest value up to some cutoff you are comfortable with. That's about the best you can do.

And no, under those circumstances, I would not trust any of those genes to be truly "differentially expressed" - what's that phrase, "trust, but verify"? (was that a Reagan catchphrase?). As you and I have discussed, there are times when the study clearly dictates a need for verification. A DE analysis without replicates is clearly one of those.

P.S. in a situation where replicates were truly impossible, I'd prefer to run traditional Affy cartridge arrays. At least with probe sets per gene you can do some fairly rigorous statistical analysis of observed gene summary differences (R tools like SScore were developed just for that very situation in fact).

[chadn737]

Honestly, if a lab is unwilling or unable to pay for replicates, then they should not be doing these sorts of experiments. Yes there are specific exceptions. In some cases, like patient samples, but this is the exception not the rule.

[sdriscoll]

agreed. i always recommend replication...and I try to put it into perspective when people ask me. if you were doing any other type of experiment, like something imaging based or behavior based, would you be eager to believe results from a single slide or a single animal? i doubt it. RNA-seq was confusing at first for some researchers because of the amount of data returned and the cost. It's hard to keep in mind that the common experimental design rules of sample size still apply.

[upper]

I have no replication to do cuffdiff.I believe bioreplication is reliable.I will test.