SEQanswers

Go Back   SEQanswers > Applications Forums > Epigenetics



Similar Threads
Thread Thread Starter Forum Replies Last Post
ChIP-Seq: ChIP-Seq Data Analysis: Identification of Protein-DNA Binding Sites with SI Newsbot! Literature Watch 0 12-02-2011 04:51 AM
ChIP-Seq: ChIP-chip versus ChIP-seq: Lessons for experimental design and data analysi Newsbot! Literature Watch 0 03-02-2011 02:50 AM
W-ChIPeaks: A web application for processing ChIP-chip and ChIP-seq data buckeye3947 Bioinformatics 0 01-24-2011 01:05 PM
Minimum Coverage for ChIP-Seq & RNA-Seq canis Illumina/Solexa 7 07-17-2009 01:34 AM
PubMed: An integrated software system for analyzing ChIP-chip and ChIP-seq data. Newsbot! Literature Watch 0 11-04-2008 05:03 AM

Reply
 
Thread Tools
Old 12-02-2011, 12:14 AM   #1
ETHANol
Senior Member
 
Location: Western Australia

Join Date: Feb 2010
Posts: 308
Default Minimum Criteria to Publish ChIP-seq data

There seems to be a disconnect between what I think should be required to publish ChIP-seq data and what I see published in a lot of papers. When I am reviewing papers I get that feeling that I am that jerk reviewer who's request go beyond what is reasonable, so I usually tone my comments down to closer to what I see published.

As with most stuff the bar will be raised as broader scientific community gains a better understanding of the pitfalls of the assay. Just like the first ChIP-seq papers seem completely inadequate today.

Here are some issues I think are worth discussing:
1) What is enough depth and how is it determined?
2) What are the mandatory controls. Input (my preference) but others use IgG.
3) Is one antibody enough? I don't think so because it doesn't control for antibody cross-reactivity.
4) Validation of data. Most people seem to pick their top peaks and check by qPCR, is that okay?
5) Number of replicates. They should be biological replicates in my opinion.

In order to avoid being that unreasonable 'third reviewer' and also to make sure the stuff being published is true, I'd like to hear people's opinions.
http://www.youtube.com/watch?v=-VRBWLpYCPY
__________________
--------------
Ethan
ETHANol is offline   Reply With Quote
Old 12-02-2011, 01:04 AM   #2
mudshark
Senior Member
 
Location: Munich

Join Date: Jan 2009
Posts: 138
Default

good questions!

Quote:
Originally Posted by ETHANol View Post
1) What is enough depth and how is it determined
there are both objective estimates (e.g. http://www.ncbi.nlm.nih.gov/pubmed/19029915) and experience that allow for a proper evaluation.

Quote:
Originally Posted by ETHANol View Post
2) What are the mandatory controls. Input (my preference) but others use IgG.
either input or IgG are mandatory. I prefer input as IgG has its own bias.

Quote:
Originally Posted by ETHANol View Post
3) Is one antibody enough? I don't think so because it doesn't control for antibody cross-reactivity.
well, it is difficult to raise a 'good' antibody in the first place. I think it is too strict to ask for a second antibody (as long as there is not another antibody available). Antibody specificity is a huge issue and to the best of my knowledge all attempts to control antibody quality even in huge ChIP mapping consortia kind of miserably failed. i.e. there are lots of 'black sheep' antibodies around, even in consortia databases. One major problem is the question how to define a good antibody and to design proper specificity controls. A classic is the story of H3K27me/H3K9me antibodies. Many researchers claimed they had the specific one, and most of them were wrong.

Quote:
Originally Posted by ETHANol View Post
4) Validation of data. Most people seem to pick their top peaks and check by qPCR, is that okay?
The question is 'what' is validated using qPCR: It is a validation of the Seq procedure and not a biological validation in the first place. I frequently have the impression that scientists are mixing up technical and biological validations and are happy with the pretty uninformative qPCR. Different biological experiments should tell you if your ChIPSeq data are making any sense.

Quote:
Originally Posted by ETHANol View Post
5) Number of replicates. They should be biological replicates in my opinion.
It depends very much on the experiment. A simple description of a binding profile with an established antibody and a standard procedure performed in an experienced lab will most likely look exactly the same if repeated. More than 2 replicates are not required and I would even accept a one-sample experiment if the data is fitting the expectation/other peoples data, other experiments in the manuscript.

If however ChIPSeq is performed in a 'real' experiment (comparing two different states - mutant vs wildtype, treatment non-treatment) biological replicates are always essential. At least the main message of this experiment has to be validated using replication. I see many many publications in which replication has not been performed even in highest level journals. I tend to wonder who were the referees as letting such studies pass is irresponsible and ignorant.
mudshark is offline   Reply With Quote
Old 12-02-2011, 06:38 AM   #3
ffinkernagel
Senior Member
 
Location: Marburg, Germany

Join Date: Oct 2009
Posts: 110
Default

Quote:
2) What are the mandatory controls. Input (my preference) but others use IgG.
Controversial topic.
In my order of preference:
Knockout. IgG from preimmune serum of same rabbit. IgG. Input.

4)
Validation in the DNA you sequenced is fairly useless, biological replicates every time.
Also, you need to go at least down to the threshold you're using for filtering, just looking at your N top peaks is not enough.

5) Completely agree with mudshark.
Though I consider a chipseq fishing expedition (identify candidates that might be differential ) followed by validation of 'those we cared about' in 3-5 further biological replicates permissible, if your story has nothing to gain from finding all differential events.
ffinkernagel is offline   Reply With Quote
Old 12-02-2011, 07:17 AM   #4
ETHANol
Senior Member
 
Location: Western Australia

Join Date: Feb 2010
Posts: 308
Default

Nice points guys.

On the multiple antibody issue, I completely disagree. To most all histone mods there are several antibodies commercially available. To proteins, antibodies are very easy and inexpensive to make. You can nearly always make antibodies way better then anything commercially available. It just takes 90 days (or less) so a little advanced planing is required. But I never see people doing it so I let it slide when I'm reviewing papers. I know from my own work, even the cleanest of clean antibodies sometimes have significant cross-reactivity.

The thing is is that while ChIP-seq is rapidly becoming less of a thing you publish on it own and more of a fishing expedition, it is still the center point of many publications in some well respected journals.

It is nice seeing the bar raised pretty quickly with ChIP-seq data.

While I usually set the bar at, does the data support the conclusions made in the paper, what about when someone else uses that data in another publication?
__________________
--------------
Ethan
ETHANol is offline   Reply With Quote
Old 12-02-2011, 08:36 AM   #5
ffinkernagel
Senior Member
 
Location: Marburg, Germany

Join Date: Oct 2009
Posts: 110
Default

Quote:
To proteins, antibodies are very easy and inexpensive to make.
For some proteins it is not easy - sometimes it's even quite non-trival to get enough purified protein for the immunization (meaning that while there's a commercial antibody, making your own is not an option).
And the animal cost should be included in your considerations.
So generally, I'd prefer a knock out control to a second antibody.

Though I have to ask - has anybody ever done a successful knock-down control for a ChIPseq experiment?
ffinkernagel is offline   Reply With Quote
Old 12-02-2011, 12:06 PM   #6
ETHANol
Senior Member
 
Location: Western Australia

Join Date: Feb 2010
Posts: 308
Default

Not to digress too much but if your antigen doesn't express well just try a different part of the protein. Animal cost (one rabbit) is less then the price of your average commercial antibody.
Always works like a charm for me:
http://ethanomics.files.wordpress.co...production.pdf
And then you can affinity purify it and get burn a whole in your western blot antibody with no background:
http://ethanomics.files.wordpress.co...rification.pdf

And there is always peptide antibodies and that requires zero work on the end of the researcher. They usually are not as good but most all commercial antibodies are raised against peptides. But again a quick affinity purification will almost always get you a better antibody then you can buy.

On the second issue, knock-down controls have not worked for me in the past so I stopped trying them. It seems, at least with the proteins I was working with, there was still enough protein in the cell to give a good ChIP signal. I have seen it work for others though.
__________________
--------------
Ethan
ETHANol is offline   Reply With Quote
Old 12-03-2011, 03:35 AM   #7
mudshark
Senior Member
 
Location: Munich

Join Date: Jan 2009
Posts: 138
Default

I agree that in theory it is not difficult to get 2 different polyclonals but consider

a) that in some labs (like mine) people work on more than 20 proteins. instead of having 2 polyclonals each we'd rather like to have different species for IF for example. And the monoclonals usually do not work well in ChIP.

b) And it is not always easy to get good antibodies without cross reactivity. Just as an example for one of our favourite proteins we do not have any good ChIP antibody after trying for more than 10 years, peptide, full length, monoclonals.. everything. Sure more than 20 attempts.

c) Also the expression of fragments or full length proteins for immunization is not always working. And affinitiy purification is not really something that improves the situation (in our hands), not for ChIP. I usually recommend crude serum for ChIPSeq.

As I said, it is nice to have more than one antibody, but as a referee I would never ask for a second one. It is not standard, there are other ways to estimate the validity of the mapping profiles and it takes a long long time to get a second batch.

As regards knock-down. Did it once with RNAi. I tell you the profiles look exactly the same. I guess it is due to (always!) incomplete knockdown. And you amplify the remaining 5-10% in the library prep. Not a usefull control.
mudshark is offline   Reply With Quote
Old 12-04-2011, 04:35 AM   #8
rory
Member
 
Location: Cambridge, UK

Join Date: Aug 2008
Posts: 28
Default

ETHANol, you are right to be asking these questions, and you are not being unreasonable in raising the standards for ChIP studies! Questioning work that you do not have confidence in is the opposite of irresponsible.

I particularly agree with mudshark about replicates. Unless the ChIP is really ancillary to the study, I don't see how a single replicate is ever acceptable. Having looked at hundreds of ChIPs, there is just too much experimental variance to have any confidence unless similar results are obtained when the experiment is repeated. This extends to peak callers, as there is so much variance between peak callers on the same data, as has been established in the literature many times. Sites identified from a single replicate with a single peak caller should not be acceptable. As others have suggested, if you are interested in differential binding, the need for replicates is absolutely essential. Right now we are looking at doing some power calculations to determine how many replicates are needed to identify differentially bound sites with an acceptable sensitivity, and aim for at least three separate ChIPs in each comparison group in order to get meaningful confidence statistics.

Regarding your other issues, depth is becoming less of an issue – for most transcription factors, a decent signal will be apparent within 20M reads, and we’re not seeing many experiments with fewer than that. This is another place where replicates can be helpful, as the replicates could be combined and a saturation analysis performed, but this may be overkill. Depth is a bigger issue for ChIPs that cover wide regions of the genome, like some histone marks (e.g. H3K27me3). For controls, one reasonable control should do it, be it Input (cleanest to deal with), IgG, or some untreated condition (e.g. vehicle). It is not always clear how deep to sequence an Input, but nothing has been published that I am aware showing it needs to go deeper than the ChIPs. We probably should be more concerned about identifying copy number variations in the sample, especially with cell line and clinical cancer tissues – significantly highly-bound regions should be checked to see if there is a copy number anomaly at that position in one of the conditions but not the other.

The antibody issues are well covered by other commentators, but if there is a specific reason to be concerned about cross-reactivity, it may be reasonable to check without doing a whole ChIP-Seq with another antibody. For example, do a knock-down of the protein of interest only and do a blot or stain showing nothing else is picked up by the antibody. Likewise, validation can take many forms. qPCR helps calibrate the false discovery rate, but if a specific biological question is being asked about an identified set of enriched regions, there are usually some consistently checks that help give confidence the intervals are meaningful, such as motif analysis for transcription factors, or gene set enrichment/pathway analysis of nearby genes to see if they match the expectations of the study.

At this moment, I think the onus is on reviewers to raise the quality and reproducibility of ChIP-Seq experiments, and ETHANol is asking exactly right questions!
rory is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 04:52 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO