Adding NGS data to the UCSC genome browser

aer · 01-21-2011, 06:39 AM

Hi everybody,
I´m a phd student from Germany analyzing the functional mechanism of a transcription factor and trying to find out new target genes. We used ChIP-seq analysis to identify the new target genes. Now, I´m trying to work with and understand the NGS data: using the UCSC genome browser I would like to see the differences between my sample and the negative control. By adding new cumstom tracks I can upload one set of data in wig format (=one chromsome).
Can anybody tell me how I am able to upload TWO sets of data in wig format to the UCSC genome browser so I can se the differences between the sample and control???
Thanks a lot!

Gangcai · 01-21-2011, 07:05 AM

Just upload both of them to UCSC (submit one file each time), then you can see both of them on Genome Browser.

aer · 01-24-2011, 02:41 AM

Thank you! But I still have a problem: First, I uploaded the data for the X-chromosome for my sample. When finished the page "manage custom tracks" appears. I marked the data with an x and clicked on "add custom tracks" and submitted the data for the X-chromosome for my negative control. But after a while this error pops up: needMem: trying to allocate 528709463 bytes (limit: 500000000). What can I do?

ulz_peter · 01-24-2011, 03:15 AM

Did you do the Wig to bigWig conversion. This is recommended for large datasets. A tool to convert the file can be downloaded here:

http://hgdownload.cse.ucsc.edu/admin/exe/

hope that helps...

Gangcai · 01-24-2011, 03:18 AM

That means the wig file is too large to be submitted to UCSC. You can try to convert wig file to bigwig by the wigToBigWig program provided by UCSC. Hope it works.

aer · 01-24-2011, 03:44 AM

Thanks a lot!
But these applications are only for Linux or MacOSX. Are there any conversion programs running on a Windows platform (32 Bit)???

ulz_peter · 01-24-2011, 04:21 AM

You could download the jksrc.zip package which contains the source code of each of the utilities
(http://hgdownload.cse.ucsc.edu/admin/jksrc.v245.zip) and compile them with a native Windows C-Compiler (actually I don't know any but that should be easy to find using Google).
Other solution would be the installation of cygwin (http://www.cygwin.com/) and emulation of a linux terminal within windows. You should be able to start the bigTobigWig program within Cygwin.

Hope that helps...

aer · 01-26-2011, 07:54 AM

Thank you! But unfortunately I wasn´t really successful..
I heard about installing the UCSC genome browser locally, then I should be able to handle my data in wig format I was told. Does anybody know something about the possibility to install the UCSC genome browser to you local pc??? Or how to do it???
Thanks a lot!

JohnK · 01-26-2011, 08:30 AM

Quote:

Originally Posted by aer

Hi everybody,
I´m a phd student from Germany analyzing the functional mechanism of a transcription factor and trying to find out new target genes. We used ChIP-seq analysis to identify the new target genes. Now, I´m trying to work with and understand the NGS data: using the UCSC genome browser I would like to see the differences between my sample and the negative control. By adding new cumstom tracks I can upload one set of data in wig format (=one chromsome).
Can anybody tell me how I am able to upload TWO sets of data in wig format to the UCSC genome browser so I can se the differences between the sample and control???
Thanks a lot!

i would convert thy wig files to .bw format and get thee self a host of some sort so thee can upload thy bigWigs to thou's server and then thy may hurriedly view thy data set without having to endure the painful wait when uploading .wigs to UCSC.

Bruins · 01-26-2011, 08:55 AM

Quote:

Originally Posted by JohnK

i would convert thy wig files to .bw format and get thee self a host of some sort so thee can upload thy bigWigs to thou's server and then thy may hurriedly view thy data set without having to endure the painful wait when uploading .wigs to UCSC.

Also, getting your local UCSC browser seems like a great idea, but you have 32 bits windows. That just might be a problem. Haven't installed UCSC locally myself but if it does work on your local win it'll be veeeeeeeeeeeery slow I think.

Eh, as for the 'I need Linux' problem: google into the wonderful world of dual boot or vmware. I can help you with dual boot, but have no experience with vmware. Ubuntu would be a nice choice. Having your own linux lying around somewhere helps in NGS.

biznatch · 01-27-2011, 12:08 PM

How big is your wig file? Is it a variableStep or fixedStep wig? Unless you have very, very few data points, a fixedStep will be a smaller file, about half the size of variable in my experience, since the fixed step has 1 column of data and the variable step has 2. All my fixedStep .wig files (1 per chromosome), are maximum a few hundred mb, which is under the 500mb UCSC Browser limit (although even files close to but under the limit may not work, but I haven't had any problems).

After generating my .wig files I compress them with gzip to .gz files, then each is about 5-15mb per file. They upload quickly, but it takes a minute or 2 for the UCSC genome browser to process them.

Galaxy can convert wig to bigwig, I haven't tried it though.

I don't know if it works for bigwig files, but for my wig and bed files I sometimes upload them to dropbox and use the sharing link to upload them to UCSC. Usually this is when someone else in my lab wants to view the files, so I can just send them the link.

I've looked in to it very briefly, but setting up your own local UCSC genome browser looks like it really takes some know how...although I've only been using linux for 2 months so maybe it's not that bad. Anyways, one of the things I like about the Browser is all the other available tracks, which I guess you'd then have to download so that might negate some of the advantage.

aer · 02-01-2011, 12:04 AM

Hi biznatch,
my wig files are about 350MB to 750MB, depends on the chromosome. But the point is that I would like to upload to sets of data and this is always above the 500mb-limit of the UCSC browser. What´s the difference between fixedStep and variableStep wig file?
Is it possible to upload zipped data files??? I have to try..
By the way, I haven´t been successful in converting my wig files into bigwig.. I will have a look on Galaxy, thank you very much!

ulz_peter · 02-01-2011, 02:35 AM

Short question:
What about the great standalone genome browsers (IGV, Savant, GenomeBrowse, IGB, ...)
At least IGV and Savant are capable of displaying WIG files as well as BAM files from Alignments.

IGV: http://www.broadinstitute.org/software/igv/
Savant: http://www.savantbrowser.com/

As these are Java programs they work in Windows as well (IGV even comes with a BAT windows startscript, don't know for sure for Savant)
You can keep your data on your computer even without extensive memory.

If you need additional data from the UCSC Genome Browser you can download it via the Table Browser function as a bed file and display it within IGV or Savant as well.

That's the way I would probably go...

aer · 02-01-2011, 08:49 AM

Thanks for the list with alternative genome browsers! IGV looks very promising..

I have one more question to the wig to bigWig conversion: What is "chrom.sizes" or where can I found the needed information for this?
Thanks!

biznatch · 02-01-2011, 10:54 AM

Quote:

Originally Posted by aer

Hi biznatch,
my wig files are about 350MB to 750MB, depends on the chromosome. But the point is that I would like to upload to sets of data and this is always above the 500mb-limit of the UCSC browser. What´s the difference between fixedStep and variableStep wig file?
Is it possible to upload zipped data files??? I have to try..
By the way, I haven´t been successful in converting my wig files into bigwig.. I will have a look on Galaxy, thank you very much!

You can upload zipped files, but I've found that if the unzipped version of the file is over 500mb it still won't work. This is because the UCSC browser allocates a certain amount of time to process each uploaded file, and the time taken to unzip and process the file will take too long.

The UCSC browser help section describes the difference between fixed and variable step wig files. http://genome.ucsc.edu/goldenPath/help/wiggle.html

ulz_peter · 02-01-2011, 10:31 PM

Hi aer.

As faras as I know, the chrom.sizes is a text-based file having chromosome names in the first column and it's size in the second. I used the chromInfo.txt file from UCSC Genome browser for the bed to BigBed Conversion, it can be downloaded here (hg19 version):
http://hgdownload.cse.ucsc.edu/golde...romInfo.txt.gz

I don't remember if you need to delete the third column before using it with the conversion tools...

aer · 02-04-2011, 01:55 AM

Hi ulz_peter,
thank you very much! The BigBed conversion worked and the IGV tool was a very good advice. With IGV I can work with my BAM files and it works very well.
Thanks again to all!

Jim Robinson · 06-27-2011, 07:09 AM

Hi,

I realize this is an old thread, but the latest IGV release (2.0.3) now supports reading bigwig and bigbed files directly, either local or remote.

-- Jim

aer · 06-27-2011, 07:43 AM

Thank you very much for the advice!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
UCSC genome browser download	ashwatha	Bioinformatics	3	07-25-2011 08:58 PM
RNA-Seq: ENCODE whole-genome data in the UCSC genome browser (2011 update).	Newsbot!	Literature Watch	1	11-24-2010 02:08 PM
UCSC Genome Browser	ECHo	Illumina/Solexa	0	02-25-2010 08:21 PM
SeqGenomeBrowser : A mini genome browser for NGS data	knightfeng	Bioinformatics	7	02-12-2010 03:37 AM
Local UCSC genome browser server	baohua100	Bioinformatics	1	08-16-2009 04:10 PM

01-21-2011, 06:39 AM	#1
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Adding NGS data to the UCSC genome browser Hi everybody, I´m a phd student from Germany analyzing the functional mechanism of a transcription factor and trying to find out new target genes. We used ChIP-seq analysis to identify the new target genes. Now, I´m trying to work with and understand the NGS data: using the UCSC genome browser I would like to see the differences between my sample and the negative control. By adding new cumstom tracks I can upload one set of data in wig format (=one chromsome). Can anybody tell me how I am able to upload TWO sets of data in wig format to the UCSC genome browser so I can se the differences between the sample and control??? Thanks a lot!

01-27-2011, 12:08 PM	#11
biznatch Senior Member Location: Canada Join Date: Nov 2010 Posts: 124	How big is your wig file? Is it a variableStep or fixedStep wig? Unless you have very, very few data points, a fixedStep will be a smaller file, about half the size of variable in my experience, since the fixed step has 1 column of data and the variable step has 2. All my fixedStep .wig files (1 per chromosome), are maximum a few hundred mb, which is under the 500mb UCSC Browser limit (although even files close to but under the limit may not work, but I haven't had any problems). After generating my .wig files I compress them with gzip to .gz files, then each is about 5-15mb per file. They upload quickly, but it takes a minute or 2 for the UCSC genome browser to process them. Galaxy can convert wig to bigwig, I haven't tried it though. I don't know if it works for bigwig files, but for my wig and bed files I sometimes upload them to dropbox and use the sharing link to upload them to UCSC. Usually this is when someone else in my lab wants to view the files, so I can just send them the link. I've looked in to it very briefly, but setting up your own local UCSC genome browser looks like it really takes some know how...although I've only been using linux for 2 months so maybe it's not that bad. Anyways, one of the things I like about the Browser is all the other available tracks, which I guess you'd then have to download so that might negate some of the advantage. Last edited by biznatch; 01-27-2011 at 01:45 PM.

06-27-2011, 07:09 AM	#18
Jim Robinson Member Location: Boston, MA Join Date: May 2009 Posts: 75	IGV & Bigwig / bigbed Hi, I realize this is an old thread, but the latest IGV release (2.0.3) now supports reading bigwig and bigbed files directly, either local or remote. -- Jim

01-21-2011, 07:05 AM	#2
Gangcai Member Location: Shanghai, China Join Date: Nov 2009 Posts: 30	Just upload both of them to UCSC (submit one file each time), then you can see both of them on Genome Browser.

01-24-2011, 02:41 AM	#3
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Thank you! But I still have a problem: First, I uploaded the data for the X-chromosome for my sample. When finished the page "manage custom tracks" appears. I marked the data with an x and clicked on "add custom tracks" and submitted the data for the X-chromosome for my negative control. But after a while this error pops up: needMem: trying to allocate 528709463 bytes (limit: 500000000). What can I do?

01-24-2011, 03:15 AM	#4
ulz_peter Senior Member Location: Graz, Austria Join Date: Feb 2010 Posts: 219	Did you do the Wig to bigWig conversion. This is recommended for large datasets. A tool to convert the file can be downloaded here: http://hgdownload.cse.ucsc.edu/admin/exe/ hope that helps...

01-24-2011, 03:18 AM	#5
Gangcai Member Location: Shanghai, China Join Date: Nov 2009 Posts: 30	That means the wig file is too large to be submitted to UCSC. You can try to convert wig file to bigwig by the wigToBigWig program provided by UCSC. Hope it works.

01-24-2011, 03:44 AM	#6
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Thanks a lot! But these applications are only for Linux or MacOSX. Are there any conversion programs running on a Windows platform (32 Bit)???

01-24-2011, 04:21 AM	#7
ulz_peter Senior Member Location: Graz, Austria Join Date: Feb 2010 Posts: 219	You could download the jksrc.zip package which contains the source code of each of the utilities (http://hgdownload.cse.ucsc.edu/admin/jksrc.v245.zip) and compile them with a native Windows C-Compiler (actually I don't know any but that should be easy to find using Google). Other solution would be the installation of cygwin (http://www.cygwin.com/) and emulation of a linux terminal within windows. You should be able to start the bigTobigWig program within Cygwin. Hope that helps...

01-26-2011, 07:54 AM	#8
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Thank you! But unfortunately I wasn´t really successful.. I heard about installing the UCSC genome browser locally, then I should be able to handle my data in wig format I was told. Does anybody know something about the possibility to install the UCSC genome browser to you local pc??? Or how to do it??? Thanks a lot!

02-01-2011, 12:04 AM	#12
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Hi biznatch, my wig files are about 350MB to 750MB, depends on the chromosome. But the point is that I would like to upload to sets of data and this is always above the 500mb-limit of the UCSC browser. What´s the difference between fixedStep and variableStep wig file? Is it possible to upload zipped data files??? I have to try.. By the way, I haven´t been successful in converting my wig files into bigwig.. I will have a look on Galaxy, thank you very much!

02-01-2011, 02:35 AM	#13
ulz_peter Senior Member Location: Graz, Austria Join Date: Feb 2010 Posts: 219	Short question: What about the great standalone genome browsers (IGV, Savant, GenomeBrowse, IGB, ...) At least IGV and Savant are capable of displaying WIG files as well as BAM files from Alignments. IGV: http://www.broadinstitute.org/software/igv/ Savant: http://www.savantbrowser.com/ As these are Java programs they work in Windows as well (IGV even comes with a BAT windows startscript, don't know for sure for Savant) You can keep your data on your computer even without extensive memory. If you need additional data from the UCSC Genome Browser you can download it via the Table Browser function as a bed file and display it within IGV or Savant as well. That's the way I would probably go...

02-01-2011, 08:49 AM	#14
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Thanks for the list with alternative genome browsers! IGV looks very promising.. I have one more question to the wig to bigWig conversion: What is "chrom.sizes" or where can I found the needed information for this? Thanks!

02-01-2011, 10:31 PM	#16
ulz_peter Senior Member Location: Graz, Austria Join Date: Feb 2010 Posts: 219	Hi aer. As faras as I know, the chrom.sizes is a text-based file having chromosome names in the first column and it's size in the second. I used the chromInfo.txt file from UCSC Genome browser for the bed to BigBed Conversion, it can be downloaded here (hg19 version): http://hgdownload.cse.ucsc.edu/golde...romInfo.txt.gz I don't remember if you need to delete the third column before using it with the conversion tools...

02-04-2011, 01:55 AM	#17
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Hi ulz_peter, thank you very much! The BigBed conversion worked and the IGV tool was a very good advice. With IGV I can work with my BAM files and it works very well. Thanks again to all!

06-27-2011, 07:43 AM	#19
aer Junior Member Location: Germany Join Date: Jan 2011 Posts: 8	Thank you very much for the advice!