data format and how to analysis

bostonian · 06-19-2012, 08:48 PM

Hi,

I am a complete newbie, so please bear with me if my question seems too silly.

I recently downloaded a RNAseq dataset from GEO, it contains paired data files, and in the following format:

readID Seq 0-mi**** 1-mi**** 2-mi**** chr start end strand
HWUSI-EAS230-R:2:99:1151:1802#0/1 GAGCTCATTGGTGGCGTGGTGGCCTTGACCTTCCGG 1 0 0 chr10 70914936 70914971 -
HWUSI-EAS230-R:2:44:642:495#0/1 TTGGCTGCCTTCTGGGGTGAACTTTCTGCTATTTCC 0 0 1 chr7 47298110 47298145 -

......

I googled around but could not figure out what format it is, and how to proceed to analyze. My goal is to produce gene expression values (RPKM) from these data files.

Thanks so much for your help.

dpryan · 06-20-2012, 02:13 PM

It'd help if you mentioned the GEO accession number. They probably mention more about the format in the GEO entry. If nothing else, you could probably convert that to SAM/BAM format (the difficulty being whether or not the mappings are actually unique) and then calculate the RPKM. Alternatively, if they provide no further information (or the original fastq file), you could convert that to a multi-fasta file and then realign it.

bostonian · 06-22-2012, 06:40 AM

Hi, Thanks a lot for your response. The dataset is GSE22260. I do not see any mentioning about the data formet, but maybe it is only because I do not know where to look. Could you also advise how to convert what I have to SAM/BAM format? or how to convert to multi-fasta files? I am fairly familiar with R, so I am hoping I will be able figure the later analysis after I have data in BAM format. Thanks again.

westerman · 06-22-2012, 08:01 AM

I believe that these are output from "Illumina Genome Analyzer Pipeline". I am not sure if there is a converter from this format to SAM/BAM. Multi-fastA would be trivial. Something like:

cut -f 1,2 file.name | sed -e 's/^/>/' -e 's/\t/\n' > output.file

or

perl -nale 'print ">$F[0]\n$F[1]"' file.name > output.file

Then you could map the reads against a current human genome and go from there.

bostonian · 06-22-2012, 08:40 AM

Thank, Rick. I am trying to avoid doing alignment myself. Since each sequence comes with chromosome number and genomic coordinates in the data file, I assume they are already aligned (?). Is there any tool that I can use to convert the data into some format so that I can read them into some R packages, such as ShortRead or others (?), for further analysis?

dpryan · 06-22-2012, 08:53 AM

As Rick suggested and they state in the data processing section of each sample, the files are from the Genome Analyzer Pipeline (there are two files for each sample, since this is paired-end data). If you were willing to redo the alignment, you can download the reads from SRA and convert them to fastq with fastq-dump. Alternatively, it wouldn't be difficult to write a small program in perl/python/R/whatever to read in both read files at once and output to SAM. Just make sure that corresponding lines in each file are actually meant to go together (that is, ensure that read names match), as if they kept reads where only one end maps then you would otherwise get really screwy results. So, just put a * in the base quality field and make up an appropriate fake MAPQ value along with the correct flag and you're good to go.

westerman · 06-22-2012, 10:22 AM

@bostonian

I do not use ShortRead however reading the manual ShortRead uses the readAligned part of bioconductor. I am not sure that readAligned will read in your file directly however the fields are similar and I suspect that with some simple data manipulation you could get them read in.

BTW: It is this type of data manipulation that makes bioinformatics such a joy. :-)

rboettcher · 10-18-2013, 04:36 AM

@bostonian

I am currently struggling with the same issue, did you find a solution for your problem?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
ChIP-Seq: Enabling Data Analysis on High-Throughput Data in Large Data Depository Usi	Newsbot!	Literature Watch	1	04-18-2018 10:50 PM
problem on file format in ChIP-Seq data analysis	sp_wade	Bioinformatics	5	03-08-2012 06:43 AM
[NGS - analysis of gene expression data] Machine Learning + RNAseq data	Chuckytah	Bioinformatics	7	03-05-2012 04:16 AM
RNA-seq analysis raw data format	neetu	Bioinformatics	8	10-14-2011 03:23 AM
Looking process to convert gff3 format into ace format or sam format	andylai	Bioinformatics	1	05-17-2011 03:09 AM

06-19-2012, 08:48 PM	#1
bostonian Junior Member Location: Boston Join Date: Jun 2012 Posts: 4	data format and how to analysis Hi, I am a complete newbie, so please bear with me if my question seems too silly. I recently downloaded a RNAseq dataset from GEO, it contains paired data files, and in the following format: readID Seq 0-mi** 1-mi 2-mi** chr start end strand HWUSI-EAS230-R:2:99:1151:1802#0/1 GAGCTCATTGGTGGCGTGGTGGCCTTGACCTTCCGG 1 0 0 chr10 70914936 70914971 - HWUSI-EAS230-R:2:44:642:495#0/1 TTGGCTGCCTTCTGGGGTGAACTTTCTGCTATTTCC 0 0 1 chr7 47298110 47298145 - ...... I googled around but could not figure out what format it is, and how to proceed to analyze. My goal is to produce gene expression values (RPKM) from these data files. Thanks so much for your help.

06-20-2012, 02:13 PM	#2
dpryan Devon Ryan Location: Freiburg, Germany Join Date: Jul 2011 Posts: 3,480	It'd help if you mentioned the GEO accession number. They probably mention more about the format in the GEO entry. If nothing else, you could probably convert that to SAM/BAM format (the difficulty being whether or not the mappings are actually unique) and then calculate the RPKM. Alternatively, if they provide no further information (or the original fastq file), you could convert that to a multi-fasta file and then realign it.

06-22-2012, 06:40 AM	#3
bostonian Junior Member Location: Boston Join Date: Jun 2012 Posts: 4	Hi, Thanks a lot for your response. The dataset is GSE22260. I do not see any mentioning about the data formet, but maybe it is only because I do not know where to look. Could you also advise how to convert what I have to SAM/BAM format? or how to convert to multi-fasta files? I am fairly familiar with R, so I am hoping I will be able figure the later analysis after I have data in BAM format. Thanks again.

06-22-2012, 08:01 AM	#4
westerman Rick Westerman Location: Purdue University, Indiana, USA Join Date: Jun 2008 Posts: 1,104	I believe that these are output from "Illumina Genome Analyzer Pipeline". I am not sure if there is a converter from this format to SAM/BAM. Multi-fastA would be trivial. Something like: cut -f 1,2 file.name \| sed -e 's/^/>/' -e 's/\t/\n' > output.file or perl -nale 'print ">$F[0]\n$F[1]"' file.name > output.file Then you could map the reads against a current human genome and go from there.

06-22-2012, 08:40 AM	#5
bostonian Junior Member Location: Boston Join Date: Jun 2012 Posts: 4	Thank, Rick. I am trying to avoid doing alignment myself. Since each sequence comes with chromosome number and genomic coordinates in the data file, I assume they are already aligned (?). Is there any tool that I can use to convert the data into some format so that I can read them into some R packages, such as ShortRead or others (?), for further analysis?

06-22-2012, 08:53 AM	#6
dpryan Devon Ryan Location: Freiburg, Germany Join Date: Jul 2011 Posts: 3,480	As Rick suggested and they state in the data processing section of each sample, the files are from the Genome Analyzer Pipeline (there are two files for each sample, since this is paired-end data). If you were willing to redo the alignment, you can download the reads from SRA and convert them to fastq with fastq-dump. Alternatively, it wouldn't be difficult to write a small program in perl/python/R/whatever to read in both read files at once and output to SAM. Just make sure that corresponding lines in each file are actually meant to go together (that is, ensure that read names match), as if they kept reads where only one end maps then you would otherwise get really screwy results. So, just put a * in the base quality field and make up an appropriate fake MAPQ value along with the correct flag and you're good to go.

06-22-2012, 10:22 AM	#7
westerman Rick Westerman Location: Purdue University, Indiana, USA Join Date: Jun 2008 Posts: 1,104	@bostonian I do not use ShortRead however reading the manual ShortRead uses the readAligned part of bioconductor. I am not sure that readAligned will read in your file directly however the fields are similar and I suspect that with some simple data manipulation you could get them read in. BTW: It is this type of data manipulation that makes bioinformatics such a joy. :-)

10-18-2013, 04:36 AM	#8
rboettcher Member Location: Berlin Join Date: Oct 2010 Posts: 71	@bostonian I am currently struggling with the same issue, did you find a solution for your problem?