SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
small part of all 1000 sequences from 1000genomes data needed? culmen Bioinformatics 11 11-07-2012 11:23 AM
1000Genomes YRI trio, NCBI36/37, rs334 raonyguimaraes Bioinformatics 1 12-14-2011 06:28 AM
VCFtools Vcf.pm problem - broken VCF header on 1000genomes data naumenko.sa Bioinformatics 1 07-08-2011 05:17 AM
Get many Bam Files from 1000genomes dongshong Bioinformatics 0 02-20-2011 08:55 PM
Illumina sequencing data from 1000genomes thsuk1 Illumina/Solexa 1 09-08-2010 03:48 AM

Reply
 
Thread Tools
Old 09-04-2011, 10:47 PM   #1
giror
Junior Member
 
Location: St. Louis

Join Date: Jun 2011
Posts: 6
Default 1000genomes Accessibility

Hi all,
I am interested in a particular genomic region that I would like to retrieve from all 1103 mapped genomes that are available today. I used this line in a python script to extract my region:

os.system('samtools view -b ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/{0} 2:69,130,000-69,160,000 > {1}'.format(sample, fn))

It works about 20% of the time, but other times I get this kind of responses:
[kftp_connect_file] 331 Anonymous login ok, send your complete email address as your password.
[main_samview] fail to open "ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/data/HG00611/alignment/HG00611.mapped.ILLUMINA.bwa.CHS.low_coverage.20101123.bam" for reading.

OR
[kftp_connect_file] 227 Entering Passive Mode (130,14,250,10,195,247).
[main_samview] fail to open "ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/data/HG00628/alignment/HG00628.mapped.ILLUMINA.bwa.CHS.low_coverage.20101123.bam" for reading.

OR
[kftp_connect_file] 331 Anonymous login ok, send your complete email address as your password.
[bam_header_read] EOF marker is absent. The input is probably truncated.
[bam_header_read] invalid BAM binary header (this is not a BAM file).
[main_samview] fail to read the header from "ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/data/HG00614/alignment/HG00614.mapped.ILLUMINA.bwa.CHS.low_coverage.20101123.bam".

Has anyone previously encountered these types of errors? Are there any easy fixes?

Thanks in advance for any help!
-giror

p.s. I have also tried the new 1000genomes slicer tool for some of these bam files only to get a 404 error.
giror is offline   Reply With Quote
Old 09-05-2011, 10:09 AM   #2
Jim Robinson
Member
 
Location: Boston, MA

Join Date: May 2009
Posts: 75
Default

I had similiar errors with the ncbi ftp servers, they don't seem to like getting multiple "REST" commands in quick succession. In my case I was doing this from java and inserted a delay after closing the stream for one query, and sending the "REST" command for the next. Unless you want to try patching samtools to do the same you might try forcing a ~3 second delay between queries for successive regions.

You might also want to post this question on the samtools-help email list.

Jim
Jim Robinson is offline   Reply With Quote
Old 09-05-2011, 09:18 PM   #3
giror
Junior Member
 
Location: St. Louis

Join Date: Jun 2011
Posts: 6
Default

Thanks a lot! That worked perfectly.
giror is offline   Reply With Quote
Old 09-06-2011, 03:40 AM   #4
laura
Senior Member
 
Location: Cambridge UK

Join Date: Sep 2008
Posts: 151
Default

As a note for the data slicer unfortunately due to the vagueries of the EBI firewall it only works for ftp sites within the EBI but it will work for http anywhere

thanks
laura is offline   Reply With Quote
Reply

Tags
1000genomes, samtools

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 07:08 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2021, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO