SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
FTP file upload to Galaxy via SFTP? jjw14 Bioinformatics 10 10-27-2013 07:26 AM
Upload .bam file to Galaxy, display on UCSC biznatch Bioinformatics 3 12-20-2011 04:12 PM
upload error in local instance of galaxy Ami Bioinformatics 2 10-19-2011 09:53 AM
FTP upload in galaxy neetu Bioinformatics 1 10-18-2011 06:17 AM
Upload genome index to Galaxy for Bowtie alignment? jjw14 Bioinformatics 0 06-08-2010 09:22 AM

Reply
 
Thread Tools
Old 12-28-2011, 01:09 PM   #1
ericguo
Junior Member
 
Location: new haven

Join Date: Sep 2011
Posts: 9
Default upload > 30 GB file on Galaxy

Hi all,

Has anyone successfully uploaded files > 30 GB onto Galaxy? I tried it a few times using ftp commands on a unix cluster. The first two times, ftp stopped at 12 GB. The third time it stopped at 28 GB.

It gives me an error message: "broken pipe". I am not sure its the problem of the cluster that I am using or Galaxy.

I was think to break my file into smaller files, then upload them one by one. Then join them on Galaxy. Does anyone know what program on Galaxy merges several fastq files into a single one?

Thanks in advance for the help!

-Eric
ericguo is offline   Reply With Quote
Old 12-28-2011, 03:57 PM   #2
maubp
Peter (Biopython etc)
 
Location: Dundee, Scotland, UK

Join Date: Jul 2009
Posts: 1,541
Default

Quote:
Originally Posted by ericguo View Post
I was think to break my file into smaller files, then upload them one by one. Then join them on Galaxy. Does anyone know what program on Galaxy merges several fastq files into a single one?
The generic text file concatenation tool in Galaxy can be used to combined multiple FASTQ files.
maubp is offline   Reply With Quote
Old 12-29-2011, 08:01 AM   #3
SES
Senior Member
 
Location: Vancouver, BC

Join Date: Mar 2010
Posts: 274
Default

Quote:
Originally Posted by ericguo View Post
It gives me an error message: "broken pipe". I am not sure its the problem of the cluster that I am using or Galaxy.
This means your connection was broken and could be do to a firewall or something with your configuration. You could try ftp in passive mode and see if that works (the command is "pftp"). Otherwise, I would try to split the files and upload them. You can then use the "-i" flag with ftp to upload them all without prompting.
SES is offline   Reply With Quote
Old 01-03-2012, 07:07 AM   #4
KevinLam
Senior Member
 
Location: SEA

Join Date: Nov 2009
Posts: 197
Default

U can consider running alightment of split fastq files before merging bam files .. If upload is slow
KevinLam is offline   Reply With Quote
Old 01-03-2012, 07:20 AM   #5
yumtaoist
Member
 
Location: lanzhou

Join Date: Dec 2011
Posts: 10
Default

Quote:
Originally Posted by ericguo View Post
It gives me an error message: "broken pipe".
"broken pipe" means the network is disconnected. This is very common after a long time...
yumtaoist is offline   Reply With Quote
Old 01-03-2012, 11:35 AM   #6
ericguo
Junior Member
 
Location: new haven

Join Date: Sep 2011
Posts: 9
Default

thanks guys.
ericguo is offline   Reply With Quote
Old 03-20-2012, 03:55 PM   #7
deepthi
Junior Member
 
Location: canada

Join Date: Mar 2012
Posts: 3
Default

Hi Guys,
I just installed galaxy but I could not upload any files using Get Data->Upload files from computer . The error showing is

An error occurred running this job: Traceback (most recent call last):
File "/export/home/deepthi/galaxy-dist/tools/data_source/upload.py", line 10, in ?
import galaxy.model
File "/export/home/deepthi/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in ?
import galaxy.datatyp

Is it something problem with the python eggs?
deepthi is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 09:46 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO