upload > 30 GB file on Galaxy

[ericguo]

Hi all,

Has anyone successfully uploaded files > 30 GB onto Galaxy? I tried it a few times using ftp commands on a unix cluster. The first two times, ftp stopped at 12 GB. The third time it stopped at 28 GB.

It gives me an error message: "broken pipe". I am not sure its the problem of the cluster that I am using or Galaxy.

I was think to break my file into smaller files, then upload them one by one. Then join them on Galaxy. Does anyone know what program on Galaxy merges several fastq files into a single one?

Thanks in advance for the help!

-Eric

[maubp]

I was think to break my file into smaller files, then upload them one by one. Then join them on Galaxy. Does anyone know what program on Galaxy merges several fastq files into a single one?

The generic text file concatenation tool in Galaxy can be used to combined multiple FASTQ files.

[SES]

It gives me an error message: "broken pipe". I am not sure its the problem of the cluster that I am using or Galaxy.

This means your connection was broken and could be do to a firewall or something with your configuration. You could try ftp in passive mode and see if that works (the command is "pftp"). Otherwise, I would try to split the files and upload them. You can then use the "-i" flag with ftp to upload them all without prompting.

[KevinLam]

U can consider running alightment of split fastq files before merging bam files .. If upload is slow

[yumtaoist]

It gives me an error message: "broken pipe".

"broken pipe" means the network is disconnected. This is very common after a long time...

[ericguo]

thanks guys.

[deepthi]

Hi Guys,
I just installed galaxy but I could not upload any files using Get Data->Upload files from computer . The error showing is

An error occurred running this job: Traceback (most recent call last):
File "/export/home/deepthi/galaxy-dist/tools/data_source/upload.py", line 10, in ?
import galaxy.model
File "/export/home/deepthi/galaxy-dist/lib/galaxy/model/__init__.py", line 13, in ?
import galaxy.datatyp

Is it something problem with the python eggs?