SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics
Similar Threads
Thread Thread Starter Forum Replies Last Post
Free & Open Environment for RNA-seq analysis: Galaxy (http://usegalaxy.org) jgoecks RNA Sequencing 27 11-07-2016 12:02 AM
NAR: PileLineGUI a desktop environment for handling NGS genome position files. gonvader Literature Watch 0 06-09-2011 02:59 AM
Free Partek RNA-Seq & ChIP-Seq Data Analysis Workshops mgabrielli@partek.com Events / Conferences 0 04-05-2011 07:28 AM
Free Avadis NGS webinar on 23 February 2011 Strand SI Events / Conferences 4 03-15-2011 01:54 AM
ChIP-Seq: The Poisson Margin Test for Normalization-Free Significance Analysis of NGS Newsbot! Literature Watch 0 03-10-2011 02:00 AM

Reply
 
Thread Tools
Old 03-26-2010, 12:39 PM   #21
nekrut
Member
 
Location: Penn State

Join Date: Apr 2009
Posts: 22
Default

Quote:
Originally Posted by tweist View Post
This is a great platform having many useful tools available from the same place and interoperable. However, uploading big dataset to Galaxy is a challenge and a road block. I tried to upload a ILMN dataset of a few Gb and not yet done after a day and half. Any plan to enable Aspera-like tools to accelerate the data transfer speed?

Also, I think a blast tool that allows search against arbitrary database would be useful too. Any plan to add this feature?

Great job!
Hi, upload limitations are going to be a problem for all of us as it is not Galaxy issue per se. To upload your files try this:
- gzip them
- put them to a web-accessible loocation
- paste a URL pointing to your files to upload tool as shown in the attached image

Aspera is incredibly expensive to license, and we're not convinced it really accelerates things that much. There are some open source alternative we will be taking a look at

Yes, blasting against custom db is on the to do list as numerous people want this to happen.
Attached Images
File Type: jpg Picture 5.jpg (20.3 KB, 58 views)
nekrut is offline   Reply With Quote
Old 03-31-2010, 09:34 AM   #22
tweist
Junior Member
 
Location: boston, usa

Join Date: Aug 2008
Posts: 3
Default

Hi,

Have you looked at open source alternative such as UDT to address the data transfer bottleneck? Also, is there a limit on how much data that I can save in my galaxy account? Thanks!
tweist is offline   Reply With Quote
Old 04-01-2010, 04:57 AM   #23
nekrut
Member
 
Location: Penn State

Join Date: Apr 2009
Posts: 22
Default

Quote:
Originally Posted by tweist View Post
Hi,
Also, is there a limit on how much data that I can save in my galaxy account? Thanks!
No limit as of now
nekrut is offline   Reply With Quote
Old 04-08-2010, 11:55 PM   #24
ezilybored
Junior Member
 
Location: Cardiff

Join Date: Dec 2009
Posts: 9
Default

Quote:
Originally Posted by nekrut View Post
Hi, upload limitations are going to be a problem for all of us as it is not Galaxy issue per se. To upload your files try this:
- gzip them
- put them to a web-accessible loocation
- paste a URL pointing to your files to upload tool as shown in the attached image

Aspera is incredibly expensive to license, and we're not convinced it really accelerates things that much. There are some open source alternative we will be taking a look at

Yes, blasting against custom db is on the to do list as numerous people want this to happen.
Hi there, sorry if i am being naive but will galaxy be able to read the files as gzips? a colleague of mine tried recently and said that it wasnt able to read the file format.

Ben
ezilybored is offline   Reply With Quote
Old 04-09-2010, 12:14 AM   #25
KevinLam
Senior Member
 
Location: SEA

Join Date: Nov 2009
Posts: 203
Default

I have uploaded a gzipped 300 MB .qual file and it works for doing box plot. perhaps your fren didn't complete the upload?
KevinLam is offline   Reply With Quote
Old 04-09-2010, 04:16 AM   #26
nekrut
Member
 
Location: Penn State

Join Date: Apr 2009
Posts: 22
Default

Quote:
Originally Posted by ezilybored View Post
Hi there, sorry if i am being naive but will galaxy be able to read the files as gzips? a colleague of mine tried recently and said that it wasnt able to read the file format.

Ben
Yes, you can. The best way to get data in is to upload gzipped files via URL (http and ftp both work). See post #21 in the thread (scroll up).
nekrut is offline   Reply With Quote
Old 04-13-2010, 02:12 AM   #27
James
Member
 
Location: Cardiff

Join Date: Mar 2010
Posts: 23
Default

I'm having a similar problem to Ben, I have illumina fastq files that are gzipped. I can upload them in a few minutes but then when I try to do the groomer i get this error:

An error occurred running this job: Traceback (most recent call last):
File "/galaxy/home/g2main/galaxy_main/tools/fastq/fastq_groomer.py", line 37, in <module>
if __name__ == "__main__": main()
File "/galaxy/home/g2main/galaxy_main/tools/fastq/fastq_groomer.py", line 18, in main

Is there some sort of facility to unzip gz files? Thanks, J
James is offline   Reply With Quote
Old 04-13-2010, 06:52 AM   #28
blankenberg
Junior Member
 
Location: Penn State

Join Date: Mar 2010
Posts: 4
Default

Quote:
Originally Posted by ezilybored View Post
Hi there, sorry if i am being naive but will galaxy be able to read the files as gzips? a colleague of mine tried recently and said that it wasnt able to read the file format.

Ben

Quote:
Originally Posted by James View Post
I'm having a similar problem to Ben, I have illumina fastq files that are gzipped. I can upload them in a few minutes but then when I try to do the groomer i get this error:

An error occurred running this job: Traceback (most recent call last):
File "/galaxy/home/g2main/galaxy_main/tools/fastq/fastq_groomer.py", line 37, in <module>
if __name__ == "__main__": main()
File "/galaxy/home/g2main/galaxy_main/tools/fastq/fastq_groomer.py", line 18, in main

Is there some sort of facility to unzip gz files? Thanks, J
Galaxy does support the upload of gzip'd files.

Galaxy does not yet accept .tar(.gz) nor .zip files. Each file needs to be gzip'd individually (without tar'ing) in order for it it work with the upload tool at this time.

Support for .tar(.gz) and .zip files is being investigated.

Please don't hesitate to let us know if we can be of further help. Thanks for using Galaxy.
blankenberg is offline   Reply With Quote
Old 04-14-2010, 10:23 AM   #29
Calico
Member
 
Location: Houston

Join Date: Jan 2010
Posts: 13
Default

Is there any way that Galaxy can be used to analyze data from a Helicos machine?
Calico is offline   Reply With Quote
Old 04-14-2010, 10:42 AM   #30
nekrut
Member
 
Location: Penn State

Join Date: Apr 2009
Posts: 22
Default

Quote:
Originally Posted by Calico View Post
Is there any way that Galaxy can be used to analyze data from a Helicos machine?
We have not see the data ourselves yet, but I just asked Helicos guys to send us a few files. Do you have any to share?
nekrut is offline   Reply With Quote
Old 04-14-2010, 01:19 PM   #31
Calico
Member
 
Location: Houston

Join Date: Jan 2010
Posts: 13
Default

Not yet, but I am about to subject some samples to Helicos sequencing.
Calico is offline   Reply With Quote
Old 04-15-2010, 02:17 AM   #32
steven
Senior Member
 
Location: Southern France

Join Date: Aug 2009
Posts: 269
Default

Quote:
Originally Posted by nekrut View Post
We have not see the data ourselves yet, but I just asked Helicos guys to send us a few files. Do you have any to share?
I do not know about the raw output from the Heliscope, but what i have seen was fasta format for the reads and BED format for the alignments. Some Helicos datasets are available there: http://open.helicosbio.com/mwiki/index.php/Datasets (a free registration is maybe required though). Actually, Helicos RNA-seq data can also be obtained from the UCSC Table Browser and sent to Galaxy, or from the UCSC ftp download page.
steven is offline   Reply With Quote
Old 04-15-2010, 09:05 AM   #33
ohofmann
Member
 
Location: Melbourne, Australia

Join Date: Jan 2009
Posts: 37
Default

Problem with Helicos is that it doesn't come as FASTQ (quality scores are assigned to reads, not nucleotides within the reads). The best (only?) way to handle this is to convert the original format to FASTA after filtering at a given quality cutoff and take it from there. There are also tools to create 'fake' FASTQ files by setting the quality score to a uniform value if a tool absolutely requires FASTQ.

You almost need to use their filtering workflow anyway as it gets rid of a number of artifacts which are otherwise difficult to remove.
ohofmann is offline   Reply With Quote
Old 04-21-2010, 09:15 AM   #34
brian.g
Junior Member
 
Location: Chicago

Join Date: Apr 2010
Posts: 1
Default

UDT (udt.sf.net) should work. It is a library with C++/JNI/.net API and only has a simple command line tool. You can also look at HSCP (http://sourceforge.net/projects/hscp/), which is a SCP-like tool but uses UDT for data transfer. Both UDT and HSCP are open source and free to use.


Quote:
Originally Posted by nekrut View Post
Hi, upload limitations are going to be a problem for all of us as it is not Galaxy issue per se. To upload your files try this:
- gzip them
- put them to a web-accessible loocation
- paste a URL pointing to your files to upload tool as shown in the attached image

Aspera is incredibly expensive to license, and we're not convinced it really accelerates things that much. There are some open source alternative we will be taking a look at

Yes, blasting against custom db is on the to do list as numerous people want this to happen.
brian.g is offline   Reply With Quote
Old 04-22-2010, 09:06 AM   #35
nekrut
Member
 
Location: Penn State

Join Date: Apr 2009
Posts: 22
Default

Quote:
Originally Posted by brian.g View Post
UDT (udt.sf.net) should work. It is a library with C++/JNI/.net API and only has a simple command line tool. You can also look at HSCP (http://sourceforge.net/projects/hscp/), which is a SCP-like tool but uses UDT for data transfer. Both UDT and HSCP are open source and free to use.
Just talked to Bob Grossman. Will try...
nekrut is offline   Reply With Quote
Old 05-05-2010, 10:16 PM   #36
nilshomer
Nils Homer
 
nilshomer's Avatar
 
Location: Boston, MA, USA

Join Date: Nov 2008
Posts: 1,285
Default

I have been playing with Galaxy on my local machine for the past few days. It is great piece of software, and was relatively easy to plug in new tools. Thank-you again.
nilshomer is offline   Reply With Quote
Old 05-06-2010, 04:33 AM   #37
nekrut
Member
 
Location: Penn State

Join Date: Apr 2009
Posts: 22
Default

Nils:

Thanks for sending us your wrappers. They will be on public site shortly!
nekrut is offline   Reply With Quote
Old 01-24-2022, 07:50 PM   #38
Abigail41
Junior Member
 
Location: usa

Join Date: Jan 2022
Posts: 2
Default

If there is anything the site or community can do to support this effort, don't hesitate
Abigail41 is offline   Reply With Quote
Old 01-25-2022, 09:54 PM   #39
Abigail41
Junior Member
 
Location: usa

Join Date: Jan 2022
Posts: 2
Default

I think it is important to include bfast as perm/bowtie do not do gapped alignment.
Abigail41 is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -8. The time now is 05:20 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2022, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO