SEQanswers

Go Back   SEQanswers > Sequencing Technologies/Companies > Illumina/Solexa



Similar Threads
Thread Thread Starter Forum Replies Last Post
Problems with Bio-linux/Ubuntu 14.04 Kimist99 Bioinformatics 16 04-24-2015 04:25 AM
Same experiment name = unable to download from Basespace? wlchew Illumina/Solexa 3 04-24-2015 03:16 AM
Problems using wgsim in Linux/Ubuntu Kimist99 Bioinformatics 12 11-11-2013 06:30 PM
BaseSpace for MiSeq epistatic Illumina/Solexa 5 11-27-2012 04:48 PM
Core Cluster Setup - Linux, Ubuntu, Rocks, Data Storage, BluArc quantrix Bioinformatics 16 10-05-2012 09:46 AM

Reply
 
Thread Tools
Old 04-24-2015, 03:22 AM   #1
cement_head
Senior Member
 
Location: Oxford, Ohio

Join Date: Mar 2012
Posts: 253
Default Download BaseSpace runs from an Ubuntu (Linux) box

Hello,

Am I correct in assuming that in order to download data files (FastQ) from BaseSpace, I have to manually run the python downloader script using my run ID and a token????????

In other words, you'd have to be running WinBlows to have this downloaded automagically?

- Andor
cement_head is offline   Reply With Quote
Old 04-24-2015, 03:33 AM   #2
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

http://seqanswers.com/forums/showthread.php?t=47633

Companies do not have an obligation to support all OS's. Illumina seems to have concentrated their user facing development on Windows.
GenoMax is offline   Reply With Quote
Old 04-24-2015, 03:46 AM   #3
cement_head
Senior Member
 
Location: Oxford, Ohio

Join Date: Mar 2012
Posts: 253
Default

Quote:
Originally Posted by GenoMax View Post
http://seqanswers.com/forums/showthread.php?t=47633

Companies do not have an obligation to support all OS's. Illumina seems to have concentrated their user facing development on Windows.
I don't think this a matter of "supporting all OS's". When the overwhelming amount of software for bioinformatics is Linux based, that should be the primarily supported OS, not other operating systems. Many companies provide the ability to download a file securely regardless of OS. In other words, without having to install an OS (*.exe) specific downloader. For example, the downloader could have been JAVA based.
cement_head is offline   Reply With Quote
Old 04-24-2015, 04:33 AM   #4
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

I don't use basespace so I don't know about the "automagical download" you are referring to (is that a separate application?). Looking at this link from the thread posted above Illumina seems to be treating Windows/OS X/Linux equally: https://support.basespace.illumina.c...run-downloader
GenoMax is offline   Reply With Quote
Old 04-24-2015, 06:51 AM   #5
cliffbeall
Senior Member
 
Location: Ohio

Join Date: Jan 2010
Posts: 144
Default

Apparently this is a new thing? It seemed to work direct through the browser on Linux about 6 months ago.

There is an OSX downloader but it wasn't Snow Leopard compatible which left out my slightly older Mac Pro. At that point I looked at the python thing but decided it looked too complicated and it was easier to find a different machine.

There is a blog post from April 11th talking about "New Download Experience" and asking for feedback so I left a comment asking for a Linux port. Maybe if enough other people do the same?
cliffbeall is offline   Reply With Quote
Old 04-24-2015, 07:54 AM   #6
cement_head
Senior Member
 
Location: Oxford, Ohio

Join Date: Mar 2012
Posts: 253
Default

Quote:
Originally Posted by cliffbeall View Post
Apparently this is a new thing? It seemed to work direct through the browser on Linux about 6 months ago.

There is an OSX downloader but it wasn't Snow Leopard compatible which left out my slightly older Mac Pro. At that point I looked at the python thing but decided it looked too complicated and it was easier to find a different machine.

There is a blog post from April 11th talking about "New Download Experience" and asking for feedback so I left a comment asking for a Linux port. Maybe if enough other people do the same?
Yes, you are correct - it used to work on Linux from within the browser. I can't login to their stupid website
cement_head is offline   Reply With Quote
Old 04-24-2015, 08:03 AM   #7
cement_head
Senior Member
 
Location: Oxford, Ohio

Join Date: Mar 2012
Posts: 253
Default

Quote:
Originally Posted by GenoMax View Post
I don't use basespace so I don't know about the "automagical download" you are referring to (is that a separate application?). Looking at this link from the thread posted above Illumina seems to be treating Windows/OS X/Linux equally: https://support.basespace.illumina.c...run-downloader
Yeah, problem is, that downloads EVERYTHING - image files, everything!
cement_head is offline   Reply With Quote
Old 04-24-2015, 08:08 AM   #8
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

If you only need the fastq files why aren't you grabbing them directly from the MiSeq (if you have direct access to MiSeq of course)?

They should be in the D:\Illumina\MiSeq Analysis\Flowcell ID\Data\Intensities\BaseCalls directory (if I am remembering path right). Apologies in advance, if use of BaseSpace somehow turns local analysis off.
GenoMax is offline   Reply With Quote
Old 04-24-2015, 05:38 PM   #9
kcchan
Senior Member
 
Location: USA

Join Date: Jul 2012
Posts: 184
Default

It's still possible to download files without their download program. You will need to go into each file and download them one by one.
kcchan is offline   Reply With Quote
Old 04-26-2015, 12:31 PM   #10
cement_head
Senior Member
 
Location: Oxford, Ohio

Join Date: Mar 2012
Posts: 253
Default

Quote:
Originally Posted by GenoMax View Post
If you only need the fastq files why aren't you grabbing them directly from the MiSeq (if you have direct access to MiSeq of course)?

They should be in the D:\Illumina\MiSeq Analysis\Flowcell ID\Data\Intensities\BaseCalls directory (if I am remembering path right). Apologies in advance, if use of BaseSpace somehow turns local analysis off.
No, you're right - but some of my collaborators are 800 miles away
cement_head is offline   Reply With Quote
Old 04-26-2015, 12:36 PM   #11
cement_head
Senior Member
 
Location: Oxford, Ohio

Join Date: Mar 2012
Posts: 253
Default

Quote:
Originally Posted by kcchan View Post
It's still possible to download files without their download program. You will need to go into each file and download them one by one.
So, I tried the python script and it started downloading EVERYTHING (all images files, etc). Is there a way to get the python script to pull only the FASTQ files?
cement_head is offline   Reply With Quote
Old 04-26-2015, 10:17 PM   #12
kcchan
Senior Member
 
Location: USA

Join Date: Jul 2012
Posts: 184
Default

You have to go into the Sample section (tube icon on the left sidebar) and click on the file name for each file (and not the checkbox). It will give you the error message "File is either not text-based or too large for viewing, please download the raw file." but also contain a download button to get the file via a normal download. It'll get pretty tedious if you have a lot of samples.
kcchan is offline   Reply With Quote
Old 04-27-2015, 07:45 AM   #13
cliffbeall
Senior Member
 
Location: Ohio

Join Date: Jan 2010
Posts: 144
Default

This appeared on the Illumina blog post I linked to earlier, with a video of their upcoming tool:
Update (04/24/2015): We have received feedback regarding the inability of using the new downloader on Linux machines. At the BaseSpace WWDC today at the Broad Institute we announced that a new feature called BaseMount will be released soon. BaseMount enables you to mount your BaseSpace data as an Linux read-only FUSE-based virtual file system. You can do normal file system operations and can download files from a command line. We have a detailed demo below.
Hopefully this will solve peoples's issues.
cliffbeall is offline   Reply With Quote
Old 05-08-2015, 11:51 AM   #14
eco32i
Member
 
Location: NYC

Join Date: Jul 2012
Posts: 10
Default

An ugly workaround we use for this (with NextSeq) is to mount a target directory on a linux box as a Samba share on NextSeq and use the Windows download tool from NextSeq to save .fastq files to that share.

But yes, it is annoying.
eco32i is offline   Reply With Quote
Old 07-23-2015, 08:01 PM   #15
greghroberts
Junior Member
 
Location: San Diego

Join Date: Jul 2015
Posts: 1
Default BaseMount is now available

BaseSpace team just released early access to BaseMount. It allows you to mount your entire account as a filesystem.

More info at:
https://basemount.basespace.illumina.com
greghroberts is offline   Reply With Quote
Old 07-24-2015, 06:01 AM   #16
eco32i
Member
 
Location: NYC

Join Date: Jul 2012
Posts: 10
Default

That's great news.
From my (very brief) test it works fine.
eco32i is offline   Reply With Quote
Old 08-04-2015, 10:57 AM   #17
jdv
Junior Member
 
Location: Wisconsin, USA

Join Date: Dec 2014
Posts: 3
Default

Another cross-platform command-line alternative is my bsclient software:

http://sourceforge.net/projects/bsfetch/

It was written prior to the existence of BaseMount when I wanted an interactive tool for headless server downloads, a niche which BaseMount also fills. One advantage is that not all users will have mount permissions on the server they're using - a requirement for BaseMount but not for bsclient.
jdv is offline   Reply With Quote
Old 08-06-2015, 12:23 PM   #18
cement_head
Senior Member
 
Location: Oxford, Ohio

Join Date: Mar 2012
Posts: 253
Default

BaseMount is now regular access:

http://blog.basespace.illumina.com/2...for-basespace/
cement_head is offline   Reply With Quote
Old 05-29-2017, 10:01 AM   #19
hr3y
Junior Member
 
Location: MX

Join Date: May 2017
Posts: 7
Unhappy basemount is weird.

Hi everyone, I was wondering if anyone has experience with mounting their BaseSpace account and regularly using it (reading, copying from it, etc). I was just trying to change the permissions of the directory where I mounted it and I can't.... even though I am the server's admin.
hr3y is offline   Reply With Quote
Old 05-30-2017, 06:41 AM   #20
thermophile
Senior Member
 
Location: CT

Join Date: Apr 2015
Posts: 243
Default

I have yet to figure out how to mount basemount so it's writable but am not sure if that's related to the server that I'm mounting on rather than basemount itself. The server I"m using uses fusermount which seems to do very strange things to basemount (ie. to unmount basemount, I have to chmod my entire home directory to 775 not just the folder containing the mountpoint).
__________________
Microbial ecologist, running a sequencing core. I have lots of strong opinions on how to survey communities, pretty sure some are even correct.
thermophile is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 01:05 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO