SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
Time & Cost of using 1 MiSeq Machine to do 16s rDNA (V2/V4) Seq on 300 Samples/Month vs92 Illumina/Solexa 28 10-09-2015 11:07 AM
Trouble running Tophat & Cufflinks on Galaxy with non-UCSC-supported genome NSher Bioinformatics 2 12-29-2011 07:05 PM
Galaxy tools - local installation doc.ramses Bioinformatics 0 01-12-2011 12:32 AM
Quick and dirty? scientifica Bioinformatics 5 01-27-2010 11:53 PM
Helicos sequencing machine data & format required balamudiam Helicos / Direct Genomics 4 10-27-2009 08:28 PM

Reply
 
Thread Tools
Old 09-26-2012, 12:47 AM   #1
sam.paech
Junior Member
 
Location: Australia

Join Date: Sep 2012
Posts: 1
Default Quick & dirty Galaxy installation in a virtual machine

QUICK & DIRTY GALAXY INSTALLATION IN A VIRTUAL MACHINE

(with basic tools, hg19 reference, ftp upload)


### DISCLAIMERS: ###
1. This build is for someone who wants to just get galaxy up and running and be able to use it, with minimal Linux expertise, minimal time & cost, and on existing hardware.
2. There is ZERO focus on security or data privacy in this build. See the title.
3. I'm not in any way affiliated with the Galaxy team. I just find this style handy for basic setup of my Galaxy VMs and thought others might benefit.
4. This build should work on any hardware capable of running virtualbox & 64-bit VMs (RAM & CPU permitting).
5. The links WILL break & become outdated over time, so amend as necessary.
6. Sorry about the mess.
7. Enjoy =)


**** RESOURCES ****
http://wiki.g2.bx.psu.edu/Admin/NGS%20Local%20Setup
http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Dependencies
http://www.agapow.net/science/bioinformatics/galaxy/installing-galaxy
http://wiki.g2.bx.psu.edu/Admin/Config/Upload%20via%20FTP

### OS Setup ###
[Install ubuntu 11.10 desktop 64 bit in virtualbox, vmware, or bare metal. Virtualbox is free and cross-platform.]
- Give the VM's hard disk a bit of space, as the tools & data will eat it up. I'd say at least 100 GB.
- root user: galaxy/galaxy
[optional for vmware]: install vmware tools & reboot

[Once OS is installed and at command line:]
sudo apt-get update
sudo apt-get install openssh-server
sudo nano /etc/ssh/sshd_config
[add the line:]
UseDNS no
[save & exit (ctrl-x, y, enter)]
sudo /etc/init.d/ssh restart


### Static ip ###
[optional] Give the server a static IP & set network settings appropriately
- We'll use 10.71.10.26 here for simplicity, but you could equally set up a DNS name

### Galaxy setup ###
mkdir ~/galaxy-python
ln -s /usr/bin/python ~/galaxy-python/python
export PATH=~/galaxy-python:$PATH
sudo apt-get install mercurial
hg clone https://bitbucket.org/galaxy/galaxy-dist/
sudo apt-get install python-dev -y
sudo apt-get install openjdk-6-jre -y
sudo apt-get install build-essential -y
cd ~/galaxy-dist
[let the run script do its initial configuration:]
sh run.sh

[open a second terminal to continue]

*** IF we get errors with building eggs, we can install them manually like so:
python scripts/scramble.py -e Whoosh
python scripts/scramble.py -e Cheetah
python scripts/scramble.py -e twill
python scripts/scramble.py -e WebHelpers
python scripts/scramble.py -e docutils
python scripts/scramble.py -e python_lzo
python scripts/scramble.py -e ssh
python scripts/scramble.py -e GeneTrack
python scripts/scramble.py -e Paste
python scripts/scramble.py -e PasteScript
python scripts/scramble.py -e nose
python scripts/scramble.py -e pycrypto
***

[Assuming galaxy is up & running, we now need to kill it manually:]
[find the PIDs of "sh run.sh" and "python ./scripts/paster.py serve universe_wsgi.ini", and use the kill command:]
ps -au
kill [PIDs]

[open galaxy server to other machines on the network:]
nano universe_wsgi.ini
[add the line to [server:main] section:]
host = 0.0.0.0
[save & exit]
sh run.sh

[access from another machine's web browser, e.g.:]
http://10.71.10.26:8080


### Configure Galaxy to run with postgresql ###
[first, kill galaxy manually]
sudo apt-get install postgresql
sudo -u postgres psql postgres
\password postgres
[enter "postgres" as the password]
ctrl-d
psql -h localhost -U postgres
CREATE DATABASE galaxy_prod;
GRANT ALL ON DATABASE galaxy_prod TO postgres;
ctrl-d
nano ~/galaxy-dist/universe_wsgi.ini
[add the following line to the [app:main] section of the config file:]
database_connection = postgres://postgres:postgres@localhost:5432/galaxy_prod
[save & exit]
cd ~/galaxy-dist
python scripts/scramble.py -e psycopg2
sh run.sh
[tables will migrate to postgresql]
###############


### Configure FTP upload ###
chmod 777 /home/galaxy/galaxy-dist/database/files
nano ~/galaxy-dist/universe_wsgi.ini
[add the lines to [app:main] section:]
ftp_upload_dir = /home/galaxy/galaxy-dist/database/files/
ftp_upload_site = 10.71.10.26
[save & close]
createuser -SDR galaxyftp -h localhost -U postgres
[password "postgres"]
psql -h localhost -U postgres -d galaxy_prod
ALTER ROLE galaxyftp PASSWORD 'dbpassword';
GRANT SELECT ON galaxy_user TO galaxyftp;
ctrl-d
sudo apt-get install proftpd-mod-pgsql -y
[run proftpd: from standalone]
sudo nano /etc/proftpd/proftpd.conf

[Lines to modify:]
ServerName "Galaxy FTP"
Umask 000

[Lines to add (AFTER "Include /etc/proftpd/modules.conf" line):]
Include /etc/proftpd/sql.conf
SyslogFacility DAEMON
SyslogLevel debug
DefaultRoot ~
CreateHome on 777 dirmode 777
AllowStoreRestart on
<Limit SITE_CHMOD>
DenyAll
</Limit>
<Limit RETR>
DenyAll
</Limit>
AuthPAM off
SQLPasswordEngine on
SQLPasswordEncoding hex
SQLEngine on
SQLBackend postgres
SQLConnectInfo galaxy_prod@localhost postgres postgres
SQLAuthTypes SHA1
SQLAuthenticate users
SQLDefaultHomedir /var/opt/local/proftpd
SQLUserInfo custom:/LookupGalaxyUser
SQLNamedQuery LookupGalaxyUser SELECT "email,password,'512','512','/home/galaxy/galaxy-dist/database/files/%U','/bin/bash' FROM galaxy_user WHERE email='%U'"

[save & quit]
sudo nano /etc/proftpd/modules.conf
[uncomment the following:]
LoadModule mod_sql.c
LoadModule mod_sql_postgres.c
LoadModule mod_sql_passwd.c

[save & exit]
sudo service proftpd start
kill galaxy & restart it
###############


### Start Galaxy automatically at bootup ###
sudo nano /etc/rc.local
[add the lines:]
sleep 10
sh ~/galaxy-dist/run.sh --daemon
[save & exit]
###############




**** STARTING & STOPPING GALAXY MANUALLY ****
[Once installed, galaxy can/should be started as a daemon:]
sudo sh ~/galaxy-dist/run.sh --daemon

[To stop galaxy when running as daemon:]
sudo sh ~/galaxy-dist/run.sh --stop-daemon

[To get galaxy status:]
sudo sh ~/galaxy-dist/run.sh --status

[Restart galaxy:]
sudo sh ~/galaxy-dist/run.sh --monitor-restart
*********************************************





########################
configure external tools:
########################


# Initial setup #
sudo mkdir /usr/local/apps
sudo chmod 755 /usr/local/apps
sudo chown galaxy /usr/local/apps

### Blast ###
sudo apt-get install ncbi-blast+
###############

### FastQC ###
cd /usr/local/apps
wget http://www.bioinformatics.babraham.ac.uk/projects/fastqc/fastqc_v0.10.1.zip
unzip fastqc_v0.10.1.zip
cd FastQC
chmod 755 fastqc
sudo ln -s /usr/local/apps/FastQC/fastqc /usr/local/bin/fastqc
cd ~/galaxy-dist/tool-data/shared/jars
mkdir FastQC
cd FastQC
ln -s /usr/local/apps/FastQC/fastqc fastqc
###############

### samtools ###
sudo apt-get install samtools
################

### fastx-toolkit ###
sudo apt-get install fastx-toolkit
################

### bwa ###
[NOTE: if bwa is updated, all reference indexes must be regenerated.]
cd /usr/local/apps
wget http://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2
tar -xjvf bwa-0.6.2.tar.bz2
cd bwa-0.6.2
sudo make
sudo ln -s /usr/local/apps/bwa-0.6.2/bwa /usr/local/bin/
sudo ln -s /usr/local/apps/bwa-0.6.2/*.pl /usr/local/bin/
################

### picard-tools ###
cd /usr/local/apps/
mkdir picard-tools
cd picard-tools
wget http://downloads.sourceforge.net/project/picard/picard-tools/1.74/picard-tools-1.74.zip
unzip picard-tools-1.74.zip
cd picard-tools-1.74.zip
chmod +x *
cd ~/galaxy-dist/tool-data/shared/jars
mkdir picard
cd picard
ln -s /usr/local/apps/picard-tools/picard-tools-1.74/*.jar ./
################

### tophat & cufflinks ###
cd /usr/local/apps
wget http://tophat.cbcb.umd.edu/downloads/tophat-2.0.4.Linux_x86_64.tar.gz
tar -xvf tophat-2.0.4.Linux_x86_64.tar.gz
wget http://cufflinks.cbcb.umd.edu/downloads/cufflinks-2.0.2.Linux_x86_64.tar.gz
tar -xvf cufflinks-2.0.2.Linux_x86_64.tar.gz
[only for older versions of ubuntu < 11.10]:
sudo apt-get install libboost*
sudo ln -s /usr/local/apps/cufflinks-2.0.2.Linux_x86_64/* /usr/local/bin/
sudo ln -s /usr/local/apps/tophat-2.0.4.Linux_x86_64/* /usr/local/bin/
###############

### bowtie ###
cd /usr/local/apps
wget http://downloads.sourceforge.net/project/bowtie-bio/bowtie/0.12.8/bowtie-0.12.8-linux-x86_64.zip
unzip bowtie-0.12.8-linux-x86_64.zip
sudo ln -s /usr/local/apps/bowtie-0.12.8/bowtie /usr/local/bin/bowtie
sudo ln -s /usr/local/apps/bowtie-0.12.8/bowtie-build /usr/local/bin/bowtie-build
sudo ln -s /usr/local/apps/bowtie-0.12.8/bowtie-inspect /usr/local/bin/bowtie-inspect
###############

### bowtie2 ###
cd /usr/local/apps
wget http://downloads.sourceforge.net/project/bowtie-bio/bowtie2/2.0.0-beta7/bowtie2-2.0.0-beta7-linux-x86_64.zip
unzip bowtie2-2.0.0-beta7-linux-x86_64.zip
sudo ln -s /usr/local/apps/bowtie2-2.0.0-beta7/bowtie2 /usr/local/bin/bowtie2
sudo ln -s /usr/local/apps/bowtie2-2.0.0-beta7/bowtie2-align /usr/local/bin/bowtie2-align
sudo ln -s /usr/local/apps/bowtie2-2.0.0-beta7/bowtie2-build /usr/local/bin/bowtie2-build
sudo ln -s /usr/local/apps/bowtie2-2.0.0-beta7/bowtie2-inspect /usr/local/bin/bowtie2-inspect
###############





########################
Data & reference genomes:
########################

2bit conversion program:
###############
mkdir /usr/local/data/twoBitToFa
cd /usr/local/data/twoBitToFa
wget http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/twoBitToFa
chmod 755 twoBitToFa
sudo ln -s /usr/local/apps/twoBitToFa /usr/local/bin/
###############

adding a reference genome (fasta) & samtools index:
###############
sudo mkdir /usr/local/data
sudo chown galaxy /usr/local/data
cd /usr/local/data
mkdir samtools
mkdir samtools/hg19/
cd samtools/hg19/
wget http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/hg19.2bit
twoBitToFa hg19.2bit hg19.fa
samtools faidx hg19.fa
nano ~/galaxy-dist/tool-data/sam_fa_indices.loc
[add the line:]
index hg19 /usr/local/data/samtools/hg19/hg19.fa
[save & close]
nano ~/galaxy-dist/tool-data/faseq.loc
[add the line:]
hg19 /usr/local/data/samtools/hg19
[save & close]
###############

bowtie reference indexes:
###############
[make sure ~8-10 GB of RAM is available before building indexes, or it may take forever]
cd /usr/local/data/samtools/hg19
bowtie-build -f hg19.fa hg19
mkdir /usr/local/data/bowtie
mkdir /usr/local/data/bowtie/hg19
mv *.ebwt /usr/local/data/bowtie/hg19/
nano ~/galaxy-dist/tool-data/bowtie_indices.loc
[add the line:]
hg19 hg19 hg19 /usr/local/data/bowtie/hg19/hg19
[close & save]
[restart galaxy]
###############


bowtie2 reference indexes:
###############
mkdir /usr/local/data/bowtie2
mkdir /usr/local/data/bowtie2/hg19
cd /usr/local/data/bowtie2/hg19
wget ftp://ftp.cbcb.umd.edu/pub/data/bowtie2_indexes/incl/hg19.zip
unzip hg19.zip
nano ~/galaxy-dist/tool-data/bowtie2_indices.loc
[add the line:]
hg19 hg19 hg19 /usr/local/data/bowtie2/hg19/hg19
[close & save]
[restart galaxy & check the reference genome dropdown list in bowtie2]
###############

BWA reference index:
###############
[NOTE: if bwa is updated, all reference indexes must be regenerated.]
mkdir /usr/local/data/bwa
mkdir /usr/local/data/bwa/hg19
cd /usr/local/data/samtools/hg19
bwa index -a bwtsw hg19.fa
mv hg19.fa.* /usr/local/data/bwa/hg19
mv /usr/local/data/bwa/hg19/hg19.fa.fai /usr/local/data/samtools/hg19

[MAKE SURE hg19.fa.fai STAYS IN samtools/hg19 DIR]

nano ~/galaxy-dist/tool-data/bwa_index.loc
[add the line:]
hg19 hg19 hg19 /usr/local/data/bwa/hg19/hg19.fa
[close & save]
[restart galaxy & check the reference genome dropdown list in bowtie2]
###############
sam.paech is offline   Reply With Quote
Old 08-21-2014, 02:15 AM   #2
dokadya
Junior Member
 
Location: bangalore

Join Date: Mar 2014
Posts: 5
Default

Hey,
Thanks for tutorial. It made my work easy.
Few of the links and steps are changed in above, I will try to put it, as soon as I complete it.

My question is How to upload data using FTP ?
as per your tutorial, what is user name and password ?
how should i connect it ?

Waiting for positive reply.
Thank You.
dokadya is offline   Reply With Quote
Reply

Tags
galaxy, ubuntu, virtual machine, virtualbox

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 03:36 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO