SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
bowtie2 sam files problem cz0013 Bioinformatics 4 07-03-2014 10:18 AM
Bowtie2 problem rakscalloni Bioinformatics 1 04-28-2014 07:14 AM
SAM (from bowtie2) to BAM problem using samtools amango Bioinformatics 10 04-15-2013 12:52 AM
Basic problem while using BWA miguelangel Bioinformatics 2 12-13-2012 12:31 AM
I need basic help with basic questions re: analysis rd69 General 3 02-16-2012 03:11 PM

Reply
 
Thread Tools
Old 07-31-2014, 12:48 AM   #1
prolife
Junior Member
 
Location: Italy

Join Date: Apr 2014
Posts: 4
Default Bowtie2 basic problem!

Hi

I'm new in the bioinformatic field and I have a basic problem

I have a read,in fastq format, ready to align on my reference genome (human genome), but I'm not able to do that!

I've understand that I must to indexing my genome and create a basename before convert my fastq in a sam file.

I need your help, I've read the tutorial but I haven't the sufficient bioinformatic knoledge to do that by myself.

Thanks
prolife is offline   Reply With Quote
Old 07-31-2014, 01:18 AM   #2
mastal
Senior Member
 
Location: uk

Join Date: Mar 2009
Posts: 667
Default

Look at the 'Getting started' section of the Bowtie webpage and work through their example.

You can also download pre-built versions of the genome index from the links on the Bowtie webpage, including links to Illumina's iGenomes.
mastal is offline   Reply With Quote
Old 07-31-2014, 02:09 AM   #3
prolife
Junior Member
 
Location: Italy

Join Date: Apr 2014
Posts: 4
Default

Solved, thanks!
prolife is offline   Reply With Quote
Old 08-20-2014, 07:34 AM   #4
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

Hi,
i'm trying to index GRCh37 with Bowtie2. RAM is 1.5Gb and indexing has been ongoing for more than 48 hours.
Is it normal?
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 09:19 AM   #5
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,707
Default

No, Bowtie2's indexing is really slow, but not THAT slow. When you say "RAM is 1.5Gb" do you mean the amount of RAM used by the process, or total memory on the computer? Also, knowing the exact operating system and CPU type would be very helpful.

Also look at the CPU utilization for that process (e.g. with "top").
Brian Bushnell is offline   Reply With Quote
Old 08-20-2014, 09:29 AM   #6
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

The total memory of computer is 4Gb while 1.5Gb is the memory allocated for ubuntu that I use as virtual machine. What do you intend for CPU type. Processor is i3, 1.8Ghz.
Is "top" a command that I have to digit?
Now indexing is running, so I can't work on shell!!
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 09:38 AM   #7
Brian Bushnell
Super Moderator
 
Location: Walnut Creek, CA

Join Date: Jan 2014
Posts: 2,707
Default

I don't think 1.5Gb is enough for Bowtie2 to index or map to Human. If you have a 64-bit OS, you should kill the process, give Ubuntu all the memory, and restart it.
Brian Bushnell is offline   Reply With Quote
Old 08-20-2014, 09:40 AM   #8
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

1.5GB RAM is not enough for this process (or the VM). Plus you are probably running 32-bit Ubuntu, which will become a problem with human genome size datasets. You should find an alternate machine with better hardware to do this.
GenoMax is offline   Reply With Quote
Old 08-20-2014, 09:49 AM   #9
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

No I'm running 64-bit Ubuntu. So do you think I might solve this problem allocating the entire RAM to Ubuntu virtual machine?
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 09:55 AM   #10
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

How much actual RAM do you have (4?) on this machine? Since the host OS is using some of it you will not be able to allocate all 4GB for the VM.
GenoMax is offline   Reply With Quote
Old 08-20-2014, 10:01 AM   #11
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

On my PC I have 4Gb of RAM. I suppose it could not be possible to allocate all 4Gb for VM. What can I do to improve performance of my PC?
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 10:07 AM   #12
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

Maybe I could expand RAM of my PC. What do you think about it?
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 10:11 AM   #13
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

Are you able to purchase more RAM? Please be aware that trying to do human data alignments in a VM is going to be a difficult task. You may be able to run a single alignment thread in the VM.

From Bowtie Manual

Quote:
Bowtie 2 indexes the genome with an FM Index (based on the Burrows-Wheeler Transform or BWT) to keep its memory footprint small: for the human genome, its memory footprint is typically around 3.2 gigabytes of RAM.
GenoMax is offline   Reply With Quote
Old 08-20-2014, 10:16 AM   #14
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

Or I could allocate 3Gb of RAM for VM? It could improve indexing and other analyses?
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 10:19 AM   #15
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

I have a Mac with 4GB of RAM and 1333Mhz. It is my personal machine but do you think I could use this computer to run analysis or Mac is not good?
Excuse me but I'm a newbie.
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 10:21 AM   #16
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

and do you think that expanding RAM is not the solution because VM is also a problem to run alignment?
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 10:28 AM   #17
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

simobioinfo: I do not want to discourage you but if you are able to use institutional resources to do the actual computing you would find the experience much more enjoyable. You can certainly try and get started on the Mac and depending on your experience you may want to move to another resource using your Mac as the terminal to connect to that resource.

Note: Since you have a Mac you strictly do not need to use a VM. OS X is unix compatible and there are binary versions of tuxedo suite programs available that will natively run on OS X.

You can download pre-formatted indexes/annotations for human genome from the iGenomes site: http://support.illumina.com/sequenci...e/igenome.html This is a big download but hopefully you are using institutional network and would not have bandwidth issues.

Last edited by GenoMax; 08-20-2014 at 10:32 AM.
GenoMax is offline   Reply With Quote
Old 08-20-2014, 10:47 AM   #18
simobioinfo
Member
 
Location: italy

Join Date: Aug 2014
Posts: 38
Default

I would like to use institutional resources but I have been wait for a new PC for 2 years (in Italy research is very difficult) and now I have to start my work. So I think that using my Mac could be a faster and more reliable solution. Are you agree?
simobioinfo is offline   Reply With Quote
Old 08-20-2014, 11:01 AM   #19
biocomputer
Member
 
Location: Canada

Join Date: Dec 2013
Posts: 62
Default

I think your indexing is taking so long because with only 1.5GB allocated to the VM, bowtie2-build doesn't have near enough RAM so it is having to use a swapfile which will lead to thrashing and will make the process take a really, really long time. I just indexed the mouse genome mm10 for Bowtie2 using default settings and it used 5GB of RAM while building the index and took 4 hours on a large server, but even on a reasonably fast desktop computer as long as you have enough RAM it should be able to build overnight. I've indexed the mouse genome mm9 on an i7-2720QM in 6-8 hours. But anyways, as has already been mentioned you can download pre-built indexes. So you'll either have to download them or upgrade your RAM.

Also why are you using a VM? Bowtie2 is available for Linux, Windows, and Mac so you should be able to run it natively. With only 4GB of RAM you still won't have enough to build the index but when it comes to aligning then 4GB may be enough as long as you don't run many other processes at the same time.

Last edited by biocomputer; 08-20-2014 at 11:05 AM.
biocomputer is offline   Reply With Quote
Old 08-20-2014, 11:02 AM   #20
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,077
Default

Starting with your Mac is certainly under your control.

Are you running a new(er) version of OS X (10.7.x or later)? Working natively in OS X would allow you to avoid the VM issue entirely.
GenoMax is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 10:27 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO