SEQanswers

Go Back   SEQanswers > General



Similar Threads
Thread Thread Starter Forum Replies Last Post
parallel de novo assembler tmy1018 Bioinformatics 3 10-22-2012 09:31 AM
Hadoop for NGS sequence analysis samanta General 0 09-03-2011 12:07 PM
Reproducibility of CLC de novo assembler corthay Bioinformatics 1 06-03-2010 06:07 AM
De Novo Short Read Assembler? doxologist De novo discovery 18 05-21-2010 06:55 AM
looking for reference genome based assembler for short-reads zchou Bioinformatics 3 12-16-2009 09:13 PM

Reply
 
Thread Tools
Old 09-08-2011, 12:16 PM   #1
samanta
Senior Member
 
Location: Seattle

Join Date: Feb 2010
Posts: 109
Default Contrail - a hadoop-based de novo sequence assembler

Hello all,

Most bioinformatics researchers get stuck with the question of how to buy a computer with enough RAM to process their NGS data, because RAM is very expensive. It is not easy to get approval for buying a $100K computer from the managers, when they think everything can be done using $1K laptop and Microsoft excel.

Internet companies like Google developed algorithms to process terabytes and petabytes of data very rapidly and give users the search results. They use clusters of commodity computers with inexpensive disks (hard drive is cheap) using an approach called MapReduce. MapReduce framework is available for free under Hadoop framework distributed by Apache foundation.

Few months back, I came across a genome assembly program called 'contrail' that uses Hadoop to assemble large quantities of NGS data, and it is scalable. When I speak to bioinformaticians about trying out Hadoop instead of buying large and expensive RAM-based machine, I usually hit a hard wall, because the words like Hadoop, MapReduce etc. are foreign to them. So, today I wrote a post to explain setting up and run contrail on your own machine using Hadoop. It is written in such a way that even if you never used Hadoop etc., you can mechanically execute the steps and will be able to assemble the reads in test library in a short time in your own Windows or Unix box. I am hoping that once researchers start to feel that Hadoop approach is easy and scalable for large data sets, they will be able to develop their own programs and the whole community will benefit.

This post discusses how to use contrail assembler -

http://www.homolog.us/blogs/2011/09/...t-uses-hadoop/

This post discusses how to set up and run Hadoop for a simple sequence analysis example -

http://www.homolog.us/blogs/2011/08/...o-get-started/


Please note that I am not associated with the researchers, who wrote contrail, and never spoke to them or met them. It is the only example I found for de Bruijn assemblers and decided to try it out.
__________________
http://homolog.us
samanta is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 09:04 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO