SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics
Similar Threads
Thread Thread Starter Forum Replies Last Post
BioPerl Issues noobie Bioinformatics 0 02-19-2013 07:36 PM
hi-seq issues dummyseq Illumina/Solexa 0 11-30-2012 06:50 AM
Bioanalyzer issues julihepple Ion Torrent 2 04-24-2012 10:31 AM
Data issues... cggj Bioinformatics 1 06-09-2010 07:00 PM
BWA memory issues tgenahmet Bioinformatics 3 06-04-2009 02:55 AM

Reply
 
Thread Tools
Old 06-26-2013, 10:33 AM   #1
DrWorm
Member
 
Location: St. Louis, MO

Join Date: Apr 2013
Posts: 17
Default seq_crumbs memory issues

Hey all,

I've been loving the seq_crumbs package for trimming and filtering my Illumina data, and was excited to download the latest update that can handle paired end data. Unfortunately, when I tried to use the tools I noticed that the memory requirements had increased dramatically. I expected a modest increase in processing paired end data compared to single end files, but it was jumping from 8G for single end to 80G for pe. Has anyone else encountered this issue or am I doing something totally wrong?

Also, does anyone else know of tools for quality filtering and complexity filtering that maintain read pairs? I'm aware of trimmomatic and trim_galore for quality trimming, but I didn't think they do filtering based on the quality/complexity of the read overall.

Thanks!
DrWorm is offline   Reply With Quote
Old 07-18-2013, 10:47 PM   #2
peio
Junior Member
 
Location: Spain

Join Date: Mar 2009
Posts: 6
Default

Quote:
Originally Posted by DrWorm View Post
Hey all,

I've been loving the seq_crumbs package for trimming and filtering my Illumina data, and was excited to download the latest update that can handle paired end data. Unfortunately, when I tried to use the tools I noticed that the memory requirements had increased dramatically. I expected a modest increase in processing paired end data compared to single end files, but it was jumping from 8G for single end to 80G for pe. Has anyone else encountered this issue or am I doing something totally wrong?

Also, does anyone else know of tools for quality filtering and complexity filtering that maintain read pairs? I'm aware of trimmomatic and trim_galore for quality trimming, but I didn't think they do filtering based on the quality/complexity of the read overall.

Thanks!
Hi DrWorm,

I am one of the developers of the seq_crumbs package. We're interested in the memory hogging issue.
Versions prior to 0.1.8 had memory related bug that we solved in the 0.1.8 version.

seq_crumb uses python's standar library for multiprocessing and seems that versions of python <=2.7.3 are affected by a bug related with memory consumption.

With python 2.7.4 in the latest ubuntu the memory use stays stable but with latest debian and python 2.7.3 memory use grows untill the process is finished.

Which version of python are you using?
p.
peio is offline   Reply With Quote
Old 07-19-2013, 05:40 AM   #3
DrWorm
Member
 
Location: St. Louis, MO

Join Date: Apr 2013
Posts: 17
Default

I believe we're using Python 2.6.5. I'll request an update and see if that makes a difference.

Thanks for the advice!!
DrWorm is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -8. The time now is 09:46 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2022, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO