SEQanswers

Go Back   SEQanswers > Applications Forums > Metagenomics



Similar Threads
Thread Thread Starter Forum Replies Last Post
bbduk memory error gastonviarengo Bioinformatics 1 05-10-2021 10:38 AM
BBduk for NexteraXT arivers Bioinformatics 3 05-07-2018 11:52 AM
BBDUK and BBMERGE, which goes first chloe1005 Bioinformatics 1 04-12-2018 04:43 AM
bbduk.sh barcode filter lamon Bioinformatics 8 11-18-2017 04:51 AM
Trimmomatic vs bbduk.sh mslider Bioinformatics 1 04-18-2017 11:10 AM

Reply
 
Thread Tools
Old 09-22-2021, 11:01 AM   #1
connieR
Junior Member
 
Location: CA

Join Date: Sep 2021
Posts: 2
Default BBDuk Out of Memory

Hello! I am running BBDuk from BBTools to remove reads that map to a mammal host genome (2.3G). I am running the bbduk.sh script on just 1 sample (an R1 and R2; each 12G). Initially, I kept getting an "java.lang.OutOfMemoryError: Java heap space" or "java.lang.OutOfMemoryError: GC overhead limit exceeded" errors, so I started requesting 100G+ of memory for my slurm jobs. The jobs run until the time limit I set is reached (3 days), and they do not produce any output files or even temporary files. The last job I ran (1 node, 15 cores, 300G of memory) is below:

bbduk.sh in1=R1.fastq in2=R2.fastq out1=clean.R1.fastq out2=clean.R2.fastq outm1=matched1.fq outm2=matched2.fq ref=CC.genomic.fna stats=stats.txt k=31 prealloc hdist=1 rskip=4 -Xmx255g

The output I received:
Executing jgi.BBDuk
Version 38.87

Initial size set to 2142000000
36.158 seconds.
Initial:
Memory: max=273804m, total=273804m, free=85629m, used=188175m

slurmstepd: error: *** JOB CANCELLED DUE TO TIME LIMIT ***

---------------------
I have also tried adding the rskip=4 and qskip=4 flags to no avail. I ran the same code on just the first 1000 lines of my fastq file and also received the "out of memory" issue.

Any idea for what might be going on? Why is this task taking so long and requires so much memory for 1 sample? What I can do to diagnose the issue and actually be able to successfully run my code? I have yet to successfully run the BBDuk code on my data.
connieR is offline   Reply With Quote
Old 09-23-2021, 03:03 PM   #2
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,127
Default

You should use a different program for this purpose. Look up "bbsplit.sh". BBDuk is not the right program for this application.
GenoMax is offline   Reply With Quote
Old 09-24-2021, 11:37 AM   #3
connieR
Junior Member
 
Location: CA

Join Date: Sep 2021
Posts: 2
Default

Thank you for your reply. That is what I was suspecting. I will look into other tools and/or bbsplit!!
connieR is offline   Reply With Quote
Old 09-24-2021, 11:59 AM   #4
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 7,127
Default

Quote:
Originally Posted by connieR View Post
Thank you for your reply. That is what I was suspecting. I will look into other tools and/or bbsplit!!
See: http://seqanswers.com/forums/showthread.php?t=41288
GenoMax is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 01:30 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2021, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO