SEQanswers

Go Back   SEQanswers > Bioinformatics > Bioinformatics



Similar Threads
Thread Thread Starter Forum Replies Last Post
MiSeq run with low Q30 and high cluster PF haroldnunez Illumina/Solexa 12 01-06-2017 07:34 AM
HISAT2 vs. TopHat2: Discrepancies between Hg19 & Hg38 Sbamo RNA Sequencing 0 01-26-2016 06:05 AM
MiSeq Run with low Cluster PF Help dross11 Illumina/Solexa 13 03-23-2015 09:01 AM
Run cd-hit-para.pl on sge cluster lzlgboy Bioinformatics 0 01-07-2015 02:19 PM
Run cuffdiff in a computer cluster satp Bioinformatics 1 07-12-2012 12:20 PM

Reply
 
Thread Tools
Old 04-29-2016, 03:14 AM   #1
ebe
Junior Member
 
Location: France

Join Date: Apr 2013
Posts: 6
Default pbl to run hisat2 on a computer cluster with hg19

Hi,
I want to map RNAseq data (from 10 to 30 Go, single and paired-end) on complete human genome with hisat2. To do that, I use a cluster (15 nodes, 24 cores each one, 64Go RAM, workload manager SLURM).
I succeed to align RNAseq files (20Go single reads or 2x20Go paired-end reads) against one human chromosome (chr7), indexed with annotation according to the manual (exons and splice sites extracted by adapted python files) with 1 node/24 cores (with hisat2 option -p 24) or 2 nodes/48 cores (-p 48).
I succeed to map my data against partial genome (chromosomes 1 to 10, without annotation).
But when I want to align with the full human genome (only the chromosomes, index without annotation), ( several tests with 2 to 12 nodes, 12 or 24 cores/node), the execution fails with a error message like this :
slurmstepd: Step 8878.0 exceeded memory limit (3652264 > 2744320), being killed
srun: Exceeded job memory limit
srun: Exceeded job memory limit
srun: Force Terminated job 8878
srun: Job step aborted: Waiting up to 2 seconds for job step to finish.
slurmstepd: *** STEP 8878.0 CANCELLED AT 2016-03-30T11:16:32 ***
srun: error: thau016: task 0: Killed

I understand that I don't have enought memory, even with this cluster. So does anyone know features to align such data against complete human genome? Is the problem something else?
Thanks!
ebe is offline   Reply With Quote
Old 04-29-2016, 04:44 AM   #2
wdecoster
Member
 
Location: Antwerp, Belgium

Join Date: Oct 2015
Posts: 97
Default

Although I'm unfamiliar with your system my interpretation is that you have a software restriction on the memory usage, rather than a hardware limitation.

Last edited by wdecoster; 04-29-2016 at 05:16 AM.
wdecoster is offline   Reply With Quote
Old 04-29-2016, 04:50 AM   #3
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 6,815
Default

I concur with @wdecoster. It appears that you are not asking for memory specifically with your SLURM command line (--mem something). That cluster config should be enough to do the alignment though I would suggest not going overboard with the cores. Even though 24 may be available per node they may become I/O starved quickly.
GenoMax is offline   Reply With Quote
Old 04-29-2016, 07:58 AM   #4
ebe
Junior Member
 
Location: France

Join Date: Apr 2013
Posts: 6
Default

Hi,

in the cluster documentation, the --mem option is fixed to the memory limit of the node by default.
Anywy, I'm going to precise --mem option.
Only for information, I have precised these SLURM options: --nodes, --ntasks-per-node and --ntasks-per-core, with the '-p' hisat2 option according to asked nodes.

Had anyone used hisat2 on a cluster?
ebe is offline   Reply With Quote
Old 04-29-2016, 08:17 AM   #5
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 6,815
Default

I have used HISAT2 with LSF on a cluster.

The --mem may not be right option (not a SLURM user). You may want to check with your local cluster admins about the correct option to use in this case. You appear to be asking for just 3.6M (if the numbers are in bytes or 3.6G if in kb) and in either case your nodes have plenty of RAM so it should work.
GenoMax is offline   Reply With Quote
Reply

Tags
cluster, genome alignment, hisat2, memory requirement

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 05:14 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO