Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • pbl to run hisat2 on a computer cluster with hg19

    Hi,
    I want to map RNAseq data (from 10 to 30 Go, single and paired-end) on complete human genome with hisat2. To do that, I use a cluster (15 nodes, 24 cores each one, 64Go RAM, workload manager SLURM).
    I succeed to align RNAseq files (20Go single reads or 2x20Go paired-end reads) against one human chromosome (chr7), indexed with annotation according to the manual (exons and splice sites extracted by adapted python files) with 1 node/24 cores (with hisat2 option -p 24) or 2 nodes/48 cores (-p 48).
    I succeed to map my data against partial genome (chromosomes 1 to 10, without annotation).
    But when I want to align with the full human genome (only the chromosomes, index without annotation), ( several tests with 2 to 12 nodes, 12 or 24 cores/node), the execution fails with a error message like this :
    slurmstepd: Step 8878.0 exceeded memory limit (3652264 > 2744320), being killed
    srun: Exceeded job memory limit
    srun: Exceeded job memory limit
    srun: Force Terminated job 8878
    srun: Job step aborted: Waiting up to 2 seconds for job step to finish.
    slurmstepd: *** STEP 8878.0 CANCELLED AT 2016-03-30T11:16:32 ***
    srun: error: thau016: task 0: Killed

    I understand that I don't have enought memory, even with this cluster. So does anyone know features to align such data against complete human genome? Is the problem something else?
    Thanks!

  • #2
    Although I'm unfamiliar with your system my interpretation is that you have a software restriction on the memory usage, rather than a hardware limitation.
    Last edited by wdecoster; 04-29-2016, 04:16 AM.

    Comment


    • #3
      I concur with @wdecoster. It appears that you are not asking for memory specifically with your SLURM command line (--mem something). That cluster config should be enough to do the alignment though I would suggest not going overboard with the cores. Even though 24 may be available per node they may become I/O starved quickly.

      Comment


      • #4
        Hi,

        in the cluster documentation, the --mem option is fixed to the memory limit of the node by default.
        Anywy, I'm going to precise --mem option.
        Only for information, I have precised these SLURM options: --nodes, --ntasks-per-node and --ntasks-per-core, with the '-p' hisat2 option according to asked nodes.

        Had anyone used hisat2 on a cluster?

        Comment


        • #5
          I have used HISAT2 with LSF on a cluster.

          The --mem may not be right option (not a SLURM user). You may want to check with your local cluster admins about the correct option to use in this case. You appear to be asking for just 3.6M (if the numbers are in bytes or 3.6G if in kb) and in either case your nodes have plenty of RAM so it should work.

          Comment

          Latest Articles

          Collapse

          • seqadmin
            Strategies for Sequencing Challenging Samples
            by seqadmin


            Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
            03-22-2024, 06:39 AM
          • seqadmin
            Techniques and Challenges in Conservation Genomics
            by seqadmin



            The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

            Avian Conservation
            Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
            03-08-2024, 10:41 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, Yesterday, 06:37 PM
          0 responses
          12 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, Yesterday, 06:07 PM
          0 responses
          10 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 03-22-2024, 10:03 AM
          0 responses
          51 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 03-21-2024, 07:32 AM
          0 responses
          68 views
          0 likes
          Last Post seqadmin  
          Working...
          X