Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • AddOrReplaceReadGroups error (that is probably a more general system or Java error)

    In running Picard Tools' AddOrReplaceReadGroups, I got an error that included the following:

    Code:
    Exception in thread "main" net.sf.samtools.util.RuntimeIOException: java.io.IOException: No space left on device
    This sounds like I have run out of disk space, but I haven't. My command was the following:

    Code:
    java -d64 -Xmx25g -jar /home/efoss/source_codes/picard-tools-1.82/AddOrReplaceReadGroups.jar \
    	INPUT=AML_PAERAH_73407_RNA_partialperlclean_noMT.sam \
    	OUTPUT=AML_PAERAH_73407_RNA_partialperlclean_noMT_AORRGaddedsorted.sam \
    	SORT_ORDER=coordinate \
    	RGID=RGID_PAERAH_73407 \
    	RGLB=PAERAH_73407_library \
    	RGPL=Illumina \
    	RGPU=run_barcode_arbitrary \
    	RGSM=AML_PAERAH_73407_RNAseq \
    	RGCN=FHCRC \
    	RGDS=AML_PAERAH_73407_RNAseq \
    	RGDT=2012-12-22
    I found some threads discussing these problems, but couldn't find a solution there. Can anyone suggest a solution? The full error message is pasted below.

    Thank you.

    Eric


    Code:
    [efoss@gizmod64:~/sequencing/MAE_in_AML$ java -d64 -Xmx25g -jar /home/efoss/source_codes/picard-tools-1.82/AddOrReplaceReadGroups.jar \
    >         INPUT=AML_PAERAH_73407_RNA_partialperlclean_noMT.sam \
    >         OUTPUT=AML_PAERAH_73407_RNA_partialperlclean_noMT_AORRGaddedsorted.sam \
    >         SORT_ORDER=coordinate \
    >         RGID=RGID_PAERAH_73407 \
    >         RGLB=PAERAH_73407_library \
    >         RGPL=Illumina \
    >         RGPU=run_barcode_arbitrary \
    >         RGSM=AML_PAERAH_73407_RNAseq \
    >         RGCN=FHCRC \
    >         RGDS=AML_PAERAH_73407_RNAseq \
    >         RGDT=2012-12-22
    [Mon Dec 24 12:29:53 PST 2012] net.sf.picard.sam.AddOrReplaceReadGroups INPUT=AML_PAERAH_73407_RNA_partialperlclean_noMT.sam OUTPUT=AML_PAERAH_73407_RNA_partialperlclean_noMT_AORRGaddedsorted.sam SORT_ORDER=coordinate RGID=RGID_PAERAH_73407 RGLB=PAERAH_73407_library RGPL=Illumina RGPU=run_barcode_arbitrary RGSM=AML_PAERAH_73407_RNAseq RGCN=FHCRC RGDS=AML_PAERAH_73407_RNAseq RGDT=2012-12-21T16:00:00-0800    VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false
    [Mon Dec 24 12:29:53 PST 2012] Executing as efoss@gizmod64 on Linux 3.2.0-24-generic amd64; OpenJDK 64-Bit Server VM 1.6.0_24-b24; Picard version: 1.82(1311)
    INFO    2012-12-24 12:29:53     AddOrReplaceReadGroups  Created read group ID=RGID_PAERAH_73407 PL=Illumina LB=PAERAH_73407_library SM=AML_PAERAH_73407_RNAseq
    
    [Mon Dec 24 12:30:01 PST 2012] net.sf.picard.sam.AddOrReplaceReadGroups done. Elapsed time: 0.13 minutes.
    Runtime.totalMemory()=1910898688
    FAQ:  [url]http://sourceforge.net/apps/mediawiki/picard/index.php?title=Main_Page[/url]
    Exception in thread "main" net.sf.samtools.util.RuntimeIOException: java.io.IOException: No space left on device
            at net.sf.samtools.util.SortingCollection.spillToDisk(SortingCollection.java:228)
            at net.sf.samtools.util.SortingCollection.add(SortingCollection.java:150)
            at net.sf.samtools.SAMFileWriterImpl.addAlignment(SAMFileWriterImpl.java:170)
            at net.sf.picard.sam.AddOrReplaceReadGroups.doWork(AddOrReplaceReadGroups.java:100)
            at net.sf.picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:177)
            at net.sf.picard.cmdline.CommandLineProgram.instanceMainWithExit(CommandLineProgram.java:119)
            at net.sf.picard.sam.AddOrReplaceReadGroups.main(AddOrReplaceReadGroups.java:66)
    Caused by: java.io.IOException: No space left on device
            at java.io.FileOutputStream.write(Native Method)
            at org.xerial.snappy.SnappyOutputStream.writeInt(SnappyOutputStream.java:105)
            at org.xerial.snappy.SnappyOutputStream.dump(SnappyOutputStream.java:126)
            at org.xerial.snappy.SnappyOutputStream.flush(SnappyOutputStream.java:100)
            at org.xerial.snappy.SnappyOutputStream.close(SnappyOutputStream.java:137)
            at net.sf.samtools.util.SortingCollection.spillToDisk(SortingCollection.java:219)
            ... 6 more
    efoss@gizmod64:~/sequencing/MAE_in_AML$

  • #2
    I've had issues when sorting if my tmp space fills up. Did you try specifying the tmp dir?

    Comment


    • #3
      Originally posted by aaronh View Post
      I've had issues when sorting if my tmp space fills up. Did you try specifying the tmp dir?
      Hi aaronh,

      I'm afraid I don't know what the tmp dir is. Googling around, it looks like it's a temporary directory used for 'scratch' space. How can I specify it?

      Since posting this, I ended up finding these two links.

      Hudson is repeatedly failing after building few projects with following stack trace with error "No space left on device", though there is enough space on disk. There are no limits of quotas on any ...


      One of our development servers went down today. Problems started with deployment script that claimed that claimed “No space left on device”, although partition was not nearly full. If you ever run into such trouble – most likely you have too many small or 0-sized files on your disk, and while you have enough disk […]


      I couldn't get the solution in the second link to work, but what I read here suggested that there was not a lack of disk space, as one might think from the error message, but instead a shortage of 'i-nodes' (don't know anything about those either). That suggested to me that using a more powerful system might fix things. I was running my commands on 12 cores with 48 gigs of ram. I switched to a more powerful linux server and then everything worked fine. So that's good, but I still would like to understand what is going on here. Also, use of the more powerful server for tasks that can take a long time (> 20 minutes) is discouraged, so my solution to use it is suboptimal.

      Thanks.

      Eric

      Comment


      • #4
        Try using java -Djava.io.tmpdir=/path/to/tmpdir -jar Add...
        I've used the TMP environment variable to control things like where gcc writes it's temporary files, but I can't seem to find an equivalent for java's createTempFile API. Does such an environment

        Some linux servers have a dedicated tmp partition that can fill up with all the little files that the sorting creates before merging. You can see if your server has this by using the 'df' command and also see where you default tmp space is by using the 'env' command.

        Comment


        • #5
          Hi aaronh,

          Thanks very much for the help. I really appreciate it.

          Best wishes,

          Eric

          Comment

          Latest Articles

          Collapse

          • seqadmin
            Strategies for Sequencing Challenging Samples
            by seqadmin


            Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
            03-22-2024, 06:39 AM
          • seqadmin
            Techniques and Challenges in Conservation Genomics
            by seqadmin



            The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

            Avian Conservation
            Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
            03-08-2024, 10:41 AM

          ad_right_rmr

          Collapse

          News

          Collapse

          Topics Statistics Last Post
          Started by seqadmin, Yesterday, 06:37 PM
          0 responses
          8 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, Yesterday, 06:07 PM
          0 responses
          8 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 03-22-2024, 10:03 AM
          0 responses
          49 views
          0 likes
          Last Post seqadmin  
          Started by seqadmin, 03-21-2024, 07:32 AM
          0 responses
          67 views
          0 likes
          Last Post seqadmin  
          Working...
          X