Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by rufessor View Post
    I may be answering in part my question- or at least clarifying my question.
    I am no longer certain that the sleeping process list actually really effectively did anything to the cache- its almost always used to capacity by linux so I guess my question is-

    what the heck are those 40 + sleeping processes doing- and are they effectively actually consuming any resource whatsoever. Do they really hold ram or is it just cached...

    Would be curious.
    Most modern linux distros manage memory internally so one can't depend on output of htop/top alone. With a TB of RAM you should have no worries about memory consumption

    See: http://www.linuxatemyram.com/

    Comment


    • Hey everyone,

      I've used trimmomatic to clean up my reads but when I use the cleaned files in tophat, I get an error. When I examined a bit closer I discovered that trimmomatic converted this:

      @D3VDZHS1:119:H036PADXX:1:1202:12533:34018 2:N:0:GGACTCCTTAGATCGC
      ATAGACAAATGCCTGCAACAACGCAGGGATCTCTTTCCCGGTAAACCAACCGTCGTCATTGAAGATATGCATGCTGGCTCGGGTATCCCATTGCTGATAC
      +
      @@CADDFFBBFHHJIGIJJIIJIBDDGE@ABGEHGGGIJIGF:CFFHGIJG<?8ADBDB@AC;.;>A>>@>:@ACC@@?C<BB(0>(:@(4::@(+4>A>
      @D3VDZHS1:119:H036PADXX:1:1202:12611:34155 2:N:0:GGACTCCTTAGATCGC
      GGTATCAACGCAGAGTACTTTTTTTTTTTCTTTTTTTTTTTTTTTTTTAAAGGAAAACCAGACAAATCATGAAGCCACATACGCTAGAGAAGCTCAATAC
      +
      B@@DFFDFHHGHHGIEHHIJIIJJJJJJI)BFGG):BCDDDDDDDDB#####################################################
      @D3VDZHS1:119:H036PADXX:1:1202:12731:34205 2:N:0:GGACTCCTTAGATCGC
      TAATAAATCCGCTACCGACGCTGACTAACATTTCGCGATCGTTCATCGCATCACCAAAGGCCGTGCAATCGCGCAACGATAAACCTAAATGTTGGGTCAG
      +
      @@CFFFFFHHHHHIJJIIJIIIJJJGIIJJJJJIIJJBAHG@GHEHFGF=BCEEEC?AC?AAB8888@CC??>BBDD>BBBBCDDAACDCDDC@8<?CBC


      TO:

      @D3VDZHS1:119:H036PADXX:1:1202:12533:34018 2:N:0:GGACTCCTTAGATCGC
      ATAGACAAATGCCTGCAACAACGCAGGGATCTCTTTCCCGGTAAACCAACCGTCGTCATTGAAGATATGCATGCTGGCTCGGGTAT+
      @@CADDFFBBFHHJIGIJJIIJIBDDGE@ABGEHGGGIJIGF:CFFHGIJG<?8ADBDB@AC;.;>A>>@>:@ACC@@?C<BB(0>
      @D3VDZHS1:119:H036PADXX:1:1202:12731:34205 2:N:0:GGACTCCTTAGATCGC
      TAATAAATCCGCTACCGACGCTGACTAACATTTCGCGATCGTTCATCGCATCACCAAAGGCCGTGCAATCGCGCAACGATAAACCTAAATGTTGGGTCAG
      +
      @@CFFFFFHHHHHIJJIIJIIIJJJGIIJJJJJIIJJBAHG@GHEHFGF=BCEEEC?AC?AAB8888@CC??>BBDD>BBBBCDDAACDCDDC@8<?CBC

      Obviously trimmomatic didn't put + on the next line and now tophat can't read the line properly. This has happened in multiple files. Does anyone know if a) this is normal for trimmomatic and I need to fix this manually or if b) I did something wrong to cause it?

      My input code:

      Code:
      java -jar /path/to/Trimmomatic-0.32/trimmomatic-0.32.jar PE -threads 8 -phred33 -trimlog Sample1trimlog sample1_R1.fastq sample1_R2.fastq sample1_R1_TP.fastq sample1_R1_TU.fastq sample1_R2_TP.fastq sample1_R2_TU.fastq ILLUMINACLIP:/path/to/Trimmomatic-0.32/adapters/adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:36
      Thanks for your help!

      Comment


      • Originally posted by travelk View Post
        Obviously trimmomatic didn't put + on the next line and now tophat can't read the line properly. This has happened in multiple files. Does anyone know if a) this is normal for trimmomatic and I need to fix this manually or if b) I did something wrong to cause it?
        a) No. Not normal.

        b) Probably. But your command line looks ok and nothing obvious is popping up.

        Comment


        • Originally posted by travelk View Post
          Obviously trimmomatic didn't put + on the next line and now tophat can't read the line properly. This has happened in multiple files. Does anyone know if a) this is normal for trimmomatic and I need to fix this manually or if b) I did something wrong to cause it?
          It is certainly not normal - it's one of the strangest trimmomatic issue i have heard of. None the less, it should be possible to track down. Some questions:

          1) Does this happen consistently on the same lines of the same files if they are run more than once?
          1.1) If so, can you isolate and send me a short example where it happens?

          2) What OS are you using?

          Thanks,

          Tony.

          Comment


          • Ok, I re-ran the files using the exact same script on the exact same files as suggested to check if it happened consistently on the same line... but now there is no problem with the new output files and tophat runs them just fine. So, I'm not sure what I did the first time around to have the files corrupt like that.

            Thank you nevertheless for taking the time to help me!

            Comment


            • Trimmomatic is not working correctly in paired end mode:

              Read names in output files are not in the correct order. Correct phase was lost at read #27 and there are additional phase changes thereafter. Command line was as follows:

              java -jar /opt/Trimmomatic-0.32/trimmomatic-0.32.jar PE -threads -phred33 -trimlog Trim_Lesion6.txt 6_S1_L001_R1_001_clip2.fastq 6_S1_L001_R2_001_clip2.fastq 6_S1_L001_R1_001_paired.fastq 6_S1_L001_R1_001_unpaired.fastq 6_S1_L001_R2_001_paired.fastq 6_S1_L001_R2_001_unpaired.fastq MINLEN:40

              Comment


              • The command seems OK, except that you haven't specified the number of threads.

                Does trimmomatic give any error messages?

                Is there something wrong with the format of read 27 in one of your files
                that causes it to be read incorrectly?

                Do your 2 input files have the same number of reads, in the same order?

                What does the trimmomatic log file indicate is happening when the output files get out of phase?

                Comment


                • Originally posted by mastal View Post

                  Do your 2 input files have the same number of reads, in the same order?
                  This was the problem. Interesting, there was, to my knowledge, no stipulation in the original publication, or in the online manual, that the program requires input files with perfectly paired reads. Maybe I'm stupid but I would have thought that a program that is designed to take raw reads, quality trim them and sort them in to paired and unpaired datasets would realize that raw data coming off a sequencing machine often has large numbers of unpaired reads to start off with. As it stands, it appears I have to run one script to cull unpaired mates and then run Trimmomatic. How inefficient is that?

                  Comment


                  • Originally posted by drdna View Post
                    Maybe I'm stupid but I would have thought that a program that is designed to take raw reads, quality trim them and sort them in to paired and unpaired datasets would realize that raw data coming off a sequencing machine often has large numbers of unpaired reads to start off with. As it stands, it appears I have to run one script to cull unpaired mates and then run Trimmomatic. How inefficient is that?
                    Inefficient indeed. But I want to know what type of machine you have that has large numbers of unpaired reads? My miSeqs and hiSeqs always pair reads -- assuming that I tell them that the project is paired.

                    Comment


                    • We have a MiSeq which gives us scads of high quality data but almost never gives us perfectly paired reads. I'll have to check with our tech to see if she sets any kind of paired data flag. Where would that be?

                      Comment


                      • Settings would be in the sample sheet.

                        Are you getting the reads directly from the sequencer or via BaseSpace? The latter may be doing some sort of trimming for you. Because we have a hiSeq and pre-existing pipelines we do not use BaseSpace but rather just grab the raw reads. Thus I am not familiar with BaseSpace but I do know that it can do a lot of useful processing.

                        Comment


                        • Only way I would see that happen is you get consistently bad quality reads on one end that are removed by MiSeq reporter/BaseSpace. Perhaps you should ask the tech to turn off on-instrument analysis (adapter trimming etc) and you can do that offline.

                          Comment


                          • That's probably it - we use BaseSpace. My guess is that BaseSpace is filtering out pairs with poor quality. I'll have to look into that.

                            Comment


                            • Thanks for the insights. BaseSpace does a good job of adaptor trimming, demultiplexing. It's probably more efficient to just run the reads through a script to pair them up properly before downstream processing.

                              Comment


                              • repair.sh from BBTools will do that easily: http://seqanswers.com/forums/showpos...8&postcount=61

                                Comment

                                Latest Articles

                                Collapse

                                • seqadmin
                                  Techniques and Challenges in Conservation Genomics
                                  by seqadmin



                                  The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

                                  Avian Conservation
                                  Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
                                  03-08-2024, 10:41 AM
                                • seqadmin
                                  The Impact of AI in Genomic Medicine
                                  by seqadmin



                                  Artificial intelligence (AI) has evolved from a futuristic vision to a mainstream technology, highlighted by the introduction of tools like OpenAI's ChatGPT and Google's Gemini. In recent years, AI has become increasingly integrated into the field of genomics. This integration has enabled new scientific discoveries while simultaneously raising important ethical questions1. Interviews with two researchers at the center of this intersection provide insightful perspectives into...
                                  02-26-2024, 02:07 PM

                                ad_right_rmr

                                Collapse

                                News

                                Collapse

                                Topics Statistics Last Post
                                Started by seqadmin, 03-14-2024, 06:13 AM
                                0 responses
                                33 views
                                0 likes
                                Last Post seqadmin  
                                Started by seqadmin, 03-08-2024, 08:03 AM
                                0 responses
                                72 views
                                0 likes
                                Last Post seqadmin  
                                Started by seqadmin, 03-07-2024, 08:13 AM
                                0 responses
                                80 views
                                0 likes
                                Last Post seqadmin  
                                Started by seqadmin, 03-06-2024, 09:51 AM
                                0 responses
                                68 views
                                0 likes
                                Last Post seqadmin  
                                Working...
                                X