Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Finding optimal reads generated on SOLiD:

    Hi,

    We're having some issues with the data coming off the SOLiD. Our people are unable to tell me which primary.* files to use and the 'latest' primary.* file doesn't always have the 'latest' write-to time stamp (unix time attribute) off the machine- nor is it the best quality. Similarly, the primary.* directories may differ across the run from sample to sample and so one sample might use primary.20101016etc on one sample and another sample of the same exp/lib/run might use primary.20101014 and both are present in each directory. These are all attributable to doing reanalysis mid-seq during a run.

    To make matters worse, the way the files are being copied is creating an entire system/network of directories that I have to go through to find the proper primary.* file for mapping. Using the wrong one results in very poor mapping. Could someone shed some light on a manual to look through or a way to ID the best-run and best primary* file to use off a run? Finally, is there a way to name these directories prior to export off the SOLiD? Thanks.

    J

  • #2
    If you disable the auto-export and export them manually using scp you can rename the files when you copy them. This doesn't solve the problem of finding which primary.* to use, though in our case it's always been the one with the largest number (most recent timestamp).

    Comment


    • #3
      Thanks, mrawlins. Maybe someone can also shed some light on the primary* issue...

      Comment


      • #4
        Sorry not quite getting you.
        But if you mean how to set a rule to find the proper F3.csfasta files to use.
        then yes I know wat you mean.

        I usually do a
        find . -iname *.stats

        only the correct reads dir with a .stats file contains the F3 that I need.

        I can't understand why they can't put the timestamp in the filename instead of the directory as well.
        such a pain
        http://kevin-gattaca.blogspot.com/

        Comment


        • #5
          Originally posted by KevinLam View Post
          I can't understand why they can't put the timestamp in the filename instead of the directory as well.
          such a pain
          Perhaps because the processing software would then have to deal with different names for F3 and R3 (or F3/F5) files. In other words, currently the files look something like:

          primary.date1/reads/run_F3_sample.csfasta
          primary.date2/reads/run_R3_sample.csfasta

          Or however you have your experiment set up. The point is that the file names (not the directories) look similar thus making it easy to switch files in and out of the pipeline. Whereas with file stamps in the file names it would look like:

          primary.date1/reads/run_F3_sample_date1.csfasta
          primary.date2/reads/run_R3_sample_date2.csfasta

          Not much of a change but perhaps something that Bioscope does not want to handle. Although, granted, making the individual files a lot more trackable.

          Anyway the above is just being "the devil's advocate". Getting back to the original question:

          Our people are unable to tell me which primary.* files to use and the 'latest' primary.* file doesn't always have the 'latest' write-to time stamp (unix time attribute) off the machine- nor is it the best quality.
          The only time you should get multiple primary.* directories for the same primer is when you doing, as you say, a reanalysis mid-run. In this case the primary.* directory with the latest date in its name should be the reanalysis one ... which is presumably the analysis you want. If it turns out to not be the one you want (e.g., because of lower quality scores) then use a previous one.

          In our (Purdue Genomics) case the only time we do a reanalysis mid-run is when something is screwed up -- i.e., we are desperately trying to save a run from a total meltdown. Fortunately the SOLiD technology is really good at being able redo parts of a run and extract data from a potentially failed run. However there have been cases where we redid a partial part of run, did not like that, re-did that part again, and ended up with even worse results. At that point we had a jumble of primary.* directories from which to choose the best of many poor results. You may be in a similar situation -- i.e., none of the results will be that great. But one of "your people" should be able to say, via looking at the heat maps and the cycle scans, which primary.* has the best chance of containing good data.

          I know that the above doesn't really answer your question "... [a] way to ID the best-run and best primary* file to use ..." but this is in part because any time a reanalysis is done then implies that there is likely to be no "best". You have a non-typical case and thus are charting a non-typical path.

          Comment


          • #6
            Originally posted by westerman View Post
            Perhaps because the processing software would then have to deal with different names for F3 and R3 (or F3/F5) files. In other words, currently the files look something like:

            primary.date1/reads/run_F3_sample.csfasta
            primary.date2/reads/run_R3_sample.csfasta

            Or however you have your experiment set up. The point is that the file names (not the directories) look similar thus making it easy to switch files in and out of the pipeline. Whereas with file stamps in the file names it would look like:

            primary.date1/reads/run_F3_sample_date1.csfasta
            primary.date2/reads/run_R3_sample_date2.csfasta

            Not much of a change but perhaps something that Bioscope does not want to handle. Although, granted, making the individual files a lot more trackable.

            Anyway the above is just being "the devil's advocate". Getting back to the original question:



            The only time you should get multiple primary.* directories for the same primer is when you doing, as you say, a reanalysis mid-run. In this case the primary.* directory with the latest date in its name should be the reanalysis one ... which is presumably the analysis you want. If it turns out to not be the one you want (e.g., because of lower quality scores) then use a previous one.

            In our (Purdue Genomics) case the only time we do a reanalysis mid-run is when something is screwed up -- i.e., we are desperately trying to save a run from a total meltdown. Fortunately the SOLiD technology is really good at being able redo parts of a run and extract data from a potentially failed run. However there have been cases where we redid a partial part of run, did not like that, re-did that part again, and ended up with even worse results. At that point we had a jumble of primary.* directories from which to choose the best of many poor results. You may be in a similar situation -- i.e., none of the results will be that great. But one of "your people" should be able to say, via looking at the heat maps and the cycle scans, which primary.* has the best chance of containing good data.

            I know that the above doesn't really answer your question "... [a] way to ID the best-run and best primary* file to use ..." but this is in part because any time a reanalysis is done then implies that there is likely to be no "best". You have a non-typical case and thus are charting a non-typical path.
            Thanks, Westernman. That's my very issue. I can't tell which primary* directory is the right one and there's no real way for me to tell unless there is no reanalysis done and only one directory to choose from. I've had cases where there were reanalyses and the latest was invalid as you said, which resulted in terrible mapping results and a waste of my time and our cluster's time. I'll talk to my SOLiD tech to try and figure out a way to analyze the heat maps and cycle scans and pass that along to our technician. That has to be the solution. Thanks a lot!

            Comment

            Latest Articles

            Collapse

            • seqadmin
              Strategies for Sequencing Challenging Samples
              by seqadmin


              Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
              03-22-2024, 06:39 AM
            • seqadmin
              Techniques and Challenges in Conservation Genomics
              by seqadmin



              The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

              Avian Conservation
              Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
              03-08-2024, 10:41 AM

            ad_right_rmr

            Collapse

            News

            Collapse

            Topics Statistics Last Post
            Started by seqadmin, Yesterday, 06:37 PM
            0 responses
            8 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, Yesterday, 06:07 PM
            0 responses
            8 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 03-22-2024, 10:03 AM
            0 responses
            49 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 03-21-2024, 07:32 AM
            0 responses
            67 views
            0 likes
            Last Post seqadmin  
            Working...
            X