Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • SCS2.6_RTA1.6 issues

    Hello:

    Has anyone had problems with the new RTA software? We have been pleased with the analysis (the new image analysis is really spectacular), but have seen a variety of strange phenomenon on several different Dells. Some of the strange things we have seen are long delays in network transfer during the run (causing the run to stop itself when the D drive is full), delays in analysis and data transfer at the beginning of the run (ie, no analysis or transfer until cycle 7 or 8), and in general random behavior of when the first extracted/called/scored bases come up. In general everything has come out OK but it's a little unsettling. I've sent dozens of logs to illumina with not much insight in return, and now am just trying to get a sense of how widespread this is. Any info or related war stories are appreciated!

    Charlie

  • #2
    Charlie,

    We have experienced exactly those problems with delayed transfer & paused runs on our latest run. I contacted Illumina and they tried to pass it off as problems with our network. The instrument we are having trouble with was originally delivered as a 1G Genome Analyzer (i.e. the original model) but has been upgraded twice now to make it a GA IIx. However it still has the older Dell 690 workstation with a 930 GB D: drive array. Our second instrument, delivered as a IIx has a Dell T7500 with a 1.8 TB D: drive. Which type of workstation do you have on your instrument?

    Another issue may be how dense your clusters are. The run we are having trouble with has cluster densities of ~680,000/mm^2. There are known issues with RTA 1.6 at cluster densities higher than 700,000/mm^2 (as in crashing the system entirely). What are your cluster densities like?

    Here are some other things I noticed:

    While it is true that the copying of images to the network was slower while the run was going on, once RTA had finished processing read1 the copy rate increased dramatically, to the rate normally seen before.

    Images, even after being copied to the network location aren't being deleted from the local drive in a timely fashion.

    Illumina has dumped robocopy for transfer and deletion of images; this function is now handled by RTA as well. It looks to me like all this processing is pushing the workstation beyond its limits.

    Any other users experiencing this? If so what model Dell workstation do you have and what types of cluster densities were involved?

    Comment


    • #3
      Thanks for your response, it's always comforting to know one is not alone in these tribulations! All our workstations are Dell 690's with the 930 TB drives, and one is pretty old, about 2.5 years. The one having the most problems actually is a computer we got directly from the sequencing factory in Hayward, supposedly all road tested and functional.

      Some of our lanes are pretty dense, maybe this is part of it but so far no indication from tech support that this could be a problem. We have been getting decent data from 380-400 K clusters/tile (old notation!) with the new software, it would be a shame if we couldn't routinely exploit that density.

      In at least one of the cases, completion of the run and analysis did not speed up the transfer, which lagged for several days.

      Like you, the first comment I usually get from tech support is it must be the network, or maybe the cable that has been working for 2 years suddenly is having problems.

      I do think Illumina recognizes this, so maybe a fix is on the way. But I also think that it's pushing the Dell's to their computational limit and so may never resolve completely.

      Comment


      • #4
        Charlie;

        Long time no see. I think I might have a few words of advice on the RTA2.6/computer issues.

        First off, the new software pushes the computer really hard, especially at high densities. A few configuration changes on the Dell can usually help quite a bit. First off, I turn off indexing on drive D: which really seems to help. I also make sure to turn off the recycle bin for drive D and disable system restore which speeds things up quite a bit. If your FSE hasn't already done it, make sure that the raid card firmware and drivers are up to the latest as well. If you have any anti-virus software defiantly get rid of it as well as any other processes that shouldn't be running on the PC.

        I also nag on the technicians to keep the D drive empty. Shouldn't matter but if it fills up the run will stop. In your case I'd double check the system logs as well to make sure the computer isn't throwing any errors during the run. A few RAID errors from a failing drive can really slow things down. Other than that, make sure your network to the analysis servers is in good shape and the analysis servers can take data on the samba share even while running an analysis.
        Hope this helps,

        Brad

        Comment


        • #5
          Originally posted by cnicolet View Post
          I do think Illumina recognizes this, so maybe a fix is on the way. But I also think that it's pushing the Dell's to their computational limit and so may never resolve completely.
          Maybe it's time bring IPAR back. Mine's sitting here gathering dust.

          Comment


          • #6
            Hi basickler:
            Nice to hear from you. As usual, you have intelligent helpful things to say. I've just done everything you suggested, all those WERE enabled. It will be very interesting to see if that helps, but at least it's something constructive to try. Why aren't the people in tech support giving this advice? Sounds a lot better than "check your cable" or "check your network." One more question, I have two runs going now (we have 3 machines!), I'm assuming I should not change any of those settings until the runs are complete. Is that correct? Thanks again!

            Charlie

            Comment


            • #7
              It shouldn't matter if you change these while the runs are going but I'd probably wait until RTA isn't doing anything or the run is over to turn off indexing on D:. It takes a while because it has to recursively go through all the files in D. In fact you probably want to wait until the computer is idle, clear out D:\Runs then make the change.
              If you need to uninstall anything, definately wait.

              Brad
              Last edited by basickler; 03-18-2010, 09:57 AM.

              Comment


              • #8
                Thanks, I'll just wait. Meanwhile I'll keep my fingers crossed and hope for the best--my tried and true facility strategy!

                Comment


                • #9
                  Hi Charlie et al,
                  We ran some titration flowcells when we upgraded and have found that although we can get crazily high cluster densitites that the actual yield stay pretty flat, PF rate goes down as cpt goes up.
                  I see this as giving us more leeway in getting quantitation correct as now a very wide range of concentration gives the same yield.
                  Anyone else run titrations of the same sample?
                  James.

                  Comment


                  • #10
                    I am so happy I found this post. We are having the exact same problems; runs pausing due to lack of space on the D: drive. Tech support has blamed our network multiple times over the phone and through email and has closed our open cases. It is so frustrating that they never mentioned that other sites are experiencing the exact same issue and that the problem is software related and NOT my network. Big thumbs down to Illumina tech support for blaming my network that has worked flawlessly for the past 2 years and not offering even the slightest hint that it could be software related.

                    Like the post above, maybe hooking up the IPAR paperweight collecting dust in the corner may help expand the local hard drive space...

                    Comment


                    • #11
                      We've also experienced some pausing and even once a full system crash due to this. I'm also glad to see others are experiencing it. We have dell precision 690s.

                      Comment


                      • #12
                        James;

                        I've found that RTA 1.6 PF flattens out at around 260,000-280,000 clusters/tile. If you really push it you can get higher than that, but at a hit in quality. We've been shooting for a max cluster density of 300,000 for high quality runs and we usually get 45-50GB of sequence at that density in a 2x100 run.

                        Brad

                        Comment


                        • #13
                          I know this doesn't solve the problem, but as a safe-guard, you can always upgrade the drives in the Dell to 2TB each. We're about to do this just in case we have network issues. Our FAS tells us that the instrument is safe to sit in 'paused' mode for several days until the network is up/clear, but I'd rather not risk it again... espeically given that the upgrade cost is only around AU$800.

                          Comment


                          • #14
                            Originally posted by basickler View Post
                            James;

                            I've found that RTA 1.6 PF flattens out at around 260,000-280,000 clusters/tile. If you really push it you can get higher than that, but at a hit in quality. We've been shooting for a max cluster density of 300,000 for high quality runs and we usually get 45-50GB of sequence at that density in a 2x100 run.

                            Brad
                            is it your impression that this decrease in quality and "PF flattening out" is a optics problem or a software problem? I'm wondering how far the user community feels that they can push their current machines.

                            Comment


                            • #15
                              We had the same issues even with previous versions of the software (BTW, 1.6 is GApipeline version not RTA version. Latest RTA version is 2.5, I think).

                              But at that time we were saving images per each Run (temporarily). After dropping images we haven't had any more issues.

                              We used perfmon to monitor what the machine was doing and we saw the IO against disk was extremely high.
                              -drd

                              Comment

                              Latest Articles

                              Collapse

                              • seqadmin
                                Strategies for Sequencing Challenging Samples
                                by seqadmin


                                Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
                                03-22-2024, 06:39 AM
                              • seqadmin
                                Techniques and Challenges in Conservation Genomics
                                by seqadmin



                                The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

                                Avian Conservation
                                Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
                                03-08-2024, 10:41 AM

                              ad_right_rmr

                              Collapse

                              News

                              Collapse

                              Topics Statistics Last Post
                              Started by seqadmin, 03-27-2024, 06:37 PM
                              0 responses
                              13 views
                              0 likes
                              Last Post seqadmin  
                              Started by seqadmin, 03-27-2024, 06:07 PM
                              0 responses
                              11 views
                              0 likes
                              Last Post seqadmin  
                              Started by seqadmin, 03-22-2024, 10:03 AM
                              0 responses
                              53 views
                              0 likes
                              Last Post seqadmin  
                              Started by seqadmin, 03-21-2024, 07:32 AM
                              0 responses
                              69 views
                              0 likes
                              Last Post seqadmin  
                              Working...
                              X