Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Google Cloud Platform

    Hey guys!

    I was asked to test a little this Google Cloud Platform. I'm just login to free trial, but I would like to have opinions of people who have already used it.
    Is it like amazon cloud, where you can by processors and memory ram depending on the algorithms you need to use?
    Does it has programs installed? Pipelines? Applications?
    If you have a computer cluster where you can install everything you need to assembly and annotated your eukaryotic genome, is there any advantage in the cloud at all?

    Thank you so much for your time and feedback!

  • #2
    Originally posted by Marcela Uliano View Post
    Hey guys!
    If you have a computer cluster where you can install everything you need to assembly and annotated your eukaryotic genome, is there any advantage in the cloud at all?

    Thank you so much for your time and feedback!
    You are likely to stumble on a project that cannot be handled by the local cluster ("planet scale" computing, think thousands of cores for 100,000 genomes project). That is when something like google genomics would come in handy. Google has done some interesting work with 1000 genomes data but I can't find a public source that describes the results. There is a brief mention here: https://www.genomeweb.com/informatic...isanal-factory

    Comment


    • #3
      Some thoughts ..
      #1 I still really like Big Servers and nearby beowulf style computing.
      Typically there's a pipe and that pipe is going to limit what you can do.
      If that pipe is always saturated by transferring terabytes back and forth from the cloud, things are going to slow down.
      Being able to launch 500 virtual cloud machines isn't really solving the bandwidth problem. Not if 25 other people using the same pipe are doing the same thing.

      #2 The internet is bursty. If you enjoy the "circling doughnut" or "please wait ... buffering", then you're ready for the cloud.

      #3 This is going to be very expensive.

      #4. Google's omnivision is getting a little scary. It's getting to be like ... Jeremy Bentham's Panoption ... or like George Orwell's 1984 where TV watches you. I just don't trust the brand anymore. Clever marketing is not going to fix this today.

      #5. There's liability and privacy concerns using patient data. The cloud is a real problem.

      Someday, somebody is going to get "cloud" right.
      Last edited by Richard Finney; 11-24-2014, 09:13 AM.

      Comment


      • #4
        I am only going to comment on technical thoughts mentioned in Richard's post.

        1. Presumably anyone wanting to do a true "planet-scale" project would have a direct peering connection with Google and/or a major internet backbone provider. For less ambitious projects/normal users, the bottleneck will likely be the connection on the user-end (rather than Google's).

        2. Google (in all likelihood, I don't know for sure) does not use public internet for internal connectivity so once the data is in the Google cloud the circling doughnut may actually indicate a running process on a large scale

        3. Costs are advertised here: https://cloud.google.com/genomics/pricing. $22 per TB per month for storage.

        5. Privacy and liability are issues that the cloud providers are coming to grips with. There are mechanisms to sign agreements that would cover this aspect (http://googlecloudplatform.blogspot....-entities.html). Since local legal counsel will be involved in setting up such an agreement I doubt Google would use/access user data beyond terms spelled out in the BAA.

        Comment


        • #5
          Originally posted by Richard Finney View Post
          Some thoughts ..
          .

          #5. There's liability and privacy concerns using patient data. The cloud is a real problem.
          Get over it already. How's your credit card doing these days at Target?

          Comment


          • #6
            Bumping this thread.

            Anybody used google cloud genomics?

            Any thoughts?

            Is Amazon or Microsoft doing any genomics cloud projects?

            Comment


            • #7
              Some folks (I know locally) have been using Google compute very successfully and they are extremely happy about the possibilities. It is not inexpensive but if one keeps things in perspective (e.g. where else can you find resources to analyze 5000 RNAseq samples in a day or so) then the possibilities are only limited by available budget.

              Google cloud storage is also compelling. Looking at using it for data backup (BTW: Veritas NetBackup can use google cloud storage as a target).

              Comment


              • #8
                Well. I am very hazy on this so bear with me this is a naive question ...

                This not trivial and would demonstrate a computationally big, practical test case ...

                I guess the question(s) I have are ... if the API interface is to objects , then are the methods to access these objects user definable?
                Can I for instance call a method to realign a bwa aligned hg19 bam files to a hg38 novoalign bam file? User must, of course, do the work of gluing programs together to do this : oldbam->fastq->novalaign->newbam . Can theses sorts of programs be run out "in the cloud" ?

                ... and ...

                Is docker or other virtual machine images usable in the google cloud?
                Last edited by Richard Finney; 03-10-2016, 10:34 AM.

                Comment


                • #9
                  I referred to "Google compute" and not "Google genomics" in my post, even though you were asking about google genomics.

                  Since the workflows are specific to sites we are using our own (not the default on genomics) via google compute with VM's.

                  Docker containers are usable on google compute. There is a nice container management system (kubernetes) available.

                  Comment


                  • #10
                    I happen to be a Google employee now. I do not work directly with the cloud or genomics groups, but I would be happy to relay questions, and will post the responses here.

                    Comment


                    • #11
                      Originally posted by Brian Bushnell View Post
                      I happen to be a Google employee now. I do not work directly with the cloud or genomics groups, but I would be happy to relay questions, and will post the responses here.
                      I wonder if you or someone else will be maintaining BBTools.

                      Comment


                      • #12
                        I'm still maintaining it on weekends

                        Comment

                        Latest Articles

                        Collapse

                        • seqadmin
                          Strategies for Sequencing Challenging Samples
                          by seqadmin


                          Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
                          03-22-2024, 06:39 AM
                        • seqadmin
                          Techniques and Challenges in Conservation Genomics
                          by seqadmin



                          The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

                          Avian Conservation
                          Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
                          03-08-2024, 10:41 AM

                        ad_right_rmr

                        Collapse

                        News

                        Collapse

                        Topics Statistics Last Post
                        Started by seqadmin, Yesterday, 06:37 PM
                        0 responses
                        8 views
                        0 likes
                        Last Post seqadmin  
                        Started by seqadmin, Yesterday, 06:07 PM
                        0 responses
                        8 views
                        0 likes
                        Last Post seqadmin  
                        Started by seqadmin, 03-22-2024, 10:03 AM
                        0 responses
                        49 views
                        0 likes
                        Last Post seqadmin  
                        Started by seqadmin, 03-21-2024, 07:32 AM
                        0 responses
                        66 views
                        0 likes
                        Last Post seqadmin  
                        Working...
                        X