Google Cloud Platform

Marcela Uliano · 11-24-2014, 03:20 AM

Hey guys!

I was asked to test a little this Google Cloud Platform. I'm just login to free trial, but I would like to have opinions of people who have already used it.
Is it like amazon cloud, where you can by processors and memory ram depending on the algorithms you need to use?
Does it has programs installed? Pipelines? Applications?
If you have a computer cluster where you can install everything you need to assembly and annotated your eukaryotic genome, is there any advantage in the cloud at all?

Thank you so much for your time and feedback!

GenoMax · 11-24-2014, 04:11 AM

Quote:

Originally Posted by Marcela Uliano

Hey guys!
If you have a computer cluster where you can install everything you need to assembly and annotated your eukaryotic genome, is there any advantage in the cloud at all?

Thank you so much for your time and feedback!

You are likely to stumble on a project that cannot be handled by the local cluster ("planet scale" computing, think thousands of cores for 100,000 genomes project). That is when something like google genomics would come in handy. Google has done some interesting work with 1000 genomes data but I can't find a public source that describes the results. There is a brief mention here: https://www.genomeweb.com/informatic...isanal-factory

Richard Finney · 11-24-2014, 08:03 AM

Some thoughts ..
#1 I still really like Big Servers and nearby beowulf style computing.
Typically there's a pipe and that pipe is going to limit what you can do.
If that pipe is always saturated by transferring terabytes back and forth from the cloud, things are going to slow down.
Being able to launch 500 virtual cloud machines isn't really solving the bandwidth problem. Not if 25 other people using the same pipe are doing the same thing.

#2 The internet is bursty. If you enjoy the "circling doughnut"

or "please wait ... buffering", then you're ready for the cloud.

#3 This is going to be very expensive.

#4. Google's omnivision is getting a little scary. It's getting to be like ... Jeremy Bentham's Panoption ... or like George Orwell's 1984 where TV watches you. I just don't trust the brand anymore. Clever marketing is not going to fix this today.

#5. There's liability and privacy concerns using patient data. The cloud is a real problem.

Someday, somebody is going to get "cloud" right.

GenoMax · 11-24-2014, 09:13 AM

I am only going to comment on technical thoughts mentioned in Richard's post.

1. Presumably anyone wanting to do a true "planet-scale" project would have a direct peering connection with Google and/or a major internet backbone provider. For less ambitious projects/normal users, the bottleneck will likely be the connection on the user-end (rather than Google's).

2. Google (in all likelihood, I don't know for sure) does not use public internet for internal connectivity so once the data is in the Google cloud the circling doughnut may actually indicate a running process on a large scale

3. Costs are advertised here: https://cloud.google.com/genomics/pricing. $22 per TB per month for storage.

5. Privacy and liability are issues that the cloud providers are coming to grips with. There are mechanisms to sign agreements that would cover this aspect (http://googlecloudplatform.blogspot....-entities.html). Since local legal counsel will be involved in setting up such an agreement I doubt Google would use/access user data beyond terms spelled out in the BAA.

Geneus · 12-04-2014, 05:26 AM

Quote:

Originally Posted by Richard Finney

Some thoughts ..
.

#5. There's liability and privacy concerns using patient data. The cloud is a real problem.

Get over it already.

How's your credit card doing these days at Target?

Richard Finney · 03-09-2016, 04:56 PM

Bumping this thread.

Anybody used google cloud genomics?

Any thoughts?

Is Amazon or Microsoft doing any genomics cloud projects?

GenoMax · 03-10-2016, 06:25 AM

Some folks (I know locally) have been using Google compute very successfully and they are extremely happy about the possibilities. It is not inexpensive but if one keeps things in perspective (e.g. where else can you find resources to analyze 5000 RNAseq samples in a day or so) then the possibilities are only limited by available budget.

Google cloud storage is also compelling. Looking at using it for data backup (BTW: Veritas NetBackup can use google cloud storage as a target).

Richard Finney · 03-10-2016, 09:30 AM

Well. I am very hazy on this so bear with me this is a naive question ...

This not trivial and would demonstrate a computationally big, practical test case ...

I guess the question(s) I have are ... if the API interface is to objects , then are the methods to access these objects user definable?
Can I for instance call a method to realign a bwa aligned hg19 bam files to a hg38 novoalign bam file? User must, of course, do the work of gluing programs together to do this : oldbam->fastq->novalaign->newbam . Can theses sorts of programs be run out "in the cloud" ?

... and ...

Is docker or other virtual machine images usable in the google cloud?

GenoMax · 03-10-2016, 11:01 AM

I referred to "Google compute" and not "Google genomics" in my post, even though you were asking about google genomics.

Since the workflows are specific to sites we are using our own (not the default on genomics) via google compute with VM's.

Docker containers are usable on google compute. There is a nice container management system (kubernetes) available.

Brian Bushnell · 03-11-2016, 06:16 PM

I happen to be a Google employee now. I do not work directly with the cloud or genomics groups, but I would be happy to relay questions, and will post the responses here.

nucacidhunter · 03-11-2016, 08:12 PM

Quote:

Originally Posted by Brian Bushnell

I happen to be a Google employee now. I do not work directly with the cloud or genomics groups, but I would be happy to relay questions, and will post the responses here.

I wonder if you or someone else will be maintaining BBTools.

Brian Bushnell · 03-12-2016, 12:34 AM

I'm still maintaining it on weekends

Hashtag Orange · 06-30-2021, 05:29 AM

Google cloud platform offers services for computing, storage, networking, machine learning, IoT which stands for the internet of things as well as for security and developers tools.
Google cloud platform is an excellent platform must go for it.

sekar · 05-11-2022, 03:16 AM

Cloud platform is stored database in cloud storage its like software, technology, networking, etc.

generally promote our website or Domain world wide

annagreysd · 05-11-2022, 04:44 AM

In the reverse Martingale, players double their https://dota2-bets.net/dota-2-skin-betting/ wager after winning. They continue to do so until they are satisfied with their wins. However, to be successful with this strategy, you will need to pick the right color each time, else you lose your earnings.However, the key to success in the game is your ability to remain disciplined. If you keep to a strategy and it is successful, know when to pull out to avoid losing all your skins. Have control of yourself to prevent addiction and huge losses.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
MetaVelvet Google Forum	vanillasky	Bioinformatics	0	04-14-2014 04:59 AM
Google Genomics	Richard Finney	Bioinformatics	3	03-07-2014 11:32 PM
iOmics - Cloud based NGS data analysis platform	geschickten	Introductions	0	10-30-2011 06:51 AM
SRA Database in Google cloud hosted by DNAnexus	GenoMax	Bioinformatics	0	10-12-2011 05:09 AM
Google spreadsheet for NGS stats	avilella	General	5	03-09-2010 12:20 PM

03-10-2016, 09:30 AM	#8
Richard Finney Senior Member Location: bethesda Join Date: Feb 2009 Posts: 700	Well. I am very hazy on this so bear with me this is a naive question ... This not trivial and would demonstrate a computationally big, practical test case ... I guess the question(s) I have are ... if the API interface is to objects , then are the methods to access these objects user definable? Can I for instance call a method to realign a bwa aligned hg19 bam files to a hg38 novoalign bam file? User must, of course, do the work of gluing programs together to do this : oldbam->fastq->novalaign->newbam . Can theses sorts of programs be run out "in the cloud" ? ... and ... Is docker or other virtual machine images usable in the google cloud? Last edited by Richard Finney; 03-10-2016 at 09:34 AM.

05-11-2022, 03:16 AM	#14
sekar Junior Member Location: india Join Date: May 2022 Posts: 4	Cloud platform is stored database in cloud storage its like software, technology, networking, etc. generally promote our website or Domain world wide __________________ Apply Personal Loan Online Personal Loan Apply Credit Card Online business loan Online Apply Doctors Loan Online

05-11-2022, 04:44 AM	#15
annagreysd Junior Member Location: Alabama Join Date: Apr 2018 Posts: 1	In the reverse Martingale, players double their https://dota2-bets.net/dota-2-skin-betting/ wager after winning. They continue to do so until they are satisfied with their wins. However, to be successful with this strategy, you will need to pick the right color each time, else you lose your earnings.However, the key to success in the game is your ability to remain disciplined. If you keep to a strategy and it is successful, know when to pull out to avoid losing all your skins. Have control of yourself to prevent addiction and huge losses. __________________ To begin to wager dota 2 live odds on LoL matches, it doesn't damage to get a smidgen of comprehension of the construction of nearby esports first. The super worldwide competition in League of Legends is customarily viewed as the mainland association or LSL.

11-24-2014, 03:20 AM	#1
Marcela Uliano Member Location: Cambridge, UK Join Date: Apr 2012 Posts: 19	Google Cloud Platform Hey guys! I was asked to test a little this Google Cloud Platform. I'm just login to free trial, but I would like to have opinions of people who have already used it. Is it like amazon cloud, where you can by processors and memory ram depending on the algorithms you need to use? Does it has programs installed? Pipelines? Applications? If you have a computer cluster where you can install everything you need to assembly and annotated your eukaryotic genome, is there any advantage in the cloud at all? Thank you so much for your time and feedback!

11-24-2014, 08:03 AM	#3
Richard Finney Senior Member Location: bethesda Join Date: Feb 2009 Posts: 700	Some thoughts .. #1 I still really like Big Servers and nearby beowulf style computing. Typically there's a pipe and that pipe is going to limit what you can do. If that pipe is always saturated by transferring terabytes back and forth from the cloud, things are going to slow down. Being able to launch 500 virtual cloud machines isn't really solving the bandwidth problem. Not if 25 other people using the same pipe are doing the same thing. #2 The internet is bursty. If you enjoy the "circling doughnut" or "please wait ... buffering", then you're ready for the cloud. #3 This is going to be very expensive. #4. Google's omnivision is getting a little scary. It's getting to be like ... Jeremy Bentham's Panoption ... or like George Orwell's 1984 where TV watches you. I just don't trust the brand anymore. Clever marketing is not going to fix this today. #5. There's liability and privacy concerns using patient data. The cloud is a real problem. Someday, somebody is going to get "cloud" right. Last edited by Richard Finney; 11-24-2014 at 08:13 AM.

11-24-2014, 09:13 AM	#4
GenoMax Senior Member Location: East Coast USA Join Date: Feb 2008 Posts: 7,143	I am only going to comment on technical thoughts mentioned in Richard's post. 1. Presumably anyone wanting to do a true "planet-scale" project would have a direct peering connection with Google and/or a major internet backbone provider. For less ambitious projects/normal users, the bottleneck will likely be the connection on the user-end (rather than Google's). 2. Google (in all likelihood, I don't know for sure) does not use public internet for internal connectivity so once the data is in the Google cloud the circling doughnut may actually indicate a running process on a large scale 3. Costs are advertised here: https://cloud.google.com/genomics/pricing. $22 per TB per month for storage. 5. Privacy and liability are issues that the cloud providers are coming to grips with. There are mechanisms to sign agreements that would cover this aspect (http://googlecloudplatform.blogspot....-entities.html). Since local legal counsel will be involved in setting up such an agreement I doubt Google would use/access user data beyond terms spelled out in the BAA.

03-09-2016, 04:56 PM	#6
Richard Finney Senior Member Location: bethesda Join Date: Feb 2009 Posts: 700	Bumping this thread. Anybody used google cloud genomics? Any thoughts? Is Amazon or Microsoft doing any genomics cloud projects?

03-10-2016, 06:25 AM	#7
GenoMax Senior Member Location: East Coast USA Join Date: Feb 2008 Posts: 7,143	Some folks (I know locally) have been using Google compute very successfully and they are extremely happy about the possibilities. It is not inexpensive but if one keeps things in perspective (e.g. where else can you find resources to analyze 5000 RNAseq samples in a day or so) then the possibilities are only limited by available budget. Google cloud storage is also compelling. Looking at using it for data backup (BTW: Veritas NetBackup can use google cloud storage as a target).

03-10-2016, 11:01 AM	#9
GenoMax Senior Member Location: East Coast USA Join Date: Feb 2008 Posts: 7,143	I referred to "Google compute" and not "Google genomics" in my post, even though you were asking about google genomics. Since the workflows are specific to sites we are using our own (not the default on genomics) via google compute with VM's. Docker containers are usable on google compute. There is a nice container management system (kubernetes) available.

03-11-2016, 06:16 PM	#10
Brian Bushnell Super Moderator Location: Walnut Creek, CA Join Date: Jan 2014 Posts: 2,707	I happen to be a Google employee now. I do not work directly with the cloud or genomics groups, but I would be happy to relay questions, and will post the responses here.

03-12-2016, 12:34 AM	#12
Brian Bushnell Super Moderator Location: Walnut Creek, CA Join Date: Jan 2014 Posts: 2,707	I'm still maintaining it on weekends

06-30-2021, 05:29 AM	#13
Hashtag Orange Junior Member Location: Gurugram Join Date: Jun 2021 Posts: 1	Google cloud platform offers services for computing, storage, networking, machine learning, IoT which stands for the internet of things as well as for security and developers tools. Google cloud platform is an excellent platform must go for it.