59
Tel Aviv - Oct 2013

Google Cloud Platform Update

Embed Size (px)

DESCRIPTION

These slides are made for the 2013 DevFest talks. It covers the main blocks of Google cloud platform: App engine, Compute Engine, storage options and more.

Citation preview

Page 1: Google Cloud Platform Update

Tel Aviv - Oct 2013

Page 2: Google Cloud Platform Update

Google Cloud Platform

Google Cloud Overview

+GreenIdoDeveloper Advocate, Google

App Engine

Page 4: Google Cloud Platform Update

Agenda

●Part I: Background & Motivation

●Part II: Google App Engine

●Part III: Cloud Platform Updates

Page 5: Google Cloud Platform Update

About Scale● Google's search index

○ (2012)

● YouTube

○ (2013)

● YouTube... monthly uniques?

○ (2013)

Page 6: Google Cloud Platform Update

Innovation

Page 7: Google Cloud Platform Update

Scaling & Innovation

Image courtesy Google/Connie Zhou

Image courtesy Google Maps

*AND* security

Page 8: Google Cloud Platform Update

Cloud industry service levels

PaaS

SaaSGoogle Apps Salesforce, SugarCRM, NetSuite

Yahoo!Mail YOUR NEXT APP?

Google Apps Script force.com

Google App Engine Cloud Foundry, OpenShift

Windows Azure, Heroku DotCloud, Engine Yard

Google BigQuery, Cloud SQL, Cloud Datastore, Translate, Prediction

Amazon RDS, DynamoDB; Microsoft SQL (Azure) Database

Amazon EC2, S3 Rackspace, Joyent

Google Compute Engine, Cloud Storage

GoGrid, CloudLayerIaaS

Page 9: Google Cloud Platform Update

Google Cloud Platform service levels

PaaS

SaaSGoogle Apps Salesforce, SugarCRM, NetSuite

Yahoo!Mail, Hotmail YOUR NEXT APP?

Google Apps Script force.com

Google App Engine Cloud Foundry, OpenShift

Windows Azure, Heroku DotCloud, Engine Yard

Google BigQuery, Cloud SQL, Cloud Datastore, Translate, Prediction

Amazon RDS, DynamoDB; Microsoft SQL (Azure) Database

Amazon EC2, S3 Rackspace, Joyent

Google Compute Engine, Cloud Storage

GoGrid, CloudLayerIaaS

Page 10: Google Cloud Platform Update

● Comprehensive, integrated platform

● Extension of Google internal infrastructure○ Internal services created for us

SHARED with YOU

● Google makes improvements to their cloud○ ...and who gets them first...

What is the Google Cloud Platform?

Page 11: Google Cloud Platform Update

Google Cloud Platform services

Page 12: Google Cloud Platform Update

Google worldwide network

Page 13: Google Cloud Platform Update

"This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds." -- S. Levy/Wired (Oct 2012)

Our cloud runs our business

Page 14: Google Cloud Platform Update

Google Cloud Platform services

Page 15: Google Cloud Platform Update

Agenda

●Part I: Background & Motivation

●Part II: Google App Engine

●Part III: Cloud Platform Updates

Page 16: Google Cloud Platform Update

➢ Hardware failure? ➢ Traffic spikes? Scaling? ➢ Patches & upgrades? ➢ Network issues? ➢ License management?

Building a traditional web application

Page 17: Google Cloud Platform Update

Why does App Engine exist?

Page 18: Google Cloud Platform Update

●Easy to build

●Easy to manage

●Easy to scale

● Enabling greater developer productivity!● Focus on building your app, not sys admin● Let us wear the pagers!● BUT don't confuse App Engine w/IaaS

services!

App Engine to the rescue!!

Page 19: Google Cloud Platform Update

Python, Java, Go, PHP not enough for you?

Page 20: Google Cloud Platform Update

Specialized APIs/Services for the Sandbox

Cloud StorageImages

Mail Channel Task Queue

Memcache Datastore URL Fetch

User Service

Page 21: Google Cloud Platform Update

Hardware failure, traffic spikes, scaling, software patches,

upgrades handled by Google... and yes, we’ll operate it for you!

How does it work?

Page 22: Google Cloud Platform Update

Administration Console

Page 23: Google Cloud Platform Update

App Engine - By the Numbers*

● 300,000+ Active developers

● 3,000,000+ Active applications

● Half of world's Internet addressestouch an App Engine server(per week)

* per month except as otherwise noted

Page 24: Google Cloud Platform Update

App Engine - Larger Numbers*

7,500,000,000+Hits (per day)

4,500,000,000,000+Datastore requests

* per month except as otherwise noted

Page 25: Google Cloud Platform Update

Some App Engine users

Page 26: Google Cloud Platform Update

● 3.6MM DAUs on FB● 1.9MM DAUs on MS● Add Orkut, Bebo,

Hi5, Friendster, Hyves, Ning…

Long-term scaling...

Page 27: Google Cloud Platform Update

Short-term scaling...

Images courtesy Gigya

Page 28: Google Cloud Platform Update

Official blog & live stream apps hosted on App Engine

On Wedding day...Blog app served:

● Up to 2k requests per second● 15 million pageviews ● 5.6 million visitors

Live stream app served:● Up to 32k requests per second● 37.7 million pageviews ● 13.7 million visitors

goo.gl/F1SGc

“Royal” scaling

Page 29: Google Cloud Platform Update

App Engine! -> Cloud Endpoints!

Not all apps have web UI

Page 30: Google Cloud Platform Update

App Engine as Mobile Backend

● Pulse’s mobile app○ Android○ iOS○ Kindle

● Awards (2011)○ Apple’s App Store Hall of Fame○ Time Magazine’s Top 50 iOS apps

● 30MM+ users, 10MM+ stories read/day● Thousands of QPS, 100M daily requests● Use of App Engine cost-effective

● Get started building your mobile backend:○ developers.google.com/cloud/samples/mbs

Page 31: Google Cloud Platform Update

Gaming... share data, mobile or web!

Page 32: Google Cloud Platform Update

Multimedia scaling...

Page 33: Google Cloud Platform Update

Google Cloud Endpoints● APIs for web and mobile backends made easy

● Watch the tutorial & get the code at cloud.google.com/mobilebackendtutorial

Page 34: Google Cloud Platform Update

Popular App Engine use cases

● Mobile/Tablet○ App backends○ Cloud persistence

● Social/Mobile Games○ Speed, scale○ API integrations○ Personals/dating

● Consumer Web Apps○ Unpredictable traffic○ Scale

● Apps in Academia○ Any course where

students build web or mobile apps

○ Research projects○ IT/Operational apps

● Business Apps○ Enterprise○ Java runtime○ Web or Mobile

Page 35: Google Cloud Platform Update

“Vendor lock-in”● What is it?

○ Systems which inherently make it very difficult or impossible to migrate (data / logic) to other systems

● Applies to App Engine?○ Yes & no.○ Can’t get something for nothing.○ Goal: take advantage of

Google's infrastructure○ Price: need to write against

Google APIs. Make sense?

● Not in Google’s DNA○ Peter Magnusson post http://goo.

gl/X2UyEz

Page 36: Google Cloud Platform Update

● “Must use” Google APIs○ Not necessarily

● Web app alternatives○ Java: servlets, JSP○ Python: Django, web2py,

Tipfy, Bottle, Pyramid

● Data Storage○ Cloud SQL (MySQL)○ Python: Django ORM*○ Java: JPA/JDO ORMs

● SDKs: open source

● APIs○ Java: JSR stds, Python: stdlib

● Datastore bulkloader○ Easily up/download your data

● Open source backend systems○ Python: AppScale; TyphoonAE○ Java: CapeDwarf/JBoss - TCK○ "Private cloud"

Fighting lock-in

Page 37: Google Cloud Platform Update

Pricing

Google Prediction API

Google Storage

PricingTier

Monthly Cost

99.95% SLA

Paid Support

Free $0

Paid pay-per-use *

Premier $150USD+ * *

● cloud.google.com/pricing● Please compare - “Apples to Apples”

Page 38: Google Cloud Platform Update

● SAS 70, SSAE 16, ISAE 3402, ISO 27001:2005 Certified

● Choose from US- or EU-based service

Security Compliance & Localization

Page 39: Google Cloud Platform Update

● Integration w/Compute Engine, BigQuery, Cloud Datastore● Java Servlet 3.0 support + Blobstore migration tool● VM Runtime App Engine-managed backends via Compute

Engine● developers.google.com/appengine/features/#roadmap_features

App Engine Roadmap

Page 40: Google Cloud Platform Update

Getting Started

Google Prediction API

Google Storage

What WhereProduct Info cloud.google.com/products

Docs, downloads, etc. developers.google.com/appengine

Create/manage your apps appengine.google.com

Help from experts on anything technical, including App Engine stackoverflow.com

Cloud Blog googlecloudplatform.blogspot.com

Social google.com/+googlecloudplatform @GoogleCloud

● The docs are great, but what about teaching materials?

Page 41: Google Cloud Platform Update

Cloud Playground

●Try App Engine in your browser...cloud-playground.appspot.com

Page 42: Google Cloud Platform Update

Agenda

●Part I: Background & Motivation

●Part II: Google App Engine

●Part III: Cloud Platform Updates

Page 43: Google Cloud Platform Update

Connect to Google Cloud Platform

BigQuery

Google Cloud Storage

Google App Engine

ComputeEngine Cloud SQL

Prediction

???

CloudDatastore

Translate

Page 44: Google Cloud Platform Update

Google Compute Engine

● Need to go outside App Engine boundaries?

● On-demand VMs (RHEL/CentOS, Debian)○ 1-8 virtual cores○ 1.8-6.5GB RAM/core○ Private VM network

● 3 storage types○ Local/scratch: 420GB○ Persistent disk: 10TB○ Cloud Storage: unlimited

● 3 interfaces○ Command-line interface○ Web-based interface○ Your code via REST API

Page 45: Google Cloud Platform Update

Compute Engine performance

Source: Scalr/gigaom.com (May 2013)

Real-world Performance Observation

Page 46: Google Cloud Platform Update

Google Cloud Platform

Google Cloud Storage● Fast, scalable, highly-available object store (US or EU)● Strong read-your-write consistency● Easy, flexible authentication & sharing (OAuth2, ACLs)● REST API, App Engine API, Web UI, command-line● developers.google.com/storage

Google Cloud SQL● MySQL-compatible relational cloud database● App Engine API or externally via JDBC● developers.google.com/cloud-sql

Page 47: Google Cloud Platform Update

Google Cloud Platform

Google Cloud Datastore● NoSQL database at Google scale● Low-level interface (access via HTTP/RPC)● Familiar native datastore for App Engine● developers.google.com/datastore

Google Translate● Translate text into other languages programmatically● Build multilingual apps with Google translation algorithms● Familiar RESTful interface● developers.google.com/translate

Cloud Datastore

Translate

Page 48: Google Cloud Platform Update

Google Cloud Platform

Google BigQuery● Large scale query and analysis cloud service● Query multi-terabyte datasets in seconds● SQL-like query language● Based on Google's internal Dremel project● cloud.google.com/bigquery-tour

Google Prediction● Machine Learning service in the cloud● Supervised learning● Train models, get "predictions"● "Magic?" Possibly.● developers.google.com/prediction

Prediction

Page 49: Google Cloud Platform Update

Fusion Tables● Think: Google Maps + Excel/Sheets on steroids● Aggregate disparate data in spreadsheet & visualize● google.com/fusiontables

Page 50: Google Cloud Platform Update

Course Builder MOOC

Page 51: Google Cloud Platform Update

Course Builder MOOC

● Open-source MOOC software○ Easily customizable○ code.google.com/p/course-builder

● Courses hosted on Google App Engine○ Run your own MOOCs: your code, your control○ No need to rely on commercial vendor(s)

● More information○ googleresearch.blogspot.com/2012/09/helping-world-to-teach.html

● Sample course○ powersearchingwithgoogle.com

Page 52: Google Cloud Platform Update

CloudCourse class registration

● App Engine app for course offerings & registration● http://goo.gl/o8NRSk

Page 53: Google Cloud Platform Update

Google Apps Script

● script.google.com

● JavaScript baby! outside the browser, in Google’s cloud.

● Browser based IDE

● Deeply integrated with Google Apps & services

○ Automate repetitive tasks

○ Manipulate data across multiple tools

Page 54: Google Cloud Platform Update

Apps Script example

Page 57: Google Cloud Platform Update

Ahh! WAIT, there’s one more thing...

Page 58: Google Cloud Platform Update

Cloud Platform Starter Pack

● Developers from affiliated partners can

receive up to $2,000USD of credit towards

cloud usage○ Up to $1,000 for Google App Engine

○ Up to $1,000 for Google Compute Engine + others

● Apply at cloud.google.com/starterpack and use promo code: gdg-in