Google Cloud Platform, Google App Engine, and Google Compute Engine

GDG Fresno User GroupSeptember 4th 2014

Bitwise Industries Mural District

Service variety

Cloud platform

• Brief history• Roles of Google, Amazon, Microsoft, IBM, HP…• Service levels

Service levels

• IaaS: Infrastructure as a Service• PaaS: Platform as a Service• SaaS: Software as a Service

SaaS

PaaS

IaaS

Infrastructure as a Service

• Simplest form: leasing a physical or virtual server box: RackSpace, SoftLayer

• Includes– Hardware: servers, network, routers, load balancers,…– Software: operating systems, databases (storage),

application servers• Amazon AWS (Amazon Web Services) (+ S3 (Simple

Storage Service) + EC2 (Elastic Cloud Compute))• Microsoft Azure: VM Role• Google: GCE - Google Compute Engine*

Platform as a Service

• The provider takes care some higher level functions in the service stack

• Instead of getting servers, you get an application framework

• Less control over the lower level service elements, but the abstraction should result in less hassle and more focus on the goal

• Google: GAE - Google App Engine*• Azure Web Role, Worker Role, Reporting Services,

etc.

Software as a Service

• Software deployed on the internet• Designed for end-users• Delivered through the web• The back-end automatically scales, fault-tolerant

persistence• Usually API (Application Programming Interface) is

available for usage or feature extension• Example– Gmail, Google Docs, Google Spreadsheet– Office 365

IaaS / PaaS / SaaS

Control + Cost Efficiency + Savings

IaaS / PaaS / SaaS

SaaS

PaaS

IaaS

Leve

l of C

ontr

ol

Leve

l of A

bstr

actio

n

Google Compute Engine (GCE)

• IaaS level mainly• Minute-by-minute billing (10 minutes minimum)• Variety of virtual hardware selections (CPU

config and mem size)• Standard or custom VM images• Can be accessed through command line and

RESTful API• Came too late?

Demo

• Spinning up a VM• Selecting VM type• Selecting OS image

Earlier Demo Last Time

• Hadoop cluster with GCE• Last time we saw a word count example (hello

world of hadoop demos)• Needs extra work– Download and preparation of the hadoop distribution

itself– Applying patches– Generating keys

• But in the end Google also provides a set of tools which make life much easier

Hadoop

• Hadoop is an open-source software framework that supports data-intensive distributed applications

• A Hadooop cluster is composed of a single master node and multiple worker nodes

• Hadoop is written in Java, utilizes JVMs• Has two main services:– Storing large amounts of data: HDFS, Hadoop

Distributed File System– Processing large amounts of data: implementation of

the MapReduce programming model

Name node

HDFS

MetadataStore

Data node Data node Data node

Node 1 Node 2

Block A Block B Block A Block B

Node 3

Block A Block B

Name nodeHeart beat signals and

communication

Job / task management

Jobtracker

Data node Data node Data node

Tasktracker Tasktracker

Map 1 Reduce 1 Map 2 Reduce 2

Tasktracker

Map 3 Reduce 3

Google App Engine

• PaaS level member of Google Cloud Platform• It’s for developing, hosting, and scaling out

web applications• Supported languages:– Python– Java (+ JVM langs Groovy, JRuby, Scala, Clojure)– Go (experimental)– PHP (experimental)

Google App Engine

• Python– Django, CherryPy, Pyramid, Flask, web2py,

webapp2, and more– CGI adapters

• Java– Servlet 2.5 technology, Jetty Web Server– JSP, Java Server Faces, JPA, JDO, Spring

Framework, Struts 2*, Grails*, ….

Google App Engine

• Restrictions:– Can only execute code which respond to HTTP

requests– Answer cannot take longer than 60 seconds!– Normally only pure Python modules (no C or

Pyrex)*– …

GAE Example

• High score chart in the cloud• Java servlets• Google Datastore (NoSQL) for the data, non

relational• https://

github.com/59DAYSOFCODE/Project-Deal-ORound/tree/master/cards/server/Server/src/main

• (not open source unfortunately)

Web.xml<?xml version="1.0" encoding="utf-8"?><web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">

<servlet><servlet-name>DataStoreSendDataEndPoint</servlet-name><servlet-class>core.DataStoreSendDataEndPointServlet</servlet-class></servlet><servlet><servlet-name>DataStoreFetchEndPoint</servlet-name><servlet-class>core.DataStoreFetchEndPointServlet</servlet-class></servlet><servlet-mapping><servlet-name>DataStoreSendDataEndPoint</servlet-name><url-pattern>/sendData</url-pattern></servlet-mapping><servlet-mapping><servlet-name>DataStoreFetchEndPoint</servlet-name><url-pattern>/fetchData</url-pattern></servlet-mapping>

</web-app>

Servlets

• DataFetchEndpointServlet.java• DataStoreSendDataEndPointServlet.java

Google Cloud Platform Live Events

Google slides

• https://speakerdeck.com/googlecloudplatform/keynote-cloud-developer-roadshow-2014– Flexibility and Automatic Management– Big Data and Real-Time

• https://speakerdeck.com/googlecloudplatform/iaas-divide-cloud-developer-roadshow-2014– Today: cloud as a binary choice– Blurring the line between IaaS and PaaS with

managed VMs

Demo: Sudoku Solver

• Peter Norvig’s solver algorithm:– http://norvig.com/sudoku.html

• Final demo:– http://compute-demo.appspot.com

• Demo source code:– https://github.com/GoogleCloudPlatform/appengi

ne-opencv-sudoku-python

Ease

• Deploy:gcloud preview app deploy

• Application specific setting1. from google.appengine.api import modules2. modules.set_num_instances(42)

Google Cloud Platform

Thank you!

• Questions?

Google has perks

A screw > nothing is perfect