node.js on Google Compute Engine

Google confidential │ Do not distribute


Running Node.js on Google Compute Engine

Arun Nagarajangoogle.com/+ArunNagarajan@entaq

Intro

Google Cloud Platform

Live demo

Questions

Agenda: May 8th, 2014

2

4

3

1

75years

1957 2003 2013

500

25years

10years

(average age of a company joining the S&P 500)

The decreasing cost of storage enables virtually limitless storage in the cloud. $600 can buy enough storage for the world’s music*.

(Source: McKinsey Global Institute May 2011)

Computing as a utility is now available for easy purchase, provided from massively efficient data centers.

(Source: Nicholas Carr, The Big Switch, 2008)

The internet allows for a model of real-time access to new innovation, information and applications from a wide range of devices.

Affordablecapacity

On-demandcomputing

Instantaccess

Big Trends

*more accurately, one could store 3.6 million music tracks at 3.5 MB each, assuming ~$50/TB HDD storage price. Noted by John Wilkes!

Google confidential | Do not distribute

“People tend to overestimate what can be done in one year and to underestimate what can be done in five or ten years.”Amara's Law

Google confidential | Do not distribute

Cloud Platform is built on the same infrastructure that powers Google.

For the past 15 years, Google has been building out the world’s fastest, most powerful, highest quality infrastructure on the planet.

Images by Connie Zhou

Wired, 'Google Throws Open Doors To Its Top Secret Data Center', October 2012

Google's Platform"[Google's] ability to build, organize, and operate a huge network of servers and fiber-optic cables with an efficiency and speed that rocks physics on its heels.

This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds."

- Wired

72 hours

100 Million Gigabytes

Over 425 Million

1/4 second

Video uploaded to YouTube every minute

Google's "Caffeine" Search index, circa 2010

Number of active Gmail users

How long it takes for Google to respond to a search query

Google and Big Data

$2.9B in additional data center investments worldwideInvesting In Our Infrastructure

Store data with reliability, redundancy and consistency

Go from data to meaning

At scale

Fast

Google white papersGoogle File System (2003)MapReduce: Simplified Data Processing on Large Clusters (2004)BigTable: A Distributed Storage System for Structured Data (2006)Dremel: Interactive Analysis of Web-Scale Datasets (2010)Machine Translation (2004-2011)

2008

2010

2002

2004

2006

2012

GFSMapReduce

Big Table

ColossusSpanner

DremelFlumeJava

Pregel

What Did We Learn? To Innovate!

Application Runtimes & Services● Iterate & deploy fast● Scale to global demand● Standards compliant

Data Services● Data Intelligence● Designed for Big Data● High Performance

Data Storage and Distribution● Global Resilient Architecture● Global Edge Distribution● Huge Secure Capacity

Global Data Centre & Networks

● Highly Resilient, Efficient & Performant● 3rd Largest Server Manufacturer● 2nd Largest Global Data Network

Google Research Publications referenced are available here: http://research.google.com/pubs/papers.html

Company’s ApplicationsGoogle Products etc...

App Engine,Cloud Endpoints

BigQuery,Flume

Cloud Storage,Cloud SQL,

Cloud DataStore

Compute Engine,Cloud Networking,

Cloud DNS

GRTE, Stubby, BorgCron

Sawmill, Dremel, MapReduce

Colossus, BlobStore, Spanner, BigTable, F1

Borg, GFE, GSLB, GTC

http://research.google.com/pubs/papers.html

IaaSInfrastructure-as-a-Service

PaaSPlatform-as-a-Service

SaaSSoftware-as-a-Service

Cloud Computing

Managed by Companies Managed by Google

Cloud Computing

SaaSSoftware-as-a-Service

ApplicationsDataRuntimeMiddlewareOSVirtualizationServersStorageNetworking



IaaSInfrastructure-as-a-Service


PaaSPlatform-as-a-Service

PackagedSoftware


Storage

Cloud Storage Cloud SQL Cloud Datastore

Compute

Compute EngineApp Engine

App Services

BigQuery Cloud Endpoints

Google Cloud Platform

Compute Engine

● Google’s Infrastructure-as-a-Service

● Create Linux VMs, and connect to them as you

would any other remote machine

● Static + Dynamic Addressing

● Persistent disks and snapshotting

● Live migration

● Network and firewall creation

● Load balancing

● Regional and zonal resources for high availability

Compute Engine Features


“Google Compute Engine is not just fast. It’s Google fast. In fact, it’s a class of fast that enables new service architectured entirely.”

- Sebastian Stadil, Scalr

Scalr

● Built on Chrome's JavaScript runtime

● Building fast, scalable network applications.

● Uses an event-driven, non-blocking I/O model

● Lightweight and efficient

● Perfect for data-intensive real-time applications

● Run across distributed devices.

Node.js


Compute Engine


● console.developers.google.com

● create project

● setup billing

● Install gcloud tool -

○ curl https://sdk.cloud.google.com | bash

Compute Engine Setup

● gcloud components update compute

● gcloud auth login

● gcloud config set project eastern-button-574

● gcloud compute instances create arun-hello-world --zone europe-west1-a

● gcloud compute firewalls create allow-http --description "Incoming http allowed." --allow tcp:80

● gcloud compute ssh arun-hello-world --zone europe-west1-a

Alternatives:

● gcutil - still around but gcloud is focus

● REST API - very powerful

● UI - easy to use

Compute Engine Create Instance


Cloud Credit $500 Code - “dr-in”


● sudo apt-get install python g++ make checkinstall● mkdir ~/src && cd $_● wget -N http://nodejs.org/dist/node-latest.tar.gz● tar xzvf node-latest.tar.gz && cd node-v*● ./configure● sudo checkinstall (make sure to fix version)

Compute Engine node.js installation

application: my-applicationruntime: govm: true…manual_scaling: instances: 1vm_settings: machine_type: n1-standard-1 apt_get_install: redis-server vim

Built using Modules https://developers.google.com/appengine/docs/python/modules/Limited preview - https://developers.google.com/cloud/managed-vms

Managed VMs (limited preview)

https://developers.google.com/appengine/docs/python/modules/

https://developers.google.com/appengine/docs/python/modules/

https://developers.google.com/cloud/managed-vms

Google confidential │ Do not distribute Images by Connie Zhou

Technology

node.js on Google Compute Engine