Portrait of the Developer as the Artist - OpenTour Sofia

Portrait of the developeur as

@chanezon

French

Polyglot

Server Side

San Francisco

Developer Relations

Patrick ChanezonSenior Director

Developer Relationschanezonp@vmware.com

@chanezon

Very freely inspired from

... and a few years as an Enterprise consultant in France before

moving to California

2010

- George became CIO

What happened while

George was playing

Golf

Dreams Of my childhood

30

Accelerando / Singularity, in a Galaxy far far away

§Even if we automate ourselves out of a job every 10 years§ ...I don’t think the singularity is near!

31

Moore's Law is for Hardware Only

§Does not apply to software§Productivity gains not keeping up with hardware and bandwidth§Writing software is hard, painful, and still very much a craft

32

Predictions

“The future is already here — it's just not very evenly distributed” William Gibson

33

Architecture Changes: 60’s Mainframe

Architecture Changes: 80’s Client-Server

Architecture Changes: 90’s Web

Architecture Changes: 2010’s Cloud, HTML5, Mobile

Back to Client Server: Groovy Baby!

38

What is Cloud Computing?

39

Cloud According to my daughter Eliette

Cloud Stack - Classic Pyramid

40

Platform As A Service

Infrastructure As A Service

SoftwareAs A Service

Cloud Stack - By Value

41

InfrastructureAs A Service

SoftwareAs A Service

Platform As A Service

Cloud Stack - History

§What does cloud mean, 4 main angles• Software 1994 Netscape• Infrastructure 2002 Amazon AWS• Platform 2008 Google• Development now!

42

§ Industrialization of hardware and software infrastructurelike electricity beginning of 20th century

§But software development itself is moving towards craftmanship

Cloud started at Consumer websites solving their needs

• Google, Amazon, Yahoo, Facebook, Twitter

• Large Data Sets

• Storage Capacity growing faster than Moore’s Law

• Fast Networks

• Vertical -> Horizontal scalability

• Open Source Software

• Virtualization

• Cloud is a productization of these infrastructures

• Public Clouds Services: Google, Amazon

• Open Source Software: Hadoop, Eucalyptus, Ubuntu, Cloud Foundry

44

Infrastructure

IaaS/Virtualization getting mainstream

§AWS, Joyent, Rackspace,...§Open Source projects: OpenStack, DeltaCloud, Eucalyptus§Automation: Chef, Juju, Cloud Foundry BOSH§Standardization? DMTF§ Inside the Firewall, Virtualization: VMware, Microsoft, Xen, KVM§60% of workloads are virtualized in 2012§Easy to provision, manage instance...BUT§Still need to manage backups, software stacks, monitor, upgrades

45

With Infrastructure, you still need to build your own platform

§Need to build a distributed platform on top of you infrastructure§Story of the AWS meltdown from last summer

• http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html

• http://www.readwriteweb.com/cloud/2010/12/chaos-monkey-how-netflix-uses.php

• http://news.ycombinator.com/item?id=2477296

• http://stu.mp/2011/04/the-cloud-is-not-a-silver-bullet.html

§Twilio, Smugmug, SimpleGeo survived it because they built their own distributed platform on top of IaaS

§Enterprise customers want to consider Infrastructure like CDNs• Multi Cloud usage

• Based on Open Source de facto standards, or full standards whenever that happens

46

47

Platforms

Platforms

§Raise the Unit of currency to be application & services instead of infrastructure

§Google App Engine, Cloud Foundry, Heroku, CloudBees, Amazon Elastic Beanstalk, Microsoft Azure, AppFog

§Single or a few languages, services

§Start of Multi language Polyglot platforms

§Enabler for Agile Developers -> Create Business value faster

§Lack of standards: risk, vendor lock-in

§Enterprise needs:•Control, customizability

•Private/Hybrid Cloud

•Avoid lock-in

48

Agile Development Processes

Agility as a survival skill

§Consumer software is becoming like fashion

•Phone apps, social apps, short lifetime, fast lifecycles

•A/B testing

§Enterprise

•Clay shirky situational apps

§Kent Beck, Usenix 2011 Talk, “Software G-Forces: the effects of acceleration”change in software process when frequency grows

§Cloud Platforms enables an Agile culture, driver for innovation

•Scalability is built in the platforms

•Can iterate faster

•Focus on design

§Cloud Platforms lets developers focus on driving business value

50

Main Risk: Lock-In

51

Welcome to the hotel californiaSuch a lovely placeSuch a lovely facePlenty of room at the hotel californiaAny time of year, you can find it here

Last thing I remember, I wasRunning for the doorI had to find the passage backTo the place I was before’relax,’ said the night man,We are programmed to receive.You can checkout any time you like,But you can never leave!

Cloud Foundry: The Open PaaS

52

Clou

d Pr

ovide

r Int

erfa

ce

Application Service Private Clouds

PublicClouds

MicroClouds

Data Services

Other Services

Msg Services

vFabric Postgres

vFabric RabbitMQTM

• Open Source: Apache 2 Licensed• multi language/frameworks• multi services• multi cloud

53

CloudFoundry.COM – Multi-Tenant PaaS Operated by VMware

Frameworks

Services

vCenter / vSphere

CloudFoundry.COM (beta)

Infrastructure

54

Micro Cloud FoundryTM – Industry First Downloadable PaaS

Single VM instance of Cloud Foundry that runs on a developer’s MAC or PCNew release v116 last weekTracks CloudFoundry.com

Frameworks

Services

Micro Cloud Foundry

Your Laptop/PC

55

Open Source

56

CloudFoundry.ORG – Community Open Source Project

github.com/cloudfoundry

.NET x 2 PHP JRuby Python Rails 2.x Clojure Erlang Haskell

§Memcached§SQL Server§Neo4j§CouchDB§VirtualBox§Mono§Rack

57

CloudFoundry.ORG – Community Open Source Project

DownloadCode

Setup Environment

Deploy Behind Firewall

Tool Chain &Scripts

Apache2 license

Your Infrastructure

Cloud Foundry BOSH

CloudFoundry.ORG

Community Frameworks Contributions

Community Services Contributions

58

Cloud Foundry Open Source Workflow

Reviewers +2/-2

test verification score ✓ ✗ git push triggers test execution

Gerrit Code Review

reviews.cloudfoundry.org

Jenkins CI

ci.cloudfoundry.org

github

github.com/cloudfoundry

+1/-1 Committers

on +2 and ✓ change pushed to github

cloudfoundry.com

production updated ~2x/week

Open Source Advantage: speed

§http://code.google.com/p/googleappengine/issues/detail?id=13

59

• https://github.com/cloudfoundry/vcap/pull/25

Open Source Advantage: breadth

60

61

Ecosystem

62

Broad Industry Investment

63

Cloud Foundry - Making Multi-Cloud a Reality

Public Cloud Operators

.COM

Management and Private Cloud Distributions

Bare metal

Cloud Foundry Architecture

Architectural Goals

65

No single point of failure

Architectural Goals

65

No single point of failure Distributed state

Architectural Goals

65

No single point of failure Distributed state Self healing

Architectural Goals

65

No single point of failure Distributed state Self healing Horizontally scalable

Architectural Goals

65

Architectural Principles

66

Loose coupling

Architectural Principles

66

Loose coupling Event-driven

Architectural Principles

66

Loose coupling Event-driven Asynchronous

Architectural Principles

66

Loose coupling Event-driven Asynchronous Non-blocking

Architectural Principles

66

Loose coupling Event-driven Asynchronous Non-blocking Idempotent

Architectural Principles

66

Loose coupling Event-driven Asynchronous Non-blocking Idempotent Eventually consistent

Architectural Principles

66

Loose coupling Event-driven Asynchronous Non-blocking Idempotent Eventually consistent Language-independent communication

Architectural Principles

66

Cloud Foundry Layers

67

Cloud Foundry Layers

Clients

67

Cloud Foundry Layers

Inner Shell

Clients

67

Cloud Foundry Layers

Outer Shell

Inner Shell

Clients

67

Cloud Foundry Layers

Infrastructure as a Service

Outer Shell

Inner Shell

Clients

67

Cloud Foundry Layers

Hardware (CPU, Memory, Storage, Network)

Infrastructure as a Service

Outer Shell

Inner Shell

Clients

67

Cloud Foundry Core

Components

Dynamically discoverable components No inter-component dependencies

• Launch in any order• Scale up and down independently

Monitor using HTTP end points

Design Principles

69

Cloud Foundry Logical View

Routers

CloudControllers App

Services

App

HealthManagerExecution Agents (DEA)

Pool

Messaging

UsersDevelopersvmc

71

Cloud Foundry BOSH

72

500 – 5,000 VMs 40+ unique node types 75+ unique software packages 75+ unique environments 2x/week cf.com updates 24x7x365 non-stop operation No-downtime deployments Reliable, robust, repeatable deployments, updates, capacity adjustments

Small teams manage many instances

Production Grade Cloud Foundry Clusters

Google style problem è Google style solution

cloudfoundry.com

production, staging, stress, qa, dev

73

Cloud Foundry BOSH is an open source tool-chain for release engineering, deployment, and lifecycle management of large scale distributed services•Prescriptive way of creating releases and managing systems and services• It is not a collection of shell scripts, not a pile of Perl

Built to deploy and manage production-class, large scale clusters•Production grade Cloud Foundry clusters: 500+ VMs, 40+ jobs, 75+ packages •Multi-node, multi-tier, complex clusters: e.g., our Gerrit/Jenkins Cluster

Built for devops usage and scale by a crack team of veterans•A project, not a product: command line interface, YAML, etc. •Continuous improvement, iterative development, rough edges

Cloud Foundry BOSH

github.com/cloudfoundry/bosh

74

BOSH: under the hood

bosh cli

redis natsdb

director healthmon

IaaS CPI

workers

stemcellagent

blobs

“BOSH is deployed by BOSH”

cloudfoundry.com

BOSH User

active jobs

disk

75

IaaS neutral by design

CPI: code completefunctional status: “work in progress”

vSphere: battle tested implementation, thousands of deployments

vCloud Director: “work in progress”, 2H 2012

contribute: github.com/cloudfoundry/bosh

Cloud Provider Interface (CPI)

Cloud Foundry BOSH

github.com/piston/openstack-bosh-cpi

76

Lessons for Developers

Predictions

•Software is becoming like fashion, design rules•Welcome to Babel, use the best tool for the job, embrace multiple language & heterogeneity

•Our jobs will change, build yourself out of your current job•Sysadmin jobs will morph, there will be less of them•Many opportunities open when you embrace change

77

Things to forget

•First normal form•Waterfall model•Single server deployment•Single language skill•Build everything from scratch•Build custom infrastructure

78

Things to learn

•Agile•Take risks, fail often, fail fast and learn•API Design: create the API first•UI Design, Javacript, HTML5, CSS3•A/B Testing•Open Source, Open Standards•Architecture, Distributed Computing (CAP theorem, 8 fallacies)•Cloud Platforms and APIs•Multiple types of languages (imperative, object, functional, logic)•Ability to encapsulate domain knowledge in a DSL•Build on the shoulders of giants: reuse, REST APIs•Pick your battles, choose what you need to build yourself to add value

•Learn to live in a box (embrace platform limitations) to think outside the box

•Use an App Store for distribution79

Redemption

The (Happy) End

Register today

93

http://cloudfoundry.com/signup

cfopentoursofiaUse Promo Code

To avoid approval queue waiting timeDownload Micro Cloud Foundry locallyftp://192.168.5.5/micro-v116-20121101.000204.zipEvents we do on Lanyrdhttp://lanyrd.com/2012/cloud-foundry-open-tour-sofia/