Next Generation Software Development: Continuous Cloud Deployment · PDF file · 2013-04-18Next Generation Software Development: Continuous Cloud Deployment @SachaLabourey CEO, CloudBees,

Next Generation Software Development:Continuous Cloud Deployment

@SachaLaboureyCEO, CloudBees, Inc.

April 18, 2013 – PhillyJUGPhoto credit: @romainguy

©2013 CloudBees, Inc. All Rights Reserved

2

• Born in 1975 in Switzerland

• JBoss– Clustering lead – 2001– GM Europe - 2003– CTO - 2005

• Red Hat– JBoss acquisition in June 2006– co-GM of Red Hat’s middleware division– Left Red Hat in April 2009

• CloudBees– Started in April 2010– About 40 bees in 7 countries

Introduction – Sacha Labourey


X

3

• What’s the life of a developer today?• Challenges ahead…

– Ideal solution!• Cloud 101• Read my lips: SERVICE

– Software vs. Service– Continuous Cloud Delivery

• Reality Check• Quick demos

Agenda


Life of a Developer Today


5

• Release v1.0

• Gather requirements for v2.0 • Work on v2.0 • Release v2.0don’t screw it up: once it is out, it is out.

• Have your customer upgrade– Support v1.0 for years

• Support and patch dozen of “branches” in parallel

• Gather requirements for v3.0 • Work on v3.0

Typical Software Development

Mo

nth

s/Y

ears

Packaged Software


6

Developers vs. Sys Admins (aka Love Story)

Photo Credit: Joseph Viles/ The CW Photo Credit: http://www.sysadminday.com/people.html


http://www.sysadminday.com/people.html

7

M. Developer

Interests & Hobbies:• Give a try to the last framework• Put in the production application

the last shiny object seen at a conference

• Participate to the Scala vs. Ceylon debate, static vs. dynamic typing, etc.

• Dream of meeting with James Gosling

M. Sys Admin/ops

Interests and Hobbies:• Not to be beeped at 2am in the

morning• Not to be beeped at 4am in the

morning• Participate to the Debian vs. Fedora

vs. Ubuntu debate• Dream of meeting with Linus

Torvald

<sarcastic mode=“on” />


8

Development? Friction!

SourceCode

Managemnt

JenkinsContinuousIntegration

UnitTest

FunctionalTest

LoadTest

IntegrationTest

Developer

“Cloud you please provision a server for me?”

“Could I get X, Y and Z installed on that server?”

“Could you please patch my server?”

“Could I please get a database dump?”

Could I get a new Git server?

Where could I run my Jenkins integration tests?

IT


9

Deployment? Friction!

Configure OS Secure OS Setup JVM

Install AS on OS

Configure AS Secure AS

provision nodes

Install load-balancer (LB)

Configure LB

Secure LBSetup BC for cluster

QA cluster

Configure DNS

Provision node:

Provision cluster:

Provision node

Install DB Configure DB

Secure DBSetup BC for DB

Provision DB:

Provision cluster

Provision DBDeploy schema

Deploy App to each node

Deploy App:

(not a transparent update process)

Analyse sec. bulletins

Analyse QA bulletins

Analyse AS/OS logs

maintain OS maintain JVM maintain AS

maintain DB

Manage environment (recurring)

maintain FWValidate/QA stack

Measure App performance

Acquire Hardware

Provision node

Update Cluster Update LB

Metering/Scaling (recurring)

Update Security


10

Friction, friction, friction

provision nodes

Install load-balancer (LB) Configure LB

Secure LBSetup BC for cluster QA cluster

Configure DNS

Provision cluster:

Provision cluster

Provision DB Deploy schema


Deploy App:



Install AS on OS

Configure AS Secure AS

provision nodes

Install load-balancer (LB)

Configure LB

Secure LBSetup BC for cluster

QA cluster

Configure DNS

Provision node:

Provision cluster:

Provision node

Install DB Configure DB

Secure DBSetup BC for DB

Provision DB:

Provision cluster

Provision DBDeploy schema


Deploy App:



Analyse QA bulletins

Analyse AS/OS logs


maintain DB


maintain FWValidate/QA stack


Acquire Hardware

Provision node



Update Security


11

Why do we do software development again?


12

To serve the BUSINESS obviously!

Business Needs

SpecificationImplementa-tion

Deployment


13

IT serves the BUSINESS?

Increase Revenues

Implement e-commerce site

JVM? Ruby? PHP? Java? Scala?Play? Groovy? Lift? Jruby? CFML? Grails? Spring? EE6?JBoss? Tomcat? Glassfish? Jetty? MySQL? MongoDB? CouchDB? Cassandra? JMS? AMQP? Memcache? Ehcache? Quartz? Lucene? Solr? Jquery? GWT? iPhone? Android? JIRA? Redmine? SVN? Git? Maven? Ant/Ivy? Jenkins? Artifactory/Nexus? etc.

AMD/Intel? HP? DELL? SPARC? UX? AWS? Linux? Windows? Solaris? RHEL? Ubuntu? Debian? Arch? EXT2/3? NFS/CIFS? Backup? Chef? Puppet? Continuous Deployment? Apache HTTPD? Mod_jk? Mod_proxy? Nginx? Haproxy? Firewall? SSL? Server provisioing? Load-balancing? Failover? Monitoring? Management? Alerting? Escalation procedure? etc.

15 seconds

2 weeks

18 months


14

14

Didn’t we simply ask him to build an e-commerce site?

15

• Lots of FRICTION• Less time spent on

DELIVERING VALUE

And that’s about to get worse…

Software Development got Difficult


Challenges ahead!


17

• In parallel, demand for new needs are growing fast!– Mobile– Big Data– Social– Local– …

At the same time…


18

Entering the post-PC era

Thanks to Mary Meeker and Matt Murphy, From David Skok presentation @ MassTLC


19

19

Source: Mary Meeker , 2011, www.kpcb.com/internettrends2011


20

Small but big!


Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends

21

India just made the switch!


Source: Mary Meeker , 2012, http://www.kpcb.com/insights/2012-internet-trends

22

• Drivers are accelerating the need for NEW apps– Mobile, Mobile, Mobile, Social, Big Data, Etc.

• Time-to-market is more important than ever!– “I need it for yesterday!”– Can’t afford 18m cycles

Plus, you must be in DISCOVERY MODE(admit it: you don’t really know what’s needed!)

Drivers & Acceleration


+

+Jav

a.lang

.Stack

Overfl

owErro

r

23

• You are (t)asked– To do many more new applications– To deliver them much earlier– To deliver incremental changes very quickly– To measure whether what you are doing works– To scale quickly the few apps that will be successful– To have them interact with an increasing number of

different devices

… Yet, nothing else should change?

Impact


Java.l

ang.St

ackOve

rflowE

rror

Looking For a Solution?


25

• You should be able to EXPERIMENT• That means

– Get a first release out FAST (i.e. TTM)– TRY lots of different things (i.e. DISCOVERY)

• KEEP what works• KILL what doesn’t

– ITERATE FAST (i.e. GO GO G0)

Ideally…


26

• In theory, fast-iteration is great! But…

FAST_ITERATION + FRICTION => HEAT!

Not feasible in your current environment You need a no-friction environment

(hint: this is what the cloud is about)

“ Yes, but… ”


Cloud 101

XaaS


28

Traditional software stackWe have done this for 20 years!


Server

Hypervisor

VM

AS

JVM

LB

Application

You

29©2013 CloudBees, Inc. All Rights Reserved

Servers

OS

VM

AS

JVM

LB

ApplicationsCloud Computing: How to do it? Who does what?

Cloud Provider?

30

Easy: « Just use a XaaS! »


31

• SalesForce.com, Zendesk, NetSuite, etc.• Focuses on recurring IT needs

– CRM, e-mail, ERP, etc.– Very quick setup, but very limited

customization– Not for developers

• THINK– Standard applications

SaaS


32

• AWS – the most popular example• Server Lego blocks

– servers, storage blocks, DNS, IP addresses, etc.• Development environment:

– You rebuild your stacks “in the cloud”– Flexible but cOmpLi CaTed

• THINK– Custom-built systems

IaaS


33

ü Directly managing your IaaS resources provides you with resource elasticity (CAPEX-free), but…

O The amount of soft-IT typically required to do so is … higher!– All of the traditional IT activities remain

(maintain/patch/monitor OS+JVM+AS+DB++, etc.)– + cloud-specific items: elasticity/security/automation

• I can read your mind: this is where you will start• My advice? Do it and then move on…

IaaS – Consequences


34

• Cloud concepts are applied to Applications– On-demand, pay-as-you-go, elasticity, etc.– No need to handle updates, patches, scalability, failover, etc.

• This is about APPLICATIONS– NOT about systems– NOT about OS, middleware, etc.– You focus on your app, the PaaS

orchestrates the underlyinginfrastructure so “it happens”

• THINK– CUSTOM APPLICATIONS

PaaS


And how using the Cloud will help me?


36

• We focus a lot on the P/I/S of XaaS• But not enough on the … S!

• We are entering a SERVICE-ERA– IaaS - You are CONSUMING IT resources– SaaS - You are CONSUMING an online Service– PaaS - You are CONSUMING development services

This is not about software anymore!

The Cloud is refocusing IT roles


37

• What’s the big difference?– They’re just a long series of 1 and 0 at the end of the day,

right?

• A Services is not just “hosted software”– i.e. take your current software, host it, and keep doing what

you’ve been doing to date

Software vs. Service


38

Software != Service


Not the same …

[ requirement gathering | development |

QA |success measurement |

release | support ]

… processes

39

Packaged Software vs Cloud Software Delivery

Development Methodology VariesBonus: “We use CI”

Unachievable without Continuous Integration and Continuous Delivery


40

• Release v1.0

• Gather requirements for v2.0 • Work on v2.0 • Release v2.0don’t screw it up: once it is out, it is out.

• Have your customer upgrade– Support v1.0 for years

• Support and patch dozen of “branches” in parallel

• Gather requirements for v3.0 • Work on v3.0

• Release v0.1• Implement micro-feature, deploy,

measure success, keep or kill • Implement micro-feature, deploy,

measure success, keep or kill • Patch a bug• Implement micro-feature, deploy,



measure success, keep or kill

To Be Or Not To Be…M

on

ths/

Yea

rs

Day

s/W

eeks

Packaged Software As-A-Service


41

Time

v1.0 Release

v2.0 Release

Time

v1.0.x ContinuousReleases

Risk/Cost

Risk/Cost

Packaged Software As-A-Service

Lower Risk and Cheap Experimentation

Same basic value prop as agile, but code is being released continuously


42

• Integrated & automated “pipeline”• Always be in a “release-ready” state

Continuous Cloud Delivery


Code Build Test Stage Deploy

#Fail#Fail #Fail#Fail

43

Complex Scenarios are possible!

43

SQE Job

Database Tests

Compliance Tests

Stress Tests

Developer Job

Developer Tests

Functional Tests

X-Team Tests

Security Tests

Access Tests

Production Job

Backup Production

Deploy

Monitor

Developer Org

QA Org

Production Org

Promote

Promote


44

• Your Code Your Service• Always be in a “deploy-ready state”

– This requires a NO-friction environment• Enables high-frequency iterations

– Code, test, deploy, measure, keep-or-kill

Continuous Cloud Delivery


Why am I telling you all of this?

Enter into the world of CloudBees!

©2012 Cloud Bees, Inc. All Rights Reserved

46

The CloudBees Platform

cloudbees.com

Test

Stage

Build

Code

JenkinsExecutor

Fo

rg

e

Re

po

si

to

ri

es

Git

SVN

mvn

CloudBees API

Grand Central Web Console

SDK

SessionClustering

MySQL

Router

Application

PartnerEcosystem


Code Test Stage Deploy ManageBuild

Develop Deploy

Iterate

47

What You Interact With When You Use CloudBees

47

CLI/SDKGrandCentral Web

Console IDE



Integrated « add-ons » for DEV and RUN


Flexible Deployments

cloudbees.com

Amazon - US

Amazon - EUR

AmazonVirtual Private CloudUS or EUR

HP Cloud ServicesOpenStack

• Fully integrated with Amazon AWS• Provided as a SERVICE, nothing for you to maintain/patch

SDK

Developers

EnterpriseDatacenter

50

ClickStartsGet started in a snap!


51

ClickStacksReuse or create your own software stack


CustomerManaged

CommunityManaged

CloudBeesCurated

Fork

Push Push Clone

52

Development? No friction!

DeveloperCloudBees

PaaS


53

Deployment? No Friction!


Install AS on OS Configure AS Secure AS

provision nodesInstall load-balancer (LB)

Configure LB

Secure LB Setup BC for cluster QA cluster

Configure DNS

Provision node:

Provision cluster:

Provision node Install DB Configure DB

Secure DB Setup BC for DB

Provision DB:

Provision cluster Provision DB Deploy schema


Deploy App:



Analyse QA bulletins Analyse AS/OS logs


maintain DB


maintain FW Validate/QA stack


Acquire Hardware Provision node



Update Security

(transparent, multi-version,HA, scalable)

• No need to care about servers, load-balancers, firewalls, backups, etc.

• The environment is constantly managed and monitored

• Scalability happens in realtime

• Integrated failover/HA

• We do « Ops », you do « Dev »

Deploy app to traditional Java platform Deploy app to CloudBees

DEPLOY


54

Value à la carte – NO FRICTION

OURYOUR TEAMYOUR PROJECT


55

Typical Mobile App with Java back-end

???CustomMobile

Back-end

Fast-IterationDiscovery

StableNO-discovery


56

Continuous Cloud Delivery at CloudBees

Jenkins Core

Jenkins Plugins

Staging

Production

Web Console

SDK

API

Identity

Provisioning

Monitoring

Messaging

Services Platform

Generic App Container

Database

Router

Curated Stack

Git

SVN

Forge

Unit Tests

StageCross Service Tests UAT Prod

As needed, multiple

times per day


Is this real?!?

©2013 Cloud Bees, Inc. All Rights Reserved

58

• Lose it!– Mobile application– >> millions of users– > 30,000 transactions per minute, at peak time

• And… only 4 employees and …2 developers!– No IT, no servers, no DevOps– Complete focus on SOFTWARE DEVELOPMENT– Anything else is handled by CloudBees

• Unmatched productivity level!– This is possible TODAY– Will you wait for your competitors to shoot first?

Example: Lose it!


59

Yeah, but the cloud is just for startups!

0

2

4

6

8

10

12

Estimated Amazon Web Services Revenue ($B)

Source: Macquarie Capital; Morgan Stanley


60

… Or not!

0

2

4

6

8

10

12

Estimated Amazon Web Services Revenue ($B)

Source: Macquarie Capital; Morgan Stanley

88% CAGR(2009-2012)

FY12: VMW: $4.3B, RHT: $1.1B Dell Server, Storage, Networking: $11B


61

• This is « the industrialization of IT »• Went there, done that (many times):

« I am against the cloud »

Specialisation, standardisation, regulation, etc.

economy of scale, higher availability, etc.

… unstoppable!


First Demo• Complete Java Server Application…

• (Backbone.js front-end, REST back-end, DB)

• … with full lifecycle automation …• (Code, build, test, deploy)

• … and do it in a single click!

ClickStart

63

cloudbees.com

What’s up with this new framework!

Get started in a snap! “ClickStart”

CloudBeesGit

Repo

Database

Build/Testdb:create

App

PartnerServicePartnerService

CloudBees API

Grand Central Web

Console

app:create

gi t cl one Jenkins job create

Bind

Bind

Trigger

Deploy

Wow! At yourservice

©2013 CloudBees, Inc. All rights reserved

64

cloudbees.com

Continous Delivery at work

CloudBeesGit

Repo

Database

Build/Test

App

PartnerServicePartnerService

Bind

Bind

TriggerDeployI want to make some

changes!

Local GitRepo gi t cl one

gi

t

co

mm

it

gi t push

Ch

an

ge

s

Wow! At yourservice


#fail #fail #fail

Mobile!

Code Build App Test Beta

DeployGADeploy

• Get started fast!• Team Work• Automatically

push new code to test

#fail

• Build on several targets: iOS, Android, etc.

• If it fails, stop

• Test is costly!• Easy-frst, then

increase!• Basic test• Multi-device• Gesture• Etc.

• If it fails, stop

• Deploy in the feld on real users

• Gather input• Retrieve dumps• Etc.• If it fails, stop

• D-Day! • Push apps to

real-life users!• If it fails, stop

Build Matrix• Testing mobile application requires testing on lots

of different devices, form factors, etc.

• Create one build job per form factor/generation?

• No!

• Jenkins Build Matrix!

Jenkins Build Matrix

http://wiki.cloudbees.com/bin/view/DEV/Matrixed_Jenkins_Android_Builds

So much more!

Getting started• Lots of new things to learn!

• (delegation of trust, access to legacy data, latency, elasticity, iterativity, etc.)

• Identify an application

• Start small & Non-critical

• Focus on « Learning »

• And extend from there

• (and enjoy!)

70

• We are entering a SERVICE-era• The Cloud == the commoditization of IT

– Read: it is inevitable• Packaged software is dead

– Too long cycles, no ability to measure, etc.• How can you remain competitive?

I’m ready to deploy! Are you?

TIME TO WAKE-UP!


Thank you

@SachaLabourey

“It is easier to ask forgiveness than it is to ask permission”-- Grace Hopper

Documents

Next Generation Software Development: Continuous Cloud Deployment · PDF file · 2013-04-18Next Generation Software Development: Continuous Cloud Deployment @SachaLabourey CEO, CloudBees,