An Integrated Pipeline for Private and Public Clouds with Jenkins, Artifactory and Pivotal Cloud Foundry by Jamie O’Meara

An Integrated Pipeline for Private and Public Clouds with Jenkins, Artifactory and Pivotal Cloud Foundry

An Integrated Pipeline for Private and Public Clouds with Jenkins, Artifactory and Pivotal Cloud Foundry

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Cloud Native Enthusiast22 years of building stuff .

: @JamieOMeara Who am I?

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Software is Changing Industries

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/

Software is a difficult business. It can 1-2 years to build an enterprise data center app, the risk is high and the return on investment tends to be all or nothing. The appeal of software is it changes industries. If you look at the companies listed theyve all used software to change their industry. For example, Uber using a mobile app changed our perspective on car services and transportation in the city. Square changed our mind set on how and where we can conduct a business transaction. And Netflix changed the way we consume entertainment and media.

Weve entered an era in software development where the emphasis is on rapid delivery. The companies listed are disruptive in their respective industries for a number of reasons. One thing they share in common is a very active software delivery pipeline. For example, Square and Airbnb have released their iOS applications 9 and 12 times since January of this year respectively.

Cloud Foundry is Changing How Apps are Built and Operated

#jenkinsconf


I think its fair to say if software is changing industries then Cloud Foundry is changing How apps are built and operated. The companies listed here recognize cloud native apps will be built and operated differently then traditional apps.

What is Cloud Native?http://tinyurl.com/CN-Platform

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/This is Cloud Native

Structure (teams, skills, pairing, size) Values (test, releases) Experience

Frequent releases delivered to users User Feedback Loops and Metrics

Cloud Native Platform Release Engineering

CultureCommunicationTechnology

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Getting Started

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/The Cloud Native Engine

Cloud Native Operations

Cloud Native Applications

ReleaseEngineering

The Developer Experience

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Cloud Native Platform

NetworkingPolicy, Access, Services

ComputeUtilization, Consumption

Storage as a ServiceConsumption, ServicesCloud Native Applications

App Routing

External Connectivity

Concurrency

Services

Service Registration/Discovery

Logs

Metrics Creation

External Configuration

Failure Isolation

DisposableCloud Native Operations

Containerization

Orchestration/Scheduling

Security

Provisioning/Lifecycle Mgmt

Scaling

Logs/Metrics

Role Mgmt

LB and App Routing

Services

Health Mgmt

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Release EngineeringDeliverContinuous DeploymentContinuous DeliveryHybrid CloudConfiguration ManagementRunBlue/Green DeploymentBaselined DeliveryProductionObject-OrientedTest Driven DevelopmentContinuous Integration

Build

#jenkinsconf


I believe the transformation in software development and delivery leads to higher quality software. For years, weve focused on creating higher quality software. Unit test, functional tests, UI tests and integration tests is where we spent the earlier years of agile software development. We shifted to new methodologies like Test Driven Development to change our mind set about the software we produce. Continuous integration allowed us to feel confident about the software we wrote and validate its ability to pass testing. Continuous deployment and delivery are ushering in a new set of standards helping us to continue to validate our software builds and ensure the delivery of those builds is as successful as the build phase. This approach needs to work across delivery boundaries. If we deliver to AWS today we must be able to feel confident our delivery to on-premise tomorrow is not compromised by proprietary APIs.Configuration management is also striving for a better approach to software delivery. Gone are the days of property files and isolated black box deployments. Configuration management is moving to the source control repo. Configuration settings are provided by the platform and can be changed and version controlled.Techniques like Blue/Green deployments are making organizations reconsider the definition of a production deployment. Using these techniques reduces the risk of a bad deployments, lessens the stress and burden on developers and operators and increases the feedback loop on creating better software.

Release Engineering for PWS

15500 Application Instances (Containers)35,000 Active Users

2~5000 Routes

320 Zero-downtime Platform Upgrades in 2015

4https://github.com/cloudfoundry/cf-release/tags

#jenkinsconf


In my opinion, a key element to continuous delivery is to use Platform as a Service. I think these 4 key elements are an absolute must for evaluating a platform,

A platform should have a strong, vibrant open source community. Cloud Foundry is the most popular open source platform today. A platform provides a clean abstraction from underlying details of a Cloud or IaaS provider. It should be portable across Cloud/IaaS providers. The platform should provide a layer to scale at the cloud and application level. An API centric approach is always good for developers who are truly the owner and stewart to the application.

Blue/Green Deployment

CF RouterBlueBlueGreenBlueGreenCF RouterCF RouterGreenCF Router

1234

app.example.com

app.example.comapp-new.example.comapp.example.com

app-new.example.com

app.example.com

app.example.com

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Simple Pipeline Demo

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Cloud Foundry Foundationhttp://maps-dev.cfapps.io

App ConsoleOps ConsoleMarketPlaceOrgs

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Simple Demo Flow

Cloud Foundry

Public Foundation

Private FoundationDevelopersIDE

DeployJobDeployJob

cf pushcf push

Organization

Dev

Test

Productionhttp://maps-dev.piv.io

Organization

Dev

Test

Productionhttp://maps-dev.cfapps.io

Lattice

#jenkinsconf


#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Best Practices/LessonsUse the CB Jenkins CF pluginHash commits and Build numbers for versioning.Canonical Route Names for easy of use and external access.Flyway for database migrations (flywaydb.org)SCM for config details and env variables.CD design includes multi-cloud for portability and scale.

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/Future Designs and TopicsJenkins WorkflowsAsynchronous pipelines and check pointsOrchestration of micro services.Docker images via Cloud Foundry DiegoOn-premise cloud scale deployment of Jenkins on Cloud Foundry.CF plugin auto-pilot (Alternative)plugins.cloudfoundry.org

#jenkinsconf

Unless otherwise indicated, these slides are 2013-2015Pivotal Software, Inc. and licensedunder aCreative Commons Attribution-NonCommercial license:http://creativecommons.org/licenses/by-nc/3.0/References and Sourcehttp://github.com/omearaj/App-Pipeline

#jenkinsconf
