Quantifying DevOps Adoption Empirically for Demonstrable ROIDES-1720

Insight

Operations

Interaction

Insight

Operations

Interaction

Innovation

and Speed

Optimize

Friction Reduction with multi-speed IT

Optimize

Innovate

Focus on Operational Costs

• Consolidation and modernization

• Operations automation

• Risk and compliance mgmt

Empowered Process Team

Legacy Infrastructure

Focus on Speed and Agility

• Assemble solutions from verified

components

• Fast development and redeployment

• User first delivery model

Systems of Engagement

Rapid Service Creation Team

PaaS Engine

Insight

Operations

Hybrid

Enterprise

Mobile

Social

Interaction

Service Integration Team

Brokerage Services

DevOps

API Economy

Cloud Service Brokers

Systems of Record

Reduction

Why DevOps is Important

Line-of-

businessCustomer

1

3

2

1. Get ideas into production fast

2. Get people to use it

3. Get feedback

Adopt DevOps approach to continuously

manage changes, obtain feedback and ,

deliver changes to users

Eliminate any

activity that is not

necessary for

learning what

customers want

What DevOps Practices should Address…

Manual Effort

Unproductive Work Poor Visibility

Long Wait Times

Lets Walk through a Story…

• Meet Julie

• Julie works in the LOB and has an Idea for a new

solution to address a key business need.

…Julie is daunted by the current process:

Business Need

•Document

•Weeks

Approval

Process

•Meetings

•Weeks

Hardware

Purchase

•Negotiations

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

Traditional Product Development

Months before you find out whether the product meets the need

Business Need

•Document

•Weeks

Approval

Process

•Meetings

•Weeks

Hardware

Purchase

•Negotiations

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

Introducing IaaS

• Months before you find out whether the product meets the need

• Hardware provisioning is undifferentiated heavy lifting

Business Need

•Document

•Weeks

Approval

Process

•Meetings

•Weeks

Hardware

Purchase

•Negotiations

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

IaaS

Cloud

IaaS Summary

• Months before you find out whether the product meets the need

• Hardware provisioning is undifferentiated heavy lifting

Business Need

•Document

•Weeks

Approval

Process

•Meetings

•Weeks

Hardware

Purchase

•Negotiations

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

IaaS

Cloud

Business Need

•Document

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

From months to

weeks is good.

Can we do better?

IaaS based Product Development

• Weeks before you find out whether the product meets the need

Business Need

•Document

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

IaaS based Product Development with DevOps

• Improved efficiencies (including automation) between development and

deployment has a significant impact in consistency and speed

Business Need

•Document

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

Dev

Ops

IaaS based Product Development with DevOps

Business Need

•Document

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

Automation is great and

reduces the time to days

But I want more!

Dev

Ops

Introducing PaaS for Feature Delivery

Business Need

•Document

•Weeks

Software

Development

•Specifications

•Weeks

Deployment

and Testing

•Reports

•Weeks

Customer

Feedback

•Doesn’t work!

•Weeks

DevOpsPaaS

w/DevOp

s

PaaS based Feature Development

Business Need

•Discussion

•Days

• Design

Thinking

Software

Development

•Code

•Weeks-Days

Customer

Feedback

•Fix this Bit!

•Hours

Days!

That is what I’m

talking about!

15

“The Technology Tree”

DevOps “Technology Tree” (Phase 1)

Phase 1 Goal:

Continuous

Testing

Starting Point

Step #1: Automated

Builds Step #2: Automated Functional Testing

as part of build process

Step #3: Code Scanning and Security

scanning as part of build process Step #4: Shift Left Testing (Integration

Testing before Functional Testing)

Step #5: Automated application

deployment to Test Environment

Step 6: On-Demand Test

Environment Provisioning

DevOps “Technology Tree” (Phase 2)Phase 2 Goal:

Continuous

Delivery

Starting Point

Step #1: Continuous

TestingStep #2: Automated application deployment to test

environment along with middleware configuration

(standard topologies)

Step #3: Automated application

deployment and middleware

configuration across DEV, QA, and Prod

Application Portfolio Release

Management

Step #5: Self Service build,

provision, and deploy (Cloud)

Step #4: Manage Environments

through automation

What do you see?

52% of users

don’t have

confidence in their information1

42% of managers

use wrong

informationat least once a week2

59% of managers

miss

informationthey should have used2

ProductProfitability

Product Variance

And HealthWorkforce

OptimizationDistributed and

Outsourced Development

Contradictions result from the quest for information

Metrics That Matter

Metrics That Matter

Category Metrics

Cost / Budget Maintenance/Innovation Ratio (% spent on bug fixes/technical debt vs new capabilities)

Assignment scope (Number of function points supported by person)

Budget underage/overage

Time / Speed Velocity Planned (Throughput of team per iteration/milestone)

Velocity Actual (Historical Velocity)

Feature cycle time (Time / Effort for developing a feature)

Turnaround Time for Changes (CR or Defect -> Production)

On-time delivery % (# of projects delivered on time / Total # of projects)

Quality / Reliability Uptime/MTBF (Mean time between failures)

Defect Backlog (# of defects not fixed in a release)

Total Tests (# of tests for application)

Total % Tests Automated

Test Escapes (# of defects not found in testing)

Support Call Volume

Defect Density (# of defects per Function point or KLOC/KELOC)

Regression (# of defects fixed that have creeped back into system)

Requirements Coverage (Total # of Requirements vs # of Requirements implemented)

Requirements Volatility (Measuring changes in requirements by lifecycle)

System Utilization (Measures Resource Utilization during performance/stress testing)

Test Case Efficiency (Ratio of Defects found in Test Cases vs total Defects)

Size (Underlying) Story Points

KLOC, KELOC

Function Points

Backfiring (KELOC -> Function Point mapping)

Code Complexity (# of methods that have >15 linear independent paths)

Percent Reuse ((Reused code count / (Reused code count + total code count)) * 100)

Rational Enterprise DevOps solutions save money and increase resource capacity

• By increasing staff capacity through productivity improvements

– Examples include:

• Faster code development and maintenance

• Faster and improved impact analysis and testing

• Build automation

• Quality Management and Test Automation

• Configuration Management

• Time to market

• By discovering defects earlier in the lifecycle thus reducing the repair costs

– Examples include:

• Improved Requirements Analysis and Management

• Code Analysis

• “Shift Left” strategy. Using transparency, continuous integration and testing to identify issues early

23

Quality

Productivity

Monetizing the Value of Productivity Improvement

How do you measure value?

– Lines of Code (LOC) and Function

Point (FP) – a lot of baseline data available, but poor indicators of value.

– Story Points – used by agile teams,

often no baseline data exists…

– Business Points¹ – few measure it, but

focus on value. Often, no baseline data

exists…

• Solution:

• Measure Velocity, that is, how much faster is the team working

now relative to solution adoption.

• To calculate velocity of your agile team, simply add up the estimates of the features, user stories, requirements or backlog items successfully delivered in an iteration.

24

¹While Story Points reflects the size of the implementation effort, Business Points

reflect the business value associated with each story / epic / scenario / …

Monetizing the Value of Quality Improvement

Two key drivers:

1. Fewer defects found post deployment

– A result of better test coverage, automated and frequent testing

2. The average defect is found earlier in the lifecycle, when the

cost of finding them is lower

25

Case Study: Enterprise DevOps Cost/Benefit Analysis

26

IBM Rational DevOps Business Case Assumptions

• Average staff costs $65/hr.

• 400 users

• 5-year investment analysis

• Adoption Rates:

–30% after 12 months

–60% after 24 months

–90% after 36 months

–100% after 48 months

Business Value Model – Productivity Impact

29

$3.4 million annual savings through combined FTE time

Increase software delivery capacity of 26 FTE from 3.3% productivity improvements

Expected Productivity Impact Causation

30

Business Value Model – Quality Impact

31

Based upon other Client implementations and recorded results, this initial draft proposes a left shift in defect detection and resulting $1.8m cost reduction

Expected Quality Impact Causation

Business Value Model

Cost Benefit Summary

Business Value Assessment

Preparing a Business Value Assessment includes the following steps:

1.Initial presentation to get Client participation

2.Interview with Client to capture processes and metrics

3.Presentation slides, analysis document & spreadsheet

4.Review and acceptance with Client / Sponsor

The BVA is a consultative sales method that

documents the business impact with the client of a

proposed solution. Following this, it compares the

cost of a proposed solution to the business value so

that an ROI can be presented.

The BVA communicates and prioritizes the

proposed solution business value to the executive

and financial level just as the technical validation

sells to the practitioner and technology level

.

34

System Z DevOps Workshop DescriptionThe System Z DevOps Workshop is a no cost ‘discovery’ workshop to identify gaps in the client’s current System Z software delivery capabilities and provide them with a customized, prioritized list of DevOps practices and a roadmap for their adoption to support faster and more efficient enterprise software delivery.

The target audience for the workshop are executives and senior development and operations managers. The workshop is designed to last 4 to 6 hours. It is preceded by a 30 minute pre-call and followed up with a 2 hour presentation of results and recommendations.

The workshop is meant to be a strategic initiative to help the customer develop a pragmatic approach for adoption of Enterprise DevOps best practices building on initiatives and capabilities already in place

Next Steps / Resources

• DES-4657: Enterprise DevOps Cost/Benefit Analysis Workshop at InterConnect

• Take the DevOps Practices Self Assessment

http://www.ibm.com/ibm/devops/us/en/

• Additional Resources

http://www.ibm.com/devops

https://www.ibm.com/developerworks/devops/

36

Notices and Disclaimers

37

Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission

from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of

initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS

DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE

USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY.

IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers

have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in

which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials

and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or

their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and

interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such

laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

Notices and Disclaimers Con’t.

38

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not

tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products.

Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the

ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT

NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual

property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®,

FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG,

Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®,

PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®,

StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business

Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM

trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

Thank YouYour Feedback is Important!

Access the InterConnect 2016 Conference Attendee

Portal to complete your session surveys from your

smartphone,

laptop or conference kiosk.