What you should know about software measurement platforms

IT Leader’s Guide to Enterprise Software Quality Solutions

What you need to know about the engine that will power your software quality initiative

CAST Copyright © 2014 - 2016 1

Contents

I. Taking the First Step

II. Software Measurement Essentials

III. Do-It-Yourself Solutions

IV. Enterprise Solutions

V. Total Cost of Ownership Analysis


Taking the First Step

Congratulations! Your organization,

from developers up to executives, has

officially recognized that software quality

measurement is a key business value

driver. Not only is this a demonstration

of your company’s maturity, it is a

strategic inflection point that, if executed

properly, will propel you toward

sustainable returns.

Companies who have thoughtfully

deployed software quality measurement

platforms have experienced:

Improved collaboration between IT

and business

Reduced time-to-market

Expanded revenue generation

opportunities

Increased user and customer

satisfaction

Now, you just have to make sure that

your new strategic focus on software

quality has the platform to help you

accurately measure the most relevant

metrics and provide the right visibility

to the right people.

Sounds easy, right? Well, it depends

on what you use to power your

initiative.

3

Software Quality Measurement Essentials • Essential Capabilities

• Solution Breadth and Depth


Software Quality Measurement Essentials

There are many offers today in the

code analysis market space. How do

you choose the right one?

Here’s a quick look at the most

essential capabilities of a software

quality measurement solution.

Solutions that fail to offer all of these

capabilities will not deliver the value

you are seeking.

A comprehensive software quality measurement platform must address all of your goals. A “good-enough” approach may give you “not-enough” results.

Measurement precision

Wide technology scope

Accurate analytical methods

Management visibility

Enterprise scalability

Measurements must be consistent over

time, and trends must be meaningful

Ability to analyze the technologies within

your most critical application

Accurate analysis must examine the

entire application system and in between

layers, beyond a single code unit

Easily scalable to all critical applications

and accessible by 100 or more staff

Have intuitive analytics for management

to quickly track performance & hotspots

Software Quality Essentials Description Consequences of “Good-Enough”

Approach

Ambiguous measurements cannot help

you make the right business decisions.

Exposes critical applications to high

risk, or incur costs for custom solutions

Inaccurate and irrelevant

measurements will force wrong and

costly decisions

Inefficiently run several instances,

reports are manually aggregated

Quality program will lose momentum

without management involved

Follow industry standards for quality

measurements and best practices (e.g.

OMG, CISQ, SEI)

Rooted in industry standards Program will be challenged by all

involved and quickly lose credibility


Breadth and Depth of the Solution A truly meaningful and valuable software quality measurement platform must have breadth and depth.

Wide technology coverage that tidily addresses your most critical applications

Capability to analyze all layers of your critical applications; from code, to component, to system

Breadth

Depth

import java.applet.*;

import java.awt.*;

public class SecondApplet extends Applet {

static final String message = "Hello

World";

private Font font;

// One-time initialization for the

applet

public void init() {

font = new Font("Helvetica",

Font.BOLD, 48);

}

// Draw the applet whenever necessary.

Do some fancy graphics.

public void paint(Graphics g) {

// The pink oval

g.setColor(Color.pink);

g.fillOval(10, 10, 330, 100);

// The red outline. java doesn't

support wide lines, so we

Unit Level • Code quality at the developer

level

Component Level • Entire technology layer

• Inter-application invocation

• Single technology

System Level • Entire application

• Interaction between components

• Architectural cohesion

6

Do-It-Yourself Solutions • The Lure of “Simplicity”

• Tangible Costs of DIY

• Hidden Costs of DIY


Beware of the Lure of “Simplicity”

Developing your own aggregation platform using statistics from single-technology code analyzers may appear to be a simple and frugal option to power your software quality program. But, they do not meet many of software quality measurement essential capabilities, and can cost you more long term.

Many technical organizations are drawn to

single-technology analyzers when they

begin to formulate their strategy around

software quality measurement. While its

low up front costs and seemingly endless

customization potential make these

analyzers an intriguing way to approach

software quality, there are a myriad of

downsides.

A Component of the Whole

To start, code analyzers are only a

component of a complete software quality

measurement platform. While licensing

the analyzers may be cheap, many

companies have quickly learned that they

have to either invest development

resources in creating a complex

aggregation platform or purchase an

commercially-available aggregator in

order to effectively track and report on

business-relevant metrics.

“Close-Enough” Measurements

Aggregating results from individual

analyzers may lead you down the wrong

path. While data form individual

analyzers can be aggregated, it does not

paint the whole picture. Issues at the

code level, do not have the same impact

as issues at the system-level, where

interactions between components can

cause catastrophic system failure. Only

looking at measurements at the code level

is a in accurate approach that may prompt

you to spend resources fixing a problem

that has very little business value.

Unfortunately, in this case, the whole is

not greater than the sum of its parts.

You Own the Platform - Completely

Analyzer components have to be updated

frequently when new versions are

released. New releases can destabilize

the platform, or worse – completely

change the results of the measurements

you have been taking.

You see, independent components do not

have to follow a coherent roadmap.

Product development is driven by a single

technology or language, and not based on

your overall enterprise needs.


The Tangible Costs of DIY

$1.2M

INITIAL DEVELOPMENT 5 Technical Staff

$240K

PROJECT MANGEMENT 1 Project Manager

$300K

SOFTWARE QUALITY 1 Software Quality Consultant

BUILD

$340K / year

PLATFORM R&D 2 Technical Staff + Hardware

$120K / year

TRAINING & SUPPORT 1 Technical Staff

$180K / year

STRATEGY & MANGEMENT 1 Application Owner

$75K / year

SOFTWARE QUALITY 0.5 Software Quality Consultant

MAINTAIN

So, what are the costs of building and operating your own code analyzer aggregator platform? We did the math for you. These estimates are based on

resources located in the U.S.

We also assume that the

platform was built with a “good-

enough” approach. A platform

that satisfies the “software

quality measurement essentials

checklist” (p. 4) will likely cost

exponentially more.

$715K / year $1.74m over 2 years

CAST Copyright © 2014 - 2016

1. Project Scope Creep – Currently

available single-technology code

analyzers primarily address Java. If

your applications are in C, COBOL,

.Net, or other commonly used

languages, you may need to purchase

or extend additional custom

analyzers. This means a significant

expansion in the scope of the project.

2. Scarce Expertise – Doing it yourself

doesn’t automatically mean

inexpensive. Many companies hire

expensive code quality consultants in

order to complete the platform.

3. Inability to Scale – Many

aggregators work very well at an

individual developer or small team

scale. However, an organizational-

wide adoption of software analysis

requires sharing of information, and

easily accessible visibility.

4. Opportunity Cost – We estimate that

it takes five resources over two years

to build an adequate software quality

analysis platform (10 man-years).

That means these resources have to

deprioritize tasks that may have better

returns.

5. Waiting for Return – Typically the

value of software quality analysis and

the process that implement to address

it begin to materialize six to 12

months after initial implementation.

Building your own means that the

return is much further away.

6. Licensing Legality – Although the

initial license had no cost, many

companies face legal action due to

mismanagement of code analyzer

components—a serious drain on

resources, money, and talent.

7. Component Updates – New releases

of code analyzers must be

researched, before upgrading. You

may face several new releases within

a very short time frame, misaligned

with your release schedule.

8. Version Control – Some components

in your application may be written

using the same technology, (e.g.

JDK), but in different versions (e.g.

JDK 5 and JDK 6). When code

analyzers are updated they may

change language version support,

requiring you to have more than one

instance of the analyzer.

9

The Hidden Costs of DIY

Building and supporting a software quality measurement platform using single-technology analyzers is not a small task. The length and hidden costs of the projects are alarming.

10

Enterprise Software Quality Measurement Solutions • Advantages of an Enterprise Solution

• Managed Services – A Prudent Approach


Advantages of an Enterprise Solution

Investing in a balanced and refined software quality measurement platform produced by a company with sound expertise and business understanding is the wiser option.

Enterprise software quality

measurement solution

Software Quality Expertise

A provider who has dedicated all of its time

and energy in software quality solutions will

likely deliver a more refined and valuable

product, than a solution that is built in-house

with individual code analyzers. By buying

an enterprise solution you can accelerate

toward returns through their specialization in

the space.

Customer Driven Road map

You have a bigger voice in your provider’s

roadmap decisions than with single-

technology components where thousands of

voices are competing to be heard. With a

larger share-of-voice you have more control

overall how future versions of a product fit

your needs.

Dedicated Support

When you do business with a solution

provider, you are the boss. Not only do you

have access to expert user support, you can

also enlist help for implementation,

adoption, and operation of the platform from

people who really know it well.

Lower Cost of Ownership

With no need for development resources

and simple update packages, enterprise

solutions will cost you less to own and

operate in the longer term. This also allows

your company to focus on its core

competency, selling the products and

services you know best.


Managed Services – A Prudent Approach

Risk has become a centerpiece in enterprise software with the growing need of companies’ to innovate through new solutions.

Managed services allow companies to quickly implement and try new software at low risk, and, for many, has become the only way to buy new software.

Implementing and operating an enterprise

software quality measurement platform by

using the provider’s managed services

can have many benefits.

1. Shift Risk to Provider

Let your provider take on the risks of their

own solution. Traditionally, to onboard a

new software solution, you may need to

dedicate project management resources,

staff to operate and support the new

application, and invest in some additional

hardware.

With managed services, your provider will

provide the man power and hardware to

deploy the software. In the case of

software quality measurement platforms,

this means that they will:

Stand up physical and logical

infrastructure

Define the code gathering and

analysis process

Design the consumption model

Determine channels of

communication

It also means that you do not have to hire

more staff, or worry about what will

happen to them if the project is aborted.

2. Greater Budget Control

Buying software in a managed services

model means that a large portion of the

funds can be sourced from the operational

expenses bucket instead of capital

expenses. This enables companies to

make faster decisions to start or end a

project.

3. Accelerate to Time to Value

Because the providers are experts in how

to deploy and operate their software, they

will enable to you get ramped up and start

measuring your software quality in a

meaningful way in much less time than if

you managed it on your own. This helps

you accelerate toward your payback

period and vastly increases your returns.

4. Capture Opportunity Cost

A managed services model allows you to

focus on your key value-adding activities.

Operational distractions such as bugs,

performance issues, and upgrades are all

handled seamlessly by the provider

without your involvement.

CAST Copyright © 2014 - 2016

About CAST “CAST’s holistic system approach, looking at the

architecture, transactions, control, and data flow across multiple technologies, may be very

beneficial, with numerous

engineering studies showing that bad software engineering practices in the ways components are interrelated and interact…account for only 10% of total defects, but can lead to 90% of production issues.”

CAST is a pioneer and world

leader in software analysis and

measurement, with unique

technology resulting from more

than $100 million in R&D

investment. CAST introduces

fact-based transparency into

application development and

sourcing to transform it into a

management discipline. More

than 250 companies across all

industry sectors and geographies

rely on CAST to prevent business

disruption while reducing hard IT

costs and software risk. CAST is

an integral part of software

delivery and maintenance at the

world's leading IT service

providers. Founded in 1990,

CAST is listed on NYSE-Euronext

(Euronext: CAS) and serves IT

intensive enterprises worldwide

with offices in North America,

Europe and India.

www.castsoftware.com

http://www.castsoftware.com/