Collaborative Quality Management

Greater confidence. Improved business performance. Delivering software quality as a

strategic competency

IBM Rational Quality Manager

www.strongback.us 2

Agenda

Challenges

Executive summary & the Jazz vision

Rational Quality Management

Customer results

www.strongback.us

Rain Sensing Wiper: Example of a System Design FailureIndividual Systems Worked, But Failed When Integrated

3

Windshield provided by local supplier Incompatible with the operation

range of the sensor No captured requirement for

proper system calibration (i.e., verifying sensor and windshield compatibility) Cars were sent to customers with

non-functioning wiper system

Initial diagnostics designated software as culprit for malfunction Mechanics couldn’t test software behavior Other components (electronic control unit, sensor, and windshield) functioned normally when tested independently Failure was not of individual components, but in the interaction at a system

level

www.strongback.us 4

The Cost of Poor Software FunctionalityMoody’s Investment Services’ $1.7 billion dollar defect

Software bug incorrectly assigns positive safe credit ratings to $4 Billion of high risk complex debt securities

Moody’s market value tumbles $1.7 Billion in one day

www.strongback.us 5

The Cost of Poor Software PerformanceA failed business-critical application can have a major impact on business

www.strongback.us 6

No. of Cha nge sM odule A 12M odule B 9M odule C 15M odule D 10M odule E 43M odule F 6

Applying automation where it matters the most

Test Suites covering Many critical Requirements i.e. high business value.

Test Suites

No. of Requirements

Test Suites covering All critical Requirements“20% of Test cases covering 80% of Business value” (Barry Boehm)

Test Suites with high Requirements coverage

Test Suites with low contribution

“80% of the defects come from 20% of the modules” (Barry Boehm)

=> Most changed modules are often most defect-prone (Mockus, French)

www.strongback.us 7

Agenda

Challenges



Customer results

www.strongback.us 8

Delivering greater value from your investments in software

Executive summaryHelp customers continuously improve their ability to achieve desired business outcomes and deliver enduring software and product quality Cut risk and cost

Collaborate seamlessly to reduce rework and the cost of bugs with integrated processes aligned to business goals

Speed time to market Automation reduces error-prone tasks and improve

consistency and operational efficiency

Make confident decisions Manage to business objectives. Anywhere. In real-time with

automated data collection and analysis using customized quality reports and dashboards

www.strongback.us 9

Successful businesses will manage software and systems delivery as a robust business process

Achieve common goals by optimizing how

people work

Increase controland efficiency by

integrating workflows

Collaborate

Continuously improveby measuring progress

in real time

ReportAutomate

www.strongback.us

Team Concert

Collaborative Software Development and Software Delivery Agility

Collaborative SCM, Change Management, Agile Planning, Dashboards / Metrics, Lifecycle

Process Management

Quality ManagerLifecycle Quality Management

Coordinate quality assurance plans, processes and resources

Enable ALM across your organization

Requirements Composer

Elicit, capture, elaborate, discuss and review requirements

Business Expert and Stakeholder Collaboration

StorageCollaboration

QueryDiscovery

Administration: Users, projects,

process

Best Practice Processes

Presentation:Mashups

FutureIBM

Capabilities

Product & Project

Management

Application Lifecycle

Management Engineering& Software

Tools

BusinessPlanning &AlignmentYour

existing capabilities

3rd-PartyJazz

Capabilities

Compliance& Security

Existing / Future Investments

ClearCase, ClearQuest, RequisitePro..

Software Delivery Platform ProductsOther owned Vendor Products

COTS, Business Management, etc

Custom Developed

Open Services for Lifecycle Collaboration

www.open-services.net

Open Services Standards based framework

www.strongback.us

Co

nsu

me

Co

nsu

me

Co

nsu

me

Co

nsu

me

Integration framework for ALM protecting existing and future customer investments

Requirements Management

Services

Change Management

Services

Quality Management

Services

Open Services for Lifecycle CollaborationArchitecture Management

Services

Publishing Services

Pu

bli

sh

Co

nsu

me

Pu

bli

shPu

bli

sh

Pu

bli

sh Pu

bli

sh

c

Existing Rational Offerings

New Rational/ IBM Offerings

Business PartnerOfferings

Co

nsu

me

Co

nsu

me

Co

nsu

me

Co

nsu

me


Services

Change Management

Services

Quality Management

Services

Open Services for Lifecycle CollaborationArchitecture Management

Services

Publishing Services

Pu

bli

sh

Co

nsu

me

Pu

bli

shPu

bli

sh

Pu

bli

sh Pu

bli

sh

c




c




www.strongback.us 12

Lower the cost of delivering quality solutionsOrchestrate across teams with ALM integration for maximum transparency and traceability of assets

Architect Developer

REST API

QualityManagement

REST API

Requirements Definition & Management

Change & Configuration Management

REST API

Association

Changes; notification

Tester finds defects,

Defect statusDefects closed

Requirement linked to work itemAnalyst Tester

● Coordinate analysts, developers and testers

● Collaborate in context and track status

● Align teams on project plans● Which requirements are in the development plan?

● Track builds and the impact of defects● What build is ready for test? ● What defects are blocking test from proceeding?

● Track status with C/ALM dashboard mashups ● Transparency across teams and artifacts


Centralized test management hub & full lifecycle support

Collaboration

Presentation:Mashups Discovery Query

Storage

Administration:Users, projects, process

Best Practice Processes

IBM Collaborative Application Lifecycle Management

Quality Dashboard

Test Management

Create Plan

Build Tests

Manage Test Lab

Execute Tests

Report Results


DefectManagement

FunctionalTesting Performance

TestingWeb Service

Quality

CodeQuality

Security andCompliance

Open Platform

homegrown

Open Lifecycle Service IntegrationsJava System z, i

SAP .NETTest DataQuality


Stakeholder and team coordination reduces mistakes Risk identification and management leads to educated

prioritization decisions Test traceability linked to business requirements improves

customer satisfaction

Running tests earlier leads to reduced repair costs Running more tests in less time improves coverage Reducing manual labor leads to fewer testing errors Lab configuration automation improves efficiency and asset

utilization

Real-time dashboards enable proactive risk management Customizable reports facilitate ongoing process improvement

IBM Rational Quality ManagerA central hub for business-driven software quality

Mitigate business risk with collaboration

Improve operational efficiency with automation

Make confident decisions with effortless reporting

IBM Rational Quality Manager

CONTINUOUS test plan participate

AUTOMATED context GOVERNANCEuse case distributed access dashboardssynchronize EASY HANDOFF trace LABUTILIZATION functional PERFORMANCEsecurity compliance

www.strongback.us

Gartner MarketScope for Application Life Cycle ManagementIBM Rational earns “Strong Positive” – the highest possible rating

IBM earns “Strong Positive”―the highest possible rating

The MarketScope is copyrighted November 11, 2010 by Gartner, Inc. and is reused with permission. The MarketScope is an evaluation of a marketplace at and for a specific time period. It depicts Gartner's analysis of how certain vendors measure against criteria for that marketplace, as defined by Gartner. Gartner does not endorse any vendor, product or service depicted in the MarketScope, and does not advise technology users to select only those vendors with the highest rating. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. The MarketScope graphic was published by Gartner, Inc. as part of a larger research note and should be evaluated in the context of the entire report. The Gartner report is available upon request from IBM.

Efficient coordination and automation of the delivery process requires new, collaborative approaches to the planning, measurement, execution, control and reporting of activities.

These new approaches are what differentiate current application life cycle management (ALM) tools, and what make ALM processes vital to leading-edge development activities.

ALM is what enables sustainable agile practices. ALM creates a management framework providing consistent, auditable records of the decisions and activities of agile teams.


Agenda

Challenges



Customer results


Customer Speak!

Cut risk and cost Collaborate seamlessly to reduce rework and the cost of bugswith integrated processes aligned to business goals

Lower the cost of delivering quality solutionsOrchestrate across teams with ALM integration for maximum transparency and traceability of assets

Unify the team through real-time, in-context collaboration

A single, dynamic quality contract provides clear and accountable direction

* Source: IBM

“Testing consumes 20% to 40+% of the average

software application life cycle effort”*

I just got a budget cut, what testing should I eliminate? What impact will it have on

application production quality?

“Some large projects have found that 41% of all

defects have their origin in bad requirements.”*

Avoid disruption and achieve better business stability and project delivery predictabilityAchieve quality objectives by understanding and controlling sources of risk


Customer Speak!

Speed time to marketAutomation reduces human error and improve operational efficiency

Leverage investments in existing tooling and infrastructure

Consolidate and reduce costs with support for heterogeneous environments and business partner offerings

Optimize lab efficiency and asset utilizationSave on test lab overhead, infrastructure and duration costs

Ramp up productive teams day-1 Productivity boosters automate repetitive, mundane tasks for rapid ROI

•Sources: *IBM, ** Standish Group

“Testers are spending 40% of their time building

test environments”*

“Testing can account for 30% to 50% of

maintenance efforts”*

“We should reuse requirements and test

artifacts and save time and money”


Unify the team through real-time, in-context collaborationA single, dynamic quality contract provides clear and accountable direction Unify the entire team with a shared

view of quality assets Central location for assets (e.g.,

business objectives requirements, resources, platform, exit criteria)

Comprehensive dynamic planning and updates Integrated process workflow, not

artifacts drives team activities, hand-offs, reviews/approvals and sign-offs

Know what others are doing and what others expect of you

Task management for individuals and team


Know you are testing the right things

Requirements driven testingKnowing what to test

Requirements tracking built into the test management tooling

Customizable attributes enable you to track what is important to your team

Real-time impact analysis of requirements changes

Traceability of test results to user needs

www.strongback.us

Comprehensive dynamic planning and updatesProcess flow, not artifacts drives team activities

Word based Test Plan

Other Test Plans

Rational Quality Manager Plan

21

Live dynamic documentation

Defines test process

and strategy

Defines responsibilities

Activity based versus hierarchy

Business level reporting against quality objectives

www.strongback.us

Base project decisions on qualitative risk analysis

22

Collaborative risk based testingRisk management and prioritization

Risk assessments captured in Test Plan and Test Cases

Collaboration planning of risk mitigation strategy

Test Case will contain a risk failure score and a risk priority score

Documented risk related decisions


Test Configurations

4 languages4 browser types5 databases5 application servers

400 Combinations!

Pairwise Optimizations

Test the right 20 combinations

Test Platform Environment Management

Configuration awareness

Focus your environment coverage

Document your environment coverage

Gain agreement across the projectTest the right cases instead of

everything. Plan optimal execution

Test coverage optimizationFocus resources on testing the right combinations

www.strongback.us

Source: GBS Test Practices study, 2005-2008, over 846 projects

Typical Scenario

Defects

13, 9, 7, 6, 4, 2

11, 7, 2

12, 9, 7, 4, 2

13, 11, 9, 8, 2

Ideal Scenario

Defects

13, 9, 7, 6, 4, 2

11

12, 9

8

Reduce project cost and time from duplicates rework

24

Duplicate defect detectionImprove visibility and tracking of defects

www.strongback.us

Tests based onrequirements ensure

deliverables meet customer expectations

Quality is conformance to requirements

Quality Manager

DOORS

RequirementsManagement

Test DesignTest

Execution

RequisitePro

RequirementsComposer

Test PlanningTest Status

Process Automation and Increased FocusThe test team is working against the right set of requirements

Integrated Requirements Management


DOORS

Rational Quality Manager

RequirementsManagement

Test Status Test Planning

Test Execution Test Design

Integrating DOORS with IBM Rational Quality Manager “Tests based on requirements

ensure deliverables meet customer expectations”

1. Plan Tests Early Plan tests for each requirement as the

requirement is written.

2. Conduct Tests Early Perform tests as early as possible in the

development process.

3. Relate Tests to Requirements Trace tests back to the requirements they

are design to check.

4. Relate Defects to Requirements Trace defects back to the requirements that

they show are not satisfied.

5. Measure Progress against Requirements

Set targets and measure the progress of testing in terms of those requirements that are shown to be satisfied or are not satisfied.

www.strongback.us

Maximizing efficiency of manual testing

27

Integrated Manual test authoring and execution

Manual test author and execute Step by step capture

and execution of manual tests

Assisted data entry Keyword support for

integrated manual and automated testing

Rich defect capture during execution, including screenshot and attachments

Simple intuitive interface for quick test execution


1. Increase repeatability through automated test playback

2. Test more critical functions faster with automation

3. Automatically deploy your test environment and schedule the execution of your test Suites

4. Track and communicate progress and regressions throughout the testing lifecycle

Integrated Functional and Regression test execution

Accelerate test execution and deepen test coverage through automated test

execution

www.strongback.us

1. Capture and track Business SLAs in your test plans

2. Create realistic user workloads that exercise the key business transactions

3. Deploy the environment and schedule execution of your workload

4. Identify the root cause of performance problems

5. Communicate results and areas of non-compliance to the stakeholders

Integrated Performance test execution

Comprehensive test planning, authoring, and analysis to identify and manage the risk of

application performance failures

29

Rational Performance Tester

Test Authoring and Control

Load Generators

System Under Test

Web Server

App Server dB Server

www.strongback.us

Work smarter, save on test lab overhead, infrastructure and duration

costs 30

40% Lab Setup40% Lab Setup 60% Test Time60% Test Time

20% Setup20% Setup 80% Test Time Free Cycles 80% Test Time Free Cycles

Non-automated tasks or steps

(50% of setup effort)

Application setup with RTLM (30-50% reduction)

System setup with TPM (15-20% reduction)

Ad-hoc or varying workflows and setup tasks

50% reduction in effort for test system deployment

20% gain

Before lab automation

With lab automation

Source: IBM

Test lab automation and managementInsight and control over the test lab

Manage Verify that I have the

resources required to fulfill my test plan

Deploy Deliver the configurations

my teams require for test

Optimize Analyze patterns to

minimize cost and maximize utilization


Customer Speak!

Make confident decisions Manage to business objectives. Anywhere. In real-time with automated data collection and analysis for custom quality reports and dashboards

Confidently deliver incremental quality improvementsManage, measure and improve quality software delivery capability with a proven, repeatable approach

Achieve project quality objectives each and every timeComplete traceability across quality assets

Make informed decisions and proactive changewith real-time analysis and actionable reportingMeasure and manage quality, project and team status performance and results

“We can do better, but don’t know what's

not working, how bad it is, or where to start.”

“2/3 of executives make more than half of their

decisions based on ‘gut feel’ rather than

verifiable information”*

“77% of managers are aware of bad decisions

made due to lack of access to accurate

information”*

*Source: Business Week

www.strongback.us

Drive continuous and measured improvement

32

Assess and measure against Organizational policies

Assessing status Standard Objectives Reuse across Test

Plans All working toward

same objectives Measures against

business objectives

www.strongback.us

Make the right decisions at the right time

33

Make confident decisions with effortless reportingClosed Loop Analysis & Reporting

Customizable reports and dashboards Reduce escalating cost of information

gathering

Reduce risk by identifying trends before they become issues

Raise enterprise visibility and transparency to reduce costs and risk

Measures the effectiveness of processes and practices to improve organizational and business outcomes


A customized report

Cognos reportin the dashboard

Make informed decisions and proactively change with real-time analysis and actionable reportingMeasure and manage quality, project and team status performance and results Measure quality process and project

outcomes Real-time intelligence based on IT

industry best-practice metrics, dashboards and models

Inform quality decisions and drill into issues Alerts and automated analysis focuses

owner to take action on root causes

52 out-of-the-box, customizable Cognos test management reports

Take real-time action on relevant quality and project data Proven business intelligence backbone

automates collection and analysis to improve lifecycle productivity

www.strongback.us

Rational Quality Manager Open Ecosystem Today

Quality Manager

Managing mobile applications testing across a global handset test environment

Supporting bidirectional integration with the Jira change management system

Enabling greater quality and productivity with automated SOA governance support

Accelerating test cycles with virtual machine management and execution

Requirements Rational Requirements Composer Rational ReqPro Rational DOORS

Builds, WorkItems and Defects Rational ClearQuest Rational Team Concert Rational BuildForge

Quick Test ProfessionalLoadRunner

Provisioning Tivoli Provisioning Manager (TPM) Tivoli Service Request Manager TADDM

Reporting Rational Insight

RapidRep

Certify

TMAP Process

35

Automated Testing Rational Functional Tester Rational Performance Tester Rational Service Tester for SOA

Quality Rational AppScan Tester Edition Test RealTime Rational Robot Rational Rhapsody


Agenda

Challenges



Customer results


Customer Industry: Computer Services

SolutionClient SituationDefined and implemented a common set of IBM Rational tools to support end-to-end integration, including requirements gathering, configuration and change management, and defect management. Defined a common workflow to streamline processes. Established goals to drive tool adoption and metrics to measure productivity. Three key pilots drove process improvements and efficiencies:Automation of regression testingIncreased team productivity and collaborationWeb scanning and remediation

Benefits

Realized over US$3 million in productivity savings

90% of developers, 80% of architects and project managers, 50% of testers using the Rational tools

31 migrations completed from third party tools to Rational, saving on license costs

Displaced shareware and homegrown tools created common processes across projects, reusable assets

Organization faced three main challenges: speed, quality and increased time to value needing to:Deliver software projects to operations on time and within budget with greater reliability.Ensure business critical applications met end-user requirements and performance criteria, while meeting regulatory standards.Leverage a common development environment, reduce duplicate tools and processes, increase efficiencies and realize benefits earlier.

Rational Software

Rational AppScan Enterprise Edition, Rational Asset Manager, Rational ClearCase, Rational ClearQuest, Rational ClearQuest and Functional Testing, Rational Developer for System z, Rational Functional Tester, Rational Performance Tester, Rational Portfolio Manager, Rational Quality Manager Standard, Rational RequisitePro, Rational Software Architect, Rational Team Concert Standard

Over US$3mproductivity

savings


Customer Industry: Telcommunications

SolutionClient Situation

Organization’s project portfolio was consistently growing in size which caused internal pressures to deliver its projects faster while decreasing with necessary internal resources. However, due to the complexity of many of the projects, its siloed IT environment constricted such improvements. Moving forward, the organization needed to adopt new software technologies to create a fully integrated project delivery system that could deliver projects faster, on or under budget, and with fewer internal staff resources.

Software

Rational ClearCase, Rational ClearQuest, Rational Quality Manager, Rational RequisitePro

Adopting a comprehensive suite of IBM Rational software solutions provided the company with complete management traceability for software lifecycles and integrated the development environment to allow for simplified resource sharing between projects and teams. Software disciplines included configuration management, change management enterprise test management, and requirements management software.

Benefits

Able to complete development projects faster and within, or under, budget. In 2003, company completed only 36 projects. In 2006, it completed 67 projects with cost savings of US$3.7 million.

Reduced average project cycle time by 76 days while increasing overall software quality.

Improved relationship with its stakeholders by: delivering project-based benefits faster with improved use of its IT budget and better focus on deliver high business value.

Reduced average project cycle

time by 76 days


Customer Industry: Healthcare

Solution

Rational DOORS + Rational Quality Manager

The company deployed DOORS in 2005 and has been using the tool to manage their project requirements effectively. Using consultative selling methods, the sales team identified their need to generate dynamic reports for FDA compliance which showed requirements and test information on the same page and avoid manual processes that used to take days to produce.

Rational DOORS Web Access To increase visibility of requirements across the organization and avoid installation and training costs, the company also decided to use DOORS Web Access. Allowing other teams (testers, developers and business stakeholders) to review requirements and provide feedback during the requirements management process.

Client Situation

A global leader in the development, manufacturing and marketing of innovative medical products sells a comprehensive range of products in more than 60 countries worldwide through direct offices and a network of distributors.

Challenges:Reduce time spent in creating reports to demonstrate compliance with FDA

Establish an audit capability that has credibility and increases consistency

Improve collaboration between requirements analysts, developers and test teams across geographical boundaries

Be able to view end to end traceability on a single window

12 month ROI

www.strongback.us

Previously in this series…..

Collaborate, Integrate, Automate -- Revolutionize Your Distributed Development Environment

When: Wednesday, June 15, 2011 at 2:00 p.m. EDT http://blog.strongbackconsulting.com/2011/06/collaborative-lifecycle-management.html

A Single Source of Truth -- Aligning Business Priorities and IT Project DeliverablesWhen: Wednesday, June 22, 2011 at 2:00 p.m. EDThttp://blog.strongbackconsulting.com/2011/06/requirements-management-in-clm-world.html

www.strongback.us

What's Next

Review the whole of CLM on our blog site and previous webcasts

Review & Download RQM or the full CLM at Jazz.net

https://jazz.net/projects/rational-quality-manager/

Have us do a deep dive with you and your team Contact us for more details

Kenny: [email protected] 386-232-8746

42www.strongback.us

Learn more at:IBM Rational softwareRational launch announcementsRational Software Delivery PlatformAccelerate change & deliveryDeliver enduring qualityEnable enterprise modernization

Rational trial downloadsdeveloperWorks RationalLeading InnovationIBM Rational TVIBM Business PartnersIBM Rational Case Studies

Ensure Web security & complianceImprove project successManage architectureManage evolving requirementsSmall & midsized businessTargeted solutions