Michela Piergentili, INFN Genova

F. P. Brooks,

“No Silver Bullet - Essence and Accidents of Software Engineering”,IEEE Computer 20(4):10-19, April, 1987

As we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or in management technique, that by itself promises even one order-of-magnitude improvement in productivity, in reliability, in simplicity. ...Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any - no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware. We cannot expect ever to see twofold gains every two years. ...Although we see no startling breakthroughs - and indeed, I believe such to be inconsistent with the nature of software - many encouraging innovations are under way. A disciplined, consistent effortdisciplined, consistent effort to develop, propagate, and exploit these innovations should indeed yield an order-of-magnitude improvement. There is no royal road, but there is a road.

Michela Piergentili, INFN Genova

Experience with Software Process in Physics Experiments

Experience with Software Process in Physics Experiments

S. Guatelli, B. Mascialino, L. Moneta,I. Papadopoulos, A. Pfeiffer, M.G. Pia,

M. Piergentili

CERN - INFN Genova

Monte Carlo 2005 Topical MeetingChattanooga, April 2005

Michela Piergentili, INFN Genova

Challenges

Complexity of physics and experiments

Geographically distributed teams and computing resources

Mission critical applications

Quality and reliability for sensitive applications

Michela Piergentili, INFN Genova

Trends in software engineering

Develop a disciplinediscipline for software engineering

Shift attention from computing technology to

software process as the way to achieve quality, lower costs

Define quantitative standards to evaluate software capability maturity:

SEI’s Software Capability Maturity Model ISO 15504

The culture of software process is not widely spread in the physics research environment yet Software is still perceived as “writing code”

The success of software projects is mainly left to individual efforts (CMM: “heroic”, level 1)

Widely used in software industry

Michela Piergentili, INFN Genova

Process framework

An incrementalincremental and iterative iterative software life-cycle

Based on the Unified Process (Booch, Jacobson and Rumbaugh)Rational Unified Process: UP + practical guidance and tools

Customized Customized to the specific development environment

Equivalent to CMM / ISO 15504 Level 3 at least

Michela Piergentili, INFN Genova

Projects adopting the RUPPilot project: Anaphe (Data Analysis Tools project, CERN/IT Division)

– a playground to learn the RUP and to adapt it to the physics research environment

RUP in projectsRUP in projectsGeant4 Low Energy Electromagnetic PhysicsGeant4 Electromagnetic Physics Validation ProjectStatistical ToolkitBrachytherapy Simulation & DosimetryIMRT Geant4-based Dosimetry SystemREMSIM Simulation (radioprotection, Aurora Programme)Bepi Colombo Simulation (planetary astrophysics)Solar System Radiation Environment Generator

RUP in an organizationRUP in an organization (encompassing several projects)Geant4 Advanced Examples (in progress)

Michela Piergentili, INFN Genova

Retain all the best practices suggested by the RUP– Iterative development– Management of the requirements– Component-based architecture– Visual modeling with UML– Continuously verify quality– Change management

Identify essential activities, artifacts and key roles– for every RUP activity: is it justified in our environment? What are the

benefits? How much does it cost?

Justify all artifacts to be produced in relation to the project objectives

Guidelines for process tailoringGuidelines for process tailoring

Michela Piergentili, INFN Genova

Essential artifacts

VisionRisk List

Development PlanDevelopment Case

CVS repository

User RequirementsArchitecture Model

Design ModelTest PlanTest SuitePrototype

ToolsGuidelines

The systemDocumentation

+ refinement of artifacts from previous phase

The productRelease Notes

Training Material+ refinement of artifacts from previous phases

+ refinement of artifacts from previous phases

Other optional artifacts may be produced in some projects

…much more than just the code!…much more than just the code!

A minimal subset identified within the large set of RUP artifacts

Michela Piergentili, INFN Genova

Metrics Quantitative process controlObjective evaluations are the key to software process improvement

Requirements

Number or user requirements / use cases

Requirement status

Traceability

Analysis & Design

Number of subsystems, packages, classes

Number of methods in a class

Inheritance tree depth

Implementation Code size

Method size

Fraction of design implemented

Deployment New features implemented per release

Released defect levels

Documentation/examples coverage

Schedule Estimated vs. actual task duration

Estimated vs. actual duration between milestones

Schedule estimating accuracy

Process Process capability level

Process deviations

Number of metrics collected

Test Number of unit/system tests

Fraction of code covered by unit testing

# of defects found by unit testing

# of defects found by integration testing

# of defects found by system testing

Fraction of deliverables reviewed

Number of defects found by reviews

Defect status

Defect recurrence

Defect source count

Project management

Number of tasks planned and completed

Number of reviews, walkthroughs

Staff size and profile

Staff turnover

Support Number of support requests

Support request aging

Average resolution time

Re-opened support requests

Michela Piergentili, INFN Genova

Some metricsSome metrics

95% projects delivered on time, on budget and with all the features originally planned

– CHAOS Report 2000 (Standish Group):

• 28% software projects failed

• 49% challenged

• 23% successful

5% failure due to neglect of applying the process– no way to enforce the application of the process onto independent researchers

Michela Piergentili, INFN Genova

A case studyA case studyDevelopment of simulation + analysis of a dosimetric system for IMRT

– Software requirements: functionality + high quality for medical application

– Master thesis work, duration: 1 year– Manpower: 1 developer + 1 process specialist – Developer: no prior software knowledge

D = 0.005; p-value = 1

Results– on-time delivery of code, documentation, physics results– all high and medium priority requirements satisfied– ~30 billion simulation events produced and analysed– high quality physics results from the software (paper in

IEEE-NSS 2004)– 355 page MSc. thesis– fraction of code discarded during the development period: 0

Michela Piergentili, INFN Genova

ConclusionConclusion

A rigorous software process is the key technology enabling development teams to achieve success

The RUP can be effectively adapted to the peculiar physics research environment

– a powerful, but flexible process framework

A RUP-based software process has been adopted by several physics projects in may research fields (fundamental physics, space science, astrophysics, medical physics, statistics)

Success rate is 95% (to be compared to CHAOS 23%)