Software Process Simulation Modeling - KAISTse.kaist.ac.kr › ... › Software-Process-Simulation-Modeling.pdf · 2013-05-01 · Software Process Simulation Modeling HyeonJeong Kim

Software Process Software Process Simulation Modelingg

HyeonJeong Kim2009. 2. 252009. 2. 25

ⓒ KAIST SE LAB 2009

Content

Part I: Definition of Software Process Simulation M d li (SPSM)Modeling (SPSM)Part II: Overall procedure to build SPSMPart III: Research issues

2/27ⓒ KAIST SE LAB 2009

Definition of SPSM(1/3)( / )

Software ProcessA set of procedures and methods

• Define the relationship of tasks[1]

Al f l h l i d[2]Also a part of process, people, technology triad[2]

Process

Task, activities

P l TechnologyPeople Technology

People with skills, training, and motivation

Tool and equipment



Simulation modelingA method to specify real system at some abstraction level

<Real world> <Simulation model>Mapping

System uncertainty

Characteristics

Stochastic simulation

Corresponding concept

Change over time

Learning mechanism

Dynamic behavior

Feedback loop



Software Process Simulation model (SPSM)System specification describes how the process works

• SPSM should contain all three elements of triad

Process

<Project environments

T h l

<Project environments in SPSM>

People Technology


Overall procedures to build SPSMp

D fi th bj ti f i l tiDefine the objectives of simulation

Gather model elements

Requirements analysis

Deriving qualitative information Formal modelingDesign

Implement the simulatorImplementation

Verification

ValidationTesting(V&V) Calibration

Collect the data from simulatorAnalysis


Analyze the collected datay

Define the objectives(1/2)Requirements analysis

j ( / )

Determine why the simulator is neededMain categories of the objectives in SPSM[3]

• Most simulators are made for supporting the manager’s decision makingdecision making

Strategic managementP ss imp m nt nd

Planning

Process improvement and technology adoption

Control and operational management


Define the objectives(2/2)Requirements analysis

j ( / )

Objectives Strategic Planning Control & SPI & tech. j gmanagement

goperational management

adoption

Area that Deciding on the Predicting/forec Project tracking Forecasting the objective deals with

organizational strategies or policies

asting some project parameters across time

and oversight on software project during enactment

impact of a potential process(tech) change before across time enactment change before enactment

Example • Decide development

• Predicting staffing levels

•Predicting which activity

•Analyzingimpacts of the development

site (cross sites vs. centralized at one site)

D id t ff

staffing levels across time and periodical re-planning

which activity threats product quality

impacts of the inspection on product quality•Analyzingi t f • Decide staff

strategies (in-house vs. outsource)

impacts of automation of test design on effortoutsourc ) ffort


Gather the model elements(1/2)Requirements analysis

( / )

Model elementsIdentified information elements

• To be needed for the defined objectives in simulation

Simulation model

Input Output

Model abstractionProcessp

parametersp

Variables

People Technology

Model scope


Gather the model elements(2/2)Requirements analysis

( / )

Categories Model scope Result variables Model Input g pabstraction

pparameters

Definition Boundary of the simulation model

Information elements needed

Key elements of software

Information elements

in organizational view and time-span

to answer the key questions related to the objectives

process and project environments

related to result variables and model abstractionsobjectives abstractions

Example •Organizationalview- less than one

•Effort/cost•Schedule•Quality(# of

•Key process components and their level

•Software size(KSLOC)•Defect less than one,

one, multiple projects(teams)

T

Quality(# of defects)•Productivity•ROI(Return on

)

their level•Sequence of the activities•Available

( ff

Defect detection efficiency•Defect removal

d • Time-span- short,

medium, long

Investment)•Staff utilization rate (staffing needs)

resources(staff, hardware) •Iteration loops

and injection rates•Requirements creeping pattern

10/27

(staffing needs) creeping pattern•Resource constraints


Designing the simulation modelDesign

g g

What to do in designDeriving the qualitative information

• Define a cause-effect relationship from input to result variables

F l d li f h li i i f iFormal modeling for the qualitative information

<Deriving qualitative info.> <Formal modeling>g q g

a

b

Informal description Formal description

“Roof size depends on size of dog house”

Roof size a = 1.2 x dog house size b


Deriving the qualitative information(1/2)

Design

Qualitative informationContain a cause-effect relationship between parameters

It d ib th t i b h i f l t ( )• It describes the atomic behavior of real system(process)

Causal diagram (cause-effect diagram)[4]

• Useful diagram to describe qualitative information• Useful diagram to describe qualitative information

Notation Meaning Example

If X increases (decreases), then Y increases(decreases)X Y

+Effort Schedule

+

If X increases (decreases), then Y d (i )X Y

-Productivity Effort

-


decreases (increases)X Y y

Cause-effect relationshipParameters

Deriving the qualitative information(2/2)

Design

Example of a causal diagram[4]

Time remaining

Schedule –Schedule pressure

Work to be doneOvertimeTime per task

++–

Feedback loop

Completion rateFatigue

Error +

+

+–p

Productivity

Error prevention rate

+

–

––

+

• Useful to identify the important dependencies for entire model• Feedback loopFeedback loop

– Represent learning mechanism from previous value13/27ⓒ KAIST SE LAB 2009

Formal modelingDesign

g

Simulation model needs a formal description of t b h isystem behavior

Mathematical representation is required • Based on observation of real projects or literatures

Qualitative information Formal description

Effort Schedule

+

-Productivity Effort


Implement a simulatorImplementation

p

SimulatorComputer program to interpret the simulation model’s instruction

Main simulation methods to build simulator for ftsoftware processSystem DynamicsDiscrete Event-based SimulationHybrid simulation


Method to build simulator(1/5)Implementation

( / )

System dynamics(SD)A method to enhance understanding dependencies between factors in complex system[4][5]

E lExample

• Project environments which change dynamically and continuously over time and their changes affect other variablescontinuously over time and their changes affect other variables

– In SD, all the parameters are automatically re-calculated on every single time unit



( / )

System dynamics (Cont’d)Result of simulator depends on single time unit

Schedule pressurepressure

Calculation Total time Accuracy<Simulator performance>

Calculation # in one run

Total time Simulation

Accuracy

8 times Long High4 times Short Low

Time

Short time unit

4 times Short LowLong time unit

Time-unit



( / )

Discrete Event-based Simulation(DES)A method to enhance understanding interactions between components of complex system[6]

St t f ft h d b• States of software process are changed by– Start/end of an activity or reception/release of an artifact

» Irrelevant to time » Events represent interactions between process activities

Analysis Design…

Events



( / )

SD vs. DES for software process

SD(System Dynamics) DES(Discrete Event-based Simulation)

Advantages-Dependency bet. parameters-Feedback loops

-Easily describe discrete process steps -Attributes of each

titFeedback loops process entity

Disadvantages-Difficult to model discrete process steps, and -Difficult to make Disadvantages p p ,attributes of each process entity

feedback loops

Project environments Process based on

Provide a hybrid simulation method to combine SD

Project environments based on feedback loop

Process based on events


Provide a hybrid simulation method to combine SD and DES


( / )

Hybrid simulation methodCombine DES and SD

• DES is responsible for describing software processSD i ibl f d ibi f db k l• SD is responsible for describing feedback loop

Example• Choi’s work[7]• Choi s work[7]

– Combine SD into DES framework


Verification and validationTesting

Real project SimulatorSimulation

model

Validation Verification

project model

Validation Verification

SimulatorSimulation

model spec CompareSimilar

result?

Calibration

Implementation ResultCalibrationInputs

Similar

Real project

Simulationmodel

Simulator Produce

Similar result?

Calibration


Result analysis(1/2)Analysis

y ( / )

Sensitivity analysisPurpose

• Determining which input parameters have a significant impact on the result[8]on the result[8]

– Identified main parameters should be carefully treated in management

Method• For each input parameter• For each input parameter

– Its minimum and maximum value are simulated under setting other parameters to have a fixed value


Result analysis(2/2)Analysis

y ( / )

Sensitivity analysis(Cont’d)Example

• Effect of input parameters of GSD* on duration to complete software project[9]software project[9]

Dur

atio

n

T 3 i ifi l d i f GSD% overlap Distance Culture Language Distribution Familiarity Meeting

D

– Top 3 significant elements on duration for GSD» Overlapping working hour» Distribution overhead» Distance


*GSD: Global Software Development

Research issues(1/2)( / )

Define the objectives of simulation Developing new simulator and Define the objectives of simulation

Gather model requirements and analyze them

panalyzing the results

•For developing a general

Build qualitative model

Collect quantitative information

F p g gsimulation model- ‘plug and play’ model components

Implement the simulator

p•For new developmentprocess

- GSD, business process i h SW Verification

ValidationCalibration

with SW process, …•For deciding managerialpolicies

Req Creeping

Collect the data from simulator

- Req. Creeping•Re-planning- Resource allocation


Analyze the collected data

Research issues(2/2)( / )

Define the objectives of simulationDefine the objectives of simulation

Gather model requirements and analyze them

Build qualitative model

Collect quantitative information

Implement the simulatorV&V for SPSM with limited data

h d Verification

ValidationCalibration

•With minimum data sets, provide the useful, proven SPSM

Collect the data from simulator Focus on analyzing the results•Combine with statistics in


Analyze the collected data stochastic simulation model

References[1] M.Paulk, et al., “Key practices of the capability maturity model”, Version 1.1,

Technical Report CMU/SEI-93-TR-25. SEI, CMU, Pittsburgh, PAp , , g ,[2] M. Chrissis, et al., “CMMI: Guildlines for Process Integration and Product

Improvement”, Addison-Wesley, 2003[3] M Kellner R Madachy and D Raffo “Software process simulation modeling:[3] M. Kellner, R. Madachy, and D. Raffo, Software process simulation modeling:

Why? What? How?”, JSS, Vol. 46, No. 2/3, 1999[4] J. Sterman, “Business Dynamics”, McGrawHill, 2004[5] P j t D i A I t t d A h” P ti H ll 1991T Abd l[5] Project Dynamics: An Integrated Approach”, Prentice Hall, 1991T. Abdel-

Hamid and S. Madnick, “Software [6] B. Zeigler, H. Pracehofer, and T. Kim, “Theory of Modeling and Simulation”,

A d i P 2000Academic Press, 2000[7] K. Choi, D. Bae, and T. Kim, “An Approach to a Hybrid Software Process

Simulation Formalism”, SPIP, 2006[8] A. Law, “Simulation Modeling & Analysis”, McGrawHill, 2007[9] S. Setamanit, W. Wakeland,and D. Raffo, “Using simulation to evaluate global

software development task allocation strategies”, SPIP,2007p g



Documents

Software Process Simulation Modeling - KAISTse.kaist.ac.kr › ... › Software-Process-Simulation-Modeling.pdf · 2013-05-01 · Software Process Simulation Modeling HyeonJeong Kim