Technical briefing on Software Release Planning

.5

Xavier FranchGroup of Software and Service Engineering

Universitat Politècnica de CatalunyaBarcelona, Spain

[email protected]

Guenther RuheSoftware Engineering Decision Support Laboratory

University of CalgaryCalgary, Alberta, Canada

[email protected]

Contents

• INTRODUCTION OF PARTICIPANTS• PART I. BACKGROUND• PART II. STATE OF THE ART • PART III. THE EVOLVE APPROACH • PART IV. CONCLUSIONS

ICSE 2016, Austin, TX 2

Contents



Attendees


Contents



Context – Software Evolution


Continuing Change — an [E-type] system must be continually adapted or it becomes progressively less satisfactory (Law 1)

Continuing Growth — the functional content of an [E-type] system must be continually increased to maintain user satisfaction over its lifetime (Law 6)

Laws of Software EvolutionManny Lehman (1925 – 2010)

what/when/how to evolve?ReleasePlanning

The planning onion

ICSE 2016, Austin, TX 9M. Cohn, Agile Estimating and Planning. Prentice Hall PTR, 2006.

Software release planning – definition


Software release planning – “critical process of deciding which features are implemented in which releases”

G. Ruhe. Product Release Planning, CRC Press 2010

Release planning – Strategic + operationalStrategic release planning – “selection and assignment of

requirements in sequences of releases such that important technical and resource constraints are fulfilled”

Operational release planning – “development of the identified features in a single software release”

Svahnberg et al. A Systematic Review on Strategic ReleasePlanning Models. IST 52(3), 2010

Example: SENERCON


• Partner of the SUPERSEDE H2020 project• Service provider for energy savings based in Berlin with

more than 75.000 users• After developing more than 20 services, still success is

unpredictable, concluding that:– the success of a service mostly depends on fulfilling personal

and individual needs of the end-user– mismatch QoS – QoE The Black Box Problem

Example: SENERCON


• Main reasons:– lack of detailed knowledge about QoE

• currently, email + hotline only– not having a systematic release planning approach in place

• currently, based on expert judgement

• Goal: a cost-effective exchange hub users developers– contextualized user feedback– discovery of service usage patterns– combine feedback with context

• Once this information is known:– features’ value easier to quantify– systematic release planning may be put in place

What is a feature


“A logical unit of behavior specified by a set of functional and non-functional requirements”

J. Bosch. Design and Use of a Software Architecture. ACM Press 2000

“A distinguishable characteristic of a concept (system, compo-nent, etc. ) that is relevant to some stakeholder of the concept”

K. Czarnecki, U.W. Eisenecker. Generative Programming: Methods, Tools and Applications. Addison-Wesley 2013

“A set of logically related requirements that provide a capability to the user and enable the satisfaction of business objectives”

K. Wiegers, J. Beatty. Software Requirements (3rd ed.), Microsoft Press 2013

Typical features


• Core functionality of the domain– prime prerequisite for a company’s business

• Demanded by the market

• Requested by a specific customer

T. Berger et al. What is a Feature? A Qualitative Study of Features in Industrial Software Product Lines. SPLC 2015

Good and bad features


Reasons for considering a feature as “good”:• Customer satisfaction• Distinct functionality• Well implemented and error free

What makes a feature “bad”:• Result of time pressure and rushed development• Compromises emerging during implementation• Duplicated and superfluous features• High volatility

T. Berger et al. What is a Feature? A Qualitative Study of Features in Industrial Software Product Lines. SPLC 2015

Software release planning – A deeper view


Amou

nt o

f im

plem

ente

d fu

nctio

nalit

y

Amou

nt o

f sug

gest

ed fu

nctio

nally

Releases

• Corrective• Adaptive• Perfective• PreventiveISO/IEC/IEEE 14764:2006

Software release planning – why?


Release decisions

ICSE 2016, Austin, TX

Software release planning - Why difficult?


Information is Uncertain Inconsistent Incomplete Fuzzy

Decision space Large size High complexity Dynamically changing

Multiple objectives Usability Value Time-to-market Frequency of use Risk

Hard & soft constraints on Time Effort Quality Resources

Main challenges in release planning

• Product management underestimated/not sufficiently established• Product release planning process immature• Product release planning not synchronized with other processes• Lack of systematic re-planning• Lack of transparency of release decisions• Lack of definition of planning goals/alignment with business goals• Lack of stakeholder involvement • Lack of resource consideration• More re-active than pro-active planning mode• Impact of better release content unclear• Impact (value) of individual features unclear• Planning for just the next release


Contents



Approaches to Software Release Planning

Two main categories• manual (“on-the-fly”) approaches

– rely on humans’ ability to negotiate between conflicting objectives and constraints

– mainly reported as experience reports

• analytical approaches– formalize the problem– apply computational algorithms to

generate best solutions– mainly reported as scientific technical

papers


G. Ruhe, M.O. Saliu. The Art and Science of Software Release Planning. IEEE Software 22(6), 2005

On-the-fly approaches


• emphasis on improving the decision process– make estimates as accurate as possible– provide stakeholders a voice

• emphasis is in the next release– planning long term is more difficult

Example: a case in Ericsson

V.T. Heikkilä et al. Continuous Release Planning in a Large-Scale Scrum Development Organization at Ericsson. XP 2013ICSE 2016, Austin, TX 24

• Ericsson node development unit– traffic management in telecommunication networks– large systems– combining hardware and software

• Large projects– 20 development teams fro Finland and Hungary– every team had 6-7 members– following Scrum

• The products– yearly public releases– 2 internal versions per release, 2 internal deadlines for

maintenance updates

Team structure


The release planning process


Reported benefits• Increased flexibility

– feature development schedule not tied to release schedule– decreased development lead time

• Eliminate waste in the planning process– early identification of too expensive or unfeasible features

save development resources

• Increased developer motivation– early involvement of developers in the feature planning

process


Remaining challenges• Misalignment with “the old way” of planning

– product manager still asking for long-term feature development plans

• increasing detail of FCS

• Managing non-feature specific work– non-feature specific problem reports, system documentation,

external change requests, …

• Low prioritization of system improvement work wrtimplementing new features– some store points saved for system improvements


Limitations of on-the-fly approaches

• Informal process• Informal decisions• But: > 1.000.000.000.000 possibilities already in case of

20 objects and three periods


Analytical approaches

F = {f(1), ..., f(N)}Set of features Set of constraints

X = {x(1), ..., x(N)} Release plan

x(j) = assigned release

C = {c(1), ..., c(M)}

RPmaximise some utilityor objective function


Analytical approaches

F = {f(1), ..., f(N)}Set of features Set of constraints

X = {x(1), ..., x(N)} Release plan

x(j) = assigned release

C = {c(1), ..., c(M)}

RPmaximise some utilityor objective function

What informationis processed?

Which results are produced?

How is the plan computed?


Example – release planning in agile projectsApproach Iterations Precedences Risk Change mgmt. Planning

[1] Multi Preced, coupling Some Some Heuristic

[2] Multi Preced Yes No Greedy

[3] Single Preced, coupling No Yes Exact

[4] Single Preced No Some Exact

[5] Multi Preced, coupling Yes No Exact

[6] Multi Preced, coupling Yes No Exact

[7] Multi Preced, anchor, coupling Yes Yes Exact

[1] D. Greer, G. Ruhe. Software release planning: an evolutionary and iterative approach. IST 46, 2004[2] M. Denne, J. Cleland-Huang. Software by Numbers. Prentice Hall, 2004[3] M.O. Saliu, G. Ruhe. Supporting software release planning decisions for evolving systems. SEW 2005[4] C. Li et al. An integrated approach for requirement selection and scheduling in software release

planning. REJ 15, 2010[5] A. Szoke. Conceptual scheduling model and optimized release scheduling for agile environments. IST

53, 2011[6] G. van Valkenhoef et al. Quantitative release planning in extreme programming. IST 53, 2011[7] M. Golfarelli, S. Rizzi, E. Turrichia. Multi-sprint planning and smooth replanning: An optimization

model. JSS 86, 2013

State of the art

Main source: systematic literature review until 2008

• 24 release planning models found– 14 original and 10 extensions, mostly from 1998

• Three main groups– EVOLVE-family + ReleasePlanner tool @ University of Calgary– SERG @ Lund University– Center of Organization and Information @ Utrecht University


Svahnberg et al. A Systematic Review on Strategic ReleasePlanning Models. IST 52(3), 2010

State of the art

Extension: ongoing non-systematic literature review until 2015 by the GESSI research group at UPC• snowballing based approach

– forward snowballing from Svahnberg et al.’s SLR C1– backward snowballing from C1– focus on selected journals and conferences– contributions from industry also sought

• Final selection: 16 new methods found in the period 2009-2015


New methods in a nutshell (sample)


NRP for eXtreme Programming dealing with some uncertaintyMultsprint planning in an agile contextCombining a planning algorithm with a scheduling methodEfficient algorithm for NRP in the projects with large sets of requirementsNRP in large scale agile organizationsCalculate the impact of uncertainty with time constraints in release planningDefine new releases in agile environments taking into accountprevious iterationsEfficient NRP algorithm based on model checking consideringdependenciesImplement a risk-aware NRP algorithm

Input: constraints and factors


SoftfactorsRisk factors Value factors

Resource consumption factors

Stakeholders’ influence factors

Svahnberg et al. A Systematic Reviewon Strategic Release

Planning Models. IST, 52(3), 2010

Requirement dependencies

Quality constraints

Budget and cost constraints

Resource constraints

Effort constraints

Time constraints

Hardconstraints

Input: constrains and factors (prevalence in 2010)


Requirement dependencies (75%)

Quality constraints(8.3%)

Budget and cost constraints(29.1%)

Resource constraints(33.3%)

Effort constraints(50%)

Time constraints(16.7%)

SoftfactorsRisk factors

(12.5%)Value factors

(37.5%)

Resource consumption factors (20.8%)

Stakeholders’ influence factors (29.2%)

28 methods

Hardconstraints

Input: constrains and factors (prevalence in 2015)


Requirement dependencies (75%)(75%)

Quality constraints(8.3%) (5%)

Budget and cost constraints(29.1%) (17.5%)

Resource constraints(33.3%) (37.5%)

Effort constraints(50%) (50%)

Time constraints(16.7%) (25%)

SoftfactorsRisk factors

(12.5%) (17.5%)Value factors(37.5%) (42.5%)

Resource consumption factors (20.8%) (17.5%)

Stakeholders’ influence factors (29.2%)(22.5%)

40 methods

Hardconstraints

Output

• Scope– one release vs. multiple releases

• Object of planning– features; user stories; requirements

• Prioritization– none– ordinal– must-have, should-have, could-have

• Time scheduling• Developer assignment


Objective function

• Optimization of the value given by the features while managing the resources and fulfilling all possible constraints

• How to measure value:– business value, stakeholder satisfaction, urgency, risk

minimization, technical debt, return on investment, …

• How to measure resources– personnel, availability– considering size/complexity of features

• Constraints– dependencies, time constraints, …


A lot of computational approaches…


Greedyalgorithms

Pareto opti-mal fronts

Monte-Carlo simulation

Knapsackproblem

Branch and bound Backbone

basedalgorithms

Graph trans-formation

AHPClustering

Example – greedy solution (1)

• Principle: always add a feature to the solution that maximizes value while not violating any constraint or requiring more resources than available

• Greedy algorithms: building a good global solution as the sequence of local optimum choices at every moment



• Input:– Set of features, F = {f(1), …, f(N)}– Resource consumption, cost: F Integer– Estimated values, value: F Integer– Set of cost capacity for each release:

totalCost: Integer Integer

• Output:– Release planning, Release: Integer {Integer}, s.t. all sets

are pair-wise disjoints


G. Ruhe. Product Release Planning, CRC Press 2010


ICSE 2016, Austin, TX 44G. Ruhe. Product Release Planning, CRC Press 2010

Example – Multi-sprint planning in Scrum (1)

• Given a set S of m sprints and a set U of n user stories, maximise a solution z for the m sprints

• Goals: – customer satisfaction– coupling management– criticality risk management

• Strategy– generalized assignment model

M. Golfarelli, S. Rizzi, E. Turrichia. Multi-sprint planning and smooth replanning: An optimization model. JSS 86, 2013


Example – Multi-spring planning in Scrum (2)

Example of input:


ID Name Deps. and coupling

Utility Comple-xity

Criticalityrisk

Uncert. risk

s1 Fee configuration s1->s2 80 5 Low Low

s2 Cash cost computation 0.3 s2+s10 85 2 Medium Medium

s3 Import from DBMS 75 2 Medium Medium

s4 Parameterization logic 30 1 Medium Medium

s5 Amortization mask 60 2 No No

s6 Exchange computation 60 2 Low Medium

s7 Exchange import from SAP s7->s6 60 7 Low Low

s8 Mngmt . control reporting 85 4 Medium Low

s9 Operational reporting 100 10 Low Medium

s10 Scenario management mask 0.3 s2+s10 65 3 Low Low

Example – Multi-spring planning in Scrum (3)

• Objective function z:

– uj: utility of story j– rj

cr: criticality risk of story j– xij = 1 if story j is included in sprint i– G: set of coupling groups of stories; Al: a set of coupled

stories– al: affinity between stories in Al

– yijl: number of stories of Al affine to story j and included insprint i


Example – Multi-spring planning in Scrum (4)the sum of stories’ complexity(considering uncertainty risks) fitsinto each sprint capacity

each story is planned in exactly onesprint

each forced story is planned in theplanned sprint

correct consideration of OR- and AND-dependencies among features

restricting values of affine stories


Summary: Analytical vs. on-the-fly planning


Caracteristics Analytical methods On-the-flyTime horizon Next release, but applicable more

generalNext release

Objectives Flexible, but typically value-based Vague and not explicitly described

Stakeholder involvement Not directly supported Opportunistic and by communication

Solution method Greedy heuristic, linear programming, simulation, ..

Intuition and experience-based

Quality of solutions Good on average, but unknown for specific case

Difficult to judge. The more risky, the more complex the problem

Feature dependencies Typically not considered Implicitly, hard to consider for more complex problems

Human resource constraints

If at all, then just cumulative effort Implicitly, hard to consider for more complex problems

What-if analysis(explicit support)

No No

Integrated tool support Limited No

Summary


• On-the-fly approaches criticised due to the difficulty of taking into account all knowledge implied by software release planning

• Conversely, analytical approaches criticised either because:– Too simple to be useful

• Lack of information considered• Over-simplifications (e.g. requirement dependencies)

– Too complex to be adopted• Learning curve• Lack of trust in result

Contents



Release planning – Art or Science?


• Art:Focus on the human intuition and communication for handling tacidknowledge

• Science:Emphasis on formalization of the problem and application of computational algorithms to generate best solutions.

What’s the problem?


“The mere formulation of a problem is far more essential than its solution, which may be merely a matter of mathematical or experimental skills. To raise new questions (and), new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advances in science.”

(Albert Einstein, 1879-1955)

Optimized release planning – How it began


EVOLVE: Greer, D. and Ruhe, G., Software Release Planning: An Evolutionary and Iterative Approach, Information and Software Technology, Vol. 46 (2004), pp. 243-253.

What constitutes a release plan?

Max{ F(x, α) = (α - 1) F1(x) + α F2(x) subject to 0 ≤ α ≤ 1, x from X}

StakeholdersWeightings for stakeholdersScores of stakeholders towards urgency (F1) and value (F2)X composed of- effort constraints- coupling and precedence constraints (between features)

Optimized release planning – How it began


F1(x) is a penalty function defined for plan x describing the degree of violation of the monotonicy property between all pairs of features

F2(x) is a benefit function based on feature scores of the stakeholders and the actual assignment of the feature according to the plan under consideration.

value(n,p) = value_score(n,p)(K – x(n) +1)

Empirical analysis

• EVOLVE was initially based on genetic search offered by Palisade’s RiskOptimizer

• Early industrial feedback (Corel, Siemens)• Development of our own GA (emphasis on avoiding

premature convergence)• Empirical studies with 200 to 700 requirements comparing

the GA with running ILOG’s CPLEX• Better solutions for LP solver in reasonable time • Known level of optimality• Development of our own solution method utilizing open

source optimization combined with knapsack-type of heuristic for B&B

• New approach more flexible and with higher level of diversification among top solutions.


EVOLVE II: Three phases

• Phase 1 - Modeling: – Formal description of the

(changing) real world to make it suitable for computational intelligence based solution techniques

• Phase 2 - Exploration: – Application of computational

techniques to explore the solution space, to generate and evaluate solution alternatives

• Phase 3 - Consolidation: – Human decision maker evaluates

current solution alternatives– Match with implicit objectives

and constraints


Computational Intelligence

Interation 1 Release 1

Release 2Interation 2

Interation 3 Release 3

Human Intelligence

Evolution everywhere

• Evolutionary software development (iterative, incremental)

• Evolutionary solution algorithms • Evolutionary problem solving (synergy between art and

science)


The diversification principle


A single solution to a cognitive complex problem is less likely to reflect the actual problem when compared to a

portfolio of qualified solutions being structurally diversified

Consolidation


Preparation

1

Planning criteriaweights

2Pre-selection of

features

3

Prioritization of features

4

Voice-of-thestakeholder analysis

5

Technology constraints

7

Resource estimation6

Optimization8

Quality and resource analysis

9Excitement analysis

10

Stakeholderevaluation of plans

12What-if-analysis

11

Final plan decision13

dependencybetween stepsmandatory step

optional step

set of logicallylinked steps

feedback link

Stakeholder-centric release planning –Method EVOLVE II

Criteria for feature selection

• Customer satisfaction• Customer dissatisfaction• Risk of implementation• Risk of acceptance• Financial value • Cost• Time to market • Volatility • Frequency of use • Ease of use


Feature dependencies

• For given features A, B, and C, we distinguish eight types of dependencies:


Pre-assignment of features to releases


Maximization of stakeholder feature points


Stakeholder

weight

Score(n,q)

Criteria

weight

SCORE(n)

Releases

weight

sfp(n,x)

TSFP(x)

Features

score(n,p,q)

Plan x


• Resource class 1: A resource type r belongs to class 1 if the feature related consumption of the resource is limited to exactly the release in which the feature if offered. Resources of this class are called local based on its spending mode.


Consumption(k,r,x) = ∑n: x(n)=k consumption(n,r) ≤ Capacity(k,r)


• Resource class 2: A resource type r belongs to class 2 if the feature related consumption of the resource can be distributed across different release periods. Resources of this class are called global based on its spending mode.


∑ n=1..N wx (n,k,r) consumption(n,r) ≤∑ Capacity(k,r) for all releases k = 1…K

0 ≤ wx (n,k,r) ≤ 1 for all n,k,r

∑ k = 1 .. K wx (n,k,r) = 1 for all n,r

Comparison of EVOLVE II with other methods


Method

Characteristics EVOLVE II on-the-fly planning [van den Akker et al. ‘08]

Time horizon Flexible Next release Next release

Objectives Flexible in the number and type of criteria

Vague and not explicitly described

Maximize financial value function

Stakeholder involvement Strongly supported with explicitly assigned individualized tasks at the different stages

Opportunistic and by communication

Not directly supported

Solution method Specialized integer programming with additional heuristics

Intuition and experience-based Integer linear programming (ILP)

Quality of solutions Five near optimal alternative solutions with known level of optimality

Difficult to judge. The more risky, the more complex the problem

Near-optimal solutions based on ILOG

Feature dependencies Precedence and coupling Implicitly, hard to consider for more complex problems

Precedence,coupling, either or dependencies

Human resource constraints number, type and granularity of the resources

Implicitly, hard to consider for more complex problems

Yes, including staffing of teams

What-if analysis(explicit support)

Yes No Yes

Integrated tool support ReleasePlanner 2.0 No Prototype based on usage of ILOG

EVOLVE II tool support - ReleasePlanner


ReleasePlanner™ - Main components


Use cases

1. Project definition: stakeholders, criteria, features, resources, estimates, capacities, number of releases, permissions

2. Feature prioritization3. Most controversial features4. Alternative plan generation5. Feature dependencies6. Excitement analysis for a given plan 7. Customization of plans8. Comparison between two selected plans9. JIRA: Import of issues and subsequent plan generation10. Change of data in JIRA and synchronization11. Innovation planning where stakeholder represent competitors12. Service portfolio planning13. When to release planning14. Feedback-driven planning15. Planning functional versus quality requirements


Contents




From closed to open world planning

Open innovation

• An (open) approach for integration of internal and external ideas and paths to market that merges distributed knowledge and ideas into production processes.


Release Planning – Information needs


Information needs

Type of release planning problem

Feat

ures

Feat

ure

depe

nden

cies

Feat

ure

valu

e

Stak

ehol

der

Stak

ehol

der o

pini

on a

nd

prio

ritie

s

Rele

ase

read

ines

s

Mar

ket t

rend

s

Reso

urce

con

sum

ptio

ns

and

cons

trai

nts

What to release × × × × × × ×

Theme based × × × × × × ×

When to release × × × × × × ×

Consideration of quality requirements × × × × × × ×

Operational release planning × × ×

Consideration of technical debt × × × ×

Multiple products × × × × × × ×

Analytic open innovation

• Open innovation with emphasis on analytics (processes, tools, knowledge, techniques, decisions).


How much planning is enough?


Perfection of information 100%

Valu

e an

d co

st o

f add

ition

al in

form

atio

n

(Harrison 1987)

Benefit

Cost

Cost-benefit

ROI the better the more often investments are used

Pro’s of investment

• Pro-active evaluation of impact of decisions

• Support to find the most promising decision alternatives

• Transparency

• Understandability

• Reducing the impact ofhuman bias

• Reducing the risk of failure

• Increasing the chance ofsuccess

77ICSE 2016, Austin, TX

Con’s from investment

• Additional effort on decision-making

• Additional effort on information retrieval

• Effort to become familiar with some support tool(s)

• Unavoidable uncertainty (depending on scope)



Summary• Basic assumption: The more qualified processes and

support is provided, the better the chance to find an appropriate decision.

• Benefit of a mature release planning process:– Better customer satisfaction– Higher competitiveness of

products– Transparency of decisions– Ability to adjust to change– Alignment to business

objectives– Higher predictability of

results


Acknowledgements

• This work has been partially funded by the SUPERSEDE H2020 project (2012-2015) under contract nb. 644018

• The first presenter wants to thank D. Ameller and C. Farré at UPC for their work in the topic of the tutorial

• The second presenter acknowledges the support provided by NSERC and the collaboration with Maleknaz Nayebi on this topic.


.5

Xavier FranchGroup of Software and Service Engineering

Universitat Politècnica de CatalunyaBarcelona, Spain

[email protected]

Guenther RuheSoftware Engineering Decision Support Laboratory

University of CalgaryCalgary, Alberta, Canada

[email protected]