Transitioning Towards Continuous Experimentation in a Large Software Product and Service ...

Sezin Gizem Yaman1, Fabian Fagerholm1, Myriam Munezero1, Jürgen Münch1,2, Mika Aaltola3, Christina Palmu3, Tomi Männistö1

1 2 3

Transitioning Towards Continuous Experimentation in a Large Software Product and Service

Development Organisation – A Case Study

Continuous experimentation is a software development approach

where research and development activities are driven by constantly

conducting experiments with product value*.

* Rissanen, O., Münch, J.: Continuous Experimentation in the B2B Domain: A Case Study.* Fagerholm,F.,Guinea,A.S.,Mäenpää,H., Münch, J.: The RIGHT model for Continuous Experimentation.

HOWEVER,guidence is lacking on how development teams in large organisations

with complex business partnership networds can transition towards continious experimentaiton.

As our aim was to observe the introduction of continuous experimentation in a company, we conducted an actual experiment round with a real product.

Transition

Small teams

Small-scale experiments Experimentation

TargetCollaborationInitial circumstances

The experiment Persistence

Initial Circumstances

Multiple stakeholders- Select a focus

Evolving product- Allocate few resources

Need to limit the risks- Start small

Small teamsSmall-scale experiments

Champions

Brainstorming on possible experiments

Simplistic UI experiment as a start

[It’s better to] start experimenting with something small. [...] It’s more important to start now. Practice will make it perfect.” (Technical coach)

Identifying an experimentation target

Utilise existing resources

Numerous discussions

Prioritize and analyse the feature

Designing and executing the experiment

Pilot run

Rerun the experiment

Expert advice

Emphasize “learning”

“Fix the experiment [the] best way you can and run it again. You can learn so much with each experiment.” (Technical coach)

Collaborating with experts

Avoid mistakes, biases

Faster start

Iterative learning

Persistence

Learn from the failures

Keep practising

Scale up

“One should not dwell on temporary failures, but keep practicing.” (The teams)

Transition Small teams

Small-scale experiments

Experimentation Target

Collaboration

Initial circumstances

The experiment

Persistence

“Experimentation made it clear to the teams that there is no need to debate between opinions and assumptions as you

can quickly test them with an experiment.” (Technical coach)

Reference

Sezin Gizem Yaman, Fabian Fagerholm, Myriam Munzero, Jürgen Münch, Mika Aaltola, Christina Palmu, Tomi Männistö. Transitioning Towards Continuous Experimentation in a Large Software Product and Service Development Organization – A Case Study. In Proceedings of the 17th International Conference on Product-Focused Software Process Improvement (PROFES 2016), LNCS. Springer-Verlag, 2016.

Get the article here: http://bit.ly/corporateexperimentation

sezin.yaman@helsinki.fi

www.cs.helsinki.fi/en/people/yamanwww.cs.helsinki.fi/group/ese/

Sezin Yaman

University of HelsinkiEmpirical Software Engineering Helsinki Research Group

Discussion:

What other ways a software company can transition towards continuous experimentation?