Evaluative learning cycle

Written by Helen Titchen Beeth - based on material offered by the Living Wholeness Institute.

Introduction

When working with innovation, especially in large and static systems, how do we nurture new ideas and help them grow and scale out without forcing them or killing them? How do we evaluate progress and transition from one phase to the next? How do we keep learning together and keep the wider system in the loop without unleashing it’s immune system to attack the new?

INTRODUCING THE ECO-CYCLE

Context: Management circles are familiar with the ‘S-curve’ model which describes the process of growth and maturity of industries and organisations. The eco-cycle model introduces two other aspects of living systems which are not considered by the S-curve, namely the ‘back loop’: death and conception, destruction and renewal.

It is important to understand that the eco-cycle describes a healthy dynamic, an ongoing movement that is happening all the time and that has the potential to spiral up after each iteration, if we but learn and evolve as we go.

Imagine what could happenIf I was so consciousAs to ask the better questions.To know that they exist,To see what they look like,To hear what they say.To discover where they take you.Beneath the surface, beyond the benign inert, To incubate the seeds of our highest potential.Why not ask the better questions?Fear, conditioning, cost.Why not ask the better questions?Is that the better question?

Poem by Mary Heneghan

We can distinguish four phases, starting with the two better known elements of the aforementioned S-curve:

Growth - this is a phase of high energy, lots of new ideas and trial-and-error learning. Seeds - the germs of new ideas - are sprouting all around and we are beginning to see what the new garden might look like. Resources are spread over a variety of projects or activities, and it is time to carefully consider which conditions and new structures are needed to allow the most promising ones to scale out. At the growth stage, it is not yet reasonable to expect large-scale impact.

Conservation - after a while the open space becomes crowded, competition starts to require efficiency. Focus shifts to consolidating and conserving resources; the forest has grown to maturity. A large organisation like the European Commission, which has been around for over 50 years, shows the properties of the conservation stage. It knows how to maintain its systems (whether good or bad) and it has impact and scale. We hear a lot of stories about how the EC system is seeking to consolidate its functioning, even with fewer resources. The positive side of conservation is preserving the story and the legacy, maintaining continuity, taking care of knowledge transfer. Here we need to continually reconnect with purpose.

Creative destruction (often also called ‘creative closure’) – the best analogy for this phase is the forest fire, needed to renew the whole ecosystem. While the system might appear to have been completely destroyed, this is not the case. An organisation might have become stagnant, unable to open up sufficient space for new

initiatives to survive. This release phase may require dismantling systems and structures that have become too rigid, have too little variety and are not responsive to the current needs of the context. It is always about the need to let something go.

Generation – In the forest, after the fire, the seeds now have the heat, light and open space to start growing. The germination phase needs very specific conditions: tender sprouts need sheltered conditions. When that germination is the seed of a new idea, you often don’t know what will grow out of it. Research done in the field of venture philanthropy suggests that a success rate of 1 idea in every 10 is an excellent outcome. In organisational terms, then, this phase is not about increasing efficiency or even effectiveness. Instead, the need is to create connections, mobilise resources and skills to create the next generation of effective (and eventually efficient) processes and services.

In a healthy system, there will be evidence of all four phases simultaneously present in different parts of the system. Often, in growth models, only the growth and conservation phases are seen and intentionally engaged with. You keep growing, bigger is better, progress is upwards. We see this focus in a lot of economic paradigms as well: there’s no back loop of release and regeneration phases. In the eco-system, this is the recognition that things have to change and things die.

The crossroads, where the back loop crosses the front loop, often represents those moments of transition between one thing and another, one state and the next. For example, when a project is being wound down (creative closure), or when somebody leaves,

it leads to an opening. This stage is an opportunity to learn what we need to let go of, what needs to change in order to move forward. It’s a clearing of the space.

USING THE ECO-CYCLE AS A FRAMEWORK FOR EVALUATING INNOVATION

When incubating innovative initiatives inside a stable, established organisation, those tender new shoots can die very quickly if they’re not appropriately supported and sustained. Particularly important is the question of how to evaluate a new initiative so that it can continue to grow, as opposed to being seen as a failure for not having the desired the impact, because it isn’t monitored in a way that could support it.

Using the eco-cycle as an evaluative framework can resolve this difficulty. Setting a project in the context of its maturation process can help us to evaluate its progress more effectively and realistically by offering a more appropriate range of questions to ask in the different phases.

Setting the stage - The initial stages of a project are about clarifying the context, scope and purpose. This phase can last anything from days to years, depending on the complexity and scope of the inquiry.

Questions to ask:

• What is the need that has arisen? Whose need is it?• What are we in pursuit of? What are we really trying to

achieve?

The outcome of this phase is having a clear purpose.

Developmental stage - this is the stage when we really start to understand that we are onto something. It’s not yet time to ask if we have been successful, because we don’t yet know what success might look like. We are still evolving what we’re doing; still trying to understand it. Once we know what we are in pursuit of, we need to allow ourselves the time and space to really envision what it would mean to have achieved our goal, and how we will know when we get there. The art lies more in finding the right questions to pursue than which specific outcomes to pursue - those will begin to show as the developmental stage progresses. The process is iterative - as the project develops, the answers will change.

Questions to ask:

• What is emerging that is new?• What are we learning that we can begin to use?• How will we know when we have reached our goal?• What are the elements that will show us that we are achieving

our purpose?

Outcomes of this stage include:

• an incipient strategy or pathway of change• action areas and activities inside those areas• success criteria or performance indicators.

One thing that is crucial to understand at this stage is that the innovating team or system must come up with its own performance indicators - this cannot be done by somebody else.

Growth stage - when we start to see what’s working and we want to scale it up, it is time to use a different form of evaluation. Now that we know what we’re working with, and we know how to track it more specifically, we no longer need to work with the questions in the same way. This stage calls for ‘formative evaluation’.

The question to ask is:

• What is working that can be scaled up?

The outcome of this stage is prototypes.

We often underestimate the time it takes to perfect a prototype so that it can be scaled out successfully. Formative evaluation is often placed on innovations much too early, before they are ready. Often we scale things up before they have the right form, or we scale up the wrong thing, or we scale it up when it still has a big deficiency in it. The more ambitious your goal, the longer it is likely to take to get there!

Conservation stage - once we reach the conservation stage, we really know what we’re working with, and the scale of work is such that we can evaluate it in the conventional way. We can see the impact of the work, there is a new story to tell and ‘summative’ evaluation is appropriate.

Here, questions to ask include:

• What have we achieved?• What new story can we tell?

The outcome at this stage is metric reports and policy development.

Creative closure - Once we have been working in these different ways for a while, we can start to see the impact. Just because we are succeeding, though, it doesn’t mean that we just keep repeating what we’ve been doing. It is time to learn from our impact, and to ask new questions, to scale up again, or to go to the next level of our work. We call it creative closure because there are invariably things we must stop doing if we are to strategically move forward.

Questions to ask:

• What has our impact been?

• What do we need to let go of, and what do we need to scale up, in order to strategically move forward at a higher level?

The outcome at this stage can be to ask new questions and scale out to a new level.

APPLYING THE EVALUATIVE LEARNING CYCLE IN PRACTICE

This evaluative learning cycle is iterative; it is always in movement. The key capacity is learning. Just because an external evaluator comes in to evaluate something doesn’t necessarily mean that people are learning. That’s why this approach is different: as we begin to learn - and get better at it - we begin to see what’s actually needed. Typically, we have an experience, and at the end of the day we come together to track that experience and see what we need to do the next day - it’s a very fast feedback loop.

The 3 key questions are: What? So what? and Now What?

• What has happened? Sometimes we have all been in an experience together, and can bring our unique perspectives to bear on it, enriching and deepening our collective understanding. Sometimes we have all been in different experiences, related to the field of our innovation. In which case, we bring all our different experiences back to the team and expand our collective understanding.

• So what does that mean? It’s not enough to share the ‘what’, if we don’t then go on to make sense of the experience together, in the light of our shared purpose. This is where the collective learning happens.

• And now what are we going to do? The more complex the project, the fewer steps ahead we can envision at any moment. It is enough to see the minimum elegant next step at any time.

When we are in a new stage of any project or initiative, it is important to practice that a lot, to get better and better at it. This enables us to work with the new levels of complexity of change in a much more stable way. This iterative evaluation cycle creates not stability, but a sense of continuity in our learning.

In each evaluation phase, we gather information and collect evidence, using many different modalities. Whenever we are developing something new, we really don’t know what we’re doing! It is therefore crucial to use practices like journalling, recording conversations, documenting the HOW at each step in as many ways as possible. Because although we don’t know what we’re doing as we’re doing it, we can look back at it later and understand: Ah! That’s what we were talking about! Having some clarity in retrospect, we can more effectively (and

efficiently) crystallise an idea and do something with it. This is another part of the practice.

Public health warning:

This framework of evaluating learning is offered as a practice! It must be brought into a living context and worked with in ways that are real. Just hanging the slogan in your office will not have the desired effect. It is only meaningful if you use it - try it, stop, learn, reflect, keep trying. Rome was not built in a day - but once it was there, it changed the world.