Developmental Evaluation… an emerging approach, discipline ,

methodology, disposition…

Rita Ndagire KizitoCTLM 30 March 2016

• Developmental evaluation is an emerging approach to evaluating innovations (or

interventions) …that are in a continuous state of unpredictable change and development (Patton, 2011; Reynolds, 2014).

• Builds on Patton’s (1978) earlier concept of ‘utilisation-focused’ evaluation.

• A form of design thinking that situates evaluation differently ,as “ not simply part of quality assurance, but as a key design, feedback and planning tool”

(Leonard, Fitzgerald & Riordan, 2015)

• Outcomes and goals are already known

• Seeks to establish if progress is being made towards these goals

Conventional Evaluation(Formative or Summative Evaluation

Developmental Evaluation

• Seeks to ‘design’ the goals

• Asks what works for whom, and in what circumstances?

(Leonard, Fitzgerald & Riordan, 2015)

Conventional Evaluation

Plan Act

(Formative Evaluation)

(Summative Evaluation

Evaluate

DevelopmentalEvaluation

Evaluate

Time

Plan

Act

Evaluate

“these two types of thinking are seen to be mutually exclusive, but developmental evaluation is about holding them in balance” ( Gamble, 2006)

Developmental evaluation

merges

Evaluation

…about critical thinking

Development

…about creative thinking

Which type of evaluation is appropriate ?

“what ? So what? Now what ?”

Forms of evaluation

Object of

evaluation

Developmental

(In the initial stages

and along the way)

Formative

(along the way)

Summative

(at the end or

end of a

section)

Processes

and

procedures

Have we misjudged

something?

Has something

unexpected

occurred?

Do we need to

change our

assumptions?

Is there

anything we can

adjust to

improve what is

happening?

Delivery - Did

things happen

the way we

planned for

them to

happen?

Quality- how

well did things

get done?

(The next five slides are from a presentation by Prof Jeff Jawitz, 2015, UCT)

Jeff Jawitz, 2015

Object of

evaluation

Developmental

(In the initial

stages

and along the

way)

Formative

(along the way)

Summative

(at the end or

end of a section)

Participant

satisfaction

and

experience

How well did

we understand

participant

expectations?

Can we identify

and address some

of the negative

aspects of

participant

experience?

Can we identify

and take

advantage of

positive aspects of

participant

experience?

Did participants

report positive or

negative

experiences of the

course?

Jeff Jawitz, 2015

Object of

evaluation

Developmental

(In the initial

stages

and along the

way)

Formative

(along the

way)

Summative

(at the end or

end of a section)

Participant

performance

and

achievements

(learning?)

How well did we

judge the

capabilities of

the participants?

How well have

we been able to

stretch

participants in

terms of

extending their

capabilities and

understanding?

Participant

engagement –

to what extent

did

participants

engage with

elements of

the

programme

- what was the

quality of

engagement?

Did participants

achieve the levels of

performance that we

expected them to

achieve?

Jeff Jawitz, 2015

Object of

evaluation

Developmental

(In the initial

stages

and along the

way)

Formative

(along the

way)

Summative

(at the end or

end of a

section)

Project

effectiveness

How realistic are

we in expecting

significant results

from the

effective design

and delivery of

projects within

the institutional

context?

Can we

improve

support for

project

design?

What levels of

effectiveness

of

participant’s

project was

achieved?

Jeff Jawitz, 2015

Object of

evaluation

Developmental

(In the initial

stages

and along the

way)

Formative

(along the

way)

Summative

(at the end or

end of a

section)

Task: Select a few items that you wish to evaluate related to your project. Identify three questions that reflect the three different evaluation focusses for each of these items.

Jeff Jawitz, 2015

References

• Jawitz Jeff ( 2015). Presentation at TAU Fellowship workshop• Leonard, S. N., Fitzgerald, R. N., & Riordan, G. (2015). Using

developmental evaluation as a design thinking tool for curriculum innovation in professional higher education. Higher Education Research & Development, 1-13.

• Patton, M.Q. (1978). Utilization-focused evaluation. Beverley Hills, CA: Sage.

• Patton, M.Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford Press

• Reynolds, M. (2014). Equity-focused developmental evaluation using critical systems thinking. Evaluation, 20(1), 75–95. doi:10.1177/1356389013516054 .