Upload
tim-hoogenboom
View
385
Download
2
Embed Size (px)
DESCRIPTION
Citation preview
Evaluation of the Intervention!\or
Realistic Evaluation after your taking action in the problem situation
PoliticianPrisons Work!
Evaluator What is it in the infinite complex
social system of prisons that make people change their behaviour
Two perspectives on reality
Fallacies of Experimental Evaluation
pre-test treatment post-test
experimental group O1 X O2
control group O1 O2
What causes something to happen has nothing to do with the number of times it happens
(Sayer, 1992, p.165)Sayer, A.R., ‘Method in social science, a realist approach’, Routledge, London, 1992
Prisons Work!
Treatment Group
Control Group
Normative Base Rate
0%
10%
20%
30%
40%
50%
60%
Recedivism Rates
Recedivism Rates
Succesionistic regularity, but what works?
Trivial?
MISQ, December 2004
ICIS, December 2007
Why Realist Evaluation• Departure: Social problems are embedded in a stratified
reality
• Rationale: Not repeating programs as a craft, but evaluating program theories as epistemic devices
• Generative causality: Knowing what works for whom in what circumstances (resources and reasoning)
• Realistic: Don’t study the program, study the inner workings
MC
xR
y
Pawson, R. and Tilley, N., ‘Realistic Evaluation’, Sage, Thousand Oaks, California, 1997
Theory-driven Evaluation
Generative Regularity
Prison Workings!Prison education (x) leads a reduction in recidivism (y)
Reasoning or Resources M1: self-realization • M2: last chance M3: second chance M4: social acceptability • M5: moral or civic responsibility M6: cognitive change
‘Certain type of prisoner’ C1: Disadvantaged • C2: The crime committed (pedophilia, pyromania) C3: First Offenders C4: Youth prisons, Isolation Cells, Federal Prisons, Death sentenced
• mediocre hypothesis
M
C
xR
y
Pawson, R. and Tilley, N., ‘Realistic Evaluation’, Sage, Thousand Oaks, California, 1997
Another Wicked Problem: Multiple Explanations
Making sense of it:• Theory adjudication
weeding out rival explanations, with focus on improvement
• Configurationalapplication in practice
• Middle-rangetransferability; patterns of repeat victimization
• Provisionaltested but not proven; soft system and wicked problems
Pawson, R. and Tilley, N., ‘Realist Evaluation’, in S Mathison (ed.), Encyclopedia of Evaluation, Sage, Thousand Oaks, California, 2004
The upside of Realistic Evaluation• Dealing with real or wicked problems• Design needs Evaluation
Did the intended outcome actually materialize?
The downside• No clear-cut recipe• Difficult language and concepts
Fitness for MIW
Camera-surveillance in Car Parks
• A form of situational crime prevention
• Visual surveillance technology monitoring variety of environments/activities
• Growth estimated at 15-20% per year (Davies, 1996)
• In car parks CCTV is used to prevent and reduce crime
Opdracht
• Experimentele Evaluatie
Camera’s werken!
• Realistische Evaluatie (opdracht):
Wat is het in cameratoezicht waardoor mensen hun gedrag aanpassen – Bedenk de mechanismes– Bedenk de contexten
Realistische Analyse
Mechanismen• Caught in the act• You’ve been framed• Nosy parker• Effective deployment• Publicity mechanism• Time for crime• Memory jogging• Appeal to be cautious
Context• Criminal clustering• Style of usage• Lie of the land• Alternative targets• Resources• Surveillance Culture
Do you believe that
Hypothesizing about contexts, mechanisms and outcomes and testing these empirically
Is more valuable than
Comparing the rates of crime before and after CCTV
Then we succeeded