Evaluating Health Promotion: Why Does

Context Matter?

Louise Potvin, PhD

Chair Community Approaches and Health Inequalities

Université de Montréal

2nd Brazilian Seminar on Health Promotion Effectiveness

Rio de Janeiro, May 14th 2008

Acknowledgements

• Sherri L. Bisset, PhD candidate (U. de Montréal) whose thesis provides empirical illustrations for this presentation

• Contributors to: Health Promotion Evaluation Practice in the Americas: Values and Research (forthcoming). New York: Springer.

Thesis

• Health promotion programs are systems of actions that operate as socio-technical networks: they produce innovations by creating and strengthening new linkages between technical devices and local actors

• The role of evaluation is to produce knowledge to guide future action regarding parts of the socio-technical network, in the same or other contexts

• Evaluation’s important role is to document links between technical devices, local actors, conditions and actions, and changes in local context

Plan

1. Programs in context: health promotion as a socio-technical network

2. Program implementation: Operating the expansion and consolidation of a socio-technical network

3. Program evaluation: context matters

Part 1:Health Promotion as Socio-

Technical Network

The Ottawa Charter: Two Innovations

• Definition of health– Health is produced in every day life; linked to

access to local resources/conditions

• Principles of action– Participation & Empowerment: legitimacy of

non expert, local knowledge – Intersectoral: health producing resources are

accessible through non health sector providers

A Social Definition of Program

• Programs are social constructions: a tinkering of previously unrelated (or loosely related), and disparate human and non human components:– Knowledge (model of action; best practice; local

culture)– People (staff; target population; partners)– Problem (determinants, consequences)– Technical devices (compound; manual; meeting

minutes)– Activities (meetings; courses; celebrations)– Resources (financial; material; human)

• A mix of “local” and “imported” elements• Need a social process to explain how such

heterogeneous elements can hold together

The Actor-Network Theory (ANT)

• Michel Callon; Bruno Latour; John Law• Two underlying stories (J. Law):

1. Relational materiality: « Entities take their form and acquire their attributes as a result of their relations to other things… divisions and distinctions are understood as effects. They are not given in the order of things »

2. Performativity: « Entities are preformed in, by, and through, those relationships. »

Socio-Technical Network

• Focus on actions and interactions: Both actors and networks are outcomes and linked through action.

• Actors: no ontological rupture between human and non-human (technical) entities; both are capable of agency. Both are defined by, and define, the network

• Two orientations of action: stabilising existing relations and making new connections

• Translation links heterogeneous entities into a network

Socio-Technical Network of Health Promotion: School-Based Nutrition

Problem: feeding children

Mothers: time & skills

Lunch in schools

Problem: lack of success in school

Problem: skills & Interest in food

Nutritionists

Nutrition workshops

Didactic links with School subjects

External funder

School teachers &principalsMothers: support in

workshop

University healthresearcher

University educationresearcher

Ministry of Education

Other school boardsBrazilian health

promotion evaluators

Part 2:Expansion and consolidation of

a socio-technical network

Translation• Operation of linking the network’s

heterogeneous entities• Ongoing interpretations/reinterpretations by

actors of their roles and of the innovative product, going from their respective interests and their power relations and leading to the elaboration of compromises

• Four operations: – Problematization– Interessement– Enrolment– Mobilization

Problematization

• Setting in motion of actors around a provisional and minimum project

• Definition of the problem/situation by the project or innovation promoters,

• Identification of affected actors, their interests and the issues linking them

• Assingment of roles and identities• HEALTH PROMOTION: assigning health

meanings to non-health entities

Interessement

• Set of strategies adopted by the various actors with a view to:– rallying the other actors around a shared objective– defining their role

• Interessement strategies seek to align actors’ new identities and roles with their interests

• Interessement strategies take shape in material devices

• HEALTH PROMOTION: actions and artefacts that concretely link existing entities with health; knowledge, partnership agreements,

Enrolment

• Enrolment occurs when the actors take on a role in the network in line with the problematization

• Successful interessement gives rise to negotiation which leads to acceptance of a precise role enabling the network’s consolidation

• HEALTH PROMOTION: integration of new health-related roles and identities by networks entities

Mobilization• Mobilization concerns the involvement of a

critical mass of actors in the action system so that innovation becomes relevant, useful, indispensable

• Actor mobilization, above and beyond their representatives, leads to network extension

• In contrast, the absence of solidity of representatives leads to controversies

• HEALTH PROMOTION: capacity to displace entities and orient their actions in a health-related direction

Translation

PROBLEMATIZATION

INTERESSEMENTMOBILIZATION

ENROLMENT

CONTROVERSIES

An Example: Petits Cuistot-Parents en Réseaux

Problem: lack of success in school

Problem: skills & Interest in food

Nutritionists

Nutrition workshops

Didactic links with School subjects

External funder

School teachers &principalsMothers: support in

workshop

Part 3:Context and Program

Evaluation

Evaluation and Socio-Technical Networks

• Socio-technical networks are performative: they acquire reality through action

• Movements in socio-technical networks: consolidating existing linkages or expansion through new linkages

• Evaluation: systematic knowledge to inform movements for consolidation or expansion of network

Knowledge to Support Consolidating Network

• Focus on the internal context of programs

• Modeling existing entities (numbers, identities, roles) their links (strength, meaning), actions (number, nature, roles) and controversies (nature, solutions; translations);

• Methods: systematization• Complementarity of qualitative and

quantitative information

Knowledge to Support Expanding Network

• Networks expansion : enrolling new entities through new or renewed linkages

• Focus on the link between existing network (internal context) and external context

• The distinction between external and internal is contingent

Lessons from Health Promotion Evaluation Practices in the

Americas• Research methods are not « context-

neutral »; evaluation activities are part of a program context and there are meaningful linkages between evaluation and program

• Innovative evaluation practices strengthen links between local context, evaluation and program:

• Evaluators have difficulties to reflect on their practice beyond the technical aspects of cellecting empirical data