© SEESAC, 2006

Name, Organisation, Event

Location, Date

Monitoring and evaluation during SALW Awareness

programmes

© SEESAC, 2006

Part 1 – 30 minutesPart 1 – 30 minutes

1. Define monitoring and evaluation

2. Explain the differences between them

3. Explain why they are important

© SEESAC, 2006

Part 2 - 45 minutesPart 2 - 45 minutes

1. Describe the different levels at which monitoring and evaluation can be carried out

© SEESAC, 2006

Part 3 – 40 minutesPart 3 – 40 minutes

1. List some common tools for doing monitoring and evaluation

2. Explain why it is best to use a mix of tools

3. List some common criteria for selecting monitoring and evaluation tools

© SEESAC, 2006

Part 1Part 1

© SEESAC, 2006

Exercise 1Exercise 1

1. What is monitoring?

2. What is evaluation?

3. What are the similarities and differences?

4. Is it important to do M&E, and if so, why?

© SEESAC, 2006

An informal explanation: Monitoring and evaluation are two forms of

activity which aim to measure how your work is progressing

Measurement during a project = monitoring Measurement after a project = evaluation

What are monitoring and What are monitoring and evaluation?evaluation?

© SEESAC, 2006

A formal definition (SASP 2)…A formal definition (SASP 2)…

Monitoring and evaluation (M&E) =

“Collecting and analysing information….

…to determine whether those groups engaged by a SALW Awareness programme have,…..

…as a result of the intervention,….

…changed their awareness of, and attitudes and behaviour towards, SALW,….

… in line with the stated programme objectives.”

© SEESAC, 2006

Table 9, (SASP 2)…Table 9, (SASP 2)… 

WHAT IT INVOLVES WHEN IT OCCURS WITHINTHE PROGRAMME CYCLE

Monitoring

Tracking progress towards the achievement of objectives, in order to identify what is working and what isn’t working so well, allowing a degree of adaptability in strategy and tactics as appropriate.

Ongoing.

Evaluation

A more structured and formal process of reviewing achievements, in order to make judgements about past effectiveness and learn from experience to improve future practice.

At fixed times – for projects lasting more than 18 months, this would normally include a mid-term review as well as an evaluation conducted at the completion of the project.

© SEESAC, 2006

Why are they important?Why are they important?

Test for effectiveness Better appreciation of area where working Learn how events have affected the work Adapt programme both during project life-

cycle (M) and before next phase (E) Identify good practice to use elsewhere Information to share with others

The programme cycleThe programme cycle 

Feasibility study

Analysing and planning

EvaluationDesigning

activities and materials

Field-testing

Implementingactivities

Monitoring and reviewing

Needs and capacity assessment

Planning for monitoring and evaluation

© SEESAC, 2006

The time difference…The time difference…

RESEARCH

+

ANALYSISDESIGN IMPLEMENT EVALUATE

- Monitoring -

- Evaluation -

© SEESAC, 2006

Part 2Part 2

© SEESAC, 2006

Imagine you are doing SALW Imagine you are doing SALW Awareness …Awareness …

© SEESAC, 2006

Levels of M&E (SASP 2)Levels of M&E (SASP 2)

LEVEL OF EVALUATION PURPOSE KEY QUESTIONS

Activities To assess how well the programme has been organised and whether resources have been used efficiently.

Outcomes To identify changes in knowledge, attitudes and behaviour among target groups that can be reasonably attributed to the programme.

Impact To explore how a particular programme may have made a difference to the lives of specific groups of people, e.g. better security.

- Are we sending people the correct messages?- Are the messages reaching the right people?

- Are there any signs that knowledge, attitudes and beliefs are changing?- Are there any signs that behaviour is changing?

- What impact has the programme had in terms of security / casualties etc.?

© SEESAC, 2006

Activity monitoring example..Activity monitoring example..

Monitoring a TV spot Watch TV!

Monitoring a community meeting Phone the local organiser

afterwards

© SEESAC, 2006

Outcome monitoring example…Outcome monitoring example…

Balkan Youth Union (BYU) 5th April 2003, central Belgrade BYU and children destroyed 500 toy

weapons Puppet show T-shirts MUP information leaflets to support

collection

© SEESAC, 2006

RTS and RTV B-92, as well as to the journalists of dailies DANAS, BLIC and POLITIKARTS and RTV B-92, as well as to the journalists of dailies DANAS, BLIC and POLITIKARTS and RTV B-92, as well as to the journalists of dailies DANAS, BLIC and POLITIKA

© SEESAC, 2006

Outcome monitoring example…Outcome monitoring example…

Outcome evaluation by: Letters to BYU (hundreds) Media coverage of interviews with the public

© SEESAC, 2006

Impact monitoring and evaluation…Impact monitoring and evaluation…

Difficult Other factors

Casualty figures Crime levels Observe weapons visibility

© SEESAC, 2006

Six questions…Six questions…

1. How has your awareness programme reduced the number of weapons casualties resulting from weapons in target communities?

2. Are the messages being promoted reaching the right people?3. Are there any signs of changes in practice or behaviour?4. How has your awareness programme changed security in

targeted communities?5. Are there any signs that knowledge, attitudes and beliefs are

changing?6. Are the messages being promoted the right ones?

© SEESAC, 2006

Part 3Part 3

© SEESAC, 2006

Things to consider:Things to consider:

Cost Staffing Skill-levels Representativeness Geographic coverage Depth of explanation Access to social groups Level of participation

© SEESAC, 2006

Common M&E ToolsCommon M&E Tools

Interviews Focus groups Questionnaires Secondary (desk) research Participatory methods

Map exercise…Map exercise… 

© SEESAC, 2006

Group Exercise…Group Exercise…

Group 1: Cheap Give a deep understanding of target group’s feelings about SALW Capture women’s views well

Group 2: Allow generalisations to be made about the wider population Allow respondents to participate

Group 3: Build the skills of respondents Capture information about unexpected impacts