Project Evaluation and

Recycling

Prepared by: Michael John D. Sison MDM student

Presented to: Josefina B. Bitonio, DPA

Professor

DM 211 Project

Development and

Management

RANGESAMY, 2008

Project recycling

Topics at glance

Project tools and

techniques

Project evaluation

and recycling

d. Stages

of project life

Project Evaluation and Recycling

Monitoring is defined as the routine collection and use of data to assess progress in achieving programmed objectives. These data are generally derived from program records

What is monitoring?

What is a project?

Project

Manager

A project is a

package of:

Objectives

Outputs

Activities

inputs

Project

Staff Project

Beneficiaries

Mercado: 2005

What is the origin of evaluation?

Evaluation has its origin in the Latin word “Valupure” which means the value of a particular thing, idea or action (Rengasamy, 2008).

Evaluation is the process of analyzing program and project inputs, transformation processes , and the effect and impact of outputs and outcomes against explicitly stated goals and objectives (Mathur and Inayatullah as cited by Miclat, 2005).

What is evaluation?

Differences between monitoring and evaluation (United Nations Evaluations Group, accessed 2014)

Project evaluation assesses activities that are designed to perform a specified task in a specific period of time. For example, a three-day workshop on volunteerism, a one-year fundraising program, a three-year HIV prevention intervention, a five-year career development innovation, or an ongoing newcomer youth orientation and integration training program are all projects that you can evaluate (Zarinpoush, 2006).

What is Project

Evaluation?

• To assess project results

• To improve project management and process planning

• To promote learning

• To understand different stakeholders’ perspectives

• To ensure accountability

In general, the purpose of monitoring & evaluation based in Rengasamy (2008) can be:

1. Efficiency refers to the amount of time and resources put into the project relative to the outputs and outcomes. A project evaluation may be designed to find out if there was a less expensive, more appropriate, less time-consuming approach for reaching the same objectives. 2. Effectiveness describes whether or not the research process was useful in reaching project goals and objectives, or resulted in positive outcomes. 3. Relevance or appropriateness describes the usefulness, ethics, and flexibility of a project within the particular context.

Underlying reasons (Rengasamy, 2008) for monitoring and evaluating are frequently framed in terms of:

What will happen if we mix the three?

It will enable judgment about whether the outputs and outcomes of the project are worth the costs of the inputs. Effectiveness, efficiency and appropriateness can be considered for the different methods, tools and approaches rather than questioning the value of the research approach as a whole (Rengasamy, 2008).

What to Monitor and

Evaluate (Rengsamy,

2008)? Outputs

• describe the concrete and tangible products of the research as well as the occurrence of the research activities themselves.

Processes

• describe the methods and approaches used for the project.

Impact

• describes overall changes that occur in the community to which the project is one of many contributing factors.

Reach

• describes who is influenced by the project and who acts because of this influence.

Outcomes

• describe the changes that occur within the community or with the project managers that can be attributed, at least in part, to the project process and outputs.

Understanding the

condition of the

community/target group

before the project was

initiated is useful in order

to provide a point of

comparison for monitor

and evaluating changes

that occur during the

project. Baseline survey

conducted at the

beginning of the project

can provide a point of

reference for comparison

and for understanding

change in the

community/target group.

Program Project Logmap

Mercado:2005

User

(Community)

Effects Personal

Outcomes Organization

Impacts Social

Improved:

1. Capacity

2. Efficiency

3. Effectiveness

4. Performance

5. Productivity

6. Quality

7. Profitability

8. Empowerment

9. Access

10. Availability

Increased:

1. Per capita

income

2. Employment

raised

3. Life expectancy

4. Literacy rate

Decreased

1. Mortality rate

2. (In)fertility rate

Consumers

Objectives

Improved:

1. Awareness

2. Knowledge

3. Attitude

4. Skills

5. Practice

Purpose of Evaluation (Rengasamy, 2008)

From an accountability perspective:

It is to make the best possible use of funds by the program managers

who are accountable for the worth of their

programs.

Measuring accomplishment in

order to avoid weaknesses and future mistakes.

VERSUS From a knowledge perspective:

Understanding people’s participation & reasons for the same.

It is to establish new knowledge about social problems and the effectiveness of policies and programs designed to alleviate them.

Evaluation helps to make plans for future

work.

Examples of Evaluation Purpose Statements (Zarinpoush, 2006)

• To assess the degree to which project objectives were achieved.

• To document the lessons learned.

• To provide recommendations for project development and improvement.

• To examine the changes that resulted from doing the project.

• To provide input to guide decision making for the upcoming renewal and extension of project funding.

Principles of Evaluation (Rengasamy, 2008):

It has a sense of

continuity.

It has a sense of

continuity.

It is inexpensive

It has a minimum hindrance to day-to-day work.

It has a total

participation, thus,

shows program totality.

It can be shared.

Preparing the Evaluation Study Proposal (Miclat, 2005)

Theoretical Framework

• The launching pad of the study from the abstract into the empirical world.

Methodology

• It discusses the methods and procedures to be used in the study.

Research Designs

• The blueprint of procedures that enable a researcher to arrive at valid and reliable findings and conclusions.

Sampling Technique

• Involves the population or universe to guarantee the validity of the study.

Research Instrument

• The data gathering device a researcher uses to gather data relevant to the study.

Treatment of Data

• The procedures on how the data would be analyzed taking into consideration the objectives of the study.

Additional two items on Methodology (Miclat, 2005)

The preparation of a research budget to carry out the study successfully should be stipulated.

Funding Requirements

The preparation of a time schedule in the conduct of the various activities of the evaluation study to maximally use time, money and manpower should also be included.

Timetable

●Impact evaluation

●Ex- post evaluation or (Summative / Terminal / Final)

●Final evaluation

3. PROGRAM COMPLETION

STAGE

●Monitoring Evaluation or Ongoing / interim.

●Concurrent evaluation

2. PROGRAM MONITORING STAGE

1. PROGRAM PLANNING STAGE

STAGES IN PROJECT EVALUATION (RENGASAMY, 2008)

●Pre – investment

evaluation

●Formative evaluation

●Ex – ante evaluation or

Early / Formulation

●Pre-project evaluation

●Exploratory evaluation

●Need assessment

●Formative evaluation

●Ex – ante evaluation or

Early / Formulation

●Pre-project evaluation

●Exploratory evaluation

●Need assessment

CRITERIA FOR DEVELOPMENT ASSISTANCE (RANGESAMY, 2008)

STEPS IN EVALUATION (RANGESAMY, 2008)

Learning about the program

Creating on evaluation plan and evaluation indicators

Briefing the concerned people

about the evaluation plan and indicators

Revising and elaborating the evaluation plan

Initiating evaluation Utilizing/Sharing

information

Implementing the Evaluation Study (Miclat, 2005)

Data Gathering

and Processes

Writing the Research

Report

Presentation and Analysis

of Data

Finding and Conclusions

Types of Evaluation:

Evaluation can be categorized under different headings (Rangesamy, 2008): A) By timing (when to evaluate) ●Formative Evaluation ●Summative Evaluation B) By Agency. Who is evaluating? ●Internal Evaluation ●External Evaluation C) By Stages ●On going ●Terminal ●Ex – post

Internal or External Evaluation (based on Rangesamy, 2008)

Internal Evaluation

It is also known as Enterprise Self

Audit.

It is a continuous process which is done at various points and in respect of various aspects of the working of an agency by the agency staff itself i.e. staff board members and beneficiaries.

VERSUS Internal

Evaluation

This is done by outsiders /Certified Management Audit.

Some donors may send consultants in order to see how far the standards laid down are put into practice.

Inter agency evaluation. In this type two agencies mutually agree to evaluate their program by the other agency.

Inter agency tours.

Desired Situation Sustained benefits

and impact

Present Situation

Mid-Term review End-of project or

final evaluation Ex-post or impact

evaluation

Time

Source: Rangesamy, 2008

Tools/ Techniques based on Rangesamy, 2008:

● First Hand Information ● Formal/Informal Periodic Reports ● Evaluation is also carried out through formal/informal reports *Formal Reports consist of: -Project Status Report -Project Schedule Chart -Project Financial Status *Informal reports such as: ● Anonymous Letters, Press Reports, Complaints by Beneficiaries & Petitions, etc. ● Graphic Presentations ● Standing evaluation review committees ● Project Profiles

Methods of Evaluation

Areas of Evaluation (Rangesamy, 2008):

Purpose: The review the objectives of the agency / project and how far these are being fulfilled. Programs: Aspects like number of beneficiaries, nature of services rendered to them, their reaction to the services, effectiveness and adequacy of services etc. may be evaluated. Staff: The success of any welfare program / agency depends upon the type of the staff an agency employs. Their attitude, qualifications, recruitment policy, pay and other benefits and organizational environment. These are the areas which help to understand the effectiveness of the project / agency.

Financial Administration:

The flow of resources and its consumption is a crucial factor in any project / agency. Whether

the project money is rightly consumed any over spending in some headings, appropriation and

misappropriation. These are some of the indicators that reveal the reasons for the success or

failures of any project.

General:

Factors like public relations strategies employed by the project /agency, the constitution of the

agency board or project advisory committee and their contribution future plans of the agency

are important to understand the success or failures of any project.

PROJECT EVALUATION (RANGESAMY, 2008)

Analysis on how successful the project has been in Transforming the means (i.e. the resources and inputs allocated to the project) through project activities into concrete project results Provides the stakeholders with information on

inputs/costs per unit produced

Overall Objectives

Efficiency

Means + Preconditions

Activities+ Assumptions

Results + Assumptions

Project Purpose + Assumptions

Change

utilization

action

allocation

Analysis on how well the production of project

results Contributes to the achievement of the

project purpose,

i.e.: Are there clear Indications of changes and

improvements that benefit the beneficiaries of the

project?

Uses base-line information on the pre project

situation as a starting point

Effectiveness

Impact

Analysis of the overall effects of the project

Analysis of the contribution of the project purpose to

the overall objectives

Focus on long-term changes in the environment of the

project “Collection” and analysis of information at

the levels of communities and society at large

focusing on the final beneficiaries of the project

Also analysis of unintended impacts (negative

and positive)

Views About Evaluation (Rangesamy, 2008):

● Evaluation as an analysis – determining the merits or deficiencies of a program, methods and processes. ● Evaluation as an audit – systematic and continuous enquiry to measure the efficiency of means to reach their particular preconceived ends. In the Agency Context: ● Evaluation of administration means appraisal or judgment of the worth and effectiveness of all the processes (e.g. planning, organizing, staffing, etc.) designed to ensure the agency to accomplish its objectives.

What is Project

Recycling?

Definition of Project Recycling

This is a process of iterations and reiteration. This is a post project activity wherein a separate planning based on results will have to be prepared. Depending upon the results of the project reports/analyses a project may be replicated or modified to suit the time, place, beneficiaries of the intended project (CLSU). The results of outputs and outcomes in terms effects and impacts are eventually recycled and feed-backed as inputs in the planning process (Miclat, 2005).

Is there a need for publication and dissemination of the evaluation?

Publication and dissemination of the evaluation

Project Evaluation and Recycling Model (Sison, 2014)

Data Gathering

and Processes

Writing the Research Report

Presentation and Analysis

of Data

Finding and Conclusions

Publication and Dissemination of

the Evaluation

Plan Update

Project Recycling, Re-modification

or Removal?

Embark on ‘new’ Project

Evaluation

Evaluation Study Implementation by Miclat, 2005

Stages of Project Life (Bitonio, 2014)

Project Initiation

Project Planning

Project Execution

Project Closure

Communication

What are your answers?

• What went right?

• What went wrong?

• How do you make the next project process better?

• Best Practices?

At the end of the Project

Project Initiation

Project Planning

Project Execution

Project Closure

Post Implement

Review

Monitor and Control Plan the

Project

Define the Project

Communication

• Client Satisfaction input/report

• Formal acceptance

• Project Schedule (Actual vs.

Estimated)

• Lessons Learned

Project Life with Recycling Cycle (Sison, 2014)

Communication

Publication and Dissemination

of the Evaluation

Plan Update

Project Recycling, Re-

modification or Removal?

Embark on another Project

with ‘new’ criteria for

Project Evaluation

Project Initiation

Project Planning

Project Execution

Project Closure

Post Implement

Review

Monitor and Control

Plan the Project

Define the Project

Closing the Project

Closing the Project

• Closing projects involves gaining stakeholder and customer acceptance of the final products and services, and bringing the project to an orderly end.

• Closing includes verifying that all of the deliverables are complete, planning for transition of products and services into the organization, and preparing a final presentation and report.

• It is also important to reflect on what can be learned to improve future projects. As philosopher George Santayana said, “Those who cannot remember the past are condemned to repeat it.”

Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures

Closing a project involves

creating:

– Administrative closure procedures

– Contract closure procedures

– Final products, services, or results

– Updates to organizational process

assets

Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures

• In closing out a project, you must prepare:

– A customer acceptance/project completion

form

– A final report and presentation

– A transition plan (provided as part of the final

report)

– A lessons-learned report (after a “sticky note”

party)

• Organizing an event/celebration/luncheon, etc.

for the project team to celebrate a job well done.

Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures

Sample Table of Contents for a Final Project Report

Sample Customer Acceptance

/Project Completion Form

Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures

Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures

Sample of Lessons Learned Report

Lessons Learned Report Cont.

Source: sbuweb.tcu.edu/mackay/INSC%2040813/Lectures

References:

BOOK Miclat, Jr. Eusebio F. Strategic Planning in Education: Making Change Happen. Rex Book Store, Inc. 2005. WEB Rengasamy, Srinivasan. There is No Management without Monitoring. slideshare http://www.slideshare.net/srengasamy/project-monitoring-evaluation-s-presentation?utm_source=slideshow&utm_medium=ssemail&utm_campaign=download_notification Posted Oct. 11, 2008 Accessed March 10, 2014 Bitonio, Josefina B. Project Development and Management 2014 ________________. Project Recycling. Chapter7.doc syllabus clsu Zarinpoush, Fataneh Project Evaluation Guide for Nonprofit Organizations. Imagine Canada. 2006 ________________________. Toolkit to Combat Trafficking in Persons. United Nations Evaluation Group. 08-58296_tool_10-1.pdf

buweb.tcu.edu/mackay/INSC%2040813/Lectures

#ProjEvalReport

to evaluate!