MED5208 Monitoring Tools and Techniques

8/9/2019 MED5208 Monitoring Tools and Techniques

1/13

MED5208 Monitoring Tools and Techniques

Course objectives

The general objective of this course is to enable the learners to understand, appreciate and apply

various monitoring tools and techniques in project managementAt the end of this course the learners should be able to

Understand and appreciate monitoring as a component of project/program management

cycle

Understand, appreciate and apply different analytical tools in monitoring projects

Discuss and recommend the appropriate monitoring tools and techniques for various

activities

Construct and analyze project diagrams e.g. networks, charts, tables etc

Understand and appreciate ethical issues in the monitoring and evaluation

Answer why, when, what, who, how and resources of monitoring

Course contents

1. Project monitoring fundamentals

Definition of monitoring

Purpose and importance of monitoring project performance

Planning and developing monitoring systems

Developing a monitoring team

Indicators and generally monitored indicators in projects

Process and criteria for monitoring

2. Project monitoring techniques (planning phase)

Log frame(logical framework)

Work breakdown structures(WBS)

Uses of log frames and WBS

Preparation/development of log frames and WBS

3. Project network

Definition of network as a tool of monitoring projects

project activity, event and dummy


2/13

Activity -on-arrow and activityon-node approach for CPM analysis

Developing and analyzing project network

4. Critical path method

Definition of CPM the concept and rationale behind CPM

Use of forward and backward pass

Estimating activities duration and causes of variation in activity duration

Project time reduction procedure(crashing)

Rationale for reducing project time

Calculating slacks/floats

5. Program evaluation and review technique (PERT)

Concept and rationale behind PERT

The three estimation times involved in PERT; Optimistic Time Estimate (TOPT), Most

Likely Time Estimate (TLIKELY), and Pessimistic Time Estimate (TPESS).

The PERT Mathematics(calculations)

6. The Gantt chart

Definition of Gantt chart as a tool of monitoring projects

Developing and applying Gantt chart in monitoring project

Advantage and disadvantages of Gantt chart in project monitoring process

7. Organization responsibility chart techniques (ORCT)

Understanding ORCT as a technique of monitoring projects

Developing ORCT

Using ORCT in assigning responsibilities

Challenges in assigning and sequencing

8. Estimating resource and costs requirement

Matching resources with the activity(resources loading vs. activity duration

Types of resources

Elapsed time, labor time and production time

Budgeting and budget preparation

Monte Carlo analysis


3/13

9. Data collection tools and techniques in project monitoring process

Definition, developing and applying various data collection tools such as:-

Observation, Questionnaire and Interview

Survey, Sampling, Case study and Participation

10. Data analysis tools and techniques in project monitoring process

Understand and appreciate quantitative data analysis tool and techniques such as:-

Frequency distribution(histogram in excel)

Descriptive statistics

Comparing meansstatistical testing

Cross tabulations

Correlations

Linear regressions

Text analytic

Understand and appreciate qualitative data analysis tools and techniques such as:-

Typology, Taxonomy, Constant Comparison /Grounded Theory, Analytic Induction,

Logical Analysis /matrix analysis, Quasi-statistics Event Analysis/Microanalysis,

Metaphorical Analysis, Content Analysis, Domain Analysis etc

NB: - These are guideline only and the learners are advised to read widely!!!


4/13

Chapter one

Project monitoring fundamentals

What is monitoring

Monitoring is a periodically recurring task already beginning in the planning stage of a project or

program. Monitoring allows results, processes and experiences to be documented and used as a

basis to steer decision-making and learning processes. Monitoring is checking progress against

plans. The data acquired through monitoring is used for evaluation. It is the systematic and

routine collection of information from projects and programs for four main purposes:-

To learn from experiences to improve practices and activities in the future;

To have internal and external accountability of the resources used and the results

obtained;

To take informed decisions on the future of the initiative;

To promote empowerment of beneficiaries of the initiative.

According toPhil Bartle (2007),monitoring is very important in project planning and

implementation because it provides information that will be useful in:-

Analyzing the situation in the community and its project;

Determining whether the inputs in the project are well utilized;

Identifying problems facing the community or project and finding solutions;

Ensuring all activities are carried out properly by the right people and in time;

Using lessons from one project experience on to another; and

Determining whether the way the project was planned is the most appropriate way of

solving the problem at hand.

Phil Bartle (2007), definesmonitoring as the regular observation and recording of activitiestaking place in a project or program. It is a process of routinely gathering information on all

aspects of the project. To monitor is to check on how project activities are progressing. It is

observation; systematic and purposeful observation. Monitoring also involves giving feedback

about the progress of the project to the donors, implementers and beneficiaries of the project.


5/13

Reporting enables the gathered information to be used in making decisions for improving project

performance.

Developing a monitoring plan

The type quaintly and quality of information should be thought of carefully before planning the

monitoring system. The project manager usually prepare annual work plan that translate the

project document into real concrete tasks. The work plans should describe in details the delivery

of inputs, the activities to be conducted and the expected results. They should clearly indicate

schedules and the persons responsible for providing the inputs and producing results. The work

plan should be used as the basis for monitoring the progress of program/project implementation.

As a management tool, monitoring should be organized at each level of management. Monitoring

systems should be linked to manual plans. A first step in designing a monitoring plan is to

identify who needs what information for what purpose how frequently and in what form. To

develop an effect monitoring system, the following steps might be followed-:

Decide what should be monitored. The careful selection of monitoring indictors organizes

and focuses the data collection process.

Select methods to track indicators and report on progress (observation, interviews,

stakeholder meetings, routine reporting, field visits, etc(how to gather information)

The monitoring plan should include who will gather the information and how often.

Project staff at various levels will do most data collection, analysis and reporting. Staff

should agree on what the monitoring report should include. (When to gather information

by whom)

Progress reports should be reviewed by project staff and major stakeholders. Feedback

should be collected by project managers on a regular basis.

The monitoring plan should indicate the resources needed to carry out project monitoring.

Needed funds and staff time should be allocated to ensure effective implementation.

Drawing up a monitoring plan will help the organization to think about how it will carry out

monitoring over the year or over the lifetime of a project.

It will consider:


6/13

When will different information be collected?

Who will carry out and manage the activities?

How will information be collected, analyzed and presented?

How will monitoring and evaluation findings be used in short-term planning and

decision making, and quality reviews?

You will then be able to prioritize the information you need to collect regularly and on an

occasional basis by being clear about the key questions, such as:

How are resources being used?

How appropriate is the management structure?

How well are we meeting identified needs?

How do we fit within a network of services?

How well have we met our expected outcomes?

What were the unexpected outcomes?

What lessons did we learn?

Organizations may focus on different questions at different points in time, and they may need

different types of information gathering.

The importance of planning

Planning is a key stage in the monitoring and evaluation cycle. Organizations should think about:

Why do we want to do it and for whom?

What will we monitor and evaluate?

How will we do it?

Who will do it and do we have the right skills?

When will we do it?

What resources will we need, including outside support?

What will we do with the information we get?

Organizations themselves, and their stakeholders, will usually want to find out how the

organization or project is meeting its planned aims and objectives. These need to be clearly

stated in the early stages of planning.


7/13

Defining aims and objectives will help you agree on planned outcomes. Many funders and

commissioners will ask for outcome information as part of their monitoring requirements.

Developing a monitoring team

Another important part of the planning stage is picking your monitoring team with the data

collection and analysis skills. The team must be aware of their duties and responsibilities Take

great care, especially if you have team-members imposed on you by the project brief. Selecting

and gaining commitment from the best team members - whether directly employed, freelance,

contractors, suppliers, consultants or other partners - is crucial to the quality of the project, and

the ease with which you are able to manage it. Generally try to establish your team as soon as

possible. Identifying or appointing one or two people even during the terms of reference stage ispossible sometimes. Appointing the team early maximizes their ownership and buy-in to the

project, and maximizes what they can contribute. But be very wary of appointing people before

you are sure how good they are, and not until they have committed themselves to the project

upon terms that are clearly understood and acceptable. Don't imagine that teams need to be full

of paid and official project team members. Some of the most valuable team members are

informal advisors, mentors, helpers, who want nothing other than to be involved and a few words

of thanks. Project management on a tight budget can be a lonely business - get some help from

good people you can trust, whatever the budget.

To plan and manage large complex projects with various parallel and dependent activities you

will need to put together a 'Critical Path Analysis' and a spreadsheet on MS Excel or equivalent.

Critical Path Analysis will show you the order in which tasks must be performed, and the relative

importance of tasks. Some tasks can appear small and insignificant when they might actually be

hugely influential in enabling much bigger activities to proceed or give best results. A Gantt

chart is a useful way of showing blocks of activities over time and at a given cost and for

managing the project and its costs along the way. Various project management software are

available, much of which is useful, but before trying it you should understand and concentrate on

developing the pure project monitoring skills, which are described in this process. The best

software in the world will not help you if you can't do the basic things.


8/13

Merely assigning people to tasks doesnt create a project monitoring team. Any good project

manager knows that a team is a collection of people who are committed to common goals and

who depend on one another to do their jobs. Monitoring teams consist of members who can and

must make a valuable and unique contribution to the project.

As soon as you identify your project monitoring team members, take steps to define and establish

your teams identity as well as its operating practices. Develop the following elements, making

sure your team understands and accepts them:-

Goals:What the team as a whole and members individually hope to accomplish

Roles:Each members areas of specialty, position on the team, authority, and assignments

Processes:The techniques that team members will use to perform their project tasks

Relationships:The attitudes and behaviors of team members toward one another

As soon as people join the team, have them review the approved project plan to reinforce the

projects goals, clarify the work planned, confirm the feasibility of time and resource estimates,

and identify any potential problems. Meet as a group to discuss peoples thoughts and reactions,

after theyve reviewed the plan.

Team members who contributed to the proposal can remind themselves of the projects

background and purpose, their planned roles, and the work to be done. They can also identify

situations and circumstances that may have changed since the proposal was prepared and then

review and reassess project risks and risk-management plans.

New team members can understand the projects background and purpose, find out about their

roles and assignments, raise concerns about time frames and budgets, and identify issues that

may affect the projects success.

Team members commit to your project when they believe their participation can help them

achieve worthwhile professional and personal goals. Help team members develop and buy into a

shared sense of the project goals by doing the following:

Discuss the reasons for the project, its supporters, and the impact of its results

Clarify how the results may benefit your organizations clients.

Emphasize how the results may support your organizations growth and viability.

Explore how the results may impact each team members job.

Encourage people to think about how their participation may help them achieve personal goals,

such as acquiring new skills and knowledge, meeting new people, increasing their visibility in


9/13

the organization, and enhancing their opportunities for job advancement. Obviously, projects

arent only about helping team members achieve personal benefits. However, when team

members can realize personal benefits while performing valued services for the organization, the

members motivation and commitment to project success will be greater.

Indicators and generally monitored indicators in projects

Indicators

The English Language Dictionary describes an indicator as an instrument which gives you

information

In line with this description, indicators come into the picture in a development co-operation

context at the moment that specific information is required. In this context, different definitions

are being used. According to OECD/DAC, an indicator is: A quantitative or qualitative factor or

variable that provides a simple and reliable means to measure achievement, to reflect changes

connected to an intervention, or to help assess the performance of a development actor (DAC

Glossary of Key Terms in Evaluation, May 2002). According to the definition adopted by

USAID, an indicator is:a variable, which purpose it is to measure change in a phenomena or

process

The European Commission describes (planning) indicators as: a description of the projects

objectives in terms of quantity, quality, target group(s), time and place.

An indicator provides evidence that a certain condition exists or certain results have or have not

been achieved (Brizius & Campbell, p.A-15). Indicators enable decision-makers to assess

progress towards the achievement of intended outputs, outcomes, goals, and objectives. As such,

indicators are an integral part of results.

Examples of types of indicators for monitoring and evaluating CFSC initiatives

Input indictor:

Funds covering the planned communication activities

Qualified staff


10/13

Process indicator:

Number of participatory radio program aired

Number of people reached through popular theatre activities

Output indicator:

Percentage of participants by men and women

Exposure to needed information/messages

Expanded public and private dialogue

Outcome/impact indicator:

ICT increasingly used for dialogue and debate

Percentage of men and women who know about voting procedures (DANIDA,

2005)

Some examples of these considerations are:

Availability of data: Some data may be considered privileged information by agencies,

projects, or government officials.

Data may be available only on aggregated levels or already calculated into indicators that maynot be the ideal indicators for your program or activities.

Resources: Ideal indicators might require collecting data to calculate an unknown denominator,

or national data to compare with project area data, or tracking lifetime statistics for an affected

and/or control population, etc.

The cost of collecting appropriate data for ideal indicators is prohibitive.

Human resources and technical skills may be a constraint as well.

Programmatic and external requirements: Indicators may be imposed from above by those

not trained in monitoring and evaluation techniques.

Reporting schedules may not be synchronized (e.g. fiscal vs. reporting year).

Different stakeholders priorities may diverge.

Standardized indicatorsshould be used if available.


11/13

In general, program should stay away from indicators that activities cannot affect, that are

too vague, that do not currently exist and cannot realistically be collected, or that do not

accurately represent the desired outcome. (Gage and Dunn, 2009)

When quantitative indicators of success cannot be identified, qualitative methods offer a

valuable alternative.When it is difficult or not possible to measure benefits or risks in

simple, quantitative terms, it is almost always possible to gather qualitative data, such as

information on the perspectives of health care providers and women who come for services. In

many cases, qualitative indicators provide more relevant information with respect to the success

and effectiveness of the intervention. (Bott, Guedes and Claramunt, 2004).

Information on the perspectives of women and girls (rights-holders) and service providers

(governmental duty-bearers or NGOs) is essential.Evaluation efforts and indicators should

include the perspectives of both clients and providers. Information from women accessing or

attempting to access services is critical for assessing the effectiveness of an intervention.

Select a set of indicators that pertain to the identified objectives for the program. If the

intervention focuses on training providers, select indicators related for example to providers

knowledge, attitudes and practices, ability to provide care, ability to make referrals, or others

depending on the specific objective.

How many indicators are enough?

Some guidelines to follow when selecting indicators:

At least one or two indicators per result (ideally, from different sources)

At least one indicator for every core activity (e.g. training, airing of TV spot)

No more than 8-10 indicators per area of significant program focus

Use a mix of data collection strategies and sources

Process versus result/impact indicators: It is important to remember the difference between

process and results indicators.

Process Indicators are used to monitor the number and types of activities carried out. Examples

include:-

The number and types of services provided

The number of people trained

The number and type of materials produced and disseminated


12/13

The number and percentage of female clients screened

Results Indicatorsare used to evaluate whether or not the activity achieved the intended

objectives or results. Examples include:-

Results indicators can be developed at the output, outcome and impact levels. Output

indicators illustrate the change related directly to the activities undertaken within the program

(e.g. percentage of traditional leaders in community x who completed the training on

international human rights standards related to violence against women and girls whose

knowledge improved.)

Outcome indicatorsrelate to change that is demonstrated as a result of the program

interventions in the medium-to-longer term (e.g. the number of decisions in the informal justice

system of community x related to violence against women that reflect a human rights-based

approach.)

Impact indicatorsmeasure the long-term effect of program interventions (e.g. the prevalence of

violence against women and girls in community x.)

An important issue that needs to be resolved in order to monitor project progress is how to define

success. Commonly, organizations are able to track how many events they have held, and how

many people have participated (outputs), but not how people have changed their attitudes or

behaviors as a result (outcomes), especially over time.

The main indicator of impactshould be a reduction in the prevalenceand incidenceof

violence, but that takes years to achieve and to measure. Monitoring and evaluation frameworks

and plans should incorporate both process and results indicators.

Process and criteria for monitoring

The process of monitoring starts getting information on the background of the project to

determine the scope of what is to be monitored, understanding the purpose of the monitoring

process and what is to be achieved. It also determines the methodology to be used. Precisely,

before developing the tools for monitoring process start by determining the central question that

the process is going to address whereby the specific objectives are selected. After the tools have

been developed, monitoring is conducted and a report written to be disseminated to the people or

organization in need of the report. Finally, you can choose to include the schedule reflecting the

timing, costing and the responsibilities and so on. Monitoring process include identification,


13/13

Documents

MED5208 Monitoring Tools and Techniques