Program Evaluation

DR. MAJED WADI

Objectives

Design necessary parameters used for program evaluation

Accept different views of program evaluation

Realize the effect of program evaluation parameters on his/her peers

Program evaluation A ‘‘program’’, in medical education, it could be

class session, a course, or a clerkship rotation in medical school or the whole of an educational program.

Evaluation is defined in the Collins English Dictionary (1994) as ‘‘the act of judgment of the worth of . . .’’.

Program evaluation is the ‘‘systematic collection and analysis of information related to the design, implementation, and outcomes of a program, for the purpose of monitoring and improving the quality and effectiveness of the program.’’

Reasons of program evaluation

To determine the effectiveness of programmes for participants

To document that programme objectives have been met

To provide information about service delivery that will be useful to programme staff and other audiences

To enable programme staff to make changes that improve programme effectiveness

Models of program evaluation

Kirkpatrick’s modelLogic modelCIPP Model

Kirkpatrik’s model

It focuses on learner outcomes

1. learner satisfaction or reaction to the program;

2. measures of learning attributed to the program (e.g. knowledge gained, skills improved, attitudes changed);

3. changes in learner behavior in the context for which they are being trained; and

4. the program’s final results in its larger context.

Logic model

CIPP Model

Designing program evaluation To design and conduct an evaluation plan one should

consider the Purpose of evaluation Audiences Negotiation and client involvement Key methods(standards) Ethical consideration Data management Time line and resources implications Reporting

Methods of data collection

Questionnaires / checklists Interviews Documentation review Observation Focus groups Case studies

Kirkpatrik’s model Level 1: learner satisfaction or reaction to the program

Methods: questionnaires, interview Level 2: measures of learning attributed to the program

Methods: analysis of exams Level 3: changes in learner behavior in the context for which

they are being trainedMethods: work-based observation

Level 4: the program’s final results in its larger contextMethods: evaluation of patients outcomes and improved

health care