Session VI

Evaluation & Wrap Up

Evaluation

Activities directed at collecting, analyzing, interpreting, and communicating information about the effectiveness of social programs undertaken for the purpose of improving social conditions.

Evaluation: Uses Assess the effectiveness of an

ongoing program in achieving its objectives

Distinguishes a program's effects from those of other forces

Aims at program improvement through a modification of current operations.

Evaluation Types Formative Evaluation

Strengthens or improves the object (program) by examining

the delivery of the program or technology,

the quality of its implementation the assessment of the organizational

context, personnel procedures and inputs.

Evaluation Types

Summative Evaluation Examines the effects or outcomes of

programs, Determines the overall impact.

Evaluation Types

Static Evaluation of a completed program is

conducted without the involvement of the target group.

Evaluation Types

Dynamic Interaction between the target group

and the program. Trochim (1996) suggests an

evaluation that occurs throughout the entire "life cycle" of the program.

Evaluation Program Domains

Needs Assessment Assessment of program theory Assessment of program process Impact Assessment Efficiency Assessment

Evaluation Standards

Utility Clear identification of audiences Writing clear and understandable

reports Timeliness Statement of evaluators’ biases and

qualifications Taking responsibility for the use of the

evaluation.

Evaluation Standards

Feasibility evaluation should be

Realistic Prudent Diplomatic Frugal

Evaluation Standards

Propriety an evaluation should be

conducted legally ethically with due regard for the welfare of those

involved in and effected by an evaluation.

Evaluation Standards

Accuracy Technical adequacy of evaluative

information validity reliability data control Conclusion drawing objectivity

Evaluation Structure The question the evaluation is to

answer. The methods and procedures the

evaluation will use to answer the questions.

The nature of the evaluator- stakeholder relationship.

Purpose

Evaluations may be preformed to determine information on various aspects of the program such as determining effectiveness, providing supporting information for

advocates, providing information for funding

decisions, etc.

Stakeholder Needs The evaluator must determine what

aspect of the program is to be assessed and successfully negotiate the contract with the stakeholders in order to present an effective, timely and useful report.

Circumstances Programs and evaluations are

generally not standardized to the point where a formula for evaluation can be applied. Some considerations vital to a successful evaluation are the stage of program development, the administrative and political context the conceptualization and

organizational structure

Resources

In order to perform an evaluation that meets expectations, the evaluation must have access to the appropriate resources. Adequate time, funding, personnel, and access to records and data are vital to the development of an effective and useful evaluation.

Challenges

Determining the type of evaluation to be preformed.

Determining the issues surrounding the evaluation.

Negotiating stakeholders’ expectations of the finished product.

Challenges Allocating necessary funding, staff,

access to data, and time constraints. Upholding utility, feasibility,

accuracy and propriety standards to ensure a useful report.

Maintaining an appropriate balance between considerations if the circumstances of the evaluation change.

Resources for Evaluation Design NC Cooperative Extension

Extension Tourism County Extension Directors

NCDENR-State Trails Program Staff County Tourism Directors Local Government Officials Non Profit Organizations Others

Website Usage

Summary by Month

MonthDaily Avg Monthly Totals

Hits Files Pages Visits Sites KBytes Visits Pages Files Hits

Jun 2006 56 38 19 7 120 193486 181 460 931 1356

May 2006 99 73 36 9 185 448063 289 1122 2283 3077

Apr 2006 98 68 35 9 168 405556 278 1077 2062 2960

Mar 2006 164 114 60 10 271 410130 337 1862 3552 5103

Feb 2006 439 247 184 11 219 440637 328 5154 6925 12319

Jan 2006 184 122 64 11 261 423587 331 1923 3661 5529

Dec 2005 27 21 12 3 3 5546 7 25 43 55

Totals232700

51751 11623 19457 30399

References http://www.cals.ncsu.edu/agexed/aee577/inde

x.html http://

www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_e.htm

http://hagar.up.ac.za/catts/ole/eel/concept-map/types.html#e

Speaker Contact Information

Jan Traskjan_trask@ncsu.eduParks, Recreation & Tourism

ManagementNorth Carolina State University