Developing your Evaluation Toolkit

DEVELOPING YOUR EVALUATION TOOLKIT

Focus on Learning 2013

Paula Buskard and Sandy Odrowski

AGENDA

Session Outcomes

Overview of what/why we evaluate

Aligning evaluation methods to outcomes

Evaluation Planning

After the Evaluation: Debriefing

Q & A

SESSION OUTCOMES

By the end of this session the participant will have the ability to:

Select suitable evaluation tools that are learner–centred and align to learning outcomes

Identify best practice assessment tools that prepare the instructor for writing great evaluations.

Identify ways to prepare the learner for evaluation.

Discuss ways to conduct effective post test debriefing for both learner and instructor.

PLACE MAT ACTIVITY

Each member will takes a square/triangle and individually writes 3 questions/concerns they have about evaluation.

RECORD THE GROUPS TOP 3 CONCERNS IN THE CENTRE

EVALUATION –WHAT IS IT??

FORMATIVE

diagnostic tool (Where are learners at? What have they got so far? Do I need to go over something again?)

help learners take stock of and assess their own progress towards learning goals

SUMMATIVE:

assess a learners achievement of meeting the learning outcomes

determine grades

AN ANALOGY OF LEARNING...9

Formative Summative

SOMETHING TO THINK ABOUT

What is Needed to know (Learning Outcome)

What is Nice to know(Filler)

WHAT TO EVALUATE?

The Course Learning Outcomes describe clearly what

learners will know and be able to do at the end of the course.

They are performance-based and

results oriented.

THE GOLDEN CHAIN

SNOWBALL EXERCISE

Each group move to a posted Learning Outcome

Take 2 minutes (I will call time) to identify evaluation methods that could be used to measure learning.

When time is called, move to the next Learning Outcome and repeat.

Once each group has had the opportunity to identify evaluation methods for each Learning Outcome let us know.

Share in large group.

WHAT DID YOU LEARN TODAY

Working with a partner, fill in the Handout labeled Evaluation Strategy Matching Exercise.

EVALUATION METHODSADVANTAGES AND CONCERNS

In your group:

Take a moment to discuss the Advantages and Concerns regarding the evaluation method listed on the flip chart paper at your table.

Record your findings and be prepared to discuss with the larger group

EVALUATION PLANNING

THE GOODS

“Will this be on the test?”

“What do I really have to know?”

“How do I answer this question?”

“Why did I get this question wrong?”

What do you want your learners to know?

FOUR POINTS TO CONSIDER:

1 – Measure what is most important rather than what is easiest to measure

2 - Be aware of the learning embedded in evaluation

3 – Understand the evaluator’s power

4 – Take care not to reduce the curriculum to only what is evaluated

GUIDELINES FOR PLANNING

Start early (holidays, sequence, scaffolding)

Determine the number of items and time limitations

Construct the question- make sure that it measures learning and is linked to content objectives/outcomes

Edit the questions for errors in spelling, sentence structure, etc.

GUIDELINES FOR PLANNING CONT’D…

Group the questions according to type and topic (make easier ones first)

Write directions (include marking scheme)

Have the answers “ready” for post test debrief

Review

HOW TO WRITE A GOOD EVALUATION

Learner-Centered and Authentic Assessment Checklist

Developing Good Test Questions

PREPARING YOUR LEARNERS

THE GOLDEN CHAIN?

Develop a business plan for a small non profit organization.

Lecture and homework: practice multiple choice questions from the text

Evaluation: Final Exam: multiple choice.

Course Learning Outcome

Learning and practice methods

Evaluation Method

THE GOLDEN CHAIN?

Develop a business plan for a small non profit organization.

Lecture: types of business and parts of a business plan/. Case studies

Student develop a business plan for a mock or real non profit organization

Course Learning Outcome

Learning and practice methods

Evaluation Method

DEBRIEFING AND FEEDBACK

TEACHER DEBRIEF

Exercise: In your small groups answer these 2 questions:

What could teachers consider after a test?

What could teachers do to help their learners after a test?

TEACHER DEBRIEF: CLEAN UP

Did my learners understand what was asked on the test?

Were there questions that the learners missed that were the fault of the teaching? How will I remedy this?

Do the results indicate that the learners were prepared for the evaluation?

TEACHER DEBRIEF: CLEAN UP CONT’D

• If I use this procedure again, what changes will I make?

• What other methods of evaluation might be more appropriate?

• Is this an evaluation that must be returned?

• Is there evidence of academic dishonesty? If so, what can I do about it (this time and in the future?)

LEARNERS DEBRIEF

Quickly go over the test and provide the correct answers to all of the questions.

Discuss the outstanding questions – those that most students answered well, those that some may have got stuck on, etc.

Open the floor for focused discussions (learning opportunity).

After a reasonable time is spent on a class discussion, invite students to make appointments with you to discuss individual concerns.

24 hour rule

EVALUATION WORKS BEST WHEN…

Provide clear, objective assignment guidelines

Provide samples, if possible, of desired product

Provide a marking rubric showing where the marks will come from

Tie evaluation strategy directly to the learning outcome being measured Domain/level

ANY QUESTIONS/CONCERNS LEFT?

ANY OTHER QUESTIONS?

DO YOU WANT TO LEARN MORE?

Durham College Assessment Using Technology

Eastern Region ABC program Excellent Evaluation Resources

Developing Good Test Questions

DO YOU WANT TO LEARN MORE?

Andrus, C and Bloor, D (2005) Teaching tools

Gross-Davis, B. (2001). Tools for teaching. San Francisco: Jossey-Bass

Fenwick T. and Parsons, J. (2000). The art of evaluation: A handbook for educators and trainers. Toronto: Thompson Educational Publishing

http://www.naacls.org/docs/announcement/writing-objectives.pdf

http://rubistar.4teachers.org/index.php

http://pareonline.net/getvn.asp?v=7&n=3