Embed Size (px)
Citation preview
DEVELOPING ASSESSMENT INSTRUMENTS
Presented by Larry F. CobbApril 29, 2014
Background
In this presentation, we will discuss the elements associated with how designers construct various types of assessments instruments.
Objectives
Determine how to distinguish the different types of criterion-referenced tests.
Determine what to consider when deciding the best format for the assessment instrument.
Four Types of Criterion-Referenced Tests
Entry Skills Test – given before instruction Pretests – administered before learners
begin instruction for efficiency Practice Tests – active leaner participation
during instruction Posttests – administered following
instruction
Test Design
Matching the learning domain with an item or assessment task type is a primary consideration.
There are a lot of different kinds which makes careful analysis very important.
Mastery Levels
A prespecified level of task performance, with no gradations below it, that defines satisfactory achievement of an objective
An example of mastery level is may be defined as the level required to be successful on a job.
Test Item Criteria
There are four categories of test item qualities that should be considered during the creation of test items and assessment tasks. Goal-Centered Criteria Learner-Centered Criteria Context-Centered Criteria Assessment-Centered Criteria
Goal-Centered Criteria
Focused on the content of the instruction. Specific criteria in this area include:
congruence between content in materials and objectives
adequacy of content coverage and completeness
authority accuracy currency objectivity
Learner-Centered Criteria
Focused on the appropriateness of instructional materials for the target group. The learner analysis documentation should provide the foundation for this evaluation. Specific criteria in this area include the appropriateness of the materials for the learners with regards to their:
vocabulary and language levels developmental, motivation, and interest levels backgrounds and experiences special language or other needs
Context-Centered Criteria
Focused on the adequacy of existing materials (do they need to be adapted or enhanced prior to use?). Materials can be evaluated to determine whether the following items are included and adequate/complete:
pre-instructional materials content sequencing and presentation student participation and congruent practice exercises feedback assessments follow-through directions for enhancing memory and
transfer delivery system and media formats learning guidance to move students from one
component/activity to the next..
Assessment-Centered Criteria
Focused on the appropriateness of existing materials for the instructional and performance context. Judge if existing material can be adopted; if not, you are in instructional materials development business. Criteria in this area include:
the authenticity of the materials for context and learners the feasibility of the materials for settings and budget. Here
examine the technical quality of existing materials with regards to: packaging graphic design and typography durability legibility audio and video quality interface design navigation functionality
Mastery Criteria
It is very important to determine right amount of items to achieve mastery.
There are many variables depending on the objectives and area by which we are determining mastery. An example would be golf and mastery being determined based on the golf course.
Test Item Format and Performance Objectives
The behavior specified in the objective provides clues to the type of item or task that can be used to test the performance.
Certain types of performance can be tested in different ways, and some test item formats can assess specified performance better than others.
Objective Tests
Include test items that are easy for learners to complete and designers to score.
The answers are short, scored as correct or incorrect, and judging correctness of an answer is straightforward.
Objective Tests
Writing Objective Test Items – Designers can use the 4 main criteria in developing effective objective test items.
Sequencing Items – There is no guide in place for order placement of items.
Writing Directions – Tests should include clear, concise directions
Alternative Assessment Instruments for Performances,
Products, and Attitudes Writing Directions - learners for
performance and products should clearly describe what is to be done and how
Developing the Instrument Identify the elements to be evaluated. Paraphrase each element. Sequence the elements on the instrument Select the type of judgment to made by the
evaluator Determine how the instrument will be scored.
Alternative Assessment Instruments for Performances, Products, and Attitudes (con’t) Identify, Paraphrase, and Sequence
Elements – Elements to be judged are taken directly from the behaviors included in the performance objectives. Each element should be paraphrased for
inclusion on the instrument. Developing the Response Format –
Determine how the evaluator will make and record the judgments
Portfolio Assessments
Portfolio assessment is defined as the process of meta-evaluating the collection of work samples for observable change or development.
Congruence in the Design Process
All must be considered during this part of the process Materials for Evaluating the Design Procedures for Evaluating the Design
Lifelong Learner Summary
As a lifelong learner, it is important to continue to evaluate was and use criterion-referenced tests to determine and help attain objects that are set by different standards, especially when attempting to reach levels to mastery.
