Embed Size (px)
Citation preview
Designing and Conducting Formative Evaluations
LaSheena Nation
Describe the purposes for and various stages of formative evaluation of instructor-developed materials, instructor-selected materials and instructor-presented instruction
Describe the instrument used in a formative evaluation
Develop an appropriate formative evaluation plan and construct instruments for a set of instructional materials or an instructor presentation.
Collect data according to a formative evaluation plan for a given set of instructional materials or instructor presentation.
Objectives
Background
Studies have demonstrated that simply trying materials with a single learner and revising the materials on the basis of that data can make a significant difference in the effectiveness of materials. This step in the instructional design model emphasizes the necessity of gathering data from members of the target population about the use and effectiveness of materials using that information to make the materials more effective.
Formative Evaluation Design
The purpose of the Formative Evaluation Design is to pinpoint specific errors in the materials in order to correct them, the evaluation design – including instruments, procedures and personnel– needs to yield information about the location of and the reasons for any problems
Formative Evaluation
Formative Evaluation is the collection of data and information can be used to improve the effectiveness and efficiency of instruction.
Formative Evaluation Designs
Ask yourself these 5 questions
1. Are the materials appropriate for the type of learning outcome?
2. Do the materials include adequate instruction on the subordinate skills and are these skills sequenced and clustered logically?
3. Are the materials clear and readily understood by the target audience?
4. DO the materials motivate the target audience and are they relevant to the needs and interest of the study?
5. Can the materials be managed efficiently in the manner they are mediated?
Role of the Subject-Matter, Learning, and Learner Specialist in Formative Evaluation
It is important to have instruction reviewed by an outside specialist
A subject matter expert (SME) should be able to comment on the accuracy and currency of instruction.
A specialist in the type of learning outcome involved can critique the instruction to enhance the targeted type of learning
Also, share instruction with a specialist who is familiar with the target audience, this person can look through the instruction with the eyes of the target audience and give an appropriate reaction and provide insight.
One–to–One Evaluations with Learners
The one-to-one is the 1st step of formative evaluation, its purpose is to identify and remove the most obvious errors in the instruction and to obtain initial performance indications and reactions to the content by learners.
The three main criteria and the decision designers will make during the evaluation are as the follows:Clarity: The message or what information is presented
to the target audienceImpact: What is the impact of the instruction on the
individuals learner’s attitude and achievement of the objective goals?
Feasibility: How feasible is the instruction given the available resources (time/ context)?
Criteria of One-to-One
Criteria Examples
The 2nd step in formative evaluation.There are two primary purposes for small group
evaluations.1. Determine the effectiveness of changes made
following the one-to-one evaluation and to identify any remaining learning problems that learners may have.
2. Determine weather learners can use the without interacting with the instructor.
Small Group Evaluation
Be sure to include a sample the includes diversity of subgroups that exist in your
target population:Low, average and high-achieving studentsLearners with various native languagesLearners who are particular with certain
procedures and who are notYounger or inexperienced learners and more
mature learners
Small Group Evaluations
The 3rd step in formative evaluation. This step evaluates the program and products in the setting in which it is intended to be used.
Determines weather the changes in instruction are made after the small group step were effective.
Determines if instruction can be used in the context it was intended for.
Field Trail
This is used to determine if learners can use the newly learned information/skills within the site where the skills are ultimately required.
Did the skills transfer?How are the skills used?How can training be improved?What physical, social, managerial factors enhanced
or prohibited the use of the skills?
Formative Evaluation in the Performance Context
The instructor chooses existing materials be there is seldom tie or resources to conduct all faces of the evaluation. Then after proceed to the field trail.
The instructor that conducts the field trail is able to observe the progress and attitudes of the learners using a set of adopted or adapted materials.
Formative Evaluation of Selected Materials
This is determine whether the instruction is effective and decide how to improve it. This
functions similar to the field trial. The instructor must be concerned with entry skills,
prior knowledge and attitudes of learners. The evaluation of instructional procedures can also
be evaluated during this process.
Formative Evaluation of Instructor- Led Instruction
The information that is associated with the collection of date during the field trail of original instructional materials applies equally to the data collection procedures used in the evaluation of selected materials and instructional procedures.
Data Collection for Selected Materials and Instruction –Led Instruction
The designer should keep several concerns about the formative evaluation context and the learners who participated in the evaluation process when planning and implementing data collection procedures.
Concerns Influencing Formative Evaluation
The data collected in the formative evaluation can help the designer make the least tentative decision about their particular unit of continued instruction.
Problem Solving during Instructional Design
