ID Project Final 12-10-2010 Evaluation

EDTECH 503 Instructional Design Project Final

Copyright © 2010, Dianne R. Johnson‐Wojnicki Page 34

Part 6. Formative Evaluation Plan

Part 6a. Expert Review

I discovered through Forum Posts that one of my Instructional Design classmates, Tammy Price has taught High School Floriculture for eight years as well as having worked in the Floral Industry. I approached Tammy early in the design project to see if she would be interested in participating as my SME and she agreed. On December 2, 2010 I forwarded all the pertinent information to her as well as an Expert Review Survey. Her review and critique is expected by December 8, 2010.

Part 6b. OnetoOne Evaluation

In order to conduct the One‐to‐One Evaluation I would ask for student council girl volunteers from the 5th through 8th grades. This will help to ensure I get a sampling which represents learners of average, learners slightly above average, and learners slightly below average skills, (Smith & Ragan, 2005). For this evaluation I would select two or three of the most articulate and thoughtful volunteers. I would give a copy of the written material, Google Docs presentation, declarative knowledge assessment, video presentations, and materials to perform the learning tasks. Since the point of this activity is to find misspellings, printing errors, ambiguous directions and the like; I would ask them if they could take time to not only work their way through the material to accomplish the task but also to make notes in their copy as to typographic errors or sentences and concepts they felt unclear or ill‐presented. At the end of their evaluation I would ask them if they felt there was sufficient content for them to accomplish the learning task. I would also inquire about their individual overall satisfaction and sense of accomplishment.

Part 6c. Small Group Evaluation

After the One‐to‐One Evaluation revisions have been applied I would plan to have a local 6th grade Girl Scout troupe participate using the material to accomplish the Small Group Evaluation. Girl Scout troupes are always looking for service project to create and bestow upon the community. Not to mention they typically have a large budget for such endeavors. Since this phase of evaluation is to determine how well the material works when presented without designer involvement and whether or not it will accomplish its goal with a more diverse set of participants; the learning materials would be given to the troupe leader and she would conduct the class. As the designer I would participate only as an observer. During this time I would take notes as to;

1. The level of acceptance received by the material. 2. How well the learners grasped the overall concepts. 3. How well the learners were able to perform the tasks given the information presented. 4. If the learners had sufficient time to accomplish individual goals as well as the overall goal.



At a subsequent Girl Scout meeting I would supply a written questionnaire, very similar to the Special Occasion Boutonniere and Corsage Creation Rubric, to determine specific deficiencies perceived on an individual basis. The information gathered through my observations during the Girl Scout class as well as the questionnaire would be applied to the learning material before the Field Trial.

Part 6d. Field trial

The only way I could envision to successfully conduct Field Trials of the magnitude described in the text, i.e. instructional material should be tried with at least thirty students at this stage (Smith & Ragan, 2005), would be to offer the learning task materials to one of the other schools within the area. A school with a small or non‐existent art program would be good candidate. In the event one of the schools would consider the presentation of the Instruction I would utilize the learner base to establish the significance of the revisions made in the previous evaluation phases, determine if the material is sufficient for a large audience, as well as if there are any gaps in the overall organization of the learning task. Since, I as the designer would not be present during the presentation of the Field Trial I would need to organize a communication and feedback conduit with the administration at the participating school. I would prepare a list of questions for the facilitator/instructor as well as questions for the learners, very similar to the Special Occasion Boutonniere and Corsage Creation Rubric. It would be through this conduit that I would gather the information necessary to make revisions based on this final phase of evaluation. Some additional information I would want to gather is as follows:

1. Where there any administrative or technical problems with the material or format. 2. Do the learners have the expected entry level skills? 3. Was the information presented in the Google Docs Presentation: “Are You Ready to Prepare

and Assemble Flowers?” enough to fill the gaps to acquire the next step. 4. Was the assessment for the presented in the Google Docs Presentation: “Are You Ready to

Prepare and Assemble Flowers?” enough determine acquisition of knowledge? 5. Was the information presented in the Ubloom Video Library Presentation from the Floral

Design Institute featuring Leanne Kessler entitled “Basic Wiring and Taping Techniques, Mastering an Essential Florist Skill”, enough to successfully perform the expected tasks.

6. Was the information presented in the Ubloom Video Library Presentation from the Floral Design Institute featuring Leanne Kessler entitled “Boutonniere Basics”, enough to successfully perform the expected tasks?

7. Was the information presented in the Ubloom Video Library Presentation from the Floral Design Institute featuring Leanne Kessler entitled “The Easiest Wrist Corsage Ever”, enough to successfully perform the expected tasks?

8. If the learners had sufficient time to accomplish individual goals as well as the overall goal.



Part 7. Formative Evaluation Report

Part 7a. Evaluation Survey or Rubric

Boutonniere and Corsage Making Project Expert Review Survey

The objective of this project is to teach a group of 6th grade girls how to design, prepare and assemble a boutonniere and corsage for a special occasion. These sessions are highly procedural in nature and demand a high degree of psychomotor skill.

Project Goal

After participating in two one‐and‐one‐half hour after school sessions the learners will be able to use the appropriate flowers, tools and materials to prepare, design, and assemble a boutonniere and corsage for a Special Occasion.

Please evaluate the instructional materials provided which were developed to support this leaning task.

1. Instructor Guide – a template to guide the instructor/facilitator.

2. Google Docs presentation; “Are You Ready to Prepare and Assemble Flowers” ‐ created specifically for these sessions to cover the declarative knowledge gaps that may exist to help ensure a successful learning experience. The flowers, flower parts, filler and foliage covered are domain specific to these sessions and these learners; Part 5a ‐ Learning Materials.

3. Ubloom Video Library Presentation from the Floral Design Institute featuring Leanne Kessler entitled “Basic Wiring and Taping Techniques, Mastering an Essential Florist Skill” ‐ presenting the essentials of flower preparation using an expert’s concise and effective approach; Part 5a ‐ Learning Materials.

4. Ubloom Video Library Presentation from the Floral Design Institute featuring Leanne Kessler entitled “Boutonniere Basics” ‐ presenting the preparation and assembly of a boutonniere using an expert’s concise and effective approach; Part 5a ‐ Learning Materials.

5.. Ubloom Video Library Presentation from the Floral Design Institute featuring Leanne Kessler entitled “The Easiest Wrist Corsage Ever” ‐ presenting the preparation and assembly of a simple wrist corsage using an expert’s concise and effective approach; Part 5a ‐ Learning Materials.

6. Formative and/or Summative Assessment materials Matrix ‐ used determine the acquisition of the procedural and psychomotor skill; Part 5b ‐ Formative and/or Summative Assessment materials.

7. Critical Flower Parts, Foliage and Filler Post‐Presentation Assessment; Appendix B – Critical Flower Parts, Foliage and Filler Post‐Presentation Assessment

8. Session Rubric ‐ used for overall session evaluation to determine if the objectives where met, indicate improvements required and the level of satisfaction of the learners; Appendix C – Session Rubric



Thank you for taking the time to provide this critical feedback!

A copy of the live form can be found in Appendix D ‐ Expert Survey. The live form can be accessed using the following link http://bit.ly/ExpertSurvey.

Part 7b. Report the results of the expert review.

The following is the list of questions posed to the expert as well as her responses.

Q. Does the task seem achievable in the estimated time given the age and prior knowledge level of the perspective learners?

A. Mostly achievable

Q. What component, if any could be compressed or modified to accomplish the goal?

A. Students might need additional practice time to perfect the method of using Floral Tape. They could practice on a plain wire (#20 gauge or heavier) a few times first before attempting to Floral Tape the flowers. The age of the learners and their dexterity level will help determine this.

Q. Does the amount of Declarative Knowledge covered in the Google Docs presentation; “Are You Ready to Prepare and Assemble Flowers” of the class seems sufficient given the prior knowledge expressed in the Needs Analysis Survey?

A. Yes, covered

Q. What, if any additional Declarative Knowledge elements need to be covered to prepare the learners for the task?

A. You covered everything the learners need to know to begin their project. No additional Declarative Knowledge is needed.

Q. Does the Declarative Knowledge material presented appear to be accurate and complete?

A. Yes, accurate and complete

Q. Is the material presented in the most effective order for the learners to successfully accomplish the task?

A. Yes, effective

Q. What order would be more effective and efficient for success?

A. Definitely correct order. You have the steps listed in the best order I believe. Every step builds on the next.

http://bit.ly/ExpertSurvey



Q. Is the specific material covered in the Ubloom Video Library presentations; “Basic Wiring and Taping Techniques”, “Boutonniere Basics”, “The Easiest Wrist Corsage Ever” sufficient for the learner’s to accomplish the learning goal?

A. Mostly, sufficient

Q. What, if any addition information would be necessary?

A. Just make sure there is a demonstration component for the instructor to complete each project/step live and in front of the learners. The instructor needs to be confident in using the steps and completing the project.

Q. Is the creation of the prototype enough hands on experience to enable the learner’s to successfully prepare the boutonniere?

A. Almost enough

Part 7c. Comments on Change

Based on the critical feedback provided by the Subject Matter Expert I feel relatively confident that the learning task is achievable. The expert’s comments indicate that the content is accurate and complete to accomplish the task.

In terms of changes, it appears that an additional step should be added to the first session. Taking into account the age of the learners and their dexterity level the expert believes that some additional practice on the taping technique would be appropriate. I would probably add a step to have the learners perform some a taping activity using a plain wire (#20 gauge or heavier) before they actually wire and tape the carnation to create the prototype.

The other change I would make is to the instructor guide to add specific instructions that the instructor must perform each step of the process prior to and in concert with the learners. This will help the instructor’s sense of confidence as well putting the learners at ease and reassuring them of the task’s achievability.

As I have learned during my tenure in Instructional Design (EDTECH 503) and as stated rather eloquently in the Instructional Design Model of G. Morrison, S. Ross and J. Kemp; ID is an uninterrupted series with revisions constantly taking place (Morrison, Ross, & Kemp, 2007). I am quite confident that each presentation of this learning task will foster innovations and modifications to the fundamental core of the material.

Documents

ID Project Final 12-10-2010 Evaluation