512 Evaluation Planning

8/2/2019 512 Evaluation Planning

1/8

EVALUATION PLANNING

The management team at KNA would like to ensure that each learner can competently

deliver the customer orientation, create a communication plan and execute that plan at

some point following the WBI. They would also like to know how much time each

learner is spending on the WBI and associated planning assignments. Finally, the

designer has recommended an evaluation of the WBI by the learners. Obtaining

information regarding the course design and appeal is considered important for the

planning of future professional development including revisions of Communication

Strategies. The questions we will seek to answer are:

How effective was the WBI in equipping learners for creating and executing a

communication plan? How competent are learners in delivering the customer

orientation?

On average, how much time are learners spending on the WBI course activities?

How well does the WBI appeal to the audience?

Table 2 includes the criteria that will be considered for evaluation of Communication

Strategies.

Evaluation Criteria Explanation Data Sources

Effectiveness:Achievement of WBI goals

Determine if learners are able to:Create a communication plan

Deliver the customer orientationExecute the communication plan

Assignments, quizzes andfinal communication plan from

WBI

Review of customerorientation (mock or actual)


2/8

( )

Communication Strategies will be evaluated from two primary orientations as described

by Fitzpatrick (2004), objective and expertise. Meeting the objectives is of primary

concern of the stakeholders, therefore, internal and external evaluations will contributeto determining if that took place.

Formative Evaluation

Communication Strategies General Stakeholders

Primary Stakeholders Designer and instructor: Latanya Blakley is responsible for the design,

development and implementation of this WBI. She is also the author of thisdesign document and will oversee the formative and summative evaluationprocesses.

Students: the adult learners, over 90 franchisees are directly impacted and

partially responsible for the successful implementation of this WBI. Someparticipants of this population will directly impact the design decisions because oftheir participation in the formative evaluation process

Secondary Stakeholders Bradley Lucas is the branch manager for all of the franchisees participating in

this WBI. He will be involved in the formative evaluation providing feedback andguidance to the designer. He is also one of the recipients of this design

document.

Minoru Takahashi is the regional manager for all of the franchisees participatingin this WBI. He will review the results of the both the formative and summative


3/8

Evaluation criteria andcategories

Specific questions Methods and Tools

EffectivenessGoals

Is the information accurate?Are the goals and objectivesclear and achievable?Are the goals and contentappropriate for Web delivery?

End-user (surveys)Expert (checklists)Extant data (exercisesand final plan)

Content Is the information completeand accurate, covering thecontent properly?Are objectives, instructionalactivities, and final projectcongruent?Is there a match amongcontent among objectives,

activities, and assessmenttools?Are reference citationsprovided?Do the activities promotelearning?

End-user (surveys)Expert (checklists)Extant data (exercisesand final plan)

Technology Do the technologyapplications function

properly?Were materials easy toaccess by students?Are copyright and intellectual

End-user (surveys)Expert (checklists)

Observations


4/8

Evaluation criteria andcategories


EfficiencyGoals

Are the goals stated clearlyand concisely?Is the purpose stated clearlyand concisely?Is there congruence betweenthe instructional goals andcontent?


Content Is the content informationclearly and conciselypresented?Is it appropriate to thediscipline? Is it timely, up-to-date?


Technology Is access to the instructor or

other learners provided?Is the website structuredappropriately?Do the technologyapplications function easilyand efficiently?

End-user (surveys)

Technical Expert(checklists)Observation

Message Design Is the organization andstructure of the message

coherent?Do the titles and subtitles helporganize the content?Are there asynchronous and


Observation


5/8

Evaluation criteria and

categories


AppealGoals

Are goals relevant tolearners?


Content Is the content interesting?Is the content challenging?Is the content enjoyable?


Technology Are there any typographical,spelling, grammar andpunctuation errors?Is navigation easy?Is navigation intuitive?Are materials easy to accessby students?Are materials easy for the

instructor to modify?

End-user (surveys)Technical Expert(checklists)Observation

Message Design Are the message and themedia pleasing?Is the vocabulary level andtone appropriate for thecontent and audience?Are the screens unclutteredwith good use of white space?

Is the color, typeface,emphasis used appropriatelyand to enhance learning?Do supporting graphics and



6/8

Evaluators, Methods and Ongoing Decisions

Evaluators

Evaluator/designer: Latanya Blakley is the main evaluator. She has taken courses inthe design of WBI and has designed several courses online for a larger population ofthis audience. Through her masters program at Boise State University Ms. Blakley hasalso taken several courses on evaluation methods in theory. She has also worked for

KNA, for over four years, and therefore has knowledge of the operation.

Expert reviewer: Joanna Dickenson is an associate from the IT team, who will reviewthe WBI prototype for technical functionality. She supports one of the companywebsites and has a background in graphic design. She has also been on the staff forclose to 10 years.

End-user reviewer(s): Several learners from the Communication Strategies course will

be selected to participate in the formative evaluation. Each review were will beconsidered a subject matter expert to cause they will have operated the business andutilize the practices addressed in this training. Participants will be chosen based ontheir level of experience, their comfort with technology and whether they can commit tocompleting all of the tasks. The exact number and names of the participants will bedetermined at a later date.Evaluation Methods and Schedule

Formative evaluation data and feedback on the design will be collected and examinedthroughout the design and development process. A small group of learners will reviewthe materials before the prototype is created. Based on the results of this review, a firstround of changes will be applied After the prototype has been created another set of


7/8

Preliminary Planning for Summative Evaluation

Evaluation Criteria Main Questions Data Sources

Effectiveness Did the final project meetcourse objectives?

Do learners believe the

course was worthattending?

Performance activities

Surveys

Efficiency How long did learnersparticipate in the WBI persession or activity?

Learner login and logouttimes

Surveys

Appeal Did learners enjoy thecourse?

Are learners willing to takeadditional WBI courses forprofessional development?

Surveys


8/8

Communication Strategies Timeline

Task Person(s)Responsible

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8

Conduct pre-planningactivities

Tanya JADKJFKA

Write objectives Tanya I andDDFAAFAA

Write assessment itemsand tools

Tanya DFAAD-------

Review objectives andassessment forformative evaluation

Tanya, Bradley ------------------

Cluster objectives, CreateWBI strategy worksheet

Tanya JKJKJKJKJKL

Evaluate the WBIstrategies Tanya, Bradley,End-users JKJKJKJKJKJK

Identify potential mediaEvaluate media choices

Tanya, Joanne FASFAFAFAFAFDFDAFD

Flowchart and storyboardReview flowchart andstoryboard

Tanya, Bradley DDFADFAFDAFAFAFDAFDAFAFAAF

Convert prototypes to WBIReview website and WBI

Tanya, Joanne,End-users

KKKKJKJKJKJ KJKLJLJJKJKJLK;JLJK;JKLJKJKJLKJJ

Test and troubleshoot WBI

and site

Tanya, Joanne JKJKLJKLJKJKJKJKJKJKJ

Final project due Tanya JJKJKJKJKJ

Items in bold are part of the review process

Documents

512 Evaluation Planning