Quest Atlantis Quest Atlantis Homepage Tutorial ProjectHomepage Tutorial Project

Min-joung KimMin-joung Kim

Ming MaMing Ma

Pil-Won OnPil-Won On

Project DescriptionProject Description

Target AudienceTarget Audience LanguageLanguage

EnglishEnglish

AgeAge 9 to 12 year-old9 to 12 year-old

Gender Gender Boys : Girls = 5: 1 Boys : Girls = 5: 1

Pre-requisite knowledgePre-requisite knowledge Minimum pre-experience of using the 3-D Minimum pre-experience of using the 3-D

interface (logging in the QA)interface (logging in the QA) Starting a computer and a program, using a Starting a computer and a program, using a

mouse and a keyboardmouse and a keyboard

ContextContext

Mainly in a local Mainly in a local Boys and Girls ClubBoys and Girls Club with fast internet connection.with fast internet connection.

Users should be registered in QA.Users should be registered in QA.

Bannero Name of Questero Emoticon

EmailoLink to pop-up email pageoMessage for any new mail

Navigation Links Links to other side-bar pages

Side-bar Contents Title and summary information

Pop-Up LinkOpens content in pop-up window with complete details for reviewing or editing

ObjectivesObjectives

ObjectivesObjectives

After going through the QA Home Page tutorial, learners will be After going through the QA Home Page tutorial, learners will be able to:able to:

Identify each feature on the side-bar pageIdentify each feature on the side-bar page BannerBanner

Identify the owner of home pageIdentify the owner of home page.. Identify what the emoticon is.Identify what the emoticon is.

EmailEmail Identify the label of email.Identify the label of email. Identify the number of new email.Identify the number of new email.

Navigation linksNavigation links Identify what “home” is for. Identify what “home” is for. Identify what “friends” is for.Identify what “friends” is for. Identify what “quests” is for.Identify what “quests” is for.

Pop-up linkPop-up link Identify the link to a pop-up.Identify the link to a pop-up.

Utilize the function of “home” page editing. Utilize the function of “home” page editing.

in 80% of in 80% of successful completion without any help from mentors.successful completion without any help from mentors.

Merrill’s 5 Stars Instructional Merrill’s 5 Stars Instructional DesignDesign

Problem: Problem: Utilize the function of the side-bar (Utilize the function of the side-bar (exex))

Activation: Use analogy of a “ House ” (Activation: Use analogy of a “ House ” (ex1ex1, , ex2ex2))

Demonstration: Show as a movie, guided practice Demonstration: Show as a movie, guided practice ((ex1ex1, , ex2ex2))

Application: Practice part - skateboarding gameApplication: Practice part - skateboarding game((ex1ex1))

Integration: The learners actually apply their Integration: The learners actually apply their learning for the real side-bar. learning for the real side-bar.

QA training team’s 6QA training team’s 6thth star star

MotivationMotivation Talking headTalking head Familiar environmentFamiliar environment Skateboarding game Skateboarding game

Use one of the avatars in QAUse one of the avatars in QA Visual reinforcementVisual reinforcement

Award: 4 points for completionAward: 4 points for completion Animation: flash moviesAnimation: flash movies

Expert review of paper Expert review of paper prototypeprototype

ConductConducted expert review with Lauryl ed expert review with Lauryl and Dr. Frickand Dr. Frick

After the review, added different After the review, added different words for feedback. For example, words for feedback. For example, “good job”, “great”, “excellent”, etc. “good job”, “great”, “excellent”, etc.

Changed “friends, quests, and home” Changed “friends, quests, and home” in the questions with other words to in the questions with other words to avoid giving hints to learners. avoid giving hints to learners.

Paper Prototype usability testPaper Prototype usability test

Leaner 1 Learner 2 Learner 3 Learner 4

Age 12 9 9 12

Gender Male Male Male Male

Nationality American American American American

Pre-experience with QA

1 year 1 year A few month

s

3 times

Information of the subjectsInformation of the subjects

Scale of reactionnareScale of reactionnare

5. I will tell my friends to play with this tutorial.

Paper Prototype usability testPaper Prototype usability testInstructional Effectiveness (n=4)

Learner Pre-assessment Score

Post-assessment Score

# 1 8 out of 9 9 out of 9

# 2 8 out of 9 9 out of 9

# 3 6 out of 9 7 out of 9

# 4 4 out of 9 7 out of 9

Novice learners (#3 and #4)

Mean SD Mean SD

5 (56%)

1.41 7 (78%

)

0

Overall Mean SD Mean SD

6.5 (72%)

1.91 8 (89%

)

1.15

Satisfaction with Instruction (n=4)

Reactionnaire Mean Score (3 point scale)

2.8

2.8

2.8

2.4

Mean SD

2.6 (87%)

0.3

Mean SD

2.7 (90%)

0.2

Paper Prototype usability testPaper Prototype usability test

Comparison of pre-test and post-test resultsComparison of pre-test and post-test results

Chart 1. Mean scores of pre-test and post-testfor Quest Atlantis Side-bar Tutorial(n=4)

6.5

8

0

1

2

3

4

5

6

7

8

9

pre-test post-test

Sco

re

Paper Prototype usability testPaper Prototype usability test Comparison of pre-test and post-test results: Comparison of pre-test and post-test results:

individual performanceindividual performance

8 8

6

4

9 9

7 7

0

1

2

3

4

5

6

7

8

9

10

1 2 3 4

Learner

Sco

re (

n=

9)

Pre-test

Post-test

Paper Prototype usability Paper Prototype usability testtest

Problem ImprovementLearners failed the question about “home” page

Explain more effectively and in detail what “home” is about

Learners failed the question about navigating to different pages

Explain the navigation function of the buttons more effectively, with simulated animation

Learners failed the question about editing the “home” page

Teach how to edit “home” procedure more effectively: review the sequence one more time after a movie, and let learners practice one more time

Revisions on improving instructional effectives

Paper Prototype usability Paper Prototype usability testtest

Problem Improvement

Learners were easily distracted when there was only audio (read by the testers) but no visual character in the tutorial

Use a talking head

Revision on improving learner satisfactionsatisfaction

Paper Prototype usability testPaper Prototype usability test

Problem Improvement

Learners did not know where they were expected to go between scenes

Use consistent “Go” button to lead learners

Learners found reading difficult

Reduce the amount of text – use more audio

Learners were confused by some wording in the tutorial

Change misleading wording

Revisions on improving usability

Computer prototype usability Computer prototype usability testtest

Leaner 1 Learner 2 Learner 3

Age 12 12 11

Gender Male Male Female

Nationality American American American

Pre-experience with QA

2 months Less than a month

Less than a month

Information of the subjectsInformation of the subjects

Computer prototype usability Computer prototype usability testtest

Instructional Effectiveness (n=3) Satisfaction with Instruction (n=3)

Learner Pre-assessment Score

Post-assessment Score

Reactionnaire Mean Score

# 1 0 out of 8 7 out of 8 3

# 2 0 out of 8 6 out of 8 3

# 3 1 out of 8 8 out of 8 3

Overall Mean SD Mean SD Mean SD

0.3(4.2%)

0.6 7(88%)

1 3(100%)

0

ResultsResults

Computer prototype usability Computer prototype usability testtest

Comparison of pre and post test resultsComparison of pre and post test results

Chart 2 Mean score of pre-test and post-test for Quest Atlantis Side-bar Tutorial (n=3)

0.3

7

0

2

4

6

8

pre-test post-test

Sco

re

Computer prototype usability Computer prototype usability testtest

012

3456

789

1 2 3

Learner

Sco

re (

n=

8)

Pre-test

Post-test

Comparison of pre and post test results: individual Comparison of pre and post test results: individual performanceperformance

Computer prototype usability Computer prototype usability testtest

Problem Improvement

Hesitation on the action of clicking the small objects which is linked to.

Make the objects more prominent in such ways: Making the object bigger, providing an instant effect.

Confusion between links and non-links

Get rid of unnecessary links which make the appearance of a mouse cursor a pointing finger

Frustration with limited choices on practice part.

Provide more options of choices.

RevisionRevision

Questions or Comments?Questions or Comments?