"Think-Aloud" Usability TestNetSmartz.org
Inquiry Topic: Child Online Safety & Parental Measures
Presented by:
Dixie Davis
Laurie Miller
Mark Pella
Francis Tan
EME 6635: Inquiry & Measurement
Dr. Vanessa Dennen
Spring 2008
Increasing growth of children (ages 5-17) using the
Internet for school and entertainment.
Problem: Greater access to the Internet has exposed
children to objectionable & dangerous content
Inquiry Background
(pornography, predators, violence, identity theft).
Question: What are parents doing to keep their children
safe when they are using the Internet at home? How
effective are these measures?
Usability Testing Focus
• 70% of parents confident in
ability to gather Internet
safety information
• Assess parental ability to
I know where to go to get information regarding
Internet safety
St rongly A g ree3 2 %
St rongly D isag ree0 %
D isag ree12 %
U ndecid ed8 %
• Assess parental ability to
locate materials concerning
online safety for children
using the Internet.
• Resource Website Focus:
www.netsmartz.org
Ag ree4 8 %
Protocol ScriptWelcome:"Thank you for participating in today’s usability test. This is an exciting study that we are conducting to
research child safety online. This usability test helps assess parental ability in locating online resources that
contain online safety materials. It should not take long to complete. We are testing the site, not you. We are
conducting this as a “think aloud” protocol. You will speak your thoughts aloud while completing the task. I
will observe and take notes."
Agenda:"There are a few steps to today’s usability test. First, I would like to gather your demographic background
(used survey questions) Second, you will be asked to complete three separate, but related, tasks using a web
browser. You will discuss aloud your thought process as you work in completing the task objective."
Entrance Questions:"Do you have any questions that need clarification from me? I want to ensure your comfort and
understanding of your role in the study. I want to make this a rewarding experience for you, so do not hesitate
to ask if you need anything."
Exit Questions:"Do you have any questions for me? Any general comments regarding the site? Regarding this test? Thank
you for your time & willingness to participate. You have made a valuable contribution to our efforts."
Protocol TasksTask 1: Using a search engine
Protocol TasksTask 2: Locate & Register for NetSmartz Bulletin
Protocol TasksTask 3: Locate & Download Safety Pledge
Test Findings
“Are you kidding me? HaHa.
Where do I even begin?”
“This looks like a crayon box
threw-up.”threw-up.”
– Test Subject A
Test Subjects
• Three females, one male.
• Ages ranged from 34 (youngest) to 41 (oldest).
• Three with post graduate degrees, one a high school • Three with post graduate degrees, one a high school
graduate.
• Two with IT backgrounds.
Success Rate: 100%
• All four subjects able to complete the three tasks.
• On average, testers spent nearly 6 minutes completing
tasks. They spent the most time on Task 2: Finding
Bulletin.Bulletin.
• All had initial difficulty finding either the bulletin or the
safety pledges.
• All initially had difficulty
navigating the site.
• Quick Links – List of
Popular Destinations only
Ah found it! (Sigh) It is so well hidden!”
– Test Subject B
Popular Destinations only
available from main page.
• Left-hand menu bar not
intuitive.
Conclusions• Test showed parents could use site to sign up for bulletin
and download safety pledges.
• Website could include search box to allow items to be
more easily found. “I don’t know if you’re looking for
suggestions, but I wish they had a key word box.” –suggestions, but I wish they had a key word box.” –
Subject C.
• Quick Links list could be included on all topic pages, not
just main page.
Reflections•Overall the Usability test was more fruitful than many
of us thought.
•Maybe we should have done a hybrid of Retrospective
and Concurrent Think Aloud Protocols instead of solely
Concurrent Think Aloud Protocol.Concurrent Think Aloud Protocol.
•The preparedness level for the administrator of a
Usability Test is similar to that of an Interview.
Lessons Learned•Reiterate the fact the participants should use the
NetSmartz website in the directions for each tasks.
•Be more specific about what pledge we want them to
downloaddownload
•It is difficult to not give a hint when the participant is
frustrated
•Following Think -Aloud protocol is difficult for some
participants.