Kasey AlpertInto to Web PublishingJacquie LamerUsability Test Assignment3/10/2015

Usability Test for KansasCity.com

Test Preparation

I prepared for the test by first going through each of the tasks that the participants

would be asked and complete them as if I was testing myself. The first task was to find what

films were nominated for best picture for the Academy Awards. I typed “best picture academy

awards” into the search tool that is in the header on KansasCity.com. The first article that

showed up was a photo gallery for the Oscars. I had to scroll through 6 pages on the photo

slideshow to find the names of the films that were nominated for best picture. The next

assigned task was to find out when the Oscars air on TV. Before I went to go search for the

information, I noticed that what I was looking for was in the caption of the photo that was still

up from the last task. The assignment for task 3 was very easy, in my opinion. The task was to

find the section of the site titled “816”, click on an article and go through the process of

ordering reprints of it. I first hovered over the “Living” tab, and then when I didn’t find what I

was looking for I went over to the “News” tab and found the “816” section. There is a circular

button with an arrow encased in it at the top of the article, and I hovered over it. I was right in

guessing that this button would display a link to order reprints. Once the link had been clicked,

it takes you to a different website where reprints can be ordered. I thought finding the reprint

link was straightforward and simple. However, task 4 proved to be more difficult. The task was

to determine the events that will take place over the following week in Jackson and Cass

counties. I went through a couple different searches, typing “upcoming events in Jackson

county” and “events calendar” through the search tool, but they weren’t helpful. I went to the

“News” tab and then clicked on “816”, and I found an article that had some recent events that

happened in Jackson and Cass County, but the article didn’t mention future events. In the end, I

couldn’t find exactly what the task was asking for. The next task was to search for a job in a

specific city using a keyword. There is a “Classified” tab that I went to that places a search for a

job using a keyword, and the city and state can also be specified. When I clicked search, the

jobs page didn’t show up. Instead KansasCity.com reloaded back to the home page. So instead

of searching the jobs through the Kansas City Star’s website, I clicked a link titled “View All Top

Jobs”, which is located right under the job search under the “Classified” tab. This took me to

careerbuilder.com where I could then search for a job in Kansas City. The last task was to share

a job listing on social media. Completing this task proved to be difficult. I could not locate a link

to share a listing on any social media site. Some job listings had the company Facebook or

Twitter feed on the side, but the only way I could find to share a job listing was to email it.

Choosing Participants

Tester 1: Karla Everman

Karla Everman is 19 years old, and is from the Kansas City area and a fulltime

student and part time tutor at Northwest Missouri State University. I chose her as a

tester because I knew that she would have an interest in participating. She spends

approximate 8 hours a day on the internet with 10% of her time spent browsing and

90% on social media. Karla is a high experience tester.

Environment for Tester 1

Location of test: The location of the test was in Karla’s room at her desk. Her

room was chosen because this is where she does most of her internet browsing.

Physical environment: There was an overhead light that was on, as well a

another overhead light in the background and a lamp nearby. Karla had her

phone on vibrate and was placed on the desk, to the left of her computer. The

room was silent; there were no other people in the room and the air

conditioning was off.

Technical environment: The computer used was a HP EliteBook laptop with a

screen resolution of 1366x768. Karla used Google Chrome for the test and did

not have any add ons installed. The internet connection speed was

approximately 5Mb/s.

Tester 2: Brenna McLeod

Brenna is a 21 year old from the Kansas City area and is a full time student at

Northwest Missouri State University. She spends about 40 hours a week on the internet,

and only 20% of that time is on social media. I chose her as a tester because I thought

she would be a good high experience user because she works part time with her job that

is back home, and all of her projects are on the computer.

Environment for Tester 2

Location of test: The location of the test was in Brenna’s room at her desk. Her

room was picked because that is where most of her internet browsing is done.

Physical environment: There was an overhead light on, a lamp, another

overhead light on, and Christmas lights were on. There was a window open, and

some slight noise could be heard from the people outside.

Technical environment: The computer used is a HP EliteBook laptop. The screen

resolution is 1366x768 and the internet connection was about 5Mb/s. For the

majority of the test Chrome was the browser that was used, and an adblock ad-

on was in-use. Internet Explorer was used for the last half of the assigned tasks.

Test Results

Initial Site Thoughts:

Tester 1 Report: Karla got to the site by typing the url into a google search. Once

she got to the site, she read through all of the articles that were displayed on the

home page. She scrolled by using the mousepad. When Karla got to the bottom

of the page, she noticed the bottom navigation and thought that it was a good

thing for the site to have.

Tester 2 Report: Brenna first noticed the temperature display when she reached

Kansascity.com. She said that, “there are lots of big things, and a lot going on but

it doesn’t seem too crowded”. She also used the mousepad to scroll and she said

that it seemed like she was scrolling forever. Once Brenna reached the bottom of

the page she remarked that, “it doesn’t seem too hard to find things specific, but

it could be crazy if you’re just browsing”.

Task 1: Determine the following information about the 87th Annual Academy Awards.

Tester 1

What films are nominated for best picture?: Karla first went over the

“Entertainment” tab and read through all of the topics before clicking on

“Movies”. She scrolled using the mousepad, and went through the entire page

until she got to the bottom. When she didn’t find what she was looking for she

went back to the top of the page and hovered over the “News” tab, then the

“Home” tab, reading through each of the topics under the tabs. She then went

back over to the “Entertainment” tab and clicked on “TV”. Karla started reading

through each of the titles of the articles and got distracted by the titles that were

displayed on the right side of the page as she scrolled through the whole page.

Her phone vibrated at this point, and she seemed to give up on the task after she

checked her phone. We moved on to the next task.

- Answer: Didn’t finish task.- Completed: No.

When will the Oscars air on Television?: She first went to the “Entertainment”

tab and after reading through the topics, and then clicked on “Star TV Listings”.

She scrolled using her mouspad, and read through the tv listings. Karla took time

looking through the listings, searching for when the Oscars would be airing. She

went through many pages of the tv listings before saying “there should be an

easier way of doing this”. She thought maybe she had passed the listing for the

Oscars so she went back up to the top of the page and started going through the

listings again. At this point I knew that the task was not going to be completed so

I advised that we move on.

- Answer: Did not find the answer.- Completed: No.

Tester 2

What films are nominated for best picture?: Brenna first went to the

“Entertainment” tab and clicked on the section “Movies”. She started scrolling

using the mousepad and was looking at the headlines first. She asked: “can I use

the search bar in the site”? I told her that that was acceptable and she typed in

“academy awards and movies” into the search tool. She said that it would be

helpful to filter out the results. Brenna then searched “academy awards”, and

when she didn’t find what she wanted she searched “Oscars”. She then spent

some time reading through all of the headlines to see if an article contained the

information that she needed. Brenna then did a couple more searches, typing

“87th Oscars” then trying “best picture”. The top link that came up when she

searched for the best picture was a photo gallery that had Oscar information.

She said, “I don’t think this is right”. She did another search for “Oscars best

picture” and scrolled through the results. She was starting to get frustrated and

said “it’s really buried; not the first thing that comes up”. At this point my phone

vibrated, and I hoped that it hadn’t distracted her. She was still looking through

articles and eventually found the answer.

- Answer: “American Sniper, Birdman, Boyhood, The Grand Budapest Hotel, The Imitation Game, Selma, The Theory of Everything, and Whiplash”.

- Completed: Yes.

When will the Oscars air on Television?: She searched “Oscars air time” in the

search tool. She mentioned the photo gallery that she had seen during her

search in the previous task, and said that it probably contained the information

she was looking for. She clicked on a different photo gallery that contained

pictures of celebrities from the red carpet event of the Academy Awards. Brenna

clicked through a couple of images until one of them said the date that they

were taken.

- Answer: “February 22”- Completed: Yes.

Recommendations to improve UX for this task, linked to Jakon Nielsen’s 10

Usability Heuristics: Both testers thought that the site was busy, and caused

some distractions. I would suggest cleaning up the design of the placement of

the articles. The heuristic that most aligns with this idea is “aesthetic and

minimalist design”. When searching through a topic, such as “816”, there are

multiple places where articles are displayed. The featured articles should be off

to the side more and not displayed throughout the page. It becomes distracting

and overwhelming when browsing.

[see next page for screenshot example]

Original

Proposed

Task 2: Find the section of the site titled “816” which covers community news

Tester 1

In this section, find a story of interest and go through the steps to order

reprints of the article. Go as far as possible without actually ordering: Karla

hovered over the “Living” tab and read through the available topics, then she

went over to the “Entertainment” tab and did the same thing. Then she found

that the “816” topic is located under the “News” section. Once she got into the

“816” topic she quickly found an article of interest. After she found the article,

her phone went off and she quickly checked it before returning to her task. She

immediately hovered over the arrow button and clicked on the order reprints

link. Then Karla clicked on the “Request reprint” link on the external page and

quickly found the form to fill out.

- Completed up to ordering: Yes.

Tester 2

In this section, find a story of interest and go through the steps to order

reprints of the article. Go as far as possible without actually ordering: Brenna

started by looking in the “Home” tab before finding the section in the “News”

tab. Once she clicked on “816” she scrolled down to the bottom of the page and

noticed the link that read “Print” in the bottom navigation. She said “that’s

probably to order the paper”, so she went and clicked on an article. There was a

pop-up that said she must sign in as a member to get unlimited access. When

she exited the pop-up the site just reload back to the home page. So, she closed

down Chrome and opened up Internet Explorer. The first thing she said when

she got to Kansascity.com was that the website “looks obnoxious with the ads”.

When she used chrome, she had an adblock on. She then got back to the same

exact article by doing all the steps already stated above. Once Brenna got to the

article, she scrolled down to the bottom of the page while saying that she was

“looking for text that says reprint”. Then, she scrolled back to the top and found

the icon with the arrow on it and clicked the link to order reprints. She followed

the link all the way until the form was on the screen. After, Brenna mentioned

that she thought that task would be simple, but it turned out not to be.

- Completed up to ordering: Yes.

Recommendations to improve UX for this task, linked to Jakob Nielsen’s 10

Usability Heuristics: Hiding the reprint link in a button makes it difficult for some

users to find. I would suggest placing the link directly on the article page, so that

once a visitor reads the article, and reaches the bottom of it, they can clearly see

the text to order a reprint. A link to print the article should also be added to stay

consistent with expectation. This would be an example of the “Recognition

rather than recall” heuristic.

[see next page for screenshot example]

Task 3: Determine what events will take place in the following week in Jackson and Cass counties.

Tester 1

Determine what events will take place in the following week in Jackson and

Cass counties: Karla went over to the “News” tab and clicked on the “Local

News” section. Then it occurred to her that she could have used the search tool

within the website to help guide her search. She typed “Jackson County” into the

search tool and then scrolled through the results. When she didn’t find what she

wanted she typed “upcoming events in Jackson co”. This search provided Karla

with events in Jackson Co., but they were not upcoming ones. She kept scrolling

through the search results and went on to page 2. There on page 2 she found a

Original

Proposed

The icons that are at the end of an article

calendar of events for Jackson and Cass Co. She read through it and got

distracted by the titles of the events.

- Answer: Read off each of the titles until I told her to stop.- Completed: Yes.

Tester 2

Determine what events will take place in the following week in Jackson and

Cass counties: Brenna first looked under the “News” tab and got distracted by

the different topics. She said she was looking through all of the tabs for a

calendar, and as she was looking, ads were popping up which were distracting.

She ended up clicking the “local news” section under the “News” tab. Brenna

said that she couldn’t find a calendar anywhere. So, she typed “calendar” into

the search tool and then clicked on an article that was on how to submit a

calendar, which wasn’t helpful. Then she did a few searches, typing

“neighborhood news”, then “upcoming events”. Brenna then filtered the results

by latest date posted. With this still not delivering the results she wanted, she

became frustrated and annoyed. Brenna said that, “I just want there to be a

calendar”. Then she wanted to move on to the next task and didn’t want to try

to figure this one out anymore. So, she gave up.

- Answer: None, gave up.- Completed: No.

Recommendations to improve UX for this task, linked to Jakob Nielsen’s 10

Usability Heuristics: There is no calendar for visitors to look at. Both of my

testers would have benefited from being able to visit a calendar that listed all of

the upcoming events. The upcoming events page is buried deep within the site,

and I am sure that many visitors have never been able to find it, or find it only

after looking for a long time. I would suggest that the site follows the heuristic

“Match between system and the real world”, and add a calendar that can be

visible to visitors.

[see next page for screenshot example]

Add a link that when clicked, will take the visitor to a calendar of events.

Original

Proposed

Task 4: Determine the following information related to finding a job.

Tester 1

Search for a job in any specific city using any keyword: First Karla went to the

“Classifieds” tab and then went over all of the other tabs before beginning to

type into the search bar. She didn’t finish typing because the titles of the articles

distracted her. She started scrolling down the page and then read through the

bottom navigation links when she got there. Then she went back over all of the

tabs again before clicking the “jobs” link at the top right of the page. At this point

it took her to an external page hosted by careerbuilder.com. Karla typed in her

keyword “Accountant” and also typed in a city and selected a category as well.

Then she scrolled through most of the job listings before clicking on one.

- Answer: Was able to find a job by typing in a keyword.- Completed: Yes.

Find a job listing and share it using social media: With the search results still up,

Karla clicked on one. She clicked on the company Facebook icon which took Karla

to the company Facebook page. She looked around and then hit the back button

to go back to her search results. She then scrolled through the twitter feed that

is directly on the Kansas City site. Karla then clicked the link to view all jobs and

then hit the back button. Her phone vibrated and she picked it up, hit a button to

silence it, and took if off the desk. Then she clicked on a different job link than

the one before, but she was having trouble finding how to share the listing. She

decided that she would try to email it instead, and then she gave up on searching

anymore.

- Answer: Couldn’t find out how to share a job listing.- Completed: No.

Tester 2

Search for a job in any specific city using any keyword: Brenna hovered over the

“Business” tab before clicking on the “Jobs” link at the top right of the page. It

then took her to the page where she could search for jobs. She typed in

“Psychology jobs” and put “Sarasota, FL” for the location and was able to find

many job listings.

- Answer: Was able to find a job using a keyword.- Completed: Yes.

Find a job listing and share it using social media: With all of the listings from the

previous task still up on the screen, Brenna clicked on a listing. She noted that

“there are ads everywhere” on the screen. She noticed that the listing could be

emailed. She clicked the company Facebook link and then went back to the job

listing she had clicked. Brenna said that she “expected it to be next to the email,

for continuity”. When she scrolled through the listing she noticed how all of the

text was on the left side of the page, and thought that the text should really be

covering more of the site. When she couldn’t figure out how to share the listing

she said that “this looks like a terrible site”. This site is, however, not directly on

Kansascity.com. It is on careerbuilder.com but there is a header that links back to

Kansascity.com.

- Answer: Was not able to share a listing on social media.- Completed: No.

Recommendations to improve UX for this task, linked to Jakob Nielsen’s 10

Usability Heuristics: When viewing any article on KansasCity.com, there were

buttons before the article started that were for sharing to social media sites. To

stay consistent, the jobs site needs to have the same kind of buttons, even if this

is on careerbuilder.com and not kansascity.com. This aligns with the

“Consistency and Standards heuristic.

These are the icons on the job listings. There is nothing to indicate a share button for social media.

These are the buttons on kansascity.com. There are clear buttons for Facebook, Twitter, and Google+.

This is what I would suggest gets done to the buttons. A share button should be added, so that visitors who want to share a job link on a social media page they can.

Conclusions

Completion rate: Karla Everman had a completion rate of 50%. She completed 3

out of the 6 assignment tasks. Brenna McLeod had a 67% completion rate. She

completed 4 out of the 6 tasks.

Similarities and differences: Both of my testers completed the tasks in many

similar ways. They both used the search tool to help them find information on

the site. They both also took the time to scroll through the pages while trying to

find the right article or information. Both Karla and Brenna used the jobs link at

the top right corner of the site instead of going under the “Classifieds” tab. My

testers also went about completing the tasks in different ways. Brenna right off

the bat used the search tool, while Karla didn’t use it until after completing 3

tasks. Karla also spent more time reading through the articles and the other

content on the site than Brenna did. Brenna didn’t let the frustration of the tasks

get to her as much as Karla did. Once Karla got frustrated, she was ready to be

done, while Brenna tried harder to finish the task. Karla tended to get more

distracted with the site than Brenna did. One of the obvious reasons why they

went about completing the tasks differently is because they are different people.

Different people search the internet different ways. I also think that the amount

of distractions a person can take affects their ability to focus. Some people are

able to have focus when there are elements that could pose as a distraction. This

was clear with my testers. Karla was easily distracted with all of the articles and

content on the site, while Brenna has a little more focus. The threshold for

focusing on the task at hand and not getting distracted made itself known during

this test. It is important to remember that different people get distracted by

different things. So, making sure that a site doesn’t have a lot going on and has a

minimalist design (following the aesthetic and minimalist design heuristic) is a

good way to ensure that the website has good user experience.