System Interface Evaluation:Goodreads

Group 3April 17, 2012

Group 3 Members

Lauren Long, Evaluator• Cataloger,

Transylvania Co. Library, Brevard, NC

• ITRL Student

JULIE FORKNER,Evaluator• Director, E.G.

Fisher Public Library

Athens, TN• ITRL Student

ANGELA GLOWCHESKI,Evaluator • Information

Specialist, Lumpkin Co. Public Library Dahlonega, GA

• ITRL Student

SUSAN MACRELLIS,Leader• Director, East Ridge City Library Chattanooga, TN• ITRL Student

MARILYN PONTIUSEvaluator• Branch Manager,Hancock War MemorialBranch LibraryWashington Co, MD• ITRL Student

Leader Responsibilities

Planning the evaluation method Developing two tasks to be performed by

each group member (Evaluator) Meeting with evaluators to discuss tasks,

responsibilities, when and where to perform walkthrough, use of Nielsen’s severity rating scale

Performing a walkthrough for each task and creating a task analysis to use as benchmarks for comparison to Evaluator walkthroughs.

Evaluator Responsibilities

Perform each task individually Identify potential problems and errors Use Nielsen’s severity rating scale Provide solutions for solving the problems and

errors Document the feelings and emotions Generate a usability report as a team

Cognitive Walkthrough Overview

Based on what was learned in Assignment 2: Evaluate interface frequently used by adult

service librarians Evaluate a freely available reader’s

advisory interface

Two Phases: Preparatory Phase Analysis Phase

Walkthrough Methodology Preparatory Phase

Define the interfaceDefine the tasksDefine the target usersDefine the correct sequence of actions to

complete tasks

Favorite Genres generated from user input

Keyword searching

What friends are reading and what they have to say about it

Free Stuff

Book Clubs

Preparatory Phase: Define the Interface

Task 1

Task 2

A user is looking for a book in the Goodreads website. He wants to find the name of the author of this book: Jonathan Strange & Mr. Norrell. Find the author’s name and write it down on the task instruction sheet.

Find how many books there are in the “Blossom Street” series written by Debbie Macomber. What is the title of the most recent book in the series? Find the title name and write it down on the task instruction sheet.

Preparatory Phase: Define the Tasks

Adult services librarian working in a public library Has obtained a college degree, and in many cases a

master’s degree Performs tasks related to reader’s advisory Utilizes a reader’s advisory information system Seeking information

Preparatory Phase: Define the Users

TASK 1

Preparatory Phase: Correct Sequence of Actions0. A user is looking for a book in the Goodreads website. He wants to find the name of the author of this book: Jonathan Strange and Mr. Norrell. If you find the author's name, write it down on the task instructions sheet.

1. Access Goodreads website1.1. Open browser1.2. Enter Goodreads URL in address bar (http://www.goodreads.com)2. Access Goodreads search box2.1. From home page scroll down to search box2.2. Click inside search box3. Enter search terms in search box (Jonathan strange and mr. norrell)3.1. Hit enter key or click on magnifying glass icon4. From results page select first record returned: Jonathan Strange and Mr. Norrell by Susanna Clarke4.1. Write down name of author on task instruction sheet, Susanna Clarke

TASK 2

Preparatory Phase: Correct Sequence of Actions

0. Find how many books there are in the "Blossom Street" series written by Debbie Macomber. What is the title of the most recent book in the series? Write down number of books in series and title of most recent one on the task instructions sheet.

1. Access Goodreads website 1.1. Open browser1.2. Enter Goodreads URL in address bar (http://www.goodreads.com)2. Access Goodreads search box2.1. From home page scroll down to search box2.2. Click inside search box3. Enter search terms in search box (Debbie Macomber)3.1. Hit enter key or click on magnifying glass icon4. From first page of results, select second record returned: The House on Blossom Street (Blossom Street #1). 4.1. Click on book title or book jacket photo5. Go to Blossom Street series page.5.1. Click on series name hyperlink following book title (Blossom Street #1)5.2. Write down number of books in series (8) and title of latest book A Turn in the Road

Walkthrough Methodology Analysis Phase

Evaluators perform tasks and record dataCollect and compare data to correct actionsConstruct success story with defense of

credibilityConstruct failure story

Analysis Phase:Task Instruction Worksheet Example

Action Problem Actual or Potential?

Nielsen Severity

Cause of Problem

Possible Solution

Feelings/Emotions

1. Go to www.googreads.com

2. Enter “Jonahan Strange and” in title search box

2.1 Error returned

Unexpected error report: Did you mean jonah an strange and

Actual 1 Misspelled name

Improve “did you mean” function to recognize “jonahan” as Jonathan”

Surprised. Confused by message, but recognized the “did you mean” probably ment a spelling error.

Analysis Phase –Consolidation Spreadsheet

Allows problems and errors to be easily identified

Create best success story with credibility data

Create worst failure story with problem areas identified

Task Data Consolidation and AnalysisTask 1 Example: Compare Actions

Correct Action Eval 1 Eval 2 Eval 3 Eval 4

1. Go to www.goodreads.com

1. Go to www.goodreads.com

1. Go to www.goodreads.com

1. Go to www.goodreads.com

1. Go to www.goodreads.com

2. Enter “Jonathan Strange and Mr. Norrell” in title search box

2. Enter “Jonathan Strange and” in title search box

2. Entered “Jonathan Strange” in title search box

2. Enter “Jonathan Strange and Mr. Norrell” in title search box

2. Enter “Jonathan Strange and Mr. Norrell” in title search box

2.1 Error returned

Summary & Conclusion Where we are

Evaluators – perform cognitive walkthroughs and fill-in findings on task instruction sheet;

Discuss in group meeting;Begin formulating narration (findings) for final

report;Review / edits for final report;Submit final report.

Questions/Comments for the Group

Group 3 thanks you for listening!

LAUREN LONGJULIE FORKNER

ANGELA GLOWCHESKI

SUSAN MACRELLIS

MARILYN PONTIUS