IRB Web Site Usability TestFinal Report

English 3367 Web Usability Testing Team

Prepared for:  Donna Peters, Project Sponsor Human Subjects Research CoordinatorTexas Tech UniversityInstitutional Review Board 

Introduction

Background

Purpose

Project Overview

Methodology

Qualitative• Heuristic evaluation

• Site map

Quantitative• Lab tests

• Post test

Heuristic EvaluationVisibility of System Status Does the web site tell you where you are?

Match of system and the real world Does the web site use realistic language and conventions?

User control and freedom Does the web site allow the user to control activities?

Consistency with standards Does the web site present information and terms consistently?

Error prevention Does the web site make it hard or easy to make mistakes?

Recognition Rather Than Recall Does the web site make you remember things or make things obvious?

Flexibility and Efficiency of Use Does the web site make it easy to do things in more than one way?

Minimalist Design Does the web site encourage interaction?

Error Recovery Are error messages plainly stated with clear solutions?

Help and Documentation Does the web site present help effectively?

Heuristic Evaluation ResultsCharacteristic Observation

Visibility of system status Yes, there is a title page for each main subject. Title tag is not being effectively used: currently, using URL address, could repeat title or be more descriptive over the site

Match of system and the real world

 Technical wording mixed with common understandable language

User control and freedom  No, feels like there are too many options and if mouse moves just a bit, there are too many options with other sub-categories

Consistency with standards  The Tech pages are consistent with tables and text, but if it is linked out, the pages change or documentation is too wordy to understand

Error prevention  Very easy to make mistakes, no clarity to where user is on site.

Recognition rather than recall

 No, overload of information in technical terms, needed information is hidden quite well.

Flexibility and efficiency of use

 No, there is no return to main page/back button; there is really no good navigation.   There is also no site map.

Minimalist design  No, site seems to be very cold and technical, thus making the user shy away from interaction.

Error recovery  There is an email to contact the webmaster for any questions to the site, but there is not a Q&A section available for common questions.

Help and documentation  There is a lot of documentation but it is not organized well.  It seems like there is too much documentation, which makes the user shy away from trying to find the page he/she needs.  Wording and organization is poor.

Test Objectives

1. Terminology. Do users understand the terminology on the IRB site? 2. Resource types. Do users understand the differences between resource types as described on the 

site and how that affects the availability of items? 3. Organization. Is the site organization effective and usable for students trying to locate institutional 

review process information? Are users able to identify parts of the site relating to IRB and differentiate it from other parts of the site (ORS)? 

4. Navigation. Is the navigation of the site efficient for an IRB approval task? Are users aware of their current location in the site and how to return to a prior point in the process? Can site be accessed from different locations by different users? 

5. Page layout. Are the page layouts confusing or distracting? Are they too similar or too different—too dense, or too sparse? 

6. Form usage. Are the users made aware of the function of the various forms they need to fill out?  Can users understand and be able to properly complete the checklist? 

7. User control. Do users feel engaged and “in control” when using the web site? 8. Quality of writing. Is the site well-written overall and clear enough for users to understand the 

information they see? 9. Links and controls. Are hyperlinks and controls always spotted and recognized as such on each 

page? 

User Personas

Faculty, “Jeffery” Graduate Students, “Emily”

Age 34, Married 27, Single

Education PhD in Psychology Working towards PhD in Sociology

Work Experience

Full-time professor at Texas Tech University

Worked as a Grant Writer for Planned Parenthood from 2001-2004

Residence Off-campus town home Off-campus apartment, Lubbock, Texas

Computer Experience

Moderate, computer use on daily basis mostly for research and e-mails

Moderate, proficient with Microsoft Office and Access, but otherwise does not like to use technology

IRB Experience

Uses IRB website regularly, has trouble finding coversheet form.

Has vast writing experience, but has never used the IRB before

Priorities Use IRB website to find forms needed for human research and locate funding for research projects

Needs to learn how to have her study approved in a timely manner; avoid driving to campus; retrieve forms from the internet; save money

Tasks

Task 1: Find the “Getting Started page” Link

Task 2: (F) Using the “Exemptions” page for IRB approval

Task 2: (G) “What are some requirements for Principle Investigators?”

Task 3: “Is TTU approval needed if TTU is not funding your research?”

Task 4: What are the elements of a consent form?

Task 5: Who would you contact to get additional information and how

Tasks Linked to Test Objectives

Graduate Scenario/Task List:

Scenario 1: Allocated time, 10 minutesYou are thinking about doing research involving human subjects and are required to obtain IRB permission before proceeding.  Locate the “How to Get Started” link on the site.  http://www.ors.ttu.edu/Newors/newhome/home/trymain.html  

Relevant test objectives

X 1. Terminology X 4. Navigation 7. User control

2. Resource types X 5. Page layout 8. Quality of writing

X 3. Organization 6. Form usage X 9. Links and controls

User Test Results

Task 1: Find the “Getting Started page” Link

• 4 out of 5 were unable to find the “Getting Started” page

Task 2: (F) Using the “Exemptions” page for IRB approval

• 2 out of 3 had difficulty finding and using the form

Task 2: (G) “What are some requirements for Principle Investigators?”

• 2 out of 2 had difficulty answering the question

Task 3: “Is TTU approval needed if TTU is not funding your research?”

• 4 out of 5 answered the question incorrectly

Task 4: What are the elements of a consent form? Link

• 3 out of 5 were not able to answer correctly

Task 5: Who would you contact to get additional information and how

• 5 out of 5 were able to answer correctly

Post Test Results

Key Findings

• Language unclear in pages and in forms

• Navigation of site difficult and navigation bar confusing

• Introductions to forms unclear and skimpy

• Users did not know where to start

• Overall site poorly organized

Language Not Clear

http://media.english.ttu.edu/faculty/barker/3367irb/language.wmv

Introduction Not Useful 

http://media.english.ttu.edu/faculty/barker/3367irb/introduction.wmv

Introduction to forms need more information

Navigation Unclear

http://media.english.ttu.edu/faculty/barker/3367irb/navigation.wmv Improve navigation by reducing redundant links.

“How To Get Started” Not Useful

http://media.english.ttu.edu/faculty/barker/3367irb/howto.wmv

Make “How to Get Started” a series of steps with links.

Site Map Evaluation

“The site is mostly a collection of documents…”

Organize the site around recognized user persons

Site Poorly Organized

Organize the site around recognized users• Tester -using the site to find information on testing

procedures and policies • Testee -interested in participating in the testing process • Eraider member (password needed)-links that can only

be used by those with an eraider account • Administrator (password possibility) staff who will be

conducting or overseeing testing and research, and may need administrative access to use the link

• External link -leads to links that are not directly controlled by texas tech (or ORS & IRB website)

Recommendations

• Clarify definitions of terms Link

• Make “How to Get Started” useful

• Improve navigation by reducing redundant links

• Create informative overviews for forms

• Reorganize the pages according to user types

Conclusion

Project Mission: make educated recommendations concerning the effectiveness of the ORS Human Subjects website

Project Methods: heuristic evaluation, qualitative and quantitative measurements, usability testing

Subjects Studied: terminology, resource types, organization, navigation page, layout, form usage, user control, quality of writing, links and controls

Recommendations:

• Clarify definitions of terms

• Make “Getting Started” useful

• Improve navigation by reducing redundant links

• Create informative overviews for forms

• Organize the site around user needs

Final Report

Please view our entire report at:

http://irbusability.pbwiki.com/

Questions?