Case study - usability evaluation Howell Istance

Case study - usability evaluationHowell Istance

SOFT3057 - Interactive Systems

ELISE II - Web- based Image Retrieval System

This allows users to enter search terms which relate to descriptions of images stored in image collections e.g. ‘sunflower’







Methods proposed for usability evaluation of ELISE II

Structured expert reviewing Use of cut-down version of cognitive walkthrough technique

Observation of users completing standard tasks Studied via a number of user trials

Extended in-situ testing Use of ELISE II in normal work during 1 month Participant keeps diary of system usage


User trials – pilot testing

Used 2 graphics designers Insufficient scope in the tasks used to test the advanced

search and the browse facilities Requires additional tasks which require these to be used


Participants in evaluation trials

Limerick 5 librarians, 1 student ULB 10 librarians DMU 2 graphics designers (pilot), 2 lecturers, 2 research

students V&A 1 researcher, 2 cataloguers, 2 curators Tilburg 1 librarian, 2 cataloguers, 1 manager


Procedures for user trials

introduction to the trial and its purpose initial interview to establish background and nature of

image use initial exploration phase formal explanation of the functions search for last image actually used Searches for 3 sample images post-trial interview


Macro Usability Issues

How well does the system support the user in formulating a query?

Are the retrieval results presented in such a way that enables the user to comprehend the relevance of the results to his search needs?

Is the user supported in making advanced queries? Can the image retrieval system be integrated into the

working environment of the user?


Search strategies

Most – began with restrictive search (several terms and boolean operators), removed terms successively of increase number of hits, only if no hits found then different generic term used

Few – began with generic term and added terms to reduce hits

Very few – used browse facility to check search term Many tried to minimise numbers of record retrieval in view

of long wait


Support in formulating query

Level of information about individual collections, type of images and search terms

Truncation not supported Language of collections not clear Order of precedence in Boolean expression evaluation

collections visible to user or one large virtual collection?


Assessing relevance of search results

Order of image retrieval unclear Detail of retrieved images Local copies of image records Decision to buy image records



Support for Advanced search

Understanding meaning of index fields Information about individual collections Complexity of search expressions


Integration with user’s normal working environment

Speed of image retrieval and system reliability Saving retrieved images General usefulness


Micro usability issues

Consistency Appropriate user support Minimal user input Reduce user memory load Feedback Flexibility Navigational support


Navigation

Mechanisms inconsistent For example..

No obvious means of returning to the simple search page from the search results page

Clicking on buttons containing the numbers of hits to view the record list is not obvious

Requires consistent mechanism to move query – search – view sequence,

Consistency in object used, in labelling, in location


System feedback

System messages are too small in size appear in a non-prominent part of the page language and content is often very obscure

e.g ERROR: HTTP return error – Broker not found=?HTTP/1.1 500 Internal Server error..

empty return – contact the administrator - meaningless to most users


Login Facility

Location of command not prominent Feedback about outcome (success or failure) is poor Dialogue box persists on-screen with no visible change

during login attempt User encouraged to quit rather than logging off

Have separate login page


Documents

Case study - usability evaluation Howell Istance