What Do Usability Evaluators Do in Practice?An Explorative Study of Think-Aloud Testing

2

14 tænke højt test 7 virksomheder

3

A B C D E F G0

1

2

3

4

5

6

7

8

Arbejder med usabilityEvaluatorer til stede Observerede test

Virksomheder

4

Dataindsamling

Hypotese24 timer og 54 minutter

5

Dataanalyse

641 segmenter 35 nøgleord

6 grupper

6

Analysis of results from a test session

Analyse er ustruktureret 9Analyse er ufuldstændige 9Analyse som et sammendrag med brugeren3

7

Analysis of results from a test session

”Let’s sum up...”

8

Confirmation of known issues as a test focus

På udkig efter kendte problemer8Evaluatorerne havde forudset problemer5

9

Confirmation of known issues as a test focus

Person 1:

”I think we agree on many of the issues”

Person 2:

”Yes – I have already written the chapter, I just need the

ammunition”

10

Practical realities influencing tests

Tekniske problemer 8Ikke færdige prototyper 6

11

Practical realities influencing tests

”Now...Let us see...[searches in prototype paper

sheets] ...these are brand new, so I have not looked at them before”

12

Questions asked during a test

Problemer bliver forklaret, ikke oplevet 13Ledende spørgsmål 13Unødvendige eller indlysende spørgsmål 10

13

Questions asked during a test

”Did you notice this column?”

”What did you expect to see?”

14

Trying to meet labratory-style scientific standards

Evaluatorerne vil have lignende forhold for brugere 5Stive eller kunstige procedurer 3

15

Trying to meet labratory-style scientific standards

”We have to make sure that all the users

get the same questions”

16

Uncovering usability problems or utility concerns

Brugere peger på anvendelighed eller mangel på 10Evaluatorerne peger på anvendelighed problemer 7

17

Uncovering usability problems or utility concerns

”This is just to tell you that I would

not do it like this!”

18

19

System failure...

20

31 af 404 = 7,6 %