End-User Perceptions of Formal and Informal Representations of

Web Sites Jason Hong

Francis LiJames Lin

James Landay

Group for User Interface ResearchUC Berkeley

CHI 2001

April 5, 2001

Design Representations

Differing levels of detail

Low-fidelity (Lo-fi)High-fidelity (Hi-fi)

Static Images

Hi-fi representations– encourage discussion of details

fonts colors graphics pixel-level layout

Lo-fi representations– encourage discussion of high-

level issues information architecture navigation flow overall page layout

Interactive Prototypes

Hi-fi prototypes– use hi-fi representations

Lo-fi prototypes– use lo-fi

representations– often on paper– faster to create than hi-fi

prototypes

End-User Perceptions

How would end-users judge a web site design, depending on the fidelity of the site’s prototype?

Hypotheses

Between hi-fi and lo-fi prototypes– people will find many of the same usability problems

Hypotheses (cont’d)

Between hi-fi and lo-fi prototypes– people will judge the following attributes differently:

professional functional finished likely to change

Hypotheses (cont’d)

Between hi-fi and lo-fi prototypes – people will rank the importance of visual vs. non-

visual design issues differently visual issues: font, color, graphics non-visual issues: general location of navigation bars,

overall page layout, organization of information

Experiment Design

Each participant evaluated one site with hi-fi, other site with lo-fi

Within-subjects

Hi-fi Lo-fi Total

Site A 4 4 8

Site B 4 4 8

Total 8 8 16Be

twee

n-

sub

ject

s

Experiment Design

For hi-fi prototypes– downloaded two actual web sites– pared down product list– greeked some text out– hardwired shopping cart data

Experiment Design

For lo-fi prototypes– made electronic representations in DENIM instead

of using paper– exported DENIM pages to GIF + image maps– touched up sketches to improve handwriting– added HTML text fields by hand

Experiment Design

Participants asked to do 5 tasks per web site– find a specific piece of information– find another specific piece of information– find information deeper in site– add two particular items to shopping cart– purchase items in shopping cart

Experiment Design

After each web site, participants:– gave overall impressions of site design

(scale: 0–10) professional, finished, functional, likely to change

– ranked ten possible ways of improving web site 6 were non-visual improvements 4 were visual improvements

– made comments, suggested improvements to web site

Significant differences found(p < .05, 2 tailed independent t-test)

Evaluated each site separately– between-subjects analysis

Are the two web sites the same?

– between hi-fi sites attractiveness quality security likelihood of changing

– between lo-fi sites ease of use efficiency complexity attractiveness usefulness

Usability Problems

Many of the same usability problems were found in both hi-fi and lo-fi– e.g., Add to Shopping Cart / Registration problem

Lo-fi prototypes– sufficient for task-based usability evaluation– faster and easier to create

Perceptions of Design Attributes

People did judge attributes differently depending on fidelity

Lo-fi representations look:– less professional– less functional– less finished– more likely to change

Rankings of Visual and Non-visual Issues

Importance of visual vs. non-visual issues did not correlate with fidelity of prototype– electronic medium may raise expectations– quality of sketches may affect perception

Future Work

Study earlier stages of design cycle Task-based vs. non-task-based evaluation Investigate effect of medium on perception

– paper sketches– scanned paper sketches– electronic sketches– printed electronic sketches

Conclusions

Many of the same usability problems found in hi-fi and lo-fi representations

Lo-fi representations look:– less professional– less functional– less finished– more likely to change

No correlation between importance of visual vs. non-visual improvements and fidelity of representation

For More Information

http://guir.berkeley.edu/pubs/chi2001/formal-informal.pdf

Thanks!

Web Site Design Process

Experiment Design

After each task, participants were asked how well site supported task (scale: 0–10)– easy– efficient– clear– helpful

More descriptive text in links 6

Improved navigation bars 4

Clearer indicators of what can be clicked on

9

Clearer scheme for organizing pages on the web site

8

Better page layout 5

More streamlined shopping cart and checkout

10

More emphasis on web site brand 7

Better use of fonts 2

Better use of colors 1

Better use of graphics 3

Vis

ual

Non

-Vis

ual

More descriptive text in links 6 5.5

Improved navigation bars 4 5.5

Clearer indicators of what can be clicked on

9 9

Clearer scheme for organizing pages on the web site

8 9

Better page layout 5 5.5

More streamlined shopping cart and checkout

10 9

More emphasis on web site brand 7 5.5

Better use of fonts 2 2

Better use of colors 1 2

Better use of graphics 3 2

Vis

ual

Non

-Vis

ual

More descriptive text in links 6 5.5

Improved navigation bars 4 2

Clearer indicators of what can be clicked on

9 9

Clearer scheme for organizing pages on the web site

8 9

Better page layout 5 5.5

More streamlined shopping cart and checkout

10 9

More emphasis on web site brand 7 5.5

Better use of fonts 2 2

Better use of colors 1 2

Better use of graphics 3 2

Vis

ual

Non

-Vis

ual

7.25

2.875

Participant Informal FormalNon-Visual Visual Non-Visual Visual

1 7.25 2.875

2

3

4

5

Mean

Participant Informal FormalNon-Visual Visual Non-Visual Visual

1 7.25 2.875 6.667 3.75

2

3

4

5

Mean

Participant Informal FormalNon-Visual Visual Non-Visual Visual

1 7.25 2.875 6.667 3.75

2 6.083 4.625 6.083 4.625

3 7.25 2.875 6.083 4.625

4 5.5 5.5 3.75 8.125

5 3.75 8.125 4.917 6.375

Mean

Participant Informal FormalNon-Visual Visual Non-Visual Visual

1 7.25 2.875 6.667 3.75

2 6.083 4.625 6.083 4.625

3 7.25 2.875 6.083 4.625

4 5.5 5.5 3.75 8.125

5 3.75 8.125 4.917 6.375

Mean 5.967 4.8 5.5 5.5