Web browser usability test

Lim Zhen Ling Samantha

Ng Yan Ting

Wong Tak Wee

Google chrome• Developed by Google• Released in September 2008• Cool features

– Minimalistic design– Rearrange tabs in different

windows by drag and drop– No need to delete history:

Incognito mode– Google Suggest: All-in-one

address bar– No need for download manager:

All-in-one browsing tab– Top 9 web sites: Convenient

access to your recent/favourite web pages

Google chrome• Developed by Google• Released in September 2008• Cool features

– Minimalistic design– Rearrange tabs in different

windows by drag and drop– Privacy: Incognito mode– Google Suggest: All-in-one

search, history and address bar – No need for download manager:

All-in-one browsing tab– Top 9 web sites: Convenient

access to your recent/favourite web pages

– Highlighted textboxes

web browser usability testing

• Objectives • Pre-testing assessment

– Assessment criteria– Cognitive walkthrough– Pilot test

• Tasks selection: Final revision• Procedure• Results • Limitations • Improvements• Conclusion

Our objectives

• Improve usability of Google Chrome web browser for current and future users

• Create positive user experience

Our objectives

• Improve usability of Google Chrome web browser for current and future users

• Create positive user experience

• Objectives of HCI research (Shneiderman, 1987)↓ time of learning↓ rate of errors↑ retention over time↑ user satisfaction

Web browser assessment criteria

• Web performance• Security• User interface design/Usability

Web browser assessment criteria

• Performance• Security• User interface design/Usability

– Intuitiveness – Convenience– Accessibility

Cognitive walkthrough: Intuitiveness

Decreased learning time and visibility of functions

• Consistency• Informative feedback• Direct manipulation of

system• Interface real estate

management• Overall look and feel

Cognitive walkthrough: Intuitiveness

Decreased learning time and visibility of functions• Consistency ×

– With older, more familiar web browsers

Inactive mode

Active mode

√ Minimalist approach overloading of function buttons × violate intuitiveness

Google Chrome

Internet explorer

Firefox

√ Minimalist approach overloading of function buttons × violate intuitiveness

Cognitive walkthrough: Intuitiveness

Decreased learning time and visibility of functions• Consistency

– With older, more familiar web browsers

To open/close bookmarks bar: Right click

√ direct manipulation with the system inconsistent layout and function × low ‘visibility’ violate intuitiveness

Cognitive walkthrough: Intuitiveness

Cognitive walkthrough: Convenience

• Shortcuts• Easy reversal of actions• Reduce short term memory

load• Error prevention and

solving• User locus of control

Cognitive walkthrough: Convenience• Shortcuts• Easy reversal of actions• Reduce short term memory

load ו Error prevention and

solving• User locus of control

Minimalist design reorganisation of functions require learning ST memory load increases

Cognitive walkthrough: Accessibility• Important or commonly

used functions – Find “Help”

Cognitive walkthrough: Accessibility• Important or commonly

used functions

Minimalist design remove redundant coding search time increasesMinimalist design remove redundant coding area of access decreases time needed for selection increases

Cognitive walkthrough: Accessibility• Important or commonly

used functions • Bookmarks bar

Small area of access violate Fitts’Law time needed for selection increases

To close bookmarks bar: Right click

To open bookmarks bar: Right click

Tasks Selection : Final

Cognitive walkthrough pilot testing finalise tasks

Task 1

Go to http://www.hardwarezone.com, and stop the page before it finishes loading.

Press the Esc button on the keyboardClick on the button

× Low visibility

Task 1

Close the bookmarks bar, and after you have done so, show the bookmarks bar again.

Right-click on the bookmarks bar to close right click on the slit to open × Fitt’s Law

Go to “Tools”

Task 2

Task 2

Task 2

Make and export a bookmark

Drag URL address to bookmarks bar go to “Bookmarks manager” go to “Tools” click on “Export bookmarks”

Click on star icon to bookmarkGo to “Tools” click on “Export bookmarks”

× accessibility - muted suggests inactivity; low visibility

Task 3

Task 3

Task 3

Save a web page on to the desktop

Select “Document” “Save page as…” Check for file on desktop

Task 4

Task 4

Internet ExplorerMozilla Firefox

Locate the “Help” option.

Go to “Tools” Select “Help”

Task 5

Task 5

Participant Profile & Procedures

Participant Profile• 6 participants

• 3 males & 3 females• 1st Time/Non-Google

Chrome Users• NTU Undergraduates• 20~26 years old • Basic level of computer

and internet expertise

Tasks identified are generally common functions usedby participant group.

Procedures

• Venue• Tutorial Room 111 and 113• Quiet, controlled environment

Venue

Venue

Equipment

• Acer Aspire 4710• Google Chrome Version 1.0• MORAE 3• Debut Video Capture Software• Sony Cyber-shot Dsc-s600 • www.hardwarezone.com• Pre/Post-test Questionnaires• Video Release Form

Observation Methods• MORAE 3

– Screen Activity, Mouse Clicks• Debut Video Capture Software• Notetaker’s Guide

– Navigation Pathways, Facial Expressions, Success/Failure etc.• Questionnaire/Feedback Form

– Likert Scale– Open-ended

feedback

RESULTS

Findings

  Task 1 Task 2 Task 3 Task 4 Task 5

M1 2 2 1 2 2

M2 2 2 0 2 2

M3 2 0 0 2 2

F1 2 0 0 2 2

F2 1 1 2 1 2

F3 1 2 0 2 2

*Success rating: 0 – Not completed, 1 – Completed with difficulty or help, 2 – Easily completed

Success ratings

Findings

• Task 3 > Task 2 > Task 1 > Task 4 > Task 5

• Highest completion rate: Task 5– All participants completed it with ease

• Lowest completion rate: Task 3– Only 2 participants completed it

• Only 1 of them completed it with ease• The other completed with some difficulty or help

FindingsDifficulty ratings

Findings

• Emotional responses provide information about the tasks too– Whether MATCH with self-reported difficulty ratings

AND success rates

• Participants were confused by some of the tasks– Especially Task 2– And also Task 1 and 3

Results analysis

• By:– Task design – how it affects the performance and

success ratings

– User expectations – how user experience and knowledge affects performance and success ratings

Results analysis

• Task design– Certain tasks inherently more complicated –

demands a longer action sequence• E.g. Tasks 2 & 3

– Other tasks are much simpler – shorter action sequence of 1 or 2 steps only – better performance

• E.g. Tasks 1, 4, & 5

Participant ID

Task 1 Task 2 Task 3 Task 4 Task 5

Mean success rating

1.666667 1.166667 0.5 1.833333 2

Results analysis

Mean success ratings

Results analysis

• User expectations– Mental models developed from use of their

familiar web browsers• E.g. Internet Explorer, Mozilla Firefox, Safari

– Mental models would then affect performance of tasks on Google Chrome

• The smaller the discrepancy between Google Chrome and familiar web browser, the better the performance, the lower the difficulty rating

Results analysisParticipants’ familiar web browsers and Difficulty ratings

Difficulty rating

ID Familiar web browser Task 1 Task 2 Task 3 Task 4 Task 5 Mean

M1 Firefox 1 2 4 1 2 2

M2 IE 1 2 3 1 1 1.6

M3 IE 3 5 5 2 2 3.4

F1 Safari 3 5 5 2 3 3.6

F2 Firefox 2 2 1 3 1 1.8

F3 IE 3 4 5 1 1 2.1667

Average – Firefox 1.5 2 2.5 2 1.5 1.9

Average – IE 2.3 3.7 4.3 1.3 1.3 2.6

P.S. Safari is left out of the comparison due to its much lower % of use among web browser users.

Results analysis

• Task 1: Stop a web page from loading– Location of X button

• Firefox - Left• IE and Google Chrome - Right

– Consistency for IE users: similar location• IE users can more readily identify and navigate around

that area for “Stop’ option

Results analysis

• Task 1: Stop a web page from loading– Difference between IE and Google Chrome

• X button doesn’t double up as the load button, coloured in red (redundancy effect)

– IE users perform better?• No!• 2 participants who completed the task with difficulty or

help– 1 used Firefox, 1 used IE

Results analysis

• Task 1: Stop a web page from loading– IE closer to Google Chrome

• IE users SHOULD give lower difficulty ratings• IE users’ average: 2.3• Firefox users’ average: 1.5• Firefox users give lower difficulty ratings INSTEAD!

Results analysis

• Task 1: Stop a web page from loading– IE users:

• NO better performance• NO lower difficulty rating

– Explained by:• Pressing “Esc” button can also stop a page from loading

– Hence IE users who employ that action sequence, instead of clicking on “Stop” button, would not be expected to perform better because of consistency in design layout

Results analysis

• Task 2: Open / close bookmarks bar– Bookmarks bar

• Firefox: right-click on top options toolbar, and check/uncheck “Bookmarks toolbar”

• IE: no bookmarks bar/toolbar!

Results analysis

• Task 2: Open / close bookmarks bar– Firefox users are expected to have an advantage

since of the presence of similar “Bookmarks toolbar”

• No! - 2 users of Firefox and 2 users of IE completed the task

– 1 of the users of Firefox completed it with difficulty or help.

Results analysis

• Task 2: Open / close bookmarks bar– Firefox closer to Google Chrome

• Firefox users SHOULD give lower difficulty rating• Task 2

– Firefox users’ average: 2 – IE users’ average: 3.7

• Firefox users INDEED give lower difficulty ratings

Results analysis

• Task 2: Open / close bookmarks bar– Firefox users:

• Similar performance as IE users• Lower difficulty ratings

– Explained by:• Similar layout design – consistency

– Firefox users recognize Google Chrome’s bookmarks bar and its associated functions more easily

Results analysis

• Task 3: Make and export bookmark– Action sequence

• Firefox– Make bookmark: click on muted star– Export bookmark: go to “Bookmarks” option ”Organize

Bookmarks” ”Import and backup” “Export HTML”

• IE– Make bookmark: click on yellow star with a plus sign “Add

to favourites”– Export bookmark: click on yellow star with a plus sign

“Import and Export…”

Results analysis

• Task 3: Make and export bookmark– Google Chrome: action sequence similar to Firefox’s

• Make bookmark: click on muted star• Export bookmark: go to “Tools” button go to “Bookmarks

manager” highlight/click on bookmark click on “Tools” option select “Export bookmarks”

– Only 2 participants who managed to complete the task • Both Firefox users

Results analysis

• Task 3: Make and export bookmark– Firefox closer to Google Chrome

• Firefox users SHOULD give lower difficulty rating• Task 3

– Firefox users’ average: 2.5 – IE users’ average: 4.3

• Firefox users INDEED give lower difficulty ratings

Results analysis

• Task 3: Make and export bookmark– Firefox users:

• Better performance• Lower difficulty ratings

– Explained by:• Similar action sequence – consistency

– Firefox users hence more familiar with task

What are the things you liked about Google Chrome?

“Sleek look, nice aesthetics..”

“Stylish and simple in design…”

“Looks sleek and tidy…”

“…look sleek and clean..”

“Faster speed…”

What are the things that you did NOT like about Google Chrome?

“Slightly cumbersome in its usability, especially with regards to bookmarks..”

“Too difficult to find various functions…”

“No gadgets, add-ons..”

“…icon not coloured…”

Feedback

• What people liked:– Mainly looks, aesthetics– Level of emotion: Visceral

• What people disliked:– Navigation, usability– Level of emotion: Behavioral

Would you recommend Google Chrome to your friends?

0

0.5

1

1.5

2

2.5

3

Yes No Maybe

Usability criteria

• Intuitiveness– Consistency– Feedback– Overall look and feel

• Convenience:– Shortcuts– Easy reversal of actions – Reduce cognitive load– User locus of control: error prevention and feedback

• Accessibility

Scope and severity ratings

• Scope– Local problems– Global problems

• Severity 1 - prevents completion of a task

2 - causes significant delays in completing a task

3 - causes minor usability problems, but users can complete the task

4 - minor annoyance - does not significantly impact usability, but should be corrected if time allows.

Tasks Usability Problems No. Of participants affected

Severity Scope

2, 3, 4, 5 Participants face difficulty in completing tasks associated with options classified under “Documents” and “Tools”

4 1 Global

2, 3, 4, 5 Classification of task options was not intuitive i.e. participants could not find the export bookmark option and the functions for bookmarks bar under bookmarks manager

4 1 Global

3 Participants could not find the export bookmarks option

4 1 Local

1, 2, 3, 4, 5 Web browser window closes without warning 3 2/4* Global

2 Participants were unaware that bookmarks bar can be closed and opened via right click

6 2 Local

3 Participants could not identify bookmarking option

4 3 Local

1 Participants could not find the stop button without prompt

2 3 Local

2 Participants could not identify the bookmarks bar without prompt

6 3 Local

1 - prevents completion of a task2 - causes significant delays in completing a task3 - causes minor usability problems, but users can complete the task4 - minor annoyance - does not significantly impact usability, but should be corrected if time allows.* Depending on the task user was performing

RECOMMENDATIONS

4 out of 6 of the participants felt that Google Chrome’s minimalist design (larger web pages) improved their web browsing experience

retain its minimalist designImprove on its existing features based on feedback and current usability problems

Reorganisation of options

Reorganisation of options

Direct manipulation of bookmarks bar

Redundancy coding for “Documents” and “Tools” option

Google Chrome

Open these the next time I use Google Chrome

Do not show me this dialog again

Alert window

Bookmarks bar button/options

Stop button

CONFOUNDS AND LIMITATIONS

Limitations/suggested changes

• Participant selection– Small sample size– Population stereotypes

• Asians• University undergraduates

– Not representative of all users

• Practice effect• Testing environment and

procedure– Discrepancy with the real world

• Computer system• Usage conditions

– Presence of test administrator– Obtrusive equipment

• Repeat usability testing– in a different population

– reorder task sequence– Include practice tests prior to

actual testing

– Naturalistic environment

Web browser evaluation

User performance test Feature test Web compatibility test Configuration test Security test

√

CONCLUSION

Google Chrome with 3D

• Version tested: early stage of development

• Minimalist design aimed at the everyday user– Differs from Chromium– Relatively sound product

• Make adjustments to accommodate users accustomed to browser giants IE and Firefox

• Google Chrome team: make adjustments as we go along– Google Chrome Blog– Google Chrome Beta– Google Chrome with 3D