DFAIT Usability Final 6 Apr08

Usability testing of DFAIT website

Prepared for: Barry Nesbitt, DFAITContract number: 08170-070584/001/CY

Version 1.2 – April 16, 2008

Une version française du sommaire exécutif est disponible sur demande

Authors: Gord Hopkins, Mike Atyeo, Scott Smith & Paula Preston

Neo Insight usability evaluation - 2Copyright 2008 Neo Insight Inc.

Table of contents Executive summary 3

Task performance summary 19

Usability evaluation results 31 Consular 33 eDiscussion and START 56 Trade negotiations and agreements 74 International youth program 93 Russian mission site 113

Contact information 129


Introduction Background

The Department of Foreign Affairs and International Trade (DFAIT) continues to make changes to its websites with the goal of creating a unified web presence to better address the needs of both domestic and international clients

This work includes implementation of new design, navigation and content; plus development of a new information architecture

Objectives Test the usability of the current design and information architecture plus

some proposed wire frames Gather user feedback to provide direction for the future evolution of the

websites tested Test the ease with which users can navigate the websites to find

information, the suitability of the link labels and terminology, and the overall appeal of the website designs

Executive Summary


Method Recruiting

26 participants in each of 5 audiences were recruited and tested on different DFAIT content:

General: Adult, international traveller (Consular website) University: Policy students/professors (eDiscussion and START) Business: Involved in international trade (Trade Negotiations) Youth: Between 18-25 years (International Youth Program) Russian: New arrivals in Canada (Russia mission site)

Remote usability testing All participants were tested in their own environments (home or

office) using remote testing software (UserVue from TechSmith). Screen interactions and participant audio was captured for later analysis Participants were asked to try and complete 5 to 7 typical tasks Measured ability to complete tasks (with or without hints) and their level

of satisfaction with the way the website supported the task Testing took place from March 10th-14th, 2008 Participants were paid an honorarium for their participation

Executive Summary

For details, see DFAIT Facilitator Guide v1.0 Master 09Mar08.doc


Strengths – Across all areas testedArea of DFAIT Description of Strength

Consular Crisp, clear layoutGood search engineGood coverage of travel warnings and alerts

eDiscussion and START

Simple layout with clear headings for most sectionsSTART offers good imageryGood coverage of Canada’s activities abroad

Trade Negotiations and Agreements

Good access to wide range of international trade infoRegularly updated contentAbility to get notifications of updates

International Youth Program

Good collection of programs and associated countriesProvides access to opportunities of value to youthHelps expand understanding of international issues

Russian Mission site

Good info on working/studying/visiting/moving to CanadaExpands awareness of Canada’s cultureHelps people understand English terminology

Executive Summary


Scorecard Comparison of task completion, satisfaction and usability issues

Executive Summary

Area Critical Major MinorYouth 2 4 9

Consular 1 6 10

Russia 4 0 4

Trade 3 5 9

Policy 1 6 9

Task Completion Rate by Area Tested

0% 20% 40% 60% 80% 100%

Policy

Trade

Russian

Consular

Youth

Complete Complete with Hints Incomplete

Satisfaction Rating by Area Tested

0% 20% 40% 60% 80% 100%

Policy

Trade

Russian

Consular

Youth

Satisfied Marginal Dissatisfied


Consular summary

Executive Summary

Task performance overview 7 tasks tested – 1 Critical and 1 Major task problem which included:

Difficulty finding potential issues associated with travelling with dual citizenship Not finding warning information in Travel Report because some people did not

scroll

Usability overview 1 Critical, 6 Major and 10 Minor issues – Critical and Major included:

Content missed because it was not where expected Travel Report information missed by some users

Country list not visible without scrolling Table of contents overlooked

Too many competing and redundant links on the Home page Difficult to discriminate some sections

Key Information vs. Essential Resources vs. Quick Links Verbose and unstructured text in some areas


Consular key recommendations

Executive Summary

Key recommendations Re-examine information architecture in a few key areas to ensure content

is where it is expected to be Change layout of the Travel Reports country listing to ensure people see

the start of country listings without having to scroll Eliminate redundant links from the Home page and organize it into clearly

differentiated sections Build on the existing categories of before, during or returning from travel

Help people focus on the country of interest and the stage in their travel Provide links to Offices Abroad or Embassies on the Home page Visually integrate content on the right hand side to make it more

noticeable


eDiscussion and START summary

Executive Summary

Task performance overview 7 tasks tested – 1 Critical and 5 Major task problems which included:

Not able to find START program from main DFAIT Home page Not able to find specific funding details for Global Peace and Security Fund Difficulty locating information on how to submit a Policy Position Paper Difficulty finding the current eDiscussion and using the email alert form Difficulty finding video about Children at War Not able to access specific contact information about Sudan activities


No links to funding eligibility information Title of current eDiscussion difficult to find Policy Paper information not expected under eDiscussions No links from summary of Policy Position Papers to actual papers Annotation of Video Library link is misleading eNewsletter subscription form confusing to use Global Issues not an obvious link to look for the START program


eDiscussion and START key recommendations Key recommendations

Support key tasks associated with looking for funding information Provide easy access to Eligibility and How to Apply information

Highlight the current eDiscussion title more prominently Clarify the relationship between eDiscussions and Policy Position Papers

Change the Policy eDiscussions section to Policy Discussions to better accommodate the range of content found there

Policy Position Papers, Online Discussions, Video Interviews, etc. Provide cross-links to related information

e.g. – Link to How to Submit a Policy Position Paper from the summary listing of papers

Make link labels match user expectations regarding destination content Simplify the eNewsletter subscription form Re-evaluate where the START program is located or change the Global

Issues link label to something which would better convey the notion of Programs or Canada’s Involvement abroad

Ensure navigation breadcrumbs are used consistently throughout the site

Executive Summary


Trade negotiations summary

Executive Summary

Task performance overview 5 tasks tested – 2 Critical task problems which included:

Unable to associate finding information about business travel with Trade Negotiations and Agreements

Difficulty finding information about trade barriers


Business travel regulations not expected under Trade Negotiations link Business travel information inconsistent in different parts of website Trade barriers not associated with the link label of Market Access Report ExportSource and InfoExport terms do not communicate well Valuable right hand content and links often overlooked Links often not arranged for ease of scanning, nor in the user’s language Search was not working during testing


Trade negotiations key recommendations Key recommendations

Provide cross-links to business travel regulations on the Services for Travellers pages

Ensure consistency of information by taking a more database approach – maintain only one valid source for Business Travel Requirements which may be accessed from several locations

Consider changing the Market Access Report label to Trade and Investment Barriers

Replace internal terms, like ExportSource and InfoExport, with more meaningful terms

Visually integrate information in right hand column with central content Arrange links into bulleted lists so they can be scanned easily Ensure links are provided at the point where the content or resources

are described – e.g. Trade Barrier descriptions should link to database Reduce the amount of text and structure content for easy scanning

Executive Summary


International youth program summary

Executive Summary

Task performance overview 5 tasks tested – 1 Critical and 0 Major task problems which included:

Unable to find link to the International Youth Program from DFAIT home page


International Youth Program not expected under Foreign Policy section Program descriptions provide no links to more details or country opportunities Link to International Job Opportunities in Engineering goes to a generic

University website Home page Content being described is often missing links to actually get to the described

information Key information missing from French version of How Our Programs Work page Right side navigation is often overlooked


International youth key recommendations Key recommendations

Provide cross-links from the Travel and Business sections and consider changing the name to something like Work or Study Abroad

Link program titles on Program Descriptions page to more specific information about the program and participating countries

Allow people to choose a program of interest and then see what countries support it OR choose a country of interest and then see what programs are supported

Ensure links to resources are provided at the point when the resources are being described

Ensure the same information is available in both official languages Visually integrate right hand links so it is clear that they are related to the

central content Relocate the Country drop-down menu to a more noticeable position and

change the prompt to Select Country Place related links at relevant points throughout the sub-site rather than

grouping them under the vague label Other Links

Executive Summary


Russian mission site summary

Executive Summary

Task performance overview 5 tasks tested – 2 Critical task problems which included:

Difficulty getting to the Trade area of the website Difficulty determining what documentation is required for moving from

Armenia to Canada

Usability overview 4 Critical, 0 Major and 4 Minor issues – Critical included:

Link to Trade area of website is missing from main menu Key links to the Trade area of the site are not visible because they are

unexpectedly on the right hand side Not all English menu items are on the Russian-language version of menu Links on the right hand side are not evident to users


Russian mission site key recommendations Key recommendations

Put a link to the Trade area on the Russian-language version Map menu items exactly between the Russian and English versions to

facilitate people learning the corresponding English terminology to use Move key links from the right hand side into the central area and

visually tie in the content in the right hand column Put links to the Trade Commissioner service in the central area Ensure all links are evident to the users – underline if necessary Highlight the currently active link on the left hand menu when

appropriate Ensure the link labels correspond to the headings on the destination

pages

Executive Summary


Future directions Continue to develop a task-based structure to the websites

Structure text and lists of links so they are easy to scan Directly link to the resources being described Examine user’s steps in achieving their goals and help them complete

each step towards their goal Remove verbose descriptions and allow people to get to the actions as

quickly as possible Test all terminology for it’s ability to communicate as intended, especially

link labels – consider doing a Customer Carewords analysis www.CustomerCarewords.com

Further improve navigation and user experience Explore opportunities to integrate and simplify information, maintaining a

consistent look and feel across the multiple sub-sites Continue to refine the effectiveness of search by managing search based

on analysis of search logs and ongoing user testing

Executive Summary


Next steps

1. Address the critical and major usability issues from this testing

2. Address typical user experience issues1. Put the users’ most frequent and important tasks front and centre2. Move departmental content out of the way3. Reduce clutter by eliminating redundant links4. Adopt common style sheets to be used across all sub-sites5. Visually integrate the right hand content better with central content6. Provide context-sensitive and specific contact information7. Provide a search box on every page

3. Consider conducting a full Customer Carewords analysis www.customercarewords.com to optimize terminology, information architecture, and users’ task flow management

Executive Summary


Task performance

summary


Task performance summary The following tables and charts show participant performance and

satisfaction ratings for each of the audiences These tasks were chosen in consultation with the stakeholders to

represent the range of tasks the websites should support Critical, major or minor performance issues are highlighted by colour

Out of the 30 tasks investigated in this project, 18 exhibited some level of performance issue

7 were critical 6 were major 5 were minor

The recommendations found throughout the rest of this document are designed to address the usability issues observed

MINOR <80 % completed without hints

MAJOR <60 % completed without hints

CRITICAL <40 % completed without hints


Consular – Task listTask Label Task Description

5 Greek citizenshipCRITICAL ***

[Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.aspYou are an18 year old Canadian but your are eligible for dual citizenship because your parents are Greek citizens. Find out if there are any issues you should be aware of before visiting Greece.

1 Kenya travel MAJOR **

[Start from DFAIT home page] http://www.international.gc.ca/index.aspx You have heard about riots in Kenya and are worried about your friend who is currently travelling there. Find out what the current recommendations are for travelling in this area.

3 Guide for womenMINOR *

[Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.aspA woman you know is planning to travel abroad alone. Find a guide specifically designed to help women prepare for travelling abroad.

2 Poland officeMINOR *

[Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.aspYou are travelling in Poland and have had your passport stolen. Find out how to contact the Canadian authorities in Poland.

4 Romanian embassy [Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.aspYou need to quickly contact the Canadian Embassy in Romania. Find the contact details.

7 Travel with children [Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.aspYou are planning to travel outside Canada with your child, but without your spouse. Find out if there are any special arrangements which need to be made.

6 Drugs and travel [Start from Consular home page] http://www.voyage.gc.ca/consular_home-en.aspYour doctor recently prescribed some medications for you. Find out what you need to do to make sure you don’t run into any problems when travelling with them.




http://www.voyage.gc.ca/consular_home-en.asp

http://www.international.gc.ca/index.aspx







Consular – Task summary

Mean Time on Task (min) - Consular

0 1 2 3 4

OVERALL

Drugs and travel

Travel w ith children

Romanian embassy

Poland office

Guide for women

Kenya travel

Greek citizenship

Task Completion by Task - Consular

0% 20% 40% 60% 80% 100%

OVERALL

Drugs and travel

Travel with children

Romanian embassy

Poland office

Guide for women

Kenya travel

Greek citizenship


Satisfaction by Task - Consular

0% 20% 40% 60% 80% 100%

OVERALL

Drugs and travel

Travel with children

Romanian embassy

Poland office

Guide for women

Kenya travel

Greek citizenship



eDiscussion/START – Task listTask Label Task Description

7 START fundingCRITICAL ***

[Start from START home page] http://www.international.gc.ca/fac/START-GTSR/index.aspx?lang=enYou are working for an international organization focused on stopping human trafficking. Find out how much funding is available to support global peace and security this year and if you are eligible for any of this funding.

3 Email alertsMAJOR **

[Start from DFAIT home page] http://www.international.gc.ca/index.aspxYou’re interested in keeping up to date on discussions about international affairs. Find out what topics are currently being discussed and arrange to be notified whenever new information is added to the discussion.

4 Child war videosMAJOR **

[Start from DFAIT home page] http://www.international.gc.ca/index.aspxYou’ve heard about interesting videos on children in war. Find these videos to view.

5 Sudan infoMAJOR **

[Start from DFAIT home page] http://www.international.gc.ca/index.aspxYou would like to know more about Canada's reconstruction and peacekeeping efforts in Sudan. Find out who you would contact to get more specific information.

2 Policy paperMAJOR **

[Start from DFAIT home page] http://www.international.gc.ca/index.aspxYou’ve heard it’s possible to submit a policy position paper. Find out what format is acceptable and to whom you would submit it.

6 START achieveMAJOR **

[Start from DFAIT home page] http://www.international.gc.ca/index.aspxYou have been asked to write a report on Canada's Stabilization and Reconstruction Task Force (START) to better understand Canada's capacity to respond to international crises. Find a document describing this task force’s achievements over the past year.

1 Share ideasMINOR *

[Start from DFAIT home page] http://www.international.gc.ca/index.aspx You’re interested in sharing your thoughts on Canada’s foreign policies. Find out how to participate in any ongoing exchange of ideas and how to register so you can start contributing.




http://www.international.gc.ca/fac/START-GTSR/index.aspx?lang=en








eDiscussion/START – Task summary

Mean Time on Task (minutes) - Policy

0 1 2 3 4 5 6

OVERALL

Share ideas

START achieve

Policy paper

Sudan info

Child war videos

Email alerts

START funding

Task Completion by Task - Policy

0% 20% 40% 60% 80% 100%

OVERALL

Share ideas

START achieve

Policy paper

Sudan info

Child war videos

Email alerts

START funding


Satisfaction by Task - Policy

0% 20% 40% 60% 80% 100%

OVERALL

Share ideas

START achieve

Policy paper

Sudan info

Child war videos

Email alerts

START funding



Trade negotiations – Task listTask Label Task Description

1 Mexico documents CRITICAL ***

[Start from DFAIT home page] http://www.international.gc.ca/index.aspxYou know Canada has a business agreement with Mexico and you are flying to Mexico do some business. You know you need a passport, but what other formal documentation is required?

5 Trade barriersCRITICAL ***

[Start from TNA main page] http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu=You are a Canadian producer of Canola Oil and would like to know which countries currently have trade barriers which would impact your ability to export to those countries.

2 WTO panel [Start from TNA main page] http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu=You’ve heard that a World Trade Organization panel was set up to settle some of the disputes over US agricultural subsidies. Find out when this panel was established.

4 Update alerts [Start from TNA main page] http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu=How could you arrange to be notified by email of any updates to the Trade Negotiations and Agreements website.

3 Korea free trade [Start from TNA main page] http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu=You are interested in doing business with Korea. Determine the status of a free trade agreement with Korea and who to contact for more information.





http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu

http://www.international.gc.ca/trade-agreements-accords-commerciaux/index.aspx?lang=en&menu_id=9&menu


Trade negotiations – Task summary

Mean Time on Task (minutes) - Trade

0 2 4 6 8 10

OVERALL

Korea free trade

Update alerts

WTO panel

Trade barriers

Mexico docs

Task Completion by Task - Trade

0% 20% 40% 60% 80% 100%

OVERALL

Korea free trade

Update alerts

WTO panel

Trade barriers

Mexico docs


Satisfaction by Task - Trade

0% 20% 40% 60% 80% 100%

OVERALL

Korea free trade

Update alerts

WTO panel

Trade barriers

Mexico docs



Int’l youth program – Task listTask Label Task Description

1 Travel in HollandCRITICAL ***

[Start from DFAIT home page] http://www.international.gc.ca/index.aspx You are interested in traveling and doing a bit of work in Holland. Find out what opportunities are offered in that country.

3 IYP int’l jobsMINOR *

[Start from IYP home page]http://www.international.gc.ca/culture/123go/menu-en.aspYou are an engineering student looking for work abroad. Find out how many opportunities the International Youth Program has for you.

2 IYP eligibilityMINOR *

[Start from IYP home page]http://www.international.gc.ca/culture/123go/menu-en.aspYou are a permanent resident in Canada. Find out if you are eligible to participate in the International Youth Program.

4 Work in Belgium [Start from IYP home page]http://www.international.gc.ca/culture/123go/menu-en.aspYou are 31 years old. Are you eligible for a Working Holiday visa in Belgium?

5 SWAP comparison [Start from IYP home page]http://www.international.gc.ca/culture/123go/menu-en.aspWhat is the difference between SWAP and the Working Holiday Program?






Int’l youth program – Task summary

Mean Time on Task (minutes) - Youth

0 1 2 3 4

OVERALL

SWAP comparison

Work in Belgium

IYP eligibility

IYP int'l jobs

Travel in Holland

Task Completion by Task - Youth

0% 20% 40% 60% 80% 100%

OVERALL

SWAP comparison

Work in Belgium

IYP eligibility

IYP int'l jobs

Travel in Holland


Satisfaction by Task - Youth

0% 20% 40% 60% 80% 100%

OVERALL

SWAP comparison

Work in Belgium

IYP eligibility

IYP int'l jobs

Travel in Holland



Russian mission site – Task listTask Label Task Description

4 Trade sectionCRITICAL ***

[Start from Russia site home page]http://www.dfait-maeci.gc.ca/missions/russia-russie/menu-rus.aspFind out how to contact the trade section of the embassy.

1 Immigration docs CRITICAL ***

[Start from preferred search engine] I’m from Armenia and I want to know if there is any special documentation/visa requirement for moving to Canada.

5 Art exhibit [Start from Russia site home page]http://www.dfait-maeci.gc.ca/missions/russia-russie/menu-rus.aspFind out if there is a Canadian Art Exhibit currently showing in Russia.

3 Info sessions [Start from Russia site home page]http://www.dfait-maeci.gc.ca/missions/russia-russie/menu-rus.aspFind out if there are any information sessions you could attend to find out more about moving to Canada to live.

2 Embassy hours [Start from Russia site home page]http://www.dfait-maeci.gc.ca/missions/russia-russie/menu-rus.aspYou are in Russia, find the hours of operation of the Embassy in Moscow.

6 Russian embassy [Start from DFAIT home page]http://www.international.gc.ca/index.aspx From this page, you would like to get to the Canadian Embassy in Russia web site.




http://www.dfait-maeci.gc.ca/missions/russia-russie/menu-rus.asp



Russian mission site – Task summary

Mean Time on Task (minutes) - Russia

0 2 4 6 8 10

OVERALL

Russian embassy

Embassy hours

Info sessions

Art exhibit

Immigration docs

Trade section

Task Completion by Task - Russia

0% 20% 40% 60% 80% 100%

OVERALL

Russian embassy

Embassy hours

Info sessions

Art exhibit

Immigration docs

Trade section


Satisfaction by Task - Russia

0% 20% 40% 60% 80% 100%

OVERALL

Russian embassy

Embassy hours

Info sessions

Art exhibit

Immigration docs

Trade section



Usability evaluation

results


Ratings guide and prioritization Guide to ratings

Usability issues are prioritized and given a rating as either Critical, Major or Minor.

Rating system and colour coding of annotations

Usability issue either prevents the user from completing their task or causes significant confusion as to the operation of the user interface.

Usability issue results from significant inefficiency in completing common tasks (e.g. navigation, layout, terminology, inconsistencies, etc.

Usability issue does not prevent completion but does cause some inefficiency or annoyance with prolonged use.

Positive usability element which helps people accomplish their tasks efficiently, effectively, or with great satisfaction.

Other comment, suggestion, or opportunity for helping to make the website, product or service more useful and usable.

Critical

Major

Minor

Strength

Comment


Usability evaluation results

Consular


Section overview – Consular (1 of 4) Usability evaluation

1 Critical, 6 Major and 10 Minor issues

Summary of observations Strengths

Layout of page is clear and crisp Search engine worked well in most instances Clear, salient and visible warning indications

Critical usability issue Critical content is in different sections from that expected by users

Recommend: Move information to the sections expected by users or provide clear cross-reference links


Section overview – Consular (2 of 4) Summary of observations (continued)

Major usability issues Too many competing links on the Home page

Recommend: Minimize the number of links by creating clear areas of focus for different types of information

Difficult to discriminate several similar looking links on the Home page Recommend: Provide only one salient link on the Home page to Drugs and

Travel and one to Children and Travel Essential Resources section difficult for users to see in bottom right

Recommend: Remove redundancies and move Essential Resources to a more prominent location – possibly integrate into other sections

Travel Reports are being missed because nothing is visible above the fold Recommend: Reduce the amount of text before listing country names

People are not aware of Travel Report information because the Table of Contents is often not recognized Recommend: Integrate Table of Contents into the central content area

Redundant content can often hide valuable resources Recommend: Make descriptive text more concise, structured and bulleted to

facilitate scanning



Minor usability issues The Map and Travel Report drop-down menu on Home page were not well

utilized – makes it difficult to justify the value of the map given the large amount of screen real-estate it takes up Recommend: Eliminate the map and make the Travel Report drop-down menu

more salient, including possibly changing the name No link to Embassies or Offices Abroad from the Home page

Recommend: Provide a link on the Home page to Offices Abroad Popular Travel Reports section not used

Recommend: Reduce the amount of competing information on the Home page Obscure titles makes finding useful guides difficult

Recommend: Use descriptive titles for link labels – e.g. Travel Tips for Women rather than Her Own Way

Logos do not link to Home pages Recommend: Link logos to their respective Home pages



Minor usability issues (continued) Passport page is not task oriented

Recommend: Provide a short list of annotated links which take people directly to the content pertinent to their task

Long drop-down menus are difficult to use Recommend: List countries in a fashion similar to that used for the Country

Reports but arranged into 3 or 4 columns Colour mapping of recommendations does not follow standard convention

Recommend: Use conventional colour mappings for Cautions and Warnings Unstructured text difficult to scan for desired information

Recommend: Provide suitable sub-headings and provide easy-to-see links to additional detail

Drop-down menus not accessible via keyboard Recommend: Use a submit button to make drop-down menus more accessible


User comments – Consular Information of most interest to participants

Documentation requirements, country information, travel planning and being able to check requirements with a local embassy

Would come to DFAIT website if concerned about travelling to countries which may be in conflict or which adjoin countries at war but wouldn’t think to go to the website if travelling to Europe

Travel agent currently uses phone book to get contact information for local embassies

Other information needs Travel agent wanted to know more about transferring “odd items”, like

those needed for handicapped travel – may be more of an airline issue


Poor use of Home page real estate

Minor: Map not being used. The map provides good access to countries in each section but it is not being used. Only two people clicked on the map. It takes up a lot of valuable screen real estate for the amount of value it provides.Recommend: Eliminate the map and provide easier access to the more commonly accessed countries. If the map is kept, allow people to zoom into a specific region to continue their selection of a given country using the map.

Minor: Travel Report drop-down menu not used. This valuable short-cut was not seen or used by any of the participants.Recommend: Make this feature much more salient. The phrase Travel Report may not be communicating the best.

Minor: No link to embassies abroad. Several people expected to find a link to Canada’s offices abroad on the left navigation bar.Recommend: Provide a link on the Home page to Offices Abroad, including their contact information.


Too many competing links on Home page

Major: Too many competing links on the home page. People often had difficulty deciding on where to look for information on the Home page because very similar links exist in several locations.Recommend: Minimize the number of links by creating clear areas of focus for different types of information.

Comment: Why are travel reports and updates spread over a wide area of the Home page.

Strength: Dual coding of Travel Warnings using both colour and an asterisk helps all users, including those with colour perception anomalies.


Similar links vie for attention

Major: Difficult to discriminate what look like redundant links. People found multiple links to what they perceived to be the same information.These multiple links clutter the home page and make it difficult for people to choose which link to follow.Recommend: Provide only one link on the home page to information about Drugs and Travel and one about Children and Travel.


Essential Resources often not seen

Minor: Popular Travel Reports not used. No one made use of the link to the Kenya travel report under Popular Travel Reports.Recommend: Reduce the amount of competing information on the Home page.

Major: Essential resources lost on bottom right. People had difficulty finding links within Essential Resources. Many people dismiss right-hand content as being unimportant.Recommend: Remove redundancies and move Essential Resources to a more prominent location.

Minor: Obscure titles do not help people find information. No one was clear what the link Her Own Way would lead to.Recommend: Use more descriptive link labels – e.g. Travel tips for women. Make people more aware of valuable guides.


Search engine works well

Strength: Search engine worked well, returning a manageable number of hits.

Strength: Desired document was the first hit and other hits were relevant to issues with women travelling in general.


Country listing for Travel Reports not visible

Major: No Travel Report countries visible without scrolling. A couple of people thought this was just descriptive text and did not notice the country listings which were below the fold (not visible without scrolling). Two of them clicked on the Travel Reports and Warnings link in the left-hand navigation bar from this page, thinking they were in the wrong place.Recommend: Reduce the amount of text before the listing of country names. Also ensure that people are aware of what page they are on so they don’t click on a link to the same page. The link should not be active and it should be highlighted in some way to show it is the current page.

Minor: Logos do not link to home page. People expect to be able to click the logos to get to Home pages.Recommend: Links logos to the respective Home pages.


Passport page not task oriented

Minor: Passport page is not task oriented. People did not see link buried at the bottom of this page for dealing with a lost passport. The text shown does not support common tasks of determining eligibility, acquiring a passport, renewing a passport, reporting a lost passport, etc.Recommend: Provide a short list of annotated links which take the user directly to the information relevant to their task – in this case, reporting a lost passport.


Drop-down menu difficult to useMinor: Long drop-down menus are difficult to use. People were able to use the long drop-down menu of country names, but with difficulty. Recommend: Provide access to Government Offices Abroad using a listing more similar to that used for Country Reports (see below). One improvement would be to list the countries in 3 or 4 columns, rather than in rows with vertical separators.


Table of Contents not seen or used

Major: People not aware of breadth of information in Travel Report. Several people did not notice the Table of Contents. They started scrolling and, because it was so long, they left before they found the information they were looking for – e.g. Embassy contacts or Local Laws.Recommend: Put the Table of Contents in the middle with two columns. The left column should contain the most often sought after information. Combine Warnings and Recommendations with Safety and Security as people can’t discriminate them.

Strength: Good to have a very clear, salient and visible warning.

Minor: Colour of status message does not support the message. Blue was not associated with Caution. Recommend: Use conventional colours to emphasize the recommendations – like the red used in the lower example. In this case, make Caution an amber colour.


Redundant content hides valuable resources

Major: Redundant content can hide valuable resources. No one found this document on Dual Citizenship. It does an excellent job of explaining the Risks and Problems.Recommend: Make the descriptive text on the main page more concise and bulleted. Move the Dual Citizenship link from the bottom to the top of this section.

Comment: People expected this page to contain a list of countries in which Dual Citizenship can be a problem and provide links to the relevant information.


Text not structured for ease of scanning

Comment: It would be good to list some of the key conditions of which people need to be aware.

Critical: Information not where expected. Most of the people tested expected the information regarding dual citizenship issues to be under the heading of Entry/Exit Requirements.Recommend: Move this information to the Entry/Exit Requirements section.

Minor: Text difficult to scan for desired information. People often had trouble finding the relevant information in this section because there is so much unstructured text.Recommend: Provide suitable sub-headings regarding things like Legal Rights, Drug and Alcohol Laws, Compulsory Military Service, etc. Instead of simply stating “You are subject to local laws”, it would be better to provide a listing of some of the major differences and a link to more detail.


Drop-down menus not keyboard accessible

Minor: Drop-down menus not keyboard accessible. People trying to one of these drop-down menus using just the keyboard can only access the first entry – Afghanistan.Recommend: This is not a major issue because there are alternate ways of getting to the same information. These drop-down menus could be made accessible by requiring the use of a Go or Submit button.


Highlight videos – Consular Highlight video descriptions

Difficulties with finding the link to Canadian offices abroad First person (travel agent) goes to Passports and misses the link to

Canadian offices abroad. She then tries While Abroad and Problems. She finds Passport Services Abroad and then clicks on Canadian offices abroad the second time around.

Second person goes to the Travel Report for Poland and clicks on Canadian offices abroad. He does not notice that the Travel Report has the information he’s seeking.

Third person reads right over the Canadian offices abroad link but is unsure where to go next. She looks at Loss of Belongings and Passports and then wanders aimlessly. After a prompt she finds the Canadian offices abroad link.

View video (Consular-PassportStolen.wmv – 5:56)



Problems interpreting and using Travel Reports First person (travel agent) goes to Travel Reports and then clicks on

Travel Reports again, not aware she is already there. Second person goes to Travel Reports and doesn’t notice any country

listings (they are just off-screen). When asked to return to the page, she sees the country listings but doesn’t think they will provide the information about potential dangers in travelling in a particular country. She starts searching a variety of other locations but is very unsure of where to look.

View video (Consular-TravelReports.wmv – 3:46)



Difficulties finding Contact Information First person misses Contact Info link on the main page even though it

is there. Second person misses the Contact Info link on the main page of the

Romanian Embassy and then can’t find it on any other page. Third person has difficulty finding Contact Info for the Romanian

Embassy. He doesn’t see the Contact Info in the Travel Report. Instead he clicks on Canadian offices abroad and then has to get back to Romania.

View video (Consular-ContactInfo.wmv – 3:18)



Difficulty find publication for women travelling alone First person thinks publication should be renamed using more

meaningful terms. Second person starts scanning a variety of links. She goes to Before

You Go but is surprised she can’t find anything. She tries FAQ and Quick Links, then starts to get frustrated.

The third person, even after a hint, cannot find Essential Resources or the Her Own Way publication.

View video (Consular-HerOwnWay.wmv – 5:22)



Difficulties trying to find any issues with citizens travelling to Greece Person looks under Dual Nationality but encounters difficulties once she

starts looking under Foreign Offices in Canada. View video (Consular-GreekCitizens.wmv – 1:34)



eDiscussion and START


Section overview - eDiscussion (1 of 4) Usability evaluation



eDiscussions page has simple layout with clear headings for most sections START offers good imagery and coverage of Canada’s activities abroad

Critical usability issue No links to information about eligibility or how to participate in funding programs

Recommend: Provide readily accessible links to Eligibility and How to Apply information

Major usability issues Title of current eDiscussion difficult to find

Recommend: Move current eDiscussion to the top of page and give it a sub-heading under Policy eDiscussions

Policy Paper details difficult to find – not expected under eDiscussions Recommend: Consider changing the section name to Policy Discussions with sub-

sections for Online Discussions, Position Papers, Videos, etc.


Section overview - eDiscussion (2 of 4) Summary of observations (continued)

Major usability issues (continued) No links from summary of Policy Position Papers to actual papers

Recommend: Provide links to relevant resources, like a list of Policy Position Papers, how to submit a Policy Position Paper, etc

Annotation associated with Video Library is misleading Recommend: Change link label to something more generic, like Video clips on a

variety of international issues eNewsletter subscription form is confusing

Recommend: Clarify and simplify form to make it clear which fields and buttons to use

Global Issues not an obvious link to find the START program Recommend: Consider a label change to Foreign Policy Programs or Canada’s

Involvement or something similar


Section overview – eDiscussion (3 of 4) Summary of observations (continued)

Minor usability issues Relationship between eDiscussions and Policy Position Papers is unclear

Recommend: Clarify the relationship between Policy Position Papers and eDiscussions

Key words for some video titles are not in the primary scanning area Recommend: Place key words at the beginning of the title – e.g. Children in War –

Monitoring and Reporting on Violations Difference between Foreign Policy and Canada in the World is unclear

Recommend: Consider changing sub-link labels to help differentiate the two – e.g. Activities by Country and Region

Contact information is non-specific to content areas Recommend: Provide specific, context-sensitive contact information rather than

general enquiry contact information Active Home links on the Home page creates confusion

Recommend: Have only one Home link and make it inactive when the user is on the Home page


Section overview – eDiscussion (4 of 4) Summary of observations (continued)

Minor usability issues (continued) Sub-sites change look and feel plus link to different Home pages

Recommend: Maintain a consistent look and feel through the website and sub-sites

eNewsletter link leads to an unexpected page regarding RSS and Travel Recommend: Ensure eNewsletter subscription leads to simple subscription page

and provide extra links to RSS or Travel information Navigation breadcrumbs disappear on some pages

Recommend: Provide consistent breadcrumbs throughout the website Multimedia link is associated with more than just Podcasts

Recommend: Change link label to Podcasts and provide cross-links to other multimedia content


User comments – eDiscussion eDiscussion landing page comments

Clear headings segment sections well Good to have University-specific link Would like to see more University and NGO content Don’t like that visited links fade and are no longer obvious

START landing page comments Not so interested in news as in links to content News content makes sense Didn’t notice that red blocks on right were links, thought they were titles Would like information on how to get access to funding Assumed there would be brochure describing the Peace and Security

Funding Program


Position Paper not expected with eDiscussionMajor: Current eDiscussion not visible enough. A few people had difficulty finding, or did not find, this information about the Current eDiscussion.Recommend: Move this information to the top and give it a sub-heading under Policy eDiscussions. Put the subscribe links under the Participate header.

Major: Policy paper not thought of as part of an eDiscussion. People had difficulty finding how to submit a Position Paper because they did not associate a physical paper with an eDiscussion. They thought of the latter was only onlineRecommend: Consider changing the section name to Policy Discussions with sub-sections for online discussions, position papers, videos, etc. Also highlight the phrase Policy Position Paper and make it a link, similar to the way Participation Guidelines are handled under Participate.


Missing link from summary to specific papers

Major: No links from summary of policy position papers. People were surprised to find no links from this summary page to the specific Policy Papers or to how to submit a Policy Paper.Recommend: Provide links to relevant resources – e.g. list of Policy Papers, how to submit a Policy Paper, etc.

Minor: Relation between eDiscussions and Policy Position papers unclear. People did not understand how eDiscussions (thought to be an online discussion) related to Policy Position Papers. They did not think the two were related.Recommend: Clarify the relationship between Policy Position papers and eDiscussions.


Video annotation misleading

Major: Annotation confusing to some users. Half the people did not expect to find videos regarding Violations against Children in War in this section because they did not associate this type of content with foreign policy thinkers.Recommend: Change annotation to something more generic, like Video clips on a variety of international issues.

Strength: Most people saw this link quite quickly.


Key words not in primary scan area

Minor: Key words not in scanning area. People often missed finding these videos on Children in War because the title is quite long and the key words are at the end of the sentence. People often scan only the first 3 or 4 words on the left.Recommend: Place key words at the beginning of the title – e.g. Children in War – Monitoring and Reporting on Violations.

Strength: It is good to group a number of common interviews under a smaller set of themes.

Comment: If this list continues to grow it might be worthwhile providing an index to the themes or providing a simple find feature, just for the videos.


Contact information not specific

Minor: Contact information is non-specific. People were expecting to find someone to contact regarding Canada in Sudan but instead got a generic list of contacts (see below).Recommend: Provide context-sensitive contact information to help people find the appropriate resources for a given topic.

Minor: Change of website unexpected. People were confused by this different looking website which still seemed to be dealing with Foreign Affairs.Recommend: Maintain a consistent look and feel throughout the website and sub-sites.

Minor: Home links to current page. People were confused when they clicked on Home and nothing happened. They did not expect this to be the Home page.Recommend: Have only one Home link and make it inactive when the user is on the Home page.


Subscription form confusingMinor: Destination page does not match user expectation. People expected to see a form to sign up for an eNewsletter but encountered RSS feeds and Travel information instead.Recommend: Ensure the title and content of the destination page match the link label.

Major: Unclear as to which form fields to use. Several people had difficulty interpreting this form. They often didn’t see the button at the bottom of the page. Two people thought the second email field was for Unsubscribing because of the text above it.Recommend: Clarify the form to make it more evident which fields and buttons to use.

Minor: No breadcrumbs available. People expected consistent breadcrumbs.Recommend: Provide consistent breadcrumbs.


Multimedia link implies more than podcasts

Major: Global Issues not an obvious link to get to START program. Only two people found the START program under Global Issues without a hint. The others thought it might be under Canada in the World. They did not associate Global Issues with programs.Recommend: Consider providing a link to Foreign Policy Programs or changing the link label to something like Canada’s Involvement.

Minor: Multimedia link associated with more than just Podcasts. Some people looked under the Multimedia link to find videos and were surprised there were none, nor links to video resources.Recommend: Change the link label to Podcasts if that is all it contains and provide cross-links to other multimedia.


No links to participation informationStrength: Details about the program and how to apply are readily available.

Critical: No links to how to participate in program. People could not find any information on program eligibility or how to apply for funding for the GPSP or GPOP programs. There is a lot of description but no links for people to take action.Recommend: Provide readily accessible links to help people determine their eligibility for the program and how to apply for funding.


Distinction between major sections unclear

Minor: Difference between sections unclear. A few people were confused as to the difference between the section on Foreign Policy and that of Canada in the World. Recommend: The distinction may be made clearer by changing some of the sub-link labels – e.g. Activities by Country and Region.


Highlight videos – eDiscussions and START Highlight video descriptions

Can’t find current eDiscussion Person scans the page but still cannot find the current topic of

discussion. View video (Policy-CurrentDiscussion.wmv – 1:15)

Difficulty using the subscription page for the email alerts Several people have difficulty interpreting the page for email alerts.

They are unsure which fields they need to fill in. View video (Policy-eNewsSubscription.wmv – 1:15)

Policy Position Papers not expected with eDiscussions People comment on not expecting Policy Position Papers being

associated with eDiscussions. They think the later is strictly online and does not apply to submission of a paper.

View video (Policy-PositionPaper.wmv – 2:12)



Video annotation is misleading Several people have difficulty finding videos of Children in War because

the annotation misleads them. It talks about interviews from foreign policy thinkers which they do not associate with Children in War.

View video (Policy-VideoAnnotation.wmv – 2:36) Media Room confused with multimedia content

Person goes to Media Room looking for multimedia content. She was expecting audio/visual content.

View video (Policy-VideoLocation.wmv – 1:20)



Difficult to find START program People are not expecting the START program to be under a link called

Global Issues. They associate Global Issues more with news and current affairs.

View video (Policy-FindingSTART.wmv – 1:18) New windows and different look unexpected

Person is not expecting a new window to open with something that looks like a whole new web-site. She is trying to find out about Canada’s involvement in Sudan.

View video (Policy-SudanSubsite.wmv – 0:26) Multiple Home pages can be confusing

Person is confused by the Home link taking her to the same page she is already on. She expects it to take her to the DFAIT home.

View video (Policy-HomeConfusion – 3:08)



Trade negotiations


Section overview – Trade (1 of 4) Usability evaluation



Provides access to a wide range of information affecting international trade Regularly updated with ability to get notified of updates

Critical usability issues Business travel regulations not expected under Trade Negotiations link

Recommend: Provide a cross-link to business travel information from the Services for Travellers pages

Business travel information inconsistent in different parts of website Recommend: Ensure consistency of information found in different parts of website

and put information where people expect to find it Trade barriers not associated with the Market Access Report

Recommend: Consider changing name to Trade and Investment Barriers


Section overview – Trade (2 of 4)

Summary of observations (continued) Major usability issues

Link labels like ExportSource and InfoExport do not communicate well Recommend: Create more meaningful link labels which communicate the destination

content better Valuable right hand content and links are often overlooked (x2)

Recommend: Explore ways to integrate the right side content with the central content Links often difficult to scan and not in user language

Recommend: Arrange key links in a bulleted list in the central scan area of the page Search was not working during the test sessions

Recommend: Fix search engine and provide more helpful error messages which clearly state the reason for a problem and how to correct the situation


Section overview – Trade (3 of 4) Summary of observations (continued)

Minor usability issues Difficult to read and discriminate multi-line link labels in left navigation

Recommend: Increase spacing between link labels or use a fine rule line Confusion about Export and Import Controls link on left and right

Recommend: Make the relationship clear between links which are repeated on a page

Several pages have active links back to the page the user is on Recommend: Do not provide an active link to the current page

Logo does not link to Home page as expected Recommend: Link the logo to the Home page

Descriptive text often does not provide links to content being described Recommend: Provide active links at the point the user is reading about the

content or resource provided


Section overview – Trade (4 of 4) Summary of observations (continued)

Minor usability issues (continued) Key words are buried in prose and neither highlighted nor linked

Recommend: Highlight key words and structure text for ease of scanning Fields required for Trade Barriers search are unclear

Recommend: Clarify instructions so people start with the minimum input required Some pages are quite verbose with little structure to support quick scans

Recommend: Make text more concise and use sub-titles and bulleted lists to structure the text for easier scanning

Table of Content links in the right column are often not seen Recommend: Move the On this Page links into a more central area


User comments – Trade Comments on Trade Negotiation and Agreements landing page

Perception that home page is cluttered and contains of a lot of text “Le site est lourd”, “La page est chargée”

Would like to see a search box Navigation breadcrumbs are good to have Would like to see Archives to see how trade and negotiations have

changed over time with different countries Right hand topics are useful but graphics take away from importance News in central area is good to see how things are changing Titles are often not related to what you find Surprised to see Guide for Canadian Municipalities under International

Trade Agreements “Links on left is what I choose from, not sure of what is on the right”

Assumes content in centre changes rapidly, content on right less so Thinks elements on the right are “flavour of the month” things

Trade Matters is a very vague phrase


Link labels ambiguousMajor: Right hand content often overlooked. Several people did not see the content on the right hand side. They did not see Dispute Settlement even when asked to scan the page for anything related to Dispute Settlement. They commented that they typically avoid the right hand column as it is usually advertising or peripheral information.Recommend: Explore ways to integrate the right side with the central content more. Reduce the use of images on the right as they often lead to the impression that this is just advertising for other sites.

Minor: Difficult to read link labels. It was unclear to users where a link label started and ended without rolling the mouse over the link. Recommend: Increase the spacing between link labels a bit or use a fine rule line between them.

Major: Link labels are ambiguous. These link labels often caused confusion. People were unsure what they would find under ExportSource or InfoExport. They also thought that Export and Import Controls covered more than just Canada.Recommend: Create more meaningful link labels which give people an accurate expectation of the underlying content.


Multiple links with same name confusing

Minor: Confusion about difference between Export and Import Controls on left and right. Several people queried why there was something about Export and Import Controls on both the left and right. They were unsure if they were the same. Recommend: If the intent is to elaborate on the content behind a link label, then create a direct mapping to the left navigation bar and explain each of the links in order. This way people will see the relationship.


Business travel info not expected under Trade

Critical: Unexpected to look under Trade Negotiations and Agreements for business travel information. Not one person thought to look under Trade Negotiations and Agreements for information about business travel to Mexico. Even after a hint, people felt they were in the wrong area when they got to NAFTA. All people initially wanted to look under Services for Travellers (see next page).Recommend: Provide a link to this information from the Services for Travellers content pages.


Travel information inconsistentCritical: Information is inconsistent with respect to business travel. Information from the NAFTA section (top left) provides a listing of documentation required for a business traveller. However, it appears to differ from that provided on the voyage.gc.ca site (bottom right). The latter seems to imply the need for business, work and visitor visas plus a tourist card. Recommend: Clarify exactly what is required for business travel in both places and ensure the information is consistent. Provide a clear separation of business and pleasure travel requirements. People expect to find this information in the Entry/Exit Requirements section of the Mexico Travel Report.


Right hand links difficult to scan

Major: Links on right difficult to scan and not in user language. Some people had difficulty finding the link to Cross Border Movement of Business Persons. This is not the first area they scan (they scan the centre first) and they had to interpret that this link was relevant to business travel regulations.Recommend: Replace the centre description of the topics with the links on the right. Each link can then be put on a single line, making them easier to scan. Remove all right side links. Reword Cross Border Movement of Business Persons to something like Business Travel Regulations.

Minor: Active link to current page. People get confused when they click on an active link and nothing seems to happen.Recommend: Remove this as an active link when the user is already on this page.


Search not working

Major: Search not working. Several people wanted to use search but it always returned an error condition during testing. The error message provided no help to the user as to what had gone wrong or how to correct the situation.Recommend: Fix search engine. If an error does occur, ensure the person understands what kind of error has occurred and what they can do about it, if anything.

Minor: Logo does not link to home page. People would often click the logo and expect it to link to the DFAIT home page, but it does not.Recommend: Link the logo to the Home page.


Trade barriers & Market Access not associated

Critical: Trade barriers not associated with the link International Market Access Report. Four out of five people could not find information on trade barriers without a hint.Recommend: Consider changing the name to Trade and Investment Barriers. Most of the text references this and it is closer to the language of the user.

Major: Right menu items only seen as last resort. People typically scanned the central content before noticing the links on the right.Recommend: Integrate access to the database and reports into the central content.


Descriptions with no links to action

Minor: Description of trade barrier database but no link. People usually scanned the central content and found the tool they were looking for but could not access it. There were no active links in the central area. They had to look elsewhere. Recommend: Provide a clear sub-title for the database and make the title an active link to the database.

Minor: Key words are not highlighted or linked. People are scanning for trade or investment barriers, not a database. Recommend: Highlight the key words and the action – e.g. Search Trade & Investment Barriers. Also make this the link to the database so people don’t have to look elsewhere to find the link. The same recommendation applies to the modifications which were made after testing (see below).

Comment: More… is ambiguous. Make the title the link and eliminate the need for this more… link.


Required form fields unclear

Minor: Required fields unclear. Half the people tested tried to fill in a Sector and Product Code when they were not needed.Recommend: Clarify the instructions so people start with the minimum information first. This is to prevent them from overly constraining the search and not finding anything because of a mis-categorization of the Sector or Product Code.


Verbose text makes it difficult to scan

Minor: Right side links not used. People did not see the links on the right which would have got them directly to the contact information. Instead they had to scroll 5 or 6 pages to find the information. Two people gave up before they reached that point.Recommend: Move the On this page links into a more central location.

Minor: Page is quite verbose with little structure to help scanning. People want to be able to quickly scan the page and pick up the key information but they found this page to be too long and dense. Recommend: Tighten up the text and make more use of sub-titles and bulleted lists to organize the key points.


Opportunity to elaborate on link labels

Comment: There is an opportunity to use this page real estate to annotate the links which are also found on the left. This would help to disambiguate the content associated with each link. There is also an opportunity to provide sub-links. These would help to define the parent link content and provide direct access to some of key information in each area. This is especially important for some of the more generic links, like Goods, Services and Further Opportunities.


Highlight videos – Trade negotiations Highlight video descriptions

Difficulty finding business travel document requirements Several people do not expect to find information about business travel

requirements under Trade Negotiations and Agreements. One person thinks he’s in the wrong place when he gets into the NAFTA content on the site. Another person has difficulty navigating the information on the destination page for Cross Border Movement of Business Persons.

View video (Trade-BusinessTravelRegs.wmv – 6:53)

Difficulties finding Trade Barrier information People scan the central content but cannot find links to the content

they want. They comment on Market Access Report not being an obvious choice for getting at Trade Barriers. They suggest using the term Trade Barriers instead of Market Access. The last person expresses frustration at not being able to find anything to do with Trade Barriers.

View video (Trade-Barriers.wmv – 3:08)


Highlight videos – Trade negotiations Highlight video descriptions

Export/Import label confusions People comment on not knowing the differences between

ExportSource, InfoExport, and Export and Import Controls. People also question the difference between Export and Import Controls links on the left versus the right.

View video (Trade-ExportImportConfusion.wmv – 1:51)

Right hand content often missed People have difficulty finding link to information on Dispute Settlement

even though it is in the right column. One person contends a dispute should not be under a section about agreements.

View video (Trade-RightSideNotSeen.wmv – 3:08)

Circular links back to the current page Person explains his confusion about clicking on a link which seems to

do nothing. He finally realizes he has gone back to the same page but can’t figure out why the link would be active.

View video (Trade-LinkLooping.wmv – 1:33)



International youth program


Section overview – IYP (1 of 4) Usability evaluation



Simple collection of programs and participating countries List of opportunities around the world which appeal to youth

Critical usability issues International Youth Program was not expected under Foreign Policy

Recommend: Provide cross-links from Travel and Business sections and possibly change the name to Work or Study Abroad

No links from the Program descriptions to details or participating countries Recommend: Link titles of program descriptions to specific details regarding the

program and which countries participate


Section overview – IYP (2 of 4)

Summary of observations (continued) Major usability issues

Link related to specific content goes to a generic Home page instead Recommend: Provide a link directly to the listing of International Jobs on University

website Textual descriptions of actions to take but no links to help user take action

Recommend: Provide “deep links” to the detailed information or appropriate resources at the point when the information is first presented

Key information missing from French version of How Our Programs Work page Recommend: Ensure the same information is presented in both official languages

Right side navigation is being missed by some people Recommend: Visually integrate right hand links with central content to make the

relationship to the current content clear


Section overview – IYP (3 of 4) Summary of observations (continued)

Minor usability issues Country drop-down menu not seen or used

Recommend: Position the menu below the IYP header and change the prompt to read Select Country

No direct links provided to the described content (x3) Recommend: Provide direct links from the descriptive text to the associated

content and/or actions Active link to current page makes it look like link does nothing

Recommend: Do not have an active link to the current page and provide an indication of which link corresponds to the active page

Link labelled Other Links is not very meaningful Recommend: Place related links at relevant points throughout the sub-site rather

than grouped under a generic title – e.g. link to Passport Office when talking about the requirement for a Passport


Section overview – IYP (4 of 4) Summary of observations (continued)

Minor usability issues (continued) Difficult to compare program descriptions – not consistently formatted

Recommend: Provide a matrix so people can compare programs across a number of common dimensions

Common synonyms for countries are not provided in country list Recommend: Provide common synonyms for country names in list

FAQ list is long and unstructured Recommend: Group related questions together and label the grouping


User comments – IYP Appropriateness for youth

Image gives good idea of lifestyle A couple of people felt more imagery of youth having fun in typical

situations abroad would be more engaging Current picture shows older people and looks too serious

Not sure what Choose in drop-down menu means Prefer websites which are not too flashy – buttons are easy to read Would like menu across top with drop-down menus Not sure why print ads are on the website Would like information in a more straightforward manner – have to

rely too much on ride side menu bar Would like search box on each page


International Youth Program difficult to find

Critical: Location for International Youth Program unclear. 80% of people could not find where to look for information about working holidays. No one expected it to be under Foreign Policy. They tended to want to look under Services for Travellers.Recommend: No one expected to find this kind of information on the DFAIT website. We have no clear recommendation on how to address this issue other than through media promotion and changing the link name to Work or Study Abroad. Cross links could also be provided from the voyage.gc.ca and Services for Business sections.


Right side navigation often missed

Major: Right side navigation is missed by some people. About half the people ignored the right hand navigation and focused on the left hand navigation and centre. This led some of them to leave the IYP pages and go to sections like Employment and Training, which is further down on the left.Recommend: Links on right need to be more visually integrated with central content. It must also be clear that these links are the local navigation for the current topic area.


No direct links to described contentMinor : Drop-down menu not seen or used. Not one person noticed or used the drop-down menu shortcut to countries. Even if someone noticed it, the word Choose did not communicate it’s use.Recommend: Position the drop-down menu below the header for the International Youth Program on the right or move it to the centre. Change the prompt from Choose to Select Country.

Minor : No direct links provided to described content. People read the centre area but could not directly link to the information of interest. They had to look elsewhere to find the associated links. Recommend: Provide direct links from the descriptive text shown here to the associated content.

Strength: The image worked well at conveying the kind of experiences provided by the programs.


Active link to current page

Minor : Active link to current page. People did not think this was the Home page for the International Youth program. They clicked on this link even though they were already on the page. Recommend: Do not create multiple Home pages. If this is a main page, then name it accordingly in the page heading in the centre. Ensure the link label uses the same wording. Ensure there are no active links to the current page. Provide an indicator in the navigation bar to show which link is currently active.

Minor : Other Links not very meaningful. People had no expectations about what they would find under Other Links.Recommend: Place related links at the appropriate point within the pages of this sub-site. For example, link to Passport office when talking about the requirement to have a passport.


Link to generic home page rather than specific

Major: Link goes to generic home page instead of specific content. People had difficulty following up on the link to Engineering positions abroad because the link goes to the generic UBC Engineering Home page, not the ICE relevant content.Recommend: Provide a direct link to the listing of International Jobs rather than making people figure out how to navigate a new website.


No links from program titles to details

Minor : Difficult to compare working holiday programs. People had difficulty determining the essential differences between the Working Holiday Programs versus the SWAP Working Holiday program. They were not together, nor were the descriptions consistent in terms of describing the what, who and why of the programs. Recommend: Consider providing a matrix so people can see how the various programs compare on a number of different dimensions.

Critical: No links to programs. People expected to be able to link to the program to find more details about eligibility, how to apply, etc.Recommend: Make the titles links to the details related to each of the programs.


Description of process but no action links

Major : Text description of what to do but no links to take action. People could read about how the programs work but there were no embedded links to help them take the actions described, such as finding a job in the country, applying for a visa or contacting the country’s embassy.Recommend: Help people not only understand what process is involved but help them take action by providing “deep links” directly to the appropriate resources, forms, etc.

Comment: Most of the people coming to this page will not care about any of this first paragraph. They will want to find out what programs are available, if they are eligible, and how to proceed.


Key information missing from French page

Major: Key information missing from French version of page. French participant was not able to complete the task regarding eligibility requirements because the French version of the page was missing some critical information presented on the English version.Recommend: Ensure the same information is conveyed in both languages.


No active links from program/country listing

Minor : People looking for other common names for country. A couple of people missed Netherlands because they were scanning for Holland.Recommend: Provide common alternate names for countries, when they exist – e.g. Netherlands (Holland).

Minor : No active links in listing of programs and participating countries. When people found the combination they wanted in the description – e.g. working holiday in Netherlands – they could not directly link to that country’s program. Recommend: Make the country names at the top link directly to the information about the specific program and country combination. List the countries in 2 or 3 easy-to-scan columns under each type of program.


FAQ list long and unstructured

Minor : List is long and unstructured. People found it inefficient to scan the FAQ. Recommend: Group related questions together, provide sub-headings for these groups, eliminate the Q: in front of each one and make it a bulleted list. For example, group all the questions into categories of participation, applying, costs, programs, etc.


List of programs not linked to details

Minor : Listing of programs not linked to details. This was not tested but has the same problem as in other areas – a list of programs which are not linked to the descriptions. People should not have to look elsewhere for the links. Recommend: Link these list items to their respective program descriptions.


Opportunity for cross-links

Comment: Opportunity to put cross-links in to the International Youth Program from here, since it applies to this country.


Highlight videos – Int’l Youth Program Highlight video descriptions

Difficulty finding the path to the International Youth Program Several people have difficulty finding their way to the International

Youth Program from the DFAIT home page. They don’t associate a working holiday with a link labelled as Education and Youth. The tendency is to look under Travel or Business services.

View video (IYP-LocNotObvious.wmv – 3:29)

Expectation that program titles would link to more information Person expects program descriptions to link to more details. She is not

aware of the links on the right hand side. View video (IYP-ProgramsNotLinked.wmv – 0:55)


Highlight videos – Int’l Youth Program Highlight video descriptions

Difficulty finding eligibility information Person expects to find eligibility information under the FAQ link. She

has no clue where to find it elsewhere. She tries to venture off-site. View video (IYP-EligibilityCan'tFind.wmv – 1:13)

Active link to current page Person clicks on link to International Youth Program – Home even

though she is already on that page. View video (IYP-ClickHomeFromHome.wmv – 0:16)



Russian mission site


Section overview – Russia (1 of 2) Usability evaluation



Good info on working/studying/visiting/moving to Canada Expands awareness of Canada’s culture Helps Russian immigrants understand English terminology

Critical usability issues The link to the Trade area of the site is missing from the main menu

Recommend: If access to the Trade area is important, put a link to it on the Russian-language page as is available on the English version

Not all English menu items are on the Russian-language version of the menu Recommend: Match the menu items exactly in the Russian-language version; if no

translation is available, divert users to the English page Links on the right hand side are not evident to users

Recommend: Move links to key tasks into the centre area, or visually tie links on the right to the centre content; underlining links may help increase the visibility of clickable words


Section overview – Russia (2 of 2) Summary of observations (continued)

Critical usability issues (continued) Key links to the Trade area of the site are not visible because they are on the

right hand side Recommend: Put links to the Trade Commissioner service in centre content OR

visually tie the links to centre content so users can notice them

Minor usability issues Trying to get to the Russia site from a search engine can be difficult

Recommend: Offer links to search Mission pages from the DFAIT main search page. Determine users’ key search phrases and use them on Mission pages so Google-users can reach the correct page directly

Links on right hand items do not appear clickable Recommend: Make links to key content underlined

There is no indication of where the user is in the site Recommend: Provide a visual distinction for menu items at left when the user is on

the page to which the link corresponds; extend breadcrumbs throughout the site Link labels do not match the heading on the page to which they lead

Recommend: Make sure link wording matches the headings on the destination page


Closing question: Translation? – Russia At the end we asked users – “does the translation make

sense?” These are their responses: Match the English menus

Russian menu items are not complete (match English menus) Russian is missing links About Canada, Doing Business With Canada,

Armenia and Uzbekistan Match the English content

Put the English information in Russian – people need to clarify Provide more in the Russian section of the website If people don’t know English it is handy

Check the wording The Russian words are borrowed from English Some of the endings of words seemed odd Some spelling mistakes


Strengths of the pages we tested

Comment: This was the Russian-language version we tested.

Strength: People could easily find their way back to the home page. Breadcrumbs helped.

Strength: Main links are often in the centre content. Participants primarily scanned the centre area.Strength: People

appreciate having the same words in English and Russian, in the same order, as they flip back and forth between languages.

Strength: The left-hand menu is used consistently in Russian anyway. People quickly got used to it.

Strength: The picture helped one user remember the cultural link.


Finding the Trade area in Russian

Critical: The link to the Trade area of the site is missing from the main menu. Several users noted that they expect the main menu to be the same in Russian as it is in English. The Doing Business With Canada menu item is not on the Russian-language menu.Recommend: If access to the Trade area is important, put a link to it on the Russian-language page as done on the English version.

Comment: Perhaps if enough users suggest the CLF-translation of Contact Us then Google Translate will more accurately reflect this translation.

Comment: The Contact Us page did not provide Trade-related contacts. For this task, only one user tried to click on the CLF button.


Armenia needs to be on the Russian page

Critical: Not all English menu items are on the Russian-language version of the menu – in this case, the menu items for Armenia and Uzbekistan. All participants said it should be. As one said “there are Russian-speaking people in Armenia”. Though one said Uzbeks may be less likely to speak English than Russian. One user said pages should be “synchronized with English pages”.Recommend: Match the menu items exactly in the Russian-language version. If no translation is available, divert users to the English page.


Finding the Trade area on the right sideCritical: Key links to Trade services are not visible because they are on the right hand side. Two people explained that they “do not pay attention” to items on the right. Another “was looking on other parts of the page”. Another said they “always look on the centre” because the right hand side “looks a little like advertisements”. One said it might help if the colour of the heading bar was red like other heading bars.Recommend: Put links to the Trade Commissioner service in centre content. Or, visually tie the links to centre content so users can notice them.

Comment: Note what the page text looks like when the user enlarges the text of the page using their browser. Links span more than one line, adding to the time it took to scan these links. Note the CLF bar also spans more than one line.


Right-hand links are not evident

Critical: Links on the right hand side are not evident to users, in this case links to an Information Session. All participants said they do not look on the right, but scan the middle of the page; “the left is for main links, the middle for main links for this page, and the right I don’t pay attention to”. Another was “dissatisfied” on this task, but would have been “very satisfied” if the menu item had been on the Russian pages. Even the angle-bracket which normally draws the eye, did not help in this case.Recommend: Move links to key tasks into the centre area, or visually tie links on the right to the centre content. Underlining links may help increase the visibility of clickable words.

Minor: Links on right hand items do not appear clickable. Two participants were not sure what text in the Trade box was clickable. Recommend: Make links to key content underlined so users can scan for them and compare them easily with other key links.


An unfruitful search experience

Comment: This user expected to see a link to a consulate at this point. She reached this voyage.gc.ca page after clicking the Consular site on the search engine.

Strength: DFAIT ranks high on a Google search for visa information.

Minor: Trying to get to the Russia site from a search engine can be difficult – in this case when searching for Armenia visa information. This Google search about Armenia points to a Youth page on the main DFAIT site. After two fruitless on-site searches, this user clicked the “Consular site”. Reaching voyage.gc.ca, the user could not find the Russia site.Recommend: Offer links to search Mission pages from the DFAIT main search page. Determine users’ key search phrases and use them on Mission pages so Google-users can reach the correct page directly.

But, it leads to a 404 error.


Users need to know where they are

Minor: There is no indication of where the user is in the site. Breadcrumbs helped in some circumstances. When asked “do you know where you are in the site”, users did not always know. In some cases, users would browse deep in the content, which they appreciated, but moving to related pages in that area was difficult.Recommend: Provide a visual distinction for menu items at left when the user is on the page to which they link. Extend breadcrumbs throughout the site.


Link doesn’t match target heading

Strength: The link matches the heading in English.

Comment: It is fixed! The page content was updated after the test – so the link now matches in Russian.

Minor: Links are not the same as the heading to which they lead. In this case, a user took a while to figure out that the link and destination page were related.Recommend: Make sure link wording matches the headings on the page to which they lead the user.


Home page link to business cooperationComment: This page caused a little confusion. Clicking this home page link leads to a page about a trip to Russia instead of business development in Russia. Users recovered easily, following the instruction to try the English site instead.

Comment: It may be linked to Hotels and trip to Russia because on the English page the link is for Canadians Abroad.

Note: This is a translation of the Russian.


Highlight videos – Russian mission site Highlight video descriptions

Difficulty getting to the Trade area of the site First user scans the embassy and consulate but says “there is

nothing about business or trade”. On the bilateral relations page, she expects a link to the trade section – but there is none. To show how much effort this adds to browsing, she says “I would probably – without your help – click on every link to find the Trade portion”.

Second user does not see the Trade links because he “was looking on a different part of the page” and says the link “should be in the middle” of the page. He says it would be “better if it was more clearly visible … in the centre… or a better colour” or one of the links in the centre.

View video (Russia-HardToFindTrade.wmv – 3:08)


Highlight videos – Russian mission site Highlight video descriptions (continued)

Not all English menu items are on the Russian-language version First user explains that “only half the links were present” making it difficult

to find the Trade section of the site, and comments “I’m always disappointed when I find some information in one language that is missing in the other one”.

Second user is very dissatisfied because it is hard to find information related to Armenia; “If Armenia were listed on the left, like in English, it would be easier, I would get there”.

Third user can not find information for people from Armenia, because “in Russian version there is no such link”, but he expects there should be.

View video (Russia-MenuMismatch.wmv – 2:08)


Highlight videos – Russian mission site Highlight video descriptions (continued)

Links on the right hand side are not evident to users First user scans the whole page for a minute, not noticing the right hand

item. She reads every other link on the page, before she finally notices the link. She explains: “I expected it on the left … but it was on the right”; the left is for main links for the site – the centre is for the main links for this page – the right-hand item is “on the side I don’t pay attention to”.

Second user does not notice the trade link on the right hand side because items on the right “look a little like advertisements”. She expects the main information to be in the centre. “I would pay more attention to the red column – grey doesn’t pop out”.

Third user is pointed to the page with the link on the upper right. He does not notice the link on the right hand side. Instead he clicks a link to the main site and says “I don’t see anything about the Trade department – there is nothing”. The user finds the Trade contacts but almost leaves because there are no trade-related contacts in the primary scanning area. The user explains why it was difficult to see the link to the Trade section: “the right hand side is for extra information… supposed to find everything in the middle”.

View video (Russian-MissRight.wmv – 5:53)

Phone (613) 271-3001

Email [email protected]

Web www.neoinsight.com

Mail Suite 374300 Earl Grey DriveOttawa, OntarioCanada K2T 1C1

Gord Hopkins

Mike Atyeo

Scott Smith

Strategic design(613) [email protected]

User Experience Specialist(613) [email protected]

Competitive strategy(613) [email protected]

Contact information

Documents

DFAIT Usability Final 6 Apr08