Embed Size (px)
Citation preview
Kate Kenyon @kate_kenyon
Content Usability Testing: Getting to Good
Hello, I’m Kate‘80s – bookish child of an IT dev and a PM‘90s – linguist, magazine writer ‘00s – marketer, journalist, technologist, content strategist
I design content, and I design the teams and systems that make it.
What we’re covering today Effective content delivers better experiences 5
10Creating a content testing process
Case Studies
20
Approach10
Wrap Up
2
We are here
Lunch is here
The Goalis to deliver a better customer experienceusing effective content
Effective content delivers better experiences
5
Creating a content testing process 10
Case Studies
20
Approach 10
Wrap Up
2
Testing content shows the gap between
strategy and practice
Testing provides data for content decision making
Testing content informs IA, CS, and Design
Image: http://www.socialmediaconnections.biz/wp-content/uploads/2015/12/responsive_web_design.jpg
Testing content keeps stakeholders aligned
Image courtesy of Shutterstock
Testing content explodes expectations
Testing enables content to become a quantifiable, manageable asset with
genuine business value
The Challengeis to create a content testing processthat will yield regular, usable data
Effective content delivers better experiences
5
10
Case Studies
20
Approach
10
Wrap Up
2
Creating a content testing process
Testing is part of a cycle, not a process Plan
Make
TestReview
Publish
Testing content needs to be a repeatable exercise
Plan
Make
TestReview
Publish
Plan
Make
TestReview
Publish
Plan
Make
TestReview
Publish
Plan
Make
TestReview
Publish
It’s not the testing, it’s what you do with the resultsRepeated testing builds data around how content works on your site, and what you should be making.
That data needs to be part of your decision making processes
Plan
Make
TestReview
Publish
Old production processes + a bit of testing won’t deliver the benefits you’d
like.
You need a plan to change your processes, before you start testing.
Approach Effective content delivers better experiences
5
Creating a content testing process 10
Case Studies
20
Approach10
Wrap Up
2
How I do testing
Image courtesy of Shutterstock
• Content testing principles• Mapping the landscape• Testing the things that matter• Writing test cases• Running tests• Writing up findings
Content Testing Principles
Image courtesy of Shutterstock
• Content is there to do a job. Know what that job is, and test for how well the content is doing that.
• Test by asking questions that are important to your key people. These could be customers or the business, or both.
• Test using questions with definitive answers.
Mapping the landscape
Image courtesy of Shutterstock
Test for the things that matter
Image courtesy of Shutterstock
• What is the significant influencing driver within the business?
• How do your testing goals align with that driver?
• What will you do with the results of the testing?
• Who will potentially be impacted by the testing?
Writing test cases
• Write tasks: one task per question• Questions must have ONE definitive answer• No clues in the questions• Use plain language and common speech• Be concise• Make tasks independent from each other• Make sure tasks don’t require confidential
information.
Writing test cases: samples
• How many days holidays is a new, full-time employee entitled to in UK offices?
• Who should be the first person you contact if you have an accident while on site with a client?
• How many bottles of hair dye did the company sell in the UK in 2015?
Running tests
Image courtesy of Shutterstock
Running Tests• Choose volunteers carefully: screen for prior
knowledge and bias. • Give volunteers plenty of notice, and tell them in
advance what will happen.• Stress that you’re testing the site, not them.• Check your environment! Equipment, connections,
facilities, room bookings• Have water, pens, paper, tissues, food available.
Recording results• Use VIDEO! Jing is free, easy screen recording
software. There are many others, and for remote testing, GotoMeeting has worked well.
• Ask participants to comment and narrate as they go
• Have one test observer sit beside the participant to guide and lead the test
• Have another observer sit behind them to one side, so they can look over their shoulder, and note down any behaviours or comments.
Writing up results
Image courtesy of Shutterstock
• Use a meaningful format. Powerpoint, doc, video: choose a format that will best deliver the message to the intended audience
• Exec summaries are essential, and recommendations.
• If quoting from participants, maintain anonymity, and quote accurately.
• Provide the data in appendices
Some notes on things I’ve learned during testing
Testing: have a clear view of who is involved
Image courtesy of Shutterstock
Be clear that this is about making hard choices
Image: https://www.lighthousewebdesigns.com/wp-content/uploads/2015/04/bad-content.png
Testing content makes you align what you’ve got with what you want
Case Studies Structured content delivers better experiences
5
Creating a content testing process 10
Case Studies
20
Approach10
Wrap Up
2
Mapping the landscape
Image courtesy of Shutterstock
• A global intranet site hosting over 100,000 pages of content
• One (rather ancient) implementation of Sharepoint
• Thousands of employees, across multiple countries
• A head of employee comms fighting to get her message across
• No way of showing the impact that the intranet was having.
Image courtesy of Shutterstock
RISKImage courtesy of Shutterstock
Mapping the landscape
Image courtesy of Shutterstock
How effective is the Bank’s intranet in supporting Global Standards? Does the intranet help employees implement their day-to-day activities, or actively contribute to risk within the Bank? Cognifide, a technology and consulting firm, was commissioned to conduct content usability research to look answering these questions. The research was conducted between July and August 2016, and focussed on how well Bank employees could use the intranet to answer 5 questions related to Global Standards.It found that, on average, employees were only able to find the answers to just over half (51%) of the questions. This means that for this test the failure rate was 49%, a significant exposure to risk for the Bank.
Writing test cases
Image courtesy of Shutterstock
• Asking each user to find the answers to a set of questions relating to Global Standards. They had to use the intranet only to complete the task.
• Questions were worded to have an unambiguous answer.
• Additional self-assessment questionnaire to provide a subjective view on how easy it was to do the tasks
Outcome Meaning Metrics
Success The information can be found in a reasonable amount of time. User can complete the task/give the right answer in the allotted time.
FailThe information cannot be found in a reasonable amount of time. User cannot complete the task/give
the right answer in the allotted time, or gives up.
Disaster The information is confusing or misleading. User completes a different task/gives the wrong answer.
Participants & Location
Image courtesy of Shutterstock
• Participants were recruited among Bank new joiners who had completed their mandatory training. They took part anonymously and they are identified in this report by a progressive number (Participant 1, Participant 2, etc.).
• Anonymity is a pre-requisite for this type of research, to ensure participants can freely express their opinion. All participants were already familiar with the intranet, and most of them stated that they use the intranet daily for their job.
• Sessions were run at the Bank in a common meeting room, without external observers, and they were attended by the participants, one at a time. Two facilitators from Cognifide were attending as well, one running the session and another one taking notes.
Running tests
Image courtesy of Shutterstock
Task # Question
Task 1 What are the goals of the Bank’s Global Standards?
Task 2 If you suspect an account is being used for money laundering, can you tell me one of the ways to escalate your concern?
Task 3 Can you name three unusual activities that might indicate a customer is involved in money laundering?
Task 4 A customer wants to send money to their relatives in Syria: are they allowed?
Task 5 Following an Unusual Activity Report, a customer asks to see their financial crime risk rating. Can you informally tell them?
Writing up results
Image courtesy of Shutterstock
• Report written, with exec summary, methodology, test cases, results per question, overall results, outcomes and recommendations
• PowerPoint of recommended next actions, with timelines and rough costs
Results into Actions
Image courtesy of Shutterstock
• Data used to build business case for $10m investment in a new intranet CMS
• Data used to acquire new funding for UX review, Content Strategy work and new intranet governance
• Repeat testing scheduled to show changes following updates.
Well, that’s a wrap!
Plan
Plan for the smallest start
Design for outcomes, not problems
Image courtesy of Shutterstock
Run
Start fast
Expect headwinds as results appear
Image courtesy of Shutterstock
Repeat, often
Don’t overthink it
Hold your ideaslightly
Expect to retestmonthly, quarterly, yearly
Image courtesy of Shutterstock
Final things!PLAN:
agree on what the goal isplan how to get thereknow what you’ve got to work withknow how you measure ‘good’
REPEAT:keep moving on, like water
RUN:brief producers fullykeep track of what’s liveown your errors, and successes
