Best practices for launching site redesigns

Making Leaders Successful Every Day

June 23, 2010

Best Practices For Launching Site Redesignsby Harley Manningfor Customer Experience Professionals

www.forrester.com

© 2010, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To purchase reprints of this document, please email [email protected]. For additional information, go to www.forrester.com.

For Customer Experience Professionals

ExECutivE SuMMaRyWhen companies redesign sites, users’ aversion to change results in problems like spikes in call center traffic and social media backlash. That’s because changes in site design force users to learn new interfaces and new process flows in order to accomplish their goals. Customer experience professionals can minimize the pain of launching a redesign by following best practices like engaging users during the design process and educating executives on what to expect.

taBLE oF ContEntSEven The Perfect Redesign Can Turn Into The Perfect Nightmare

the Science Behind Why People Dislike Change

Best Practices For Minimizing Problems Caused By Redesigns

RECoMMEnDationS

Start By Rolling Out A Truly High-Quality Site

notES & RESouRCESForrester interviewed customer experience professionals at companies that had recently launched redesigns of all or part of their sites.

Related Research Documents“Web Site Reviews: What, Why, and When” april 9, 2009

“Low-Cost user Research and usability testing techniques” February 3, 2009

“Scenario Design: a Disciplined approach to Customer Experience” July 19, 2004

June 23, 2010

Best Practices For Launching Site Redesignsby Harley Manningwith Richard Gans and Shelby Catino

2

2

7

mailto:[email protected]

www.forrester.com

www.forrester.com

http://www.forrester.com/go?docid=54217&src=57159pdf





© 2010, Forrester Research, inc. Reproduction ProhibitedJune 23, 2010

Best Practices For Launching Site Redesigns For Customer Experience Professionals

2

EvEN THE PERFECT REDESIgN CAN TuRN INTO THE PERFECT NIgHTMARE

There’s no way around it: Changing the design of a site is risky. Site owners we’ve interviewed report a list of problems that include huge spikes in call center volume as people call to get help or complain (or both), deep declines in customer satisfaction ratings, and social media backlash. For example, Yahoo!’s customer satisfaction rating dropped 5% in 2006 from the previous year because its redesigned site didn’t resonate with users.1 And when Facebook launched a redesign in March 2009, 1.7 million of its users joined a user group called “Petition Against the New Facebook” within the next two weeks.2

The Science Behind Why People Dislike Change

Firms may have the best intentions when making changes to their sites, but users don’t always see it that way. Why is change so hard for users to deal with? Changes that result from site redesigns can degrade the online customer experience because they:

· Place a cognitive burden on users. Redesigns require users to learn low-level, page-specific details like what a particular button or link will do. Users must also learn higher-level task concepts — such as what they have to do to open an account — in order to complete their goals.3

· Make two key usability metrics decline. Usability experts judge sites based on a handful of measurable human factors. These include the time it takes to learn how to use the site and the time it takes to complete a task. Both of these metrics move in the wrong direction for experienced users encountering a redesign for the first time. With the old design, the time required to learn a task like buying a product or finding favorite content was zero — they already knew how to do it! With the new design, expert users become novices and have to start learning from scratch.

· Fill users with frustration and anxiety. Urgent tasks aggravate these problems. Even at the best of times, customers are easily frustrated by high-involvement tasks such as transferring funds between accounts. That’s because the tasks can be inherently complex and the downside risk of getting them wrong is high (“Where did my money disappear to?”). Now imagine rushing to meet a bill payment deadline or trying to buy that last-minute anniversary gift only to find that the familiar interface changed. Is it any wonder that switchboards light up in the call center?

BEST PRACTICES FOR MINIMIzINg PROBlEMS CAuSED By REDESIgNS

Fortunately, there are best practices that reduce problems related to redesigns. We spoke with customer experience professionals who successfully manage the risk of their site changes, both large and small, through a disciplined approach to rollouts. After analyzing their advice, we conclude that firms should:

© 2010, Forrester Research, inc. Reproduction Prohibited June 23, 2010


3

· Engage users during the design process. Involving your users throughout the design process not only improves the quality of the finished product but also creates advocates. For example, online card sorting exercises improve menu redesigns by revealing how real users think about navigating a site. And because online card sorting is so efficient, compared with the original model of having users literally sort pieces of paper, some companies involve thousands of customers in the process — at an affordable price.

· Preview the design with the relevant user base. Customer experience professionals should let users take control of when they learn a new design by sharing their beta version well in advance of a launch. An email campaign and a prominent link on the home page (or the landing page that’s closest to the change) does the trick (see Figure 1 and see Figure 2). Preview versions should include a feedback mechanism so users can report a final round of flaws and, just as importantly, feel heard (see Figure 3). Companies can estimate how long to leave up their betas (or “previews” or “demos”) by examining how frequently their most important customers visit the part of the site that’s about to change.

· Educate executives on what to expect. Customer experience professionals should take the time to brief executives on what they’re changing, why they’re changing it, and the steps they’ve taken to ensure quality (e.g., usability testing). The briefing should include highlights of the testing results and verbatim user feedback that supports the quality of the change. Most importantly, the customer experience team should remind executives that users dislike change and that, despite all of these efforts, they should expect emails, phone calls, letters, social media backlash, and a spike in call center traffic — all of which will eventually go away.4

· Prepare call centers for a spike in traffic. Real-world data points range from no detectable spike — which is rare — to a 300% spike in the first month, tapering off to zero over the next five months. The size of the spike will depend on the number of people affected by the change, the extent of the change, the quality of the design, and the quality of the communication program leading up to the change (see Figure 4). The length of the spike will depend on the length of time it takes for all affected users to learn how to accomplish their goals (see Figure 5). Firms can help prepare their call center teams by including them in the redesign project team from the beginning as well as during the quality assurance process. Call center teams should also be provided with the results of usability testing, a list of changes and fixes that were made as a result of the usability testing, and a list of potential problems that you expect users may complain about.



4

Figure 1 Hoover’s Directed users to Preview its new Site via an Email Campaign

Source: Forrester Research, Inc. 57159

Source: Hoover’s email campaign

Hoover’s included aprominent link tothe new design.



5

Figure 2 Bloomberg Directed users to Preview its new Site via a Link on the Home Page


Source: Bloomberg Web site



6

Figure 3 Hoover’s allowed users to Provide Feedback Regarding its new Site


Source: Hoover’s beta Web site



7

Figure 4 a Formula For Estimating the Size of a Call Center Spike as a Result of Change


To estimate the size of the spike, calculate A x B x C x D, where:

The percent of user population affected by change = A

The degree of change = B (≤ 1.00 where 1.00 means total overhaul)

The quality of improvement = C (≤ 1.00 where lower means better)

The quality of communication program = D (≤ 1.00 where lower means better)

Figure 5 Factors For Estimating the Length of a Call Center Spike as a Result of Change


To estimate the timing of the spike, look at the size of your user populationthat will encounter the change in a given time period. For example:

Population A visits site daily = bigger spike, shorter duration

Population B visits once per month = smaller spike, longer duration

R E C o M M E n D a t i o n S

START By ROllINg OuT A TRuly HIgH-QuAlITy SITE

if customer experience professionals assume that their new site is really good in an absolute sense, that’s probably a bad assumption. virtually every site Forrester tests has basic design flaws that could have been avoided.5 to create a positive experience the first time out of the door, firms should:

· Conduct upfront user research and testing. Many problems that result from a redesign can be avoided by conducting upfront user research, such as ethnographic research and persona creation. these robust research methods can be combined with various testing methods — including prototyping, user experience reviews, and low-cost research techniques like surrogate user interviews. together, these techniques help firms understand user goals that relate to the business purposes of their various channels.6

· Pick a Scenario Design process and enforce it consistently. in order to design a superior experience, a firm needs to understand who its users are, what their goals encompass, and what they need to do in order to achieve their goals. Forrester calls this framework Scenario Design.7 Scenario Design helps companies avoid the pitfalls that come from designing based on a product-centric view of the world and instead design based on a customer-centric view



8

of the world. Firms should adopt a Scenario Design process and enforce it across the entire organization and throughout the entire project life cycle — helping keep customers top of mind.

· Be willing to put rollouts on hold, based on user feedback. all too often, firms get caught up in the fact that they’ve invested a great deal of time and money in an initiative and want to get it live even if it doesn’t meet their users’ needs. Firms must listen to the feedback they receive — especially when it comes from actual users during beta testing — and be willing to make changes based on this feedback. For example, a giant financial services company’s personalized customer portal never made it out of beta because it received a lukewarm reception.

ENDNOTES1 This 5% drop was seen according to the American Customer Satisfaction Index annual eBusiness report,

which is released by the University of Michigan’s Stephen M. Ross School of Business (http://www.bus.umich.edu/), Ann Arbor, Mich. This national survey is part of a study of 70,000 customers who rank their experiences with more than 200 companies in 45 industries. It is updated quarterly. Source: Chantal Todé,

“Yahoo’s Customer Satisfaction Takes a Hit,” DMNews, August 14, 2006 (http://www.dmnews.com/yahoos-customer-satisfaction-takes-a-hit/article/92328/).

2 Even though 1.7 million users only amounted to around 1% of Facebook’s total 175 million user base, it still represents a large number of dissatisfied users, which can further increase due to the nature of social media. In addition, Facebook’s own new layout poll — called “Vote on the new Facebook layout” — showed how unpopular the redesign was with users. Out of more than 1.2 million votes, only just more than 75,000 gave a thumbs up to the new layout. Source: Daniel Ionescu, “Facebook Redesign Revolt Grows to 1.7M,” PC World, March 23, 2009 (http://abcnews.go.com/Technology/PCWorld/story?id=7149776).

3 These two types of knowledge that users need to require are “syntactic knowledge” and “semantic knowledge.” In general, it’s better to minimize the amount of syntactic (low-level) knowledge that users must learn in order to use a site because it doesn’t benefit the users. For example, the users can’t carry that knowledge with them to a new site and get value from it there. Requiring users to learn semantic knowledge (e.g., task flow) can be even more problematic. To ease the burden, designers need to relate the new concepts to what the users already know, describe the model for use (e.g., showing a diagram of a process like applying for a loan online), and provide contextual help. For more information on this topic, refer to one of our favorite books on design, Designing the User Interface. Source: Ben Shneiderman and Catherine Plaisant, Designing the User Interface, Addison Wesley, 2004.

4 Customer experience professionals should provide realistic estimates of the size of problems. For example, when expecting a spike in call center traffic, executives should be offered an estimate of the size and length of the spike. One way to arrive at these estimates is to look at historical norms for past redesigns and then consider the size of the audience affected by the current redesign.



9

5 Because Forrester’s Web Site User Experience Review methodology is intended to diagnose potential problems rather than simply rating sites as “acceptable,” only a small proportion of the sites we review achieve a score of 25 or greater. Across more than 1,200 evaluations completed to date, the average score is 0.3.

6 During these uncertain economic times, it’s even more important to satisfy customers by offering them a high-quality Web experience. Web site reviews (also known as expert reviews, heuristic evaluations, or scenario reviews) offer an efficient way to identify — and then remove — usability problems that lead to lost sales and customer dissatisfaction. Companies can gauge their online user experience with an expert review methodology like Forrester’s Web Site User Experience Review, which is available online (http://www.forrester.com/cxpbenchmark). For more information about what constitutes a Web Site User Experience Review, why these reviews are valuable tools for identifying site design flaws, and when they should be used as part of a mix of usability inspection techniques, see the April 9, 2009, “Web Site Reviews: What, Why, And When” report.

It’s important to realize that even during a slumping economy when firms can expect shrinking budgets, user research and testing don’t have to become cost-cutting casualties. To learn more about several lower-cost user research and usability testing methods that can be employed to gain insight into users, see the February 3, 2009, “Low-Cost User Research And Usability Testing Techniques” report.

7 Firms know that customer experience is important — but they deal with it haphazardly. As a result, customers suffer through needlessly painful interactions. That’s why firms need a more disciplined approach to customer experience. Forrester recommends that companies adopt Scenario Design, a concept built on a simple assumption: No experience is inherently good or bad, and it can only be judged by looking at how well it helps customers achieve their goals. This approach requires companies to continually ask — and answer — three questions: Who are your users?; what are their goals?; and how can you help them achieve those goals? See the July 19, 2004, “Scenario Design: A Disciplined Approach To Customer Experience” report.





Forrester Research, Inc. (Nasdaq: FORR)

is an independent research company

that provides pragmatic and forward-

thinking advice to global leaders in

business and technology. Forrester

works with professionals in 20 key roles

at major companies providing

proprietary research, customer insight,

consulting, events, and peer-to-peer

executive programs. For more than 26

years, Forrester has been making IT,

marketing, and technology industry

leaders successful every day. For more

information, visit www.forrester.com.

Headquarters

Forrester Research, Inc.

400 Technology Square

Cambridge, MA 02139 USA

Tel: +1 617.613.6000

Fax: +1 617.613.5000

Email: [email protected]

Nasdaq symbol: FORR

www.forrester.com

M a k i n g L e a d e r s S u c c e s s f u l E v e r y D a y

57159

For information on hard-copy or electronic reprints, please contact Client Support

at +1 866.367.7378, +1 617.613.5730, or [email protected].

We offer quantity discounts and special pricing for academic and nonprofit institutions.

For a complete list of worldwide locationsvisit www.forrester.com/about.

Research and Sales Offices

Forrester has research centers and sales offices in more than 27 cities

internationally, including Amsterdam; Cambridge, Mass.; Dallas; Dubai;

Foster City, Calif.; Frankfurt; London; Madrid; Sydney; Tel Aviv; and Toronto.

www.forrester.com

mailto:[email protected]