Web Analytics Workshop

Making Your UX Process Effective and Persuasive

with Web AnalyticsO’Neil | Rahul

Somesh Rahul@SomeshRahul

Daniel O’Neil@phoenix1189

Namaste! We’re Information Architects at The Understanding Group (TUG)

How Somesh Got Here

How Daniel Got Here

Intended Takeaways

1. Be able to apply a framework for balancing quantitative and qualitative research methods.

2. Have a grasp of several key Google Analytics tools that are most relevant to UX practices.

3. Learn through labs, workshops and case studies how web analytics is applied to actual UX projects.

Why Web Analytics

What Web Analytics Can and Can’t do

Analytics track actions, not intent!

Web Analytics and User Experience

Behaviors can infer intent Quantitative guides Qualitative

Search Analytics for Your Site

by - Lou Rosenfeld

The Web Analytics Framework

Business Goals and User Needs

Market, Audience, Seasonality

Testable On-Site Behavior

Website Analytics Works Best When They are Measuring the Distillation of True Value

Business Goals / User Needs

Website GoalsDescribed by: Hypotheses

MarketAudience

Seasonality

Constrained and Organized by:Filters, Segments, Time

Testable On-Site Behavior

Tested by:

Descriptive Analytics

Flow, Page Navigation, Nonstatistical narratives

Statistical Analytics

A/B testing of Page Variations and Dimension Segments

Goals and Hypotheses

Business Goals / User Needs

Website GoalsDescribed by: Hypotheses

Websites Run on GoalsA “Goal” is a measurable outcome resulting in a user completing some desired activity on your website. Typical goals are:

– Confirmation page at the end of a sales transaction.

– Thank-you page after filling out a contact or quote request form.

– Application or content downloads.

– Playing a game or watching a video on a site.

The Best Goals are Existentially Critical

– Is it a “Holiday Bonus” question?

– If this goal stopped happening, would your organization (or your department) still exist?

– Most companies should have a few goals filtered through many segments.

BUT...Goals Can’t Say Why or How

– Goals describe behavior badly.

– Goals can’t describe intent at all.

Hypotheses Link Goals, Behavior, and Theories about Intent.

Hypotheses

A hypothesis suggests a functional change based on a theory of action that has a measurable outcome.

Goals and Hypotheses

– Goals link clear up or down numbers to the outcome of a specific site behavior.

– Hypotheses provide the testable narrative about how the user’s experience on the site affects those goals.

– The testable narrative does not have to BE a goal, but should specifically be IN SERVICE OF a goal.

Filters and Segments

MarketAudience

Seasonality

Constrained and Organized by:Filters, Segments, Time

User Segments

shops like consumer

designer involved

in specifying VIP atcurrentcontract

customer

VIP atcontractprospect

?investor

Problems User Segments Addresses

Problems User Segments Addresses

Filters

- Todo: something about filters here

5minutes

Descriptive Analytics

Testable On-Site Behavior

Tested by:

Descriptive Analytics

Flow, Page Navigation, Nonstatistical narratives

Statistical Analytics

A/B testing of Page Variations and Dimension Segments

User Flows

- Structure User Interviews- Create User Journey- Find Path of Least Resistance

Stand-alone

Integrated

External / Social

Statistical Analytics

Testable On-Site Behavior

Tested by:

Descriptive Analytics

Flow, Page Navigation, Nonstatistical narratives

Statistical Analytics

A/B testing of Page Variations and Dimension Segments

What is Statistically Significant?

Determining whether the differences seen in data is more than random chance.

Why Use It?

- Addresses the HiPPO problem.- Saves time by getting to outcomes faster.- Uncovers subtle effects.- Confronts our own biases about aesthetic and design.

Quantifying a “measurable outcome”

If goals have been set up properly, outcomes can be measured using simple statistics. And simple is all we need!

The recommended statistical method for UX professionals is the A/B test.

Appropriate A/B Tests Should:

- Be immediately apparent to anyone looking comparing the pages.

- Be defined in a functional UX way.- Represent a set of coherent conceptual changes against

a single hypothesis.

Typical A/B Test Candidates

Question Testing For Best Testing Tool

Is the navigation layout affecting conversion rate?

Conversion rate by template

Google Analytics Experiements (Not out of the box but you can hack it)

Which of two landing pages performs better?

Conversion rate by page version

Google Analytics Experiments

Which User segment converts better

Conversion rate compared by User segment

Advanced segments, Confidence Interval test

What Statistics Don’t Tell You

- Why a test failed. This can be just as critical as a success.

- Why it succeeded. - How to thoughtfully create testable hypotheses.

Marrying User Experience & Web Analytics

Your UX Process

Abstracted UX Process

Discovery Research and Analysis

Design and Testing

Discovery

Discovery

● Establish clearly the “Why” and “Who” for the site.

● Organizational goals are articulated and prioritized.

● The audience is clearly identified.● The ultimate measures of success

are agreed upon.

Research and Analysis

Research and Analysis

● Research how your users approach your current site.

● Evaluate the website’s design and information architecture.

● Synthesize the details into high-level models that represent both user needs and a high-level information architecture.

Design and Testing

Design and Testing

● Specify the site structure.● Determine the ways in which the

goals will be achieved through site structure.

● Test.

Qualitative and Quantitative Research

Discovery Research and Analysis

Design and Testing

Qualitative Research (UX)

Quantitative Research (WA)

Stakeholder InterviewsIntention Modeling

User InterviewsPersonasUser Journeys

PrototypesLive Testing

Hypothesis GenerationGoal Context

User SegmentsUser Flows

A/B Tests

Thank You!