The Definitive Guide to Qualitative Analytics

Quantitative Analytics

Qualitative Analytics

OPTIMIZE YOUR APP WITH

APPSEE UX APP ANALYTICS

The Definitive Guide to Qualitative Analytics

Introduction

Why Did We Write This Guide?

There’s a unique transformation occurring in the realm of mobile app

analytics. Analytics are not only becoming more specified but also more

robust in terms of what mobile teams are able to track and analyze.

Specifically, now app professionals have the ability to actually see and

qualify specific user behavior instead of just broadly defining it by numbers.

This is thanks to the advent of qualitative app analytics. We wanted to help

people understand exactly why this change is happening, what it means,

and how they can harness its full value.

Who is This Guide for?

This guide is for mobile product managers, CTO’s, mobile developers, UX

researchers, UI designers, mobile marketers, mobile startup leaders, or any

other type of “appreneur.” You do not need to have a Masters in Statistics

or Data Science in order to get the most from this guide. All you need is the

ambition to build the best app possible and the willingness to learn and

grow. Now, let’s get to the good stuff…

https://www.appsee.com/

The Rise of App Analytics | Page 4

The Rise of App Analytics (cont.) | Page 5

Quantitative Analytics: A Break Down | Page 7

The Shortcomings of Quantitative Analytics | Page 10

The Need for Qualitative Analytics | Page 12

Qualitative Analytics in Action (Use Cases) | Page 15

Comparison Graph: Quantitative vs. Qualitative | Page 21

Table of Contents


It’s hard not to gape at the sheer mushrooming of the mobile app

ecosystem. As Eric Schmidt, Executive Chairman of Google, so perfectly

put it, “The trend has been mobile was winning; it’s now won.” Just think,

Schmidt said this back towards the end of 2013- almost three years ago.

Look how much growth has even occurred since then! The stats are

extensive and astonishing.

To start, US daily smartphone screen time has actually exceeded television

time. On top of that, now 80% of internet users own a smartphone and of

those users, 89% of their media time is spent within apps. By 2020, App

Annie projects that gross revenue across all app stores will exceed $101

billion globally. Plus, in terms of simple annual growth, from 2014 to 2015

global total time spent in android apps grew by a staggering 63%. The

amazing statistics go on and on. We can confidently look back over the last

3 years and reaffirm that mobile has definitely won but we can also say that

apps have been the most valuable player!

The Rise of App Analytics

What is qualitative analytics?

Page 12Page 4


Page 5

Just how have apps risen as the “most valuable player” in the mobile

industry? An upsurge of unique, innovative products and increased

smartphone adoption certainly play a role, but are just the tip of the

iceberg. Under the surface practically every app uses some sort of app

analytics.

Why? Well to start off the concept of “If you build it, they will come” lasted,

if anything, for a hot minute. In this exploding app ecosystem, mobile

publishers quickly learned that if they want to stand out from the millions

of other apps out there, they can’t just rely on building a “great app”. They

must engage, monetize, and retain your users (who are some of the most

selective and impatient users the digital world has ever seen). The only

way to successfully do this is to know what’s going on inside your app- and

you can only obtain this knowledge via app analytics. Even interviewing

your potential users and holding focus groups won’t give you the insights

you are seeking. As Jakob Nielsen so perfectly worded it, you must “pay

attention to what users do, not what they say.” By gathering valuable data

on your app’s real users right from the get-go, you will optimize your app

more efficiently and with more certainty.

Thanks to the explosion of apps and speedy specification of apps’ needs,

there are a multitude (hundreds) of app analytics options out there. While

these systems might differ greatly in specific attributes, capabilities, plans

& pricing, and platform support, they typically can be categorized as

“Quantitative” (traditional) app analytics.

The Rise of App Analytics (cont.)


Page 12


Page 6 /22The Definitive Guide to Qualitative Analytics

But staying true to the spirit of the rapidly evolving mobile app world,

there’s already a “new” category of app analytics on the scene-

“Qualitative” app analytics. Now the big question is which one is going to

deliver the type of exceptional value that will allow you to fully understand

your users and what they’re doing, effectively optimize and troubleshoot

your app, and ultimately keep your app not just up to par but prevailing

in the ultra-competitive app ecosystem? In this guide we will illuminate

the characteristics and capabilities of both types of analytics, explore

the reasons behind the advent of Qualitative app analytics, and help you

successfully conclude which type of analytics best suits your needs. Let’s

get started.


Page 12


Page 6


So let’s start with what exactly is “Quantitative Analytics”? To obtain the best

understanding, let’s first look at the definition of the word “quantitative.”

Merriam Webster notes the definition as follows:

1. of, relating to, or expressible in terms of quantity

2. of, relating to, or involving the measurement of quantity or amount

3. based on quantity; specifically of classical verse: based on temporal

quantity or duration of sounds

Numbers, numbers, numbers- that is the crux of the definition. So when

it comes to quantitative app analytics, it means you are analyzing and

gathering data in terms of quantities. All of the data and information

obtained through this type of analytics can be measured and written down

with numbers. Typically, one would utilize quantitative analytics to get hard

aggregate insights on certain user actions and trends. Below are some

examples of when you would use quantitative analytics:

Quantitative Analytics: A Break Down

Page 7


Page 12


1. Basic usage reporting

a. How many total users do you have?

b. How many times is your app launched on a daily basis?

c. How many new users do you have? What is the ratio of new to

returning users?

d. How long is the average user session?

e. How many users never return to your app after their first session?

f. How many of your users come from a particular country, a particular

campaign, a particular site?

2. Basic conversion reporting

a. How many users made it from Step A to Step B within a specific

conversion funnel?

b. How many users dropped off between Step A and Step B within a

specific conversion funnel?

c. How many users completed a certain conversion funnel? What is your

CR for a particular funnel?

3. Basic user trend reporting

a. How many times do users use a particular feature?

b. How many users quit your app on a particular screen?

c. How many users accept or deny a particular in-app permission?

Page 8


Page 12



Notice a trend? The term “how many” is often used when asking

quantitative oriented questions. Now although the fundamentals of

quantitative analytics are to gather large amounts of numerical data,

let’s not forget how far quantitative analytics has come in terms of how

those numbers are obtained and displayed. Today, the technique of data

visualization, the presentation of data in a pictorial or graphical format,

plays a big role in quantitative analytics. Basically it enables mobile

professionals to better grasp difficult concepts and identify new patterns

within all the data on their app. There is no doubt that data visualization can

certainly help you pinpoint trends and correlations within your data, but at

the end of the day it cannot tell you the “whys” behind all those numbers.

This leads us into our next section: The Shortcomings of Quantitative

Analytics


Page 12


Page 9


Numerical data is of course important, but there is so much more that

you can (and need) to learn about your app. Essentially, this is where

quantitative analytics falls short, as it only satisfies your number based

inquires. As noted earlier, quantitative data can help you pinpoint

numerical-based concerns such as quit rate of certain screens, but cannot

really go beyond that in terms of analysis.

For example, let’s say you have a quit rate of over 50% for your login screen.

Clearly the majority of your users are not wanting to login to your app-

that’s certainly important to know. Unfortunately, with quantitative analytics

you have absolutely no idea as to why they are quitting. No matter whether

the data is displayed in a table or as a beautiful pie chart, at the end of

the day it is still a quantitative measurement and nothing more. In order to

hopefully understand the reason(s) behind this particular metric, you will

probably have to invest many hours, implement multiple tests and design

iterations, and potentially conduct additional primary user research. That’s

a lot- all just to get one answer for one bit of data.

The Shortcomings of Quantitative Analytics


Page 12


Page 10



Ultimately, a table of data can only take us so far, and often times the most

actionable insights cannot be deduced from numeric data. Due to the fact

that quantitative analytics was practically the only solution on the market

since the onset of mobile analytics, many mobile professionals initially

presumed that they were tracking and measuring their app in the best

way possible. Logically as the mobile app market proliferated, technology

evolved, and user experience standards skyrocketed, mobile professionals

quickly began to notice that “something” was missing in terms of their

analysis of their app. Their numeric data is only supplying them with a

glimpse of what is going on in their app. And of course, as anyone would,

they want the full picture.

Enter- the need for Qualitative App Analytics.


Page 12


Page 11


To properly grasp the essence of qualitative analytics, let’s again start out

with a simple definition:

1. Pertaining to or concerned with quality or qualities

2. Of or relating to how good something is: of or relating to the quality of something

Can qualities such as the color of one’s eyes, feelings on one’s birthday,

or softness of one’s cat be delineated by numbers? No- these are traits,

sentiments, and conditions.

So when it comes to the analysis of your app, what is the one essential

element that cannot be represented by numbers?

User experience.

User experience (UX) encompasses all aspects of the end-user’s

interaction with your product. Because the UX of your app is a qualitative

measurement, you need to essentially see what your users are

experiencing and how they behave in order to actually understand it.

Only qualitative app analytics, or as many now call it, “UX app analytics”

can measure this extremely critical and nuanced KPI.

The Need for Qualitative Analytics


Page 12


Capturing Visual Data

Drilling down to the individual user level

Displaying Data in a Comprehensive Manner

USER RECORDING

00:23

1

3

2

SETTINGS

Send feedback

Sign out

About

CHECKOUT

Phone

Name

Address

Crashing Screens

Touch Heatmap

*From this point on you will see us utilize “Qualitative Analytics” and “UX App Analytics” interchangeably. Fundamentally, UX App Analytics is a type of qualitative analytics. In the case of mobile app software, qualitative analytics specifically focus on assessing user experience.

Let’s take a look at the key pillars that define the essence of UX app analytics*.


Page 12


Page 13


The Need for Qualitative Analytics (cont.)

0%

10%

20%

30%

0-3 sec 3-10 sec 10-30 sec 30-60 sec 1-3 min 3-10 min

Ne

w u

sers

Time spent within App

Create account

Login

NEW USER SESSION PLAYBACK USER #0014

00:23 iPhone 6iOS 8.1.2 wifi May 2 4:24pm

TIMELINEAction Time

Login 00:00

00:10Shopping List

(Appsee’s User SessionRecording Feature)

Quantitative VS Qualitative

So now you have a basic comprehension of:

1. Why there’s a considerable need for qualitative analytics.

2. What are the foundations of qualitative analytics.

To further highlight the power of qualitative analytics for your app, take

a look at this common app assessment scenario and the kind of data

obtained from each type of analytics. Below, a mobile app professional

is trying to understand new user behavior within their app.


Page 12


Page 14


Qualitative Analytics in Action (Use Cases)

Not only can qualitative analytics greatly help you with analyzing and

optimizing your user onboarding experience, it can also address a plethora

of app optimization concerns.

Let’s dive in to some specific case studies:

Crashes You can save your dev team over “20% additional

hours” like eBay did by harnessing the power

of qualitative analytics for crash analysis and

repairs. With qualitative data on crashes, such as

crash video recordings paired with symbolicated

crash reports, mobile development teams no

longer have to pour over crash logs for endless

hours. Instead, they can see the exact sequence of actions that led to

the crash with their own eyes and then easily pinpoint the precise line

of code that needs to be fixed. No more guesswork when it comes to

understanding and resolving crashes- which means a better overall user

experience for your users.


Page 12 Page 15


Checkout Process Any m-commerce app understands the utter

importance of their checkout process and

screen. The slightest flaw or confusing element

can cost them a sale. In order to effectively and

thoroughly enhance this critical facet of your app,

you must go beyond quantitative data. Exactly

why did your user abandon purchase? Did they

experience a payment issue, did a form field freeze on them, did they have

trouble adding a promo code? All of these questions can only be answered

by UX app analytics. With UX app analytics you can filter users that did not

end up making a purchase and watch their session recordings. In seconds

you'll be able to detect the issue(s) that hindered them from converting.

BUY

User Support Maybe you have a unique IoT app that allows

users to control their home heating system or

monitor their baby’s vitals? Your user support

team is even doubly crucial in this case seeing

as your app regulates a real-world, tangible

device in a realm that is relatively new in the

digital world. Basically, your mobile application is

your user’s home base and communicates the value of your IoT device. To

maintain a high level of performance, connectivity, and usability, your user

support team needs to know exactly how your users are behaving within

your app and what they are experiencing. Again, user experience is not

something that can be represented by numbers. By watching user session

recordings of real-world interactions with your app, your support team can

see the exact experiences of users, which users were affected by particular

issues, and reach out and resolve those matters accordingly. This can allow

your support team to be drastically more precise, personal, and prompt-

meeting your users’ high standards and even exceeding them.


Page 12 Page 16


Search Accuracy For any app with an extensive search function,

such as the standard travel app, you need to

remain on top of your game in terms of search

accuracy, auto-correction/predictions, and

search display results. Whether your search

function is fully optimal can make the difference

between a conversion and a drop-off. What are

the most often searched terms, how are the results appearing (are you

providing too many results per page?), are users confused by results or

straining to find the search they want? Surprise- another instance where

only qualitative analytics can provide you with the actionable insights you

need regarding your search function.

Secure Sign-Up The majority of financial and banking companies

are going mobile nowadays and it makes total

sense. Users want to be able to access banking

and financial functionalities on the go, and not

have to wait to physically be at a desktop. Yet

that doesn’t mean that there isn’t a sense of

apprehensiveness from the user when it comes

to logging in to these types of apps, especially due to mobile security

issues. Basically, a secure sign-up/account login screen can make or break

the adoption of your app and retention of your users. Maybe your users are

struggling to complete a certain “CAPTCHA” or password confirmation? Or

maybe your users are quickly abandoning your app due to a “suspicious”

login glitch? Once more, you can utilize qualitative analytics to understand

how your users are experiencing your app and affim that your signup/login

screen remains a safe, gateway and not a barrier.


Page 12 Page 17


In-app Ads Let’s say you have a sports app, with your main

source of revenue being in-app ads. To put it

simply, no matter how much your monetization

tool and programmatic ad network “promise” that

they provide the best control and transparency in

the business, it is oftentimes not enough. When it

comes to monitoring your in-app ads, there are a

lot of moving pieces to keep track of, especially

if your app has a large amount of impressions to fill. Once an unsuitable ad

sneaks its way into your app, it can damage your brand name, ruin the UX,

trigger app abandonment, and spark negative reviews. A qualitative UX

tool, like user session recordings, can allow you to see through your users’

eyes how the ads (from banners to videos) are actually appearing on your

app’s screens.

These insights can reveal whether an ad potentially interrupted your users’

experience/journey, had a technical or quality issue, and/or led the person

to instantly quit your app among many other results. Many top apps are

beginning to harness the potential of qualitative analytics for monitoring

of in-app ads. For example, when the renowned sports app 365 Scores

utilized UX app analytics to monitor there in-app ads, they were able to

decrease user complaints by 44% and discontinue work with two unsuitable

ad networks.


Page 12 Page 18


Gesturization Let’s say you have a gaming app with a unique

gesture pattern. You want to analyze how your

users grasp this gesture, their natural movement

on your screens, and whether the gesture is

ultimately the best choice for your user base.

Maybe users from a specific country interpret

the gesture differently than another geo? Maybe some users are lacking

the fine motor skills to successfully execute this gesture? Or maybe some

users are just frustrated with the gesture? Numerical data is not going to

provide you with actionable answers on your gestures. UX app analytics

systems, like Appsee, will provide you with the qualitative answers you

need to optimize your gestures and app usability. Via Appsee’s user session

recordings and automatic tagging of specific gestures, you will be able to

see for yourself exactly how specific users complete or don’t complete your

gesture. On an aggregate level of analysis, Appsee’s visual touch heatmaps

can highlight all the gestures users are completing on each screen and

where they focus the most. It can also reveal every instance and location

of an unresponsive gesture on each screen. Innately, gestures are a very

physical, nuanced user action that in order to be properly optimized, must

be analyzed on the qualitative level.


Page 12 Page 19


Fake Installs Many mobile teams will run paid campaigns

to acquire downloads/new users. This is a

great tactic to boost visibility and increase

your user base, but it also has its inherent risks.

Unfortunately, there is always the potential that

some acquired users from paid campaigns can

actually be fake users/bots. The only way you

can truly separate the “wheat from the chaff” is by actually seeing how

users behave in your app. Surprise- you can only accurately execute this

by employing qualitative analytics. Qualitative analytics will allow you to

effectively pinpoint behaviors (or lack of behavior) that is characteristic

of fake users and then identify which source they originated from. This

will enable you to fine-tune your work with particular ad networks and

significantly boost your ROI.


Page 12 Page 20


In order to fully grasp the value of qualitative analytics, you must

understand that it is a powerful complement to quantitative analytics. As

discussed, quantitative allows you to identify important trends, issues,

and actions within your app on a numerical basis. Then, qualitative

analytics augments this data by supplying the crucial “whys” behind the

numbers. Basically, in order to paint the clearest picture of what’s going on

inside your app, all successful qualitative analytics systems reference at

least some quantitative data.

Below we have another nifty visual that illuminates how qualitative

analytics effectively expands on quantitative.

The Marriage of Quantitative and Qualitative

QUALITATIVE

QUALITATIVE

Funn

el Metrics Crash M

etrics

Usability Metrics User Navigatio

n Met

rics

Why d

id u

sers

dro

p off

your funnel a

t a certa

in

stage

Why did a crash occur and

what sequance of events

led up to the crash

How are your users actually

using your app and where do

they encounter usability issuesHow is

each u

ser n

avigatin

g

thro

ugh your app and w

hat does

their

journ

ey look lik

e?

QUANTITATIVE


Page 12


Page 21


So are you ready to get the ball rolling and gain the best insights possible on your app? Choose the definitive leader in qualitative app analytics- Appsee.

With Appsee's unique user recordings and touch heatmaps, you can make the most informed decisions regarding your app's user experience and spend less time sifting through mountains of data. Appsee also automatically detects all

screens, buttons and user actions within your app without requiring you to select what to measure in advance. No more asking questions of your analytics system-

just pure, unambiguous answers regarding your app and its users.

Integrated with Fabric? You can easily install Appsee’s free trial with just one click.

You can also simply begin our Free Trial via our site. All it takes is one line of code and less than 5 minutes. Now that’s user experience at its finest.

Get Started Now

Appsee is trusted by these amazing mobile teams:

The Industry Leader in Qualitative App Analytics

[email protected]

https://www.appsee.com/start?utm_source=qualitative_eBook&utm_medium=premium_content&utm_campaign=free_trial_button&cpnid=701b0000000JzMJ

https://fabric.io/kits/ios/appsee?utm_source=qualitative_eBook&utm_medium=premium_content&utm_campaign=fabric_link&cpnid=701b0000000JzMJ

https://www.appsee.com/start?utm_source=qualitative_eBook&utm_medium=premium_content&utm_campaign=free_trial_button&cpnid=701b0000000JzMJ


Documents

The Definitive Guide to Qualitative Analytics