A blueprint for mobile app optimization

How Best-in-Class App Developers Use Data to Develop Amazing Mobile Apps

A BLUEPRINT FOR A/B TESTING AND OPTIMIZATION ON NATIVE MOBILE APPS

2


EVALUATING MOBILE STRATEGY

QUANTIFYING GOALS ON MOBILE

HOW CONVERSION RATE OPTIMIZATION (CRO) FITS IN

TESTING

ANALYTICS

SEGMENTATION

PERSONALIZATION

COMPARING UX & OPTIMIZATION: WEB VS. MOBILE

UX MEETS CRO ON THE WEB

UX MEETS CRO ON MOBILE

CASE STUDY: TASK RABBIT - WEB VS. MOBILE

TECHNICAL DYNAMICS ON THE WEB VS. MOBILE

CRO ON MOBILE: ACQUISITION

OPTIMIZING FOR APP STORES

OPTIMIZING FOR REGISTRATION/KEY ACTIONS

CRO ON MOBILE: RETENTION

TECHNICAL CONSIDERATIONS

PRODUCT CONSIDERATIONS

CONCLUSION

0 1

03

02

04 05

06

CONTENTS00

3


A majority of Americans now own a smartphone of some kind –

positioning mobile as one of the most significant opportunities for

consumer and B2B technology companies.

It’s no surprise that the app market is growing, with more than one

million apps both in the Apple App Store and Google Play Store.

How do you make your app stand out from the crowd? The first

step is to craft a best-in-class user experience.

Optimizely has already built a market-leading platform for

creating best-in-class user experiences on the web. With users

spending an increasing amount of time on mobile devices, it’s

become critically important for businesses to be able to deliver

amazing user experiences that span the web and mobile.

Delivering on this, however, is easier said than done. Many budding

app developers will soon learn that while their web and mobile

users are often the same people, those people have very different

goals and behaviors when engaging on mobile vs. when engaging

on the web.

That’s why we wrote this guide – to provide you with the most

comprehensive and in depth resource to learn:

(1) What makes an awesome mobile user experience

(2) How to evolve your mobile strategy to create highly engaging

products

(3) How to optimize that user experience for conversions

WHILE WEB AND MOBILE USERS ARE

OFTEN THE SAME PEOPLE, THOSE

PEOPLE HAVE VERY DIFFERENT GOALS

AND BEHAVIORS WHEN ENGAGING ON

MOBILE VS. WHEN

ENGAGING ON THE WEB.

1. HTTP://PEWINTERNET.ORG/REPORTS/2013/SMARTPHONE-OWNERSHIP-2013.ASPX

1

4


01 EVALUATING MOBILE STRATEGY

With smartphone penetration rates as high as 74% , your com-

pany’s mobile strategy is more important than ever. Mobile-only

companies like Instagram, WhatsApp, and Snapchat have forged a

path to billion-plus dollar valuations. Not to mention, today’s most

innovative firms like Eventbrite, an event management platform,

and Percolate, software for content marketers, are investing heav-

ily in developing mobile-focused solutions.

What’s your mobile strategy? How can your brand make the

strongest impact possible?

2. HTTP://MASHABLE.COM/2013/08/27/GLOBAL-SMARTPHONE-PENETRATION/

2

5


JUST-FOR-FUN APPS ARE

EVERYWHERE — WHICH MAKES IT

TOUGH FOR APP DEVELOPERS

TO STAND OUT.

Years ago, a successful mobile strategy meant ‘having an app.’ But

the landscape has quickly changed. Hollywood success stories are

becoming less and less common. Just-for-fun apps are everywhere

— which makes it tough for app developers to stand out.

The secret to a successful mobile app is no secret at all. Plain and

simple, top mobile apps delight users by 1: making their lives easier

and/or 2: providing entertainment.

Consumers crave elegant user experiences, workflow tools, and

highly engaging games. They’re looking for resources to be more

productive, opportunities for brain breaks, and touch points to

stay connected with friends and family.

That’s why analytics, A/B testing, and conversion rate optimization

are extremely important to the app design process. You need

to be learning about your users, testing your assumptions, and

exploring areas for improvement.

QUANTIFYING GOALS ON MOBILE

Before building an app, product owners will need to clearly define

their revenue objectives. This process starts with identifying key

metrics that align with an engaging – and profitable – mobile user

experience.

Start by thinking about the metrics that define success on a

desktop. Some of the user experience objectives (i.e. user engage-

ment and monetization) will be similar. You’ll want your customers

to be engaging with the product in a way that drives revenue. At

the same time, you need to approach your products goals with a

mobile-first mindset, understanding that you’re reimagining your

products for a completely new device.

Pay attention to the following metrics when designing for mobile:

1. DAILY ACTIVE USERS (DAUS): DAU measures your app’s

engagement levels – how many unique users visit the app daily?

A high DAU count is a sign that your app is something that people

need (or really love). For instance, users might love your game

and decide to play it, every day, on their morning train commutes.

Users might rely on your app to organize their notes (like

Evernote), book vacation rentals (like HotelTonight), and organize

their content marketing programs (like Percolate).

6


2. AVERAGE RATING & REVIEWS: This metric quantifies what

users think of your app and directly impacts performance in app

marketplaces.

App reviews are also a great place for developers to research user

concerns -- which can directly influence your brand positioning

and future product releases.

So how do you get users to review your app?

Well, you ask them.

But here’s the thing. Simply telling users to “leave a great review”

won’t be enough. In addition to prompting users to review your

app, you need to think about how to position these requests within

your user experience.

Most apps will generally prompt a user to leave a rating or review

upon launching the app - interrupting the general user flow. A bet-

ter way to prompt the user is after they have finished doing what

they need to do within your app -- or after they’ve accomplished

something that gives them a sense of achievement (like reaching a

high score within a game). For example, FIFA 2012 prompts you to

leave a review for the app after you have won a major tournament.

Apple’s App Store uses app ratings as a factor in calculating Top Chart rankings. It’s tough for apps to bounce back from a low ranking -- even if they’re downloaded fairly often.

7


4. RETENTION: Retention is calculated as DAUs divided by

number of downloads. It’s a key driver for apps with in-session

monetization opportunities. Are your DAUs increasing as down-

loads increase? If not, it is worth digging deeper into why users are

abandoning your app.

Apps using the freemium model as their main source of revenues

must be sure that they have the proper mechanics in place to

maximize retention and upselling.

As an example, Clash of

Clans (a popular mobile

game) offers virtual

currency and cleverly

timed push notifications.

Some app companies

have even determined

the length of time it takes

to monetize a user and

will even serve ads to

competing games when

a user has past that time

frame and still has yet to

monetize.

3. DOWNLOADS: Downloads of an app are the equivalent of an

account sign-up on the web and play a crucial role in the app’s

ranking. It’s important to monitor downloads with DAU to ensure

that both are trending positively. If you see a spike in downloads,

do you see a similar spike in DAU? If not, you may have to revisit

your retention features (and usage levels) or test different push

notification messages.

As an example, the Carrot To-Do app does a great job of using

humor in its push notifications to pull users back into the app.

Virtual currency and push notifications in Clash of Clans

Carrot To-Do in the App Store

8


5. APP CRASHES / FAILURES: While difficult to track,

it is important to have crash analytics built into your app.

Unfortunately, the default Google Play and Apple analytics do not

capture all crashes, so it’s important to use an additional crash

analytics tool.

To circumvent a crash

or feature request from

turning into a negative

review, app developers are

implementing feedback

functionality so users can

provide performance issues

or new feature suggestions

directly within the app.

For example, Tweetbot, a popular Twitter client, implements a

feedback mechanism within their settings page.

Tweetbot’s in-app feedback mechanism

Negative bug-related app review left by a Tweetbot user

9


HOW CONVERSION RATE OPTIMIZATION (CRO) FITS IN

02 Conversion rate optimization describes the structured and sys-

tematic approach to improving the performance of your website

or mobile app through analytics and user feedback. Simply put,

it’s the practice of running experiments to 1: find out why users

aren’t doing what you want them to do, 2: test your assumptions,

3: iterate, and 4: test again.

CRO is a long-established process that began in the early days

of the Internet -- long before smartphones entered the picture.

The process starts with understanding your company’s unique

objectives and key performance indicators (KPIs). Techniques such

as A/B testing and qualitative research allow business owners to

use data to make informed user-driven decisions.

To understand how CRO fits in with mobile, you need to

understand its application on the web. Before we jump back into

mobile-specific workflows, let’s start with a quick refresher by

reviewing the following nuts and bolts of CRO.

10


TESTING

A/B tests allow companies to compare page versions (or ele-

ments) against one another. Each page variation is then measured

by its effectiveness in comparison to the other. For example, a

marketer may compare a page with a green call-to-action (CTA)

to a page with a red CTA. A/B tests focus on only one element at

a time.

ANALYTICS

Analytics empower you to assess your current conversion rate,

identify trouble areas, and design an execution action.

Pay attention to the following metrics:

• CONVERSION RATE: This metric bridges the connection between

website traffic and revenue.

Are users taking the action that you would like them to? If not,

analyze your conversion flow and start testing the page(s) with

the biggest drop off. This metric will help you make sure that

users are completing desired goals within your app (making

purchases, returning quickly, encouraging friends to download

the app, etc.)

• BOUNCE RATE: This metric captures the percentage of visitors

who leave your website after viewing a single page.

If your web pages have a high bounce rate, it could be that

users are expecting something different than what you are

presenting. A high bounce rate could be a symptom of a con-

fusing, cruddy, or less-than-intuitive app design.

Bounce rates will tell you what’s happening, but you need to dig

deeper to understand why. Qualitative research -- with a service

like UserTesting -- can help you understand what users are

expecting from your website or mobile product.

• EXIT RATE: This metric captures the percentage of visitors who

leave your website after viewing a specific page.

Each page on your website will have its own exit rate. Pay close

attention to the exit rate on your crucial pages such as the

homepage, pricing page, and checkout flow. If you see a high

exit rate on a page during the checkout flow, it may be worth

testing that page to see if you can improve user retention.

• AVERAGE TIME ON SITE: This engagement metric gives you an idea

of how long people are staying on your website.

Are visitors who spend more time on your site more likely to

convert? Or are users spending a lot of time on your site, but

your conversion rate is very low? Optimizing these metrics will

reveal whether you need to make changes or test different call

to actions. Focus your attention to variables that are likely to

improve user engagement.

• AVERAGE PAGE VIEWS: This engagement metric tells you how

many pages the average visitor goes through before leaving

your website.

Average page views is another metric to measure in your con-

version optimization. Are visitors who see more pages on your

11


site more likely to convert? Are visitors who read 3 or more blog

posts likely to convert into trial users? If so, it is worth testing a

call to action at the end of each blog post.

Know your KPIs. These metrics will help you develop laser-focused

optimization experiments. On the web, these goals will typically

include leads, user account sign-ups, e-commerce checkouts, and

direct sales.

Although a completely different platform, on mobile, you will find

similar goals that relate to user engagement, retention, conversion,

and revenue. Example metrics to optimize include downloads,

conversions from free users into paid customers, and retention

rates.

KNOW YOUR KPIS. THESE METRICS WILL

HELP YOU DEVELOP LASER-FOCUSED

OPTIMIZATION EXPERIMENTS.

12


SEGMENTATION

No two visitors are alike. One visitor may be new to your brand

while the other may be a long-time customer. A well-optimized

website ensures that your brand aligns with both user experiences.

Businesses will typically design segments based on the following

characteristics:

• Where is the user in the sales funnel

• New versus repeat visitor

• The type of customer (small business vs. enterprise)

• Decision maker or influencer in purchase decision

• Demographic variables like gender and age

• Levels of technical sophistication

PERSONALIZATION

Personalization can be thought of both from a technical per-

spective and also from a marketing or promotional perspective.

From a technical point of view, personalization involves displaying

information about the user that has been saved from prior visits.

App developers can then create algorithms to behaviorally target

users with app features (or offers) that they’ll find relevant.

Amazon has done a great job of personalizing each user’s experi-

ence. Simple things such as displaying the visitor’s name, showing

the products previously viewed by the user, and present products

a visitor may be interested in based on past data.

Personalization can also be incorporated in your static marketing

or promotional copy. You can use surveys to find common

objections about your service or features that customers find most

valuable. You can use these findings to craft a more personalized,

human message. Speak in a language that resonates with your

audience. Your audience will see that your brand fully understand

their needs and that you have the right product to solve their

problems.

The bottom line of personalization is to show the right marketing

message to the right audience at exactly the right time -- in a

human, engaging, and personally relevant way. Present each user

with a unique experience on your website.

13


COMPARING UX & OPTIMIZATION: WEB VS. MOBILE

03User experience (UX) involves more than just aesthetics. This

concept encompasses the totality of how your user engages with

your website, product, or app. What’s important to keep in mind

is that user experiences span multiple touch points, especially for

customers who engage with your brand across multiple platforms.

You need to pay attention to the following:

• How easy your site or app is to use

• How fast your product is

• How much friction exists when users complete key actions

UX and CRO go hand-in-hand. Across platforms, you need to

guide your audiences through the conversion funnel.

There are a few major differences when it comes to designing

conversion optimized user experiences on the web versus on

mobile. In this section, you will discover the key differences and

how to best optimize for each screen.

14


UX MEETS CRO ON THE WEB

On the web, we are primarily stationary with our keyboard and

mouse. Key distractions include other browser tabs, IMs, email

notifications, and other software programs.

On the web, you’ll be focusing primarily on two aspects of UX:

1. Reducing friction in the

form of wasted clicks, excess

pages, slow page loads, and

other friction points that

cause users to give up.

2. Reducing doubt and

indecision from the user’s

mind that may cause them to

not convert.

DESIGN FLOWS NOT PAGES

Too often, we prematurely turn our focus to page design and

information architecture, when we should focus on the user flows

that need to be supported by our designs.

When starting your next web project, consider starting with a

detailed look at the objectives of the user and the business. This

practice will allow you to scheme the various flows that address

both parties.

User objectives could range from finding case studies to learning

more about a product to making a purchase. While business

objectives include email sign-ups to filling out a form to a phone

call.

Visitors to your website come from different sources with varying

levels of brand engagement. Each visitor has her own goals in

mind and it’s up to you to map

each user flow to the appropriate

conversion funnel.

It’s important to prioritize quality

over quantity of users.

In other words, it’s better to have

100 engaged, high-converting

users than 1,000 who use your

app once and then disappear.

High-quality users are especially important when using paid

acquisition channels. You want to make sure you are attracting

users that you can ultimately monetize.

Focus your efforts on the user flows that will have the biggest

impact on your visitors and result in the biggest gain for your

business.

IT’S BETTER TO HAVE 100 ENGAGED,

HIGH-CONVERTING USERS THAN

1,000 WHO USE YOUR APP ONCE

AND THEN DISAPPEAR.

LANDING PAGE

EMAILSIGNUP

SOCIAL MEDIA SHARE

15


BEST PRACTICES FOR UX DESIGN While there is no exact formula for the perfect landing page, there are common elements that you should monitor to maximize your conversions.

• MAIN HEADLINE – The landing page headline should comple-ment the source of the traffic. This is especially true when you are driving paid traffic to your landing page.

• SECONDARY HEADLINE – Try to address a specific point that is related to the content of the landing page rather than having a vague or uninteresting headline.

• COMPLIMENTARY VIDEO OR PHOTO – Customer testimonial videos, product demos, or strong visuals can have a positive impact on visitors.

• SOCIAL PROOF – Build trust with your visitor by incorporating trust symbols such as testimonials, press mentions, number of customers, and guarantee banners.

• STRONG CALL TO ACTIONS – The primary call to action button should stand out from the rest of the page and contain words that your visitors might be searching for such as “free,” “buy,” “download,” etc.

• STREAMLINED LINKS – Simplicity is key to an optimized landing page. Keep links to other parts of your website to a minimum, as they distract your visitors and can have a negative impact on conversions.

• CONTINUOUS TESTING – Continue to test different headlines, call to actions, and videos to see what resonates with your visitors. In addition, consider testing two completely different site designs. This comparison can have a big impact to your long-term success.

UX MEETS CRO ON MOBILE

Although you are working with a much smaller screen size, the

same basic web UX principles apply to mobile.

It is still important to start with the objectives of the user. Starting

with a goal-first approach, focus on designing your user flows

before thinking about what you’re crafting on each screen. You

also want to reduce friction in the form of wasted taps, excess ads

or screens, big app sizes, and complicated user flows.

DESIGN IS ESSENTIAL ON MOBILE

Apple’s culture revolves around design excellence. It’s no coinci-

dence that Apple’s apps have surged in popularity -- leading some

to argue that design is the most critical component in building a

successful app. Here are some tips to get you started:

• 1. POSTPONE SIGN-UP TO AVOID ABANDONMENT: App developers

frequently struggle to create interfaces with continuous user

engagement. That’s because consumers are non-committed.

In fact, the average iOS and Android user only downloads three

to five apps per month.

On top of that, 26% of the apps are only opened once and never

used again, and another 48% are opened 10 times or less.

Moreover, mobile users have become weary of apps that auto-

matically post activity to social media. To earn consumer trust,

app developers are moving towards a new trend, requiring users

to sign up after they’ve become dedicated users. Forcing a user

to sign up too early may result in one-star reviews and complete

abandonment.

16


• 2. CONSIDER GESTURES: The Apple iOS Human Interface Guidelines

emphasize that an app should allow users to directly manipulate

on-screen objects. By simply allowing users to tap, pinch,

or swipe the screen you will find that your users are more

engaged. It will be easier for users to experience the results of

their actions.

In an iOS app, people experience direct manipulation when

they:

• Rotate or otherwise move the device to affect on-screen

objects

• Use gestures to manipulate on-screen objects

• Can see that their actions have immediate, visible results

• 3. ENFORCE CONSTRAINTS: One of the biggest challenges in mobile

design is undoing the years of designing on the desktop and

the web with its large screen sizes. On the web, it makes total

sense to create long pieces of content or forms with many

fields -- users are sitting at a desk and have the patience (and

computing power) to absorb more information.

On mobile, however, every new page or field can add more

complexity to the app and make for a poor user experience.

Product owners need to approach app design with a ‘mobile-

first’ mindset. In other words, stop thinking like you’re designing

for the web. Mobile presents a completely new user experience.

You need self-control when designing mobile apps. Here’s

what we mean by ‘enforcing constraints’:

Developers can avoid the ‘content and information overload’

problem by stripping the app – its screen and elements – to

the bare essentials. Great mobile design empowers users to

do more with less. For example, your app can allow users to

perform all the necessary features within the app with just the

use of one thumb. Or remove all navigation within your app and

allow the user to use gestures to navigate through your app.

Key Gestures on mobile Devices

17


Let’s take the two to-do list apps in the screenshot as examples.

The one of the left appears cluttered and hard to navigate. And

it is not completely clear how a user can mark a task as com-

plete. In addition, there is wasted screen space at the bottom

where the “Home” and “Add” buttons reside.

But the one on the right, Clear, shows all the tasks in a neat

manner and uses the iPhone’s built-in gestures to manipulate

your to-do. Users can pull down on the app to add a new to-do

list or swipe to the left to mark a task as complete.

• 4. RESPECT THE TAP: Every swipe, tap, and pinch that a user takes

will have a lasting impact on UX. Each small gesture should

help a user make progress. Think: lots of return for very minimal

effort.

Sam Shank, CEO of Hotel Tonight, illustrates that booking a hotel

on the app requires only three taps and a swipe – totaling about

eight seconds. Compare those stats to the apps of Priceline (52

taps, 102 seconds) and Hotel.com (40 taps, 109 seconds). It is

no wonder that Hotel Tonight has exploded as a mobile only

company.

A cluttered to do list app (left) vs. a simplified, gestures-first to do list app (right)

18


CASE STUDY: TASKRABBIT - WEB VS MOBILE

Let’s take a look at TaskRabbit and

the key differences between their

UX on the web and on mobile.

HOMEPAGE: On the web,

TaskRabbit has a lot more

screen real estate, so they have

decided to promote becoming a

TaskRabbit, posting a task, and

“TaskRabbit for Business.” On

mobile the company focuses on

having users sign up or log in

immediately upon launch.

TASKS SCREEN: In this screen, you

will notice that the web version

has a list of all the different

categories of tasks. On the web,

TaskRabbit can also use more

visual content along with a clear

call to action.

DESKTOP HOMEPAGE

DESKTOP TASKS SCREEN

MOBILE HOMEPAGE

MOBILE TASKS SCREEN

19


POST A TASK SCREEN: The web version of this screen is the basic

form that most audiences are accustomed to seeing.

In the mobile version, each section of the form – Location,

Supplies, Deadline, etc. – has its own screen and users have to nav-

igate back to complete the entire form. While this UX adds more

taps and screens to the overall flow, the app varies the gestures of

each screen so users are engaged with each section.

DESKTOP POST A TASK SCREEN MOBILE POST A TASK SCREEN

TECHNICAL DYNAMICS ON THE WEB VS.

MOBILE

Eric Ries in his book The Lean Startup pop-

ularized the term minimum viable product

(MVP) - a development technique in which a

new product or website is created with the few

features necessary to satisfy early adopters.

The final, complete set of features is only

designed and developed after considering

feedback from the product’s initial users.

While the concept of a minimum viable

product has exploded on the web, it is still very

difficult to build minimum viable products on

mobile.

On the web, changes can be implemented and

pushed to production in a matter of minutes.

On mobile, however, developers must account

for each app store’s review process, time for

users to upgrade their apps, and the nuances

of each mobile platform.

To further explain the technical dynamics of

developing and designing for the web versus

mobile, we asked someone who has familiarity

building products in both spaces.

20


Steve P. Young led Product Marketing for SmartShoot - an online

marketplace of professional photographers and filmmakers. He

was the lead product manager of SmartShoot’s bid platform

that allowed anyone to submit a photo or video brief and receive

quotes from the network.

He also is an app developer with more than 10 apps in the App

Store and Google Play Store. Furthermore, he hosts a podcast,

Mobile App Chat, where he’s interviewed over 100 app developers

including Tweetbot, Shazam, Facebook Messenger, and many

more.

Q&A WITH STEVE P. YOUNG

OPTIMIZELY: What are the key differences in building on the web versus

mobile?

STEVE: The main difference is speed of development. On the web, there

are many tools that already exist that allow us to quickly build prototypes

and get it in front of users. Some tools are so easy to use that marketers

these days can test their business hypotheses without the help of

engineering.

However, on mobile those tools do not exist. Development requires time to

code, publish on the app stores, and analyze the results.

In addition, on mobile you have to test on different devices with varying

screen sizes and computing power.

And with the recent release of iOS 7, many app developers had to

overhaul their entire design. Can you imagine Chrome or Firefox dictating

how a website should look and operate?

OPTIMIZELY: What about the key differences in user experience?

STEVE: On the web, users are a little more forgiving. When you reach a

404 page, you can just hit the back button and find your way around a

website. Moreover, many websites provide an online chat feature, so users

can immediately get answers in case they are lost.

This is certainly not the case on mobile. If an app isn’t intuitive and its

design confuses the user, there’s no way to get an immediate response

from the developer. Also if an app crashes, users are generally quick to

leave a negative review or simply delete the app.

OPTIMIZELY: From your experience what have you learned about the

mobile space?

STEVE: It’s definitely an exciting space to be in. Things are constantly

changing and you are hearing more success stories from indie developers.

Through my 100+ interviews, here are three key lessons about building

apps:

• Be a feature: Some of the best apps in the world solve one specific

need. On the web, we like our tools to have a lot of features. However,

on mobile we are happy when our apps to just do one thing really

well.

• Don’t distill the web onto mobile: With such a small screen size,

don’t try to distill what you have on the web onto a mobile screen.

The experience should be completely different and you should take

advantage of the touch screens and incorporate taps and swipes into

your interface.

• Build a minimum viable experience/value: It is incredibly hard to

rapidly prototype on mobile, so think about creative ways you could

prototype the experience or value of your app. Don’t start with

interface of your app, start with the experience you’d like the user to

have with your app.

21


CRO ON MOBILE: ACQUISITION

04

Thanks to Optimizely, A/B testing and optimization on the web

requires little technical assistance and presents few barriers.

Mobile is a different story.

Marketers on the web have easier access to data and with the help

of tools can make changes on the fly without engineering help.

On mobile, marketers and product managers are more reliant on

engineering teams and do not have the freedom to make changes

like they do on the web.

22


Fortunately, there are some non-technical “hacks” that marketers

and product managers can do to help optimize adoption and

engagement with their apps.

Before we do that let’s take a look at a typical conversion funnel

for an app:

OPTIMIZING FOR APP STORES

App Store Optimization (ASO) is the equivalent of search engine

optimization (SEO) for mobile apps. The goal is to make it easy

for users to find your app in the App Store through careful selec-

tion of keywords that specific user intents.

The nice thing about ASO is that it requires very little technical

knowledge and often times can be optimized without the need of

an engineering team. However, it’s important to note that ASO is

not a real-time A/B test. You need to compare data from before

and after the change.

In fact, when performing ASO, you are analyzing past results with

current results to see if the change you made had an impact.

App Store Optimization tests can involve:

• APP ICON: It’s important to test your icon even before releasing

the app. You can buy some ads on mobile advertising networks

such as Admob or online advertising through Google Adwords

and Facebook. What you want to find out is which app icon is

getting the highest click through rate.

• APP NAME: According to Nielsen, 63% of Android and iOS users

have utilized search to discover new apps. That’s why it’s

important to optimize your app name with highly sough after

keyword phrases. ASO tools such as Straply and Sensor Tower

are great places to start. In the online marketing world, having

keywords in the title of your web page helps with SEO. In the

app store, the same principle applies with your app name.

Awareness

App Store Search

Download

Open App

Registration/Key Action

Usage &Retention

3. HTTP://WWW.SLIDESHARE.NET/VICTORI98PT/STATE-OF-MOBILE-Q32011-BY-NIELSEN%20

3

23


• KEYWORDS: On the web, keyword meta-data no longer influences

search rankings. An app’s keyword data, however, actually has an

impact on its search rank. Along with your app’s name, having

relevant, high-traffic words in your keyword meta-data can help

improve your apps search ranking.

• SCREENSHOTS: While the term may imply a screen capture of

your app, you should think of your screenshots as a banner

advertisement. Both the Google Play and Apple App Store

prominently display screenshots rather than an app’s description.

Besides the app icon, your app’s screenshot is the strongest

visual representation of your app, so you want to draw the user

to download it. Again, just like the app icon, you can test using

traditional banner ads to see which screenshot yields the highest

click throughs.

OPTIMIZING FOR REGISTRATION/KEY ACTIONS

A recent study from Compuware shows that 80-90% of apps are

deleted after only one use. Mobile customers are unforgiving. If you

do not make an immediate first impression, it’s likely that your app

will be deleted.

Without an optimized onboarding experience, you are not only

losing users, but also an opportunity to monetize your app.

Let’s unpack this a little more. Unlike the web where we have mul-

tiple tabs open and going to a website takes a matter of seconds,

on mobile we have to go through the trouble of finding your app,

downloading it, and then remembering to actually open it.

The user has gone through three steps just to see what your app

has to offer, so retaining as much of that 90% as possible is critical.

How do you ensure you make the best first impression possible?

Make sure your app does not make some of these common mis-

takes we see from app developers during the onboarding process:

REGISTER FIRST, TRY LATER: Apps that require registration before

using it, lose up to 56 p ercent of its users. Mobile users want to

see and engage with your app before deciding to join.

LENGTHY REGISTRATIONS: The mobile screen is small enough, so

requiring a lot of information from your users will have a negative

effect on their usage. You may want to use multiple screens to

allow the user to enter in bite-size information. For example,

having three screens with 2-3 fields each is much better than

having one screen with 9 fields.

LONG LOAD TIME: Mobile customers have a low tolerance for unsta-

ble and buggy apps. Apple has emphasized design and UX within

its default apps, so users have come to expect elegant designs and

intuitive user flows.

WITHOUT AN OPTIMIZED ONBOARDING EXPERIENCE, YOU ARE NOT ONLY LOSING USERS, BUT ALSO AN OPPORTUNITY TO MONETIZE YOUR APP.

24


CRO ON MOBILE: RETENTION

05

As outlined in this paper, mobile app optimization can yield huge

benefits for app developers. At the same time the reality of today’s

marketplace is that users are interacting with brands and products

across desktop web and native mobile apps.

However given the technical constraints of executing an A/B test or

a personalization campaign within a native mobile app, many app

developers will simply not test at all. This leaves valuable valuable

opportunities on the table for better conversion rates, higher

engagement, and increased revenue.

25


That’s why Optimizely is investing in product enhancements that

enable customers to optimize end-to-end experiences across all

devices, websites, and apps their visitors are using. Our goal is to

reduce the friction required to run in-app experiments that lead to

optimized user experiences that achieve the goals app developers

have set out to accomplish.

TECHNICAL CONSIDERATIONS

Before deciding on a mobile app optimization strategy, it’s import-

ant to consider the technical challenges mentioned above. There is

no mobile app equivalent to the DOM (Document Object Model),

which can be manipulated by JavaScript. This introduces the

complexity of requiring engineering support to test changes that

would be much more straightforward to implement on the web.

Moreover companies are becoming increasingly reliant upon third

party SDKs for analytics, monetization, and optimization, all of

which require technical knowledge.

Here are some key technical considerations. While not specific to

A/B testing and optimization, they are important to keep in mind:

APP STORE REVIEW: Each app store has its own review cycle. On

Google Play it could be a matter of hours, where on the Apple App

Store, it could be anywhere from 5-7 days. This delay makes it hard

for developers to push changes. Control remains squarely in the

hands of the platforms. While tools like Parse can make it possible

to release updates quicker, it is still difficult for a non-technical

marketer or product owner to have control over an A/B test or

feature change.

USERS MUST UPGRADE APP: The landscape changed a bit when Apple

introduced automatic updates with iOS7, but it is still no guarantee

that users have this featured turned on. Some power users have

actually turned this feature off to save battery life. Even with the

feature turned on, it is hard to predict when the update will reach

every user.

USERS MAY NOT BE ONLINE: We have all had connection problems

here and there. While this may not be a huge concern, it is still

something to consider when releasing an update. You can’t always

guarantee that users will have Internet connection, so you need to

ensure that your app has the proper messaging in place to address

offline issues. Users can always upgrade when they’re back online,

but in the interim, they may feel frustrated. This one moment of

frustration has the potential to wreak havoc on user engagement.

As a marketer, you may think -- ‘well, they’ll hop online later.’

Keep in mind, however, that consumers want instant results and

gratification.

PRODUCT CONSIDERATIONS

Technical considerations aside, we don’t see much of a difference

between A/B testing and optimization on the web vs. mobile. As

mentioned above, Optimizely is expanding its already market-lead-

ing web optimization platform to support mobile app A/B testing

and optimization.

We built Optimizely for mobile apps using the same philosophy as

we did when building our web product. Of course, it’s also import-

ant to recognize and account for the fact that there are tactical

differences when optimizing across devices.

26


When choosing your native mobile app optimization strategy, it’s

important to keep these tenets in mind:

CROSS-PLATFORM. Users are the same across web and mobile, but

their behaviors and motivations are different. That’s why it is

important to choose a native app optimization framework that

works across both the web and mobile devices. This framework

should allow you to craft different experiences to cater to

device-specific needs, while still carrying over information between

web and mobile to make sure the experiences is consistent no

matter where the user is engaging.

FAST. The app development space is one of the fastest-paced that

exists today. It’s never been more important to build apps that

react to the needs of your users and to the market in general.

Any app optimization platform you choose should take this into

account. It should be easy to roll out changes without a code push.

It should minimize dependence on developer time as much as

possible.

ANALYTICS AND INSIGHTS. Experimentation and optimization is no

good without concrete data to take action on. When choosing an

optimization platform, it’s critical to pick one that will give you the

insights an analysis you need to make decisions using the data

that’s being generated.

SPEED, STABILITY, AND SECURITY. Mobile apps are different than

websites in that they’re ultimately end-to-end programs that users

are installing on their devices. Once an app is installed, it needs to

be able to function and scale independently. It’s important that any

solution you choose does not compromise the fidelity of your app,

because that will ultimately reflect on your app’s rating and, by

extension, its future success.

INTEGRATES YOUR ECOSYSTEM. If your app is like most apps, it’s

already leveraging third party SDKs for things like analytics and

crash reporting. For something as fundamental as A/B testing

and optimization, it’s critical to choose a platform that integrates

with the most commonly used mobile app SDKs out there. Your

team shouldn’t have to relearn a new set of tools or define a new

workflow just to optimize your app. Barriers like that can prevent

you from being able to implement a culture of experimentation

within your organization.

USERS ARE THE SAME ACROSS WEB AND MOBILE, BUT THEIR BEHAVIORS AND MOTIVATIONS ARE DIFFERENT.

27


CONCLUSION

It’s never been more important to deliver compelling experiences

that delight and engage users on mobile devices. For years,

marketers and product managers have embraced a culture of

experimentation and testing, and as a result, web experiences are

becoming increasingly effective at improving users’ experiences

while also helping companies accomplish their goals.

However this culture of experimentation has not yet made it’s way

into the realm of mobile app development. This is largely due to

the reality that implementing a mobile app optimization strategy

typically requires a prohibitively high level of involvement from

engineering teams who could otherwise be working on feature

development.

In developing its mobile app optimization framework, Optimizely’s

goal is to reduce the necessity to involve engineering teams to

the extent that it can be adopted in the mobile app community as

heavily as it has been adopted in the web community.

Learn more about Optimizely’s mobile app optimization platform

at http://optimizely.com/mobile.

http://optimizely.com/mobile

ABOUT OPTIMIZELY

Optimizely is the world’s leading experience optimization platform,

providing A/B testing, multivariate testing, and personalization for

individuals and organizations with and without technical expertise.

The platform’s ease of use empowers organizations to conceive

of and run experiments that help them make better data-driven

decisions. With targeting and segmentation using powerful real-

time data, Optimizely meets the diverse needs of any business

looking to deliver unique experiences to their visitors.

OPTIMIZELY.COM/MOBILE

Email: [email protected]

Phone: 1-800-252-9480

Facebook: facebook.com/optimizely

Twitter: twitter.com/optimizely

Amsterdam OfficeNes 761012 KE Amsterdam

San Francisco Office631 Howard Street, Suite 100San Francisco, CA 94105

To learn more about Optimizely for Mobile Apps, visit

ABOUT THIS GUIDE A Blueprint for A/B Testing and Optimization on Native Mobile Apps

Written By:Sean OliverSenior Product Marketing Manager, Optimizely@SeanOliver

Designed By: Jon Saquing@jsaq

Thank you to Ural Cebeci, Steve Young, and Ritika Puri.

https://www.optimizely.com/mobile

mailto:hello%40optimizely.com?subject=Hello%20Optimizely%21

http://www.facebook.com/optimizely

http://www.twitter.com/optimizely