How growth teams are revolutionizing UX and product development

Scaling Startup Marketing

How growth teams are revolutionizing UX and product development#UTwebinar @UserTesting

Casey Winters

Former Product Lead on the Growth Team, Pinterest

UserTestings on demand customer insights platform enables companies to create powerful products and exceptional customer experiences.

#UTwebinar

TALK TRACK - OVERALL SLIDEWe help over 1,400 companies, including all of the top 10 web properties, get insights when they need them.

ALTERNATIVE SLIDES: There are other slides at the end of this presentation that can be altered based on the type of customer you are speaking with.2

Laurie Kuhn () - [email protected] Confirming that this would be the one logo slide. It has all three elements: big brands (see them, yes.) logos reflecting a specific industry? logos reflecting company size? can you share your thoughts about how many logos per industry and per company size, and for this slide - what they are?Kiley Rundle () - This is the main logo slide, yes. Im not sure how Phil broke them down when he sent me the additional logos, it was just titled "Corporate". Perhaps we could separate them by rows, so that the first 3 could be big brands, the next two could be company size, and then the last row could be specific industry.

Generally when I am creating a logo slide, I mix them around based on the size, shape and color of the logo, which is what i did with this slide, but if you want it to be controlled by the sales person then it may not look as presentable.Laurie Kuhn () - i love breaking them down by how we would talk about it - big guys, then industry would be second. people want to see that we have customers in their line of business, that we "get" them. and then finally one row about their size. perfect!Kiley Rundle () - great!

We Partner With Thousands Of Customers

#UTwebinar

TALK TRACK - OVERALL SLIDEWe help over 1,400 companies, including all of the top 10 web properties, get insights when they need them.

ALTERNATIVE SLIDES: There are other slides at the end of this presentation that can be altered based on the type of customer you are speaking with.3

Laurie Kuhn () - [email protected] Confirming that this would be the one logo slide. It has all three elements: big brands (see them, yes.) logos reflecting a specific industry? logos reflecting company size? can you share your thoughts about how many logos per industry and per company size, and for this slide - what they are?Kiley Rundle () - This is the main logo slide, yes. Im not sure how Phil broke them down when he sent me the additional logos, it was just titled "Corporate". Perhaps we could separate them by rows, so that the first 3 could be big brands, the next two could be company size, and then the last row could be specific industry.

Generally when I am creating a logo slide, I mix them around based on the size, shape and color of the logo, which is what i did with this slide, but if you want it to be controlled by the sales person then it may not look as presentable.Laurie Kuhn () - i love breaking them down by how we would talk about it - big guys, then industry would be second. people want to see that we have customers in their line of business, that we "get" them. and then finally one row about their size. perfect!Kiley Rundle () - great!Across All Platforms, At All Stages Of Development

Watch customers interact with:

DESKTOPSMARTPHONETABLETWEBSITES

APPS & PROGRAMS

PROTOTYPES & WIREFRAMES

SURVEYS

PHYSICAL PRODUCTS

Anywhere people interact with your brand:AT HOMEAT A STOREON THE GOOn any platform or device:

#UTwebinar

8 Casey WintersFormer Product Lead on the Growth Team, Pinterest Casey Winters has been working on growing technology companies for over a decade and now advises tech companies on their growth strategies. Prior to advising, Casey spent the last three years leading growth product efforts at Pinterest, helping the company grow to over 150 million monthly active users. Prior to Pinterest, Casey was the first marketer at Grubhub, an online food delivery service, helping the company grow from three cities to over 500.@onecaseman

#UTwebinar

TALK TRACK

Before we jump into the agenda, Id love to take a second to see who we have on the line. Lets do quick intros where you share your role and how youd work with us if we kick off our partnership.

Thanks everyone. Im [Name], and Im an Account Executive here at UserTesting. We work with lots of [your title] on a regular basis so youre in good hands. In fact, just last [week/month] I was chatting with [company name or similar company (if its true)]

From talking with [SDR name] I understand that you [have X need or are looking for X or are interested in X]. ADD more content here from disco call. Get confirmation that what we discovered is the case. How we solve those challenges

Is there anything else that youd like to make sure you come away with after our call?

By the end of our conversation well make sure you have the information you need.

Lets start by quickly talking about who we are

DESIGN NOTES:Static image OK, no animation needed. This will be the slide that shows while people load into a call. It will need to be customized with the UserTesting persons information and photo who is leading the demo.

Meeting Feedback: Get a photo of each sales rep headshot. Plan to shoot on X day around All Hands. (Stef creating plan)5

Why Growth Teams are Revolutionizing UX and Product Development

CASEY WINTERS

Casey Winters | @onecaseman | caseyaccidental.com

6

Casey Winters | @onecaseman | caseyaccidental.comThe purpose of a growth teamScale usage of a product which has product market fit

Build a playbook on how to do 1

So, before we get into that, lets talk about the purpose of a growth team. From a business perspective, its goals is to scale usage of a product which has product market fit. If you dont have product market fit, you shouldnt be worried about growth. You should be worried about getting to product market fit. Now I could talk about what that looks like, and I have in previosu presentations, but for today, well assume we all have product market fit and want to growth.

The second business goal of a growth team is to build a playbook on how to scale, optimize it, and socialize it to the rest of the organization.

You may here this purpose described in different ways, like managing the inflow and outflow of users.7

Casey Winters | @onecaseman | caseyaccidental.comSo why growth teams?Why not marketing teams?The biggest growth drivers tend to come from within the product

Marketing teams dont have access or expertise in changing the product

So why growth teams? Why not marketing teams? Marketers used to be responsible for getting customers, and product teams responsible for making a product that is a good experience for them. What changed?8

Why is growth different from traditional product development & UX?Most product teams are built to create value or improve value provided to customersGrowth teams focus on connecting more people to the existing value of a productThis requires a different mindset that is more iterative and analytical than traditional product developmentMore experiments or shots on goalIt also requires teams to balance business value and user value much more directlyCasey Winters | @onecaseman | caseyaccidental.com

9

What does a growth team looklike at scale?Sub-teams devoted to key areas, usually around parts of the funnelCarved off real estate from core product for swim lanesSpecial teams start to emerge at larger companies focused on key friction areas without swim lanesCasey Winters | @onecaseman | caseyaccidental.com

So, if we know not to focus on total signups, what is growth about then? I separate it into five buckets, but for some businesses, it may just be three or four. Those buckets are acquisition, activation, retention, referral, and revenue. Dave McClure calls these startup marketing for pirates, or AARRR.

Now, when is it five and when is it three buckets? Well, for a business like GrubHub, acquisition and activation are the same thing. You only sign up to order food, on which youre activated. One could also make the argument theres also no difference between retention and monetization for GrubHub. The only way to use the product is to order food, so any sort of retention is monetization. Thats not exactly true, because monetization doesnt just mean that you make money off a user, but how much. And even for a business like GrubHub, things like upsells of drinks and appetizers, or tweaking the sorting algorithm so that the restaurants you make the highest commission off can make a business more money off the same type of retention.

Now, lets compare that to Pinterest. We get plenty of people who sign up, but dont ever pin things. We have about the most healthy rate of acquisition to activation Ive ever seen for a business, but theres still a gap. Also, someone will be able to engage on Pinterest without necessarily having that be a monetizable activity. So retention and monetization are also separate things to track.

Now, you may notice that awareness is not on this list. Its not because awareness isnt important. Its because its not quantifiable without methods that are too expensive for a startup to think about, and because its actually a bad thing to think about most of the time for a startup. Ive seen so many companies waste money on bad marketing chalking it up to awareness. You cant operate lean and yet throw dollars down a hole in the name of awareness. Its best to pretend it doesnt matter.10

What does team structure look like?Dedicated PM, designers, engineers, analysts, researchers, marketers, operations for each sub-team


11

SEO Friendly Landing Pages


So Im going to start with SEO, which may seem weird. SEO is really about two things. Authority and relevance. Authority is mainly generated by the quantity and quality of external links to your page and domain. Relevance is about whether the keywords people are searching for are prevalent on your page. Were talking about UX today, so well focus on relevance

For relevance, at both Pinterest and GrubHub we did the same thing. We took all of our content, and we made targeted keyword landing pages from them. So, at GrubHub, we had all these local restaurants, and we could create new pages for them for any geo or cuisine combination, so "nob hill chinese delivery".

This is UX work, but you have two constituents, the user and the search engine bot. The bot has to understand what youre relevant for the page to reach users.

12

SEO Friendly Landing Pages


At Pinterest, users did this themselves by organizing Pins into boards on different topics. But a board was one person's opinion on what was relevant, whereas Pinterest had all this data on what the most relevant Pins were already. So, we created our own topic pages that organized the best Pins on each topic, like James Turrell. And both search engines and users responded well. More traffic, and better conversion on the pages. Now 20% of Pinterests traffic from Google goes to these pages, and they have the highest conversion of any page type into signup.

13

But conversion was still terrible


We started receiving a lot of traffic directly to these content pages from referring sources like Google, Facebook et al. These pages did not ask for signup really at all, and as a result, few people did.

Gift Wrap put a translucent black banner at the bottom of the screen asking for signup, and stopped the scroll of Pins at 25. Also, if you clicked on a specific Pin, we asked you to signup to see it. It basically gave you a free taste, then made you sign up to see all the good stuff we have. This took a couple of days and had over a 50% improvement on conversion rate on the first experiment.14

Where are people actually landing on your service?


This is what happened when you clicked on an individual Pin on a page. Super simple test to validate an idea. Huge gains for the company.

Now, this is one of those user value vs. business value tradeoffs I talked about. For the user, the better short term experience is letting them see all the content. But its at the expense of the long term value, which is Pinterest learning what you like and recommending the most relevant content to you on a regular basis.

So, once this works, we iterate on the UX to make it perform better and feel better. 15

Once you find a winning idea, iterate on that idea until you cant anymore.


Gift Wrap was a very quick project with quick returns, and it provided us a framework for what to do when people land on content pages that increases conversion. So, from there, we iterated on it a ton, polished designs, and this is what we have today. We also noticed it worked better on different page types, so what we did is we made all the page types look as close to each other as possible.16

What do I get if I do what you want?


17

WOM via Product Quality

Clear Value PropositionsCasey Winters | @onecaseman | caseyaccidental.com

So, people might be surprised to see compelling value prop what I talk about first. Isnt that a marketing or product job? Well, growth teams are constantly testing to find the simplest way to describe their products.

Robinhood is a great example of this. When you land on their home page, you see this. Could not be clearer what the value is, which is traditionally hard for financial products, and that its on your phone. By the way, this image actually changes colors and phones to show its on both iOS and Android.

At Pinterest, we really struggled with this. If you tell people youre tool that will show them ideas on how to imprve their life, people are like yeah, theres a whole section of the bookstore for that called self-help. I dont need that. So you can see that we figured out people seeing content they liked is what mattered, so we showed them that and made the value prop the ability to see more of those types of ideas.

18

The Mobile EquationMobile signup vs. App installDo an AB test


While signing up people on the web is great, mobile is a bigger opportunity. The first thing we tried to learn is our mobile equation. If someone lands on your mobile website, are you better off asking them to sign up then and there or prompt them to install the app immediately. So, we did an AB test. What we found is that pushing for an app download dramatically decreases signups, but actually drives more engaged users over the long term. So it was better for our business to push for app download as soon as possible.

By the way, Im intentionally showing you the shitty MVPs so you can see how growth teams test. We dont polish until weve validated its worth polishing.19

App Interstitial Experiment Frenzy


So, we tried what most people tried: interstitials. After a bunch of iterations, we landed on this. Something that rotates the content behind the interstitial at the top, but still prompts for app download at the bottom. While this resulted in less signups for Pinterest, we ended up with more engaged users because those users started on a better experience with the app. So, its important to remember what youre optimizing for, which is usage, not signups. This may not be true for every business. You need to measure whether asking for mobile web usage or prompting mobile app usage is the right move for your business. What works for us here may not work for, say, Airbnb, who has a very different business.

Now, this interstitial tactic is falling out of favor with Google, and in November, they switched to marking sites with app interstitials as non-mobile friendly (even though they do it themselves). So we now have a header instead of an interstitial.

Now, we also went a step further. If you continue in the browser and click on anything, well deep link you if you have the app or send you to the App Store/Google Play if you do not (this is called Seamless). We explicitly made the call that the mobile web was not the best experience, and we didnt want that to be someone first experience with Pinterest. These experiments combined had our biggest impact on mobile growth.

This type of work also impacts your organic ranking on the App Store/Google Play. And without spending any money on it like apps like Game of War with their television commercials and app installs, were consistently the top ranked independent app on the App Store/Google Play. So, we like to joke like one engineer on our team drove more app installs than Kate Upton.20

Optimizing further downthe mobile funnel


We go a step further and let you scroll endlessly, but as soon as you click on something, we tried either opening the app or sending you to the App Store or Google Play. This worked extremely well, but it was a bit of a janky experience. So, we created this primer telling you what we doing. The primer itself increased conversion rates.21

What about when they return to their browser?


One of the things we noticed is that as an app is downloading, people do other things. They may then forget to open the app. So, when you return to your browser after going to the App Store/Google Play, we remind you about our decision to download the app.

Some people do this via JavaScript, leaving some lame pages behind that dont even work when you return to the browser. 22

Putting it all together


So heres the whole flow with our latest design.23

What about when they open the app?


What about when they open the app for the first time? We talked about reducing friction to drive retention, what about for driving conversion. The first thing we did is preserve the context from what you were looking at on mobile web when we asked you to download the app. Were using Branch Metrics to preserve this context. Instead of showing a generic home page, we show you exactly the same context you were looking at before. We do this on both platforms.24

Casey Winters | @onecaseman | caseyaccidental.comLets go back to signup

Even though we had built a custom flow from the mobile web all the way to an initial home feed where every individual component experiment had increased active users, we felt there was a lot more on the table.Our signup flow was a bunch of fields, and we asked for it all at once.25

Casey Winters | @onecaseman | caseyaccidental.comGoing multi-step

Even though we had built a custom flow from the mobile web all the way to an initial home feed where every individual component experiment had increased active users, we felt there was a lot more on the table.Our signup flow was a bunch of fields, and we asked for it all at once. We tried splitting it up into steps, and saw more completed signups. Its easier to digest one question at a time on mobile. We added a progress bar, so you knew it wouldnt take a ton of time.

So our fields now are email, name, password, age, gender. So, then we asked what could we get rid of from a step perspective? Well, you can read the email on Android and pre-populate it. So, we did that. So, then we asked, could we get rid of passwords? Yes, we could. A password is not needed as we send an authentication token on signup that we match with an email address every time someone attempts to login to a device. If you login on a device you have logged into before, we already have that token. If someone then wants to sign in on web, we can send a push to the device or an email token.

Then, we went further. Could we get rid of name? On Android, you can read contact list for the full name of the user or infer based on email address. All of this improved mobile conversion. Then, we said, what if the person didnt have an email address? This especially is true internationally and with young people. So, we tried accepting phone numbers, and this improved conversion as well. We also switched the copy here to continue from signup so we could prevent web users from accidentally signing up with new accounts. We check the email, and if you exists, we either ask for your password or let you in. If not, we ask you to go through the signup flow. This allowed us to remove a login button.

26

Casey Winters | @onecaseman | caseyaccidental.comGoing multi-step

So our fields now are email, name, password, age, gender. So, then we asked what could we get rid of from a step perspective?27

Casey Winters | @onecaseman | caseyaccidental.comOptimizing multi-step login

So our fields now are email, name, password, age, gender. Well, you can read the email on Android and pre-populate it. So, we did that. On previous versions of Android, this was a permission you got on app download. On Marshmallow, you ask in context. This may create different behavior, so watch out for it. Internationally, most people are not on marshmallow, and wont be for a while.

We also tested continue instead of signup because we had trouble with people accidentally creating second account.28


This gave us the ability to ask for age without adding a step, which was helpful for our monetization team.

29

Casey Winters | @onecaseman | caseyaccidental.comOptimizing multi-step login

Now, when youre transferring your business from web to mobile, if youre successful, youll have a lot of current users switch to mobile. They were accidentally creating second accounts frequently. Now, when you type in an email, or we read it, when you hit continue, we check to see if we can log you in first, and if so, send you to the login flow.

30


So, then we asked, if were reading the email address, can we read the name too? So, we did that. Were also testing inferring the name from the email address. This is the same permission on Android as is fetching the email address. Its about accessing the contact list.31

Lets go back to signup


So, then we asked, could we get rid of passwords? Yes, we could. A password is not needed as we send an authentication token on signup that we match with an email address every time someone attempts to login to a device. If you login on a device you have logged into before, we already have that token. If someone then wants to sign in on web, we can send a push to the device or an email with a login token. Now, on Android prior to Marshmallow, you get push permissions by default. On Marshmallow and for iOS, you have to ask in context. Since very few of our users are on Marshmallow, we dont worry about priming, but that is a frontier well need to address.

Now were down to three steps on Android while adding in an additional field during the process. This is all rolling out now. And on iOS were going to test name inference and passwordless soon.

All of this improved mobile conversion. Then, we said, what if the person didnt have an email address? This especially is true internationally and with young people. So, we tried accepting phone numbers, and this improved conversion as well. We also switched the copy here to continue from signup so we could prevent web users from accidentally signing up with new accounts. We check the email, and if you exists, we either ask for your password or let you in. If not, we ask you to go through the signup flow. This allowed us to remove a login button.

32

Optimizing Personalized ContentAsk people which topics theyre interested inCustomize these with any data you have on the user


33

Optimizing Onboarding and EducationGet people to the core value your product provides as fast as possible (but not faster)

Dont be afraid to educate (contextually)


Onboarding and education may be a big or small opportunity given how complicated your product is. At Grubub, 85% of people ordered the same day they got an account. So we never talked about it. At Pinterest, it was a huge focus of ours. So, whats the goal of onboarding? To get people to the core value your product provides as fast as possible (but not faster)

So, if you look at this screenshot of Pinterest, this is what happens after I click those topics from a few slides earlier. I am presented the core product experience, a feed of content that matches my interests. Its not a blank screen. This is called addressing the cold start problem.

You hear a common idiom thrown around startups that if your design requires education, its a bad design. It may sound smart, but personally, I think its bullshit. Dont be afraid to educate if it helps your users connect to the core value, but try to show not tell. Youll notice here Pinterest is educating users on what they can do in a contextual way. Its not making you go through five screens that tell you everything Pinterest can do. As you experience the product, its telling you what things are for and what can you do. After you scroll, you see this, including these pulsing circles indicating the content is clickable.34

Simplifying FurtherWe saw in research we were educating on too many things, and users were getting lost

We made a short list of critical elements of the product to create value (not explain how it works!), and removed everything else


The Pinterest activation team still had a problem on their hands though. We saw in research that we were throwing too many concepts at people, and people were getting lost and not connectin to that core value of seeing cool content on topics you care about that you didnt know existed. So they made a list of the critical elements of the product required for users to get value (not to sufficiently explain the service) and removed everything else from the experience for the first 30 days. You can see in this example Pin descriptions are removed from the feed.35

Email User ExperienceTest manually, then automate and personalize

Stop sending email like a marketer. Send email like a personal assistantRight contentRight timeRight amount

Subject lines and calls to action matter; design doesn't as much


All right lets talk email. Emails are a key driver of retention. They wont solve your retention problem, but they will certainly help if you do them right. At every company I have been at, people hated email and didnt want to send them to their customers. When they finally did, they saw lifts. You are not your customer. You get more email than they do. Emails help them if theyre connected to the core value of why they use your product. At Pinterest, I made this mistake. I set up campaigns with emails that explained all of the things Pinterest could do. People dont care about what Pinterest can do. They care about seeing cool content related to their interests. So we replaced those emails with popular content in topics of interest for each user, and our retention increased.

So, if youre starting emails, you want to try to do some things manually to see what works. When I started at Grubhub, they werent sending any email, so I emailed the entire user base asking what they wanted to hear about from us. They said new restaurants to order from and new deals. We didnt have many deals, so I started manually curating new restaurants once a week by city to prove it was valuable. Once it was driving thousands of orders, we started a personalized program for each persons address with multiple email types that rotated over time.

The goal is to get a problem that sends like emails not like a marketing interrupting you, but like a personal assistant making your life easier. When do most people think about ordering food? 4PM. So thats when Grubhub sends suggestions.

Subject lines: they matter. One of our engineers tested 4,500 variants of subject lines for our emails. Hundreds of thousands of additonal WAUs resulted from this. Some emails open rates went up 40%. We did a major redesign of our emails before this project, and no metrics moved meaningfully.

36

Notifications & BadgingStart with transactional, and take baby steps to test other types of messages

Notifications are hard to unsubscribe from, so people will just delete your app

Copy matters, as does landing experience

Badging is severely under-utilized, especially on Android

Pinterest built a completely new experience to leverage badging and notification landing pages


Notification are a bit more sensitive. I suggest starting with transactional message, and then baby stepping into other messages.

Notifications are hard to subscribe from, so people will delete your app

Copy matters a lot for push notifications, so test variations. And dont just drop them into the app. Drop them some place that matches the intent of the push.

And utilize badging of your app icon. It drives re-engagement, and no one is doing it on Android because they dont realize the APIs are manufacturer specific. We did a long term holdout for badging, and saw a 4% sustained lift in DAUs after a year.37

Loyalty/Engagement ProgramsThese programs should be profit drivers not cost centers, unless building a moat


The biggest misconception of loyalty programs is that they are actually rewards for customers. They are there to increase the profits made by the company. Now, whats important to realize about engagement programs is you have to know what behavior youre trying to change. I group customers into four buckets, and see what the biggest opportunity is for the business. For frequent but non-loyal users, you have to figure out a way to incentivize the usage they are doing not on your platform to come to your platform. For infrequent, but loyal users, you need to incentivize additional use cases for them with your product. For customers that are infrequent and non-loyal, thats a bigger issue. You usually need to create additional products, services, use cases, and incentives.

At GrubHub, our biggest opportunity was here. People ordered delivery frequently, but not always with us. 38

Find your bucket to focus onApartments.com was definitely infrequent, non-loyal

For Grubhub, we had to figure out biggest opportunity

Yummy Rummy


So, Apartments.com was definitely in the infrequent, non-loyal segment. People look for apartments a max of once a year. Thats very infrequent. So, when they do a search again, they usually forget how they found the last. What Craigslist did is provide that kind of value in tons of other areas so people used it regularly. That meant when they needed a new apartment, it was already top of mind. So, all we could do at Apartments.com was build in some moving services and some get to know your new neighborhood services through partners. We tried to create an apartment living section, but it was not that important a need.

For GrubHub, we needed to learn which segment to focus on. So we surveyed users to understand their loyalty, and data mined our user base to understand frequency. Once we had an understanding that we were frequent, non-loyal, we had to understand the use cases that people used delivery, but not Grubhub. Once we talked to a bunch of users, we got some major reasons. Then we surveyed our non-loyal users again to get statistical representation of what biggest reason for ordering delivery, but not using Grubhub was. Was it a product issue, a coverage issue, or just habit. It was just habit.

We thought we could change habits, but the unit economics of loyalty programs suck for marketplaces. When someone spends $30 with Grubhub, Grubhub doesnt make $30; it makes $3-5. So you model out what you can spend and how much you can influence. In our case, traditional models didnt work. 25 orders would get you a free drink. Not great. So we got more creative. We made it a game you played on every three orders with a 25% chance of winning and a variable reward. You generally want to shift to variable rewards in engagement programs instead of static awards, because people become accustomed to static rewards, so they cease to influence behavior. You should get a lawyer if you do this, as there are so many weird gotchas. Like having to post a bond for each state, like being classified as a sweepstakes, which means you need post bonds for each state you operate in, you cant AB test, and you need to be able to play without purchase.

So at GrubHub, we were classified as a sweepstakes, so we couldnt AB test. So we could either do it in one state and not another, or pulse it, like the McDonalds monopoly game. We chose the latter.39

ReferralsNeed a strong value proposition for sharing (generally money or a better product experience for them if friend joins)Need to understand if one to one or mass referrals make sense for your business


Referrals is all about creating a viral loop. You generally need to have a strong reason to get people to share, generally money or an improved product for them or status. So, if you bring one person to your site, and they invite two people, if everyone of those people invites two more people, you have a self-sustaining viral loop. This rarely happens. But, your viral factor is likely going to be less than one, but not insignificant.

Everlane is a lesser known example. Everlane is an online retailer of their own branded clothing. They created an invite system where if you invited someone who signed up, you got early access, but if you invited 15 people who signed up, you got one of their shirts. And if you invited 50 people who signed up, you got free shipping for life. They had 100,000 sign-ups pre-launch from this by optimizing not for every person to invite a friend or two, but for a select few influencers to invite dozen up to even thousands.

40

Internet

How growth teams are revolutionizing UX and product development