Is real-time research the real deal

Craig StrudleyResearch Director

Is real-time research the real deal?Or is there greater power in a

considered response?

Is real-time research the real deal?2

Or should businesses looking for a more complete view, combine in-the-moment research with considered response mechanisms?

As companies find more and more and ways of doing business, it is inevitable that market researchers are also challenged to find new and improved ways of capturing customer feedback. One clear trend has been the emergence of technologies that seek to understand the customer experience in almost real-time. Indeed, at Future Thinking we have worked with our clients across a number of sectors to develop new and innovative tools capable of delivering insights as they happen.

But does this mean that the ‘old’ methods no longer have a place? Should we abandon traditional email and telephone surveys that ask opinions days or weeks after an event?

Here we shall explore the idea that the answer to this question is a resounding “no” and that sensible businesses, looking for a more complete view, will combine in-the-moment research with considered response mechanisms.

Is real-time research the real deal?

Is real-time research the real deal? 3

IMPERFECT MEMORIES

Traditional research techniques, whether Quantitative or Qualitative, often rely on a participant’s memory in relation to a product or service received. Of course, memories are often imperfect and may have subsequently been influenced by external factors. By conducting research at the point of consumption, we can get a much truer reflection of the experience. Modern techniques have made ‘real time’ research a reality without the need to spend a fortune on ethnographic studies.

Voice of the customer research programmes that rely purely on feedback gathered days or weeks after an event are going to restrict the insight available, making it harder to make significant improvements.

REAL-TIME INSIGHT = REAL-TIME IMPROVEMENT

Another obvious benefit associated with in-the-moment research is that the timeline from response to insight can be reduced from weeks into hours. It doesn’t always follow that, just because feedback is collected at the point of experience, it is available to a client at the same time - but it’s certainly possible. Future Thinking have the ability to create sophisticated portals for our customers; by uploading data almost as soon as it is collected, they are able to track responses. In some cases, this allows businesses to implement fixes almost as issues occur.

The rise of real-time research

There are a number of factors why traditional techniques are being replaced; two of the main advantages relate to accuracy and timeliness.


What is more important – the actual experience or how the experience is remembered?

Real-time research is all about understanding experiences as they happen and not how they are remembered. Of course, this makes sense if you want to understand various touchpoints along the customer journey and where the ‘pain’ points are. However, this rather dismisses the importance of the customer’s memories, even if they are sometimes inaccurate.

For instance, when you are coming to renew a contract or potentially repurchase a product, what is more important – the actual experience or how the experience is remembered? When asked whether you would recommend a company by a friend, you are going to base your answer on your current feelings; recommendation and loyalty are based on what we remember.

So, while we would expect experiences to be closely linked with memories, in some respects, the memory of the event or product is actually more important.

Memories are important


Journey’s endWhen it comes to memories, it is widely believed that the ending is extremely important; if fact, many would say that it is the final outcome that dominates. Many real-time research tools only deal with individual touchpoints; so, while they are excellent at identifying parts of a customer journey where things need improvement, they don’t always deal with the crucial end-point.

So, real time data is good but the power of a considered response after the event should not be under-estimated. A negative heat of the moment reaction does not necessarily mean a lost customer if the business follows up in the correct way; in this case, the journey’s end might be viewed as being after the company’s response e.g. an apology or compensation. On the flipside, positive responses can also be misleading; what if a ‘fixed’ pipe starts leaking again a day later? Or if a new television breaks down two weeks after purchase?

EXAMPLE 1

Recently, we moved house and so I called our bank to change our address. Immediately after a pleasant call with a helpful advisor, I was sent a short text survey to rate my experience. I was still ‘in-the-moment’ – I could remember exactly what had happened and excellent scores were duly given.

Several weeks later, we discovered that our address had not in fact been changed on the bank’s system. Confidential correspondence, including credit cards and pin numbers, had been sent to our previous house. The bank stated human error – the ‘excellent’ advisor had “messed-up”. No follow-up survey was sent – although the experience at the time was positive, my memories at the conclusion are negative. Here, the research does not reflect the reality.

Sometimes research can be too quick – almost too real-time. There is still power in a considered response down the line after the dust has settled.

EXAMPLE 2

Future Thinking conduct a huge amount of travel-related research e.g. on trains and buses – we can literally talk about ‘journey’s end’. Interviewing on-board using handheld tablets is a great way of conducting this research. Data can be uploaded to our systems as soon as a reliable signal is available. The client can have access to the results very swiftly but, perhaps more importantly, the responses themselves are collected in-the-moment. We are not relying on the customer to try and remember the cleanliness of the carriage, the timeliness of the train and so on – they can report back on what they are actually experiencing.

So, where is the drawback? With real-time research, there is very often a point when a snapshot is taken and the ‘results’ submitted. What happens if, at the end of the journey, the train halts outside the station and is delayed? What if the participant wants to use the toilet after the survey is conducted and finds it blocked?

Again, it very often makes sense to combine the very powerful findings from the on-board survey with those collected at the end of it all, based on final memories.


There are other powerful reasons why the ‘old ways’ should not be abandoned – here are three:

1. FLESHING OUT THE DETAIL

Real-time research is very often quantitative in nature, involving a small number of questions. Even if one of these questions is open-ended, inviting comment, it is sometimes very difficult for businesses to understand the ‘why’ behind the results. Automated text analysis tools are improving but they still struggle with many sentiments and cannot reliably understand the nuances of language, such as sarcasm. Traditional surveys are often longer and thus able to capture more detailed information; this enables advanced analytics such as key driver analysis and better understanding of the issues.

Although it doesn’t have to be the case, many real-time surveys also rely on technology such as IVR, SMS, website pop-ups, automatically generated emails and so on. Although some might consider it old-school, the use of telephone interviews is a far more personable experience and one which customers react well to. An actual

human being makes customers feel more valued than impersonal text messages or IVR. In addition, trained interviewers can probe answers more fully to get to the bottom of the issue. They can also check that the customer understands the question and the scale used.

And we shouldn’t forget Qualitative research either; combining real-time data with a series of focus groups or depth interviews will also add detail beyond what is possible from many new techniques.

Further advantages of traditional approaches


Further advantages of traditional approaches

2. CONTROLLING THE SAMPLE

Well-designed research programmes allow us to define who will be interviewed so we can ensure that the sample is representative. Real-time feedback tools are often ‘self-selecting’ and therefore skewed to customers with certain characteristics. This is certainly an issue for businesses with an older and perhaps less tech savvy customer base. Not everyone has a smartphone; there is still a place for online and CATI.

Worse still, many in-the-moment techniques are reliant on the individual being rated to start the process. This could be as simple as a waiter delivering a comments card with the bill after a meal or it could be a call centre advisor putting a customer through to an IVR survey. There will be a reticence to do this if it is obvious a customer’s experience has been poor – this is human nature. It can be argued that more traditional approaches, with a random sample generated after the fact, remove this issue and are more representative. This is truly independent research.

3. KNOWING YOUR PARTICIPANTS

Very often, real-time techniques don’t allow the researcher to ask demographic questions and really understand who they are speaking to. In some cases, the response can be linked back to customer information held by the company but that is not always true, especially if the data is collected using social media.

In contrast, more traditional surveys will often include detailed questions about a respondent, even allowing segments to be defined. Very often, customer details provided in the sample can be utilised so that teams have the ability to match survey results to individual customer journeys and touchpoints. Using this sample, we can cut the data in many different ways - extracting maximum insight. Importantly, we can also cross reference detailed survey results with future behaviour; are Customer Effort Scores or NPS really the best indicators of loyalty?

Real-time feedback tools are often ‘self-selecting’ and therefore skewed to customers with certain characteristics


Real-time research can be the real deal but there is definitely still power in the considered response

The intention of this paper was never to argue that somehow the old techniques are better than the new - far from it. To reiterate, Future Thinking have developed our own tools for in-the-moment research and real-time feedback.

However, we also recognise that memories are often different to the actual experience. We understand the importance of the ‘journey’s end’. We truly believe that this is not a case of either-or and that organisations really should consider combining the benefits of real-time research with further in-depth analysis on data collected further down the line. So, to answer the question posed right at the start, real-time research can be the

real-deal but there is definitely still power in the considered response.

At Future Thinking we have many years of experience and a vast toolkit of approaches; traditional and innovative. We can work with you and your company to ensure that you maximise the benefits of research in a cost effective way.

In summary: Not either-or, but both

Future Thinking takes a consultative approach to market research with commercial focus driving everything we do. That’s why we focus our attention on the three key areas that drive competitive advantage: Launch, Communicate, Experience.

We’re a global company of researchers, marketeers, statisticians, strategists, innovators, creatives and industry experts, integrating qual, quant and analytics through the latest technologies, to deliver research that engages audiences and drives action.

Our mission is to deliver consumer and business insights that tells stories, inspires action and travels within an organisation, long after the debrief.

For more information please contact John Whittaker +44(0)1865 336 463 [email protected]

The AuthorCRAIG STRUDLEY RESEARCH DIRECTOR, EXPERIENCE

Craig has been working in the world of market research for over 20 years. During that time he has worked with numerous clients across a range of sectors. Whereas it’s fair to say that all have shared the common goal of measuring and improving customer experience, the solutions have often been very different. Craig is a firm believer in working with companies to design or adapt bespoke research programmes that meet individual objectives. He joined Future Thinking in 2010 where he is currently a Research Director in the Experience team.

[email protected]

futurethinking.com Follow us on Twitter: @FutureThinkHQ

Documents

Is real-time research the real deal