Evaluation of user experiences and

perceptions of the ‘Smokeline’ stop-smoking telephone advice

service

February 2010

Kate Sewel, Jane Eunson & Lorraine Murray

Ipsos MORI Scotland

Published by NHS Health Scotland, Woodburn House, Canaan Lane, Edinburgh, EH10 4SG © NHS Health Scotland 2008

The opinions expressed in this publication are those of the author/s and do not necessarily reflect those of NHS Health Scotland.

Contents Summary ............................................................................................ 1 1 Introduction and sample profile ............................................... 3 1.1 Introduction ..............................................................................................3 1.2 Aims and objectives of the research.........................................................3 1.3 Methodology.............................................................................................3 1.3.1 Quantitative IVR survey............................................................................3 1.3.2 Follow-up qualitative interviews................................................................4 1.3.3 Mystery shopping .....................................................................................6 1.4 Sample Profile ..........................................................................................6 1.4.1 IVR Survey ...............................................................................................6 1.4.2 Follow-up sample .....................................................................................9 2 Main Findings ........................................................................ 11 2.1 Overall satisfaction .................................................................................11 2.2 Advocacy................................................................................................13 2.3 Quality of communications .....................................................................14 2.3.1 Advisor attitude ......................................................................................14 2.3.2 Understanding callers' needs .................................................................15 2.3.3 Relevance of information or advice provided .........................................17 2.3.4 Quality of information or advice provided ...............................................20 2.3.5 Advisor awareness of LanguageLine assistance ...................................22 2.3.6 Likelihood of acting on information or advice provided...........................23 2.3.7 Views on the Information pack ...............................................................24 3 Recommendations for service development.......................... 26 Appendices....................................................................................... 31 Appendix 1 – Interactive Voice Recognition (IVR) questionnaire........................32 Appendix 2 – Follow-up interview topic guide .....................................................33 Appendix 3 – Mystery Shopping Scenarios ........................................................34 Appendix 4 – Mystery shopping evaluation form ................................................35

1

Summary Overall, the Smokeline service is viewed extremely positively by callers. The attitude of advisors and their deep understanding of the realities of smoking, and stopping smoking, were particularly appreciated. The provision of high quality and relevant advice and information, coupled with the empathy demonstrated by advisors, results in callers feeling well supported in their quit efforts. Callers feel that they have received advice which they will, and in many cases do, act upon. Callers are particularly appreciative of the service when they feel that they have received personalised support, i.e. when Smokeline advisors take the time to establish their smoking history, smoking routines, and detail of any previous attempts to quit, etc. Particular mention was made of advisors showing interest in why and when they smoke(d), which enabled them to help callers design tailored quit plans and coping strategies. Smokeline is currently better equipped to help people who are at the outset, or planning stage, of a quit attempt, as opposed to those who have already stopped smoking and are seeking support to help them stay stopped. Callers who are further along their quit journey (e.g. have quit for a number of weeks or months) are, overall, slightly less satisfied with Smokeline. However, this difference is small, and the majority of these callers do feel the service is helping and encouraging them in their quit attempt. There were very few instances of advisors providing factually incorrect information. Where this did happen, it tended to be in relation to specific circumstances where there is less agreement on best practice, most notably, in the provision of advice concerning Nicotine Replacement Therapies (NRT) use among pregnant women. Although almost all callers feel that their needs are being met, the evaluation identified scope for Smokeline advisors to provide significantly more information or advice than a caller initially requests or expects. Individual callers will not know about all the information and advice with which they could be provided, but the follow-up interviews and the mystery shopping exercise revealed inconsistencies in the amount of information and advice provided in calls. Advisors should view every call to Smokeline as an opportunity to arm callers with as much information and advice as possible to aid their quit attempt. While a key aim of Smokeline is to signpost people to local stop smoking services (LSSS), there were instances of advisors doing this and then

2

immediately terminating the call. While advisors may be of the view that callers will receive further information and advice at the LSSS, it should not be assumed that this will be the case. Some callers do not subsequently attend and so are no further forward on their quit journey. Provision of additional information and advice, such as the effects and availability of NRT products, hints and tips on how to stop, or a discussion of the health benefits of stopping, may mean that the caller is better equipped to quit even if they do not, for whatever reason, attend a LSSS. Respondents were generally of the view that the LSSS signposting aspect of Smokeline worked efficiently. Those who asked for their contact details to be passed to a LSSS (for the service to contact the caller) tended to report that contact had occurred promptly. However, there is scope for advisors to take more time to discuss LSSS with callers who are initially reluctant about attending. The evaluation identified instances where advisors did not probe reluctant callers' views or beliefs about what LSSS entail, and why they were reluctant to attend. However, there is clearly a fine line between exploring concerns, reassuring or encouraging callers, and ensuring LSSS are not ‘pushed’ too hard. Over-endorsement of LSSS may alienate callers. Thus, while promoting the effectiveness of these services, it is important to ensure that callers are not given the impression that advisors think this is the only way to quit, or that it is all they will get told about if they call again. A number of the follow-up participants noted that they had resorted to smoking in the morning 'because Smokeline is not open'. However, further research would be required to determine whether there would actually be sufficient demand for the service before midday (e.g. between 7am and 9:30am) to make this the best use of resources.

1 Introduction and sample profile 1.1 Introduction This report presents the findings of an evaluation of user experiences and perceptions of the ‘Smokeline’ stop-smoking telephone advice service. Commissioned by NHS Health Scotland, the evaluation comprised a quantitative Interactive Voice Recognition (IVR) survey of Smokeline callers, a series of follow-up qualitative telephone interviews with IVR respondents, and a mystery shopping exercise. Fieldwork was conducted November 2009 – January 2010.

1.2 Aims and objectives of the research The overall aim of the research was to evaluate the effectiveness of Smokeline in supporting callers in their attempts to quit smoking. The results will help inform the future development of the service. Specifically, the evaluation was designed to assess:

• user perceptions of their experience of calling Smokeline • the relevance of information and advice provided • the accuracy of information and advice provided • the effectiveness of the service in supporting quit attempts.

1.3 Methodology The study comprised three distinct elements: a quantitative IVR survey of 298 callers, a series of 37 follow-up qualitative telephone interviews, and 32 mystery shopping calls to Smokeline.

1.3.1 Quantitative IVR survey For the duration of the fieldwork period (16 November 2009 – 20 January 2010), advisors recruited participants for the IVR survey (see Appendix 1 for full questionnaire). Taking into consideration the transfer protocol (Figure 1.1), all eligible callers were invited to participate upon completion of their call to Smokeline, and consenting participants were immediately transferred to the survey. Figure 1.1: Transfer Protocol

3

Eligible callers:

• All genuine interactive callers over the age of 16 • Callers phoning on behalf of someone else

Ineligible callers:

1. Any caller aged under 16 years old 2. Any non interactive (hoax or silent) callers 3. Any caller who is distressed (i.e. too upset or angry) 4. Any caller who says they have already completed the survey (after a previous call to Smokeline during the survey period)

4

The IVR survey was designed by Ipsos MORI and NHS Health Scotland and was run by Eckoh. The questionnaire explored:

• overall satisfaction with, and advocacy of, the Smokeline service • relevance of, and likelihood to act upon, advice or information provided • callers' perception of advisor knowledge and understanding of their

specific needs • callers' perception of the quality of information provided.

The survey also recorded: the age and sex of respondents; their current smoking status; their reason(s) for calling Smokeline; the number of times they had phoned the service in the last year; whether they were offered details of a local stop smoking service; and, whether they requested further information to be sent to them. Respondents were also afforded the opportunity to leave a message about their experience of calling Smokeline. On completion of the survey, respondents were asked whether they would be willing to be contacted to take part in a follow-up telephone interview to explore in more detail their experience of calling Smokeline, and if so, to leave their contact details. The survey questions were read out by one of Eckoh’s in-house voice artists and respondents provided their answers by pressing buttons on their telephone keypad. For example, for Q1 ‘How satisfied or dissatisfied were you with Smokeline today’, ‘1’ would be pressed to indicate very satisfied, ‘2’ for fairly satisfied, ‘3’ for neither satisfied nor dissatisfied, ‘4’ for fairly dissatisfied, and ‘5’ for very dissatisfied. The IVR survey took around 6 minutes to complete. The survey was live from 16 November 2009 until 20 January 2010. This resulted in 296 people completing the first question, 235 answering the last main question, and 222 completing the entire survey. These drop-out rates are a function of IVR surveys: an automated response system does not have the same ability to build rapport and encourage an individual to complete a survey. It is easier for a respondent to disconnect a call from a machine than from an actual person.

1.3.2 Follow-up qualitative interviews Interviewees for the follow-up component were recruited from those who had participated in the IVR survey and indicated that they were willing to be re-contacted.

5

Potential participants were contacted 7-14 days after their call to Smokeline. The 7-14 day timescale was designed to ensure that they had been given time to reflect on their call and act on the information or advice provided, but were still likely to remember specifics of the call. The interviews allowed for a more in-depth exploration of issues than was possible in the quantitative survey and mystery shopping exercise, while also providing an opportunity to discuss issues not amenable to quantitative exploration. Specifically, the follow-up interviews were designed to explore:

• callers’ perceptions of satisfaction with the helpline and information provided

• likelihood and experience of acting upon information provided • how participants feel Smokeline might best improve its performance in the

future. At the outset of the interviews, participants were asked some open-ended, general questions concerning their level of satisfaction with their call to Smokeline. They were then probed further for particular examples of the various aspects of the service, in order to identify the specific aspects that influence their overall views. The interviews explored what information the caller had been provided with, what actions, if any, they had taken in the intervening period, why they had taken the steps they had, and their future intentions regarding their smoking behaviour. For example, if they were a smoker at the time of their call, whether they had attempted to quit smoking or whether they had reduced consumption or made any other changes to their smoking habits (e.g. delaying smoking in certain situations). A topic guide for the interviews was designed by Ipsos MORI in consultation with NHS Health Scotland (see Appendix 2). Fifty-five people consented to participating in a follow-up interview, and provided a telephone number on which they could be contacted. A total of 37 follow-up interviews were conducted between 30 November 2009 and 28 January 2010. Sixteen respondents were not contactable and two said that they no longer wished to participate in the follow-up component – the reason each provided related to their personal circumstances and not their quit attempt. The interviews tended to last between 15 and 30 minutes. Where respondents consented, interviews were digitally recorded and transcribed for analysis. In order to maximise participation of respondents in different circumstances (e.g.

6

full-time/shift workers), where necessary, respondents were called at least ten times, at different times of day and on different days of the week.

1.3.3 Mystery shopping The mystery shopping exercise was designed to evaluate the call-handling skills of advisors as well as the accuracy and quality of the information and advice they provide. A key purpose of the exercise was to 'quality assure' information provided about pharmacotherapies, Nicotine Replacement Therapies and one-to-one and group support services. Eight mystery shopping scenarios were created by Ipsos MORI, in conjunction with NHS Health Scotland. (See Appendix 3 for detail of the scenarios and the number of calls made using each). In order that each caller appeared genuine, there was slight variation of the detail of each call within each scenario. This ensured that all calls were as convincing as possible, even if the same scenario was answered by the same advisor on more than one occasion. For example, the mystery shoppers were provided with different stories about how many cigarettes they smoke(d) per day, how many years they had smoked for, and previous quit attempts.

A total of 32 mystery shopping calls were conducted by Ipsos MORI interviewers between 23 November and 27 January 2010. Where possible, the mystery shoppers were current or ex-smokers. They also represented a cross section of callers to Smokeline in terms of sex and age. To ensure evaluation of call handling on different days and at different times, specific dates and times were specified for each mystery shopping call. An evaluation form designed by Ipsos MORI in collaboration with NHS Health Scotland (see Appendix 4) was produced for the mystery shoppers to record the details of their call. The completed forms were then judged against a set of criteria provided by NHS Health Scotland.

1.4 Sample Profile

1.4.1 IVR Survey Respondents to the IVR survey were aged between 17 and 76, with an average age of 43.9 years. Overall, 46% of respondents were male and 53% were female (where sex is known – 61 out of 296 callers did not answer this question). Figure 1.2 shows the age and gender profile of respondents. As illustrated, the 45-59 age group is the largest. There were a similar number of males and females in the younger two groups, and more females than males in the older two groups.

7

Figure 1.2

Age and gender profile of IVR respondents

1412 14

6

13

11

19

10

0

5

10

15

20

25

30

35

16-34 35-44 45-59 60+

% re

spon

dent

s

FemaleMale

The majority of respondents to the IVR survey had not called Smokeline in the last 12 months: 19% had called on one previous occasion, and 9% had called more than once before. Figure 1.3 shows where IVR respondents were on their quit journey. As illustrated, around a quarter of respondents (26%) had already stopped smoking when they completed the IVR survey (and were looking for advice on staying stopped), while the remainder were roughly divided between those who had already set a quit date (38%) and those who had not (yet) set quit date (33%).

8

Figure 1.3

Stage of quit journey

Never smoked (calling behalf of someone else)

3%

Already stopped26%

Set a quit date38%

Not (yet) set a quit date33%

Female callers were more likely than male callers to have already stopped smoking (35% compared with 15%). Similarly, older callers were more likely to have stopped (39% of those aged 60+ compared with 15% of those aged 16-34). The majority of IVR respondents (71%) were current smokers: 26% indicated that they used to smoke but had quit, and 3% had never smoked. Almost all respondents (89%) were calling for themselves: 8% were calling on behalf of someone else and 3% were calling both for themselves and for someone else. As Figure 1.4 illustrates, the most frequently mentioned motive for phoning Smokeline was to obtain 'general advice on giving up smoking'. Around half were calling to obtain ‘information about stop smoking aids like nicotine replacement patches, lozenges or chewing gum etc. or prescription medicines'. Slightly fewer were calling for details of a local stop smoking service. A sizeable number were also calling for advice in staying stopped (including advice about coping with withdrawal symptoms).

9

Figure 1.4

Q8. What are your reasons for calling Smokeline today?

71

48

30

4957

0

10

20

30

40

50

60

70

80

General advice Stop smokingaids

Details of a localservice

Coping withwithdrawal

Staying stopped

% re

spon

dent

s

There are therefore two very distinct groups of callers; those calling to obtain advice and information about how to (prepare to) stop smoking, and those calling for advice, support, and encouragement once they have already quit.

1.4.2 Follow-up sample The relatively low volume of callers to Smokeline across the fieldwork period meant that all those who left contact details formed the follow-up sample and there was no selection involved. Figure 1.5 shows the age and gender profile of the follow-up participants. As illustrated, 54% of follow-up participants were male and 46% were female. As with the IVR survey, the 45-59 age group is the largest. There were exactly the same number of males and females in the older two age groups, more males in the 16-34 age group, and more females in the 35-44 age group.

10

Figure 1.5

Age and gender profile of follow-up participants

7

3

64

2

5

6

4

0

2

4

6

8

10

12

14

16-34 35-44 45-59 60+

% re

spon

dent

s

FemaleMale

The profile and levels of satisfaction of those who agreed to take part in the follow-up and those who did not were very similar. Thus, the follow-up sample was representative of all respondents.

11

2 Main Findings 2.1 Overall satisfaction Caller perceptions of their experience of calling Smokeline were overwhelmingly positive. At the outset of the IVR survey, respondents were asked to indicate (on a five point scale from 'very satisfied' to 'very dissatisfied'), how satisfied or dissatisfied they were 'with Smokeline today'. As illustrated in Figure 2.1, the vast majority of respondents were satisfied with the service they received; 79% were 'very satisfied', and a further 13% were 'fairly satisfied'. In contrast, only 4% reported that they were dissatisfied. Figure 2.1

Q1. Overall, how satisified or dissatisified were you with Smokeline today?

79

2 2

13 4

0

10

20

30

40

50

60

70

80

90

Very satisfied Fairly satisfied Neither/nor Fairly dissatisfied Very dissatisfied

% r

espo

nden

ts

There were no notable differences by gender, age, whether they had previously called Smokeline, or which call centre (Glasgow or Brora) handled their call. Satisfaction was high among those phoning for advice about stopping smoking and among those looking for advice on staying stopped. However, as shown in Figure 2.2, those who had already stopped were less likely than those who had not (yet) stopped to be ‘very satisfied’ with the service they received from Smokeline.

12

Figure 2.2

Q1. Overall, how satisified or dissatisified were you with Smokeline today?

7770

86 868685

74

0

10

20

30

40

50

60

70

80

90

100

Currentlysmoke

Nowstopped

Generaladvice

Details oflocal service

NRT info Coping withwithdrawal

Stayingstopped

% v

ery

satis

fied

The follow-up interviews reaffirmed callers experiences of calling Smokeline as overwhelmingly positive: the service is viewed as an extremely valuable resource with respondents emphasising that it had encouraged them in their quit effort. In many cases, Smokeline exceeded expectations. This was particularly evident among callers who perceived Smokeline to be simply a phoneline to contact for provision of literature:

I just thought they would say 'I’ll send you a booklet, there you are kind of thing', but he didn't.

(Female, 35-44, already stopped smoking) The exceptions, instances where callers were less satisfied, generally relate to external factors, and caller expectations which may not match the Smokeline remit. Particular examples are the expectation of provision of medical advice, provision of NRT products, and a lack of LSSS or pharmacists to refer people to, in remote areas of the country. In such instances, caller satisfaction is immediately limited and they are frustrated that recommended approaches to help them quit are not easily accessible to them. Further, during the fieldwork period, the NHS Stop Smoking Quit Kit was made available to people resident in England. Having seen advertisements for the Quit Kit, a few participants had called Smokeline with the intention of ordering it. Although not necessarily dissatisfied, these callers were disappointed to discover

13

the kit is not available in Scotland. While this is not directly related to the service provided by Smokeline, it would be useful for Smokeline advisors to be made fully aware of smoking cessation campaigns in England so that they are best equipped to manage callers' expectations. When pushed to provide recommendations for specific aspects of the Smokeline service participants felt could be improved upon, the hours of operation were mentioned. While follow-up participants acknowledged that it is convenient that Smokeline is open until midnight, and opening hours will be dictated by resources, it was suggested that the service offered could be improved by extending the opening hours:

The only thing I hated was that it’s only open from twelve during the day to twelve midnight. I tried to phone yesterday morning, it must have been about quarter past nine and I was like, 'oh my God', I was gobsmacked, 'oh my God, my Smokeline is not open till twelve o’clock today' …. I think your best times [to open] are probably morning, afternoon and evening, because when you get up in the morning you want a cigarette, the afternoon you have your lunch and then you want a cigarette, the evening time you have your dinner and you want a cigarette.

(Female, 35-44, already stopped smoking)

While a number of the follow-up participants noted that they had considered or resorted to smoking in the morning 'because Smokeline is not open', and morning opening was certainly something that callers felt would help them in their individual quit efforts, there is insufficient evidence to make a clear recommendation that the hours should be extended. While opening earlier in the day would undoubtedly aid the national quit effort, further research would be required to determine whether, and at what specific time(s), there would be sufficient demand for the service to make it the best use of resources.

2.2 Advocacy Respondents were also asked to indicate how likely they would be to 'recommend Smokeline to a friend or relative who wanted to give up smoking'. Reflecting the high levels of satisfaction, the vast majority (86%) said they would be 'very' (76%) or 'fairly' (10%) likely to do so. In contrast, only 4% said they would be 'very' or 'fairly' unlikely to recommend the service. Reflecting the slightly lower levels of overall satisfaction among respondents who had already stopped smoking, this group of callers were slightly less likely to say that they would recommend the service (13% were unlikely to, compared with 4% overall).

14

Generally, follow-up participants said that they would use the service again if they wanted to and they would also recommend it to others; indeed, a few participants said that they had already recommended the service to someone else. Further, participants recounted occasions of passing on information and advice to others, potentially helping the national quit effort, but also providing scope for inaccurate relaying of (personalised) information received from Smokeline.

2.3 Quality of communications

2.3.1 Advisor attitude Respondents were overwhelmingly satisfied with Smokeline advisors. In terms of call handling, there were very high levels of satisfaction with their customer service skills, participants commonly described advisors as being 'polite', 'pleasant', and 'courteous':

I work on phones myself, so it makes a big difference when you hear someone who has a good customer service and stuff ... it’s quite easy to talk to someone like that.

(Male, 16-34, already stopped smoking) Further, advisors were complimented on speaking, and explaining things, clearly. Prior to phoning Smokeline, there was a concern among the follow-up participants that advisors would be ‘impersonal’, ‘judgemental’, ‘unsympathetic’, or ‘preachy’. In contrast, their actual experience was far more positive and rewarding:

You didn't really feel under pressure ... it was like talking to a friend. (Male, 16-34, wants to stop smoking, has not (yet) set a quit date)

It just helps you know that other people are concerned or interested in the fact that you want to stop smoking.

(Female, 45-59, wants to stop smoking, has not (yet) set a quit date)

As well as appreciation of their customer service skills, respondents welcomed the friendly and encouraging approach of Smokeline advisors, and felt they had established genuine rapport with them:

It was probably lucky the phone call was shortish, because I could have confessed everything, she just had a really nice manner, she was very friendly and supportive.... you know, sometimes you just have people you feel you can open up to.

15

(Female, 35-44, already stopped smoking)

This empathy between advisors and respondents was regularly cited as one of the most important aspects of the Smokeline experience:

He didn't treat me as ‘just another one'. He was compassionate and he knew my name first-hand which was great as well. I wasn’t just a person on the other side of the phone, which was brilliant.

(Female, 35-44, wants to stop smoking, has set a quit date)

To some callers, simply having contacted Smokeline, and talked to someone about stopping smoking made them feel they had moved on to the next stage of their quit journey. Regardless of whether or not it resulted in an immediate change in smoking behaviour, the fact that they had taken the step to phone the service was sufficient to make some callers feel that they were doing something positive about quitting smoking. The advisors further encouraged respondents, discussed with them the health and financial benefits they could expect, made them feel good about making the call, and made them realise that if ‘thousands of people can do it’, so can they. Respondents were ‘pleasantly surprised’ at how useful an initial call to Smokeline had been, to the extent that some reported ‘excitement’ at the realisation of the variety and extent of support available:

I felt excited when I put the phone down, because I felt this is something I’m really aiming for and it just made it feel a little bit easier.

(Female, 35-44, wants to stop smoking, has set a quit date)

There was a view that an initial call had helped people to ‘make up their mind’ about quitting:

I thought, well if I made the phone call, I can just do it. (Female, 60+, already stopped smoking)

2.3.2 Understanding callers' needs Respondents to the IVR survey were asked to rate (on a five point scale from 'very good' to 'very poor'), their advisor's understanding of their needs. As illustrated in Figure 2.3, the vast majority of respondents rated their advisor's understanding of their needs as 'good' (92%). In contrast, only 3% rated their advisor's understanding of their needs as 'poor'.

16

Figure 2.3

Q2. How would you rate the advisor's understanding of your needs?

80

1 2

512

0

10

20

30

40

50

60

70

80

90

Very good Fairly good Neither/nor Fairly poor Very poor

% re

spon

dent

s

Respondents who called for information on NRT (who are more likely to be in the early stages of their quit attempt), were particularly likely to feel that their advisor understood their needs (88% said 'very good'), while again, those who called for support in staying stopped (i.e. those further along in their quit attempt) were slightly less positive about their advisor's understanding of their needs (71% said 'very good'). The mystery shopping exercise and follow-up interviews provided further insight into the high ratings for advisors' understanding of caller needs. In particular, callers felt Smokeline advisors listened, and responded, to what they said, and were impressed with their apparent understanding of quitting smoking and its impact on day-to-day life. Advisors were viewed as empathetic and sensitive to the needs of individual callers, and a common theme was that respondents felt advisors displayed an understanding of what they were experiencing. There was a prevailing view that an advisor who has previously smoked themselves is best able to understand the needs of a caller:

The only thing I would say is, I would hope to speak to somebody who knows what it's like to stop smoking.

(Female, 35-44, already stopped smoking)

17

The first man I spoke to had stopped smoking and he was talking about the patches, so I could relate to him because I was very early into stopping smoking.

(Female, 35-44, already stopped smoking)

However, advisor empathy was strong even when the advisor had never been a smoker:

[The advisor] actually knew the feelings and what you would be going through when trying to cut down... but he actually was never a smoker himself, which was brilliant.

(Female, 35-44, wants to stop smoking, has set a quit date) I asked her if she was actually a non smoker herself and she said she couldn’t tell me. She didn't confess that she had been, but in a few words, kind of said, 'I know where you’re coming from' …. she may not have been [an ex-smoker], but it was enough of an emotional resonance. It touched base and I thought, 'oh that’s great'. It was just what I needed to hear at the time.

(Female, 35-44, already stopped smoking) Criticism arose when it was felt an advisor lacked a basic grasp of the day-to-day realities of being a smoker and quitting smoking. For example, one mystery shopper felt the advisor was unable to fully understand her needs when the advisor asked her how much a packet of cigarettes cost. Further, the follow-up interviews revealed that participants felt understood if they encountered support and compassion from the advisor: an apparent absence of empathy and understanding on the part of an advisor could attract criticism. Respondents who had phoned Smokeline in order to obtain support in staying stopped tended to give slightly lower ratings for their advisor’s understanding of their needs. There was a view that Smokeline is not designed to help people who have already stopped smoking:

You still need support for a while after you’ve stopped. Sometimes I feel it’s a line that deals with people who are still smoking and I’m phoning up saying ‘I’m still stopped, I’ve stopped smoking for six weeks’. Sometimes you will get congratulations, but sometimes you think it’s only for people who are trying to stop smoking, it’s not actually for people who have stopped …. I just keep getting the same advice over again, it’s not really working. It works for so long and then I get kind of fed up listening because I do it and it still doesn’t work.

(Female, 35-44, already stopped smoking)

18

However, in the follow-up interviews, those who had already stopped smoking were particularly likely to provide praise for the service when an advisor took the time to discuss with them why they stopped in the first place. Callers found this a powerful reminder of their motivation. There was also praise for advisors who took the time to simply engage in a 'pep talk’ or to help someone through a craving by taking the time to ‘talk them through it’ and ‘take their mind off it’:

I’d given up smoking for about nine days when I phoned and it was just for a bit of support more than anything. That day I was really, really, bad and I wanted to take the patch off and have a cigarette. When I phoned, the girl I spoke to was really, really, good, very helpful with me …. She was telling me how long it would last and that it would pass. [She told me that] the notion only lasts for a few minutes and then it kind of goes away which is very helpful.

(Female, 35-44, already stopped smoking)

2.3.3 Relevance of information or advice provided Respondents were asked to rate (on a five point scale from 'very good' to 'very poor') the relevance, to them, of the information or advice provided by Smokeline. As illustrated in Figure 2.4, the vast majority of respondents rated the relevance as 'good' (90%). In contrast only 4% rated the relevance of the information or advice provided as 'poor'. Figure 2.4

Q3. How would you rate the relevance, to you, of the information or advice provided?

73

2 2

6

17

0

10

20

30

40

50

60

70

80

Very good Fairly good Neither/nor Fairly poor Very poor

% r

espo

nden

ts

19

Respondents contacting Smokeline in order to obtain information on NRT were particularly positive about the relevance of the information or advice with which they were provided (80% said 'very good'). Indeed, the follow-up interviews revealed that participants were overwhelmingly impressed with advisors' knowledge of NRT products and, in particular, the variety of products available, where and how they can be obtained, and how they can be used. For the 'Secret smoker' mystery shopping scenario, for example, the advisor was able to suggest the caller try microtabs because 'they can be placed under the tongue and are small so people won’t notice'. Follow-up participants also mentioned that if they told an advisor that they had previously found that one type of NRT was ineffective, they were able to offer advice on other types of NRT products, or combining different products. In providing advice about NRT products, advisors were well aware of the boundaries of their remit and extent of their own expertise; if medical advice was required, advisors would refer the caller to a qualified healthcare professional such as a GP or pharmacist. Further, there was consensus that where advisors took time to establish and understand callers needs, they were able to provide information, advice and support that was felt to be tailored to the individual. Establishing smoking history, smoking routines and detail of previous attempts to quit led to the caller feeling they were receiving personalised support. Particular mention was made of advisors showing interest in why and when they smoke(d), which in turn enabled advisors to help them callers design tailored quit plans and coping strategies.

She asked me how long I smoked for, how many cigarettes I smoked, I didn't expect those questions, I was just expecting answers, but it was good that she brought it up.

(Male, 16-34, already stopped smoking)

[The advisor] was really good. She asked when my first cigarette was. As soon as I get up it’s a coffee and cigarette. She said ‘well, change your [routine], instead, have a shower and get changed’ …. She told me that instead of having a coffee, maybe have a drink of juice or water, or a cup of tea instead of coffee.

(Female, 35-44, already stopped smoking)

Conversely, where it appeared that an advisor had not established a caller's individual circumstances, there was criticism in relation to the relevance of information and advice provided.

20

2.3.4 Quality of information or advice provided The quality of information or advice provided by Smokeline was also rated very highly. As illustrated in Figure 2.5, 93% of respondents to the IVR survey rated the quality of the information or advice provided as 'good', with only 3% rating it as 'poor'. Figure 2.5

Q4a. How would you rate the quality of the information or advice provided?

79

1 2

514

0

10

20

30

40

50

60

70

80

90

Very good Fairly good Neither/nor Fairly poor Very poor

% re

spon

dent

s

Respondents calling Smokeline for general advice, or for information on NRT products were particularly positive (83% and 84% respectively said 'very good'). The follow-up component revealed that these respondents were impressed with advisors' apparent knowledge of the various options available to them in terms of products and services. Those calling for support in staying stopped were slightly less positive (74% said 'very good'). However, respondents' ratings of quality of information are not evidence that information provided is actually of high quality. The mystery shopping was particularly useful in analysing this issue; it identified instances of variability in both depth and quality of information provided. In terms of accuracy and quality of information, the 'Pregnancy' scenario calls resulted in advisors providing conflicting information about which NRT products were deemed suitable for pregnant women, appearing unclear of current

21

guidelines that ‘Pregnant women can use most forms of NRT … but usually it would be better to use gum or lozenges/micro-tablets rather than patches’1. Of the five callers, one caller was told ‘gum and lozenges are the only NRT products suitable for pregnant women', one was told ‘only microtabs are suitable’, another was told that 'patches cannot be used and microtabs are normally recommended'. A fourth caller was told that NRT 'can't generally be used by pregnant woman' but that she should check at a LSSS. The fifth caller was also referred to another healthcare professional for further information. There were also varying responses to the 'pregnant' mystery shoppers saying that they thought they could reduce their smoking but not stop completely. The advisors tended to offer praise instead of using the opportunity to explain that stopping smoking completely is recommended. Indeed, instead of explaining that it is easier to quit than it is to cut down, that the withdrawal symptoms are likely to be less severe, and that a quit date should be set, one mystery shopper was told that cutting down 'would be a good start'. There was also variation in the amount of information and advice provided to the ‘pregnant’ mystery shoppers. The following types of information were offered across the calls: contact details for a LSSS; general hints and tips; advice about the available information pack; and alternative sources of information and advice (e.g. GP and pharmacist). However, no single caller was furnished with all this information. Similarly, the mystery shopping scenario 'I've heard about groups and would like more information about them', obtained responses of differing quality and substance. One approach was to provide the caller with contact details for a LSSS and terminate the call. On other occasions, the time was taken to establish a caller's perceptions of what the LSSS might be like, allowing the advisor to provide information to them about what they would actually entail. On some occasions, advisors discussed not only services, but pharmacotherapies, including NRT, or provided other useful information and advice to aid respondents' quit efforts. For example, providing general hints and tips, and discussing the advised approach of setting a quit date. The 'Go it alone' scenario also elicited differing responses. Some advisors were immediately accepting of a mystery shopper’s desire to 'go it alone', on other occasions advisors probed for further information (i.e. why they felt this way) and as a result were able to provide further encouragement (i.e. reinforce the message of being four times more likely to quit when combining aids and groups). Similarly, there was variation in the extent to which follow-up

1 Mystery Shopper Analysis Framework, as provided by NHS Health Scotland (January 2010)

22

respondents who expressed hesitation about LSSS were probed about their views, and encouraged to attend. Adopting a probing approach allows advisors to clarify misconceptions and encourage callers to consider the option of attending a LSSS, as well as providing an opportunity to equip them with alternative advice. As such, regardless of whether or not they attend a LSSS, the option has been fully explored and discussed and the caller has been equipped with advice and information that might help change their smoking behaviour regardless of whether or not they attend a LSSS.

2.3.5 Advisor awareness of LanguageLine assistance Two of the mystery shopping calls required the caller to display a poor level of English, with the aim that they would be offered the services of a LanguageLine assistance interpreter. One call required a Cantonese translator while the other required a Polish translator. The Cantonese caller was initially told an interpreter was not available, then following some hesitation, the advisor changed their mind, suggesting that this might be an option, and a translator was brought into the call. The quality and relevance of information and advice received during the call was somewhat compromised. To some extent this was due to information and advice being ‘lost in translation’. For example, the mystery shopper heard the advisor mention NRT patches, gum, lozenges, inhalators, nasal spray and microtabs. However, lozenges, inhalators and microtabs were not translated. Further, the Cantonese mystery shopper had previously made a call to Smokeline and she felt that the call she made with LanguageLine assistance provided her with much less detail than their previous call (which had been in English). For example, LSSS were only mentioned to her after she probed for alternatives to NRT and no hints or tips were provided unprompted. The Polish caller, despite acting confused at everything the advisor said, was at no point made aware of the availability of the LanguageLine assistance service. The only information provided to the Polish caller was that NRT patches and gum are available and they could help him stop smoking. He was advised to contact his GP to further discuss the NRT options available to him. Due to the language barrier, no other specifics were discussed during the course of the call. Although the mystery shopper felt that the advisor was trying their hardest given the circumstances, the lack of provision of comprehensive information meant that the service provided to the mystery shopper was of much lesser quality than that generally provided to callers who can speak English.

23

Advisors should therefore be made aware of LanguageLine assistance and advised to make sure that they provide comprehensive information and advice, but to provide it in short chunks, with adequate pauses, so that the translator can retain the information and relay it accurately.

2.3.6 Likelihood of acting on information or advice provided Respondents were also asked to indicate (on a five point scale from 'very likely' to 'very unlikely') how likely they were to act on the information and/or advice provided by Smokeline. The vast majority of respondents (94%) felt they were 'likely' to act on information or advice with which they were provided. Only 4% reported that they would be 'unlikely' to act (Figure 2.6). This reflects the high satisfaction with, and perceived relevance of, the information and advice provided. Figure 2.6

Q5. How likely are you to act on the information or advice you were provided with today?

81

04

213

0

10

20

30

40

50

60

70

80

90

Very likely Fairly likely Neutral Fairly unlikely Very unlikely

% re

spon

dent

s

Respondents phoning Smokeline to obtain details of a LSSS were particularly likely to say they would act on information or advice provided (86% said 'very likely'). On the other hand, those calling for advice on coping with withdrawal symptoms were slightly less positive (74% said 'very likely'), as were those calling for support in staying stopped (8% said 'very unlikely'). The follow-up participants were particularly positive about hints and tips provided by Smokeline. Hints and tips which callers appeared to find particularly useful were: drink water, fruit juice, or lemon/lime cordial with water; go for a walk/take

24

dog for a walk; keep busy/do something to take your mind off smoking; brush your teeth/use mouthwash; and, undertake exercise.

When I get up first thing in the morning, I used to be cigarette, cigarette, cigarette and [the advisor] said to use chewing gum or mints, a strong mouth wash and orange juice in the morning, that’s what really helped.

(Male, 16-34, already stopped smoking) Participants said that they had initially been unconvinced by some of these tips (e.g. brushing teeth or using mouthwash) but had changed their views after having tried them. The follow-up interviews allowed exploration of actual experience of acting upon information and advice provided by Smokeline (within 7-14 days of their call). Although respondents who phoned Smokeline to obtain details of a LSSS were particularly likely to say they would act on information or advice provided, by the follow-up interview, some who had intended to contact a group had followed through on this, others had not. Among those who had not, some said they simply 'hadn't got around to it' and others had decided that they were not going to attend a LSSS at present. This finding serves to reinforce the recommendation that Smokeline advisors could be better at providing additional information to callers. Advisors referring callers to LSSS should not think the caller will receive all the available advice at a LSSS, because if the caller subsequently decides not to go, they are no further along in their quit attempt.

2.3.7 Views on the information pack Sixty-two per cent of respondents had requested the Smokeline information pack to be mailed to them. Those still smoking at the time of their call were particularly likely to have requested it. This may be because those who had already quit are also likely have called Smokeline before – in which case they may have requested the information in a previous call. However, the follow-up component revealed that participants tended to view the information pack as only being applicable to people who are at the beginning of their quit attempt, in helping them prepare to quit. Nonetheless, those who had already quit commented on how useful the pack is in helping them quit. There is therefore a role for advisors to explain the kind of information contained in the pack, and how it will be of use to people who have already stopped smoking. There is also scope for Smokeline to brand a different information pack specifically aimed at helping people stay stopped.

25

Participants who recalled ordering the pack were generally of the view that it had arrived promptly. Among respondents who had reviewed the information pack by the time of their follow-up interview, it was received very favourably. Information about the various NRT products available and the effects on the body of stopping smoking were seen as particularly useful. The fact that the materials have 'normal people' sharing their experience of quitting also appealed. The materials offer a 'round-the-clock' source of information, advice and encouragement. The booklet was frequently 'kept to hand' and re-read, and was viewed as being ‘like having [an advisor] with you’, motivating readers to 'stay on track' with their quit attempt. It was also noted that the information pack provides support outwith the noon until midnight operating hours of the phoneline service.

26

3 Recommendations for service development While respondents were overwhelmingly satisfied with the Smokeline service, the evaluation has identified some areas for service development. Improving service response to callers who have already stopped smoking The Smokeline service was initially conceived to help people who want to quit smoking – those at the outset of their quit attempt, as opposed to those who are further along their quit journey. A recurring theme to emerge throughout the evaluation is that Smokeline remains better equipped to support people who are at the outset of a quit attempt, and slightly less equipped to support people who have already stopped smoking. People who were further along their quit journey (e.g. had quit for a number of weeks or months) were, overall, slightly less satisfied with Smokeline. However, this difference is small, and the majority of these callers do feel the service is helping and encouraging them. It should also be noted that the higher level of satisfaction among prospective and recent quitters may reflect the fact that those at the outset of a quit attempt, who are contacting Smokeline in order to obtain information to help their effort, are likely to be more enthusiastic than callers who are some weeks or months in to quitting and are demotivated and 'fed-up' that they are still struggling and requiring support. This is an aspect Smokeline can focus on – both in terms of providing 'new' information and advice, as well as providing positive support to these callers. Such callers, for example, could be provided with information and advice they had not previously been made aware of: they could be made aware of LSSS and the benefits of attending them (especially in conjunction with pharmacotherapies), or provided with hints and tips to help them avoid ‘danger’ situations and help keep them motivated. Further, a number of callers in this situation found a little extra encouragement particularly motivating. Advisors providing praise to callers for how long they had managed to stop for, reiterating that the ‘worst is over’, and engaging in a discussion to remind the caller of why they had stopped smoking in the first place, were viewed as motivating. Discussions about the financial benefits and the rewards callers could treat themselves to as a result of quitting, and reassurance about the health benefits they will be experiencing by having stopped (e.g. a reminder that after 72 hours bronchial tubes relax, or that after two weeks circulation will increase, and the

27

fact that after one year the risk of having a heart attack will have substantially decreased) were also viewed as useful. The appreciation of these discussions indicates the potential value of a call-back service, whereby after a period of time (e.g. 2-3 months) the caller receives a call from a Smokeline advisor to see how they are getting along with their quit effort. The advisor can then provide further information, advice and encouragement as required. The current information pack may go someway to providing callers who have already quit with additional advice and support. However, many callers who had already quit had not requested the pack because they felt that it would only be of use to people at the outset of their quit effort. While this may be the principal focus of the pack, callers who had utilised the pack generally felt that it also contained useful reassurance after they had stopped. In light of this, Smokeline could review the information provided to people who have already stopped, and advisors could rectify the misperception that the information is only useful for those who are at the outset of a quit attempt. Further, there is scope for Smokeline to design a version of the information pack aimed specifically at people who have already quit, but are looking for help in staying stopped, and branding it as such. Advisors displaying an understanding of the realities of being a smoker and quitting smoking Advisors were viewed as empathetic and sensitive to the needs of individual callers. There was a prevailing view that an advisor who has smoked themselves is best able to understand the needs of a caller. However, advisor empathy was generally strong even among advisors who said they had never been a smoker. Criticism arose when it was felt that an advisor lacked a basic grasp of the day-to-day realities of being a smoker and quitting smoking. It may be useful therefore, if an advisor does not have first-hand experience of quitting themselves, that they are able to relate to, and discuss, the experiences of friends, members of their family, or other Smokeline callers. Smokeline advisors have to be current non-smokers, and have been encouraged not to reveal personal details, in order that they focus on the caller’s needs. However, the feedback suggests that callers actually find that hearing the experiences of other people was useful to them. Many commented that they felt further encouraged that their advisor was able to say something about their own quit attempt.

28

Consistency in amount and type of information provided to callers The follow-up interviews and the mystery shopping exercise revealed that there is some inconsistency in how much information callers are provided with when they phone Smokeline. For example, if someone contacts Smokeline with a specific query about an NRT product, or to get details of a LSSS, on some occasions, their query would be answered and the call would be terminated. On other occasions, advisors would provide more comprehensive information and advice to such callers – engaging them in a discussion about the health and lifestyle benefits of stopping, and providing hints and tips to aid their quit effort. Similarly, if an advisor referred a caller elsewhere for further information and advice (e.g. to a GP or pharmacist), the call tended to be very short. In particular, if a caller contacted Smokeline to enquire about a LSSS, or appeared very keen on the idea of such a service when told about it, they were sometimes referred to their local service and the call was terminated without any further discussion. While a key aim of Smokeline is to signpost smokers to LSSS, there is scope for Smokeline to take the opportunity to furnish the caller with additional advice, information, and encouragement, such as hints and tips and steps towards quitting, and the health and lifestyle benefits of stopping. This comprehensive approach would ensure that if a caller does not subsequently act on the information given in relation to their initial query, (e.g. they subsequently do not attend a LSSS), they have nonetheless been provided with information and encouragement, and are potentially one step further along their quit journey. Provision of accurate information and advice There were very few instances of Smokeline advisors providing inaccurate information. Where this did happen, it tended to be in relation to specific circumstances where there is less agreement on best practice, most notably, in the provision of advice concerning NRT use among pregnant women. Advisors seemed unclear about which NRT products are suitable for pregnant women, and there should be further advisor training on this specific issue.

29

Awareness of LanguageLine assistance and how to make best use of the service It is important to bear in mind that the findings regarding advisor awareness, and use, of LanguageLine assistance are based only upon two mystery shopping calls. However, the quality and relevance of information and advice received during these two calls was somewhat compromised. It appeared that not all advisors were aware of the translation service. Further, some of the loss of quality of information and advice occurred during translation – the Smokeline advisor would provide some specific information or advice in English, but this would not be translated. However, it also appeared that the advisor did not provide as much information to the translator as they could have. This is perhaps an artefact of a conversation involving a translator; the advisor may feel awkward. However, there is therefore scope for Smokeline to both ensure advisors are aware of LanguageLine assistance, but also that they are advised how to make best use of the service. Advisors should be made aware that information and advice should be provided in short chunks, with pauses, to ensure the translator can retain and relay all information and advice accurately. However, the advisor should ensure that they provide as much information as they normally would in the course of a call. Provision of 'personalised' advice Callers who felt that they had been provided with advice tailored to their needs were particularly positive about their experience of Smokeline. Where advisors had taken the time to establish detail such as smoking history, smoking routines, and previous attempts to quit, callers felt that they had received information and advice tailored to their situation. Smokeline advisors should therefore adopt a probing approach with all callers. Advisors should probe on the callers reactions to their suggestions (e.g. attendance at a LSSS, use of a particular form of NRT etc.) in order that they can reassure the caller, clarify any misperceptions they may have, and provide encouragement and suitable alternatives where necessary. Operating hours There was a desire among callers for the Smokeline service to be open before midday. A number of follow-up participants felt this would help them in their quit effort. It was suggested, in particular, that the service should be available when people are getting up in the morning/after breakfast – a time when people often require extra support as their cravings are particularly bad. Those who were previously heavy smokers are most affected by this.

30

While opening earlier in the day would undoubtedly aid the national quit effort, further research would be required to determine whether there would actually be sufficient demand for the service before midday to make this the best use of resources. The relationship between smoking cessation and mental health There is evidence that people with mental health problems are more likely to smoke than the general population, and less likely to be able to quit using the mainstream programmes on offer. Further, people with mental health problems experience particular difficulty when stopping smoking, suffering from more severe withdrawal symptoms. Indeed, a number of follow-up participants explained that they experience mental health problems. There was no evidence that these callers were any less satisfied than average, or that they felt that their needs had not been met. However, given the complex relationship between mental health and stopping smoking, it would prove valuable for Smokeline to ensure advisors are sufficiently aware of the relevant issues, and trained in dealing with, and providing useful advice to, people who experience mental health problems.

31

Appendices

32

Appendix 1 – Interactive Voice Recognition (IVR) questionnaire

Hello. Thank-you for agreeing to take part in the Smokeline caller opinion survey. The survey takes around 5 minutes to complete. Please answer the questions based on the call you have just made. Remember, your responses are completely anonymous and confidential. Please respond using the numbers on your telephone keypad. To repeat a question at any time, please press the star key.

Scale Q1 For the following question, please answer using a scale of 1 to 5, where 1 is “very satisfied”, 2 is “fairly satisfied”, 3 is “neither satisfied nor dissatisfied”, 4 is “fairly dissatisfied” and 5 is “very dissatisfied”. To repeat the scale, press the hash key now.

Q1 Overall, how satisfied or dissatisfied were you with Smokeline today? Remember, 1 is very satisfied and 5 is very dissatisfied.

Scale Q2-Q4

For the following three questions, please answer using a scale of 1 to 5, where 1 is “very good” , 2 is “fairly good”, 3 is “neither good nor poor”, 4 is “fairly poor” and 5 is “very poor”. To repeat the scale, press the hash key now.

Q2 How would you rate the advisor’s understanding of your needs? Remember, 1 is very good and 5 is very poor. Q3 How would you rate the relevance to you of the information or advice provided? Again, 1 is very good and 5 is very poor. Q4 How would you rate the quality of the information or advice provided? Thank you. If you were offered details of a local stop smoking service, please press 1. If not, press 0 Scale Q5&Q6

For the following two questions please use a scale of 1 to 5 where 1 is “very likely” and 5 is “very unlikely”.

Q5 How likely are you to act on the information or advice you were provided with today? Remember, 1 is very likely and 5 is very unlikely. Q6 How likely would you be to recommend Smokeline to a friend or relative who wanted to give up smoking? Again, 1 is very likely and 5 is very unlikely.

Q7 Thank you. If you called Smokeline today on behalf of someone else, please press 1. If you called Smokeline for information or advice for yourself please press 2. If you called Smokeline to get information or advice for both yourself and someone else, please press 3.

We would just like to know why you called Smokeline today. You can select as many reasons for calling as you want.

Q8a If you called for general advice on giving up smoking, please press 1, if not, press 0 Q8b If you called to get details of a local stop smoking service, please press 1, if not, press 0

Q8c If you called to get information about stop smoking aids like nicotine replacement patches, lozenges or chewing gum etc. or prescription medicines, please press 1, if not, press 0

Q8d If you called for advice about coping with withdrawal symptoms, please press 1, if not, press 0

Q8e If you called because you have quit smoking and wanted to speak to an advisor for support in staying stopped, please press 1, if not, press 0.

Q9 If you requested further information such as a leaflet or DVD to be sent in the post to you, please press 1, if not, press 0

Q10 If you have anything else you would like to say about your experience of using the Smokeline service today, please press 1. If you do not wish to leave a message, please press 0.

If code 1 @Q10

Please leave your message after the tone. When you have finished leaving your message please press the star key to continue. If you wish to re-record your message please press 1.

Q11 Please enter your age. For example, if you are 35 years old, please press the 3 key then the 5 key. Q12 Please press 1 if you are male and 2 if you are female

Q13 Please indicate the number of times have you phoned Smokeline in the last 12 months – not including today. If you have not phoned Smokline before today, please press 0.

Q14 Please press 1 if you smoke, 2 if you previously smoked but have now stopped, and 3 if you have never smoked. If code 1 @Q14, ask Q15

Please press 1 if you want to stop smoking and have set a quit date, press 2 if you want to stop smoking but have not set a quit date, press 3 if you are not sure if you want to stop smoking at the moment, or press 4 if you don’t want to stop smoking at the moment.

Q16a

Ipsos MORI, the independent research organisation conducting this survey on behalf of NHS Health Scotland would like to speak in more detail to some callers about their experience of calling Smokeline. The findings will be used to make improvements to the Smokeline service. If you think you might be interested in taking part in a 20 minute telephone conversation with a researcher, in the next two weeks or so, please press 1. If you do not wish to speak to a researcher, please press 0. To repeat this question, press the star key.

If code 1 @Q16a

Thank you. Please leave your name, telephone number and postcode after the tone. When you have finished leaving your details please press the star key to continue. If you wish to re-record your message please press 1.

Closing Thank you for participating in this survey, your feedback is very much appreciated. Goodbye.

33

Appendix 2 – Follow-up interview topic guide Introduction

• Introduce self, Ipsos MORI – reiterate that you are a social researcher not a stop-smoking specialist • We’ve been commissioned by NHS Health Scotland to look at the Smokeline service, find out

customers experiences and suggest ways in which the service could be improved • Thank them very much for taking part in the survey and agreeing to take part in follow up • Will take around 20-30 mins • Emphasise confidentiality and anonymity • Request permission to record

Background to call

• How did you find out about Smokeline? • Why did you call? • What prompted you to call on that day in particular? • What did you hope to get from the call? • Had you ever called before? • (If applicable), probe on stage of quit journey at time of call: • Had you already stopped smoking? Set a quit date? Tried to stop before?

Experience of the call Tell me about the call……

• What was the advisor like? [Possible prompts if ness: helpful? friendly? encouraging? understood you? knowledgeable? off-putting? annoying? didn’t understand/listen to you?]

• What did you ask about? • What questions were you asked? • What are the main things you remember about what you were told? [Possible prompts if ness:

Groups, NRT, prescription medication, tips for reducing stress, health benefits, information about withdrawal symptoms?]

• Was information clearly explained and easy to understand? [In particular, any detailed information provided about different NRT, medicines, or groups]

• Was there anything that you didn’t understand? • How did you feel during the call? • Was it what you expected? • What would you say was good about the call? • What would you say was not so good about the call? • Were you told anything that surprised you? • Were you asked about anything that you didn’t expect to be asked about? How did you feel about

that? After the call

• Can you remember how you felt immediately after you had made the call? • Tell me what’s happened since? [probe for stages in quit journey and whether they feel any actions

were as a result of the call or would have happened anyway] • Have you been able to act on any of the advice or information you were given?

34

• What was the most helpful thing you were told? • Was there any advice or information you were given that you haven’t used? • What was the least helpful thing you were told? • Have you changed your views about anything since you called? [probe: things they didn’t think would

work/help before] Postal information

• Were you sent anything from Smokeline [probe: ‘Aspire’ booklet or ‘We can help you to stop smoking’ DVD]?

• Have you read/watched it? • How did you find it?

Ideas for improvement

• Have you called again? Would you call again? • What do you think Smokeline could do to improve the service? • Probes: • Should the advisors do anything differently? • Anything about the way the advisors speak to you? • Questions you are asked? • Information you are given? • Advice you are given? • Opening hours?

Is there anything you think is particularly good about Smokeline that they should keep the same? Is there anything else you want to say ? About Smokeline? About stopping smoking?

Thank and close

Appendix 3 – Mystery Shopping Scenarios 'Go it alone': Has smoked for years, thinking about quitting for first time after medical concerns expressed by GP. Reluctant to consider groups or other services and wants to go it alone. (5 calls)

35

'I've heard about groups and would like more information about them': Has smoked for years, has tried to quit a few times before. A friend has mentioned groups, so interested in finding out what’s available locally. (5 calls)

'Concerned I might relapse': Has already stopped smoking but are concerned might relapse. Contacting Smokeline for support in staying stopped. (5 calls)

'Pregnancy': Just found out she is pregnant with her first child. Has been told by a friend with kids that she should quit for the health of the baby. Wants to know the facts about effects of smoking on her baby. Thinks she could cut down but isn’t confident about quitting completely. (5 calls)

'I've heard about prescription drugs that can stop you straight away': Has tried quitting before a number of times using NRT but never stayed quit very long. Fed up and wants something that reduces the need for willpower – has heard that there are a couple of prescription drugs now that can stop you straight away. (5 calls)

'LanguageLine assistance': Poor English-language skills, so will need LanguageLine assistance. Wants general information on quitting, wonders if there are any services available in their language. (2 calls) 'Secret smoker': Friends and family do not know you smoke so need to quit in secret. Cannot attend regular groups, wondering what other options are available. (2 calls) 'Work shifts and travel': Do shift work so cannot attend regular group. Also travel between Scotland and Poland frequently so needs something to keep him going in both places. Main motivation is financial benefits. (3 calls)

Appendix 4 – Mystery shopping evaluation form

Evaluation of Smokeline Mystery shopping evaluation form

36

The main aim of this mystery shopping exercise is to evaluate the accuracy and quality of the information and advice provided by Smokeline advisors, as well as to evaluate their call-handling skills. A key purpose is to ‘quality assure’ the information provided about the various medications (including prescription-only medications such as Zyban and Champix plus Nicotine Replacement Therapies) and group and one-to-one support sessions. Please bear this in mind while you are completing this form. We do not expect you to know all about medications and support groups etc., but the project researchers will be assessing the quality of the service through the information you record on this form so please also provide as much feedback as possible about:

• information, hints and tips, or advice you were given • the advisor's understanding of the specific needs you outlined • empathy of the advisor • overall call handling

Interviewer name: Interviewer number: / Scenario number: Caller number: Your (real life) telephone number (in case we need to contact you)

Section One: Pre-planned answers to the demographic questions you will be asked about at the end of the phone call Please complete this section with the details that you used during the call.

37

Please have these details prepared before the call and, unless you have to change these details in order to fit with your scenario, please use your own real life details. Date of birth Day Month Year Ethnicity Address, including postcode Gender: Male Female Employment status Number of cigarettes smoke per day Length of time smoked Section Two: Date and time of call Date of call: Day Month Year Time at start of call (please use 24 hour clock): Hours Minutes

38

Time at end of call (please use 24 hour clock): Hours Minutes Section Three: Call handling and advisor empathy Q1: For each of the statements below, please tick one response option a) It was easy to understand the information/advice you were given

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

b) The advisor was friendly

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

c) The advisor was helpful

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

d) The advisor was polite

Strongly agree

Tend to agree

39

Neither agree nor disagree

Tend to disagree

Strongly disagree

e) The advisor listened to you

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

f) The advisor dealt with you efficiently (i.e. not overly hurried or dawdling)

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

g) The advisor spoke clearly

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

h) The advisor was calm

Strongly agree

Tend to agree

40

Neither agree nor disagree

Tend to disagree

Strongly disagree

i) The advisor probed for additional information in order to understand your situation

Strongly agree

Tend to agree

Neither agree nor disagree

Tend to disagree

Strongly disagree

Q2: Were you asked whether you would like to be sent any further information about stopping smoking (e.g. be sent literature/DVD through the post)? Yes No Q3: How would you rate the advisor’s understanding of your needs? (please tick one box only) Very good Fairly good Neither good nor poor Fairly poor Very poor Add any comments: Q4: How would you rate the relevance, to your scenario, of the information or advice provided? (please tick one box only) Very good

41

Fairly good Neither good nor poor Fairly poor Very poor Add any comments:

42

Section Four: Details of local stop smoking services Q5: If you were offered details of a local stop smoking service/s, please record as much information as possible about these services below. (If you were offered details of more than 3 services, please record this information on an attached sheet) Details of Service 1 Name of Service: Location of service: Description of what service offers:

Any other information you were given about the service: (Please also note whether you had to ask for this information or whether it was given to you unprompted)

43

Details of Service 2 Name of Service: Location of service: Description of what service offers:

Any other information you were given about the service: (Please also note whether you had to ask for this information or whether it was given to you unprompted)

44

Details of Service 3 Name of Service: Location of service: Description of what service offers:

Any other information you were given about the service: (Please also note whether you had to ask for this information or whether it was given to you unprompted)

45

Section Five: Information about stop-smoking medicines, NRT, and aids to help quitters Q6: If you discussed prescription-only medications (such as Zyban/bupropion or Champix/varenicline), or other stop smoking aids such as nicotine replacement therapies (NRT, like patches, gum, lozenges, inhalators etc.), please record as much information as possible about which ones you were told about and what you were told about each one below. Please also note whether you had to ask for this information or whether it was given to you unprompted. (Please continue on an attached sheet if necessary). Q7. Were you told about Zyban (bupropion)? Yes No Q8. Was Zyban recommended to you? Yes No Q9. Were you told how you can get hold of Zyban? Yes No Q10. If you were told about Zyban, what were you told about it?

Q11. What, if anything, were you told about how Zyban works – and how long you should use it for? – Please indicate whether each piece of information was prompted or unprompted.

46

Q12. What, if anything, were you told about how safe it is to use Zyban? – Please indicate whether each piece of information was prompted or unprompted. Q13. What, if anything, were you told about the success of using Zyban? – Please indicate whether each piece of information was prompted or unprompted. Q14. Were you told about any side effects of Zyban? Yes No Q15. What, if any, side effects of Zyban were you told about? Q16. Were you told about Champix (varenicline)? Yes No Q17. Was Champix recommended to you? Yes No Q18. Were you told how you can get hold of Champix? Yes No

47

Q19. If you were told about Champix, what were you told about it? – Please indicate whether the information you received was prompted or unprompted.

Q20. What, if anything, were you told about how Champix works – and how long you should use it for? – Please indicate whether each piece of information was prompted or unprompted.

Q21. What, if anything, were you told about how safe it is to use Champix? – Please indicate whether each piece of information was prompted or unprompted.

Q22. What, if anything, were you told about the success of using Champix? – Please indicate whether each piece of information was prompted or unprompted.

48

Q23. Were you told about any side effects of Champix? Yes No Q24. What, if any, side effects of Champix were you told about? – Please indicate whether each piece of information was prompted or unprompted. Q25. Were you told about any of the following nicotine replacement therapy (NRT) products? Patches Yes No Gum Yes No Lozenges Yes No Inhalators Yes No Nasal spray Yes No Microtabs (tablets) Yes No

49

Please record in the boxes below what, if anything, you were told about the various NRT products – please record any information you were told about (i) NRT generally (ie how many forms there are, safety in general and/or for particular groups of the population), how successful they are in comparison with each other, how to get them, etc, and (ii) each type of NRT discussed (how they work, how long you use them for, etc), (iii) whether they were recommended to you, and if so, whether one particular form was recommended over the others.

GENERAL INFORMATION ABOUT NRT

NICOTINE PATCHES

GUM

LOZENGES

INHALATORS

50

NASAL SPRAY

MICROTABS / SUBLINGUAL TABLETS

Information in general / comparisons of pharmacotherapies Q26. Were comparisons, including pros and cons, of each type of pharmacotherapy/medication/smoking cessation aid (ie Zyban, Champix, NRT) discussed with you? Yes No Please write in the box what was discussed – include any information you were given about the pros and cons of any pharmacotherapy/medication/smoking cessation aids discussed: Q27. Was one particular type of pharmacotherapy/medication/smoking cessation aid (ie Zyban, Champix, NRT) favoured over the others? Yes No

Please write in the box what was discussed – include any information about which particular type of pharmacotherapy/medication/smoking cessation aid was favoured and why:

51

Q28. Was it strongly recommended to you to use pharmacotherapies/medication/smoking cessation aids in conjunction with group/1:1 support to improve your chances of success at stopping smoking? Yes No Please write in the box what was discussed – include the reasons you were given for combining aids/medicines with group or 1:1 support: Q29. Was the effect of combining pharmacotherapies/medication/smoking cessation aids with support services explained to you? Yes No

Please write in the box what was explained to you:

52

Section Six: Coping with withdrawal symptoms and stress Q30. Were you told anything about coping with withdrawal symptoms and /or stress? Yes No Q31. If yes, what were you told about withdrawal symptoms (likelihood of getting them i.e. that not everyone gets them and it’s not related to how long you’ve smoked or how many you smoke or whether you’ve experienced them before; coping with them; how long they might last; and any problems you might face?) – Please indicate whether each piece of information was prompted or unprompted. Q32. Were you told the following stress reducers? If you feel like smoking… Y N … ‘walk away from it’ - take a break, have a drink of water or juice … try deep breathing and stretching … phone a friend who makes you laugh, or watch a comedy video … do something active, like go for a walk, attend a fitness class, etc.

53

Q33. Please record any other advice, hints or tips you were given about reducing stress while you quit smoking in the box below – Please indicate whether each piece of information was prompted or unprompted. Section Seven: Provision of general information, hints and tips Q34. Were you told any of the following pieces of information? Y N … nicotine reduces feelings of hunger, so you may feel hungrier when you stop smoking ………………. … nicotine speeds up the rate at which your body burns calories so you may find that you do not burn calories as quickly as before ………………………… food will start tasting better when you stop smoking …………… most people who stop smoking tend to consume more calories – largely through snacking between meals more often, especially on sweet foods………………………………………………………… .. …

54

Q35. Were you advised… Y N … to avoid snacks like biscuits and cakes, and instead to try fruit, breadsticks, crackers or oatcakes etc. … to eat lots of fruit and vegetables at mealtimes … to get more active Q36. Were you told to plan rewards? (for example, to reward yourself for managing 1 day/1 week/1 month without smoking by, for example, ………. going for a massage, having a night out, going for a walk in the country, phoning a friend, going bowling, watching TV, booking a holiday, going shopping, going to the cinema etc.) Q37. If you were told to plan rewards, what were you told? – Please indicate whether each piece of information was prompted or unprompted.

Q38. Were you told about the benefits giving up smoking has in terms of saving you money? Yes No If yes, what were you told? – Please indicate whether each piece of information was prompted or unprompted.

55

Q39. Were you told that changing your routines (for example, cutting out caffeine or alcohol) will help you stay stopped? Yes No If yes, what were you told? – Please indicate whether each piece of information was prompted or unprompted.

Q40. Were you told about the health/fitness benefits of giving up smoking? (For example, were you told information like: ‘it will make breathing easier’, ‘it will improve circulation’, ‘quitting will put you at less risk of a heart attack’, ‘it will improve your energy levels’, etc.) Yes No If yes, what were you told? – Please indicate whether each piece of information was prompted or unprompted. Q41. Were you given any of the following helpful hints about stopping smoking? … if you need to put something in your mouth, try sugar-free chewing gum or something healthy ………………… Yes No and non-fattening like a carrot … find something to fiddle with if you need to do something with your hands …………………………………... Yes No

56

… try inhaling and exhaling in the way you would if you were puffing on a cigarette ………………… Yes No … try drinking fruit juice or eating fruit when you feel like a cigarette ……………………………………….. Yes No Q42. If you were given any other helpful hints, or DOs and DON’Ts about quitting, please record them in the box below. Please also indicate whether each piece of information was prompted or unprompted. Continue on an attached sheet if necessary. Q43: If you were provided with information about any effects smoking can have on the health of babies, please record as much information as possible below. Please also note whether you had to ask for this information or whether it was given to you unprompted. (Please continue on an attached sheet if necessary)

57

Section Eight: Scenario Specific Questions Please answer the questions relating to the specific scenario you called Smokeline about Again, please note whether information was prompted for or provided unprompted. Scenario One Q44. What was the advisor's response when you said you do not want to consider groups and services but would rather 'go it alone'?

Q45. Did the advisor ask why you do not want to consider groups and services but would rather 'go it alone'?

Q46. What advice, if any, were you given about groups and services?

Q47. What advice, if any, were you given about 'going it alone'?

58

Q48. Did the advisor mention pharmacotherapies/medicines/stop-smoking drugs to you? Yes No Q49. What, if anything, were you told about combining pharmacotherapies/ medicines/stop-smoking drugs with group or 1:1 support services?

Scenario Two Q50. What was the advisor's response when you said that you had tried to quit previously?

Q51. What was the advisor’s response when you said that your friend had mentioned groups to you?

Q52. Did the advisor offer to pass your details on to support services instead of only offering to give you the contact details of local service/s so you can contact them?

59

Yes No Please ensure that you have recorded full details of the information you were given about groups in Section 4 of this form. Scenario Four Q53. What was the advisor's initial response when you said that you smoke and you are pregnant? Q54. What was the advisor's response when you said that you had been told by a friend that you should quit for the health of your baby? Q55. What specifically did you tell the advisor what your friend had told you? Q56. What was the advisor's response to what you said your friend had told you?

60

Q57. What facts were you given about the effects of smoking on your baby?

Q58. What was the advisor's response when you said that you thought you could cut down but weren't confident about quitting completely?

Q59. What, if anything, did the advisor say about using NRT when pregnant?

Scenario Five Q60. What was the advisor's response when you said that you had tried to quit previously?

61

Q61. What information were you given about your experience and use of that form of NRT (if any), about other forms of NRT (if any), prescription drugs/medications/smoking cessation aids, and any other aids, to help you stop smoking straight away?

Q62. What was the advisor’s response when you said you had heard about prescription drugs/medications/smoking cessation aids that can help you stop smoking straight away?

Please ensure that you have recorded full details of the information you were given about NRT products and other prescription drugs/medications/smoking cessation aids in Section 5 of this form. Section Nine: Everyone to answer these questions Q63: Were you asked at the end of your phone call whether you wanted to take part in a further survey? Yes No Q64: Please note below, or on an attached sheet, any other observations or comments about your experience of calling Smokeline.

62

I confirm that this evaluation was carried out in accordance with the market research society code of conduct on Mystery Shopping. Interviewer signature: Thank you, now please double check the responses on this form and return it in the pre paid envelope included in your pack.