How do respondents explain WTP responses? A review of the qualitative evidence

The Journal of Socio-Economics 37 (2008) 1427–1442

How do respondents explain WTP responses?A review of the qualitative evidence

Rachel Baker a, Angela Robinson b,∗, Richard Smith b

a Newcastle University, United Kingdomb University of East Anglia, United Kingdom

Received 17 May 2006; received in revised form 23 March 2007; accepted 10 April 2007

Abstract

Alongside a growing body of empirical research relating to willingness to pay (WTP) valuations ofthe environment, health and safety, there is mounting evidence of embedding, framing effects and otheranomalies in responses. Gaining an understanding into how respondents arrive at WTP values is crucial todetermining the possible reasons for such anomalies and helping to construct more ‘valid’ WTP instruments.

This paper reports a comprehensive literature review of qualitative research conducted alongside the elici-tation of WTP values in the areas of environment, transport safety and health. Our review revealed a paucity ofwork in this area and the need for further in-depth studies of this kind. Despite a wide range of studies in differ-ent sectors, with different focus in terms of the nature of the goods in question and the objectives of the quali-tative studies, we identify four preliminary themes: mental accounting, lack of trust, moral outrage and moralsatisfaction. The relevance of such findings for the design and interpretation of WTP studies is discussed.© 2007 Elsevier Inc. All rights reserved.

JEL classification: D61; I10; Q51

Keywords: Willingness to pay; Qualitative research; Systematic review

1. Introduction

The recent treasury ‘green book’ urges greater emphasis on the quantification of monetary ben-efits in all sectors of the UK economy, including the environment, transport and health, promptingrenewed interest in the use of willingness to pay (WTP) in these sectors. The international body ofempirical research relating to WTP is extensive and the potential problems associated with WTP

∗ Corresponding author at: School of Medicine, Health Policy & Practice, University of East Anglia, Norwich NR47TJ, United Kingdom.

E-mail address: [email protected] (A. Robinson).

1053-5357/$ – see front matter © 2007 Elsevier Inc. All rights reserved.doi:10.1016/j.socec.2007.04.001

mailto:[email protected]

dx.doi.org/10.1016/j.socec.2007.04.001

1428 R. Baker et al. / The Journal of Socio-Economics 37 (2008) 1427–1442

well documented. For example, insensitivity to scale and scope – where respondents state muchthe same WTP for different levels of benefit – otherwise known as ‘embedding’ (Jones-Lee et al.,1995; Dubourg et al., 1997; Olsen et al., 2004). Such findings are at odds with the neo-classicaltheory of consumer behaviour (in which WTP is rooted) and have been uncovered in studies con-ducted in a range of different fields, including environment (e.g. Loomis et al., 1993), transportsafety (e.g. Jones-Lee et al., 1995) and health care (e.g. Olsen et al., 2004).

Despite this mounting empirical evidence of anomalies in WTP responses, relatively few qual-itative studies have been carried out in order to determine how respondents answer such questions.Yet, without this, it is difficult to explain apparent anomalies in responses by observing empiricalresults alone, and an examination of the qualitative evidence is crucial to our understanding ofthe nature of what is being elicited through WTP responses.

Whilst the nature of the good differs markedly in different settings (for example, from pure‘public’ goods in many environmental studies, to essentially ‘private’ reductions in own risk in anumber of safety studies), the pervasiveness of anomalies such as embedding suggests that certainqualitative findings may be common across contexts. In this paper, we therefore review thosequalitative studies carried out alongside the elicitation of WTP values in the fields of environment,safety and health.

The use of qualitative methods to explore responses to preference elicitation techniques is inits infancy and presents particular challenges. This paper therefore also describes the range ofqualitative techniques typically deployed in this type of study.

In sum, the objectives of this review are three-fold:

• Identify studies from the environment, safety and health literature that use qualitative methodsto investigate WTP responses.

• Identify common themes in qualitative responses that may explain anomalies in WTP responsesacross the various contexts.

• Describe the range of qualitative techniques deployed in the identified studies.

2. Methods

2.1. Search strategy

The search strategy was designed in two parts. Contained within the first parentheses are a listof qualitative terms which have been adapted from Petticrew and Roberts (2006). These qualitativeterms are then combined with search terms used to describe willingness to pay methods. The fullsearch is given below.

(“qualitative research” or “qualitative stud*” or ethno* or phenomenolog* or “groundedtheor*” or “participant observ*” or “purposive sampl*” or “content analy*” or “them*analy*” or “constant compar* method*” or “theor* sample*” or “discourse analy*” or“focus group*” or “verb* protoco*”) and (“willingness to pay” or “WTP” or “contingentvaluation”)

Electronic databases searched were: Web of Knowledge, Econlit, Medline, WorldCat, ERIC,PsychInfo, ASSIA, Environmental sciences and pollution management, Social science abstracts,and Water resources abstracts.

Titles and abstracts were initially reviewed by one of the authors (RB) to determine whetherpapers reported qualitative results relating to a WTP study. Those studies that used qualitative

R. Baker et al. / The Journal of Socio-Economics 37 (2008) 1427–1442 1429

methods only at the study design stage were not retrieved. Where it was clear from the abstractthat the qualitative data referred to written comments on a questionnaire, publications were notretrieved. Papers which did not include qualitative data, or which were not WTP studies wereexcluded.

Preliminary investigation (by RB) indicated that qualitative studies carried out alongside WTPexercises often receive scant attention in journal articles, which tend to focus on the quantitativeempirical results (see also Coast et al., 2004). We considered that substantive qualitative evidencemay also be found in book chapters, reports and theses and so the provisional list of papers wassent to a number of ‘key’ people in the field (those we considered likely to have conducted studiesthemselves and/or be familiar with the subject area) who were asked to identify any missingreferences.

All papers obtained were reviewed independently by two of the three authors guided by a dataextraction form (available from the authors on request). At this stage papers were once againexcluded if, despite indication from the abstract, they did not report qualitative investigation ofempirical WTP responses.

2.2. Interpretative reviewing

We take a thematic approach in this review, identifying prominent themes and summarisingthese under thematic headings (see Dixons-Woods et al., 2005 for an account of interpretativeas opposed to integrative reviewing). Two of the authors (AR and RB) coded the findings, iden-tified and agreed the common themes through a discursive process akin to qualitative analytictechniques. The quantitative and qualitative methods used are tabulated and all papers that con-tributed data to each theme are listed in the text. Whilst we identify the recurrent themes, it isimportant to acknowledge that the themes as we present them are, to some extent, theory driven.Given the background of all three authors in health economics (and qualitative methods) our inter-pretation of the findings reported are unavoidably guided by our shared disciplinary perspective.Our stated interest in the ‘validity’ of the WTP method further shapes the themes which naturallyfocus on the exploration of anomalous findings.

3. Results

After duplicates were discarded, the electronic search identified a total of 229 papers. Afterreview on title and abstract, 34 of the 229 were retrieved for analysis. A search of these papers’references identified one additional paper [39]. Contact with experts in the area led to the identifi-cation of 2 book chapters, 1 report and 1 conference paper, giving a total of 39 papers for review.The list of the 39 papers reviewed is given in Appendix A. Five of these had been conducted ina health context, 26 were environmental studies and 3 were concerned with transport safety. Ofthe remaining 5 papers, 2 were in the development field, 1 in agricultural economics and 2 inexperimental economics.

A number of papers deployed focus groups techniques only as a means of developing thedescription of the good for purposes of questionnaire design or for gaining further insights intohow respondents perceived the good after the valuation exercise [1,2,13,18,19,25–28]. Otherpapers were commentaries on the use of qualitative techniques in WTP studies, but did notinclude actual data collected alongside an empirical study [14,20,22,23,29]. One study, whilst itappeared to be a qualitative study was in fact a quantitative analysis [24]. One study used focusgroups to elicit preferences over a novel funding scheme [33]. Whilst it was considered that the


papers from the agricultural and development fields were suitable for inclusion in the review (asthe ‘goods’ shared common features with those in the other papers), two papers [6,9] describinglaboratory experiments involving money lotteries were judged to be quite different in focus. Thesepapers were excluded from analysis.

The results reported in this paper relate to the 21 remaining publications, the essential featuresof which are given in Table 1. As expected, given the high number of environmental studies,most have been concerned with valuing public, rather than private, goods. Table 1 also shows thediverse methods used to elicit WTP responses with both question format and payment vehiclevarying across studies.

The objectives of the qualitative components to these WTP studies (where stated) variedaccording to the focus of each study. Some set out to investigate the relevance of theoreticalconstructs to the qualitative analysis of reasons behind WTP responses [e.g. 5,34]. Others adoptedmore general objectives of ‘capturing respondents’ motivations’ (p. 122) and ‘what meanings theyattach to their answers’ (p. 123) [36] or whether ‘respondents feel that their WTP figures were agood way of representing their values . . .’ (p. 47) [16]. Still other studies used qualitative methodsto explore or challenge responses which were apparently ‘irrational’ [12,17,35].

The extent to which the qualitative data were formally analysed and analytic methods reported,also varied a good deal. Whilst there are papers that could be described as predominantly qualitativein nature [16,32,38], other papers in this review are essentially quantitative reports with thequalitative component of the studies described only briefly in the paper [3,4,21].

The range of policy contexts, research questions and qualitative methods used clearly rulesout a ‘qualitative meta analysis’ of the papers. Whilst we have adhered to practices of systematicreviewing in our search and data abstraction, we have selected to take a thematic approach (Dixons-Woods et al., 2005) to the presentation of results, as described in the methods section. An alternativeapproach would have been to ‘select-out’ papers that did not meet pre-set criteria, such as indicatorsof study quality, which would have reduced the papers reviewed to a very small number, the resultsof which may have been amenable to further analysis. However, our aim is to be inclusive at thisstage, to assess the extent and the nature of the existing research in this area; we are interested inhighlighting all those studies that qualitatively explore WTP responses, however disparate, andthe themes we report are drawn from the range of papers. Additionally we identify a small subsetof papers which are comparable in their specific investigation of ‘embedding’.

3.1. Qualitative methods used

As anticipated, a range of different qualitative approaches was used to investigate the responsesgiven to WTP questions. Table 2 lists each paper and the methods used for data collection andanalysis, where this is stated in the paper. The studies varied in relation to the timing of datacollection, the methods used to generate qualitative data and the approach to data analysis.

Five of the 21 studies used concurrent qualitative data collection methods. These includedthink aloud techniques, or else the use of interviews and focus groups where a verbal discussionof WTP values was simultaneous with completion of a WTP questionnaire. Fourteen studies usedretrospective techniques, such as interviews and focus groups that took place at some intervalafter the completion of the WTP questions. The two remaining studies used a combination ofboth concurrent and retrospective methods.

The analytic methods listed in Table 2 range from relatively quantitative approaches, suchas content analysis, to more open ended ‘grounded’ qualitative approaches, such as discourseanalysis. Whilst a description of analysis was scant in many of the papers, in others it is described

R.B

akeretal./T

heJournalofSocio-E

conomics

37(2008)

1427–14421431

Table 1References included: quantitative methods used

Study Field Country Population The good Format of WTP Payment vehicle Test of embedding?

3. Bateman et al. (2001) Environment UK General Protection of the NorfolkBroads from salineinundation (pub)

Dichotomouschoice

Out of pocket

4. Beattie et al. (1998) Transport safety UK General (a) Reduction in own riskof road traffic death andinjury (private)

(a) Paymentscale

(a) Out of pocket (a) Yes

(b) Reduction in numbersof deaths on roads (pub)

(b) Open-ended (b) Contribution tosafety scheme

(b) Yes

5. Blamey (1998) Environment Australia General Protection of SouthAustralian Coorang fromgroundwater drainage(pub)

One off payment$50

Water rates/various

7. Brouwer et al. (1999). Environment UK General Protection of thebroadlands from salineinundation (pub)

Dichotomouschoice

Taxation

8. Burgess et al. (1998). Environment UK General + specialinterest

Wildlife enhancementscheme on marsh land(pub)

Bidding Taxation

10. Cameron (1997) Environment Australia General + specialinterest

Improved water quality ofthe Hawkesbury-Nepeanriver (pub)

Open ended Taxation

11. Chilton et al. (2004) Environment/health UK General Improvement in health viareduction in air pollution(pub)

Payment card Higher prices on goodsand VAT

Yes

12. Chilton et al (2003) Transport safety UK General Reduction in numbers ofdeaths on roads (pub)

Open-ended Contribution to safetyscheme

Yes

15. Chilton & Hutchinson(2003)

Environment Ireland General Expansion of afforestationand planting of peatland(pub)

Open-ended Out of pocket Yes

16. Clark et al. (2000) Environment UK General + specialinterest

Wildlife enhancementscheme on marsh land(pub).

Bidding Taxation

1432R

.Baker

etal./The

JournalofSocio-Econom

ics37

(2008)1427–1442

Table 1 (Continued )

Study Field Country Population The good Format of WTP Payment vehicle Test of embedding?

17. Currie et al (2005) Health Canada General Expansion of threedifferent health careprogrammes (pub)

Payment card Taxation/voluntarycontribution

Yes

21. Jones-Lee et al. (1995) Transport safety UK General + student Reduction in own risk ofroad traffic death andinjury (private)

Payment scale Out of pocket Yes

30. Philip et al. (2005) Environment UK General Protection of wildlife andendangered species (pub)

Payment card Taxation

31. Powe et al. (2004). Environment UK Special interest A biodiversity scheme onland owned by Southernwater (pub)

Open-ended Willingness to foregoreduction in water bill

32. Schkade & Payne(1994)

Environment USA General Protection of migratorywaterfowl from waste-oilholding ponds (pub)

Open ended Higher petrol prices Yes

34. Shiell, & Rush (2003) Health Canada Convenience Vaccination programmesfor (a) self (private) and (b)others (pub)

Payment card Out of pocket Yes

35. Shiell, & Gold (2002) Health UK Convenience Vaccination programme forself (private)

Payment card Out of pocket Yes

36. Shiell & Gold (2003) Health UK Convenience Vaccination programme forself (private)

Payment scale Out of pocket Yes

37. Smith (2007) Health Australia Population Payment to avoid healthstates (private)

Open ended Out of pocket

38. Svedsater (2003) Environment UK General + student Reduction in globalwarming by reducing greenhouse gases (pub)

Open ended Taxes or higher prices

39. Vadnjal & O’Connor(1994)

Environment New Zealand General Avoidance of developmentof an undeveloped Island(pub)

Open ended Out of pocket

Paper 4 covers two studies – A and B – that took two different approaches. Study 4b is also reported in paper 12. In the table above, ‘private’ refers to a good which accrues benefit onlyto the individual consumer of their household, whilst ‘pub’ refers to a good that, once provided, also accrues benefit to the wider ‘community’ (defined variously across the differentstudies).

R.B

akeretal./T

heJournalofSocio-E

conomics

37(2008)

1427–14421433

Table 2References included: qualitative methods used

Number and abbreviatedreference

Qualitative methods

Data collection: timing Data collection: method Data analysis (as reported)

Concurrent Retrospective Focusgroup

Interview Thinkaloud

Writtenresponses

Thematicanalysis

Contentanalysis

Verbalprotocolanalysis

Discourseanalysis

Groundedtheorymethods

Other

3. Bateman et al. (2001) X X NS4. Beattie et al. (1998) X X X X NS5. Blamey (1998) X X NS7. Brouwer et al. (1999) X X Majority

views8. Burgess et al. (1998) X X NS10. Cameron (1997) X X NS11. Chilton et al. (2004) X X X12. Chilton et al. (2003) X X X X15. Chilton & Hutchinson

(2003)X X X

16. Clark et al. (2000) X X X17. Currie et al. (2005) X X X21. Jones-Lee et al.

(1995)X X NS

30. Philip et al. (2005) X X X X31. Powe et al. (2004) X X X X NS32. Schkade and Payne

(1994)X X X X

34. Shiell & Rush (2003) X X X35. Shiell and Gold

(2002)X X X

36. Shiell and Gold(2003)

X X X

37. Smith (2007) X X X38. Svedsater (2003) X X X X X X39. Vadnjal and O’Connor

(1994)X X Lexicological

analysis


in some detail [6]. Seven papers did not state the analytic approach taken. The qualitative methodsused will impact on the nature of the data generated in a number of different ways.

3.2. Qualitative findings: thematic analysis

In this section, we describe the prominent themes identified across the 21 papers under thesubheadings: ‘Mental accounting’, ‘Lack of trust’, ‘Moral outrage’, and ‘Moral satisfaction’.

3.3. Mental accounting

Mental accounting describes the tendency for respondents to set a mental budget from whichthe payment for the good is to be made. The manner in which this notional budget is set appearsto take on a number of different forms. First, a number of studies report a notional budgetbased on an amount that would not seriously disrupt normal expenditure and saving patterns[4,11,12,15,17,21,37,38]. This was variously expressed as money that would ‘not be missed’ orwas ‘not significant’.

Alternatively, respondents considered an amount they normally contribute towards some otherworthy cause and use this as the basis for their response [30,32,37]. In some cases, this ‘benchmark’other cause would seem to have little in common with the good under valuation. For example, whenasked about their WTP in higher petrol prices in order to protect migratory wildfowl, respondentsin the Schkade and Payne study considered what they paid towards a range of disparate causes,including MADD (Mothers Against Drunk Drivers), and ‘the presidential thing’. When askedabout their WTP to avoid a poor health state, a respondent in the study by Smith is quoted asconsidering how much they currently give to the Salvation army [37].

A related strategy that appears to have been used to set the notional mental budget is to usesome ‘benchmark’ good as a comparison and consider how much that good would cost [21,35].In one study that set out to elicit WTP for vaccination prior to taking a foreign holiday [35],respondents set this limit by considering the cost of other vaccinations or holiday insurance.Similarly, in a study of WTP for reduction in the risk of suffering non-fatal road accident, certainrespondents focused on the cost of specific safety features, such as cycle helmets and seatbelts astheir benchmark [21].

Mental accounting is found in studies considering both public [4,11,12,15,17,38] and private[41, 21,35,37] goods, so would appear to be un-related to the nature of the good under consid-eration. The common feature of those studies reporting mental accounting of this kind is thatthe self-imposed budget is related to insensitivity to scale and scope where tested explicitly (seeTable 1). That is, once they have set themselves a limit (which may have little to do with howthey value the good or even their total income), they state much the same WTP regardless of themagnitude of the benefit derived. And, of course, responses that are anchored to some benchmarkor other – such as the contribution to other causes or the cost of travel insurance – will necessarilybe dependent on the choice of benchmark.

3.3.1. Lack of trustA common theme to emerge from the studies reviewed is respondents expressing scepticism

that the benefits under valuation will actually be realised and a lack of trust in the agency charged

1 Paper 4 considered two separate studies involving both private and public goods. The existence of mental accountingis reported in both contexts.


with delivering the good [5,7,11,12,16,30–32,38]. Such sentiments are closely related to thepayment vehicle used in the study.

Several of the studies in which respondents revealed a lack of trust, or scepticism, makeuse of a taxation payment vehicle in one form or another [5,7,11,16,30,38]. In one study thatset out to value a reduction in the health effects associated with air pollution via higher pricesand VAT, protest zero responses were related to a lack of trust that the money would be spentto bring about the benefit and that current tax revenues ought to be re-deployed [11]. Anotherstudy using taxation as the payment vehicle funding a wildlife enhancement scheme also foundconcerns over how the money would be spent [16]. The use of taxation as the payment vehi-cle also results in concerns over the contribution of others and the ‘fairness’ of the system[33].

Whilst such findings suggest that the use of taxation as a payment vehicle is problem-atic, scepticism and lack of trust are also reported in studies using private companies as thepayment vehicle [31,32]. For example, in a study using willingness to forego a reduction inwater rates in order to improve biodiversity on land owned by the water company, respon-dents expressed a lack of trust in the water company and scepticism that they would use themoney to bring about the promised improvement [31]. Whilst it seems plausible that lack oftrust would result in protest zero responses, and that link was made in one study that set outto explain such responses [11], the information reported in other studies was generally insuf-ficient to link qualitative and quantitative findings in this way. The common feature of thestudies discussed in this section is that all studies relate to a good that is provided publiclyand respondents appear to be questioning the link between their WTP and the provision of thegood.

3.3.2. Moral outrageThis theme describes the moral outrage – or indignation – expressed by respondents when

asked to place a value on a good they feel it is inappropriate to consider in monetary terms. Forexample, in a study to improve water quality in a river to render it safe for swimming, boatingand other recreational activities, respondents objected to turning the river into a commodity [10].Similarly, in a study that set out to elicit WTP to prevent development of an attractive islandbeauty spot, respondents rejected the notion that a monetary value may be placed on the islandstating that it is ‘just there’, ‘beyond value’ or ‘priceless’ [39]. Elsewhere, it is reported that usingmonetary sums to value the environment appears to be a concept that is ‘alien’ to respondents[16].

Allied to these concerns about attaching a monetary value to the environment are issues sur-rounding property rights and who should pay. For example, respondents expressed the view thatpolluters of the river [10] or the environment [11,38] should pay and that the island ‘belongedto everyone’ and that ‘no one had the right’ to own or develop it [39]. Similarly, respondents inother studies expressed the view that the water and petrol companies respectively ought to payfor the environmental protection scheme, rather than the consumer [31,32].

Whilst attaching a value to the environment may appear to be particularly problematic, moraloutrage has also been uncovered in studies that set out to value a reduction in road traffic deaths[12]. Here, respondents expressed distaste for trading lives off against money and rejected thenotion of equating numbers of lives saved with the value of the safety scheme. As with lack oftrust, it seems plausible that moral outrage would be linked to zero responses, but it is not possibleto determine this from the results reported in the papers reviewed here. Again, all studies includedin this section relate to goods that are provided publicly.


3.3.3. Moral satisfactionMoral satisfaction – or warm glow – describes the tendency of respondents to express general

support for a ‘good thing’ or ‘worthy cause’, rather than their valuation of the good in question.Evidence that payments were symbolic or donations to a worthy cause was uncovered in a numberof studies [4,10–12,15–17,32,38]. For example, in a study that set out to value reducing globalwarming via a reduction in green house gases, respondents made reference to more generalenvironmental issues as well as to charitable contributions [38]. In another study valuing theplanting of trees, respondents who had stated the same WTP for two different sized plantingschemes said that they were making a general contribution towards the good cause regardless ofthe number of trees planted [15].

As in the case of mental accounting, the common feature of the studies reporting moral satisfac-tion is the link to insensitivity to scale and scope where tested explicitly (see Table 1). For example,Schkade and Payne [32] reported marked insensitivity to the number of migratory wildfowl savedand moral satisfaction was considered to be a major contributory factor. Moral satisfaction wasalso considered to be a contributory factor towards the marked insensitivity to the magnitude ofthe benefit in studies valuing road safety, both when the benefit was described as a public andprivate good [4,12]. In the same way that ‘saving the planet’ is seen as a good thing, so too itappears that safety may be seen as ‘a good thing’, irrespective of the level of benefit yielded.

3.4. Embedding papers

We have highlighted four common themes that may offer a partial explanation for the pervasiveanomalies that arise in WTP responses. We show that two of the themes – mental accounting andmoral satisfaction – appear to be associated with the presence of embedding, where that hasbeen tested. In three of these papers ‘reflective’ methods were used to explore explicitly thephenomenon of embedding with individual respondents. That is, respondents were ‘confronted’with apparently anomalous responses and asked to discuss the reasoning behind the answers theygave. As these papers were more homogeneous in their objectives than the remainder of those inthe full review, we elected to have a closer look at these papers in order to determine whether itwas possible to draw out differences, as well as similarities, in the qualitative data.2

In the study by Currie and colleagues, respondents were asked about their WTP to bring about2 different expansions in a cancer programme that would treat 300 and 450 additional patientsrespectively [17]. Reasons for being WTP more (or not) for the larger programme were exploredin follow up interviews (see Table 2). Respondents in the road traffic safety study [12] were askedabout their WTP towards 2 safety schemes preventing 5 and 15 deaths respectively and thenasked to discuss the relative amounts contributed to each. The study by Shiell & Gold [35] wassomewhat different in that respondents were asked what they would be WTP for two vaccinesseparately and for a combined vaccine delivering both at once. Respondents were then asked tocompare the sum of their WTP amounts for the two separate vaccines with that for the combinedvaccine in order to explain ‘part-whole bias’, a common manifestation of embedding.

As above, Currie and colleagues identified moral satisfaction (labelled ‘contribution to thecause’ in this study), and mental accounting (labelled ‘budget constraints’ in this study) as twofactors contributing towards insensitivity to the numbers treated by the two different cancerprogrammes. In explaining their rational for stating the same WTP for 2 programmes treating 300

2 It is not practical to detail differences in findings across the full set of papers as much of the qualitative material relatesto the specific good under investigation and would amount to reproducing the entirety of qualitative findings here.


and 450 patients, respondents were also found to be ‘inferring differences in the programmes’.In particular, respondents suggested that the smaller programme must provide a better qualityservice than the larger one. Alternatively, the smaller programme must be ‘inefficient’, so couldachieve the same numbers as the other if resources were used to better effect. These findings seemto suggest that respondents did not perceive that their (collective) contributions would determinethe size of the programme, but were struggling to understand why one programme would besmaller than the other, given a fixed ‘cancer’ budget.

The road traffic safety study [12] reported findings that fitted into each of the four commonthemes, as outlined above. In addition, the qualitative results suggested that respondents haddifficulties conceptualising the impact of the programme in terms of how the lives would besaved, what contribution others would make and who would have died. As in the Currie study,this suggests that respondents were struggling to see the link between their stated WTP and theprovision of the good.

Whilst Shiell and Gold report the presence of mental accounting (see above), the most commonreason for valuing the composite vaccine less than its component parts was expectation about whatwas ‘a fair price to pay’. That is the cost of the combined vaccine ‘ought’ to be lower than the twosold separately because people ‘expect’ a discount for bulk and ‘2-in-1 shampoo is cheaper thanbuying it separately’ p. 256. This suggests that respondents were looking to the ‘real world’ forcues as to what appropriate responses to the various questions ought to be. The next most importantcategory was ‘availability bias’ which the authors describe as the second (more serious) strain ofthe disease looming larger when the composite vaccine was valued with protection against thefirst (less serious)–strain seen as a ‘bonus’. Hence, they were willing to pay more for the extraprotection offered by the first vaccine in combination but not as much as they had said they wouldpay for that vaccine taken alone. Such findings are clearly related to the sequential nature of thespecific task respondents were faced with in that study.

Such detailed analysis indicates that there are likely to be differences as well as similaritiesin qualitative findings, even across papers with broadly similar objectives. That is, there are‘context specific’ considerations that are unlikely to be identified in the search for commonthemes across disparate studies. It seems plausible that certain of the findings (such as the ‘2-in-1shampoo’ argument or assumptions about ‘better quality care’) only came to light due to thedetailed examination of anomalous responses undertaken in these studies. On the other hand, theresults of these studies may be driven by a degree of ‘post hoc rationalisation’ with respondentsseeking to offer ‘rational’ explanations for apparently anomalous answers. Discussion of therelative merits of different qualitative approaches is beyond the scope of this paper, but this wouldseem to be an important consideration in the design of future studies.

4. Discussion

It has been argued elsewhere that WTP studies may be less problematic in certain contexts,such as health, than in an environmental context—where the vast majority of WTP studies havebeen conducted (Smith, 2004). The argument is that certain goods, such as health care, may bemore akin to simple private goods than to the type of complex, public goods that often feature inenvironmental studies. On the face of it, a number of the findings reported above (for example,moral satisfaction and lack of trust) would appear to be intrinsically linked to the provisionof public goods, so may seem of little relevance outside of environmental economics. We do,however, believe the picture is a good deal more complex than a simple private/public dichotomyand that a number of issues raised above are of general relevance to the use of WTP.


First, a number of WTP studies carried out in the areas of health and safety ask respondents toanswer in their capacity as citizens and express their values for programmes that are essentiallypublic goods (Acton, 1976; Olsen and Donaldson, 1998; Olsen et al., 2004). It seems plausiblethat certain of the problems associated with eliciting WTP for public goods will be pertinentto all studies describing ‘the good’ in this way, whether or not the good is truly public in thesense understood by economists (i.e. non-rival and non-excludable). There seems little reason tosuppose that warm glow effects – so pervasive elsewhere – will not affect responses whenever thegood is described in a ‘public’ manner. That is, providing, for example, health care is seen as agood thing, irrespective of the scale or scope of the programmes on offer, a consideration whichmay help explain the embedding found by Olsen et al. (2004).

It has been recommended that WTP studies ought to use the payment vehicle which bestreflects the real world funding mechanism (Arrow et al., 1993; Mitchell and Carson, 1993; Gafni,1991). In the case of health, safety and other publicly provided ‘goods’ in the UK, it followsthat a general taxation payment vehicle ought then to be used as this is how such schemes arepredominantly funded. The results presented here, however, indicate that this may be problematicwhenever there is concern over how additional tax revenues will be spent and/or how revenuesare currently deployed. Of course, using a ‘hypothecated’ or ‘ring-fenced’ tax may alleviatethese problems to some extent, but there would remain a challenge in convincing respondentsthat ring-fencing would work in practice. Hence, whilst taxation may be the appropriate pay-ment vehicle from the policy perspective, it may be inappropriate for the purposes of studydesign.

It also appears that the use of a payment vehicle that involves a contribution to a public fund –whether via taxation or some other mechanism – may leave respondents struggling to understandthe relationship between their stated WTP amount and the provision of the good. Questions suchas, ‘will the rich pay more?, what if others don’t pay?, won’t I have to pay anyway?’ suggest thatrespondents often just do not ‘get’ the way in which the contingent market is meant to operate.This highlights the important point that there is more to conducting a CV study than simplyasking a WTP question: the contingent market has got to be properly set up by the researcherand understood by the respondent (see, for example, Bateman et al., 2002; Smith, 2003). Whilstthis may be relatively straightforward in the case of an out of pocket payment for a private good,it may be a good deal more complex when the payment mechanism is via some public fund. Itseems to us that a number of the findings reported in the studies reviewed here may reflect alack of attention by the researcher to the nature of the contingent market they are expecting therespondent to engage with.

It is difficult to see how the problems associated with mental accounting may be over-come and it seems likely that such anomalies will also affect responses to WTP questionsin health. In stating a WTP that leaves normal expenditure and savings patterns untouched,respondents are avoiding making difficult trade offs between the good being valued and all theother things they value. This phenomenon has been reported elsewhere as a possible explana-tion of observed insensitivity in WTP responses (Smith, 2005). If this is a robust phenomenon,then it is crucial to consider the implications for the design of WTP studies in a number ofcontexts.

The extent to which such anomalies may be ‘designed out’ of WTP studies remains an openquestion. Whilst it is clear that WTP questions must be properly piloted, the contingent marketadequately set up by the researcher and understood by the respondent, it is less obvious thatimproved study design will necessarily be able to overcome problems inherent in WTP studies.Recent studies have advocated giving participants in WTP studies more time to consider their


responses and the use of preliminary focus groups as an opportunity for respondents to deliberateand ask questions [26,27]. Whilst the argument in favour of eliciting more considered responsessounds compelling, certain of the studies included in this review did make use of preliminary focusgroups and still reported marked embedding in WTP responses [4,12]. Although it seems thatgiving respondents time to think and discuss the programmes in more detail will not necessarilyovercome problems such as embedding, the use of more in-depth techniques in eliciting WTPvaluations remains an important area for future research.

Appendix A

References: The 39 papers reviewed (21 relevant papers in bold)

1. Asenso-Okyere, W.K., Osei-Akoto, I., Anum, A., Appiah, E.N. (1997). Willingness to pay forhealth insurance in a developing economy. A pilot study of the informal sector of ghana usingcontingent valuation. Health Policy 42: 223–237.

2. Bardsley D., Edwards-Jones (2006). Stakeholders’ perceptions of the impacts of invasive exoticplant species plant species in the Mediterranean region GeoJournal 65: 199–210.

3. Bateman, I.J., Langford, I.H., Jones, A.P., Kerr, G.N. (2001). Bound and path effects indouble and triple bounded dichotomous choice contingent valuation. Resource and EnergyEconomics 23: 191–213.

4. Beattie, J., Covey, J., Dolan, P., Hopkins, L., Jones-Lee, M., Loomes, G., Pidgeon, N.,Robinson, A., Spencer, A. (1998). On the contingent valuation of safety and the safety ofcontingent valuation. Part I. Caveat Investigator. Journal of Risk and Uncertainty 17: 5–25.

5. Blamey, R. (1998). Contingent valuation and the activation of environmental norms. Eco-logical Economics 24: 47–72.

6. Brown, T. (2005). Loss aversion without the endowment effect, and other explanations for theWTA-WTP disparity, Journal of Economic Behavior & Organization, 57: 367–379.

7. Brouwer, R., Powe, N., Turner, R.K., Bateman, I.J., Langford, I.H. (1999). Public attitudesto contingent valuation and public consultation. Environmental Values 8: 325–347.

8. Burgess, J., Clark, J., Harrison, C.M. (1998). Respondents’ evaluations of a CV survey:a case study based on an economic valuation of the wildlife enhancement scheme, PevenseyLevels in East Sussex. Area 30: 19–27.

9. Burton, A. et al. (2000). The WTP/WTA discrepancy: a preliminary qualitative examination.Environmental Values 9, 4 (November 2000): 481–91.

10. Cameron, J.I. (1997) Applying socio-ecological economics: a case study of contingentvaluation and integrated catchment management. Ecological Economics 23: 155–165.

11. Chilton et al. (2004). Valuation of health benefits associated with reductions in air pol-lution, final report, DEFRA www.defra.gov.uk.

12. Chilton S., Covey J., Hopkins L., Jones-Lee M., Loomes G., Pidgeon N., Robinson A.,Spencer A. Valuing the value of life: a case of constructed preferences? In Todd, Z., Nerlich,B.b, McKeown, S., Clark, D.C. (Eds.). Mixing methods in psychology, London: Routledge,2003.

http://www.defra.gov.uk/


13. Chilton, S.M., Hutchinson, M.G. (1999a). Exploring divergence between respondent andresearcher definitions of the good in contingent valuation studies. Journal of Agricultural Eco-nomics 50: 1–16.

14. Chilton, S.M., Hutchinson, W.G. (1999b). Do focus groups contribute anything to the contin-gent valuation process? Journal of Economic Psychology 20: 465–483.

15. Chilton, S.M., Hutchinson, W.G. (2003). A qualitative examination of how respondentsin a contingent valuation study rationalise their Wtp responses to an increase in the quantityof the environmental good. Journal of Economic Psychology 24: 65–75.

16. Clark, J., Burgess, J., Harrison, C.M. (2000). “I struggled with this money business”:respondents’ perspectives on contingent valuation. Ecological Economics 33: 45–62.

17. Currie G, Donaldson C, Lee H. Using mixed methods to explore the issue of scope sensi-tivity in contingent valuation, Paper Presented to Health Economists’ Study Group (HESG)Meeting, Newcastle, 2005.

18. Davis, J. (2004). Assessing community preferences for development projects: are willingness-to-pay studies robust to mode effects? World Development 32: 655–672.

19. Dong, H.J., Mugisha, F., Gbangou, A., Kouyate, B., Sauerborn, R. (2004). The feasibility ofcommunity-based health insurance in burkina faso. Health Policy 69: 45–53.

20. Hutchinson, W.G., Chilton, S.M., Davis, J. (1995). Measuring nonuse value of environmentalgoods using the contingent valuation method—problems of information and cognition and theapplication of cognitive questionnaire design methods. Journal of Agricultural Economics 46:97–112.

21. Jones-Lee M., Loomes G., Robinson, A. Why did two theoretically equivalent methodsproduce two very different values? In Schwab, Christie and Soguel (Eds.), Contingent Val-uation, Transport Safety and Value of Life. Kluwer: Boston, 1995.

22. Kaplowitz, M (2000), Identifying ecosystem services using multiple methods: lessons fromthe Mangrove Wetlands of Yucatan, Mexico, Agriculture and Human Values v17, n2 (June 2000):169–79.

23. Kaplowitz, M.D., Hoehn, J.P. (2001). Do focus groups and individual interviews reveal thesame information for natural resource valuation? Ecological Economics 36: 237–247.

24. Kassardjian, E, Gamble J, Gunson A, Jaeger S (2005). A new approach to elicit consumers’willingness to purchase genetically modifies apples, British Food Journal 107:, 541–555.

25. Kenyon, W., Nevin, C. (2001). The use of economic and participatory approaches to assessforest development: a case study in the Ettrick Valley. Forest Policy and Economics 3: 69–80.

26. Kontogianni, A., Skourtos, M.S., Langford, I.H., Bateman, I.J., Georgiou, S. (2001). Integrat-ing stakeholder analysis in non-market valuation of environmental assets. Ecological Economics37: 123–138.

27. Langford, IH, Georgiou, S, Day R, Bateman, I.J (1999). Comparing perceptions of riskand quality with willingness to pay: a mixed methodological study of public preferences forreducing health risks from polluted coastal bathing waters, Risk, Decision & Policy, 4: 201–220.


28. Loomis, J., Kent, P., Strange, L., Fausch, K and Covich, A (2000). Measuring the total eco-nomic value of restoring ecosystem services in an impaired river basin: results from a contingentvaluation survey, Ecological Economics 33: 103–117.

29. MacMillan D, Philip L, Hanley N, Alvarez-Farizo B (2002). Valuing the non-market benefitsof wild goose conservation: a comparison of interview and group-based approaches, EcologicalEconomics, 43: 49–59.

30. Philip, L, MacMillan, D (2005). Exploring values, context and perceptions in contingentvaluation studies: The CV market stall technique and willingness to pay for wildlife conser-vation, Journal of Environmental Planning and Management, 48 (2): 257–274.

31. Powe, N. et al. (2004). Putting action into biodiversity planning: assessing prefer-ences towards funding source. Journal of Environmental Planning and Management 47(2):287–301.

32. Schkade, D.A., Payne, J.W. (1994). How people respond to contingent valuationquestions—a verbal protocol analysis of willingness-to-pay for an environmental-regulation.Journal of Environmental Economics and Management 26: 88–109.

33. Scott, A. et al. (2003). Visitor payback: panacea or pandora’s box for conservation in the UK?Journal of Environmental Planning and Management 46: 583–604.

34. Shiell, A & Rush (2003). Can willingness to pay capture the value of altruism? an explo-ration of Sen’s notion of commitment, Journal of Socio-Economics 32, 6 (2003): 647–60.

35. Shiell, A., Gold, L. (2002). Contingent valuation in health care and the persistence ofembedding effects without the warm glow. Journal of Economic Psychology 23: 251–262.

36. Shiell A., Gold L. (2003). If the price is right: vagueness and values clarification in con-tingent valuation Health Economics 12: 909–919.

37. Smith R.D. (2006). The relationship between reliability and size of willingness-to-payvalues: a qualitative insight Health Economics 16: 211–216.

38. Svedsater, H. (2003). Economic valuation of the environment: how citizens make senseof contingent valuation questions. Land Economics 79: 122–135.

39. Vadnjal D., O’Connor (1994). What is the value of Rangitoto Island?, Environmentalvalues, 3: 369–380.

References

Acton, J.P., 1976. Evaluating public programs to save lives: the case of heart attacks. RAND Corporation, Santa Monica,Report No. R950RC.

Arrow, K., Solow, R., Portney, P., Leamer, E., Radner, R., Schuman H, 1993. Report of the NOAA panel on contingentvaluation. Federal Register 59, 1062–1191.

Bateman, I., Carson, R., Day, B., Hanemann, M., Hanley, N., Hett, T., Jones-Lee, M., Loomes, G., Mourato, S., Ozde-meroglu, E., Pearce, D., Sugden, R., Swanson, J., 2002. Economic Valuation with States Preference Techniques: AManual. Department for Transport Publications, Edward Elgar.

Coast, J., McDonald, R., Baker, R., 2004. Issues arising from the use of qualitative methods in health economics. Journalof Health Services Research and Policy 9 (3), 171–176.

Dixons-Woods, M., Agarwal, S., Jones, D., Young, B., Sutton, A., 2005. Synthesising qualitative and quantitative evidence:a review of possible methods. Journal of Health Services Research & Policy 10 (1), 45–53.


Dubourg, W.R., Jones-Lee, M.W., Loomes, G, 1997. Imprecise preferences and survey design in contingent valuation.Economica 64, 681–702.

Gafni, A, 1991. Willingness to pay as a measure of benefits: relevant questions in the context of public decision makingabout health care programmes. Medical Care 29, 1246–1252.

Jones-Lee, M.W., Loomes, G., Philips, P.R., 1995. Valuing the prevention of non-fatal road injuries: contingent valuationvs. standard gambles. Oxford Economic Papers 47, pp. 676–695.

Loomis, J., Lockwood, M., DeLacy, T., 1993. Some empirical evidence on embedding effects in contingent valuation offorest protection. Journal of Environmental Economics and Management 24 (1), 45–55.

Mitchell, R.C., Carson, R., 1993. Current issues in the design, administration and analysis of contingent valuation surveys.Working Paper, Department of Economics, University of California, SanDiego, Cal.

Olsen, J.A., Donaldson, C, 1998. Helicopters, hearts and hips: using willingness to pay to set priorities for public sectorhealth care programmes. Social Science and Medicine 46, 1–12.

Olsen, J.A., Donaldson, C., Pereira, J, 2004. The insensitivity of ‘willingness-to-pay’ to the size of the good: New evidencefrom health care. Journal of Economic Psychology 25, 445–460.

Petticrew, M., Roberts, H., 2006. Systematic Reviews in the Social Sciences: A Practical Guide. Blackwell, Oxford.Smith, R.D., 2003. Construction of the contingent valuation market in health care: a critical assessment. Health Economics

12 (8), 609–628.Smith, R.D., 2004. Contingent valuation in health care: Is there more to it than just valuing health status? Paper Presented

to the Health Economists’ Study Group, Glasgow, July 2004.Smith, R.D., 2005. Sensitivity to scale in contingent valuation: the importance of the budget constraint. Journal of Health

Economics 24, 515–529.

Documents

How do respondents explain WTP responses? A review of the qualitative evidence