Click here to load reader

Survey Measurement of Preference Parameters Presentation by Miles Kimball Osaka University, 2004

  • View

  • Download

Embed Size (px)

Text of Survey Measurement of Preference Parameters Presentation by Miles Kimball Osaka University, 2004

  • Survey Measurement of Preference ParametersPresentation by Miles KimballOsaka University, 2004

  • OutlineWhat We Want to KnowA New Approach: Survey Measurement of Preference ParametersChallenges in the Survey Measurement of Preference ParametersWhere to Go from Here

  • What We Want to Know: A Partial Wish-List We All Have in CommonTime PreferenceElasticity of Intertemporal SubstitutionAltruism/EnvyLabor Supply Elasticities Risk AversionPrudenceTemperance

    Flow Utility Overall Present Discounted Value of UtilityMarginal Propensity to ConsumeShadow Interest RateShadow WageRisk Aversion of Value Function

  • The Need for a New ApproachDespite decades of research using data on actual economic choices, there is still no consensus on the values of many basic preference parameters.In the absence of persuasive identification of the values of preference parameters, convenient calibrations continue to reassert themselves. log utilityadditive separability zero or infinite labor supply elasticity

  • Advantages of Hypothetical ExperimentsExogenousLarge enough to overcome frictionsSimplified situations highlight different aspects of preferencesClose connection to economic theoryClose connection to economic intuition

  • The Disadvantage of Hypothetical ExperimentsMany economists have doubts about whether people can really answer these questions meaningfully.These doubts deserve to be taken seriously, without letting them paralyze the research program.This requires dealing seriously with the cognitive limitation of respondents and measurement error.

  • Challenges in the Survey Measurement of Preference Parameters

    Cognitive limitations of respondentsMeasurement errorLinking to economic theorySurvey Mechanics

  • Evidence that Survey Respondents Have Important Cognitive Limitations

    Question order and the precise wording of questions often matter.First response on paper and pencil questions, last response in phone interviewsYes-man effectStatus quo biasAnchoring (effects of the response categories)Overweighting of recent eventsGuessing what the interviewer would think was reasonable

  • Strategies for Dealing with Respondents Cognitive LimitationsGet many peoples reactions to a question, particularly the reactions of professional survey methodologists.Make a vignette (story) for the hypothetical situation that is as concrete and believable as possible given the objective.Break the question down into pieces that are each individually easy to understandeven at the cost of complicating the later analysis of the question.When the question still seems too hard, go back to the drawing board and design a new question to get at the desired concept.

  • A SloganTo maximize the reliability of survey answers to difficult questions, substitute researcher effort for respondent effort wherever possible.

  • How to Gauge How Seriously Cognitive Limitations Might Be Affecting Responses

    Pretest.Look at nonresponse rates.Debrief respondents. (Why did you answer the way you did on that question?)Try different variants of the same basic question.Vary the wordingVary the orderVary cutpoints and the unfolding sequence

  • Measurement ErrorEven after doing everything possible to make the questions as easy as possible for respondents to understand and answer, there will still be measurement error. Fortunately, econometric techniques can deal with this measurement error.The appropriate econometric techniques are different from those we are used to because survey measurement of preference parameters often yields more information about the characteristics of the measurement error than other econometric applications.

  • Linking to Economic TheoryEconomic theory gives precise cardinal definitions of elasticities and other parameters that make questions about preference parameters quantitative. This distinguishes economics from psychology, which typically has only ordinal concepts. Enjoying this strength of economic theory requires designing hypothetical situations to match the economic theory as closely as possible.

  • Survey MechanicsGetting a representative sample.Getting a high response rate in order to avoid serious biases.Taking into account the advantages and disadvantages of different modes.In person (highest quality, highest cost) Mail (low response rates a problem, at least in the U.S., and no interviewer feedback on how respondents are handling questions) Phone (often the interior optimum)Internet (emerging method with great potential)

  • Case Study #1: Risk AversionQuestion Design: Risk over permanent income to get at the underlying utility function.Discrete choices to reduce the cognitive burden.Rewritten to avoid status quo bias.Vignette to motivate the hypothetical situation.

  • Case Study #1: Risk Aversion(continued)ImplementationAsked in multiple waves to get a measure of measurement error. Imputation of a cardinal value to each category taking this into account. Econometric adjustment to OLS that takes into account the measurement error and its.Starting point and order of unfolding varied.Small risk questions. Variant questions for retired respondents.

  • Case Study #2: Labor SupplyQuestion Design: We use the theoretical relationship relationship between income and substitution effects.Income effects are the easiest for people to think about.Winning the lottery is an obvious vignette, since people really think about that possibility.The lottery is large and an easily understood size.The decision is broken down into quit or not, then if not quit, reduce hours or not and by how much. The question sequence is designed to match the family structure.

  • Case Study #2: Labor Supply(continued)ImplementationAsked in multiple waves of the HRS to allow us to get at measurement error. (This aspect is still unanalyzed.) We make an analytical and econometric adjustment for the special status of quitting.We have put some variant questions on the Survey of Consumers, but due to lack of funding for survey time, only a fraction of everything that should be looked at.

  • Case Study #3: Time Preference and the EISQuestion Design: Choice over consumption profiles to get at preferences. (In contrast to choices over money that get at the individuals shadow interest rate.)Limitation to two time periods and discrete choices to reduce the cognitive burden. Elicitation of second choices to gain more total information. Bar graphs to make the numbers more vivid. The preamble is a compromise, trying to control for inflation and health care costs but not try the respondents patience.

  • Case Study #3: Time Preference and the EIS(continued)

    Initial ImplementationInitial implementation on the Wave I of the HRS had an inconsistency between the consumption growth rates and the ABCDE labels. Also there was not enough resolution. In person interviews allowed use of graphs.

  • Case Study #3: Time Preference and the EIS(continued)

    Revised ImplementationThe HRS has a mailout that has a reasonable response rate because of the preexisting relationship. We implemented a revised version of the question with more resolution and a systematic relationship between ABCDE labels and consumption growth rates. In the ongoing analysis, we have statistically modeled status quo bias (in this case the tendency to continue answering the same letter, regardless of true underlying preferences) and measurement error. The procedure for dealing with the measurement error also easily handles the discrete choice aspect.

  • Case Study #3: Time Preference and the EIS(continued)

    Internet redesign and implementation Bob Willis has headed a project experimenting with internet interviewing. This will allow us to test variants of the time preference/intertemporal substitution question, including a variant with continuous choice over consumption bars using a mouse.This is still in the design stage. Technical hurdles remain.

  • Where to Go From HereIssues in the Choice of a StrategyOverall Strategy RecommendationHigh Priority Topic Areas

  • Issues in the Choice of a StrategyKey Question #1: How much time and effort should be spent nailing down each preference parameter? (The tradeoff between thoroughness and getting many things done all at once.)Key Question #2: What is the best mode?Key Question #3: What is the best way to tap into existing wisdom and resources for survey measurement?

  • Overall Strategy Recommendation #1: Put Thoroughness FirstOur discipline of Economics is skeptical enough of survey measures of preference parameters that it is important to be thorough and do the best job possible in measuring each preference parameter to have an impact. Getting many things done all at once is of no avail if the answers are not seen as credible. Even a booster of survey measures of preference parameters should only be fully persuaded of the results after doing the most careful design, implementation and analysis possible. In the future these measures might guide the policy for nations. Since surveys in many nations can use the measures, it is worth paying a high fixed cost at the design stage.

  • Overall Strategy Recommendation #2: Phone Survey Initially + Eventually an Internet Survey.

    Getting a representative sample is quite valuable given the interest in the population average values of preference parameters. This can be achieved well be either phone or in-person surveys, but phone surveys are cheaper and more practical.Internet surveys can do everything mail surveys can do and more and are more flexible for experimental work. This is an emerging mode. Efforts are being made to figure out ho

Search related