Upload
merlien-institute
View
1.097
Download
0
Embed Size (px)
DESCRIPTION
Presented at the 2nd European Conference on Qualitative Research for Policy Making, 26-27 May 2011, Belfast, UKOrganised by Merlien Institute
Citation preview
DELIVERING EFFECTIVE
QUALITATIVE RESEARCH
FOR POLICY MAKING
GIUSEPPE A. VELTRI, INSTITUTE FOR PROSPECTIVE TECHNOLOGICAL STUDIES (IPTS)
QUALITATIVE RESEARCH FOR POLICY MAKING 2011, BELFAST 26-27 OF MAY.
QUALITATIVE RESEARCH
AND POLICY MAKERS
• Although there is an increasing recognition of the usefulness of qualitative research in policy making environments, the general understanding does not see it as a stand alone research endeavor.
• Based on my experience, qualitative research is rather well received in a context of mixed methodologies, but it is not perceived as funds-worthy if not connected to a quantitative study.
• Oppositional views of qualitative vs. quantitative research are fruitless.
May 27, 2011
POLICY MAKING
ENVIRONMENT
• The policy making environment does a large use of quantitative study to have ‘evidence-based policies’.
• There has been an increase of the use of qualitative study because of the failure of quantitative indicators or to support the adoption of particularly controversial quantitative measurements.
• Policy makers are responsive to the argument of having a more balanced share of quantitative and qualitative studies.
May 27, 2011
QUALITATIVE AND QUANTITATIVE RESEARCH
May 27, 2011
QUALITATIVE RESEARCH
MAIN FEATURES
May 27, 2011
LIMITS OF QUALITATIVE
RESEARCH
(from the perspective of a policy maker, perceived as such)
• Not generalizability (and in general misunderstanding of qualitative research epistemological position)
• Lack of methodological rigor
• Lack of transparency
• Time consuming
• Complexity expansion rather than reduction
• Low rhetorical and argumentative value in conflict of claims
May 27, 2011
QUALITATIVE RESEARCH
AND POLICY MAKING
May 27, 2011
RISKS OF
QUANTIFICATION
• Overconfidence in mathematical models
• This uncritical acceptance amounts to the assumption that reality is identical to our rational reconstruction of reality
• Because they are logically consistent, mathematical models screen out ambiguity
• Ambiguity is real, but quantitative models have little to no room for it. Ambiguity arises whenever there are two (or more) courses of action that are equally important yet conflict with one another
May 27, 2011
RISKS OF
QUANTIFICATION /2
• Putting a situation into numbers enforces the belief that things are linear and events are necessarily comparable..
• ..since for any two numbers one is larger and one is smaller. Suppose that you modeled teaching on a scale that goes from 1 to 10. You would conclude that a teacher with a score of 8.3 was better than one with 6.8, but this would inevitably ignore all kinds of qualitative factors
• Any system that involves human behavior, like economics or finance, is inherently self-referential.
• We are all participants in, not just observers of, these systems. Self-referential systems are notoriously difficult to control and predict because they are often chaotic, not deterministic. Most mathematical models do not take this element of self-reference seriously and anyhow chaotic systems, by definition, cannot be predicted
May 27, 2011
RISKS OF
QUANTIFICATION /3
• "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
• Campbell's Law was published in 1976 by Donald T. Campbell, an experimental social science researcher and the author of many works on research methodology.
• Very relevant for policy.
May 27, 2011
WHY MIXED METHODS?
• As much as they are powerful, quantitative methodologies are not immune from shortcomings and they embed the risks of quantification that are not always fully understood by users of quantitative data.
• Those risks of quantification are very relevant for policy makers
• Triangulation is the methodological principle to adopt and argue in support of.
May 27, 2011
TRIANGULATIONS
• Data triangulation
• Data triangulation involves the use of different sources of data/information
• Investigator triangulation
• It involves using several different investigators/evaluators in an evaluation project.
• Theory triangulation
• It involves the use of multiple professional perspectives to interpret a single set of data/information
May 27, 2011
TRIANGULATIONS /2
• Methodological triangulation
• It involves the use of multiple qualitative and/or quantitative methods to study the issue. If the conclusions from each methods are the same, then validity is established.
• Environmental triangulation
• This type of triangulation involves the use of different locations, settings and other key factors related to the environment in which the study took place such as time of the day, day of the week or season of the year.
May 27, 2011
COSTS
• Qualitative studies are on average cheaper than quantitative studies.
• Policy makers are less concerned about a cost and more about how to justify that cost.
• There is nothing more expensive than ill-designed policy
May 27, 2011
HOT SPOTS FOR
QUALITATIVE RESEARCH
• Qualitative research has two main functions: it can be part of the so called ‘Evidence based policies’; It can be a valuable tool in Evaluation Research
• The focus in qualitative research on exploring meanings and behavior in depth, identifying diverse perspectives, capturing processes and social contexts, and using flexible methods, means that qualitative research is seen as being capable of making a distinctive contribution to policy evaluation.
May 27, 2011
HOT SPOTS FOR QUALITATIVE
RESEARCH /2
• Ex-ante assessment
• Ex-ante assessment involves undertaking an evaluation of the conditions for the launch of a program or institution.
• Cross cultural studies
• In international organizations where the diversity of social, political and institutional contexts is high (e.g. EU)
May 27, 2011
May 27, 2011
Presented at the 2nd European
conference on Qualitative Research for
Policy Making, 26 -27 May 2011, Belfast
For more information
Please visit: http://www.merlien.org