18
DELIVERING EFFECTIVE QUALITATIVE RESEARCH FOR POLICY MAKING GIUSEPPE A. VELTRI, INSTITUTE FOR PROSPECTIVE TECHNOLOGICAL STUDIES (IPTS) QUALITATIVE RESEARCH FOR POLICY MAKING 2011, BELFAST 26- 27 OF MAY.

Delivering effective qualitative research for policy making

Embed Size (px)

DESCRIPTION

Presented at the 2nd European Conference on Qualitative Research for Policy Making, 26-27 May 2011, Belfast, UKOrganised by Merlien Institute

Citation preview

Page 1: Delivering effective qualitative research for policy making

DELIVERING EFFECTIVE

QUALITATIVE RESEARCH

FOR POLICY MAKING

GIUSEPPE A. VELTRI, INSTITUTE FOR PROSPECTIVE TECHNOLOGICAL STUDIES (IPTS)

QUALITATIVE RESEARCH FOR POLICY MAKING 2011, BELFAST 26-27 OF MAY.

Page 2: Delivering effective qualitative research for policy making

QUALITATIVE RESEARCH

AND POLICY MAKERS

• Although there is an increasing recognition of the usefulness of qualitative research in policy making environments, the general understanding does not see it as a stand alone research endeavor.

• Based on my experience, qualitative research is rather well received in a context of mixed methodologies, but it is not perceived as funds-worthy if not connected to a quantitative study.

• Oppositional views of qualitative vs. quantitative research are fruitless.

May 27, 2011

Page 3: Delivering effective qualitative research for policy making

POLICY MAKING

ENVIRONMENT

• The policy making environment does a large use of quantitative study to have ‘evidence-based policies’.

• There has been an increase of the use of qualitative study because of the failure of quantitative indicators or to support the adoption of particularly controversial quantitative measurements.

• Policy makers are responsive to the argument of having a more balanced share of quantitative and qualitative studies.

May 27, 2011

Page 4: Delivering effective qualitative research for policy making

QUALITATIVE AND QUANTITATIVE RESEARCH

May 27, 2011

Page 5: Delivering effective qualitative research for policy making

QUALITATIVE RESEARCH

MAIN FEATURES

May 27, 2011

Page 6: Delivering effective qualitative research for policy making

LIMITS OF QUALITATIVE

RESEARCH

(from the perspective of a policy maker, perceived as such)

• Not generalizability (and in general misunderstanding of qualitative research epistemological position)

• Lack of methodological rigor

• Lack of transparency

• Time consuming

• Complexity expansion rather than reduction

• Low rhetorical and argumentative value in conflict of claims

May 27, 2011

Page 7: Delivering effective qualitative research for policy making

QUALITATIVE RESEARCH

AND POLICY MAKING

May 27, 2011

Page 8: Delivering effective qualitative research for policy making

RISKS OF

QUANTIFICATION

• Overconfidence in mathematical models

• This uncritical acceptance amounts to the assumption that reality is identical to our rational reconstruction of reality

• Because they are logically consistent, mathematical models screen out ambiguity

• Ambiguity is real, but quantitative models have little to no room for it. Ambiguity arises whenever there are two (or more) courses of action that are equally important yet conflict with one another

May 27, 2011

Page 9: Delivering effective qualitative research for policy making

RISKS OF

QUANTIFICATION /2

• Putting a situation into numbers enforces the belief that things are linear and events are necessarily comparable..

• ..since for any two numbers one is larger and one is smaller. Suppose that you modeled teaching on a scale that goes from 1 to 10. You would conclude that a teacher with a score of 8.3 was better than one with 6.8, but this would inevitably ignore all kinds of qualitative factors

• Any system that involves human behavior, like economics or finance, is inherently self-referential.

• We are all participants in, not just observers of, these systems. Self-referential systems are notoriously difficult to control and predict because they are often chaotic, not deterministic. Most mathematical models do not take this element of self-reference seriously and anyhow chaotic systems, by definition, cannot be predicted

May 27, 2011

Page 10: Delivering effective qualitative research for policy making

RISKS OF

QUANTIFICATION /3

• "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

• Campbell's Law was published in 1976 by Donald T. Campbell, an experimental social science researcher and the author of many works on research methodology.

• Very relevant for policy.

May 27, 2011

Page 11: Delivering effective qualitative research for policy making

WHY MIXED METHODS?

• As much as they are powerful, quantitative methodologies are not immune from shortcomings and they embed the risks of quantification that are not always fully understood by users of quantitative data.

• Those risks of quantification are very relevant for policy makers

• Triangulation is the methodological principle to adopt and argue in support of.

May 27, 2011

Page 12: Delivering effective qualitative research for policy making

TRIANGULATIONS

• Data triangulation

• Data triangulation involves the use of different sources of data/information

• Investigator triangulation

• It involves using several different investigators/evaluators in an evaluation project.

• Theory triangulation

• It involves the use of multiple professional perspectives to interpret a single set of data/information

May 27, 2011

Page 13: Delivering effective qualitative research for policy making

TRIANGULATIONS /2

• Methodological triangulation

• It involves the use of multiple qualitative and/or quantitative methods to study the issue. If the conclusions from each methods are the same, then validity is established.

• Environmental triangulation

• This type of triangulation involves the use of different locations, settings and other key factors related to the environment in which the study took place such as time of the day, day of the week or season of the year.

May 27, 2011

Page 14: Delivering effective qualitative research for policy making

COSTS

• Qualitative studies are on average cheaper than quantitative studies.

• Policy makers are less concerned about a cost and more about how to justify that cost.

• There is nothing more expensive than ill-designed policy

May 27, 2011

Page 15: Delivering effective qualitative research for policy making

HOT SPOTS FOR

QUALITATIVE RESEARCH

• Qualitative research has two main functions: it can be part of the so called ‘Evidence based policies’; It can be a valuable tool in Evaluation Research

• The focus in qualitative research on exploring meanings and behavior in depth, identifying diverse perspectives, capturing processes and social contexts, and using flexible methods, means that qualitative research is seen as being capable of making a distinctive contribution to policy evaluation.

May 27, 2011

Page 16: Delivering effective qualitative research for policy making

HOT SPOTS FOR QUALITATIVE

RESEARCH /2

• Ex-ante assessment

• Ex-ante assessment involves undertaking an evaluation of the conditions for the launch of a program or institution.

• Cross cultural studies

• In international organizations where the diversity of social, political and institutional contexts is high (e.g. EU)

May 27, 2011

Page 17: Delivering effective qualitative research for policy making

[email protected]

THANK YOU!

May 27, 2011

Page 18: Delivering effective qualitative research for policy making

May 27, 2011

Presented at the 2nd European

conference on Qualitative Research for

Policy Making, 26 -27 May 2011, Belfast

For more information

Please visit: http://www.merlien.org