Quality Issues in Mixed-Methods Research

Alan Bryman,

Management Centre,

University of Leicester, UK.

Quality Criteria in Qualitative Research

• Significant development

• Inappropriateness of traditional criteria

• Two approaches– Adapt traditional criteria– Alternative criteria

What about Mixed-Methods Research?

• No criteria for appraising mixed-methods research exist (Sale and Brazil, 2004)

• How should we approach quality criteria for mixed-methods research?

Three Possible Approaches

1. Convergent criteria

2. Separate criteria

3. Bespoke criteria

• Problem of lack of agreement on qualitative criteria

• Not a focus of concern among interviewees

A Contingency Approach

• Insight from organization studies

• Each approach to quality criteria may be appropriate in some circumstances but not others

Quantitative or Qualitative Dominant (Teddlie, 2005)

Quantitative or Qualitative Dominant

• Quantitative dominant – qualitative research generates hypotheses to be tested or to develop measurement instruments– Traditional criteria

• Qualitative dominant - quantitative research simply for background data or amount of quantitative data is minimal – Alternative, qualitative criteria

• Convergent criteria – use same criteria for both components

Integrated or Separate?

• Different research questions for quantitative and qualitative components?– Separate criteria

• Integration of quantitative and qualitative components– Bespoke criteria

Bespoke Criteria

• Two sets of criteria

1. Tashakkori & Teddlie (2003) – emphasis on inference quality

– Does an inference meets the fundamental criteria required for it to be credible and/or defensible?

– Blends internal validity (traditional) and trustworthiness and credibility (alternative)

Bespoke Criteria

2. Sale & Brazil (2004) Truth value = internal validity + credibility Applicability = external validity/ generalizability +

transferability Consistency = reliability + dependability Neutrality = objectivity + confirmability

• Rooted in criteria for quantitative and qualitative research

• Other criteria specific to mixed-methods?• Not always appropriate or needed anyway

Bespoke Criteria

• Appear to be obvious solution but…– Not always appropriate– Lack of agreed qualitative criteria– Unplanned outcomes of mixed-methods

research

Formulation of bespoke criteria for mixed-methods research very difficult