BRIDGING THE RESEARCH-PRACTICE GAP THOUGH EVIDENCE-BASED MANAGEMENT AND SYSTEMATIC REVIEW

Friday, Aug 1 2014 12:30PM - 2:30PM at Pennsylvania Convention Center

Room 202 B (Program Session #: 109 | Submission: 15526)

Prof. Rob Briner and Prof. David Denyer

@daviddenyer | @Rob_Briner | @CenterforEBMgt | #EBMgt

Welcome!

1. A brief introduction to evidence-based practice (EBM) and the role of systematic reviews (SR) in other fields

2. The nature, purposes and processes involved in evidence-based management (EBMgt) and SR– Plenary discussion: what questions do you have about EBP, EBMgt

and SR?

3. The language of evidence-based management– Table discussions

4. Teaching EBMgt and developing SR skills– Table discussions

5. Academic and practitioner collaboration though EBMgt and SR – Table discussion

6. Resources available to develop EBMgt and SR

1. A brief introduction to evidence-

based practice (EBM) and the role of systematic reviews (SR) in other

fields

Evidence-based practiceand SRs not unusual inother fields

• All practitioners use some of types of evidence in their practice• Main problem is not using enough good quality evidence nor critically

appraising it for its trustworthiness• Started in medicine with BMJ editorial (1991) stated that: ‘‘only about

15% of medical interventions are supported by solid scientific evidence’’• Spread to many other areas of practice

What is evidence-based practice?

• “Evidence-based management is about making decisions through the conscientious, explicit, and judicious use of four sources of information: practitioner expertise and judgment, evidence from the local context, a critical evaluation of the best available research evidence, and the perspectives of those people who might be affected by the decision.” (Briner et al., 2009, p. 19)

What is evidence-based practice?

Traditional literature reviews  

• Do not usually focus on a specific or practice-relevant question• Have diverse aims and purposes• Adopt a wide variety of approaches and structure• Do not use particular methods or do not explicitly state the methods

used to conduct the review• Are more prone to bias because, for example, authors can select

studies that support their views and ignore those that do not• Are less likely to identify the best available evidence• Do not aim to be comprehensive by including all relevant and available

research• Are not replicable nor easy to update

Systematic literature reviews  

• Always focus on specific and usually practice-relevant questions• Have similar aims and purposes in that they all focus on answering specific

questions• Adopt similar approaches and a structure based on a set of broad principles• Use particular and explicitly stated methods in order to:

– Search for and identify relevant literature– Make decisions about what research to include or exclude– Judge the quality and relevance of the research– Integrate or synthesize findings

• Are less prone to bias, because, for example, decisions about what to include and exclude are made explicitly

• Are more likely to identify the best available evidence• Aim to be comprehensive by including all relevant and available research within

explicitly-stated boundaries or constraints• Are replicable and easy to update

Systematic literature reviews: Medicine

• Cochrane Collaboration founded in 1993 it aims to help people make well-informed decisions by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of interventions in all areas of health care

• Cochrane Database of Systematic Reviews– 1995 36 reviews– 1999 500 reviews– 2001 1000 reviews – 2004 2000 reviews + 1400 published protocols (plans)– 2012 5000+ reviews

• Reviews prepared by healthcare professionals who volunteer (10000 people worldwide)

• Cochrane Review Groups• Application and debate about quality standards

Systematic literature reviews: Social Policy

• Campbell Collaboration founded in 2000 • An independent, non-profit, and largely voluntary collaboration • Mission is help people make well-informed decisions by preparing,

maintaining and disseminating systematic reviews in education, crime and justice, and social welfare.

2. The nature, purposes and processes

involved in evidence-based management (EBMgt) and SR

The steps in Evidence-Based Practice  

1. Asking: Translating a practical issue or problem into an answerable question

2. Acquiring: Systematically searching for and retrieving the evidence

3. Appraising: Critically judging the trustworthiness and relevance of the evidence

4. Aggregating: Weighing and and pulling together the evidence

5. Applying: Incorporating the evidence in the decision-making process

6. Assessing: Evaluating the outcome of the decision taken

The mains steps in conducting a Systematic Review 

1. Identify and clearly define the question the review will address.

2. Determine the types of studies and data that will answer the question.

3. Search the literature to locate relevant studies.

4. Sift through all the retrieved studies in order to identify those that meet the inclusion criteria (and need to be examined further) and those that do not and should be excluded.

5. Extract the relevant data or information from the studies.

6. Critically appraise the studies by assessing the study quality determined in relation to the review question.

7. Synthesize the findings from the studies.

8. Consider potential effects of publications or other biases.

Plenary discussion

• What questions do you have about EBP, EMBgt and SR?

6. Resources available to develop

EBMgt and SR

Plenary discussion

• What questions do you have about EBP, EMBgt and SR?

3. The language of evidence-based

management

Table discussions

• What does 'evidence' mean? • What different connotations does it have? • What does 'evidence-based' mean?

Misinterpretations and barriers

Misinterpretations:• EBMgt is done by scholars not practitioners• EBMgt is a single rigid formulaic method• EBMgt is only concerned with effect size, randomised-

controlled trials and meta-analysis etc.

Barriers• Preoccupation with ‘new knowledge’ and ‘new

questions’• Evidence aware/informed or based?• Integration with experience and other forms of evidence

4. Teaching EBMgt and developing SR

skills

Table discussions

• Why are SRs useful and important - academically and practically?

• How do (or could) you teach students and others to do this?

• Share examples of curricula, assignments, etc.

How EBMgt and SR skills can be taught to different groups  

• Cranfield DBA• Co-producing SRs

Engagement

Research

Dissemination

Exploitation

Evaluation

From

Engagement

Integration?Integration?

How EBMgt and SR skills can be taught to different groups  

• Cranfield DBA, executive education• Co-producing SRs

Engagement

To

Dissemination

Exploitation

Research

Evaluation

Integration

5. Academic and practitioner

collaboration though EBMgt and SR

Table discussions

• How can EBMgt be done in practice? How common is it?

• What are the barriers to managers becoming more evidence-based?

• How can academics work with organizations and managers to produce systematic reviews that address practical questions?

HOW organisations ARE commissioning systematic reviews from academics 

• AWA – Advanced Workplace Associates• Network for Business Sustainability

6. Resources available to develop

EBMgt and SR

CEBMa Website

Other websites and publications

BRIDGING THE RESEARCH-PRACTICE GAP THOUGH EVIDENCE-BASED MANAGEMENT AND SYSTEMATIC REVIEW

@daviddenyer | @Rob_Briner | @CenterforEBMgt | #EBMgt

Extra slides

Some people care about research impact

– Hambrick - What if the Academy actually mattered?– Huff - Mode1/2 knowledge production– Van de Ven - Engaged scholarship– Bartunek - Collaborative research– Pearce - What do we know and how do we really know it?– Rousseau - Evidence-based Management

42

The situation

The research conducted by most business school academics does not have a marked influence on the practice of management

(Currie, Knights and Starkey, 2010)

The current evidence-base

In general, managers prefer to update their knowledge and skills by reading popular management books, typically written by consultants, former executives or journalists

(Pfeffer and Fong, 2002; Rynes, Bartunek and Daft, 2001)

Evidence-based management: integrate different forms of knowledge

“Evidence-based management is about making decisions through the conscientious, explicit, and judicious use of four sources of

information: practitioner expertise and judgment, evidence from the local context, a critical evaluation

of the best available research evidence, and the perspectives of those people who might be

affected by the decision.”

(see Briner, Denyer, Rousseau, 2009)

Hierarchy of evidence?