SLP is a scientific field SLPs are clinical scientists · 12/12/2013 2 What ASHA says…. (ASHA position statement, 2005) “It is the position of the American Speech- Language-Hearing

12/12/2013

1

Evidence-Based Practice:

Definitions and Practical

Application

Patrick Coppens, Ph.D., CCC-SLP SUNY Plattsburgh

GSHA

Atlanta

February 8, 2014

Disclosures

Relevant financial relationship:

I am receiving an honorarium from GSHA for

this presentation

Relevant nonfinancial relationship:

none to disclose

SLP is a scientific field

Pseudoscience is “a body of belief and practices but seldom a field of active enquiry; it is tradition bound and dogmatic rather than forward looking and exploratory”

(Bunge, 1984, p. 41).

Science Pseudoscience

Objective (testable) Subjective (untestable)

True scientific method May sound “scientistic”.

No evidence. Belief-based.

Evolves with knowledge Does not change. Based

on traditions, anecdotes.

Science & Pseudoscience

SLPs are clinical scientists

Gather information about client

Observe and measure behaviors

Apply therapy

Draw clinical conclusions based on measurements

Write up results

EBP provides a strategy to ensure that all clinical decisions are of the highest quality and represent the best possible service to the client

Why EBP?

It’s the ethical thing to do!

Clinical decisions based on sound evidence.

Minimizes intuition and other unsupported claims = “data-driven care.”

Best care for best outcome.

Reduce disparities and variation in care

Recognizes that not all evidence is created equal!

Limits the value of “expert opinion.”

Explicitly includes the client’s values, preferences, etc.

Why EBP?

Everybody wins when EBP is applied! ◦ clinicians are ethical, accountable

◦ clients are well-served

◦ insurance companies get a good service that works for their rehabilitation $

12/12/2013

2

What ASHA says…. (ASHA position statement, 2005)

“It is the position of the American Speech-

Language-Hearing Association that audiologists

and speech-language pathologists incorporate

the principles of evidence-based practice in

clinical decision making to provide high quality

clinical care.”

“In making clinical practice evidence-based,

audiologists and speech-language pathologists—

◦ acquire and maintain the knowledge and skills that are

necessary to provide high quality professional

services, including knowledge and skills related to

evidence-based practice.”

What ASHA says…. (ASHA Code of Ethics, 2010)

Principle of Ethics I – Rule B

“Individuals shall use every resource … to

ensure that high-quality service is provided”

Principle of Ethics II

“Individuals shall honor their responsibility to

achieve and maintain the highest level of

professional competence and performance”

Barriers to EBP use

Reported

Problem

Solutions

Access ?

Time ?

Lack of evidence

or

Insufficient evidence ?

Contradictory

evidence ?

Limited training in

EBP and research. Congratulations!

That’s why you are here!

Lack of information

literacy skills. Congratulations!


Barriers to EBP use:

one caveat….

Reported

Problem Solution?

Time • Not only the responsibility of the SLPs.

• The PARIHS framework (Promoting

Action on Research Implementation in

Health Services) recognizes

“Organizational Culture and Climate” as

partly responsible for the good

implementation of EBP. (Kitson et al., 1998)

• Successful implementation =

Evidence + Context + Facilitation

• Advocacy is the solution here (at the

individual and ASHA levels)

EBP: Skills to hone…

Scientific thinking

◦ Always doubt observed

relationships: a brain is easy to fool!!!

◦ Be a skeptic (including for your own work).

◦ Always think of alternative explanations.

Learn to say “why?”

◦ Some clinicians readily trust information

reported by authority figure or friends.

Armed with your scientific and critical thinking skills, it is

now time to tackle EBP…

http://amzn.to/d1Mzvi

12/12/2013

3

Evidence-Based Practice Evidence-Based Practice

Clinical Decision

Practice Based

Evidence

Best external scientific evidence

Patient preferences and values

http://www.asha.org/Members/ebp/web-tutorial

Dollaghan (2007); Lof (2011)

Topics to be discussed…

EBP components

1. Patient values, preferences, circumstances

2. Best external evidence:

A. Asking the right question

B. Finding the information

C. Evaluating the evidence

i. Strength of rationale

ii. Strength of design

iii. Strength of methods

3. Practice-based evidence


B. Evaluating the evidence


EBP components

1. Patient values, preferences, circumstances

2. Best external evidence: A. Asking the right question






3. Practice-based evidence A. Asking the right question


Clinical Decision

Patient preferences and values

1. Patient Values,

Preferences, Circumstances

We know how to do this: make it functional.

EBP (Dollaghan, 2007):

1. Choice of goals: find agreed upon objectives, but

may require counseling.

2. Choice of approach: all must be based on EBP, but

client preferences and/or

circumstances may tip the

balance.

Possible ethical dilemma:

client requests a discredited

approach.

Do

llag

ha

n (

20

07

).




12/12/2013

4

Patient Values, Preferences,

Circumstances

Conclusions:

What should we do?

1. Listen to the client/family

2. Understand needs but also limitations

(financial, transportation, support, etc.)

3. Develop common goals but without

compromising your prognosis. Counsel if

needed.

4. Use form to compare 2 possible Tx

approaches.

Barriers to EBP use

Reported

Problem

Solutions

Access ?

Time ?

Lack of evidence

or

Insufficient evidence ?

Contradictory

evidence ?

Limited training in

EBP and research. Congratulations!


Lack of information

literacy skills. Congratulations!



EBP components 1. Patient values, preferences, circumstances

2.Best external evidence: A. Asking the right question






3. Practice-based evidence



Clinical Decision

Best external scientific evidence

2. Best External Evidence





The PICO question:

Population/Patient

Intervention

Comparison

Outcome


“Which is the best treatment for aphasia?”

In aphasic adults (P) does Semantic Feature Analysis Tx (I) lead to significantly improved naming (O) as compared to no treatment (C)?

Including all 4 characteristics will: make the information gathered more relevant for

the particular client

facilitate the search process.

Trade-off: level of specificity will increase relevancy but

make literature search more difficult.

12/12/2013

5


E.g. (Gillam & Gillam, 2008):

Which type of intervention, computer based (I),

group pullout (C), or individual (C), provided to

preschool children with speech and language

impairments (P) results in the greatest

improvement on measures of phonemic

awareness (O)?

practice a PICO question:


For example, look at: ASHA EBP compendium :

http://www.asha.org/members/ebp/compendium/ ASHA evidence maps: http://www.ncepmaps.org ANCDS websites:

http://www.ancds.org/index.php/practice-guidelines-9 http://aphasiatx.arizona.edu/

B. Finding the information For example, look at: Public databases: http://scholar.google.com/ http://www.tripdatabase.com/ http://www.speechbite.com/ http://highwire.stanford.edu/ TBI resources: http://www.psycbite.com

Cochrane collaboration: http://www.cochrane.org Contact your local university. Contact the author.


Look at:




Judge the importance

of the results

Importance of critical

and scientific thinking

There are good resources

available

There are forms (or create your own)

Do

llag

ha

n (

20

07).

Le

mo

nce

llo &

Fa

nn

ing

(2

011

).

http://www.asha.org/members/ebp/compendium/

http://www.ncepmaps.org/

http://www.ancds.org/index.php/practice-guidelines-9





http://aphasiatx.arizona.edu/

http://scholar.google.com/

http://www.tripdatabase.com/

http://www.speechbite.com/

http://highwire.stanford.edu/

http://www.psycbite.com/

http://www.cochrane.org/

12/12/2013

6

Gill

am

& G

illam

(2

00

8).



Is the review of the literature

thorough? Have the authors

ignored some important element?

Is there a reasonable research

question based on the lit review?

Is the question clinically relevant for

your purpose?



ASHA levels of evidence

Level Description

Ia Well- designed meta-analysis of >1 randomized

controlled trial

Ib Well-designed randomized controlled study

IIa Well-designed controlled study without

randomization

IIb Well-designed quasi-experimental study

III Well-designed non-experimental studies (e.g.,

correlations, case studies)

IV Expert committee report, consensus conference,

clinical experience of respected authorities



12/12/2013

7



• Essentially 3 broad avenues of

inquiry:

• Statistical issues: do the stats fit the design?

• Internal validity issues: are there alternate

explanations for the observed results?

• External validity issues: are the results

generalizable to other individuals?

Statistical issues

E.g.:

• alpha = 0.05

• Correlation and causation

• Between-subject vs. within-subject

Internal validity threats: E.g.:

An external variable intervenes during the experiment.

Maturation or spontaneous recovery effect. Precision of measurement: validity and reliability of tests and measures,

calibration of instruments. Inter- and intra-rater reliability.

Unequal groups. Floor & ceiling effects.

0

5

10

15

20

25

30

1 2

External validity issues: E.g.:

Is the sample representative? You can only

generalize to the same subjects.

You can’t generalize to other settings.

Multiple Treatment Interference: if there are

multiple steps or sequential treatments, the

generalization can only occur to people who

receive the same sequence of steps.

Examples for practice:

Find the possible confounding variables:

An investigator measures language comprehension in

10 male and 10 female elderly subjects without

dementia in the presence of 4 different levels of

ambient noise.

An investigator asked severe stutterers to have a

conversation with a close friend and a conversation

with a stranger in the clinical setting to investigate

the effect of conversation partner on stuttering

frequency.

Best External Evidence

Conclusions:

What can we do?


Practice PICO.


Get familiar with the websites and databases.

Rely on guidelines, systematic

reviews, meta-analyses.

Use local university contacts.

12/12/2013

8


Conclusions:

What can we do?

C. Evaluating the evidence Use critical thinking.

Use scientific thinking.

Develop an easy-to-use form.

Practice evaluating articles.

Update statistical knowledge, get familiar with internal and external validity threats (e.g., general research method books).

And most importantly, do not do this for all clients at once! (Robey, 2011)


Robey (2011): A medley:

◦ Clinicians “came to EBP as competent and

experienced clinicians (and) were engaged in

ongoing professional-development learning

activities”

◦ “the process of EBP begins with clinicians’ …

choosing a certain aspect of practice for

enhancement”

◦ “target only one clinical decision for

improvement … and then move to another”

◦ “…must enforce realistic limits on their time”

Barriers to EBP use

Reported

Problem

Solutions

Access •Use databases. Use ASHA. •Ask your local university.

Time • Lots of review articles exist, USE THEM.

• Use a simple evaluation form.

• Tackle 1 topic at a time.

Lack of evidence

or

Insufficient evidence

•Ask a different question for your search

• Seek closest possible applicable evidence. • generate your own evidence (see below)

Contradictory

evidence • Which is strongest?

Limited training in EBP

and research.

Congratulations!


Lack of information

literacy skills.

Congratulations!



EBP components 1. Patient values, preferences, circumstances

2. Best external evidence:







3.Practice-based evidence A. Asking the right question


Clinical Decision

Practice Based

Evidence

3. Practice Based Evidence

Complements external evidence: effectiveness (clinical setting) instead of efficacy (controlled environment).

This must be more than subjective experience: “Practice-Based Evidence.”

The same critical and scientific thinking must be applied to clinical work. Controls are still necessary to draw reasonable conclusions.

If there is no evidence, provide it! But you need a supported rationale.


The same PICO principle applies to daily

clinical application:

In a chronic patient with Broca’s aphasia

(P) does Semantic Feature Analysis Tx

(I) lead to significantly improved

naming (O) as compared to traditional

stimulation approach (C)?

In this case, you are attempting to answer the

question yourself

12/12/2013

9

Do

llag

ha

n (

20

07).


The problem is to defend against

confounding variables.

how confident am I that the therapy

caused the observed improvement as opposed to

another competing variable (maturation)?

2 areas to watch:

Measurement

Design

B. Evaluating the evidence:

Measurement

Establish a stable pre-Tx baseline.

Make sure your measurements are valid ◦ define your scoring protocol carefully

◦ use other scorers or multiple scorers (inter-rater

reliability)


Design

The traditional pre/post design (or ABA)

has problems: it is difficult to conclude on

the success of the therapy.


Design


Design What we like to see:

12/12/2013

10


Design B. Evaluating the evidence:

Design


Design


Design


Design B. Evaluating the evidence:

Design

12/12/2013

11


Design Practice Based Evidence

Conclusions: What can we do?

A. Use a client-specific PICO question. B. Apply the same critical and scientific

thinking to your clinical work: ◦ Watch quality of measurements

◦ Watch design set up

◦ Look for confounding variables.

C. Have a supported rationale for trying something new.

D. If there is no evidence in the lit., report yours!

Barriers to EBP use

Reported

Problem

Solutions

Access • Use databases. • Ask your local university.

Time • Lots of EBP articles exist, USE THEM.

• Tackle 1 topic at a time (Robey 2011).

Lack of evidence

or

Insufficient evidence

• Generate your own evidence: Always

have a sound rationale, try it, and report it! • Ask a different question for your search

• Seek the closest possible applicable evidence.

Contradictory

evidence

• Which is strongest? (see Evaluating the Evidence)

Limited training in

EBP and research.

• Congratulations! That’s why you are here.

Lack of information

literacy skills.

• Congratulations! That’s why you are here.

[email protected]

References American Speech-Language-Hearing Association.

(2005). Evidence-based practice in communication disorders [Position Statement]. Available from www.asha.org/policy.

Dollaghan, C. A. (2007). The handbook for evidence-based practice in communication disorders and sciences. Baltimore, MD: Paul Brookes.

Gillam, S., & Gillam, R. (2008). Teaching graduate students to make evidence-based decisions. Topics in Language Disorders, 28(3), 212-228.

Goldacre, B. (2008). Bad science. New York, NY: Faber & Faber.

References Kitson, A., Harvey, G., & McCormack, B. (1998).

Enabling the implementation of evidence based practice: a conceptual framework. Quality in Health Care, 7, 149-158.

Lemoncello, R., & Fanning, J. L. (2011, November). Practice-based evidence. Seminar presented at the ASHA meeting. San Diego, CA.

Lof, G. L. (2011). Science-based practice and the speech-language pathologist. International Journal of Speech-Language Pathology, 13(3), 189-196.

Lum, C. (2002). Scientific thinking in speech and language therapy. Mahwah, NJ: Lawrence Erlbaum.

Robey, R. (2011). Treatment effectiveness and evidence-based practice. In L. L. Lapointe (Ed.), Aphasia and related neurogenic language disorders (pp. 197-210). New York, NY: Thieme.

http://www.asha.org/policy

Documents

SLP is a scientific field SLPs are clinical scientists · 12/12/2013 2 What ASHA says…. (ASHA position statement, 2005) “It is the position of the American Speech- Language-Hearing