Critical Thinking for Helping Professionals: A Skills-Based

Workbook. Fourth Edition. 2017

Instructor’s Manual

EILEEN GAMBRILL

LEONARD GIBBS

OXFORD

UNIVERSITY PRESS

2

CONTENTS

PART 1: Critical Thinking as a Guide to Decision Making

Exercise 1 Making Decisions About Intervention

Exercise 2 Reviewing Your Beliefs About Knowledge

Exercise 3 Controversy: Invaluable for Problem Solving and Learning

Exercise 4 Critical Thinking and Advocacy

PART 2: Recognizing Propaganda: the Importance of Questioning Claims

Exercise 5 Critically Appraising Human Services Advertisements

Exercise 6 Does Scaring Youth Help Them “Go Straight”?

Exercise 7 Detecting Misleading Problem Framing

Exercise 8 Following the Money

Exercise 9 The Language of Propaganda

PART 3: Increasing Your Skills in Avoiding Fallacies, Biases, and Pitfalls in Decision

Making

Exercise 10 Using the Professional Thinking Form

Instructions for Reasoning-in-Practice Games

Exercise 11 Reasoning-in-Practice Game A

Exercise 12 Reasoning-in-Practice Game B: Group and Interpersonal Dynamics

Exercise 13 Reasoning-in-Practice Game C: More Biases

Exercise 14 Preparing a Fallacies Film Festival

Exercise 15 Fallacy Spotting in Professional Contexts

Exercise 16 Avoiding Group Think

PART 4: Evidence Informed Decision Making

Exercise 17 Applying the Steps in the Process in Evidence-Based Practice

Exercise 18 Working in Interdisciplinary Evidence-Based Teams

Exercise 19 Preparing Critically Appraised Topics`

Exercise 20 Involving Clients as Informed Participants

Exercise 21 Asking Hard Questions: Enhancing Assertive Skills

Exercise 22 Evaluating Service Outcomes

Exercises 23.1 and 23.2 Reviewing Your Expertise

PART 5: Critically Appraising Research

Exercise 24 Evaluating Effectiveness Studies: How Good is the Evidence?

Exercise 25.1 Critically Appraising Reviews

Exercise 25.2 Critically Appraising Practice Guidelines

Exercise 26 Critically Appraising Self-Report Measures

Exercise 27.1 Estimating risk and Making Predictions

3

Exercise 27.2 Critically Appraising a Risk Measure

Exercise 28 Evaluating Diagnostic Tests

Exercise 29 Evaluating Research Regarding Causes

PART 6: Reviewing Decisions

Exercise 30 Critically Appraising Arguments

Exercise 31 Critical Thinking as a Guide to Making Ethical Decisions

Exercise 32 Reviewing Intervention Plans

PART 7: Improving Educational and Practice Environments

Exercise 33 Encouraging a Culture of Thoughtfulness

Exercise 34 Evaluating the Teaching of Critical Thinking Skills

Exercise 35 Forming a Journal Club

Exercise 36 Encouraging Continued Self Development Regarding the Process of

Evidence-Informed Practice and Policy

Exercise 37 Increasing Self Awareness of Personal Obstacles to Critical Thinking

4

PREFACE

Critical Thinking for the Helping Professionals has a single purpose: To improve the

reasoning of practitioners as they carry out their life-affecting work. Though developed

primarily for use in research methods and practice courses, the exercises are intended to sharpen

day-to-day reasoning among practitioners in the helping professions through exercises whose

principles apply across the curriculum. We suggest that you get an overview of the exercises

then decide how each may serve you best. Many exercises are designed to help students spot and

counter faulty reasoning that may result in questionable decisions. This Instructor’s Manual

contains descriptions of each exercise including a brief Overview, its Purpose (learning

objectives), Materials Required, Time Required, Suggestions for Using the Exercise, and

Possible Answers to Follow-Up Questions. Exercises in the Workbook rely on examples from

social work, education, psychology, counseling, nursing, and medicine. Consequently, this

Workbook should be useful to a variety of disciplines and professions. The exercises

demonstrate principles universal to all helping professions. They are designed so that students

can remove selected portions from the Workbook to hand in as assignments for you to review.

Room is provided on material to be completed by students and handed in for the student’s name,

date, and instructor. We welcome your suggestions for improvement and descriptions of your

experience with the exercises. We are sure that many can be improved.

A workbook format provides practical hands-on experience designed to help students to

transfer principles of critical thinking skills to practice. For example, students may learn how to

avoid common fallacies in reasoning such as anchoring and insufficient adjustment. If, after

entering practice, they avoid making snap judgments in the initial minutes of their contact with

clients, then we might say that content about anchoring and insufficient adjustment transferred to

real-life settings. Transfer does not happen automatically. Deliberate practice is needed to

develop and maintain skills (Rousmaniere, Goodyear, Miller, & Wampold, 2017). Active

participation is needed in a “culture of thoughtfulness” (Perkins, 1992; Lipman, 1991). (See also

Brudvig et al, 2013; Horsley et al., 2011; Tiruneh et al, 2014.)

Arrange many opportunities to apply critical thinking to diverse situations. The more

chances that students get to apply new knowledge, the better.

Make learning situations as similar as possible to real-life ones.

Use varied examples that represent different situations in which knowledge and skills are

of value.

Make learning situations vivid and realistic. Games, simulation exercises, and laboratory

work can enhance participation and transfer.

Teach transfer skills. Remind students of the forest (other situations outside of class)

while teaching about trees (specific lessons). Emphasize general principles and arrange

many opportunities to practice them. Teach students “metacognitive skills that enhance

transfer such as asking “Can I use _____ here?”

Expect a tension between demands that you “cover the material” and your efforts to teach

critical thinking skills. Because teaching critical thinking implies learning at the evaluation,

synthesis, and analysis levels, expect students who have been learning at the recognize, recall,

and translate levels to express their uncertainty and discomfort, especially when it comes to

5

evaluating their critical thinking skills. Be prepared to feel a bit lonely and misunderstood at

times. Critical inquiry involves asking questions others may avoid asking such as “Does it

work? How do you know?” If you teach students to evaluate arguments, to look for evidence

before tossing off a conclusion, to avoid being taken in by emotional appeals, to spot common

fallacies in professional reasoning, and to locate evidence for themselves; then you and your

students may feel out of step in educational settings in which unexamined pronouncements may

be the norm.

Although recent years have witnessed increasing revelations of the prevalence of inflated

claims in the professional literature, including peer reviewed publications (e. g., Ioannidis, 2016),

combatting such claims is an ongoing challenge. Let’s say you and your students have just

completed Exercise 5, which demonstrates how a propagandistic human service advertisement is

the polar opposite of critical appraisal of claims and arguments. Your students may encounter an

instructor who shows the same advertisement to support a claim that a program works. Consider

another example. Students often ask if they can miss a class to go to professional conferences.

These conferences may concern a new treatment method. You may say, “… Of course, but will

you ask about the effectiveness of the method at some time during the conference? In years of

making this request, only one student had the courage to ask; our students have thought it

impertinent to ask about effectiveness.

Focus on the purpose of critical thinking: to help clients and to avoid doing harm.

Keeping these purposes in mind should help you and your students to have the courage to ask

questions about claims, to devote the time and care necessary to critique claims, to learn how to

spot and avoid faulty reasoning, to examine assumptions, analyze arguments, and listen carefully

to the ideas of others.

References

Brudvig, T. J. Dirkes, A., Dutta, P., & Rane, K. (2013). Critical thinking skills in health care

professional students: A systematic review. Journal of Physical Therapy Education,

27, 12-26

Horsley, T., Hyde, C., Santesso, N., Parkes, J., Milne, R., & Stewart, R. (2011). Teaching

critical appraisal skills in healthcare settings. Cochrane Database of Systematic

Reviews 2011, Issue 11.

Rousmaniere, T., Goodyear, R. K., Miller, S. D., & Wampold, B. E. (Eds.) (2017). The cycle

of excellence: Using deliberate practice to improve supervision and training.

Hoboken, NJ: John Wiley & Sons.

Tiruneh, D. T., Verburgh, A., & Elen, J. (2014). Effectiveness of critical thinking instruction

in health education: A systematic review of intervention studies. Higher Education

Studies, 4, 1-17.

`

1

__________________________________________________________________________

Exercise 1: MAKING DECISIONS ABOUT INTERVENTION

________________________________________________

OVERVIEW

Respondents compare criteria they would like their doctor to rely on when making

recommendations with criteria they themselves rely on when selecting assessment and

intervention methods in their own practice. This exercise encourages readers to be explicit about

the criteria they value and to reflect on differences that may emerge in criteria used in different

situations. A lack of generalization of knowledge and skills from one area to another is a key

obstacle in encouraging critical thinking. Data gathered from a sample of social workers can be

seen in Gambrill and Gibbs (2002).

PURPOSE

1. To review criteria for making practice decisions.

2. To compare criteria used in a personal decision and criteria used in one's own practice. We

hope that this exercise will help you to demonstrate to students that professionals are often

inconsistent in the evidentiary standards they use. That is, when it concerns their own

interest (compared with the interests of clients), they may rely on more rigorous criteria.

MATERIAL REQUIRED

1. Making Decisions About Intervention questionnaire (Practice Exercise 1).

2. Scoring Instructions.

TIME REQUIRED

Five to ten minutes for the questionnaire and 30 minutes for discussion.

SUGGESTIONS

This exercise has had the most impact when respondents were given no background explanation

and were just asked to complete the questionnaires. First, hand out Part 1 of Exercise 1 and ask

students to note their name on the form and complete it. Collect these forms. Then hand out Parts

2 and 3 (regarding their clients) and ask them to put their names on them and complete the

questionnaire. We have found that when students are asked what criteria they would like their

dentist or doctor to use in making a decision that affects them personally (or a family member or

friend), they opt for criteria based on scientific data. When making decisions about their own

clients they often appeal to intuition and anecdotal experience (Gambrill & Gibbs, 2002).

Respondents usually score highest on Situation #1, indicating a preference for scientific criteria

2

in personal situations. There are many reasons why different criteria may be appealed to (e.g.,

there may be no controlled experimental studies). Helpers may not realize they use different

criteria when making personal and professional decisions.

POSSIBLE ANSWERS TO FOLLOW-UP QUESTION

If your scores differs in the three situations, why do you think this is so? Respondents have had

three responses to this question:

"I didn't think of the ethical implications of demanding higher standards for evidence for

myself than for my own clients! "

We suggest a hearty, "You've gotten the point of the exercise." Review problems with

basing beliefs on questionable criteria such as testimonials (item 7), case examples (item

3), what fits personal style (item 5), or what is usually offered (item 6). This exercise

provides an opportunity to discuss the difference between analytic and intuitive thinking

described in Part I of the Workbook. Clients prefer counselors who base decisions on past

track records of personal experience and research (O'Donohue, et al, 1989). This exercise

can also be used to discuss criteria relied on to select assessment and evaluation methods.

It can be combined with a discussion of related ethical concerns.

"I don't have time to search the literature."

Possible reply: "Computer technology and seeking high-quality reviews makes it easier to

discover practice-related research." (See for example, Cochrane and Campbell

databases.)

"The same quality of evidence does not exist for problems in social services as in

medicine."

Possible reply: "This may be so, but we are still ethically responsible to know what

evidence does exist and to evaluate progress with each client. Research in the helping

professions shows that we may harm as well as help clients .

Scoring Instructions

Score answers in each situation on the Making Decisions About Intervention questionnaire

(Exhibit 1.1) using the following key (scores may range from -7 to + 10):

For each of items 1, 2, 3, 5, 6, 7, 9 _________

marked, subtract 1 point

For item 4 marked, give 3 points _________

For item 8 marked, give 5 points _________

For item 10 marked, give 2 points _________

3

Add these values, being sure to subtract __________

the negative values, for a total

1

_____________________________________________

Exercise 2:REVIEWING YOUR BELIEFS ABOUT KNOWLEDGE

__________________________________________________________________

OVERVIEW

This exercise offers readers an opportunity to review their beliefs about what knowledge is and

how it can be obtained. Readers review questions about knowledge and learning to identify

beliefs that may interfere with helping clients. Some questions are from W. Gray (1991)

Thinking Critically About New Age Ideas. Our beliefs about knowledge influence the criteria we

use to review arguments. Consider debates about different ways of knowing ("What is

evidence?). Our beliefs about knowledge influence the criteria we use to make decisions. It does

little good to encourage students to consider research related to life-affecting decisions if they

don't believe any knowledge is available and are not aware of informed consent obligations or do

not value these. Some questions such as “Is there a God?", cannot be falsified -- we can't find

out if they are accurate or not. Others such as: "Is cognitive behavioral treatment of depression

more effective than medication in decreasing depression?", can be investigated.

PURPOSE

To critically review beliefs about knowledge (what it is and how to get it) so that available

knowledge will be considered in helping clients and avoiding harm. Additional items have been

included in this new version to highlight the need for skepticism in reviewing all material,

including the peer-reviewed literature (see items 3, 6, 9). NOTE: missing text in No. 9 is “found

to be accurate.”

MATERIALS REQUIRED

1. Reviewing Your Beliefs About Knowledge questionnaire (Practice Exercise 2).

2. Suggested Answers and Reasons for Beliefs About Knowledge.

TIME REQUIRED

Five to ten minutes for the questionnaire and 30-60 minutes for discussion.

SUGGESTIONS

This exercise addresses beliefs related to critical inquiry. Its importance can be highlighted by

emphasizing that our beliefs influence our actions, both in our work and personal lives. It might

be useful to tie Exercises 1 and 2 together because if students don't believe there is any

knowledge to draw on in social work for example but want a physician to base decisions on

experimental evidence when they have a medical problem of their own, then we have a

contradiction. Structural theorists such as Mullaly (1993) emphasize the importance of helping

2

students to identify contradictions between their beliefs and their actions (e.g., abhorring

oppression but valuing a capitalistic economic system that encourages it). This exercise also

provides an opportunity for you to review your beliefs about knowledge. And it provides an

opportunity to consider the role of ignorance as well as knowledge, including its strategic use.

Suggested replies when discussing answers:*

1. Disagree. It is true that our beliefs influence what we see. The methods of science are

designed to ferret out bias. For example, people who disagree about putting fluoride in

water can search for results of laboratory tests and experiments regarding fluoride and tooth

decay.

2. Agree. It is true that there are certain kinds of questions that we may never be able to

answer (e.g., Is there life after death?). However, there are many things that we can find out

about. If we develop the means, the tools, the measures, we can often discover what we do

not now know. It was inconceivable at one time that a surgeon could ever transplant a

human heart; now we do so regularly. At one time we knew nothing about the structure of

the moon; now we have gone there and returned with samples of its structure. The inconceivable

often becomes commonplace.

3. Disagree: Hiding limitations is common (see Ioannides, 2005, 2016). This emphasizes the

importance of critically appraising research for yourself.

4. Disagree. Everything is not possible. By definition, a circle is not a square. It is not

possible to walk through walls or to levitate. Some events are much more likely than

others. Skepticism helps us to determine which events are most likely.

5. Disagree: Ignorance and knowledge are not mirror images. They may for example, be

used for different purposes in different ways.

6. Disagree: False claims are common, even in peer-reviewed publications (e. g., see

retractionwatch.com).

7. Disagree. It is true that there is much we do not know about behavior. However, by testing

our assumptions about behavior we have discovered much about behavior as illustrated in

books describing the principles of behavior. We are beginning to understand biochemical

influences that affect synapses in human nerve cells. Social norms make some behavior

predictable.

8. Disagree: Both knowledge and ignorance can be strategically used to influence people.

For an example of the latter, see Oreskes and Conway (2010).

9. Disagree: Much is of questionable accuracy, so it is important to review it carefully.

10. Agree. Not all claims can be investigated through scientific methods (e.g., Does God exist?

Is it evil to assault people?). However, many claims are testable scientifically. Such claims

should be put in a testable form. Vague questions cannot be answered.

3

11. Agree: For example this permits informed consent and suggests research directions. Can you think

of an example from your own life?

12. Disagree. Neither helpers nor their clients can know whether services are useful unless they

evaluate progress. Consider a doctor who is treating a patient for diabetes and wants to test the

patient's blood sugar. How can the patient or the doctor know if the treatment is working without

data? The same goes for depression and other problems. Informed consent requires helping clients

determine if services have resulted in outcomes clients value.

13. Disagree: Such experience often does not provide the careful comparison required to critically test

claims about what is true and what is not.

14. Agree: Consider for example neuroscience, astrology.

15. Disagree: Interest groups do influence social policy. However, policymakers often review

scientific evidence related to a problem. Also, many citizens seek and review research data related

to questions.

16. Disagree. Science is a way of approaching problem-solving. It involves critically testing

claims. If an element of intelligent life came to us from one of Jupiter's moons, that

creature's society would have to guess and test as our scientists do to solve problems.

Scientific reasoning (guessing and testing) has been used all over our planet by persons of

all races to solve problems. Eskimos survived because they relied on basic principles of

science (Nelson, 1993).

*We thank Michael Hakeem, Professor-Emeritus of the University of Wisconsin at Madison, for

his suggestions on some of the answers.

_____________________________________________________________________________________

Exercise 3: CONTROVERSY ______________________________________________________________________________

OVERVIEW

Making informed decisions often requires consideration of different views. Enhancing related

skills will help students to participate effectively in discussions and arrive at well-reasoned

decisions.

PURPOSE

To enhance students’ skills in constructive controversy.

MATERIAL REQUIRED (Depends on Activity Selected) (see p. 83-84).

1. Guidelines for discussion in text.

2. Access to Internet/databases.

TIME REQUIRED

Depends on activity selected.

SUGGESTIONS

Seek other material that requires constructive controversy. Procon.org lists many debate topics.

See also newseumed.org; from provocative to productive: teaching controversial topics. Steps in

constructive controversy suggested by Johnson and Johnson (undated) include the following:

1. Organize students into groups of four, divide each into two pairs, assign pro position to one

and con to the other.

2. Ask each pair to research and prepare a position statement and to present their position.

3. Engage in open discussion.

4. Reverse perspectives and present other position, adding new information as available.

5. Synthesize and integrate best evidence and reasoning in a joint position statement (p, 116).

Promote genuine discussion by making sure students are interested in the topic. See also

www.crit.umich.edu. “Discussion-based teaching and handling controversial topics in the

classroom.” Some tips include: allow everyone a chance for input, do not interrupt other (unless

violating norms), establish limits for one person to talk, hand out cards with helpful hints (e. g.,

“Were reasons for a position given?”).

Johnson, D. W. & Johnson, R. T. (2015). Constructive controversy: Energizing learning. Small

Group Learning in Higher Education (pp. 114-121).

____________________________________________________________________________________

Exercise 4: CRITICAL THINKING AND ADVOCACY

__________________________________________________________________

OVERVIEW

Advocacy in which practitioners pursue enhancement of quality of services and/or clients’ lives

is an important component of being a helping professional. There are many kinds of advocacy

including urging staff in other agencies to provide requested help, pursuit of legal action, and

seeking agency changes that benefit clients.

PURPOSE

To emphasize the importance of advocacy efforts; to give students an opportunity to engage in

related planning/action.

MATERIAL REQUIRED

1. Exercise 4.

2. Content in text.

TIME REQUIRED

Depends on activity selected.

SUGGESTIONS

Emphasize the importance of the need for advocacy and the wisdom of working together with

others to increase success. Students may have special interests, for example involving clients in

advocacy efforts, or making the reporters to expose avoidable miseries. As a start they may

work together with others to clearly depict a harmful situation, describe who it affects and who

or what organizations are responsible for the unjust situation, and suggest remedies.

1

_______________________________________________________

Exercise 5: CRITICALLY APPRAISING HUMAN-SERVICES

ADVERTISEMENTS _____________________________________________________________________________

OVERVIEW

This exercise uses a persuasive advertisement and a form for rating this. Discussion questions

illustrate concerns about using advertisements to guide decisions.

PURPOSE

1. To demonstrate characteristics of human-services advertisements.

2. To teach students to rate such evidence against criteria for identifying advertisements.

MATERIALS REQUIRED FOR ACTIVITY 1

1. There are probably groups of practitioners, hospitals, and organizations in your area that

produce advertisements for their programs that you can use for teaching material. You

could draw on promotional material on the Internet and/or in magazine and newspapers that

advertise weight loss, study skills, smoking cessation, and other types of self-help

programs. Videotaped presentations at professional conferences may meet the criteria for

an advertisement: A charismatic and well-known person describes an intervention, presents

the method in an entertaining way, and does not raise the issue of effectiveness. We

recommend collecting audiovisuals of related material to use for demonstrations,

examinations, and quizzes.

2. DVD or videotape player and screen or monitor.

3. Copy of Human-Services Advertisement Spotting Form (Practice Exercise 5).

4. Scoring Instructions.

TIME REQUIRED

For Activity 1 about ten minutes to show the promotional material, five minutes to rate it, five

minutes to score the Human-Services Advertisement Spotting Form, and 20 minutes for

discussion.

SUGGESTIONS

Step 1. Obtain an example of a human-services advertisement. See for example website of

Rogers Memorial Hospital.

Step 2. Ask the class the following:

"Please watch this and circle your answers to the ten items on the Human-Services

2

Advertisement Spotting Form:" (Give the class a few minutes to examine the form before

you begin showing the advertisement.) Try not to imply any bias for or against the

material.

Step 3. Show the material and allow the class a few minutes to complete the form.

Step 4. Score the responses. (See scoring instructions below.) The higher the number of points

on the form, the more likely the source is a human-services advertisement. Generally, if

the source scores more than five points it is an advertisement.

SUGGESTED ALTERNATE PROCEDURE

Show the advertisement before students are given the Workbook. Show them the following

questions before presenting the material and allow participants to refer to them as they view

material.

1. What is the one central conclusion that the makers of the advertisement would have you

draw regarding their program?

2. Based on this material, would you refer your clients to this facility? Yes_ No _

3. Why or why not? (Explain your answer to question 2.)

SCORING THE SUGGESTED ALTERNATE PROCEDURE

For question 1, give five points for identifying the conclusion that the program is

effective, or that persons should refer their clients to the program because it is effective.

For question 2, give five points for a "no" answer.

For question 3, give five points for every methodological concern raised, including:

allusion to an emotional appeal, use of case example or testimonial, manner or style of

the presenter, use of music, one-sidedness, absence of data regarding effectiveness of

the program, need to measure success, experimental designs that might be used to

evaluate the program. Rank (1982, p. 147) identified five universal aspects of

advertising that include the following: 1) Attention-Getting, 2) Confidence-Building, 3)

Desire-Stimulating, 4) Urgency-Stressing, and 5)Response-Seeking. (The Key for

Exercise 6, may help here).

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

Which features of an advertisement does the material demonstrate?

_____________________________________________________________________________

Answers to Activity 1 Exercise 5 _____________________________________________________________________________

3

For each "Yes" answer to items 1, 3, 4, 5, 8, or 9 on the Human-Services Advertisement Spotting

Form give 1 point. For each "No" answer to items 2, 6, 7, or 10, give 1 point. Total the number

of points. Scores can range from 0 to 10. Record this value on the bottom of the Human-Services

Advertisement Spotting Form. If the source scores more than 5 points, it is an advertisement; the

higher the point value, the weaker the evidence.

1

__________________________________________________________________

Exercise 6: DOES SCARING YOUTH HELP THEM "Go STRAIGHT"?

_________________________________________________________________

OVERVIEW

This exercise evaluates skills in reasoning critically about an emotional appeal regarding the

claim that a delinquency prevention method works. It evaluates respondents' ability to apply

basic principles of critical thinking to a practice decision. Participants watch promotional

material and answer questions related to whether they would refer clients to the program. The

material violates most major tenets for thinking critically about the effectiveness of a treatment

program. It is pure propaganda. The exercise relies on the "Scared Straight" film or videotape

(not included but hopefully available in university libraries or interlibrary loan). This exercise

can be used as a teaching lesson and as a measure of critical thinking.

PURPOSE

To measure skills in thinking critically about case material when faced with a powerful

emotional argument that a method works. This case material was selected because its artful

emotional appeal has no value as scientific evidence regarding the program's effectiveness; yet,

its central theme argues constantly and persuasively that the program is effective. The viewer

who can resist its emotional appeals, vagueness, one-sided presentation, oversimplifications,

persuasive manner, case examples, testimonials, appeals to numbers, and so on, without being

sucked into this illogical morass, demonstrates powerful critical thinking skills.

MATERIALS REQUIRED

1. A VHS videotape or DVD player or laptop.

2. A VHS or DVD videotape that describes the Juvenile Awareness Project at Rahway State

Prison in New Jersey. The "Scared Straight" material is available at many libraries.

Otherwise, the film or videotape can be purchased. Showing the film/video on a large

screen is more effective than watching it on a small laptop screen.

3. A suitable room, where lighting can be adjusted so that respondents can sit comfortably

while viewing the film or videotape as they take notes. The room should provide sufficient

space between respondents to discourage distractions by a neighbor.

4. Two sheets of 8.5-by-ll-inch lined paper (one for notes and one for the respondent's

answer).

5. Blackboard or projector to display questions for easy reference during the test.

6. Pen (scoring and reliability checks are more legible with ink than with pencil).

7. Scoring instructions. Up to 50 points are possible-5 points each for items 1 and 2, and 40

for item 3. (See scoring instructions below.)

2

TIME REQUIRED

About fifty minutes to show the material, 15 minutes to answer, 10 minutes to score responses,

and 20 minutes for discussion.

SUGGESTIONS

We suggest two uses for this exercise. Many, in spite of their efforts to think critically, are taken

in by its nonstop emotional appeal. This presentation is the antithesis of good evidence about the

effectiveness of a program. Anyone who would subject clients to the program described by

saying "yes" to item 2 without better evidence has been taken in by emotional appeals providing

no empirical evidence regarding the program's effectiveness. Indeed, research suggests that

"Scared Straight" programs do more harm than good (Petrosino, Turpin-Petrosino, & Buehler,

2003).

This exercise can also be used as a measure of critical thinking. Studies have found high inter-

rater reliability. Two undergraduate research assistants independently scored responses from

students in five research methods classes from four universities. Their Pearson r reliability

coefficients were: r = .96, n = 19; r = .89, n = 17; r = .83, n = 18; r = .87, n = 35; r = .90, n = 26

(Gibbs, Gambrill, Blakemore, Begun, Keniston, Peden, & Lefcowitz, 1995). Students in labs of a

Social Work Research course at the University of Wisconsin at Eau Claire, without any prior

training, scored their own and their partner's responses with these r values: r = .98 (n = 7), r =

.65 (n = 7); r = .64 (n = 9). Students (n = 20) in an honors course on Thinking Critically about

Social Problems at the same university scored themselves and a partner without any prior

training and obtained an r of .94.

Try not to offer any cues that might imply a positive or negative attitude toward the material. A

matter-of-fact style works best. Set up and test the audiovisual equipment before students enter

the room, and have all materials available. Hand each student the pen and two sheets of paper,

and explain: "I would like you to react to some case material. You may use one of these pieces of

paper for your notes. The other is for your answer to three questions that I will give you. Please

write clearly."

Either write the questions in Practice Exercise 6 (in the Workbook) on the blackboard or project

them. All respondents should be able to refer to the questions during testing, but should not be

allowed to take a copy of the questions from the classroom (to avoid compromising the

measure).

Read the questions aloud to make sure that respondents are familiar with them. If students ask

questions, tell them the following: "Please reread the question carefully, and base your

interpretation on what the question implies to you. I can't help you with an interpretation because

it might bias the result." Tell respondents: "When the audiovisual material is finished you will

have 15 minutes to write your answers to the two questions." (Be sure to set your timer when

they begin and give them exactly 15 minutes.) After the 15 minutes, inform them: "The fifteen

minutes are up. Please finish the sentence that you are writing and hand in your paper."

3

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

1. What was your score?

This exercise has been administered to students in five research methods classes at four

different universities with these results:

n = 19, mean = 6.6, sd = 3.6; n = 17, mean = 8.6, sd = 8.6; n = 18, mean = 12.1, sd = 3.7; n =

35, mean = 10.8, sd = 7.1; n = 26, mean = 30.2, sd = 7.4 (Gibbs, Gambrill, Blakemore,

Begun,, Keniston, Peden, & Lefcowitz, 1995). In classes in which students have had no

exposure to critical thinking, it is common for all to get five or fewer points.

2. What is the main kind of appeal in the "Scared Straight" material? The scoring key identifies

the dominant forms of evidence as pure propaganda.

3. Why did you respond as you did to the "Scared Straight" presentation?

Discuss the powerful emotional appeals, including violent street language, physical

intimidation, close-up to juvenile's tearful face, slamming doors, vivid news accounts, Peter

Falk's tough-sounding talk and voice, before/after talks with juveniles, careful editing,

dramatic camera angles, and so on.

4. Do you think the Juvenile Awareness Project might have produced harmful (iatrogenic)

effects? Share results of Petrosino, Turpin-Petrosino and Buehler (2003)'s study.

5. Is this exercise a valid test of critical thinking? Explain your answer.

We think that it is because it asks for a decision based on evidence.

SCORING INSTRUCTONS

Up to 50 points are possible: 5 points for each of items 1 and 2; 40 for item 3.

Item 1

Give 5 points if the respondent states in any way that the program is effective at preventing

delinquency, otherwise no points. Here are examples of five-point responses:

The results of this program 'are unparalleled by conventional treatment methods.

The juveniles really go straight.

The program is effective.

The program helps juveniles.

The program really works.

Give zero points for other conclusions that do not state that the program is effective at

delinquency prevention, for example:

The program is inexpensive.

4

The Lifers enjoy their work.

Convicts and prison officials can work together.

I would be scared too.

Item 2

Give 5 points for a "No" answer. Give zero points for a "Yes" answer.

Item 3

It would be a daunting task to name and explain all of the fallacies of reasoning in the "Scared

Straight" audiovisual. However, ten prominent fallacies are named and explained below. Give 2

points for using a key word from the list of key words, but give only 2 points for the first occur-

rence of a key word in each group. Give 2 points for each explanation that clearly describes how

a particular fallacy is present in the presentation.

1. Key word: Emotional Appeal, Visual Emotional Appeal, Appeal to the Emotions,

Shocking Emotional Appeal (2 points for just one of these).

Explanation: This presentation is a nonstop assault on the emotions. Examples include

shocking street language, slamming doors, homosexual taunts, vivid descriptions of

atrocities in prison life, shots from an angle showing huge inmates towering over

juveniles, a tear in the eye of a juvenile; these are all ways that audiovisuals heighten

emotional impact. Such emotional appeals are distracting and irrelevant to the central

question: Does the program work? Give 2 points for any statement that critiques such

emotional appeals as reasons to accept a claim, or notes that they distract attention from a

critical review of evidence concerning program effectiveness.

2. Key word: Status, Titles, Well-known Person, Familiar Person, Ad Verecundium, Appeal

to Authority (2 points for just one of these).

Explanation: Very prominent people, including the warden, Judge Nicola, and actor Peter

Falk, say the program is effective. Saying so doesn't make it so. Just because prominent

people say the program prevents delinquency does not mean it does. Prominent people

are bound by the same rules of evidence as anyone else. Give 2 points if the respondent's

answer states somehow that the status of persons arguing for the program's effectiveness

does not assure that their argument is true.

3. Key word: Manner, Style, Sincerity, Sincere (2 points for just one of these).

Explanation: The fallacy of manner assumes that the persuasive manner in which a

person presents an argument assures the argument's validity. Examples include inmate's

talk about their sincere efforts to help the juveniles (e.g., "my apostolate work"), and how

the announcer had a sincere way of describing the program and its effects. Give 2 points

only if the respondent comments about the manner of the inmates, officials, or juveniles

and debunks the idea that the manner supports the claim that the program is effective.

5

4. Key word: Vagueness, Vague Term, Undefined Term, Outcome Not Defined, Outcome

Not Measured (2 points for just one of these) .

Explanation: Although the term "straight" is used throughout the presentation, "straight"

is never defined. Nor is the term "successful." What precisely does "straight" mean? Who

determined this outcome? What criteria for "straight" were used? Were court records

consulted? Where? Claims that a program works must be judged against a clearly

defined, valid, and reliable outcome measure. Give 2 points for any answer that implies

confusion about what "straight" or "success" means, or if the respondent asks for a

definition of this term.

5. Key word: Testimonial(s) (2 points for using this term).

Explanation: Testimonials involve asking recipients of a program to tell how the program

has affected them. Testimonials are a weak form of evidence because: (1) Those giving

testimonials are subject to pressure to say positive things about the program, (2) those

chosen to give testimonials may have been hand-picked to say favorable things, (3)

testimonials rely on emotional appeal because the viewer can identify with the people

involved in the program, (4) editors can add and remove sections of material (film, tape

or written material) to support a favored position, and 5) a testimonial is a sample of 1 so

violates the law of large numbers. (See Exercise __.) The juveniles gave testimonials

when they described how scared they were and how they were going to stop their

delinquent behavior. Give up to 2 points for mentioning anyone of the above weaknesses.

t

6. Key word: Ignorance of Possible Harmful Effects, Possible Iatrogenic Effects, Possible

Harm, May Have Harmed (2 points for just one of these).

Explanation: Not only may the program have been ineffective, it may have harmful

effects; that is, make the delinquents more delinquent. Give 2 points for any mention that

the program might harm the delinquents.

7. Key word: Case Example, A Few Cases, Case Example Fallacy, Just a Few Cases (2

points for just one of these).

Explanation: Case example involves showing just one or a few unrepresentatively chosen

individuals and generalizing to all such individuals. Case example is weak because (1) it

encourages us to rely on specific emotion-arousing material (e. g., the muscular juvenile

who was stared down and said "You would kill me") that may bear no relation to

soundness of an argument, (2) cases can be hand-picked because their behavior will

support a bias, and (3) the sample is small (violates the law on large numbers). Give 2

points for recognizing the weaknesses of presenting 17 hand-picked cases and then

generalizing what may be true of these individuals to the whole program.

8. Key word: Appeal to Numbers, Popularity, Peer Pressure, Bandwagon, Because So

Many, Because So Few (2 points for just one of these).

6

Explanation: Appeal to numbers is a fallacy when the number of persons exposed to the

treatment is given in hopes of convincing the viewer to use the method. An appeal to

numbers can argue that many have been exposed to the treatment, and therefore more

should be. An appeal can also be based on the idea that few such wonderful programs are

now available, and others should hurry and start one. Give 2 points for correctly

identifying that claims that thousands have passed through the program, or that only a

handful of such programs exist, say nothing about the program's effectiveness.

9. Key word: Ignorance of Better Evidence, Not Asking for Better Evidence, Ignorance of a

Hierarchy of Evidence, Accepting Poor Evidence, Ignorance of Scientific Evidence,

Ignoring Evidence (2 points for just one of these).

Explanation: The propaganda appeals in "Scared Straight" are among the weakest that

could be given regarding the effectiveness of a program. Give 2 points for requesting any

form of scientifically better evidence (e.g., a study that includes random assignment to

treated and control groups, objective observers, keeping systematic records of outcome,

reported loss of follow-up rate, reliable and valid outcome measure, and so on).

10. Key word: Lack of Objectivity, Bias, Biased, One-Sided, Stacking the Deck, Loading the

Dice (2 points for just one of these).

Explanation: Critical thinking requires that evidence and counterevidence be examined in

an evenhanded way. One must look at all evidence and arguments regarding the

effectiveness of a program, not just evidence favoring it. This presentation argues

nonstop, from Peter Falk's introduction to Judge Nicola's conclusion, that the program is

effective. Give 2 points if the respondent refers to how one-sided the evidence was or

expresses a need for evidence about how the program might not have been effective.

__________________________________________________________________

Exercise 7: DETECTING MISLEADING PROBLEM FRAMING

__________________________________________________________________

OVERVIEW

What problems are selected for attention and how they are framed influences clients’ options. It

is important to increase students’ understanding of the social construction of problems and

identify what problems may be ignored.

PURPOSE

To enhance students’ understanding of the importance of thinking about who decides what

situation comprises a problem and what kind. To enhance understanding of political, social and

economic factors related to problem construction.

MATERIAL REQUIRED

1. The century of the self by Adam Curtis (2001) is available on YouTube. Decide what parts

you want students to watch if they do not have time to watch the entire program.

TIME REQUIRED

Time to watch and discuss presentation.

SUGGESTIONS

Try not to rush this activity. You must decide whether to show this in class or to ask students to

watch it on YouTube. Discussion time is important.

__________________________________________________________________

Exercise 8: FOLLOWING THE MONEY

__________________________________________________________________

OVERVIEW

Many corporations are involved in the helping professions. An interest in profit may contribute

to goal displacement including promotion of services that harm rather than help clients.

Following the money (who gets paid for what) is valuable in understanding sources of goal

displacement.

PURPOSE

To increase students’ attention to pursuit of profit in the helping professions which may

compromise quality of services.

MATERIALS FOR ACTIVITY 1

1. Access to the Internet.

2. Content in text.

TIME REQUIRED

Time on the Internet will be required for this in addition to class presentation time for “follow-

the-money” reports.

SUGGESTIONS

Model tracking of money involved in a service for students. This will introduce them to valuable

web resources they can use in their searches. You could also introduce students to key resources

of fraud and corruption such as Transparency International.

__________________________________________________________________

Exercise 9: THE LANGUAGE OF PROPAGANDA

_________________________________________________________________

OVERVIEW

Language influences what we do and feel as Orwell (1984) describes in Politics and the English

Language (1946).

PURPOSE

To increase students’ awareness of how words affect emotions and decisions.

MATERIAL FOR ACTIVITY 1

1. Content in text including list of language ploys.

2. Material to be reviewed.

TIME RQUIRED

Depends on how many students review their material.

SUGGESTIONS

Review language ploys in class. Have students read the Politics of the English language (1946)

and discuss this in class, applying content to current topics in the helping professions. Ask

students to read On Bullshit (Frankfurt, 1986) and discuss how bullshit differs from lying.

Encourage students to identify language ploys and discuss how students can minimize the

influence of language. See also other suggested activities. Ask students to consult social

psychological studies regarding effects of language on emotions,

1

__________________________________________________________________

Exercise 10: USING THE PROFESSIONAL THINKING FORM

__________________________________________________________________

OVERVIEW

This 25-item short-answer questionnaire contains brief vignettes that may or may not contain a

practice fallacy (common error in reasoning). The clinical reasoning, critical thinking, and

informal logic literature describes common errors. The Professional Thinking Form (PTF) can be

used to generate c1assroom discussion. It can be self-scored or used as a research instrument.

PURPOSE

1. To test students' skill in identifying common practice fallacies.

2. To teach how to identify fallacies in reasoning about practice.

MATERIALS REQUIRED

1. The Professional Thinking Form (Practice Exercise 5).

2. Ballpoint pen (for greater clarity than pencil for reliability checks).

3. Scoring key (see Answers to Exercises).

TIME REQUIRED

Up to 30 minutes for the PTF, 10 minutes to score it, 20 minutes for discussion.

SUGGESTIONS

The Professional Thinking Form was developed as a measure and teaching tool. It contains 25

brief vignettes illustrating common fallacies and pitfalls that cloud thinking across the helping

professions (e.g., attacking the person, accepting case examples as evidence, selection bias).

Documentation showing how PTF's fallacies negatively affect judgments can be found in related

literature (e. g., in medicine and psychology). PTF's vignettes illustrate fallacious thinking in

examples from social work, community organization, mental health, child welfare, chemical

dependency, family therapy, and other areas.

Each of the vignettes, including two nonfallacy items (5 and 9), and duplications of two fallacies

(4 and 10, 14 and 19) occupy just a few lines of text. Longer excerpts would limit PTF's

classroom utility and make it difficult to score. Shorter excerpts best illustrate just one practice

fallacy.

For discussions in large classes, items can be projected onto a screen for item-by-item

examination. Students can read each item on the screen or refer to their own copy, then comment

2

on what they think might be the fallacy. In our workshops for professionals, we ask participants

to write a response to about four items at a time. We then discuss each item relative to the

scoring key's explanation. Participants contribute additional instructive, entertaining examples of

each.

SUGGESTIONS FOR USING PTF AS A MEASURE

Although PTF was designed as a teaching tool, it can be used as a measure of critical thinking

about practice. PTF's format and some of its scoring instructions are based on an instrument by

Logan (1967). The instrument has undergone several revisions since it was first developed

(Gibbs & Werner, 1988). The version immediately prior to this one was published in Scientific

Reasoning for Social Workers (Gibbs, 1991, pp. 54-59, 274-278). The present version clarifies

scoring criteria, rewrites weak items (based on a Spearman rho item-analysis), and changes the

total possible point value from 115 to 100 points. If used as a research instrument, we

recommend that respondents be informed that "you are completing a measure of critical,

professional, analytical, scientific thinking. . ." and that these accompanying instructions be

given:

1. Your responses will not be used to grade you nor to reflect positively or negatively on you in

any way.

2. Please keep your responses brief and to the point. When you finish, turn your paper over and

leave it that way. You have 40 minutes to complete your responses. (Few people require 40,

but allowing this time takes the pressure off respondents).

Reliability and Validity

Inter-rater reliability has been checked several times. The first reliability trial was conducted by

Gibbs and Werner (198S) on responses from social work and nursing students. This yielded a

Pearson r = .89 for the first trial (p < .0001, n = 21), and r = .89 for a second trial (p < .0001, n =

31). Spearman rho for independent scoring of each of 20 items on PTF ranged from .46 to .99

(mean = .78, sd = .13) on the first trial and from .65 to .95 (mean = .80, sd = .08) on the second

trial. The current version has been revised in light of results from item analysis. Three items were

revised extensively (because they were weak), and two new nonfallacy items were added.

Our primary claim to PTF's content validity rests on how well items and references in the key

reflect fallacies and concepts in the critical thinking and clinical reasoning literature. Thelen

(1993) tested correlations between PTF and both the Watson-Glaser Critical Thinking Appraisal

(1980) and cumulative grade point average for 97 senior undergraduate nursing students at the

University of Wisconsin at Eau Claire.

Pearson r correlations were .36 (p < .001) and .18 (p < .10) respectively. Presently, we know of

no applications to more or less competent criterion groups. It might be informative to see if

practitioners with different levels of training and experience score differently on PTF.

Scoring PTF For Measurement Purposes

3

Scoring takes about six minutes for experienced scorers. It can go faster with fewer errors if

respondents use a nongooey, legible ballpoint pen rather than pencil to record their answers.

Items are scored from 0 through 4. Here is a way to assign point values for each item:

No points: The item is not answered, nor is the thinking in the vignette questioned in a

specific way, nor is a key word used (e.g., answers to Situation 1: "OK" or "I like to attend

entertaining workshops where the presenters are funny").

One point: The answer raises approximately the same question as the one raised in the

explanation but does not use a key word (e.g., answer to Situation 1: "I wonder about accepting:

Steinberg's argument based on the way she presents it").

Two points: The respondent uses a key word identically, or almost identically, to one or

more words in the Key Words to be Located in Answers Column (e.g., answers to Situation 1:

mannerisms, manner, charismatic, charisma).

Or, the answer raises the same question that is raised in the explanation, but does not use

a key word (e.g., answer to Situation 1: "Steinberg's way of presenting herself does not assure

that her argument is true. We need to see her evidence").

Three points: A close or approximate answer gives a key word.

Four points: Both a key word and the explanation match the key (e.g., answer to

Situation 1: "Steinberg's style of presentation does not assure that her argument is true. We need

to see her evidence").

POSSIBLE ANSWERS TO FOLLOW-UP QUESTION

Do any of PTF's vignettes reflect real situations particularly well? Which one(s)?

Suggestion: You might inform respondents that most of the items reflect real events that have

been altered to protect confidentiality.

4

Answers to Practice Exercise 10

Give 2 points for a key word (only 2 points for one key word in each item). Key words are listed

in bold print above the explanation section. For an explanation that states essentially the same

answer as in the explanation, give 3 points. For an answer that approximates the explanation,

give 2 points. For any answer that questions the item at all, give 1 point. For not questioning the

item at all, give zero points.

1. Manner, style, charisma, stage presence, likeable, likability.

The convincing way a speaker presents an argument does not mean the idea is valid.

The argument must be critically examined.

2. New, newness, novel, tried and true, tradition.

Accepting a method based on newness, or traditional use, ignores the fundamental issue

of whether the method works.

3. Uncritical documentation, relying on citation alone, not examining the document.

Just because an idea can be cited does not prove a point. Examine the contents in the

reference to see if it supports the argument.

4. Ad hominem, personal attack, against the person, attacking the person.

The coworker rejects Hajdasz's evidence because of his personal life, not because of his

evidence.

5. OK, not a fallacy, no fallacy, all right, no problem.

The thinking here is sound. Key terms need definition before an argument can be

understood, so asking for a definition is necessary here.

6. Experience, appeal to experience, all evidence is equally good, ignorance of

scientific evidence, ignoring evidence.

The worker does not discriminate between sound scientific evidence, with its

procedures for controlling bias, and personal experience. Where clinicians' judgments

are compared with actuarial findings (research findings and statistical methods), the

latter consistently outperform the former (Grove & Meehl, 11996).

7. Popularity, peer pressure, bandwagon, numbers, because everybody.. .

Widespread use of a method does not demonstrate its effectiveness. In fact, some once-

popular treatments have been shown to be harmful (e.g., the drug DES caused cancer in

children of some women exposed to it (Berendes & Lee, 1993). When the drug was

5

popular, its advocates argued strongly in its favor and defended it from critics (Dutton,

1988).

8. Appeal to authority, ad verecundium, status titles, degrees, famous, fame.

Because an argument is presented by someone with status or titles does not mean the

argument is true or valid. Famous persons are bound by the same rules for evidence as

everyone else

9. OK, not a fallacy, no fallacy, all right, no problem.

This is not a fallacy because it states a truth about the helping professions: Factors other

than the treatment often account for improvement (Kassirer & Kopelman, 1991, pp.

171-186). Sound causal inferences about the effects of an intervention can be made

only after careful experimentation, painstaking observation, and record-keeping.

10. Ad hominem, personal attack, against the person, attacking the person.

Ms. Washington's argument is rejected because of her personal appearance. Instead, the

validity of her argument should be examined.

11. Objectivity, objective, honest(ly), bias (ed) , vested interest, cheating in science,

unethical.

Consultants should follow scientific principles, without allowing their client's self-

interest to influence the findings.

12. Either/or, only two sides, only two alternatives, false dilemma.

The agency supervisor limits thinking about the problem to allowing discussion of only

one alternative or the other. Both alternatives may be inappropriate, or some other

alternative may be best. A combination of the two alternatives may be appropriate.

13. Testimonial.

Clients who give a testimonial are subject to bias in their statements about treatment

effectiveness. An answer to this item gets full credit by listing any bias (e.g., small

unrepresentative sample, emotional appeal, pressure by a therapist to give a positive

report, biased selection of clients, the testimonial provides no standard for comparison

with other individuals.

14. Hasty generalization, possibly biased sample, unrepresentative, sweeping

generalization.

6

Those who called the planners may not have reflected the whole community's

sentiments. The callers may just have been the most vocal. Reference to the need for a

survey scores two points.

15. Vagueness, unclear term, undefined term.

What does "depressed" mean? Before making a judgment about a client, specific

criteria for making such a judgment should be identified. Some criteria for estimating

degree of depression are specific and reliable.

16. Case example, single case.

Neither what is generally true of thousands of Vietnam-era clients, nor whether a

treatment generally benefits them, or any group of clients, can be inferred from a single

case. Single cases have emotional appeal. Case examples help to foster the erroneous

feeling that, if you know one you know them all. They may be selected purposely to

support almost any generalization.

17. Two questions, double-barreled, two-headed, ambiguous.

This question is confusing because it poses two questions at once.

18. Leading, loaded, biased question.

This question implies a specific answer, thus limiting the client's freedom to give an

accurate answer.

19. Sweeping generalization, hasty generalization.

This general statement (sweeping generalization) is made without any evidence to

support it. Give 2 points for asking any methodological question, for example, "What

does electroconvulsive shock imply? Have studies been done to evaluate this treatment

method? 'Most effective' relative to what?"

20. Tautology, word defines itself, “progress” defines “progress.”

By defining “progress” as being “progress” or “independence” as being

“independence” the reader does not know what specific behaviors are required to gauge

improvement (Fearnside & Holther, 1959, p. 137).

21. Post hoc, after this, temporal order, because treatment precedes improvement.

Simply because improvement follows treatment does not mean that treatment caused

the improvement. Other factors may have caused the improvement; or improvement

may have occurred in spite of the treatment.

7

22. Jargon.

The worker uses professional words that the client cannot readily understand.

23. Regression, regression fallacy, regression to the mean.

Individuals scoring extremely low or high on pretest tend to score closer to the mean on

posttest. This is because extreme scores often have a high error component.

24. Selection bias, biases selection of clients.

RARC’s 75% treatment success may be due to self-selected, higher-functioning clients at

the RARC agency.

25. The probability that abuse will occur (positive predictive value of the ST), given the

Donohue’s Family Positive ST score is:

(Prevalence)(Sensitivity)

=

(Prevalence)(Sensitivity) + (1-Prevalence)(1-Specificity)

(.95)(.03)

=

(.95)(.03) + (.05)(.97)

1

__________________________________________________________________

INSTRUCTIONS FOR REASONING-IN-PRACTICE GAMES

__________________________________________________________________

OVERVIEW

In the RIP Games students work together in small groups to identify fallacies and cognitive

biases in brief practice vignettes. Teams take turns acting out parts in the vignettes.

Countermeasures and "what you could say" in relation to each one are shown in the Answers

sections. Game A concerns common informal fallacies in reasoning. (See also pp. 134-136 of

text.)

PURPOSE

1. To have fun.

2. To learn how to identify and avoid common fallacies or pitfalls in reasoning related to

practice.

MATERIALS REQUIRED

1. Steps and suggested activities.

2. Fallacy definitions for game being played.

3. Answers for game being played.

4. Watch or timer.

TIME REQUIRED

One hour of homework to prepare to play the game by reading the fallacy definitions and game

rules, five minutes in class to review the game rules, up to 90 minutes to play the game (the

game can end at any time based on an agreed-upon time limit).

SUGGESTIONS

We have experience with these games in a variety of contexts including undergraduate and

graduate classes and with faculty and field instructors involved in professional education.

1. We recommend that you follow the vignettes in sequence as they appear in the Workbook.

The order of the vignettes in the Workbook matches the order in the Answers to Exercises

section. Once you are familiar with the game, you might try the option below with the cards

as they appear in this section of the Instructor's Manual.

2. You could duplicate the pages with the vignettes in this manual on card stock at a local copy

shop and make your own sets of cards for each game. You could print each set of game cards

2

on a different color paper. Students could be asked to do the same. Answers could be written

on the back of each card as students practice spotting and countering fallacies before class

use. Or you could make a master list of answers for each game on one card and refer to this

as needed. You could include a brief definition of the fallacy illustrated in each vignette. You

may want to point out the relationship of particular fallacies to particular cognitive

illusions/biases (Pofl, 2004). For example, vivid case examples and testimonials are

"available" in our memory.

A card format allows mixing of cards from games. For example, you may want to select a

given set of cards reflecting a special interest (e.g., in a particular setting). With a card format

it is easy to practice in everyday settings outside of class. The card format makes it easy to

add examples (just write down the situation on a card and number in sequence).

3. Be sure that the moderator has time to read and understand the game rules so that when

questions arise they can be answered quickly. It helps to have an enthusiastic moderator with

a sense of humor.

4. Be sure the moderator has an accurate clock and holds to agreed upon time limits. Initial

vignettes may require a little more time compared to later ones, while participants get used to

the game. Permitting protracted discussions bogs the game down.

5. Make certain that the game players have both the time (approximately one hour) and

the motivation to read each game's rules and the fallacy definitions carefully before

playing the game. On two occasions, where the players were handed the game and were

given only about fifteen minutes to read the fallacy explanations, the game was extremely

difficult with extensive disagreements about which fallacy each vignette represented, even

though the audiences were highly educated. On the other hand, freshman undergraduates had

remarkably consistent judgments (i.e. teams get 80 to 90 points) where they had been given

the time to study the game's fallacies.

6. We tried to design vignettes so they most represent a single fallacy, but features of more than

one fallacy can be found in many vignettes. Students who disagree with the suggested answer

to a situation have an obligation to describe and offer cogent reasons for their view.

7. Emphasize that participants are playing the game not to win over their worthy opponents, but

to learn about how to identify and avoid fallacies and biases in making decisions so they can

be better helpers. Though the game is light-hearted, it concerns matters of grave importance.

If participants share this spirit, they will be more tolerant of (or even welcome) competing

views. Emphasize the importance of understanding why a particular fallacy or bias applies to

a given example, rather than merely agreeing with the views of other students. Encourage

participants to take responsibility for describing why they believe the fallacy or bias they

select applies to a vignette. Coach students to ask for reasons for beliefs if confronted with

someone who merely labels a vignette with a fallacy without describing how it applies.

Mary Ann Sontag divided her class of 70 students into 14 teams of five students each. She

does not hand out the vignettes. At odd moments during the semester she asks two teams to

3

go to the front of the class to compete in evaluating three practice vignettes. Roles assigned

to other class members include two scorekeepers (one for each team), one timer, one coach

for each team (to determine when all team members have made a choice and to guard against

cheating), and two crowd control leaders (who lead cheers for their team). The rest of the

class cheers and observes. Sontag gives each team member one point for an answer that

agrees with the key and also gives each team five points if they are unanimous. She makes

sure every team has a chance to play in the first round, then reduces the field to winners and

losers, then plays successive rounds until the final round, where competitors get prizes. This

encourages class attendance (students never know when there will be a few minutes to play

around and it encourages students to keep reviewing fallacy definitions so they can play the

game at a moment's notice).

8. We provide feedback to the groups as they play to catch and correct incorrect beliefs (e.g., a

correct answer based on an incorrect account of why the fallacy applies). We support astute

questions and statements. We encourage groups to make note of disagreements for further

discussion with the class as a whole. You may also overhear exchanges as you circulate

among the groups and note positive and negative ways to handle differences of opinion,

which may reflect some of the very fallacies being considered (e.g., ad hominum appeals).

If Students Play the Game by Themselves

Students could work through each vignette and keep a score of their "hits" (correct fallacy

spotting) and "misses." They could formulate a response to each item and compare their

responses to the "What You Could Say" suggestion.

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

Do any of this game's vignettes reflect real situations particularly well? Which one(s)?

We have found that items remind participants of real-life situations, particularly participants who

have had some practice experience.

RESEARCH REGARDING RIP's EFFECTIVENESS

Duncan Whyte of the University of South Carolina, following a Solomon Four Group design,

randomly assigned masters degree students in a research methods class to play the RIP games or

to a control. He found among the 136 participants in the study, as measured on the Professional

Thinking Form (see Exercise 10), that those playing the RIP Games scored statistically

significantly higher at posttest on PTF and had statistically significantly higher change scores

over controls. There was no statistically significant difference across controls at posttest among

controls who took a pretest compared with those who did not (possibly explainable by low

statistical power).

1

__________________________________________________________________

Answers to Exercise 11: REASONING-IN-PRACTICE GAME A:

COMMON PRACTICE FALLACIES

__________________________________________________________________

GAME A: NO.1

FALLACY 2

Testimonials. The client said that the treatment method worked, based on her own

personal experience. Other relevant data are not reviewed such as others who experienced

the method and did not improve, or got worse, or those who did not participate in the

treatment and improved (perhaps even more). Individuals who give testimonials may be

hand picked from among many treated individuals. Testimonials can capture the

emotions with their "I'm just like you..." appeal.

Countermeasure. Search for reports of controlled studies to evaluate the effects of the

treatment. Both case examples and testimonials involve partiality in the use of evidence-

looking at just part of the picture.

What you could say. Do you know if there are any data comparing results of people who

participated in this program and those who did not?

GAME A: NO. 2

FALLACY 3

Vague Description of Problem and Outcome. What is a "chemical dependency"? There is

no mention of a specific outcome measure, specific observable behavior, or measurement

procedure while claiming to know that the client improved.

Countermeasures. Clearly describe hoped-for outcomes (what, where, when, who, how

often). Client concerns and related outcomes should be so clearly described that different

people agree on their occurrence.

What you could say. Could you give me an example of what you mean by "chemical

dependency problem"? Could you tell me what you mean by "co-dependency"?

GAME A: NO.3 *

FALLACY 5

Confirmation bias: only successes are focused on. This is an example of partiality in the

use of evidence. You look only for a piece of a puzzle. This will not yield sound

inferences.

2

Countermeasures. To draw a valid conclusion about the agency's success in placing

children, you must know how many were placed. In addition to considering successes,

look for failures, persons not treated who got better, and those not treated who got

worse. Decrease reliance on memory. Keep records of successes, failures, those not

treated and improved, and those not treated and not improved. The latter two groups

might be estimated by reading literature about what generally happens to untreated

persons. It would also be valuable to know the proportion of successful placements made

by other agencies and by those placed through other means (e.g., attorneys). Look

fearlessly at all the evidence, not just the data that support a hypothesis (i.e., cases where

the treatment worked). How else can an accurate judgment be made?

What you could say. What we should find out is our rate of success. If we tried to place

100 children over the past two years and only successfully placed 20 during that time,

this would be a yearly success rate of 20% -not very impressive.

GAME A: NO.4

FALLACY 1

Case Example. Nothing can be said with any confidence about how widespread welfare

cheating might be based on one case, vivid or not. Such vivid cases often appear in news

stories.

Countermeasures. To make accurate generalizations, collect a random sample from a

population that is large enough to offer an accurate guide to what is true of the

population. Several methods may be used to gather this information including

observations and surveys.

What you could say. I don't think one case example tells us much. We should find out

what percentage of all people in our state who are on welfare engage in fraud.

GAME A: NO.5

FALLACY 10

Post Hoc Fallacy (After This, Therefore, Because of This). (See also Fallacy 9.) Group

members' family feelings may change over time for reasons other than the treatment.

Families may seek help during a worse phase in their better-worse-better-worse-better

cycles.

Countermeasures. Think of other possible causes for client improvement, or

deterioration, before taking credit (or blame). For example, you may think that your

client acquired a new social skill as a result of your program, but your client may have

matured naturally into better social skills. You may believe that cognitive-behavioral

therapy helped a depressed, middle-aged male client, but the client may have improved

because he saw a psychiatrist who prescribed an antidepressant. A break in very

hot weather may have been responsible for a decrease in street violence rather than your

3

community crisis team's efforts to head off violence. There are cyclical problems that get

worse, improve, and again get worse (Skrabanek & McCormick, 1998). Well-designed

studies can help to rule out these and other explanations for client change.

What you could say. (See answer to #1.) How do we know that the measure used (IFF) is

valid?

GAME A: NO.6

FALLACY 4

Softhearted, Therefore Softheaded. To assert that one is either a people-person or a

critical thinker ignores that one can (and should) be both. One might argue that

practitioners, who keep up with the related practice literature and who critically review

the quality of evidence and arguments, do so precisely because they do care enough

about clients to want to be as effective as possible.

Countermeasures. Be hardheaded (analytical, scientific, datafocused) because you are

softhearted (care about what helps people).

What you could say. This implies that if you consult practice related research, you cannot

also have "people skills." This is "either/oring." You can do both.

GAME A: NO.7

FALLACY 5

Confirmation Bias. The author requested the library helpers to find evidence that supports

the effectiveness of family-based treatment. What were they to do with evidence that did

not support its effectiveness?

Countermeasures. Be skeptical of anyone who presents just one side of anything. The

world is not that simple. Seek and present multiple positions and diverse sources of data.

What you could say. What do other studies report? We should look at all relevant studies,

not just studies that support effectiveness.

GAME A: NO.8

FALLACY 10

Post Hoc Fallacy (After This, Therefore, Because of This). (See also Fallacies 1 and 9.)

Simply because improvement followed cognitive-behavioral therapy does not show that

the therapy was responsible. Other events might account for the improvement (e.g.,

taking antidepressant medication).

Countermeasures. (See those described in No. 5.)

4

What you could say. This change could be due to regression effects. (See Game C.) The

client was in the "severe range" at intake. He would tend to return to his "mean" score.

GAME A: NO. 9*

FALLACY 8

Oversimplifications. What is a "brain disorder?" What does "very anxious" mean?

(vagueness).

Countermeasures. Watch out for oversimplifications such as the above. What are other

possibilities here and other options?

What you could say. I wonder if you "jumped the gun." I wonder what is going on in

Mrs. A's life that may account for her anxiety.

GAME A: NO. 10

FALLACY 7

Appeal to unfounded authority. Unless Dr. Morrison has collected better data, the survey

data provides the best guide here.

Countermeasures. Compare the evidentiary status of different sources of data.

What you could say. Could you tell me if Dr. Morrison has collected or knows about

larger survey sets regarding service needs of Hmong families?

GAME A: NO. 11

NONFALLACY ITEM

Nonfallacy Item. The sources of opinions are described; the source offers relevant

evidence for the conclusions, and the conclusions drawn do not go beyond the data.

What you could say. Let's take a close look at the study design. We have a comparison of

two treatments. Were subjects (Ss) randomly assigned? If not, this is a problem. What did

comparisons receive? Is a measure of attitudes a valid reflection of behavior (e.g.,

attempted suicide)? There are problems with this report. However, it's better than no data

at all about the program.

GAME A: NO. 12

FALLACY 4

Softhearted, Therefore Softheaded. To assert that you can either apply scientific thinking

and use valid measures for client progress or treat clients with respect implies that one

can't be both.

5

Countermeasure. Be hardheaded (analytical, scientific, data focused) because you are

softhearted (care about what helps people).

What you could say. I've seen studies that show that clients like concrete feedback about

progress. For example, Campbell (1988) interviewed clients and found that they valued

the concrete feedback from single-subject data more than the practitioner's opinion about

client progress. Also, critical appraisals of interventions show that intuition and good

intentions are not a good guide to degree of progress (e. g.., see Jacobson, Foxx, &

Mulick, 2005).

GAME A: NO. 13

FALLACY 5

Confirmation bias -- Focusing on successes only. To draw a valid conclusion about the

Department's job placement rate, they would need to examine the proportion of students

in the class who got jobs. If there were 60 students in the class and 30 obtained jobs, only

half would be employed.

Countermeasures. Look at all the evidence. Keep records of successes and failures.

What you could say. How many students graduated? If we knew, we could figure out the

percentage now employed in social work.

GAME A: NO. 14

FALLACY 6

Newness/Tradition. The speech therapist asserts that facilitated communication should be

used simply because it is new. All methods have been new at one time (those that were

later shown to be dismal failures as well as those shown to be effective

Countermeasures. Point out that being new or old does not make an idea about practice

valid. Ask to see evidence and data to judge effects of methods.

What you could say. Is there any evidence (e.g., experimental studies involving random

assignment) showing that facilitated communication is effective?

GAME A: NO. 15

FALLACY 1

Case Examples. A case example was presented by the drug company to support its claim

that the drug is effective. They would probably not present a client who did not improve.

Think of the possibility that there may be many unsuccessful cases. Case examples do not

provide support for claims of effectiveness.

Post Hoc (After This, Therefore, Because of This). The client's depression might have

been cyclical, passing through acute, less acute, phases; so when the medication was

6

stopped the client might have improved anyway. The client's marriage or some other

feature of his life might have changed.

Countermeasures. Search for experimental studies. This is not to say that case material

cannot be a vital part of professional education. Case material can be used to demonstrate

practice skills and apply them. For example, a videotape of an interview with an adoles-

cent mother may demonstrate interview techniques. A field instructor may model a

technique for family therapy. Such use of case material constitutes a valuable part of

social work education. The problem arises when generalizations to all clients are made

from case examples.

What you could say. One case example doesn't tell us anything about whether the relief in

depression was due to Wellbutrin. Also, many key items are vague (erectile failure,

decreased libido, feeling more energetic, sexual performance is normal).

GAME A: NO. 16

FALLACY 7

Appeal to unfounded authority -- Uncritical documentation. Simply giving a reference

does not cast any light on a method's success. This reference might consist purely of case

examples, testimonials, and biased presentations by advertisers who seek to profit from

functional communication training. You don't know because nothing is said about how

the authors drew their

conclusion. Seek out a high quality review of related studies (e.g., Jacobson, Foxx, &

Mulick, 2005).

Countermeasures. If you want to assess the accuracy of an assertion that is merely

referenced, you'll need to read that reference for yourself.

What you could say. Could you give me some details about their study? This would help

me to evaluate the basis for their claim.

GAME A: NO. 17

FALLACY 3

Vague Descriptions of Outcomes. How was self-esteem measured? How is its level

determined? Without greater clarity, we can't tell if there has been improvement.

Countermeasures. Clearly describe desired outcomes. Descriptions of outcomes should

be so clearly stated that we can agree on when they have been attained. They should

answer the questions what, when, who, how often?

What you could say. Could you clarify what you mean by "generally their self-esteem

improves"? What does "generally" mean? How was self-esteem measured? Is this

measure valid? What do we know about changes in self-esteem of women who did not

attend the program?

7

GAME A: NO. 18

FALLACY 7

Appeal to unfounded authority - Uncritical documentation. Citing a reference tells us

nothing about the method by which the authors drew their conclusion. For all we know,

this reference may have nothing to do with learning transfer.

Countermeasures. If you want to assess the accuracy of an assertion that is merely

referenced, you'll need to read it for yourself.

What you could say. Could you give me some details about their study? This would help

me to evaluate the basis for their claim.

GAME A: NO. 19

FALLACY 6

Newness/Tradition. The child welfare worker asserts that open adoptions should be tried

because the practice is new. No data are described regarding this method's success.

Countermeasures. Point out that being new or old does not make an idea or practice

valid. Ask to see evidence and data to judge effects of methods.

What you could say. Is there any evidence (e.g., experimental studies involving random

assignment).

GAME A: NO. 20

FALLACY 5

Oversimplification: there could e many reasons why people use controlled substances.

Simply because improvement followed treatment does not mean that the chiropractor’s

efforts led to change. Many lower back problems are self-limiting. They come and go

without any intervention.

Countermeasures. (See No. 9.)

What you could say: What other circumstances may encourage substance abuse?

1

__________________________________________________________________

Exercise 12: REASONING-IN-PRACTICE GAME B: GROUP AND

INTERPERSONAL DYNAMICS

__________________________________________________________________

OVERVIEW

Students work together in small groups to Identify fallacies in brief vignettes. The vignettes

concern the following contexts: special education, mental health, child welfare, corrections,

administration, health, substance abuse, education, geriatrics, and juvenile court.

PURPOSE

1. To have fun.

2. To learn how to identify and avoid fallacies that occur in case conferences, group meetings,

interdisciplinary teams, and staffings.

MATERIALS REQUIRED

Same as for Exercise 11.

TIME REQUIRED

Same as for Exercise 11.

SUGGESTIONS

Same as for Practice Exercise 11. Please note that the majority of the vignettes in Game B are

starred (*). These items have parts to be acted out. We think this adds realism to a game that

concerns group dynamics. Try to encourage interaction, even in nonstarred items, by having

different people read the vignette.

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

Do any of this game's vignettes reflect real situations particularly well? Which one(s)?

We suggest that you encourage participants to recall their experiences in groups that seek to

accomplish a task (e.g., case conference, political action, interdisciplinary team, and planning).

2

Answers to Exercise 7: Game B

______________________________________________________________________________

GAME B: NO.1 *

FALLACY 6

Groupthink. This is groupthink because there was no serious discussion of the views of

the special education teacher. This denied the group useful information.

Countermeasures. Janis (1982, pp. 262-265) suggests three ways to counter Groupthink

include the following: (1) have the leader assign the role of critical evaluator to some of

its members; (2) make it clear at the beginning of an important discussion that the leader

will be impartial to the group's decision; and (3) for important decisions, set up

independent committee to gather evidence and deliberate independently of other groups.

Each committee should be led by a different person.

What you could do: Discuss how the team could encourage serious discussion of

dissenting points of view.

GAME B: NO. 2*

FALLACY 10

Slippery-slope: Assuming that giving money to one group of necessity requires giving

more money to many more groups when it does not require this.

Countermeasures. Note the lack of necessity.

What you could say. If we have good reason to increase funds to the elderly we do not

have to justify giving more funds to other groups. After all, we are a service for the

elderly.

GAME B: NO.3

FALLACY 2

Begging the question: No additional information is given.

Countermeasure: Note the circular nature of the reply.

What you could say. But what is “order”?

GAME B: NO.4

FALLACY 1

At the Person (Ad Hominem). Diana presented A 15-year-old report which should have

been updated (what does current literature report). But, rather than note this point, Lizzie

and Polly called her a "know-it-all."

3

Countermeasures. When making decisions, we should focus on our goals and on

examination of related evidence. This involves arguing at the issue, or at the thing (Ad

Rem).

What you could say.

Sandra: Reliability is important but we should search for more recent data. And, even if

we can reliably rate BPD, is this a valid diagnosis? Does it help us to understand the

etiology of related behaviors and to select effective intervention methods?

GAME B: NO. 5*

FALLACY 3

Diversion (Red Herring). The psychologist raised a vital question about the client's

suicide potential, but the discussion was sidetracked by talking about a joke. Hopefully,

the group got back to the subject.

Countermeasures. Gently bring the discussion back to the point at issue (e.g., We were

just talking about …).

What you could say.

Clinical Psychologist. The reason I am worried about Mr. Meyer's potential for suicide is

that the last two times his depression lifted, he did make a suicide gesture. (She returns

to her concern.)

GAME B: NO. 6

NONFALLACY ITEM 11

Nonfallacy: The supervisor praises the worker for sharing this information and suggests

next steps to consider more carefully the exact practices/policies questioned.

GAME B: NO. 7

FALLACY 7

Stereotyping. The bottom line in judging any method is, Does it work? The popularity of

a method does not mean it is effective.

Countermeasures. Accurately describe behavioral methods (e. g., Gambrill, 2013).

What you could say. Manipulation refers to controlling people in unfair ways.

Behavioral methods such as constructing alternative repertoires that compete with

undesired ones, are used to help participants to enhance the quality of their lives.

GAME B: NO.8*

FALLACY 8

4

Either-Or. Believing that there are only two choices (file a formal complaint, initiate a

full investigation/do not initiate investigation) may limit the care to this family, possibly

overlooking alternatives such as vocational counseling for the father and/or mother,

referral to income maintenance programs, and referral to the Veterans Administration.

Countermeasures. Identify several options about what might be done. Ask each group

member to independently identify possible courses of action. Assure them that whatever

they write will be read anonymously and discussed seriously.

What you could say. We have hearsay evidence from a friend of one of the neighbors.

This is shaky. However, we are obliged to investigate all allegations of child abuse.

Why don't we get in touch with this neighbor? I think we have to do something.

GAME B: NO. 9*

FALLACY 2

Begging the question. No additional information is given.

Countermeasures. Note the circular nature of the reply.

What you could say.

Jenny: Could there be other reasons for this behavior? Do we know the antecedents and

consequences related to his behavior? And could you give me examples of "erratic

behavior"?

GAME B: NO. 10

FALLACY 4

Stereotyping. The female workers are prejudging a job applicant without viewing his

work performance, qualifications, references, and experience because they have an

opinion about all males. If they attacked the candidate individually by calling him a

name, or by bringing up an irrelevant or untrue argument against him, they would be

arguing At the Person (Ad Hominem).

Countermeasures. Evaluate each person's competencies without relying on preconceived

notions based on their membership in a group or class. Stereotyping fosters racism,

sexism, classism, and ageism (among other "isms").

What you could say. It's easy to fall into making decisions based on gender, ethnicity,

class or race. However we have to be careful not to do that and try to recognize our

biases. We have no evidence that male social workers don't do as well in hospital social

work as women. I think we have to be careful not to impose unfair and inaccurate

stereotypes on job candidates.

5

GAME B: NO. 11*

FALLACY 9

Straw Man Argument. Richard has argued for matching alcoholics with the most

appropriate treatment, not that all alcoholics are unique. Onesmo examines the idea that

all alcoholics are unique. Both people should stay with the original topic: Should

alcoholics be matched systematically with their most effective treatment? Onesmo might

have avoided the red herring if he had asked? "Are you arguing that all alcoholics are

unique?"

Countermeasures. When discussing vital issues, carefully listen to the other person's

position. Restate the position in your own words as accurately as you can and ask

whether you have restated it accurately and if so, then react.

What you could say.

Richard: The more specific we can be about what works with which particular kinds of

individuals, the better chance we have of matching intervention recommendations to

specific people. I haven't seen any data showing that all people experience the same

physiological symptoms of withdrawal after they have build up a tolerance for alcohol.

Anyway, I don‟t' think this is the key point in relation to matching people with

treatments. Have I missed something?

GAME B: NO. 12

FALLACY 7

Bandwagon. Professionals can be wrong. Turning to a new method may be premature

in light of later evidence. Examine the evidence before adopting a new method.

Countermeasures. Critically appraise any popular notion, particularly if it concerns

treatment. Search for related studies, including those that provide counterevidence and

counterarguments before you decide.

What you could say. If we did classify more children into this category, what effect

would it have? Would it help us to select effective services?

GAME B: No. 13*

FALLACY 1

At the Person (Ad Hominem). The social work trainee described referred to reports about

how to help residents with Parkinson's Disease; the nurse described the social worker as

naive. This was an attack at the person, not an examination of the argument presented.

Countermeasures. When making decisions about client care, we should seek and

critically appraise related evidence. This involves arguing at the issue, or at the thing (Ad

Rem). A standard for judging such evidence can be found in the Quality of Study Rating

Form, or by playing the RIP Game to see if someone's argument avoids fallacies.

6

What you could say. I think we have to consider the results of this study, especially if it

was well designed and shows that our methods are not as effective as other methods.

Lois: Could you bring this study in so we can read it?

GAME B: NO. 14*

NONFALLACY ITEM

Not a Fallacy. The presenter has made claims for effectiveness of his method but

provided no related evidence. The social workers raised questions regarding the

evidentiary status of claims made. (See vignettes for example of what they might ask.)

GAME B: NO. 15*

FALLACY 5

Manner or style. A sincere, warm, entertaining way of presenting an idea can encourage

professional listeners to pay attention, but the manner or style of the presenter does not

assure the accuracy of the presenter's ideas. Examine the evidence when thinking about

client care, in spite of distractions. One type of stylistic distraction is someone whose

mannerisms offend but whose ideas are sound.

Countermeasures. In matters of importance involving client care, base judgments and

decisions on the evidence presented, not on the speaker's style or lack of it, even if the

idea comes from an "odd" person. The utility and soundness of the person's argument is

what matters.

What you could say. That may be, but I'd really like to find out if there's any evidence

that validation therapy really helps the client to become more oriented to time, place, and

person. Mr. Diggelman may be a nice person. The question is, Does validation therapy

work? Will it help our clients?

GAME B: NO. 16*

FALLACY 3

Diversion (Red Herring). Instead of replying to the supervisor's concern the worker goes

off on a tangent.

Countermeasures. Focus on the issue at hand -- this worker came in late.

What you could say. My concern here is you being late. This is the second time this

week you have been late.

GAME B: NO. 17

FALLACY 5

7

Manner or Style. Ms. Carey„s confident manner may distract attention from the accuracy

of what she says.

Countermeasures. Base judgments and decisions on evidence presented, not on the

speaker's style or lack of it. The utility and soundness of the person's

argument is what matters.

GAME B: NO. 18*

FALLACY 8

Either-Or. This form of reasoning narrows the range of services that could help the

family, including temporary housing with a church member, referral to income

maintenance, help finding a job, and so on.

Countermeasures. Consider several working hypotheses about what might be done. Each

group member could independently write down possible courses of action. Assure them

that whatever they write will be read anonymously and discussed seriously.

What you could say.

Judge Calhoun: Couldn't there be another alternative? How about going out and talking

further with the family?

GAME B: NO. 19*

FALLACY 8

Either-Or. This form of thinking limits the decision to one of only two alternatives.

Countermeasures. Consider several options about what might be done. Ask each group

member to independently write down possible courses of action. Assure them that

whatever they write will be read anonymously and discussed seriously.

What you could say.

Jonathan: I don't think it's an either-or matter. There maybe some other options we can

pursue.

GAME B: NO. 20*

FALLACY 9

Straw Man Argument. Mike misrepresents Antonio's argument and then attacks the

misrepresentation. Antonio says there may be unintentional cues that can shape a child's

testimony and wonders if this happened; Mike says that Antonio is implying he would

intentionally mislead a child to give false testimony; Mike then attacks this idea.

Reflecting, or restating, the other person's argument for accuracy before reacting to it can

help avoid a Straw Man Argument.

8

Countermeasures. When discussing vital issues, carefully listen to the other person's

position. Restate the position in your own words as accurately as you can. Ask if you

have restated it accurately, and then react.

.

What you could say.

Antonio: I did not say anything about your intentionally misleading a child into giving

false testimony. What I did say is that research suggests that regardless of our intentions

we may lead witnesses, especially children, into saying that are certain things that are not

accurate.

GAME B: NO. 21

FALLACY 7

Bandwagon. Increasing use of a method does not mean it is effective. Only evidence

regarding its effectiveness for particular lessons, with particular students,

in relation to particular learning goals can answer the question of effectiveness.

Countermeasures. Critically appraise claims. Examine counterevidence and

counterarguments as well as supporting evidence before you decide.

What you could say.

Faculty member: Let's find out if problem-based learning is effective in enhancing

clinical reasoning skills on-the-job. Is there a systematic review of related research?

1

_________________________________________________________________________

Exercise 13: REASONING-IN-PRACTICE GAME C: COGNITIVE BIASES _________________________________________________________________________

OVERVIEW

Students work together in small groups to identify cognitive biases described in the literature on

clinical reasoning, decision-making, decision analysis, and judgment. A few of the best

understood and most commonly described biases are the subject of interest in this game. (Please

see related references in the literature.) Vignettes come from the following disciplines: special

education, medicine, social work, nursing, research methodology, psychology, chemical

dependency counseling, and corrections.

PURPOSE

1. To have fun.

2. To learn how to identify and to avoid common errors in reasoning that influence judgments

and decisions.

MATERIALS REQUIRED

,

Same as for Exercise 11.

TIME REQUIRED

Same as for Exercise 11.

SUGGESTIONS

Same as for Exercise 11.

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

Do any of the game's vignettes reflect real situations particularly well?

Which one(s)?

We suggest that participants reflect on their practice experience or as recipients of services.

2

______________________________________________________________________________

Answers to Exercise 13 GAME C:

______________________________________________________________________________

GAME C: NO.1 *

FALLACY 1

Hindsight Bias is evident in the special education teachers' looking back and recalling

successful placements. A true evaluation of predictive accuracy would be to examine all

records for cases where judgments to place or not to place, were made.

Countermeasures. (1) Be mindful that there is a tendency, when looking back, to

overestimate the accuracy of your predictions; and (2) keep your own records

prospectively (before the fact) of your predictions, and consult studies that assess risk

(i.e., make predictions, record predictions, and then see what happened). These

countermeasures were suggested by Hogarth (1987, p. 149).

What you could say. I think we have to look at the group as a whole rather than taking

individual examples. Doing so will let us see what our track record has been so far and

how we can improve it.

GAME C: NO. 2*

FALLACY 5

Overlooking Regression Effects. The lowest five percent may have had a particularly bad

day at the time of the pretest because they were ill, emotionally stressed, or for some

other reason; therefore, on posttest, their scores would regress upward toward the mean at

posttest.

Countermeasure. Be skeptical of studies that single out an extreme group for

observation. Ideally, submit half of the extreme group to intervention, the other half to a

control or alternate intervention; then posttest both groups for comparison.

What you could say.

Second Child Care Worker: I don't think we can assume that this shows that our

enrichment program helps school kids. We would expect those who scored low to score

higher on the next testing; they would approach their "average level." We would have to

conduct an experiment (e.g., randomly distribute children to two different programs, one

which had the enrichment program, and one which did not) and then see if there are any

differences in outcomes between these two groups. Or, maybe we could carry out a

number of experimental single-case studies to discover whether the enrichment program

really is effective in delivering what it promises.

3

GAME C: NO.3 *

FALLACY 3

Framing Effects. The probabilities are identical for the two procedures; the patient is

drawn to surgery because it is described in terms of likelihood of success.

Countermeasure. Describe both negative and positive consequences for all alternatives.

What you could say.

Doctor: Before you make a decision about what to do, I think I should point out these

two estimates are the same.

Patient: What do you mean?

Doctor: In the first case, 25% of the time the operation is not a success; and the same is

true with cortisone.

GAME C: NO. 4*

FALLACY 2

Fundamental Attribution Error. Joe does not recognize environmental influences on

behavior, and so he misses the chance to intervene at the community level to help

children.

Countermeasure. Be informed about how the environment influences behavior and help

clients to alleviate environmental stress and/or learn to cope with it.

What you could say.

Maria: I agree that there are many factors that influence school performance, including

the extent to which parents support education, the quality of neighborhoods, and cultural

and ethnic norms and values. However, levels of ingested lead correlate with reduced

educational performance. I think we have to consider this and try to do something about

it. This doesn't mean other factors are not important.

GAME C: NO.5

FALLACY 4

Overconfidence. No evidence is provided for this claim.

Countermeasure. The psychologist should make predictions and keep track of which

ones turn out to be accurate.

GAME C: NO.6

NONFALLACY ITEM

Not a Fallacy. The nurse administrator has formed an opinion based on data and explains

why.

4

What you could say.

Listener: Well, this information is helpful. Thank you for finding this report. Do you

know if there are other studies that bear on this decision, especially ones that may have

contradictory findings?

GAME C: NO. 7*

FALLACY 1

Hindsight Bias Penny did not keep a record of her predictions ahead of time to test their

accuracy; and even now she cannot state specifically what her indicator of high risk

might be.

Countermeasures. (See No. 1)

What you could say.

Maureen: It's always easy to say "I knew it" in retrospect. What is difficult is to predict

what will happen in advance. If you really want to test how accurate your predictions are,

make them right after the first group meeting and test their accuracy by seeing what

happens over a number of people.

GAME C: NO.8

FALLACY 3

Framing Effects. If one question emphasizes a possible positive outcome, and another

emphasizes a possible negative consequence, the client will probably choose the former.

Countermeasure. Note that the 10% and 90% are inter-related.

GAME C: NO.9

FALLACY 6

The Law of Small Numbers. A single case, with which the social worker is familiar, may

not be representative of most cases. (See also discussion of case examples in Exercise 6.)

Countermeasures. Give more weight to conclusions based on large numbers of

representative, carefully studied clients, and less weight to intimate experience with just a

few clients.

What you could say.

Listener: I don't think one example tells us as much about the population at large as does

a study of 50 people. A particular student may be quite atypical. We have to consider the

size of the sample and how it was selected in deciding how informative it may be in

relation to the population of all students.

5

GAME C: NO. 10

FALLACY 5

Overlooking Regression Effects. The problem is not that the top ten relaxed, but that their

initial scores might have overestimated their ability due to chance factors like lucky

guessing, or having particular knowledge that most reflected the test's first form;

therefore, on retesting, these ten were likely to test out lower

Countermeasures. See No. 2.

What you could say.

Listener: I think what we have here is regression toward the mean. The people who

scored high on the test would be expected to score lower the second time around.

Therefore I don't think the changes in test scores can be attributed to relaxing during the

training and not paying attention.

GAME C: NO. 11 *

FALLACY 4

Overconfidence: Monitoring progress contributes to positive outcome; self reports of

clients may be inaccurate and should be compared to data gathered via self monitoring or

observation.

Countermeasure. Be informed about important practice issues.

GAME C: NO. 12

FALLACY 9

Gambler's Fallacy. Each trip is independent of the others; so on the next trip the success

of the trip is not more nor less certain than for those previously.

Countermeasure. For truly independent events (a fair coin, birth of boy or girl in a given

hospital), what happened on the previous trial cannot affect the next in the series. Aside

from clinical reasoning, the gambler's fallacy is an important one to understand now that

gambling has been legalized in many states. No matter how many times you enter the

lottery, your chances of winning the next time you play will be the one in one million (or

whatever odds are stated on the ticket) no matter how many times you have played in the

past. This is something to understand and to convey to clients, because many spend

money that they can ill afford on gambling.

What you could say.

Listener: You mention that three staff from your office have gotten through with backup

support and only minor confrontations. What about others? What's the base rate of

confrontations?

GAME C: NO. 13*

6

FALLACY 7

Ignoring Prevalence Rate. The protective service worker ignores the fact that the

prevalence rate for abuse among families in the community would be much lower than

would be the reabuse rate for protective service clients, who have most likely already

abused a child. In the former; there would be many false positives (abuse predicted but

none found).

Countermeasure. Though the mathematics to show why this is true can take up a whole

chapter (Gibbs, 1991, p. 217-237), the implications of base rate (proportion of the group

who have a given characteristic or problem that we want to identify) cannot be ignored.

In the first situation (protective service clients), where those to be identified constitute a

large proportion with the characteristic (child abuse), the identification job is easier. In

the second, where the same characteristic is used to try to identify a very few individuals

among a much larger group of individuals (all families in the community), the assessment

and identification task is much harder. In the latter situation, low base rate, there will be

many more false positives (persons judged to have the problem).

What you could say.

Researcher: Well I don't think we could do this because the population is different. The

90% rate involves an identified population (protective service clients). If we apply this to

all families in a community in which the base rate is much lower, you will get a high

false positive rate and nowhere near the 90% rate obtained with the identified population.

This could result in many false allegations.

GAME C: NO. 14

FALLACY 11

Availability. This represents an actual case from the medical literature, whereby a

surgeon did not think of a possibility outside the realm of surgical experience-namely,

that the patient's problem could be a psychiatric one.

Countermeasure. Try to think of alternatives that do not come to mind readily. Consult,

where possible, surveys that record the relative frequencies of events. (See Arkes, 1981,

p. 323.)

What you could say.

Listener: You've been very clear and candid about what you've done and not done. This

will help me remember to consider psychological causes of physical complaints.

GAME C: NO. 15

FALLACY 10

Anchoring and Insufficient Adjustment. The social worker would make better judgments

if she kept an open mind while doing an assessment until sufficient information was

7

obtained to make a judgment

Countermeasures. Arkes (1981, p. 327) suggests using strategies to encourage alternative

hypotheses about a client. For example, you may entertain several hypotheses.

What you could say.

Listener: You might be unduly influenced by the course you just completed? Anchoring

effects (what we think first) influence what we then do or see. I try to suspend my initial

assumptions temporarily and look for information that supports alternative views so I

don't overlook something important. When I concentrate on my first impressions without

questioning them, I'm likely to miss things.

GAME C: NO. 16

FALLACY ITEM 11

Affective Bias: Here the emotions of a professional influence reactions toward clients.

Countermeasures: Attend to your emotions: Have you had an upsetting experience that

could affect your work? If so, use your emotion management skill to calm down.

GAME C: NO. 17

FALLACY 10

Anchoring and Insufficient Adjustment. This hospital social worker prematurely closes

off channels of information that may aid decision making. In this case, additional

information might come to light that is not in the chart. The client's condition and prefer-

ences may have changed since the last entry in the chart.

Countermeasures. Arkes (1981, p. 327) suggests using strategies to ensure alternative

hypotheses about a client. For example, one might resolve, when beginning a group

meeting, to entertain several hypotheses about what may be the group's principal interest

at the meeting. Resolve not to formulate an opinion until each member of the group has

had a chance to speak.

What you could say.

Listener: I find that information I need to consider possible options is often missing on

the hospital chart. Some of my clients, for example, can return to live with relatives.

GAME C: NO. 18*

FALLACY 6

The Law of Small Numbers. The vivid and recent experience with a single case does not

carry as much power as does a larger sample of individuals who participated in an

evaluation study.

8

Countermeasure. Give more weight to conclusions based on large numbers of

representative, carefully studied clients, and less weight to intimate experience with just a

few clients.

What you could say.

First Child Psychologist: I can see how this is a vivid case example, but I don't think it

takes away from the findings of the research study I described. First, we don't know if

this psychologist implemented paired reading appropriately. Also, it may not work with

all children but may work with most. I think we have to look at its overall ''batting

average."

GAME C: NO. 19

FALLACY 9

Gambler's Fallacy. Because the lives of offenders on a probation officer's caseload

progress pretty much independently of each other, the fact that there is a run of recidivists

does not imply that the next case is more or less likely to recidivate than those before.

Countermeasures. See Item 12.

What you could say.

Second Probation Officer: I'm not sure you could say this. Each of these individuals is

independent; what happens to one is not related to what happens to another. The

probability that the next one will be a success is no higher than the probability that the

next one will be a failure.

GAME C: NO. 20

FALLACY 8

Omission bias: This refers to viewing acts of omission such as failure to use effective

methods as less concerning than acts of commission, such as use of a harmful method. It

is important to keep track of both omissions and commissions.

Countermeasure: Keep track of both lost opportunities to help clients by offering

effective methods as well as occasions in which clients are harmed by use of methods.

GAME C: NO. 21*

FALLACY 7

Ignoring Prevalence Rate. The community probation officer's clients are at lower risk

(lower base rate) than those released from the maximum security prison. That is, there is

a higher proportion of recidivists in the prison than among those on probation in the

community. Consequently, there might be a high false positive rate for predictions on the

measure (recidivism predicted but does not happen) in the probation officer's caseload.

9

Countermeasures. See Item 13.

What you could say.

Corrections Policy Maker: I don't think that would be a good idea because the two

groups are different. (See Item 13.)

___________________________________________________________________________

Exercise 14: PREPARING A FALLACIES FILM FESTIVAL

________________________________________________________________

OVERVIEW

Based on examples and literature from the RIP Games, the Professional Thinking Form, and

other literature, participants write a two page paper that defines a chosen fallacy, describes how

to avoid that fallacy, and includes an original 30-to-60-second script for a vignette. They then act

out their vignette before a small group while being videotaped. These vignettes are edited and

then shown to an entire class whose members try to name the fallacy. They can be posted on

YouTube. This is one of the exercises students enjoy the most.

PURPOSE

1. To familiarize teams with a practice fallacy.

2. To help the whole class learn more about fallacies.

MATERIALS REQUIRED

1. List of Practice Fallacies and Pitfalls (Practice Exercise 14).

2. Access to a library containing related material.

3. Videotape camera (your university might have an instructional media center whose experts

could videotape the students' vignettes and edit them).

4. VHS videotape player.

5. Television monitor.

6. Props (to add realism to the vignettes).

7. A room for videotaping free from outside noises and interruptions. (We use a university

classroom and a lot of imagination.)

TIME REQUIRED

Thirty minutes from one class period to practice acting the vignettes, one class period to

videotape the vignettes and one class period to show the Fallacies Film Festival and to give

awards.

SUGGESTIONS

We find that students enjoy putting the fallacy numbers from the Practice Fallacies and Pitfalls

list on slips of paper, folding them up, and then picking one out of a hat that identifies the fallacy

for their assignment. Students are assigned to two-member teams according to where they are

seated in the class. If there is an odd number, one group can have three members. Practicing the

vignettes the week before videotaping makes them go more smoothly, but still, actors need a

practice run the day of videotaping to warm up. The taping works best before a live but quiet

2

audience. A microphone can be used to compensate for soft voices, but actors should avoid

rustling their scripts next to it. Visual aids should be brief, large, and legible.

In preparation for the festival, your university's Media Development Department can edit and

splice the vignettes from all the labs into one tape. This tape includes the vignette itself, a five-

second break, and a caption that gives the title of the fallacy. During the festival we ask the class

to guess the fallacy being portrayed before showing them the title. The Festival works best if

most of the class doesn't know ahead of time which fallacy is being portrayed. Multiple sections

of a class can meet for a Fallacies Film Festival to watch and to guess all fallacies. The videotape

can be stopped after each vignette for students to guess the fallacy and discuss their reasons.

As the vignettes are shown during the Fallacies Film Festival, the fallacy names and the names

of the actors are noted. When the last vignette has been shown, a secret ballot among the class is

taken to see which vignette wins a prize. At the beginning you might want to note the following

rating criteria on the blackboard:

The vignette accurately and clearly depicted the fallacy yes _ no _

The presentation used a costume yes _ no _

The presentation used special effects yes _ no _

The presentation used other props yes _ no _

You guessed the fallacy's name yes _ no _

The vignette was brief (about 1 min.) yes _ no _

The actors were enthusiastic yes _ no _

Prizes have included a book about critical thinking for each student. Showing students the

awards before the competition may help to motivate them.

Grading Suggestions

We grade the paper and the vignette together for each team. Students' assignments describe the

following criteria for grading:

Your grade will be based on your fallacy's documentation, clarity of its definition,

appropriateness of description of how to avoid the fallacy, accuracy and clarity of the

script's demonstration of the fallacy, realism (e.g., use of visuals and props), and

entertainment value of your portrayal of the fallacy.

Documentation could be evaluated on the basis of appropriate references noted, including

references not included in the Workbook. In addition, grades could be based on a numerical

score reflecting (1) how accurately and clearly they define the fallacy, (2) the percentage of

students who successfully identified it in class, and (3) how clearly and accurately they describe

how to avoid the fallacy. You could add a score for the percentage of votes the vignette received

as "best" in the Fallacies Film Festival, or for how well the actors portrayed the fallacy and

entertained the group while doing so. The grade could be based on the total of these scores.

3

POSSIBLE ANSWERS TO FOLLOW-UP QUESTION

What have you learned from this exercise?

Suggested Lessons

1. These fallacies occur in the helping professions.

2. Illustrating the fallacies brings them to life.

3. One fallacy is often mixed with others.

1

__________________________________________________________________

Exercise 15: FALLACY SPOTTING IN PROFESSIONAL CONTEXTS

__________________________________________________________________

OVERVIEW

Students select a quote from a source (e.g., Internet journal, class lecture, book) that they think

demonstrates a fallacy that is relevant to the helping professions. In writing, they give the full

quote and source, name and define the fallacy, and describe why they believe this represents a

fallacy. This exercise provides opportunities to highlight the consequences for clients of

accepting claims based on faulty reasoning. You may use it more than once during a term. We

collect good examples students provide for illustrative and practice purposes. Tie this exercise to

a discussion of Walton’s definition of fallacy in Exercise 30.

PURPOSE

To hone skills in spotting fallacies in professional contexts. This exercise provides an

opportunity for students to dip into the professional literature, relevant sources on the Internet,

and related articles in newspapers and practice critiquing content.

MATERIALS REQUIRED

1. Fallacy Spotting in Professional Contexts form (Exhibit 15.1).

2. Definitions for fallacies in Exercises 11, 12 and 13.

3. Access to a library containing professional journals and books.

4. Description of APA citation guidelines.

TIME REQUIRED

Students may spend an hour locating a fallacy. Many attach a copy of the relevant material to the

fallacy spotting form. The exact source of the material and data (e.g., website) must be noted.

Students can save time by highlighting key material with a color liner. Additional time is

required to review definitions of relevant fallacies, and complete the fallacy spotting form.

Lastly, discussion of each fallacy will occupy additional time. About five minutes may be given

to each student to describe or read the case material, define the fallacy, and describe how the

fallacy applies to the example selected.

SUGGESTIONS

We require students to seek recent material (2010 to the present). Students can use quotes from

lectures. We encourage students to critique our own writing and class lectures. We stress that it

must be an exact quote and be placed in context. It is not necessary to identify the instructor

(who may be you). You could ask students to duplicate their completed fallacy spotting form and

2

give a copy to other students. This material can be used as the basis for critiques in small work

groups that allow each student to review many examples. Your task could be to provide feedback

as you circulate among the groups. Caution students to limit their selection to one key claim and

to include exact quotes and not take content out of context. Sometimes students pull out a large

section which has many arguments and do not identify what argument they are addressing. Have

students type their responses. It is easier to follow their reasoning.

In reviewing student reports, see if the critique is relevant to content quoted. Does the quote

illustrate a fallacy? Have they correctly identified the main fallacy illustrated? Have they clearly

described why this fallacy applies? Occasionally, students just write down the names of some

fallacy with no indication of how it applies to the quoted material (and in some cases they may

not apply at all to the example). Review the importance of understanding how the fallacy applies.

POSSIBLE ANSWERS TO FOLLOW-UP QUESTION

What have you learned from this exercise?

Suggested points to emphasize

1. Experienced, well-educated professionals can engage in faulty reasoning.

2. People who usually reason well can fall into faulty reasoning.

3. Errors in reasoning related to decisions can have serious implications for clients, so we

should cultivate accurate reasoning and practice spotting faulty reasoning to avoid errors.

____________________________________________________________________________

Exercise 16: AVOIDING GROUP THINK ____________________________________________________________________________

OVERVIEW

Professionals often participate in a variety of group meetings related to their work. Decisions in

these meetings may be compromised by a number of ploys designed to skew the decisions in a

direction that does not forward well-reasoned decisions.

PURPOSE

This exercise provides an opportunity for students to become aware of the phenomena of group

think and to learn to identify and avoid indicators of group think.

MATERIALS REQUIRED

1. Practice Exercise 11.

2. Access to group meeting

TIME REQUIRED

Depends on length of meeting observed.

SUGGESTIONS

The more realistic the demonstration of group think indicators and harmful consequences, the

better. The more practice in detecting and responding to group think tactics the better. Be sure

to give corrective feedback following practice opportunities.

REVIEWING STUDENTS’ WORK

Note the frequency of group think indicators. Which ones occur most often? Which remedies

did students select? Was there any change in the rate of group think indicators following the use

of remedies?

1

__________________________________________________________________

Exercise 17: APPLYING THE STEPS IN EVIDENCE-INFORMED

PRACTICE

__________________________________________________________________

OVERVIEW

Sackett, Richardson, Rosenberg and Haynes (1997) describe the process of EBP. (See also

Straus, et al., 2011; Gibbs, 2003; Gambrill, 2013). We also owe a debt to Gray (2001), author of

Evidence-Based Health Care.

Be prepared for objections to this exercise such as clinicians' workloads prohibit electronic

searching and the volume of material will overwhelm anyone who searches for answers to

practice questions. Electronic searches become ever easier over the years, as described in the

text and using methodological filters makes the process more rapid.

PURPOSE

To provide practice in carrying out the process of evidence-based practice.

MATERIALS REQUIRED

1. Copies of relevant Boxes in the exercise.

2. Access to a computer and printer.

3. Access to the World Wide Web.

4. Access to relevant databases.

TIME REQUIRED

We suggest a whole class. Ideally, multiple opportunities would be given over the class as well

as in other classes to search for answers to practice and policy questions.

SUGGESTIONS

Practice ways to apply methodological filters described in Exercise 17. Demonstrate searches in

real time regarding actual questions for all to see, as projected on a large screen. Then have

members of the class try their hand at searching for all to watch on the screen while being

critiqued by the search expert. Then provide guided experience for all students to sharpen search

skills. Students could draw on questions that arise in their field work.

2

We suggest that you buy and carefully read Evidence-Based Medicine (2011). Though the

examples concern medicine, good reasoning is good reasoning, and EBM's principles can and are

being adapted to apply to multiple areas.

REVIEWING STUDENTS’ WORK

What obstacles occur in carrying out the process? Keep track of these and help students acquire

relevant skills. Did students find the best evidence? What kind of search problems occurred?

_______________________________________________________________________________

Exercise 18: WORKING IN INTERDISCIPLINARY EVIDENCE-BASED

TEAMS

______________________________________________________________

OVERVIEW

Decision making often occurs in teams. Exercise 18 should be carried out by those who have a

background in applying the steps in the process of evidence-based practice.

PURPOSE

To enhance skills in working in evidence-based teams; applying the process of EBP in deciding

what to do in response to a case scenario. Grading criteria for this exercise may include the

following and are stated explicitly to students.

The team picked a leader who coordinated the team’s efforts.

Question identified a client/patient type.

Question identified a course of action (in this case an intervention).

Question identified an alternate course of action.

Question identified a desired outcome.

The search record included appropriate search terms.

The search record included appropriate methodological filters.

The search record included a list of databases searched.

The team applied the appropriate form for rating the study (e. g., PRISMA).

The team located a key resource (preferably a high-quality systematic review).

The team addressed the impact of intervention relative to number needed to treat or an

odds ratio.

This exercise could be used for an examination as well as for practice of related skills.

MATERIAL REQUIRED

1. Content in text for Exercise 18.

2. Computer with access to Internet with relevant databases.

TIME REQUIRED

In groups this exercise can be completed in half an hour. A single individual might be given an

hour.

SUGGESTIONS

2

Students in each group can divide up the databases and investigate different ones simultaneously.

Attached is a copy of a permission form that students could sign so that you could video their

work. This form was approved by attorneys at the University of Wisconsin System and by

attorneys at Brooks/Cole Thomson learning.

REVIEWING STUDENTS’ WORK

How could team interaction be improved? Was the best source found? How could search efforts

be improved?

3

INSTRUCTIONS GIVEN TO STUDENTS BEFORE COMPLETING THE EXERCISE

Dear Students,

Here are your teams:

Team 1 ___________ _____________ ___________ ____________

Team 2 ___________ _____________ ___________ ____________

Team 3 ___________ _____________ ___________ ____________

Suggestions for preparing for the team meeting:

Select a leader who will keep the group on track using the sequence of steps in the

process.

Select a person who will record your work to answer each question on the examination

sheet.

Select a timekeeper who will call off the time remaining to be sure you use an even

amount of time for reach question (30 minutes for each question, one hour total only).

Don’t get bogged down in interpersonal dynamics; stay focused on the task of answering

the question together as a team as best you can.

Review the process as described in Exercise 18.

Practice locating sources on different databases efficiently.

Refer to relevant methodological review forms (e. g., CONSORT) to guide critical

appraisal of best source found but do not get bogged down in reviewing quality of

research.

Your task is to locate the best (most rigorous and relevant source) to answer your

question. Depending on the question, this will be one of the following:

Systematic review or meta-analysis of randomized control trials

A high quality randomized controlled trial

Quasi-experimental study

Multiple single case (N of 1) studies.

Expert opinion

4

Release for Use of Written Materials Generated in EBP Workshop

By entering into a written materials release dated ___________ described here, I

_________________ (please print your name) consent to the use of written material (such as

COPES questions and search strategies) generated by me during a class or workshop presented

by _____________________ for the purpose of training students, professional social workers,

and others in the helping professions. I give my consent to the use of the written materials in

instructional materials, including textbooks, training manuals, and journal articles, prepared by

___________ for teaching evidence-based practice. My consent is given with the understanding

that ____________ may publish the instructional materials. Any copyrights, trademarks, or

patents will remain with ________________ or any third parties with whom he may contract.

I understand that my cooperation is purely voluntary. Neither providing nor withholding my

permission will affect by grade or the number of continuing education units I receive for

participating in the class/workshop in any way.

Signed ________________________________________ Date ______________

5

WORDING WHEN USED AS AN EXAMINATION

Team 1:______________ ___________ _____________ ____________

Team 2:______________ ___________ _____________ ____________

Team 3:______________ ___________ _____________ ____________

Have one person record the group’s work on this sheet making sure all sign the examination

sheet. Give this to your instructor with timesheet.

Background:

You have rudimentary knowledge of how to pose and answer questions to make well-reasoned

decisions. Apply the process we have studied to make your recommendations. Work as a team

to accomplish your task. Work through the problem in sequence, answering each question in

turn. (Work as a group on one question at a time. Budget your time and spend about half the

time on each question. You have one hour, no more.)

Question #1: Delinquency Prevention (30 minute limit).

Assume that you have taken a job as a probation-parole officer working with juvenile clients

who have been adjudicated by a local juvenile court. Your supervisor at your agency has asked

you for your opinion about whether juvenile, who are served by your probation-parole agency,

should participate in a delinquency prevention program patterned after the one in the popular

video titled “Scared Straight.” This video shows an innovative program put on by “lifers”

serving a life sentence that is intended to literally scare the delinquents straight.

1. What is your COPES (PICO) question?

2. Please record your search plan here including terms to mark key concepts and to include

appropriate MOLES (methodological filters)

3. Please record, if necessary on additional sheets, the most successful search histories or

history for your group, including databases searched, terms applied, and numbers of hits, to

locate your best document.

6

4. Best source found. Give complete reference APA style.

5. How sound is your best source relative to criteria on the appropriate evidence rating form?

Please include a brief paragraph to summarize your assessment of the evidence quality.

6. What would you tell your supervisor about trying the Scared Straight program to prevent

delinquency?

7. Can you calculate Number Needed to Treat (NNT) for any studies?

Question 2 (30 minute limit).

Assume that one of you has taught for several years, and you now find yourself the principal of a

middle school and high school that includes grades 7 through 12. You are concerned about

alcohol misuse among young people thorough experience with several tragic deaths. You

wonder what primary prevention program (primary means preventing the initial occurrence of a

problem) would most effectively prevent alcohol misuse among young people. What approach

would you try?

1, What is your COPES (PICO) question?

2. Record your search plan here including terms to mark key concepts including appropriate

methodological filters.

7

3. Record, if necessary on additional sheets, the most successful search histories or history for

your group including databases searched, terms applied, and number of hits to locate your

best document.

4. Best source found (give complete citation APA style).

5. How informative is your best source relative to criteria on the appropriate rating form?

Include a brief paragraph summarizing your assessment of evidence quality.

6. What intervention does this source support? Can you calculate NNT?

Please sign your names below.

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

______________________________________________________________________________

_____________________________________________________________________________

Exercise 19: PREPARING A CRITICALLY APPRAISED TOPIC (CAT)

_____________________________________________________________________________

OVERVIEW

Summaries of research related to a practice or policy question are helpful in making practice

decisions. You can use the format of a CAT to share what is found. Each student prepares a

critically appraised topic. Students may contact a helping professional to solicit a question. A

contact person who works in a helping profession could be suggested who might provide a

question.

PURPOSE

This exercise provides an opportunity to pose and answer an important practice question using

the format for a CAT

MATERIALS REQUIRED

1. Relevant material on the Internet regarding preparing a CAT including a CAT outline.

2. Access to relevant databases.

3. Access to appropriate rating forms regarding research reports.

4. Models of CATs.

TIME REQUIRED

Depends on how swiftly key research is located and students‟ skills in preparing a CAT. It also

depends on whether students work in terms. Getting students up to speed so they can do their

CATs may take multiple meetings. A class of eight students spent five classes practicing

working as a team while searching. Students were given case material and asked to pose a well-

built question and to work in the computer lab to locate their best source; critically appraise it,

and come to a conclusion about what to recommend.

SUGGESTIONS

Guide students as needed based on CATs available. Carefully review all answers regarding

completeness and rigor of search and accuracy of conclusions and provide corrective feedback.

2

CAT OUTLINE

Critically Appraised Topic Outline (use visual aids, no more than 10 minutes).

1. State who you are, source of your question, where that person works, and why the question

is worth answering (e. g., presenting concern of client).

2. State your well-structured question.

3. Describe your search plan.

4. Describe your search history.

5. Give your best source. (Give full reference APA style.) Include evaluation on the

appropriate rating form.

6. Based on this source, how would you answer the question? The evidence may not be

sufficient to come up with a recommendation. There may be contradictory results. The

results may be clear regarding positive effects but the intervention may create harmful

effects; if so, how do you weight their relative impact?

3

SHORT EXERCISE

Tick Bite

A family is in the church parking lot one Sunday morning walking toward church. Mary turns to

her friend and says, „I feel something on my neck. It might be a tick.” Her friend says “It is a

tick. I can see the legs moving.” Mary called the triage nurse at the local hospital because she

was concerned about Lime‟s Disease. The nurse told Mary to remove the tick, apply some

antibiotic, and watch for a distinctive “bull‟s eye rash” that might appear around the bite. If the

rash appears or the area grows red and inflamed, Mary should come in for tests.

At the drug store, Mary‟s friend buys antibiotic cream and a small magnifying glass with a sharp

tipped tweezer attached. When they get home, Mary‟s friend removes the tick with the tweezers

leaving its head embedded in Mary‟s neck. Mary‟s friend applies the antibiotic. Their question

concerns whether they removed the tick properly or should they have used some other method.

1. What is the COPES question?

2. Record your search history including databases, search terms, MOLES, number of

documents.

3. Best source found (use APA style).

4. Critical appraisal of best source applying appropriate evidence rating sale.

4

5. Recommendation for action and reasons

6. Did Mary and her friend do the right thing to treat the tick bite?

WORKING IN TEAMS

Divide the class into two groups. Give one group a problem like the one above. Give another the

task of observing how the team approaches the task. Have the observers look for whether the

task group followed the spirit of inquiry described in the quote below from the Wright Brothers

(100th

Anniversary of Flight Dec. 17, 2003) and report on what they observed about the group‟s

spirit of inquiry.

Wilbur and Orville had a technique for arguing that made others think them eccentric. People

often heard them arguing, yelling at each other, gesturing. Then observers saw them quiet down

for a moment but begin to argue again but switching sides in the argument, Wilbur arguing

Orville‟s point, Orville arguing Wilbur‟s point. In this manner, the Wrights avoided getting

stuck when solving a problem. In fact, they solved a whole series of problems that eluded the

best engineers of their time … Part of their genius was in how they argued. Spratt complained

that the brothers [Wilbur and Orville‟s] method of “rounding the corners” by switching sides in

the middle of an argument struck him as dishonest, Wilbur replied:

It is not my intention to advocate dishonesty in argument nor a bad spirit

in a controversy. No truth is without some mixture of error, and no error

so false but that is possesses some elements of truth. If a man is in too big

a hurry to give up an error he is liable to give up some truth with it, and in

accepting the arguments of the other man he is sure to get some error with

it. Honest argument is merely a process of mutually picking the beams

and motes out of each other‟s eyes so both can see clearly…. (Howard, p.

110).

From: Howard, F. (1998). Wilbur and Orville: A biograph of the Wright Brothers. Mineola,

NY: Dover.

5

EXAMPLE OF FINAL EXAMINATION FOR COURSE

Final Examination of Practice Application of Evidence-Based Practice. The class was

expected to work as a team in the computer laboratory to answer both questions within the

time limit (one hour). The class was informed that they would share the grade. The

grading criteria included: (1) inclusion of all four elements of a well-built question; (2) use

of appropriate methodological filters; (3) description of search history including search

terms and databases searched; (4) use of appropriate form for rating evidence quality; 5)

located appropriate marker document or better document; (6) accurate critical appraisal

of materials; (and (7) conclusion based on the best evidence that included discussion of

some index of treatment effect size. Work as a group on one question at a time. You have one

hour to answer both questions. Budget your time and spend about half the time on each

question.

Question #1: Family Disagreement

A family in the Eau Claire Area has an ongoing argument. The wife believes that a motorized

electric powered tooth brushing will remove more plaque from teeth. The father and son believe

that manual tooth brushing will remove plaque just as well or better. Your job is to help the

family resolve the disagreement by searching for the current best evidence.

1. Describe your COPES question here.

2. Record your search plan here including terms to mark key concepts. Include appropriate

quality filters.

3. On additional sheets record the search histories for your group including databases

searched, terms applied, and number of hits.

4. Best source found (APA style).

5. How rigorous is your best source relative to criteria described on an appropriate evidence

rating form? Include a brief paragraph to summarize your assessment.

6. Bottom line.

6

Question #2

A physician has discovered a primary basal cell carcinoma on your friend‟s nose that has been

confirmed by a laboratory test. The tumor is about the size of this “0” on the edge of his nostril.

Your friend wonders if the doctor should remove the tumor surgically, use radiotherapy, or use

cryotherapy.

1. Describe your COPES question here.

2. Record your search plan here including terms to mark key concepts and appropriate quality

filters.

3. On additional sheets, record the search histories for your group including databases

searched, terms applied, and number of hits.

4. What tis your best source (APA style)?

5. Describe your best source relative to criteria on an appropriate rating form. Include a brief

paragraph to summarize your assessment.

6. What intervention does this source support, if any?

7. Bottom line?

__________________________________________________________________

Exercise 20: INVOLVING CLIENTS AS INFORMED PARTICIPANTS

__________________________________________________________________

OVERVIEW

Clients are often not involved as informed participants in decisions made; they may be

misinformed or uninformed about recommended services and alternatives.

PURPOSE

This exercise provides an opportunity for students to understand the requirements for informed

consent and to implement this using the form in Box 20.1 illustrating what information should be

included in accurately informing clients.

MATERIALS REQUIRED

1. Practice Exercise 20 and Box 20.1.

2. Access to relevant databases.

TIME REQUIRED

This exercise could take a few hours. Students must do a search related to outcomes selected.

They must also gather information regarding agency services. This information may not be

accessible.

SUGGESTIONS

Discuss the challenges to involving clients as informed participants. For example, some clients

may not want to be involved as informed participants or staff may not have the knowledge or

skills to involve clients as informed participants. Engage students in a discussion of when

informed consent is an ethical obligation even if a client does not want to be fully informed, for

example in child welfare. Discuss possible objections from agency administrators to using the

form in Box 20.1.

REVIEWING STUDENTS WORK

What obstacles arose? How did students respond to them? What excuses did they use for not

providing clear information?

_____________________________________________________________________________

Exercise 21: ASKING HARD QUESTIONS

__________________________________________________________________

OVERVIEW

Evidence-informed practice requires asking questions such as "Is there any evidence that this

service does more good than harm?" Some people view questions as adversarial and obstructive

rather than as essential for helping clients make informed decisions. Staff may be expected to

follow the “party line” and may be punished for asking vital questions.

PURPOSE

1. To identify questions that should be asked.

2. To illustrate their relevance to informed consent and evidence-informed practice.

3. To enhance student's skills in raising questions in an effective manner.

MATERIALS REQUIRED

1. Practice Exercise 21, including Boxes 1.4 and 1.1.

TIME REQUIRED

This depends on how many practice opportunities are offered in raising questions and gaining

feedback.

SUGGESTIONS

A reluctance to pose questions may be due to anxiety created by fears of rejection and retaliation.

Such fears may be warranted. Decreasing this fear may require desensitization via exposure; that

is giving students repeated practice in raising questions in increasingly difficult situations. A

reluctance to pose questions may also be due to a lack of concern with clients' right to competent

services. Students may be reluctant to raise questions because they do not consider them

important.

REVIEWING STUDENTS’ EXPERIENCES

What were common obstacles to raising hard questions? How skilled were students in raising

questions? What are important self-development goals?

__________________________________________________________________

Exercise 22: EVALUATING SERVICE OUTCOMES

__________________________________________________________________

OVERVIEW

Most professionals are employed in some sort of organization such as an agency. Whether a

direct line practitioner, administrator or supervisor, skill in evaluating service outcomes is

important. Do services do more good than harm? Only if services offered are carefully

evaluated regarding quality and outcome, can they be systematically improved.

PURPOSE

1. To highlight the importance of carefully evaluating agency services.

2. To provide guidelines for evaluating service as well as practice in using them.

MATERIALS REQUIRED

1. Practice Exercises 22.1 and 22.2.

2. Access to relevant databases.

TIME REQUIRED

These exercises require students to gather information. This may require visiting an agency and

reading agency reports. Students may also search for research regarding a particular service.

SUGGESTIONS

Evaluation is a sensitive topic. Obstacles to careful evaluation such as administrators who may

not want to reveal what is done to what effect because of concerns about future funding should

be discussed. Caution students to "pick their battles" regarding when and how to try to gather

evaluative data. They are in low power positions in agencies compared to administrators and

senior clinicians. Administrators may not look favorably on involving clients as informed

participants. Give students practice in raising evaluation issues diplomatically. They could draw

on Getting to Yes by Fischer & Ury, 2011. Engage students in a probing discussion of how

services are and should be evaluated from a client’s point of view.

__________________________________________________________________

Exercises 23.1 and 23.2: REVIEWING YOUR EXPERTISE

__________________________________________________________________

OVERVIEW

Clinical expertise is drawn on to integrate the many sources of information pertaining to a

decision. Such expertise is a key ingredient of being an effective practitioner. This exercise

introduces students to components of expertise and related research.

PURPOSE

To provide an opportunity for students to become familiar with research regarding expertise and

to reflect on their expertise. That is, observation during structured interviews is needed to

accurately estimate expertise as is knowledge regarding outcome of services to review their own

expertise.

MATERIALS REQUIRED

1. Material in Exercise 23.

TIME REQUIRED

Depends on activity. (See Practice Exercises 23.1 and 23.2.)

SUGGESTIONS

Engage in students in a review of Box 23.1. Ask students to complete the Checklist in 23.1 (pp.

253-254) and identify two competencies they would like to enhance and prepare a plan for doing

so. Review these plans in terms of likely effectiveness.

Ask students to complete Exercise 23.2 (Errors) and discuss kinds of errors. Ask them to

indicate what errors are of most concern to them and discuss how they could be decreased.

Engage students in a discussion regarding expertise and time in practice and experience. Does

effectiveness increase with time and experience? See Goldberg, S. B. et al (2016). Do

psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a

clinical setting. J of Counseling Psychology, 63, 1-11.

1

_____________________________________________________________________________

Exercise 24: EVALUATING EFFECTIVENESS STUDIES: HOW GOOD IS THE

EVIDENCE?

____________________________________________________________________________

OVERVIEW

Students select (or are given) an outcome study related to practice. They then rate the study using

3 different forms.

PURPOSE

1. To identify the hallmarks of a sound intervention study.

2. To examine effect size.

MATERIALS REQUIRED

1. Quality of Study Rating Form (QSRF) Box 24.1.

NOTE: Error in QSRF #6. Should read “subjects randomly selected (not assigned).”

2. An intervention study.

TIME REQUIRED

This depends on whether students are asked to locate an outcome study themselves that concerns

a client or a study is given to them to rate. In the former case, students may spend time searching

for a relevant study. About 60 or more minutes will be needed to review criteria, 60 minutes to

read and rate the study, and 30 minutes to discuss ratings. Reviewing potential biases will

require time. (See Box 5.1, page 258).

SUGGESTIONS

Students differ in their knowledge about research methods. You can use the Quality of Study

Rating Form to provide an overview of major features of an intervention study. If students select

an outcome study, ask them to attach a copy of the study to the rating form.

You can expect reasonable inter-rater agreement for most items on the QSRF among students

who independently rate the same study if you explain criteria on the form. If you want students

to calculate ESl and ES2, they need specific instructions about which effect sizes to compute for

what outcomes at what outcome interval; they need some statistical sophistication for more

complex articles. Many articles omit essential information, so these indices cannot be computed.

Consult additional sources as needed for interpreting ES1 and ES2.

2

DISCUSSION QUESTIONS

1. Discuss variations in evaluations among students.

2. What are Effect Sizes? Was there agreement? Were they accurately calculated? Discuss

controversies about using effect sizes.

3. Would students recommend this intervention? Ask them to explain their reasons.

4. What have students learned? We hope responses include: (1) it is possible to take some

guesswork out of evaluating the quality of a research report, and (2) students working

independently can arrive at reasonably consistent appraisals for the same study.

____________________________________________________________________________

Exercise 25.1: CRITICALLY APRAISING RESEARCH REVIEWS: HOW GOOD IS

THE EVIDENCE?

_____________________________________________________________________________

OVERVIEW

Locating high-quality research reviews regarding questions is an important skill.

PURPOSE

This exercise provides practice in locating and apprising research reviews. Characteristics of

high quality reviews and sources are described including the Cochrane and Campbell Libraries.

Students should be warned that reviews may be misleading (see Ioannidis, 2016).

MATERIALS REQUIRED

1. A research review to evaluate.

2. QUORUM guidelines. See also p. 60 in Geyman, Deyo and Ramsey (2000); Greenhalgh,

2010).

3. Access to relevant databases.

TIME REQUIRED

One to two hours.

SUGGESTIONS

Discuss the various kinds of reviews and their purposes (e. g., Greenhalgh, et al., 2011; Pettigrew

& Roberts, 2006). Give students sources of high-quality reviews and practice in locating them.

Select questions of interest to students to motivate them to search effectively for rigorous

reviews. Carefully review concerns discussed by Ioannidis (2016).

REVIEWING STUDENTS WORK

How skilled are students in locating and critiquing research reviews? What kind of errors do

they make? Is access to relevant databases a problem? Can access be improved? How rigorous

were students’ evaluations? How effective were their searches? What obstacles did they

encounter? Have services used in students’ field work agencies been critically appraised and

results made accessible to all involved parties, including clients?

DISCUSSION QUESTIONS

Discuss common flaws in research review. Read and discuss Ioannidis (2016).

_____________________________________________________________________________

Exercise 25.2: CRITICALLY APPRAISING PRACTICE GUIDELINES

_____________________________________________________________________________

OVERVIEW

Practice guidelines are promoted on many websites and by many organizations. Are all well

argued?

PURPOSE

To enhance skills in critically appraising practice guidelines.

MATERIALS REQUIRED

1. A practice guideline to evaluate.

2. Guidelines for reviewing (e. g., Greenhalgh, 2011).

3. Access to relevant databases.

TIME REQUIRED

Depends on whether students locate a guideline of interest or you give them one to evaluate.

Evaluation will take 60 minutes and class discussion 60 minutes.

SUGGESTIONS

Select a guideline of interest to students. Encourage students to identify interested parties (e. g.,

pharmaceutical companies). (See also Exercise 8.)

REVIEWING STUDENTS WORK

What errors to students tend to make when reviewing guidelines? What were their beliefs about

guidelines prior to reading related critiques?

DISCUSSION QUESTION

Why do practice guidelines differ in quality?

____________________________________________________________________________

Exercise 26: CRITICALLY APPRAISING SELF-REPORT MEASURES

______________________________________________________________________________

OVERVIEW

Social workers may rely on self report measures in their work, perhaps from other professionals

such as psychologists. How valid are these?

PURPOSE

1. To increase skills in accurately appraising the validity of self-report measures.

2. To increase skills in identifying sources of bias in self-report measures.

3. To give students practice in critically appraising a self-report measure.

MATERIALS REQUIRED

1. A self-report measure.

2. Practice Exercise 26.

3. Description of reliability and validity in text.

4. Access to relevant databases.

TIME REQUIRED

An in depth discussion of reliability and validity will take at least an hour or two of class time,

and students should review material at home. Give students opportunities to assess their

knowledge about these important concepts.

SUGGESTIONS

Review the many sources of bias in self-report measures and give students repeated opportunities

to detect them. As always, emphasizing the possible negative effects on clients of ill-advised

decisions based on biased self report data may enhance motivation to attend to such possibilities.

Developing a sound understanding of reliability and validity and their relationship is important

and professionals can draw on this knowledge throughout their careers. To tap

misunderstandings, ungraded quizzes offer valuable corrective feedback.

REVIEWING STUDENTS’ WORK

Do students understand different kinds of reliability and validity and the relationship between

reliability and validity? How accurate are they in their appraisal of research articles in relation to

these concepts?

1

____________________________________________________________________________

Exercise 27.1: ESTIMATING RISK AND MAKING PREDICTIONS ____________________________________________________________________________

OVERVIEW

This exercise identifies common pitfalls in estimating risk and making predictions (e.g., relying

on anecdotal experience and ignoring base rates). Students complete an exercise translating

probabilities into natural frequencies to estimate risk.

PURPOSE

1. To introduce basic concepts concerning risk assessment and decision making.

2. To encourage further reading about related concepts and common errors.

MATERIALS REQUIRED

1. Exercise 27.1.

2. Answers.

TIME REQUIRED

Fifty minutes (30 for calculations, 20 for discussion).

SUGGESTIONS

This exercise highlights the value of using natural frequencies to estimate risk. Problems in

estimating risk include a lack of information about prevalence, sensitivity, and specificity. We

cannot predict what an individual will do based on group data; we can only describe the baserate

data of the group. Predictions based on actuarial data are more accurate than those relying on

clinical intuition (Grove & Meehl, 1996). See also description of use of predictive risk modeling

in child welfare.

DISCUSSION QUESTIONS

Do social workers consider the prevalence of a problem among an agency's clients as well as

sensitivity and specificity of a test's results in estimating the predictive value of a positive test

result?

Research exploring accuracy of professionals in estimating predictive value of a test show that

they overestimate their accuracy (Gigerenzer, 2002; Welch, 2004). Eddy (1982) found that 95 of

100 physicians estimated the predictive value of a positive mammogram (X-ray test for breast

cancer) to be 75%, where in reality it was only .077, or about 8% (p. 253).

2

Is the predictive value of a positive test greater where the prevalence rate is high or where it is

low?

__________________________________________________________________________________

Exercise 27.2: CRITICALLY APPRAISING A RISK MEASURE

____________________________________________________________________________

OVERVIEW

You may have to estimate risk, for example of child abuse or suicide. Being informed about

how best to do so is important to protect clients from actions taken based on uninformed or

misinformed bases.

PURPOSE

To enhance skills in estimating risk.

MATERIALS REQUIRED

A risk measure of interest to students.

TIME REQUIRED

At least 60 minutes.

SUGGESTIONS

Be sure to select a measure that pertains to tasks students confront.

DISCUSSION QUESTIONS

1. How accurate is this measure?

2. Is the measure widely used?

3. Does use of this measure do more good than harm?

1

_______________________________________________________________

Exercise 28: EVALUATING DIAGNOSTIC TESTS ___________________________________________________________________________

OVERVIEW

This exercise involves students in reviewing a diagnostic test. Reflex dilation was a test devised

to determine whether children had been sexually abused. The history of the helping professions

provides hundreds of examples of the use of inaccurate diagnostic tests resulting in harm to

clients. The example in this exercise is one that occurred in England. This exercise illustrates the

importance of considering concepts such as false positives and false negatives that may sound far

removed from the everyday world of practice, but in fact, influence clients' lives. This exercise

can be used to raise key questions about diagnostic measures.

PURPOSE

To highlight questions that should be raised about diagnostic tests and the dangers of using

questionable tests (they do not measure what they claim to measure).

MATERIALS REQUIRED

Description of reflex dilation test in workbook.

TIME REQUIRED

Thirty minutes.

SUGGESTIONS

Discuss the life-affecting implications of faulty decisions based on invalid tests including both

false positives (e.g., the child has been abused but is judged to have been be abused) and false

negatives (the child has been abused and judged not to have been abused). Discuss implications

for children, parents, and professionals.

POSSIBLE ANSWERS TO QUESTIONS

1. What questions should be raised about this test?

Suggestion: Review Exercise 28, which presents major issues related to this exercise. We

recommend laying out the decision on a 2 X 2 table, with Reflex Dilation Test Result

(positive, negative) on the left, and True State (child abused, child not abused) across the top.

Thus, there will be four cells, two that describe errors (false positive, false negative) and two

that describe accurate judgments (true positive, true negative). In this example, we do not

2

have all the data needed to fill in the four cells. You could accompany this exercise with data

in the literature that suggest the false positive and false negative rates of alleged child abuse.

We hope that the exercise will prevent students from making the kinds of errors shown here

and will highlight how easy it is to do so by asking questions about false positive and false

negative rates.

2. Would you use this test? Please explain your answer.

Suggestion: The bottom line here would be the predictive value (see Exercise 28).

_____________________________________________________________________________

Exercise 29: EVALUATING RESEARCH REGARDING CAUSES

_____________________________________________________________________________

OVERVIEW

Professionals try to identify the causes of clients' concerns. This exercise provides an

opportunity to discuss some of the complications and sources of bias in discovering causes.

PURPOSE

1. To identify types of causes.

2. To give students experience in identifying these.

3. To identify common errors and their consequences.

4. To learn about helpful criteria when evaluating research studies exploring causation.

MATERIALS REQUIRED

1. A relevant article to review.

2. Guidelines for reviewing related articles.

3. Practice Exercise 29.

4. Access to relevant databases.

TIME REQUIRED

Depends on the complexity of the article reviewed.

SUGGESTIONS

You could review the major kinds of causes (e.g., biological, psychological, sociological and so

on) as well as possible interactions among them as in multi-causal accounts. Common kinds of

oversimplifications and their effects on decisions can be reviewed as well as how to spot these.

REVIEWIGN STUDENTS’ WORK

What errors do students make in reviewing material regarding causation? Were certain kinds of

reports, for example, those including pictures of brains, especially misleading?

1

_____________________________________________________________________________________________________________________

Exercise 30: ANALYZING ARGUMENTS

_________________________________________________________________

OVERVIEW This exercise provides guidelines for analyzing arguments as well as practice opportunities.

Accepting or rejecting an argument regarding client care can have profound implications for

client welfare.

PURPOSE

To increase skills in argument analysis.

MATERIALS REQUIRED

1. Argument Analysis Form (Exhibit 30).

2. Guidelines for Evaluating Arguments in Exercise 30.

3. Arguments to analyze.

TIME REQUIRED

This will vary depending on how many arguments you review.

SUGGESTIONS

This exercise provides an additional opportunity to develop students' argument analysis skills.

You could use it in class and/or as a homework assignment. We find it helpful to precede this

exercise with class practice in which practice-related arguments are identified, and premises as

well as warrants related to conclusions are reviewed in terms of their acceptability, relevance,

sufficiency, and rebuttal (see guidelines in the workbook). You could hand out model argument

analyses to highlight valuable criteria for reviewing arguments. Damer (1995) stresses that no

matter how often he emphasized the importance of rebuttal, many students do not include this in

their reply. Reasons given include not wanting to reveal weak points in their arguments.

You could break students into groups of three or four and ask them to prepare an argument

analysis which is then presented to the entire class. Draw on Richard Paul's ideas for involving

all students. For example, rather than designating someone in each group to report back to the

entire class, you could tell the class that you will call randomly on one of the members in each

group to present the group's "thinking." This will increase the likelihood that every person in a

group will be ready to present.

Highlight the consequences of basing decisions on faulty arguments in a range of contexts

including work with individuals, families, groups, organizations and communities. You could

2

include a discussion of different kinds of explanations for behavior (e.g., developmental,

biological, psychological, sociological) as they may be involved in the warrants students give to

link premises to conclusions. Initial arguments should be short until students become skilled in

identifying the key elements in arguments.

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

1. What was the most difficult part of completing your argument analysis?

Suggestion: Common problems include accurately evaluating relevance, acceptability, and

sufficiency of premises related to a conclusion. Other problems include overlooking key

premises, separating one argument from another, determining whether a statement presents

an argument (it may just make a claim), and clearly identifying conclusions.

2. Did you come up with effective rebuttals to your argument?

Suggestion: Review rebuttals students offer. Can stronger ones be made? Have they

forgotten to provide rebuttals? If so, ask them why.

3. Discuss relevance of different goals of an argument such as inquiry, deliberation, or

persuasion.

1

______________________________________________________________________________

Exercise 31: CRITICAL THINKING AS A GUIDE TO MAKING ETHICAL

DECISIONS

______________________________________________________________________________

OVERVIEW

This exercise draws on the vignettes in Exercises 11, 12, and 13 to review ethical questions that

arise in everyday practice. Ethical issues involved in each vignette are suggested. Many writers

have emphasized the value of critical thinking in arriving at moral and just decisions (e.g.,

Baron, 1994; Brookfield, 1987; Gambrill, 2005; Paul, 1993). Considering the multiple interests

of different parties increases the likelihood of arriving at equitable decisions.

PURPOSE

To illustrate the value of critical thinking as a guide to making ethical decisions in helping

contexts.

MATERIALS REQUIRED

1. Vignettes for Exercises 11, 12, and 13.

2. Checklist of Ethical Concerns (see Box 31.1).

3. Suggested Ethical Issues in vignettes.

TIME REQUIRED

This will vary depending on how many examples are reviewed. To avoid getting "stalled" on one

item, we recommend setting a ten-minute time limit for discussing each item.

SUGGESTIONS

You could select vignettes that involve particular kinds of concerns and ask students to identify

related ethical issues. You could precede this by a discussion of factors that may influence

decisions such as agency policy, client wishes, the wishes of significant others, legal regulations,

as well as professional codes of ethics.

Students can hone their argument analysis skills by preparing arguments for their positions. You

can ask them to clearly identify the premises from which they argue that a certain ethical

principle is involved and/or has been violated in a certain situation. Students can work in groups

to identify ethical issues in vignettes and provide arguments for their beliefs, and then share their

results with the rest of the class.

2

REGARDING FOLLOW-UP QUESTIONS

Check whether students think important ethical issues have been overlooked in the examples

given. If so, what are they and what are examples?

Encourage an open discussion of different views. Ask students if they believe professionals are

ethically bound to think critically about decisions they make. We hope that, among others, three

reasons will emerge: (1) because critical thinking involves weighing evidence about what helps

clients, it would be unethical not to think critically about practice decisions; (2) because critical

thinking implies intellectual honesty, critical thinkers will be more likely and able to inform

clients more accurately about options, costs, risks, and possible outcomes of different plans; and

(3) because critical thinking implies fairness and objectivity, critical thinkers will be less likely

to impose stereotypes and prejudices on others.

______________________________________________________________________________

Possible Answers (See also Box 31.1.)

Game A: Common Practice Fallacies

1. Ethical issue: The recommendation made is based on one individual's experience with an

intervention; it offers no guidance as to whether the method will help anyone else.

2. Ethical issue: Professionals have a responsibility to be clear. Vagueness decreases the

likelihood of understanding client concerns and identifying related causes. It increases the

likelihood that helpers will, unwittingly, end up talking about different things.

3. Ethical issue: This administrator is trying to get an accurate picture of success rates-an

admirable professional aim.

4. Ethical issue: Whether intentionally or not, this politician uses a propagandistic method to

question a welfare program. Politicians should present issues, questions, and problems

fairly and avoid propaganda ploys, such as relying on vivid, atypical case examples.

5. Ethical issue: Here a professional makes a claim of effectiveness based on pre-post data.

This violates our obligation to base recommendations on sound evidence.

6. Ethical issue: Supervisors should think critically about claims made and educate their staff

to do the same. This supervisor is either/or-ing.

7. Ethical issue: This example illustrates a lapse on the part of an author of an article in a

professional journal. Rather than examining all articles related to a question, - both pro and

con - this author used only those in support of one position. This will not help readers judge

the effectiveness of family-based treatment.

8. Ethical issue: See #5.

3

9. Ethical issue: No evidence is provided that anxiety is a brain disorder.

10. Ethical issue: Appeal to unfounded authority.

11. Ethical issue: This vignette illustrates a concern to make decisions based on available

evidence.

12. Ethical issue: This statement is contradicted by data that shows that tracking progress

contributes to positive outcome (Boswell et al., 2013; Lambert, 2015).

13. Ethical issue: What happens with the rest? Administrators are ethically bound to provide

accurate (rather than misleading) information.

14. Ethical issue: Newness does not equate with effectiveness.

15. Ethical issue: Professional organizations such as the American Psychiatric Association (the

publishers of the journal) have an obligation to inform rather than mislead, not only in their

published articles but in their human-services advertisements. The vague claims made in

this ad are based on one case example.

16. Ethical issue: Professionals are obliged to present sufficient information about a study so

that we can judge for ourselves whether related conclusions are warranted.

17. Ethical issue: Relies on vague data.

18. Ethical issue: Vagueness is also a problem here. No information is given about the study.

19. Ethical issue: Newness does not equate with effectiveness.

20. Ethical issue: Overconfidence and vagueness reigns here.

Game B: Group and Interpersonal Dynamics

1. Ethical issue: Careful consideration of different possibilities at case conferences helps to

avoid wrong choices. Focusing on her goal (to help clients) will give this professional the

courage to pursue a culture of thoughtfulness and ignore any putdowns that occur along

the way.

2. Ethical issue: Basing decisions on small samples in artificial settings and overlooking

data based on observation in real-life settings may result in poor decisions and (in this

example) can result in injury to staff.

3. Ethical issue: Professional helpers should base decisions on the best evidence available

and question decisions based on weak grounds.

4

4. Ethical issue: Professionals are ethically bound to consider the quality of evidence related

to decisions. Lizzie and Polly do not; they argue ad hominem rather than ad rem. They

should share objections they have and not whisper to each other.

5. Ethical issue: Professionals have an obligation to focus on making sound decisions

during case conferences and to avoid sidetracks that can get in the way of making them.

6. Ethical issue: Professionals have an obligation to avoid misunderstanding of theories and

related data that may help clients.

7. Ethical issue: Professionals have an ethical responsibility to base decisions on sound

rather than weak criteria. Tradition is not a sound basis for making decisions.

8. Ethical issue: Professionals are legally and morally obliged to follow up on reports of

child abuse, and should do so in a way that protects the rights of alleged perpetrators.

False allegations may have devastating consequences.

9. Ethical issue: No additional clear information is provided for this premature diagnosis.

10. Ethical issue: Stereotyping is not only ethically wrong but also may be illegal.

11. Ethical issue: Professionals have a responsibility to match clients and recommended

interventions in a way that maximizes success. Also, strawman arguments do not

forward careful decisions.

12. Ethical issue: Professionals have a responsibility to critically review diagnostic labels to

make sure they help (i.e., point to etiology and/or effective interventions) more than harm

(i.e., obscure possible options). It is unethical to stigmatize people by assigning labels

that provide no effective intervention guidelines. Negative labels may follow children

through their educational careers.

13. Ethical issue: What is the evidentiary status of claims in these reports? Also because a

person has a certain diagnosis does not necessarily entail spilling food. Professionals

have an ethical obligation to make decisions based on evidence and well-argued theory.

14. Ethical issue: Those who make claims about the effectiveness of intervention methods

should clearly describe relevant evidence. Also, professionals are obliged to raise

questions about evidentiary claims that affect clients’ lives.

15. Ethical issue: Professionals have an obligation to base decisions on sound criteria rather

than weak criteria such as manner of presentation.

16. Ethical issue: Professionals are obliged to respond appropriately to criticism of work-

related behavior -- admit to being late.

17. Ethical issue: See #15.

5

18. Ethical issue: There may be a third option (leave children in the home and provide

services needed to safeguard children). Professionals have an obligation to select the

least intrusive effective methods. Placing children in foster care is an intrusive

intervention.

19. Ethical issue: Professionals have an obligation to seek options that maximize the

likelihood of helping clients. This means avoiding either-or framing that may result in

overlooking promising options.

20. Ethical issue: Professionals have an obligation to avoid distractions where life-affecting

decisions are at hand.

21. Ethical issue: Attention to evidentiary issues is neglected.

Game C: Clinical Reasoning

1. Ethical issue: Failure to consider the role of hindsight bias may result in poor decisions.

2. Ethical issue: Regression effects are ignored which may result in an ill-advised decision.

3. Ethical issue: Professionals have an obligation to avoid misleading effects of how a

choice is "framed."

4. Ethical issue: Professionals have an obligation to be informed about contextual factors

related to client concerns in order to arrive at a sound assessment. This means

considering not only the client's family interaction, cultural norms, and values, but also

environmental factors such as lead based paint residues.

5. Ethical issue: Actually, research regarding psychotherapy shows that helpers are not

very accurate in detecting deterioration (Shimokawa and colleagues, 2010).

6. Ethical issue: This nurse is attending to evidentiary issues.

7. Ethical issue: See #1.

8. Ethical issue: See #3.

9. Ethical issue: Professional helpers have an obligation to base decisions on sound

evidence. 0063

10. Ethical issue: See #2.

11. Ethical issue: Outcome monitoring is related to progress (Lambert, 2015). Thus ignoring

this is an ethical lapse..

6

12. Ethical issue: This caseworker has fallen into the gambler’s fallacy. Professionals have

the right to freedom from undue danger in carrying out their responsibilities. Social

workers sometimes go into situations that even the police have refused to enter.

13. Ethical issue: See #2.

14. Ethical issue: Professionals have an obligation to conduct a thorough assessment. This

requires questioning initial assumptions and searching for alternative views.

15. Ethical issue: Professionals have an obligation to avoid confirmation biases.

16. Ethical issue: Professionals have an obligation to make sure their emotional reactions do

not impede high quality services.

17. Ethical issue: Professionals have an obligation to avoid confirmation biases.

18. Ethical issue: Professionals should base decisions on the best evidence available.

19. Ethical issue: Professionals have an obligation to be knowledgeable about cognitive

biases such as the gambler's fallacy that may influence decisions.

20. Ethical issue: Professionals should attend to harms of omission as well as harms of

commission.

21. Ethical issue: See #2.

Reference

Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2013). Implementing routine

outcome monitoring in clinical practice: Benefits, challenges, and solutions.

Psychotherapy Research. DOI:10.1080/10503307.2013.817696

Lambert, M. J. (2015). Progress feedback and the OQ-System: The past and the future.

Psychotherapy, 52, 381-390.

Shimokawa, K., Lambert, M. J., & Smart, D. (2010). Enhancing treatment outcome of patients

at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy

quality assurance system. Journal of Consulting & Clinical Psychology, 78, 298-311.

_____________________________________________________________________________________________________________

Exercise 32: REVIEWING INTERVENTION PLANS _________________________________________________________________________

OVERVIEW

This exercise engages students in reviewing criteria for selecting intervention plans. The

checklist included is relevant to all practice levels (e.g., individual, family, community, or

organization). Intervention plans succeed or falter depending on how soundly they are conceived.

This exercise takes a detailed look at criteria that should be considered when selecting plans such

as: How intrusive is the plan? Are cultural differences considered? How effective is it likely to

be?

PURPOSE

To encourage students to think critically about intervention plans.

MATERIALS REQUIRED

1. Checklist for Reviewing Intervention Plans (Exhibit 32).

2. An example of a plan.

TIME REQUIRED

Thirty minutes to apply the checklist and one hour for discussion. If students are asked to select

examples from the professional literature, more time will be required for library search.

SUGGESTIONS

Ask students to apply this checklist to one of their own clients, one that you give to students, or

to an example they locate in the literature. All students should have access to examples reviewed

so that they can decide for themselves which criteria apply. They should read case examples

beforehand to make the most productive use of class time. One member of the class could take

the lead in applying the criteria to an example. Students could work in teams of three to review

an example and apply the criteria and then present their views to the class. The class could

review the examples as they present their material.

1

_________________________________________________________________________

Exercise 33: ENCOURAGING A CULTURE OF THOUGHTFULNESS

_________________________________________________________________________

OVERVIEW

This exercise provides an opportunity for students to plan how to enhance a culture of

thoughtfulness (Perkins, 1992) in their work settings guided by a list of possibilities. Students

complete a questionnaire that allows them to evaluate the extent to which their work

environment encourages critical thinking, identify two barriers, and describe specific steps to

overcome them.

PURPOSE

To become familiar with indicators of a culture of thoughtfulness as a first step toward

identifying constructive changes to pursue.

MATERIALS REQUIRED

1. Culture of Thoughtfulness Scale (Exhibit 33).

2. Access to a work setting.

TIME REQUIRED

Fifteen minutes to complete the form, 30 to 40 minutes to discuss it.

SUGGESTIONS

There are several ways to proceed. You might ask all agency staff to fill out the form

anonymously; record the mean score for each item; then discuss three major strengths (highest

mean score items) and three weak items (lowest mean score items). You could ask students to

complete the form individually to review the culture in their field work agency. You could

complete the form yourself in relation to your own work environment. Are there items on the

questionnaire that students believe should not be there? What are their reasons? Can they make

a good argument that the characteristic would not contribute to a culture of thoughtfulness? Are

there items they would add? Can they make a good argument for this? You could have students

who work in the same or in similar agencies compare their ratings. Reasons for differences can

then be explored.

Ask students to select one change they would like to pursue in their agency and help them to

identify this clearly and to design a plan for pursuing it. Encourage students to draw on their

knowledge and skills related to organizations in planning how to encourage a culture of

thoughtfulness. For example, the better students understand the formal and informal incentive

2

systems and communication channels in an organization (e.g., the better their "contingency

analysis" skills for identifying what antecedents and consequences support or punish behaviors

of interest), the more likely they are to accurately identify leverage points for improvement.

At this point there are no norms for this instrument; so it cannot be used for comparative

purposes.

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

1. Which three items (highest scores) are your agency's greatest?

Suggestion: Focus first on strengths. This demonstrates intellectual charity (looking for the

best) and fairmindedness. Ask for specific examples of items noted (e.g., behaviors and

events) that demonstrate why an item was selected.

2. Which three items (lowest scores) are your workplace's greatest weaknesses?

Suggestion: Emphasize that the purpose here is not to expose errors and to punish the

guilty, but to examine how a culture of critical inquiry could be enhanced. Ask for specific

examples (behaviors, events) that demonstrate why given items on the questionnaire were

marked low.

3. What is the usual fallacy rate in case conferences?

Suggestion: Review fallacies detected and discuss how they could be avoided.

4. Suggestion: Here, too, you could review data collected and discuss

how to avoid fallacies.

1

_____________________________________________________________________________

Exercise 34: EVALUATING THE TEACHING OF CRITICAL-THINKING SKILLS

______________________________________________________________________________

OVERVIEW

This exercise highlights characteristics of a learning environment in which critical thinking is

modeled and supported. Students (and/or instructors) can use the Teaching Evaluation Form in

the exercise to rate the extent to which an instructor encourages critical thinking. Relevant

behaviors include encouraging questions and providing alternative views on controversial issues.

PURPOSE

To help students and instructors to evaluate the extent to which their classroom environments

encourage critical thinking.

MATERIALS REQUIRED

Teaching Evaluation Form (Exhibit 34).

TIME REQUIRED

Ten minutes to complete the form, 30 minutes to discuss item content.

SUGGESTIONS

An item-by-item review will offer the most useful feedback on particular items. For example,

you could collect all questionnaires and calculate the mean scores for each item as well as an

overall score. Students must be assured of confidentiality so they will be free to express their

opinions (e.g., not put their names on the questionnaire). These mean scores (on each item and

the overall score) can serve as discussion topics. The characteristics noted on this questionnaire

that model and support critical thinking may depart radically from common practices. We

showed an earlier version to a group of university instructors and urged that it be used as a way

to evaluate their performance. The group found only eight items acceptable as criteria to rate

their own performance. They rejected the instrument because: "We don't teach that way." (For a

valuable source on teaching see Brookfield, 1990.)

This exercise encourages both students and instructors to think about the consequences-

especially to clients-of teaching "that way."

2

POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS

1. Which item seems most important as a characteristic for an instructor who teaches critical

thinking?

Suggestion: We have no norms for this measure; so we suggest that you discuss the content

of the item and specific reasons for ratings given. Have each student identify an important

item and explain their choice by stating why the item identifies a trait of a critical thinker.

Students could also discuss obstacles to enhancing an atmosphere of critical inquiry in the

classroom and how to overcome them.

2. Which item seems least important as a characteristic for an instructor who teaches critical

thinking?

Suggestion: Have each student identify an item and compare and discuss them. Is this item

really unimportant to teaching critical thinking?

____________________________________________________________________________

Exercise 35: FORMING A JOURNAL CLUB

_____________________________________________________________________________

OVERVIEW

This exercise describes the purpose and helpful requirements of a Journal Club including the fun

one can have.

PURPOSE

To help readers set up a successful Journal Club that contributes to the effectiveness of their

services.

MATERIALS REQUIRED

1. Practice Exercise 35.

2. Access to Exercise 35 in text. (Preparing CATS.)

3. Access to relevant databases.

TIME REQUIRED

Depends how often the group meets and what tasks they take on.

SUGGESTIONS

Make this fun as well as useful. Seek out valuable resources described in the literature about

how to make journal clubs interesting and successful. Increasing students’ skills and motivation

in this area will be important for life-long learning.

REVIEWING STUDENTS’ WORK

What characteristics of successful journal clubs did students incorporate? Which ones did they

neglect? How much did they enjoy the group meetings? Did everyone participate? Were CATS

valuable in making decisions?

______________________________________________________________________________

Exercise 36: ENCOURAGING CONTINUED SELF DEVELOPMENT REGARDING

THE PROCESS OF EVIDENCE-INFORMED PRACTICE

______________________________________________________________________________

OVERVIEW

Effective professional practice requires keeping up-to-date with new developments that may

affect practice and policy decisions.

PURPOSE

To highlight the importance of this and to give readers some guidelines for self development in

relation to the process of evidence-informed practice.

MATERIALS REQUIRED

1. Box 36.1.

2. Practice Exercise 36.

3. Access to relevant databases.

TIME REQUIRED

About one hour to complete Practice Exercise 36 and one hour to have a discussion.

SUGGESTIONS

Note the importance and interest value of life-long learning. Encourage students to draw on

literature concerning self-management (e. g., how to collect clear data regarding baseline levels

of key skills such as posing clear questions, how to increase desired behaviors and how to

display progress, for example on graphs, see Watson & Thorp, 2013).

REVIEWING STUDENTS’ WORK

What skills did students select for self-improvement? How accurate are they in assessing their

competency levels of given skills? Are their self-management plans for increasing skills

informed by literature on self-management (Watson & Tharp, 2015)?

____________________________________________________________________________

Exercise 37: INCREASING SELF AWARENESS OF PERSONAL OBSTACLES TO

CRITICAL THINKING

______________________________________________________________________________

OVERVIEW

There are great impediments to offering clients effective services. Professionals are not

necessarily trained to think critically about decisions they make and there are a number of

personal obstacles that may get in the way including authoritarian dispositions and inflated

beliefs about one's competencies (over confidence).

PURPOSE

To help students identify and decrease personal obstacles to evidence-informed decision making.

MATERIALS REQUIRED

1. Box 37.1

2. Practice Exercise 37.1

TIME REQUIRED

About one hour to complete Practice Exercise 37 and one or more hours to discuss particular

personal obstacles students mention.

SUGGESTIONS

Reviewing the kinds of personal obstacles may be a good place to start (e. g., motivational, poor

self-management skills). You would ask students to read material describing how common some

obstacles are, such as self-inflated assessment of competency (e. g., Dunning, Heath, & Suls,

2004, Psychological Science in the Public Interest, 5, 69-106).

REVIEWING STUDENTS’ WORK

What obstacles do students mention most often? Are there certain obstacles that are important to

review that no one talks about – taboos?