Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Critical Thinking for Helping Professionals: A Skills-Based
Workbook. Fourth Edition. 2017
Instructor’s Manual
EILEEN GAMBRILL
LEONARD GIBBS
OXFORD
UNIVERSITY PRESS
2
CONTENTS
PART 1: Critical Thinking as a Guide to Decision Making
Exercise 1 Making Decisions About Intervention
Exercise 2 Reviewing Your Beliefs About Knowledge
Exercise 3 Controversy: Invaluable for Problem Solving and Learning
Exercise 4 Critical Thinking and Advocacy
PART 2: Recognizing Propaganda: the Importance of Questioning Claims
Exercise 5 Critically Appraising Human Services Advertisements
Exercise 6 Does Scaring Youth Help Them “Go Straight”?
Exercise 7 Detecting Misleading Problem Framing
Exercise 8 Following the Money
Exercise 9 The Language of Propaganda
PART 3: Increasing Your Skills in Avoiding Fallacies, Biases, and Pitfalls in Decision
Making
Exercise 10 Using the Professional Thinking Form
Instructions for Reasoning-in-Practice Games
Exercise 11 Reasoning-in-Practice Game A
Exercise 12 Reasoning-in-Practice Game B: Group and Interpersonal Dynamics
Exercise 13 Reasoning-in-Practice Game C: More Biases
Exercise 14 Preparing a Fallacies Film Festival
Exercise 15 Fallacy Spotting in Professional Contexts
Exercise 16 Avoiding Group Think
PART 4: Evidence Informed Decision Making
Exercise 17 Applying the Steps in the Process in Evidence-Based Practice
Exercise 18 Working in Interdisciplinary Evidence-Based Teams
Exercise 19 Preparing Critically Appraised Topics`
Exercise 20 Involving Clients as Informed Participants
Exercise 21 Asking Hard Questions: Enhancing Assertive Skills
Exercise 22 Evaluating Service Outcomes
Exercises 23.1 and 23.2 Reviewing Your Expertise
PART 5: Critically Appraising Research
Exercise 24 Evaluating Effectiveness Studies: How Good is the Evidence?
Exercise 25.1 Critically Appraising Reviews
Exercise 25.2 Critically Appraising Practice Guidelines
Exercise 26 Critically Appraising Self-Report Measures
Exercise 27.1 Estimating risk and Making Predictions
3
Exercise 27.2 Critically Appraising a Risk Measure
Exercise 28 Evaluating Diagnostic Tests
Exercise 29 Evaluating Research Regarding Causes
PART 6: Reviewing Decisions
Exercise 30 Critically Appraising Arguments
Exercise 31 Critical Thinking as a Guide to Making Ethical Decisions
Exercise 32 Reviewing Intervention Plans
PART 7: Improving Educational and Practice Environments
Exercise 33 Encouraging a Culture of Thoughtfulness
Exercise 34 Evaluating the Teaching of Critical Thinking Skills
Exercise 35 Forming a Journal Club
Exercise 36 Encouraging Continued Self Development Regarding the Process of
Evidence-Informed Practice and Policy
Exercise 37 Increasing Self Awareness of Personal Obstacles to Critical Thinking
4
PREFACE
Critical Thinking for the Helping Professionals has a single purpose: To improve the
reasoning of practitioners as they carry out their life-affecting work. Though developed
primarily for use in research methods and practice courses, the exercises are intended to sharpen
day-to-day reasoning among practitioners in the helping professions through exercises whose
principles apply across the curriculum. We suggest that you get an overview of the exercises
then decide how each may serve you best. Many exercises are designed to help students spot and
counter faulty reasoning that may result in questionable decisions. This Instructor’s Manual
contains descriptions of each exercise including a brief Overview, its Purpose (learning
objectives), Materials Required, Time Required, Suggestions for Using the Exercise, and
Possible Answers to Follow-Up Questions. Exercises in the Workbook rely on examples from
social work, education, psychology, counseling, nursing, and medicine. Consequently, this
Workbook should be useful to a variety of disciplines and professions. The exercises
demonstrate principles universal to all helping professions. They are designed so that students
can remove selected portions from the Workbook to hand in as assignments for you to review.
Room is provided on material to be completed by students and handed in for the student’s name,
date, and instructor. We welcome your suggestions for improvement and descriptions of your
experience with the exercises. We are sure that many can be improved.
A workbook format provides practical hands-on experience designed to help students to
transfer principles of critical thinking skills to practice. For example, students may learn how to
avoid common fallacies in reasoning such as anchoring and insufficient adjustment. If, after
entering practice, they avoid making snap judgments in the initial minutes of their contact with
clients, then we might say that content about anchoring and insufficient adjustment transferred to
real-life settings. Transfer does not happen automatically. Deliberate practice is needed to
develop and maintain skills (Rousmaniere, Goodyear, Miller, & Wampold, 2017). Active
participation is needed in a “culture of thoughtfulness” (Perkins, 1992; Lipman, 1991). (See also
Brudvig et al, 2013; Horsley et al., 2011; Tiruneh et al, 2014.)
Arrange many opportunities to apply critical thinking to diverse situations. The more
chances that students get to apply new knowledge, the better.
Make learning situations as similar as possible to real-life ones.
Use varied examples that represent different situations in which knowledge and skills are
of value.
Make learning situations vivid and realistic. Games, simulation exercises, and laboratory
work can enhance participation and transfer.
Teach transfer skills. Remind students of the forest (other situations outside of class)
while teaching about trees (specific lessons). Emphasize general principles and arrange
many opportunities to practice them. Teach students “metacognitive skills that enhance
transfer such as asking “Can I use _____ here?”
Expect a tension between demands that you “cover the material” and your efforts to teach
critical thinking skills. Because teaching critical thinking implies learning at the evaluation,
synthesis, and analysis levels, expect students who have been learning at the recognize, recall,
and translate levels to express their uncertainty and discomfort, especially when it comes to
5
evaluating their critical thinking skills. Be prepared to feel a bit lonely and misunderstood at
times. Critical inquiry involves asking questions others may avoid asking such as “Does it
work? How do you know?” If you teach students to evaluate arguments, to look for evidence
before tossing off a conclusion, to avoid being taken in by emotional appeals, to spot common
fallacies in professional reasoning, and to locate evidence for themselves; then you and your
students may feel out of step in educational settings in which unexamined pronouncements may
be the norm.
Although recent years have witnessed increasing revelations of the prevalence of inflated
claims in the professional literature, including peer reviewed publications (e. g., Ioannidis, 2016),
combatting such claims is an ongoing challenge. Let’s say you and your students have just
completed Exercise 5, which demonstrates how a propagandistic human service advertisement is
the polar opposite of critical appraisal of claims and arguments. Your students may encounter an
instructor who shows the same advertisement to support a claim that a program works. Consider
another example. Students often ask if they can miss a class to go to professional conferences.
These conferences may concern a new treatment method. You may say, “… Of course, but will
you ask about the effectiveness of the method at some time during the conference? In years of
making this request, only one student had the courage to ask; our students have thought it
impertinent to ask about effectiveness.
Focus on the purpose of critical thinking: to help clients and to avoid doing harm.
Keeping these purposes in mind should help you and your students to have the courage to ask
questions about claims, to devote the time and care necessary to critique claims, to learn how to
spot and avoid faulty reasoning, to examine assumptions, analyze arguments, and listen carefully
to the ideas of others.
References
Brudvig, T. J. Dirkes, A., Dutta, P., & Rane, K. (2013). Critical thinking skills in health care
professional students: A systematic review. Journal of Physical Therapy Education,
27, 12-26
Horsley, T., Hyde, C., Santesso, N., Parkes, J., Milne, R., & Stewart, R. (2011). Teaching
critical appraisal skills in healthcare settings. Cochrane Database of Systematic
Reviews 2011, Issue 11.
Rousmaniere, T., Goodyear, R. K., Miller, S. D., & Wampold, B. E. (Eds.) (2017). The cycle
of excellence: Using deliberate practice to improve supervision and training.
Hoboken, NJ: John Wiley & Sons.
Tiruneh, D. T., Verburgh, A., & Elen, J. (2014). Effectiveness of critical thinking instruction
in health education: A systematic review of intervention studies. Higher Education
Studies, 4, 1-17.
`
1
__________________________________________________________________________
Exercise 1: MAKING DECISIONS ABOUT INTERVENTION
________________________________________________
OVERVIEW
Respondents compare criteria they would like their doctor to rely on when making
recommendations with criteria they themselves rely on when selecting assessment and
intervention methods in their own practice. This exercise encourages readers to be explicit about
the criteria they value and to reflect on differences that may emerge in criteria used in different
situations. A lack of generalization of knowledge and skills from one area to another is a key
obstacle in encouraging critical thinking. Data gathered from a sample of social workers can be
seen in Gambrill and Gibbs (2002).
PURPOSE
1. To review criteria for making practice decisions.
2. To compare criteria used in a personal decision and criteria used in one's own practice. We
hope that this exercise will help you to demonstrate to students that professionals are often
inconsistent in the evidentiary standards they use. That is, when it concerns their own
interest (compared with the interests of clients), they may rely on more rigorous criteria.
MATERIAL REQUIRED
1. Making Decisions About Intervention questionnaire (Practice Exercise 1).
2. Scoring Instructions.
TIME REQUIRED
Five to ten minutes for the questionnaire and 30 minutes for discussion.
SUGGESTIONS
This exercise has had the most impact when respondents were given no background explanation
and were just asked to complete the questionnaires. First, hand out Part 1 of Exercise 1 and ask
students to note their name on the form and complete it. Collect these forms. Then hand out Parts
2 and 3 (regarding their clients) and ask them to put their names on them and complete the
questionnaire. We have found that when students are asked what criteria they would like their
dentist or doctor to use in making a decision that affects them personally (or a family member or
friend), they opt for criteria based on scientific data. When making decisions about their own
clients they often appeal to intuition and anecdotal experience (Gambrill & Gibbs, 2002).
Respondents usually score highest on Situation #1, indicating a preference for scientific criteria
2
in personal situations. There are many reasons why different criteria may be appealed to (e.g.,
there may be no controlled experimental studies). Helpers may not realize they use different
criteria when making personal and professional decisions.
POSSIBLE ANSWERS TO FOLLOW-UP QUESTION
If your scores differs in the three situations, why do you think this is so? Respondents have had
three responses to this question:
"I didn't think of the ethical implications of demanding higher standards for evidence for
myself than for my own clients! "
We suggest a hearty, "You've gotten the point of the exercise." Review problems with
basing beliefs on questionable criteria such as testimonials (item 7), case examples (item
3), what fits personal style (item 5), or what is usually offered (item 6). This exercise
provides an opportunity to discuss the difference between analytic and intuitive thinking
described in Part I of the Workbook. Clients prefer counselors who base decisions on past
track records of personal experience and research (O'Donohue, et al, 1989). This exercise
can also be used to discuss criteria relied on to select assessment and evaluation methods.
It can be combined with a discussion of related ethical concerns.
"I don't have time to search the literature."
Possible reply: "Computer technology and seeking high-quality reviews makes it easier to
discover practice-related research." (See for example, Cochrane and Campbell
databases.)
"The same quality of evidence does not exist for problems in social services as in
medicine."
Possible reply: "This may be so, but we are still ethically responsible to know what
evidence does exist and to evaluate progress with each client. Research in the helping
professions shows that we may harm as well as help clients .
Scoring Instructions
Score answers in each situation on the Making Decisions About Intervention questionnaire
(Exhibit 1.1) using the following key (scores may range from -7 to + 10):
For each of items 1, 2, 3, 5, 6, 7, 9 _________
marked, subtract 1 point
For item 4 marked, give 3 points _________
For item 8 marked, give 5 points _________
For item 10 marked, give 2 points _________
3
Add these values, being sure to subtract __________
the negative values, for a total
1
_____________________________________________
Exercise 2:REVIEWING YOUR BELIEFS ABOUT KNOWLEDGE
__________________________________________________________________
OVERVIEW
This exercise offers readers an opportunity to review their beliefs about what knowledge is and
how it can be obtained. Readers review questions about knowledge and learning to identify
beliefs that may interfere with helping clients. Some questions are from W. Gray (1991)
Thinking Critically About New Age Ideas. Our beliefs about knowledge influence the criteria we
use to review arguments. Consider debates about different ways of knowing ("What is
evidence?). Our beliefs about knowledge influence the criteria we use to make decisions. It does
little good to encourage students to consider research related to life-affecting decisions if they
don't believe any knowledge is available and are not aware of informed consent obligations or do
not value these. Some questions such as “Is there a God?", cannot be falsified -- we can't find
out if they are accurate or not. Others such as: "Is cognitive behavioral treatment of depression
more effective than medication in decreasing depression?", can be investigated.
PURPOSE
To critically review beliefs about knowledge (what it is and how to get it) so that available
knowledge will be considered in helping clients and avoiding harm. Additional items have been
included in this new version to highlight the need for skepticism in reviewing all material,
including the peer-reviewed literature (see items 3, 6, 9). NOTE: missing text in No. 9 is “found
to be accurate.”
MATERIALS REQUIRED
1. Reviewing Your Beliefs About Knowledge questionnaire (Practice Exercise 2).
2. Suggested Answers and Reasons for Beliefs About Knowledge.
TIME REQUIRED
Five to ten minutes for the questionnaire and 30-60 minutes for discussion.
SUGGESTIONS
This exercise addresses beliefs related to critical inquiry. Its importance can be highlighted by
emphasizing that our beliefs influence our actions, both in our work and personal lives. It might
be useful to tie Exercises 1 and 2 together because if students don't believe there is any
knowledge to draw on in social work for example but want a physician to base decisions on
experimental evidence when they have a medical problem of their own, then we have a
contradiction. Structural theorists such as Mullaly (1993) emphasize the importance of helping
2
students to identify contradictions between their beliefs and their actions (e.g., abhorring
oppression but valuing a capitalistic economic system that encourages it). This exercise also
provides an opportunity for you to review your beliefs about knowledge. And it provides an
opportunity to consider the role of ignorance as well as knowledge, including its strategic use.
Suggested replies when discussing answers:*
1. Disagree. It is true that our beliefs influence what we see. The methods of science are
designed to ferret out bias. For example, people who disagree about putting fluoride in
water can search for results of laboratory tests and experiments regarding fluoride and tooth
decay.
2. Agree. It is true that there are certain kinds of questions that we may never be able to
answer (e.g., Is there life after death?). However, there are many things that we can find out
about. If we develop the means, the tools, the measures, we can often discover what we do
not now know. It was inconceivable at one time that a surgeon could ever transplant a
human heart; now we do so regularly. At one time we knew nothing about the structure of
the moon; now we have gone there and returned with samples of its structure. The inconceivable
often becomes commonplace.
3. Disagree: Hiding limitations is common (see Ioannides, 2005, 2016). This emphasizes the
importance of critically appraising research for yourself.
4. Disagree. Everything is not possible. By definition, a circle is not a square. It is not
possible to walk through walls or to levitate. Some events are much more likely than
others. Skepticism helps us to determine which events are most likely.
5. Disagree: Ignorance and knowledge are not mirror images. They may for example, be
used for different purposes in different ways.
6. Disagree: False claims are common, even in peer-reviewed publications (e. g., see
retractionwatch.com).
7. Disagree. It is true that there is much we do not know about behavior. However, by testing
our assumptions about behavior we have discovered much about behavior as illustrated in
books describing the principles of behavior. We are beginning to understand biochemical
influences that affect synapses in human nerve cells. Social norms make some behavior
predictable.
8. Disagree: Both knowledge and ignorance can be strategically used to influence people.
For an example of the latter, see Oreskes and Conway (2010).
9. Disagree: Much is of questionable accuracy, so it is important to review it carefully.
10. Agree. Not all claims can be investigated through scientific methods (e.g., Does God exist?
Is it evil to assault people?). However, many claims are testable scientifically. Such claims
should be put in a testable form. Vague questions cannot be answered.
3
11. Agree: For example this permits informed consent and suggests research directions. Can you think
of an example from your own life?
12. Disagree. Neither helpers nor their clients can know whether services are useful unless they
evaluate progress. Consider a doctor who is treating a patient for diabetes and wants to test the
patient's blood sugar. How can the patient or the doctor know if the treatment is working without
data? The same goes for depression and other problems. Informed consent requires helping clients
determine if services have resulted in outcomes clients value.
13. Disagree: Such experience often does not provide the careful comparison required to critically test
claims about what is true and what is not.
14. Agree: Consider for example neuroscience, astrology.
15. Disagree: Interest groups do influence social policy. However, policymakers often review
scientific evidence related to a problem. Also, many citizens seek and review research data related
to questions.
16. Disagree. Science is a way of approaching problem-solving. It involves critically testing
claims. If an element of intelligent life came to us from one of Jupiter's moons, that
creature's society would have to guess and test as our scientists do to solve problems.
Scientific reasoning (guessing and testing) has been used all over our planet by persons of
all races to solve problems. Eskimos survived because they relied on basic principles of
science (Nelson, 1993).
*We thank Michael Hakeem, Professor-Emeritus of the University of Wisconsin at Madison, for
his suggestions on some of the answers.
_____________________________________________________________________________________
Exercise 3: CONTROVERSY ______________________________________________________________________________
OVERVIEW
Making informed decisions often requires consideration of different views. Enhancing related
skills will help students to participate effectively in discussions and arrive at well-reasoned
decisions.
PURPOSE
To enhance students’ skills in constructive controversy.
MATERIAL REQUIRED (Depends on Activity Selected) (see p. 83-84).
1. Guidelines for discussion in text.
2. Access to Internet/databases.
TIME REQUIRED
Depends on activity selected.
SUGGESTIONS
Seek other material that requires constructive controversy. Procon.org lists many debate topics.
See also newseumed.org; from provocative to productive: teaching controversial topics. Steps in
constructive controversy suggested by Johnson and Johnson (undated) include the following:
1. Organize students into groups of four, divide each into two pairs, assign pro position to one
and con to the other.
2. Ask each pair to research and prepare a position statement and to present their position.
3. Engage in open discussion.
4. Reverse perspectives and present other position, adding new information as available.
5. Synthesize and integrate best evidence and reasoning in a joint position statement (p, 116).
Promote genuine discussion by making sure students are interested in the topic. See also
www.crit.umich.edu. “Discussion-based teaching and handling controversial topics in the
classroom.” Some tips include: allow everyone a chance for input, do not interrupt other (unless
violating norms), establish limits for one person to talk, hand out cards with helpful hints (e. g.,
“Were reasons for a position given?”).
Johnson, D. W. & Johnson, R. T. (2015). Constructive controversy: Energizing learning. Small
Group Learning in Higher Education (pp. 114-121).
____________________________________________________________________________________
Exercise 4: CRITICAL THINKING AND ADVOCACY
__________________________________________________________________
OVERVIEW
Advocacy in which practitioners pursue enhancement of quality of services and/or clients’ lives
is an important component of being a helping professional. There are many kinds of advocacy
including urging staff in other agencies to provide requested help, pursuit of legal action, and
seeking agency changes that benefit clients.
PURPOSE
To emphasize the importance of advocacy efforts; to give students an opportunity to engage in
related planning/action.
MATERIAL REQUIRED
1. Exercise 4.
2. Content in text.
TIME REQUIRED
Depends on activity selected.
SUGGESTIONS
Emphasize the importance of the need for advocacy and the wisdom of working together with
others to increase success. Students may have special interests, for example involving clients in
advocacy efforts, or making the reporters to expose avoidable miseries. As a start they may
work together with others to clearly depict a harmful situation, describe who it affects and who
or what organizations are responsible for the unjust situation, and suggest remedies.
1
_______________________________________________________
Exercise 5: CRITICALLY APPRAISING HUMAN-SERVICES
ADVERTISEMENTS _____________________________________________________________________________
OVERVIEW
This exercise uses a persuasive advertisement and a form for rating this. Discussion questions
illustrate concerns about using advertisements to guide decisions.
PURPOSE
1. To demonstrate characteristics of human-services advertisements.
2. To teach students to rate such evidence against criteria for identifying advertisements.
MATERIALS REQUIRED FOR ACTIVITY 1
1. There are probably groups of practitioners, hospitals, and organizations in your area that
produce advertisements for their programs that you can use for teaching material. You
could draw on promotional material on the Internet and/or in magazine and newspapers that
advertise weight loss, study skills, smoking cessation, and other types of self-help
programs. Videotaped presentations at professional conferences may meet the criteria for
an advertisement: A charismatic and well-known person describes an intervention, presents
the method in an entertaining way, and does not raise the issue of effectiveness. We
recommend collecting audiovisuals of related material to use for demonstrations,
examinations, and quizzes.
2. DVD or videotape player and screen or monitor.
3. Copy of Human-Services Advertisement Spotting Form (Practice Exercise 5).
4. Scoring Instructions.
TIME REQUIRED
For Activity 1 about ten minutes to show the promotional material, five minutes to rate it, five
minutes to score the Human-Services Advertisement Spotting Form, and 20 minutes for
discussion.
SUGGESTIONS
Step 1. Obtain an example of a human-services advertisement. See for example website of
Rogers Memorial Hospital.
Step 2. Ask the class the following:
"Please watch this and circle your answers to the ten items on the Human-Services
2
Advertisement Spotting Form:" (Give the class a few minutes to examine the form before
you begin showing the advertisement.) Try not to imply any bias for or against the
material.
Step 3. Show the material and allow the class a few minutes to complete the form.
Step 4. Score the responses. (See scoring instructions below.) The higher the number of points
on the form, the more likely the source is a human-services advertisement. Generally, if
the source scores more than five points it is an advertisement.
SUGGESTED ALTERNATE PROCEDURE
Show the advertisement before students are given the Workbook. Show them the following
questions before presenting the material and allow participants to refer to them as they view
material.
1. What is the one central conclusion that the makers of the advertisement would have you
draw regarding their program?
2. Based on this material, would you refer your clients to this facility? Yes_ No _
3. Why or why not? (Explain your answer to question 2.)
SCORING THE SUGGESTED ALTERNATE PROCEDURE
For question 1, give five points for identifying the conclusion that the program is
effective, or that persons should refer their clients to the program because it is effective.
For question 2, give five points for a "no" answer.
For question 3, give five points for every methodological concern raised, including:
allusion to an emotional appeal, use of case example or testimonial, manner or style of
the presenter, use of music, one-sidedness, absence of data regarding effectiveness of
the program, need to measure success, experimental designs that might be used to
evaluate the program. Rank (1982, p. 147) identified five universal aspects of
advertising that include the following: 1) Attention-Getting, 2) Confidence-Building, 3)
Desire-Stimulating, 4) Urgency-Stressing, and 5)Response-Seeking. (The Key for
Exercise 6, may help here).
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
Which features of an advertisement does the material demonstrate?
_____________________________________________________________________________
Answers to Activity 1 Exercise 5 _____________________________________________________________________________
3
For each "Yes" answer to items 1, 3, 4, 5, 8, or 9 on the Human-Services Advertisement Spotting
Form give 1 point. For each "No" answer to items 2, 6, 7, or 10, give 1 point. Total the number
of points. Scores can range from 0 to 10. Record this value on the bottom of the Human-Services
Advertisement Spotting Form. If the source scores more than 5 points, it is an advertisement; the
higher the point value, the weaker the evidence.
1
__________________________________________________________________
Exercise 6: DOES SCARING YOUTH HELP THEM "Go STRAIGHT"?
_________________________________________________________________
OVERVIEW
This exercise evaluates skills in reasoning critically about an emotional appeal regarding the
claim that a delinquency prevention method works. It evaluates respondents' ability to apply
basic principles of critical thinking to a practice decision. Participants watch promotional
material and answer questions related to whether they would refer clients to the program. The
material violates most major tenets for thinking critically about the effectiveness of a treatment
program. It is pure propaganda. The exercise relies on the "Scared Straight" film or videotape
(not included but hopefully available in university libraries or interlibrary loan). This exercise
can be used as a teaching lesson and as a measure of critical thinking.
PURPOSE
To measure skills in thinking critically about case material when faced with a powerful
emotional argument that a method works. This case material was selected because its artful
emotional appeal has no value as scientific evidence regarding the program's effectiveness; yet,
its central theme argues constantly and persuasively that the program is effective. The viewer
who can resist its emotional appeals, vagueness, one-sided presentation, oversimplifications,
persuasive manner, case examples, testimonials, appeals to numbers, and so on, without being
sucked into this illogical morass, demonstrates powerful critical thinking skills.
MATERIALS REQUIRED
1. A VHS videotape or DVD player or laptop.
2. A VHS or DVD videotape that describes the Juvenile Awareness Project at Rahway State
Prison in New Jersey. The "Scared Straight" material is available at many libraries.
Otherwise, the film or videotape can be purchased. Showing the film/video on a large
screen is more effective than watching it on a small laptop screen.
3. A suitable room, where lighting can be adjusted so that respondents can sit comfortably
while viewing the film or videotape as they take notes. The room should provide sufficient
space between respondents to discourage distractions by a neighbor.
4. Two sheets of 8.5-by-ll-inch lined paper (one for notes and one for the respondent's
answer).
5. Blackboard or projector to display questions for easy reference during the test.
6. Pen (scoring and reliability checks are more legible with ink than with pencil).
7. Scoring instructions. Up to 50 points are possible-5 points each for items 1 and 2, and 40
for item 3. (See scoring instructions below.)
2
TIME REQUIRED
About fifty minutes to show the material, 15 minutes to answer, 10 minutes to score responses,
and 20 minutes for discussion.
SUGGESTIONS
We suggest two uses for this exercise. Many, in spite of their efforts to think critically, are taken
in by its nonstop emotional appeal. This presentation is the antithesis of good evidence about the
effectiveness of a program. Anyone who would subject clients to the program described by
saying "yes" to item 2 without better evidence has been taken in by emotional appeals providing
no empirical evidence regarding the program's effectiveness. Indeed, research suggests that
"Scared Straight" programs do more harm than good (Petrosino, Turpin-Petrosino, & Buehler,
2003).
This exercise can also be used as a measure of critical thinking. Studies have found high inter-
rater reliability. Two undergraduate research assistants independently scored responses from
students in five research methods classes from four universities. Their Pearson r reliability
coefficients were: r = .96, n = 19; r = .89, n = 17; r = .83, n = 18; r = .87, n = 35; r = .90, n = 26
(Gibbs, Gambrill, Blakemore, Begun, Keniston, Peden, & Lefcowitz, 1995). Students in labs of a
Social Work Research course at the University of Wisconsin at Eau Claire, without any prior
training, scored their own and their partner's responses with these r values: r = .98 (n = 7), r =
.65 (n = 7); r = .64 (n = 9). Students (n = 20) in an honors course on Thinking Critically about
Social Problems at the same university scored themselves and a partner without any prior
training and obtained an r of .94.
Try not to offer any cues that might imply a positive or negative attitude toward the material. A
matter-of-fact style works best. Set up and test the audiovisual equipment before students enter
the room, and have all materials available. Hand each student the pen and two sheets of paper,
and explain: "I would like you to react to some case material. You may use one of these pieces of
paper for your notes. The other is for your answer to three questions that I will give you. Please
write clearly."
Either write the questions in Practice Exercise 6 (in the Workbook) on the blackboard or project
them. All respondents should be able to refer to the questions during testing, but should not be
allowed to take a copy of the questions from the classroom (to avoid compromising the
measure).
Read the questions aloud to make sure that respondents are familiar with them. If students ask
questions, tell them the following: "Please reread the question carefully, and base your
interpretation on what the question implies to you. I can't help you with an interpretation because
it might bias the result." Tell respondents: "When the audiovisual material is finished you will
have 15 minutes to write your answers to the two questions." (Be sure to set your timer when
they begin and give them exactly 15 minutes.) After the 15 minutes, inform them: "The fifteen
minutes are up. Please finish the sentence that you are writing and hand in your paper."
3
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
1. What was your score?
This exercise has been administered to students in five research methods classes at four
different universities with these results:
n = 19, mean = 6.6, sd = 3.6; n = 17, mean = 8.6, sd = 8.6; n = 18, mean = 12.1, sd = 3.7; n =
35, mean = 10.8, sd = 7.1; n = 26, mean = 30.2, sd = 7.4 (Gibbs, Gambrill, Blakemore,
Begun,, Keniston, Peden, & Lefcowitz, 1995). In classes in which students have had no
exposure to critical thinking, it is common for all to get five or fewer points.
2. What is the main kind of appeal in the "Scared Straight" material? The scoring key identifies
the dominant forms of evidence as pure propaganda.
3. Why did you respond as you did to the "Scared Straight" presentation?
Discuss the powerful emotional appeals, including violent street language, physical
intimidation, close-up to juvenile's tearful face, slamming doors, vivid news accounts, Peter
Falk's tough-sounding talk and voice, before/after talks with juveniles, careful editing,
dramatic camera angles, and so on.
4. Do you think the Juvenile Awareness Project might have produced harmful (iatrogenic)
effects? Share results of Petrosino, Turpin-Petrosino and Buehler (2003)'s study.
5. Is this exercise a valid test of critical thinking? Explain your answer.
We think that it is because it asks for a decision based on evidence.
SCORING INSTRUCTONS
Up to 50 points are possible: 5 points for each of items 1 and 2; 40 for item 3.
Item 1
Give 5 points if the respondent states in any way that the program is effective at preventing
delinquency, otherwise no points. Here are examples of five-point responses:
The results of this program 'are unparalleled by conventional treatment methods.
The juveniles really go straight.
The program is effective.
The program helps juveniles.
The program really works.
Give zero points for other conclusions that do not state that the program is effective at
delinquency prevention, for example:
The program is inexpensive.
4
The Lifers enjoy their work.
Convicts and prison officials can work together.
I would be scared too.
Item 2
Give 5 points for a "No" answer. Give zero points for a "Yes" answer.
Item 3
It would be a daunting task to name and explain all of the fallacies of reasoning in the "Scared
Straight" audiovisual. However, ten prominent fallacies are named and explained below. Give 2
points for using a key word from the list of key words, but give only 2 points for the first occur-
rence of a key word in each group. Give 2 points for each explanation that clearly describes how
a particular fallacy is present in the presentation.
1. Key word: Emotional Appeal, Visual Emotional Appeal, Appeal to the Emotions,
Shocking Emotional Appeal (2 points for just one of these).
Explanation: This presentation is a nonstop assault on the emotions. Examples include
shocking street language, slamming doors, homosexual taunts, vivid descriptions of
atrocities in prison life, shots from an angle showing huge inmates towering over
juveniles, a tear in the eye of a juvenile; these are all ways that audiovisuals heighten
emotional impact. Such emotional appeals are distracting and irrelevant to the central
question: Does the program work? Give 2 points for any statement that critiques such
emotional appeals as reasons to accept a claim, or notes that they distract attention from a
critical review of evidence concerning program effectiveness.
2. Key word: Status, Titles, Well-known Person, Familiar Person, Ad Verecundium, Appeal
to Authority (2 points for just one of these).
Explanation: Very prominent people, including the warden, Judge Nicola, and actor Peter
Falk, say the program is effective. Saying so doesn't make it so. Just because prominent
people say the program prevents delinquency does not mean it does. Prominent people
are bound by the same rules of evidence as anyone else. Give 2 points if the respondent's
answer states somehow that the status of persons arguing for the program's effectiveness
does not assure that their argument is true.
3. Key word: Manner, Style, Sincerity, Sincere (2 points for just one of these).
Explanation: The fallacy of manner assumes that the persuasive manner in which a
person presents an argument assures the argument's validity. Examples include inmate's
talk about their sincere efforts to help the juveniles (e.g., "my apostolate work"), and how
the announcer had a sincere way of describing the program and its effects. Give 2 points
only if the respondent comments about the manner of the inmates, officials, or juveniles
and debunks the idea that the manner supports the claim that the program is effective.
5
4. Key word: Vagueness, Vague Term, Undefined Term, Outcome Not Defined, Outcome
Not Measured (2 points for just one of these) .
Explanation: Although the term "straight" is used throughout the presentation, "straight"
is never defined. Nor is the term "successful." What precisely does "straight" mean? Who
determined this outcome? What criteria for "straight" were used? Were court records
consulted? Where? Claims that a program works must be judged against a clearly
defined, valid, and reliable outcome measure. Give 2 points for any answer that implies
confusion about what "straight" or "success" means, or if the respondent asks for a
definition of this term.
5. Key word: Testimonial(s) (2 points for using this term).
Explanation: Testimonials involve asking recipients of a program to tell how the program
has affected them. Testimonials are a weak form of evidence because: (1) Those giving
testimonials are subject to pressure to say positive things about the program, (2) those
chosen to give testimonials may have been hand-picked to say favorable things, (3)
testimonials rely on emotional appeal because the viewer can identify with the people
involved in the program, (4) editors can add and remove sections of material (film, tape
or written material) to support a favored position, and 5) a testimonial is a sample of 1 so
violates the law of large numbers. (See Exercise __.) The juveniles gave testimonials
when they described how scared they were and how they were going to stop their
delinquent behavior. Give up to 2 points for mentioning anyone of the above weaknesses.
t
6. Key word: Ignorance of Possible Harmful Effects, Possible Iatrogenic Effects, Possible
Harm, May Have Harmed (2 points for just one of these).
Explanation: Not only may the program have been ineffective, it may have harmful
effects; that is, make the delinquents more delinquent. Give 2 points for any mention that
the program might harm the delinquents.
7. Key word: Case Example, A Few Cases, Case Example Fallacy, Just a Few Cases (2
points for just one of these).
Explanation: Case example involves showing just one or a few unrepresentatively chosen
individuals and generalizing to all such individuals. Case example is weak because (1) it
encourages us to rely on specific emotion-arousing material (e. g., the muscular juvenile
who was stared down and said "You would kill me") that may bear no relation to
soundness of an argument, (2) cases can be hand-picked because their behavior will
support a bias, and (3) the sample is small (violates the law on large numbers). Give 2
points for recognizing the weaknesses of presenting 17 hand-picked cases and then
generalizing what may be true of these individuals to the whole program.
8. Key word: Appeal to Numbers, Popularity, Peer Pressure, Bandwagon, Because So
Many, Because So Few (2 points for just one of these).
6
Explanation: Appeal to numbers is a fallacy when the number of persons exposed to the
treatment is given in hopes of convincing the viewer to use the method. An appeal to
numbers can argue that many have been exposed to the treatment, and therefore more
should be. An appeal can also be based on the idea that few such wonderful programs are
now available, and others should hurry and start one. Give 2 points for correctly
identifying that claims that thousands have passed through the program, or that only a
handful of such programs exist, say nothing about the program's effectiveness.
9. Key word: Ignorance of Better Evidence, Not Asking for Better Evidence, Ignorance of a
Hierarchy of Evidence, Accepting Poor Evidence, Ignorance of Scientific Evidence,
Ignoring Evidence (2 points for just one of these).
Explanation: The propaganda appeals in "Scared Straight" are among the weakest that
could be given regarding the effectiveness of a program. Give 2 points for requesting any
form of scientifically better evidence (e.g., a study that includes random assignment to
treated and control groups, objective observers, keeping systematic records of outcome,
reported loss of follow-up rate, reliable and valid outcome measure, and so on).
10. Key word: Lack of Objectivity, Bias, Biased, One-Sided, Stacking the Deck, Loading the
Dice (2 points for just one of these).
Explanation: Critical thinking requires that evidence and counterevidence be examined in
an evenhanded way. One must look at all evidence and arguments regarding the
effectiveness of a program, not just evidence favoring it. This presentation argues
nonstop, from Peter Falk's introduction to Judge Nicola's conclusion, that the program is
effective. Give 2 points if the respondent refers to how one-sided the evidence was or
expresses a need for evidence about how the program might not have been effective.
__________________________________________________________________
Exercise 7: DETECTING MISLEADING PROBLEM FRAMING
__________________________________________________________________
OVERVIEW
What problems are selected for attention and how they are framed influences clients’ options. It
is important to increase students’ understanding of the social construction of problems and
identify what problems may be ignored.
PURPOSE
To enhance students’ understanding of the importance of thinking about who decides what
situation comprises a problem and what kind. To enhance understanding of political, social and
economic factors related to problem construction.
MATERIAL REQUIRED
1. The century of the self by Adam Curtis (2001) is available on YouTube. Decide what parts
you want students to watch if they do not have time to watch the entire program.
TIME REQUIRED
Time to watch and discuss presentation.
SUGGESTIONS
Try not to rush this activity. You must decide whether to show this in class or to ask students to
watch it on YouTube. Discussion time is important.
__________________________________________________________________
Exercise 8: FOLLOWING THE MONEY
__________________________________________________________________
OVERVIEW
Many corporations are involved in the helping professions. An interest in profit may contribute
to goal displacement including promotion of services that harm rather than help clients.
Following the money (who gets paid for what) is valuable in understanding sources of goal
displacement.
PURPOSE
To increase students’ attention to pursuit of profit in the helping professions which may
compromise quality of services.
MATERIALS FOR ACTIVITY 1
1. Access to the Internet.
2. Content in text.
TIME REQUIRED
Time on the Internet will be required for this in addition to class presentation time for “follow-
the-money” reports.
SUGGESTIONS
Model tracking of money involved in a service for students. This will introduce them to valuable
web resources they can use in their searches. You could also introduce students to key resources
of fraud and corruption such as Transparency International.
__________________________________________________________________
Exercise 9: THE LANGUAGE OF PROPAGANDA
_________________________________________________________________
OVERVIEW
Language influences what we do and feel as Orwell (1984) describes in Politics and the English
Language (1946).
PURPOSE
To increase students’ awareness of how words affect emotions and decisions.
MATERIAL FOR ACTIVITY 1
1. Content in text including list of language ploys.
2. Material to be reviewed.
TIME RQUIRED
Depends on how many students review their material.
SUGGESTIONS
Review language ploys in class. Have students read the Politics of the English language (1946)
and discuss this in class, applying content to current topics in the helping professions. Ask
students to read On Bullshit (Frankfurt, 1986) and discuss how bullshit differs from lying.
Encourage students to identify language ploys and discuss how students can minimize the
influence of language. See also other suggested activities. Ask students to consult social
psychological studies regarding effects of language on emotions,
1
__________________________________________________________________
Exercise 10: USING THE PROFESSIONAL THINKING FORM
__________________________________________________________________
OVERVIEW
This 25-item short-answer questionnaire contains brief vignettes that may or may not contain a
practice fallacy (common error in reasoning). The clinical reasoning, critical thinking, and
informal logic literature describes common errors. The Professional Thinking Form (PTF) can be
used to generate c1assroom discussion. It can be self-scored or used as a research instrument.
PURPOSE
1. To test students' skill in identifying common practice fallacies.
2. To teach how to identify fallacies in reasoning about practice.
MATERIALS REQUIRED
1. The Professional Thinking Form (Practice Exercise 5).
2. Ballpoint pen (for greater clarity than pencil for reliability checks).
3. Scoring key (see Answers to Exercises).
TIME REQUIRED
Up to 30 minutes for the PTF, 10 minutes to score it, 20 minutes for discussion.
SUGGESTIONS
The Professional Thinking Form was developed as a measure and teaching tool. It contains 25
brief vignettes illustrating common fallacies and pitfalls that cloud thinking across the helping
professions (e.g., attacking the person, accepting case examples as evidence, selection bias).
Documentation showing how PTF's fallacies negatively affect judgments can be found in related
literature (e. g., in medicine and psychology). PTF's vignettes illustrate fallacious thinking in
examples from social work, community organization, mental health, child welfare, chemical
dependency, family therapy, and other areas.
Each of the vignettes, including two nonfallacy items (5 and 9), and duplications of two fallacies
(4 and 10, 14 and 19) occupy just a few lines of text. Longer excerpts would limit PTF's
classroom utility and make it difficult to score. Shorter excerpts best illustrate just one practice
fallacy.
For discussions in large classes, items can be projected onto a screen for item-by-item
examination. Students can read each item on the screen or refer to their own copy, then comment
2
on what they think might be the fallacy. In our workshops for professionals, we ask participants
to write a response to about four items at a time. We then discuss each item relative to the
scoring key's explanation. Participants contribute additional instructive, entertaining examples of
each.
SUGGESTIONS FOR USING PTF AS A MEASURE
Although PTF was designed as a teaching tool, it can be used as a measure of critical thinking
about practice. PTF's format and some of its scoring instructions are based on an instrument by
Logan (1967). The instrument has undergone several revisions since it was first developed
(Gibbs & Werner, 1988). The version immediately prior to this one was published in Scientific
Reasoning for Social Workers (Gibbs, 1991, pp. 54-59, 274-278). The present version clarifies
scoring criteria, rewrites weak items (based on a Spearman rho item-analysis), and changes the
total possible point value from 115 to 100 points. If used as a research instrument, we
recommend that respondents be informed that "you are completing a measure of critical,
professional, analytical, scientific thinking. . ." and that these accompanying instructions be
given:
1. Your responses will not be used to grade you nor to reflect positively or negatively on you in
any way.
2. Please keep your responses brief and to the point. When you finish, turn your paper over and
leave it that way. You have 40 minutes to complete your responses. (Few people require 40,
but allowing this time takes the pressure off respondents).
Reliability and Validity
Inter-rater reliability has been checked several times. The first reliability trial was conducted by
Gibbs and Werner (198S) on responses from social work and nursing students. This yielded a
Pearson r = .89 for the first trial (p < .0001, n = 21), and r = .89 for a second trial (p < .0001, n =
31). Spearman rho for independent scoring of each of 20 items on PTF ranged from .46 to .99
(mean = .78, sd = .13) on the first trial and from .65 to .95 (mean = .80, sd = .08) on the second
trial. The current version has been revised in light of results from item analysis. Three items were
revised extensively (because they were weak), and two new nonfallacy items were added.
Our primary claim to PTF's content validity rests on how well items and references in the key
reflect fallacies and concepts in the critical thinking and clinical reasoning literature. Thelen
(1993) tested correlations between PTF and both the Watson-Glaser Critical Thinking Appraisal
(1980) and cumulative grade point average for 97 senior undergraduate nursing students at the
University of Wisconsin at Eau Claire.
Pearson r correlations were .36 (p < .001) and .18 (p < .10) respectively. Presently, we know of
no applications to more or less competent criterion groups. It might be informative to see if
practitioners with different levels of training and experience score differently on PTF.
Scoring PTF For Measurement Purposes
3
Scoring takes about six minutes for experienced scorers. It can go faster with fewer errors if
respondents use a nongooey, legible ballpoint pen rather than pencil to record their answers.
Items are scored from 0 through 4. Here is a way to assign point values for each item:
No points: The item is not answered, nor is the thinking in the vignette questioned in a
specific way, nor is a key word used (e.g., answers to Situation 1: "OK" or "I like to attend
entertaining workshops where the presenters are funny").
One point: The answer raises approximately the same question as the one raised in the
explanation but does not use a key word (e.g., answer to Situation 1: "I wonder about accepting:
Steinberg's argument based on the way she presents it").
Two points: The respondent uses a key word identically, or almost identically, to one or
more words in the Key Words to be Located in Answers Column (e.g., answers to Situation 1:
mannerisms, manner, charismatic, charisma).
Or, the answer raises the same question that is raised in the explanation, but does not use
a key word (e.g., answer to Situation 1: "Steinberg's way of presenting herself does not assure
that her argument is true. We need to see her evidence").
Three points: A close or approximate answer gives a key word.
Four points: Both a key word and the explanation match the key (e.g., answer to
Situation 1: "Steinberg's style of presentation does not assure that her argument is true. We need
to see her evidence").
POSSIBLE ANSWERS TO FOLLOW-UP QUESTION
Do any of PTF's vignettes reflect real situations particularly well? Which one(s)?
Suggestion: You might inform respondents that most of the items reflect real events that have
been altered to protect confidentiality.
4
Answers to Practice Exercise 10
Give 2 points for a key word (only 2 points for one key word in each item). Key words are listed
in bold print above the explanation section. For an explanation that states essentially the same
answer as in the explanation, give 3 points. For an answer that approximates the explanation,
give 2 points. For any answer that questions the item at all, give 1 point. For not questioning the
item at all, give zero points.
1. Manner, style, charisma, stage presence, likeable, likability.
The convincing way a speaker presents an argument does not mean the idea is valid.
The argument must be critically examined.
2. New, newness, novel, tried and true, tradition.
Accepting a method based on newness, or traditional use, ignores the fundamental issue
of whether the method works.
3. Uncritical documentation, relying on citation alone, not examining the document.
Just because an idea can be cited does not prove a point. Examine the contents in the
reference to see if it supports the argument.
4. Ad hominem, personal attack, against the person, attacking the person.
The coworker rejects Hajdasz's evidence because of his personal life, not because of his
evidence.
5. OK, not a fallacy, no fallacy, all right, no problem.
The thinking here is sound. Key terms need definition before an argument can be
understood, so asking for a definition is necessary here.
6. Experience, appeal to experience, all evidence is equally good, ignorance of
scientific evidence, ignoring evidence.
The worker does not discriminate between sound scientific evidence, with its
procedures for controlling bias, and personal experience. Where clinicians' judgments
are compared with actuarial findings (research findings and statistical methods), the
latter consistently outperform the former (Grove & Meehl, 11996).
7. Popularity, peer pressure, bandwagon, numbers, because everybody.. .
Widespread use of a method does not demonstrate its effectiveness. In fact, some once-
popular treatments have been shown to be harmful (e.g., the drug DES caused cancer in
children of some women exposed to it (Berendes & Lee, 1993). When the drug was
5
popular, its advocates argued strongly in its favor and defended it from critics (Dutton,
1988).
8. Appeal to authority, ad verecundium, status titles, degrees, famous, fame.
Because an argument is presented by someone with status or titles does not mean the
argument is true or valid. Famous persons are bound by the same rules for evidence as
everyone else
9. OK, not a fallacy, no fallacy, all right, no problem.
This is not a fallacy because it states a truth about the helping professions: Factors other
than the treatment often account for improvement (Kassirer & Kopelman, 1991, pp.
171-186). Sound causal inferences about the effects of an intervention can be made
only after careful experimentation, painstaking observation, and record-keeping.
10. Ad hominem, personal attack, against the person, attacking the person.
Ms. Washington's argument is rejected because of her personal appearance. Instead, the
validity of her argument should be examined.
11. Objectivity, objective, honest(ly), bias (ed) , vested interest, cheating in science,
unethical.
Consultants should follow scientific principles, without allowing their client's self-
interest to influence the findings.
12. Either/or, only two sides, only two alternatives, false dilemma.
The agency supervisor limits thinking about the problem to allowing discussion of only
one alternative or the other. Both alternatives may be inappropriate, or some other
alternative may be best. A combination of the two alternatives may be appropriate.
13. Testimonial.
Clients who give a testimonial are subject to bias in their statements about treatment
effectiveness. An answer to this item gets full credit by listing any bias (e.g., small
unrepresentative sample, emotional appeal, pressure by a therapist to give a positive
report, biased selection of clients, the testimonial provides no standard for comparison
with other individuals.
14. Hasty generalization, possibly biased sample, unrepresentative, sweeping
generalization.
6
Those who called the planners may not have reflected the whole community's
sentiments. The callers may just have been the most vocal. Reference to the need for a
survey scores two points.
15. Vagueness, unclear term, undefined term.
What does "depressed" mean? Before making a judgment about a client, specific
criteria for making such a judgment should be identified. Some criteria for estimating
degree of depression are specific and reliable.
16. Case example, single case.
Neither what is generally true of thousands of Vietnam-era clients, nor whether a
treatment generally benefits them, or any group of clients, can be inferred from a single
case. Single cases have emotional appeal. Case examples help to foster the erroneous
feeling that, if you know one you know them all. They may be selected purposely to
support almost any generalization.
17. Two questions, double-barreled, two-headed, ambiguous.
This question is confusing because it poses two questions at once.
18. Leading, loaded, biased question.
This question implies a specific answer, thus limiting the client's freedom to give an
accurate answer.
19. Sweeping generalization, hasty generalization.
This general statement (sweeping generalization) is made without any evidence to
support it. Give 2 points for asking any methodological question, for example, "What
does electroconvulsive shock imply? Have studies been done to evaluate this treatment
method? 'Most effective' relative to what?"
20. Tautology, word defines itself, “progress” defines “progress.”
By defining “progress” as being “progress” or “independence” as being
“independence” the reader does not know what specific behaviors are required to gauge
improvement (Fearnside & Holther, 1959, p. 137).
21. Post hoc, after this, temporal order, because treatment precedes improvement.
Simply because improvement follows treatment does not mean that treatment caused
the improvement. Other factors may have caused the improvement; or improvement
may have occurred in spite of the treatment.
7
22. Jargon.
The worker uses professional words that the client cannot readily understand.
23. Regression, regression fallacy, regression to the mean.
Individuals scoring extremely low or high on pretest tend to score closer to the mean on
posttest. This is because extreme scores often have a high error component.
24. Selection bias, biases selection of clients.
RARC’s 75% treatment success may be due to self-selected, higher-functioning clients at
the RARC agency.
25. The probability that abuse will occur (positive predictive value of the ST), given the
Donohue’s Family Positive ST score is:
(Prevalence)(Sensitivity)
=
(Prevalence)(Sensitivity) + (1-Prevalence)(1-Specificity)
(.95)(.03)
=
(.95)(.03) + (.05)(.97)
1
__________________________________________________________________
INSTRUCTIONS FOR REASONING-IN-PRACTICE GAMES
__________________________________________________________________
OVERVIEW
In the RIP Games students work together in small groups to identify fallacies and cognitive
biases in brief practice vignettes. Teams take turns acting out parts in the vignettes.
Countermeasures and "what you could say" in relation to each one are shown in the Answers
sections. Game A concerns common informal fallacies in reasoning. (See also pp. 134-136 of
text.)
PURPOSE
1. To have fun.
2. To learn how to identify and avoid common fallacies or pitfalls in reasoning related to
practice.
MATERIALS REQUIRED
1. Steps and suggested activities.
2. Fallacy definitions for game being played.
3. Answers for game being played.
4. Watch or timer.
TIME REQUIRED
One hour of homework to prepare to play the game by reading the fallacy definitions and game
rules, five minutes in class to review the game rules, up to 90 minutes to play the game (the
game can end at any time based on an agreed-upon time limit).
SUGGESTIONS
We have experience with these games in a variety of contexts including undergraduate and
graduate classes and with faculty and field instructors involved in professional education.
1. We recommend that you follow the vignettes in sequence as they appear in the Workbook.
The order of the vignettes in the Workbook matches the order in the Answers to Exercises
section. Once you are familiar with the game, you might try the option below with the cards
as they appear in this section of the Instructor's Manual.
2. You could duplicate the pages with the vignettes in this manual on card stock at a local copy
shop and make your own sets of cards for each game. You could print each set of game cards
2
on a different color paper. Students could be asked to do the same. Answers could be written
on the back of each card as students practice spotting and countering fallacies before class
use. Or you could make a master list of answers for each game on one card and refer to this
as needed. You could include a brief definition of the fallacy illustrated in each vignette. You
may want to point out the relationship of particular fallacies to particular cognitive
illusions/biases (Pofl, 2004). For example, vivid case examples and testimonials are
"available" in our memory.
A card format allows mixing of cards from games. For example, you may want to select a
given set of cards reflecting a special interest (e.g., in a particular setting). With a card format
it is easy to practice in everyday settings outside of class. The card format makes it easy to
add examples (just write down the situation on a card and number in sequence).
3. Be sure that the moderator has time to read and understand the game rules so that when
questions arise they can be answered quickly. It helps to have an enthusiastic moderator with
a sense of humor.
4. Be sure the moderator has an accurate clock and holds to agreed upon time limits. Initial
vignettes may require a little more time compared to later ones, while participants get used to
the game. Permitting protracted discussions bogs the game down.
5. Make certain that the game players have both the time (approximately one hour) and
the motivation to read each game's rules and the fallacy definitions carefully before
playing the game. On two occasions, where the players were handed the game and were
given only about fifteen minutes to read the fallacy explanations, the game was extremely
difficult with extensive disagreements about which fallacy each vignette represented, even
though the audiences were highly educated. On the other hand, freshman undergraduates had
remarkably consistent judgments (i.e. teams get 80 to 90 points) where they had been given
the time to study the game's fallacies.
6. We tried to design vignettes so they most represent a single fallacy, but features of more than
one fallacy can be found in many vignettes. Students who disagree with the suggested answer
to a situation have an obligation to describe and offer cogent reasons for their view.
7. Emphasize that participants are playing the game not to win over their worthy opponents, but
to learn about how to identify and avoid fallacies and biases in making decisions so they can
be better helpers. Though the game is light-hearted, it concerns matters of grave importance.
If participants share this spirit, they will be more tolerant of (or even welcome) competing
views. Emphasize the importance of understanding why a particular fallacy or bias applies to
a given example, rather than merely agreeing with the views of other students. Encourage
participants to take responsibility for describing why they believe the fallacy or bias they
select applies to a vignette. Coach students to ask for reasons for beliefs if confronted with
someone who merely labels a vignette with a fallacy without describing how it applies.
Mary Ann Sontag divided her class of 70 students into 14 teams of five students each. She
does not hand out the vignettes. At odd moments during the semester she asks two teams to
3
go to the front of the class to compete in evaluating three practice vignettes. Roles assigned
to other class members include two scorekeepers (one for each team), one timer, one coach
for each team (to determine when all team members have made a choice and to guard against
cheating), and two crowd control leaders (who lead cheers for their team). The rest of the
class cheers and observes. Sontag gives each team member one point for an answer that
agrees with the key and also gives each team five points if they are unanimous. She makes
sure every team has a chance to play in the first round, then reduces the field to winners and
losers, then plays successive rounds until the final round, where competitors get prizes. This
encourages class attendance (students never know when there will be a few minutes to play
around and it encourages students to keep reviewing fallacy definitions so they can play the
game at a moment's notice).
8. We provide feedback to the groups as they play to catch and correct incorrect beliefs (e.g., a
correct answer based on an incorrect account of why the fallacy applies). We support astute
questions and statements. We encourage groups to make note of disagreements for further
discussion with the class as a whole. You may also overhear exchanges as you circulate
among the groups and note positive and negative ways to handle differences of opinion,
which may reflect some of the very fallacies being considered (e.g., ad hominum appeals).
If Students Play the Game by Themselves
Students could work through each vignette and keep a score of their "hits" (correct fallacy
spotting) and "misses." They could formulate a response to each item and compare their
responses to the "What You Could Say" suggestion.
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
Do any of this game's vignettes reflect real situations particularly well? Which one(s)?
We have found that items remind participants of real-life situations, particularly participants who
have had some practice experience.
RESEARCH REGARDING RIP's EFFECTIVENESS
Duncan Whyte of the University of South Carolina, following a Solomon Four Group design,
randomly assigned masters degree students in a research methods class to play the RIP games or
to a control. He found among the 136 participants in the study, as measured on the Professional
Thinking Form (see Exercise 10), that those playing the RIP Games scored statistically
significantly higher at posttest on PTF and had statistically significantly higher change scores
over controls. There was no statistically significant difference across controls at posttest among
controls who took a pretest compared with those who did not (possibly explainable by low
statistical power).
1
__________________________________________________________________
Answers to Exercise 11: REASONING-IN-PRACTICE GAME A:
COMMON PRACTICE FALLACIES
__________________________________________________________________
GAME A: NO.1
FALLACY 2
Testimonials. The client said that the treatment method worked, based on her own
personal experience. Other relevant data are not reviewed such as others who experienced
the method and did not improve, or got worse, or those who did not participate in the
treatment and improved (perhaps even more). Individuals who give testimonials may be
hand picked from among many treated individuals. Testimonials can capture the
emotions with their "I'm just like you..." appeal.
Countermeasure. Search for reports of controlled studies to evaluate the effects of the
treatment. Both case examples and testimonials involve partiality in the use of evidence-
looking at just part of the picture.
What you could say. Do you know if there are any data comparing results of people who
participated in this program and those who did not?
GAME A: NO. 2
FALLACY 3
Vague Description of Problem and Outcome. What is a "chemical dependency"? There is
no mention of a specific outcome measure, specific observable behavior, or measurement
procedure while claiming to know that the client improved.
Countermeasures. Clearly describe hoped-for outcomes (what, where, when, who, how
often). Client concerns and related outcomes should be so clearly described that different
people agree on their occurrence.
What you could say. Could you give me an example of what you mean by "chemical
dependency problem"? Could you tell me what you mean by "co-dependency"?
GAME A: NO.3 *
FALLACY 5
Confirmation bias: only successes are focused on. This is an example of partiality in the
use of evidence. You look only for a piece of a puzzle. This will not yield sound
inferences.
2
Countermeasures. To draw a valid conclusion about the agency's success in placing
children, you must know how many were placed. In addition to considering successes,
look for failures, persons not treated who got better, and those not treated who got
worse. Decrease reliance on memory. Keep records of successes, failures, those not
treated and improved, and those not treated and not improved. The latter two groups
might be estimated by reading literature about what generally happens to untreated
persons. It would also be valuable to know the proportion of successful placements made
by other agencies and by those placed through other means (e.g., attorneys). Look
fearlessly at all the evidence, not just the data that support a hypothesis (i.e., cases where
the treatment worked). How else can an accurate judgment be made?
What you could say. What we should find out is our rate of success. If we tried to place
100 children over the past two years and only successfully placed 20 during that time,
this would be a yearly success rate of 20% -not very impressive.
GAME A: NO.4
FALLACY 1
Case Example. Nothing can be said with any confidence about how widespread welfare
cheating might be based on one case, vivid or not. Such vivid cases often appear in news
stories.
Countermeasures. To make accurate generalizations, collect a random sample from a
population that is large enough to offer an accurate guide to what is true of the
population. Several methods may be used to gather this information including
observations and surveys.
What you could say. I don't think one case example tells us much. We should find out
what percentage of all people in our state who are on welfare engage in fraud.
GAME A: NO.5
FALLACY 10
Post Hoc Fallacy (After This, Therefore, Because of This). (See also Fallacy 9.) Group
members' family feelings may change over time for reasons other than the treatment.
Families may seek help during a worse phase in their better-worse-better-worse-better
cycles.
Countermeasures. Think of other possible causes for client improvement, or
deterioration, before taking credit (or blame). For example, you may think that your
client acquired a new social skill as a result of your program, but your client may have
matured naturally into better social skills. You may believe that cognitive-behavioral
therapy helped a depressed, middle-aged male client, but the client may have improved
because he saw a psychiatrist who prescribed an antidepressant. A break in very
hot weather may have been responsible for a decrease in street violence rather than your
3
community crisis team's efforts to head off violence. There are cyclical problems that get
worse, improve, and again get worse (Skrabanek & McCormick, 1998). Well-designed
studies can help to rule out these and other explanations for client change.
What you could say. (See answer to #1.) How do we know that the measure used (IFF) is
valid?
GAME A: NO.6
FALLACY 4
Softhearted, Therefore Softheaded. To assert that one is either a people-person or a
critical thinker ignores that one can (and should) be both. One might argue that
practitioners, who keep up with the related practice literature and who critically review
the quality of evidence and arguments, do so precisely because they do care enough
about clients to want to be as effective as possible.
Countermeasures. Be hardheaded (analytical, scientific, datafocused) because you are
softhearted (care about what helps people).
What you could say. This implies that if you consult practice related research, you cannot
also have "people skills." This is "either/oring." You can do both.
GAME A: NO.7
FALLACY 5
Confirmation Bias. The author requested the library helpers to find evidence that supports
the effectiveness of family-based treatment. What were they to do with evidence that did
not support its effectiveness?
Countermeasures. Be skeptical of anyone who presents just one side of anything. The
world is not that simple. Seek and present multiple positions and diverse sources of data.
What you could say. What do other studies report? We should look at all relevant studies,
not just studies that support effectiveness.
GAME A: NO.8
FALLACY 10
Post Hoc Fallacy (After This, Therefore, Because of This). (See also Fallacies 1 and 9.)
Simply because improvement followed cognitive-behavioral therapy does not show that
the therapy was responsible. Other events might account for the improvement (e.g.,
taking antidepressant medication).
Countermeasures. (See those described in No. 5.)
4
What you could say. This change could be due to regression effects. (See Game C.) The
client was in the "severe range" at intake. He would tend to return to his "mean" score.
GAME A: NO. 9*
FALLACY 8
Oversimplifications. What is a "brain disorder?" What does "very anxious" mean?
(vagueness).
Countermeasures. Watch out for oversimplifications such as the above. What are other
possibilities here and other options?
What you could say. I wonder if you "jumped the gun." I wonder what is going on in
Mrs. A's life that may account for her anxiety.
GAME A: NO. 10
FALLACY 7
Appeal to unfounded authority. Unless Dr. Morrison has collected better data, the survey
data provides the best guide here.
Countermeasures. Compare the evidentiary status of different sources of data.
What you could say. Could you tell me if Dr. Morrison has collected or knows about
larger survey sets regarding service needs of Hmong families?
GAME A: NO. 11
NONFALLACY ITEM
Nonfallacy Item. The sources of opinions are described; the source offers relevant
evidence for the conclusions, and the conclusions drawn do not go beyond the data.
What you could say. Let's take a close look at the study design. We have a comparison of
two treatments. Were subjects (Ss) randomly assigned? If not, this is a problem. What did
comparisons receive? Is a measure of attitudes a valid reflection of behavior (e.g.,
attempted suicide)? There are problems with this report. However, it's better than no data
at all about the program.
GAME A: NO. 12
FALLACY 4
Softhearted, Therefore Softheaded. To assert that you can either apply scientific thinking
and use valid measures for client progress or treat clients with respect implies that one
can't be both.
5
Countermeasure. Be hardheaded (analytical, scientific, data focused) because you are
softhearted (care about what helps people).
What you could say. I've seen studies that show that clients like concrete feedback about
progress. For example, Campbell (1988) interviewed clients and found that they valued
the concrete feedback from single-subject data more than the practitioner's opinion about
client progress. Also, critical appraisals of interventions show that intuition and good
intentions are not a good guide to degree of progress (e. g.., see Jacobson, Foxx, &
Mulick, 2005).
GAME A: NO. 13
FALLACY 5
Confirmation bias -- Focusing on successes only. To draw a valid conclusion about the
Department's job placement rate, they would need to examine the proportion of students
in the class who got jobs. If there were 60 students in the class and 30 obtained jobs, only
half would be employed.
Countermeasures. Look at all the evidence. Keep records of successes and failures.
What you could say. How many students graduated? If we knew, we could figure out the
percentage now employed in social work.
GAME A: NO. 14
FALLACY 6
Newness/Tradition. The speech therapist asserts that facilitated communication should be
used simply because it is new. All methods have been new at one time (those that were
later shown to be dismal failures as well as those shown to be effective
Countermeasures. Point out that being new or old does not make an idea about practice
valid. Ask to see evidence and data to judge effects of methods.
What you could say. Is there any evidence (e.g., experimental studies involving random
assignment) showing that facilitated communication is effective?
GAME A: NO. 15
FALLACY 1
Case Examples. A case example was presented by the drug company to support its claim
that the drug is effective. They would probably not present a client who did not improve.
Think of the possibility that there may be many unsuccessful cases. Case examples do not
provide support for claims of effectiveness.
Post Hoc (After This, Therefore, Because of This). The client's depression might have
been cyclical, passing through acute, less acute, phases; so when the medication was
6
stopped the client might have improved anyway. The client's marriage or some other
feature of his life might have changed.
Countermeasures. Search for experimental studies. This is not to say that case material
cannot be a vital part of professional education. Case material can be used to demonstrate
practice skills and apply them. For example, a videotape of an interview with an adoles-
cent mother may demonstrate interview techniques. A field instructor may model a
technique for family therapy. Such use of case material constitutes a valuable part of
social work education. The problem arises when generalizations to all clients are made
from case examples.
What you could say. One case example doesn't tell us anything about whether the relief in
depression was due to Wellbutrin. Also, many key items are vague (erectile failure,
decreased libido, feeling more energetic, sexual performance is normal).
GAME A: NO. 16
FALLACY 7
Appeal to unfounded authority -- Uncritical documentation. Simply giving a reference
does not cast any light on a method's success. This reference might consist purely of case
examples, testimonials, and biased presentations by advertisers who seek to profit from
functional communication training. You don't know because nothing is said about how
the authors drew their
conclusion. Seek out a high quality review of related studies (e.g., Jacobson, Foxx, &
Mulick, 2005).
Countermeasures. If you want to assess the accuracy of an assertion that is merely
referenced, you'll need to read that reference for yourself.
What you could say. Could you give me some details about their study? This would help
me to evaluate the basis for their claim.
GAME A: NO. 17
FALLACY 3
Vague Descriptions of Outcomes. How was self-esteem measured? How is its level
determined? Without greater clarity, we can't tell if there has been improvement.
Countermeasures. Clearly describe desired outcomes. Descriptions of outcomes should
be so clearly stated that we can agree on when they have been attained. They should
answer the questions what, when, who, how often?
What you could say. Could you clarify what you mean by "generally their self-esteem
improves"? What does "generally" mean? How was self-esteem measured? Is this
measure valid? What do we know about changes in self-esteem of women who did not
attend the program?
7
GAME A: NO. 18
FALLACY 7
Appeal to unfounded authority - Uncritical documentation. Citing a reference tells us
nothing about the method by which the authors drew their conclusion. For all we know,
this reference may have nothing to do with learning transfer.
Countermeasures. If you want to assess the accuracy of an assertion that is merely
referenced, you'll need to read it for yourself.
What you could say. Could you give me some details about their study? This would help
me to evaluate the basis for their claim.
GAME A: NO. 19
FALLACY 6
Newness/Tradition. The child welfare worker asserts that open adoptions should be tried
because the practice is new. No data are described regarding this method's success.
Countermeasures. Point out that being new or old does not make an idea or practice
valid. Ask to see evidence and data to judge effects of methods.
What you could say. Is there any evidence (e.g., experimental studies involving random
assignment).
GAME A: NO. 20
FALLACY 5
Oversimplification: there could e many reasons why people use controlled substances.
Simply because improvement followed treatment does not mean that the chiropractor’s
efforts led to change. Many lower back problems are self-limiting. They come and go
without any intervention.
Countermeasures. (See No. 9.)
What you could say: What other circumstances may encourage substance abuse?
1
__________________________________________________________________
Exercise 12: REASONING-IN-PRACTICE GAME B: GROUP AND
INTERPERSONAL DYNAMICS
__________________________________________________________________
OVERVIEW
Students work together in small groups to Identify fallacies in brief vignettes. The vignettes
concern the following contexts: special education, mental health, child welfare, corrections,
administration, health, substance abuse, education, geriatrics, and juvenile court.
PURPOSE
1. To have fun.
2. To learn how to identify and avoid fallacies that occur in case conferences, group meetings,
interdisciplinary teams, and staffings.
MATERIALS REQUIRED
Same as for Exercise 11.
TIME REQUIRED
Same as for Exercise 11.
SUGGESTIONS
Same as for Practice Exercise 11. Please note that the majority of the vignettes in Game B are
starred (*). These items have parts to be acted out. We think this adds realism to a game that
concerns group dynamics. Try to encourage interaction, even in nonstarred items, by having
different people read the vignette.
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
Do any of this game's vignettes reflect real situations particularly well? Which one(s)?
We suggest that you encourage participants to recall their experiences in groups that seek to
accomplish a task (e.g., case conference, political action, interdisciplinary team, and planning).
2
Answers to Exercise 7: Game B
______________________________________________________________________________
GAME B: NO.1 *
FALLACY 6
Groupthink. This is groupthink because there was no serious discussion of the views of
the special education teacher. This denied the group useful information.
Countermeasures. Janis (1982, pp. 262-265) suggests three ways to counter Groupthink
include the following: (1) have the leader assign the role of critical evaluator to some of
its members; (2) make it clear at the beginning of an important discussion that the leader
will be impartial to the group's decision; and (3) for important decisions, set up
independent committee to gather evidence and deliberate independently of other groups.
Each committee should be led by a different person.
What you could do: Discuss how the team could encourage serious discussion of
dissenting points of view.
GAME B: NO. 2*
FALLACY 10
Slippery-slope: Assuming that giving money to one group of necessity requires giving
more money to many more groups when it does not require this.
Countermeasures. Note the lack of necessity.
What you could say. If we have good reason to increase funds to the elderly we do not
have to justify giving more funds to other groups. After all, we are a service for the
elderly.
GAME B: NO.3
FALLACY 2
Begging the question: No additional information is given.
Countermeasure: Note the circular nature of the reply.
What you could say. But what is “order”?
GAME B: NO.4
FALLACY 1
At the Person (Ad Hominem). Diana presented A 15-year-old report which should have
been updated (what does current literature report). But, rather than note this point, Lizzie
and Polly called her a "know-it-all."
3
Countermeasures. When making decisions, we should focus on our goals and on
examination of related evidence. This involves arguing at the issue, or at the thing (Ad
Rem).
What you could say.
Sandra: Reliability is important but we should search for more recent data. And, even if
we can reliably rate BPD, is this a valid diagnosis? Does it help us to understand the
etiology of related behaviors and to select effective intervention methods?
GAME B: NO. 5*
FALLACY 3
Diversion (Red Herring). The psychologist raised a vital question about the client's
suicide potential, but the discussion was sidetracked by talking about a joke. Hopefully,
the group got back to the subject.
Countermeasures. Gently bring the discussion back to the point at issue (e.g., We were
just talking about …).
What you could say.
Clinical Psychologist. The reason I am worried about Mr. Meyer's potential for suicide is
that the last two times his depression lifted, he did make a suicide gesture. (She returns
to her concern.)
GAME B: NO. 6
NONFALLACY ITEM 11
Nonfallacy: The supervisor praises the worker for sharing this information and suggests
next steps to consider more carefully the exact practices/policies questioned.
GAME B: NO. 7
FALLACY 7
Stereotyping. The bottom line in judging any method is, Does it work? The popularity of
a method does not mean it is effective.
Countermeasures. Accurately describe behavioral methods (e. g., Gambrill, 2013).
What you could say. Manipulation refers to controlling people in unfair ways.
Behavioral methods such as constructing alternative repertoires that compete with
undesired ones, are used to help participants to enhance the quality of their lives.
GAME B: NO.8*
FALLACY 8
4
Either-Or. Believing that there are only two choices (file a formal complaint, initiate a
full investigation/do not initiate investigation) may limit the care to this family, possibly
overlooking alternatives such as vocational counseling for the father and/or mother,
referral to income maintenance programs, and referral to the Veterans Administration.
Countermeasures. Identify several options about what might be done. Ask each group
member to independently identify possible courses of action. Assure them that whatever
they write will be read anonymously and discussed seriously.
What you could say. We have hearsay evidence from a friend of one of the neighbors.
This is shaky. However, we are obliged to investigate all allegations of child abuse.
Why don't we get in touch with this neighbor? I think we have to do something.
GAME B: NO. 9*
FALLACY 2
Begging the question. No additional information is given.
Countermeasures. Note the circular nature of the reply.
What you could say.
Jenny: Could there be other reasons for this behavior? Do we know the antecedents and
consequences related to his behavior? And could you give me examples of "erratic
behavior"?
GAME B: NO. 10
FALLACY 4
Stereotyping. The female workers are prejudging a job applicant without viewing his
work performance, qualifications, references, and experience because they have an
opinion about all males. If they attacked the candidate individually by calling him a
name, or by bringing up an irrelevant or untrue argument against him, they would be
arguing At the Person (Ad Hominem).
Countermeasures. Evaluate each person's competencies without relying on preconceived
notions based on their membership in a group or class. Stereotyping fosters racism,
sexism, classism, and ageism (among other "isms").
What you could say. It's easy to fall into making decisions based on gender, ethnicity,
class or race. However we have to be careful not to do that and try to recognize our
biases. We have no evidence that male social workers don't do as well in hospital social
work as women. I think we have to be careful not to impose unfair and inaccurate
stereotypes on job candidates.
5
GAME B: NO. 11*
FALLACY 9
Straw Man Argument. Richard has argued for matching alcoholics with the most
appropriate treatment, not that all alcoholics are unique. Onesmo examines the idea that
all alcoholics are unique. Both people should stay with the original topic: Should
alcoholics be matched systematically with their most effective treatment? Onesmo might
have avoided the red herring if he had asked? "Are you arguing that all alcoholics are
unique?"
Countermeasures. When discussing vital issues, carefully listen to the other person's
position. Restate the position in your own words as accurately as you can and ask
whether you have restated it accurately and if so, then react.
What you could say.
Richard: The more specific we can be about what works with which particular kinds of
individuals, the better chance we have of matching intervention recommendations to
specific people. I haven't seen any data showing that all people experience the same
physiological symptoms of withdrawal after they have build up a tolerance for alcohol.
Anyway, I don‟t' think this is the key point in relation to matching people with
treatments. Have I missed something?
GAME B: NO. 12
FALLACY 7
Bandwagon. Professionals can be wrong. Turning to a new method may be premature
in light of later evidence. Examine the evidence before adopting a new method.
Countermeasures. Critically appraise any popular notion, particularly if it concerns
treatment. Search for related studies, including those that provide counterevidence and
counterarguments before you decide.
What you could say. If we did classify more children into this category, what effect
would it have? Would it help us to select effective services?
GAME B: No. 13*
FALLACY 1
At the Person (Ad Hominem). The social work trainee described referred to reports about
how to help residents with Parkinson's Disease; the nurse described the social worker as
naive. This was an attack at the person, not an examination of the argument presented.
Countermeasures. When making decisions about client care, we should seek and
critically appraise related evidence. This involves arguing at the issue, or at the thing (Ad
Rem). A standard for judging such evidence can be found in the Quality of Study Rating
Form, or by playing the RIP Game to see if someone's argument avoids fallacies.
6
What you could say. I think we have to consider the results of this study, especially if it
was well designed and shows that our methods are not as effective as other methods.
Lois: Could you bring this study in so we can read it?
GAME B: NO. 14*
NONFALLACY ITEM
Not a Fallacy. The presenter has made claims for effectiveness of his method but
provided no related evidence. The social workers raised questions regarding the
evidentiary status of claims made. (See vignettes for example of what they might ask.)
GAME B: NO. 15*
FALLACY 5
Manner or style. A sincere, warm, entertaining way of presenting an idea can encourage
professional listeners to pay attention, but the manner or style of the presenter does not
assure the accuracy of the presenter's ideas. Examine the evidence when thinking about
client care, in spite of distractions. One type of stylistic distraction is someone whose
mannerisms offend but whose ideas are sound.
Countermeasures. In matters of importance involving client care, base judgments and
decisions on the evidence presented, not on the speaker's style or lack of it, even if the
idea comes from an "odd" person. The utility and soundness of the person's argument is
what matters.
What you could say. That may be, but I'd really like to find out if there's any evidence
that validation therapy really helps the client to become more oriented to time, place, and
person. Mr. Diggelman may be a nice person. The question is, Does validation therapy
work? Will it help our clients?
GAME B: NO. 16*
FALLACY 3
Diversion (Red Herring). Instead of replying to the supervisor's concern the worker goes
off on a tangent.
Countermeasures. Focus on the issue at hand -- this worker came in late.
What you could say. My concern here is you being late. This is the second time this
week you have been late.
GAME B: NO. 17
FALLACY 5
7
Manner or Style. Ms. Carey„s confident manner may distract attention from the accuracy
of what she says.
Countermeasures. Base judgments and decisions on evidence presented, not on the
speaker's style or lack of it. The utility and soundness of the person's
argument is what matters.
GAME B: NO. 18*
FALLACY 8
Either-Or. This form of reasoning narrows the range of services that could help the
family, including temporary housing with a church member, referral to income
maintenance, help finding a job, and so on.
Countermeasures. Consider several working hypotheses about what might be done. Each
group member could independently write down possible courses of action. Assure them
that whatever they write will be read anonymously and discussed seriously.
What you could say.
Judge Calhoun: Couldn't there be another alternative? How about going out and talking
further with the family?
GAME B: NO. 19*
FALLACY 8
Either-Or. This form of thinking limits the decision to one of only two alternatives.
Countermeasures. Consider several options about what might be done. Ask each group
member to independently write down possible courses of action. Assure them that
whatever they write will be read anonymously and discussed seriously.
What you could say.
Jonathan: I don't think it's an either-or matter. There maybe some other options we can
pursue.
GAME B: NO. 20*
FALLACY 9
Straw Man Argument. Mike misrepresents Antonio's argument and then attacks the
misrepresentation. Antonio says there may be unintentional cues that can shape a child's
testimony and wonders if this happened; Mike says that Antonio is implying he would
intentionally mislead a child to give false testimony; Mike then attacks this idea.
Reflecting, or restating, the other person's argument for accuracy before reacting to it can
help avoid a Straw Man Argument.
8
Countermeasures. When discussing vital issues, carefully listen to the other person's
position. Restate the position in your own words as accurately as you can. Ask if you
have restated it accurately, and then react.
.
What you could say.
Antonio: I did not say anything about your intentionally misleading a child into giving
false testimony. What I did say is that research suggests that regardless of our intentions
we may lead witnesses, especially children, into saying that are certain things that are not
accurate.
GAME B: NO. 21
FALLACY 7
Bandwagon. Increasing use of a method does not mean it is effective. Only evidence
regarding its effectiveness for particular lessons, with particular students,
in relation to particular learning goals can answer the question of effectiveness.
Countermeasures. Critically appraise claims. Examine counterevidence and
counterarguments as well as supporting evidence before you decide.
What you could say.
Faculty member: Let's find out if problem-based learning is effective in enhancing
clinical reasoning skills on-the-job. Is there a systematic review of related research?
1
_________________________________________________________________________
Exercise 13: REASONING-IN-PRACTICE GAME C: COGNITIVE BIASES _________________________________________________________________________
OVERVIEW
Students work together in small groups to identify cognitive biases described in the literature on
clinical reasoning, decision-making, decision analysis, and judgment. A few of the best
understood and most commonly described biases are the subject of interest in this game. (Please
see related references in the literature.) Vignettes come from the following disciplines: special
education, medicine, social work, nursing, research methodology, psychology, chemical
dependency counseling, and corrections.
PURPOSE
1. To have fun.
2. To learn how to identify and to avoid common errors in reasoning that influence judgments
and decisions.
MATERIALS REQUIRED
,
Same as for Exercise 11.
TIME REQUIRED
Same as for Exercise 11.
SUGGESTIONS
Same as for Exercise 11.
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
Do any of the game's vignettes reflect real situations particularly well?
Which one(s)?
We suggest that participants reflect on their practice experience or as recipients of services.
2
______________________________________________________________________________
Answers to Exercise 13 GAME C:
______________________________________________________________________________
GAME C: NO.1 *
FALLACY 1
Hindsight Bias is evident in the special education teachers' looking back and recalling
successful placements. A true evaluation of predictive accuracy would be to examine all
records for cases where judgments to place or not to place, were made.
Countermeasures. (1) Be mindful that there is a tendency, when looking back, to
overestimate the accuracy of your predictions; and (2) keep your own records
prospectively (before the fact) of your predictions, and consult studies that assess risk
(i.e., make predictions, record predictions, and then see what happened). These
countermeasures were suggested by Hogarth (1987, p. 149).
What you could say. I think we have to look at the group as a whole rather than taking
individual examples. Doing so will let us see what our track record has been so far and
how we can improve it.
GAME C: NO. 2*
FALLACY 5
Overlooking Regression Effects. The lowest five percent may have had a particularly bad
day at the time of the pretest because they were ill, emotionally stressed, or for some
other reason; therefore, on posttest, their scores would regress upward toward the mean at
posttest.
Countermeasure. Be skeptical of studies that single out an extreme group for
observation. Ideally, submit half of the extreme group to intervention, the other half to a
control or alternate intervention; then posttest both groups for comparison.
What you could say.
Second Child Care Worker: I don't think we can assume that this shows that our
enrichment program helps school kids. We would expect those who scored low to score
higher on the next testing; they would approach their "average level." We would have to
conduct an experiment (e.g., randomly distribute children to two different programs, one
which had the enrichment program, and one which did not) and then see if there are any
differences in outcomes between these two groups. Or, maybe we could carry out a
number of experimental single-case studies to discover whether the enrichment program
really is effective in delivering what it promises.
3
GAME C: NO.3 *
FALLACY 3
Framing Effects. The probabilities are identical for the two procedures; the patient is
drawn to surgery because it is described in terms of likelihood of success.
Countermeasure. Describe both negative and positive consequences for all alternatives.
What you could say.
Doctor: Before you make a decision about what to do, I think I should point out these
two estimates are the same.
Patient: What do you mean?
Doctor: In the first case, 25% of the time the operation is not a success; and the same is
true with cortisone.
GAME C: NO. 4*
FALLACY 2
Fundamental Attribution Error. Joe does not recognize environmental influences on
behavior, and so he misses the chance to intervene at the community level to help
children.
Countermeasure. Be informed about how the environment influences behavior and help
clients to alleviate environmental stress and/or learn to cope with it.
What you could say.
Maria: I agree that there are many factors that influence school performance, including
the extent to which parents support education, the quality of neighborhoods, and cultural
and ethnic norms and values. However, levels of ingested lead correlate with reduced
educational performance. I think we have to consider this and try to do something about
it. This doesn't mean other factors are not important.
GAME C: NO.5
FALLACY 4
Overconfidence. No evidence is provided for this claim.
Countermeasure. The psychologist should make predictions and keep track of which
ones turn out to be accurate.
GAME C: NO.6
NONFALLACY ITEM
Not a Fallacy. The nurse administrator has formed an opinion based on data and explains
why.
4
What you could say.
Listener: Well, this information is helpful. Thank you for finding this report. Do you
know if there are other studies that bear on this decision, especially ones that may have
contradictory findings?
GAME C: NO. 7*
FALLACY 1
Hindsight Bias Penny did not keep a record of her predictions ahead of time to test their
accuracy; and even now she cannot state specifically what her indicator of high risk
might be.
Countermeasures. (See No. 1)
What you could say.
Maureen: It's always easy to say "I knew it" in retrospect. What is difficult is to predict
what will happen in advance. If you really want to test how accurate your predictions are,
make them right after the first group meeting and test their accuracy by seeing what
happens over a number of people.
GAME C: NO.8
FALLACY 3
Framing Effects. If one question emphasizes a possible positive outcome, and another
emphasizes a possible negative consequence, the client will probably choose the former.
Countermeasure. Note that the 10% and 90% are inter-related.
GAME C: NO.9
FALLACY 6
The Law of Small Numbers. A single case, with which the social worker is familiar, may
not be representative of most cases. (See also discussion of case examples in Exercise 6.)
Countermeasures. Give more weight to conclusions based on large numbers of
representative, carefully studied clients, and less weight to intimate experience with just a
few clients.
What you could say.
Listener: I don't think one example tells us as much about the population at large as does
a study of 50 people. A particular student may be quite atypical. We have to consider the
size of the sample and how it was selected in deciding how informative it may be in
relation to the population of all students.
5
GAME C: NO. 10
FALLACY 5
Overlooking Regression Effects. The problem is not that the top ten relaxed, but that their
initial scores might have overestimated their ability due to chance factors like lucky
guessing, or having particular knowledge that most reflected the test's first form;
therefore, on retesting, these ten were likely to test out lower
Countermeasures. See No. 2.
What you could say.
Listener: I think what we have here is regression toward the mean. The people who
scored high on the test would be expected to score lower the second time around.
Therefore I don't think the changes in test scores can be attributed to relaxing during the
training and not paying attention.
GAME C: NO. 11 *
FALLACY 4
Overconfidence: Monitoring progress contributes to positive outcome; self reports of
clients may be inaccurate and should be compared to data gathered via self monitoring or
observation.
Countermeasure. Be informed about important practice issues.
GAME C: NO. 12
FALLACY 9
Gambler's Fallacy. Each trip is independent of the others; so on the next trip the success
of the trip is not more nor less certain than for those previously.
Countermeasure. For truly independent events (a fair coin, birth of boy or girl in a given
hospital), what happened on the previous trial cannot affect the next in the series. Aside
from clinical reasoning, the gambler's fallacy is an important one to understand now that
gambling has been legalized in many states. No matter how many times you enter the
lottery, your chances of winning the next time you play will be the one in one million (or
whatever odds are stated on the ticket) no matter how many times you have played in the
past. This is something to understand and to convey to clients, because many spend
money that they can ill afford on gambling.
What you could say.
Listener: You mention that three staff from your office have gotten through with backup
support and only minor confrontations. What about others? What's the base rate of
confrontations?
GAME C: NO. 13*
6
FALLACY 7
Ignoring Prevalence Rate. The protective service worker ignores the fact that the
prevalence rate for abuse among families in the community would be much lower than
would be the reabuse rate for protective service clients, who have most likely already
abused a child. In the former; there would be many false positives (abuse predicted but
none found).
Countermeasure. Though the mathematics to show why this is true can take up a whole
chapter (Gibbs, 1991, p. 217-237), the implications of base rate (proportion of the group
who have a given characteristic or problem that we want to identify) cannot be ignored.
In the first situation (protective service clients), where those to be identified constitute a
large proportion with the characteristic (child abuse), the identification job is easier. In
the second, where the same characteristic is used to try to identify a very few individuals
among a much larger group of individuals (all families in the community), the assessment
and identification task is much harder. In the latter situation, low base rate, there will be
many more false positives (persons judged to have the problem).
What you could say.
Researcher: Well I don't think we could do this because the population is different. The
90% rate involves an identified population (protective service clients). If we apply this to
all families in a community in which the base rate is much lower, you will get a high
false positive rate and nowhere near the 90% rate obtained with the identified population.
This could result in many false allegations.
GAME C: NO. 14
FALLACY 11
Availability. This represents an actual case from the medical literature, whereby a
surgeon did not think of a possibility outside the realm of surgical experience-namely,
that the patient's problem could be a psychiatric one.
Countermeasure. Try to think of alternatives that do not come to mind readily. Consult,
where possible, surveys that record the relative frequencies of events. (See Arkes, 1981,
p. 323.)
What you could say.
Listener: You've been very clear and candid about what you've done and not done. This
will help me remember to consider psychological causes of physical complaints.
GAME C: NO. 15
FALLACY 10
Anchoring and Insufficient Adjustment. The social worker would make better judgments
if she kept an open mind while doing an assessment until sufficient information was
7
obtained to make a judgment
Countermeasures. Arkes (1981, p. 327) suggests using strategies to encourage alternative
hypotheses about a client. For example, you may entertain several hypotheses.
What you could say.
Listener: You might be unduly influenced by the course you just completed? Anchoring
effects (what we think first) influence what we then do or see. I try to suspend my initial
assumptions temporarily and look for information that supports alternative views so I
don't overlook something important. When I concentrate on my first impressions without
questioning them, I'm likely to miss things.
GAME C: NO. 16
FALLACY ITEM 11
Affective Bias: Here the emotions of a professional influence reactions toward clients.
Countermeasures: Attend to your emotions: Have you had an upsetting experience that
could affect your work? If so, use your emotion management skill to calm down.
GAME C: NO. 17
FALLACY 10
Anchoring and Insufficient Adjustment. This hospital social worker prematurely closes
off channels of information that may aid decision making. In this case, additional
information might come to light that is not in the chart. The client's condition and prefer-
ences may have changed since the last entry in the chart.
Countermeasures. Arkes (1981, p. 327) suggests using strategies to ensure alternative
hypotheses about a client. For example, one might resolve, when beginning a group
meeting, to entertain several hypotheses about what may be the group's principal interest
at the meeting. Resolve not to formulate an opinion until each member of the group has
had a chance to speak.
What you could say.
Listener: I find that information I need to consider possible options is often missing on
the hospital chart. Some of my clients, for example, can return to live with relatives.
GAME C: NO. 18*
FALLACY 6
The Law of Small Numbers. The vivid and recent experience with a single case does not
carry as much power as does a larger sample of individuals who participated in an
evaluation study.
8
Countermeasure. Give more weight to conclusions based on large numbers of
representative, carefully studied clients, and less weight to intimate experience with just a
few clients.
What you could say.
First Child Psychologist: I can see how this is a vivid case example, but I don't think it
takes away from the findings of the research study I described. First, we don't know if
this psychologist implemented paired reading appropriately. Also, it may not work with
all children but may work with most. I think we have to look at its overall ''batting
average."
GAME C: NO. 19
FALLACY 9
Gambler's Fallacy. Because the lives of offenders on a probation officer's caseload
progress pretty much independently of each other, the fact that there is a run of recidivists
does not imply that the next case is more or less likely to recidivate than those before.
Countermeasures. See Item 12.
What you could say.
Second Probation Officer: I'm not sure you could say this. Each of these individuals is
independent; what happens to one is not related to what happens to another. The
probability that the next one will be a success is no higher than the probability that the
next one will be a failure.
GAME C: NO. 20
FALLACY 8
Omission bias: This refers to viewing acts of omission such as failure to use effective
methods as less concerning than acts of commission, such as use of a harmful method. It
is important to keep track of both omissions and commissions.
Countermeasure: Keep track of both lost opportunities to help clients by offering
effective methods as well as occasions in which clients are harmed by use of methods.
GAME C: NO. 21*
FALLACY 7
Ignoring Prevalence Rate. The community probation officer's clients are at lower risk
(lower base rate) than those released from the maximum security prison. That is, there is
a higher proportion of recidivists in the prison than among those on probation in the
community. Consequently, there might be a high false positive rate for predictions on the
measure (recidivism predicted but does not happen) in the probation officer's caseload.
9
Countermeasures. See Item 13.
What you could say.
Corrections Policy Maker: I don't think that would be a good idea because the two
groups are different. (See Item 13.)
___________________________________________________________________________
Exercise 14: PREPARING A FALLACIES FILM FESTIVAL
________________________________________________________________
OVERVIEW
Based on examples and literature from the RIP Games, the Professional Thinking Form, and
other literature, participants write a two page paper that defines a chosen fallacy, describes how
to avoid that fallacy, and includes an original 30-to-60-second script for a vignette. They then act
out their vignette before a small group while being videotaped. These vignettes are edited and
then shown to an entire class whose members try to name the fallacy. They can be posted on
YouTube. This is one of the exercises students enjoy the most.
PURPOSE
1. To familiarize teams with a practice fallacy.
2. To help the whole class learn more about fallacies.
MATERIALS REQUIRED
1. List of Practice Fallacies and Pitfalls (Practice Exercise 14).
2. Access to a library containing related material.
3. Videotape camera (your university might have an instructional media center whose experts
could videotape the students' vignettes and edit them).
4. VHS videotape player.
5. Television monitor.
6. Props (to add realism to the vignettes).
7. A room for videotaping free from outside noises and interruptions. (We use a university
classroom and a lot of imagination.)
TIME REQUIRED
Thirty minutes from one class period to practice acting the vignettes, one class period to
videotape the vignettes and one class period to show the Fallacies Film Festival and to give
awards.
SUGGESTIONS
We find that students enjoy putting the fallacy numbers from the Practice Fallacies and Pitfalls
list on slips of paper, folding them up, and then picking one out of a hat that identifies the fallacy
for their assignment. Students are assigned to two-member teams according to where they are
seated in the class. If there is an odd number, one group can have three members. Practicing the
vignettes the week before videotaping makes them go more smoothly, but still, actors need a
practice run the day of videotaping to warm up. The taping works best before a live but quiet
2
audience. A microphone can be used to compensate for soft voices, but actors should avoid
rustling their scripts next to it. Visual aids should be brief, large, and legible.
In preparation for the festival, your university's Media Development Department can edit and
splice the vignettes from all the labs into one tape. This tape includes the vignette itself, a five-
second break, and a caption that gives the title of the fallacy. During the festival we ask the class
to guess the fallacy being portrayed before showing them the title. The Festival works best if
most of the class doesn't know ahead of time which fallacy is being portrayed. Multiple sections
of a class can meet for a Fallacies Film Festival to watch and to guess all fallacies. The videotape
can be stopped after each vignette for students to guess the fallacy and discuss their reasons.
As the vignettes are shown during the Fallacies Film Festival, the fallacy names and the names
of the actors are noted. When the last vignette has been shown, a secret ballot among the class is
taken to see which vignette wins a prize. At the beginning you might want to note the following
rating criteria on the blackboard:
The vignette accurately and clearly depicted the fallacy yes _ no _
The presentation used a costume yes _ no _
The presentation used special effects yes _ no _
The presentation used other props yes _ no _
You guessed the fallacy's name yes _ no _
The vignette was brief (about 1 min.) yes _ no _
The actors were enthusiastic yes _ no _
Prizes have included a book about critical thinking for each student. Showing students the
awards before the competition may help to motivate them.
Grading Suggestions
We grade the paper and the vignette together for each team. Students' assignments describe the
following criteria for grading:
Your grade will be based on your fallacy's documentation, clarity of its definition,
appropriateness of description of how to avoid the fallacy, accuracy and clarity of the
script's demonstration of the fallacy, realism (e.g., use of visuals and props), and
entertainment value of your portrayal of the fallacy.
Documentation could be evaluated on the basis of appropriate references noted, including
references not included in the Workbook. In addition, grades could be based on a numerical
score reflecting (1) how accurately and clearly they define the fallacy, (2) the percentage of
students who successfully identified it in class, and (3) how clearly and accurately they describe
how to avoid the fallacy. You could add a score for the percentage of votes the vignette received
as "best" in the Fallacies Film Festival, or for how well the actors portrayed the fallacy and
entertained the group while doing so. The grade could be based on the total of these scores.
3
POSSIBLE ANSWERS TO FOLLOW-UP QUESTION
What have you learned from this exercise?
Suggested Lessons
1. These fallacies occur in the helping professions.
2. Illustrating the fallacies brings them to life.
3. One fallacy is often mixed with others.
1
__________________________________________________________________
Exercise 15: FALLACY SPOTTING IN PROFESSIONAL CONTEXTS
__________________________________________________________________
OVERVIEW
Students select a quote from a source (e.g., Internet journal, class lecture, book) that they think
demonstrates a fallacy that is relevant to the helping professions. In writing, they give the full
quote and source, name and define the fallacy, and describe why they believe this represents a
fallacy. This exercise provides opportunities to highlight the consequences for clients of
accepting claims based on faulty reasoning. You may use it more than once during a term. We
collect good examples students provide for illustrative and practice purposes. Tie this exercise to
a discussion of Walton’s definition of fallacy in Exercise 30.
PURPOSE
To hone skills in spotting fallacies in professional contexts. This exercise provides an
opportunity for students to dip into the professional literature, relevant sources on the Internet,
and related articles in newspapers and practice critiquing content.
MATERIALS REQUIRED
1. Fallacy Spotting in Professional Contexts form (Exhibit 15.1).
2. Definitions for fallacies in Exercises 11, 12 and 13.
3. Access to a library containing professional journals and books.
4. Description of APA citation guidelines.
TIME REQUIRED
Students may spend an hour locating a fallacy. Many attach a copy of the relevant material to the
fallacy spotting form. The exact source of the material and data (e.g., website) must be noted.
Students can save time by highlighting key material with a color liner. Additional time is
required to review definitions of relevant fallacies, and complete the fallacy spotting form.
Lastly, discussion of each fallacy will occupy additional time. About five minutes may be given
to each student to describe or read the case material, define the fallacy, and describe how the
fallacy applies to the example selected.
SUGGESTIONS
We require students to seek recent material (2010 to the present). Students can use quotes from
lectures. We encourage students to critique our own writing and class lectures. We stress that it
must be an exact quote and be placed in context. It is not necessary to identify the instructor
(who may be you). You could ask students to duplicate their completed fallacy spotting form and
2
give a copy to other students. This material can be used as the basis for critiques in small work
groups that allow each student to review many examples. Your task could be to provide feedback
as you circulate among the groups. Caution students to limit their selection to one key claim and
to include exact quotes and not take content out of context. Sometimes students pull out a large
section which has many arguments and do not identify what argument they are addressing. Have
students type their responses. It is easier to follow their reasoning.
In reviewing student reports, see if the critique is relevant to content quoted. Does the quote
illustrate a fallacy? Have they correctly identified the main fallacy illustrated? Have they clearly
described why this fallacy applies? Occasionally, students just write down the names of some
fallacy with no indication of how it applies to the quoted material (and in some cases they may
not apply at all to the example). Review the importance of understanding how the fallacy applies.
POSSIBLE ANSWERS TO FOLLOW-UP QUESTION
What have you learned from this exercise?
Suggested points to emphasize
1. Experienced, well-educated professionals can engage in faulty reasoning.
2. People who usually reason well can fall into faulty reasoning.
3. Errors in reasoning related to decisions can have serious implications for clients, so we
should cultivate accurate reasoning and practice spotting faulty reasoning to avoid errors.
____________________________________________________________________________
Exercise 16: AVOIDING GROUP THINK ____________________________________________________________________________
OVERVIEW
Professionals often participate in a variety of group meetings related to their work. Decisions in
these meetings may be compromised by a number of ploys designed to skew the decisions in a
direction that does not forward well-reasoned decisions.
PURPOSE
This exercise provides an opportunity for students to become aware of the phenomena of group
think and to learn to identify and avoid indicators of group think.
MATERIALS REQUIRED
1. Practice Exercise 11.
2. Access to group meeting
TIME REQUIRED
Depends on length of meeting observed.
SUGGESTIONS
The more realistic the demonstration of group think indicators and harmful consequences, the
better. The more practice in detecting and responding to group think tactics the better. Be sure
to give corrective feedback following practice opportunities.
REVIEWING STUDENTS’ WORK
Note the frequency of group think indicators. Which ones occur most often? Which remedies
did students select? Was there any change in the rate of group think indicators following the use
of remedies?
1
__________________________________________________________________
Exercise 17: APPLYING THE STEPS IN EVIDENCE-INFORMED
PRACTICE
__________________________________________________________________
OVERVIEW
Sackett, Richardson, Rosenberg and Haynes (1997) describe the process of EBP. (See also
Straus, et al., 2011; Gibbs, 2003; Gambrill, 2013). We also owe a debt to Gray (2001), author of
Evidence-Based Health Care.
Be prepared for objections to this exercise such as clinicians' workloads prohibit electronic
searching and the volume of material will overwhelm anyone who searches for answers to
practice questions. Electronic searches become ever easier over the years, as described in the
text and using methodological filters makes the process more rapid.
PURPOSE
To provide practice in carrying out the process of evidence-based practice.
MATERIALS REQUIRED
1. Copies of relevant Boxes in the exercise.
2. Access to a computer and printer.
3. Access to the World Wide Web.
4. Access to relevant databases.
TIME REQUIRED
We suggest a whole class. Ideally, multiple opportunities would be given over the class as well
as in other classes to search for answers to practice and policy questions.
SUGGESTIONS
Practice ways to apply methodological filters described in Exercise 17. Demonstrate searches in
real time regarding actual questions for all to see, as projected on a large screen. Then have
members of the class try their hand at searching for all to watch on the screen while being
critiqued by the search expert. Then provide guided experience for all students to sharpen search
skills. Students could draw on questions that arise in their field work.
2
We suggest that you buy and carefully read Evidence-Based Medicine (2011). Though the
examples concern medicine, good reasoning is good reasoning, and EBM's principles can and are
being adapted to apply to multiple areas.
REVIEWING STUDENTS’ WORK
What obstacles occur in carrying out the process? Keep track of these and help students acquire
relevant skills. Did students find the best evidence? What kind of search problems occurred?
_______________________________________________________________________________
Exercise 18: WORKING IN INTERDISCIPLINARY EVIDENCE-BASED
TEAMS
______________________________________________________________
OVERVIEW
Decision making often occurs in teams. Exercise 18 should be carried out by those who have a
background in applying the steps in the process of evidence-based practice.
PURPOSE
To enhance skills in working in evidence-based teams; applying the process of EBP in deciding
what to do in response to a case scenario. Grading criteria for this exercise may include the
following and are stated explicitly to students.
The team picked a leader who coordinated the team’s efforts.
Question identified a client/patient type.
Question identified a course of action (in this case an intervention).
Question identified an alternate course of action.
Question identified a desired outcome.
The search record included appropriate search terms.
The search record included appropriate methodological filters.
The search record included a list of databases searched.
The team applied the appropriate form for rating the study (e. g., PRISMA).
The team located a key resource (preferably a high-quality systematic review).
The team addressed the impact of intervention relative to number needed to treat or an
odds ratio.
This exercise could be used for an examination as well as for practice of related skills.
MATERIAL REQUIRED
1. Content in text for Exercise 18.
2. Computer with access to Internet with relevant databases.
TIME REQUIRED
In groups this exercise can be completed in half an hour. A single individual might be given an
hour.
SUGGESTIONS
2
Students in each group can divide up the databases and investigate different ones simultaneously.
Attached is a copy of a permission form that students could sign so that you could video their
work. This form was approved by attorneys at the University of Wisconsin System and by
attorneys at Brooks/Cole Thomson learning.
REVIEWING STUDENTS’ WORK
How could team interaction be improved? Was the best source found? How could search efforts
be improved?
3
INSTRUCTIONS GIVEN TO STUDENTS BEFORE COMPLETING THE EXERCISE
Dear Students,
Here are your teams:
Team 1 ___________ _____________ ___________ ____________
Team 2 ___________ _____________ ___________ ____________
Team 3 ___________ _____________ ___________ ____________
Suggestions for preparing for the team meeting:
Select a leader who will keep the group on track using the sequence of steps in the
process.
Select a person who will record your work to answer each question on the examination
sheet.
Select a timekeeper who will call off the time remaining to be sure you use an even
amount of time for reach question (30 minutes for each question, one hour total only).
Don’t get bogged down in interpersonal dynamics; stay focused on the task of answering
the question together as a team as best you can.
Review the process as described in Exercise 18.
Practice locating sources on different databases efficiently.
Refer to relevant methodological review forms (e. g., CONSORT) to guide critical
appraisal of best source found but do not get bogged down in reviewing quality of
research.
Your task is to locate the best (most rigorous and relevant source) to answer your
question. Depending on the question, this will be one of the following:
Systematic review or meta-analysis of randomized control trials
A high quality randomized controlled trial
Quasi-experimental study
Multiple single case (N of 1) studies.
Expert opinion
4
Release for Use of Written Materials Generated in EBP Workshop
By entering into a written materials release dated ___________ described here, I
_________________ (please print your name) consent to the use of written material (such as
COPES questions and search strategies) generated by me during a class or workshop presented
by _____________________ for the purpose of training students, professional social workers,
and others in the helping professions. I give my consent to the use of the written materials in
instructional materials, including textbooks, training manuals, and journal articles, prepared by
___________ for teaching evidence-based practice. My consent is given with the understanding
that ____________ may publish the instructional materials. Any copyrights, trademarks, or
patents will remain with ________________ or any third parties with whom he may contract.
I understand that my cooperation is purely voluntary. Neither providing nor withholding my
permission will affect by grade or the number of continuing education units I receive for
participating in the class/workshop in any way.
Signed ________________________________________ Date ______________
5
WORDING WHEN USED AS AN EXAMINATION
Team 1:______________ ___________ _____________ ____________
Team 2:______________ ___________ _____________ ____________
Team 3:______________ ___________ _____________ ____________
Have one person record the group’s work on this sheet making sure all sign the examination
sheet. Give this to your instructor with timesheet.
Background:
You have rudimentary knowledge of how to pose and answer questions to make well-reasoned
decisions. Apply the process we have studied to make your recommendations. Work as a team
to accomplish your task. Work through the problem in sequence, answering each question in
turn. (Work as a group on one question at a time. Budget your time and spend about half the
time on each question. You have one hour, no more.)
Question #1: Delinquency Prevention (30 minute limit).
Assume that you have taken a job as a probation-parole officer working with juvenile clients
who have been adjudicated by a local juvenile court. Your supervisor at your agency has asked
you for your opinion about whether juvenile, who are served by your probation-parole agency,
should participate in a delinquency prevention program patterned after the one in the popular
video titled “Scared Straight.” This video shows an innovative program put on by “lifers”
serving a life sentence that is intended to literally scare the delinquents straight.
1. What is your COPES (PICO) question?
2. Please record your search plan here including terms to mark key concepts and to include
appropriate MOLES (methodological filters)
3. Please record, if necessary on additional sheets, the most successful search histories or
history for your group, including databases searched, terms applied, and numbers of hits, to
locate your best document.
6
4. Best source found. Give complete reference APA style.
5. How sound is your best source relative to criteria on the appropriate evidence rating form?
Please include a brief paragraph to summarize your assessment of the evidence quality.
6. What would you tell your supervisor about trying the Scared Straight program to prevent
delinquency?
7. Can you calculate Number Needed to Treat (NNT) for any studies?
Question 2 (30 minute limit).
Assume that one of you has taught for several years, and you now find yourself the principal of a
middle school and high school that includes grades 7 through 12. You are concerned about
alcohol misuse among young people thorough experience with several tragic deaths. You
wonder what primary prevention program (primary means preventing the initial occurrence of a
problem) would most effectively prevent alcohol misuse among young people. What approach
would you try?
1, What is your COPES (PICO) question?
2. Record your search plan here including terms to mark key concepts including appropriate
methodological filters.
7
3. Record, if necessary on additional sheets, the most successful search histories or history for
your group including databases searched, terms applied, and number of hits to locate your
best document.
4. Best source found (give complete citation APA style).
5. How informative is your best source relative to criteria on the appropriate rating form?
Include a brief paragraph summarizing your assessment of evidence quality.
6. What intervention does this source support? Can you calculate NNT?
Please sign your names below.
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
_____________________________________________________________________________
Exercise 19: PREPARING A CRITICALLY APPRAISED TOPIC (CAT)
_____________________________________________________________________________
OVERVIEW
Summaries of research related to a practice or policy question are helpful in making practice
decisions. You can use the format of a CAT to share what is found. Each student prepares a
critically appraised topic. Students may contact a helping professional to solicit a question. A
contact person who works in a helping profession could be suggested who might provide a
question.
PURPOSE
This exercise provides an opportunity to pose and answer an important practice question using
the format for a CAT
MATERIALS REQUIRED
1. Relevant material on the Internet regarding preparing a CAT including a CAT outline.
2. Access to relevant databases.
3. Access to appropriate rating forms regarding research reports.
4. Models of CATs.
TIME REQUIRED
Depends on how swiftly key research is located and students‟ skills in preparing a CAT. It also
depends on whether students work in terms. Getting students up to speed so they can do their
CATs may take multiple meetings. A class of eight students spent five classes practicing
working as a team while searching. Students were given case material and asked to pose a well-
built question and to work in the computer lab to locate their best source; critically appraise it,
and come to a conclusion about what to recommend.
SUGGESTIONS
Guide students as needed based on CATs available. Carefully review all answers regarding
completeness and rigor of search and accuracy of conclusions and provide corrective feedback.
2
CAT OUTLINE
Critically Appraised Topic Outline (use visual aids, no more than 10 minutes).
1. State who you are, source of your question, where that person works, and why the question
is worth answering (e. g., presenting concern of client).
2. State your well-structured question.
3. Describe your search plan.
4. Describe your search history.
5. Give your best source. (Give full reference APA style.) Include evaluation on the
appropriate rating form.
6. Based on this source, how would you answer the question? The evidence may not be
sufficient to come up with a recommendation. There may be contradictory results. The
results may be clear regarding positive effects but the intervention may create harmful
effects; if so, how do you weight their relative impact?
3
SHORT EXERCISE
Tick Bite
A family is in the church parking lot one Sunday morning walking toward church. Mary turns to
her friend and says, „I feel something on my neck. It might be a tick.” Her friend says “It is a
tick. I can see the legs moving.” Mary called the triage nurse at the local hospital because she
was concerned about Lime‟s Disease. The nurse told Mary to remove the tick, apply some
antibiotic, and watch for a distinctive “bull‟s eye rash” that might appear around the bite. If the
rash appears or the area grows red and inflamed, Mary should come in for tests.
At the drug store, Mary‟s friend buys antibiotic cream and a small magnifying glass with a sharp
tipped tweezer attached. When they get home, Mary‟s friend removes the tick with the tweezers
leaving its head embedded in Mary‟s neck. Mary‟s friend applies the antibiotic. Their question
concerns whether they removed the tick properly or should they have used some other method.
1. What is the COPES question?
2. Record your search history including databases, search terms, MOLES, number of
documents.
3. Best source found (use APA style).
4. Critical appraisal of best source applying appropriate evidence rating sale.
4
5. Recommendation for action and reasons
6. Did Mary and her friend do the right thing to treat the tick bite?
WORKING IN TEAMS
Divide the class into two groups. Give one group a problem like the one above. Give another the
task of observing how the team approaches the task. Have the observers look for whether the
task group followed the spirit of inquiry described in the quote below from the Wright Brothers
(100th
Anniversary of Flight Dec. 17, 2003) and report on what they observed about the group‟s
spirit of inquiry.
Wilbur and Orville had a technique for arguing that made others think them eccentric. People
often heard them arguing, yelling at each other, gesturing. Then observers saw them quiet down
for a moment but begin to argue again but switching sides in the argument, Wilbur arguing
Orville‟s point, Orville arguing Wilbur‟s point. In this manner, the Wrights avoided getting
stuck when solving a problem. In fact, they solved a whole series of problems that eluded the
best engineers of their time … Part of their genius was in how they argued. Spratt complained
that the brothers [Wilbur and Orville‟s] method of “rounding the corners” by switching sides in
the middle of an argument struck him as dishonest, Wilbur replied:
It is not my intention to advocate dishonesty in argument nor a bad spirit
in a controversy. No truth is without some mixture of error, and no error
so false but that is possesses some elements of truth. If a man is in too big
a hurry to give up an error he is liable to give up some truth with it, and in
accepting the arguments of the other man he is sure to get some error with
it. Honest argument is merely a process of mutually picking the beams
and motes out of each other‟s eyes so both can see clearly…. (Howard, p.
110).
From: Howard, F. (1998). Wilbur and Orville: A biograph of the Wright Brothers. Mineola,
NY: Dover.
5
EXAMPLE OF FINAL EXAMINATION FOR COURSE
Final Examination of Practice Application of Evidence-Based Practice. The class was
expected to work as a team in the computer laboratory to answer both questions within the
time limit (one hour). The class was informed that they would share the grade. The
grading criteria included: (1) inclusion of all four elements of a well-built question; (2) use
of appropriate methodological filters; (3) description of search history including search
terms and databases searched; (4) use of appropriate form for rating evidence quality; 5)
located appropriate marker document or better document; (6) accurate critical appraisal
of materials; (and (7) conclusion based on the best evidence that included discussion of
some index of treatment effect size. Work as a group on one question at a time. You have one
hour to answer both questions. Budget your time and spend about half the time on each
question.
Question #1: Family Disagreement
A family in the Eau Claire Area has an ongoing argument. The wife believes that a motorized
electric powered tooth brushing will remove more plaque from teeth. The father and son believe
that manual tooth brushing will remove plaque just as well or better. Your job is to help the
family resolve the disagreement by searching for the current best evidence.
1. Describe your COPES question here.
2. Record your search plan here including terms to mark key concepts. Include appropriate
quality filters.
3. On additional sheets record the search histories for your group including databases
searched, terms applied, and number of hits.
4. Best source found (APA style).
5. How rigorous is your best source relative to criteria described on an appropriate evidence
rating form? Include a brief paragraph to summarize your assessment.
6. Bottom line.
6
Question #2
A physician has discovered a primary basal cell carcinoma on your friend‟s nose that has been
confirmed by a laboratory test. The tumor is about the size of this “0” on the edge of his nostril.
Your friend wonders if the doctor should remove the tumor surgically, use radiotherapy, or use
cryotherapy.
1. Describe your COPES question here.
2. Record your search plan here including terms to mark key concepts and appropriate quality
filters.
3. On additional sheets, record the search histories for your group including databases
searched, terms applied, and number of hits.
4. What tis your best source (APA style)?
5. Describe your best source relative to criteria on an appropriate rating form. Include a brief
paragraph to summarize your assessment.
6. What intervention does this source support, if any?
7. Bottom line?
__________________________________________________________________
Exercise 20: INVOLVING CLIENTS AS INFORMED PARTICIPANTS
__________________________________________________________________
OVERVIEW
Clients are often not involved as informed participants in decisions made; they may be
misinformed or uninformed about recommended services and alternatives.
PURPOSE
This exercise provides an opportunity for students to understand the requirements for informed
consent and to implement this using the form in Box 20.1 illustrating what information should be
included in accurately informing clients.
MATERIALS REQUIRED
1. Practice Exercise 20 and Box 20.1.
2. Access to relevant databases.
TIME REQUIRED
This exercise could take a few hours. Students must do a search related to outcomes selected.
They must also gather information regarding agency services. This information may not be
accessible.
SUGGESTIONS
Discuss the challenges to involving clients as informed participants. For example, some clients
may not want to be involved as informed participants or staff may not have the knowledge or
skills to involve clients as informed participants. Engage students in a discussion of when
informed consent is an ethical obligation even if a client does not want to be fully informed, for
example in child welfare. Discuss possible objections from agency administrators to using the
form in Box 20.1.
REVIEWING STUDENTS WORK
What obstacles arose? How did students respond to them? What excuses did they use for not
providing clear information?
_____________________________________________________________________________
Exercise 21: ASKING HARD QUESTIONS
__________________________________________________________________
OVERVIEW
Evidence-informed practice requires asking questions such as "Is there any evidence that this
service does more good than harm?" Some people view questions as adversarial and obstructive
rather than as essential for helping clients make informed decisions. Staff may be expected to
follow the “party line” and may be punished for asking vital questions.
PURPOSE
1. To identify questions that should be asked.
2. To illustrate their relevance to informed consent and evidence-informed practice.
3. To enhance student's skills in raising questions in an effective manner.
MATERIALS REQUIRED
1. Practice Exercise 21, including Boxes 1.4 and 1.1.
TIME REQUIRED
This depends on how many practice opportunities are offered in raising questions and gaining
feedback.
SUGGESTIONS
A reluctance to pose questions may be due to anxiety created by fears of rejection and retaliation.
Such fears may be warranted. Decreasing this fear may require desensitization via exposure; that
is giving students repeated practice in raising questions in increasingly difficult situations. A
reluctance to pose questions may also be due to a lack of concern with clients' right to competent
services. Students may be reluctant to raise questions because they do not consider them
important.
REVIEWING STUDENTS’ EXPERIENCES
What were common obstacles to raising hard questions? How skilled were students in raising
questions? What are important self-development goals?
__________________________________________________________________
Exercise 22: EVALUATING SERVICE OUTCOMES
__________________________________________________________________
OVERVIEW
Most professionals are employed in some sort of organization such as an agency. Whether a
direct line practitioner, administrator or supervisor, skill in evaluating service outcomes is
important. Do services do more good than harm? Only if services offered are carefully
evaluated regarding quality and outcome, can they be systematically improved.
PURPOSE
1. To highlight the importance of carefully evaluating agency services.
2. To provide guidelines for evaluating service as well as practice in using them.
MATERIALS REQUIRED
1. Practice Exercises 22.1 and 22.2.
2. Access to relevant databases.
TIME REQUIRED
These exercises require students to gather information. This may require visiting an agency and
reading agency reports. Students may also search for research regarding a particular service.
SUGGESTIONS
Evaluation is a sensitive topic. Obstacles to careful evaluation such as administrators who may
not want to reveal what is done to what effect because of concerns about future funding should
be discussed. Caution students to "pick their battles" regarding when and how to try to gather
evaluative data. They are in low power positions in agencies compared to administrators and
senior clinicians. Administrators may not look favorably on involving clients as informed
participants. Give students practice in raising evaluation issues diplomatically. They could draw
on Getting to Yes by Fischer & Ury, 2011. Engage students in a probing discussion of how
services are and should be evaluated from a client’s point of view.
__________________________________________________________________
Exercises 23.1 and 23.2: REVIEWING YOUR EXPERTISE
__________________________________________________________________
OVERVIEW
Clinical expertise is drawn on to integrate the many sources of information pertaining to a
decision. Such expertise is a key ingredient of being an effective practitioner. This exercise
introduces students to components of expertise and related research.
PURPOSE
To provide an opportunity for students to become familiar with research regarding expertise and
to reflect on their expertise. That is, observation during structured interviews is needed to
accurately estimate expertise as is knowledge regarding outcome of services to review their own
expertise.
MATERIALS REQUIRED
1. Material in Exercise 23.
TIME REQUIRED
Depends on activity. (See Practice Exercises 23.1 and 23.2.)
SUGGESTIONS
Engage in students in a review of Box 23.1. Ask students to complete the Checklist in 23.1 (pp.
253-254) and identify two competencies they would like to enhance and prepare a plan for doing
so. Review these plans in terms of likely effectiveness.
Ask students to complete Exercise 23.2 (Errors) and discuss kinds of errors. Ask them to
indicate what errors are of most concern to them and discuss how they could be decreased.
Engage students in a discussion regarding expertise and time in practice and experience. Does
effectiveness increase with time and experience? See Goldberg, S. B. et al (2016). Do
psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a
clinical setting. J of Counseling Psychology, 63, 1-11.
1
_____________________________________________________________________________
Exercise 24: EVALUATING EFFECTIVENESS STUDIES: HOW GOOD IS THE
EVIDENCE?
____________________________________________________________________________
OVERVIEW
Students select (or are given) an outcome study related to practice. They then rate the study using
3 different forms.
PURPOSE
1. To identify the hallmarks of a sound intervention study.
2. To examine effect size.
MATERIALS REQUIRED
1. Quality of Study Rating Form (QSRF) Box 24.1.
NOTE: Error in QSRF #6. Should read “subjects randomly selected (not assigned).”
2. An intervention study.
TIME REQUIRED
This depends on whether students are asked to locate an outcome study themselves that concerns
a client or a study is given to them to rate. In the former case, students may spend time searching
for a relevant study. About 60 or more minutes will be needed to review criteria, 60 minutes to
read and rate the study, and 30 minutes to discuss ratings. Reviewing potential biases will
require time. (See Box 5.1, page 258).
SUGGESTIONS
Students differ in their knowledge about research methods. You can use the Quality of Study
Rating Form to provide an overview of major features of an intervention study. If students select
an outcome study, ask them to attach a copy of the study to the rating form.
You can expect reasonable inter-rater agreement for most items on the QSRF among students
who independently rate the same study if you explain criteria on the form. If you want students
to calculate ESl and ES2, they need specific instructions about which effect sizes to compute for
what outcomes at what outcome interval; they need some statistical sophistication for more
complex articles. Many articles omit essential information, so these indices cannot be computed.
Consult additional sources as needed for interpreting ES1 and ES2.
2
DISCUSSION QUESTIONS
1. Discuss variations in evaluations among students.
2. What are Effect Sizes? Was there agreement? Were they accurately calculated? Discuss
controversies about using effect sizes.
3. Would students recommend this intervention? Ask them to explain their reasons.
4. What have students learned? We hope responses include: (1) it is possible to take some
guesswork out of evaluating the quality of a research report, and (2) students working
independently can arrive at reasonably consistent appraisals for the same study.
____________________________________________________________________________
Exercise 25.1: CRITICALLY APRAISING RESEARCH REVIEWS: HOW GOOD IS
THE EVIDENCE?
_____________________________________________________________________________
OVERVIEW
Locating high-quality research reviews regarding questions is an important skill.
PURPOSE
This exercise provides practice in locating and apprising research reviews. Characteristics of
high quality reviews and sources are described including the Cochrane and Campbell Libraries.
Students should be warned that reviews may be misleading (see Ioannidis, 2016).
MATERIALS REQUIRED
1. A research review to evaluate.
2. QUORUM guidelines. See also p. 60 in Geyman, Deyo and Ramsey (2000); Greenhalgh,
2010).
3. Access to relevant databases.
TIME REQUIRED
One to two hours.
SUGGESTIONS
Discuss the various kinds of reviews and their purposes (e. g., Greenhalgh, et al., 2011; Pettigrew
& Roberts, 2006). Give students sources of high-quality reviews and practice in locating them.
Select questions of interest to students to motivate them to search effectively for rigorous
reviews. Carefully review concerns discussed by Ioannidis (2016).
REVIEWING STUDENTS WORK
How skilled are students in locating and critiquing research reviews? What kind of errors do
they make? Is access to relevant databases a problem? Can access be improved? How rigorous
were students’ evaluations? How effective were their searches? What obstacles did they
encounter? Have services used in students’ field work agencies been critically appraised and
results made accessible to all involved parties, including clients?
DISCUSSION QUESTIONS
Discuss common flaws in research review. Read and discuss Ioannidis (2016).
_____________________________________________________________________________
Exercise 25.2: CRITICALLY APPRAISING PRACTICE GUIDELINES
_____________________________________________________________________________
OVERVIEW
Practice guidelines are promoted on many websites and by many organizations. Are all well
argued?
PURPOSE
To enhance skills in critically appraising practice guidelines.
MATERIALS REQUIRED
1. A practice guideline to evaluate.
2. Guidelines for reviewing (e. g., Greenhalgh, 2011).
3. Access to relevant databases.
TIME REQUIRED
Depends on whether students locate a guideline of interest or you give them one to evaluate.
Evaluation will take 60 minutes and class discussion 60 minutes.
SUGGESTIONS
Select a guideline of interest to students. Encourage students to identify interested parties (e. g.,
pharmaceutical companies). (See also Exercise 8.)
REVIEWING STUDENTS WORK
What errors to students tend to make when reviewing guidelines? What were their beliefs about
guidelines prior to reading related critiques?
DISCUSSION QUESTION
Why do practice guidelines differ in quality?
____________________________________________________________________________
Exercise 26: CRITICALLY APPRAISING SELF-REPORT MEASURES
______________________________________________________________________________
OVERVIEW
Social workers may rely on self report measures in their work, perhaps from other professionals
such as psychologists. How valid are these?
PURPOSE
1. To increase skills in accurately appraising the validity of self-report measures.
2. To increase skills in identifying sources of bias in self-report measures.
3. To give students practice in critically appraising a self-report measure.
MATERIALS REQUIRED
1. A self-report measure.
2. Practice Exercise 26.
3. Description of reliability and validity in text.
4. Access to relevant databases.
TIME REQUIRED
An in depth discussion of reliability and validity will take at least an hour or two of class time,
and students should review material at home. Give students opportunities to assess their
knowledge about these important concepts.
SUGGESTIONS
Review the many sources of bias in self-report measures and give students repeated opportunities
to detect them. As always, emphasizing the possible negative effects on clients of ill-advised
decisions based on biased self report data may enhance motivation to attend to such possibilities.
Developing a sound understanding of reliability and validity and their relationship is important
and professionals can draw on this knowledge throughout their careers. To tap
misunderstandings, ungraded quizzes offer valuable corrective feedback.
REVIEWING STUDENTS’ WORK
Do students understand different kinds of reliability and validity and the relationship between
reliability and validity? How accurate are they in their appraisal of research articles in relation to
these concepts?
1
____________________________________________________________________________
Exercise 27.1: ESTIMATING RISK AND MAKING PREDICTIONS ____________________________________________________________________________
OVERVIEW
This exercise identifies common pitfalls in estimating risk and making predictions (e.g., relying
on anecdotal experience and ignoring base rates). Students complete an exercise translating
probabilities into natural frequencies to estimate risk.
PURPOSE
1. To introduce basic concepts concerning risk assessment and decision making.
2. To encourage further reading about related concepts and common errors.
MATERIALS REQUIRED
1. Exercise 27.1.
2. Answers.
TIME REQUIRED
Fifty minutes (30 for calculations, 20 for discussion).
SUGGESTIONS
This exercise highlights the value of using natural frequencies to estimate risk. Problems in
estimating risk include a lack of information about prevalence, sensitivity, and specificity. We
cannot predict what an individual will do based on group data; we can only describe the baserate
data of the group. Predictions based on actuarial data are more accurate than those relying on
clinical intuition (Grove & Meehl, 1996). See also description of use of predictive risk modeling
in child welfare.
DISCUSSION QUESTIONS
Do social workers consider the prevalence of a problem among an agency's clients as well as
sensitivity and specificity of a test's results in estimating the predictive value of a positive test
result?
Research exploring accuracy of professionals in estimating predictive value of a test show that
they overestimate their accuracy (Gigerenzer, 2002; Welch, 2004). Eddy (1982) found that 95 of
100 physicians estimated the predictive value of a positive mammogram (X-ray test for breast
cancer) to be 75%, where in reality it was only .077, or about 8% (p. 253).
2
Is the predictive value of a positive test greater where the prevalence rate is high or where it is
low?
__________________________________________________________________________________
Exercise 27.2: CRITICALLY APPRAISING A RISK MEASURE
____________________________________________________________________________
OVERVIEW
You may have to estimate risk, for example of child abuse or suicide. Being informed about
how best to do so is important to protect clients from actions taken based on uninformed or
misinformed bases.
PURPOSE
To enhance skills in estimating risk.
MATERIALS REQUIRED
A risk measure of interest to students.
TIME REQUIRED
At least 60 minutes.
SUGGESTIONS
Be sure to select a measure that pertains to tasks students confront.
DISCUSSION QUESTIONS
1. How accurate is this measure?
2. Is the measure widely used?
3. Does use of this measure do more good than harm?
1
_______________________________________________________________
Exercise 28: EVALUATING DIAGNOSTIC TESTS ___________________________________________________________________________
OVERVIEW
This exercise involves students in reviewing a diagnostic test. Reflex dilation was a test devised
to determine whether children had been sexually abused. The history of the helping professions
provides hundreds of examples of the use of inaccurate diagnostic tests resulting in harm to
clients. The example in this exercise is one that occurred in England. This exercise illustrates the
importance of considering concepts such as false positives and false negatives that may sound far
removed from the everyday world of practice, but in fact, influence clients' lives. This exercise
can be used to raise key questions about diagnostic measures.
PURPOSE
To highlight questions that should be raised about diagnostic tests and the dangers of using
questionable tests (they do not measure what they claim to measure).
MATERIALS REQUIRED
Description of reflex dilation test in workbook.
TIME REQUIRED
Thirty minutes.
SUGGESTIONS
Discuss the life-affecting implications of faulty decisions based on invalid tests including both
false positives (e.g., the child has been abused but is judged to have been be abused) and false
negatives (the child has been abused and judged not to have been abused). Discuss implications
for children, parents, and professionals.
POSSIBLE ANSWERS TO QUESTIONS
1. What questions should be raised about this test?
Suggestion: Review Exercise 28, which presents major issues related to this exercise. We
recommend laying out the decision on a 2 X 2 table, with Reflex Dilation Test Result
(positive, negative) on the left, and True State (child abused, child not abused) across the top.
Thus, there will be four cells, two that describe errors (false positive, false negative) and two
that describe accurate judgments (true positive, true negative). In this example, we do not
2
have all the data needed to fill in the four cells. You could accompany this exercise with data
in the literature that suggest the false positive and false negative rates of alleged child abuse.
We hope that the exercise will prevent students from making the kinds of errors shown here
and will highlight how easy it is to do so by asking questions about false positive and false
negative rates.
2. Would you use this test? Please explain your answer.
Suggestion: The bottom line here would be the predictive value (see Exercise 28).
_____________________________________________________________________________
Exercise 29: EVALUATING RESEARCH REGARDING CAUSES
_____________________________________________________________________________
OVERVIEW
Professionals try to identify the causes of clients' concerns. This exercise provides an
opportunity to discuss some of the complications and sources of bias in discovering causes.
PURPOSE
1. To identify types of causes.
2. To give students experience in identifying these.
3. To identify common errors and their consequences.
4. To learn about helpful criteria when evaluating research studies exploring causation.
MATERIALS REQUIRED
1. A relevant article to review.
2. Guidelines for reviewing related articles.
3. Practice Exercise 29.
4. Access to relevant databases.
TIME REQUIRED
Depends on the complexity of the article reviewed.
SUGGESTIONS
You could review the major kinds of causes (e.g., biological, psychological, sociological and so
on) as well as possible interactions among them as in multi-causal accounts. Common kinds of
oversimplifications and their effects on decisions can be reviewed as well as how to spot these.
REVIEWIGN STUDENTS’ WORK
What errors do students make in reviewing material regarding causation? Were certain kinds of
reports, for example, those including pictures of brains, especially misleading?
1
_____________________________________________________________________________________________________________________
Exercise 30: ANALYZING ARGUMENTS
_________________________________________________________________
OVERVIEW This exercise provides guidelines for analyzing arguments as well as practice opportunities.
Accepting or rejecting an argument regarding client care can have profound implications for
client welfare.
PURPOSE
To increase skills in argument analysis.
MATERIALS REQUIRED
1. Argument Analysis Form (Exhibit 30).
2. Guidelines for Evaluating Arguments in Exercise 30.
3. Arguments to analyze.
TIME REQUIRED
This will vary depending on how many arguments you review.
SUGGESTIONS
This exercise provides an additional opportunity to develop students' argument analysis skills.
You could use it in class and/or as a homework assignment. We find it helpful to precede this
exercise with class practice in which practice-related arguments are identified, and premises as
well as warrants related to conclusions are reviewed in terms of their acceptability, relevance,
sufficiency, and rebuttal (see guidelines in the workbook). You could hand out model argument
analyses to highlight valuable criteria for reviewing arguments. Damer (1995) stresses that no
matter how often he emphasized the importance of rebuttal, many students do not include this in
their reply. Reasons given include not wanting to reveal weak points in their arguments.
You could break students into groups of three or four and ask them to prepare an argument
analysis which is then presented to the entire class. Draw on Richard Paul's ideas for involving
all students. For example, rather than designating someone in each group to report back to the
entire class, you could tell the class that you will call randomly on one of the members in each
group to present the group's "thinking." This will increase the likelihood that every person in a
group will be ready to present.
Highlight the consequences of basing decisions on faulty arguments in a range of contexts
including work with individuals, families, groups, organizations and communities. You could
2
include a discussion of different kinds of explanations for behavior (e.g., developmental,
biological, psychological, sociological) as they may be involved in the warrants students give to
link premises to conclusions. Initial arguments should be short until students become skilled in
identifying the key elements in arguments.
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
1. What was the most difficult part of completing your argument analysis?
Suggestion: Common problems include accurately evaluating relevance, acceptability, and
sufficiency of premises related to a conclusion. Other problems include overlooking key
premises, separating one argument from another, determining whether a statement presents
an argument (it may just make a claim), and clearly identifying conclusions.
2. Did you come up with effective rebuttals to your argument?
Suggestion: Review rebuttals students offer. Can stronger ones be made? Have they
forgotten to provide rebuttals? If so, ask them why.
3. Discuss relevance of different goals of an argument such as inquiry, deliberation, or
persuasion.
1
______________________________________________________________________________
Exercise 31: CRITICAL THINKING AS A GUIDE TO MAKING ETHICAL
DECISIONS
______________________________________________________________________________
OVERVIEW
This exercise draws on the vignettes in Exercises 11, 12, and 13 to review ethical questions that
arise in everyday practice. Ethical issues involved in each vignette are suggested. Many writers
have emphasized the value of critical thinking in arriving at moral and just decisions (e.g.,
Baron, 1994; Brookfield, 1987; Gambrill, 2005; Paul, 1993). Considering the multiple interests
of different parties increases the likelihood of arriving at equitable decisions.
PURPOSE
To illustrate the value of critical thinking as a guide to making ethical decisions in helping
contexts.
MATERIALS REQUIRED
1. Vignettes for Exercises 11, 12, and 13.
2. Checklist of Ethical Concerns (see Box 31.1).
3. Suggested Ethical Issues in vignettes.
TIME REQUIRED
This will vary depending on how many examples are reviewed. To avoid getting "stalled" on one
item, we recommend setting a ten-minute time limit for discussing each item.
SUGGESTIONS
You could select vignettes that involve particular kinds of concerns and ask students to identify
related ethical issues. You could precede this by a discussion of factors that may influence
decisions such as agency policy, client wishes, the wishes of significant others, legal regulations,
as well as professional codes of ethics.
Students can hone their argument analysis skills by preparing arguments for their positions. You
can ask them to clearly identify the premises from which they argue that a certain ethical
principle is involved and/or has been violated in a certain situation. Students can work in groups
to identify ethical issues in vignettes and provide arguments for their beliefs, and then share their
results with the rest of the class.
2
REGARDING FOLLOW-UP QUESTIONS
Check whether students think important ethical issues have been overlooked in the examples
given. If so, what are they and what are examples?
Encourage an open discussion of different views. Ask students if they believe professionals are
ethically bound to think critically about decisions they make. We hope that, among others, three
reasons will emerge: (1) because critical thinking involves weighing evidence about what helps
clients, it would be unethical not to think critically about practice decisions; (2) because critical
thinking implies intellectual honesty, critical thinkers will be more likely and able to inform
clients more accurately about options, costs, risks, and possible outcomes of different plans; and
(3) because critical thinking implies fairness and objectivity, critical thinkers will be less likely
to impose stereotypes and prejudices on others.
______________________________________________________________________________
Possible Answers (See also Box 31.1.)
Game A: Common Practice Fallacies
1. Ethical issue: The recommendation made is based on one individual's experience with an
intervention; it offers no guidance as to whether the method will help anyone else.
2. Ethical issue: Professionals have a responsibility to be clear. Vagueness decreases the
likelihood of understanding client concerns and identifying related causes. It increases the
likelihood that helpers will, unwittingly, end up talking about different things.
3. Ethical issue: This administrator is trying to get an accurate picture of success rates-an
admirable professional aim.
4. Ethical issue: Whether intentionally or not, this politician uses a propagandistic method to
question a welfare program. Politicians should present issues, questions, and problems
fairly and avoid propaganda ploys, such as relying on vivid, atypical case examples.
5. Ethical issue: Here a professional makes a claim of effectiveness based on pre-post data.
This violates our obligation to base recommendations on sound evidence.
6. Ethical issue: Supervisors should think critically about claims made and educate their staff
to do the same. This supervisor is either/or-ing.
7. Ethical issue: This example illustrates a lapse on the part of an author of an article in a
professional journal. Rather than examining all articles related to a question, - both pro and
con - this author used only those in support of one position. This will not help readers judge
the effectiveness of family-based treatment.
8. Ethical issue: See #5.
3
9. Ethical issue: No evidence is provided that anxiety is a brain disorder.
10. Ethical issue: Appeal to unfounded authority.
11. Ethical issue: This vignette illustrates a concern to make decisions based on available
evidence.
12. Ethical issue: This statement is contradicted by data that shows that tracking progress
contributes to positive outcome (Boswell et al., 2013; Lambert, 2015).
13. Ethical issue: What happens with the rest? Administrators are ethically bound to provide
accurate (rather than misleading) information.
14. Ethical issue: Newness does not equate with effectiveness.
15. Ethical issue: Professional organizations such as the American Psychiatric Association (the
publishers of the journal) have an obligation to inform rather than mislead, not only in their
published articles but in their human-services advertisements. The vague claims made in
this ad are based on one case example.
16. Ethical issue: Professionals are obliged to present sufficient information about a study so
that we can judge for ourselves whether related conclusions are warranted.
17. Ethical issue: Relies on vague data.
18. Ethical issue: Vagueness is also a problem here. No information is given about the study.
19. Ethical issue: Newness does not equate with effectiveness.
20. Ethical issue: Overconfidence and vagueness reigns here.
Game B: Group and Interpersonal Dynamics
1. Ethical issue: Careful consideration of different possibilities at case conferences helps to
avoid wrong choices. Focusing on her goal (to help clients) will give this professional the
courage to pursue a culture of thoughtfulness and ignore any putdowns that occur along
the way.
2. Ethical issue: Basing decisions on small samples in artificial settings and overlooking
data based on observation in real-life settings may result in poor decisions and (in this
example) can result in injury to staff.
3. Ethical issue: Professional helpers should base decisions on the best evidence available
and question decisions based on weak grounds.
4
4. Ethical issue: Professionals are ethically bound to consider the quality of evidence related
to decisions. Lizzie and Polly do not; they argue ad hominem rather than ad rem. They
should share objections they have and not whisper to each other.
5. Ethical issue: Professionals have an obligation to focus on making sound decisions
during case conferences and to avoid sidetracks that can get in the way of making them.
6. Ethical issue: Professionals have an obligation to avoid misunderstanding of theories and
related data that may help clients.
7. Ethical issue: Professionals have an ethical responsibility to base decisions on sound
rather than weak criteria. Tradition is not a sound basis for making decisions.
8. Ethical issue: Professionals are legally and morally obliged to follow up on reports of
child abuse, and should do so in a way that protects the rights of alleged perpetrators.
False allegations may have devastating consequences.
9. Ethical issue: No additional clear information is provided for this premature diagnosis.
10. Ethical issue: Stereotyping is not only ethically wrong but also may be illegal.
11. Ethical issue: Professionals have a responsibility to match clients and recommended
interventions in a way that maximizes success. Also, strawman arguments do not
forward careful decisions.
12. Ethical issue: Professionals have a responsibility to critically review diagnostic labels to
make sure they help (i.e., point to etiology and/or effective interventions) more than harm
(i.e., obscure possible options). It is unethical to stigmatize people by assigning labels
that provide no effective intervention guidelines. Negative labels may follow children
through their educational careers.
13. Ethical issue: What is the evidentiary status of claims in these reports? Also because a
person has a certain diagnosis does not necessarily entail spilling food. Professionals
have an ethical obligation to make decisions based on evidence and well-argued theory.
14. Ethical issue: Those who make claims about the effectiveness of intervention methods
should clearly describe relevant evidence. Also, professionals are obliged to raise
questions about evidentiary claims that affect clients’ lives.
15. Ethical issue: Professionals have an obligation to base decisions on sound criteria rather
than weak criteria such as manner of presentation.
16. Ethical issue: Professionals are obliged to respond appropriately to criticism of work-
related behavior -- admit to being late.
17. Ethical issue: See #15.
5
18. Ethical issue: There may be a third option (leave children in the home and provide
services needed to safeguard children). Professionals have an obligation to select the
least intrusive effective methods. Placing children in foster care is an intrusive
intervention.
19. Ethical issue: Professionals have an obligation to seek options that maximize the
likelihood of helping clients. This means avoiding either-or framing that may result in
overlooking promising options.
20. Ethical issue: Professionals have an obligation to avoid distractions where life-affecting
decisions are at hand.
21. Ethical issue: Attention to evidentiary issues is neglected.
Game C: Clinical Reasoning
1. Ethical issue: Failure to consider the role of hindsight bias may result in poor decisions.
2. Ethical issue: Regression effects are ignored which may result in an ill-advised decision.
3. Ethical issue: Professionals have an obligation to avoid misleading effects of how a
choice is "framed."
4. Ethical issue: Professionals have an obligation to be informed about contextual factors
related to client concerns in order to arrive at a sound assessment. This means
considering not only the client's family interaction, cultural norms, and values, but also
environmental factors such as lead based paint residues.
5. Ethical issue: Actually, research regarding psychotherapy shows that helpers are not
very accurate in detecting deterioration (Shimokawa and colleagues, 2010).
6. Ethical issue: This nurse is attending to evidentiary issues.
7. Ethical issue: See #1.
8. Ethical issue: See #3.
9. Ethical issue: Professional helpers have an obligation to base decisions on sound
evidence. 0063
10. Ethical issue: See #2.
11. Ethical issue: Outcome monitoring is related to progress (Lambert, 2015). Thus ignoring
this is an ethical lapse..
6
12. Ethical issue: This caseworker has fallen into the gambler’s fallacy. Professionals have
the right to freedom from undue danger in carrying out their responsibilities. Social
workers sometimes go into situations that even the police have refused to enter.
13. Ethical issue: See #2.
14. Ethical issue: Professionals have an obligation to conduct a thorough assessment. This
requires questioning initial assumptions and searching for alternative views.
15. Ethical issue: Professionals have an obligation to avoid confirmation biases.
16. Ethical issue: Professionals have an obligation to make sure their emotional reactions do
not impede high quality services.
17. Ethical issue: Professionals have an obligation to avoid confirmation biases.
18. Ethical issue: Professionals should base decisions on the best evidence available.
19. Ethical issue: Professionals have an obligation to be knowledgeable about cognitive
biases such as the gambler's fallacy that may influence decisions.
20. Ethical issue: Professionals should attend to harms of omission as well as harms of
commission.
21. Ethical issue: See #2.
Reference
Boswell, J. F., Kraus, D. R., Miller, S. D., & Lambert, M. J. (2013). Implementing routine
outcome monitoring in clinical practice: Benefits, challenges, and solutions.
Psychotherapy Research. DOI:10.1080/10503307.2013.817696
Lambert, M. J. (2015). Progress feedback and the OQ-System: The past and the future.
Psychotherapy, 52, 381-390.
Shimokawa, K., Lambert, M. J., & Smart, D. (2010). Enhancing treatment outcome of patients
at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy
quality assurance system. Journal of Consulting & Clinical Psychology, 78, 298-311.
_____________________________________________________________________________________________________________
Exercise 32: REVIEWING INTERVENTION PLANS _________________________________________________________________________
OVERVIEW
This exercise engages students in reviewing criteria for selecting intervention plans. The
checklist included is relevant to all practice levels (e.g., individual, family, community, or
organization). Intervention plans succeed or falter depending on how soundly they are conceived.
This exercise takes a detailed look at criteria that should be considered when selecting plans such
as: How intrusive is the plan? Are cultural differences considered? How effective is it likely to
be?
PURPOSE
To encourage students to think critically about intervention plans.
MATERIALS REQUIRED
1. Checklist for Reviewing Intervention Plans (Exhibit 32).
2. An example of a plan.
TIME REQUIRED
Thirty minutes to apply the checklist and one hour for discussion. If students are asked to select
examples from the professional literature, more time will be required for library search.
SUGGESTIONS
Ask students to apply this checklist to one of their own clients, one that you give to students, or
to an example they locate in the literature. All students should have access to examples reviewed
so that they can decide for themselves which criteria apply. They should read case examples
beforehand to make the most productive use of class time. One member of the class could take
the lead in applying the criteria to an example. Students could work in teams of three to review
an example and apply the criteria and then present their views to the class. The class could
review the examples as they present their material.
1
_________________________________________________________________________
Exercise 33: ENCOURAGING A CULTURE OF THOUGHTFULNESS
_________________________________________________________________________
OVERVIEW
This exercise provides an opportunity for students to plan how to enhance a culture of
thoughtfulness (Perkins, 1992) in their work settings guided by a list of possibilities. Students
complete a questionnaire that allows them to evaluate the extent to which their work
environment encourages critical thinking, identify two barriers, and describe specific steps to
overcome them.
PURPOSE
To become familiar with indicators of a culture of thoughtfulness as a first step toward
identifying constructive changes to pursue.
MATERIALS REQUIRED
1. Culture of Thoughtfulness Scale (Exhibit 33).
2. Access to a work setting.
TIME REQUIRED
Fifteen minutes to complete the form, 30 to 40 minutes to discuss it.
SUGGESTIONS
There are several ways to proceed. You might ask all agency staff to fill out the form
anonymously; record the mean score for each item; then discuss three major strengths (highest
mean score items) and three weak items (lowest mean score items). You could ask students to
complete the form individually to review the culture in their field work agency. You could
complete the form yourself in relation to your own work environment. Are there items on the
questionnaire that students believe should not be there? What are their reasons? Can they make
a good argument that the characteristic would not contribute to a culture of thoughtfulness? Are
there items they would add? Can they make a good argument for this? You could have students
who work in the same or in similar agencies compare their ratings. Reasons for differences can
then be explored.
Ask students to select one change they would like to pursue in their agency and help them to
identify this clearly and to design a plan for pursuing it. Encourage students to draw on their
knowledge and skills related to organizations in planning how to encourage a culture of
thoughtfulness. For example, the better students understand the formal and informal incentive
2
systems and communication channels in an organization (e.g., the better their "contingency
analysis" skills for identifying what antecedents and consequences support or punish behaviors
of interest), the more likely they are to accurately identify leverage points for improvement.
At this point there are no norms for this instrument; so it cannot be used for comparative
purposes.
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
1. Which three items (highest scores) are your agency's greatest?
Suggestion: Focus first on strengths. This demonstrates intellectual charity (looking for the
best) and fairmindedness. Ask for specific examples of items noted (e.g., behaviors and
events) that demonstrate why an item was selected.
2. Which three items (lowest scores) are your workplace's greatest weaknesses?
Suggestion: Emphasize that the purpose here is not to expose errors and to punish the
guilty, but to examine how a culture of critical inquiry could be enhanced. Ask for specific
examples (behaviors, events) that demonstrate why given items on the questionnaire were
marked low.
3. What is the usual fallacy rate in case conferences?
Suggestion: Review fallacies detected and discuss how they could be avoided.
4. Suggestion: Here, too, you could review data collected and discuss
how to avoid fallacies.
1
_____________________________________________________________________________
Exercise 34: EVALUATING THE TEACHING OF CRITICAL-THINKING SKILLS
______________________________________________________________________________
OVERVIEW
This exercise highlights characteristics of a learning environment in which critical thinking is
modeled and supported. Students (and/or instructors) can use the Teaching Evaluation Form in
the exercise to rate the extent to which an instructor encourages critical thinking. Relevant
behaviors include encouraging questions and providing alternative views on controversial issues.
PURPOSE
To help students and instructors to evaluate the extent to which their classroom environments
encourage critical thinking.
MATERIALS REQUIRED
Teaching Evaluation Form (Exhibit 34).
TIME REQUIRED
Ten minutes to complete the form, 30 minutes to discuss item content.
SUGGESTIONS
An item-by-item review will offer the most useful feedback on particular items. For example,
you could collect all questionnaires and calculate the mean scores for each item as well as an
overall score. Students must be assured of confidentiality so they will be free to express their
opinions (e.g., not put their names on the questionnaire). These mean scores (on each item and
the overall score) can serve as discussion topics. The characteristics noted on this questionnaire
that model and support critical thinking may depart radically from common practices. We
showed an earlier version to a group of university instructors and urged that it be used as a way
to evaluate their performance. The group found only eight items acceptable as criteria to rate
their own performance. They rejected the instrument because: "We don't teach that way." (For a
valuable source on teaching see Brookfield, 1990.)
This exercise encourages both students and instructors to think about the consequences-
especially to clients-of teaching "that way."
2
POSSIBLE ANSWERS TO FOLLOW-UP QUESTIONS
1. Which item seems most important as a characteristic for an instructor who teaches critical
thinking?
Suggestion: We have no norms for this measure; so we suggest that you discuss the content
of the item and specific reasons for ratings given. Have each student identify an important
item and explain their choice by stating why the item identifies a trait of a critical thinker.
Students could also discuss obstacles to enhancing an atmosphere of critical inquiry in the
classroom and how to overcome them.
2. Which item seems least important as a characteristic for an instructor who teaches critical
thinking?
Suggestion: Have each student identify an item and compare and discuss them. Is this item
really unimportant to teaching critical thinking?
____________________________________________________________________________
Exercise 35: FORMING A JOURNAL CLUB
_____________________________________________________________________________
OVERVIEW
This exercise describes the purpose and helpful requirements of a Journal Club including the fun
one can have.
PURPOSE
To help readers set up a successful Journal Club that contributes to the effectiveness of their
services.
MATERIALS REQUIRED
1. Practice Exercise 35.
2. Access to Exercise 35 in text. (Preparing CATS.)
3. Access to relevant databases.
TIME REQUIRED
Depends how often the group meets and what tasks they take on.
SUGGESTIONS
Make this fun as well as useful. Seek out valuable resources described in the literature about
how to make journal clubs interesting and successful. Increasing students’ skills and motivation
in this area will be important for life-long learning.
REVIEWING STUDENTS’ WORK
What characteristics of successful journal clubs did students incorporate? Which ones did they
neglect? How much did they enjoy the group meetings? Did everyone participate? Were CATS
valuable in making decisions?
______________________________________________________________________________
Exercise 36: ENCOURAGING CONTINUED SELF DEVELOPMENT REGARDING
THE PROCESS OF EVIDENCE-INFORMED PRACTICE
______________________________________________________________________________
OVERVIEW
Effective professional practice requires keeping up-to-date with new developments that may
affect practice and policy decisions.
PURPOSE
To highlight the importance of this and to give readers some guidelines for self development in
relation to the process of evidence-informed practice.
MATERIALS REQUIRED
1. Box 36.1.
2. Practice Exercise 36.
3. Access to relevant databases.
TIME REQUIRED
About one hour to complete Practice Exercise 36 and one hour to have a discussion.
SUGGESTIONS
Note the importance and interest value of life-long learning. Encourage students to draw on
literature concerning self-management (e. g., how to collect clear data regarding baseline levels
of key skills such as posing clear questions, how to increase desired behaviors and how to
display progress, for example on graphs, see Watson & Thorp, 2013).
REVIEWING STUDENTS’ WORK
What skills did students select for self-improvement? How accurate are they in assessing their
competency levels of given skills? Are their self-management plans for increasing skills
informed by literature on self-management (Watson & Tharp, 2015)?
____________________________________________________________________________
Exercise 37: INCREASING SELF AWARENESS OF PERSONAL OBSTACLES TO
CRITICAL THINKING
______________________________________________________________________________
OVERVIEW
There are great impediments to offering clients effective services. Professionals are not
necessarily trained to think critically about decisions they make and there are a number of
personal obstacles that may get in the way including authoritarian dispositions and inflated
beliefs about one's competencies (over confidence).
PURPOSE
To help students identify and decrease personal obstacles to evidence-informed decision making.
MATERIALS REQUIRED
1. Box 37.1
2. Practice Exercise 37.1
TIME REQUIRED
About one hour to complete Practice Exercise 37 and one or more hours to discuss particular
personal obstacles students mention.
SUGGESTIONS
Reviewing the kinds of personal obstacles may be a good place to start (e. g., motivational, poor
self-management skills). You would ask students to read material describing how common some
obstacles are, such as self-inflated assessment of competency (e. g., Dunning, Heath, & Suls,
2004, Psychological Science in the Public Interest, 5, 69-106).
REVIEWING STUDENTS’ WORK
What obstacles do students mention most often? Are there certain obstacles that are important to
review that no one talks about – taboos?