Why Should You Take a Clinical Research Course?

Audiovisual set-up slide:

If you are hearing my voice now, know how to advance slides, and how to end the slide show, you can skip this slide.

To hear my voice and to see the slides best, you need to view the slides using Power Point’s “view slide show” setting. When in Power Point, go to “view” on the top bar of the power point display. Select “slide show” so the slide occupies your entire screen.

To advance slides, double click the down arrow key on your key board (not the arrow on the “2” key). You can see and hear the preceding slide by pressing the up arrow key. The slides are NOT set to advance automatically so you can spend as long on each as you wish.

To leave the slide show, move your cursor around slowly near the lower left edge of the screen until a small, translucent box appears on the slide. Click it for options including “end show”.

If you are in the full screen “view slide show” and your computer’s sound is on, you should be hearing my voice right now. If you can’t hear me, something is wrong with your sound system.

I’m Rich Sherman.

I have been president of AAPB and have been involved in biofeedback for over 30 years so am familiar with many of the problems which happen when people try to perform biofeedback without an adequate background. I’m going to tell you why you need to know just a bit about research and the placebo response to perform biofeedback effectively.

Why should you go to all the time and effort to take a course in Clinical Research??

Richard A. Sherman, Ph.D.

Dean, College of Psychophysiology, University of Natural Medicine.

rsherman@nwinet.com

(This slide appears in all 4 basic skills talks so skip it if you already read it.)

You are attending one of four brief talks in the series on “what you need to know to perform biofeedback effectively”.

The series contains talks on why:

1. You need to know how biofeedback devices actually record physiology and how to set the devices to produce useful recordings (Biofeedback Instrumentation).

2. You need to know how the body works in order to use biofeedback effectively to change physical and mental functioning. Otherwise, you may try to do training which can’t work well or can’t be done at all (A&P / Human Biology).

3. You need to be able to tell if there is enough solid evidence supporting the use of a technique for you to even try it, let alone charge for it, and to tell whether the results you see or hear about are due to placebo or treatment effects (Basics of clinical research).

4. Everybody needs a basic biofeedback course before doing biofeedback. You need to know when an introductory level general biofeedback or neurofeedback course doesn’t provide the depth you need to effectively incorporate biofeedback into interventions for the kinds of disorders you treat (e.g. chronic pain, incontinence, muscle rehabilitation, etc.).

(This slide appears in all 4 basic skills talks so skip it if you already read it.)

Before we get started, let’s be sure you and I have the same general concept of what biofeedback is.

Biofeedback includes (a) performing accurate psychophysiological recordings using standard techniques to evaluate known problems with levels and patterns of function of one or more known physiological systems and then (b) immediately showing these levels and patterns to a client as they are recorded so he or she can learn to recognize and control them. Numerous placebo controlled studies with long term follow-ups have proven that biofeedback is effective for many disorders.

The crucial points here are (1) that biofeedback includes actual learning by the client and (2) that widely recognized problems in function are corrected.

Biofeedback devices are NOT magic machines recording unverified physical problems using methods not explainable through normal scientific means. They do not magically cause the body to change its functioning through unproven methods to fix problems which can not be demonstrated through standard tests. No scientifically acceptable studies have shown devices making such claims to be effective.

If you are not familiar with psychophysiological recording and biofeedback, please see the introductory lecture in the “Want to know what biofeedback and psychophysiology are?” section on this site.

CredibilityHow do you know that the technique you were taught, watched, heard about, etc. actually works?

You need to know how to evaluate clinical research.

Is the common use of a technique an indication of its efficacy?

No! Increasing numbers of popular techniques are being shown to be ineffective.

If a manufacturer says it works – does that mean it does?

(Doesn’t the FDA stop anybody from selling a device

that doesn’t work?) No and NO!!!

If the big person on campus uses it and pushes it, does that mean it works or just adds to its credibility? Just adds credibility!

Does the FDA control quality of behavioral interventions? NO!

Keeping an open mind

The weight of evidence

You need to be able to balance all the factors to make your own decision about whether a technique is efficacious or even

worthy of a trial

Many of the techniques currently in use will change over the next few years. More studies supporting or detracting the use of a myriad of techniques will be published.

Dozens of new techniques for assessment and control will be promulgated by your colleagues, clinicians in other professions, patients, and “superiors”.

You need to know how to read the clinical literature defensively in order to separate the few gold nuggets from the mass of useless, misleading chaff.

Thus, you need to be able to recognize a good study when you see one and - at least as important - recognize the crucial flaws in a clinical study.

You need to have sufficient skills in searching the literature and assessing quality to tell when it is worth trying a technique and when it is worth discouraging the use of others.

Do you actually know how effective you are as a clinician?Droves of studies conducted by neutral observers, including several performed by my group, have shown that nearly all clinicians from every field think they are far more effective than they actually are. It is human nature to remember the successes and not realize that patients who don’t come back are frequently failures even through they tell you that they are better as they walk out your door.

My group has the advantage of not only being able to conduct large surveys but also of having longitudinal records of virtually all of the patients seen in our medical system (the US military) over decades. Thus, we can easily trace the progress of large numbers of patients having various disorders through numerous treatments.

Some years ago, we audited a biofeedback group who had spent nearly a decade treating a variety of patients with medical conditions. They felt that they were very successful in treating patients with medical problems such as headache. However, our audit showed that few of their patients showed improvement with biofeedback beyond the placebo level and that any changes were usually brief.

Biofeedback has been proven effective for many of the conditions they were treating. They simply didn’t know how and didn’t recognize their own limitations.

Lack of CredibilityA huge problem for acceptance of biofeedback is the prevalence of unsubstantiated (unproven, unvalidated) interventions incorporating an incredible variety of techniques which are talked about, used, and charged for as though there was a reasonable body of clinical evidence supporting their efficacy.

The combination of (a) there not being many properly designed studies with adequate numbers of subjects and sufficiently long follow-ups to be convincing and (b) the prevalence of unsubstantiated techniques which are being charged for has led many health care providers to be very wary of biofeedback.

In this time when previously accepted medical and surgical techniques are falling by the way-side in droves because good studies and patient audits are proving their ineffectiveness, the lack of credibility truly harms the acceptance of biofeedback.

Key elements of a credible clinical study / publication:

1. Adequate diagnosis of the subjects

2. Adequate pre-treatment baseline to establish symptom variability.

3. Objective outcome measures relevant to the disorder.

4. Intensity of the intervention sufficient to produce an effect.

5. Way to check whether the intervention was successful (drug taken properly, behavioral technique successfully learned and then used).

6. Sufficient patient-subjects so result is credible.

7. Appropriate design for the question (e.g. single group, controls, believable placebo, etc.).

8. Sufficient descriptive statistics so results are clear.

9. Long enough follow-up so duration of results can be established.

Many organizations such as the American Psychological Association & AAPB have adopted requirements such as the following for determining that a treatment has been shown to be efficacious:

a. Two studies with appropriate design and controls (group design or a series of single case design studies).

b. Studies conducted by different researchers. c. Studies demonstrate clinical efficacy. The new

treatment must be shown to be efficacious in comparison with medication, a placebo, or another treatment. The treatment must be shown to be equally effective to an established treatment for the same problem.

d. Waiting list controls are not sufficient for demonstrating efficacy. e. The diagnostic criteria must be clearly specified.

Criteria for accepting a new technique as efficacious:

(No expectation effect.)

Fatal Flaws in conducting research

Fatal flaws happen when the investigators don’t know enough about (1) the disorder they are working with, (2) their recording techniques, (3) assessing outcome measures to ask the right question, and (4) the basic elements of research design!

You need to look for these fatal flaws when you read a study or hear about an idea. If you are going to incorporate a technique you read or hear about into your practice, you need to know enough about the above factors to evaluate the idea and to apply it.

A common fatal design flaw is - Failure to anchor the study outcome measures to the

population having the problem being investigated

A good example is an early study on behavioral treatment of cancer.The design compared a group therapy intervention to the records of similar patients. The outcome measure was years of survival with significance determined by the difference in average number of years of survival of each group.

The investigators reported that the group receiving group therapy survived significantly longer than the control group and concluded that group therapy was probably the reason for the longer survival.

The investigators did not compare the survival rates of their tiny groups to the huge data base of similar cancer patients starting at the same stages of the same type of cancer. In fact, (a) their groups were so small that known variability in survival would have made it very likely that one group would have an average survival time far longer than the other just by chance and (b) their failure to review well known life table data on survival times caused them to miss the crucial point that the apparently longer survival of several participants was not out of line with the population. Sadly, the controls died earlier than would be expected.

Numerous studies have now shown behavioral interventions to be useless for cancer survival.

Failure to analyze the data correctly because the investigators don’t understand the outcome measures or human variability!

For example, typical ESP studies have participants try to guess which one of five possible shapes – such as those on the cards shown below – a sender is thinking about. Each shape is on a card which can’t be identified from behind and there are usually 25 cards in the deck. Thus, the participant has a one in five chance of guessing a card correctly. If there is no ESP, the participant would be expected to randomly guess five of the 25 cards.

If somebody guesses far more or less than 5 they may be declared to have ESP.

Regression to the mean tells us that a greater or lesser number occurs by chance if sufficient people are tested and that the person guessing unusually more or less than chance is likely to guess at about chance level at the next test. Unfortunately for proponents of ESP, this is just what happens nearly all the time.

Know the tools and equipment – the recording methodology!

a. Use of the Minnesota Multiphasic Personality Inventory (MMPI) to evaluate low back pain.

Ludicrous use pointed out by an orthopedic surgeon who read the test! Many questions relate to actual pain symptoms! b. "Alpha" EEG for anxiety: Alpha frequency waves look just like muscle tension from the eyes! The early biofeedback devices couldn’t tell the difference and neither could the early clinicians so all they did was teach people to increase eye muscle tension by rolling up their eyes. The treatments worked because (1) people learned to sit quietly in a dim room and (2) effective relaxation exercises were given as homework!

Both “EOG” and alpha waves look

like this.

Know the physiology and the disorder being worked with!

a. Lack of repeated measures in hypertension studies.

BP

Weekly Sessions

b. Finger temperature baseline instability(stressed college kids doing class demonstration gradually relaxing – blood flow and temp down with stress – up with relaxation!)

Habituation controls

Untreated controls

Treated patients

Temp

Time in minutes Stressed relaxed

1 8

The table shows the results of a clinical study which used biofeedback to treat a disorder. The authors sent only the left column of the table in with their paper for review and concluded that biofeedback did not work for the disorder. However, I was able to find sufficient data in the paper to construct the middle and right columns. The middle column shows the number of subjects who learned the skill vs. improvement while the right hand column shows the number of subjects who did not learn the skill vs. improvement. It is obvious that most subjects did not learn the skill but that most of those who did learn the skill showed clinical improvement while most of those who did not learn the skills did not show clinical improvement!

Original results of the study

Subjects who learned the skill

Subjects who did not learn the skill

Subjects who showed no improvement 30

6

24

Subjects who improved 21

18

3

Know the difference between learning a technique and changes in intensity of a clinical problem:

Non-specific / Placebo effectsThe bane of uncontrolled clinical studies!

or

why all the “new” treatments work.

New treatments are initially tested using the pretreatment baseline – intervention – post treatment baseline (A-B-A) design with no control group. So, changes due to natural fluctuations in the disorder’s intensity (as with acute low back pain) and non-specific / placebo effects are missed.

(30% placebo cure rate for headache)

Non-specific effects include patient-therapist bonding, placebo effects, changes with time, expectations, etc.

Follow-ups are usually too short to observe the placebo effects wearing off.

A good placebo control includes at least:

1. Treatment expectation effects.

2. Placebo effect from the belief that the treatment can work / is working.

3. Habituation to the treatment environment and sufficient duration to elucidate changes with time.

4. Good therapist-patient bonding with the therapist giving general support and expectation that the treatment will work.

PL

Rx

The Crucial need for Controlled Studies

Controls - especially realistic placebo controls - are critical because of (1) the placebo effect and (2) changes in problems with time.

Ex: Dohrman and Laskin (1976) conducted a study of 24 patients diagnosed as having jaw area pain related to muscle tension. Eight patients were in a placebo group and sixteen were given biofeedback.

Three quarters of the patients treated with biofeedback showed “significant improvement of clinical symptoms and required no further treatment”.

Sounds great right?

continued

Need for placebo controls - continued

Unfortunately for people who say that controls aren’t needed,

half of the controls had just as good a result!!

There was no long term follow-up so there was no opportunity to know whether the placebo effect wore off (it usually lasts six months or so) or whether the pain simply returned on its own.

I strongly recommend that readers unfamiliar with the great power of the placebo effect on pain read a review of this phenomenon (e.g. Elton et al 1983).

An especially good article is by Finniss & Bendetti (Pain 114, 3 – 6, 2005) which discusses mechanisms of the placebo response and their impact on clinical trials and clinical practice. A good article demonstrating the power of the placebo response is the one by Price et al (Pain 127, 63 – 72, 2007) which shows that placebo analgesia results in large reduction in pain-related brain activity in irritable bowel syndrome patients. This is one of many studies beginning to come out which show that changing how the brain processes pain changes pain perception regardless of the source of the pain or the method used to change the brain’s processing.Discover and Scientific American magazines have had several articles on brain scans showing that placebos stop pain by changing processing of the signals (e.g. Epstein in Discover, page 26, January 2006; Choi in Scientific American page 36 November 2005; and Ruvin in Discover April 2006).For more on the brain and pain, you may want to look at Nicoll and Alger’s article on the brain making its own pain relievers (Scientific American, December, 2004).

Why open studies don’t show that a technique is actually effective:

Single subject and single group designs are important to demonstrate that a change in important outcome measures such as pain intensity, ability to walk further, etc. takes place from beginning to end of the study period so it is worth continuing to a much more expensive, complex design.

Evaluation of a technique’s efficacy can’t stop at open studies because the change could just as easily be due to time alone or placebo effects.

This is why single group studies showing that interventions for low back pain such as (a) chiropractic manipulation and (b) low back surgery help low back pain patients have little to no credibility in the medical community.

As soon as a control for change with time is introduced, it turns out that subjects receiving chiropractic do no better that those not receiving any treatment. (Most of the comparison studies are “population change” based in which changes during a chiropractic study are compared with changes expected of the population.)

As soon as a comparison control with other treatments is included, it turns out that surgery for low back pain is no better than well designed, intense behavioral and strengthening programs. When both are compared with no intervention, their results are less than impressive.

Fatal flaws in typically tiny “controlled” studies

of behavioral interventions

(1) waiting list control groups (a) do not have expectation / non-specific / placebo effects and (b) the people on the waiting lists are frequently very different from those in the study. They have sometimes turned down participation in the behavioral study, are too poor to make the required trips, or are involved in other situations.

(2) sample sizes too small to adequately represent true patterns in the general population of people with the disorder are common. The typical 10 subject behavioral study is too small to either represent typical patterns in the population of sick people or to avoid looking good or bad because of unusual reactions by a few people very sensitive or insensitive to the treatment.

(3) follow-ups are generally, but not always, too short to observe placebo effects wearing off.

Why you should not trust the

“weight of clinical experience”

“If the technique is in use now, it must be efficacious” = bunk!

When the idea of washing hands before surgery came out, “everybody” knew it was dumb and certainly wouldn’t help. It was only accepted after a comparative controlled study counting surgical survival.

Virtually all of the techniques which were in use around WWI are gone. Everybody “knew” they worked and laughed at the new ideas. Nearly all of the drugs and techniques which everybody knew worked during the WWII era are gone.

How about now? The FDA’s insistence on proving a new drug to be efficacious is NEW and unique on the face of the planet. It doesn’t cover surgical or behavioral techniques unless they incorporate devices.

continued

Weight of evidence - continued

Many of today’s accepted techniques espoused by “experts” have never been subjected to controlled study so nobody knows how many are actually useless, doing harm, or are relatively worse than competing techniques.

At least some of the “alternative and complimentary” techniques of today will survive to be the standard techniques of tomorrow and we’ll all look back and laugh at the old techniques which we swear by today.

With a bit of luck, better education of clinicians, and enforcement of the law, more and techniques will have to prove themselves effective before they reach wide use.

Weight of evidence - continued

Ex: Continued use of biofeedback for stroke rehabilitation without concurrent training to rectify learned non-use.

At least two excellent controlled studies (Basmajian et al 1987, Inglis et al 1984) and a major review of controlled studies (de Pedro-Cuesta et al 1992) demonstrated that biofeedback does not add anything to standard physical therapy for stroke rehabilitation. One of the studies was done by a world renowned expert in rehabilitation medicine who is also the father of rehabilitation biofeedback (Basmajian), so it is very likely that he knew what he was doing.

Thus, “clinical experience” is not a guide to deciding whether a treatment is effective.

Credibility alertWhile you are listening to discussions of alternative techniques, please keep credibility in mind.

Positive signs of credibility to watch for during this discussion include:

(1) numerous articles by different authors supporting use of a

technique

(2) when many articles have good, realistic, placebo controls

(3) the articles are double blind – not single blind and the

evaluations are done by a neutral team

(4) the patients were randomly assigned to the alternative technique

(not people who showed up wanting it)

(5) the articles are published in mainstream journals with high

reputations rather than only in a journal supported by

practitioners of that technique

Well, I certainly hope I’ve provided some convincing support for the idea that people need a course in clinical research to do a good job at biofeedback – or at least given you enough hints so you can do your job better after attending this brief talk.

Just a note on typical clinical research courses: Most are designed for people who are going to be biologists or perform psychological studies. They don’t concentrate on how to recognize or perform good studies in the real biofeedback clinical environment without a ton of money and time. So, be careful to take a course relevant to your interests.

If you have questions – especially about the best ways to learn this material if you are going to be doing biofeedback rather than basic research, please feel free to contact me.

Thanks for listening and, especially, for keeping an open mind!

Rich Sherman

rsherman@nwinet.com