Applied & Preventive Psychology 6:205-209 (1997). Cambridge University Press. Printed in the USA. Copyright © 1997 AAAPP 0962-1849/97 $9.00 + .10

Intervention research: Integrating practice guidelines with dissemination strategies A rejoinder to Paul, Stuve, and Cross

PATRICK W. CORRIGAN AND STANLEY G. MCCRACKEN

University of Chicago Center for Psychiatric Rehabilitation

Abstract

Paul, Stuve, and Cross (this issue) misunderstood the importance of the token economy described by Morisse et al. (1996) because they evaluated it solely from a clinical research perspective. Intervention research is a broader paradigm that describes the development of effective interventions via a series of research activities including basic psychological research, clinical research, and dissemination research. Given the wisdom of intervention research, we address three points made by our colleagues: (a) mental health staff should not adapt empirically validated practice guidelines to meet the needs of their program; (b) total-quality-management strategies like Interactive Staff Training should not drive dissemination efforts because they have little empirical support; and (c) user-friendly programs are inappropriate and unethical because they ignore consumer interests.

Key Words: Practice guidelines, Dissemination strategies, Intervention research, Token economy

If You Build It Will They Come?

Paul and Lentz 's (1977) social-learning program (SLP) is one of many empirically validated practice guidelines for programs that serve persons with severe mental illness. The SLP, and clinical research programs like it, typically do not include knowledge-dissemination strategies that are needed to transpose these practice guidelines for other settings. The distinction between practice guidelines and dissemination strategies has been described in articles on a variety of topics including knowledge dissemination and usage (Martinez-Brawley, 1995; Rich, 1991), technology transfer (Backer, 1991), adaptive expertise (Smith, Ford, & Koz- lowski, 1997) and intervention research (Rothman, 1989). We briefly summarize the components of one of these dis- semination models--intervention research--so readers can better appreciate the value of the token economy at Elgin

This article was made possible in part by a grant from the U.S. Depart- ment of Education and the Illinois Department of Mental Health and De- velopmental Disabilities. We thank Andrew Garman, Daniel Giffort, Paul Holmes, and Joseph Mehr for helpful comments.

Send correspondence and reprint requests to: Patrick W. Corrigan, University of Chicago Center for Psychiatric Rehabilitation, 7230 Arbor Drive, Tinley Park, IL 60477. E-mail may be sent to pcorriga@ mcis.bsd.uchicago.edu.

205

State Hospital (Morisse, Batra, Hess, Silverman, & Corri- gan, 1996). We then address three points made by Paul, Stuve, and Cross (this issue): (a) Mental health staff should not tinker with, or otherwise adapt, empirically validated practice guidelines; (b) total quality management strategies like Interactive Staff Training have no empirical support and should not, therefore, be touted as dissemination strate- gies; (c) addressing staff requests for user-friendly programs ignores the consumer's interest and, therefore, is inappropri- ate and unethical.

The Range of Research Activities Defined by Intervention Research

Several research methodologies are needed to provide a complete picture of the effects of a specific intervention in the real world. These methodologies describe the following three sequential stages (Thomas & Rothman, 1994).

. Basic psychological research: This stage addresses fundamental questions about human behavior such as "How do reinforcers help persons learn?" or "What is the relationship between areas of the brain and cognitive functions?" Psychopathologists frequently adopt these principles to describe and understand the

206 Corrigan and McCracken

various syndromes of persons with psychiatric dis- ability.

2. Clinical research: This stage describes the activity of applied scientists who transpose basic principles of human behavior into rules for planned change. For example, fundamental principles of operant condi- tioning led to the development of the token economy that targeted deficits in social and instrumental func- tioning (Ayllon & Azrin, 1968).

3. Dissemination research: Investigations in this stage address questions of user-friendly technology. For example, how can we package relatively esoteric in- terventions, developed in resource-rich academic set- tings, so line-level clinicians in settings with limited resources can implement them with ease?

Where do empirically validated practice guidelines fit in this schema? These are the products of clinical researchers. Clinical investigators need to remember that clinical re- search does not occur separately from the other enterprises of intervention research.

The rules and values that describe good methodology vary across intervention-research enterprises (Thomas & Rothman, 1994). Research that addresses questions of basic psychology and clinical practice is dominated by internal validity issues, whereas dissemination investigators concern themselves with external validity. Dissemination re- searchers are especially interested in the generalizability of an intervention; is Treatment A relevant for a large number of consumers suffering a particular problem and for the professionals charged with providing care for these consum- ers? In addition, dissemination research is concerned with the usability of an intervention. Effective interventions are valuable when they are user friendly to the consumers and clinicians who must use them.

Because internal and external validity are inversely re- lated, dissemination research focusing on external validity is often judged harshly by clinical investigators. This judg- ment neither promotes healthy doses of research on dissem- ination strategies nor encourages discussion of alternative research designs that might address dissemination questions (Rooney, 1994).

Intervention Research and the Social-Learning Program

We now address three points made by Paul, Stuve, and Cross (this issue) in light of the wisdom of intervention research.

Do Not Tinker with Practice Guidelines Paul and his colleagues (this issue) acknowledge the

range of research that distinguishes clinical investigations from dissemination studies. "Agencies funding research on clinical interventions have popularized distinctions between

efficacy, emphasizing internal validity in controlled 'set- tings,' and effectiveness, emphasizing external validity in 'real-world' situations" [italics in original] (Paul et al., this issue, p. 197). Paul, Stuve, and Cross seem to discount this distinction by arguing that the single study described by Paul and Lentz (1977) addresses all clinical and dissemina- tion issues. They conclude, therefore, that SLP is sufficient to address all issues needed to transfer technologies to other settings.

We agree that Paul and Lentz's (1977) study on the SLP is an excellent example of clinical research. No one research project stands alone in addressing the questions posed by intervention research, however. Clinical research only makes sense when considered as a hypothesis generated by basic psychological research. The 1977 study on SLP is significant because it tested hypotheses generated by oper- ant psychology. Clinical research is important only when it is replicated; the SLP study replicated earlier clinical studies on the token economy (Atthowe & Krasner, 1968; Ayllon & Azrin, 1968).

Most clinical research does not adequately address ques- tions about generalizability and usability. Morisse and her colleagues (1996) from Elgin listed the exigencies of their hospital that required changing the SLP to meet the needs and resources of their setting. For example, Morisse and colleagues believe that the staff demands of the SLP exceed available full-time equivalents (FTE) at Elgin. Paul, Stuve, and Cross replied that existing staffing patterns were suffi- cient to operate the SLP. Their reply assumes that running the SLP is the only responsibility of Elgin staff, however. Paul and colleagues said that administrators need to free up five FTEs from other responsibilities to carry out the obser- vational strategies that support the SLP on two units. This is 20 FTEs for an eight-unit hospital! Moreover, staff need to devote several months to intensive training before starting clinical care. Most mental health administrators are unable to reroute valuable human resources so easily.

Paul and his colleagues (this issue) may have used the values of their 1977 study to make these assertions about staffing and other issues. The nature of inpatient care has changed tremendously since then; for example, the overall number of inpatients and their length of stay has dropped during this time (Manderscheid & Sonnenschein, 1990). The 1977 study did not foresee the development of a new generation of antipsychotic medications (Meltzer, 1995) and assertive community treatment (Test, 1992), which have, along with other forces, drastically changed the venue for services from hospital to community. The 1977 study did not incorporate family members as essential players in the service of relatives with severe mental illness (Mueser & Glynn, 1995). The 1977 study did not foresee the expanding role of consumers and empowerment in the provision of services (Corrigan & Penn, in press).

These insufficiencies are not the fault of a study con- ducted in the 1970s. They are the realities that a consultant

Rejoinder to Paul et al. 207

of the 1990s must consider. It is the findings of dissemina- tion research that informs consultants how to integrate clini- cal research with contemporary realities.

Where Is the Data Supporting Interactive Staff Training? Morisse et al.'s article (1996) described the use of Inter-

active Staff Training (IST), a knowledge-dissemination pro- gram. The steps of IST include identifying staff priorities for program development, selecting a program committee and champion to spearhead development decisions, setting up a frame within which program decisions are made, facili- tating a pilot of the newly designed program, and setting up a quality assurance program to evaluate the impact of the program (Corrigan & McCracken, 1995a, 1995b, 1997). Paul, Stuve, and Cross argue that our assertions about IST are premature and have little respectable data supporting them. Their critique is once again based on the values of clinical research and ignores the broader agenda of interven- tion research.

The worth of a dissemination strategy does not lie with research on that strategy alone. Broader research programs that support heuristics of the dissemination strategy must also be described. IST developed out of research on partici- pative decision making and total quality management (Cor- rigan & McCracken, 1995a). Participative decision making and total quality management have been extensively exam- ined by organizational researchers who study the worlds of business and the military (Hackman & Wagerman, 1995; Spector, 1986). Mental health investigators with interest in dissemination strategies have extrapolated this research by examining the effect of participative decision making and total quality management on program development and con- sumer outcome (Hunter & Love, 1996; Magura et al., 1988; Sluyter & Barnette, 1995). This research has provided a broad set of evaluation indicators that address questions in which clinical researchers are typically not interested. Are staff members satisfied by the dissemination procedure or treatment innovation? Do staff members learn new behav- iors and concepts from the dissemination procedure? Does learning new behaviors and concepts change staff attitudes about work or their work behavior? Do changes in work attitudes and behavior lead to improved consumer outcome?

IST does not rest on prior research alone. The model that supports IST has been tested in several studies. Results have shown that staff identified lack of teamwork, institutional red tape, and burnout as barriers to setting up treatment programs using practice guidelines (Corrigan, Holmes, & Luchins, 1995; Corrigan, Holmes, Luchins, Buican, et al., 1994; Corrigan, Kwartarini, & Pramana, 1992; Corrigan, McCracken, Kommana, Edwards, & Simpatico, 1996; Gar- man et al., in press). Subsequent studies have examined components of IST. Research using a dismantling design showed that staff reliably identify their priorities in program development (Corrigan, Holmes, Luchins, Parks, et al., 1994). Research showed staff agree about who would make

a good champion for program development (Corrigan, Holmes, & Luchins, 1993). Two studies have looked at the effect of IST on team and staff. One study showed signifi- cant reductions in staff burnout and increments in attitudes about program development (Corrigan, McCracken, Ed- wards, Kommana, & Simpatico, in press). A second study showed that staff behavior related to the treatment program improved significantly, as did consumer response to that program (Corrigan, Holmes, Luchins, Basit, et al., 1995). Like most interventions, IST is a work in progress, a work with a good base of direct empirical support.

User-Friendly Programs Are Unethical Paul, Stuve, and Cross (this issue) challenge the value of

questions like: "Are staff members satisfied with the dis- semination procedure?" or "If staff learn new behaviors and concepts, does this change staff attitudes about work or their work behavior?" Paul et al. argued that concerns about user- friendly interventions are inappropriate because they dis- count the importance of consumer participation in program development. Their critique presents a limited description of IST and dissemination strategies like total quality man- agement.

Staff satisfaction within a treatment program should not be dismissed as irrelevant self-concern. Staff satisfaction is an essential variable for setting up and maintaining a treat- ment innovation. Staff who do not experience an innovation as beneficial are likely to devalue the corresponding pro- gram development; programs in this kind of ambience soon die (Corrigan et al., 1992; Corrigan et al., 1996).

Recognizing that an innovation needs to be user friendly does not mean staff exclude the consumer's viewpoint. In fact, consumer input is fundamental to total quality manage- ment (Deming, 1986; Sluyter & Mukherjee, 1993) and in- teractive staff training (Corrigan & McCracken, 1995b, 1997). Staff members and consumers alike will be frustrated if a newly developed program does not serve the needs of the clientele for whom it was designed. Treatment teams participating in IST are encouraged to make consumers ac- tive members of the committee that decides which innova- tions to integrate into their current treatment program (Cor- rigan & McCracken, 1997).

Summary

The fundamental disagreement between our two positions might be reduced to a single question: Should treatment teams be encouraged to adapt empirically validated practice guidelines to fit their perceptions of consumer needs and program restrictions? Paul, Stuve, and Cross state a re- sounding "No !" Any change in empirically validated prac- tice guidelines render them ineffective.

We believe the wrong question has been asked. Do not ask whether treatment teams should adapt practice guide- lines; treatment teams do. Staff in state hospitals, day

208 Corrigan and McCracken

treatment programs, psychosocial clubhouses, supported employment programs, and assertive community treatment teams are not passive objects. They have attitudes about effective intervention and the barriers to these interventions. If we are to have any effect on programs being developed by these staff, then we must not ignore their opinions. Dissem- ination research provides teams with ways to shape these innovations into user-friendly technologies while retaining much of the innovation's potency. Subsequent quality as-

surance programs conducted by the team test the effect of the program developed by that team.

These facts put the work of Morisse and her colleagues at Elgin in a different light. We salute them for developing an incentive program that has fared well under subsequent pro- gram evaluation. We also acknowledge the editors of Ap- plied and Preventive Psychology for providing a venue for discussing the tensions between practice guidelines and dis- semination strategies.

REFERENCES

Atthowe, J. M., & Krasner, L. (1968). Preliminary report on the application of token reinforcement procedures (token economy) on a "chronic" psychiatric ward. Journal of Abnormal Psychol- ogy, 73, 37-43.

Ayllon, T., & Azrin, N. (1968). The token economy: A motivation system for therapy and rehabilitation. New York: Appleton- Century Crofts.

Backer, T. E. (1991). Knowledge utilization: The third wave. Knowledge: Creation, Diffusion, Utilization, 12, 225-240.

Corrigan, P. W., Holmes, E. P., & Luchins, D. (1993). Identifying staff advocates of behavioral treatment innovations in state psy- chiatric hospitals. Journal of Behavior Therapy and Experimen- tal Psychiatry, 24, 219-225.

Corrigan, P. W., Holmes, E. P., & Luchins, D. (1995). Burnout and collegial support in state psychiatric hospital staff. Journal of Clinical Psychology, 51, 703-710.

Corrigan, P. W., Holmes, E. P., Luchins, D., Basit, A., DeLaney, E., Gleason, W., Buican, B., & McCracken, S. (1995). The effects of interactive staff training on staff programming and patient aggression in a psychiatric inpatient unit. Behavioral Interventions, 10, 17-32.

Corrigan, P. W., Holmes, E. P., Luchins, D., Buican, B., Basit, A., & Parks, J. J. (1994). Staff burnout in psychiatric hospitals: A cross-lagged panel design. Journal of Organizational Behavior, 15, 65-74.

Corrigan, P. W., Holmes, E. P., Luchins, D., Parks, H., DeLaney, E., & Kayton-Weinberg, D. (1994). Setting up inpatient behav- ioral treatment programs: The staff needs assessment. Behav- ioral Interventions, 9, 1-12.

Corrigan, P. W., Kwartarini, W. Y., & Pramana, W. (1992). Bar- riers to the implementation of behavior therapy. Behavior Mod- ification, 16, 132-144.

Corrigan, P. W., & McCracken, S. G. (1995a). Psychiatric reha- bilitation and staff development: educational and organizational models. Clinical Psychology Review, 15, 699-719.

Conigan, P. W., & McCracken, S. G. (1995b). Refocusing the training of psychiatric rehabilitation staff. Psychiatric Services, 46, 1172-1177.

Corrigan, P. W., & McCracken, S. G. (1997). Interactive staff training: Rehabilitation teams that Work. New York: Plenum Press.

Corrigan, P. W., McCracken, S. G., Kommana, S., Edwards, M., & Simpatico, T. (1996). Staff perceptions about barriers to inno- vative behavioral rehabilitation programs. Cognitive Therapy and Research, 20, 541-551.

Corrigan, P. W., McCracken, S. G, Edwards, M., Kommana, S., &

Simpatico, T. (in press). Changing staff attitudes about behav- ioral rehabilitation programs. Psychiatric Services.

Corrigan, P. W., & Penn, D. L. (in press). Disease and Discrimina- tion: Two paradigms that describe severe mental illness. Jour- nal of Mental Health.

Deming, W. E. (1986). Out of the crisis. Cambridge: MIT Center for Advanced Engineering Study.

Garman, A. N., Corrigan, P. W., Norris, R., Bachand, S., Lam, C., & McCracken, S. (in press). How mental health and develop- mental disabilities staff prioritize training and development needs. Community Mental Health Journal.

Hackman, J. R., & Wagerman, R. (1995). Total quality manage- ment: Empirical, conceptual, and practical issues. Administra- tive Science Quarterly, 40, 309-342.

Hunter, M. E., & Love, C. C. (1996). Total quality management and the reduction of inpatient violence and costs in a forensic psychiatric hospital. Psychiatric Services, 47, 751-754.

Magura, S., Goldsmith, D. S., Casriel, C., Lipton, D. S., Goldstein, P. J., Spunt, B. J., & Strng, D. L. (1988). Patient-staff gover- nance in methadone maintenance treatment: A study in partici- pative decision making. International Journal of the Addic- tions, 23, 253-278.

Manderscheid, R. W., Sonnenschein, M. A. (1990). Mental Health, United States (DHHS Publication No. ADM 90-1708). Wash- ington, DC: U.S. Government Printing Office.

Martinez-Brawley, E. E. (1995). Knowledge diffusion and transfer of technology: Conceptual premises and concrete steps for hu- man services innovators. Social Work, 40, 670-682.

Meltzer, H. Y. (1995). Atypical antipsychotic drug therapy for treatment-resistant schizophrenia. In S. R. Hirsch & D. R. Weinberger (Eds.), Schizophrenia (pp.485-502). Oxford: Blackwell.

Morisse, D., Batra, L., Hess, L., Silverman, R., & Corrigan, P. (1996). A demonstration of a token economy for the real world. Applied & Preventive Psychology, 5, 41-46.

Mueser, K. T., & Glynn, S. M. (Eds.). (1995). Behavioral family therapy for psychiatric disorders. Boston: Allyn and Bacon.

Paul, G. L., & Lentz, R. J. (1977). Psychosocial treatment of chronic mental patients: Milieu versus Social-Learning Pro- grams. Cambridge: Harvard University Press.

Rich, R. F. (1991). Knowledge creation, diffusion, and utilization. Knowledge: Creation, Diffusion, Utilization, 12, 319-337.

Rooney, R. H. (1994). Disseminating intervention research in aca- demic settings--A view from social work. In J. Rothman & E. J. Thomas (Eds.), Intervention research: Design and develop- ment for human service (pp. 353-368). New York: Haworth.

Rejoinder to Paul et al. 209

Rothman, J. (1989). Intervention research: Application to runaway and homeless youths. Social Work Research and Abstracts, 25, 13-18.

S luyter, G. V., & B arnette, J. E. (1995). Application of total quality management to mental health: A benchmark case study. Jour- nal of Mental Health Administration, 22, 278-285.

Sluyter, G. V., Mukherjee, A. K. (1993). Total quality manage- ment for mental health services: a paradigm for the '90s. An- nandale, VA: National Association of Private Residential Re- sources.

Spector, P. E. (1986). Perceived control by employees: A recta- analysis of studies concerning autonomy and participation at work. Human Relations, 39, 1005-1016.

Test, M. A. (1992). Training in community living. In R. P. Liber- man (Ed.), Handbook of psychiatric rehabilitation. New York: Macmillan.

Thomas E. J., & Rothman, J. (1994). Intervention research: Design and development for human service. New York: Ha- worth.