14
This article was downloaded by: [University of Bath] On: 06 October 2014, At: 19:17 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Psychotherapy Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/tpsr20 Cluster Analysis of Patient Reported Psychotherapy Outcomes Mary Connolly a & Hans Strupp b a Department of Psychiatry , University of Pennsylvania b Vanderbilt University Published online: 25 Nov 2010. To cite this article: Mary Connolly & Hans Strupp (1996) Cluster Analysis of Patient Reported Psychotherapy Outcomes, Psychotherapy Research, 6:1, 30-42, DOI: 10.1080/10503309612331331558 To link to this article: http://dx.doi.org/10.1080/10503309612331331558 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms- and-conditions

Cluster Analysis of Patient Reported Psychotherapy Outcomes

  • Upload
    hans

  • View
    215

  • Download
    0

Embed Size (px)

Citation preview

This article was downloaded by: [University of Bath]On: 06 October 2014, At: 19:17Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Psychotherapy ResearchPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tpsr20

Cluster Analysis of Patient ReportedPsychotherapy OutcomesMary Connolly a & Hans Strupp ba Department of Psychiatry , University of Pennsylvaniab Vanderbilt UniversityPublished online: 25 Nov 2010.

To cite this article: Mary Connolly & Hans Strupp (1996) Cluster Analysis of Patient ReportedPsychotherapy Outcomes, Psychotherapy Research, 6:1, 30-42, DOI: 10.1080/10503309612331331558

To link to this article: http://dx.doi.org/10.1080/10503309612331331558

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Psychotherapy Research 6(1> 30-42, 1996

CLUSTER ANALYSIS OF PATIENT REPORTED PSYCHOTHERAPY OUTCOMES Mary Beth Connolly Department of Psychiatry, University of Pennsylvania

Hans H. Strupp Vanderbilt University

The purpose of the present investigation was to explore the patient’s perspective of important psychotherapy outcomes. Eighty patients, following 25 sessions of dynamically oriented psychotherapy, were asked in writing to describe the most important changes they had received from psychotherapy. Ninety distinct changes were reported by this sample. A dissimilarity matrix of these changes was analyzed using hierarchical cluster analysis. The cluster analysis revealed four clusters: improved symptoms, improved self-understanding, im- proved self-confidence, and greater self-definition. Further, these four clusters formed two superclusters corresponding to improvements in symptoms and self-concept. The results of this investigation indicate that a complete outcome battery, designed to encompass all changes important to patients treated in dynamically oriented psychotherapy, should include not only symptom change measures, but also measures of change in self-concept.

Throughout the history of psychotherapy research, questions regarding how best to define psychotherapy outcome have remained fundamental. In 1973, Malan argued that outcome is in fact “the crucial variable in psychotherapy” since other therapeutic variables are only important to the extent that they predict the outcome of treatment. Lambert and Hill (1994) describe the measurement of psychotherapy outcome as shifting over time from a focus on general ratings of improvement by the clinician to assessments of specific symptoms from multiple perspectives.

Although many psychotherapy studies have largely focused on changes in symptomatology, Strupp and Hadley (1977) suggest many patients are entering treatment, not for the purpose of eliminating specific symptoms, but rather to find meaning in their lives. The authors emphasize that there is more involved in

The preparation of this article was supported in part by National Institute of Mental Health Grant R01-MH20369 to Hans H. Strupp and National Institute of Mental Health Grant R01-MH40472 to Paul

Correspondence regarding this article should be addressed to Mary Beth COMOIIY, University of Pennsylvania Medical Center, Center for Psychotherapy Research, 3600 Market Street, Room 770, Philadelphia, Pennsylvania 19104-2648.

Crits-Christoph.

30

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

CLUSTER ANALYSIS OF OUTCOME 31

the assessment of outcome than changes in a person’s feeling state and behaviors alone. In their tripartite model of psychotherapy outcome, the authors point out that there are unique values inherent in each perspective of outcome. Therefore, evaluating psychotherapy outcome from multiple perspectives should include more than simply having patients assess the domains of outcome that experts value. Efforts should be made to include domains of outcome that patients, families and society value.

In the tripartite model, Strupp and Hadley (1977) argue that a sense of well- being and happiness are the most highly valued outcomes from the patient’s perspective. Horowitz(l979)andHorowitzandVitkus(1986)reportthatthemost common problems bringing people into psychotherapy include psychological symptoms, cognitions about self, and interpersonal problems. Horowitz points out that an evaluation of therapy outcome that focuses on changes in symptom- atology alone would be disparate with patients’ presenting complaints and the focus of many treatments, including interpersonally oriented psychotherapies.

Few empirical investigations have focused on uncovering the domains of outcome considered to be important from the patient’s perspective. In an earlier investigation of patients’ retrospective views of their psychotherapies, Strupp, Fox, and Lessler (1969) compared patients’ presenting problems with those changes considered most important by the patient following treatment. In this study, pre- senting complaints included symptoms, such as loss of interest in day-to-day living, physicalsymptoms, generalizedanxiety, and depressive feelings, but also interper- sonal problems. However, following treatment, improvements in interpersonal functioning were reported most frequently as an important change.

In an examination of both clients’ and therapists’ views of psychotherapy, Llewelyn (1988) asked patients to describe the events in therapy which were most helpful. Patients reported that reassurance/relief events and problem solu- tion events were most helpful, whereas therapists reported insight events as the most helpful. The author concludes that therapists value the process of insight, while patients value the results of the insight process, namely problem solution. Llewelyn postulates that patients are simply unable to perceive the process. Therefore, clients may be less interested in self-knowledge than in problem solution.

Elliott and James (1989) reviewed the theoretical and research literature in order to evaluate the variety of therapeutic experiences reported by both patients and therapists. The authors report that patients and therapists report two types of therapeutic impacts: task/problem solving impacts and interpersonal/affective impacts. The most common tasbproblem solving impacts reported across the 21 sources reviewed include self-understanding, guidance, self-awareness, taking responsibility, insight into others, facing reality, and self-control. The most common interpersonal/affective impacts included expression/catharsis, reassur- ance/confidence, feeling understood, and instillation of hope. Further, the au- thors reviewed five studies which specifically asked clients to describe specific outcomes of treatment. The most common treatment related changes in these five studies included increased self-esteem, symptom relief, improved interper- sonal relationships, and greater mastery. Elliott and James (1989) point out that psychotherapy outcome researchers have attended primarily to assessments of symptom change at the expense of other outcomes important to clients.

Elliott, Clark, Kemeny, Wexler, Mack, and Brinkerhoff (1990) examined the changes identified by ten depressed clients following 16 sessions of experiential

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

32 CONNOUY AND STRUPP

therapy. The authors used a grounded theory analysis of the identified changes, by which four judges independently developed categories and then joined to decide on consensus categories that best represented the changes. The authors report three major domains of outcome, including: improvements in self, im- provements in dealing with others, and improvements in life situations.

The present investigation attempts to further uncover the criteria of change considered to be important from the perspective of patients following short-term dynamically oriented psychotherapy. Although the literature reviewed above suggests a variety of important outcome criteria, including changes in interper- sonal functioning, self-perception, and symptomatology, the current investiga- tion uses an exploratory approach to discover the important changes reported by patients.

The current investigation focuses only on the retrospective view of the patient. Although the perspective of the patient prior to treatment and during treatment is equally important, the retrospective view has the advantage of including an informed perspective. Prior to treatment, a patient might have only a vague idea about the possible benefits of treatment. Following treatment, patients are in a better position to reflect on what they found most important and these factors are more likely to be related to the patient’s satisfaction following treatment. Thus, the current project focuses only on the patient’s perspective of important therapeutic changes following treatment.

The primary research question examined was: What are the categories of change considered to be important from the perspective of patients following short term dynamically oriented psychotherapy?

METHOD

PATIENTS

The subjects for the present investigation participated in the Vanderbilt I1 study of the effectiveness of Time-Limited Dynamic Psychotherapy (Henry, Strupp, Schacht, Binder, & Butler, 1993). The patients accepted into the study were considered to be representative of those who seek treatment in mental health centers or private practices. All patients were solicited through advertise- ments in local newspapers. The final sample consisted of 80 subjects between the ages of 21 and 65 who met the criteria for at least one Axis I or Axis I1 disorder. 77.5 percent of subjects were female and 48.8 percent were married. Ninety-one percent of subjects received at least one Axis I diagnosis and 67 percent of subjects received an Axis I1 diagnosis. The most common Axis I diagnoses included major depression, dysthymic disorder, and generalized anxiety disorder.

THERAPISTS

The therapists for this investigation were eight clinical psychologists and eight psychiatrists with at least two years of post-doctoral or post-residency clinical experience, respectively. All therapists were private practitioners recom- mended by senior psychiatrists and psychologists from training programs at Vanderbilt University. The mean level of post-graduate experience was five years.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

CLUSTER ANALYSIS OF OUTCOME 33

Six therapists were female and ten were male. All therapists were between the ages of 27 and 48.

TREATMENT

The Vanderbilt Psychotherapy Research Project was designed to study the effectiveness of training in time-limited dynamic psychotherapy (TLDP; Strupp and Binder, 1984). The project consisted of three cohorts. During the first cohort, each of 16 therapists saw two patients in up to 25 weekly sessions. All patients received “therapy as usual.” The second cohort represented the training phase. The 16 therapists attended weekly training sessions in TLDP and supervision. Training sessions were conducted by the authors of the TLDP manual and their associates. During this time each therapist saw one patient for up to 25 sessions. These patients were considered the “training cases.” In the third cohort, each of the 16 therapists again saw two patients in time-limited dynamic therapy for up to 25 sessions. (For more details of the Vanderbilt psychotherapy project, refer to Henry et al., 1993.)

PROCEDURES

All patients, at the completion of treatment, were asked in writing to “Describe the most important changes you have experienced.” The authors converted each answer to a list of distinct changes by converting each essay answer to a list of phrases. The original wording of each phrase was maintained. Due to missing data and cases which reported no changes, a range of one to seven phrases was collected from each of 67 patients. On average, patients reported 2.83 changes. Across this sample, a final list of 90 distinct changes was derived. Responses were considered to overlap between patients only if the same, or nearly the same, wording was used. For example, “I had more confidence in myself” and “I felt more self-confident” were both represented by the item “improved self-confidence .”

In order to reduce the data into discrete categories, 100 undergraduate students at Vanderbilt University were asked to rate the pairwise semantic similarity of each of the 90 changes, on a seven-point Likert-type scale (scale and directions provided in Appendix A). Because there are 4,005 pairs of changes, the students were randomly separated into groups of ten. Each group of ten students rated a subset of 400 pairs. The mean age of the student raters was 19.6, ranging from 18 to 31. Sixty-four percent of the raters were female and only eight percent had received previous psychotherapy.

The purpose of this task was to obtain a reduced, more descriptive, set of changes from the patient’s perspective. It was assumed that the undergraduate raters were representative of the general population in their ability to rate the semantic similarity of these items. One might argue that semantic similarity may vary as a function of age and cultural background. Therefore, an additional rater group consisting of ten adults, matched in age and sex to the original patient sample, was obtained in order to assess the generalizability of the ratings of undergraduate students. The adult group rated the semantic similarity of only one subset of 400 pairs of changes.

At intake and treatment termination, all patients completed an extensive assessment battery. Self-report measures included the Global Outcome Rating

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

34 CONNOLLY AND STRUPP

(Strupp, 1980), the Symptom Check List-90 Revised (Derogatis, 1973, the Interpersonal Dependency Inventory Wirschfeld, Klerman, Gough, Barrett, Korchin, & Chodoff, 1977), the Vanderbilt Interpersonal Locus of Control Scale (Strupp, 1980), and the Structural Analysis of Social Behavior Intrex Question- naire (Benjamin, 1983).

DATA ANALYSIS

First, interrater reliability for each of the ten subsets of judges was computed using the intraclass correlation. In addition, the intraclass correlation for the matched adult judges was computed. The average ratings for this adult judge group were correlated with the average ratings of the student judges who rated the same subset of 400 items.

A similarity matrix was constructed from the average of the judges’ direct similarity estimates for each pair of items. The matrix of similarity estimates was converted to a dissimilarity matrix by subtracting each similarity value from 8, such that high values would reflect the distance between items. The resulting dissimilarity matrix was submitted to a nonoverlapping agglomerative hierarchi- cal cluster analysis (SPSSPC+, 1993).

Ward’s linkage method (Ward, 1963) was used in this phase of the investi- gation. There is much debate in the literature regarding which type of linkage provides the most accurate solution. Schreibler and Schneider (1 985) used Monte Carlo designs to compare various linkage methods for their ability to recover true structure under various error conditions. The authors report that Ward’s method was adequate in recovering true structure.

For this investigation it was necessary to determine the appropriate number of clusters to best class@ psychotherapy outcome changes. Aldenderfer and Blashfield (1984) have suggested two heuristics for determining the number of valid clusters. First, a plot of the fusion coefficients versus the number of clusters can be used to determine the appropriate number of clusters. A flattening in the curve indicates that no new information is gained by further cluster mergers. Therefore, the clusters that exist before the flattening can be considered valid. A second heuristic involves subjectively reviewing the resultant dendrogram for the most reasonable solution based on conceptual clarity. For this investigation, both heuristics were used in conjunction to determine the appropriate number of clusters.

The resulting clusters were named using a variation of the recaptured item technique @IT; Meehl, Lykken, Schofield, & Tellegen, 1971). As in factor analy- sis, there is substantial subjectivity used in naming clusters. The RIT was used to reduce, as much as possible, the ad hoc subjectivity in cluster naming. First, six judges were given the list of items for each cluster. The judges were four Ph.D.-level psychologists and two master’s-level psychology graduate students reporting various theoretical orientations (dynamic, cognitive, and family sys- tems). Each judge was asked independently to name each cluster, so that the name best represented the majority of items in each cluster. The authors used the names suggested by the judges to decide on final cluster names.

During phase two, the recaptured-item phase, six additional judges were selected. All judges were full-time research assistants with bachelors degrees in psychology or biology. These judges were given the list of items for each cluster and the final cluster names and asked to match names and clusters.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

CLUSTER ANALYSIS OF OUTCOME 35

RESULTS

First, interrater reliability coefficients were computed for each subset of ten raters. The pooled judge intraclass correlation coefficients for the random effects model for the ten subsets of raters ranged from .74 to 234, indicating good interrater reliability. Since adequate reliability was observed, the similarity ratings were averaged across the ten raters for each comparison. The average ratings were used to construct the similarity matrix of all 90 change items.

As a check on the validity of using undergraduate students to rate semantic similarity, one subset of judges was compared with adult judges. Like the undergraduate raters, the adult judges showed good interrater reliability (ICC = .81). Furthermore, their similarity ratings correlated significantly with the undergraduates’ ratings (r = .7 1 , p < .OOl), indicating that the undergraduate raters were valid raters of semantic similarity.

To determine the appropriate level of the hierarchy, the number of clusters was plotted against the fusion coefficients (see Figure 1). Although this graph reveals a very slight flattening between the 3- and 4-cluster solutions, the lack of a definitive flattening indicated that conceptual clarity might be the best criteria for deciding the appropriate number of clusters for further exploration. A subjective examination of the dendrogram provided in Figure 2 confirmed that the 4-cluster solution revealed coherent clusters that best represented the data at a broad level. In addition, both the 7-cluster solution and the 2-cluster solution seemed to add conceptual clarity to the four broad clusters. Therefore, further analyses focused on the 4-cluster solution with special attention to the seven subclusters and two superclusters in order to best understand the identified domains of change.

The recaptured item technique was used to label the 4- cluster solution. The results of phase one of the RIT indicated that judges were in general agreement

“i 191

m - 17-

a - ; 15-

b 13-

7 -

5-

3-

8‘0 100 120 140 160

PUBION COEFFICIENT

i - 8

180

Figure 1. Plot of number of clusters versus the fusion coefficients for Ward’s method cluster analysis.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

36 CONNOUY AND STRUPP

0 5 10 1s 20 2s

Figure 2. Dendrogram for Wards method cluster analysis of psychotherapy outcomes.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

CLUSTER ANALYSIS OF OUTCOME 37

regarding the cluster names. For the first cluster, three judges chose “fewer symptoms” as the cluster name, while the other judges chose related names, such as: “greater ego strength,” “increased control over patterns/feelings,” and “self- satisfaction. ” “Improved symptoms” was selected as the final cluster name.

For the second cluster, three judges named the cluster “greater self-under- standing. The other judges again selected related cluster names, including “self-recognition,” “increased differentiation,” and “self-awareness.“ “Improved self-understanding” was chosen as the final cluster name. “Improved self-confi- dence” was selected as the final cluster name for the third cluster. Three judges selected this name while the other judges chose such names as “self-assured,” “improved self-esteem, ” and “self-acceptance.”

Cluster 4 resulted in the most varied cluster names. Judges chose such names as “greater self-assertion,” “self-autonomy,” “increased self-focus,” “self con- trol/efficacy,” and “healthy independence.” All names reflected some degree of the patient’s ability to define him/herself in relation to others. Therefore, the cluster was named “greater self-definition.”

The final cluster solution is provided in the form of a dendrogram in Figure 2. The improved symptoms cluster consisted of 39 items. This cluster breaks down conceptually into two subclusters. The “greater self-control” subcluster is best represented by such items as “more control of life,” “able to handle prob lems,” and “able to prevent problems.” The “improved psychological symptoms” subcluster is represented by such items as “improved sleep,” “decrease in depression,” and “fewer crying spells.”

The improved self-understanding cluster consisted of 18 items which are best represented by two subclusters. Representative items for the “improved recog- nition and understanding” subcluster include “understand self better,” “better able to recognize negative aspects of self,” and “better able to identlfy emotions.” The second subcluster, “interpersonal openness,” is represented by the following items: “able to express emotions” and “able to communicate needs.” Seventeen items comprise the improved self-confidence cluster. This cluster is best repre- sented by such items as “more confident in relationships,” “better able to take risks,” and “feels okay to like self.”

Finally, the greater self-definition cluster (with 16 items) can be clarified by examination of two subclusters: “greater independence” and “better boundaries in relationships.” The greater independence subcluster is best represented by items such as “able to live life for self,” “able to put self first,” and “increased independence.” Representative items for the better boundaries in relationships subcluster include “better boundaries in relationships,” “more objective in rela- tionships,” and “understand that you can’t control others.” A complete list of cluster items is provided in Figure 2.

An examination of the 2-cluster solution adds further clarity to the structure of psychotherapy outcomes reported by patients following dynamically oriented psychotherapy. At this level of the hierarchy, two superclusters corresponding to changes in symptoms and changes in self-concept emerge. Thus, the results indicate that therapeutic outcomes reported by this patient sample cover the broad domains of symptoms and self-concept, with self-concept broadly repre- senting changes in self-understanding, self-confidence, and self-definition.

The results of the cluster analysis indicate that across a sample of patients treated in TLDP there are multiple domains of outcome that are considered important. One question that remains is whether individual patients focus on

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

38 CONNOUY AND STRUPP

multiple domains of outcome. In this investigation, patients reported an average of 2.83 important psychotherapeutic changes. The number of clusters repre- sented by each patients’ change items was computed. Across the 4 clusters, patients listed items representing an average of two clusters, with a range of one to three clusters represented across the sample.

The frequency of patients reporting items in each cluster was computed in order to evaluate further which domains of outcome are important to the majority of patients. F&y-two percent (35/67) of patients reported an item in the improved symptoms cluster, 45 percent (30/67) of patients listed items from the improved self-understanding cluster, and 3 1 percent (21/67) included an item representa- tive of the greater self-definition cluster. The improved self-confidence cluster was the most frequently reported cluster, with 61 percent (41/67) of patients including an item in this domain. Within this cluster, the most common items were “improved self-confidence” and “improved self-esteem.” In summary, these results indicate that most of the patients evaluated in this sample found multiple domains of outcome to be important. Although patients varied in the specific domains they each found to be important, the most frequently reported outcome domain was improved self-confidence.

DISCUSSION

The results of the cluster analysis indicate that psychotherapeutic outcome might best be assessed through multiple criteria to account for the types of outcomes that patients treated in dynamically oriented psychotherapy value. It is evident that improvements noted as important by this patient sample include not only changes in psychiatric symptomatology, but also changes in self-understanding, self-confidence, and self-definition.

These results are consistent with earlier works by Strupp, Fox, and Lessler (1969), Horowitz (1979) and Horowitz and Vitkus (l985), who suggest that important domains of outcome include symptoms, cognitions about the self, and interpersonal problems. In addition, the investigation by Elliott et al. (1990) revealed similar domains of outcome from a sample of patients treated in experiential psychotherapy, suggesting that the domain of changes in self-con- cept may be important to patients receiving various forms of psychotherapy.

The focus of self-concept in this sample is noteworthy. Three out of four domains of outcome identified by this sample relate to how the patient feels about him/herself. These results indicate that improvements in self-concept that are important from the patient’s perspective include the patient’s ability to under- stand his/her own feelings and patterns, to feel good about him/herself, and to better define him/herself in interpersonal relationships.

Further, although interpersonal problems do not form a unique cluster as suggested by earlier theorists, an examination of the cluster solution indicates that the three clusters related to self-concept contain an interpersonal compo- nent. The improved self-understanding cluster appears to form two subordinate clusters, one consisting of items that represent general gains in understanding emotions and patterns and the other containing items that represent the patient’s ability to express him/herselfto others. The improved self-confidence cluster also contains items that reflect improvements in self-confidence that affect the patient’s interpersonal world, such as: “more outgoing,” and “able to accept

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

CLUSTER ANALYSIS OF OUTCOME 39

rejection.” Finally, the improved self-definition cluster contains items primarily related to the patient’s ability to achieve independence and define boundaries in relationships.

These results appear inconsistent with the results reported earlier by Llewelyn (1988). Unlike the patients examined by Llewelyn, this sample regarded changes in self-understanding as important. These differences may be a result of the methodologies implemented in each protocol. In Llewelyn’s investigation, patients were asked to identify important events from each session whereas in this investigation patients were asked to globally report on important changes from therapy. It is possible that patients can value global changes in self-under- standing without being able to identify the in-session events that can be consid- ered insight events.

Overall, these results indicate that from the perspective of patients receiving short-term dynamic psychotherapy, outcome consists of multiple criteria. As such, complete outcome batteries, designed to encompass all changes important to these patients, should include symptom change measures and measures of change in various aspects of the self-concept. Assessment of self-concept should further relate not only to how the patient feels about and understands him/herself but also to how the self-concept interacts with the patient’s interpersonal world. Although current research on the efficacy of psychotherapy often includes multiple domains of outcome assessed from multiple perspectives, constructs such as self-understanding, self-confidence, and self-definition are rarely assessed as important domains of outcome.

Although these results indicate important aspects of the patient’s perspective of psychotherapy outcome, some methodological limitations suggest the need for future research. First, although the undergraduate raters reveal good interrater reliability, one might question the generalizability of the raters in representing the general population. If semantic structure varies within the general population due to background variables, then the resulting clusters may not best represent the patient’s perspective of outcome. However, the significant correlation be- tween undergraduate ratings and ratings of the age-matched adult sample indi- cates that the undergraduate raters are adequate raters of semantic similarity.

The current investigation focused only on the perspective of the patient following treatment. Although this perspective has the advantage of representing an informed opinion on the patient’s part, this perspective also has limitations. First, the results include only the opinions of treatment completers. It is possible that some patients discontinued treatment because the domains of outcome that they found important were not being addressed. However, the consistency of these results with the research literature on patients’ perspectives of outcome reviewed earlier suggests that the domains of symptom change and change in various aspects of the self-concept are at least important to many patients.

Another limitation to the retrospective view is that patients may be influ- enced by their therapists’ values and the values of the treatment project. Through- out treatment, patients are implicitly taught by their therapists to value self exploration. In addition, patients are asked to complete questionnaires that may implicitly convey to the patient which domains of outcome are considered important by the investigators. In the current project, patients completed a diverse outcome battery. Notably, there were no measures that directly assessed changes in self-confidence or self-understanding. Although patients are clearly socialized through the treatment to value these constructs, these patients are also

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

40 CONNOLLY AND STRUPP

satisfied with these changes and further report changes that are consistent with the change domains valued by patients treated in alternative treatments such as experiential psychotherapy.

Another methodological limitation of this investigation is that the “import- ant changes” listed by this patient sample were elicited following dynamically oriented treatment. Therefore, the changes listed may be confounded with the changes that actually occurred during this treatment. There might be additional changes that patients find important that weren’t addressed in TLDP. ‘Future research is needed to explore additional domains of outcome which patients receiving alternative treatments find to be important. Although this list cannot be considered to encompass all change categories that may be important to psychotherapy patients, the large subject pool used along with the variability in changes listed, indicate that these categories may well represent important domains of change for patients receiving dynamically oriented treatments.

Further, this investigation utilized an exploratory approach to uncover outcome categories. As a first step, this approach proved helpful in suggesting future research directions. In this investigation, the recaptured item technique was used to reduce subjectivity in cluster naming, and also to help validate the resulting cluster solution. It is noteworthy that the phase one judges reported that the task was difficult. However, independently, the majority of judges chose the same cluster name. In most cases the other judges chose related cluster names. Additionally, the phase two judges unanimously matched the correct cluster names. The difficulty of the task may reflect some level of error in the final cluster solution. However, there seems to be a consistent structure that is evident to all judges. Given the difficulty in validating cluster solutions, these results should be replicated with additional patient samples.

Finally, these results are limited to the patient’s perspective of outcome. Although the perspective of the patient is meaningful, other perspectives of important psychotherdpy outcomes should also be explored. Future research should continue to uncover important domains of outcome from multiple per- spectives. As Strupp and Hadley (1977) suggest, families and the society at large may hold additional domains of outcome to be important that are not currently assessed in psychotherapy outcome research.

Even from the patient’s perspective, many questions remain. Do patients who receive alternative forms of treatment find the same domains of outcome presented here to be important? Further, do patients’ perspectives over the course of treatment change as a result of specific therapeutic interventions? Finally, do patients’ values vary as a function treatment variables such as treatment length and patient variables such as diagnosis, age, or gender?

In summary, researchers should continue to uncover important domains of outcome that could enrich psychotherapy outcome research. It is not enough to assess the domains of outcome that professionals believe to be central from multiple perspectives. As clinicians and researchers, we cannot assume to know what patients, families, and society value as a therapeutic response. The results presented here suggest that there are domains of psychotherapy outcome that patients value that have been overlooked by researchers. Additional research will be needed to create a comprehensive outcome battery that can do justice to the varied interests of patients, therapists, families, and society.

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

CLUSTER ANALYSIS OF OUTCOME 41

REFERENCES

Aldenderfer, M. S., & Blasllfield, R. K. (1984). Cluster analysis. Sage University Paper series on Quantitative Applications in the Social Sci- ences, series no. 07-044. Beverly Hills and London: Sage Publications.

Benjamin, L. S. (1983). The ZNTREX user’s man- ual, Parts 2 and 22 [Computer program man- ual]. (Madison, WI: lntrex Interpersonal Insti- tute, Inc., P.O. Box 55218, Madison, W 53705.)

Derogatis, L. R. (1977). SCL-90: Administration, scoring, andprocedures for manual for the revised version. Baltimore: John Hopkins Hos- pital.

Elliott, R., Clark, C., Kemeny, V., Wexler, M. M., Mack, C., & Brinkeroff, J. (1990). The impact of experiential therapy on depression: The first ten cases. In G. Lietaer, J. Rombauts, & R. Van Balen (Eds.), Client-centered and experi- ential psychotherapy towarris the nineties. Leuven, Belgium: Leuven University Press.

Elliott, R., &James, E. (1989). Varieties of client experiences in psychotherapy: An analysis of the literature. Clinical Psychology Review, 9,

Henry, W. P., Strupp, H. H., Butler, S. F., Schacht, T. E., & Binder, J. L. (1993). Effects of training in time-limited dynamic psychotherapy: Medi- ators of therapists’ responses to training.Jour- nal of Consulting and Clinical Psychology,

Hirschfeld, R. M. A,, Merman, G. L., Gough, H. G., Barrett, J . , Korchin, S. J., & Chodoff, P. (1977). A measure of interpersonal depen- dency. Journal ofPersonality Assessment, 41, 610-618.

Horowitz, L. M. (1979). On the cognitive struc- ture of interpersonal problems treated in psy- chotherapy. Journal of Consulting and Clin- ical Psychology, 47(1), 5-15.

Horowitz, L. M., &Vitkus, J. (1986). The interper- sonal basis of psychiatric symptoms. Clinical Psychology Review, 6 443-469.

Lambert, M. J., and Hill, C. E. (1994). Assessing

443-467.

GI , 441-447.

psychotherapy outcomes and processes. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change. New York: John Wdey.

Llewelyn, S. P. (1988). Psychological therapy as viewed by clients and therapists. British Jour- nal of Clinical Psychology, 27, 223-237.

Malan, D. H. (1973). The outcome problem in psychotherapy research. Archives of General Psychiaty, 29,719-729.

Meehl, P. E., Lykken, D. T., Schofield, W., & Tellegen, A. (1971). Recaptured-item-tech- nique OUT): A method for reducing somewhat the subjective element in factor naming.Jour- nal o f Experimental Research in Personality,

Scheibler, D., & Schneider, W. (1985). Monte Carlo tests of accuracy of cluster analysis algo- rithms: A comparison of hierarchical and non- hierarchical methods. Multivariate Behav- ioral Research, 20, 283-304.

Strupp, H. H. (1980). The Global Outcome Rat- ing. Unpublished manuscript, Center for Psy- chotherapy Research, Department of Psychol- ogy, Vanderbilt University.

Strupp, H. H. (1980). The Vanderbilt Znterper- sonal Locus of Control Scale. Unpublished manuscript, Center for Psychotherapy Re- search, Department of Psychology, Vanderbilt University.

Strupp, H. H., &Binder, J. L. (1984). Psychother- apy in a new key: A guide to time-limited dynamic psychotheraD. New York: Basic Books.

Strupp, H. H., & Hadley, S. (1977). A tripartite model of mental health and therapeutic out- comes. American Psychologist, 32, 187- 196.

Strupp, H. H., Fox, R. E., & Lessler, K. (1969). Patients view theirpsychotherapy. Baltimore: John Hopkins University Press.

Ward, J. (1963). Hierarchical grouping to opti- mize an objective function. Journal of the American Statistical Association, 58, 236- 244.

5, 171-190.

Received December 22,1994 Revision Received July 25, 1995

Accepted July 28,1995

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4

42

APPENDMA

INSTRUCTIONS TO RATERS

CONNOLLY AND STRUPP

THIS TASK INVOLVES RATING THE SIMILARITY OF ITEMS. EACH ITEM REPRESENTS A CHANGE THAT ONE MAY ATTRIBUTE TO PSYCHOTHERAPY. YOU WILL BE GIVEN 405 PAIRS. FOR EACH PAIR, RATE THE SEMANTIC SIMILARITY OF THE PAIR USING THE RATING SCALE PROVIDED BELOW. RECORD YOUR RATING FOR EACH PAIR ON THE LINE TO THE RIGHT OF THAT PAIR.

BE SURE TO USE THE ENTIRE RATING SCALE. SOME OF YOUR RATINGS SHOULD BE HIGH AND SOME SHOULD BE LOW. IT WILL HELP IF YOU REAL) THROUGH THE ITEMS FIRST. DO NOT SKIP ANY ITEMS. THANK YOU FOR YOUR TIME.

I - I I - I I I I 1 2 3 4 5 6 7 NOT AT ALL MODERATELY VERY SIMILAR . SIMILAR SIMILAR

Dow

nloa

ded

by [

Uni

vers

ity o

f B

ath]

at 1

9:17

06

Oct

ober

201

4