7
Sctrnd J SOC Welfare 1996 5 12-18 Copyright Q .Muiiksgaard 1996 Prinled m Drnmnrh All ri&s reserved ,TlhDl\1i +\ ,"L".*, or SOCIAL WELFARE ISSN 0907-2055 Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment Gerdner A, Soderfeldt B. Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment. Scand J SOC Welfare 1996: 5: 12-18. 0 Munksgaard, 1996. This study aims to assess the consistency of replies to questionnaires mail- ed to patients and two kinds of collaterals, i.e., social workers and sig- nificant others, at a public treatment center with socially unstable and compulsory committed patients. It compares the quantity and kind of discrepancies between replies by patients and collaterals on outcome data concerning social situation and drinking habits. It aims to measure the amount of systematic bias among factors that may explain inconsistencies between reports, especially the factors compulsory commitment, worse outcome, frequency of contact and type of collateral. The responders gen- erally agreed. Variables in which there was less agreement were explored in logistic regressions using ten explanatory variables. Significant relations did not exceed those expected by chance. Discrepancies were not system- atic in size and kind. On a six-rank ordinal scale of alcohol use or abuse, however, patients tended to underestimate the extent of their abuse. Incon- sistencies here concerned the degree but not the presence of abuse. No difference in consistency due to type of collateral was found. In conclusion, the consistency of the questionnaires was high and independent of the social situation of the patient, of compulsory commitment and of other background or treatment factors. as well as of treatment outcome and I type of collateral. A. Gerdner1-2, B. SOderfeldt3g4 " m a g i r d e n Treatment Centre, brebro, National Board on Institutional Care, 2Department of Alcohol Diseases, University of Lund, %bra County Council, 4Departrnent of International Health and Social Medicine, Karolinska Institute. Stockholm, Sweden Key words: treatment outcome evaluation; coercion: validity; mail questionnaire: multivariate analysis; collaterals Arne Gerdner, Runnagirden, Box 150 83, 5-70015 Orebro, Sweden I Accepted for publication February 23, 1995 The validity of self-reports in outcome studies of al- coholism treatment is disputed (Sobell & Sobell, 1975, 1978; Watson et al., 1984; Watson, 1985; Fuller et al., 1988; Maisto & Connors, 1992). A high degree of agreement between the replies of alcoholics in clinical samples and the replies of collaterals has been noted (Midanik, 1982; Maisto & Connors, 1992). Under-re- porting as well as over-reporting of the use of alcohol has been found (Midanik, 1982; Myers, 1983). Dis- agreements between the sources of information do not necessarily indicate that the patient is wrong and the test, the record, or the spouse is right (Maisto & Cooper, 1980; Sobell & Sobell, 1980; Midanik, 1982). Validity may be lost because of the presence of non- random error that prevents indicators from showing what they are supposed to indicate: the theoretical con- cept (Carmines & Zeller, 1979). Midanik (1982) con- cludes that discrepancies between self-reports and col- laterals seem to be a function of four factors: I) the sample under investigation, 2) the ,type of collateral, 3) the frequency of contact between the patient and the other and 4) the type of information sought. 12 Sample Consistency between the alcoholic and the collateral has been found to be good in various populations: in- and outpatient alcoholics, pre- and post-treatment patients, prisoners and college students (Maisto & Connors, 1992). Consistency is related to compliance with treatment. Moreover, patients under the influ- ence of alcohol give less accurate self-reports (Skin- ner, 1984). It has been debated whether the discre- pancies increase with a worse outcome (Watson et al. 1984; Watson, 1985; Maisto & O'Farrell, 1985; Wolb- er et al., 1990). Cognitive function and the type of abused drug, i.e., alcohol or various narcotics, are not related to greater discrepancies (Brown et al., 1992). Collaterals Awareness among patients that self-reports will be checked against other sources of information has been found to be positively related to validity (Skin- ner, 1984). Alcoholic patients claim that spouses and

Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Embed Size (px)

Citation preview

Page 1: Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Sctrnd J SOC Welfare 1996 5 12-18 Copyright Q .Muiiksgaard 1996 Prinled m Drnmnrh All ri&s reserved , T l h D l \ 1 i + \ ,"L".*, or

SOCIAL WELFARE ISSN 0907-2055 Explaining inconsistencies between

patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Gerdner A, Soderfeldt B. Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment. Scand J SOC Welfare 1996: 5: 12-18. 0 Munksgaard, 1996.

This study aims to assess the consistency of replies to questionnaires mail- ed to patients and two kinds of collaterals, i.e., social workers and sig- nificant others, at a public treatment center with socially unstable and compulsory committed patients. It compares the quantity and kind of discrepancies between replies by patients and collaterals on outcome data concerning social situation and drinking habits. It aims to measure the amount of systematic bias among factors that may explain inconsistencies between reports, especially the factors compulsory commitment, worse outcome, frequency of contact and type of collateral. The responders gen- erally agreed. Variables in which there was less agreement were explored in logistic regressions using ten explanatory variables. Significant relations did not exceed those expected by chance. Discrepancies were not system- atic in size and kind. On a six-rank ordinal scale of alcohol use or abuse, however, patients tended to underestimate the extent of their abuse. Incon- sistencies here concerned the degree but not the presence of abuse. No difference in consistency due to type of collateral was found. In conclusion, the consistency of the questionnaires was high and independent of the social situation of the patient, of compulsory commitment and of other background or treatment factors. as well as of treatment outcome and

I type of collateral.

A. Gerdner1-2, B. SOderfeldt3g4 "mag i rden Treatment Centre, brebro, National Board on Institutional Care, 2Department of Alcohol Diseases, University of Lund, %bra County Council, 4Departrnent of International Health and Social Medicine, Karolinska Institute. Stockholm, Sweden

Key words: treatment outcome evaluation; coercion: validity; mail questionnaire: multivariate analysis; collaterals

Arne Gerdner, Runnagirden, Box 150 83, 5-70015 Orebro, Sweden

I Accepted for publication February 23, 1995

The validity of self-reports in outcome studies of al- coholism treatment is disputed (Sobell & Sobell, 1975, 1978; Watson et al., 1984; Watson, 1985; Fuller et al., 1988; Maisto & Connors, 1992). A high degree of agreement between the replies of alcoholics in clinical samples and the replies of collaterals has been noted (Midanik, 1982; Maisto & Connors, 1992). Under-re- porting as well as over-reporting of the use of alcohol has been found (Midanik, 1982; Myers, 1983). Dis- agreements between the sources of information do not necessarily indicate that the patient is wrong and the test, the record, or the spouse is right (Maisto & Cooper, 1980; Sobell & Sobell, 1980; Midanik, 1982). Validity may be lost because of the presence of non- random error that prevents indicators from showing what they are supposed to indicate: the theoretical con- cept (Carmines & Zeller, 1979). Midanik (1982) con- cludes that discrepancies between self-reports and col- laterals seem to be a function of four factors: I ) the sample under investigation, 2 ) the ,type of collateral, 3) the frequency of contact between the patient and the other and 4) the type of information sought.

12

Sample

Consistency between the alcoholic and the collateral has been found to be good in various populations: in- and outpatient alcoholics, pre- and post-treatment patients, prisoners and college students (Maisto & Connors, 1992). Consistency is related to compliance with treatment. Moreover, patients under the influ- ence of alcohol give less accurate self-reports (Skin- ner, 1984). It has been debated whether the discre- pancies increase with a worse outcome (Watson et al. 1984; Watson, 1985; Maisto & O'Farrell, 1985; Wolb- er et al., 1990). Cognitive function and the type of abused drug, i.e., alcohol or various narcotics, are not related to greater discrepancies (Brown et al., 1992).

Collaterals

Awareness among patients that self-reports will be checked against other sources of information has been found to be positively related to validity (Skin- ner, 1984). Alcoholic patients claim that spouses and

Page 2: Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Explaining inconsistencies in studies of coercive alcoholism treatment

other relatives know more than employers or super- visors about their drinking habits (Marlatt et al., 1986). Subjects who disagree with spouses or super- visors tend to rate themselves more severely, yet the opposite occurs when subjects disagree with their counsellors (Freedberg & Johnston, 1980).

Contact It is assumed that less contact between patients and collaterals explains the discrepancies (Midanik, 1982; Marlatt et al., 1986). However, Maisto et al. (1979) found no such relation.

Type of information

Since alcoholics are believed to deny abuse, drinking outcome variables may be expected to be less consist- ent than social outcome variables. This expectation has not been confirmed (Maisto & Connors, 1992). It has been found that the consistency between self- reports and reports by collaterals is better concerning global ratings of drinking pattern than concerning very specific data (Marlatt et al., 1986; Maisto & Connors. 1992). The more exact the question, the more difficult it would be for them to agree. Further- more, it seems likely that the alcoholic finds it harder to remember the drinking patterns in the past months or years than in recent weeks. Events may also be recalled. but not the time when they occurred (Mid- anik 1982, 1989).

There are thus several - sometimes divergent - ob- servations about the consistency of reports from dif- ferent sources. Some factors of importance - i.e., compulsory commitment to treatment and social workers as collaterals - have not been studied. There- fore, the aims of this article are:

to describe the magnitude and kind of discrepan- cies between self and collateral reports on social and drinking outcome data; to attempt to explain inconsistencies between re- ports, especially whether compulsory commit- ment, worse outcome and frequency of contact are related to such inconsistencies; and to compare consistencies in reports with the type of collateral, i.e., social workers and significant others.

Material and methods

At the time of this study, Runnaghrden, a treatment center in mid-Sweden, was owned by the Orebro County Council and offered voluntary and coercive care to adult alcoholics and, other drug-dependent people. There was one locked detoxication and moti- vation unit containing 16 beds and three unlocked

units with 30 beds and an intensive Alcoholics Anonymous-oriented program. According to the Swedish law on the coercive treatment of addicts (LVM), an abuser can be coerced by administrative county courts into undergoing treatment at certain public institutions on medical and social grounds, if he or she refuses to accept the necessary treatment voluntarily. Thus coercion here is called “compulsory commitment” because it does not resemble the pro- cedure concerning court-ordered patients who choose treatment instead of prison.

The entire study population consisted of 521 pa- tients who were treated for a minimum of 24 hours in 1988-1990 and were given follow-up ques- tionnaires. All patients were interviewed by a social worker on admission to Runnaghrden, and a medical check-up was done by a physician. Background data were also obtained from referral documents of the social welfare agencies and compulsory commitment decisions.

Questionnaires were mailed one year or later, after discharge, to 521 patients and to their referring social agencies. There were contracts with 113 patients con- cerning a significant other to whom an additional questionnaire could be sent. All these questionnaires were based on the Hazelden 12-month questionnaires (Laundergan, 1982). For this study we chose I 1 out- come variables. Six dichotomous variables indicated social status and improvement: employed (regular work/other), occupation (skilled/unskilled), own apartment (y/n) and legal actions after discharge, such as drunk arrests, arrests for drinking and driv- ing, LVM-sentence, prison or probation (y/n), treat- ment since discharge (y/n), social welfare since dis- charge (y/n; not in the questionnaire sent to signifi- cant others). Five variables focused directly on alcohol abuse and, in some analyses, were dicho- tomized from ordinal or continuous scales in the questionnaire: overall alcohol use (abstinent versus moderate social drinkindabuse only at weekends/ abuse not only at weekenddabuse almost dailylfull- time drinking), frequency of drinking (more often/ equally often versus less oftedrnuch less oftednot used), quantity of drinking (morehame versus less/ much lesshot used), number of relapses since treat- ment (continuous; two or lesdmore), number of drinking events last month (continuous; abstinent/ others).

Study population

With at least one complete questionnaire per patient, the overall effective reply rate was 79% (409 patients). However, the reply rate was lower for each ques- tionnaire. The effective reply rate of social workers was 57% (299 replies), when 130 social workers who reported no contact were categorized as nonre-

13

Page 3: Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Gerdner et al.

sponders. Only 35% of the patients replied (183 re- plies). Of the 113 significant others, 55% replied (62 replies). In case of 109 patients (21%), effective ques- tionnaire replies were obtained from at least one col- lateral as well as from the patient. They form the se- lected study population of the present article. In 87 cases, there were replies both from the patient and the social worker and in 35 cases, both from the pa- tient and the significant other. Overlapping occurred in 13 cases.

The 109 patients studied ranged in age from 21 to 71 years (mean 41.5, SD 9.8), and 77% were men. All were alcohol abusers and 91% were severely depend- ent. One third were multidrug addicts. One fourth were compulsorily committed to care. The majority (560/0) had previously been admitted to institutions for treatment. Many had a difficult social situation: 52% had no regular job, 71% lived alone, 41% had a criminal record and 17% had no apartment of their own. The type of treatment they had received recently also differed: 65%) had completed the intensive treat- ment program, 23% had entered but dropped out of the intensive program, and 12%) had only undergone detoxication.

Another analysis of the entire study population, addressing non-reply, found that patients with a better prognosis and better outcome replied more fre- quently than other patients did (Gerdner et al., in preparation). Social workers, on the other hand, re- spond more about patients with a worse prognosis and worse outcome. No difference was noted in prog- nosis or outcome between those without any of the three questionnaire replies and those with replies. The selected study population was compared to all the other patients in the entire study population regard- ing social background, type of drug, legal and treat- ment factors. Only one of the 15 comparisons proved significant ( R 0 . 0 5 ) : duration of treatment. The en- tire study population was divided into three equally sized groups according to the time spent in the center (<35 days, 35-59 days or >60 days), although about half of those in the selected study population stayed 35-59 days, while about 25% stayed shorter and 25% stayed longer.

Analytical models

The statistical analysis was done using SPSS. Cohen’s kappa measures agreement adjusted for random agreements and is a recommended method to assess validity (Maisto & Connors 1992). Dichotomous variables were chosen in order to compare kappa for drinking outcome with kappa for social situation. Kappa values could be interpreted as follows: cO.00 “poor”, 0.00-0.20 “slight”, 0.2 1-0.40 “fair”, 0.41- 0.60 “moderate”, 0.61-0.80 “sub’stantial” and 0.81- 1.00 “almost perfect” (Landis & Koch. 1977). The

significance of kappa was tested by 95% confidence intervals based on the normal distribution.

The consistency may also be reduced by nonsys- tematic errors. The size and kind of discrepancies in replies were studied. Bivariate logistic regression was used to determine whether background factors are systematically related to differences in the replies of the patients and the social workers. In cross-tabu- lations between inconsistency and possible explana- tory factors, Cramer’s V was used as a measure of association (Liebetrau, 1983). The kind of inconsist- ency concerning dichotomous variables - i.e., the number of under-reports versus over-reports - was tested by McNemar’s test (Pagan0 & Gauvreau, 1992). The size and kind of consistency of the ordinal scales i.e., the magnitudes of overreports versus underreports, was tested by the Wilcoxon matched- pairs signed-rank test (Gibbons, 1993).

Results The agreement between the patients and the col- lateral replies was tested for the 11 selected variables (Table 1). According to the kappa values, agreement varied from fair to perfect on both the social and drinking variables. The patient replies to the “legal actions” variable showed less agreement with the re- plies of both the social worker and the significant other. The patient replies on the “relapse” variable showed less agreement with the replies of the social worker. The confidence intervals are wide due to the paucity of replies, especially from significant others. When the confidence intervals are considered, one may also question the agreements on the “treatment since discharge”, “frequency” and “quantity” vari- ables.

Table 1. Agreement between the replies of patients, social workers and significant others. Cohen’s kappa and 95% confidence intervals (GI)

Collateral Social workers Signifidant others n=87 n=35

Kappa CI 95% Kappa CI 95%

Social situation Occupation 0.65 0.43-0.87 0.70 0.43-0.97 Employed 0 87 0.75-0.99 0.80 0 54-1.00 Social welfare 0.63 0.45-0.81 Own apartment 0.74 0.50-0.98 0.65 0.01-1 .OO Legal actions 0.30 0.05-0.55 0 34 0.00-0.92 Treatment 0.42 0.19-0.66 0.62 0.28-0.95

Drinking outcome

Frequency of drinking 0.57 0.36-0.79 0.59 0.23-0.95 Overall alcohol use 0.62 0.43-0.82 0.92 0.78-1 .OO

Quantity of drinking 0.46 0.20-0.72 0.63 0.30-0.63 Relapses 0.36 0.05-0 68 1 .OO 1.004 .OO Abstinent last month 0.64 0.40-0.88 0.77 0.47-1.00

Mean values of kappa 0.57 0 70

14

Page 4: Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Explaining inconsistencies in studies of coercive alcoholism treatment

On all items but one, the confidence intervals over- lap between social workers and significant others, and there is little difference in the mean values of kappa. Thus, no difference in consistency depending on type of collateral was found.

Variables with kappac0.40 as the lower limit in the 95% confidence interval were evaluated with the bivariate logistic regression models. The dependent variables were absolute differences between the pa- tients and the social workers, i.e., the presence of discrepancy or not, on five outcome variables: legal actions, treatment, frequency, quantity and relapses. The ten independent variables were: difference in reply time (continuous), post-treatment contact be- tween the patient and the social worker (IO+/less), education (more than primary/others), program (entered intensive progradothers), compulsory committal to treatment (y/n), previous inpatient al- cohol treatment (0-1 times/2+ times), drug abuse (alcohol only/others), age (continuous), gender, al- cohol use (abstinent/others). The latter is an out- come variable found to have high agreement be- tween the patients and the collaterals when dicho- tomized. In testing the hypothesis by Watson (1984) that a worse outcome may result in low validity, it is used here as an explanatory factor for other out- come variables.

Of 50 relations, 3 were significant. A high level of education was related to more discrepancies on the questions of legal actions (odds ratio (OR)=8.17, P<O.OI) and frequency (OR=5.50, P40.05). More contact was related to differences in the replies con- cerning quantity (OR=9.41, P<0.05). For 50 com- parisons, 2.5 significant relations are expected by chance, i.e., due to mass significance.

The bivariate associations of the absolute differ- ences between the patients and the social workers ver- sus the three significantly associated variables are de-

scribed by Cramer’s V tested for significance. More contact with social workers was associated with more differences in the report on quantity (n=46, V=0.34, P<0.05). More education was associated with more differences regarding the question on legal actions (n=55, V=0.38, P<O.Ol). The combination of more contact and more education was associated with more differences in the reported frequency of drink- ing (n=25, V=0.46, P<0.05).

Considering the relative difference of magnitudes for the ordinal scales on drinking outcome and the direction of differences on the dichotomous social variables, consistency is shown in Tables 2 and 3. Over-reports means patients who give answers that indicate a better situation, which is not confirmed by the collateral. Under-reports refer to the opposite situation. +Ranks correspond to the magnitude of over-reports, while -Ranks correspond to the magni- tude of under-reports. For most relations, we could not confirm that the inconsistencies showed any ten- dency towards over-reporting nor toward under-re- porting, and the number of significant relations does not exceed what is expected by chance. Here, by using an ordinal scale with 6 ranks on the overail use of alcohol, Table 3 shows that patients tend to under- estimate the degree of their abuse when compared to the reports of social workers and significant others, although the difference between the patients and the significant others does not reach the 5% level because of the small number of cases. However, the dichot- omous variable, abuse or not, constructed from the scale shows very good agreement (Table 1). Inconsist- ency is a matter of degree, not of kind.

Discussion The main findings in this study can be summed up as follows.

Table 2. Over- or under-reporting of various social variables. Patients (P) versus social workers (SW) versus significant others (SO)

Over-reports Agreement Under-reports McNemar 2-tailed P

Occupation (P vs SW) Occupation (P vs SO) Occupation (SO vs SW) Employed (P vs SW) Employed (P vs SO) Employed (SO vs SW) Social welfare (P vs SW) Own apartment (P vs SW) Own apartment (P vs SO) Own apartment (SO vs SW) Legal actions (P vs SW) Legal actions (P vs SO) Legal actions (SO vs SW) New treatment (P vs SW) New treatment (P vsS0) New treatment (SO vs SW)

5 3 2 3 - -

11 3 1

11 2 3

10

2

-

-

47 24 12 59 23 15 56 60 . 24 16 45 23 6

46 21 7

5 1 .oo 1 0.63 3 1 .oo 1 0.63 2 0.50 - 1 .a0 2 0.23 1 0.63 - 1 .oo - 100 8 0.65 1 1 .oo 4 1 .oo 7 0.69 4 0.13 2 1 .oo

15

Page 5: Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Gerdner et al.

Table 3. Magnitudes of over- and under-reportino concerning ordinal scales on drinking outcome. Patients (P) versus social workers (SW) versus significant others (SO)

+Ranks Ties -Ranks Z Wilcoxon 2-tailed P

Overall alcohol use (P vs SW) 26 29 11 -2.95 0.003 Overall alcohol use (P vs SO) 4 21 2 -1.36 0.17 Overall alcohol use (SO vs SW) 5 11 1 -1.78 0.07 Frequency of drinking (P vs SW) 21 32 12 -1.37 0.17 Frequency of drinking (P vs SO) 6 21 1 -1.77 0.08 Frequency of drinking (SO vs SW) 4 9 3 -0.00 1 .oo Quantity of drinking (P vs SW) 20 24 12 -0.86 0.39 Quantity of drinking (P vs SO) 6 20 2 -1.54 0.13 Quantity of drinking (SO vs SW) 3 a 2 -0.40 0.69 Relapses (P vs SW) 12 10 15 -0.29 0.77 Relapses (P vs SO) 3 11 5 -0.07 0.94 Relapses (SO vs SW) 2 6 2 -0.73 0.47 Abstinent last month (P vs SW) 7 25 10 -1.16 0.26 Abstinent last month (P vs SO) 4 17 0 -1.83 0.07 Abstinent last month (SO vs SW) 2 7 3 -0.27 0.79

1. Most of the correllations between the replies of the patients and the collaterals are in substantial agreement, both about drinking outcome and so- cial situation.

2 . Significant relations between discrepancies in re- plies to 10 independent factors did not exceed that expected by chance. Patients with higher level of education and more contacts more often gave re- plies that disagreed with those of the social worker about improvement concerning frequency and quantity of drinking and the occurrence of legal actions since discharge. However, differences in the number and magnitudes of under-reports versus over-reports did not exceed that expected by chance. This would suggest that the discrepancies were not systematic.

3. No differences in consistency due to type of col- lateral was found, i.e., whether replies of the pa- tients were compared to the relies of the social workers or the significant others.

This study has strengths and weaknesses. The main shortcomings are the lack of personal interviews or biological markers and the small number of replies. Concerning the first shortcoming, it should be noted that the patients came from 68 municipalities in 15 counties of Sweden. Therefore, personal interviews and biological tests to assess alcohol intake would have been too costly and time-consuming. In screen- ing and outcome studies, interviews have been com- pared to self-administered questionnaires. No con- clusive evidence has been produced indicating any greater validity of either method (Ulrich, 1981; Skin- ner & Allen, 1983; Duffy & Waterton, 1984; Bernadt et al., 1984). However, because of this shortcoming we could not determine whether the subject was in- toxicated when he or she filled in' the form. Since the replies from collaterals did not indicate a relation be-

tween a worse outcome and a poorer consistency in replies, this does not seem to be a problem. The sec- ond shortcoming reflects the difficulty of assessing patients who have been compulsorily committed to an institution. This problem may also raise the ques- tion of whether the study population is representa- tive. Comparisons concerning background and treat- ment factors as well as another study addressing non- responders in the entire study population did not indicate selectivity (Gerdner et al., in preparation). I t should also be noted that the respondents did not know whether the collateral actually had filled in the questionnaires.

The strengths in our opinion are the use of three groups of respondents who were asked in the same way and answered the same questions independently and the evaluation of inconsistencies by looking for systematic errors.

Such variables of short duration as drinking dur- ing the preceding month were expected to show less agreement because of differences in the follow-up time. Alcoholics are traditionally expected to deny their drinking problems. We therefore expected them to agree less often with collaterals about the outcome variables assessing their drinking than those assessing their social situation. None of these expectations proved correct when all the variables were dicho- tomized. However, the six-rank ordinal scale of over- all alcohol use indicated that patients tended to re- port less abuse than the collaterals. Hence, inconsist- encies concerned the amount but not the existence of abuse.

The largest discrepancies concerned legal actions. Various explanations may account for this, such as the patients prestige bias or the collaterals' limited knowledge. However, since there were about the same number of over-reports as under-reports. this does not seem to be a systematic error.

16

Page 6: Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Explaining inconsistencies in studies of coercive alcoholism treatment

While social workers were expected to know about the presence of abuse and the various problems ag- gravated by drinking, they were not expected to know about the more specific quantity and frequency of drinking, since they were not present during the drinking events. This naturally caused large differ- ences in the reports but, when the magnitudes of under-reports were compared to those of over-re- ports, the differences were not significant. Hence, this was not a systematic error.

The word “relapse” can be interpreted in more than one way. For example, the respondents were asked to choose between the following interpreta- tions: “drinking any alcohol”, “been drunk”, “lost control”, “lost control for several days” and “other”. The respondents in all the three groups chose various interpretations. However, we detected no correlation between interpretation and number of relapses. Thus, the differences between the interpretations of “re- lapse” did not explain the discrepancies. Nor were they explained by any other factor in the regressions.

The results do not support the assumption by Wat- son et al. (1984) that validity, in general, decreases with a worse outcome. Nonabstinence was not related to consistency in any of the regressions. Nor could we confirm the suggestion by Skinner (1984) that patient compliance with treatment affects validity. According to Sobell & Sobell (1978), coercion was not related to discrepancies between patients and collaterals. In their study, coercion meant court-ordered treatment as an alternative to prison, not necessarily indicating a lack of compliance. In this study, however, coercion was a compulsory commitment decided after the pa- tient had refused voluntary treatment. Coercion in this sense was not related to consistency between re- plies of the patients and collaterals. To our knowl- edge, this matter has not been studied before. As pre- viously reported (Brown et al., 1992), the type of drug abused was not related to the consistency between self-reports and those of collaterals.

Both education and contact were chosen as ex- planatory factors on assumptions that were contra- dicted by our findings. A low level of education was expected to influence understanding of the ques- tionnaire, and less contact was expected to result in inconsistencies due to lack of knowledge (Midanik 1982; Marlatt et al., 1986; Maisto & Connors, 1992). Neither of these assumptions proved correct. In this study, more - not less - education and contact were related to discrepancies. However, these findings ap- plied to only a few of the questions asked, and to under-reports as well as over-reports.

In conclusion, the validity of questionnaire replies was found to be satisfactory in a socially unstable population, for compulsorily $omi t ted as well as for voluntary patients and independent of their treat- ment outcome and type of collateral. However, on the

ordinal scale of overall alcohol use, patients tended to underestimate the amount of their abuse, as com- pared to the reports of the social workers. As a safe- guard against bias, a conservative strategy would be to choose the least favorable of the patient’s and col- lateral’s replies. The choice of strategy should depend on the question asked.

Acknowledgments This study was supported by grants from the Orebro County Council’s Research Delegation and from the National Board of Health and Welfare.

References Bernadt MW, Mumford J, Murray RM (1984). Can accurate

drinking histories be obtained from psychiatric patients by a nurse conducting screening interviews? British Journal of Addiction 79: 201-206.

Brown J, Kranzler HR, Del Boca FK (1992). Self-reports by alcohol and drug abuse inpatients: factors affecting reliability and validity. British Journul of Addiction 87: 1013-1024.

Carmines EG, Zeller RA (1979). Reliability and validity assess- ment. Beverly Hills, Sage Publications.

Duffy JC, Waterton JJ (1984). Underreporting of alcohol con- sumption in sample surveys. The effect of computer inter- viewing in field-work. British Jorrrnul of Addiction 79: 303- 308.

Freedberg E, Johnston W (1980). Validity and reliability of al- coholics’ self-reports of use of alcohol submitted before and after treatment. Psycliologicul Reports 46: 999-1005.

Fuller RK, Lee KK, Gordis E (1988). Validity of self-report in alcoholism research: results of a veterans administration cooperative study. Alcoholism: Clinicul wxl Experimental Re- search 12(2): 201-205.

Gerdner A, Soderfeldt B. Berglund M (in preparation). Social worker as collateral - a remedy for non-response in outcome studies of treatment for alcohol dependence.

Gerdner A, Bodin L, Soderfeldt B, Berglund M (in press). Pre- diction of outcome in coerced and voluntarily treated alco- holics. Scandinuvian Journal of Social Weyare.

Gibbons JD (1993). Nonpnramerric statistics - an introduction. Series: Quantitative Applications in the Social Sciences 90. Beverly Hills, Sage University Paper.

Ldndis JR, Koch G G (1977). The measurement of observer agreement for categorical data. Eiometrics 33: 159-174.

Laundergan JC (1982). Easy does it - alcoholisni treatment out- comes. Hazelden arid the Minnesota Model. Center City, Min- nesota, Hazelden Foundation.

Liebetrau AM (1983). Measures of association. Series: Quanti- tative Application in the Social Sciences 32. Beverly Hills, SAGE University Paper.

Maisto SA, Connors GJ (1992). Using subject and collateral reports to measure alcohol consumption. In: Litten R, Allen J, ed. Measuring alcohol consumption: Chicago Humana Press Inc., pp. 73-96.

Maisto SA, Cooper AM (1980). A historical perspective on al- cohol and drug treatment outcome research. In: Sobell LC, Sobell MB, Ward E, ed. Evaluating aicohol and drug ubirse treatment effectiveness: recent advances: New York, Perga- mon Press, pp. 1-14.

Maisto SA, O’Farrell TJ (1985). Comment on the validity of Watson et a1.k ‘Do alcoholics give valid self-reports?’. Journal of Studies on Alcohol 46: 447-450.

Maisto SA. Sobell LC. Sobell MB (1979). Comparison of alco- holics’ self-reports of drinking behavior with reports of col-

17

Page 7: Explaining inconsistencies between patients and collaterals: validity in outcome studies of coercive alcoholism treatment

Gerdner et al.

lateral informants. Journal of Consulting and Clinical Psy- chology 47(1): 106-1 12.

Marlatt GA, Stephens RS, Kivlahan D, Buef DJ, Banaji M (1986). Empirical evidence of the reliability and validity of seu- reports of alcohol use and associated behaviors. Workshop on the Validity of Self-Report in Alcoholism Treatment Re- search. Washington, DC, National Institute on Alcohol Abuse and Alcoholism.

Midanik L (1982). The validity of self-reported alcohol con- sumption and alcohol problems: a literature review. British Journal of Addiction 77: 357-382.

Midanik L (1989). Perspectives on the validity of self-reported alcohol use. British Journal of Addiction 84: 1419-1423.

Myers T (1983). Corroboration of self-reported alcohol con- sumption: a comparison of the accounts of a group of male prisoners and those of their wiveskohabitees. Alcohol & Al- coholism 18: 61-74.

Pagano M, Gauvreau K (1992). Principles of biostatistics. Belmont, CA, Duxbury Press.

Skinner HA (1984). Assessing alcohol use by patients in treat- ment. In: Smart RG, Cappell HD, Glaser FB, Israel Y, Kal- ant H, Popham RE, Schmidt W, Sellers EM, ed. Research odvuncrs in alcohol and drug problems 8. New York, Plenum Press, pp. 183-207.

Skinner HA, Allen BA (1983). Does the computer make a dif- ference? Computerized versus face-to-face versus self-report

assessment of alcohol, drug and tobacco use. Journal of Con- siilting and Clinical Psychology 51: 261-215.

Sobell LC, Sobell MB (1975). Outpatient alcoholics give valid self-reports. Journal of Nervous and Mental Diseases 161: 32- 42.

Sobell LC, Sobell MB (1978). Validity of self-reports in three populations of alcoholics. Journal of Consulting and Clinical

Sobell LC, Sobell MB (1980). Convergent validity: An ap- proach to increasing confidence in treatment outcome con- clusions with alcohol and drug abusers. In: Sobell LC, Sobell MB, Ward E, ed. Evaluating alcohol and drug abuse treatment effectiveness: recent advances. New York. Pergamon Press.

Ulrich J (1981). Two methods of control over results of alcohol- ism treatment. International Journal of Rehabilitation Re- search 4 (3): 404-405.

Watson CG (1985). More reasons for a moratorium: a reply to Maisto & OFarreil. Journal of Studies on Alcohol 46: 450- 453.

Watson CG, Tiileskjor C, Hoodecheck-Schow EA, Pucei J, Ja- cobs L (1984). Do alcoholics give valid self-reports? Journal of Studies on Alcohol 46: 344-348.

Wolber G, Came WF, Alexander R (1990). The validity of self- reported abstinence and quality sobriety following chemical dependence treatment. International Journal of the Addictions

Psychology 46: 901-901.

25(5): 495-513.

18