The effectiveness of a consultation: Compliance with initial recommendations

SPECIAL ARTICLE

The Effectiveness of a Consultation

Compliance with Initial Recommendations

CYNTHIA L. SEARS, M.D.

MARY E. CHARLSON, M.D.

New York, New York

From the Departments of Medicine and Public Health, Cornell University College, New York, New York. Dr. Sears was a Henry J. Kaiser Family Foundation Fellow in General Internal Medicine. Dr. Charlson is a Henry J. Kaiser Family Founda- tion Faculty Scholar in General Internal Medicine. Requests for reprints should be addressed to Dr. Mary E. Charlson, DepaWrent of Medicine, Cornell University College, 1300 York Avenue, New York, New York 10021. Manuscript accepted November 17, 1982.

To identify the attributes of an effective consultation, 202 general medicine consultations were analyzed to assess the extent of compliance with the consultant’s initial recommendations. The overall compliance rate was 77 percent. Compliance decreased as the number of recommendations increased. The consultant made more recommendations among patients who had more complex and more severe illnesses. Although compliance did increase slgnlfi- cantly in severely ill patients (p <O.Ol), with each severity level, compliance was higher when flve or fewer recommendations were made. In fact, compliance decreased from 96 percent in severely ill patients with small consultation lists to 79 percent in those with large lists. Compliance was greatest with recommendations involving medications and least with those requiring direct physician and nursing action. Multlvarlate analysis confirmed that clinical severity of the patients’ Illnesses and the type and number of recommendations were all predictors of compliance. To promote overall compliance, consultants should limit the total number of recommendations in their Initial consultation to five or fewer, focusing on issues central to current patient care. This is especially true in severely Ill patients. Since recommendations that must be Implemented by physicians or nurses have a lower compliance rate, consultants must carefully follow up those requests.

Consultation is a common clinical activity for many internists. Acting as a consultant for hospitalized patients, the internist is commonly confronted with severely ill patients with multiple problems [ 11. The consultant-internist must convince the patient’s primary physician that the recommendations should be implemented. Consultative medicine requires a combination of clinical ability and skill in communication. Yet, there is little data about either the science or the art of consultation [ 2-61.

The effectiveness of a consultation could be defined as the extent to which the consultant’s recommendations are implemented by the referring physician. In contrast, a study of the efficacy of a consultation would consider the clinical outcome-whether the consultation did more good than harm.

To identify the attributes of an effective initial consultation, 202 consultations requested of the General Medicine Consultation Service by surgeons were reviewed. The compliance with the consultant’s recommendations was assessed.

a70 May 1993 The American Journal of Medicine Volume 74

EFFECTIVENESS OF CONSULTATION-SEARS and CHARLSON

METHODS

Assembly of Populatfon. Between July 1,1980, and March 9, 1981,212 general medicine consultations were requested by the surgical services at the New York Hospital-Cornell Medical Center. Two consultations that resulted in emergency transfer to the medical service were eliminated; eight charts could not be retrieved. Thus, 202 (95 percent) consultations were reviewed retrospectively.

The consultations were performed on the day that they were requested by either a senior resident or general medicine fellow, in conjunction with an attending physician. Within 24 hours, a typed note was placed on the patient’s chart: the note contained the reason for the consultation, a list of the patient’s problems, a numbered list of the consultant’s recommendations, and a discussion of the rationale for the recommendations [ 71. Recommendations. Deflnltion and number of recommendations: Each statement in the consultant’s list was defined as a recommendation, except for (1) those stated as “if . . . consider” or “would be interesting to obtain”; (2) those involving tests or diagnostic studies scheduled or done before the consultation; and (3) those suggesting routine admission tests.

Each statement in the consultant’s list was counted as a single recommendation, with the following exceptions in which several statements were counted as one recommendation: (1) recommendations that multiple tests be performed on blood or urine specimens obtained at the same time; (2) recommendations to continue multiple preadmission medications.

Category of r ecommw&tionsz Recommendations were divided into four categories according to who was responsible for completing the request: (1) diagnostic tests included x- rays, electrocardiograms, scans, and blood tests, all of which were performed by hospital laboratories or technical support staff; (2) medications included all recommendations involving medications and intravenous orders; (3) the nursing category included recommendations (excluding medications) that had to be carried out by the nursing staff, for example, weights, intake and output measurements, and determination of urine fractional% and (4) the physician category included recommendations that had to be performed by the physician, for example, skin tests, blood cultures, and blood gas determi- nations.

Urgency of recommendations: Recommendations were classified as crucial or routine. Crucial recommendations were those judged critical to a patient’s management, in- cluding those for which the time frame for completion was specified by the consultant as “today” or “by tomorrow,” and those that specified tests, procedures, or medications to be done before surgery. All other recommendations were routine. Examples of crucial recommendations include joint aspiration because of a suspicion of septic arthritis or heparin for a pulmonary embolus; in contrast, routine recommendations included diagnostic thoracentesis for a pleural effu- sion or antibiotics for a urinary tract infection. Compliance. The patient’s chart was searched for evidence that the recommendations were implemented. Compliance

was defined as the number of recommendations completed (ordered and done) divided by the total number of recom mendations made. For example, four of six recommendations in one patient were followed for a compliance rate of 67 percent. Consultations resuiting in no recommendations were omitted from the analysis of compliance.

Time frame: Crucial recommendations must have been completed within 24 hours from the time the recommendation was made, or preoperatively, if surgery was scheduled more than 24 hours later.

Routine recommendations must have been initiated within 72 hours, unless the consultant specified a different time frame. If there was evidence of implementation of the recommendation in the physician’s or nurses’ progress notes, order book, iab sheets, or medication sheets, credii was given for completion. Completion of recommendations such as determination of urine fractionals, calorie counts, and “fre- quent” blood tests were assessed over the first week of hospitalization.

Uncompleted recommendations may have been (1) ordered by the physician but not done or (2) never ordered. Eighty-one percent of uncompleted recommendations were never ordered by the physician. For this analysis, the 19 percent of uncompleted recommendations that were ordered but not carried out were counted as noncompliance.

Consultant confounding: The consultant actually completed 4.5 percent (56) of the recommendations. Of these, 7 percent involved the nursing category, 21 percent required direct physician action, 27 percent involved arranging or ordering diagnostic tests, and 45 percent involved medications. For this analysis, recommendations carried out by the consultant were counted as completed-in order to avoid inflating the number of uncompleted recommendations by counting them as “not done.” However, if the consultant- completed recommendations were counted as “not done” or if they were excluded from the analysis, the basic results were unchanged. Methods of Classifkatkn. Clinical severity: Patients were classified as not iii, moderately ill, or severely ill, taking into account the medical and surgical problems. Typical examples of the clinical severity classification are as follows.

Not ill included patients who had no active medical problems, those with compensated chronic illnesses, for example, chronic obstructive pulmonary disease or congestive heart failure, and those with asymptomatic, but not necessarily well-controlled, diabetes meilitus or hypertension.

Moderately ill included patients with multiple non-life- threatening management problems (for example a patient with fever, wound infection, diabetes mellitus, stable congestive heart failure, and angina). it also included those with decompensated chronic illnesses, those with symptomatic diabetes meiiitus, or uncontrolled hypertension (diastolic pressure persistently greater than 110 mm Hg).

Severely ill included patients with multiple problems that were possibly immediately life-threatening, as well as those with acute respiratory failure, pulmonary embolus, diabetic ketoacidosis, or hypertensive crisis.

Problems: All medical conditions, as listed by the consultant, were defined as problems, excluding (1) previously

May 1983 The American Journal of Medicine Volume 74 871


TABLE I Questions Posed to Referring Physicians

“Soundness” Were the recommendations made medically sound, well thought out, and intelligent?

_1_______~4___________7____________10_

Hasty and Sound but Mostly well Sound, stupid not too thought out thoughtful, and

intelligent intelligent “interacilon” Do you feel that the consultant functioned well as part of the clinical team (how did the consultant interact with you)?

.l___,_ 1-_r.4 _______ ____7 ____ ______ __., (&

Bull in a A difficult Good as team player Very useful and china shop; player but but difficult to helpful member; overbearing; with good work with at times; always cordial delusions of sugges- generally cordial and infallibility tions and understanding understanding

resolved Or inactive medical problems, for example, history of treated syphilis: (2) current surgical problems; (3) prior surgical procedures; (4) problems listed as “possible history of” or “to be ruled out.”

Operative status: The consultation was defined as preoperative when surgery was planned within one week, and postoperative if surgery had occurred less than one month earlier. A patient was classified as having had “no operation” if no surgery was planned or if surgery had occurred more than one month earlier. Assessment of the Referring Physicians. A closed-format questionnaire, which employed the visual analogue technique [8], was used to evaluate the service from the perspective of the referring physicians. Specifically, the soundness of the consultations and the nature of interaction of the consultant with the referring physician were assessed by the questions listed in Table I.

In four services (ophthalmology, otorhinoiaryngoiogy, oral surgery, and urology), all those who had requested consultations completed the questionnaire; in gynecology, more’ than half completed the questionnaire; in general surgery and orthopedics, fewer than half returned the forms. However, the physicians who completed the questionnaires had requested about 80 percent of the total consultations. Statlsticai Analysis. The difference between two groups was assessed by the two-tailed t test for means and chi- square for proportions. The difference between three or more groups was assessed by analysis of variance; F ratios had to achieve an alpha level of 0.05 for specific comparisons to be made by the Scheffe procedure [9]. Analysis of variance and regression were performed through the GLM (General Linear Models) program in the Statistical Analysis System [ lo].

RESULTS

Number of Recommendations. In total, 1,256 recommendations were made, a mean of 6.2 per patient. An average of 1 .O crucial recommendation was made per patient (range zero to six); 5.2 were routine (range zero to 19). In 6 percent (12 of 202) of the consultations, the consultant made no recommendations.

As shown in Table II, more recommendations were made in patients older than 50 years of age, among those who were more severely ill, and among those with more .problems. Multivariate analysis demonstrated that clinical severity (p <0.002) and number of problems (p <O.OOl) were significant independent predictors of the number of recommendations made by the consultant. Fewer recommendations were made in preoperative patients (p <0.05). The differences in number of rec-

TABLE Ii

Age

Characteristics of 202 General Medicine Consuitatlons on Surgical Services

Number of Mean Number of Patients per Recommendations

Group per Pallent’

Compliance

per Patient+

149 55 (27 %)i 4.5 250 169 65 (32%) 6.8 170 62 (41%) 6.9

Clinical severity Not ill 76 (38%) 3.8 Moderately ill 102 (50%) 7.3 Severely ill 24 (12%) 9.1

Number of problems None 10 (5%) 1.5 l-3 120 (59%) 4.8 L4 72 (36%) 9.2

Operative status Preoperative 126 (63%) 4.8 Postoperative 35 (17%) 6.5 No operation 41 (20%) 6.6

l Includes the 12 consultations in which no recommendations were made. + Recommendations completed (ordered and carried out) divided by total recommendations per patient. * Percentage of total number of consultations (that is, percentage of 202).

76% 72% 74%

72% 73% 83%

74% 73% 74%

75% 67% 75%

672 May 1983 The American Journal of Medicine Volume 74


TABLE III Characteristics of Recommendations

Total Number of

Recommendations Mean

per Patient Compliance per Patient

Number of Patients

Urgency Crucial Routine

Category Medications Diagnostic tests Physician Nursing

Size of List Small (one to five recommendations) Large (six or more recommendations)

210 1.0 90% 1,046 5.2 71%

388 1.9 91% 153 395 1.8 73% 155 208 1.0 65% 127 265 1.3 52% 111

275 3.1 78%

981 9.7 70%

109’ la9

a9 101

l Number of patients with given urgency, category, or list size, that is, crucial recommendation in 109 patients, routine in 189.

ommendations according to age and surgical service were explained by these variables. Clinical severity, problems, and operative status accounted for 41 percent of the variance in the number of recommendations (F = 9.27; p <O.OOl; R* = 0.409).

When crucial recommendations were analyzed separately, the overall pattern was similar. The number of problems and clinical severity were significant independent predictors of the number of crucial recommendations made (p <O.Ol, p <0.002, respectively). There was also some variation in the number of crucial recommendations made by service; fewer crucial recommendations were made for patients with burns or urologic illness (p <O.OOl, p <0.02, respectively). Compliance. As shown in Table II, compliance did not vary significantly with the age of the patient, the number of problems, or the operative status. However, compliance was significantly greater in the severely ill than in the not ill to moderately ill (T = 3.81; p <O.Ol).

As shown in Table Ill, compliance with routine recommendations was 7 1 percent: compliance with crucial recommendations was 90 percent (T = 6.2; p <O.OOl). Overall, compliance was highest-91 percent-for recommendations involving medication; it decreased to 73 percent for those involving diagnostic tests, to 65

percent for those involving physician action, and to 52 percent for those involving nursing action (F = 7.99; p <O.OOl). Recommendations ordered but not carried out amounted to 0.3 percent in the medication category, 3 percent in the diagnostic test category, 3 percent in the physician category, but almost 17 percent in the nursing category. Thus, only in the nursing category did it make a significant difference how the “ordered not carried out” were counted.

As is shown in Table Ill, compliance was significantly greater when small lists of recommendations were made (T = 4.1; p <0.05). However, as shown in Table IV, when compliance is divided into categories of 100 percent, 75 to 99 percent, and less than 75 percent, the impact of the size of the list of recommendations on compliance is more striking. In consultations with one to five recommendations, 54 percent (48) resulted in all recommendations being followed; in those with six

or more recommendations, only 7 percent (seven) resulted in all being followed. In addition, 64 percent (63) of moderately ill and 79 percent (19) of severely ill patients, but only 28 percent (19) of not ill patients, had six or more recommendations made. As shown in Table IV, the majority of consultations involving these moderately or severely ill patients had compliance rates of

TABLE IV Distribution of Consultations* by Extent of Compliance, Clinical Severity, and Size of List of Recommendations (l-5 or 26)

Extent 01 Compliance

100% 75-99 % <75%

Not Ill Moderately Ill Total 1-5

Severely Ill 16 l-5 >_6 1-5 26 1-5 16

26t 0 16 5 4 2 48 7 4 7 3 24 1 10 a 41

ia 12 15 34 0 7 33 53

Total 48 19 36 63 5 19 a9 101

l Based on the 190 consultations with recommendations. + Number of consultations.



TABLE V Compliance per Patient According to Severity and Number and Type ot Recommendations

biagnootk Medlcattml Test8 PhysIcIan Nursing All’

Small lists (one to five recommendations) Not ill or moderately ill

Severely ill

Total

Large lists (six or more recommendations) Not ill or moderately ill

Severely iii

Total

96%

(6l)t 100%

(2) 95%

(63)

87% (62) 66%

(la) 67 96

(66)

72%

(56) 66%

(4) 73%

(66)

71% (79) 64%

(18) 73%

(95)

70%

(36) 100%

(3) 72%

(41)

59%

(76) 76%

(16) 62%

(66)

54%

(26) 100%

(1) 55%

(29)

46 %

(67) 62%

(15) 51%

(62)

76%

(64) 96 %

(5) 78%

(69)

66%

(62) 79%

(19) 70%

(101)

l The overall compliance rates per group. t Numbers In parentheses are number of patients in severity class with given list size who had recommendations involving medications made.

less than 100 percent. The total number of recommendations is a powerful determinant of compliance, tending to offset the overall improved compliance rates noted when patients are perceived as being more severely ill.

Multivariate analysis revealed that list size, clinical severity, and the category of recommendations were significant and independent predictors of compliance. As shown in Table V, compliance decreased as the total number of recommendations increased. Of particular note, in severely ill patients, overall compliance rates decreased from 96 percent with a small list to 79 percent with a large list. However, compliance in each category of recommendations was greater in the severely ill than in the not ill or moderately ill patients. Regression analysis confirmed that the contributions of severity and list size were independent (p <O.Ol, p =0.056, respectively).

Regression analysis also demonstrated two other points. Regardless of list size or severity, compliance with medication recommendations was high (p <O.OOl); conversely, compliance with nursing recommendations was poor (p <0.05). Compliance was most affected by list size and clinical severity in the physician cate-

gory. Compliance within each category of recommenda-

tions was also analyzed for the effect of age. Only in the physician category did age exert even a minimal effect on compliance. Here, among the 16 severely ill patients who had consultation lists containing six or more recommendations, compliance for the eight patients younger than 70 years was 81 percent; for the eight who were older than 70 years, compliance was 69 percent.

The difference, while not statistically significant, may be clinically important. Age, number of problems, and operative status dii not affect overall compliance when list size, clinical severity, and the category of recommendations were taken into account. When crucial recommendations were analyzed separately, the effects of list size, clinical severity, and category of recommendations were unchanged. Compliance Versus Evaluation of Consultants’ Rec- ommendations. Flgure 1 shows the mean number of recommendations per patient not followed, carried out by the consultant, or carried out by the primary physician according to the service that requested the consultation. For this analysis only, the percentage compliance with recommendations was calculated by dividing the number of recommendations followed (excluding those carried out by the consultant) by the number of recommendations made. The compliance per service ranged from a high of 73 percent to a low of 52 percent, as shown at the bottom of Figure 1.

The compliance per service was compared in relation to the surgeons’ assessments of whether the recommendations made by the consultants were medically sound and well conceived (question 1, Table I), since lower rafings on soundness might correlate with lower compliance with recommendations. The different services’ soundness ratings of the general medicine consultations are shown at the bottom of Figure 1. Of the two services that rated soundness highest, one (orthopedics) actually followed more recommendations per patient and had a high compliance, whereas the other (ophthalmology) had the largest proportion of recommendations carried out by the consultants, con-



8-

O- SERWCE OWGYN GEN. ORAL

4R:x:O ENT URO EYE

SURG. SURG.

‘3outvoM-ss ” 9.0 8.0 10.0 0.0 8.0 10.0 (MID) (LO) (HI) (MID) (LO) (HI)

“/NTERACTIoN” 0.3 7.7 8.7 9.3 9.3 9.2 10.0 (LO) (LO) (MID) (till (HI) (HI) (HI)

Mean Number of Recommendations per Patient per Service :

I Not followed

Carried out by consultant

ESB Carried out by primory physician

Figure 1. Compliance with recommendations of the general medicine consultant: mean number of recommendations per patient per service followed and not followed. Compliance was calculated by dividing the number of recommendations carried out by the primary physician by the total number.

founding the estimation of compliance. Of the two services that provided the lowest ratings of soundness, one (urology) had the largest number of recommendations not followed, whereas the other (surgery) had a high compliance rate.

Since compliance has also been suggested to be a function of the tenor of the relationship between physicians, referring services’ evaluations of the cordiality of the consultants were also compared with their compliance rates. The different services’ ratings of the interaction (question 2, Table I), with the consultants are shown at the bottom of Figure 1. Two of the services (urology and ophthalmology) that provided high ratings of interaction had the lowest compliance rates, whereas two of the services (surgery and obstetrics/gynecology) that provided the lowest ratings of interaction had the highest compliance rates. Therefore, on a service- by-service basis, there was no simple relation between compliance with recommendations and either the evaluation of soundness or the nature of the physician interaction.

COMMENTS

Internists spend substantial time in complex consultative encounters [ 1,11,12]. Some have stressed the im- portance of teaching consultative technique [ 131, but guidelines are scarce [2]. Since we train young physicians “to be complete” [14] in their approach to patients, it is not surprising that long lists of recommendations were often made, particularly in sick patients with many problems. However, as the number of recommendations increased, compliance decreased. Although comprehensive evaluation is essential, ex- haustive lists of recommendations in the initial consultation may be counterproductive.

The consultant needs to be particularly cautious in generating long lists of recommendations in severely ill patients. In these clinical situations, recommendations especially important to an individual patient’s care may get lost. This may be of particular concern in the elderly, severely ill patient, particularly when the recommendations must be implemented by the physician.



Although decreased compliance in this subgroup may vices’ compliance rates did not parallel either their reflect factors that were unmeasured (for example, the evaluations of “soundness” nor their assessments of prognosis of these elderly patients or patient and family the cordiality of the physician “interaction.” It may be, desires for noninvasive care), it potentially pinpoints however, that assessing these variables on the level of patients requiring meticulous follow-up care by the a whole service is too insensitive a method of mea- consultant. suring their relationship.

Recommendations involving nursing also deserve careful follow-up evaluation by the consultant. Twenty-nine percent of suggested recommendations in the nursing category were never ordered; an addi- tional 17 percent were ordered but not completed. In part, this may reflect a lack of written documentation. Nonetheless, the criteria for completion of the recommendations were not stringent; low compliance in the nursing category appeared to reflect a lower respon- siveness on the part of both the physician and the nursing staff. In contrast, recommendations involving medications are very likely to be implemented.

Studies of consultations in psychiatry [ 15,161 and other disciplines [ 17,161 have suggested that the nature of the relationship between the referring physician and the consultant may be a more important determinant of compliance than the soundness of recommendations. We did not find this to be the case. The different ser-

It should be emphasized that the appropriateness of the recommendations made by the general medicine consultant was not evaluated; therefore, compliance must not be equated with quality of care. Nonetheless, the assessment of the therapeutic efficacy of different management strategies is a crucial task for investiga- tors in consultative medicine. There have been very few studies of the common issues that arise in general medicine consultations, for instance, preoperative management of the diabetic or hypertensive patient. The lack of data makes it difficult to assess the appropriateness of consultants’ recommendations by any objective criteria.

Comprehensive evaluation is central to all patient care, but in consultative medicine, care must be taken to present recommendations in a concise fashion. The effect on compliance of several small lists of recommendations awaits future research.

1.

2.

3.

4.

5.

6.

7.

6.

9.

REFERENCES

Mendenhall RC, Tarlov AR, Girard RA, et al: A national study of internal medicine and its specialities. II. Primary care in internal medicine. Ann Intern Med 1979; 91: 275-267.

Tumulty PA: Communicating as a consultant. In: The effective clinician. Philadelphia: WB Saunders, 1973; 45-46.

Rudd P, Siegler M, Byyny RL: Perioperative diabetic consultation: a plea for improved training. J Med Educ 1976; 53: 590-596.

Perlman LV, Kruskall MS, Rosenrweig D, et al: Process and outcome in medical consultations. Evaluation on a pulmonary service. Postgrad Med 1975; 57: 111-l 15.

MacKenzie TB, Popkis MK, Collies AL, et al: The effectiveness of cardiology consultation concordance wlth dllgnostic and drug recommendations. Chest 1961; 76: 16-22.

Byyny RL, Slegler M, Tarlov, AR: Development of an academic section of general internal medicine. Am J Med 1977; 63: 493-496.

Charlson ME, Cohen RP, Sears CL: General medicine consultation: lessons from a clinical service. Am J Med, in press.

Bond A, Lader M: The use of analogue scales in rating sub- jective feelings. Br J Med Psycho1 1974; 47: 211-216.

Kleinbaum PG. Kupper LL: Applied regression analysis and other multivariable methods. North Scituate, Massachu-

10.

11.

12.

13.

14.

15.

16.

17.

16.

setts: Duxbury Press, 1976; 271-277. Statistical analysis system. Cary, North Carolina: SAS Institute.

1960. Burnum JF: What one internist does in his practice: implica-

tions for the internist’s disputed role and education. Ann Intern Med 1973; 76: 437-444.

Moore DA, Kammerer WS, McGlynn TJ, et al: Consultations in internal medicine: a training program resource. J Med Educ 1977; 52: 323-327.

Rudd P: Contrasts in academic consultation. Ann Intern Med 1961; 94: 537-536.

Hardison JE: To be complete. N Engl J Med 1979; 300: 193-194.

Cavanaugh JL, Flood J: Psychiatric consultation services in the large general hospital: a review of a new report. Int J Psych Med 1976-77; 7: 193-207.

Krakowski A: Psychiatric consultation in the general hospital: an exploration of resistances. Dis Nerv Sys 1975; 36: 242-244.

Robbins PR. Spencer EC, Frank DA: Borne factors influencing the outcome of consultation. Am J Public Health 1970: 60: 524-534.

Wang VL: An annotated bibliography on consultation. Health Educ Monographs 1975; 3: 403-413.


Documents

The effectiveness of a consultation: Compliance with initial recommendations