JOURNAL OF CHILD AND ADOLESCENT PSYCHOPHARMACOLOGYVolume 6, Number 4, 1996Mary Ann Liebert, Inc.Pp. 215-228

Double-Blind Versus Open Evaluations ofStimulant Drug Response in Children withAttention-Deficit Hyperactivity Disorder

JOYCE SPRAFKIN, Ph.D. and KENNETH D. GADOW, Ph.D.

ABSTRACT

Although placebo controls and double-blind conditions are considered to be essential for theunbiased scientific assessment of drug effects, there is very little research on these proce-dures in the pédiatrie psychopharmacology literature. To examine the impact of controlledassessment procedures on the magnitude of observed drug effects, two groups of childrenwith attention-deficit hyperactivity disorder (ADHD) were evaluated for response tomethyIplienidate under two different assessment procedures. One group (n = 33) was partof a placebo-controlled, double-blind crossover research protocol, with randomized dosesequences, compliance checks, numerous dependent measures, written informed consent,and a considerable amount of staff involvement. The other group (n = 43) received phar-macotherapy at a community-based child psychiatry outpatient service where they were fol-lowed in a routine clinical manner, with "no treatment" as the only control condition, stan-dard fixed-dose titration, parental responsibility for data collection, use of form letters, andminimal staff involvement. Each individual in both groups received divided doses of 0.3 and0.5/0.6 mg/kg daily for a minimum of 1 week at each dose. Comparisons of teacher ratingsobtained for the two assessment procedures revealed highly similar findings. The results ofthis study are discussed with regard to both their methodological and clinical implications.

For research PSYCHOPHARMACOLOGiSTS, placebos and double-blind conditions are sine qua non for theconduct of scientific investigations and the satisfactory documentation of the safety and efficacy of ther-

apeutic agents. In fact, it would be pure heresy to even suggest that there may be situations where suchcontrols are not necessary. Many clinical psychopharmacologists, most notably those who have adopted ap-plied behavioral analysis methodologies (behavioral pharmacologists), have even argued that placebo con-

trol and double-blind procedures should be incorporated into routine clinical evaluations to ensure reliableand valid assessment of drug response (Ullmann and Sleator 1986).

Some of the earliest examples of placebo controls in drug research were studies of the effects of ethanoland caffeine in normal volunteers (e.g., Gilliland and Nelson 1934, Hollingworth 1923, Rivers 1908). Itwas not until the 1960s, however, that they actually became commonplace. The use of a placebo prepara-

Department of Psychiatry and Behavioral Science, State University of New York, Stony Brook, NY.

215

SPRAFKIN AND GADOW

tion in a clinical pharmacological study is a direct application of John Stuart Mill's method of difference,which maintains that:

If an instance in which the phenomenon under investigation occurs, and an instance in which it does notoccur, have every circumstance in common save one, that occurring in the former; the circumstances inwhich alone the two instances differ, is the effect, or the cause, or an indispensable part of the cause, ofthe phenomenon (Mill JS, A System ofLogic (Vol. 1), p. 458, quoted by Cohen and Nagel, 1934, p. 256).

Although it is impossible to create identical circumstances in a drug study, investigators are in general agree-ment that every effort should be made to achieve this goal, especially with regard to variables that are knownto influence drug response. Variables that could conceivably be relevant, but which are nevertheless be-yond manipulation, are conveniently ignored. This is best illustrated by comparing procedures for evaluat-ing pharmacological versus behavioral interventions (Gadow et al. 1986b, Gadow 1985).

One of the earliest drug studies to use double-blind procedures was an experiment to determine if theanesthetic properties of bottled ether were in fact more powerful than ether purchased in large drums (Goldand Gold 1935). Another early double-blind study examined the effects of amphetamine sulfate on the cog-nitive performance of juvenile delinquents (Molitch and Eccles 1937). During the 1950s, several psychol-ogy experiments were conducted illustrating how instructions provided by, or the behavior of the experi-menter (whether intentional or not), could influence the outcome of an experiment (Gadow et al. 1986b).In time, psychopharmacologists began to use double-blind procedures, and by the late 1960s, a growingnumber of scientists believed that their use should become a mandatory requirement for publication.

In spite of their now obvious benefits for obtaining reliable and valid information in a drug evaluation,placebo controls and double-blind procedures are not without limitations, controversy, and confusion. Inprinciple, placebo preparations should be identical to the experimental drug formulation in every way (color,size, shape, taste) except for the "active" ingredient. This sounds relatively straightforward but can be dif-ficult to achieve in actual practical application. Even more problematic for researchers is the conceptual-ization of the placebo (e.g., Gadow et al. 1986a, Jospe 1978, Grunbaum 1981, White et al. 1985), particu-larly the notion of active and inactive ingredients. Numerous studies have shown that double-blind conditionsare rarely maintained because the care provider and patient can often differentiate treatment from placeboconditions. In fact, if this were not the case, one might seriously question the efficacy of the treatment.Double-blind procedures are not only difficult to maintain, but there is relatively little consensus as to whatconstitutes a "blind" state. For example, in some research protocols, only the manufacturer knows the iden-tity of the treatment formulation, whereas in others, some individuals actually involved in patient manage-ment (but not all) know or have access to the treatment code. Blindness is therefore a continuum that rangesfrom the highly rigorous standards that characterize most federally funded drug studies to the poorly con-trolled efforts of the dilettante. The reluctance of many scientific journals to publish single-blind and opentrials no doubt contributes to authors' obfuscations and ambiguities in procedural descriptions and, at times,exaggerated claims of methodological rigor (see Aman and Singh 1986).

These commonly recognized limitations notwithstanding, it is generally believed that the failure to use

placebos and double-blind conditions results in erroneous conclusions about drug effects, most typically,exaggerated claims of efficacy. Although we do not question either the purpose or importance of controlledtrials, it is noteworthy that in the child psychopharmacology literature, there is almost no research on therelationship between methodological control and the perceived magnitude of drug effects, and the little re-search that has been done has yielded mixed results. Sulzbacher (1973) published a widely cited review ar-

ticle comparing the results of 208 studies that used controlled conditions (placebos, blind evaluations) and548 uncontrolled studies. He concluded that the likelihood of obtaining statistically significant results was

comparable for both controlled and uncontrolled studies. However, he did note that when "professionalopinion" or behavior ratings scales were the dependent variable (as compared with standardized psycho-logical tests or direct measurements of behavior), favorable drug effects were much more likely to be re-

ported. Kavale (1982) examined the findings from 135 stimulant drug studies of children with MBD/hy-peractivity and concluded that the effect sizes for controlled and uncontrolled studies were comparable. Inother words, the degree of methodological rigor was not an important variable with regard to assessing stim-

216

STIMULANTS AND ADHD

ulant drug effects. Nevertheless, there really is very little systematic research on this topic in the pédiatriepsychopharmacology literature.

We have conducted a "natural" experiment comparing the results of two different procedures for evalu-ating response to drug treatment that were used in the same child psychiatry outpatient service. One pro-cedure ("controlled experiment") involved placebos, double-blind conditions, and a highly rigorous, labor-intensive assessment battery that was part of a research protocol. The second procedure ("medication clinic")did not use placebos and relied on parents to supervise the collection of drug response data. The primarygoal for comparing the findings from the controlled experiment and the medication clinic was to determineif the latter was generating information about stimulant drug effects that was consistent with a more sci-entifically rigorous procedure.

METHODS

SubjectsThe controlled experiment sample consisted of 33 boys between 5 and 13 years old (mean ± SD, 9.03 ±

2.02) who were referred for psychiatric evaluation (often at the urging of the school personnel) for behav-ior problems or academic difficulties and who participated in one of several research protocols that exam-

ined the safety and efficacy of methylphenidate for the treatment of ADD/ADHD (Gadow et al. 1990,Gadow et al. 1995, Sverd et al. 1989). The distribution of race was as follows: Caucasian = 88%, AfricanAmerican = 9%, and other = 3%. All controlled research subjects were diagnosed as meeting DSM-III(American Psychiatric Association 1980) criteria for attention deficit disorder (ADD) with or without hy-peractivity or DSM-III-R (American Psychiatric Association 1987) criteria for attention-deficit hyperac-tivity disorder (ADHD) by a board certified child psychiatrist with 20 years research and clinical experi-ence with hyperactive children. These diagnoses were made on the basis of unstructured clinical interviewswith the parent (typically the mother), a mental status examination of the child, and an extensive battery ofteacher- and parent-completed behavior rating scales for assessing ADD/ADHD symptoms as part of a com-

prehensive clinical evaluation.The medication clinic sample was comprised of 42 boys and 1 girl between 4 and 12 years old (8.20 ±

1.95) who were evaluated for behavior or learning problems in a child psychiatric outpatient clinic, were

diagnosed with ADHD, and then referred to the medication clinic for a trial of methylphenidate. Eighty-one percent of this sample were Caucasian, 7% were African American, and 12% were of other racial/eth-nic backgrounds.

The ADHD measures employed in both the controlled experiment and medication clinic diagnosticevaluations included the IOWA Conners Teacher's Rating Scale (Loney and Milich 1982), the Primaryand Secondary Symptom Checklist (PSSC) (Loney 1984) completed by the mother, and the parent andteacher versions of the Child Symptom Inventory-3R (CSI-3R) (Gadow and Sprafkin 1994). TheCSI-3R is a rating scale based on DSM-III-R (American Psychiatric Association 1987) symptoms for13 psychiatric disorders and is used as a symptom screening device. Items are scored on a 2-point scale(never or sometimes = 0; often or very often =1). Cutoff scores are based on the number of symp-toms indicated by DSM-III-R as being necessary for a diagnosis but are used only to indicate the pres-ence and the relative frequency of these behaviors in the school and home setting. The CSI-3R hasbeen shown to be a useful clinical tool for guiding a comprehensive clinical interview (Grayson andCarlson 1991). Early in the study, some of the children were evaluated using a DSM-III version ofthis checklist (Gadow and Sprafkin 1994), which was designed, structured, and scored in a similarmanner.

All children from the medication clinic and all but 1 boy from the controlled experiment sample scoredabove cut-off on at least one of the two teacher measures of hyperactivity/ADHD: the Inattention-Overactivity subscale of the IOWA Conners Teacher's Rating Scale (clinical cut-off score >7) and theADHD index of the CSI-3R (score >7). The boy who was not rated as hyperactive by his teacher was

above cut-off on both parent measures of hyperactivity/ADHD. With the exception of 1 boy in the con-

trolled experiment sample and 2 children in the medication clinic sample, all of the children scored above

217

SPRAFKIN AND GADOW

cut-off on at least one of the two parent measures of hyperactivity/ADHD: the Hyperactivity subscale fromthe PSSC (score >2) and the ADHD index of the CSI-3R (score >7). The means and SDs for each of thesehyperactivity/ADHD measures for each sample is shown in Table 1. T tests revealed no significant (p <

0.05) differences between the controlled experiment and medication clinic samples on these diagnostic mea-

sures of hyperactivity/ADHD.In addition to ADD/ADHD, both the controlled experiment and the medication clinic samples were co-

morbid for a variety of different disorders. The majority of the children were experiencing serious academicproblems in school, and 73% of the research sample and 60% of the clinic sample were receiving some

form of special education (full-time or resource room). However, the IQs of the children in both the con-

trolled experiment (105 ± 15, range = 82-133) and medication clinic samples (102 ± 13, range = 78-132)were virtually all in the normal range. The presence of concomitant disruptive behavior disorders was as-

sessed using the Aggression subscale of the IOWA Conners Teacher's Rating Scale, the Aggression sub-scale of the Primary-Secondary Symptom Checklist, and the CSI-3R. Eighty-eight percent of the controlledexperiment sample and 61% of the medication clinic sample scored above cut-off on at least one of thethree teacher measures of aggression: the Aggression subscale of the IOWA Conners Teacher's Rating Scale(clinical cut-off >4), the Oppositional Defiant Disorder index of the CSI-3R (score >4), or the ConductDisorder index of the CSI-3R (score >2). The means and SDs for each of these aggression measures areshown in Table 1. T tests revealed no significant differences between the controlled experiment and med-ication clinic samples. However, one measure yielded a marginally significant difference; the IOWA-Aggression score was slightly higher for the controlled experiment than medication clinic sample, p < 0.08.

Similarly, 77% of the controlled experiment sample and 56% of the medication clinic sample were abovecut-off on at least one of the parent measures of aggression: the Aggression subscale of the PSSC (score>2), the Oppositional Defiant Disorder index of the CSI-3R (score >4), or the Conduct Disorder index ofthe CSI-3R (score >2). T tests on the means (shown in Table 1) of the two samples did not reveal any sig-nificant differences on these measures.

In the controlled experiment sample, 18 of the 33 children had comorbid Tourette's disorder or chronicmotor tic disorder (see Gadow et al. 1992, Gadow et al. 1995, Sverd et al. 1989). There was no systematictic status evaluation for the medication clinic sample, although it is well documented that approximatelyone-third of clinic-referred ADHD children exhibit one or more motor or vocal tics (Conners 1970, Muniret al. 1987).

Table 1. Sample Characteristics—Means (and SDs)

Controlled MedicationVariable experiment clinic p <

Age (years) 9.0(2.0) 8.2(1.9) 0.08IQ 105.4 (15.2) 102.3 (12.9) NSIOWA-IO 11.2(2.7) 11.3(2.4) NSIOWA-A 8.5 (4.5) 6.6 (4.6) 0.08MOMS-H 3.9(1.3) 3.8(1.1) NSMOMS-A 3.0(1.6) 2.5(1.8) NSCSI-3R:

ADHD (Teacher) 9.7(4.1) 9.1(3.3) NSADHD (Parent) 10.1 (2.3) 9.6 (2.6) NSODD (Teacher) 3.6(3.2) 3.1(3.1) NSODD (Parent) 3.9 (2.4) 3.7 (2.6) NSCD (Teacher) 1.1(1.3) 1.2(1.5) NSCD (Parent) 1.1(1.2) 1.4(1.7) NS

ODD = oppositional defiant disorder, CD = conduct disorder, NS =

not significant.

218

STIMULANTS AND ADHD

Children who exhibited any of the following characteristics were excluded from consideration from bothsamples: children who were believed to be too phychiatrically ill (dangerous to self or others) or who were

psychotic or who had a seizure disorder, major organic brain dysfunction, major medical illness, medical or

other contraindication to stimulant medication, or pervasive developmental disorder. None of the children ineither sample were receiving any concurrent medications during the course of the medication evaluation.

MedicationControlled experiment. All subjects received either two (n = 14; 0.3 and 0.6 mg/kg) or three (n = 19;

0.1, 0.3, and 0.5 mg/kg) doses of methylphenidate and placebo for 2 weeks each as part of their involve-ment in a research protocol. For ease of administration, individual mg doses were rounded off to the near-

est 5 mg. The upper limit for the largest dose was either 20 or 25 mg, depending on the research protocol.The mean unit doses were 9.6 ± 3.27 (0.3 mg/kg) and 16.3 ± 4.8 (0.5/0.6 mg/kg). Dose schedules were

counter-balanced and assigned on a random basis. Medication (and placebo) was administered twice daily,approximately 3.5 hours apart, 7 days a week. In most cases, the morning dose was given by parents be-fore the child left for school and the noon dose was administered by the school nurse. Medication and iden-tically matching placebos were dispensed to the parents and school nurses in dated, sealed envelopes at

2-week intervals. Parents and nurses were asked to return unused medication envelopes, which allowed us

to monitor compliance. Parents, teachers, patients, observers, and physician were "blind" to the identity ofthe dose conditions (though in 3 cases, the physician was not blind).

Medication clinic. All subjects received three doses of methylphenidate (0.1, 0.3, and 0.5 mg/kg) for 1week each following a 1-week no-treatment baseline. Doses were rounded off to the nearest 2.5 mg, andparents were instructed on how to split the 5-mg pill using a splitting device that could be purchased fromtheir local pharmacy. The mean unit doses were 8.1 ± 2.89 mg (0.3 mg/kg) and 13.1 ± 3.45 mg (0.5 mg/kg).The dose schedule followed a fixed-dose increment design (no medication, 0.1, 0.3, and 0.5 mg/kg). Eachnew condition was started on a Saturday to allow the parents to observe the child on a particular dose be-fore the school trial. Medication (and placebo) was administered twice daily, approximately 4 hours apart,7 days per week. On school days, the parents administered the morning dose before the child left for school,and the school nurse administered the second dose at mid-day.

ProcedureControlled experiment. Once medication evaluation began, the children and their mothers were required

to come to the clinic at 2-week intervals for clinical evaluation and to receive the next 2-week allotment ofmedication. At each clinical evaluation, the mother was interviewed by a physician, and the child's clini-cal status was assessed. Parents were asked to provide behavioral and side-effect ratings for their child'sbehavior during Saturday and Sunday, and the results of these ratings were discussed with the physician.Each child was also observed in school on approximately 3 to 4 days for each dose condition (2 days perweek for each 2-week treatment condition). Each subject was observed while engaged in academic seat-work in the classroom and during lunch and recess (20 to 30 minutes per setting per day).

Each child's classroom teacher completed several behavior rating scales twice per week or four timesper condition on the days that the observers were in the school. Teachers were asked to base their ratingson the child's behavior during the mornings. The classroom observers were responsible for distributing andretrieving the teacher-completed behavior rating scales.

Medication clinic. During their first medication clinic visit, each child was examined (height, weight,blood pressure, heart rate), and parents were asked about their child's medical history by the medicationclinic physician (a child psychiatry fellow). Parents were given a detailed instruction sheet that containedtheir child's medication schedule for the following 4 weeks (no treatment baseline; 0.1 mg/kg; 0.3 mg/kg;0.5 mg/kg). The duration of each dose condition was 1 week. The instructions also outlined the evaluationprocedures: (1) All new dose conditions were started on Saturdays, and parents were to complete a ratingscale on their child's behavior on Saturdays and Sundays; (2) Teachers were to be given a letter explain-ing the goals of the evaluation, the need for them to be unaware of the medication conditions, and the re-

quest to complete the attached rating scales twice per week, preferably on Tuesdays and Thursdays; (3) The

219

SPRAFKIN AND GADOW

school nurse was to be given a consent letter that authorized the nurse to administer the mid-day doseaccording to the schedule provided and requested that she not inform the classroom teacher of the doseschedule; (4) The parents were asked to schedule a brief appointment after the 0.3 mg/kg dose was startedto discuss with the physician any side effects and to receive the instruction to continue the medication eval-uation. None of the side effects reported necessitated termination of the evaluation.

The parents were given a prescription that provided them with sufficient pills for the entire evaluation.They were instructed to divide all the pills into two containers, one for themselves and one for the schoolnurse. No placebos were used, and therefore, the parents, child, school nurse, and the physician were notblind to the dose condition. However, an effort was made to keep the teachers blind to the dose conditionby (1) Giving them a form letter explaining the purpose of the evaluation and the importance of them be-ing unaware during any given week of the medication status; (2) Informing the nurses in writing that we

appreciated their not revealing the medication condition to the teacher during the evaluation; (3) Instructingthe parent in person to not reveal the medication condition; (4) Arranging the mid-day dose so it coincidedwith the lunch period, which was most often one in which the classroom teacher was not present; (5) Havingthe child go to the nurse during the baseline week so that the no-medication periods were less obvious tothe teachers.

Measures

The IOWA Conners Teacher's Rating Scale contains 10 items, 5 of which form the Inattention-Overactivity subscale and 5 of which form the Aggression subscale (Loney and Milich 1982). TheInattention-Overactivity. scale consists of 5 items: (1) fidgeting, (2) hums and makes other odd noises,(3) excitable, impulsive, (4) inattentive, easily distracted, and (5) fails to finish things he/she starts, shortattention span. In the authors' version, Item 1 is phrased "constantly fidgeting," which is the wordingin the Abbreviated Teacher Rating Scale (Conners 1973). The IOWA Conners has been shown to dif-ferentiate between the behavioral dimensions of inattention/hyperactivity and negativistic behavior(Atkins et al. 1989, Halperin et al. 1990, Loney and Milich 1982, Milich and Fitzgerald 1985, Milichand Landau 1988, Nolan and Gadow 1994). The Inattention-Overactivity scale is significantly corre-

lated with chart ratings (based on notes in child's clinical record) of hyperactivity symptoms (but not

aggression) and off-task behavior during academic seatwork activity in both simulated classroom andpublic school settings. The Inattention-Overactivity scale is temporally stable and is able to discrimi-nate between clinic-referred and control boys. The Aggression scale contains 5 items that primarilyassess oppositional behavior: (1) quarrelsome, (2) acts "smart," (3) temper outbursts, explosive and un-

predictable behavior, (4) defiant, and (5) uncooperative. The Aggression scale correlates significantlywith chart ratings of aggression (but not hyperactivity) and direct observations of noncompliant behav-ior and peer aggression.

The Peer Conflict Scale (Gadow 1986a, 1993, Nolan and Gadow 1994) contains 10 items that assess ag-gressive interactions between children. Each item is rated on a 4-point scale (Not at all = 0; Very much =

3). Research indicates that this scale is sensitive to stimulant drug effects (Gadow et al. 1990, Gadow et al.1992a, Gadow et al. 1995), correlates well with direct observations of peer aggression (Nolan and Gadow1994), and has adequate reliability (Nolan and Gadow 1994).

Side effects were assessed using the Stimulant Side Effects Checklist (SSEC) (Gadow 1986b, 1993). TheSSEC contains 13 side-effect items rated on a 4-point scale (0 = Not at all; 3 = Very much). For scoringpurposes, items are grouped as follows: Mood index (4 items), Attention-Arousal index (4 items), SomaticComplaints index (3 items), and Unusual Motor Movements (1 item). The SSEC has proven to be a usefultool for gathering side effect information from parents and teachers and has been shown to be sensitive tostimulant drug effects (Gadow et al. 1995).

Both parents and teachers completed the aforementioned measures. However, because parent-completedratings of medication effects are not as sensitive as teacher-completed ratings in detecting differences be-tween doses (Gadow et al. 1995), only the teacher data are presented here. The one exception is the SomaticComplaints index of the SSEC, which only the parents completed because 2 items pertained to observa-tions that they would generally make (i.e., changes in sleep and appetite).

220

STIMULANTS AND ADHD

RESULTS

Total sampleA series of 3 X 2 mixed-design analyses of variance (ANOVAs) in which factors were Dose (No med-

ication baseline or Placebo, Low dose = 0.3 mg/kg, Moderate dose = 0.5/0.6 mg/kg) and Protocol(Controlled Experiment vs. Medication Clinic) were performed on the Inattention-Overactivity andAggression subscales of the IOWA Conners Teacher's Rating Scale, Peer Conflict Scale, and side effects(SSEC) indices. Of primary interest was whether the teacher's perceptions of the children's responses tomedication were different for children in the controlled experiment compared with children in the medica-tion clinic. This would be evident by a significant Dose X Protocol interaction. None of the outcome mea-sures or side-effect measures yielded such an interaction. The children treated in the controlled experimentand in the medication clinic both responded to methylphenidate in a parallel manner (see Figures 1-3).

Considering the documented efficacy of methylphenidate, it is not surprising that of the drug evaluationmeasures, there was a significant main effect of dose on all three outcome measures, F(2, 146) = 62.3,50.8, and 26.4, for the IOWA Inattention-Overactivity scale, the IOWA Aggression scale, and the PeerConflict Scale, respectively (p < 0.001 for all). These means were analyzed with a series of Newman-Keuls(p < 0.05) tests. For the IOWA Inattention-Overactivity scale, the mean baseline/placebo scores (8.79)were significantly higher than the low (4.90) and moderate (4.09) dose scores, but scores for the two med-ication conditions were not significantly different from one another. For the IOWA Aggression scale, themean baseline/placebo scores (5.02) were significantly higher than the low (2.37) and moderate (1.50) doses,and scores for the moderate dose were significantly lower than the low dose. For the Peer Conflict Scale,the mean baseline/placebo scores (5.92) were significantly higher than the low (2.95) and moderate (1.93)doses, but the difference between the two medication conditions was not significant.

cu 8ooin>, 7

es 61—CD>

9 5o

'•*—'

I 4CO

cco<i>

3h

^r

• Experiment Clinic

Baseline/Placebo 0.3Dose (mg/kg)

0.5/0.6

FIG. 1. Mean teacher ratings on the Inattention-Overactivity scale of the IOWA Conners Teacher's Rating Scale as

a function of methylphenidate dose for the medication clinic versus controlled experiment samples.

221

SPRAFKIN AND GADOW

• Experiment Clinic

Vi_i_.3 0.5/0.6

Dose (mg/kg)FIG. 2. Mean teacher ratings on the Aggression scale of the IOWA Conners Teacher's Rating Scale as a function ofmethylphenidate dose for the medication clinic versus controlled experiment samples.

The parallel ANOVAs on the measures of side effects (SSEC indices for Mood, Attention-Arousal,Somatic Complaints, Unusual Motor Movements) yielded a significant main effect of dose for Mood [F(2,136) = 20.6, p < 0.0001] and Somatic Complaints [F(2, 124) = 13.4, p < 0.0001]. Newman-Keuls (p <

0.05) tests revealed that negative mood was significantly lower on the low and moderate doses than base-line/placebo, and mood ratings for the moderate dose were significantly better than the low dose. Parent-rated somatic complaints were significantly lower during baseline/placebo than for the low and moderatedoses, and treatment with the moderate dose was associated with significantly more side effects than thelow dose (see Table 2). The other two side-effect indices did not yield dose effects.

Separate samplesThe focus of this study was to determine whether the two methodologies would yield the same results.

In an exploratory set of analyses, one-way repeated-measure ANOVAs were performed separately on thedata for the controlled experiment and for the medication clinic. For the IOWA Inattention-Overactivityscale and Peer Conflict Scale scores, there were significant dose effects (p < 0.001 for both samples), andsubsequent Newman-Keuls tests (p < 0.05) yielded identical results for both samples; namely, both lowand moderate doses improved symptoms compared with no medication/placebo, but differences betweenthe 0.5/0.6 mg/kg dose and the 0.3 mg/kg dose were not significant. For the IOWA Aggression scale, al-though both assessment methods yielded a significant dose effect (p < 0.001 for both samples), the posthoc analyses were somewhat different. Using the medication clinic sample alone, one would have concludedthat low and moderate doses were equally effective in reducing oppositional/aggressive behavior and thatboth doses were superior to the no-medication baseline. However, for the controlled experiment, the analy-ses showed that the moderate dose was more effective than the low dose, which was more effective thanplacebo. When aggression was the target symptom, the findings from the controlled experiment support theclinical superiority of the moderate dose, whereas the results from the medication clinic assessment indi-cate that low and moderate doses are equally effective.

oo

COcoC/5COCD1—

<<

5.5

5.0

4.5

4.0

3.5

3.0

2.5

2.0

1.5

1.0

0.5

0.0Baseline/Placebo

222

STIMULANTS AND ADHD

6.5

6.0

5.5 |-5.0

S 4.5Oo

2 4.0O

**—

§ 3.5Oo3 3.0a>

0_C 2.5COa>2 2.0

1.5

1.0

0.5

0.0 Baseline/ Placebo 0.3Dose (mg/kg)

0.5/0.6

FIG. 3. Mean teacher ratings on the Peer Conflict Scale as a function of methylphenidate dose for the medicationclinic versus controlled experiment samples.

With regard to side effects, both protocols yielded significant dose effects for the Somatic Complaintsindex, and the means followed the same pattern (Table 2); namely, higher doses are associated with moresomatic complaints. Newman-Keuls tests (p < 0.05) revealed that for the medication clinic sample, the0.5/0.6 mg/kg dose was associated with significantly more somatic complaints than the 0.3 mg/kg dose orno medication, and the low dose produced significantly more somatic complaints than no medication.

Table 2. Means (and SDs) of the Subscaleson the Stimulant Side Effect Checklist

Condition Mood Somatic

Overall sampleBaseline/placebo0.3 mg/kg0.5 mg/kg

Controlled experimentPlacebo0.3 mg/kg0.5/0.6 mg/kg

Medication clinicBaseline0.3 mg/kg0.5 mg/kg

2.63 (2.63)1.61 (1.70)1.13(1.50)

2.52 (2.40)1.88 (1.87)1.41 (1.71)

2.72 (2.82)1.43 (1.59)0.94(1.36)

0.76 (1.01)1.22 (1.34)1.84(1.66)

1.13(1.06)1.28 (1.32)1.86(1.62)

0.57 (0.93)1.15(1.36)1.81 (1.70)

223

SPRAFKIN AND GADOW

However, Newman-Keuls analyses for the controlled experiment sample yielded more overlap betweendoses. Treatment with the moderate dose was associated with significantly more somatic complaints thanplacebo (but no more than the low dose), and the low dose and placebo produced equivalent ratings for so-matic symptoms. This pattern of results indicates that knowing that a child is taking active medication in-creases the likelihood of attributing somatic complaints to the medication.

For the Mood index, there were significant dose effects for both assessment protocols, and the means

(Table 2) followed the same pattern, indicating that affect improved with increasing dose. However,Newman-Keuls tests (p < 0.05) indicated that for the controlled experiment sample, only the 0.5/0.6 mg/kgdose was associated with better mood ratings than placebo, whereas the analysis of the medication clinicdata suggested that both doses were equally effective and superior to no medication.

The analyses of the scores for the Attention-Arousal index and Unusual Motor Movement index did not

yield significant dose effects for either protocol.

DISCUSSION

The finding that a rigorously controlled research protocol examining the dose effects of methylphenidatein children with ADD/ADHD generates highly similar data (i.e., virtually identical findings for 8 out of 9post hoc comparisons on the behavior rating scales) to routine clinical medication trials conducted in a com-

munity-based child psychiatry outpatient service would be inspirational were it not for the fear that it mightbe erroneously misinterpreted as a blessing for shoddy research and forgiveness for castigating uncontrolledstudies in the past. Lest the message of this report be misconstrued, these findings must not be interpretedas suggesting that investigators should abandon placebo controls or double-blind conditions. One can eas-

ily imagine a number of situations where one would predict the outcome to have been very different, mostnotably where drugs produce more subtle treatment effects (i.e., small effect sizes) or in the case of symp-toms that are much more difficult to evaluate (e.g., internalizing disorders) or that care providers are lesslikely to notice. Furthermore, our study evolved out of naturally occurring events and thus was not designedspecifically to address the issue of double-blind versus open evaluation methods. As previously noted, therewere differences between the two samples (age, special education status, level of aggression, tic status)which would likely have been remedied in a random assignment design. Despite these limitations, there are

at least three situations in which consideration might be given to exploring the utility of alternative drugassessment procedures: (a) to expedite data collection for use in shaping the design of controlled trials fornewly approved drugs, (b) to establish procedural standards for case reports, and (c) to obtain drug responseinformation in community-based clinical settings.

First, given the relatively long delay between the time a new drug (not approved and labeled for pédi-atrie use by the Federal Drug Administration) is actually prescribed with some regularity and the appear-ance of several controlled studies, it may be useful to explore the possibility of developing standardizeddata collection procedures for use with a network of participating physicians, a prospect that is becomingmore feasible with the advent of managed care. Participants would evaluate therapeutic and untoward ef-fects with a standard battery of drug-response measures within the context of a nonblind clinical protocol.These data would then be published with appropriate caveats and would serve as a guideline for develop-ing more sophisticated research protocols.

Second, very little attention has been given to the potential role that data-based case reports can play inshaping directions for future research. Even if no-treatment control conditions are warranted only for drugswith relatively large effect sizes and for externalizing symptoms, there would still be plenty of applicationsof this methodology in improving the quality and empirical and clinical utility of case reports. In an edito-rial published in this Journal, Wender (1993) expressed a similar notion.

Third, it has been commented on for many years that procedures for evaluating response to medicationin "real world" clinical settings leave a lot to be desired (Bennett and Sherman 1983, Bosco and Robin1976, Brülle et al. 1983, Copeland et al. 1987, Gadow 1975, 1976, 1978, Jensen et al. 1989, Loney andOrdona 1975, Robin and Bosco 1973, Sandoval et al. 1976, Sindelar and Meisel 1982, Solomons 1973,Weithorn and Ross 1975). In general, anecdotal reports from parents and occasionally teachers serve as the

224

STIMULANTS AND ADHD

primary basis for making decisions about effectiveness, dose, and side effects in everyday clinical situa-tions. The situation in community-based child and adolescent mental health facilities is not much better,but several decades of litigation have produced dramatic changes in medication-related assessment proce-dures in residential facilities for individuals with developmental disabilities (Sprague 1982, Sprague andGalliher 1988).

One reason for this circumstance is that research on applied issues in evaluating response to medicationin everyday clinical situations is one of the most neglected topics in psychopharmacology and one thatrarely receives research grant support. There is an enormous need for psychometrically sound, cost-effec-tive, and ecologically valid instruments for assessing response to psychotropic medication (to include be-havioral and somatic toxicity) and reliable procedures for determining efficacy at the individual patientlevel. In addition, however, there is the need for practical procedures for obtaining the data once the in-strumentation is available.

Despite the aforementioned problems, there is some research on ways to improve the clinical manage-ment of children receiving medication for the treatment of behavior disorders (e.g., Barkley et al. 1988,Fine and Jewesson 1989, Gadow and Nolan 1993, Gadow et al. 1991, Gadow et al. 1992b, Ottinger et al.1985, McBride 1988, Neisworth et al. 1976, Pelham and Hoza 1987, Safer and Allen 1976, Sprague andSleator 1975, Swanson and Kinsbourne 1978, Varley and Trupin 1983, Yellin and Greenberg 1981). Mostof these medication assessment procedures are designed for specialized clinics (typically university-basedresearch facilities) and are poorly suited for community-based clinicians, who would have to overcome enor-

mous obstacles to achieve the elegance of controlled trials (i.e., placebo controls and double-blind conditions).Given this situation, if one assumes that only rigorously controlled medication evaluations yield clinicallyvalid findings, there does not appear to be an incentive to collect systematic drug response information, es-

pecially if it entails modifying research-based methodologies for routine use in nonspecialized clinic settings.However, if it could be shown that a no-treatment baseline is a reasonable substitute for a placebo in certainsituations and that the failure to establish "blind" conditions does not invalidate teacher ratings (e.g., Sprafkinand Gadow 1993), then clinicians may be more willing to try standardized data collection procedures.Nevertheless, until the findings from this investigation are replicated and the situations and patient character-istics that indicate its feasibility are documented, our recommendation for the use of the medication clinic pro-cedure remains tentative.

ACKNOWLEDGMENT

This study was supported, in part, by a research grant from the Tourette Syndrome Association, Inc. andP.H.S. Grant No. MH 45358 from the National Institute of Mental Health.

The authors wish to thank Gaye Carlson, John Pomeroy, and Jeffrey Sverd for providing valuable clin-ical services related to the conduct of the medication evaluations, and Edith Nolan for assisting us with datacollection and analyses.

REFERENCES

Aman MG, Singh NN: A critical appraisal of recent drug research in mental retardation: The Coldwater studies. J MentalDeficiency Res 30:203-216, 1986American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, Third Edition. Washington,D.C., American Psychiatric Association, 1980

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, Third Edition-Revised.Washington, D.C., American Psychiatric Association, 1987Atkins MS, Pelham WE, Licht MH: The differential validity of teacher ratings of inattention/overactivity and aggres-sion. J Abn Child Psychol 17:423^135, 1989

Barkley RA, Fischer M, Newby RF, Breen MJ: Development of a multimethod clinical protocol for assessing stimu-lant drug response in children with attention deficit disorder. J Clin Child Psychol 17:14-24, 1988

225

SPRAFKIN AND GADOW

Bennett FC, Sherman R: Management of childhood "hyperactivity" by primary care physicians. J Dev Behav Pediatrics4:88-93, 1983Bosco JJ, Robin SS: Ritalin usage: A challenge to teacher education. Peabody J Educ 53:187-93, 1976Brülle AR, Barton LE, Foskett JJ: Educator/physician interchanges: A survey and suggestions. Education and Trainingof the Mentally Retarded 18:313-317, 1983Cohen MR, Nagel E: An Introduction to Logic and Scientific Method. New York, Harcourt Brace, 1934, p 256Conners CK: Symptom patterns in hyperkinetic, neurotic, and normal children. Child Dev 41:667-682, 1970

Conners CK: Rating scale for use in drug studies with children (Special Issue: Pharmacotherapy of children).Psychopharmacol Bull 9 (Special Issue):24-84, 1973

Copeland L, Wolraich M, Lindgren S, Milich R, Woolson R: Pediatricians' reported practices in the assessment andtreatment of attention deficit disorders. J Dev Beh Pediatr 8:191-197, 1987Fine S, Jewesson B: Active drug placebo trial of methylphenidate: A clinical service for children with an attentiondeficit disorder. Canad J Psychiatry 34:447-449, 1989

Gadow KD: Pills and preschool: Medication usage with young children in special education. Paper presented at themeeting of the Illinois Council for Exceptional Children, Chicago, IL, October 1975

Gadow KD: Psychotropic and anticonvulsant drug usage in early childhood special education programs: I. Phase one:

A preliminary report: Prevalence, attitude, training and problems. Paper presented at the annual meeting of the Councilfor Exceptional Children, Chicago, April 1976 (ERIC Document Reproduction Service No. ED 125 198)Gadow KD: Drug treatment with children in programs for the trainable mentally handicapped. Paper presented at themeeting of the Council for Exceptional Children, Kansas City, May 1978 (ERIC Document Reproduction Service No.ED 153 398)Gadow KD: Relative efficacy of pharmacological, behavioral, and combination treatments for enhancing academic per-formance. Clin Psychol Rev 5:513-533, 1985

Gadow KD: Peer Conflict Scale. Stony Brook (NY), State University of New York, Department of Psychiatry, 1986a(available from author)Gadow KD: Stimulant side effects checklist. Stony Brook (NY), State University of New York, Department ofPsychiatry, 1986b (available from author)Gadow KD: A school-based medication evaluation program. In: Handbook of Hyperactivity in Children. Edited byMatson JL. Needham Heights (MA), Allyn and Bacon, 1993, pp 186-219

Gadow KD, Nolan EE: Practical considerations in conducting school-based medication evaluations for children withhyperactivity. J Emotional Behav Disorders 1:118-126, 1993

Gadow KD, Nolan EE, Paolicelli LM, Sprafkin J: A procedure for assessing the effects of methylphenidate on hyper-active children in public school settings. J Clin Child Psychol 20:268-276, 1991

Gadow KD, Nolan EE, Sverd J: Methylphenidate in hyperactive boys with comorbid tic disorder: II. Short-term be-havioral effects in school settings. J Am Acad Child Adolesc Psychiatry 33:462—471, 1992a

Gadow KD, Nolan EE, Sverd J, Sprafkin J, Paolicelli L: Methylphenidate in aggressive-hyperactive boys: I. Effects on

peer aggression in public school settings. J Am Acad Child Adolesc Psychiatry 29:710-718, 1990

Gadow KD, Pomeroy JC, Nolan EE: A procedure for monitoring stimulant medication in hyperactive mentally retardedschool children. J Child Adolesc Psychopharmacol 2:131-143, 1992b

Gadow KD, Sprafkin J: Child symptom inventories manual. Stony Brook (NY), Checkmate Plus, 1994

Gadow KD, Sverd J, Sprafkin J, Nolan EE, Ezor SN: Efficacy of methylphenidate for attention-deficit hyperactivitydisorder in children with tic disorder. Arch Gen Psychiatry 52:444—455, 1995

Gadow KD, White L, Ferguson DG: Placebo controls and double-blind conditions: Placebo theory in experimental de-sign. In Methodological Issues in Human Psychopharmacology. Edited by Gadow KD, Poling AD. Greenwich (CT),JAI Press, 1986a, pp 41-83Gadow KD, White L, Ferguson DG: Placebo controls and double-blind conditions: Experimenter bias, conditionedplacebo response, and drug-psychotherapy comparisons. In: Methodological Issues in Human Psychopharmacology.Edited by Gadow KD, Poling AD. Greenwich (CT), JAI Press, 1986b, pp 85-114

226

STIMULANTS AND ADHD

Gilliland AR, Nelson D: The effects of coffee on certain mental and physiological functions. J Gen Psychol 21:339-349,1934Gold H, Gold D: The stability of U.S.P. ether after the metal container has been opened: With preliminary results of a

clinical comparison of U.S.P. ether in large drums with ether in small cans labeled "For Anesthesia." Current ResAnesthesia Analgesia 14:92-95, 1935

Grayson P, Carlson GA: The utility of a DSM-III-R-based checklist in screening child psychiatric patients. J Am AcadChild Adolesc Psychiatry 30:669-673, 1991

Grunbaum A: The placebo concept. Behavior Res Therap 19:157-167, 1981

Halperin JM, O'Brien JD, Newcorn JH, Healey JM, Pascualvaca DM, Wolf LE, Young JG: Validation of hyperactive,aggressive, and mixed hyperactive/aggressive childhood disorders: A research note. J Child Psychol Psychiatry31:455-459, 1990

Hollingworth HL: The influence of alcohol. J Abn Psychol 18:204-237, 1923

Jensen PS, Xenakis SN, Shervette RE, Bain MW, Davis H: Diagnosis and treatment of attention deficit disorder in two

general hospital clinics. Hosp Community Psychiatry 40:708-712, 1989

Jospe M: The placebo effect in healing. Lexington (MA), Lexington Books, 1978

Kavale K: The efficacy of stimulant drug treatment for hyperactivity: A meta-analysis. J Learning Disabilities15:280-289, 1982

Loney J: A short parent scale for subgrouping childhood hyperactivity and aggression. Poster presented at the annualmeeting of the American Psychological Association, Toronto, August 1984

Loney J, Milich R: Hyperactivity inattention, and aggression in clinical practice. In: Advances in Developmental andBehavioral Pediatrics (Vol 3). Edited by Wolraich M, Routh DK. Greenwich (CT), JAI Press, 1982, pp 113-147

Loney J, Ordona TT: Using cerebral stimulants to treat minimal brain dysfunction. Am J Orthopsychiatry 45:564-572,1975

McBride MC: An individual double-blind crossover trial for assessing methylphenidate response in children with at-tention deficit disorder. J Pediatr 113:137-145, 1988

Milich R, Fitzgerald G: Validation of inattention/overactivity and aggression ratings with classroom observations. JConsult Clin Psychol 53:139-140, 1985

Milich R, Landau S: Teacher ratings of inattention/overactivity and aggression: Cross validation with classroom ob-servations. J Clin Child Psychol 17:92-97, 1988

Mill JS, A System of Logic (Vol. 1), p. 458, quoted by Cohen and Nagel, 1934, p. 256

Molitch M, Eccles JP: Effect of benzedrine sulfate on the intelligence scores of children. Am J Psychiatry 94:587-590,1937

Munir K, Biederman J, Knee D: Psychiatric comorbidity in patients with attention deficit disorder: A controlled study.J Am Acad Child Adolesc Psychiatry 26:844-848, 1987

Neisworth JT, Kurtz PD, Ross A, Madie RA: Naturalistic assessment of neurological diagnoses and pharmacologicalintervention. J Learning Disabilities 9:149-152, 1976

Nolan EE, Gadow KD: Relationship between ratings and observations of stimulant drug response in hyperactive chil-dren. J Clin Child Psychol 23:78-90, 1994

Ottinger DR, Halpin B, Miller M, Demia L, Hannemann R: Evaluating drug effectiveness in an office setting for chil-dren with attention deficit disorders. Clin Pediatr 24:245-251, 1985

Pelham WE, Hoza J: Behavioral assessment of psycho-stimulant effects on ADD children in a summer day treatment

program. In: Advances in Behavioral Assessment of Children and Families (Vol 3). Edited by Prinz R. Greenwich (CT),JAI Press, 1987, pp 3-34

Rivers WHR: The Influence of Alcohol and Other Drugs on Fatigue. London, Arnold, 1908

Robin SS, Bosco JJ: Ritalin for school children: The teacher's perspective. J School Health 43:624-628, 1973

Safer DJ, Allen RP: Hyperactive Children: Diagnosis and Medical Management. Baltimore (MD), University Park Press,1976

227

SPRAFKIN AND GADOW

Sandoval J, Lambert N Yandell W: Current medical practice and hyperactivity. Am J Orthopsychiatry 46:323-334,1976

Sindelar PT, Meisel CJ: Teacher-physician interaction in the treatment of children with behavioral disorders. Intl JPartial Hospitalization 1:271-277, 1982Solomons G: Drug therapy: Initiation and follow-up. Annals NY Acad Sei 205:335-344, 1973

Sprafkin J, Gadow KD: Four purported cases of methylphenidate-induced tic exacerbation: Methodological and clini-cal doubts. J Child Adolesc Psychopharmacol 3:231-244, 1993

Sprague RL: Litigation, legislation, and regulations. In: Drugs and Mental Retardation. Edited by Breuning SE, PolingAD. Springfield (IL), Charles C. Thomas, 1982, pp 377^114

Sprague RL, Galliher L: Litigation in psychotropic medication. In: Pharmacotherapy and Mental Retardation. Editedby Gadow KD, Poling AD. Austin (TX), PRO-ED, 1988, pp 297-312

Sprague RL, Sleator EK: What is the proper dose of stimulant drugs in children? Intl J Mental Health 4:75-104, 1975Sulzbacher SI: Psychotropic medication with children: An evaluation of procedural biases in results of reported stud-ies. Pediatr 51:513-517, 1973Sverd J, Gadow KD, Paolicelli LM: Methylphenidate treatment of attention deficit disorder with hyperactivity in boyswith Tourette syndrome. J Am Acad Child Adolesc Psychiatry 28:574-579, 1989Swanson JM, Kinsbourne M: Should you use stimulants to treat the hyperkinetic child? Modern Medicine 46:71-80,1978Ullmann RK, Sleator EK: Responders, nonresponders, and placebo responders among children with attention deficitdisorder. Clin Pediatr 25:594-599, 1986

Varley CK, Trupin EW: Double-blind assessment of stimulant medication for attention deficit disorder: A model forclinical application. Am J Orthopsychiatry 53:542-547, 1983Weithorn CJ, Ross R: "Who monitors medication?" J Learning Disabilities 8:458-461, 1975Wender PH: Methamphetamine in child psychiatry [Editorial]. J Child Adolesc Psychopharmacol 3:iv-vi, 1993White L, Tursky B, Schwartz GE (eds): Placebo: Theory, Research, and Mechanisms. New York, Guilford, 1985, pp332-350

Yellin AM, Greenberg LM: Attention-deficit disorder: Monitored data-based assessment and treatment. MinnesotaMedicine 64:487^190, 1981

Address reprint requests to:Dr. Joyce Sprafkin

Department of Psychiatry and Behavioral SciencePutnam Hall, South Campus

SUNY at Stony BrookStony Brook, NY 11794-8790

228