Psychological Bulletin1990, Vol. 107, No. 2,215-225

Copyright 1990bytheAmerican Psychological Association, Inc.0033-2909/90/S00.75

Valid Theory-Testing Meta-Analyses Further Questionthe Negative State Relief Model of Helping

Norman Miller and Michael CarlsonUniversity of Southern California

Cialdini and Fultz (1990) questioned the validity of our method (Carlson & Miller, 1987) and ob-

jected to three of our reported tests of their negative state relief (NSR) model of mood-induced

increments to helpfulness. In response, we present substantial evidence that the use of judges todefine variables is a common tool in psychology and, when used within meta-analyses, consistently

meets the relevant criteria of convergent and discriminant trait validity as well as construct validity.

In addition to the 17 nonsupportive tests of the NSR model that we initially reported, multiple new

tests based directly on the more objective criteria that Cialdini and Fultz stipulate for denning NSR

variables also fail to support their model. New data they presented to challenge the discriminant

validity of judges' ratings are shown to be based on both methodologically and conceptually flawed

procedures. Finally, they reported a (nonsignificant) positive correlation between sadness-manipula-

tion check effect sizes and helpfulness effect sizes, in support of the NSR model. When we computedthis correlation in nine different ways on the basis of more extensive sets of studies, including in

some instances new studies published since our original report, it was uniformly close to zero and

nonsignificant, thereby supporting our prior meta-analytic outcome.

Although researchers now frequently use meta-analysis tosummarize research literatures, we believe that its most impor-tant strength is its capacity to test theory. Toward this end, virtu-ally all who use it have moved away from the single mean effect-size summary statement sought by the policy maker. Instead,investigators attempt to assess the extent to which given theoret-ical variables moderate or mediate variation in the dependentmeasure of interest. For this purpose, they use the method sec-tions of studies in order to code the studies in terms of theirlevels on theoretically relevant predictors (e.g.. Dillard, Hunter,& Burgoon, 1984). Sometimes the predictor is a noncontinuouscategorical distinction that can be objectively determined onthe basis of information in the original study (e.g., Isenberg,1986; Signorella& Jamison, 1986;Steblay, 1987; Wood, 1987).Often, however, an underlying dimension is continuous and re-quires inferential judgment about subjects' subjective experi-ence. Authors sometimes make these judgments themselves(e.g., Bowers & Clum, 1988; Eagley & Steffen, 1986; Hull &Bond, 1986; Mullen et al., 1985; Steele & Southwick, 1985),but often they use additional judges (e.g., Eagly & Carli, 1981;Eagly & Crowley, 1986; Eagly & Steffen, 1986; Steele & South-wick, 1985). In some instances, authors judge a new variablenot previously noted (e.g., Bowers & Clum, 1988; Hull & Bond,1986; Mullen etal., 1985; Steele & Southwick, 1985).

To assess several explanations of why negative moods in-

The preparation of this article was facilitated by National ScienceFou ndation Grant BSN 8719439 to Norman Miller and a Haynes Foun-

dation Fellowship to Michael Carlson.

Correspondence concerning this article should be addressed to Nor-man Miller, Department of Psychology, University of Southern Califor-

nia, Los Angeles, California 90089-1061.

crease helpfulness relative to neutral moods, we used such judg-ment procedures (Carlson & Miller, 1987). The judges read por-tions of the individual method sections of the experimental lit-erature in order to make continuous ratings of the contextuallevels of theoretically relevant variables. Additionally, wesought greater analytical power by using partial correlation tocontrol statistically for the effects of other variables when assess-ing the contribution of each single variable. Our application ofthese procedures consistently supported two models of moodand helping behavior, but produced no support for the negativestate relief (NSR) model (e.g., Cialdini, Darby, & Vincent,1973). Although Cialdini and Fultz (1990) have accepted meta-analysis as a useful tool for summarizing research, they objectto our own use of judges in conjunction with it, stating that thisis a "novel methodology." Both of these procedures—the use ofjudges and partial correlation analysis—are, however, not newto psychology, and a sequential use of them does not markedlyincrease their uniqueness. Because judgments were elicited onmore than a dozen variables, however, an extensive amount ofnew data was generated in addition to the effect-size estimatesof helpfulness computed from the individual studies. Our useof the term mega-analysis to describe our research was in-tended to reflect this fact: that the quantity of new data broughtto bear on the issues exceeded that of a standard literature-sum-mary meta-analysis by a factor greater than 10:1.

In addition to questioning the general validity of judges' rat-ings, Cialdini and Fultz (1990) also challenged our repeated fail-ures to confirm any aspect of the NSR model. In contestingour application of meta-analytic procedures to the NSR model,they rejected three of our tests of it, claiming that they are inap-propriate. To support their contention that judges generate in-valid information and that consequently our analyses should

215

216 NORMAN MILLER AND MICHAEL CARLSON

not be viewed as disconfirming the NSR model, they invokefive subissucs: (a) the divergent perspective effect seen in actor-observer attributions; (b) the literature on the replication of so-cial psychological experiments by use of role-playing proce-dures; (c) our use of graduate versus undergraduate students asjudges; and (d) their own computation of the correlation be-tween sadness (as reported by subjects in experimental studies)and helpfulness.

Although their article appears to be a powerful indictmentof our work, closer examination reveals its serious flaws. Withrespect to the general criticism regarding the use of judges togenerate data, we argue that when held against the only reason-able scientific criteria for assessing the validity of judges' rat-ings, interperson agreement, and construct validity (Funder,1987; Kenny & Albright, 1987), the use of judges is a valid de-nning operation and one that has been widely used and acceptedin past research. Regarding our tests of the NSR model, we pro-pose that Cialdini and Fultz's (1990) depiction of our work isincomplete and selective in that it omits discussion of numeroustests that incorporated the conceptual features they have pro-posed as necessary in a proper test. A number of new tests oftheir model, including those they specifically mandate in theircriticism of our work, continue to highlight the essential inade-quacy of the NSR model. Finally, with respect to other, morespecific criticisms, we argue that the new data they reported areconceptually and methodologically flawed and inappropriatefor their intended use; Cialdini and Putt's invocation of the ac-tor-observer and role-playing literatures reflects importantmisunderstandings of their nature; and our use of graduate-stu-dent as opposed to undergraduate-student judges does not ex-plain our failure to find support for the NSR model.

Use of Judges to Generate Meta-Analytic Data

The use of judges to define variables in psychological researchis so extensive that it hardly needs documentation. The entirefield of decision analysis assumes a basic validity in humanjudgment (von Winterfeldt & Edwards, 1986). Among notewor-thy citations, Block's (1971) application of judgment proce-dures to the longitudinal personality data of the Institute of Hu-man Development files is particularly instructive, transcendingdivergence both in the specific dimensions studied across wavesof measurement and the specific instruments used to assessthose dimensions that were common across measurementwaves by the simple expedient of having judges use Q sorts totransform individual measures into a common metric on a uni-form set of dimensions. The extensive work of Rosenthal andhis colleagues on nonverbal communication provides anotherwell-implemented example (e.g., Rosenthal, 1987; Rosenthal,Hall, DiMatteo, Rogers, & Archer, 1979; Scherer, Scherer, Hall,& Rosenthal, 1977). Other instances of the successful use ofjudges can be found in the prediction of behavior in industrialand organizational settings (e.g., Hollander, 1965; Mayfield,1972; Waters & Waters, 1970).

Within the context of meta-analysis, too, a striking feature isthe substantial construct validity seen in judges' ratings. In anappendix concerned specifically with this issue, we presentedevidence of criterion-related validitv as well as construct valid-

ity (Carlson & Miller, 1987). With the exception of the variablesof the NSR model, virtually all of the theoretically relevant con-structs defined by our judges' ratings were significantly relatedto helpfulness in the expected, theoretically meaningful way(e.g., guilt, self-absorption, objective self-awareness, responsi-bility for the mood-lowering event, target of the negative event,legitimacy-urgency and salience of the helping request, and an-ger and frustration). This basic outcome was replicated in a fur-

ther study concerned with the effects of positive mood on help-fulness (Carlson, Charlin, & Miller, 1988). Table 1 lists variablestheoretically related to mood and helpfulness (other than theNSR variables in the case of negative mood, which in each in-stance were unrelated to helping), gives a citation for each vari-able that experimentally confirms its theoretical importance,and presents the correlation between judges' ratings of the vari-able and helpfulness effect sizes (evidence that the theoreticalvariable, as defined by judges' ratings, affects mood-based help-fulness in a theoretically predicted manner). Note that guilt andself as target of the mood-inducing event interact with type ofmood as theoretically expected and thus evidence the additionalfeature of discriminant construct validity.

Table 2 presents parallel evidence for ratings elicited by oth-ers. Within it, the majority of judgmentally defined variables(approximately 85%) show construct validity. In their critique,Cialdini and Fultz (1990) did not account for how such substan-tial convergent evidence for the construct validity of judges' rat-ings in meta-analytic studies could emerge if judges cannot val-idly rate variables.

It might be argued that although judges (persons) have nor-mative or reliable views of the world, such views neverthelessreflect erroneous assumptions about the relation betweenmoods and behavior. If so, ratings of mood should show consis-tent relations with judgments about expected behavior, butthese relations should be incorrect. However, such correlationsdo not conceptually parallel our procedure, which insteadshows that judgments of subjects' subjective experiences are re-lated to the actual behavior of those subjects. The correlationspresented in Table I confirm prior experimental work and,taken together, consistently support the basic validity of ourjudgment procedure. They provide an integration that exceedsthat of the individual studies. Moreover, such validity could nothave been obtained had the ratings lacked reliability.1 To ac-count for this lack of support for their model, NSR theoristsmust argue that key NSR variables, sadness and distastefulnessof the helping act, are a unique subset about which judges make

useless ratings.2

1 At an earlier point, Cialdini (personal communication, 1984) raised

the argument that lower reliabilities for the NSR model's variables(compared with those of other models) may have contributed to its lack

of confirmation. Averaged, however, the reliabilities of the NSR vari-

ables match or exceed those of other tested models that were supported

by our procedure.2 Arguing from analogy, Cialdini (personal communication, 1984)

implied that there might be differential validity for judgments on emo-

tional dimensions as opposed to cognitive dimensions. In his view, thevariables of the other models we tested are primarily cognitive, whereas

sadness is emotional. He then evoked this distinction to account for

META-ANALYTIC VALIDITY QUESTIONS NSR MODEL 217

Table 1

Construct Validity: Correlations Between Judgmentally Rated Theoretical Variables and Helping

Effect-Size Estimates Within Two Mega-Analytic Studies

Study VariableExample of experiment providing

theoretical predictionPredictedrelation

Association withhelping effect sizes"

Negative mood and helping OSA X salience of request(Carlson & Miller, 1987)

Rogers, Miller, Mayer, & Duval(1982)

Gibbons & Wicklund (1982)

Positive

Positive mood and helping(Carlson, Charlin,& Miller, 1988)

Responsibility for causingnegative event

Guilt

Other (vs. self) as target ofnegative event

Self-absorption (given anonsalient request for help)

Pleasantness of the helping task

Extent to which positive eventenhances estimate of humannature or kindness

Oneself (vs. another) as targetof positive event

Guilt

Rogers etal. (1982)

Carlsmith & Gross ( 1 969);McMillen & Austin (1971)

Thompson, Cowan, & Rosenhan(1980)

McMillen, Sanders, & Solomon(1977)

Isen & Simmonds (1978); Forest,Clark, Mills, &Isen( 1979)

Holloway, Tucker, & Hornstein(1977); Hornstein, Lakind,Frankel,&Manne(1975)

Rosenhan, Salovey, & Hargis (1981)

Cunningham, Steinberg, & Grev(1980)

Positive

Positive

Positive

Negative

Positive

Positive

Positive

Negative

,4l,p<.001

.50,p<.001

.30,p<.01

-.27,p<.05

.34,/j<.015

.3I,/?<.015

.49,/x.OOI

-.44,p<.OOI

' Partial correlation (controlling other theoretical and control variables) of rated theoretical variable with helping effect sizes.

Problems in Deriving NSR Predictions

Cialdini and Fultz (1990) objected to the manner in which

we tested the major variables of the NSR model (viz., subjects'

age, distastefulness of the helping act, and sadness). This objec-

tion centers in the first two instances on the proper content do-

main of the model, namely, whether it refers solely to the mood

of sadness or to other negative moods as well. We confess our

own confusion on this issue, but attribute it to the key presenta-

tions of the model. On the one hand, NSR theorists have stated

repeatedly that their theory refers specifically to the single nega-

tive affect of sadness (e.g., Cialdini, Baumann, & Kenrick,

1981; Cialdini etal., 1973; Cialdini & Kenrick, 1976; Cialdini

et al., 1987; Manucia, Baumann, & Cialdini, 1984). In stark

contrast, but in harmony with the more global title negative

state relief, the model is said to apply to a "general negative

affective state" throughout a 1973 review (Cialdini et al., 1973,

pp. 503, 505, 512, and 513). Moreover, numerous outcomes

concerned with negative moods other than sadness are cited

why other models received support, whereas the NSR model did not.

Although differential validity in the judgments of emotions versus cog-nitions is possible and worthy of further research, this cannot accountfor the negative outcomes for the NSR model. Two of its three majorvariables, subjects' age and distastefulness of the helping task, are notprimarily emotional. Age can be directly coded from studies. Distaste-

fulness requires subjects (and judges) to make a cost-benefit assess-ment—a cognitive task. On the other hand, many other judged variablesthat enter into theoretical models for which we provide convergent, dis-

criminant, and construct validity are clearly emotional, for example,anger and frustration, positive affect, and guilt.

(Cialdini & Kenrick, 1976) as producing increments in helpful-

ness that are explainable by the NSR model, for example, em-

barrassment (Apsler, 1975), feelings of deviancy (Filter &

Gross, 1975), guilt (Freedman, Wallington, & Bless, 1976), per-

petration of another's harm (McMillen, 1971), and cognitive

dissonance (Kidd & Berkowitz, 1976).'

There are two possible reasons why these studies might be

cited in support of the model despite the fact that they clearly

do not contain manipulations of sadness. One is that although

the specific emotion of sadness may have particular impor-

tance, the model does in fact apply to an array of other negative

emotions as well. Alternatively, outcomes of negative mood ma-

nipulations other than sadness may have been cited as support-

ing the NSR model in the belief that sadness was a common,

underlying conceptual component of these other diverse manip-

ulations. In either case, if the studies that contain manipulations

of the specific negative moods that the model claims will disrupt

sadness-based helpfulness (e.g., frustration and anger) are ex-

cluded, a test of the model within the broad literature should

be supportive. It is inconsistent, however, to contend that a

broad set of studies supports a model and then reject a test of it

that not only includes those studies but also, in further fairness

3 Other citations of support for the model are also confusing. It is saidthat the Berkowitz and Connor (1966) studies, which failed to supportthe NSR model, do not appropriately apply to it because their manipu-lations featured failure (Cialdini & Kenrick, 1976, p. 912). Yet Weyant(197 8) is cited as supporting the model's prediction of an interactionbetween negative mood, task pleasantness, and helping despite the factthat this study manipulated mood via failure.

218 NORMAN MILLER AND MICHAEL CARLSON

Table 2

Construct Validity of Judged Variables in Psychological Meta-Analyses

Authors)

Bowers &Clum( 1988)

Eagly&Carli(1981)

Eagly&Crowley(l986)

Eagly&Steffen(1986)

Hull & Bond (1986)

Mullen etal.(1985)

Steele & Southwick ( 1 985)

Judged by

Authors

Others

Others

AuthorsOthers

Authors

Authors

Authors

Others

Judged variable

Credibility of placebocontrol

1 (high)2 (medium)3 (medium)4 (medium)5 (medium)6 (medium)7 (low)

Masculine topicover representation

Interest in topicKnowledge of topic

Beliefs about helpingCompetenceComfortableDanger

Helping likelihoodOwn behaviorStereotypic

Amount of provocationAggressive behavior

HarmGuilt/anxietyDangerOwn behaviorStereotypic

Quality of the expectancymanipulation: low(O)-high (3)

Generality of the referencepopulation

Degree of inhibitory conflictLowHighLowHigh

Dependent variable

Specific and nonspecific treatmenteffects in behavior therapy

Sex differences in susceptibilityto persuasive influence

(Male-female)(Male-female)

Gender and helping(Male-female)(Male-female)(Female-male)

(Male-female)(Male-female)

Gender and aggression

(Female-male)(Female-male)(Female-male)(Male-female)(Male-female)

Consequences of alcoholconsumption & expectancy

False consensus effect

Alcohol & social behavior

Statistic

,5 = 0.35"6 = 0.84i = 0.66I = 0.656 = 0.34I = 0.663 = 0.21

r = .31,p<.01r = .04, ns

8 = 0.58, p<. 0010 = 0.27, p<. 0010 = 0.72, p<. 001

/} = 0.49, f<. 001£ = 0.32, p < . 001Q, = 35.4, p<. 001

0 = 0.40,p<.01ft = 0.68, p<. 0010 = 1.17,p<.OOI0 = 0.92, p<. 001ft = 0.40, p<. 001

r = .07, ns</=.0009,Z = 0.1515p = .05 (one-tailed)

a = 0.140"6= 1.063« = 0.196 = 0.94

Note. This table is based on a nonexhaustive search for meta-analyses that were reported in the Journal of Personality and Social Psychology andPsychological Bulletin from January 1981-July 1988. We thank Sharon R. Gross for preparing this table." Neither the comparison of the high versus pooled medium effect sizes nor the pooled medium versus low effect sizes is significant (Z = 0.45, p =.33, and Z - 0.58, p = .28, respectively, using Rosunthal's [1984] Equation 4.27 with the modification that w ~ n because n = 2 for the low effectsize). The authors explain the low effect size in the high credible group as being due to the occurrence of less-powerful treatments within that set ofstudies.bBoth the author-rated and other-rated comparisons of the high versus low degree of inhibitory conflict are significantly different (Z = 2.37. p =.0089, and Z = 1.94,p = .0265, respectively).

to the model, partials out of their manipulations the effects of

all key conceptual components other than sadness.

Our response to this ambiguity, however, was not merely to

present the three tests cited as inappropriate. Instead, we tested

the NSR model in a variety of ways: using a full array of relevant

negative mood cases in order to maximize power; testing the

specific variable sadness as well as the more global variable neg-

ative affect; partialing out of sadness ratings the effects of other

rated mood states; testing subsets of cases in which manipula-

tions featuring high levels of either anger or frustration were

omitted; testing the model within the subset of studies contain-

ing only adult subjects; assessing the mean effect size produced

by explicit manipulations of sadness among adult subjects; and

so forth. In all, we reported at least 17 tests of the NSR model

and conducted many others. None of our tests supported the

model.

In their discussion of the three tests they dispute, Cialdini

and Fultz (1990) presented their own requirements for a proper

assessment of the three key NSR variables: age, distastefulness

of the helping task, and sadness. For the first two variables, they

argued that analyses should be limited to cases containing ex-

perimental manipulations of sadness. For the third, they stated

that the analysis should use a measure of sadness that is not

based on judges' ratings. We have performed new analyses im-

plied by these suggestions. One direct indicator of whether an

experimenter manipulated sadness is whether the author explic-

itly describes the manipulation as one that features sadness or

depression. It seems axiomatic that in such studies some degree

META-ANALYTIC VALIDITY QUESTIONS NSR MODEL 219

of sadness was indeed induced and that on the average its

amount would exceed that in studies that manipulated vari-

ables other than sadness. This expectation is strongly supported

by a mean effect size of approximately 1.5 for the sadness-ma-

nipulation checks in the subset of studies that explicitly manip-

ulated and then measured sadness.

Table 3 presents the results of nine new analyses based di-

rectly on the criteria stipulated by Cialdini and Fultz (1990).4

In each instance as well as in other unreported new analyses

based on the criteria they stipulate for a proper test, the NSR

model fails to receive support. In no case does a p value ap-

proach significance for any predictions from the NSR model,

and the direction of effect is often opposite to that specified

by it.

In summary, with the exception of NSR variables, our results

consistently show that judges' ratings exhibit construct validity

and that for a number of variables operationalized by such rat-

ings (other than those of the NSR model) the direction of their

correlation with mood-based helpfulness confirms prior theo-

rizing and experimental work. When the new tests that Cialdini

and Fultz (1990) stipulated are applied, the NSR model again

receives no support. Moreover, this disconfirmation is main-

tained in analyses that do not even use judges' ratings to define

the comparisons, but instead use objective criteria.

Specific Criticisms

We now discuss some other, more specific issues raised by

Cialdini and Fultz (1990).

Actor-Observer Differences

The first component of the more specific criticisms regarding

the validity of judges' ratings raises the divergent perspectives

effect seen in the comparison of actor and observer ratings. To

assess the relevance of this literature, it is important to reiterate

that our judges' task was to indicate the relative level of a given

variable across a set of studies.3 The literature on the divergent

perspectives of actors and observers is instead largely concerned

with mean differences between their ratings on a common scale,

not with an assessment of the correlation between their relative

ordering of an array of conditions. In contrast to instances of

divergent perspective effects that can be cited from individual

studies, when the causal attribution literature is examined

meta-analytically (Harrington & Miller, 1988), there is a strong

between-studies positive correlation between actor and ob-

server judgments (r = .68). As Funder (1987) cogently noted,

the literature on bias as a function of perspective can only ad-

dress the process of judgment, not its accuracy. Nevertheless,

despite the disclaimers of some of the researchers involved in it

(e.g., Tversky & Kahneman, 1983), many psychologists errone-

ously view this literature as challenging the correctness or docu-

menting the ineptness of human judgment. Accuracy, however,

cannot be inferred from bias (Kenny & Albright, 1987). It is

thus mistaken to focus on processes that produce bias and, hav-

ing documented them, to assume that human judgment is poor

and does not improve much on chance. Lately, although ac-

knowledging human susceptibility to bias, there is an emerging

recognition that the accuracy of observers can nevertheless be

quite high (Hastie & Rasinski, 1988; Kruglanski & Ajzen,

1983).

Role-Playing Replications of Social

Psychology Experiments

Cialdini and Fultz (1990) discussed the literature on role

playing as a second source of evidence against our reported out-

comes. We argue that this literature, too, has little bearing on

the validity of the ratings asked of our judges. Whereas our

judges were asked merely to assess the degree to which the aver-

age subject would experience sadness, anger, and a number of

other emotions and cognitions in an array of particular situa-

tions, judges in a typical role-playing study are asked a single,

but far more complex question; namely, how would the subject

behave? Presumably, to be accurate, such role players must first

assess how they would feel in a given situation, and then how

they would consequently behave. Yet the typical role-playing

instructions do not even prime or elicit thought about the medi-

ating cognitive-affective state, but instead simply elicit a behav-

ioral prediction. Moreover, Funder (1987) and McArthur &

Barron (1983) persuasively document how the verbal stimuli

that constitute the task for such role players (as opposed to more

ordinary judgment tasks) deviate from ordinary circumstances

4 To provide as favorable a test as possible far the NSR model, ouroriginal data set (Carlson & Miller, 1987) on negative mood and helpingwas expanded by treating separate helping task conditions within astudy, as well as within-studies age distinctions within the categories ofchildren and adults, as independent observations in the analysis. In our

original report, single observations were formed by collapsing over thesedistinctions in order to achieve a greater degree of statistical indepen-dence. This expansion of our original data set produced a total of 98distinct negative versus neutral mood effect-size cases. In addition, weobtained new ratings of the variable sadness (reliability = .90) and in-cluded a new, potentially more sensitive helping-task pleasantness-dis-tastefulness variable (viz., capacity of helping task to alter mood, de-nned as "the extent to which performing the particular helping taskswould tend to either lower or elevate the mood of the average subject"

[scale range: -5, would lessen mood to an extreme extent, to 5, would

elevate mood to an extreme extent; reliability = .80]). Thanks are due toAmy Marcus-Newhall and Lisa Maybrown (social psychology graduatestudents at the University of Southern California [USC]) for providingratings on these two variables. Finally, all of the eifect-size estimateswere corrected for sample size bias by applying Hedge's correction for-mula (Glass, McGraw, & Smith, 1981). None of these changes and addi-

tions significantly altered the outcomes originally reported.5 For example, in a relative sense, they judge how much guilt subjects

experienced when listening to noxious music as opposed to being toldafter they had knocked over a deck of IBM cards that as a consequenceJane will never complete her degree in lime for graduation; how distaste-ful was a helping task that required picking up an armful of books versusanswering hundreds of boring multiple choice questions; how sad wouldone feel when asked to imagine that a very close friend was dying ofcancer as opposed to being required to multiply pairs of three-digitnumbers mentally while a pneumatic jackhammer was in operation

outside a nearby window.

220 NORMAN MILLER AND MICHAEL CARLSON

Table 3

Further Tests of the Negative State Relief(NSR) Model: More Disconfirming Evidence

Based on Cialdini and Fullz 's Assessment Criteria

NSR variableCialdini and Fultz's

requirement Manner tested NSR prediction Outcome

Age (children vs. adults)

Hedonic value ofhelping task

Sadness

Test only among cases Usi ng only instances featuring an explicitfeaturing (a) a sadness manipulation (and a privatesadness helping opportunity, given children asmanipulation and subjects3), a comparison of the average(b) a private effect-size values of cases featuringhelping children (aged 12 and under) andopportunity adults (aged 13 years or older) was

undertaken.13

Test only among 1. Given the presence of an explicitsadness cases sadness manipulation (i.e., as defined

by the experimenter), zero-order andpartial correlation coefficients between"capacity of helping task to altermood" and effect size were calculated.0

2. Same as I above, but analysis waslimited to subjects aged 13 years orolder.

3. Given the presence of a study cited inone of Cialdini's own publishedreviews (Cialdini, Baumann, &Kenrick, 1981)asentailingamanipulation that is capable ofproducing an NSR effect, and givensubjects aged 13 years or older, zero-order and partial correlationcoefficients between "capacity ofhelping task to alter mood" and effectsize were calculated.

Test with a valid A dicholomous variable corresponding toindicator of whether the experimenter explicitlysadness manipulated sadness (coded value = I,

n = 36) or did not (coded value - 0,n - 62) was derived. This variable wasthen correlated with helping effect sizesamong subjects aged 13 years or older.

Larger effect sizes Children: M = -0.14 (n = Ifor adults Adults: M = 0.05 (n = 20)

;(26) = 0.67, ns

Positive correlationwith effect size

Positive correlationwith effect size

Zero-order r(34) = —.02, ns\partial r(29) = -.03, ns

Zero-order r(24) = .09, ns;partialr(l9)=.01,ns

Positive correlation Zero-order r(39) = .06, ns;with effect size partial r(34) =-.05, ns

Positive correlation Zero-order r(82) = —.05, ns;with effect size partial r(76) = .06, ns

* Because Cialdini has stated that the public versus private dimension is critical only for certain childhood age groups (Cialdini et al., 1981), bothpublic and private helping opportunities were allowed for adult cases.b To avoid confounding, instances featuring high levels of guilt (a value above the median) were eliminated from the analysis, as guilt tends toaugment helping and is largely absent among studies that used children as subjects. However, additional analytic strategies used on the sadness cases,such as including observations high in guilt, limiting the adult cases to private helping opportunities, or testing the dichotomous age variable ofchildren versus adults in a partial correlation analysis that controls for other variables, all uniformly fail to support the NSR predictions.c In conducting partial correlation analyses, the variables guilt, responsibility for the negative event, objective self-awareness, self versus other astarget of the negative event, and salience of the helping request (Carlson & Miller, 1987) were statistically adjusted for.

to undermine the normal rules of social inference.6 Conse-

quently, we view the various failures and successes of the role-

playing procedure as having no bearing on the adequacy of our

own methodology because the tasks are not comparable. Re-

cent evidence from a validity study confirms this view. Judges'

ratings of subjects' affect and cognition correlate well with ma-

nipulation checks on subjects' affect and cognition, but judges

could not predict subjects' behavior (Miller, Lee, & Carlson, in

press).

Graduate Student Judges

Cialdini and Fultz (1990) criticize our use of graduate student

judges, pointing out that they are less similar to the subjects

who served in the original studies than are undergraduate

judges, and therefore are presumably more prone to divergent

perspective effects. We chose graduate students primarily be-

cause they tend to be more conscientious and knowledgeable

than undergraduates. Our judgment task was of large magni-

tude, requiring for each of perhaps a dozen variables a separate

rating of each of 85 method sections, one by one. This makes

6 As implied, a second important distinction is that many role-playingreplication studies use a between-subjects design, with each subject rep-licating a single experimental condition. The fact that our judges, incontrast, make comparative ratings across an array of conditions pro-vides a powerful judgmental advantage in comparison to such role play-ers (e.g.. Miller, Butler, Doob, & Marlowe, 1965).

META-ANALYTIC VALIDITY QUESTIONS NSR MODEL 221

conscientiousness an especially critical requirement. Addition-ally, knowledge, not surface features of similarity to the targetsuch as age or hair color, is recognized as a key quality for judgesto possess (Rosenthal, 1987). For example, decision theoristsroutinely recommend knowledgeable judges (von Winterfeldt&Edwards, 1986). Within the domain of personality judgment,correlations between peer ratings and self-ratings typicallyrange from .30 to .60 (e.g., Anderson, 1984; Paunonen & Jack-son, 1985; Woodruffe, 1985), but increase when the judgeknows whom he or she is rating (e.g., Cloyd, 1977; Funder &Colvin, 1988; Kusyszn, 1968; Norman & Goldberg, 1966; Taft,1966). Social psychology graduate students are probably moreknowledgeable about the psychological meaning of a verbal de-scription of an experimental manipulation than are naive un-dergraduates. Nonetheless, it is worth noting that many of thestudies cited in Table 2 did use undergraduate judges success-fully.7

Data Purportedly Refuting the Discriminant Validity of

Judges' Ratings of Sadness

Our original article reported a validity correlation of .47 be-tween judges' ratings of sadness and sadness effect sizes, as com-puted from those studies that reported a manipulation checkadministered to the actual subjects (« = 14). Although Cialdiniand Fultz (1990) reported data from their own judges that con-firm this outcome, they dispute its manifest confirmation of thevalidity of judges' ratings, alleging that such results lack dis-criminant validity.8 Their basis for this contention is that otherratings they obtained on global negative affective dimensionssuch as general emotionality and strength of affect correlate ashighly with manipulation checks of experimentally inducedsadness as does judged sadness. We feel this argument is faultyfor two reasons.

First, in their procedure Cialdini and Fultz (1990) stronglyand artifactually invited halo effects by instructing their judgesto rate each study on all judgment dimensions before proceed-ing to the next study, rather than having judges rate all studieson each single dimension before proceeding to judgments onthe next as did Carlson and Miller (1987). With each judge'srating of sadness clearly visible while rating other dimensions,consistency motives will induce spuriously high correlationsamong semantically similar dimensions. Second, even if this bi-ased methodology were corrected, in this specific data set onewould still fully expect a high correlation between judges' rat-ings of emotionality and sadness-manipulation check effectsizes for the following reason; The presence of a sadness-manip-ulation check was the criterion for selecting this subset of stud-ies. Consequently, the preponderance of cases correspondinglyfeatured an explicit manipulation of sadness. Within such a set,sadness is virtually the sole source of variance in emotionality,and therefore, discriminant validity between ratings of it andother variables directly related to the strength of its induction(such as general emotionality) cannot be expected. A properassessment of the discriminant validity of sadness ratings rela-tive to ratings of general emotionality requires a set of cases inwhich other negative affects (e.g., anger, jealousy, and boredom)were sometimes strongly present in the absence of sadness. In-

stead, Cialdini and Fultz assessed discriminant validity by ana-lyzing a selected set of cases within which sadness and generalemotionality are largely, by definition, synonymous, and thencontended that the obtained high correlations between the twovariables casts doubt on judges' ability to differentiate.

Valid Evidence of Discriminant and Convergent Validity

Although our initial report of the correlation between judges'ratings and manipulation check effect sizes of sadness induc-tions was only intended to provide evidence of convergent valid-ity (i.e., agreement between judges' and subjects' ratings of sad-ness), further evidence indicates that our judges were able todiscriminate sadness from other specific negative affects. Forexample, among our rated variables, guilt, anger, and frustra-tion (all of which, like sadness, fall within the more general cate-gory of negative affect), the average correlation with the 14sadness-manipulation check effect sizes was —.08.' Evidenceshowing that judges validly differentiate conceptually relatedconstructs such as objective self-awareness and self-absorption(Carlson & Miller, 1987) and anger and frustration (Carlson,1988) further demonstrates the discriminant validity of theirratings. In the latter case, for example, manipulation checkeffect-size estimates of the degree of experienced anger (n — 33)correlated positively with judges' ratings of felt anger (r = .44,p < .01), but were unrelated to their ratings of frustration(/=-.! 6).

In addition, we have obtained positive convergent validitycorrelations between judges' ratings and manipulation checkeffect sizes of positive affect (Carlson et al., 1988), frustration,and negative affect (Carlson, 1988), as well as for a combinationof four variables including objective self-awareness, guilt, self-absorption, and anger (Carlson & Miller, 1987). When these cor-relations are pooled via the Stauffer method of combining Zscores (Rosenthal, 1984), the null hypothesis of a zero correla-tion is untenable (Z = 4.32, p < .0001). Moreover, althoughthese consistent positive correlations are substantial (range =.4-.7), they are obtained in the face of factors likely to depresstheir magnitude: (a) commonly, a restricted range is present onthe dimension being assessed because of the fact that a manipu-lation check is more likely in cases where a high level of a vari-able was intentionally manipulated; (b) nonidentical manipula-tion checks, associated with different degrees of sensitivity and

7 To provide evidence of consistency and construct validity within the

context of our own work, we had a USC freshman rate the variable guilt

within the negative-mood helping literature. We thank Jeff Rosenfeld

for this assistance. Consistent with the fundamental validity of our

method, his ratings correlated (r = .65, p < .0001) with those made by

our graduate student judges and correlated with helpfulness effect sizes

(r = .49, p < .0001) at a level similar to that of graduate student judges

(r = .56). His rating of a second new variable, felt victimization, corre-

lated negatively with helpfulness (r = -.22, p < .03) as theoretically

expected (Carlson & Miller, 1987, p. 103).8 Cialdini and Fultz (1990) neglected to report their stronger obtained

outcome of .62 when the manipulation check effect sizes are trans-

formed to normalize their distribution.9 These figures were generated with respect to the data set described

in Footnote 3.

222 NORMAN MILLER AND MICHAEL CARLSON

unique method variance, are lumped together in computing thecorrelations; and (c) self-report and other measures imperfectlyassess the construct in question, and may therefore produce sys-tematic bias in certain cases.

Correlation Between Sadness-Manipulation Check and

Helping Effect Sizes

To support a central theoretical contention of the NSRmodel, Cialdini and Fultz (1990) argued that, among adult sub-jects, sadness-manipulation check effect sizes correlate posi-tively with helping effect sizes. We question this contention.First, the reported correlation is nonsignificant—r(12) = .42,p > .05, one-tailed—and therefore could be due to samplingerror. Second, it was obtained only after excluding the Rogers,Miller, Mayer, and Duval (1982) data, whose inclusion wouldproduce a negative correlation between sadness and helpingeffect sizes, r( 16) = -.17,p = .50.'°

To assess more adequately the relation between sadness andhelping effect sizes, we increased the number of sampled obser-vations by including relevant studies published after 1982, thecutoff date for inclusion in our original meta-analysis (viz., Ber-kowitz, 1987; O'Malley& Andrews, 1983;Manuciaetal., 1984;Shaffer & Smith, 1985). In all, eight new observations wereadded, producing a total of 26 cases. In harmony with our ear-lier outcome, the resulting correlation between sadness andhelping effect sizes failed to support the NSR model, r(24) =— . l l , p = .58." In sum, when we computed this correlation inat least nine different ways on the basis of more extensive setsof studies, including in some instances new studies publishedsince our original report, it was uniformly close to zero andnonsignificant, thereby supporting our prior meta-analytic out-come.

Summary and Conclusion

To contest our application of meta-analysis to the NSRmodel, Cialdini and Fultz (1990) contended that ratings madeby judges are invalid. There are two basic meanings to validityin social judgments: first, agreement among those who are judg-ing the same thing; and second, the ability of those judgmentsto predict behavior. Campbell and Fiske (1959) referred to thefirst as trait validity, whereas the term construct validity (Cron-bach & Meehl, 1955) is commonly used in reference to the sec-ond. With respect to the first of these two basic criteria, we notonly show high interrater agreement among our judges, butmore important, substantial evidence of convergence betweenjudges' ratings and the ratings of subjects in their responses tomanipulation checks. Moreover, we present ample evidencethat judges' ratings evince Campbell and Fiske's companion cri-terion of discriminant validity. Regarding the second, constructvalidity, we show very substantial evidence that judges' ratingsdo predict the behavior of subjects in experiments. This con-struct validity not only appears consistently in our own work,but also in that of others.

In attempting to strengthen their argument concerning theinvalidity of judges' ratings, Cialdini and Fultz (1990) citednonrelevant literatures on the process of judgmental bias, not

error, namely, the literatures on the actor-observer effect androle playing as a means of replicating experimental social psy-chological research—literatures in which the subjects' tasks de-viate in important ways from those of our judges (Funder, 1987;McArthur & Barren, 1983). Moreover, Cialdini and Fultz pro-vided no explanation for how the proposed error in judges' rat-ings only works against the NSR model, whereas outcomesbased on judges' ratings of other variables consistently corre-spond to those obtained in experiments. In addition, they pre-sent no tests of effects for the study that they present to supporttheir criticism that our judges' ratings lacked discriminant va-lidity—a study that we argue is methodologically and conceptu-ally flawed. They cited a nonsignificant correlation to argue thatsadness and helping effect sizes are positively related. When wecompute this correlation in nine different ways on the basis ofless selective data sets, we find it uniformly close to zero. Finally,Cialdini and Fultz raised an objection regarding the outcomesof the 17 nonconfirming tests of the NSR model reported in ourarticle, contending that we did not test it appropriately. Cialdiniand Fultz (1990) presented their own specific criteria for propertests. However, when we use such tests, they produce additionalevidence that fails to support the NSR model.

Cialdini and Fultz (1990) argued against analyses in whichwe statistically controlled for levels on variables not manifestlypart of the NSR model by using partial correlation procedures.However, comparisons across studies must control for potentialconfounding. In the helping literature, this confounding withinparadigms is illustrated well in Eisenberg and Miller's (1987)meta-analytical review of empathy and helping. Picture-storymeasures, which fail to show empathy effects, are commonlyused in studies with children, but are rarely used in studies on

10 The six studies that provided the 14 cases for the correlation com-

puted by Cialdini and Fultz (1990) are Cunningham, Steinberg, andGrev (1980); Forest, Clark, Mills, and Isen (1979); Fried and Berkowitz

(1979); Harvey and Enzle (1981); Kidd and Marshall (1982); and

Thompson, Cowan, and Rosenhan (1980). Cialdini and Fultz discarded

the Rogers et al. data because it yielded abnormally large sadness effect

sizes. Alternate, more acceptable analytic strategies, such as includingit and either using a nonparametric correlation method or normalizing

the resulting distribution via a transformation, produce nonsignificant,

near-zero correlations between sadness and helping effect sizes, for ex-

ample, Spearman r(16) = .11, p - .66; using log-transformed sadness

effect sizes, Pearson r(16) = -.07,p = .77. In addition, the correlation

between sadness and helping effect sizes remains nonsignificant if key

rated theoretical variables are controlled for, for example, adjusting forthe variable guilt, partial r(15) = — .17, p = .52; adjusting for the vari-

ables anger and frustration, partial r( 14) = -.20, p = .47.

"When included in this larger data set, Rogers et al. (1982) is no

longer an outlier. The Z score pertaining to its largest sadness effect size

is below 2.0. Nevertheless, within this expanded data set, numerous al-

ternate analytic tacks, such as excluding the study by Rogers et al.,

/•(20) = .13, p = .57; using a nonparametic correlation procedure,

Spearman r(24) = .05, p = .82; or using log-transformed sadness effectsizes, r(24) = -.03, p = .87, uniformly failed to produce a significant

correlation between experimentally manipulated sadness and subse-

quent helpfulness. In fairness to the NSR model, the fixed mood condi-

tion within Manucia et al. (1984), for which the NSR proponents explic-

itly predicted an absence of sadness-generated helpfulness, was ex-

cluded from all of these analyses.

META-ANALYTIC VALIDITY QUESTIONS NSR MODEL 223

adults. Therefore, a review of the research on children will for

this reason alone yield a negative outcome (e.g., Underwood

& Moore, 1982), whereas other measures that show a positive

relation between empathy and prosocial behavior in adults

(when they happen to be used in studies on children) will con-

firm empathy effects in children too. Between-studies compari-

sons of age effects are particularly vulnerable to such interac-

tions. When, for instance, we control for the effects of extrane-

ous variables in our analyses of the effect of age and fail to find

support for the NSR model, such outcomes question the valid-

ity of the explanatory dynamic proposed by the model to ac-

count for mood-based interactions with age—that is, whether

the hypothesized timetable for levels of socialization (with re-

spect to self-reinforcement) does indeed interact with sadness

to carry the explanatory burden (Cialdini & Kenrick, 1976).

Finally, we argue that the theoretical importance of such

meta-analytic outcomes for the interpretation of individual

studies should not be overlooked. These outcomes suggest why,

in the face of broader, more general disconfirmation of the NSR

model, one can nevertheless obtain an outcome within an indi-

vidual study that seemingly supports it. Namely, we suggest that

in such instances researchers have (inadvertently) confounded

manipulations of NSR variables with levels of other (non-NSR)

variables that do in fact explain mood-based helpfulness, such

as objective self-awareness or attentional focus.

A promising theory should not be discarded on the basis of a

single disconfirmation or on the basis of a set of disconfirma-

tions that were all obtained within a single, narrow research

paradigm. With respect to the NSR model, however, the evi-

dence we have presented in this article, in combination with

our prior evidence (Carlson & Miller, 1987), integrates findings

from numerous individual research paradigms. Across an array

of standard procedures for quantitatively integrating individual

experimental findings, the outcomes are consistently noncon-

firming for the NSR model.12 In line with Cook and Campbell's

(1979) discussion of the Quine-Duhem thesis, it is in principle

possible to explain away each new nonconfirming outcome by

continually advancing new ad hoc hypotheses. At some point,

however, it becomes more appropriate to question the adequacy

of the unconfirmed model, especially when the same research

integration procedures do support alternative models.

12 It is also worth noting that the results of a number of individual

studies, in agreement with our meta-analytic findings, contradict theNSR model's insistence on the importance of sadness (Berkowitz, 1987;

Rogers et al., 1982; Shroeder, Dovidio, Sibicky, Matthews, & Allen,

1988; Thompson etal., 1980; Underwood etal., 1977) and pleasantness

of the helping task (Cunningham et al., 1980; Forest et al., 1979; Freed-

manetal., 1976;Harada, 1983; Wallace & Sadella, 1976).

References

Anderson, S. (1984). Self-knowledge and social inference: II. The diag-

nosticity of cognitive/affective and behavioral data. Journal of Per-

sonality and Social Psychology, 46. 294-307.Apsler, R. (1975). Effects of embarrassment on behavior toward others.

JournalojPersonality and Social Psychology, 32, 145-153.

Berkowitz, L. (1987). Mood, self-awareness, and willingness to help.

Journal of Personality and Social Psychology, 52, 721-729.

Berkowitz, L., & Connor, W. H. (1966). Success, failure, and social re-

sponsibility. Journal of Personality and Social Psychology, 4, 664-

669.

Block, J. (1971). Lives through time. Berkeley, CA: Bancroft Books.

Bowers, T. G., & Clum, G. A. (1988). Relative contribution of specific

and nonspecific treatment effects: Meta-analysis of placebo-con-

trolled behavior therapy research. Psychological Bulletin, 103, 315-

323.

Campbell, D. T, & Fiske, D. W. (1959). Convergent and discriminant

validation by the multitrait-multimethod matrix. Psychological Bul-

letin. 56, 81-105.

Carlsmith, J., & Gross, A. (1969). Some effects of guilt on compliance.

Journal of Personality and Social Psychology, 11, 232-239.

Carlson, M. (1988). Toward understanding the relationship between un-

pleasant stimulation and aggressive behavior in humans. Unpub-

lished doctoral dissertation, University of Southern California, Los

Carlson, M., Charlin, V., & Miller, N. (1988). Positive mood and helping

behavior A test of six hypotheses. Journal of Personality and Social

Psychology. 55,211-229.

Carlson, M., & Miller, N. (1987). Explanation of the relation between

negative mood and helping. Psychological Bulletin, 102,91 -108.

Cialdini, R. B., Baumann, D. J., & Kenrick, D. T. (1981). Insights from

sadness: A three-step model of the development of altruism as hedo-

nism. Developmental Review, I, 207-223.

Cialdini, R. B., Darby, B. L., & Vincent, J. E. (1973). Transgression

and altruism: A case for hedonism. Journal of Experimental Social

Psychology, 9. 502-516.

Cialdini, R. B., & Fultz, J. (1990). Interpreting the negative mood/help-

ing literature via "mega"-analysis: A contrary view. PsychologicalBulletin, 107,210-214.

Cialdini, R. B., & Kenrick, D. T. (1976). Altruism as hedonism: A social

development perspective on the relationship of negative mood state

and helping. Journal of Personality and Social Psychology, 34, 907-914.

Cialdini, R. B., Schaller, M., Houlihan, D., Arps, K., Fultz, J., & Bea-

man, A. L. (1987). Empathy-based helping: Is it selflessly or selfishly

motivated? Journal of Personality and Social Psychology, 52, 749-

758.

Cloyd, L. (1977). Effect of acquaintanceship on accuracy of personal

perception. Perceptual and Motor Skills, 44, 819-826.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design

and analysis issues for field settings. Chicago, IL: Rand McNally.

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychologi-

cal tests. Psychological Bulletin. 52, 281-302.

Cunningham, M. R., Steinberg, J., & Grev, R. (1980). Wanting to and

having to help: Separate motivations for positive mood and guilt-in-

duced helping. Journal of Personality and Social Psychology, 38,181-

192.

Dillard, J. P., Hunter, J. E., & Burgoon, M. (1984). Sequential-request

persuasive strategies: Meta-analysis of foot-in-the-door and door-in-

the-face. Human Communication Research, 10,461-488.

Eagly, A. H., & Carli, L. L. (1981). Sex of researchers and sex-typed

communications as determinants of sex difference in influenceabil-

ity: A meta-analysis of social influence studies. Psychological Bulle-

tin. 90, 1-20.

Eagly, A. H., & Crowley, M. (1986). Gender and helping behavior: A

meta-analytic review of the social psychological literature. Psycholog-

ical Bulletin, 100, 283-308.

Eagly, A. H., & Steffen, V. J. (1986). Gender and aggressive behavior A

meta-analytic review of the social psychological literature. Psycholog-

ical Bulletin, 100, 309-330.

224 NORMAN MILLER AND MICHAEL CARLSON

Eisenberg, N., & Miller, P. A. (1987). The relation of empathy to proso-

cial and related behaviors. Psychological Bulletin, 101, 91-119.

Filter, T. A., & Gross, A. E. (1975). Effects of public and private devi-

ancy on compliance with a request. Journal of Experimental Social

Psychology. II, 553-559.

Forest, D., Clark, M. S., Mills, J., & Isen, A. M. (1979). Helping as a

function of feeling state and nature of the helping behavior. Motiva-

tion and Emotion, 3, 161-169.

Freedman, J. L., Wallington, S. A., & Bless, E. (1976). Compliance with-

out pressure: The effect of guilt. Journal of Personality and SocialPsychology, 7, 117-124.

Fried, R., & Berkowitz, L. (1979). Music hath charms . . . and can

influence helpfulness. Journal of Applied Social Psychology, 9, 119-

20S.

Funder, D. C. (1987). Errors and mistakes: Evaluating the accuracy ofsocial judgment. Psychological Bulletin, 101, 75-90.

Funder, D. C., & Colvin, C. R. (1988). Friends and strangers: Acquain-

tanceship, agreement, and the accuracy of personality judgment.Journal of Personality and Social Psychology, 55, 149-158.

Gibbons, F. X., & Wicklund, R. A. (1982). Self-focused attention andhelping behavior. Journal of Personality and Social Psychology, 43,

462-474.

Glass, G. V., McGraw, B., & Smith, M. L. (1981). Mela-analysis insocial research. Beverly Hills, CA: Sage.

Harada, J. (1983). The effects of positive and negative experiences on

helping behavior. Japanese Psychological Research, 25, 47-51.

Harrington, H. J., & Miller, N. (1988). Altributionalbias: A meta-ana-

lytic evaluation of seven major hypotheses. Unpublished manuscript,University of Southern California, Los Angeles.

Harvey, M. D., & Enzle, M. E. (1981). A cognitive model of social

norms for understanding the transgression-helping effect. Journal ofPersonality and Social Psychology, 44, 866-875.

Hastie, R., & Rasinski, K. A. (1988). The concept of accuracy in social

judgment. In D. Bar-Tal & A. Kruglanski (Eds.), The social psychol-

ogy of knowledge (pp. 193-208). Cambridge, England: Cambridge

University Press-

Hollander, E. P. (1965). Validity of peer nominations in predicting a

distant performance criterion. Journal of Applied Social Psychology,49, 434^(38.

Holloway, S., Tucker, L., & Hornstein, H. (1977). The effect of social

and nonsocial information in interpersonal behavior of males: The

news makes news. Journal of Personality and Social Psychology. 35,514-522.

Hornstein, H., Lakind, E., Frankel, G., & Manne, S. (1975). Effects of

knowledge about remote social events on prosocial behavior, social

conception, and mood. Journal of Personality and Social Psychology,32, 1038-1046.

Hull, J. G., & Bond, Jr., C. F. (1986). Social and behavioral conse-

quences of alcohol consumption and expectancy: A meta-analysis.Psychological Bulletin, 99, 347-360.

Isen, A. M., & Simmonds, S. F. (1978). The effect of feeling good on a

helping task that is incompatible with good mood. Social Psychology,41, 346-349.

Isenberg, D. J. (1986). Group polarization: A critical review and meta-

analysis. Journal of Personality and Social Psychology, 50, 1141-

1151.

Kenny, D. A., & Albright, L. (1987). Accuracy in interpersonal percep-

tion: A social relations analysis. Psychological Bulletin, 102, 390-402.

Kidd, R., & Berkowitz, L. (1976). Effect of dissonance arousal on help-

fulness. Journal of Personality and Social Psychology, 33, 613-622.

Kidd, R. E, & Marshall, L. (1982). Self-reflection, mood, and helpful

behavior. Journal ofResearch in Personality. 16, 319-334.

Kruglanski, A. W., & Ajzen, I. (1983). Bias and error in human judge-

ment. European Journal of Social Psychology, 13, 1-44.

Kusyszn, I. (1968). A comparison of judgmental methods with endorse-

ments in assessment of personality traits. Journal of Applied Psychol-

ogy, 52, 227-253.

Manucia, G. K., Baumann, D. J., & Cialdini, P. B. (1984). Mood influ-

ences on helping: Direct effects or side effects. Journal of Personality

and Social Psychology, 46, 357-364.

Mayfield, E. C. (1972). Value of peer nominations in predicting life in-

surance sales performance. Journal of Applied Psychology, 56, 319-

323.

McArthur, L. Z., & Barren, R. M. (1983). Toward an ecological theory

of social perceptions. Psychological Review, 90, 215-238.

McMillen, D. L. (1971). Transgression, self-image, and compliant be-

havior. Journal of Personality and Social Psychology, 20, 176-179.

McMillen, D. L., & Austin, J. B. (1971). Effect of positive feedback oncompliance following transgression. Psychonomic Science, 24, 59-60.

McMillen, D. L., Sanders, D. Y, & Solomon, G. S. (1977). Self-esteem,

attentiveness, and helping behavior. Personality and Social Psychol-

ogy Bulletin, J, 257-261.

Miller, N., Butler, D. C., Doob, A. N. J., & Marlowe, D. (1965). The

tendency to agree: Situational determinants and social desirability.

Journal of Experimental Research in Personality, 1, 78-84.

Miller, N., Lee, J., & Carlson, M. (in press). The validity of inferentialjudgments when used in theory-testing meta-analyses. Personality

and Social Psychology Bulletin.

Mullen, B., Atkins, J. L., Champion, D. S., Edwards, C., Hardy, D.,

Shovy, J. E., & Vanderklok, M. (1985). The false consensus effect: Ameta-analysis of 115 hypothesis tests. Journal of Experimental Social

Psychology, 21, 262-283.

Norman, W. T, & Goldberg, L. R. (1966). Raters, ratees, and random-ness in personality structure. Journal of Personality and Social Psy-

chology, 4.681 -691.

O'Malley, M. N., & Andrews, L. (1983). The effect of mood and incen-

tives on helping: Are there some things money can't buy? Motivationand Emotion, 7, 179-189.

Paunonen, S. V., & Jackson, D. N. (1985). Idiographic measurement

strategies for personality and prediction: Some unredeemed promis-sory notes. Psychological Review, 92. 486-511.

Rogers, M., Miller, N., Mayer, F. S., & Duval, S. (1982). Personal respon-

sibility and salience of the request for help: Determinants of the rela-

tion between negative affect and helping behavior. Journal of Person-

ality and Social Psychology, 43, 956-970.

Rosenhan, D. L., Salovey, P., & Hargis, K. (1981). The joys of helping:

Focus of attention mediates the impact of positive affect on altruism.

Journal oj Personality and Social Psychology, 40, 899-905.

Rosenthal, R. (1984). Meta-analytic procedures for social research. Bev-erly Hills, CA: Sage.

Rosenthal, R. (1987). Judgment studies: Design, analysis, and meta-

analysis. Cambridge, England: Cambridge University Press.

Rosenthal, R., Hall, J. A., DiMatteo, M. R., Rogers, P. L., & Archer,

D. (1979). Sensitivity to nonverbal communication: The PONS test.Baltimore, MD: Johns Hopkins University Press.

Schercr, K. R., Scherer, U., Hall, J. A., & Rosenthal, R. (1977). Differ-

ential attribution of personality based on multi-channel presentation

of verbal and nonverbal cues. Psychological Research, 39, 221-247.

Schroeder, D. A., Dovidio, J. F, Sibicky, M. E., Matthews, L. L., &

Allen, J. L. (1988). Empathetic concern and helping behavior: Ego-

ism or altruism. Journal of Experimental Social Psychology, 24, 333-353.

Shaffer, D. R., & Smith, J. E. (1985). Effects of pre-existing mood on

observers' reactions to helpful and non-helpful models. MotivationandEmotion, 9, 101-122.

META-ANALYTIC VALIDITY QUESTIONS NSR MODEL 225

Signorella, M. L., & Jamison, W. (1986). Masculinity, femininity, an- Underwood, B., & Moore, B. (1982). Perspective-taking and altruism.drogyny, and cognitive performance: A meta-analysis. Psychological Psychological Bulletin, 91, 143-173.Bulletin, 100, 207-228. von Winterfeldt, D., & Edwards, W. (1986). Decision analysis and be-

Steblay. N. M. (1987). Helping behavior in rural and urban environ- kavioral research. Cambridge, England: Cambridge University Press.ments: A meta-analysis. Psychological Bulletin, 102, 346-356. Wallace, J., & Sadella, E. (1976). Behavioral consequences of transgres-

Steele, C. M., & Southwick, L. (1985). Alcohol and social behavior 1: sion: Tne effects of social recognition. Journal of Experimental Re-The psychology of drunken excess. Journal of Personality and Social search in Personality, 1, 194-197.Psychology, 48, 18-34. Waters, L. K., & Waters, C. W. (1970). Peer nominations as predictors

Taft,R. (1966). Accuracy of empathetic judgment of acquaintances and of short-termsaltperformwce. Journal of Applied Social Psychology,

strangers. Journal of 'Personality and Social Psychology, 3,600-604. '̂ 42-44.Thompson, W, Cowan, C., & Rosenhan, D. L. (1980). Focus of atten- We,yant'J; ('"8)' E*ects of™ood states cost, and benefits on helping.

tion mediates the impact of negative affect on altruism. Journal of Journal of Personahty and Socml Psychology, 36, 1169-1176.Personality and Social Psychology, 38, 291-300. Wood, W. (1987). Mteta-anaivticrev,ew of sex d,nerencesm group per-

^ . „ „ , „ „„„, „ . , formance. Psychological Bulletin, 102,53-71.Tversky, A & Kahneman, D (1983). Extensional versus mtu.t,ve rea- Woodruffei c.,, 985). Consensual validation of personality traits: Addi-

sonmg: The conjunction fallacy m probab.htyjudgment. Psychology tjona| eviden№ and individua, differences Joumal ofPersonality and

cof Ant* 90. 293-315. Soaal Psychology, 48, 1240-1252.Underwood, B., Berenson, J. F., Berenson, R., Cheng, K.., Wilson, D.,

Kulik, J., Moore, G., & Wenzel, G. (1977). Attention, negative affect Received August 19,1988and altruism: An ecological validation. Personality and Social Psy- Revision received June 1, 1989

chology Bulletin, 3. 54-58. Accepted June 16, 1989 •

Instructions to Authors

Authors should prepare manuscripts according to the Publication Manual of the American Psy-

chological Association (3rd ed.). All manuscripts must include an abstract containing a maxi-

mum of 1,000 characters and spaces (which is approximately 100-150 words) typed on a sepa-

rate sheet of paper. Typing instructions (all copy must be double-spaced) and instructions on

preparing tables, figures, references, metrics, and abstracts appear in the Manual. Also, all

manuscripts are subject to editing for sexist language.

Authors reporting original data will be required to state in writing that they have complied

with APA ethical standards in the treatment of their sample, human or animal, or to describe

the details of treatment. (A copy of the APA Ethical Principles may be obtained from the APA

Ethics Office, 1200 17th Street, N.W., Washington, DC 20036.)

Anonymous review will be first an author's option. If anonymous review is not requested in

a cover letter, it will become the prerogative of the processing editor. Authors requesting anony-

mous review are requested to include with each copy of the manuscript a cover sheet, which

shows the title of the manuscript, the authors' names and institutional affiliations, and the date

the manuscript is submitted. The first page of the manuscript should omit the author's name

and affiliation but should include the title of the manuscript and the date it is submitted. Foot-

notes containing information pertaining to the authors' identity or affiliations should be on

separate pages. Every effort should be made to see that the manuscript itself contains no clues

to the authors' identity.

APA policy prohibits an author from submitting the same manuscript for concurrent consid-

eration by two or more journals. APA policy also prohibits duplicate publication, that is, publi-

cation of a manuscript that has already been published in whole or in substantial part in another

journal. Prior and duplicate publication constitutes unethical behavior, and authors have an

obligation to consult journal editors if there is any chance or question that the paper might not

be suitable for publication in an APA journal. Also, authors of manuscripts submitted to APA

journals are expected to have available their raw data throughout the editorial review process

and for at least 5 years after the date of publication.

Manuscripts should be submitted in quadruplicate. All copies should be clear, readable, and

on paper of good quality. A dot matrix or unusual typeface is acceptable only if it is clear and

legible. Authors should keep a copy of the manuscript to guard against loss. Mail manuscripts to

the Editor, Mark I. Appelbaum, Psychological Bulletin, Department of Psychology and Human

Development, Box 159 Peabody Station, Vanderbilt University, Nashville, TN 37203.