Frustrative Nonreward in Partial Reinforcement and Discrimination Learning

Psychological Review1962, Vol. 69, No. 4, 306-328

FRUSTRATIVE NONREWARD IN PARTIALREINFORCEMENT AND DISCRIMINATION LEARNING:

SOME RECENT HISTORY AND A THEORETICAL EXTENSION1

ABRAM AMSEL

University of Toronto

The overall objective of this paperis to examine the current status of aconcept of frustrative nonreward inbehavior theory. The examinationtakes two forms: a general discussionand survey of recent work on mo-tivating and inhibiting properties ofnonreward, followed by a theoreticalextension of frustrative-nonrewardtheory to certain prediscriminationphenomena.

In the first part of the paper a briefhistorical account will be presented ofrecent changes toward an "active"conceptualization of nonreward, par-ticularly in neo-Hullian theory, the cur-rent status of such an active nonrewardconcept in explanations of partial re-inforcement effects will be set in abroader framework of partial rein-forcement theories, and some recentdata will be presented in support of theidea that discrimination learning in-volves frustrative-nonreward effects.

The second more theoretical portionof this paper stems from an earlierfrustrative conceptualization of non-reward (Amsel, 1958). It representsan attempt to extend frustrative-non-reward theory to situations in whichdiscrimination learning is preceded byvarious amounts and kinds of "priorexperience." These prior experiences

1A shorter version of this paper was pre-sented to the Psychology Colloquium of theUniversity of Illinois in May 1961. Thepreparation of this paper and much of theexperimental research to which it referswere supported by grants (G-SS27 andG-13895) from the National Science Founda-tion,

will be termed "prediscriminationtreatments" and will involve trainingto approach in the presence of one orboth (separately) of the eventualdiscriminada, under conditions ofpartial or of continuous reinforcement.

HISTORY

Hull's adoption of what he termedthe Mowrer-Miller hypothesis as abasis for a two-factor theory of inhibi-tion (Hull, 1943) was, at the sametime, the acceptance of a "passive" con-ception of nonreward and rejectionof a more active conception whichhe and others had held. Certainly,Pavlov's nonreinforcement, producinginternal inhibition, was an active con-ception. Spence's (1936, 1937) theoryof discrimination learning, clearly inthe Pavlovian tradition, also assignedspecial inhibitory properties to non-reinforcement. In some of his morerecent systematic writing, Spence(1956) points out that his position onthe Mowrer-Miller hypothesis wasnever in agreement with Hull's.

Emotional-motivational effects ofnonreward, as they relate to learning,were described in an article by Millerand Stevenson (1956), which attrib-uted the agitated behavior of rats dur-ing extinction to effects of nonrein-forcement carried over from one trialto the next. This is an example ofan active motivational conception ofnonreward. A similar emotional-moti-vational interpretation of nonrewardcan be found in Skinner (1938).

In Hull's 1943 theory, which domi-

306

FRUSTRATIVE NONREWARD 307

nated the psychology of learning in theforties, nonreinforcement was an eventwithout direct inhibitory or motiva-tional effect as it was also in theearly mathematical-model treatmentsof learning (Bush & Hosteller, 1951;Estes, 1950). By about 1950, how-ever, expressions of dissatisfactionwith the passive theoretical status ofnonreward were emerging. These ex-pressions were in theoretically orientedexperiments and in theoretical papers.Most of the early defections from 1943Hull, in this connection, were by per-sons close to the Hullian point of view.There are suggestions that Hull, him-self, was getting ready to change hisposition on this shortly before hisdeath (Hull, 1952, Ch. 5).

Rohrer's (1949) treatment of ex-tinction in terms of a "frustrationdrive" was one of the first expressionsof a desire to give nonreinforcement amore active role in inhibition.2 How-ever, while Rohrer's reference to frus-tration was specifically related to theoperation of nonreinforcement, it pro-vided no particular mechanism forfrustrative inhibition. At about thesame time, the well-known experi-ments of Virginia Sheffield (1949,1950) appeared, relating partial re-inforcement acquisition and extinctioneffects to the spacing of trials andcontaining suggestions of both frustra-tion drive and frustrative inhibition.These articles and another by Stanley(1952), along with a late experimentalpaper by Hull and others (Hull,Livingston, Rouse, & Barker, 1951)

2 It is questionable that "inhibition" is theappropriate and correct term to use in re-gard to response decrement related to frus-trative nonreward. Better terms would be"interference" or "competition," since frus-trative inhibition cannot be regarded as di-rectly affecting the excitatory tendency outof which it grows. It is rather a new ex-citatory tendency which competes with theolder one,

and Hull's (1952) last book, all con-tained at least the germ of a changetoward a more active conception ofnonreinforcement. An experimentalpaper by Denny and Dunham (1951)offered inhibition based on nonrein-forcement (frustration) as an alterna-tive to reactive inhibition to accountfor differential nonreinforcement ef-fects in a T maze.

Brown and Farber (1951) outlineda theory in which nonreward was oneof several antecedents to frustration.In this paper, the major emphasis wason the definition of frustration in termsof competing response tendencies andthe authors were concerned only withthe motivational (drive) properties offrustration, and not with' inhibitoryeffects. In the same year, a paper byAmsel (1951) made the point thatfractional anticipatory frustration (r^),which is the classically conditionedform of the frustrative reaction to non-reward, provides a mechanism for con-ceptualizing the active properties ofnonreinforcement and should be re-garded as a determiner of inhibitoryeffects. And, at about the same time,Seward (1951) published a theoreticalpaper which mentioned, without elabo-ration, the possible functional proper-ties of an anticipatory frustrationfactor.

The remainder of the decade, 1950to 1960, has seen a number of addi-tional experimental and theoretical at-tacks on the problem of nonreward.On the one hand there are experimentsand theoretical treatments by investi-gators in the Hullian tradition of be-havior theory which have sought toextend and revise Hull's theory in re-gard to the status of nonreward. Theother hand holds a number of publishedstudies representing a variety of theo-retical and nontheoretical interestswhich have contributed informationabout frustrative nonreward.

308 ABRAM AMSEL

Much of this work has been re-viewed earlier (Amsel, 1958; Lawson& Marx, 1958); but some of it is veryrecent. The experiments fall intothree categories: (a) demonstrationsof the motivating effects of frustrativenonreward, i.e., the carried-over, en-hancing effect of nonreward on thevigor of immediately following be-havior. This has been termed theFrustration Effect (FE). Severalstudies, by far the greatest number,have been directed not only at demon-strating the effect but also at examin-ing the variables determining its occur-rence and strength (Amsel, Ernhart, &Galbrecht, 1961; Amsel & Hancock,1957; Amsel & Penick, 1962; Amsel& Roussel, 1952; Bower & Stocks,1960; Longstreth, 1960; Marzocco,1951; Penney, 1960; Roussel, 1952;Seward, Pereboom, Butler, & Jones,1957; Wagner, 1959). (6) The sec-ond category consists of experimentswhich are mainly concerned with thefrustration drive stimulus, but whichalso touch on frustration reduction asreinforcement (Amsel & Prouty, 1959;Amsel & Ward, 1954; Tyler, Marx,& Collier, 1959). (c) And finally,there are a few studies which deal withthe inhibitory properties of frustrativenonreward. The inhibitory mecha-nism is fractional anticipatory frustra-tion (rF), a classically conditionedform of the goal response to frustrativenonreward. The mechanism and itstheoretical properties have been identi-fied in detail (Amsel, 1958; Spence,1960), and experiments testing impli-cations of such a mechanism are avail-able (Bower, 1961; Goodrich, 1959;Haggard, 1959; Wagner, 1961a,1961b). Of particular importance inthis connection are a series of experi-ments by Wagner (1961b) which gofar toward establishing the propositionthat frustrative nonreward has manyof the properties of punishment, and

that rp operates in many respects likefear. These experiments would supportsuch statements as (a) the greater thestrength of frustration the faster theextinction (avoidance) of a nonre-warded response; (b) cues paired withfrustrative nonreward acquire motiva-tional properties; and (c) the cessationof cues previously paired with frustra-tive nonreward serves to reinforce anew response.

There are several other experimentson nonreward (or reward prevention)as a variable in learning which providesupport for nonreward as a factor ininhibition. For example, studies byLambert and Solomon (1952) and byAdelman and Rosenbaum (1954)related extinction to the blocking ofinstrumental behavior at various dis-tances from the goal. An experimentby Adelman and Maatsch (1955)found that resistance to extinction de-pends to some extent on what the sub-ject is allowed to do when it finds noreward in the goal box, i.e., is frus-trated. The suggestion is that resist-ance to extinction is high when thesubject makes a response to frustrationwhich removes it from the frustratingsituation rather than one which keepsit in the apparatus. Ferster (1957,1958) found that stimuli signaling"time out" from positive reinforce-ment acquire aversive properties. Healso showed that the withdrawal of apositive conditioned reinforcer had thefunctional properties of punishment,suppressing the rate of respondingmuch like the presence of a negativereinforcer. These experiments pointclearly to the inhibitory (aversive)properties of nonreward.

Some recent experiments, whichshow decreasing resistance to extinc-tion with increasing numbers andmagnitudes of reinforcement, must beregarded as supporting a conceptionof nonreward as an active factor. For

FEUSTRATIVE NONREWARD 309

example, Hulse (1958) and Armus(1959) found that larger magnitudesof reinforcement in acquisition arefollowed by faster extinction. Veryrecent reports by Reynolds and Siegel(1961), by Pavlik (1961), and byWagner (1961a) were to the same ef-fect. North and Stimmel (1960) haveshown that rats reinforced 90 or 135times extinguish faster than those re-inforced 45 times. Similar resultshave been reported by Reynolds,Richter, and Carlock (1960). Theyfound faster extinction following .5-than following .1-gram reward, andfaster extinction after 66 acquisitiontrials than after 6, 12, or 30 trials.This kind of result is quite in accordwith results from studies of overlearn-ing and discrimination reversal, whichwe will look at later. Together, theyare as incompatible with early Hulliantheory, or any other theory requiringmonotonicity and positive relationshipsin these variables (e.g., the variousmathematical approaches), as were theearly partial reinforcement findings.We will come back to some of theseexperiments later, and my discussionof partial reinforcement effects willshow why a theory which includes afrustrative nonreward hypothesis mustpredict such results.

There are by now several experi-ments which can be taken to indicatethat nonreinforcement is more im-portant than reinforcement in discrimi-nation. Chronologically, these arestudies by Fitzwater (1952), Groveand Eninger (1952), Eninger (1953),Shoemaker (1953), Cantor and Spiker(1954), Grice and Goldman (1955),Birch (1955), and Solomon (1956),to mention some of the earlier ones.A very recent study which makes thissame point has been reported byLachman (1961). This is not theplace for a detailed description of thesestudies; however, the burden of evi-

dence is that avoidance of nonrewardis a more powerful factor in discrimi-nation than approach to reward; andthis is, of course, very relevant to apoint of view which makes nonrewarda determiner of inhibition. PerhapsHarlow and Hicks (1957) were re-acting not only to their own experi-mental findings but also to the weightof this kind of evidence when theyargued for a "uniprocess" rather thana "duoprocess" theory of discrimina-tion learning, the single process beinginhibition. The weakness of theirargument is that the negative process(r^, in my terms) may well depend onthe prior existence of a positive process(see Amsel, 1958; Spence, 1936,1937), and one is back to two proc-esses again.

Other examples of a return to anactive conception of nonreward outsideof the Hull-Spence camp can be foundin the recent work of Mowrer andEstes. Mowrer, in his LearningTheory and Behavior, makes frequentuse of the notion of nongoal eventsarousing "frustration" or "anger," inaddition to "disappointment" and ex-presses agreement with the position Ifavor in regard to frustrative goalevents (Mowrer, 1960, p. 409).Estes (1959) has recently acknowl-edged that statistical learning theorymust cope with the problem of anactive negative process based on non-reward in these words:

. . . I have found that in the naive animalan unreinforced trial produces no apparentchange in response probabilities, but thatafter a series of reinforced runs to a givenside of a T maze, the introduction of non-reinforcement yields a decrement in proba-bility of response to that side. . . . Appar-ently under partial reinforcement schedules,the effect of nonreinforcement varies be-tween these extremes . . ." (p. 417).

From this brief sketch of the recenthistory of the concept of nonreward,

310 ABEAM AMSEL

we move to a description of the roleof active conceptions of nonrewardin interpretations of noncontinuous(partial) reinforcement effects.

NONREWARD IN INTERPRETATIONS OFPARTIAL REINFORCEMENT

In his 1958 chapter on learning,Lawrence differentiates between "in-tertrial" and "intratrial" interpreta-tions of the Partial ReinforcementEffect (PRE)—the finding that partialor intermittent reinforcement in learn-ing leads to greater resistance toextinction than does continuous rein-forcement. Intertrial explanations at-tribute the PRE in one way or anotherto immediately carried-over traces ofstimulation from one trial to the nextin acquisition. According to such inter-pretations, it is on the basis of suchtraces that acquisition is more clearlydiscriminated from extinction by con-tinuously than by partially reinforcedsubjects. Intratrial explanations ascribethe PRE to some kind of learnedmechanism developing on each trial inacquisition and having its effect, as-sociatively, later in extinction. Therehave been two classes of intertrial ex-planations. In both cases the classescan be termed Cognitive-Expectancy(C-E) interpretations, and Stimulus-Response (S-R) interpretations.

Generally speaking, the intertrialinterpretations were an earlier develop-ment than the intratrial interpretations.Early C-E interpretations were the"common sense" expectancy hypothe-sis which emerged out of the earlyimportant experiments of Humphreys(e.g., 1939, 1940) and the discrimina-tion hypothesis developed and testedby Bitterman and his associates (e.g.,Bitterman, Fedderson, & Tyler, 1953).On the S-R side, the major intertrialexplanation (apart from the "responseunit" interpretation which applies tochained or highly massed responses

but not to discrete trial learning) wasthe Hull-Sheffield hypothesis tested bySheffield (1949). Weinstock's (1954)demonstration that partial reinforce-ment effects were not eliminated evenwhen trials were separated by 24hours, and an experiment by Wilson,Weiss, and Amsel (1955) disconfirm-ing the earlier results of Sheffieldhelped to set the stage for, and indeedto provide, intratrial explanations of,the PRE. Much later, in fact very re-cently, there has been strong contra-indication for a discrimination hy-pothesis in the work of Jenkins(1961b) and of Theios 1962).Jenkins, using pigeons in a Skinner-type situation, and Theios, using ratsin a runway, have found that the PREis sustained when a large number ofcontinuous reinforcements are inter-polated between discrete trial partialreinforcement and extinction. Thismeans that a large number of continu-ous reinforcements immediately be-fore extinction and following partialreinforcement does not neutralize thepartial reinforcement effect. Since thebasis for discriminating acquisitionfrom extinction is now the same forpartially as for continuously reinforcedsubjects, and the PRE persists, aninterpretation is required, as Theiosindicates, in terms of "relatively per-manent effects of nonreinforcementwhich can be sustained through blocksof continuous reinforcement."

Intratrial explanations of the PREconcern themselves in one way or an-other with processes following non-reward. Until recently, the intratrialexplanations were the exclusive prop-erty of S-R psychologists. Wein-stock's (1954) contiguity-interferencehypothesis was perhaps the earliest ofthese, but was followed in short orderby a variety of neo-Hullian hypothe-ses. All of these are what I wouldterm conditioning-expectancy hypothe-


ses as opposed to cognitive-expectancyhypotheses of the PRE. All dependupon constructs of the anticipatorygoal response type, representing re-sponses which develop in strengthwithin trials. There are three neo-Hullian interpretations of the PRE, allof which were first proposed to accountfor particular experimental findings.The first emerged out of a study byWilson, Weiss, and Amsel (1955) andwas subsequently elaborated and testedin some detail (Amsel, 1958). It at-tributes the PRE to the evocation offrustration by nonreward in partialreinforcement, to the conditioning ofanticipatory frustration (rF — SF) tocues in the instrumental sequence, andto the association of SF with approachresponses in acquisition. Logan's in-terpretation of the PRE (Logan,Beier, & Kincaid, 1956) is that itdepends on the extinction of rGto "postreinforcement-time cues."Kendler and his associates (Kendler,Pliskoff, D'Amato, & Katz, 1957)have offered an interpretation similarto the one I have favored, except theychoose a neutral designation of non-reward effects in acquisition and referto anticipatory nonreward and notto anticipatory frustration. Spence(1960) has recently come out in favorof the same kind of interpretation andhas employed the rff construct to ac-count for acquisition phenomena inpartial reinforcement. Consequently,it seems safe to say that a prominentinterpretation of the PRE in neo-Hullian writings is a conditioning-ex-pectancy position with identifies frac-tional anticipatory frustration as wellas fractional anticipatory reward.

Recently, Festinger (1961) hascalled attention to the importance of"insufficient rewards" in a cognitive-expectancy (cognitive dissonance) in-terpretation of the PRE. Cognitivedissonance (Festinger, 1957) is a hy-

pothesis which recognizes the activeproperties of nonreward. In its use ofdelay of reward procedures to enhanceattractiveness of a goal in a delay ofreward situation, Festinger's treatmentof insufficient rewards is very similarin conceptualization to some earlierwork of Olds (1953, 1956). On thebasis of experiments with children,Olds concluded that the reward valueof a secondary rewarding stimulus(Sr) could be increased—and hence thestrength of approach responses leadingto Sr could be increased—by delayingthe presentation of Sr after overlearn-ing. He showed that tokens whichcould ultimately be exchanged formore primary reward increased in re-ward value when their presentationwas delayed, if these delay trials fol-lowed a number of trials in which theirpresentation was immediate. Oldssuggested that such a procedure con-stitutes "practice at wanting" and thatsuch practice at wanting increases thevalue of that which is wanted (Sr).Festinger would seem to be taking thesame position in his animal research.In one of several experiments dealingwith dissonance in rats, he employs adouble runway in which a start box isseparated from a mid box by an alley,and the mid box is separated from anend box by another alley. Subjectsare delayed in the mid box in themanner of Holder, Marx, Holder, andCollier (1957) and Wagner (1959),and the finding is that the subjects de-layed in the mid box, before being al-lowed to run to the end box and findfood, will continue to run to the midbox longer on test trials than the sub-jects always fed in the mid box as wellas in the end box. Festinger's (1961)interpretation is that the delayed sub-jects develop dissonance as a result ofthe delay, and that the "dissonancetends to be reduced by developing someextra preference about something in

312 ABEAM AMSEL

the situation. The existence of thisextra preference leads to the strongerinclination to continue running duringextinction trials" (p. 9).

The position of Festinger, like theearlier one of Olds, is clearly that non-reward (or delay of reward) is anactive factor determining an increasedwant or attraction. However, thereseems to be an important difference inwhat increases in attractiveness be-tween Festinger's earlier treatmentsof dissonance in humans (Festinger,1957) and his application of the con-cept to the explanation of the PRE inrats (Festinger, 1961). While speci-ficity is not always a characteristic ofcognitive theory, it does seem asthough the 1957 treatment (and evenparts of the 1961 paper) did specifythat the violation of a cognitive expec-tation produces dissonance which isresolved by upgrading the value of theprevious goal object. However, in thetreatment of "insufficient rewards" interms of the prototype experiment withstart box, mid box, and end box, dis-sonance is said to develop in the midbox, where the subject never has foundfood; and dissonance is reduced "bydeveloping a preference for somethingabout the place where the dissonancewas introduced . . . by developingsome liking for the place where theywere not rewarded . . ." (p. 10).This latter position is used to explain,in terms of dissonance, why the sub-jects never fed in the mid box runlonger to the mid box on nonrewardtest trials than the subjects always fedthere before. The same thinking isthen carried over to explanation of thePRE. In terms of the S-R analysis,the subjects have been trained to ap-proach the mid box of Festinger'ssituation despite anticipatory frustra-tion (SF) cues because, in the periodprior to testing, they were always

finally fed in the end box. In C-Eterms the subjects developed an extrapreference for the nonreward situation—the mid box in which they had neverbeen rewarded. This would leave tobe answered such questions as: Whydoes successive nonreward result inextinction? And, how do the subjectslearn to discriminate cues leading toreward from cues leading to non-reward ?

The S-R treatment of nonreward ef-fects in partial reinforcement (Amsel,1958; Kendler, Pliskoff, D'Amato, &Katz, 1957; Spence, 1960; Wilson,Weiss, & Amsel, 1955) does not sug-gest that nonreward (or delayed re-ward) increases "wanting" in Olds'terms, or "preference about somethingin the situation" in Festinger's. Thistype of neo-Hullian approach is a de-scription of a mechanism operatingthrough anticipatory nonreward whichcan lead to either rapid extinction ofbehavior or to increased resistance toextinction, and which can either facili-tate discrimination learning or retardit. I will claim that neither increased"wanting" nor "preference"—in fact,no "common sense" cognitive inter-pretation—can account for all of theeffects which nonreward seems tohave.

The difference between the two posi-tions, in the cognitive language of Oldsand Festinger, is as follows: Whilethe C-E position has nonrewardincreasing attractiveness of the non-reward situation, the S-R positionimplies a decrease in attractiveness.The partial reinforcement effect is,according to Festinger, due to in-creased attractiveness. According tomy position it is the result of trainingto persist in behavior despite decreasedattractiveness. In some other respectsthe positions are similar: both makenonreward the important determinant


of the effect, and both stress the im-portance of developing (intratrial) as-sociative tendencies during acquisitionas determinants of ultimate resistanceto extinction. Neither necessitates anassumption that the subject discrimi-nates acquisition from extinction (Bit-terman, Fedderson, & Tyler, 1953).

The details of the S-R positionwhich I favor have been presentedelsewhere (Amsel, 1958) in the formof a sequence of hypotheses identifiedwith four stages of practice in dis-crete trial partial reinforcement or indiscrimination learning (the firstthree stages are essentially the samefor both). A fifth hypothesis whichapplies to discrimination learning hasbeen added, and this will be importantfor some data which follow. The es-sence of the position is outlined belowin relation to an apparatus we havebeen using. It was designed to dif-ferentiate the immediate motivating(energizing) effects of frustration ex-emplified by the so-called FrustrationEffect (FE) from fractional anticipa-tory frustration (rp), which representsthe inhibiting feature of frustration.The apparatus is essentially, two run-ways in series, either in a straight line(e.g., Amsel & Roussel, 1952) or inthe form of an L (e.g., Amsel &Hancock, 1957). The apparatus hasa start box (SB), a first runway(Runway 1), a first goal box (Gl), asecond runway (Runway 2), and asecond goal box (G2) in successivearrangement. Reward and nonrewardare manipulated in Gl, and the ap-paratus permits a separation of thefrustrated response and changes inrunning speed indicating rF (measuredin Runway 1), from the frustrationmotivated response and changes indi-cating the FE (measured in Run-way 2).

The sequence of hypotheses in termsof five stages of practice is as follows:

1. In Stage 1, rK —SR S is developingin Runway 1 with early rewards, andnonreward (in Gl) has no particulareffect, as measured by the FE inRunway 2.

2. After the development of rH — SBfor the Runway 1 response, nonre-wards elicit frustration, measured bythe FE in Runway 2. This is Stage 2.

3. When nonrewards elicit frustra-tion, the cues in Runway 1 previouslyevoking rK now also begin to evoke r$.In Stage 3 these antedating goal re-sponse tendencies are temporarily incompetition.

4. Since r^ and r^ cannot be elicitedseparately by differential cues in par-tial reinforcement, as they can in thelatter stages of discrimination learn-ing, and since the temporary conflictin partial reward training is resolvedin favor of running to the intermit-tently rewarding goal box, SF becomesassociated with the instrumental ap-proach response in Stage 4 of par-tial reinforcement training, providingthe mechanism for the partial rein-forcement effect. When extinction iscarried out, partially reinforced sub-jects have been trained to respond (ap-proach in the presence of antedating,

3 I employ the term, rR — SB, in the samemanner as it is used in an earlier develop-ment, and for the same reason: ". . . ro — sois a general term covering all types of ante-dating conditioned response. Separately,these might be designated r« — SB, fractionalanticipatory reward; rp — SP, fractional an-ticipatory punishment, usually termed fear oranxiety; and rp — SF, fractional anticipatoryfrustration" (Amsel, 1958, p. 102). Themeaning of TR — SH is, however, not differentfrom ra — SQ as Spence (1960) uses it in asimilar connection. All that is intended hereis a clarification of terminology in light ofthe fact that goal events may be positive(such as to elicit approach) or negative(such as to elicit avoidance).

314 ABRAM AMSEL

frustration produced stimuli), whereasconsistently reinforced subjects havenot. In discrimination situations thereis a basis for the differential elicitationof approach and avoidance by SB andSF respectively in Stage 4.

5. Following discrimination, whenTff consistently precedes nonreward,the FE following nonreward shouldgrow weaker, since nonreward is de-nned as frustrating only in the presenceof rB.

I have been suggesting that in a veryimportant area of the analysis of be-havior, the explanation of persistenceof behavior in the face of nonrewardsattributable to partial reinforcement,the role of nonreward as an activeprocess has emerged in both cognitive-expectancy and in S-R conditioninginterpretations. It would certainlyseem to me that this is an area wherethe clearer, better worked out S-R lan-guage and conceptualization might pro-vide a handle for a more workableexpectancy theory.

There is, in addition, the matter ofa basic difference between the cognitiveand S-R interpretations of the PRE:the former says nonreward increasesattractiveness; the latter says non-reward decreases attractiveness. Bothpositions can account for the PRE; bothcan account for faster extinction fol-lowing large rewards than followingsmaller rewards; both can account forfaster extinction following many re-wards than following fewer rewards.However, a cognitive theory such asFestinger proposes would have dif-ficulty with a finding of Hulse (1958)which has recently been confirmed byWagner (1961a). In Hulse's words:". . . large as compared with smallrewards produced greater resistance toextinction if partial reinforcement wereused, but less resistance to extinctionif continuous reinforcement were used. . ." (p. 56). In the cognitive lan-

guage of Festinger this would have tomean that the subjects develop "someextra preference about something inthe situation" (greater resistance toextinction) when rewards are smalland continuous; and that they do notdevelop the extra preference, in factquite the opposite, when rewards aresmall and partial. The result of Hulseand of Wagner is clearly in line withthe S-R interpretation in terms ofanticipatory responses: Extinction isfaster following continuous rewards oflarge than of small magnitudes becauseof the greater FE in extinction andsubsequent faster development of rF.However, in partial reinforcementthere is opportunity to elicit FE andto build rF during acquisition. Underthese circumstances, the larger magni-tude of reward permits the mecha-nism for the PRE—the connection ofSF to continued approach—to developmore strongly. Consequently extinc-tion should be slower following partialrewards of large magnitude than fol-lowing partial rewards of small magni-tude.

I am claiming that the S-R condi-tioning analysis of nonreward effectshas much to recommend it. The ques-tion is, How much? To break thequestion down: Does inclusion of theconcepts rK and rF in the analysis ofpartial reinforcement and discrimina-tion learning provide us with explana-tory power which a more parsimoniousapproach-avoidance analysis (e.g.,D'Amato & Jagoda, 1960; Nissen,1950) does not? Does this type ofanalysis of nonreward effects providepredictive power which a cognitivetreatment of nonreward (e.g. Fes-tinger, 1961) does not?

My attempts to convince the readerthat the answer to these last two ques-tions is "yes" will take two forms:(a) Some data will be presented whichbear on predictions already made on


the basis of this type of analysis. I willclaim that data such as these are notreadily deducible from either a lessspecific cognitive language or from amore parsimonious approach-avoidancelanguage. (£>) I will offer an exten-sion of frustrative nonreward theorydesigned to generate predictions aboutthe course of discrimination learningfollowing various prediscriminationexperiences with partial and continu-ous reward.

Before proceeding to this extensionof the theory of frustrative nonreward,a brief comment on the relation of thepresent position on nonreward to othertheories of frustration. As I haveindicated before (Amsel, 1958), ". . .we have been interested in certainactive properties of nonreward follow-ing reward and no more than this ismeant by the term frustration as weuse it" (p. 103). Nevertheless, themechanism which has been outlined toexplain the PRE, particularly the con-ditioning of rF and the connection ofSP to continued approach as a resultof intermittent reward, is certainly acandidate for the learning theorycounterpart of "frustration tolerance"as employed in social psychology andpsychopathology. The treatment offrustration from the viewpoint of theo-ries of personality is beyond the scopeof this paper; and there is no intenthere to grapple with the multiplemeanings the concept has in thosecontexts. However, it would seempossible to recast portions of the frus-tration theory of Rosenzweig (e.g.1944) which deal with nonreward(failure or delay of "gratification")into the present terminology; and thetreatment of frustration by Rotter(1954), as a discrepancy between re-ward expectancy and its occurrence,seems compatible with the mechanismshypothesized here and with the theo-retical extension I will propose.

NONREWARD EFFECTS INDISCRIMINATION LEARNING :

SOME DATA

According to the sequence of fivehypotheses on the role of nonrewardand partial reinforcement in discrimi-nation, it should be possible to showthat the FE, measured in Runway 2of a double runway, bears certain rela-tionships to various stages of discrimi-nation learning. More specifically,(a) evidence of nonreward relatedfrustration, measured in Runway 2,should precede any evidence that thesubjects are running faster to S+ andslower to S— in Runway 1, and (b) atsome point following the appearance ofdiscriminative behavior in Runway 1,a reduction in the magnitude of the FEmeasured in Runway 2 should becomeapparent.

In an unpublished doctoral disserta-tion, Ward (1961) employed thedouble runway to record discrimina-tion to positive (e.g., black) andnegative (e.g., white) cues presentedsingly in Runway 1, and the concur-rent development of frustrative effectsin Runway 2. A single runway condi-tion (Runway 1 alone) was used tostudy the same discrimination uncon-taminated by Runway 2 performance.There were 20 subjects in the doublerunway group and 10 subjects in thesingle runway comparison group.They were run six trials a day for 30days, and the intertrial interval wasabout 10 minutes. Figure 1 presentsdata for the 20 subjects in the doublerunway. The top panel shows thedeveloping FE in Runway 2. Thebottom panel shows discriminationstarting at about Day 7 (42 trials),and at a point well after the beginningof FE development on Day 3 (18trials). There is an indication that theFE almost disappears (perhaps onlytemporarily) at a point where separa-

316 ABRAM AMSEL

100

80

^ 40

i *

a.

I

100

80

2 60

20 Ss

FE - RUNWAY 2

DISC - RUNWAY 1

! \

10 15

DAYS

20 25 30

FIG. 1. The appearance and diminutionof the frustration effect in Runway 2 (toppanel) in sequential relationship to the de-velopment of the successive discriminationin Runway 1 (bottom panel). (From Ward,1961)

tion of the discrimination curves isgreatest. This occurs at about Day 15(90 trials).

It could be argued the Ward resultis an artifact: that different subjectsreach criterion at different times. An-ticipating this, I selected the data ofseven of Ward's subjects whichreached the discrimination criterion onthe same block of trials. Figure 2presents these data in 2-day blocksand shows essentially the same rela-tionships as in Figure 1.

Figure 3 presents data from thesingle runway comparison group andshows the Runway 1 discriminationunaffected by any subsequent Runway2 performance. Even in the absenceof the second runway there is a tend-ency for the N and R curves of thediscrimination to converge on latertrials (see Berry, Verplanck, &

Graham, 1943). These data of Ward,which have been duplicated in ourlaboratory as by-products of otherstudies, are taken as evidence for theinvolvement of frustrative effects indiscrimination in a manner such as ouranalysis suggests.

In another experiment using thesame double runway apparatus, Ward(1961) studied rate of discriminationlearning in Runway 1 as related tonumber of prediscrimination rewards.In this experiment, which was similarto an earlier one of Shoemaker (1953)performed in a single runway, the re-sponse of running in Runway 1 andentering Gj was rewarded 0, 12, or48 times prior to learning a black-white discrimination in Runway 1.An implication of our position is that,with initial positive tendencies to thetwo discriminanda equal, rate of dis-

100

so

I 60

£100

60

FE -RUNWAY 2

DISC • RUNWAY 1

1-2 5-6 9-10 13-U 17-18 21-22 25-26 29-30

2- DAY BLOCKS

FIG. 2. These are the relationships shownin Figure 1 for seven subjects selected be-cause they reached the criterion of discrimi-nation on the same block of trials. (Adaptedfrom Ward, 1961)


10 Ss

8DISC - RUNWAY 1

CONTROL

100

IS

I 80za, 60

z< f

5 10 15 20 25

DAYS

30

FIG. 3. The successive discrimination ina group which ran only in Runway 1. (Ex-treme N times going up to 3 or 4 secondsare not shown. Note the decline of the non-reward curve in the single runway group asin the double runway groups.)

crimination learning should be a posi-tive function of the strength of TR tothe negative stimulus. This meantthat prediscrimination trials were allrewarded in G^ and that Runway 1 wasblack on half of these trials and whiteon the other half. On discriminationtrials, Runway 1 was black when Gt

contained reward and white when Gt

contained no reward (or vice versa).Second runway performance was meas-ured on every discrimination trial fol-lowing reward or nonreward in Gt toget a measure of the FE, whosestrength at different stages could thenbe related to the developing discrimina-tive performance. Table 1 shows theresults of this experiment for bothrunning time and goal entry measuresfor two criteria of learning. A moredetailed analysis of the data in Ward'sdissertation also indicates the appear-ance of the FE before discriminationand its diminution or disappearancefollowing discrimination. These dataare in accord with, and go a littlebeyond, previously cited studies which

stress the importance of nonreward indiscrimination.

A recent experiment by Jenkins(1961a) demonstrates the importanceof nonreward effects in discriminationlearning in another way. Usingpigeons as subjects, a discrete trialprocedure, and key pecking as the cri-terion response, Jenkins demonstratedthat the positive stimulus elicits twiceas many responses to extinction follow-ing discrimination training as does thesame stimulus following an equal num-ber of continuous reinforcements. Onthe basis of an ingenious arrangementof controls, Jenkins is able to concludethat the increased resistance to extinc-tion is not due to the fact that thepositive stimulus is eventually dis-criminated from the negative, butrather to the occurrence of nonrein-forced responses in the process oflearning the discrimination.

FRUSTRATIVE NONREWARD THEORYAND PREDISCRIMINATION EFFECTS

In the Ward experiment, subjectswere exposed before a discrimination

TABLE 1MEAN DAY ON WHICH CRITERION OF DIS-

CRIMINATION REACHED BY GROUPSFOLLOWING 0, 12, OR 48 FEE-

DISCRIMINATION B+ W+TRIALS

ConditionGroup

0 12 48

F

Running time

Easy criterionHard criterion

8.012.8

6.510.3

4.58.6

3.64*5.26**

Goal entry time

Easy criterionHard criterion

710

29

4.38.1

4.16.1

36

22*14**

* £ < .05.**p < .01.

318 ABRAM AMSEL

to both of the stimuli, and approachresponses in the presence of both werealways rewarded. The Ward experi-ment represents continuous (predis-crimination) reward experience withboth discriminanda. Now let us extendfrustrative nonreward theory to a classof prediscrimination treatments broaderthan that represented by the Wardexperiment. '

Consider how discrimination learn-ing will be affected not only bycontinuous prediscrimination rewardexperience but also by partial pre-discrimination reward experience inrelation to one or to both of a pair ofdiscriminanda. I will restrict the dis-cussion, in this paper, to a black-white,successive discrimination (single pres-entation of stimuli). The problem isto make predictions about the courseof such a discrimination on the basisof some 'liistory of consistent or ofpartial reward experience in relation toone or both of the eventual discrimi-nanda. Discrimination is measuredin terms of amplitude (running time)changes to the positive and negativediscriminanda.

Table 2 lists several possible kindsof exposure to discriminanda prior toa black-white discrimination. As weexamine these cases, it is clear thattheorists of learning have been inter-

TABLE 2KINDS OF PREDISCRIMINATION EXPERIENCE

IN RELATION TO A BLACK (+)WHITE ( — ) DISCRIMINATION

One Prediscrimination Stimulus

B+ Continuous Reward, Positive StimulusW+ Continuous Reward, Negative

StimulusB± Partial Reward, Positive StimulusW± Partial Reward, Negative Stimulus

Two prediscrimination stimuli

B+ W+ Continuous Reward, Both StimuliB± W± Partial Reward, Both StimuliB — W+ Discrimination Reversal

ested mainly in prediscrimination ex-perience involving both discriminanda.An early example is the continuityversus noncontinuity experimentswhich varied amount of presolutionexperience with the discriminationproblem itself (B+W—) or with itsreverse (B—W+), and with receptororienting activity in relation to thediscriminanda (e.g., Ehrenfreund,1948; Catling, 1951; Krechevsky,1932, 1938; Lashley, 1942; Spence,1940, 1945). More recent work onprediscrimination treatments as theyinfluence later discrimination learninghas taken several forms includingHarlow's (e.g., 1949, 1950, 1957) ex-periments on learning sets, Lawrence's(1949, 1950, 1952) studies on acquireddistinctiveness of cues, and the workrelating overlearning to discriminationreversal and transfer (e.g., Birch,Ison, & Sperling, 1960; Capaldi &Stevenson, 1957; Pubols, 1956; Reid,1953). These newer interests in pre-discrimination experience have beensomewhat less oriented than the earlierones toward theoretical argument. Infact, they have to some extent tendedto bring together the earlier wide splitbetween S-R and cognitive approachesto discrimination, while investigatingmediational processes affecting dis-crimination. Perhaps a point has beenreached where it may be profitable tobegin looking at the prediscrimination-discrimination sequence in a waywhich will bring the cognitive-expect-ancy position even more into the foldof an S-R (conditioning-expectancy)interpretation. It seems to me that atreatment of discrimination phenomenain terms of anticipatory (conditioned)mechanisms does just this.

Consider the following "attributes"of prediscrimination exposure to asingle stimulus: (a) The prediscrimi-nation stimulus may become either thepositive or the negative stimulus in theeventual discrimination, (b) The re-


sponse to this prediscrimination stimu-lus may be partially or continuouslyrewarded, (c) The number of predis-crimination exposures to the stimulus-response-reward (or nonreward) se-quence may be large, small, or ofsome intermediate value.* Figure 4

* The values for number of trials, 10, 32,and 120, which appear in Figures 4, 5, and 6,and in the text in various places, are speci-fied here for purposes of exposition andbecause we are now conducting experimentswhich involve these numbers of prediscrimi-nation trials. However, these numbers oftrials only represent guesses, at the presenttime, as to when the various stages of prac-tice are reached. Rather than be pinneddown to these numbers of trials in the pre-dictions for discrimination learning whichfollow, I would prefer to set up independentindices of the existence of one stage oranother and to predict discrimination out-come on the basis of these indices ratherthan some number of trials. The index forthe first stage is relatively safe and simple.It is a stage at which nonreward producesno evidence of frustration as measured bythe FE in a double runway, or in any otherway. If there is no frustration, there canbe no rr. The presence of TV takes us intothe second stage. The choice of a numberof trials to designate the second stage ismost difficult since it must represent a verycritical phase in partial reinforcement:enough trials for rr to be present but notso many that SF is associated with approach.The independent indicator of the middlestage (TF, but SF not associated with ap-proach) is increased variability of behaviorsoon after nonreward produces the FE(Amsel, 19S8). Stopping partial rewardacquisition at such a point defines this middlestage. The late stage (identified here,conservatively, as 120 trials) has a clearindependent definition. It exists (a) whenvariability decreases again, but more particu-larly (b) when partial reward performanceexceeds continuous reward performance invigor (see Goodrich, 1959; Haggard, 1959;Spence, 1960; Wagner, 1961a). While itmight appear from earlier partial reinforce-ment experiments that 32 trials of partialreinforcement are sufficient to get us to thelast stage and produce the PRE (and there-fore are too many trials for our intermediatecondition) experiments demonstrating PREwith this many acquisition trails (e.g., Shef-field, 1949; Weinstock, 1954) have usually

^

POSITIVEIN DISCRIMINATION

PREOISCRIMINATIONSTIMULUS

NEGATIVEIN DISCRIMINATION

r ^yrfmitf-, APP / yV/'flHt Sp/lApp /

^'RM"°V / "7

Bi - B«W-

WJ -i B-W.

Ws -t B»W-

B;-.B-W.

B. afw

w.-. a-w.

VW-.B.W-

B.-.B-W.

/PARTIAL CONTINUOUS

PREDISCRIMINATION REWARD CONDITION

FIG. 4. Schema showing 24 prediscrimi-nation-discrimination sequences (12 condi-tions, each counterbalanced) resulting fromthree attributes of prediscrimination ex-posure to a single discriminandum.

shows the various possible combina-tions of a, b, and c for a black-whitediscrimination. The 12 cells of thisschema represent a matrix of possibleexperimental conditions, each counter-balanced for absolute effects of color.

What can be predicted from a be-havior theory with frustrative non-reward premises about the effects ofprediscrimination experiences in rela-tion to a single discriminandum onsubsequent black-white discrimination?Is it possible to show that all combina-tions of a, b, and c and also effects ofprediscrimination experience involvingboth discriminanda will be deduciblefrom a relatively small number of prin-ciples? In the next four sections, Iwill look at four kinds of prediscrimi-nation treatment and, for each one, tryto see what the theory leads us toexpect.

Partial Reward of a Response to OneDiscriminandum Prior to Discrimina-tion

Our guiding conception, when theprediscrimination treatment involves

involved an initial period of about 10 con-tinuous rewards. Our procedure involvespartial reward from the outset.

320 ABRAM AMSEL

partial reinforcement and a sufficientlylarge number of trials, is that antici-patory frustration produced stimuli(sF) become associated with approachresponses. (Miller, 1960, has sug-gested a similar mechanism—"learningto advance to the cue of fear"—to ac-count for the learning of resistance tofear.) The reasoning is that while Spwill at first elicit responses antagonisticto approach, it will finally become con-nected to approach tendencies. Thismeans that at some stage in partialreinforcement Sp will become part ofa stimulus complex evoking approach,and earlier evidence of conflict willdisappear.

Of the four stages of practice inpartial reinforcement previously out-lined, three are theoretically differenti-able with respect to the involvement ofrF, and therefore Sp: a stage where TRhas been conditioned, but not yet rF

(10 trials) ; a stage where both rB andrp have been conditioned and theirstimuli, SB and Sp, still evoke competing(approach and avoidance) responsetendencies (32 trials) ; and a stagewhere both TK and rF are conditionedand the stimuli they produce, SR andSF, both evoke approach tendencies(120 trials). This conceptualizationof three stages in partial reinforcementin regard to rp—Sp, along with anassumption borrowed from Miller'sanalysis of conflict and displacement(1944, 1948), will provide the basisfor some deductions which I findinteresting.

The assumption from Miller's analy-sis of conflict, for which there is goodexperimental support, is that the gen-eralization gradient for positive (ap-proach) tendencies is flatter than thatfor negative (avoidance) tendencies.If we now identify SR—»App (whichresults from TR) and Sp—>Av (whichresults from rp) as such positive andnegative tendencies, the three panelsof Figure 5, from top to bottom re-

STRENOTH OF RESPONSE TENDENCY AT

BEGINNING OF DISCRIMINATION FOLLOWING

PREDISCRIMINATION PARTIAL REWARD

TO BLACK IB'. )

iUl EARLY STABE

(10 TRIALS I

MIDDLE STAGE(32 TRIALS)

LATE STAGE1120 TRIALS)

BLACK WHITE

DISCRIMINATION STIMULUS

FIG. 5. An application of an assumptionfrom Miller's analysis of conflict to predic-tions of ease of discrimination learning fol-lowing various prediscrimination exposuresto a single discriminandum.

spectively, represent the state of af-fairs at the beginning of a discrimina-tion, when the discrimination has beenpreceded by a few, an intermediatenumber, or a large number of partialreinforcements of a response to theprediscrimination stimulus (B). Rep-resented on the left-hand side of thegraph are the relative strengths of ap-proach and avoidance tendencies tothe prediscrimination stimulus (B)through the mediating response pro-duced stimuli, SB and Sp. Shown onthe right are the generalized strengthsof these tendencies to the stimulus(W) which will be new in the dis-crimination. (If the prediscriminationstimulus were W, then B would be thenew stimulus in the discrimination andthe B and W positions on the baselinewould of course be reversed.)

A simplifying assumption, repre-sented in the bottom panel of Figure 5,is that the strength of approach, elicitedthrough rF—Sp by B and W in the late


stage, is related to the strength ofavoidance tendencies elicited throughrp—Sp in the middle stage. I am sug-gesting that the avoidance tendency toSF is "neutralized" and has the effectof an approach tendency at this stage.

Some predictions can now be maderelative to the conditions in Figure 4,and these will be restricted to theunderlined conditions. (The otherconditions simply counterbalance forany absolute effects of color and I as-sume they will not be different indirection.) Again, let me emphasizethat the predictions will differ depend-ing on the depth dimension: Wasprediscrimination training discontinuedat the stage of development of r^ butnot rF, indicated by the point of earliestappearance of FE? Or, was trainingcarried on to the intermediate conflictstage at which Sp evokes avoidance?Or, were a very large number of trialsrun, allowing SF to become connectedto approach ? Outlined below, for eachof these cases, are predictions as towhich discrimination will be learnedfaster: that in which the prediscrimi-nation stimulus remains positive in thediscrimination (B±—»B+W—), orthat in which it becomes negative(B±-»B-W+).

Let us begin with the case of dis-crimination following 120 partial re-wards. The reasoning (see Figure5) : At the start of discrimination, Bwill elicit ra and rF strongly and aboutequally. On the other hand, W, whichis new in the discrimination, will elicitTR relatively strongly because of theflat positive gradient, but will elicit rF

very weakly because of the steep nega-tive gradient. This means that whenB is made negative in the discrimini-nation after a great many prediscrimi-nation trials, B— will elicit strongrP—sp, and Sp will elicit continued ap-proach. This will slow discriminationrelative to the condition where B ismade positive in the discrimination.

Here the W— stimulus which is newin the discrimination elicits very weakrF. Consequently, there is little Spto elicit approach, little to counteractthe build-up of avoidance to W—, anddiscrimination should develop morequickly. The theory predicts the fol-lowing: After a very large number ofprediscrimination partial rewards ofan approach response to B, B+W—will produce faster discrimination thanwill B—W+. This prediction and itsrelationship to other predictions areshown in Figure 6.

The most interesting and criticalcomparison to be made for purposesof our analysis is between the 120-prediscrimination-trial case and theintermediate, 32-prediscrimination-trialcase. In the 120-trial case Sp is al-ready connected to approach, while inthe 32-trial case Sp still elicits avoid-ance. Consequently the predictions inthese two cases are reversed. The

IDz2cc

zo

i2 It- Ky oQ IQ

"• &

UJ

B»W'-»B'W-

t---

BiWJ-^B'W^"

10 32 120

NUMBER OF PRE-DISCRIMINATION TRIALS

FIG. 6. Predicted rate of discriminationlearning following various prediscriminationexposures to one or both of the eventualdiscriminanda.

322 ABEAM AMSEL

reasoning for the intermediate case isthat both rK and rF are being evokedby B at the start of discrimination, butSB elicits approach and SF elicits avoid-ance. When, after 32 trials, we switchfrom prediscrimination B± to theB+W— discrimination, B+ evokesboth approach and avoidance tenden-cies through SB and SF, and W—, newin the discrimination, evokes mainlygeneralized approach (rK) and littleavoidance (rF). When the shift isfrom B± to B—W+, there is a strongtendency for SF to elicit avoidance inrelation to the now negative B— (gen-eralizing very little to W+) and astrong tendency for SB to elicit ap-proach to the new, now positive, W+.This latter situation is much morefavorable for discrimination. The pre-diction : After an intermediate numberof partial rewards of an approachresponse to B, B+W— will pro-duce slower discrimination than willB-W+.

After a few B± trials, enough toproduce some rG but little or no FEand no rF, there should be no differ-ence in the rate of the B+W— andB—W+ discriminations.

Continuous Reward of a Response toOne Discriminandum Prior to Dis-crimination

We deal here with the underlinedconditions on the right of Figure 4:continuous (B+) prediscriminationexperience followed by a discrimina-tion in which B remains positive(B+W—) or becomes negative(B—W+). The expectations hereare most interesting when we comparethem to the predictions just madefor prediscrimination partial reward.Briefly, with increasing numbers ofrewards of approach responses to theprediscrimination stimulus, B+ shouldevoke increasing rK, and nonreward inthe subsequent discrimination should beincreasingly frustrating. Since TR gen-

eralizes strongly from B to W, both Band W should evoke rB quite stronglyand almost equally in the discrimina-tion. Nevertheless, the FE should besomewhat greater when B is made thenegative stimulus in the discriminationthan when it stays positive. However,in either case (B+-»B+W- or B+-=»B—W+), the flat gradient of gen-eralization of positive tendencies leadsto a prediction which is the same asthe earlier Ward result: increasing thenumber of continuous prediscrimina-tion rewards should produce faster dis-crimination.

In contrast to the center panel ofFigure 6 in which the relationships atthe middle and end points are reversed,the differences between the two curvesin the top panel are shown as consist-ently in favor of faster discriminationfor B+-*B+W-. This is not a pre-diction based on frustrative nonrewardfactors alone. In fact, part of this isnot a prediction at all. There is evi-dence in some unpublished data5 thata switch from B+ in prediscrimina-tion to B— in discrimination does pro-duce a greater FE in Runway 2 than aswitch from B+ to W—, as expected(Amsel & Hancock, 1957); however,an immediate generalization decrementof the Runway 1 response in switchingfrom B to W weakens response to W,which speeds the apparent separationof the B+W— discrimination curvesand retards the separation of theB—W+ curves. The suggestion isthat, in the case of prediscriminationcontinuous reinforcement, the factor ofgeneralization decrement, operatingagainst the somewhat greater FE whenthe prediscrimination stimulus be-comes negative in the discrimination,should maintain the relative advantageof B+-»B+W- over B+-»B-W+

5 The reference is to data collected bySusan Hicks at Newcomb College, TulaneUniversity, as part of an undergraduatehonors project.


for all numbers of prediscriminationtrials. In prediscrimination partial re-inforcement (middle panel of Figure6), since there is a switch-over in rela-tive positions from 32 to 120 trials, thefactor of generalization decrementshould reduce the apparent differenceat the intermediate point and increasethe reversed difference at the 120-trialpoint.

While some of the implications ofthe analysis of variables representedby the schema in Figure 4 are cur-rently being tested using a runwayapparatus with amplitude changes asthe response measures, we plan also totest these ideas in terms of rate of dis-crimination in two-choice situationswith single presentation of discrimi-nanda. In these latter situations, gen-eralization decrement effects due tointroduction of new stimuli would stillbe expected to effect running speed;however, they might have less effect ona correct-choice indicant of speed ofdiscrimination, and provide an addi-tional check on the implications of thisposition.

We have been looking at situationsin which the response to a singlestimulus cue is partially or continu-ously rewarded some number of times,and this stimulus then becomes eitherthe positive or the negative cue in adiscrimination. The last two sectionsof this paper will outline some of thefactors operating in prediscriminationexperience involving both discrimi-nanda.

Partial versus Continuous Reward ofResponses to Both Discriminanda Pre-sented Separately Prior to Discrimi-nation

The bottom panel of Figure 6 sum-marizes predictions about the rate ofdiscrimination learning following pre-discrimination experience with bothdiscriminanda presented separately. Iam suggesting that B+W-i--»B+W—

operates essentially like B+~*B+W—(or B+-H>B-W+), except that itwould not be susceptible to generaliza-tion effects, since neither B nor W isnew in the discrimination. There issome evidence related to this conditionfrom the Shoemaker and Ward experi-ments, but only for the part of the"function" covering the first twopoints. Analysis of B±W± as a pre-discrimination condition indicates adecreasing monotonic relationship be-tween rate of discrimination and num-ber of such prediscrimination trials.The reasoning is simply that partialreinforcement of approach responses toboth stimuli will result in conflictingapproach and avoidance response tend-encies to both B and W at the 32-trialstage; and the elicitation of approachresponses to SF in connection with bothB and W at the 120-trial stage.While the former represents a dif-ficult base on which to build discrimi-nation, as compared with B+W+, thelatter should be even worse for dis-crimination, since it involves themechanism for the PRE (SF connectedto approach) elicited by the negativestimulus in the discrimination. Conse-quently, a prolonged period of con-tinued approach to both B+ and W—would be expected in the discrimina-tion phase.

Discrimination Reversal as a "Pre-discrimination" Phenomenon

The evidence on discrimination re-versal is certainly not unequivocal;however, the weight of it suggests aU shaped relationship between ease ofdiscrimination learning (B+W—) andnumber of prior B—W+ trials. Thiswould look like the B±-^B+W-prediction (middle panel Figure 6)in direction. Reversal, at least upto the 32-trial point, would producesuccessively slower learning; but re-versal after many postcriterion trials,e.g., the 120-trial point, would facili-

324 ABRAM AMSEL

tate the formation of the reverseddiscrimination.

The discrimination reversal studiescited earlier have been conducted insituations involving joint presentationof stimuli. They involve two basicfindings. The first concerns ease ofdiscrimination following the first re-versal ; the second concerns the "learn-ing to learn" phenomenon—that suc-cessive reversals shorten the numbersof trials required to criterion. Forthe first of these findings, there is scantevidence in situations involving singlepresentation of stimuli (the presentcase); however the recent Birch, Ison,and Sperling study (1960) is impor-tant and relevant.

The conditions of the Birch, Ison,and Sperling experiment are closest tomeeting the boundary conditions of afrustrative nonreward analysis of re-versal as a "prediscrimination" phe-nomenon. Their subjects, rats, weregiven 11 trials each day, the first al-ways positive, the next 10 divided be-tween positive and negative accordingto random orders. Their measure wasresponse speed rather than choice, andtheir apparatus was essentially a run-way. They were trained initially toapproach a white platform and to avoida black one and, when they reached acriterion of discrimination defined interms of approach speed, half of thesubjects were switched to a B+W —discrimination immediately, the otherhalf after 12 additional days of over-training to B—W+. The averagenumber of trials to reach the originalcriterion for both groups was about 50(well beyond our 32-trial point). Thismeans that the overtrained group hadabout 180 trials to B-W+ (wellbeyond our 120-trial point) before itwas reversed and learned B+W — .The overtrained group learned the re-versed discrimination in significantlyfewer trials.

Interpreting their results, Birch,et al. (1960) state: ". . . the differ-ence between the two groups appearsto be located in their differential ratesof extinction to the previously rein-forced cue. No difference in rate ofacquisition to the newly positive cuewas found in the present data" (p. 39).These results have a rather obviousinterpretation in terms of a frustrativenonreward position; but the situationis complex and there are theoreticaldifficulties to overcome. The obviouspart of the interpretation is that theoverlearned group has developedstronger r.B to W+, which makessubsequent W— more frustrating,which facilitates the conditioning of Wto rF, which, finally, ensures the evoca-tion of avoidance responses by SF.This part is fine for the theory; how-ever, there is the matter of what hap-pens to SF, which presumably developsin the initial discrimination of theBirch, et al. experiment to the negativestimulus (B — ) and which is connectedto avoidance. Shouldn't the over-trained group have a stronger SF~*avoidance tendency in relation to Bthan the criterion group? Andshouldn't this retard the learning ofthe reversed discrimination in theovertrained group when B subse-quently becomes the positive stimulus?This difficulty remains for the theoryunless one assumes that rE (and SK—»approach) builds in strength veryslowly over many trials (at least over50) while rF (and SF—^avoidance) de-velops quickly and is maximal by about50 trials—this being the case despitethe fact that the growth of rF dependsupon the prior growth of rK. This iscertainly an interesting possibility.

SUMMARY

This paper begins with an accountof recent evidence for a concept offrustrative nonreward in behavior

FRUSTRATIVE NONREWAED 325

theory, and presents support for theposition that an "active" role must beassigned to nonreward in both motiva-tion and inhibition. This evidence isnot only from experiments specificallydesigned to study frustrative proper-ties of nonreward, but also from avariety of other kinds of experiments,including, among others, studies ofextinction as a function of magnitudeof reward and number of rewards, andexperiments in discrimination learningwhich demonstrate the relative impor-tance of nonrewarded and rewardedtrials.

The importance of an active concep-tion of nonreward in current thinkingis discussed with reference to simi-larities and differences between recentcognitive-expectancy and neo-Hullianinterpretations of the partial reinforce-ment effect. The suggestion is thatcurrent S-R conditioning interpreta-tions of these partial reinforcementphenomena, stressing the role of bothanticipatory reward and anticipatoryfrustration mechanisms, and involvinga more specific and better worked outlanguage, provide a more workable"expectancy" theory than do cognitiveinterpretations.

The contention is that current neo-Hullian approaches, employing condi-tioning-expectancy concepts, go beyondcognitive-expectancy approaches inboth specificity and predictive power.As evidence for the merit of such aposition, data are presented from dis-crimination learning experiments whichtest predictions from a conditioninganalysis of nonreward effects, and anextension of frustrative nonrewardtheory is presented which leads to pre-dictions about the course of discrimi-nation learning following various pre-discrimination experiences. Thesepredictions depend on the interactionof three attributes of prediscriminationexperience: (a) whether prediscrimi-

nation exposure is to the stimuluswhich is eventually positive or negativein the discrimination (or to both) ; ( b )whether the prediscrimination treat-ment involves continuous or partialreward; and (c) the number ofprediscrimination exposures (trials),representing different stages of de-velopment of anticipatory reward andanticipatory frustration.

REFERENCES

ADELMAN, H. M., & MAATSCH, J. L. Re-sistance to extinction as a function of thetype of response elicited by frustration./. exp. Psychol, 1955, 50, 61-65.

ADELMAN, H. M., & ROSENDAUM, G. Ex-tinction of instrumental behavior as afunction of frustration at various distancesfrom the goal. /. exp. Psychol., 1954, 47,429-432.

AMSEL, A. A three-factor theory of inhibi-tion: An addition to Hull's two-factortheory. Paper read at Southern Societyfor Philosophy and Psychology, Roanoke,1951.

AMSEL, A. The role of frustrative nonre-ward in noncontinuous reward situations.Psychol. Bull., 1958, 55, 102-119.

AMSEL, A., ERNHART, C. B., & GALBEECHT,C. R. Magnitude of frustration effect andstrength of antedating goal factors.Psychol. Rep., 1961, 8, 183-186.

AMSEL, A., & HANCOCK, W. Motivationalproperties of frustration: III. Relation offrustration effect to antedating goal fac-tors. /. exp. Psychol., 1957, 53, 126-131.

AMSEL, A., & PENICK, E. The influence ofearly experience on the frustration effect./. exp. Psychol., 1962, 63, 167-176.

AMSEL, A., & PROUTY, D. L. Frustrativefactors in selective learning with rewardand nonreward as discriminanda. /. exp.Psychol, 1959, 57, 224-230.

AMSEL, A., & ROUSSEL, J. Motivationalproperties of frustration: I. Effect on arunning response of the addition of frus-tration to the motivational complex./. exp. Psychol., 1952, 43, 363-368.

AMSEL, A., & WARD, J. S. Motivationalproperties of frustration: II. Frustrationdrive stimulus and frustration reductionin selective learning. /. exp. Psychol.,1954, 48, 37-47.

ARMUS, H. L. Effect of magnitude of re-inforcement on acquisition and extinction

326 ABRAM AMSEL

of a running response. /. exp. Psychol,1959, 58, 61-63.

BERRY, R. N., VERPLANCK, W. S., &GRAHAM, C. H. The reversal of discrimi-nation in a simple running habit. /. exp.Psychol., 1943, 32, 325-334.

BIRCH, D. Discrimination learning as afunction of the ratio of nonreinforced toreinforced trials. /. camp, physiol. Psy-chol., 1955, 66, 456-464.

BIRCH, D., ISDN, J. R., & SPERLING, S. E.Reversal learning under single stimuluspresentation. J. exp. Psychol., 1960, 60,36-40.

BlTTERMAN, M. E., FEDDERSON, W. E., &

TYLER, D. W. Secondary reinforcementand the discrimination hypothesis. Amer.J. Psychol, 19S3, 66, 456-464.

BOWER, G. H. A contrast effect in differ-ential conditioning. J. exp. Psychol., 1961,62, 196-199.

BOWER, G. H., & STOCKS, D. Graded frus-tration effect as a function of graded re-duction in reward. Paper read at Psycho-nomic Society, Chicago, 1960.

BROWN, J. S., & FARBER, I. E. Emotionsconceptualized as intervening variables—with suggestions toward a theory of frus-tration. Psychol. Bull., 1951, 48, 465-495.

BUSH, R. R., & MOSTELLEH, F. A mathe-matical model for simple learning. Psy-chol. Rev., 1951, 58, 313-323.

CANTOR, G. N., & SPIKER, C. C. Effects ofnonreinforced trials on discrimination learn-ing in preschool children. /. exp. Psy-chol, 1954, 47, 256-258.

CAPALDI, E. J., & STEVENSON, H. W. Re-sponse reversal following different amountsof training. /. camp, physiol. Psychol,1957, 50, 195-198.

D'AMATO, M. R., & JAGODA, H. Effects ofextinction trials on discrimination reversal./. exp. Psychol, 1960, 59, 254-260.

DENNY, M. R., & DUNHAM, M. D. Theeffect of differential nonreinforcement ofthe incorrect response on the learning ofthe correct response in the simple T-maze./. exp. Psychol, 1951, 41, 382-389.

EHRENFREUND, D. An experimental test ofthe continuity theory of discriminationlearning with pattern vision. /. camp,physiol Psychol, 1948, 41, 408-422.

ENINGER, M. U. The role of generalized ap-proach and avoidance tendencies in bright-ness discrimination. /. comp. physiol.Psychol, 1953, 46, 398-402.

ESTES, W. K. Toward a statistical theoryof learning. Psychol Rev., 1950, 57, 94-107.

ESTES, W. K. The statistical approach tolearning theory. In S. Koch (Ed.),Psychology: A study of science. Vol. 2.New York: McGraw-Hill, 1959.

FERSTER, C. B. Withdrawal of positive re-inforcement as punishment. Science, 1957,126, 509.

FERSTER, C. B. Control of behavior inchimpanzees and pigeons by time out frompositive reinforcement. Psychol. Monogr.,1958, 72(8, Whole No. 461).

FESTINGER, L. A theory of cognitive dis-sonance. Evanston, 111.: Row, Peterson,1957.

FESTINGEH, L. The psychological effects ofinsufficient rewards. Amer. Psychologist,1961, 16, 1-11.

FITZWATER, M. E. The relative effect ofreinforcement and nonreinforcement inestablishing a form discrimination. /.comp. physiol. Psychol., 1952, 45, 476-481.

CATLING, F. A study of the continuity ofthe learning process as measured by habitreversal in the rat. /. comp. physiol.Psychol., 1951, 44, 78-83.

GOODRICH, K. P. Performance in differentsegments of an instrumental responsechain as a function of reinforcement sched-ule. J. exp. Psychol, 1959, 57, 57-63.

GRICE, G. R., & GOLDMAN, H. M. General-ized extinction and secondary reinforce-ment in visual discrimination learningwith delayed reward. /. exp. Psychol,1955, 50, 197-200.

GROVE, G. R., & ENINGER, M. U. The rela-tive importance of approach and avoidancetendencies in brightness discriminationlearning. Paper read at Midwestern Psy-chological Association, Chicago, 1952.

HAGGARD, D. F. Acquisition of a simplerunning response as a function of partialand continuous schedules of reinforcement.Psychol. Rec., 1959, 9, 11-18.

HARLOW, H. F. The formation of learningsets. Psychol. Rev., 1949, 56, 51-65.

HASLOW, H. F. Analysis of discriminationlearning by monkeys. /. exp. Psychol,1950, 40, 26-39.

HARLOW, H. F., & HICKS, L. H. Discrimi-nation learning theory: Uniprocess vs.duoprocess. Psychol. Rev., 1957, 64, 104-109.

HOLDER, W. B., MARX, M. H., HOLDER, E.E., & COLLIER, G. Response strength asa function of delay of reward in a runway./. exp. Psychol, 1957, 53, 316-323.

HULL, C. L. Principles of behavior. NewYork: Appleton-Century, 1943.

HULL, C. L. A behavior system. NewHaven: Yale Univer. Press, 1952.


HULL, C. L., LIVINGSTON, J. R., ROUSE, R.O., & BARKER, A. N. True, sham, andesophageal feeding as reinforcements./. comp. physiol. Psychol., 1951, 44, 236-245.

HULSE, S. H., JR. Amount and percentageof reinforcement and duration of goalconfinement in conditioning and extinc-tion. /. exp. Psychol., 1958, 56, 48-57.

HUMPHREYS, L. G. The effect of randomalternation of reinforcements on the ac-quisition and extinction of conditioned eye-lid reactions. /. exp. Psychol, 1939, 25,141-158.

HUMPHREYS, L. G. Extinction of condi-tioned psycho-galvanic response followingtwo conditions of reinforcement. /. exp.Psychol, 1940, 27, 71-76.

JENKINS, H. M. The effect of discrimina-tion training on extinction. /. exp. Psy-chol., 1961, 61, 111-121. (a)

JENKINS, H. M. Resistance to extinctionwhen partial reinforcement is followed byvarious amounts of regular reinforcement.Paper read at Psychonomic Society, NewYork, 1961. (b)

KENDLER, H. H., PLISKOFF, S. S., D'AMATO,M. R., & KATZ, S. Nonreinforcementsversus reinforcements as variables in thepartial reinforcement effect. /. exp. Psy-chol., 19S7, 53, 269-276.

KRECHEVSKY, I. "Hypotheses" in rats.Psychol. Rev., 1932, 39, 516-532.

KRECHEVSKY, I. A study of the continuityof the problem-solving process. PsycholRev., 1938, 45, 107-133.

LACHMAN, R. The influence of thirst andschedules of reinforcement-nonreinforce-ment ratios upon brightness discrimination./. exp. Psychol, 1961, 62, 80-87.

LAMBERT, W. W., & SOLOMON, R. L. Ex-tinction of a running response as a func-tion of distance of block point from thegoal. /. comp. physiol Psychol, 1952, 45,269-279.

LASHLEY, K. S. An examination of the"continuity theory" as applied to discrimi-nation learning. /. gen. Psychol,, 1942,26, 241-265.

LAWRENCE, D. H. Acquired distinctivenessof cues: I. Transfer between discrimina-tions on the basis of familiarity with thestimulus. /. exp. Psychol, 1949, 39, 770-784.

LAWRENCE, D. H. Acquired distinctivenessof cues: II. Selective association in aconstant stimulus situation. /. exp. Psy-chol, 1950, 40, 175-188.

LAWRENCE, D. H. The transfer of a dis-crimination along a continuum. /. comp.physiol Psychol, 1952, 45, 511-516.

LAWRENCE, D. H. Learning. Annu. Rev.Psychol., 1958, 9, 157-188.

LAWSON, R., & MARX, M. H, Frustration:Theory and experiment. Genet, psychol.Monogr., 1958, 57, 393-464.

LOGAN, F. A., BEIER, E. M., & KINCAID,W. D. Extinction following partial andvaried reinforcement. /. exp. Psychol,1956, 52, 65-70.

LONGSTRETH, L. E. The relationship betweenexpectations and frustrations in children.Child Develpm., I960, 31, 667-671.

MARZOCCO, F. M. Frustration effect as afunction of drive level, habit strength anddistribution of trials during extinction.Unpublished doctoral dissertation, StateUniversity of Iowa, 1951.

MILLER, N. E. Experimental studies of con-flict. In J. McV. Hunt (Ed.), Personalityand the behavior disorders. New York:Ronald, 1944.

MILLER, N. E. Theory and experiment re-lating psychoanalytic displacement tostimulus-response generalization. /. ab-norm. soc. Psychol, 1948, 43, 155-178.

MILLER, N. E. Learning resistance to painand fear: Effects of overlearning, ex-posure, and rewarded exposure in context./. exp. Psychol, 1960, 60, 137-145.

MILLER, N. E., & STEVENSON, S. S. Agi-tated behavior of rats during experimentalextinction and a curve of spontaneous re-covery. /. comp. Psychol, 1936, 21, 205-231.

MOWRER, O. H. Learning theory and be-havior. New York: Wiley, 1960.

NISSEN, H. W. Description of learned re-sponses in discrimination behavior. Psy-chol Rev., 1950, 57, 121-131.

NORTH, A. J., & STIMMEL, D. T. Extinc-tion of an instrumental response follow-ing a large number of reinforcements.Psychol. Rep., 1960, 6, 227-234.

OLDS, J. The influence of practice on thestrength of secondary approach drives./. exp. Psychol, 1953, 46, 232-236.

OLDS, J. The growth and structure ofmotives. Glencoe, 111.: Free Press, 1956.

PAVLIK, W. B. The effects of deprivationschedule and reward magnitude on per-formance during acquisition and extinc-tion. Paper read at Eastern Psychologi-cal Association, Philadelphia, April 1961.

PENNEY, R. K. The effects of non-rein-forcement on response strength as a func-tion of number of previous reinforcements.Canad. J. Psychol., 1960, 14, 204-215.

328 ABRAM AMSEL

PUBOLS, B. H., JR. The facilitation of visualand spatial discrimination reversal byoverlearning. /. comp. physiol. Psychol,1956, 49, 243-248.

REID, L. S. The development of noncontinu-ity behavior through continuity learning./. exp. Psychol., 1953, 46, 107-112.

REYNOLDS, W. F., RICHTER, M. L., & CAR-LOCK, J. Resistance to extinction as afunction of reward magnitude and numberof training trials. Paper read at Psycho-nomic Society, Chicago, 1960.

REYNOLDS, W. F., & SIEGEL, M. H. Drivereversal effect as a function of magnitudeof reward. Psychol. Rep., 1961, 8, 135-141.

ROHRER, J. H. A motivational state result-ing from non-reward. /. comp. Psychol.,1949, 42, 476-485.

ROSENZWEIG, S. An outline of frustrationtheory. In J. McV. Hunt (Ed.), Person-ality and the behavior disorders. NewYork: Ronald, 1944.

ROTTEK, J. B. Social learning and clinicalpsychology. New York: Prentice-Hall,1954.

ROUSELL, J. Frustration effects as a func-tion of repeated non-reinforcements and asa function of the consistency of reinforce-ment prior to introduction of nonrein-forcement. Unpublished master's thesis,Tulane University, 1952.

SEWARD, J. P. Experimental evidence forthe motivating function of reward. Psy-chol. Bull., 1951, 58, 130-149.

SEWASD, J. P., PEEEBOOM, A. C., BUTLER,B., & JONES, R. B. The role of prefeed-ing in apparent frustration effect. /. exp.Psychol, 1957, 54, 445-450.

SHEFFIELD, V. F. Extinction as a functionof partial reinforcement and distribution ofpractice. /. exp. Psychol, 1949, 39, 511-526.

SHEFFIELD, V. F. Resistance to extinctionas a function of the distribution of extinc-tion trials. /. exp. Psychol., 1950, 40, 305-313.

SHOEMAKER, H. A. The relative efficiencyof rewarded and non-rewarded training ina black-white discrimination problem.Unpublished doctoral dissertation, Univer-sity of Colorado, 1953.

SKINNER, B. F. The behavior of organisms.New York: Appleton-Century, 1938.

SOLOMON, R. L. The externalization ofhunger and frustration drive. /. comp.physiol. Psychol., 1956, 49, 145-148.

SPENCE, K. W. The nature of discrimina-tion learning in animals. Psychol. Rev.,1936, 43, 427-449.

SPENCE, K. W. Analysis of the formationof visual discrimination habits in thechimpanzee. /. comp. Psychol., 1937, 23,77-100.

SPENCE, K. W. Continuous versus non-continuous interpretations of discrimina-tion learning. Psychol. Rev., 1940, 47,271-288.

SPENCE, K. W. An experimental test of thecontinuity and non-continuity theories ofdiscrimination learning. /. exp. Psychol.,1945, 35, 253-266.

SPENCE, K. W. Behavior theory and condi-tioning. New Haven: Yale Univer. Press,1956.

SPENCE, K. W. Behavioral theory andlearning. Englewood Cliffs: Prentice-Hall, 1960.

STANLEY, W. C. Extinction as a functionof the spacing of extinction trials. /. exp.Psychol, 1952, 43, 249-260.

THEIOS, J. The partial reinforcement effectsustained through blocks of continuousreinforcement. /. exp. Psychol, 1962, inpress.

TYLER, D. W., MARX, M. H., & COLLIER, G.Frustration stimuli in discrimination. /.exp. Psychol, 1959, 58, 295-301.

WAGNER, A. R. The treatment of inhibitionin the more complex behavioral situation.Paper read at American Psychological As-sociation, 1961. (b)

WAGNER, A. R. The role of reinforcementand nonreinforcement in an "apparentfrustration effect." /. exp. Psychol, 1959,57, 130-136.

WAGNER, A. R. Effects of amount and per-centage of reinforcement and number ofacquisition trials on conditioning and ex-tinction. /. exp. Psychol, 1961, 62, 234-242. (a)

WARD, J. S. An experimental investigationof the role of frustrative factors in dis-crimination learning. Unpublished doc-toral dissertation, Tulane University, 1962.

WEINSTOCK, S. Resistance to extinction ofa running response following partial re-inforcement under widely spaced trials./. comp. physiol. Psychol, 1954, 47, 318-322.

WILSON, W., WEISS, E. J., & AMSEL, A.Two tests of the Sheffield hypothesis con-cerning resistance to extinction, partialreinforcement, and distribution of practice./. exp. Psychol, 1955, 50, 51-60.

(Received November 20, 1961)

Documents

Frustrative Nonreward in Partial Reinforcement and Discrimination Learning