JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR

IMPULSE CONTROL IN PIGEONS'G. W. AINSLIE

HARVARD UNIVERSITY

Pigeons were given a small, immediate food reinforcement for pecking a key, and a larger,delayed reinforcement for not pecking this key. Most subjects pecked the key on more than95% of trials. Howvever, wvhen pecking a differently colored key at an earlier time preventedthis option from becoming available, three of 10 subjects consistently pecked it, therebyforcing thenmselves to wait for the larger reward. They didl not peck the earlier key when itdid not prevent this option. This is an experimental exanmple of psychological impulse anda learnable device to control it. Althotigh only a miniority of the subjects learned it, thefact that such learning is possible at all argues for a theory of delayed reward that can pre-dict change of preference as a functioni of elapsing titme.

The phenomenon of impulsiveness or failureto delay gratification is often discussed insociology (Straus, 1962), economics (Strotz,1956), and clinical psyclhology (Freud, 1958;Mischel and Metzner, 1962). An impulse canbe defined as the choice of a small, short-termgain at the expense of a large, long-term loss.It often is described by such terms as spend-thriftiness, temptation, or seduction. Althoughmany impulsive belhaviors involve the takingof risks or the weiglhing of a reward againsta punishment, their common clharacteristic isthat they trade a higlh overall expectancy ofgain for a low one. Mowrer and Ullman (1945)advanced the hypotlhesis that impulsivenessarises from a basic property of reward that canbe observed in lower animals. This property isa decline in the effectiveness of a reinforcementas it is delayed from the moment of choice,and is usually described as a steep, negativelyaccelerated curve (Kimble, 1961, pp. 140-144;Renner, 1964). Such a curve suggests thatsmall, early reinforcements may have a greatadvantage over muchi larger, delayed ones,and raises the question of how impulsive be-havior is ever prevented.The answer may lie in the shape of the de-

lay function. In the last decade, several para-

'The work reported here has been supported by agrant to Harvard University from the National Instituteof Mental Health (MH-15494-04). I thank RichardHerrnstein and Howard Rachlin for advice and DonaldLoveland for technical assistance. Reprints may be ob-tained from G. W. Ainslie, Massachusetts MentalHealth Center, 74 Fenwood Rd., Boston, Mass. 02115.

metric experiments dealing with preferencefor various amounts of reinforcement atvarious delays have generated highly concavecuirves of reinforcing effect as a function ofdelay (Chung and Herrnstein, 1967; Killeen,1970; Logan, 1965; Shimp, 1969). One pre-diction that can be made from these curves isthat preference for some alternative reinforce-ments at different delays can change simply asa function of elapsing time. For instance,Chung and Herrnstein (1967) found that foodreinforcements were preferred in inverse pro-portion to their delay. It would follow thatif a reinforcement were available at time T,and an alternative reinforcement that wasthree times as great were available at T + 3sec, a subject making the choice at T - 3 secwould choose the larger, later reinforcement,while if it chose at T - 1 sec it would choosethe smaller, earlier one.

If there exist cases where preference changessimply as a function of the time the choice ismade, they pose for a subject the problem ofbinding its own future freedom of choice. Anyeffective device for getting the later, largerreinforcement must include a means of eitherpreventing preference from changing as thesmaller, earlier reward comes close, or keepingthe subject from acting on this change. If thesubject lacked the ability to do this on its own,it could be expected to learn a device to do itif an experimenter offered one. This predictionwould not depend on the existence of "higherintelligence", but only on whether the effectof the larger reinforcement was great enough

485

1974, 21, 485-489 NUMBER 3 (MAY)

G. W. AINSLIE

to cause learning of the pre-committing deviceat the time the device was available. If thesubject's preference did not change regularlyover time, a device to bind the behavior wouldnot be differentially reinforced and thus wouldnever be learned.The author has previously found evidence

that pigeons can learn a pre-committing de-vice (described in Rachlin, 1970). Subjectscould avoid the option of getting a reduced,immediate food reinforcement by pecking akey several seconds before the option was avail-able. But only two birds did this out of 10tested. Rachlin and Green (1972) were ableto increase this proportion by presentingpigeons with two keys, a committing and anon-committing key, the consequences beingdetermined by whichever key received thebird's twenty-fifth peck. However, interpreta-tion of this experiment is complicated by thefact that the time required by most of the birdsto peck 25 times was much longer than thelongest possible delay to which they mightcommit themselves, so that to obtain "im-mediate" reinforcement they would have tostart pecking a long time before this rewardwas actually available. In order to study thepre-committing response in its most elementaryform, the present experiment again used dis-crete trials with a single key. Because a largenumber of birds could not be run for the neces-sary length of time, potential subjects werescreened to find those that learned pre-commit-ment, and those that did were studied ex-tensively.

METHODSubjectsTen White Carneaux pigeons, one of which

had been in the most recent pilot study andnine previously naive birds that had just beentrained to peck a red key for food with everypeck reinforced, were kept at 80% of theirfree-feeding weights.ApparatusThe subjects performed in a sound-proof

chamber that measured 30 by 32 by 33 cm. Inone wall was a single key, which was centeredat a height of 24 cm and required 14 g (1.4 N)to operate. The key could be illuminated frombehind by green or red 7-W Christmas-treebulbs. Ten cm beneath the key was a 5 by 6

cm niche in the wall, into which a hopper ofgrain could be raised by an electromagnet.Centered in the top of the chamber was a one-way viewing lens, of the kind often set into thefront door of apartments. It measured 1 cmin diameter and permitted the experimenter tosee most of the chamber without being seenfrom inside. No houselight was used. Whitenoise was piped into the box to mask environ-mental sounds.

General ProcedureEach trial lasted 19 sec, regardless of the

subject's behavior. Each subject had 50 trials aday. Each subject served as its own control.Seven subjects that did not meet criteria de-scribed below were eliminated. The three re-maining subjects, including the bird from thepilot study, each ran in three successive experi-ments that differed from one another only inthe kind of control condition used (Figure 1).The birds went twice from the experimentalcondition to each control condition and backagain, for a total of 12 changes over a periodof 23 months. The temporal parameters usedwere those that had been the most productivein previous pilot experiments.Experimental condition. At the beginning

of each trial, the key was illuminated green

EXPERIMENTAL

BLANK --14 SEC-FOOD

NIci :IN OPECKNO PECK 1! NO PECK

I 7.5 SEC 14 EC4 SEC 4 SEC IGREEN LUGHT BLANK WED FOOO

CONOL I

I JOOD BLANKNO PECK

BLANK 3SEC 4 EC IGDMa FOODFOOD

NO PECK 1I NO PECK7.5 SEC 4.5 SEC 1 3 8ECI 4 SEC I

UGHT BLANK 'RED FOOD'

CONTROL E

FOOO

N PFOOD'NO PECK It PECK

1 7.ESEC I 4.5 SC SECI 4 SEC IIREEN UGHT BLANK 'RED FOOD

8|fSC *BLANKIa NO PECKSEC 4SEC IRED LH' FOODCONTROt m

I 7. $ E EGREEN UGHT BLNK FOODFig. 1. Diagram of the conditions. Events proceed in

a horizontal line. A response caused a vertical move toa different horizontal line.

486

IMPULSE CONTROL IN PIGEONS

for 7.5 sec. If it was not pecked, the key wentdark after the 7.5 sec and remained so foranother 4.5 sec, after which it was illuminatedred for 3 sec. If it was still not pecked, it wentdark at the end of the 3 sec and the subjectwas given access to grain for the next 4 sec,which were the last 4 sec of the trial. Peckingthe key while it was green made the key godark and prevented it from being lit red laterin the trial. The subject was then given accessto grain for the last 4 sec of the trial. Peckingthe key while it was red made it go dark andgave the subject access to grain for the next2 sec. The key then remained dark and thesubject had no further access to grain for therest of the trial. Since it takes pigeons about1 sec to get to the grain in this kind of appa-ratus, the actual reinforcements were prob-ably about 3 sec versus 1 sec of feeding.

Control I. This was the same as the experi-mental condition, except that a peck on thekey while it was green did not prevent itfrom being lit red later in the trial. Thus, ithad no pre-committing effect. A subject wasshifted to the Control I when it had pecked thegreen key on more than 40% of trials for threeconsecutive five-day periods. It went from theControl I to the experimental condition whenit had pecked the green key on fewer than 30%of trials for three consecutive five-day periods.Two birds that pecked the red key on fewerthan 90% of trials, four birds that neverpecked the green key enough to be shiftedinto Control I, and one bird that never peckedthe green key on fewer than 95% of trials werediscarded after 3000 trials (Figure 2, bottomrow).

Control II. This was the same as the experi-mental condition, except that a peck on thekey while it was red had no effect, other thanto darken the key. Subjects thus always re-ceived the later, larger reinforcement. Thiscontrol served the same purpose as Control I,except that it would also rule out the possi-bility that the birds pecked the green key justto control the appearance of the red key per se.Beginning with the first run of Control II, allbirds were given 40 trials a day for 48 daysbefore the condition was changed, regardlessof their behavior. The reduction in the num-ber of trials per day was made to prevent thebirds from gaining too much weight, since theyhad 4-sec access to food on every trial. Thenumber of days was fixed to eliminate any

question of artifact arising from long-term,random changes in their green-key pecking.

Control III. This was the same as the experi-mental condition, except that a peck on thekey while it was green was necessary for the keyto be lit red later in that trial. Thus, not peck-ing the green key in Control III had the sameeffect as pecking it in the experimental con-dition. This condition was designed to findwhether choice was changing haphazardlyduring the period the key was green, causingpecking whenever that alternative was tran-siently preferred.

RESULTS AND DISCUSSIONEight of the 10 original birds initially pecked

the key when it was red on more than 95% oftrials. They all continued to peck it onvirtually all the trials in which it appeared foras long as they remained in the experiment.During Control II, when the key lit-up red onall trials, no bird pecked it on fewer than 95%of trials in any six-day period. During theexperimental condition and in Control I andControl III, when the appearance of the redkey depended on the subject's previous behav-ior, no subject pecked it on fewer than 95%of the trials in which it appeared in any six-day period (or five-day period, before the num-ber of trials in a day had been reduced to 40).Thus, the tendency to seek the shorter access tofood when it was immediately available re-mained strong after as many as 20,000 trials.However, casual observation through the one-way viewing lens often revealed the subjects tobe pecking the chamber wall near the keywhile it was red, before pecking the key itself.This may be an example of the "mediatingbehavior" that has been described duringdifferential-reinforcement-of-low-rate respond-ing (DRL: Bower, 1961; Kramer and Rilling,1970).The percentage of trials on which subjects

pecked the key when it was green is shownin Figure 2. Each of the three subjects thatinitially pecked it on a large proportion oftrials during the experimental condition andon a small proportion of trials during ControlI maintained or increased this difference be-tween experimental and control conditionsthroughout the experiment. During the exper-imental condition, when pecking the greenkey was required to avoid the option of getting

487

G. W. AINSLIE

so

so

40

20

w

x

-0

4tit.

42

100

so 81BID6 407 40 67

40

IA

12 12 12BLOCKS OF TRIALS

Fig. 2. Percentage of trials on which sithe key while it was green, for each fi(250 trials; six-day periods, containing 2arrow). Data from the two subjects thatcriterion response rates on the red key ar

the shorter, earlier access to food,rose to between 50 and 90% of trControl I and Control II, whengreen key did not affect the occurroption, pecking the green key felower levels. A similar fall was ,Control III, when pecking the gr(required in order for the option ofshorter, earlier access to food to oThus, three subjects worked

points in the trial to obtain and tsame event. However, the food itshave been the only important reir

I ment of this event. Recent experiments haveCONTROL

m made it clear that pigeons have a strong tend---CONTROL Z

ency to peck a key that once has beenassociated with food, even when it currentlyreduces or eliminates their food intake (Fan-tino, 1966; Gamzu and Williams, 1973; Wil-liams and Williams, 1969). Therefore, it is notpossible to say what part of the reinforcementfor pecking the key when it was red came fromthe food itself, and how much from the chanceto peck a food-related key. The attractivenessof the earlier reinforcement may have been

augmented by the chance to peck a key to getit, or changed in either direction by the recentconsumption of food during the precedingtrial. Wlhat is remarkable is that some pigeonslearnedl to peck a key at an earlier time, if andonly if this made them unable to obtain thesmaller reinforcement. At one point in the cy-cle, these subjects had a great tendency to peckthe key for an early reinforcement, but at agreater distance they worked to forestall thistendency. This seems to be a true example ofimpulsiveness and impulse control as the

oo s ~terms are generally used.Preference for pecking the key may not

have been constant during the 3-sec period it20 was red. The observation of mediating be-

haviors during this time suggests that it wasnot, since these presumably were attempts towithhold key-pecking behavior. But since anychange of preference while the key is red con-fronts the subject with the problem of impulse

. control, this distinction is not important. In12 12 either case, if the subject prefers the later,ubjects pecked larger reinforcement at first, it will need toive-day period adopt some kind of pre-commitment in order40 trials, after to get it.did not reach The majority of subjects might have failed^e omitted. to learn the pre-committing device because the

effectiveness of the reinforcements used de-this pecking clined in such a way that preference neverials. During changed. However, it may be that the differ-pecking the ential effectiveness of the reinforcements dur-rence of this ing the time the key was green was too small11 to much to allow most pigeons to learn a pecking task;seen during or these contingencies of reward may haveeen key was been too confusing for them. No conclusionsf getting the can be drawn from the negative findings.ccur. This experiment affirms and enlarges on theat different view of impulse control advanced theoreticallyto avoid the by the economist Strotz (1956), and a relatedielf may not proposal by the sociologist Becker (1960). Theiforcing ele- existence of an experimental paradigm of

I I

BIRD 54 -EXPERIMENTAL --CONTROL I -

IJI-Al '

488

IMPULSE CONTROL IN PIGEONS 489

impulse and control may permit research ona subject that has been almost purely theoreti-cal in the past.

REFERENCESBecker, H. S. Notes on the concept of commitment.

Anmerican Journal of Sociology, 1960, 66, 32-40.Bower, G. H. Correlated delay of reinforcement.

Jouirnal of Comparative and Physiological Psy-chology, 1961, 54, 196-203.

Chung, S. H. and Herrnstein, R. J. Choice and delayof reinforcement. Journal of the ExperimentalAnalysis of Behavior, 1967, 10, 67-74.

Fantino, E. Immediate reward followed by extinctionvs. later reward without extinction. PsychonomicScience, 1966, 6, 233-234.

Freud, S. Formulations on the two principles ofmental functioning. In J. Strachey and A. Freud(Eds.), The standard edition of the complete psycho-logical works of Sigmund Freud. London: Hogarth,1958. V. 12, Pp. 218-226.

Gamzu, E. R. and Williams, D. R. Associative factorsunderlying the pigeon's key pecking in auto-shapingprocedures. Journal of the Experimental Analysis ofBehavior, 1973, 19, 225-232.

Killeen, P. Preference for fixed-interval schedules ofreinforcemiient. Journal of the Experimental Analysisof Behavior, 1970, 14, 127-131.

Kimble, G. Hilgard and Marquis' conditioning andlearning. New York: Appleton-Century-Crofts, 1961.Pp. 140-144.

Kramer, T. J. and Rilling, M. Differential reinforce-ment of low rates: a selective critique. PsychologicalBulletin, 1970, 74, 225-254.

Logan, F. A. Decision making by rats: delay versusamount of reward. Journal of Comparative andPhysiological Psychology, 1965, 59, 1-12.

Mischel, W. and Metzner, R. Preference for delayedreward as a function of age, intelligence, and lengthof delay interval. Journal of Abnormal and SocialPsychology, 1962, 64, 425-431.

Mowrer, 0. H. and Ullman, A. D. Time as a deter-minant in integrative learning. Psychological Re-view, 1945, 52, 61-90.

Rachlin, H. Introduction to modern behaviorism. SanFrancisco: Freeman, 1970. Pp. 186-191.

Rachlin, H. and Green, L. Commitment, choice, andself-control. Journal of the Experimental Analysisof Behavior, 1972, 17, 15-22.

Renner, K. E. Delay of reinforcement: a historicalreview. Psychological Bulletin, 1964, 61, 341-361.

Shimp, C. P. The concurrent reinforcement of twointerresponse times: the relative frequency of aninterresponse time equals its relative harmoniclength. Journal of the Experimental Analysis ofBehavior, 1969, 12, 403-411.

Straus, M. A. Deferred gratification, social dass, andthe achievement syndrome. American SociologicalReview, 1962, 27, 326-335.

Strotz, R. H. Myopia and inconsistency in dynamicutility maximization. Review of Economic Studies,1956, 23, 165-180.

Williams, D. R. and Williams, H. Auto-maintenancein the pigeon: sustained pecking despite contingentnon-reinforcement. Journal of the ExperimentalAnalysis of Behavior, 1969, 12, 511-520.

Received 4 September 1973.(Final Acceptance 10 December 1973.)