1 23

Psychopharmacology ISSN 0033-3158Volume 231Number 15 Psychopharmacology (2014)231:2981-2988DOI 10.1007/s00213-014-3477-6

Random-ratio schedules produce greaterdemand for i.v. drug administration thanfixed-ratio schedules in rhesus monkeys

Carla H. Lagorio & Gail Winger

1 23

Your article is protected by copyright and

all rights are held exclusively by Springer-

Verlag Berlin Heidelberg. This e-offprint is

for personal use only and shall not be self-

archived in electronic repositories. If you wish

to self-archive your article, please use the

accepted manuscript version for posting on

your own website. You may further deposit

the accepted manuscript version in any

repository, provided it is only made publicly

available 12 months after official publication

or later and provided acknowledgement is

given to the original source of publication

and a link is inserted to the published article

on Springer's website. The link must be

accompanied by the following text: "The final

publication is available at link.springer.com”.

ORIGINAL INVESTIGATION

Random-ratio schedules produce greater demand for i.v. drugadministration than fixed-ratio schedules in rhesus monkeys

Carla H. Lagorio & Gail Winger

Received: 4 November 2013 /Accepted: 26 January 2014 /Published online: 23 February 2014# Springer-Verlag Berlin Heidelberg 2014

AbstractRationale Organisms emit more responses when food is pro-vided according to random as compared with fixed schedulesof reinforcement. Similarly, many human behaviors deemedcompulsive are maintained on variable schedules (e.g., gam-bling). If greater amounts of behavior are maintained by drugsof abuse when earned according to variably reinforced sched-ules, this would suggest that excessive drug-taking behaviormay be due in part to the nature of drug availability.Objectives The aim was to determine whether random sched-ules of contingent intravenous drug delivery would producemore responding than similarly priced fixed schedules.Methods Six rhesus macaque subjects responded to producecocaine (0.003–0.03 mg/kg/inj), remifentanil (0.01–1.0 μg/kg/inj), or ketamine (0.01–0.1 mg/kg/inj) according to eitherfixed or random ratio requirements that increased systemati-cally across sessions. Demand curves were generated with themost effective dose of each drug and compared across drugand schedule type.Results Cocaine and remifentanil maintained higher levelsand rates of responding when earned according to random-ratio schedules as compared with fixed-ratio schedules. Thisdifference was most pronounced when drugs were available athigh unit prices. Differences in responding across the scheduletypes generated by ketamine—a lesser-valued reinforcer—were qualitatively similar but smaller in magnitude.Conclusions The current study provides a systematic replica-tion across reinforcer type demonstrating that drugs deliveredafter a random number of responses generate more behavior

than those delivered according to a fixed schedule. The vari-able nature of the availability of drugs of abuse—particularlythose that are scarce or expensive—may be a contributingfactor to excessive drug intake by humans. This effect is mostlikely to be observed when more highly demanded(reinforcing) drugs are being consumed.

Keywords Drug self-administration . Rhesusmonkeys .

Cocaine . Remifentanil . Ketamine . Random ratio . Fixedratio . Demand curves . Response output curves . Behavioraleconomics

Introduction

It has been recognized for some time that variable schedules ofreinforcement can maintain relatively high and stable rates ofresponding as compared to behavior generated by fixed-ratioschedules (Ferster and Skinner 1957). Some forms of high andstable excessive behavior such as pathological gambling maydepend heavily on the random delivery of reinforcement. Lessis known about the relevance of variable reinforcer deliverieson other types of behavior that can be considered excessive orcompulsive, such as drug abuse. Although the variable natureof reinforcer deliveries is less clear with drug abuse than withmany gambling games, it is still the case that the amount ofbehavior required to obtain drugs is not likely to be on a fixedschedule. An important question is what role variably sched-uled payoffs may have in the development of excessive drug-taking behavior.

Previous research has assessed nonhuman subjects’ prefer-ence for variability in discrete-trial choice assays. In suchprocedures, subjects make repeated choices between two op-tions—one that is fixed and one that is variable. There areseveral dimensions that can be manipulated and assessed inthis type of procedure, including work requirement, delay toreinforcement, and amount of reinforcement. Several

C. H. Lagorio (*)Department of Psychology, University of Wisconsin-Eau Claire, EauClaire, WI 54702, USAe-mail: lagorich@uwec.edu

G. WingerDepartment of Pharmacology, University of Michigan, Ann Arbor,MI 48109, USA

Psychopharmacology (2014) 231:2981–2988DOI 10.1007/s00213-014-3477-6

Author's personal copy

investigators have reported that animal subjects prefer variableover fixed food amounts in discrete choice situations (e.g.,Logan 1965; Fantino 1967; Lagorio and Hackenberg 2012).When examining choices between fixed and variable delays toreinforcer delivery, researchers have reported robust prefer-ences for the option that is variable (e.g., Cicerone 1976;Davison 1972; Fantino et al. 1987; Herrnstein 1964; Hurshand Fantino 1973; Lagorio and Hackenberg 2010; Mazur1984, 2004). And, although not as consistent a finding, thereare several reports of preference for variable over fixed ratios,particularly at high ratio requirements (e.g., Ferster and Skin-ner 1957; Zeiler 1979). Other studies with similar aims haveexamined free-operant responding when only one scheduletype is available. For example, Madden et al. (2005) demon-strated that pigeons responding for food under closed eco-nomic conditions respond at higher rates and consume morefood when reinforcers are delivered according to a random-ratio (RR) as compared with fixed-ratio (FR) schedules, par-ticularly at high unit prices—a quantified relation betweenwork ratio and payoff amount.

To our knowledge, no studies have assessed whether RRand FR schedules engender different performances when sub-jects are working for reinforcers other than food. The currentstudy was designed to determine whether RR schedules gen-erate more behavior than FR schedules when intravenous (IV)drug delivery was used to establish and maintain responding.It was also designed to compare behavior maintained underrandom- and fixed-ratio schedules when drugs with differentreinforcing effectiveness (Hursh andWinger 1995) were used.It was therefore possible to determine whether differences inbehavior generated by FR and RR schedules would emergeonly with highly valued reinforcers such as food or cocaine.To accomplish these goals, demand curve analyses were usedto establish relative reinforcing effectiveness among differentdoses of three drugs: cocaine, remifentanil, and ketamine.Overall responses and response rates produced by each drugunder both FR and RR schedules were then compared toassess whether variably delivered drug reinforcers might en-gender higher rates of responding.

Materials and methods

Animals

Five adult rhesus monkeys (two females) were subjects in thisstudy. Each subject had a history of drug self-administrationunder FR schedules of reinforcement. The monkeys wereindividually housed in stainless steel cages measuring83.3 cm high×76.2 cm wide×91.4 cm. A metal panel(20 cm high×28 cm wide) was mounted on one side of eachcage and contained three response levers with a stimulus lightlocated 5 cm above each lever. Levers were positioned 5 cm

above the bottom of the panel and were spaced 2.5 cm apart.The stimulus lights were spaced 5 cm apart and could beindependently illuminated red, green, or yellow.

Procedures

Each monkey had an IV catheter implanted in an accessiblevein under sterile conditions and surgical anesthesia (10 mg/kgketamine and 2 mg/kg xylazine). The catheter passed subcu-taneously to the animal’s back where it exited in themidscapular region and proceeded through a flexible tubularmetal tether to the outside rear of the cage. Subjects wore aTeflon jacket (Lomir, Quebec, Canada) that protected thecatheter and allowed attachment of the tether to the animal.Located behind each cage was an infusion pump (Watson-Marlow model SCI-Q 400, Wilmington, MA), a fail-safedevice to stop the pump if a computer malfunction did notstop it automatically at 5 s, and plastic infusion bags contain-ing the drug concentration indicated for the drug dose andweight of the monkey. The catheters were connected to thissystem by way of a 0.45-μm Millipore filter.

Subjects were given access to IV drug infusions duringtwice-daily 2-h sessions (6 am and 12 pm). The start of eachsession was signaled by the onset of a red stimulus light overthe right lever. Responses on this lever resulted in a druginfusion after the schedule requirement was completed (de-scribed below). Drug infusions always lasted for 5 s and wereaccompanied by the offset of the right, red stimulus light, andthe onset of the center, green stimulus light. Following eachreinforcer delivery, there was a 10-s timeout period duringwhich all lights were extinguished and responses had noscheduled consequences.

Between three and four subjects were evaluated with eachdrug: remifentanil (monkeys MP, AL, FO, DA), cocaine(monkeys AL, FO, ST), and ketamine (monkeys AL, FO,ST, DA). Because there was more variability in respondingacross subjects when ketamine was the reinforcer, subjectswere evaluated twice with this drug. Drug dose order presen-tation was randomized and included the following: 0.00001,0.00003, 0.0001, and 0.0003 mg/kg/inj (remifentanil); 0.003,0.01, and 0.03 mg/kg/inj (cocaine); and 0.01, 0.03, and0.1 mg/kg/inj (ketamine). The three drugs were made avail-able according to both fixed- and random-ratio schedulesacross conditions. Testing of the three drugs was in a consis-tent order across subjects (remifentanil followed by cocaineand ketamine last). The other manipulations (dose of drug andRR or FR schedule type) were presented in varying ordersacross subjects. In order to generate demand functions, withineach condition, the response requirement increased every twosessions in the following ascending order: 10; 32; 100; 320;560; 1,000; and occasionally 1,780. For the random-ratioschedule, in which each response has an equal probability ofproducing a reinforcer, the programmed values were

2982 Psychopharmacology (2014) 231:2981–2988

Author's personal copy

generated using a method described by Bancroft and Bourret(2008). This method utilizes a macro (created using MicrosoftExcel) to identify a priori how many responses need to beemitted per reinforcer based on the programmed probability.For example, in an RR 10 schedule, each response has a 0.1probability of producing a reinforcer; the macro loops throughpotential successive responses made (response 1, response 2,etc.) to probabilistically determine whether a response meetsthe 0.1 criteria. When it does, that response number is record-ed and subsequent responses continue to be assessed formeeting the next reinforcement criteria based again on the0.1 probability. The response requirements programmed bythe RR schedule were generated in this manner to ensure thatthe programmed ratio closely approximated the obtained ratio.The maximum obtained values were 46; 334; 586; 1,334;3,004; 5,534; and 6,224 for each ratio, respectively.

Drugs

Lyophilized remifentanil hydrochloride was purchased fromthe University of Michigan hospital. Cocaine hydrochloridewas generously provided by the National Institute on DrugAbuse. Ketamine hydrochloride was purchased from ButlerSchein Animal Health (Dublin, OH) as a solution of 100 mg/cc in water. The drug doses were calculated as the salt form ofthe drugs. All drugs were reconstituted or dissolved in sterile0.9 % sodium chloride solution.

Data analyses

Data are assessed according to unit price (P), which is aquantified relation between the programmed response require-ment (R) and amount of reinforcement (A)—in this case, thedose of the drug (Hursh 1980). This equation was modified byHursh (1988) to account for reinforcers delivered with prob-ability p. According to this basic equation, no differences inresponding are predicted for reinforcers produced by fixed- orrandom-ratio schedules:

P ¼ R

pAð Þ ð1Þ

Rates of responding, number of responses, and overall drugconsumption were calculated for each ratio and condition. Theconsumption of each drug was normalized (Hursh andWinger1995; Hursh and Silberberg 2008) such that the point of originwas set to maximal responding (100 %)—the maximum num-ber of injections earned at the lowest cost (FR or RR 10).Consumption was normalized with the equation

Q ¼ 100� Yn

Y 10ð2Þ

such that normalized consumption (Q) at a specific ratio (Yn)was a function of consumption at the lowest price (Y10). Price(P) was normalized with the equation

P ¼ R� Y 10

100ð3Þ

where R is the ratio value (FR or RR). These data were fit to anexponential demand equation (Hursh and Silberberg 2008)using Microsoft Excel’s Solver function:

log Qð Þ ¼ log Q0ð Þ þ k � e−a�Q0�C−1� � ð4Þ

Q is the quantity consumed (number of injections multi-plied by dose in milligrams per kilogram per injection), Q0 isthe predicted level of consumption when the commodity isfreely available (and was set to 100), k indicates the range ofthe dependent variable in logarithmic units, and α (alpha) isthe critical measure reflecting rate of change in consumptionwith price, or the elasticity of the demand function. Thiselasticity measure, in turn, indicates the “essential value” ofthe reinforcer. The alpha parameters were obtained from de-mand functions that included data from whichever dose gen-erated the highest Pmax (the unit price at which peak levels ofresponding were maintained) for each of the tested drugs.

Response output functions were determined usingGraphPad Prism 5 (La Jolla, CA, USA) in order to comparethe amount of behavior generated by the two schedule types.Under ratio schedules, one can calculate response output asthe product of reinforcers earned times the cost of thosereinforcers (the fixed or random ratio requirement). UsingEq. 4 as the expression for the number of reinforcers earned,the equation for total responding is the solution of that expres-sion at any ratio value (C) multiplied by the programmed ratiovalue, as shown in Eq. 5. Separate response functions weregenerated for each reinforcer type, for the different doses ofeach drug, and for the schedule arrangement (FR or RR).

responses ¼ C � 10log Q0ð Þþk� e−a�Q0�C−1

� �h i

ð5Þ

Results

Figure 1 displays a basic comparison of the demand for eachof the three tested drugs. Curves were generated using datafrom the FR schedule condition using the dose that producedthe highest Pmax. Data in this figure were normalized acrosssubjects, such that their maximum consumption of each drugwas set at 100 % with remaining consumption at larger ratioscalculated as a percent of maximum. Alpha (α) was free to

Psychopharmacology (2014) 231:2981–2988 2983

Author's personal copy

vary, while k was fixed at 3.495 which was the empiricallyderived best-fit shared value across the three drugs. Table 1indicates the elasticity measures of the three drugs, withsmaller numbers indicating less elasticity of demand and morereinforcing effectiveness (essential value of the commodity).The relative, rank order reinforcing value of the three drugsindicated that remifentanil and cocaine were nearly equallyvalued (remifentanil being somewhat more valuable), andketamine was less effective in maintaining responding in theface of price increases. Reported R2 values indicate the across-subjects aggregate fit when curves are generated for eachsubject’s individual consumption data.

Figure 2 displays response output functions for both the FRand RR conditions when cocaine (upper left), remifentanil(upper right), and ketamine (below) were available. As con-firmed by the differences in the elasticities of the demandfunctions (as shown in Fig. 1), remifentanil and cocainemaintained similar maximum response outputs, whereas thoseproduced by ketamine were considerably lower. The variableresponse requirement producedmore responses for all of thesedrugs; this difference was particularly pronounced when ratiovalues were higher. With remifentanil and ketamine, differ-ences in FR and RR responding emerged at ratio values of320; for cocaine, differences were observed at ratio values of100. At higher ratio values, the two curves diverge with theRR schedule generating substantially more responding andhigher levels of consumption (number of injections earned,Fig. 3) for each drug. As can be seen, the difference in theamount of behavior controlled by random as compared withfixed ratio schedules was more robust in the case of cocaineand remifentanil than with ketamine. In addition, Fig. 4 showsthe rates of responding (responses per second) produced bythe three drugs under the RR and FR schedule requirements.RR schedules generated higher rates of responding, and thiseffect was particularly pronounced at ratio values above 100.

The ketamine determinations were conducted twice be-cause our initial results did not show a consistent responsepattern across subjects (i.e., one subject responded at a muchhigher rate when reinforcers were delivered on the RR sched-ule, while response rates for the other subjects were largelyundifferentiated). The same pattern emerged upon replication,with one subject (a different monkey from the one showingthe anomalous pattern in the first determination) displayinghigher rates for ketamine delivered on the RR schedule andothers displaying similar response outputs on both FR and RRschedules. Therefore, the data from both conditions are shownin aggregate in Figs. 2, 3, and 4. All other results wereconsistently demonstrated on an individual subject level;therefore, results are displayed in aggregate with standarderror measures presented in Figs. 2, 3, and 4.

As noted above, differences in responding produced by thetwo schedule types were more pronouncedwhen the unit pricewas high (i.e., when more than 100 responses were required to

produce one injection). The fact that the current study utilizedmultiple drug doses permitted a more comprehensive analysisof the effect of unit price on schedule differences. Figure 5shows this effect in greater detail, displaying the number ofdrug injections earned as a function of the response require-ment during both FR and RR determinations forremifentanil—the drug showing most pronounced scheduledifferences. Individual panels display the different drug dosesexamined. The figure represents data from fourmonkeys, withthe RR and FR determinations normalized within subject suchthat the maximum number of injections earned by one subjecton either schedule is set at 100 (considering that the RR andFR schedules did not induce reliably distinct results at thelowest ratio, normalizing in this fashion did not alter thecurves but instead permitted a better way to compare datafrom several subjects that may have differed in basal rates ofdrug intake). As can be seen, differences in responding underthe FR and RR schedules are not only more pronounced whenthe ratio value (work requirement) is higher but also when thedrug dose is lower. When the drug was relatively inexpensive(either available at a low ratio requirement or at a higher dose),subjects emitted similar numbers of responses irrespective ofthe schedule type; however, when drug access was moreexpensive, the random schedule supported higher amountsand rates of behavior. The data displayed in Fig. 5 confirm

Fig. 1 Number of injections earned as a function of unit price, normal-ized to maximum consumption within subject such that drug intake iscalculated as the percent of maximum. Curves are fit to the data usingEq. 5

Table 1 Obtained essential value (α) of remifentanil, cocaine, and keta-mine, and average overall variance accounted for (R2) by the exponentialdemand function

Drug Alpha (α) FR R2

Remifentanil 2.50 e-04 0.957

Cocaine 3.14 e-04 0.929

Ketamine 8.77 e-04 0.975

2984 Psychopharmacology (2014) 231:2981–2988

Author's personal copy

not only that RR schedules generate more responding at highratio requirements, but also that this effect is influenced byreinforcer quantity such that differences between the scheduletypes were all but eliminated when the drug dose was high(and, consequently, unit price was lower).

Discussion

The demand curve analyses of these data support earlierfindings that cocaine and remifentanil have a nearly equalreinforcing value, which is higher than that of ketamine(Hursh and Winger 1995). These data also demonstrate thatmore responses are generated by rhesus monkeys when ran-dom ratio, as compared with fixed ratio, schedules of rein-forcement are used to maintain responding, and this differenceis greatest when drugs are available at higher unit prices(either at high ratio requirements or low drug doses).

The finding that RR schedules generate greater drug de-mand at high unit prices expands and generalizes similarresults reported by Madden et al. (2005). These investigatorsused food as a reinforcer in pigeon subjects that wereresponding under closed economic conditions (i.e., their entiredaily food intake was earned during experimental sessions).They established demand functions for the two schedule typesand determined that reinforcers delivered under a variableschedule were more highly valued (i.e., had a lower calculatedα value) than reinforcers delivered under a fixed schedule.This preference has been attributed to the fact that reinforcersare occasionally delivered more immediately under variableschedules—there is the possibility of receiving a reinforcerafter only a few responses/seconds (Madden et al. 2007).Since it has been readily established that reinforcers lose theirvalue hyperbolically when delayed in the future (e.g., Mazur1984), those reinforcers that are delivered after few responsesin a variable schedule have considerably more value than

Fig. 2 Comparison of responses produced by random (open symbols) and fixed (closed symbols) ratio schedules as a function of the averageprogrammed ratio requirement across the three drugs. Curves are fit to the data based on Eq. 5. Error bars display the standard error across subjects

Psychopharmacology (2014) 231:2981–2988 2985

Author's personal copy

those that are delivered after many responses (even when theaverage payoff is held constant). This hyperbolic discountingof delayed outcomes could account for how the same drugscan maintain higher response rates and higher essential valuewhen delivered after a random number of responses. Maddenet al. (2005) extended this analysis and suggested that unitprice (when used in obtaining demand functions) may beappropriately modified by dividing it by the value of thereinforcer as obtained by the hyperbolic discounting function(Mazur 1984). When Madden et al. (2005) calculated their

demand curves using this measure of price, the differencebetween consumption under variable and fixed schedulesdisappeared.

Despite how reinforcer immediacy appears to account forwhy pigeons in the Madden et al. (2005) study respondedfaster when reinforcers were presented on a random schedule,not every experiment in the literature confirms reinforcerimmediacy as the sole mechanism. For example,

Fig. 3 Consumption of the three drugs (number of injections earned) as afunction of the average programmed ratio requirement under random(open symbols) and fixed (closed symbols) ratio schedules. Error barsdisplay the standard error across subjects

Fig. 4 Comparison of response rate (responses per second) produced byrandom (open symbols) and fixed (closed symbols) ratio schedules as afunction of the average programmed ratio requirement across the threedrugs. Error bars display the standard error across subjects

2986 Psychopharmacology (2014) 231:2981–2988

Author's personal copy

Andrzejewski et al. (2005) and Soreth and Hineline (2009)exposed pigeons to choices between random and fixed inter-val schedules, where choosing the random schedule alwaysproduced a longer delay to reinforcement than the fixedschedule. They found that the birds did not demonstrate apredicted (and theoretically appropriate) preference for thefixed schedule (with its shorter delay to reinforcement), butresponded as much as 50% of the time on the random intervaloption. Furthermore, a possible “preference for variability”agrees with neurochemical findings that unpredictable rein-forcer deliveries produce greater dopamine release in theventral striatum than do expected reinforcer deliveries(Schultz 2012). Dopamine release also correlates with delayto reinforcement (Kobayashi and Schultz 2008) suggestingthat delays to reinforcer deliveries and unpredictable

(variable) reinforcer deliveries are two potentially separablefactors that may impact reinforcer value.

In general, ketamine-maintained responding appeared lessaffected by the RR schedule as compared with behaviormaintained by cocaine or remifentanil. However, two mon-keys showed instances of extremely high rates of respondingon one of two occasions when their behavior was maintainedby 0.1 mg/kg/inj ketamine under a high RR schedule (RR1,780). We have not seen any examples of very high rates ofketamine-maintained responding under FR schedules in theyears that we have studied this drug (e.g., Winger et al. 2002;Broadbear et al. 2004). This effect mirrors that seen withcocaine and remifentanil when reinforcers were delivered aftera random number of responses at high unit prices. With theexception of these observations, however, demand for

Fig. 5 Normalized consumption of remifentanil as a function of pro-grammed response ratio (x-axis) and drug dose (different panels). Opensquares indicate injections earned under RR conditions, closed squares

under FR conditions. Data are normalized within subject to the maximumnumber of injections earned when exposed to the lowest ratio (10) ineither RR or FR conditions

Psychopharmacology (2014) 231:2981–2988 2987

Author's personal copy

ketamine was less influenced by the random schedule thanthat for cocaine and remifentanil. It is possible that, sincedemand for ketamine is more elastic (it is a relatively lower-valued commodity), responding is not well maintained at thehigh prices necessary for differences between the RR and FRschedules to emerge. Future work could assess how different-ly valued reinforcer types may influence the degree of prefer-ence for variably arranged outcomes.

Taken as a whole, the experiment described herein indicatesthat various drugs of abuse will generatemore respondingwhenthey are delivered on a random as compared to a fixed responseschedule.When the amount of responding necessary to producea valued drug dose is substantial (i.e., the unit price is high), thevariable nature of the schedule generates considerably moreresponding. The fact that drugs delivered on variable schedulesmay periodically be acquired after periods of very brief delaysmay account for the increase in reinforcer value that is seen. Ifthis information can be translated to the situation of drug abuse-related behavior in humans, it may imply that when an abuser isunsure of some aspects of drug acquisition such as the pricerequired, the quality of the drug purchased, or the amount oftime needed to wait before a drug can be used, then drugprocurement behaviors will be increased and occur at a higherrate. The uncertainty is unlikely to account entirely for theexcessive nature of drug-associated behavior in addicted indi-viduals, but it may contribute to it.

Acknowledgments This work was supported by PHS DA023992 toGW and T32 DA007268 to CL. The excellent technical assistance ofAngela Lindsey, Matthew Zaks, Kathy Carey Zelenock, and Yong GongShi is gratefully acknowledged.

Conflict of interest The authors have no conflict of interest to declare.

References

Andrzejewski ME, Cardinal CD, Field DP, Flannery BA, Johnson M,Bailey K, Hineline PN (2005) Pigeons’ choices between fixed andvariable interval schedules: utility of variability. J Exp Anal Behav83:129–145. doi:10.1901/jeab.2005.30-04

Bancroft SL, Bourret JC (2008) Generating variable and random sched-ules of reinforcement using Microsoft Excel macros. J Appl BehavAnal 41:227–235. doi:10.1901/jaba.2008.41-227

Broadbear JH, Winger G, Woods JH (2004) Self-administration of fen-tanyl, cocaine and ketamine: effects on the pituitary-adrenal axis inrhesus monkeys. Psychopharmacology (Berl) 176:398–406. doi:10.1007/s00213-004-1891-x

Cicerone RA (1976) Preference for mixed versus constant delay ofreinforcement. J Exp Anal Behav 25:257–261. doi:10.1901/jeab.1976.25-257

Davison MC (1972) Preference for mixed-interval versus fixed-intervalschedules: number of component intervals. J Exp Anal Behav 17:169–176. doi:10.1901/jeab.1972.17-169

Fantino E (1967) Preference for mixed- versus fixed-ratio schedules. JExp Anal Behav 10:35–43. doi:10.1901/jeab.1967.10-35

Fantino E, Arbaraca N, Ito M (1987) Choice and optimal foraging: testsof the delay reduction hypothesis and the optimal diet model. In:Commons ML, Kacelnik A, Shettleworth SJ (eds) Foraging.Quantitative analyses of behavior Erlbaum. Lawrence ErlbaumAssociates, Hillsdale

Ferster CB, Skinner BF (1957) Schedules of reinforcement. Prentice-Hall, Englewood Cliffs

Herrnstein RJ (1964) Aperiodicity as a factor in choice. J ExpAnal Behav7:179–182. doi:10.1901/jeab.1964.7-179

Hursh SR (1980) Economic concepts for the analysis of behavior. J ExpAnal Behav 34:219–238. doi:10.1901/jeab.1980.34-219

Hursh SR (1988) A cost-benefit analysis of demand for food. J Exp AnalBehav 50:419–440. doi:10.1901/jeab.1988.50-419

Hursh SR, Fantino E (1973) Relative delay of reinforcement and choice. JExp Anal Behav 19:437–450. doi:10.1901/jeab.1973.19-437

Hursh SR, Silberberg A (2008) Economic demand and essential value.Psychol Rev 115:186–198. doi:10.1037/0033-295X.115.1.186

Hursh SR, Winger G (1995) Normalized demand for drugs and otherreinforcers. J Exp Anal Behav 64:373–384. doi:10.1901/jeab.1995.64-373

Kobayashi S, Schultz W (2008) Influence of reward delays on responsesof dopamine neurons. J Neurosci 28:7837–7846. doi:10.1523/jneurosci.1600-08.2008

Lagorio CH, Hackenberg TD (2010) Risky choice in pigeons andhumans: a cross-species comparison. J Exp Anal Behav 93:27–44.doi:10.1901/jeab.2010.93-27

Lagorio CH, Hackenberg TD (2012) Risky choice in pigeons: preferencefor amount variability using a token-reinforcement system. J ExpAnal Behav 98:139–154. doi:10.1901/jeab.2012.98-139

Logan FA (1965) Decision making by rats: uncertain outcome choices. JComp Physiol Psych 59:246–251. doi:10.1037/h0021850

Madden GJ, Dake JM, Mauel EC, Rowe RR (2005) Labor supply andconsumption of food in a closed economy under a range of fixed-and random-ratio schedules: tests of unit price. J Exp Anal Behav83:99–118. doi:10.1901/jeab.2005.32-04

Madden GJ, Ewan EE, Lagorio CH (2007) Toward an animalmodel of gambling: delay discounting and the allure ofunpredictable outcomes. J Gambl Stud 23:63–83. doi:10.1007/s10899-006-9041-5

Mazur JE (1984) Tests of an equivalence rule for fixed and variablereinforcer delays. J Exp Psychol Anim Behav Process 10:426–436. doi:10.1037/0097-7403.10.4.426

Mazur JE (2004) Risky choice: selecting between certain and uncertainoutcomes. Behav Anal Today 5:190–203

Schultz W (2012) Risky dopamine. Biol Psychiatry 71:180–181. doi:10.1016/j.biopsych.2011.11.019

Soreth ME, Hineline PN (2009) The probability of small schedule valuesand preference for random-interval schedules. J Exp Anal Behav 91:89–103. doi:10.1901/jeab.2009.91-89

Winger G, Hursh SR, Casey KL, Woods JH (2002) Relative reinforcingstrength of three N-methyl-D-aspartate antagonists with differentonsets of action. J Pharmacol Exp Ther 301:690–697. doi:10.1124/jpet.301.2.690

Zeiler MD (1979) Output dynamics. In: Zeiler MD, Harzem P (eds)Advances in analysis of behaviour: reinforcement and the organiza-tion of behavior. Wiley, Chichester

2988 Psychopharmacology (2014) 231:2981–2988

Author's personal copy