Presentation and validation of “The Learning Game,” a toolto study associative learning in humans

James Byron Nelson & Anton Navarro &

Maria del Carmen Sanjuan

# Psychonomic Society, Inc. 2014

Abstract This article presents a 3-D science-fiction-basedvideogame method to study learning, and two experimentsthat we used to validate it. In this method, participants are firsttrained to respond to enemy spaceships (Stimulus 2, or S2)with particular keypresses, followed by transport to a newcontext (galaxy), where other manipulations can occur.During conditioning, colored flashing lights (Stimulus 1, orS1) can predict S2, and the response attached to S2 from theprior phase comes to be evoked by S1. In Experiment 1 wedemonstrated that, in accord with previous findings fromanimals, conditioning in this procedure was positively relatedto the ratio of the time between trials to the time within a trial.Experiment 2 demonstrated the phenomena of extinction,timing, and renewal. Responding to S1 was slightly lost witha context change, and diminished over trials in the absence ofS2. On early extinction trials, responding during S1 declinedafter the time that S2 normally occurred. Extinguishedresponding to S1 recovered robustly with a context change.

Keywords Video game . Human associative learning .

Conditioning . Extinction . Renewal

In this article, we present a videogame method with which tostudy learning in humans. The method is a first-person “spaceshooter,” modeled conceptually after the procedures ofIvanov-Smolensky (1927). He first instructed participantshow to respond to a stimulus (S2) and then signaled S2 withanother, S1. Over trials S1 came to evoke the response at-tached to S2. Accordingly, the game consists of two generalphases. In a “response-training” phase enemy spaceships ap-pear in a training environment and participants learn to re-spond on a keyboard, activating weapons, to repel theseenemies. After establishing responses to the enemies, partici-pants enter the “experimental phase,” for which they aretransported to galaxies containing space stations that areattacked by the enemies. Flashing lights can be programmedto appear in this phase—for example, to predict an attackingenemy. These stimuli should come to elicit the responsesattached to the enemies that they predict, responses that canbe used in making inferences about underlying processes. Ourmain goals in developing the method were to provide anentertaining medium through which to present events; tominimize researcher–participant interactions; to index learn-ing with effortful increases in responses, which we suggest hasseveral possible benefits; and to develop a simple interface tocreate experiments.

Creating entertainingmethods has challenges; tasks must besimple so as to not confound learning the task with the learningdesired to take place among stimuli within the task. In thegame, the participants’ task is simply to respond to spaceships.In an effort to keep the game entertaining while keeping thetask simple, we developed visual and musical elements com-parable to those in commercial video games. To facilitate thetask and minimize participant/researcher interactions, we in-corporated all instructions within the format and context of thegame. A participant need only be seated at the computer and beinstructed by the researcher to press the “B” key. Interactionsbetween the researcher and the participant as a source of

Electronic supplementary material The online version of this article(doi:10.3758/s13428-014-0446-2) contains supplementary material,which is available to authorized users.

J. B. Nelson :A. Navarro :M. d. C. SanjuanUniversity of the Basque Country, San Sebastian, Spain

J. B. Nelson (*)Depto. Procesos Psicológicos Básicos y su Desarrollo, Universidadde País Vasco, Avenida de Tolosa, 70, San Sebastian, Spain 20018e-mail: Jamesbyron.nelson@ehu.es

J. B. Nelsone-mail: DrJBN@hotmail.com

Behav ResDOI 10.3758/s13428-014-0446-2

variability are thus minimized (cf. Arcediano, Ortega, &Matute, 1996; Nelson & Sanjuan, 2006).

Intuitively, a game-format task could be more enjoyable thana nongame task. Though enjoyment may not increase the qualityof data (Hawkins, Rae, Nesbitt, & Brown, 2012), it seemsdesirable.Moreover, a game allows relationships between eventsto be directly perceived by the participant, as opposed to readingabout the pairing in methods such as predictive-learning (PL)tasks (e.g., Matute, Arcediano, & Miller, 1996). In PL tasks,participants typically self-advance through a series of screens onwhich stimuli, such as fictitious symptoms and diseases, arepaired through pictures and scripts. The data come from self-reports of the estimated probability or likelihood that the out-come is either caused by, or will appear given, the precedentstimuli. The results from these arguably introspective methodsare sensitive to testing procedures, such as the frequency withwhich judgments are required (e.g., Collins&Shanks, 2002) andthe phrasing of questions (e.g., Matute et al., 1996).

Our motivations for indexing learning by effortful increasesin responding are best discussed by first considering somecurrent game techniques. At least three tasks are available thatinvolve response suppression. The first is the relatively widelyused “Martians” task (Franssen, Clarysse, Beckers, van Vooren,& Baeyens, 2010, discuss its broad application). Another is thatdeveloped by Nelson and Sanjuan (2006), and the third is onedeveloped by Molet, Callejas-Aguilera, and Rosas (2007).Though visually different, these games bear strong conceptualsimilarities to each other. In each, a baseline of behavior isestablished by having participants respond on a mouse or key-board to earn points by shooting at enemies (e.g., Martians orspaceships). The stimuli (e.g., illuminated lights) are presentedthat signal an attack, in varying forms, by the on-screen targets.Participants are able to avoid the attack by suppressing their ownrate of firing. Learning is indexed by measuring the degree ofbaseline suppression evoked by the predictive stimuli.

In few computer-game-based tasks does a simple pairing ofevents lead to an increase in behavior, outside of explicitlyoperant tasks (e.g., Gamez & Rosas, 2007), in which theoutcomes depend entirely on behavior. Among those comput-er games, little is different from what is available in PL tasks.For example, in the Spy Radio task of Pineño, Ortega, andMatute (2000), participants load a number of refugees onto atruck for rescue, depending on the road conditions predictedby lights on a “Spy Radio.” In Pineño et al. (2000) andEscobar, Pineño, and Matute (2002), loading has been accom-plished by holding down a space bar for varying amounts oftime. When more than one physical response is required toload refugees, there has been no time limit (Pineño & Miller,2004). Moreover, participants read about the road-conditionoutcomes rather than perceive them directly in the game.Though it provides a novel way to present stimuli, in manyregards the procedure is not unlike PL tasks (see also the“Tank Game” of Dickinson, Shanks, & Evenden, 1984).

Our motivation to index learning with increases inresponding was threefold. First, that approach seemed con-ceptually simpler than suppression tasks, in which participantsmust comprehend reasons to emit the baseline response, andthen learn that response. Other reasons then have to be under-stood to subsequently withhold responding. Any misunder-standing on the part of participants can lead to poor perfor-mance. In a response generation task, nothing is requiredexcept the response. Instructions can address one behavior(the response) rather than two (the response and its suppres-sion). Thus, instruction complexity can be reduced. Second,the effort involved in quickly producing a vigorous responsemight be less likely to be influenced by cognitive variables atthe time of responding than are the Spy Radio and PL-typetasks—a reasonable assumption, given that inferential reason-ing processes are thought to be slow and intensive on workingmemory (see González-Martín, Cobos, Morís, & López,2012, for a discussion). Finally, a measure that involves re-sponse counts accumulated over several seconds might bemore sensitive to manipulations than are suppression tasks,in which a single behavior emitted once—removal of thefinger from the keyboard or mouse—can produce a maximumsuppression response.

The present game was programmed in C# using MicrosoftVisual Studio Express 8 and Microsoft XNA Game Studio3.1. It operates on any computer running Windows XPService Pack 3 (SP3) or later, with hardware support forMicrosoft DirectX 9.0c and pixel-shader model 3 (availablesince 2004). The game requires the Microsoft XNA 3.1Redistributable and the Microsoft .NET 3.5 framework. Thelatter is typically already present on mostWindows computersthat have received regular updates. Both are available in thedownload referenced by the link below.

The game was designed to be a simple way to createexperiments. All events are controlled via edits to a text-format parameter file. An example file and documentation ofits syntax are distributed with the game and are available in thesupplementary materials. Both the download and the supple-mentary materials include instructions on installing and usingthe game, the format of the files that are used for instructions,and the format of the resulting data files. In addition to thejournal’s website, the download is available by visiting http://drjbn.wordpress.com/the-learning-game-download-links/,where the present materials and future updates can be found.

The key graphical elements of the game that are used inexperiments are shown in Figs. 1 and 2 below. A presentationof the method is available at www.youtube.com/watch?v=Yl9Q8ThMN40.

The elements of the game will be discussed in more detailbelow as they are encountered in the flow of the game, outlinedin Table 1. Throughout, the use of the word “fixed” or“assigned” indicates that the property being described is notmodifiable.

Behav Res

Initial instructions

Participants initially see a view screen, shown in Fig. 1 (with-out the enemy crafts shown at F–I), and are instructed by theresearcher to press the “B” key to begin the game. As isoutlined in Table 1, the first event (1) is the appearance of atranslucent black message panel, not pictured in Fig. 1, thatrises between the weapons at D and C. Messages (e.g.,

instructions, cover story) are presented on this panel.Throughout the game, messages can be optionally accompa-nied with voice recordings, provided by the researcher, as isindicated by “voice/no voice” in Table 1. The panel can bedismissed by pressing the fixed “M” key, unless otherwiseindicated in the following text.

Response-training phase

Each trial optionally begins with a prestimulus instruction (2)that remains on the screen until the participant presses the “M”key. For example, in the experiments below, participants wereinformed that “something is about to appear.” Once thatinstruction is dismissed, an intertrial interval (3) elapses, dur-ing which nothing occurs and the participant simply views thewire-frame background present in Fig. 1. Throughout thegame, the backgrounds are constantly in motion, as if theparticipant’s ship was scanning the area.

Following the intertrial interval, any of the enemy space-craft shown at F–I in Fig. 1 can be programmed to appear (4).Each enemy is assigned to appear by flying in from a distancefrom the corner closest to which it is shown in Fig. 1, andmultiple ships can appear together. All enemies are protectedby translucent shields that become visible when they are

A

C

B

D

F

H

G

I

E

Fig. 1 Screen capture of the experimental apparatus. Colored lightstimuli are presented at E. A–D indicate weapons, and F–I show fourpossible enemy spaceships. The green wire-frame background is presentin response training. See the text for further details

Fig. 2 Screen captures of the four galaxies used as contexts. From top left to bottom left, clockwise, these show the Boutonia, Aesop, Nicholosia, andWagneria galaxies. See the text for further details

Behav Res

attacked. Enemies require being fired at a modifiable numberof times before they exit, ending the trial.

The appearance of the enemy(s) can be accompanied by anon-trial instruction (4) that stays on screen while the enemy ispresent (i.e., it cannot be dismissed with the “M” key). Forexample, in the present experiments, this option was used toidentify the enemy, inform the participant which weapon touse to repel it, and describe how to use the weapon.

The participants’ weapons are shown at A–D in Fig. 1.Each weapon is assigned to the enemy shown closest to it inthe figure. When an enemy is present, the assigned weaponaims at the enemy. A weapon requires accumulating 5 s ofkeypressing at a fixed rate of at least three per second to becharged. Once it is charged, it fires with continuedkeypressing when its assigned enemy is present. To provideimmediate feedback for keypressing, presses produce differ-ent sounds, assigned to each weapon, that increase in pitch. Inthe absence of an assigned enemy, pressing a weapon’s keyproduces the changes in sound pitch, but the weapon does notfire. The charging of each weapon is independent of the

others. If multiple attackers are present, the participant cansimultaneously activate multiple weapons.

The weapon located at A is assigned to the tab key, B to thebackspace key, C to the number-pad zero key, and D to the leftshift key. The layout of the keys on the keyboard matchesnumerous nationalities (http://en.wikipedia.org/wiki/Keyboard_layout) and matches the screen layout of theweapons to which they are assigned. Note that the color of aweapon bears a similarity to that of its assigned enemy.Matching the enemies’ colors and arrival locations to thoseof the weapons, as well as matching the key/weapon layouts,were manipulations to facilitate attaching responses to theenemies.

After the programmed trials elapse, response training endswith “end-of-phase” instructions (5) that are dismissed bypressing the “M” key. To illustrate, in the experiments below,participants were informed that they were ready to proceed tothe next phase of their mission.

Experimental phase

After removing the end-of-phase instructions, participantsenter the experimental phase of the game, during which theyare transported to one of the four galaxies (6) shown in Fig. 2.During transport, the message panel appears with the words“Protect the _______” with the blank being filled by one ofthe following: Boutonians, “energy station of the Aesop,”Nicholosians, or Wagnerians (the names correspond toFig. 2, moving clockwise from top left). These texts are fixedbut can appear in English or Spanish. A spinning tunnelappears and grows to encompass the view screen, giving theimpression of approaching and traveling through it. On arrivalat the next galaxy, the tunnel disappears, but the messagepanel remains visible for two more seconds and is retractedautomatically.

At this point, a modifiable intertrial interval (7) elapses,followed by a prestimulus (pre-S) period (8) that, for partici-pants, is indistinguishable from the intertrial interval. In thepre-S period, the game will record the total responses on eachsecond made on any or all of the weapon keys, key by key, atthe researcher’s discretion. During these periods, the partici-pant simply sits and observes the galaxy; each contains ananimated space station and has a different assigned, space-themed, looping instrumental musical track playing in thebackground.

Following the pre-S period, the stimulus (S) period begins,during which behavior is also recorded. Flashing lights (8) canbe programmed to appear in any of the eight darker ovals inthe panel shown at E in Fig. 1. These stimuli can come on atany point during the S period and can flash for any duration, ata fixed rate of three flashes per second, in any color producedby additive combinations of each of 255 shades of red, green,

Table 1 Outline of game events and modifiable parameters

1: Initial instructions (text, voice/no voice) [participant response requiredto proceed]

Response-training phase

For each trial

2: Prestimulus (Pre-S) instructions [participant response requiredto proceed] (presence/absence, text contents, voice/no voice)

3: Intertrial interval (duration in whole seconds)

Stimulus period

4: Enemy spaceship(s) [participant responses required to proceed](enemy spaceship types, response requirement to end trial)

4: On-trial instruction [ends with trial] (presence/absence, text,voice/no voice)

5: End-of-phase instructions (text contents, voice/no voice, responserequired to proceed)

Experimental phase

For each trial

6: Context (which of four is present throughout the trial)

7: Intertrial interval (duration in whole seconds)

8: Pre-S period (duration in whole seconds)

Stimulus period

9A-B: Lights (number of lights, start time, end time, color)[parameters above are independent for each light]

9A-B: Enemy spaceships (types, start time, end time) [parametersabove are independent of each type]

10: End screen & credits

Time flows from top to bottom. Items with sequential numbering occursequentially unless otherwise indicated. Items with the same numberingcan occur at the same time. The subscripts A and B indicate that theseevents can occur at any time with respect to each other. Variables listed inparentheses are modifiable by the researcher, and further notes are inbrackets. See the text for details

Behav Res

and blue. Black lights (R = G = B = 0) can be used to create anS period during which no lights are presented. Each light isindependent of the others. For example, they can be presentedsingly or in any combination, can overlap in time, with asyn-chronous start and stop times, or can occur one after the other.The light panel itself can be either consistently present orabsent throughout the game. When the panel is absent and alight is programmed to flash, the light appears where it isshown in Fig. 1, but in the absence of any other light-panelelements.

Up to four enemy spaceships can be programmed to appear(9) during the S period. As with the lights, each enemy isindependent of the others. An enemy can enter at any timeduring the S period and can remain for any duration. Thetiming of each enemy is independent of that of the lights (e.g.,enemies can appear before, during, or after any light). Whenan enemy appears during this phase, it attacks the space stationunless it is being attacked by the participant. It cannot fire itsown weapon through its shield.

Enemies stay on the screen for a period designated by theresearcher, independently of participants’ behavior. Once theenemy’s programmed duration times out, it flies off the screen,disappearing in the distance. If a participant does not activatethe weapon by the end of the enemy’s programmed duration,the weapon will fire once, without participant input, and theenemy will exit.

A trial ends when all events (i.e., lights, enemies) in the Speriod have ended.

Changes from galaxy to galaxy can occur between trials. Achange occurs in the same way as the initial change from theresponse-training to the experimental phase.

After execution of all of the experimental-phase tri-als, participants are informed (10) via a fixed messagethat the “danger” has passed. They are thanked for theirassistance, and the credits for the game’s creation arepresented.

With the method described above we presented two exper-iments. They were designed to establish the validity of themethod as a tool to study associative learning by characteriz-ing conditioning, extinction, and the use of the galaxies as acontextual manipulation.

Experiment 1

The first experiment demonstrated the effects of differenttiming parameters in establishing responding to S1. In pigeonautoshaping, responding is acquired more rapidly with largerratios of C/T, where C is the length of the time between trialoutcomes (“Cycle” time) and T is the time within a trial (e.g.,S1–S2 pairing, or “Trial” time; Gibbon & Balsam, 1981). Forall groups, the duration of S1 was 20 s and an S1–S2 pairinglasted 15 s, coterminating. The time between S1 presentations

averaged 10 s for Group 10–20, 20 s for Group 20–20, and40 s for Group 40–20. Minimally, responding should beacquired more rapidly with larger intertrial intervals.

Method

Participants

The participants were college-student volunteers. A total of 44participants were assigned to groups randomly, with no at-tempt at equalizing group sizes (see Nelson & Sanjuan, 2006,for a discussion). On a participant-by-participant basis, acondition was randomly selected from a pool, assigned, andreplaced in the pool, giving every condition an equal proba-bility of being assigned to each participant. We solicitedparticipants by distributing sign-up sheets with the goal ofobtaining at least ten participants per group. The sheetsallowed for more participants than were necessary, to com-pensate for “no shows,” and no participant who showed wasturned away. In that way, more participants could be placedinto conditions than were planned, but not less. Because wehad no data from the novel method with which to assessvariance, the minimum n was set to 10 for convenience inobtaining participants in a timely manner.

Apparatus

The game was run on Dell OptiPlex 755SF computersequipped with Intel E8400 processors, 256-MB ATI RadeonHD2400XT video cards, and 22-in. Dell monitors. The mon-itor aspect ratio was 1.6 (width/height), set to a resolution of1,280 × 800 pixels and 32-bit color depth.

An animated plasma effect was visible in the round win-dow to the rear left of weapon B in Fig. 1 and became morerapid when the backspace key was pressed. These visualswere not mentioned in the introduction because they werediscontinued after this experiment and do not reflect thecurrent state of the game.1

S1 was a 20-s flashing of the middle oval in the top row ofthe light panel with the color red (R = 255, G = B = 0). S2 wasa 15-s appearance of the enemy shown at G in Fig. 1. Thename “SOP Cannon” appeared on white paper attached to thebackspace key.

1 Our plans for the method involve the use of eyetracking to measure theallocation of visual attention. After Experiment 1, we realized that corre-lations between animated effects on the weapons and keypressing couldcause attention to be directed there simply due to the correlation, asopposed to the relationships of the weapons with other events in thegame. Thus, we removed the link between the animation of the plasmaand keypressing from the game for all future experiments.

Behav Res

Procedure

Response training The initial instructions were presented inthe message panel and played over headphones in a femalevoice. The instructions (1) tasked participants withprotecting galaxies from invaders, (2) instructed the par-ticipants to attack invaders first because invaders could notattack when being attacked, and (3) informed participants(a) that they would be instructed on how to use weapons toaccomplish the task, and (b) that they should press the “M”key to continue.

On pressing the “M” key, a prestimulus instruction, “In afew moments, something will appear on the screen. I will tellyou what to do. PressM to begin” appeared. After pressing the“M” key, the message panel disappeared, an intertrial intervalelapsed, and an enemy and on-trial instruction appeared. Theinstruction named the enemy as the “Learian” and informedthe participant to use the “SOP Cannon located at the top rightcorner of the screen” to repel it. The participant was told topress the key labeled “SOP Cannon” rapidly and repeatedlyuntil the weapon was charged and firing and to continuepressing until the Learian had fled. Participants begankeypressing at their volition. The panel remained until partic-ipants had successfully fired eight shots, repelling the Learian.

The next trial began with a prestimulus instructioninforming participants that (1) the weapon’s charge woulddissipate rapidly and would have to be charged at every use,(2) the enemy did not have to be present to be able to chargethe weapon, and (3) the same invader might have to berepelled many times. After participants had pressed the “M”key, the instructions disappeared, and no further instructionswere used. The intertrial interval elapsed, the Learian ap-peared and remained until the participant had repelled it, andthat sequence was repeated for four more trials. The intertrialintervals averaged 5 s (range 10 s, SD = 4.1) in this phase.

After the sixth trial, end-of-phase instructions appeared thatinformed the participants that they were “ready for patrol” andreminded them that the weapon should be charged before theinvader appeared. Participants were told that invaders mightnever appear, and that they would have to learn “certainthings” about the technology that had been “forgotten.”When participants pressed “M” the message panel wasretracted, they were transported to the Boutonia galaxy, andthe experimental phase began.

Experimental phase On each of 20 trials, S1 was paired withS2. S2 appeared after 5 s of S1 and terminated with S1. Thetotal time between the offset of the stimuli and the onset of S1for the next trial varied between groups. Group 10–20 re-ceived intervals (intertrial interval plus a 5-s pre-S period)averaging 10 s (range 9 s, SD = 2.8 s). Group 20–20 receivedintervals averaging 20 s (range 28 s, SD = 6.5 s). Group 40–20received intervals averaging 40 s (range 60 s, SD = 19 s).

Data and analysis

Responses on the backspace key were recorded. Pre-Sresponding was that measured during the pre-S period.Responding to S1 was that measured during S1, before S2was present.

Analysis of variance (ANOVA), calculated using Type III(unweighted) sums of squares, was used to analyze the effectsof interest. Follow-up ANOVAs (i.e., simple effects) wereconducted using error terms derived from the overallANOVA, with degrees of freedom adjusted according to theWelch–Satterthwaite procedures (see, e.g., Howell, 1987, pp.434–435). When the assumptions of ANOVA were not met(e.g., excessive heterogeneity of variance, distributions pre-dominately containing zeros), we used chi-square probabilityestimates from Kruskal–Wallis tests for independent samples.

We conducted simple-effect tests in the presence of inter-actions to generally characterize where differences emergedand disappeared. Our goal was to make an overall conclusionas to the advantage of some conditions over others, based, inpart, on multiple comparisons. That goal was one for which“Control of the probability of any error is unnecessarily strin-gent, as a small proportion of errors will not change the overallvalidity of the conclusion” (Benjamini & Hochberg, 1995, p.292). In addition to the protection afforded by our initialANOVAs, our criterion for rejection on each simple-effect testwas adjusted by the procedures outlined by Benjamini andHochberg to control the false discovery rate.

Effect sizes are reported as partial eta-squared for theoverall ANOVAs. Effect sizes involving significant compari-sons between two means are reported as Cohen’s d, using thepooled variances of the means being compared. Variances arereported as the pooled within-cell variance across the designbeing analyzed. For example, in a Group × Trials × Secondsdesign, the variance that we report will represent the variancepooled within each group across each second of each trial.

Results

The data supported two key conclusions. First, acquisitionwas more rapid with larger C/T ratios: Responding to S1emerged more rapidly in Groups 20–20 and 40–20 than inGroup 10–20. Second, short C/T ratios led to responding to thecontext. Responding during the pre-S period emerged in theabsence of any stimuli other than the context in Group 10–20.

Conclusion 1 The average responding to S1 on each secondof each trial, expressed as responses per second, is shown inFig. 3 in the solid lines. The figure collapses Groups 20–20and 40–20, which did not differ in the following analyses. AGroup × Trials × Seconds ANVOA revealed a three-wayinteraction, F(152, 3116) = 1.37, p = .002, η2p = .06. Thegreatest degree of variation among seconds was from the 1st

Behav Res

to the 2nd. In the later 4 s (i.e., 2–5), during which participantshad had time to perceive S1 and begin to respond, respondingwas relatively more stable and group differences were mostapparent. To reduce the number of comparisons, we collapsedacross these last 4 s for testing the effect of group.

Alphas were adjusted for the three possible group compar-isons on each of the 20 trials (i.e., 60 comparisons). Group 20–20 never differed fromGroup 40–20, and they were combinedfor presentation and further analysis. More responding in thelarger C/T-ratio groups was observed on Trials 2–6, 9, and 11,Fs(1, 157) ≥ 7.29, ps ≤ .008, dmin-max = 0.39–0.62. Theresponse variance was 8.82.

Conclusion 2 Pre-S data are shown in Fig. 3 as dashed lines.Responding in Groups 40–20 (mean = .39) and 20–20 (mean= .26) was extremely low and featuredmostly zeros (variances= 0.76 and 1.73, respectively), whereas the variance in Group10–20 was 4.89. Thus, the group comparisons utilizedKruskal–Wallis tests. For simplicity, we collapsed across sec-onds. Alpha was adjusted for 60 possible comparisons.Groups 40–20 and 20–20 never differed and were combined.Group 10–20 responded more than did the other conditionscombined on Trials 13, 14, 18, and 20, χ2s(1) ≥ 8.86, ps ≤.003. Note that adjusting responding during the S period bythat observed in the pre-S period would enhance the effectobserved between groups in responding to S1.

Discussion

The relationship of the time between trials to the time within atrial was manipulated between three groups. Conditioning to

S1 improved with larger ratios. The method also indicatedcontext conditioning. Pre-S response rates were higher inGroup 10–20 than in the others, consistent with the predic-tions from the Rescorla and Wagner (1972) model and fromGibbon and Balsam (1981). Subsequent experiments, notreported here, showed that when S2 stimuli appearedunsignaled, a slow but steady rate of responding was obtainedin the presence of the contextual stimuli alone, as was ob-served during the pre-S period in Group 10–20.

Experiment 2

Experiment 2 was designed to demonstrate a “renewaleffect” (e.g., Nelson, Sanjuan, Vadillo-Ruiz, Pérez, &Léon, 2011), to validate our context manipulations,characterize extinction, and assess whether the methodcan detect timing of events. Two groups received con-ditioning and testing with S1 in one context (A), withextinction prior to testing being conducted in the samecontext (Group AAA) or in a different one (GroupABA). Group ABA was expected to show moreresponding on test than would Group AAA. Such aresult would depend on participants’ ability to discrim-inate the contexts.

The extinction phase provided the opportunity to assesswhether participants could learn to time the events within thegame. During conditioning, S1 was present for 20 s, and S2appeared at the end of Second 5 and remained for 15 s. Ifparticipants simply expected S2 during S1, their responsesshould increase during the first 5 s and remain relatively

Fig. 3 Average responses oneach second of each trial duringthe prestimulus (pre-S; dashedlines) and stimulus (S; solid lines)periods in Experiment 1. Allgroups received a 20-s S1 thatoverlapped with a 15-s S2. S1 andS2 coterminated. The timebetween S1 presentationsaveraged 10 s for Group 10–20(solid circles), and 20 s or 40 s forGroup 20–20 and Group 40–20(combined in open circles),respectively

Behav Res

consistent across the remaining 15 s in S2’s absence. If par-ticipants learned to expect S2 around Second 5, then theirexpectation would be violated at that time, and respondingshould decrease during the latter 15 s of S1.

Method

Participants

Eighty-seven participants recruited from a college campusparticipated in the research. They were recruited as inExperiment 1 and were randomly assigned to Group ABA(n = 46) or Group AAA (n = 41), with the goal of at least 40per condition so as to allow adequate counterbalancing (i.e.,eight combinations of variables, described later) within eachgroup. The assignment of participants to groups was entirelyrandom, as in Exper iment 1 . To dis t r ibute thecounterbalancing factors as equally as possible, counterbal-ance conditions within each group were assigned to partici-pants randomly, without the condition being available forreassignment until all conditions had been assigned once toevery eight consecutive participants.

Apparatus

The same apparatus was used as in Experiment 1. TheBoutonia and Nicholosia galaxies were used as contexts.The enemy at G or H in Fig. 1 was used as S2, betweensubjects, for generality.

Procedure

Response training To lessen potential expectation of a partic-ular outcome response training used all four S2s. The gamebegan with the same cover story and end-of-phase instructionsas Experiment 1. Each of the four possible S2 stimuli appearedtwice in a row on the first eight trials. The prestimulus and on-trial instructions of the first trial of a pair, and the prestimulusinstruction of the second trial of a pair mirrored those used onthe first two trials of Experiment 1, with only the names of theS2 and weapon changed. In clockwise order beginning in thetop left of Fig. 1, the weapons were called “Extinction Fire,”“SOP Cannon,” “Lambda Ray,” and “IBO Laser.” In clock-wise order beginning at F, the enemies were labeled“Stellarian,” “Learian,” “Juk Destroyer,” and “Gluckonian.”The names of the weapons were attached to the assigned keyson the keyboard. Trial 9 began with a prestimulus instructionto practice on all invaders. No further instructions were pro-vided until the end of the phase. Referring to the craft labels inFig. 1, the training sequence was G, G, H, H, F, F, I, I, G, I, H,F, I, F, H, F, I, H, G, G.

Conditioning Participants received S1–S2 pairings in ContextA and equal exposure to Context B in the sequence ABAB orBABA. Conditioning took place as it did during the first tentrials in Group 20–20 from the previous experiment, with fiveconditioning trials each time participants were in Context A.All counterbalancing was between subjects. Within eachgroup, the enemy type (e.g., G or H), context ID (Boutoniaor Nicholosia as Context A), and context sequence (ABAB orBABA) were counterbalanced with respect to each other.

Extinction S1 was presented ten times, without S2, in ContextB for Group ABA and Context A for Group AAA. The timebetween S1 presentations averaged 20 s.

Testing All participants were tested in Context A, where S1was presented four times, without S2. The time between S1presentations averaged 20 s.

Data analysis

Responses were recorded from the backspace key for partic-ipants receiving the “Learian” as S2, and from the number-padzero key for those receiving the “Juk Destroyer.” The dataanalysis was as in the previous experiment.

Results

We reached three key conclusions. First, during extinction,responding to S1 decreased both with a context change andover trials. Second, participants timed the appearance of S2 inthe presence of S1; during both extinction and test, respondingdecreased after the point in time at which S2 should haveoccurred. Third, responding and its timing recovered when S1was tested outside of the extinction context.

To confirm the absence of group differences prior to theextinction phase, a Group (AAA/ABA) × Trials × SecondsANOVAwas conducted on the conditioning-phase data.

Responding to S1 increased across trials, particularly in thelatter 4 s, as would be expected from Experiment 1, producinga Trials × Seconds interaction, F(36, 3060) = 16.45, p < .001,η2p = .16. No effect of group emerged,F(1, 85) = 2.10, p = .15,nor any effects involving group,Fs < 1. The response variancewas 4.03.

Pre-S responding throughout the experiment was almostnonexistent. It averaged 0.14 (variance = 0.12). Pre-Sresponding never varied between groups, and will not bereported further.

Conclusion 1

The first conclusion was supported by the data from extinc-tion, shown in Fig. 4. The three-way interaction in a Group(AAA/ABA) × Trials × Seconds ANOVAwas reliable, F(171,

Behav Res

14535) = 1.91, p < .0001, η2p = .02. Within groups,responding averaged over seconds in each group differed fromthe first to the last trial, Fs(1, 255) ≥ 131.0, ps < .0001, ds ≥.84. The extinction response variance was 3.81. Betweengroups, alpha was adjusted for the 200 possible group com-parisons at each second.We observed 27 differences, with lessresponding in Group ABA than in Group AAA on Trials 1(Seconds 1–7 and 9–20) and 2 (Seconds 2–9), Fs(1, 303) ≥7.91, ps ≤ .005, dmin–max = 0.47–1.24.

Conclusion 2

Responding on Second 5 was compared to each of Seconds 6–20 in each group on each trial of extinction. Alpha wasadjusted for 300 possible comparisons. In all, 56 reliabledifferences were observed. In Group ABA, differences ap-peared on Trials 1 (Seconds 11–20) and 2 (Seconds 18–20). InGroup AAA, differences appeared on Trials 1 (Seconds 11–20), 2 (Seconds 7–20), 3 (Seconds 10–20), and 6 (Seconds12–13, 15–20),Fs(1, 2857) ≥ 6.87, ps ≤ .0088, dmin-max = 0.34–1.14. The decrease was not fatigue, since participants couldmaintain high response rates across the stimulus. On the finalconditioning trial, for example, responding during the 15-s S2averaged 5.6 (SD = 0.93).

Conclusion 3

Both the response and its timing recovered on test in GroupABA, shown at right in Fig. 4. A Group × Seconds × TrialsANOVA revealed a three-way interaction, F(57, 4845) = 7.32,

p < .0001, η2p = .08. To investigate the effect of group, alphawas adjusted for 80 comparisons. A total of 24 differencesemerged: Group ABA responded more than Group AAA onTrials 1 (all seconds) and 2 (Seconds 4–7), Fs(1, 214) ≥ 6.1, ps≤ .014, dmin-max = 0.47–1.91. Responding on Second 5 wascompared to each of the remaining seconds in each group oneach trial. Alpha was adjusted for 120 comparisons. Thisrevealed 16 differences, all decreases occurring in GroupABA on Trials 1 (Seconds 8–20) and 2 (Seconds 18–20),Fs(1, 2016) ≥ 9.15, ps ≤ .0025, dmin-max = 0.35–1.44.

Discussion

In Experiment 2, participants received conditioning and ex-tinction in the same or in different contexts. All participantswere tested where conditioning had occurred. A recovery ofthe extinguished responding was expected for those that re-ceived extinction in a different context. The experiment re-vealed that the galaxy-change context manipulation produceda small decline in responding, shown at the beginning ofextinction. Following extinction in the different context,responding recovered robustly when S1 was tested in theconditioning context.

Indications of timing were obtained during the presentationof S1 in extinction and on test. Since S2 was not present onthese trials, any responding should be based on anticipation ofS2. Responding declined after the time at which S2 wasexpected to occur, despite S1 still being present. The patternsproduced are similar to those obtained in a peak procedure(e.g., Roberts, 1981). Unlike in a peak procedure, only a

Fig. 4 Responses per second oneach second of each trial duringthe 20-s Stimulus 1 presentationduring extinction in Context A(Group AAA, open circles) orContext B (Group ABA, solidcircles) in Experiment 2.Performance at testing in ContextA is shown at the right

Behav Res

single, fixed-duration S1 was used, and more than a singleresponse was required to activate the weapon. The initialresponse increase is the accumulation of responses necessaryto activate the weapon, as opposed to the perception of theduration of time passed becoming more similar to a set inter-val. However, the subsequent decline in responding reflectedthat participants were expecting S2 earlier, rather than later, inS1.

General discussion

The present article presents a new videogame method de-signed as a general tool with which to study associative-learning phenomena involving response generation. In aninitial response-training stage, keypress responses are attachedto enemy spaceship S2s. In the experimental phase, pairingthese S2s with flashing-light S1s endows S1 with the ability toevoke the response that was attached to the S2 with which itwas paired.

Experiment 1 demonstrated that conditioning with themethod generally obeys the temporal characteristics ofPavlovian conditioning establishedwith animals. Larger ratiosof the time between trials to the duration of an S1–S2 pairingled to higher rates of conditioning, and lower ratios led tolower rates of conditioning to S1 and increases in respondingto the background contextual stimuli (i.e., context condition-ing). In Experiment 2, we investigated the effects of a contextchange on conditioning and extinction, showing that both aresusceptible to changes in the context, demonstrating the va-lidity of the context manipulation.

During trials with S1 in which S2 was not presented,participants displayed behavior indicative that they had timedthe occurrence of S2. Responding increased to a level neces-sary to activate the weapon when S2 would ordinarily appear,and then declined afterward. Evidence of timing S2 within acue has not been demonstrated in other videogame methods.Molet et al. (2007) demonstrated that participants have anappreciation for the duration of cues by showing the superpo-sition of suppression across cues of different length.Arcediano, Escobar, and Miller (2003) showed timing ofoutcomes in a computer-based task, though it did not have agame-like setting. Their task was developed specifically tomeasure the learning of temporal relationships and includedinstructions that favored participants using information abouttime. In the present method, no special methodological con-siderations were used to make the method any more suited tostudy timing than any other conditioning phenomena, and nospecial instruction was required. Though participants wereencouraged to prepare weapons prior to S2, illumination ofthe lights provided enough information to accomplish thattask. It was not necessary to learn when S2 would appear

within S1. The experiment did not address the mechanismbehind the timing, but demonstrated that its study is tractable.

S1 likely acts as a Pavlovian stimulus. It could evoke arepresentation of S2 that motivates behavior, eliciting theresponse. Responding to S1 in order to fire the weapon im-mediately upon the arrival of S2 could be further supported byavoidance learning by preventing enemies from attacking onarrival. Avoidance learning is possible, but aspects of theprocedure might minimize its contribution. The aiming char-acteristics of the weapons and the movement of the enemiesmade the weapons sometimes miss, allowing an attack. Therate of attack by S2 could be reduced, but an attack couldseldom be completely avoided. The appearance and disap-pearance of S2 was also independent of participants’ behavior.Nevertheless, to the extent that avoidance learning was in-volved, such learning has long been known to depend onPavlovian learning between the signal and the avoided event(e.g., Mowrer, 1960).

The weapon, once prepared with enough responses, wouldonly fire when S2 was present. Thus, S1 could serve adiscriminative-stimulus function, indicating when pressingwould be rewarded by the weapon firing. The timing ofparticipants’ responses suggested that even should S1 serveas a discriminative stimulus, it was accompanied by detailedknowledge of the predicted S2.

In summary, the game appears to be a conceptually simple,powerful, nonintrospective tool capable of studying a varietyof complex questions about learning. The experiments pre-sented here demonstrate a sample of the diverse types ofphenomena that can be addressed by the single method.

Author note The method and research presented here were madepossible by Grant No. PSI2011-24231 from the Spanish Ministry ofScience and Innovation and Grant No. IT-694-13 from the Basque Min-istry of Science.We thankAndrewCrook for his assistance in the creationof the 3-D models used in the game, Peter Clarke for creating the music,Paul Sountsov for math assistance, particularly in calculating 3-D ellip-soid tangent-plane normals, and Gabriel Rodriguez for his assistance withthe voice recordings. The game’s development would not have beenpossible without the members of the forum at http://xboxforums.create.msdn.com/forums/, who assisted in solving a variety of problems.

References

Arcediano, F., Escobar, M., & Miller, R. R. (2003). Temporal integrationand temporal backward associations in human and nonhuman sub-jects. Learning & Behavior, 31, 242–256. doi:10.3758/BF03195986

Arcediano, F., Ortega, N., & Matute, H. (1996). A behavioural prepara-tion for the study of human pavlovian conditioning. QuarterlyJournal of Experimental Psychology, 49B, 270–283. doi:10.1080/713932633

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discoveryrate: A practical and powerful approach to multiple testing. Journalof the Royal Statistical Society, Series B, 57, 289–300.

Behav Res

Collins, D. J., & Shanks, D. R. (2002). Momentary and integrativeresponse strategies in causal judgment. Memory & Cognition, 30,1138–1147. doi:10.3758/BF03194331

Dickinson, A., Shanks, D., & Evenden, J. (1984). Judgement of act-outcome contingency: The role of selective attribution. QuarterlyJournal of Experimental Psychology, 36A, 29–50.

Escobar, M., Pineño, O., & Matute, H. (2002). A comparison betweenelemental and compound training of cues in retrospective revaluation.Animal Learning&Behavior, 30, 228–238. doi:10.3758/BF03192832

Franssen, M., Clarysse, J., Beckers, T., van Vooren, P. R., & Baeyens, F.(2010). A free software package for a human online-conditionedsuppression preparation. Behavior Research Methods, 42, 311–317.doi:10.3758/BRM.42.1.311

Gamez, A. M., & Rosas, J. M. (2007). Associations in human instrumen-tal conditioning. Learning and Motivation, 28, 242–261.

Gibbon, J., & Balsam, P. (1981). Spreading associations in time. In C. M.Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and con-ditioning theory (pp. 219–253). New York, NY: Academic Press.

González-Martín, E., Cobos, P. L., Morís, J., & López, F. J. (2012).Interference between outcomes, spontaneous recovery, and contexteffects as measured by a cued response reaction time task: Evidencefor associative retrieval models. Journal of Experimental Psychology:Animal Behavior Processes, 38, 419–432. doi:10.1037/a0029517

Hawkins, G. E., Rae, B., Nesbitt, K. V., & Brown, S. D. (2012). Gamelikefeatures might not improve data. Behavior Research Methods, 45,301–318. doi:10.3758/s13428-012-0264-3

Howell, D. (1987). Statistical methods for psychology (2nd ed.). Boston,MA: PWS-KENT.

Ivanov-Smolensky, A. G. (1927). On the methods of examining theconditioned food reflexes in children and in mental disorders.Brain, 50, 138–141.

Matute, H., Arcediano, F., & Miller, R. R. (1996). Test questions modu-late cue competition between causes and between effects. Journal ofExperimental Psychology: Learning, Memory, and Cognition, 22,182–196. doi:10.1037/0278-7393.22.1.182

Molet, M., Callejas-Aguilera, J. E., & Rosas, J. M. (2007). Latent timingin human conditioned avoidance. Journal of ExperimentalPsychology: Animal Behavior Processes, 33, 476–483. doi:10.1037/0097-7403.33.4.476

Mowrer, O. (1960). Learning theory and behavior. New York, NY:Wiley.

Nelson, J. B., & Sanjuan, M. (2006). A context specific latent inhibitioneffect in a human conditioned suppression task. Quarterly Journalof Experimental Psychology, 59, 1003–1020.

Nelson, J. B., Sanjuan, M., Vadillo-Ruiz, S., Pérez, S., & Léon, S. P.(2011). Experimental renewal in human participants. Journal ofExperimental Psychology: Animal Behavior Processes, 37, 58–70.doi:10.1037/a0020519

Pineño, O., & Miller, R. R. (2004). Signaling a change in cue–outcomerelations in human associative learning. Learning & Behavior, 32,360–375. doi:10.3758/BF03196034

Pineño, O., Ortega, N., & Matute, H. (2000). The relative activation ofassociations modulates interference between elementally trainedcues. Learning and Motivation, 31, 128–152.

Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian condi-tioning: Variations in the effectiveness of reinforcement adnnonreinforcement. In A. H. Black & W. F. Prokasy (Eds.),Classical conditioning II: Current research and theory (pp. 64–99). New York, NY: Appleton-Century-Crofts.

Roberts, S. (1981). Isolation of an internal clock. Journal of ExperimentalPsychology: Animal Behavior Processes, 7, 242–268. doi:10.1037/0097-7403.7.3.242

Behav Res