INSTITUTE OF PHYSICS PUBLISHING NETWORK: COMPUTATION IN NEURAL SYSTEMS

Network: Comput. Neural Syst. 12 (2001) 317–329 www.iop.org/Journals/ne PII: S0954-898X(01)23519-2

Neural coding of naturalistic motion stimuli

G D Lewen, W Bialek and R R de Ruyter van Steveninck

NEC Research Institute, 4 Independence Way, Princeton, NJ 08540, USA

Received 19 January 2001

AbstractWe study a wide-field motion-sensitive neuron in the visual system of theblowfly Calliphora vicina. By rotating the fly on a stepper motor outside in awooded area, and along an angular motion trajectory representative of naturalflight, we stimulate the fly’s visual system with input that approaches the naturalsituation. The neural response is analysed in the framework of informationtheory, using methods that are free from assumptions. We demonstrate thatinformation about the motion trajectory increases as the light level increasesover a natural range. This indicates that the fly’s brain utilizes the increasein photon flux to extract more information from the photoreceptor array,suggesting that imprecision in neural signals is dominated by photon shot noisein the physical input, rather than by noise generated within the nervous systemitself.

1. Introduction

One tried and tested way to study sensory information processing by the brain is to stimulatethe sense organ of interest with physically appropriate stimuli and to observe the responsesof a selected part of the system that lends itself to measurement. Within that frameworkthere are strong incentives, both practical and analytical, to simplify stimuli. After all, shortlightflashes or constant tones are easier to generate and to capture mathematically than theever-changing complex world outside the laboratory. Fortunately, sense organs and brainsare extremely adaptive, and they apparently function in sensible ways, even in the artificialconditions of typical laboratory experiments. A further reason for using simplified stimuli isthat they are presumed to elicit simple responses, facilitating interpretation of the system’sinput–output behaviour in terms of underlying mechanism. Typically, these simple stimuliare repeated a large number of times, the measured outputs are averaged and this average isdefined to be the ‘meaningful’ component of the response. This is especially helpful in thecase of spiking neurons, where we face the embarrassment of the action potential: becausethey are an extremely nonlinear feature of the neural response we often do not really know howto interpret sequences of action potentials (Rieke et al 1997). One way to evade the questionand save tractability is to work with derived observables, in particular with smooth functionsof time, such as the average firing rate.

Although it certainly is useful to perform experiments with simplified inputs, one alsowould like to know how those stimuli that an animal is likely to encounter in nature are

0954-898X/01/030317+13$30.00 © 2001 IOP Publishing Ltd Printed in the UK 317

318 G D Lewen et al

processed and encoded. We expect animals to be ‘designed’ for those conditions, and itwill be interesting to see to what extent the brain can keep up with the range and strengthof these stimuli. In this paper we are concerned primarily with the question of how noisyneural information processing really is. This question cannot be answered satisfactorily ifwe do not study the problem that the brain is designed to solve, because for us it is hard todistinguish willful neglect on the part of the brain in solving artificial tasks from noisiness ofits components.

As soon as we try to characterize the behaviour of a sensory system in response to thecomplex, dynamic, nonrepeated signals presented by the natural world, we lose many of thesimplifications mentioned earlier. To meet the challenge we must modify both our experimentaldesigns and our methods for analysing the responses to these much more complicated inputs.Recent examples of laboratory-based approaches to the problem of natural stimulation are stud-ies of bullfrog auditory neurons responding to synthesized frog calls (Rieke et al 1995), insectolfactory neurons responding to odour plumes (Vickers et al 2001), cat LGN cells respondingto movies (Dan et al 1996, Stanley et al 1999), primate visual cortical cells during free viewingof natural images (Gallant et al 1998, Vinje and Gallant 2000), auditory neurons in song birdsstimulated by song and songlike signals (Theunissen and Doupe 1998, Theunissen et al 2000,Sen et al 2000), the responses in cat auditory cortex to signals with naturalistic statisticalproperties (Rotman et al 1999) and motion-sensitive cells in the fly (Warzecha and Egelhaaf2001, de Ruyter van Steveninck et al 2001). In each case compromises are struck betweenwell controlled stimuli with understandable statistical properties and the fully natural case.

A more radical approach to natural stimulation was taken by Roeder in the early 1960s (seeRoeder 1998). He and his coworkers made recordings from moth auditory neurons in responseto the cries of bats flying overhead in the open field. More recently the visual system of Limuluswas studied with the animal moving almost free on the sea floor (Passaglia et al 1997).

Here we study motion-sensitive visual neurons in the fly, and—in the spirit of Roeder’swork—rather than trying to construct approximations to natural stimuli in the laboratory, wetake the experiment into nature. We record the responses of H1, a wide-field direction-selectiveneuron that responds to horizontal motion, while the fly is being rotated along angular velocitytrajectories representative for free-flying flies. These trajectories indeed are quite wild, withvelocities of several thousand degrees per second and direction changes which are completewithin ten milliseconds. In analysing the responses to these stimuli we would like to usemethods that do not depend on detailed assumptions about what features of the stimulus areencoded or about what features of the spike sequences carry this coded signal. Recently,information theoretical methods were developed for analysing neural responses to repeatedsequences of otherwise arbitrarily complex stimuli (de Ruyter van Steveninck et al 1997,Strong et al 1998). In our experiments we repeat the same motion trace, lasting several seconds,and this provides us with the raw data for computing the relevant information measures, asexplained in section 2.3. We emphasize that although we repeat the stimulus many times toestimate the relevant probability distributions of responses, the measures we derive from thesedistributions characterize the information coded by a single example of the neural response.

2. Methods

2.1. Stimulus design considerations

The giant motion-sensitive interneurons in the fly’s lobula plate are sensitive primarily to suchrigid rotational motions of the fly as occur during flight (Hausen 1982, Krapp and Hengstenberg1997), and these cells typically have very large visual fields. It is this wide-field rigid rotation

Neural coding of naturalistic motion stimuli 319

that we want to reproduce as we construct a naturalistic stimulus. But what pattern of rotationalvelocities should we use? As a benchmark we will present data from an experiment where thefly was rotated at velocities that remained constant for one second each. We would, however,also like to present the fly with stimuli that are more representative for natural flight. Free-flighttrajectories were recorded in the classic work of Land and Collett (1974), who studied chasingbehaviour in Fannia canicularis and found body turning speeds of several thousand degreesper second. A recent study (van Hateren and Schilstra 1999) reports flight measurements fromCalliphora at high temporal and spatial resolution. In these experiments flies made of the orderof 10 turns s−1, and turning speeds of the head reached values of over 3000◦ s−1. In general,high angular velocities pose problems for visual stimulus displays, because even at relativelyhigh frame rates they may give rise to ghost images. In our laboratory we use a Tektronix608 display monitor with a 500 Hz frame rate; then 3000◦ s−1 corresponds to jumps fromframe to frame of 6◦, four times larger than the spacing between photoreceptors. Althoughthe frame rate used here is well above the photoreceptor flicker fusion frequency (de Ruytervan Steveninck and Laughlin 1996) the presence of ghost images may have consequences forthe encoding of motion signals. Further, the light intensity of the typical displays used in thelaboratory is much lower than outside. As an example, the Tektronix 608 induces of the order of5×104 photoconversions s−1 in fly photoreceptors at its maximum brightness. The brightnessoutside can easily be a factor of a hundred higher (Land 1981), although the photoreceptor pupilmechanism will limit the maximum photon flux to about 106 photoconversions s−1 (Howardet al 1987). Finally, the field of view of H1 is very large, covering essentially the field of oneeye (Hausen 1984, Krapp and Hengstenberg 1997), which is about 6.85 sr or 55% of the full4π sr in female Calliphora vicina (estimates based on Beersma et al (1977), see also figure 1).In practice with a display monitor it is hard to stimulate the fly with coherent motion over sucha large area and in most of our laboratory experiments we stimulate less than about 20% of thefull visual field of H1.

2.2. Stimulus apparatus

All the factors mentioned above suggest an experimental design in which the visual world canbe made to move more or less continuously relative to the fly, and this is easiest to accomplishby moving the fly relative to the world as occurs during free flight. We therefore constructed alight and compact assembly consisting of a fly holder, electrode manipulator and preamplifierthat can be mounted on a stepper motor, as shown in figure 1. This setup is rigid enough toallow high-speed rotations around the vertical axis while extracellular recordings are madefrom the H1 cell. Because it is powered by batteries the setup can be taken outside, so thatthe fly’s visual system is stimulated with natural visual scenes. The mounting and recordingstage inevitably covers some area in the fly’s visual field. During the experiment this rotatesalong with the fly, and so does not contribute to motion in the fly’s visual field. By tracing thecontours of the setup as seen from the fly, we estimate the shape and size of this overlap, asdepicted in figure 1. The setup was designed to minimize the overlap in the visual field of theleft eye. In the experiments presented here, recordings were therefore made from contralateralH1, on the right side of the head. The setup occludes only 1.52 sr, or 22%, of the visual fieldof the left eye, most of it ventral–caudal, as indicated by the heavy mesh in the right panel offigure 1.

The stepper motor (Berger-Lahr RDM 564/50) was driven by a Divistep 331.1 controllerin microstep mode, that is, at 104steps/revolution, corresponding to a smallest step size of0.036◦ or roughly 1/30th of an interommatidial angle. The Divistep controller in turn wasdriven by pulses from a custom-designed interface that produced pulse trains by reading pulse

320 G D Lewen et al

rotation axis

left eye: excluded part of visual field

fly

electrode

electrode holder and amplifier

Figure 1. Left: setup used in the outdoor experiments. The fly is in a plastic tube, head protruding,and immobilized with wax. A small feeding table is made, from which the fly can drink sugarwater.The part of the setup shown here rotates around the axis indicated at the bottom, by means of astepper motor. A silver reference wire makes electrical contact with the body fluid, while a tungstenmicroelectrode records action potentials extracellularly from H1, a wide-field motion-sensitiveneuron in the fly’s lobula plate. The electrode signals are preamplified by a Burr-Brown INA111integrated instrumentation amplifier, the output of which is fed through a slip ring system to asecond-stage amplifier and filter and digitized by a National Instruments PCMCIA data acquisitioncard in a laptop computer. The part of the setup visible in the figure is mounted on a stepper motor,which is driven by computer-controlled laboratory built electronics. Right: occlusion in the leftvisual field of the fly. The dot in the centre represents the position of the fly. The animal is lookingin the direction of the arrow and has the same orientation as the fly in the setup on the left. Themesh to the right of the heavy (blue) line (from the fly’s point of view) represents the excludedpart of the visual field of the left eye for a free-flying fly (based on Beersma et al 1977). The (red)mesh to the left represents the overlap of the left eye’s natural visual field with those parts of thesetup that rotate along with the fly, and therefore do not contribute to a motion signal. The totalvisual field of the left eye is 6.85 sr, or 0.55 × 4π . The overlap subtends about 1.52 sr, or 22% ofthe visual field of the left eye.

(This figure is in colour only in the electronic version, see www.iop.org)

frequency values from the parallel port of a laptop computer. Pulse frequency values wererefreshed every 2 ms.

To generate naturalistic motion stimuli we used published trajectories of chasing Fanniafrom Land and Collett (1974), interpolated smoothly between their 20 ms sample points. Fortechnical reasons we had to limit the accelerations of the setup, and we chose therefore torotate the fly at half the rotational velocities derived from the Land and Collett data. This maybe reasonable as Calliphora is a larger fly than Fannia, and is likely to make slower turns. Theconstant-velocity data presented in figure 2 were taken with rotation speeds ranging from about0.28◦ s−1 to 4500◦ s−1. To avoid extreme accelerations during high-velocity presentations thepulse program for the stepper motor delivered smooth 100 ms pulse frequency ramps to switchbetween velocities. For velocities below 18◦ s−1 pulses were sent to the controller at intervalslonger than 2 ms. At the lowest constant velocity used in our experiments, 0.28◦ s−1, pulseswere delivered at 128 ms intervals. The step size was small enough that a modulation of thePSTH was undetectable in the experiment.

Neural coding of naturalistic motion stimuli 321

10 100 1000 100000

outsidelaboratory

300

250

200

150av

erag

e ra

te (

spik

es/s

)

100

50

0.1 1

velocity (°/s)

Figure 2. A comparison of responses to constant velocity in a typical laboratory experiment (closedsquares), and in an outdoor setting where the fly is rotating (open circles). Average firing rateswere computed over the last 0.5 s of a 1 s constant-velocity presentation.

The experiment of figure 2 compares data from the outdoor setup to data taken inside withthe fly observing a Tektronix 608 CRT. The stimulus displayed on this monitor consisted of 190vertical lines, with intensities derived from a one-dimensional scan of the scene viewed by thefly in the outdoor experiment. The moving scene was generated by a digital signal processor,and written at a 500 Hz frame rate. As mentioned above, this gives rise to ghosting at highimage speeds when the pattern makes large jumps from frame to frame. The DSP producedthe coarse part of motion essentially by stepping through lines in a buffer memory. On top ofthis, fine displacements were produced by moving the entire image by fractions of a linewidthat each frame. The resulting motion was smooth and not limited to integer steps. The fly waspositioned so that the screen subtended a rectangular area of 67◦ horizontal by 55◦ vertical,with the left eye facing the CRT and rightmost vertical edge of the CRT approximately in thesagittal plane of the fly’s head.

2.3. Information theoretic analysis of neural firing patterns

We describe briefly a technique for quantifying information transmission by spike trains (deRuyter van Steveninck et al 1997, 2001, Strong et al 1998). We consider segments of the spiketrain with length T divided into a number of bins of width �t , where �t ranges from onemillisecond up to �t = T . Each such bin may hold a number of spikes, but within a bin nodistinction is made on where the spikes appear. However, two windows of length T that havedifferent combinations of filled bins are counted as different firing patterns. Also, two windowsin which the same bins are filled but with different count values are distinguished. We referto such firing patterns as words, WT,�t . From an experiment in which we repeat a reasonablylong naturalistic stimulus a number of times, Nr (here Nr = 200 repetitions of a Tr = 5 s longsequence), we get a large number of these words, WT,�t (t), with t the time since the start ofthe experiment. Here we discretize t into 1 ms bins, giving us 5000 words/repetition period,and 106 words in the entire experiment. From this set of words we set up word probabilitydistributions, from which we calculate total and noise entropies, and their difference, accordingto Shannon’s definitions.

(i) The total entropy, Stot(T , �t). From the list of words WT,�t (t), for all t (0 � t � NrTr),we directly get a distribution, P(WT,�t ) describing the probability of finding a word

322 G D Lewen et al

a

d 3 ×102

c 2 ×105

b 3 ×106

photons/s per

at zenith

0

0

20

40

0

20

40

0

20

40

-2000

2000

velo

city

(°/

s)

photoreceptor

tria

ltr

ial

tria

l

time (s)

0.0 0.1 0.2 0.3 0.4 0.5

Figure 3. Responses of the H1 neuron to the same motion trace recorded outside at different timesof the day. (a) Short segment of the motion trace executed by the stepper motor with the fly. Thefull segment of motion lasted 5 s, and was derived from video recordings of natural fly flight duringa chase (see methods). (b) 50 Spike rasters in response to the motion trace in (a), taken at noon.(c) As (b), but recorded about half an hour before sunset. (d) As (b), but recorded about half anhour after sunset.

anywhere in the entire experiment. The total entropy is now

Stot(T , �t) = −∑W

P (WT,�t ) log2[P(WT,�t )]. (1)

This entropy measures the richness of the ‘vocabulary’ used by H1 under theseexperimental conditions, hence the time of occurrence of the pattern within the experimentis irrelevant.

(ii) The average noise entropy, S̄noise(T , �t). If the neuron responded perfectly reproduciblyto repeated stimuli, then the information conveyed by the spike train would equal thetotal entropy defined above. There is noise, however, and this leads to variations in theresponses, as can be seen directly from the rasters in figure 3. S̄noise(T , �t) gives usan estimate of how variable the response to identical stimuli is. We first accumulate,for each instant tr in the stimulus sequence, the distribution of all those firing patternsP(WT,�t |tr), taken across all trials, that begin at tr (note that 0 � tr � Tr). The entropyof this distribution measures the (ir)reproducibility of the response at each instant tr:

Snoise(T , �t, tr) = −∑W

P (WT,�t |tr) log2[P(WT,�t |tr)]. (2)

Calculating this for each point in time and averaging all these values we obtain the averagenoise entropy:

S̄noise(T , �t) = 1

Tr

∫ Tr

0Snoise(T , �t, tr) dtr. (3)

Neural coding of naturalistic motion stimuli 323

(iii) The information conveyed by words at the given length T and resolution �t is thedifference of these two entropies:

I (T , �t) = Stot(T , �t) − S̄noise(T , �t). (4)

The coding efficiency of the spike train is the fraction of the total entropy that is utilizedto convey information:

η(T , �t) = I (T , �t)

Stot(T , �t). (5)

Small values of η(T , �t) indicate a loose coupling between stimulus and spike train, whereasvalues close to 1 imply that there is little noise entropy, so that most of the structure of thespike train is meaningful, and carries a message. Here we will not be interested in the decodingquestion, that is in what that message is, but only in how much information is conveyed aboutthe stimulus. We will then compare these values in different conditions.

It should be stressed that the information values we derive by these methods are not strictlyabout velocity. They are potentially about anything in the stimulus that is repetitive with periodTr. It is our job as experimenters to construct inputs that we think will stimulate the neuronwell, and for H1, naturalistic wide-field motion seems to be a good choice. But that does notnecessarily mean that that is the best choice. Further, the motion pattern is dynamic, and anynoiseless time-invariant operation on this signal will produce a result that has the same repeatperiod as the original. Our information measures do not distinguish these cases; specifically,our discussion is unaffected by the question of whether H1 encodes velocity, acceleration orsome nonlinear function of these variables. Questions of decoding are highly interesting, butat the same time difficult to tackle for stimuli of the type studied here, and we will leave themaside in this paper.

It is interesting to try and estimate I (T , �t) as we let T become very long, and �t veryshort, as this limit is the average rate of information transmission. Because calculating thislimit requires very large data sets, we focus here on the information transmitted in constanttime windows, T = 30 ms, as a function of �t . We choose T = 30 ms because that amountsto the delay time with which a chasing fly follows turns of a leading fly during a chase (Landand Collett 1974); the end result, that is the dependence of information transmission on �t ,was found not to depend critically on the choice of T .

To quantify noise entropy, the method described above requires that a stimulus waveformbe repeated. Although it is possible in principle to quantify information transmission basedonly on one repetition, using many repetitions is easier in practice. In a sense this mode ofstimulation is still removed from the realistic situation in which stimuli are not repeated at all.Indeed, in our experiments there are hints that the fly adapts to the stimulus somewhat over thefirst few presentations of the 5 s long stimulus. The effects of adaptation to dynamic stimuliare certainly interesting (Brenner et al 2000, Fairhall et al 2001), but in the data we presenthere we skip the first few presentations, and only analyse that part of the experiment in whichthe fly seems fully adapted to the ongoing dynamic stimulus. Inspection of the rasters in thatphase shows no obvious trends, so that the fly seems to be close to stationary conditions. Inthis regard our information measures are lower bounds, as deviations from stationarity willincrease our estimate of the noise entropy, lowering information estimates.

324 G D Lewen et al

3. Results

3.1. Operating range for naturalistic motion stimuli

In order to be sure that H1 receives no dominant motion-related signals from other modalitiesthan vision we rotated the fly either in darkness or under a cover that turned along with thefly. This did not produce discernible motion responses in H1. Strictly speaking that does notexclude possible modulatory mechanosensory input, which could be investigated in principleby presenting conflicting visual and mechanosensory stimuli. The possibility seems remote,however, and even if true it would not invalidate our conclusions about the dependence of H1’sinformation transmission on parameters of the visual stimulus.

As a first comparison between laboratory and natural conditions we present data froman experiment in which H1 was excited by one second long episodes of motion at constantvelocity. These were presented at a range of velocities from about 0.28◦ s−1 to 4500◦ s−1.Outdoors the fly was placed in a wooded environment and rotated on the stepper motor. Inthe laboratory the same fly watched a vertical bar pattern derived from a one-dimensionalscan of the natural environment in which the outdoor experiment was done. This pattern wasdisplayed on a standard Tektronix 608 monitor, with a rectangular stimulated visual area of 67◦

horizontal by 55◦ vertical. The pattern moved at the same settings of angular velocity as wereused outdoors, but the indoor and outdoor stimuli differed both in average light level and instimulated area.

Figure 2 shows the average firing rates obtained from the last half second of each velocitypresentation. At low velocities, up to about 20◦ s−1, the spike rates for both conditions are notvery different, despite the large change in total motion signal present in the photoreceptor array.Apparently the fly adapts these differences away (see Brenner et al 2000). In both experimentsthe rate depends roughly logarithmically on velocity over an appreciable range and this ispartly a result of adaptation as well (de Ruyter van Steveninck et al 1986). In the laboratoryexperiment the motion response peaks at about 100◦ s−1, whereas in natural conditions the flyencodes velocities monotonically for an extra order of magnitude, its response peaking in theneighbourhood of 1000◦ s−1. This brings H1’s encoding of motion under natural conditionsinto the range of behaviourally relevant velocities. A lack of sensitivity to high speeds has beenclaimed both to be an essential result of the computational strategy used by the fly, and to beadvantageous in optomotor course control (Warzecha and Egelhaaf 1998). These conclusionsdo not pertain to the conditions in the outdoor experiment, where H1 responds robustly andreliably to angular velocities of well over 1000◦ s−1.

3.2. Motion detection throughout the day

Figure 3 shows spike train rasters generated by H1 in three outdoor experiments, focusing on ashort segment that illustrates some qualitative points. Trace (a) shows the velocity waveform,which was the same in all three cases. The experiments were performed at noon (b), halfan hour before sunset (c) and about half an hour after sunset (d). Rough estimates of thephoton flux in a blowfly photoreceptor looking at zenith are shown beside the panels. In allexperiments the fly saw the same scene, with a spatial distribution of intensities ranging fromabout 5 to 100% of the zenith value.

The figure reveals that some aspects of the response are quite reproducible, and furtherthat particular events in the stimulus can be associated reliably with small numbers of spikes.More dramatically, the timing precision of the spike trains gradually decreases going fromthe noon experiment to the one after sunset. Higher photon rates imply a more reliablephysical input to the visual system. The figure therefore strongly suggests that the fly’s

Neural coding of naturalistic motion stimuli 325

visual system utilizes this increased input reliability to compute and encode motion moreaccurately when the light intensity increases. This statement is ecologically relevant, as theconditions of the experiment correspond to naturally occurring light levels and approximatelyto the naturally stimulated visual area. To get a feeling for the spike timing precision in thethree conditions we can simply look at the distribution of timing of the first spike generatedafter a fixed criterion time (for which we choose t = 0.28 s in (b) and (c) and t = 0.30 sin (d)). The jitter in the spike timing across different trials has a standard deviation of0.95 ms in (b), 1.4 ms in (c) and 5.8 ms in (d). The relative timing of spikes can beeven more accurate: the interval from the first to the second spike fired after the criteriontime is 2.3 ± 0.23 ms in (b), 5.0 ± 0.6 ms in (c) and 16 ± 2.4 ms in (d). Compared tothe rapid onsets and offsets of the spike activity at the higher photon fluxes, the stimulusvaries rather smoothly, which means that the time definition of spikes with respect to thestimulus can be much better than might be suggested by the stimulus bandwidth. An examplecan be seen in the rather smooth hump in the velocity waveform at about t = 0.43 s,which induces on most trials a well defined response consisting of a sharply defined pairof spikes.

We quantify these impressions using the information theoretic approach described brieflyin section 2.3. The result of this analysis is shown in figures 4(a)–(d), for the threedifferent experiments discussed above. Figure 4(c) clearly shows that the informationin a 30 ms window increases both when the light intensity goes up, and when thespikes are timed with higher accuracy. The increase in information with increasing spiketiming precision is most dramatic for the highest light levels, indicating that coding byfine spike timing becomes more prominent the better the input signal to noise ratio. Acomparison of figures 4(a) (total entropy) and figure 4(b) (noise entropy) reveals that theincrease in information content with increasing light levels is primarily due to an increasein total entropy: the neuron’s vocabulary increases in size as its input becomes betterdefined. Figure 4(d) shows that at the two highest light intensities the coding efficiencyis of order 0.5 at time resolution �t = 1 ms, increasing slightly for larger values of�t . In the darkest condition the efficiency decreases markedly for all values of timeresolution.

The right column of figure 4 compares experiments in which we took data bothoutdoors and in the laboratory. These data are from another fly, but the conditions ofthe outdoor experiment were similar to those for the first fly at the highest light level.After the outdoor experiment the fly was taken inside the laboratory, and the same velocitystimulus as the one used outside was repeated inside. In the laboratory, as before,the visual stimulus was presented on a Tektronix 608 CRT. Photoreceptors facing theCRT received about 5 × 104 photons s−1 at maximum intensity, a value in between thelight intensities seen by the first fly in the experiments just before and just after sunset(grey and black symbols in figures 4(a)–(d)). Two experiments were done indoors, onein which the picture on the monitor consisted of vertical bars with a contrast patternmeasured in a horizontal scan of the outdoor scene (filled triangles), the other a high-contrast square wave pattern with contrast = 1, and spatial wavelength = 12.5◦ (filledsquares). From figure 4(g) we see that the information transmitted by H1 is much lowerin the laboratory experiments than in the outdoor experiment, due to the smaller stimulatedarea and the lower light level. Figure 4(e) shows that the decrease in information, asbefore, is mainly due to a lowering of the total entropy. The noise entropy also decreases(figure 4(f )), but not enough to compensate. Somewhat surprisingly, the experiment withthe high-contrast pattern indoors leads to a slightly higher coding efficiency than even theoutdoorexperiment.

326 G D Lewen et al

zenith photon flux(photons/s per photoreceptor)

time resolution ∆t (ms)

codi

ng e

ffici

ency

0.0

0.2

0.4

0.6

0.8

1.0

nois

e en

trop

y (b

its)

0

2

4

6

8

10

info

rmat

ion

(bits

)

0

1

2

3

4

5

6

a

b

h

g

f

e

d

c

info

rmat

ion

(bits

)

0

1

2

3

4

5

6

3×102

2×105

3×106

time resolution ∆t (ms)

0

codi

ng e

ffici

ency

0.0

0.2

0.4

0.6

0.8

1.0

nois

e en

trop

y (b

its)

0

2

4

6

8

10

tota

l ent

ropy

(bi

ts)

0

2

4

6

8

10

location and stimulus pattern

tota

l ent

ropy

(bi

ts)

0

2

4

6

8

10

outsidelab, bar patternlab, sampled scene

1 101 10

1 10

1 10

1 10

1 10

1 10

1 1

Figure 4. Left-hand column: information theoretic quantities for the three outdoor experimentswhose rasters are shown in figure 3. The symbol shadings refer to the different conditions ofillumination in the experiments. All figures refer to a 30 ms measurement window in whichneural firing patterns are defined at time resolutions, �t , of 1, 2, 3, 5, 10, 15 and 30 ms, asgiven by the abscissae. (a) Total entropy of spike firing patterns. (b) Average noise entropy.(c) Average information transmitted by firing patterns. (d) Coding efficiency, defined as thetransmitted information divided by the total entropy. Right-hand column: the same quantitiesas plotted in the left-hand column, but now for an experiment outdoors (open symbols), and twoexperiments in the laboratory (closed symbols, see text for further description of conditions).Squares are for a moving square wave pattern of high contrast (C = 1) and spatial wavelength12.5◦; triangles are for a moving sample of the natural scene at the location where the outdoorexperiments were done. Both these stimulus patterns were generated on the cathode ray tube inthe laboratory.

Neural coding of naturalistic motion stimuli 327

4. Discussion

Outdoor illumination can easily be a hundred times brighter than anything displayed oncommon laboratory equipment, and in the outdoor experiment stimuli extend over a largefraction of the fly’s full visual field rather than being confined to a small flat monitor. Botheffects are relevant for our experiments, as the higher brightness leads to higher photoreceptorsignal to noise ratios (de Ruyter van Steveninck and Laughlin 1996), and as H1’s receptivefield covers almost a hemisphere (Krapp and Hengstenberg 1997). In moving from laboratoryto outdoor conditions, both effects increase the signal to noise ratio of the input available forcomputation of rigid wide-field motion from the photoreceptor array. The question then iswhether the fly’s brain uses this improvement in input signal quality to produce more accurateestimates of visual motion, and/or increase its operating range of motion detection. Figure 2shows that the range of velocities that are encoded increases markedly when the visual inputbecomes more reliable.

If the accuracy of information processing is limited by noise sources within the nervoussystem, we should observe a plateau, that is, information transmission should saturate at somedefined level of input signal quality. There is some arbitrariness in the choice of the level ofinput signal quality, however: in principle we can surpass any degree of accuracy of the physicalinput signal by simply increasing the light intensity, and at some point the internal randomnessof the brain’s components must become the limiting factor in information processing. However,statements about the magnitude of internal versus external noise in sensory informationprocessing are primarily meaningful in the context of reasonable, physiological levels of inputsignal quality. Those stimuli that the animal encounters naturally, taken at the high end oftheir dynamic range, would meet this criterion. For the case considered here the dynamicrange refers to light intensity, size of stimulated visual field and dynamics of motion. The datawe recorded outdoors show no sign of saturation in information transmission when the inputsignal quality increases. On the contrary, if we compare the rasters of figures 3(b) and (c), wesee that there is a marked improvement in the timing of spikes, even over the highest decade oflight intensity (2 × 105–3 × 106 photons s−1 at zenith per photoreceptor). This improvementtranslates into a significant gain in information transmission, especially at fine time resolution,as shown in figure 4(c). Thus, in computing motion from the array of photoreceptors, the fly’sbrain does not suffer noticeably from information bottlenecks imposed by internal noise, underecologically relevant conditions.

In our outdoor experiments, the information content of the spike train varies primarilyas a result of a varying total entropy (figure 4(a)). The noise entropy (figure 4(b)) appearsto be almost constant as a function of light level. One can distinguish two different ways toincrease information transmission through a channel. The first is to encode the same messagesmore accurately, the second to increase the variety of messages, keeping the accuracy of eachindividual message the same. The first scheme implies constant total entropy and decreasingnoise entropy, the second an increase in total entropy at constant noise entropy. Our data suggestthat as the visual input becomes more reliable, the fly chooses to increase the vocabulary ofH1 to encode a wider variety of features of the motion stimulus, keeping precision roughlyconstant.

Acknowledgments

We thank Naama Brenner, Steve Strong and Roland Koberle for many pleasant and enlighteningdiscussions.

328 G D Lewen et al

References

Beersma D G M, Stavenga D G and Kuiper J W 1977 Retinal lattice, visual field and binocularities in flies. Dependenceon species and sex J. Comp. Physiol. 119 207–20

Brenner N, Bialek W and de Ruyter van Steveninck R 2000 Adaptive rescaling maximizes information transmissionNeuron 26 695–702

Dan Y, Atick J J and Reid R C 1996 Efficient coding of natural scenes in the lateral geniculate nucleus: experimentaltest of a computational theory J. Neurosci. 16 3351–62

de Ruyter van Steveninck R, Borst A and Bialek W 2001 Real-time encoding of motion: answerable questions andquestionable answers from the fly’s visual system Motion Vision. Computational and Ecological Constraints edJ M Zanker and J Zeil (Berlin: Springer) pp 279–306

de Ruyter van Steveninck R R and Laughlin S B 1996 Light adaptation and reliability in blowfly photoreceptors Int.J. Neural Syst. 7 437–44

de Ruyter van Steveninck R R, Lewen G D, Strong S P, Koberle R and Bialek W 1997 Reproducibility and variabilityin neural spike trains Science 275 1805–8

de Ruyter van Steveninck R R, Zaagman W H and Mastebroek H A K 1986 Adaptation of transient responses ofa movement sensitive neuron in the visual system of the blowfly Calliphora erythrocephala Biol. Cybern. 54223–36

Fairhall A L, Lewen G D, Bialek W and de Ruyter van Steveninck R R 2001 Multiple timescales of adaptation in aneural code Adv. Neural Inform. Process. Syst. 14 at press

Gallant J L, Conner C E and van Essen D C 1998 Neural activity in areas V1, V2 and V4 during free viewing ofnatural scenes compared to controlled viewing Neuro. Rep. 9 1673–8

Hausen K 1982 Motion sensitive interneurons in the optomotor system of the fly. II. The horizontal cells: receptivefield organization and response characteristics Biol. Cybern. 46 67–79

——1984 The lobula-complex of the fly: structure, function and significance in visual behaviour Photoreception andVision in Invertebrates ed M A Ali (New York: Plenum) pp 523–59

Howard J, Blakeslee B and Laughlin S B 1987 The intracellular pupil mechanism and the maintenance of photoreceptorsignal to noise ratios in the blowfly Lucilia cuprina Proc. R. Soc. B 231 415–35

Krapp H G and Hengstenberg R 1997 A fast stimulus procedure to determine local receptive field properties ofmotion-sensitive visual interneurons Vis. Res. 37 225–34

Land M F 1981 Optics and vision in invertebrates Handbook of Sensory Physiology ed H Autrum (Berlin: Springer)pp 472–592

Land M F and Collett T S 1974 Chasing behavior of houseflies (Fannia canicularis). A description and analysisJ. Comp. Physiol. 89 331–57

Passaglia C, Dodge F, Herzog E, Jackson S and Barlow R 1997 Deciphering a neural code for vision Proc. Natl Acad.Sci. USA 94 12649–54

Rieke F, Bodnar D and Bialek W 1995 Naturalistic stimuli increase the rate and efficiency of information transmissionby primary auditory neurons Proc. R. Soc. B 262 259–65

Rieke F, Warland D, de Ruyter van Steveninck R R and Bialek W 1997 Spikes: Exploring the Neural Code (Cambridge,MA: MIT Press)

Roeder K D 1998 Nerve Cells and Insect Behavior (Cambridge, MA: Harvard University Press)Rotman Y, Bar Yosef O and Nelken I 1999 Responses of auditory–cortex neurons to structural features of natural

sounds Nature 397 154–7Sen K, Wright B D, Bialek W and Doupe A J 2000 Discovering features of natural stimuli relevant to forebrain

auditory neurons Soc. Neurosci. Abstr. 551.2Stanley G B, Li F F and Dan Y 1999 Reconstruction of natural scenes from ensemble responses in the lateral geniculate

nucleus J. Neurosci. 19 8036–42Strong S P, Koberle R, de Ruyter van Steveninck R R and Bialek W 1998 Entropy and information in neural spike

trains Phys. Rev. Lett. 80 197–200Theunissen F E and Doupe A J 1998 Temporal and spectral sensitivity of complex auditory neurons in the nucleus

HVc of male zebra finches J. Neurosci. 18 3786–802Theunissen F E, Sen K and Doupe A J 2000 Spectral–temporal receptive fields of nonlinear auditory neurons obtained

using natural sounds J. Neurosci. 20 2315–31van Hateren J H and Schilstra C 1999 Blowfly flight and optic flow II. Head movements during flight J. Exp. Biol.

202 1491–500Vickers N J, Christensen T A, Baker T and Hildebrand J G 2001 Odour-plume dynamics influence the brain’s olfactory

code Nature 410 466–70

Neural coding of naturalistic motion stimuli 329

Vinje W E and Gallant J L 2000 Sparse coding and decorrelation in primary visual cortex during natural vision Science287 1273–6

Warzecha A-K and Egelhaaf M 1998 On the performance of biological movement detectors and ideal velocity sensorsin the context of optomotor course stabilization Vis. Neurosci. 15 113–22

——2001 Neural encoding of visual motion in real-time Motion Vision. Computational and Ecological Constraintsed J M Zanker and J Zeil (Berlin: Springer) pp 239–78