SPATIAL COGNITION AND COMPUTATION, 4(3), 255–272 Copyright © 2004, Lawrence Erlbaum Associates, Inc.

Human Path Integration by Optic Flow

Timothy M. Ellmore and Bruce L. McNaughton University of Arizona

Path integration or ‘dead reckoning’ is the ability to keep track of relative position using self-motion signals that convey information about speed and direction of movement. Most animal species, including humans, exhibit some degree of path integration capability and neurophysiological studies have demonstrated that self-motion signals are sufficient to update internal representations of both position and orientation. In the present study, human subjects were required to monitor their position or orientation on the basis of unstructured optic flowfields. Trials were conducted at different speeds to examine the accuracy of path integration and rates of random error accumulation, and at two different head azimuths to prevent a confounding strategy of position updating based primarily on tracking changes in the angular declination of distant landmarks with respect to the horizon. Participants integrated the speed of visual motion to update accurately a representation of their position and orientation within the environment. Consistent with the characteristics of real-world path integration, errors accumulated linearly with the magnitude of position and orientation estimation. We conclude that coherent optic flowfields provide a sufficient basis for humans to keep track of their position and orientation relative to remembered landmarks. Keywords: Path integration, navigation, optic flow, eye movements, virtual reality.

Timothy M. Ellmore, Department of Psychology and ARL Division of Neural Systems, Memory, and Aging, University of Arizona; Bruce L. McNaughton, Departments of Psychology and Physiology, and ARL Division of Neural Systems, Memory, and Aging, University of Arizona

Correspondence concerning this article should be addressed to Bruce L. McNaughton, Ph.D., Arizona Research Laboratories, Division of Neural Systems, Memory, and Aging, University of Arizona, 384 Life Sciences North Building, PO Box 24-5115, Tucson, AZ 85724-5115; email: bruce@nsma.arizona.edu.

256 ELLMORE, MCNAUGHTON

More than a century ago, Darwin theorized that animals use a dead reckoning strategy to make return trajectories to departure points (Darwin, 1873). Later (Etienne, Maurer, and Saucy, 1988; H. Mittelstaedt and Mittelstaedt, 1982; Mittelstaedt and Glasauer, 1991; M. L. Mittelstaedt and Mittelstaedt, 1980), experiments showed that, in mammals, the linear and angular components of return trajectories are a consequence of integrating internal cues (H. Mittelstaedt and Mittelstaedt, 1973) from vestibular signals and motor efference copy (‘path integration’). Neurophysiological studies have revealed place and head direction cells in rodents and primates, and self-motion signals are sufficient to update these neural representations of location and orientation in darkness or in the absence of familiar landmarks (McNaughton et al., 1996).

In humans, either vestibular or somatosensory information is sufficient for accurate whole-body linear and angular path integration. Using only inertial otolithic signals, humans can estimate passive linear displacements (Israel and Berthoz, 1989; Israel, Chapuis, Glasauer, Charade, and Berthoz, 1993) and reproduce translational velocity profiles (Berthoz, Israel, Georges-Francois, Grasso, and Tsuzuku, 1995). Humans can also accurately reorient themselves after whole-body angular displacements around the earth-vertical axis, using only inertial signals from the semi-circular canals (Israel, Sievering, and Koenig, 1995). Somatosensory cues can compensate for vestibular system damage, and substratal information may even predominate during human locomotion (M. L. Mittelstaedt and Mittelstaedt, 2001). In a blindfolded walking task, labyrinthine-defective humans can estimate linear distance as well as normal subjects (Metcalfe and Gresty, 1992). Labyrinthine-defective subjects can also compensate for the loss of their vestibular senses by relying on somatosensory information to estimate whole-body rotations (Bles, De Jong, and De Wit, 1984).

Gibson’s theory of ecological optics proposed that an observer could recover information about the speed and direction of movement through the environment from ‘optic flow’, the changing pattern of coherent motion on the retina during locomotion (Gibson, 1950). This proposal led to the experimental demonstration that a compelling illusion of self-motion, or ‘vection’, can be elicited by a purely visual, coherently moving stimulus that occupies a large portion of the visual field (Berthoz, Pavard, and Young, 1975). Human estimates of angular displacement from optic flow alone are reasonably accurate on average, with an overall gain near 1.0, but are influenced by rotation velocity, fixation pattern, and possibly the distance of environmental surfaces and configuration of previously visible landmarks (Becker, Raab, and Jurgens, 2002; Jurgens, Nasios, and Becker, 2003; Mallot and Gillner, 2000; Probst, Krafczyk, Brandt, and Wist, 1984). Results from other studies suggest that humans can use optical velocities for accurate distance discrimination (Bremmer and Lappe, 1999; Frenz, Bremmer, and Lappe, 2003), reproduction (Bremmer and Lappe, 1999), and estimation (Redlick, Jenkin, and Harris, 2001). Humans can also accurately estimate their heading direction from optic flow (Royden, Banks, and Crowell,

OPTIC FLOW PATH INTEGRATION 257

1992; Warren and Hannon, 1988), and rely on the optic flow field to control the direction of their locomotion (Warren, Kay, Zosh, Duchon, and Sahuc, 2001).

Hypothesis and Prediction The relative contributions of optic flow and other body senses (e.g., vestibular and proprioceptive) to position estimation have been investigated in virtual and real environments in three previous studies that required human subjects to make responses that combined estimates of linear and angular distance (Bakker, Werkhoven, and Passenier, 1999; Kearns, Warren, Duchon, and Tarr, 2002; Peruch, May, and Wartenberg, 1997; Riecke, van Veen, and Bulthoff, 2002). The results of these studies suggest that humans may be able to path integrate using optic flow, but when information from body senses is available it appears to dominate.

The term “integration” corresponds to the hypothesis that the brain computes distance by cumulating successive positions along the path (M. L. Mittelstaedt and Mittelstaedt, 1980). Under this hypothesis, if one knows one’s location and orientation at time t, as well as one’s linear and angular speed and elapsed time, one’s position and orientation at time t + ∆t can be calculated. An ideal path integrator will cumulate successive estimates of translation and rotation changes to update representations of location and orientation; however, even if the successive translation and rotation change estimates are quite accurate, on average, random errors must occur during the updating process because no physical system is noise free. The individual random errors accumulate and cause the position representation to “drift” (i.e., become increasingly inaccurate). During repeat estimates of path length, integrator drift will cause the standard deviation of location estimation error to increase linearly with distance traveled (Benhamou, Sauve, and Bovet, 1990).

Neurophysiological evidence from several rodent path integration experiments (reviewed in McNaughton et al 1996) suggests there are separate neural systems for computing the linear and angular components of path integration. In the present study, we investigated, first, whether humans could estimate in two separate experiments their position or orientation relative to remembered landmarks using only self-motion information from optic flowfields, and second, whether their linear or angular estimation errors accumulated linearly with distance, consistent with behavioral observations of mammalian path integration in the real world. Most previous studies of position and orientation discrimination from optic flowfields have been conducted with participants’ view oriented to the horizon and parallel to textured or random-dot ground planes (Bremmer and Lappe, 1999; Kearns et al., 2002; Redlick et al., 2001; Royden et al., 1992; Warren and Hannon, 1988). Position estimation when the observer’s view is oriented parallel to the ground plane raises a potential confound for the study of linear path integration, as participants may use a strategy of tracking, with a single smooth-pursuit eye movement, a point representing the location of a distant visible, or previously visible, landmark during simulated movement. In the present study, position and orientation trials

258 ELLMORE, MCNAUGHTON

were conducted with the participants’ viewpoint oriented parallel to the ground plane (head-up, Figure 2a), and with the participants’ viewpoint tilted down (head-down, Figure 2b), thereby simulating a view directed to the ground in front of one’s feet, and eliminating the potential confound just described.

Method

Two experiments were conducted. Each experimental session was limited to 1 hour, during which all participants explored a virtual environment (Figure 1) by traveling at their own pace between proximal landmarks. In the first experiment, one set of participants was asked, in a series of trials after exploration, to judge their linear position with respect to previously visited landmarks. In the second experiment, another separate set of participants was asked to judge their angular orientation relative to previously visited landmarks. Position and orientation estimation trials occurred during simulated passive linear or angular displacement, and after removal of all distal and proximal cues. Trials were conducted at speeds within and beyond the range of movement experienced during self-paced exploration so that participants had to integrate the speed of optic flow and could not simply estimate their position or orientation by reproducing their movements or using a purely time-based strategy. The only source of self-motion information during a trial was the visual movement of the randomly textured ground plane. Trials were conducted at three constant velocities, and at two head azimuths (Figure 2).

Figure 1. Snapshot of a typical view of the virtual environment during exploration. The red box marks the distant landmark as a target.

OPTIC FLOW PATH INTEGRATION 259

Figure 2. Head azimuths during virtual navigation. a, During head-up estimation, the view is oriented toward the horizon and parallel to the ground plane. b, During head-down estimation, the view is tilted 45-deg toward the ground plane so that the horizon is no longer visible.

Participants A total of 48 undergraduates (24 males, 24 females) volunteered for two experiments. In the first experiment, a subset of this group (14 males, 15 females) completed position estimation trials. For the second experiment, a separate subset (10 males, 9 females) completed orientation estimation trials. Participants were recruited by way of an online experimental scheduling system from an introductory level undergraduate psychology class. Written consent was obtained from each participant in accordance with a protocol approved by the University of Arizona Human Subjects Protection Program. Each participant received 2 extra-credit points for the class.

Experimental Apparatus and Virtual Environment Participants interacted with a virtual environment (Figure 1) via a joystick while seated 71 cm in front of a 40 cm wide by 30 cm high computer monitor (59o by 44o FOV, or 13.5 pixels per 1o visual angle on the 800-by-600 pixel screen display). Head movements were minimized by use of a combination forehead/chinrest, and eye movements were recorded with an ISCAN ETL-200 infrared eye tracking system. The virtual environment consisted of an arena (diameter of 51 eye-heights) containing 10 proximal landmarks (trees, bench, mailbox, street lamp and wooden crate). The arena was bounded by a 16-sided textured stone wall, and a distal scene (mountains, sun, clouds and sky) was simulated by a high-resolution (512-by-512 pixels) sky-box (Terragen, Planetside Software, www.planetside.co.uk). The virtual environment was rendered at the monitor refresh rate of 60 Hz with a custom-written hardware-accelerated graphics program (OpenGL 1.2 on an NVIDIA GeForce2

260 ELLMORE, MCNAUGHTON

MX/MX400 graphics board). Each participant’s position, view, and vertical and horizontal eye position were recorded at each frame display of the virtual environment.

Exploration and Position and Orientation Tasks Participants actively explored the virtual environment with a joystick by approaching, and then colliding with, proximal “target” landmarks (Figure 1). Exploration served to familiarize participants with the distances among landmarks during self-paced navigation (i.e., calibrate their movements with the speed of optic flow in the environment), and to acquaint participants with the nature of the optic flowfields in each of the two head orientations. A landmark was designated as a target when a red box appeared above it. During exploration, target landmarks appeared in pseudorandom order, and participants were instructed to collide with as many target landmarks as possible during two exploration phases. Collisions were signaled by a unique sound corresponding to each landmark. During exploration, participants controlled their speed of movement through the world, but the maximum translational velocity was limited to 2.1 eye-heights/sec, and the maximum rotational velocity was limited to 136 deg/sec. For both the position and orientation experiments, participants explored the virtual world during two separate exploration phases. The first 10-min exploration phase was conducted with participants’ view in the head-up configuration. After the first exploration phase, participants completed a randomized set of either position or orientation trials presented in both head-up and head-down orientations. Following the first set of trials, participants completed a second 10-min exploration phase in which their view was varied between both head-up and head-down configurations. At the start of the second exploration phase, the participants’ view was placed in the head-up orientation. Then, after the target landmark appeared in the FOV for 2000 ms, the participants’ head orientation was tilted down during the journey to the target. Upon collision with the target, the participants’ view was tilted back to the head-up configuration until the next target landmark appeared in the FOV for 2000 ms. In this second exploration phase, the shift from head-up to head-down during travel between landmarks was done so that, in the head-up orientation, participants could see where they were going (i.e., where the landmark was), and in the head-down orientation, participants could experience the optic flowfield during self-paced navigation. After the second exploration phase, participants completed a second, final set of either position or orientation trials presented in both head-up and head-down configurations.

Each position estimation trial began with the participant positioned at one of the proximal landmarks. All of the proximal and distal landmarks were visible at the beginning of the trial, and each participant’s view was oriented toward one of the previously visited target landmarks. After 2000 ms, the distal scene and all proximal landmarks, including the target landmark, disappeared, leaving only a randomly textured ground plane. In half of the trials, the participant’s head orientation was tilted down (head-down, Figure 1d) 45o toward the ground, such

OPTIC FLOW PATH INTEGRATION 261

that only the ground plane in front of the participant was visible, while in the other half of the trials, the participant’s head orientation was maintained toward the horizon and parallel to the ground plane (head-up, Figure 1c). Then, 1000ms later, participants were passively displaced toward the previously visible target landmark at one of three speeds (1.2, 2.8, or 4.4 eye-heights/sec). Participants were instructed to press the joystick trigger when “you believe you are about to collide with the target object”. After a trigger response, the screen went blank and the participant was prompted to start the next trial when ready.

Orientation estimation trials were similar to position estimation trials, except that, after the distal scene and proximal landmarks disappeared, the participant’s position remained stationary and his/her view was rotated CW or CCW a variable number of degrees at one of two constant velocities (70 deg/sec and 170 deg/sec) in either a head-down or head-up orientation. While the two angular displacement velocities were chosen to be at the low and high range of the turn speeds experienced during the exploration phase, they tended to be higher than rotation rates used in previous studies of real-world self-rotation estimation (Israel et al., 1995; Mittelstaedt, 1995). Participants were instructed to wait for the passive rotation to end, and then “use the joystick to rotate your viewpoint such that you are facing directly the position of the previously visible target object”. Participants were not given any information about the accuracy of their position or orientation estimates.

Results

Position Estimation For position estimation trials, a linear regression of each participant’s estimated position onto the actual position of the target landmarks was used to assess position estimation accuracy. The average regression line and the averages of participants’ mean estimated positions are plotted for head-up (Figure 3a) and head-down (Figure 3b) orientations at the three passive displacement velocities, along with the 29 participants’ individual regression lines (Figures 3a, 3b insets). For each participant, a slope and intercept for an equation relating the actual positions (AP) of targets to the participant’s estimated positions (EP) was computed for each displacement velocity at each head orientation. In the head-down position, there was a systematic over-estimation of position that occurred with speed. An analysis of variance of the 29 participants’ position estimation slopes showed a main effect of displacement velocity for head-down trials (F(2,84) = 6.82, p < 0.01), but not head-up trials (F(2,84) = 1.86, p = 0.1926). For head-down trials, the most accurate updating occurred at the slowest displacement velocity of 1.2 eye-heights/sec (EP = 1.02AP + 1.54, r2 = 0.66, F(1,286) = 566.67, p « 0.0001), equivalent to a moderately paced walk by a person of average height (1.25 eye-heights/sec is about 2 meters/sec for a pedestrian 1.75 meters tall with eyes 1.6 above the ground (Cutting, Alliprandini, and Wang, 2000)). At faster displacement velocities during head-down trials, slopes increased for a speed corresponding to a fast-paced run (2.8

262 ELLMORE, MCNAUGHTON

eye-heights/sec, EP = 1.09AP + 2.43, r2 = 0.66, F(1,263) = 505.21, p < 0.01), to a speed that is out of range of naturalistic locomotion (4.4 eye-heights/sec, EP = 1.20AP + 1.96, r2 = 0.62, F(1,238) = 382.74, p < 0.01). An analysis of variance of the 29 participants’ intercepts showed no main effect of displacement velocity for head-down (F(2,84) = 0.24, p = 0.7836) or head-up trials (F(2,84) = 0.50, p = 0.6073).

A linear regression of each participant’s standard deviation of position estimation responses (i.e., random errors) onto the actual positions of the target landmarks was used to quantify the accumulation of error during position estimation. As predicted, random errors increased linearly with distance. Mean and individual regressions for participants’ random errors at increasing target distances are plotted in Figure 3 for head-up (Figure 3c) and head-down (Figure 3d) orientations at the three passive displacement velocities. An analysis of variance of the 29 participants’ random error regression slopes (rate of error accumulation with distance) showed a main effect of displacement velocity for head-down trials (F(2,84) = 11.09, p < 0.01), but not head-up trials (F(2,84) = 1.57, p = 0.2136). For head-down trials, errors accumulated linearly with distance for the slowest displacement velocity of 1.2 eye-heights/sec (F(1,263) = 30.58, p < 0.01) and the medium displacement velocity of 2.8 eye-heights/sec (F(1,256) = 36.23, p < 0.01), but not the fastest displacement velocity of 4.4 eye-heights/sec (F(1,238) = 2.34, p = 0.13).

Eye Movements During Position Estimation An analysis of vertical eye movements during position estimation trials was conducted in 8 of the 29 participants whose data was not contaminated by gross movement artifact. The eye movement traces showed a pattern of eye motion relative to the viewing screen that is consistent with a strategy of tracking the velocity of optical flow. An important difference in eye movement patterns was found between head-up (Figure 4a) and head-down (Figure 4b) position estimation trials. During head-up estimation, participants’ appeared to track, with a continual smooth pursuit eye movement, a point corresponding to the location of the previously visible target landmark. This pattern of eye movements is consistent with a strategy during which participants estimate their position relative to the previously visible target landmark by using knowledge of their eye-height and the target landmark’s angular declination relative to the horizon (Ooi, Wu, and He, 2001). During simulated movement, participants update a representation of their position relative to the target by tracking the speed of optic flow as they approach the point in space representing the target landmark. Such a strategy, which does not constitute path integration, is not possible during head-down position estimation because the horizon is not visible during simulated movement. Instead, participants must track the optical velocity of the moving ground beneath them. The eye traces during head-down estimation are consistent with this hypothesized strategy, and show short duration vertical optokinetic nystagmus accompanied by longer duration

OPTIC FLOW PATH INTEGRATION 263

Figure 3. Position estimation accuracy and error accumulation. Mean responses (colored circles) and mean of 29 individual participant regression lines (colored solid lines) for head-up (a) and head-down (b) position estimation trials at increasing target distances for three passive displacement velocities. Insets show the individual participant regression lines. The dotted diagonal black line illustrates a perfect response profile. Mean and individual regressions for participants’ random errors (SD of response error) at increasing distances and velocities are shown for (c) head-up and (d) head-down position estimation trials.

264 ELLMORE, MCNAUGHTON

Figure 4. Pattern of vertical eye movements during position estimation depends on head azimuth. Vertical eye position traces from 2 participants (top and bottom panels respectively) for head-up and head-down trials at three passive displacement velocities (red = 1.2 eye-heights/sec, blue = 2.8 eye-heights/sec, green = 4.4 eye-heights/sec). Raw traces were smoothed with a 66 ms hamming window, and gaps within a trace reflect removal of blink artifact. Deflections in a trace represent motion of the eye down the vertical axis of the screen. For head-up trials, participants’ eye movements indicated that they tended to track continuously a point representing the previously visible target landmark, suggesting that they were not necessarily path integrating in this condition. The angular declination (change in vertical visual angle) for a point at the base of the previously visible target landmark is plotted as a black dotted line for head-up trials. For head-down trials, during which the horizon is not visible, participants’ eye movements exhibited strong short-duration vertical optokinetic nystagmus accompanied by longer-duration tracking of the moving ground.

OPTIC FLOW PATH INTEGRATION 265

repetitive tracking of the ground as the participant moves closer to the target landmark. Thus, the eye-movements lend support to the idea that head-down position estimation is more characteristic of true path integration, whereas during head-up position estimation, participants may rely more on angular declination to make an estimate of position.

Orientation Estimation During orientation estimation trials, participants remained at their virtual position, and experienced passive angular rotation of their viewpoint after all distal and proximal landmark cues were removed. After passive rotation ceased, participants were instructed to reorient their view and press the joystick trigger when they thought they were directly facing the previously visible proximal landmark. For each trial, a reorientation error was computed as the angle between the participant’s viewpoint heading at the trigger press and the ideal viewpoint heading if the participant had correctly reoriented to the position of the previously viewable landmark. Circular statistics (Batschelet, 1981) were used to compute the mean vector direction m {0,360 deg} and mean vector length r {0,1} of the 19 participants who completed orientation estimation trials.

Participants accurately reoriented their view to the original heading direction after angular displacements of up to one complete rotation (360 deg) of viewpoint for head-up and head-down orientations at both slow and fast angular displacement velocities (Figures 5a-d). The modified Raleigh u test-statistic (V test (Batschelet, 1981)) indicated significant non-randomness of participant mean vector directions for head-down slow rotations (m = 0.81, r = 0.90, u = 5.55, p < 0.0001), head-down fast rotations (m = 359.81, r = 0.91, u = 4.26, p < 0.0001), head-up slow rotations (m = 13.88, r = 0.90, u = 5.24, p < 0.0001) and head-up fast rotations (m = 32.21, r = 0.94, u = 3.72, p < 0.0001). Participants’ average reorientation errors increased linearly with angular displacement up to two complete rotations of viewpoint for both head-up slow rotations (Figure 6a, red regression line, F(1,88) = 37.27, p « 0.0001) and head-up fast rotations (Figure 6a, green regression line, F(1,83) = 32.95, p « 0.0001). Errors also increased linearly for head-down slow rotations (Figure 6b, red regression line, F(1,90) = 13.28, p < 0.0004), and head-down fast rotations (Figure 6b, green regression line, F(1,84) = 17.62, p « 0.0001). Across both slow and fast rotation speeds, heading errors were significantly smaller for head-down (Figure 6a) compared to head-up (Figure 6b) trials F(1,338) = 10.03, p < 0.0017), suggesting that participants’ kept track of their orientation more accurately when more of their visual field contained coherent motion.

Discussion

While the vestibular and somatosensory contributions to human path integration have been extensively documented, the behavioral findings of the present study indicate that another sensory signal, continuous visual motion from coherent optic flowfields, may be sufficient for humans to keep track of their location and

266 ELLMORE, MCNAUGHTON

Figure 5. Heading direction after angular displacement. Mean heading after view reorientation is shown for 19 participants as colored arrows with direction and length (concentration parameter r {0,1}) components for head-up (a and b) and head-down (c and d) trials. Asterisks denote a significant non-randomness of directional responses to the expected reorientation heading of 0 degrees (modified V statistic, p < 0.01). Participants could accurately reorient their view after up to 360 degrees of angular displacement for slow (70 deg/sec) and fast (170 deg/sec) head-up and head-down trials.

OPTIC FLOW PATH INTEGRATION 267

Figure 6. Heading error after angular displacement. Mean of 19 participants' average heading error (colored circles) plotted as a function of angular displacement magnitude for two passive angular displacement velocities. The mean of the 19 individual regression lines are shown as colored lines. Heading errors increased significantly (p < 0.01) in proportion to angular displacement (up to 2 complete rotations of view) for head-up (a) and head-down (b) trials. Across speeds, errors were significantly smaller for head-down compared to head-up trials (p < 0.01). orientation. When scaling by eye-height in the virtual environment is related to the real world, position updating was found to be most accurate at a displacement velocity that resembles natural locomotion. Consistent with characteristics of real-world mammalian path integration, participants’ estimation error variance increased linearly with distance during both location and orientation estimation. The eye-movement patterns for different displacement velocities and head-orientations are consistent with a strategy of tracking the speed of visual motion, and most importantly, the strategy used during head-down position estimation resembles true path integration in that the target landmark’s angular declination relative to the horizon may not be used during any part of the target trajectory to estimate position.

Previous studies of distance estimation by optic flow did not characterize or control for possibly different eye movement strategies, and may be confounded by the use of simulated head-orientations that are oriented toward the horizon and parallel to the ground plane. The results of one study of distance discrimination from visual motion suggest no difference in humans’ ability to discriminate travel distances when the simulated viewing angle is changed (Frenz et al., 2003). A change in the angle of simulated gaze affects the mean velocity of the optic flowfield. Yet, humans may be able to compensate for this

268 ELLMORE, MCNAUGHTON

global velocity change when discriminating travel distances from visual motion. Only two studies have examined the effect of altered head azimuth on active locomotion in the real world without visual feedback. In one study, humans walked to a previously seen target at a distance of 10 m at one of five different velocities (M. L. Mittelstaedt and Mittelstaedt, 2001). For each of the velocities, subjects had to walk with five different head positions: 1) head normal (parallel to the ground, looking straight ahead), 2) head moved continuously from left to right, 3) head moved continuously up and down, 4) head tilted down, and 5) head tilted back. During the last four conditions, signals from the inertial sense organs differed from normal conditions because of the altered planes of the otoliths and canals. Path length errors depended on the walking velocity for all head positions, but the locomotor pattern was not influenced by head position. In contrast to this result, data from a study of humans tested with a head-down tilt imposed during walking to a previously seen target without vision show an increase in mean absolute error of about 20% compared to walks with normal head position (Clark and Sidaway, 1999). However, no difference in the variance of errors was found, and signed errors were not reported. It appears that, in the real world, head position has no significant influence on path length estimation during active locomotion. One reason may be that central processing of vestibular signals can deal with different head positions and head movements through a coordinate transformation of head-referenced to trunk-referenced signals (Mittelstaedt, 1997; H. Mittelstaedt and Mittelstaedt, 1991). In the present study, during the beginning portions of some long-distance head-up trajectories, participants may use a strategy of path integration when there is no, or only a small, barely noticeable angular declination change in the previously visible target location; as the trajectory continues, and the rate of angular declination of the previously visible target increases, participants’ may switch to a strategy of tracking this increasingly noticeable change with a smooth pursuit eye movement to estimation position. During head-down position estimation, participants’ responses are consistent with a velocity-dependent, error-accumulating path integration strategy. The rate of error accumulation as a function of distance showed a main effect of displacement velocity for head-down trials, but not head-up trials. In humans, the error variance of both the production and perception of temporal intervals is linearly related to the duration of the time interval being estimated (Ivry and Hazeltine, 1995). Latency, as well as distance, errors also increase linearly during human inertial linear path integration (Israel et al., 1993). In the present study, it is possible that errors accumulated less in proportion to distance during the fastest head-down position estimation trials because, when participants’ responded that they had arrived at the target landmark, less time had passed relative to the slow and medium passive velocity trials for participants’ random errors to sum. Errors in path integration may result from errors in the estimation of both velocity and elapsed time.

Gibson’s ‘optic flow’ is a construct that presumes visual information about self-motion is recovered irrespective of one’s eye movements. Consequently,

OPTIC FLOW PATH INTEGRATION 269

one may argue that, in order to claim that subjects in the present study made their responses on the basis of optic flow alone, one would need to exclude the possibility that subjects responded on the basis of some integrated measure of other sensory signals or motor effort. While the experimental design of the present study controlled for the potential contributions by vestibular signals and walking effort, the subjects were free to make eye movements. The eye movement traces in Figure 4 show abundant nystagmus. Thus, the signals representing the speed and duration of these eye movements may have contributed to the participants’ judgment of self-displacement.

In summary, while humans can navigate to hidden spatial locations using the visual relationship among distal landmarks (Jacobs, Laurance, and Thomas, 1997; Jacobs, Thomas, Laurance, and Nadel, 1998) in a manner consistent with cognitive mapping theory (O'Keefe and Nadel, 1978), the present study shows that humans can keep track of their position and orientation with respect to previously visited locations using only optic flowfields, in a manner consistent with the properties of real-world path integration. These findings, together with the observation that non-human primates can navigate using exclusively visual cues (Towers, Ellmore, and McNaughton, 2003), suggest that path integration may be investigated in the context of virtual navigation during which the only self-motion signals are the speed and direction of optic flow. The demonstration that coherent optic flowfields are sufficient to keep track of position and bearing, and that location specific visual information is unnecessary, establishes a behavioral paradigm for examining the neural mechanisms and circuits underlying path integration using the techniques of non-invasive human functional imaging, whole-brain characterization of activity-dependent immediate early gene expression, and neurophysiological recording in non-human primates. The neural mechanisms underlying mammalian path integration have only begun to be characterized. It appears that, in the rodent, both external cues and path integration interact to control the updating of the hippocampal ensemble representation for place (Gothard, Skaggs, and McNaughton, 1996). In the absence of visual input, cells encoding head direction and place maintain their firing characteristics based on self-motion, and update, apart from random drift error, as if environmental cues were still visible (McNaughton et al., 1996). In the primate, cortical neurons in dorsal MST appear to encode both place and path (Froehler and Duffy, 2002). Other experiments reveal cells in primate pre-subiculum that code for head direction (Robertson, Rolls, Georges-Francois, and Panzeri, 1999), and cells in parietal cortex that compute coordinate transformations of retinal location to head-centered and body-centered space (Andersen, 1985; Snyder, Grieve, Brotchie, and Andersen, 1998). More experiments are needed to understand fully the neural systems involved in the storage and integration of movements for flexible navigation and exploration of familiar and novel environments.

270 ELLMORE, MCNAUGHTON

Acknowledgments

We thank Erin Lindstedt for assistance with data collection and analysis, and Kalairaja Chinnaveerappan for help with eye-tracker calibration. This work was supported by PHS Grant NS20331 and by a grant from Japan Science and Technology Corporation-Core Research for Evolutional Science and Technology.

References

Andersen, R. (1985). Encoding of spatial location by posterior parietal neurons. Science, 230, 456–458.

Bakker, N., Werkhoven, P., & Passenier, P. (1999). The effects of proprioceptive and visual feedback on geographical orientation in virtual environments. Presence: Teleoperators and Virtual Environments, 8, 36–53.

Batschelet, E. (1981). Circular statistics in biology. New York: Academic Press. Becker, W., Raab, S., & Jurgens, R. (2002). Circular vection during voluntary

suppression of optokinetic reflex. Experimental Brain Research, 144, 554–557.

Benhamou, S., Sauve, J.-P., & Bovet, P. (1990). Spatial memory in large scale movements: Efficiency and limitation of the egocentric coding process. Journal of Theoretical Biology, 145, 1–12.

Berthoz, A., Israel, I., Georges-Francois, P., Grasso, R., & Tsuzuku, T. (1995). Spatial memory of body linear displacement: What is being stored? Science, 269, 95–98.

Berthoz, A., Pavard, B., & Young, L. R. (1975). Perception of linear horizontal self-motion by peripheral vision (linear vection). Basic characteristics and visual-vestibular interactions. Experimental Brain Research, 23, 471–489.

Bles, W., De Jong, J., & De Wit, G. (1984). Somatosensory compensation for loss of labyrinthine function. Acta otolaryngol, 97, 213–221.

Bremmer, F., & Lappe, M. (1999). The use of optical velocities for distance discrimination and reproduction during visually simulated self motion. Experimental Brain Research,, 127, 33–42.

Clark, R., & Sidaway, B. (1999). The effect of altered gait and head position on goal directed walking without vision. Journal of Human Movement Studies, 37, 107–126.

Cutting, J. E., Alliprandini, P. M., & Wang, R. F. (2000). Seeking one's heading through eye movements. Psychon Bull Rev, 7, 490–498.

Darwin, C. (1873). Origin of certain instincts. Nature, 7, 417–418. Etienne, A. S., Maurer, R., & Saucy, F. (1988). Limitations in the assesment of

path dependent information. Behaviour, 106.5, 81–111. Frenz, H., Bremmer, F., & Lappe, M. (2003). Discrimination of travel distances

from situated optic flow. Vision Res, 43, 2173–2183. Froehler, M. T., & Duffy, C. J. (2002). Cortical neurons encoding path and

place: Where you go is where you are. Science, 295, 2462–2465.

OPTIC FLOW PATH INTEGRATION 271

Gibson, J. J. (1950). The perception of the visual world. Boston: Houghton Mifflin.

Gothard, K. M., Skaggs, W. E., & McNaughton, B. L. (1996). Dynamics of mismatch correction in the hippocampal ensemble code for space: interaction between path integration and environmental cues. Journal of Neuroscience, 16, 8027–8040.

Israel, I., & Berthoz, A. (1989). Contribution of the otoliths to the calculation of linear displacement. Journal of Neurophysiology, 62, 247–263.

Israel, I., Chapuis, N., Glasauer, S., Charade, O., & Berthoz, A. (1993). Estimation of passive horizontal linear whole-body displacement in humans. Journal of Neurophysiology, 70, 1270–1273.

Israel, I., Sievering, D., & Koenig, E. (1995). Self-rotation estimate about the vertical axis. Acta Otolaryngol, 115, 3–8.

Ivry, R., & Hazeltine, R. (1995). Perception and production of temporal intervals across a range of durations: Evidence for a common timing mechanism. Journal of Experimental Psychology: Human Perception and Performance, 21, 3–18.

Jacobs, W., Laurance, H., & Thomas, K. (1997). Place learning in virtual space I: Acquisition, overshadowing, and transfer. Learning and Motivation, 28, 521–541.

Jacobs, W., Thomas, K., Laurance, H., & Nadel, L. (1998). Place learning in virtual space: Topological relations as one dimension of stimulus control. Learning and Motivation, 29, 288–308.

Jurgens, R., Nasios, G., & Becker, W. (2003). Vestibular, optokinetic, and cognitive contribution to the guidance of passive self-rotation toward instructed targets. Experimental Brain Research, 151, 90–107.

Kearns, M. J., Warren, W. H., Duchon, A. P., & Tarr, M. J. (2002). Path integration from optic flow and body senses in a homing task. Perception, 31, 349–374.

Mallot, H. A., & Gillner, S. (2000). Route navigating without place recognition: What is recognised in recognition-triggered responses? Perception, 29, 43–55.

McNaughton, B., Barnes, C., Gerrard, J., Gothard, K., Jung, M., Knierim, J., et al. (1996). Deciphering the hippocampal polyglot: The hippocampus as a path integration system. Journal of Experimental Biology, 199, 173–185.

Metcalfe, T., & Gresty, M. (1992). Self-controlled reorienting movements in response to rotational displacements in normal subjects and patients with labyrinthine disease. Annals of the New York Academy of Sciences, 656, 695–698.

Mittelstaedt, H. (1997). Interaction of eye-, head-, and trunk-bound information in spatial perception and control. J Vestib Res, 7, 283–302.

Mittelstaedt, H., & Mittelstaedt, M.-L. (1973). Mechanismen der orientierung ohne richtende Aussenreize. Fortschritte der Zoologie, 21, 46–58.

272 ELLMORE, MCNAUGHTON

Mittelstaedt, H., & Mittelstaedt, M.-L. (1982). Homing by path integration. In H. Papi & G. Wallraff (Eds.), Avian Navigation (pp. 290–297). Heidelberg-Berlin: Springer-Verlag.

Mittelstaedt, H., & Mittelstaedt, M.-L. (1991). From interdepdendent to independent control of head and trunk. In A. Berthoz, P. P. Vidal & W. Graf (Eds.), The Head-Neck Sensory Motor System (pp. 369–373). New York: Oxford University Press.

Mittelstaedt, M.-L. (1995). Influence of centrifugal force on angular velocity estimation. Acta Otolaryngol, 520, 307–309.

Mittelstaedt, M.-L., & Glasauer, S. (1991). Idiothetic navigation in gerbils and humans. Zool. Jb. Physiol., 95, 427–436.

Mittelstaedt, M.-L., & Mittelstaedt, H. (1980). Homing by path integration in a mammal. Naturwisseschaften, 67, 566–567.

Mittelstaedt, M. L., & Mittelstaedt, H. (2001). Idiothetic navigation in humans: Estimation of path length. Experimental Brain Research, 139, 318–332.

O'Keefe, J., & Nadel, L. (1978). The hippocampus as a cognitive map.Oxford: Oxford University Press.

Ooi, T. L., Wu, B., & He, Z. J. (2001). Distance determined by the angular declination below the horizon. Nature, 414, 197–200.

Peruch, P., May, M., & Wartenberg, F. (1997). Homing in virtual environments: Effects of field of view and path layout. Perception, 26, 301–311.

Probst, T., Krafczyk, S., Brandt, T., & Wist, E. R. (1984). Interaction between perceived self-motion and object-motion impairs vehicle guidance. Science, 225, 536–538.

Redlick, F. P., Jenkin, M., & Harris, L. R. (2001). Humans can use optic flow to estimate distance of travel. Vision Res, 41, 213–219.

Riecke, B., van Veen, H., & Bulthoff, H. (2002). Visual homing is possible without landmarks: A path integration study in virtual reality. Presence: Teleoperators and Virtual Environments, 11, 443–473.

Robertson, R. G., Rolls, E. T., Georges-Francois, P., & Panzeri, S. (1999). Head direction cells in the primate pre-subiculum. Hippocampus, 9, 206–219.

Royden, C. S., Banks, M. S., & Crowell, J. A. (1992). The perception of heading during eye movements. Nature, 360, 583–585.

Snyder, L. H., Grieve, K. L., Brotchie, P., & Andersen, R. A. (1998). Separate body- and world-referenced representations of visual space in parietal cortex. Nature, 394, 887–891.

Towers, D., Ellmore, T., & McNaughton, B. L. (2003). Spatial navigation by rhesus monkeys in a virtual environment by exclusively visual cues. Paper presented at the Society for Neuroscience, New Orleans, LA.

Warren, W. H., & Hannon, D. J. (1988). Direction of self motion is perceived from optical flow. Nature, 336, 162–163.

Warren, W. H., Kay, B. A., Zosh, W. D., Duchon, A. P., & Sahuc, S. (2001). Optic flow is used to control human walking. Nature Neuroscience, 4, 213–216.