Why do we move our eyes?

- Image stabilization in the presence of body movements.

- Information acquisition - bring objects of interest onto high acuity region in fovea.

Retinal structure

Cone Photoreceptors are densely packed in the central fovea

Visual Acuity matches photoreceptor density

Oculomotor Muscles

Muscles innervated by oculomotor, trochlear, and abducens (cranial) nerves from the oculomotor nuclei in the brainstem. Oculo-motor neurons: 100-600Hz vs spinal motorNeurons: 50-100Hz

Types of Eye Movement

Information Gathering StabilizingVoluntary (attention) Reflexive

Saccades vestibular ocular reflex (vor)new location, high velocity (700 deg/sec), body movementsballistic(?)

Smooth pursuit optokinetic nystagmus (okn)object moves, velocity, slow(ish) – typically whole field image motionup to 35 deg/sec

Vergencechange point of fixation in depthslow, disjunctive (eyes rotate in opposite directions)

(all others are conjunctive)Note: link between accommodation and vergenceFixation: period when eye is relatively stationary between saccades.

AccelerationDepth-dept gain, Precision in natural vision

Velocity

Acuity – babies

https://www.youtube.com/watch?v=KSJksSA6Q-A

Latency of vestibular-ocular reflex=10msec

It is almost impossible to hold the eyes still.

Demonstration of VOR and its precision – sitting vs standing

Miniature eye movements

Slow driftMicro-saccadestremor

Saccade latency approx 200 msec, pursuit approx 100 – smaller when there is a context thatallows prediction.

Step-ramp allows separation of pursuit (slip) and saccade (displacement)

“main sequence”: duration = c Amplitude + bMin saccade duration approx 25 msec, max approx 200msec

Factors That Control Gaze.

- TASK Defines behavioral goals, what information is relevant.

- REWARDS Oculomotor circuitry sensitive to reward/subjective value of those goals.

- UNCERTAINTY Get information. Peripheral resolution/ working memory REDUCTION decay etc

- PRIORS/ Memory Gaze targeting reflects stored knowledge.

- IMAGE Salient properties eg high contrast/ spatial outliers

1. Neural activity related to saccade

2. Microstimulation generates saccade

3. Lesions impair saccade

Brain Circuitry for Saccades

Brain Circuitry for Pursuit

Eye Tracking Methods

Developments in Eye Tracking

Head fixed /restricted: Contact lenses: mirror / magnetic coils Early infra-red systems Dual Purkinje Image tracker

Head Free: Head mounted IR video-based systems Remote systems with head tracking

Scene camera

Difficulty: optical power of eye + observer movement

Why eye movements are hard to measure.

18mm

0.3mm = 1 deg visual angle

x a

tan(a/2) = x/da = 2 tan

-1 x/d

Visual Angle

d

1 diopter = 1/focal length in meters

55 diopters = 1/.018

A small eye rotation translates into a big change in visual angle

Early Methods:

“Barlow photographed a droplet of mercury

placed on the limbus. Translations of the head

were minimized by having subjects lie on a

stone slab with their heads wedged tightly

inside a rigid iron frame”

Kowler, 1990

Measuring Eye Movements

Early methods:

“The eye is first cocainized, then the lids should

be propped apart by some form of eye-lid

fastener, of which the best is probably that in

form of a wide-opening spring with tortoise-

shell grooves for the lids.”

Delabarre,

1898

Monitoring Eye Movements; YarbusMirror mounted on eye using suction. Light bounces off mirror and is recorded on film

• Non image-based eye trackers

– Electrical/analog

– Limbus

– Magnetic search coil

Non image-based Eye Trackers

EOG

EOG

The eye is a ‘dipole’ with ~millivolts voltage difference between the retina and the cornea.

ElectroOculoGram (EOG)

Use in clinic – head not fixed

By monitoring the ‘whites of the eye’ below the iris, it is possible to determine eye position.

Vertical eye movements cause both signals to increase (up) or decrease (down).

Horizontal eye movements cause differential illumination between the right and left sensors.

Limbus Trackers

Limbus Trackers

Limbus

EOG and Limbus trackers

Good temporal resolution.

Lousy spatial resolution

High noise, drift

Mostly useful in clinic

Skalar search coils

Magnetic Search CoilsUsed for much animal work, though less so recently. Very high precision andaccuracy (few minutes of arc). Used in older human em literature.Can use similar methodology for head and hand (see Hayhoe lab)

• Image-based eye trackers

– Dual Purkinje

– Video based

Image-based Eye Trackers

Dual Purkinje Trackers

The ‘gold standard’ in eye trackers

Multiple reflections from the cornea and lens vary in a very well-defined way as the eye moves. By tracking the 1st and 4th reflections, the tracker can determine eye position with very high precision.Bill Geisler lab has a binocular tracker.

Dual Purkinje Trackers

Precision: < 1’ (~1/100 deg)Accuracy: a few min arcUpdate rate: > 500 Hz

Dual Purkinje Trackers

Usually requires bite bar but theoretically can get away with head rest.

• Video-based eye trackers:

–Head mounted

–Remote

Video-based Eye Trackers

Head mountedCamera on head views scene, another camera views eye.

Video-based Eye TrackersInfra-red video camera finds center of pupil and corneal reflection.Advantages: unconstrained viewing.Disadvantages: temporal resolution may be as low as 30 HzAccuracy never better than 0.5 deg.

RIT Wearable Eyetracker

RIT Wearable Eyetracker

Build-up neurons in the intermediate layers of the SC are activeprior to a saccade.

Cell in the superifical layers get input from the retina. This may mediateVery fast saccades – sometimes called “express saccades”

Extent of buildup neuron activity reflects stimulus probability.

Express saccades might also reflect activity in buildup neurons.

LIP: Lateral Intra-parietal AreaTarget selection for saccades: cells fire before saccade to attended object

Posterior Parietal Cortex

reaching

grasping

Intra-Parietal Sulcus: areaof multi-sensory convergence

Visual stability

Model of saccade generation: target selection depends on expected value

Trommershauser, Glimcher, Gegenfurtner, 2009

Area LIP contains a reward expectation signal which modulates the gain of visual neurons in LIP.

Reward modulation of saccadic eye movements originates from dopaminergic input to caudate nucleus.

Relation between saccades and attention.

Saccade is always preceded by an attentional shiftHowever, attention can be allocated covertly to the peripheral retina without a saccade.

Pursuit movements also require attention.

Figure 8.18 The comparator

Visual Stability

A cross seen through an aperture that moves clockwise around the boundary. Alternatively, the aperture may be stationary, and the cross move behind it. Individual views, shown on the right, are ambiguous.

Observers have no trouble with this if they have an “internal model” or schema that readily allows interpretation of the sequence.

-Saccades/Smooth Pursuit

-Planning/ Error Checking

-relates to behavioral goals

Supplementary eye fields

A subset of SEF neurons and LFPs exhibited strong modulation following erroneous saccades to adistractor. Altogether, these results suggest that SEF plays a limited role in controlling ongoing visual search behavior, but may play a larger role in monitoring search performance.

Nearby Anterior Cingulate also involved in performance monitoring.

Motor neurons for the eye muscles are located in the oculomotor nucleus (III), trochlear nucleus (IV), and abducens nucleus (VI), and reach the extraocular muscles via the corresponding nerves (n. III, n. IV, n. VI).Premotor neurons for controlling eye movements are located in the paramedian pontine reticular formation(PPRF), the mesencephalic reticular formation (MRF), rostral interstitial nucleus of the medial longitudinal fasciculus (riMLF), the interstitial nucleus of Cajal (IC), the vestibular nuclei (VN), and the nucleus prepositus hypoglossi (NPH).

Motor neurons

Pre-motor neurons

Oculomotor nucleus

Trochlear

Abducens

H

V

Pulse-Step signal for a saccade

Brainstem circuits for saccades. Omnipause neurons (OPN) in the nucleus raphe interpositus (RIP) tonically inhibit excitatory burst neurons (EBN) located in the paramedian pontine reticular formation (PPRF). When OPNs pause, the EBNs emit a burst of spikes, which activate motor neurons (MN) in the abducens nucleus (VI) innervating the lateral rectus muscle. The burst also activates interneurons (IN) which activate motor neurons on the oculomotor nucleus (III) on the opposite side, innervating the medial rectus. Inhibitory burst neurons (IBN) show a pattern of activity similar to EBNs, but provide inhibitory inputs to decrease activation in the complementary circuits and antagonist muscles. Long-lead burst neurons (LLBN) showactivity long before movement onset, and provide an excitatory input to EBNs.

Brain areas involved in making a saccadic eye movementBehavioral goal: make a sandwich (learn how to make sandwiches) Frontal cortex.

Sub-goal: get peanut butter (secondary reward signal - dopamine - basal ganglia)

Visual search for pb: requires memory for eg color of pb or location (memory for visual properties - Inferotemporal cortex; activation of color - V1, V4)

Visual search provides saccade goal. LIP - target selection, also FEF

Plan saccade - FEF, SEF

Coordinate with hands/head

Execute saccade/ control time of execution: basal ganglia (substantia nigra pars reticulata, caudate)

Calculate velocity/position signal oculomotor nuclei

Cerebellum?

RF reticular formation VN vestibular nuclePN , pontine nucleii

CerebellumOV oculomotor vermis VPF ventral paraflocculus FN fastigial nucleus

otoliths

Rotational (semi-circular canals) translational (otoliths)

target selection

signals to muscles(forces)

inhibits SC

saccade decision

saccade command(where to go)

monitor/plan movements

Function of Different Areas

H

V

Smooth pursuit& Supplementary

Brain Circuitry for Pursuit

Velocity signal

Early motion analysis

Eye Movement Research

Consequences of image motion on visual acuity: stabilized images

Metrics of saccades/ pursuit/ vergence/vor

Constancy of visual direction

Eye movements in reading/ Cognitive role of eye movements

Active Vision/Natural tasks: Fixation patterns, eye/head/hand coordination

Language comprehension in visual context

Paradigm Differences

Natural Tasks: Study small segments of behavior Multiple visual operations: transitions between operations S in control of agenda

complexity of scene

Standard approach: Repeated observations of a small time slice Single visual operation or movement

Limited complexity

Remote

Bright pupil Dark pupil

Video-based Eye Trackers

Video-based Eye Trackers

Dark pupil

Bright pupil

Bright-pupil; coaxial illumination

Two signals; pupil and corneal reflection - fairly robustto tracker movement wrt head.Lower temporal and spatial resolution than eg coils/DPI1 deg, 60-120 Hz

Embed eye image in video record to monitor quality of track.