What visual image information is needed for the things we do?

How is vision used to acquire information from the world?

Types of Eye Movement

Information Gathering StabilizingVoluntary (attention) Reflexive

Saccades vestibular ocular reflex (vor)new location, high velocity, ballistic body movements

Smooth pursuit optokinetic nystagmus (okn)object moves, velocity, slow whole field image motion

Vergencechange point of fixation in depthslow, disjunctive (eyes rotate in opposite directions)(all others are conjunctive)

Fixation: period when eye is relatively stationary between saccades.

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Catching: Gaze Patterns

CatcherThrower

saccade X

X

smooth pursuit

X

Terminology: saccadic eye movement

Catching: Gaze Anticipation

CatcherThrower

X

X X61 ms

-53 ms

Timing of departure and arrival linked to critical events

Saccade reaction time = 200ms

What is the significance of prediction?

Brain must learn the way ball moves etc and programmovement for an expected state of world. Not reacting simply to current visual information. Stimulus Response

What is meant by an “internal model”?

Photoreceptors ganglion cells LGN

Primary visual cortex other cortical areas

mid-brain brain stem muscles

Why is prediction necessary?

Analysis of visual signals takes a lot of time!

Round trip from eye to brain to muscles takes a minumumof 200 msec. Cricket ball only takes about 600 msec.Prediction gets around the problem of sensory delays.

20 deg

Accuracy of Fixations near Bounce

2D elevation

Subjects fixate above the bounce point

bounce point

How good is Internal Model?

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Poor tracking when ball is unexpectedly bouncy

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Better tracking 2 trials later.

Pursuit accuracy following bounce

0%10%20%30%40%50%60%70%80%90%

100%

1 2 3 4 5 6Trial Number

%age of time gaze on Target

tennis ball

bouncy ball

Does pursuit accuracy improve with repeated trials?Does it matter which ball is used first? What can we conclude if it does?

5 subjects

Measure proportion of time between bounce & catch that eye is close to ball

Prediction in Squash

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aH.263 decompressorare needed to see this picture.

Prediction in Squash

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Time between fixation on front wall and bounce on front wall

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

latency (seconds)

Player 1Player 2

Error between fixation and bounce-point

0

2

4

6

8

10

12

14

16

Error (degrees of visual angle)

Player 1Player 2

Percentage Pursuit from Bounce off Wall to Bounce on Floor

0

10

20

30

40

50

60

70

80

Pursuit (%)

Maximum speed for perfect pursuit and for reduced gain pursuit

0

50

100

150

200

250

300

velocity (degrees/second/second)

Player 1Player 2

These speeds are much higher than expected. Too high for a reactive system.That is, prediction is necessary. 100deg/sec = 10 deg in 100 msec.

“Reduced gain” means eye lags behind ball. “Gain” = 1 means perfect tracking

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Macular degeneration patient (blind in central 15 deg field)Fovea not necessary for smooth pursuit - can learn to use other regions

Binocular Vision

Stereoscopic information: image in the two eyes is different.This information is used to perceive the depth relations inthe scene.

When is stereoscopic information useful? - reaching and grasping- walking over obstacles- catching??

Development of stereoscopic vision - amblyopia/ astigmatism- critical period

Difference in retinal distance between the objects in the two eyes is called “retinal disparity” and is used to calculate relative depth.

Binocular Vision

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Monocular Vision

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Other information that may be useful for catching.

Motion parallax: change in relative position of objects at different depths when the head moves.

Looming: image of ball increases in size as ball gets closer. Rate of change of size can be used to calculate “time-to-contact”

Pursuit movement: keeping the eye on the ball.

target selection

signals to muscles

inhibits SC

saccade decision

saccade command

CerebellumLearning?

Planning?

signals to muscles

detect/analyzeretinal image motion

decision to pursue/attention

prediction/learning?

Supplementary eye fieldsplanning?

Even the simplest action must involve linkage between memory, vision, eye movements, and body movements.from Land et al, 1999

1. What are the questions?• Is the behavior observed by Land in cricket also true for a simple task like

catching a ball?• What eye movements are made in this case?• Do subjects anticipate the bounce point? By how much? • Do Subjects look at floor or above the bounce point?• What happens after bounce? • How do subjects adjust to different balls?• …..• Is there a difference between throwing and catching? Why?• What eye movements are made when observing others throw and catch?• Similarity between individuals? • When do the hands start to move?

• 2. Choice of task:• Catching and throwing a ball.

• 3. Procedure:• Select subject and calibrate eye tracker. Three people stand at equal distances

apart and throw the ball back and forth, with a bounce in the trajectory. Need to measure this distance.

• First throw in a predictable manner, about 8 times.• Then use a different ball, 8 trials.• Other balls…• Compare one versus two eyes???

2. Data analysis• Label your tape. Play it frame-by-frame on the VCR in the lab. • ….

• What to look for:– Describe eye movements sequence for each trial

• eg Trial 1: fixate near hands/saccade to bounce point/fixate/track portion of trajectory/fixate for last part of trajectory (??)

• Trial 2: fixate near hands/saccade to bounce point/fixate/track portion of trajectory/fixate for last part of trajectory (??)

• ….• B How regular is the sequence of movements?• C What is the timing of the saccades/fixations/tracking relative to

movement of the ball. How much do subjects anticipate the bounce point, if at all?

• D. How accurate are fixations near the bounce point? (Need to measure visual angle.)

– Compare different conditions.– What happens with the different balls? Do the eye movements change

with additional experience? How quickly do they adjust?

• Other Aspects:

– How similar are different individuals? Where would we expect similarities/ differences?

• What is the role of the pursuit movement? If pursuit is made only on final bounce, implies pursuit is used to guide hands. Maybe position of eye in head.

• Binocular information versus monocular (looming)

QuickTime™ and aYUV420 codec decompressor

are needed to see this picture.

Different gaze pattern for watchingbut still anticipate bounce and catch events.

CatcherThrower

saccade

X

X

Gaze Patterns Different when Watching

X

CatcherThrower

X

X

Watching:Gaze Anticipation

X-51 ms

-167 ms

-517 ms

Head rotation begins 200-500 msec before release