Stage 3 Project

Is there a difference in the pattern of eye movements during visual imagery, when spatial or object information is being recalled from

memory?

James Allen

Student number: 011306821

Word Count: 7918

1

Contents Page

Abstract 3

1. Introduction 4

2. Methods 102.1 Outline 10

2.2 Subjects 11

2.3 Stimuli 11

2.4 Apparatus 12

2.5 Procedure 12

2.6 Visual working memory test 14

2.7 Data analysis 14

3. Results 163.1 Scan paths 16

3.2 Percentage of fixation duration in correct quadrant 18

3.3 Number of fixations in correct quadrant 21

3.4 Total fixation duration of all subjects in all quadrants 25

3.5 Mean duration of fixations in each condition 26

3.6 Comparison of 'what/where trials first' subjects and 27

'imagine trials first' subjects

3.7 Incorrect Responses 29

3.8 Visual Working memory score 29

3.9: Subject reports 31

4. Discussion 32

Appendix 1 36

Appendix 2 37

Appendix 3 38

Appendix 4 39

Appendix 5 42

Appendix 6 43

Appendix 7 44

Appendix 8 45

Appendix 9 46

Appendix 10 47

5. References 48

2

Abstract

Patterns of eye movements during the recall of visual spatial information, and visual object

information, were investigated. Since the visual system appears to operate with two broadly separate

systems for processing spatial and object information (the ‘what and where’ pathways), it was

hypothesised that the recall of these two types of information may interact with the ocular system in

different ways. This may be reflected in a difference between whether or not eye movements are made

towards the spatial location of the shape being recalled (as it was initially stored). To test this, 20

subjects were presented with an array of 4 different geometrical shapes to memorise, on a computer

monitor. Eye movements in response to being asked to recall spatial information (e.g. “where was the

triangle?”) and object information (e.g. “what colour was the square?”) were recorded for 5 seconds per

trial (60 trials per subjects). When ‘where’ information about a shape was recalled, subjects fixated for

significantly longer in the quadrant of the screen that previously displayed the shape (t=4.35 and

t=3.60, p<0.01), and made significantly more fixations in that quadrant (t=4.55 and t=3.52, p<0.01).

This effect did not occur when ‘what’ information was recalled. Subject’s visual working memory was

controlled for, and did not interact with eye movements. The implications are discussed in terms of the

interaction of visual imagery with eye movements.

3

1. Introduction

There is a large body of experimental evidence that indicates the visual system

can be dissociated into two broadly separate pathways (see Livingstone & Hubel,

1987 for a review). These pathways can be seen to segregate at the level of ganglion

cells of the retina, and continue through the primary visual cortex, to higher visual

areas in the brain. The magnocellular pathway appears to be sensitive to high

temporal and low spatial frequencies, and appears to be best suited for processing

movement and spatial information, hence it is commonly referred to as the ‘where’

pathway. The parvocellular pathway is sensitive to low temporal and high spatial

frequencies, and appears to be suited for processing information about form and

colour, and is often referred to as the ‘what’ pathway.

Evidence for this distinction comes from many sources. Neurophysiological

studies of the primate visual cortex have provided evidence for an inferior temporal

(ventral) pathway selective for processing object information, but not for location, and

a posterior parietal (dorsal) pathway selective for spatial processing but not for form

or colour (for a review, see Van Essen, Anderson & Felleman, 1992). This was

investigated, in one study, using micro-electrodes to record action potentials from the

macaque lateral geniculate nucleus, while macaques were presented with a series of

spatial sinusoidal grating patterns (Derrington & Lennie, 1984).

Neurological damage in primates and humans produces similar results. For

example, Isseroff et el. (1982) found that lesions to the mediodorsal nucleus in rhesus

monkeys leads to an impairment on two spatial memory tasks (spatial delayed

alternation task, and a delayed-response task), whereas performance on two object

discrimination tasks (object reversal and visual pattern discrimination) was

unimpaired. Similarly, Mishkin & Manning (1978) found that lesions to the inferior

prefrontal cortex in monkeys led to a severe impairment on three object memory tasks

(delayed object alternation, delayed object matching, and delayed colour matching),

whereas lesions to the principal sulcus led to severe impairment on spatial memory

tasks. The same phenomenon is observed in humans – Bohbot et al. (1998) found that

patients with lesions to the right hippocampus (made to relieve epilepsy) selectively

impaired spatial memory performance.

4

Neuroimaging has also produced similar conclusions. PET studies have

revealed a differentiation in activation of brain areas during object tasks and spatial

tasks, which correspond to the same pathways. For example, Haxby et al. (1991)

studied regional cerebral blood flow in 11 normal subjects, while they conducted two

visual match-to-sample tasks. A face-matching task was used to test object vision, and

a dot-locating task was used to test for spatial vision. The two tasks appeared to

activate consistently different areas of the brain.

The fact that there appears to be a strong distinction between the ‘what’ and

‘where’ pathways in the visual system has prompted researchers to propose a similar

distinction in visual memory. Experimental evidence appears to support this

proposition.

For example, Tresch, Sinnamon & Seamon (1993) studied the relative

interference that a secondary task has on the performance of a primary task, when

both are carried out simultaneously by a subject. The two tasks studied were a spatial

memory task (subjects were asked to remember the location of a dot in a spatial

memory test), and an object memory task (remembering the form or colour of an

object). Spatial memory was found to be selectively impaired by a movement

discrimination spatial task, whereas object memory was selectively impaired by a

colour discrimination object task. They conclude that this is evidence for a

dissociation between a ‘what’ visual memory and a ‘where’ visual memory (Tresch et

al., 1993).

PET studies reveal differing activation of brain regions during working

memory tasks involving spatial or object tasks. Smith & Jonides (1997) used simple

memory tasks, in which subject were presented with an array of randomly-located

dots (a spatial memory task) or a pair of objects (an object memory task) After a 3

second retention period, subjects were required to recall the position of one of the

dots, or the shape of an object. The object task generally activated left hemisphere

regions (the posterior parietal region and the premotor region in the left hemisphere),

and the spatial task activated right-hemisphere regions (the posterior parietal cortex

and the anterior occipital cortex in the right hemisphere) (Smith and Jonides, 1997).

Ruchkin, Johnson, Grafman, Canoune & Ritter (1997) recorded ERP scalp

topographies whilst subjects retained and recalled object information (line drawings

of abstract faces) and spatial information (the location of a moving asterisk with

5

respect to a cross), and also found different underlying patterns of brain activation.

Mecklinger & Pfeifer (1996) also demonstrated this, by recording ERP topographies

whilst subjects conducted two versions of a delayed match-to-sample task, one using

object forms and the other using 2-dimensional spatial configurations.

Further evidence has been found in case-studies of neurologically damaged

patients. Levine, Warach & Farah (1985) presented two such patients, one of whom

showed selective damage to object memory, and the other spatial memory. ‘Patient 1’

had extreme difficulty recognising faces (including his wife), animals and

occasionally common objects, and equal difficulty in describing imagined faces,

animals and colours of objects, or drawing common items (such as a clock or an

elephant) from memory. Yet he had no difficulty in approaching or reaching for

objects, copying the arrangement of lines from a picture, describing a detailed mental

journey from one place to another, drawing a detailed plan of his house or marking

major cities on a map. This pattern of selective deficits appears to demonstrate a

dissociation of Patient 1’s ‘what’ memory, which is severely impaired, and his

‘where’ memory, which is intact.

Patient 2 showed the reverse pattern of deficits – he could identify objects,

faces and animals easily, but could not reach accurately for objects, draw a straight

line between two points and had difficulty describing the spatial relationship between

two objects presented to him. He also got lost frequently, and could not describe the

route from his home to his local shops.

Farah, Hammond, Levine & Calvanio (1988) also present a patient (L.H.) who

showed similar deficits to Patient 1 described above. Using four visual imagery tasks

(involving the recall of object shape, size, colour) and seven spatial imagery tasks

(involving the recall of the relative locations of objects, and spatial transformations of

objects). They found a clear dissociation in L.H.’s performance on object and spatial

imagery tasks – object imagery was impaired, whereas spatial imagery performance

was normal.

The present study investigated whether a dissociation between ‘what’ imagery

and ‘where’ imagery can be observed in subject’s eye movements during visual recall.

The purpose of eye movements during perception is clear - to focus particular

stimuli of interest on the fovea, so that fine spatial detail can be examined. However,

6

the question of whether eye movements during imagery reflect cognitive processes

involved in visual recall is a matter of controversy.

‘Strong’ theories posit a direct, functional role for eye movements during

imagery, such as Norton’s feature network theory (Norton & Stark, 1971a). Norton

and Stark (1971a, 1971b) recorded subject’s eye movements while they viewed large

line drawings, and again when subjects were presented with the same stimulus for

recognition. They found that subjects scanned images with a fixed, repetitive pattern

of eye movements, characteristic for each subject, and that those patterns were

observed again when subjects viewed the stimulus again for recognition.

Based on these observations, Norton proposed that an object stored in visual

memory consists of a network of separate features, which are spatially indexed by

recording the associated eye movements that were required to shift from feature to

feature. Imagery would therefore take place by a serial recall of sensory memory

traces, with a corresponding re-enactment of the eye movements required to shift

attention from one feature to the next.

Support comes from many studies. For example, Brandt and Stark (1997)

presented subjects with a series of irregularly-chequered diagrams, and were then

asked to visualise them. They observed characteristic eye movement patterns during

viewing of a diagram, as Norton & Stark had done, and that similar eye movements

were observed during imagery of the same stimulus.

Laeng and Teodorescu (2002) demonstrated that subjects who maintain their

gaze on a central fixation point during perception of a stimulus make little or no eye

movement when recalling the same stimulus. In contrast, subjects allowed to move

their eyes during perception made eye movements during recall, which are very

similar to those observed during perception. Most importantly, they demonstrated that

those subjects who were allowed to make eye movements during the perception

phase, but were forced to fixate during imagery, were less accurate in recalling details

than subjects allowed to move their eyes during imagery. Laeng and Teodorescu

conclude that this can be taken as evidence for a functional role of eye movements

during imagery – that is, eye movements may be important in the accurate recall of

visual information.

Hall (1974) presented subjects with a series of arrays of 12 geometric shapes,

and then asked to recall the shapes they had seen. During recall, subjects either fixated

7

on a cross, or were allowed to freely move their eyes. Subjects who fixated during

recall remembered fewer shapes, and had a slower recall latency (Hall, 1974).

However, there are many objections. Movement of the eyes is certainly not a

necessary condition for visual recall. Although Laeng and Teodorescu (2002) did

observe a detrimental effect of forced fixation on imagery performance, this effect

was far from total, and caused only a slight reduction in performance. Furthermore,

Norton and Stark (1971a) only found repetitive, characteristic scan paths during

recognition of stimuli in 65% of trials, and do not provide a quantitative method for

analysing scan paths. If eye movements do play a role in imagery, they are helpful,

but not necessary.

We must also consider studies that find no relationship between eye

movements and imagery performance. For example, Hale and Simpson (1970)

presented noun pairs, using a tape-recorder, to subjects whilst recording their lateral

eye-movements. Their task was to generate mental images that linked each noun pair.

Subjects were instructed either to make eye movements during visual imagery, or not

to make eye movements, and the time it took to generate images, and the reported

‘vividness’ of those images, was recorded under each condition. They found that

making eye movements made no significant difference to the vividness or latency of

generating mental images (Hale and Simpson 1970).

Janssen and Nodine (1974) presented nouns for memory storage, using a tape-

recorder, while subjects either fixated on a cross, or were allowed to move their eyes.

They found no difference in the recall performance of nouns between the two

conditions. These two studies therefore contradict the idea that eye-movements

facilitate recall of visual information.

The disagreement between these studies and those of Norton & Stark (1971a,

1971b), Laeng & Teodorescu (2002) and Hall (1974) may be due to a simple factor:

Janssen & Nodine (1974) and Hale & Simpson (1970) used auditory stimuli,

presented using a tape-recorder, whereas Norton & Stark and others used visual

stimuli. It seems that the precise type of cognitive demand strongly influences

whether or not eye movements occur during imagery, and in what spatial pattern.

The present study was designed to investigate the effects of different cognitive

demands on the pattern of eye movements during visual recall. Considering the

evidence already resented, it is reasonable to hypothesise that recall of ‘what’ visual

8

information (e.g. colour and shape) may elicit a different pattern of eye movements to

the recall of ‘where’ information. Furthermore, the requirement of a verbal response

from subjects may also be a significant factor.

This will be tested by presenting subjects with a visual array of 4 shapes, in 4

corners of a display screen. Subjects will then be asked to recall either ‘what’ info or

‘where’ info about one of the shapes, whilst their eye movements are recorded.

Subjects will also be asked to simply ‘imagine’ the colour or shape of one of the

shapes, without verbally responding.

Eye movements that subjects made during recall will be analysed to determine

what proportion are directed towards the corner of the screen that previously

displayed the shape being recalled, compared to the other corners of the screen. An

observed difference in eye movements between ‘what’ and ‘where’ imagery

conditions would be evidence that these types of visual imagery operate and interact

with the ocular system in different ways. It would also demonstrate a further

difference in how the brain processes ‘what’ and ‘where’ information.

Furthermore, if one condition elicited a higher proportion of eye movements

towards the corner being tested than another, this would suggest a greater reliance of

this type of visual recall on eye movements which are related to the spatial

arrangement of the stimulus previously stored.

We also tested for a difference between the recall of visual information when

subjects must verbally respond, and recall of the same information without subjects

responding verbally (i.e. simply ‘imagining’ the stimulus). If a difference is observed

here, this would suggest that the task of responding verbally also influences the nature

of eye movements elicited.

A confounding variable, which may influence subject’s responses, is how

good their short-term visual memory is. To control for this, each subject’s visual

working memory capacity was measured in a simple test (see section 2.6), to

investigate whether there is an interaction of memory capacity with eye movement

responses.

To minimise the influence of demand characteristics of the experimental

procedure, subjects were led to believe that pupil size was being measured rather than

eye movements. The idea of pupil measurement was reinforced during the

experiment, and eye movements were not mentioned at any time.

9

2. Methods

2.1 Outline

A series of 60 computer images were used as stimuli. Images consisted of a

grey background, with simple shapes in each corner (a triangle, a square, a circle and

a star), which were different colours (red, green, yellow and blue). After viewing each

image for 5 seconds, while eye movements were recorded by an eye-tracker, subjects

were then asked to recall information about those images.

Six types of questions were asked, which constituted six experimental

conditions (see table 1). These were designed to task visual memory in different ways.

Our primary interest is how eye-movement responses to ‘what’ questions compare to

responses to ‘where’ questions. However, of secondary interest is whether using

shape or colour as the ‘reference’ for recall will affect subject responses. For example:

- “What was the red shape?” requires that object information about what the

shape was (its shape), using a colour (red) as the reference for recall. This type

of question will be referred to as What (colour).

- “What colour was the circle?” requires that object information about what

the colour of the shape was, using its shape (circle) as a reference. This type of

question is What (shape).

Therefore there are two conditions for both ‘what’ and ‘where’ questions, one

using colour as a reference, and the other using shape (see table 1). All these

questions require a response from the subject.

Of secondary interest is what happens when similar questions are asked, but a

verbal response is not required – this is investigated using a further two experimental

conditions, in which subjects are asked to imagine a particular shape, again using

colour or shape as a reference.

10

Table 1: Summary of the 6 experimental conditions

Question type Information to be recalled‘What’ or ‘Where’

visual memory tested?

1. What was the yellow shape? Recall shape, with colour as a reference What

2. What colour was the square? Recall colour, with shape as a reference What

3. Where was the red shape? Recall location, with colour as a reference Where

4. Where was the square? Recall location, with shape as a reference Where

5. Imagine the yellow shape. Recall shape, with colour as a reference (with no verbal response)

?

6. Imagine the colour of the square.

Recall colour, with shape as a reference (with no verbal response).

?

2.2 Subjects

Twenty volunteers aged 20-25 took part in the experiment (16 females, 4

males). All subjects were psychology undergraduate students. Subjects were naïve as

to the true purpose of the experiment, as they were informed that their pupil size

would be recorded during a simple memory exercise. No subjects were colour-blind.

Subjects were divided into two equal blocks of ten:

a) Those doing ‘what’ and ‘where’ trials first, and then ‘imagine’ trials.

b) Those doing ‘imagine trials first, then ‘what’ and ‘where’ trials.

The purpose of doing this is to investigate any ‘carry-over’ effects between

conditions in which a response is required, and conditions in which it is.

2.3 Stimuli

60 unique images were used, one image for all 60 trials in one experimental

run. The dimensions of each image were 1024 x 768 pixels, with 4 shapes (a triangle,

a square, a circle and a star), each no larger than 120 x 120 pixels, in the 4 corners

(See figure 1, and appendix 1 for a more

detailed diagram). Each shape covered an

area approximately 4.6 of visual angle at the

distance tested. The arrangement of shapes in

each image was unique, and counterbalanced

across all 60 images.

The background was neutral grey. The

four shapes in each image was either red,

11

Figure 1: Example stimulus

green blue or yellow – again the combination of colours was unique for each image,

and counterbalanced. The colour values of all colours used are summarised in table 2.

The shape to be tested (the test shape) in each image was determined prior to

the experiment. All 60 test shapes were unique, and counterbalanced, with respect to

their colour, shape and location.

Table 2: RGB Colour values for images

Colour R value G value B value Hue Saturation LuminanceRed 255 0 0 0 240 120

Green 0 255 0 80 240 120Blue 0 0 255 160 240 120

Yellow 255 255 0 40 240 120Grey 128 128 128 160 0 120

2.4 Apparatus

An infra-red based eye-tracker, made by Cambridge Research Systems, was

used to record eye movements. Two separate infra-red sources are used to create two

purkinje images reflected from the subject’s eye. By tracking the relative distance

between the purkinje images and the pupil, the position of the eye (after proper

calibration) can be calculated. Eye movements were tracked at a rate of 50 hertz.

Subjects sat with their head held in position by a chinrest, in a dimly lit room.

Both the table and chair height were adjustable to enable maximum comfort for the

subject. Stimuli were presented at an 8-bit colour depth, on a 21” monitor running at

100hz. The display size was 3960 x 2920 mm. The distance between subject’s eyes

and the screen was 550mm, which gives a stimulus area of 35.7 - 29.7 of visual

angle. The eye tracker is accurate to approximately +/- 0.25 – 0.5 of visual angle.

2.5 Procedure

Subjects were informed, in the first instance, that they were taking part in an

experiment in which we would measure pupil size during a simple memory exercise.

As discussed earlier, this essential step was designed to avoid subjects becoming

aware of their eye movements, and so increasing the validity of our results.

On arrival at the laboratory, they were given an instruction sheet (see

Appendix 2), which described the experimental procedure. Depending on whether

12

subjects belonged to the ‘what and where trials first’ block or ‘imagine trials first’

block, the instruction sheet only described either the ‘what and where’ trials or the

‘imagine’ trials respectively. Further instructions were only given after completion of

the first set of trials.

Subjects were then seated in the eye-tracker. The system was first calibrated to

each subject for maximum accuracy. Subjects were asked to look directly at a series

of 20 small black dots which appeared at various locations on the screen. The position

of subject’s eyes whilst fixating on each dot was used to calibrate the eye-tracker.

Subjects were informed that this procedure enabled the apparatus to more accurately

measure pupil size.

When the subject was ready, the first set of trials began. Each trial consisted of

the following sequence:

1. Image displayed on screen for 5000 ms

2. Neutral grey screen displayed

3. Either a question was asked about the image, or the subject was asked to

imagine a shape from the image. Simultaneously, co-ordinates of their eye

movements were recorded, for 5000 ms. Accuracy of verbal responses were

also recorded.

This sequence was repeated until that set of trials was complete. For those

subjects who did ‘response-required’ trials first, the set of trials consisted of 40

randomly mixed questions:

- 10 questions asking ‘What colour was the (shape)?’

- 10 questions asking ‘What was the (colour) shape?’

- 10 questions asking ‘Where was the (shape)?’

- 10 questions asking ‘Where was the (colour) shape?’

For those subjects who did ‘imagine’ trials first, the set of trials consisted of

20 randomly mixed questions:

- 10 statements asking subjects to ‘Imagine the (colour) shape.’

- 10 statements asking subjects to ‘Imagine the colour of the (shape)’.

On completion of the first set of trials, subjects were informed of the second

set of trials. After a minute rest period, the second set of trials was conducted. After

completion, all subject were tested for their visual working memory capacity.

13

2.6 Visual Working Memory test

Working memory is often tested using the digit span task. This test has been

modified in the present experiment to test working memory capacity for visual

shapes, in which subjects memorize a string of shapes shown on cards, and must then

immediately draw them on paper. Although we cannot rule-out the possibility of

verbal encoding, we will assume that this will give a fair approximation of the visual

memory capacity of each subject.

Example stimuli can be found in appendix 3. Stimuli were presented by the

experimenter on white cards, 19cm x 8cm, with each shape printed in black and

approximately 1cm x 1cm in size. Subject and experimenter sat half a meter apart – at

this distance the shapes were easily visible. Subjects were given a number of seconds

equal to the number of shapes on the card in which to memorise each string. Starting

from 1 shape, each correct response increased the number of shapes to be tested on

the next trial. The test was continued until subjects gave two successive incorrect

answers, and the number of shapes subjects could recall correctly was recorded as the

measure of his/her visual working memory capacity.

2.7 Data Analysis

The temporal sequence of eye positions, recorded for each trial at 20ms intervals

(50hz), was initially processed to discard minor movements, and extract a series of

eye fixations and their durations. This was done by comparing each co-ordinate to the

next, to determine whether a spatial shift of 15 pixels or more (0.5 of visual angle or

more) had occurred. If the shift was smaller, there was considered to be no change in

fixation. If the shift was larger, and this deviation remained for longer than 80ms, this

was considered a change in fixation.

Each fixation was then assigned as being (see Figure 2):

1. In the same quadrant as the shape being tested and recalled. This quadrant will

be referred to from now on as the ‘correct quadrant’.

2. In one of the other quadrants of the screen. These shall be called ‘non-test

quadrants’.

14

3. Within a 100 x 100 pixel area in the middle of the screen (see Figure 2),

referred to as the ’exclusion box’. It was decided that saccades in this area

could not be considered to be definitely in any particular quadrant.

Figure 2: Diagram showing how fixations were divided into quadrants

15

3. Results

3.1 Scan paths

When subjects were presented with a stimulus for 5 seconds (the stimulus inspection

phase), their eyes made saccades to at least two, usually three shapes in that stimulus.

For a typical example, see Figure 3. Analysis of all recordings confirmed that subjects

did not simply fixate in the centre of the screen.

During the memory recall

phase, in response to a

question or a command to

imagine one of the shapes, eye

movements across the screen

were successfully recorded in

most cases. Examples of scan

path recordings during

memory recall can be seen in

Figure 4, superimposed on the

particular image which was

being recalled at the time.

Eye movements were not recorded in 24 trials, due to tracking obstructions such as

closing eyelids, blinking, or looking outside the boundary of the screen. However this

represents only 2% of trials conducted in total.

16

Figure 3: recording of a typical subject's eye movements when viewing a stimulus for 5 seconds

Figure 4: examples of eye-movement recordings during memory recall.

Figure 4A: "What colour was the star?"Figure 4B: "What was the red shape?"Figure 4C: "Where was the square?"

Figure 4D: "Where was the red shape?"

17

Figure 4A Figure 4B

Figure 4C Figure 4D

3.2 Percentage of fixation duration in correct quadrant

For each trial, the duration of time subjects fixated in the correct quadrant was

calculated as a percentage of the total duration of time they fixated in all four of the

quadrants. Therefore, a percentage of over 25% would indicate that more eye

movements were being made towards the corner of the screen which previously

displayed the shape being tested, than to any other corner. The means and standard

deviations for all 6 conditions (see table 1) are summarised in table 3. Full data can be

found in appendix 4.

Table 3: Average time spent in correct quadrant (ms), per condition, as a percentage of total time in all 4 quadrants

Figure 5 shows these data

in a bar chart. A line at

25% on the y-axis, i.e.

25% of time spent in the

correct quadrant, rep-

resents what would be

expected assuming the null hypothesis that subjects do not make more eye movements

towards the quadrant being tested. There was a slight increase over 25% for 'what'

conditions, and a much larger increase over 25% for 'where' and 'imagine' conditions.

The increase in where' and 'imagine' conditions is larger when colour is used as the

reference.

To test whether the proportion of time spent in the correct quadrant was significantly

more than 25%, 6 t-tests were carried out on the data, to test whether each condition

varied significantly from having a mean of 25%. Therefore, they tested the probability

that the data for each condition was sampled from a population having a mean of

25%, and a standard deviation equal to the standard deviation found in our data (see

table 3).

Table 4 summarises the t-values and their related one-tailed significance

values for all conditions (see appendix 5 for source data). A one-tailed test was used

because only increases over 25% were observed in the data (see figure 5), and so only

18

Condition Average % time

St. Dev.

What (colour) 26.28 % 33.30

What (shape) 27.04 % 34.97Where (colour) 44.44 % 38.60Where (shape) 36.48 % 36.86

Imagine (colour) 43.04 % 37.34Imagine (shape) 40.69 % 38.05

Figure 5: Time spent in correct quadrant as percentage of time spent in all 4 quadrants

increases were tested for. The Kolmogorov-Smirnov test was used to test whether the

data were normally distributed - the data were approximately normally distributed, but

only with a confidence of p<0.2 (KS value = 0.46, p<0.2). However, the t-test, and

ANOVA test, are robust to small departures from normality. See Appendix 6 for the

appropriate Q-Q plot.

Table 4: T-values for each condition, testing for percentage time in correct quadrant being more than 25%

Condition Tobt dfTcrit =0.05(1-tailed)

Tcrit =0.01(1-tailed)

Significant?

What (colour) 0.18 19 1.73 2.54 n.s.What (shape) 0.65 19 1.73 2.54 n.s.Where (colour) 4.35 19 1.73 2.54 p<0.01Where (shape) 3.60 19 1.73 2.54 p<0.01Imagine (colour) 4.66 19 1.73 2.54 p<0.01Imagine (shape) 4.24 19 1.73 2.54 p<0.01

19

For 'what' conditions, there was no significant increase in the average

proportion of time spent by subjects fixating in the correct quadrant, over 25%.

However for 'where' conditions, there was a highly significant increase over 25% (t=

4.35, t=3.60, p<0.01). Recalling the location of a shape, using colour as the reference,

elicited a stronger increase in the proportion of fixation durations in the correct

quadrant, than using shape as a reference (44.4%, compared to 36.5%).

Conditions which subjects were asked to 'imagine' a particular shape also

elicited significant increase in average proportion of fixations in the correct quadrant

over 25% (t=4.66, t=4.24, p<0.01).

A one-way related ANOVA (see Table 5) was also carried out on the data to

test for significant differences between conditions. Condition (what type of question /

statement was given to the subject) was found to have a highly significant effect (F=

7.23, p<0.001). Subjects also represented a significant source of variance (F=2.67,

p<0.001).

Table 5: Related ANOVA table comparing average percentage time incorrect quadrant for all 6 conditions

Source of Var.

Sum of Sq.

df Mean Sq.

F Ratio

F critical (=0.001)

Condition 6606.13 5 1321.23 7.23 F5,95 = 4.48

Subjects 9262.34 19 487.49 2.67 F19,95 = 2.59

Error 17369.06 95 182.83

Total 33237.53

119

To reveal which conditions in particular gave significantly different results, a

posthoc Tukey's honestly significant difference yardstick (R) was calculated, using

=0.01. Comparing the yardstick (R= 15.09, =0.01) with the means for each

condition (see Table 3) shows that there was a significant difference between the

'where (colour)' condition, and both 'what' conditions, and a significant difference

between the imagine (colour) condition and both 'what' conditions. Other differences

were not significant.

20

3.3 Number of fixations in correct quadrant

For each trial, the number of fixations in the correct quadrant, in the other 3

quadrants, were counted. The mean number of fixations per trial, for each condition,

and standard deviations are summarised in Table 6.

Table 6: Mean number of fixations per trial in correct Quadrant and other 3 quadrants

Condition Mean no. fix-ations in correct

quadrant

St. Dev.

Mean no. fix-ations in all other

quadrants

St. Dev.

What (colour) 1.98 2.71 4.86 3.75

What (shape) 1.77 2.54 4.92 3.70

Where (colour) 3.19 3.32 3.63 3.55

Where (shape) 2.43 2.65 4.12 3.38Imagine (colour) 3.34 3.66 3.17 2.96

Imagine (shape) 2.89 3.22 4.00 3.36

Figure 6 shows a histogram of average number of fixations in the correct quadrant,

and the average number of fixations in each of the three other quadrants (i.e. the

values from the 4th column in table 6, divided by 3), for each condition. To obtain a

meaningful comparison, the width of bar representing the number of fixations in the

other 3 quadrants is 3 times the width of the bar representing the single, correct

quadrant.

21

Figure 6: Histogram of average number of fixations in correct quadrant, and the other 3 quadrants, for each condition.

For both 'what' conditions, there was little difference between the average

number of fixations made in the correct quadrant, and the average number made in the

other three quadrants. In the 'where' conditions there were, on average, significantly

more fixations made in the correct quadrant than in the other three quadrants. The

difference was always larger when colour was the reference. The 'imagine' conditions

again appeared to elicit responses very similar to 'where' conditions - there were

significantly more fixations made in the correct quadrant than the other three

quadrants, and again the difference was larger when colour was used as the reference.

For each condition, related t-tests were used to test the significance of

differences between:

a) the average number of fixations made in the correct quadrant

and

b) the average number of fixations made in the other 3 quadrants, divided by

3. i.e. the average number of fixations made in each of the three other

quadrants.

Table 7 summarises the t-values and their related critical values. See Appendix

7 for the source data. A Kolmogorov-Smirnov test was used to test whether the data

were normally distributed - the data were normally distributed (KS value = 0.17,

p<0.00). See Appendix 8 for the appropriate Q-Q plot.

Table 7: T-values to tests significance of difference between no. of fixations in correct quadrant, and average no. of fixations in each of the other quadrants, for each condition.

Condition Tobt df Tcrit =0.05 (1-tailed)

Tcrit =0.01 (1-tailed)

Significance?

What (colour) 1.06 19 1.73 2.54 n.s.What (shape) 0.70 19 1.73 2.54 n.s.Where (colour) 4.55 19 1.73 2.54 p<0.01Where (shape) 3.52 19 1.73 2.54 p<0.01Imagine (colour) 4.01 19 1.73 2.54 p<0.01

Imagine (shape) 3.80 19 1.73 2.54 p<0.01

22

There was a significant difference between the average number of fixations in the

correct quadrant, and the average number in the other three quadrants, in the 'where'

conditions (t=4.55, t=3.53, p<0.01) and 'imagine' conditions (t=4.01, t=3.80, p<0.01)

only. The difference for 'what' conditions was not significant.

Table 8: number of fixations in correct quadrant, as a percentage of total number of fixations made, per trial, for each condition

For each condition, the number of

fixations made by subjects in the

correct quadrant was calculated as a

percentage of the total number of

fixations made. The mean

percentages, and standard deviations,

are summarised in Table 8. A related ANOVA was carried out on the data to test for significant differences in the

proportion of fixations made in the correct quadrant, between conditions. Table 9

shows the ANOVA table (see Appendix 9 for source data).

Table 9: ANOVA table of variance between conditionsfor average percentage of number of fixations in test quadrant

Source of Var.

Sum of Sq.

df Mean Sq.

F Rati

o

F critical (=0.001)

Condition

7509.32 5 1501.86

6.65 F5,95 = 4.48

Subjects 11367.14

19 598.27 2.65 F19,95 = 2.59

Error 21442.58

95 225.71

Total 40319.04

119

Condition had a significant effect on the proportion of fixations subjects made in the

tested quadrant, compared to the other quadrants (F=4.48, p<0.001). Subjects were

also a significant source of variation (F=2.59, p<0.001). A posthoc Tukey's HSD

yardstick, calculated as R=16.76 (for =0.01), revealed that there was a significant

difference between where (colour) and both what conditions, and imagine (colour)

and both 'what' conditions.

23

ConditionMean % fixations

in correct quadrant

St. Dev.

What (colour) 28.9% 19.1

What (shape) 26.5% 12.0

Where (colour) 46.8% 18.7

Where (shape) 37.1% 13.4Imagine (colour) 51.3% 17.9

Imagine (shape) 41.9% 16.9

3.4 Total fixation duration of all subjects in all quadrants

Table 10 summarises the total duration of recorded fixations made by all

subjects, across all trials, for each condition. The total duration of fixations made in

the correct quadrant, in the other 3 quadrants, and in the central exclusion box, are

shown separately.

The total recording time for all subjects in each condition should have been

1,000 seconds (10 trials per condition, lasting 5 seconds, for 20 subjects) - however,

some time will have been 'lost' due to blinks, or other disruptions to eye-tracking.

Therefore the total time recorded in each quadrant as a percentage of 1,000 seconds is

also shown, as well as the percentage of recording time which was lost.

Table 10: Total duration of fixations (ms\ by all subjects, for each condition, with percentage of total recording time per condition (1,000,000 ms)

Condition Total T in correct quad.

Percentage of total

Total T in other 3 quads.

Percentage of total

Total T in exclusion

box

Percentage of total

Percentage of T not

recorded

What (colour) 108220 10.8% 259440 25.9% 553320 55.3% 8%

What (shape) 101680 10.2% 273140 27.3% 556600 55.7% 6.8%

Where (colour) 176240 17.6% 207820 20.7% 537640 53.8% 7.9%

Where (shape) 136980 13.7% 243460 24.3% 561740 56.2% 5.8%

Imagine (colour) 206960 20.7% 241540 24.2% 411120 41.1% 14%

Imagine (shape) 218500 21.9% 291180 29.1% 363200 36.3% 12.7%

For 'what' and 'where' conditions, the time subjects were recorded fixating in

the central exclusion box was always more than 50% of the total time for each

condition.. However this is not true for the 'imagine' conditions - the time subjects

spent in the central exclusion box drops significantly. The total time spent in the

correct quadrant was also higher in both 'imagine' conditions than any other.

Between 5.8 % and 14 % of the total recording time was lost because of loss

of tracking.

The total time subjects spent in the correct quadrant is compared to the total

time subjects spent in the other quadrants in Figure 7. To make a meaningful

comparison, the bar for the total time spent in the three non-test quadrants is three

times the width of the bar representing the single, correct quadrant.

24

Figure 7: Histogram of total duration (ms) of fixations by all subjects, in each quadrant, for all conditions.

There is little difference in the total time recorded in the correct quadrant and the

other quadrants for both 'what' conditions. There is a large difference for both 'where'

and 'imagine' conditions. Again, these differences are slightly larger when colour was

the reference for recall.

3.5 Mean duration of fixations in each condition

The mean duration of fixation made by subjects, in both the correct quadrant and the

other quadrants, are compared in Table 11. Also shown are the average duration of

fixation made in the central exclusion box, and standard deviations.

Table 11: Mean duration of fixation by all subjects in test quadrant, non-test quadrants and exclusion box

Condition Tested quadran

t

St. Dev.

Non-test quadran

ts

St. Dev.

Exclusion box

St. Dev.

What (colour) 304.0 ms

283.4

292.2 ms

288.1

352.8 ms 492.7

What (shape) 312.9 ms

352.4

311.1 ms

302.5

296.9 ms 332.3

Where (colour) 285.2 ms

288.8

308.3 ms

284.8

334.6 ms 525.5

Where (shape) 294.6 ms

315.3

326.8 ms

342.6

301.1 ms 343.0

Imagine (colour) 310.8 ms

350.4

381.0 ms

447.0

519.9 ms 754.2

Imagine (shape) 378.0 459. 362.6 419. 552.7 ms 714.

25

ms 5 ms 6 6

These data are represented as a bar chart in Figure 8. It shows only small differences

in the duration of fixation that subjects made when fixating in a test quadrant,

compared to when fixating in a non-test quadrant. However, in 'imagine' conditions,

the average duration of fixation is slightly higher in the exclusion box.

Figure 8: Average duration of fixation (ms) in test quadrant, non-test quadrants, and exclusion box

3.6 Comparison of 'what/where trials first' subjects and 'imagine trials first' subjects

10 subjects were given 'what' and 'where' conditions first, which required a

verbal response, and then 'imagine' conditions, which did not. 10 different subjects

were given 'imagine' conditions first, then 'what' and 'where' conditions. It was

considered possible that this variable may influence subject's eye movement

behaviour, and so it was equally balanced across subjects in this way.

Table 12 shows the duration of fixations in the correct quadrant, as a

percentage of the duration in all quadrants, spent by subjects who ran the 'imagine'

conditions first, and the subjects who ran the 'what / where' conditions first.

26

Figure 9 shows these percentages as a bar chart. It shows that there was little

difference between the two groups of subjects.

Table 12: Average duration of fixation in test quadrant as a percentage of duration in all quadrants, for each condition, comparing 'what/where 1st' with 'imagine 1st'

Condition'Imagine'

conditions presented 1st

St. Dev.

'What / where' conditions

presented 1stSt.

Dev.

What (colour) 28.45 % 34.45 24.10 % 32.15

What (shape) 26.65 % 32.79 22.99 % 32.89

Where (colour) 37.12 % 37.31 51.30 % 38.72

Where (shape) 36.47 % 38.37 36.49 % 35.56Imagine (colour) 39.45 % 37.57 46.58 % 36.87

Imagine (shape) 35.57 % 37.76 45.92 % 37.81

Figure 9: Average % time in tested quadrant, comparing subjects who ran 'what' and 'where' trials 1st with subjects who ran 'imagine' trials 1st

A two-way, mixed-design ANOVA was used to test for a significant difference

between the two groups. Variable A was whether subjects did 'what/where' trials first

or 'imagine' trials first (the between-subjects variable). Variable B was the condition

(the within-subjects variable). Table 13 shows the ANOVA table for this calculation -

the source data can be found in appendix 10.

The effect of variable A, whether subjects did which trials first, was

insignificant, and there was no interaction with variable B (condition).

27

Table 13: 2-way, mixed-design ANOVA to compare effect of 'what/where first' with 'imagine first', across all conditions

Source of Variance

Sum of Sq.s df Mean

Sq.s F ratio Fcrit = 0.05

Fcrit = 0.01

W/W or Imagine 1st (Var. A) 792.26 1 792.26 0.94 4.96 10.0

Error AS 8470.11 10 847.01

Condition (Var. B) 6606.15 5 1321.23 4.10 2.40 3.41A x B (interaction) 1275.94 5 255.19 0.793 2.40 3.41

Error B x AS 16093.10 50 321.86

Total 33237.56 119

3.7 Incorrect Responses

The total number of incorrect responses given by all subjects, for the 'what' and

'where' conditions, are compared in Table 14. Since subjects did not respond to

'imagine' trials, the success of recall was not recorded. Table 11 shows that 'where'

questions elicited fewer incorrect responses than 'what' questions. In total, 7% (56) of

'what' and 'where' trials were answered incorrectly.

Table 14: Total number of incorrect responses, for all subjects, in 'what' and 'where' conditions (out of 200).

ConditionNo. of

incorrect responses

What (colour) 18

What (shape) 17Where

(colour) 9

Where (shape) 12

3.8 Visual Working memory score

All subjects scored between 3 and 6 on the visual working memory tests (the vWM

score). That is, subjects could remember a string of between 3 and 6 of the shapes

used on the working memory test (see appendix 3). Figure 10 shows the frequency of

subjects who had working memory scores of 3, 4, 5 and 6. Half of subjects scored 4.

28

Figure 10: Histogram of working memory test scores for all subjects

Table 15 is a summary of working memory scores for each subject, and the number of

incorrect answers given by subjects in 'what' and 'where' trials. Figure 11, a plot of

subject's visual working memory scores against the number of incorrect answers

given by subjects, reveals a moderate negative correlation (-0.42).

Table 15: Working memory scores and number of incorrect responses for each subject

Figure 11: Plot of visual working memory score against number of incorrect answers

29

Subject vWM score

No. wrong

1 4 9

2 3 2

3 4 6

4 6 0

5 6 0

6 3 5

7 5 0

8 4 3

9 5 3

10 4 9

11 4 2

12 6 1

13 4 1

14 3 2

15 5 5

16 5 0

17 4 3

18 4 3

19 4 4

20 4 9

Figures 12 shows the average percentage of fixation duration in the correct quadrant

per trial, for subjects with different working memory scores. Figure 13 shows the

same percentages for subjects who gave different numbers of incorrect responses.

These graphs show that the eye movement data did not appear to be related to

subject's working memory capacity.

Figure 12: Average percentage time in correct Figure 13: Average percentage time in correct quadrant against visual WM score quadrant against number of incorrect answers

3.9: Subject reports

When informed of the true nature of the experimental hypothesis (after conducting the

experiment), subjects reported being unaware of it - they assumed that pupil diameter

was being measured rather than eye movements.

30

4. Discussion

The present study investigated the effects of different cognitive demands on the

pattern of eye movements during visual recall. It was predicted that the different

demands of recalling ‘what’ visual information (e.g. colour and shape) and ‘where’

visual information may elicit different patterns of eye movements, in terms of whether

or not eye movements during recall of a stimulus are directed towards the spatial

location of the stimulus when it was stored (such eye movements will be referred to

from now on as Towards Eye Movements, or T.E.M.s.)

This was tested by presenting subjects with a visual array of 4 shapes, in 4

corners of a display screen. Eye movements that subjects made during a 5-second

‘recall phase’ were analysed to determine what proportion are directed towards the

corner of the screen that previously displayed the shape being recalled, compared to

the other corners of the screen.

For ‘what’ trials, there was no significant increase in the proportion of time

that subjects fixated in the correct quadrant, compared to in any of the other

quadrants. Neither were there a greater number of fixations in the correct quadrant

compared to other quadrants. We can conclude that the recall of visual information

does not appear to cause the brain to make TEMs.

In contrast, when subjects recalled ‘where’ visual information, they spent a

significantly greater proportion of time fixating in the correct quadrant, compared to

any of the other quadrants. They also made a greater number of fixations. We can

conclude that visual imagery of a stimulus which involves spatial information causes

the brain to make eye movements which are related to the spatial co-ordinates of the

stimulus when it was stored (i.e. they made TEMs).

However, although this is what the brain does, the present experiment does not

tell us whether these eye movements actually helped subjects recall the information.

This is because reaction time was not recorded due to time constraints of this project,

and the difficulty of the task was not great enough to give a high variance in

performance. This means we cannot conclude whether or not eye movements were

functionally involved in any way with visual recall.

For example, it may be, in accordance with Norton and Stark’s feature

network theory (Norton & Stark; 1971a; 1971b), that the eye movements observed in

this study were functionally involved in aiding visual working memory to recall

31

spatial information. That is, the re-enactment of the eye movements themselves help

working memory to access features of the image stored in visual memory.

Alternatively, they may be that the TEMs observed were an irrelevant by-

product of thinking about spatial locations. The eyes are almost constantly moving, so

it seems plausible to consider they moved towards the location of the stimulus when it

was stored simply in the absence of anything else relevant to look at. The present

study cannot rule out such explanations.

However, the study has revealed an important question: why did we observe

these eye movements in ‘where’ conditions, but not in ‘what’ conditions? This is an

unexplained and reliable phenomenon, worthy of further investigation.

For example, further research using the same basic procedure as the present

study, but with a more difficult task and with reaction time measured for each subject,

would reveal more information about how eye movements relate to recall

performance. If performance was strongly related to TEMs, this would suggest a more

functional role for eye movements in visual imagery.

Furthermore, the exact timing of subject responses could be recorded on a

microphone, so that eye movements could be analysed in terms of exactly when

subjects succeeded in recalling the stimulus. If TEMs were made before subjects gave

a response, this would suggest they were used to help recall the visual information. If

they were made after subjects responded, this would suggest they play no role in

helping working memory, and are a simple by-product. Unfortunately, the present

studied analysed a whole 5-second time period called the ‘recall phase’, without a

precise record of when during that 5 seconds subjects had succeeded in recalling the

stimulus.

When subjects were asked to recall and simply imagine ‘what’ information

(e.g. “imagine the colour of the square”, “imagine the green shape”), they spent

significantly more time fixating in the correct quadrant than any other quadrants, to

approximately the same extent as during ‘where’ conditions. This suggests that the

‘relaxed’ nature imagine conditions (that is, the subject only has to think), caused

subjects also to recall and imagine the location of the shapes, i.e. the ‘where’

information as well as the ‘what’ information. This would account for the TEMs

observed.

If further research was done to further investigate this, it would be more

appropriate to replace ‘imagine’ statements with replications of the same questions

32

used in the ‘what’ and ‘where’ conditions (e.g. “what colour was the square?”,

“Where was the green shape?”), and simply ask subjects not to responds to the

questions (but to think about them). In this was, the effect of making a verbal

response can be directly compared to identical trials in which subjects do not respond,

without any change in what is said to subjects. Also, it would allow testing of both

‘what’ and ‘where’ visual recall without a verbal response, whereas the present study

only investigated ‘what’ visual recall.

No significant differences were found in TEMs between conditions in which

colour was used as the reference for recall (e.g. “What was the green shape?”) and

conditions in which shape was the reference for recall (e.g. “What colour was the

triangle?”). However, there were consistently slightly more TEMs observed in

conditions using colour as the reference. This suggests that the distinction between

recalling colour information and recalling shape information may be a further variable

involved in the pattern of eye movements during visual imagery. A more powerful

experiment may be required to investigate this.

The visual working memory scores (vWM scores) for each subject appeared to

be validated, to some degree, by the negative correlation of vWM scores with the

number of incorrect answers given by subjects. A negative correlation would be

expected if the scores were valid measures of vWM capacity.

It was confirmed that subject’s vWM capacity did not influence or interact

with their eye movement responses. That is, how good each subject’s visual memory

was did not appear to influence the percentage of time subjects spent in the correct

quadrant, compared to other quadrants, during the experiment. We can say therefore

that it was not a confounding variable in the experiment.

It was also confirmed that whether we presented subjects with ‘what’ and

‘where’ trials first (in which they were required to respond), or whether we presented

them with ‘imagine’ trials first, made no significant to the variation in subject’s eye

movements. Therefore, this was also not a confounding variable in our experiment.

Subjects, however, did represent a significant source of variance in this study.

This is important in that it indicates that subjects varied significantly in the behaviour

of their eye movements during visual recall. An in-depth case-by-case analysis of eye

movement patterns would be more revealing in investigating how much subjects

differ, and in what ways, but beyond the scope of the present study.

33

Subjects were unaware of the exact nature of the experimental hypothesis, and

so we can be fairly confident that the eye movements observed were not made in

response to the task demands of the experiment – that is, that they did not make eye

movements simply because they felt like this was what was being tested for.

34

Appendix 1

Example stimulus image, showing image dimensions in pixels. 60 images in total were used, each one with a unique combination of 4 shapes (a star, circle, square and triangle) in 4 corners of the image, with 4 colours (red, green, yellow and blue).

35

Appendix 2Example instruction sheet given to subjects.

Instruction sheet

Thank you for agreeing to take part in our experiment. The first thing we will do is make sure you do not have any type of colour blindness – is you do, we may not be able to use you as a subject.

We will be measuring pupil size during a simple memory task. You will be sitting with your head held in position by a pupilometer. Please make sure you are comfortable, as the table and chair heights can be adjusted. If it needs changing, please say so -you will be in it for about 10-15 mins.

Calibration

We will then calibrate the pupilometer to your pupil. After getting a comfortable position in the pupilometer, you will see a series of 20 small black dots appear at various locations on the screen – please look directly at them in turn.

The experiment

We can then start the experiment. We will be giving you a simple memory exercise, in which you have to memorise geometric shapes presented on a computer screen. An example of what you will see is shown below:

You will see an image like this for 5 seconds – please look directly at each of the shapes, and try to memorise their colour, shape, and location.

We will then ask you a question about the image - for example, “where was the triangle?” or “what was the red shape”. You should try to answer this as quickly and accurately as you can. This sequence will be repeated a number of times.

36

Appendix 3

Test cards used in visual working memory test (See section 2.6). Real size 19 x 8 cm.

37

Appendix 4

Data tables, for each condition, of the time spent in the test quadrant as a percentage

of time spent in all 4 quadrants, on each trial and for all subjects. IR = incorrect

response - these trials were omitted from data analyses. ND = no data was recorded

from the whole trial. NQ = the whole time was spent in the exclusion box.

38

39

40

Appendix 5

Data table of duration of fixations in tested quadrant as a percentage of total duration

in all 4 quadrants, for each subject, averaged for each condition.

41

Appendix 6

Test for normality of the data: average time recorded in the correct quadrant as a

percentage of time recorded in all 4 quadrants, for all subjects over all conditions (see

Appendix 5 for data). The Q-Q plot below is a plot of the quantiles of the data's

distribution against the quantiles of the normal distribution. If the data is normally

distributed, the points cluster around a straight line.

Normal Q-Q plot of percentage of fixation duration in correct quadrant compared to other three quadrants

42

Appendix 7

Number of fixations in tested quadrant and the other 3 non-test quadrants, for each

subject, averaged across each condition.

43

Appendix 8

Test for normality of the data: average number of fixations per quadrant (see

Appendix 7 for data). The Q-Q plot below is a plot of the quantiles of the data's

distribution against the quantiles of the normal distribution. If the data is normally

distributed, the points cluster around a straight line.

Normal Q-Q plot of average number of fixations per quadrant

44

Appendix 9

Data table showing the number of fixations made in the tested quadrant, as a

percentage of number of fixations made in all quadrants, averaged for each subject,

for each condition:

45

Appendix 10

Data table of duration of fixations in test quadrant, as a percentage of total duration in

all quadrants, averaged for each subject in each condition. Subjects who conducted

'what' and 'where' conditions first are compared with subjects who conducted

'imagine' conditions first.

46

5. References

Bohbot, V.D., Kalina, M., Stepankova, K., Spackova, N., Petrides, M. & Nadel, L. (1998) Spatial memory deficits in patients with lesions to the right hippocampus and to the right parahippocampal cortex. Neuropsychologia, 36, 1217-1238e

Brandt, S.A., Stark, L.W. (1997) Spontaneous eye movements during visual imagery reflect the content of the visual scene. Journal of Cognitive Neuroscience, 9, 27-38

Derrington, A.M. & Lennie, P. (1984) Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of Macaque. Journal of Physiology, 357, 219-240

Farah, M.J., Hammond, K.M., Levine, D.N. & Calvanio, R. (1988) Visual and spatial mental imagery: Dissociable systems of representation. Cognitive Psychology, 20, 439-462

Hale, S.M. & Simpson, H.M. (1970) Effects of eye movements on the rate of discovery and the vividness of visual images. Perception and Psychophysics, 9, 242-245

Hall, D.C. (1974) Eye fixation and spatial organization in imagery. Bulletin of the Psychonomic Society, 3, 335-337

Haxby, J.V., Grady, C.L., Horwitz, B., Ungerleider, L.G., Mishkin, M., Carson, R.E., Herscovitch, P., Schapiro, M.B. & Rapoport, S.I. (1991) Dissociation of object and spatial visual processing pathways in human extrastriate cortex. Proceedings of the National Academy of Science, USA, 88, 1621-1625

Isseroff, A., Rosvold, H.E., Galkin, T., Goldman-Rakic, P. (1982) Spatial memory impairments following damage to the mediodorsal nucleus of the thalamus in rhesus monkeys. Brain Research, 232, 97-113

Janssen, W.H. & Nodine, C.F. (1974) Eye movements and visual imagery in free recall. Acta Psychologia, 38, 267-276.

Laeng, B. & Teodorescu, D.S. (2002) Eye scanpaths during visual imagery reenact those of perception of the same visual scene. Cognitive Science, 26, 207-231.

Levine, D.N., Warach, J. & Farah, M.J. Two visual systems in mental imagery: dissociation of ‘what’ and ‘where’ in imagery disorders due to bilateral posterior cerebral lesions. Neurology, 35, 1010-1018

Livingstone, M.S. & Hubel, D.H. (1987) Psychophysical evidence for separate channels for the perception of form, color, movement and depth. The Journal of Neuroscience, 7(11), 3416-3468

Marks, D.F. (1973) Visual imagery differences and eye movements in the recall of pictures. Perception and Psychophysics, 14(3), 407-412

Mecklinger, A. & Pfeifer, E. (1996) Event-related potentials reveal topographical and temporal distinct neuronal activation patterns for spatial and object working memory. Cognitive Brain Research, 4(3), 211-224

47

Mishkin, M. & Manning, F.J. (1978) Non-spatial memory after selective prefrontal lesions in monkeys. Brain Research, 143, 313-323

Norton, D. & Stark, L. (1971a) Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision Research, 11, 929-942.

Norton, D. & Stark, L. (1971b) Scanpaths in eye movements during pattern perception. Science, 171, 308-311

Ruchin, D.S., Johnson, R., Grafman, J., Canoune, H. & Ritter, W. (1997) Multiple visuospatial working memory buffers: Evidence from spatio-temporal patterns of brain activity. Neuropsychologia, 35(2), 195-209.

Smith, E.E. & Jonides, J. (1997) Working memory: A view from neuroimaging. Cognitive Psychology, 33, 5-42

Tresch, M.C., Sinnamon, H.M. & Seamon, J.G. (1993) Double dissociation of spatial and object visual memory: Evidence from selective interference in intact human subjects. Neuropsychologia, 31(3), 211-219

Van Essen, D.C., Anderson, C.H. & Felleman, D.J. (1992) Information processing in the primate visual system: An integrated systems perspective. Science, 255, 419-423.

48