Acta Psychologica 43 (1979) 455-467 @North-Holiand Publishing Company

KINETIC AND STATIC IMAGERY IN SENTENCE MEMORY

Stephen E. NEWSTEAD and Stuart BLACKLEY*

Psychology Tt@ing Group. Hymouth Polytechnic. Engkd

Received December 1978

This experiment investigates the possibility that two Werent kinds of imagery codes are used in sentence memory, one involving moving images (kinetic imagery) and the other involving stationary images (static imagery). Using a modalityspecific interference task it was shown that only sentences involving kinetic imagery were affected by the visual interference task; neither static nor low imagery sentences were so affected. fhe results arc interpreted as showing that some kind of imaginal code is used in memory, but that there are different kinds of code available. It is claimed that this result is hcon- &tent both with Paivio’s (1971) ‘dualcodmg’hypothesis and with propositional accounts of sentence memory.

Introduction

The ‘dualcoding’ hypothesis (Paivio 197 1) claims that verbal material can be remembered in two distinct forms. High imagery, concrete material has access to the visual code, while low imagery, abstract material has access only to the verbal code. This paper is primarily cancerned with the use of the visual code in remembering sentences; in other words, does sentence memory involve visual imagery? Evidence that imagery is involved comes from two main sources: firstly, there is evidence that the meaning of concrete sentences is better retained than the meaning of abstract sentences; and secondly there is evidence that visual and auditory interference tasks have different effects on the recall of concrete and abstract sentences. However, as we shall see, the evidence from neither of these sources is conclusive.

Begg and Paivio (1969) were among the first to look at the ease of recall of abstract and concrete sentences. Their experiment basically repeated Sachs (1967) investigation into memory for sentences

* The authors would like to thank Jan Dennis and Jonathan Evans for their helpful comments and advice.

456 S. E. Nnusrmd, S. Bkkkyhagw~ and sentence memory

embedded in prose, but also looked at differences between concrete and abstract sentences. They found that semantic changes were far more readily noticed in concrete sentences, but changes in wording were more easily noticed in abstract sentences. They interpreted these results as showiug that concrete sentences are remembered as visual images, which retain the overall meaning of the sentence, while abstract sentences are retained in a purely verbal form.

This conclusion can be criticised on several counts. One important possibility is that concrete sentences are simply easier to understand, and this is why their meaning is better remembered. Johnson et al. (1972) fust suggested this possibility, and gave empirical support to their idea by showing that the concrete sentences in Begg and Paivio’s study were rated as more ‘comprehensible’ than the abstract ones. Klee and Eysenck (1973) and Holmes and Langford (1976) both provide corroboratory evidence that concrete sentences are easier to compre- hend. However, an experiment by Moeser (1974) suggests that more than just comprehensibility is involved. In a series of experiments in which subjects were required to detect meaning and wording changes in sentences, she found that performance on abstract sentences was worse than on concrete ones, even with comprehensibility controlled. However, Moeser dit not find the Begg and Paivio interaction between concreteness and the type of change (meaning or wording) that was most easily detected; hence she claims her results are incompatible with the dual-coding hypothesis. Nevertheless, there is clearly something other than comprehensibility which is making concrete sentences easier to remember.

Another problem is that Begg and Paivio’s (1969) study does not directly implicate imagery in sentence memory. The findings can be explained just as easily bj a propositional account of sentence memory, if it is assumed that concrete sentences simply lead to more distinctive or to more closely interconnected propositions. Such an account is favoured by Anderson and Bower (1973), Pylyshyn (1973) and Gold- farb et al. ( 1973).

The use of interference tasks (Brooks 1967, 1968) seems more appropriate to investigate the role of imagery. If Paivio’s ‘dual-coding’ hypothesis is correct, one would expect a visual task to interfere more with concrete, high imagery sentences than with abstract sentences, and vice verJcl for a verbal-auditory interference task. Such a finding was obtained by Atwood (1971). His experiment involved remembering

S. E. Newmad, S. Ehckleyhagay and sentence memory 451

either high or low imagery phrases while responding to either a visually or an auditorily presented digit. The results showed that the visual task selectively interfered with the concrete phrases while the auditory task selectively interfered with the abstract phrases. This strongly suggests that visual imagery is involved in memory for concrete material. Confirmatory evidence comes from the work of Sasson (197 1; Sasson and Fraisse 1972). He found that recall of concrete sentences was impaired by interpolated pictures or unrelated concrete sentences. Recall of abstract sentences, however, was not so affected, but was interfered with by interpolated abstract sentences. Again, the evidence seems to suggest the use of an imagiual code in memory for concrete sentences.

However, this evidence_ has not gone unchallenged. Baddeley et al. (1975) failed to replicate Atwood’s result using adjective-noun pairs and pursuit rotor tracking as the interference task. It is also pointed out by these authors that other, unpublished experiments have also failed to replicate Atwood’s findings. Janssen (1976), however, found no such problems of replication. He used high and low imagery noun pairs and an interference task like Atwood’s, though given at retrieval rather than during presentation. Like Atwood, he found a selective interference effect.

Clearly, there are procedural differences between these experiments which might account for the contradictory results. For example, only one of the series of studies using the selective interference paradigm used sentences (Sasson 1971; Sasson and Fraisse 1972) as opposed to phrases or word pairs. There are also differences in the times at which the interference task was given. Janssen’s experiment gave the inter ference task during recall, Sasson’s experiments presented the inter- fering material after presentation, but before recall, while the remaining studies (Baddeley et al. 1975; Atwood 197 1) gave the inter- ference task during presentation of the material. However, the main difference between these various studies is probably in the interference tasks used. Janssen (1976) suggests this when he points out that the visual interference task used by Baddeley et al. (pursuit tracking) involved a strong spatial element, and this might explain why it did not interfere with concrete material which lacked this spatial component. Another possibility is that there are separate stores for retaining kinetic (moving) and static (motionless) images. Baddeley et al.‘s interference task was clearly one that involved movement, which might explain why

458 S. E. Newstead, S. BkzckIeyj.magery and sentence memov

it failer, to disrupt the largely static images evoked by the adjective- noun pairs used. Interestingly, Baddeley et al. did find that pursuit tracking interfered with another, nonverbal task which clearly involved kinetic and static imagery, the present experiment manipulated type of Janssen’s distinction between spatial and nonspatial imagery can also explain this fading.

In order to test /the possibility that there are separate stores for kinetic and static imagery, the present experiment manipulated type of imagery as an independent variable. Thus there were three types of sentence: high kinetic imagery (e.g. ‘The damaged plane plunges onto the grey tarmac’); high static imagery (e.g. ‘The pretty woman sunbathes on the sandy beach’); and low imagery (e.g. ‘The secret information influ- ences the illegal doctrine’). The effects on these of two different inter- ference tasks, one visual and one auditory, were investigated. The basic prediction is of an interaction between imagery-type and interference task, though the exact nature of the interaction will depend upon whether kinetic and static imagery are affected in similar ways by the visual interference task. Since the interference task wzs one involving movement (mental paper folding) one might expect it to prove more detrimental to kinetic imagery sentences.

The experiment

Method

Design The experiment used a 3 x 2 x Ss randomised block factorial design using

repeated measures. The factors were imagery (kinetic VS. static vb. low as rated by a panel of judges), and interference task (visual vs. auditory). Each S was presented with a mixture of sentences from each of the three imagery types followed by the fi*zst interference task, and then a second series of sentences followed by the other interference task. After each part of the experiment, memory was tested by speed and accuracy of sentence recognition using tachistoscopic presentation.

T-et sentence; 72 scmtences of similar length (seven or eight words) were constructed, all of

which fiked into the syntactic frame: the t adjective + noun + verb + (preposition) + article + adjective + noun. The experimenters judged that there were approximate- ly equal numbers of kinetic, static and low imagery sentences. All the sentences were ratbd by ten undergraduate students at Plymouth Polytechnic on two seven- point Ns; one scale measured the degree of imageability of each sentence, the

S. E. Newstead, S. Blackteyfhagety and sentence memory 459

other the degree of meaningfulness and comprehensibility. The 24 sentences selected for use in the main study were the eight rated most highly imageable from both the kinetic and static groups, and also the eight sentences given the lowest imageability ratings. The imagery ratings and comprehensibility ratings for these sentences are given in table 1.

Table 1 Mean imagery and comprehensibility ratings for sentences of each imagery type.

Imageability ratingS

Comprehensibility ratinas

Kinetic sentencea 6.1 6.1 static sentences 6.0 6.0 Low imagery sentences 1.4 3.5

T-tests revealed no significant differences between kinetic and static senbnces on either imageability or comprehensibility ratings (p> 0.3) but both of these were rated significantly higher on both scales than the low imagery sentences (PC 0.001).

24 t$pes were prepared, each containing all 24 target sentences. The sentences were divided randomly into two groups of 12, each containing fear sentences of each imagery type. Recording was done by a male speaker using flat intonation. There was a gap of approximately one and a half set between sentences.

Distractor sentences For the recognition trials, two sets of dqqtractor sentences were used. 24 Related

Distractor sentences were constructed by changing the two adjectives in each of the target sentences. For example, the sentence ‘The damaged plane plunges onto the grey tarmac’ was changed to ‘The burning plane plunges, onto the black tarmac’. 24 Unrelated Distractor items were obtained b;r ta5ng eight sentences of each imagery type from those sentences rated for imageability and comprehensibility but not selected as target sentences.

Interference task8 The visual interference task involved ‘mental paper-folding (Shepard and Feng

1972). This involves working out ‘in the head’ whether two sides of an unravelled paper cube (e.g. the two arrowed sides in fig. 1) will meet if the figure is re-folded. This task is generally agreed to involve complex visual imagery.

Thirteen such cubes were used, two to familiar& the Ss with the task, the remainder to constitute the interference task itself.

The auditory intefference task involved counting out loud backwards in threes from a three-digit number given by the experimenter.

S. 6: Nmstead, S. Ehckky/hnagety and sentence memory

. . . . . . . . . . . . . . . . . ._

: . . . . . . . . . . . . . . . . . .

a.._...............: A

Fig. 1. Umavelkd cube used in the visual interfercncc task.

The 24 Ss were male and female students from Plymouth Polytechnic.

Procedure Ss were instructed concerning the general pi’ocedure and also the interference task

that was to be given f&&They were told that they would be presented aurally with a list of sentences, following which they would be required to carry out another task (i.e. the interference task). The nature of this task was exphuned to them, and two practice trials were run. Ss were told that after this second task, they would be presented, tachistoscopiczdly, with another series of sentences, and their task would be to press a key by their left hand if the sentence was ‘old’ (one they had seen before) and the right ham. key if it was a ‘new’ sentence.

Ss were then presented with the fTt group of 12 sentences, followed immediately by the appropriate interference task. Half of the Ss received each task first, and both tasks were continued for one min. Immediately after this the reco@tion trials were presented, which consisted of 36 sentences - 12 Target SenteWes, 12 Related Distracters and 12 Unrelated Distracters in random order. Eoch sentence was presented on a two-field tachistoscope for eight sec. and the Ss’ time to sake a response as to whether the sentence was ‘old’ or ‘new’ was recorded. The Ss were then given the instructions for the interference task they had not yet received, and the whole procedure was repeated for the second group of sentences.

S. E. Newstead, S. Bhckl&nagery and sentence memory 461

Results

The mean recognition times, ignoring errors, are given in table 2. An analysis of variance on these scores revealed a main effect of imagery type (F2.46 = 5.34; pC5.01) and a significant interaction between imagery type and interference task (F2,46= 6.48; p-C 0.005). A breakdown of the main effect of imagery type showed that there was a significant overall difference between static and low imagery, but not between either of the other pairwise comparisons. The interaction effect was shown to be due primarily to the kinetic imagery condition wb’ch produced signif- icantly worse recognition under thevisual interference task (Fl 69 = 4.35; p< 0.05). Analysis of each interference task alone showed that for the v&al interference task there was a significant main effect of imagery (F2.46 = 5.47;p< O.Ol), due to the responses in the kinetic condition being significantly slower than those in the static. For the auditory interference task there was again a main effect of imagery (F2,46 = 6.28; p< 0.01) due to the low imagery condition producing significantly slower responses than either kinetic or static imagery.

Table 2 Hean recognition ,times (ii seconds) as a function of imagery type and interference task.

vlzudt8ak 3.47 3.03 3.21 Auditory task 3.06 3.09 3.47

Mean number of errors made by each S is presented in table 3. There was a similar trend in the data to that found with the recognition times, but none of the effects were significant.

Table 3 Mean number of errors made by each subject as a function of interfenwe task and imagery type.

vhal int8rferenca task

Auditory interfwence task

Kinetic static LOW

1.00 0.71 1.00 0.67 0.75 1.08

Discussion

The results all seem to point to the conclusion that the visual task selectively interfered with kinetic imagery sentences while the auditory task had a small but nonsignificant interference effect on the low

462 S. E. Newsted S. Bhzckleyflmagev end sentence memov

imagery material. This is suggested by the significant interference task x imagery type interaction which is caused largely by the poorer ~cog&on of kinetic imagery sentences in the visual interference condition. Consistent with this, if the visual task is considered alone, the kinetic imagery sentences proved significantly harder to recognise than the static sentences, with low imagery sentences coming some- where between; while for the auditory interference task, low imagery sentences were recognised worse than either kinetic or static sentences. This latter fmding is consistent with the auditory task interfering selectively 4th the low imagery sentences, though it must be remem- bered that there is no significant difference in the recognition times for these sentences under the two interference tasks.

The main effect of imagery is somewhat difficult to interpret, given the interaction, but is certainly consistent with the above interpretation. Static imagery suffers selectively from neither interference task, hence one would expect recognition to be faster for these sentences than for the other sentences. This is indeed the case, though the effect is only significant with low imagery sentences. This pattern of results is quite easily understood if one assumes, with Paivio, that low imagery sentences are inherently difficult to remember anyway. This, plus the fact that they are also subject to selective interference in the auditory task, would explain their low recognition scores overall. Kinetic sen- tences, on the other hand, do at least have the advantage of high imageability, even though they are subject to disruption from one of the interference tasks; hence their overall recognition scores are some- where between static and low sentences. Another possibility apart from this Paivio-type explanation, is in terms of comprehensibility. In the preliminary study, it was found that low imagery sentences were also rated low on comprehensibility, and this could explain why they proved difficult to remember. Notice, however, that comprehensibility cannot explain the interference effects outlined above, since the same sen- tences were used in both interference tasks.

At first sight, these results seem quite consistent with Paivio’s ‘dual- coding’ hypothesis. Certainly, there is strong support for the use of a visual code, at least for the kinetic imagery sentences, since these were selectively interfered with by a visual task. One could also argue that the results give tentative support for the use of a verbal code, since the low Imagery sentences gave markedly (though not significantly) slower recognition times under the auditory interference task. This lack of a

S. E. hkwsteud, S. Blackleyhnagery and sentence memory 463

significant auditory interference effect could perhaps be. explained by the nature of the interference task used; since this, was articulatory rather than purely verbal in nature, one might not expect it to strongly disrupt verbal coding,

However, problems arise for the dual-coding hypothesis when we consider the static imagery condition, since as far as we can tell from this experiment, neither interference task had any selective effect on this. Hence although static sentences are rated high in imageability, they are not selectively affected by the visual interference task; in fact, performance with these sentences is uniformly good throughout. In order to retain Paivio’s dual-coding hypothesis, it would seem necessary to postulate the existence of two distinct imagery codes, one for kinetic and one for static images. If this is the case, the present results become quite understandable; it is to be expected that the kinetic imagery code would be the only one affected by the visual task used in this experi- ment, since this tas’k (mental paper folding) is clearly one that involves movement. If a task involving purely static imagery could be devised, one might expect this to interfere selectively with the static sentences, though no such task was used in the present experiment, and it is difficult to determine what such a task might be.

Such modifications do seem to rescue Paivio’s hypothesis, though only at a price. The suggestion now is not one of dual-coding but of tr@Ze-coding, and so the theory loses some of its elegant simplicity. In addition one might have to postulate the existence of other visual codes besides the static and kinetic ones. Janssen (1976), as we have seen, has suggested that one can separate images into spatial and non- spatial ones, so arc we to believe that there are separate codes for each of these? It would appear that Janssen’s distinction is different to the one being made here between kinetic and static images, since spatial images are usually involved in both of these. Thus, for example, the static sentence ‘The pretty woman sunbathes on the sandy beach’ clearly involves spatial relationships, in just the same way as does the kinetic sentence ‘The damaged plane plunges onto the grey tarmac’. If one needs to postulate more and more different kinds of imagery code to explain the experimental evidence, then Paivio’s hypothesis becomes less and less plausible.

Currently the main alternative to the imagery approach is the propositional one (Anderson and Bower 1973 ; Pylyshyn 1973). The basic tenet of this approach is that all sentences are stored as elemen-

464 S. E. Newstead, S. B~ckhyjhagay and sentence memory

tary propositions. In order to explain the interference effects obtained in the present experiment it is necessary to postulate that the paper folding task also ,mvolves propositional analysis; and furthermore that this analysis is such that it interferes with the propositions underlying kinetic sentences to a greater extent than it does those underlying static sentences. Palmer (1975) has suggested one way in which tasks similar to mental paper folding might be given propositional representations. As applied to the paper folding task, his claim would be that the ..rrow- heads are represented propositionally in terms of their &stance and direction in relation to the point about which the folding is to take place. Each fold that is made involves a change in these propositional relationships and the updating of a monitor which keeps track of these changes. The more changes; that have to be made, the longer the task will take - hence Separd and Feng’s (1972) result that the further the two edges of the cube had to be moved the longer the mental paper folding task took.

The main problem with an approach such as Palmer’s is that it explains too much: using a post hoc propositional analysis such as his, any results allegedly involving mental imagery can be explained. However, it is a poor theory that can explain any experimental out- come. The explanation in terms of mental imagery could - and did - predict the paper folding results on u priori grounds. A second problem with Palmer’s approach is that it cannot explain why mental paper folding should interfere selectively with kinetic sentences. No current propositional theory would represent kinetic sentences in a way that is different in any systematic way to that given to static sentences. Hence why should paper folding interfere more with kinetic sentences than with static ones?

Thus it seems that the explanation in terms of multiple imagery codes is preferable to a propositional explanation. Against this, it has been claimed recently that one can never distinguish experimentally between imaginal and propositional forms of representations. Anderson (1978) makes the point that an experiment can only investigate a representa- tion-process pair, where the representation is the format in which information is stored (e.g., pictorially or propositionally) and the process is the operation(s) carried out on the representation (encoding, transforming and decoding). Any results can be explained in terms of the processes involved rather than the representations. Anderson’s argument is quite convincing with respect to the experimental situations

S. E. Newstead, S. Bfackkyhnagety and sentence memory 465

he considers, but it fares less well with the results from modality- specific interfehnce tasks. (It is interesting to note that both Anderson (1978) and Pylyshyn (1973) virtually ignore such tasks.) For instance, in order to explain the present results, it would have to be maintained that there is something in common between the processes involved in storing kinetic ‘sentences and those involved in mental paper folding that is not shared with the processes involved in storing static sentences. It is difficult to see what these processes might be. Furthermore, it is obviously more economical to explain the results in terms of an imagina3 representation alone rather than in terms of a complicated represerBtation-process pair.

A further distinction is sometimes made between representation and content The content refers to the kind of information that is stored rather than the format in which it is represented. With respect to the present experiment, it might be claimed that subjects encoded informa- tion about movement both when remembering the kinetic sentences and when carrying out the mental paper folding task. This can explain the present interference effects in non-imagery terms, but once again it seems very much a post hoc explanation, and it also suffers frcm rather serious problems of its own. The most serious of these concerns the way in which information about movement is stored. If it is stored as part of the proposition itself, then we are left with a theory such as Palmer’s, which as we have seen has difficulty in explaining the present results. If on the other hand movement is stored as some kind of ‘tag’ or ‘marker’ which is quite separate from the proposition, this is clearly unsatisfactory since it violates one of the basic tenets of the proposi- tional approach - that all information is represented in propositional terms. Thus neither the representation-process distinction nor that between representation and content seem to provide good grounds for rejecting the imagery explanation.

A general criticism of the interference task paradigm used in this experiment has been put forward by Phillips and Christie (1977). They point out that any interference effects obtained may be due to competition for general purpose resources, rather than for any specif- ically visual processing resources. This point is a valid one, and certainly the present results do not exclude the possibility that some of the interference effects are attributable to competition for general resources. However the fact remains that, over and above these general effects, there is a specific interference by the visual task on the visual

466 S. E. News&xi, S. Bhckleyfhagety and stntznce memory

imagery sentences, and clearly this requires explanation. And, by far the simplest and most attractive explanation of the present results is that there are two distinct imagery codes, one kinetic and one static, each of which can be selectively disrupted by at, appropriate inter- ference task.

References

Amlemon, J. R., 1978. Arguments concerning representations for mentai imagery. Psychological Revkw 85.249-277.

Adsson, J. R. and G. II. Bower, 1973. Human associative memory. New York: Winston and Sons.

Atwaod, G. E., 1971. An experhnental study of visual Imagination and memory. Cognitive Psychology 2,290-299.

Baddeley, A. D., S. Grant, E. Wit&t and N. Thompson, 1975. Imagery and visual working memory. In: P. M. A. Rabbit and S. Domlc feds.). Attention and performance V, pp. 205- 17. London: Academic Press.

Begg, I, and A. PaIvIo, 1969. Concreteness and imagery in sentence meaning. Journal of Verbal Learning and Verbal Behavior 6.821-7.

Brooks, L. R., 1967. The suppression of visualisation by reading. Quarterly Journal of Experi- mental Psychology 191,289-99.

Bra :b, L. R, 1968. Spatial and verbal components of the act of recall. Canadian Journal of PsychoIogy 22,349-68.

Goldtarb, C., J. Wirtz and M. A&field, 1973. Abstract and concrete sentences in false recog- nition. Journal of Experimental Psychology 98.25-30.

Holmes, V. M. and r. Langford, 1976. Comprehension and recall of abstract and concrete sentences. JoumaI of Verbal Learning and Verbal Behavior 15,599-606.

Janmen, W. II., 1976. SelectBe interference during the retrieval of visual images. Quarterly JoumaI of ExpaimentaJ Psychology 28.535-39.

JO~IUO~, M. K.. J. 0-i Bransford, S. E. Nyberg and J. J. Cleary, 1972. Comprehension factors in ~terpretmg memory for abstract and concrete sentences. Journal of Verbal Learning and VerhaIBehavior 11,451_).

KIee, Il. and M. W. Eysenck, 1973.Comprehension of abstract and concrete sentences. Journal OcVeIbrl Leazing and Verbal Behavior 12,522-g.

Moeser, S. I)., 1974. Memory for meaning and wording in conrrete and abstract sentences. Journal of Verbal L&dog and Verbal Behavior 13,682-97.

Rivio, A.. 1971. bnw and verbal processer. New York: Halt, Rinehart and Winston. Palmer, S. E., 1975. ViataI perception and world knowledge. In: D.A. Norman, D. E. Rumelhart

and L.N.R.ResearchGroup(eds.),Explorations in cognittion.!hn Francisco: W. N. Freeman, Dhinipa,, W. A. and D. F. M. Christie, 1977. Interference with vlsualisa~ion. Quarterly Journal

of ExpeahnentaI Psychology 29,63?-50. tiWh~Q, 2. W., 1973. W%at the mind’s eye tells the mind’s brain: a critique of mental imagery.

l%y&obgial BuBetIn 80,1-24. sodu, 1. S., 367. Rewgnition memory for syntactic aspects of connected discourse. Percep-

tirjfiand Psycbophysica 2.437-42. slrron, R. Y., 1971. Interfering images at sentence retrieval. Journal of Experimental Psychol-

ogy 89.56-62.

S. E. Newsread. S. Bklckley/hage~ and sentence memory 461

&son, R. Y. and P. Fraisse, 1972. Images in memory for concrete and abstract sentences. Journal of Experimental Psychology 94,149-S.

Shepard, R. N. and C. Feng, 1972. A chronometric study of mental paper folding. Cognitive Psychology 3,228-43.