Acta Psyehologicu 33 Attention and Performance III (A. F. Sanders, ed.) 1970,280~292 0 North-Holland Publishing Comparvy, Amsterdam

ON SELECTION IN VISUAL IMMEDIA ORY

J. M. VON WRIGHT

Institute of Psychology, University qf Turku, FWnd

ABSTKACT

The efficiency of the selection of items from visual immediate memory (sensory storage) according to different criteria was studied. Selection on the basis, of simple and general physical characteristics was efficient, * the greater the number of distinctive features to be taken into account, and the less the discriminability between or the uniformity within the categories to be distinguished, the more difficult was selection. The level of performance was determined mainly by the efficiency (speed) of the selection process, and was comparatively independent of stimulus characteristics which, though they had to be processed and reported, did not affect selection. The results are in agreement with serial models of information processing. The retention of rejected items is discussed.

1. INTROJECTION

A distinction is usually made between two kinds of stages of short-term memory for visually presented material: a sensory or pre- perceptual storage which. decays rapidly with time, and a post-percep- tual storage of subzt:~tial!y Ionger u +ration. The former will here be &.led VIS (visual Infczmation storage). Several experiments (e.g., A’JERBJKEI and SPERHJNG, 1961) have &own that the maximal time for storage of information in VIS with a bright post-exposure field is approx’imately 300 msec.

A technique for the study of information available in VIS has been developed by SPERLWG (1960). The subject (S) is shown an array of items fachistos~pically and instructed to report a part of the array, the signal specifying the part to be reported being given after the termination of visual pnzentation (as a post-stimulus cue). This method samples the informati’on that S has available when the selection signal has been processed. The total amount of information available can be estimated from the sample. This estimate, called the parfinE report,

* This research was supported by the National Research Council for the Social Scierxes, the Foundation of Signe and Ane Gyllenberg, and the Foundation of Jenny and Antti Wilhuri. I am grateful to Mr. Pekka R. Lehtiii, Mr. Keijo Tahkokallio and Mr. Ka3 Sundman for their generous help.

280

SELECTICIN IN MSU DIATE 281

rts provides an estimate of the

easure of the efficiency c4 the

by color and to report item size, or to select by size and report

e present paper is concerned with three empirical problems: (1) What is the relative efficiency of selection according to different selection criteria when the response criterion is constant? This question differs in one important respect from that investigated in a long series of experiments, initiated by M~~LPE’s (1904) study, in which Ss have been asked to attend to and report selectively various characteristics of a stimulus array. In the latter, se1 ion and response criteria were identical, e.g., S had to select and rep colour, or ,to select and report shape, etc. Thus the response criterion has varied from condition to condition, which makes intercondition comparisons difficult. In the present experiment 1 the response criterion is constant over conditions; only the criterion according to which letters are selected is varied.

er to report an item correctly, S must process at least two etied by selecti nd by response criteria. How is per-

formance aEec;tcd by the o r in which two given features arc processed when the discriminability of one feature is varied and that of the other one kept constant? (3) What, if anything, does the subject learn about the non-selected items?

2. EXPERIMENT 1

In a previous paper (VON WRIGHT, 1968a) the author has reported results from an experiment in which selection by locatlon, colour,

282 3. M. VON WRIGHT

brightness, size, and orientation ‘were compared. In the present experi- ment three additional selection criteria were used: selection between letters and numbers, between vowels and consonants, and between letters and mkror-images of letters. The method and procedure were described in detail in the earlier paper; here they are only briefl summarized.

2.1. Meikd The apparatus was a two-field mirror tachistoscope. Viewing was

binocular at a distance of 50 cm. The fixation field was dimly ilIumi- nated wi& a fixation point in the middle. S was instruc the fixation point until it was clearly in focus; he then pressed a button which itiitiated the presentation. The exposure time was 106 msec. Five msec after the termination of exposure the selection signal was delivered to the subject through earphones. The signal consisted of a short piIJ of either 1500 cps or 300 cps. (In part of the experiment a pip of 700 cps was used to indicate ‘whole report’.)

The stimuli were lettered 20 X 16 cm cards with two rows of four letters each. The letters were capitals, 10 mm high, the distance betwen the midpoints of any two adjacent letters being 15 mm.

Ex~ condition included (a) series of expositions in which S was instr~tti ta report as many letters as possible (whole report), and (b) serk of expositions in which only letters specified by the selection signal were to ‘be reported (partial report). In the latter case the partial report score was obtained by multiplying the proportion of correctly reporkd letters out of all letters in the category to be selected by the tot&l number of letters on the card (cf. SPUILING, 1960).

Responses were recorded on specially prepared response sheets whkh were placed on the table immediately below the tachistoscope. S was instructed to write down the letters immediately on hearing the choke signal, the room illumination being adequate for writing.

Ss were undergraduates, paid by the hour. They were all well practiced in this type of task. Each S served as his own control, i.e., the difference between whole and partial reports was determined for each S =parately on the basis of at least 30 trials in each condition, (For further d t ‘1 e ar s o procedure, see VON WRIGHT, 1968a.) f

2.2. Experimental condition

l’hree CondifioIls were used to suppllement the five used in the previous experiment:

EMORY 283

s of items was ran

of the items on each f numbers. The location of the

d the low tone, and the s condition .,~as previously

alf of the items on each ns~na~ts, the other half vowels.

ifkrent from their mirror

here.)

way round’. The partial report scores repo

those trials in which mirror selected were substantially lower than those reported

2.3. e&ts and discussion

The results arc summarized in table 1 together with those obtaine

TABLE 1

Average whole reports and partial reports for di rent selection criteria in experiment 1 . The II%U&S from an earlier experiment (VAN WRIGHT, 1968a) are included for

comparison.

Condi- tion Selection criterion N Ils ---__l_p__-I_r_^ .-.. ___,__~_. ._. ,...,. -. _. ___ .__ _

Partial Whole report report Diff.

Y Location 5 5 6.5 4.0 2.5 II Colour 12 11 5.7 3.5 2.2 III Size (10 mm vs 6 mm) 4 4 4.9 3.4 1.5 IV Brightness (reflectance

8% vs 23%) 4 3 4.9 3.6 1.3

VA. Orientation 0” v#rs -- 45” 4 -_ 4.0 3.8 0.2 V.B. Orientation -45” vs + 45” 4 - 3.3 3.8 - 0.5 ‘VI Letter vs number 10 2 4.0 3.7 0.3 Vl?I Vowel vs consonant 6 1 4.0 3.9 0.1 v19 6 ‘Normal’ letters from

‘normal’ vs mirror im. 4 - 3.9 3.7 0.2

ns = r;umber of Ss for whom the partial reports were consistently and significantly better than the whole reports.

284 J. M. VON WRIGHT

h the earlier paa of the experiment. It CXUI be seen that in CO~SOES

1-1~ tie .padd report scores are in most instance signifkantly higher than the mnesponding whole report scores. I.e., Ss were able to se items f&]y ,g:ff&ntly by location, colour, brightness, and size. This i&ic&es that S were here, like in studies of selective listening, able to select or ignore items efficiently on the basis of the general physical cbar,acte&tics of the items. However, the ease of selection is obviously dependent on the disc;iminability between the: actual values of the variables used as selection criteria. A pilot study with different si pairs id&ted that the ease of selection by size is a monotonic function of the dierence between the sizes involved.

The results are characteristically different for conditions V-WI. With two exceptions, d.iscussp;d below, the subjects were not able to select by orientation, or between letters and numbers, vowels and consonants, and ‘letters and mirror-images of letters: in the great majority of cases the corrected partial report scores are no better than the whole report scores. The reason for this is presumably that in these conditions v

selection requires that each item is analyzed in full before S can determine whether it is to be selected or rejected. In conditions I-IV

S had to select between two locations, two colours, two brightnesses, or two sizes. Tn conditions VI-VII the categories to be selected were less unYorm: the number-category contained 8, the consonant-category 20, and. the vowel-category 6 different shapes. Letters and numbers, like consonants and vowels, and like letters and, their mirror images, are comparatively similar as visual forms, and there are no simple and obvious features on the basis of which the: two calegories could be distinguished from one another. Correspondingly, it seems likely that in condition v the letters had to be extensively processed before their orkatation could be determined, si:nce letter-shapes contain a variety of Nferently oriented parts.

Efowever, two Ss were able to carry out efbcient selection in condition VI, and one of them in con&ion VII as well. (Individual differences in the skill. to select were, on the whole, highly consistent eve:’ (conditions.) These Ss had had more practice than the others in visual selection and search tasks. In studies on visual search it has been found that although search time usually increases with an increasing number of target items (e.g., KAPLAN and CARVELLAS, 1965), extensive training may take it possible for Ss to search for several items as fast as for one item (P~EISSER et al., 196:)). In visual search S moves his

SELECTION IN VISUAL IM IATE MEMORY 285

k could be described as ‘search 11, the two types of tasks have several

nt variables ins tistigated seem oh tasks similarly. It may therefore be

t items by several ‘features’ simul- nough and when motivated to owzver, no definite conclusions

is known about the determinants selection efficiency.

in dichotic listening experi- it requires detailed identification

e simpler th:z mber of items

essing within the limited time

is is also reflected in the comments made by Ss. For instance, when selection was by colour, trained Ss typically reported that when they heard the selection signal, the letters specified by the signal seemed to stand out and to form a definite visual pattern. In conditions V-VIII,

on the other hand, the items within any category to be selected were not experienced to have the same kind of unity, nor tu form an immediate spstial pattern. These observations lend some informal support to the notion that - at least wh selection is easy - the initial scan of what i:; stored in VIS serves to cify the location of tale items to be further: processed. The overall spatial pattern formed by the s&cted items Seems to form a frame of reference for the determination of the precise position of individual items.

experiment 2 selection of shape ;by colour and selection of colour by shape is compared, the number of shapes and colours being equal, i.e., two, A basic feature is that whlether S selects by shape or bq colour, he always has to make the sar.ne two binary decisions for each uccepted item and to make the same kind of overt response; only the order of the decisions is varied. A second independent variable is the discriminability of the two shapes involved, colours being constant.

In the experiment letters were used as shapes. An additional pilot experiment indicated that sim&ilar results are obtained when simple geometrical forms are substituted for the letter-shapes.

2815 J. M. VON WRIGHT

3.1. Method me apparatus and general features of the experiment were the same

as in experiment 1. The stimuli were lettered 20 X 16 cm cards with four rows of four letters each. In each condition two COPOUTS, blue and red, and two letters were used. Four letter pairs, chosen on the basis of pre-experiments to represent different degrees of discriminability (cf, also KUNNAPAS, 1966) were us condition I: X and 0 (maximal scriminability ) ; condition II: E and S; condition III: 0 and LJ; condition IV: E and F (minimal discritinability). Thus, in condition I each stimulus card contained 4 red X:S, 4 blue 4 red 0:s and 4 blue 0:s, randomly intermixed.

Recording of reports. n order to insure that the reports involved the same kind of response in each condition, the response sheets had the same size as the stimulus cards ‘aud contained four rows of four 10 x 10 mm squares, spaced in the same way as the letters on the stimulus cards. Each square was divided by a diagonal line from lower left to upper right. When selection was by colour, one of the two letters involved (X in condition I) wa printed above the diagonal in each square, the other letter (0) below the diagonal. When selection was by letter, the area above the diagonal in each square was red, the area below it being blue. S was asked to mark she appropriate half of each square which corresponded to a selected c,olour (letter). For instance, when selection was by colour and the signal specified ‘red’, S was to make a mark in each square corresponding to a red letter, to show whether ,the letter in question was an X or an 0.

Preserzfatiom. Each condition included four types of presentations which will here be described for condition II:

(a) Report ~j (umwkcted) colows. There was only one type of signal (post-stimulus cue), the letter-shapes being irrelevant. S was asked to mark, on a response sheet with red/blue squares, for each item whether it was red or blue.

(b) Report of (unselected) Jettem !ere was only one typ.r of signal, the colours being irrelevant. S ‘was asked to mark, on a response sheet with squares containing letters X/O, for each itern whether it was an X Or iSIlO.

(d ~~hkwz hY mbur. S was given the following instructions: “Ibe high selection signal means red. When you hear the high signal,

SELECTION IN VISUAL lh4 287

whether it is he low signal nleans e sheets were us as with presentation (b).

) Selection by letter. S was given instructions to the effect that signal he should report for each X whether it

e heard the low signal he should do like- e same response sheets were sed as with

er respects the pr~edure was the same as in experinlent I.

ary of th= main results is presented in table 2. It can bz at Ss on the avera e reported the colour of

TABLE 2

Experiment 2: average reca.h of (a) unselected colours, (b) unselected letters, (c) letters selected by colour, and (d) colours selected by letter.

(a) W G-3 (d)

Unselected Unselected Diff. Letters Colours Difi’. I’,z

colours letters I11 I selected selected (c-c) (a-b) I by colour by letter

XI0 4 i 9.6 9.2 0.4 - 3.4 3.4 -- - /

L’s 5 1

9s 9.1 0.4 .- 3.7 3.2 0.5 2 O/U 5 9.8 8.2 1.6 2 3.3 2.8 0.5 3 E/F 4 ] 10.1 6.9 3.2 2.3 0.9 4

nl = number of Ss for whom the difference (a-b) is significant, i.e., who reported significantly more unselected colours than letters.

na :=7 number of Ss for whom the differena: (c-d) is significant, i.e., who reported significantly more letters (selected by colour) than colours (selected by letter).

approximately ten items correctly. Most S:; stated that they did this by retaining an impression of the over-all pattern of one of the colours. Column (b) in the table shows that Ss coul identify a similar number of letters-in-position when the letters were easy to discriminate (condi- tions X/O and L/S); again, they usually stated that they remembered the spatial pattern formed by (one of the) letters. However, the more dticult the letters to discriminate from each other, the lower was the score. The inter-individual variance was considerable, hence straight- forward comparisons can only be made in respect to the intra-individual differences between recall of c&ours and recall of letters. This dif-

288 J. M. VON WRIGHT

fereuce &d not approach statistical signihcauce for any S in condition r (X/O) whereas it was sign&ant for each. S in condition IV (E/E;).

selection of 1e”ters hy c&zcr (co1um.n (c) in table 2) WB almost equdy eficlent in each condition: it did not matter much whether the letters to be reported were easy or difficult to discriminate f other, The efficiency of the select&? of dours by lette other hand, varied as a function 4 ,the discriminability between the letters, Thus it was a5 easy to use the letters X/O as selection criterion as it was to use the colours red/blue, whereas selection between :s and F:s was signifkantly ,inferior to selection by colour, the difference (cd) being statistically significant for each subject in condition tv (see table 2).

(The scores til table 2 are raw scores: in order to make the figures in columns (c) and (d) comparable to the partial report scores in table 1, they should be multiplied by 2. The fact that these partial reports, even when corrected, are lower than the ‘whole reports’ shown in colilmns (a) and (b) is due to the unusual nature of the latter conditions: only one dichotomous decision had to be made for each item, whereas conditions (c) and (d) required two such decisions for each accepted item. A whole report condition in which the Ss had to report both colour C& shape of each item was also included in the experiment. These whole reports were on the average 3.2 items, i.e. approx. 50 5% of the corresponclitlg average corrected partial report scores.)

In order to evaluate the influence of guessing responses, the pcr- centage of wrong responses out of all responses was computed for each case. This percentage varied considerably from one S to another but was fairly constant over conditions and types of presentation for each 6”. Thus any conclusions reached on the basis of the difference scores in table 2 (a-b, and c-d, respeotively), which were computed separately for each S, are not likely to be influenced by this factor.

men interpreting the results it should be remembered that in order to report an item correctly in the seleotion situations {conditions c odd d) S had to analyze both its colour and its form: within both conditions the same two binary decisions had to be made for each accepted item, irrespeotive of whether selection was by colour or by letter= mus, if each item had been fully analyzed before a decision was made to accept or to reject it, selection by colour and selection by form should. have been equally efhcien,t.

The results support the assumption that a two-stage process in

DIATE MORY 289

of selection was const.ant ount of letters was reported irrespective

rs in the stimulus array

4.. N $RB RETEN'rfON OF NON-SELECTED ITE

The problem of the extent to which non-selected items are processed ~II the present type of selection tcsk is not altogether easy

cause of the ditEculty to design sufficiently sensitive tests of retention. (A slight change of the task would make it possible ao

technique used by LEHT@ 1970.) In some pilot experiments rent types of tests were tried. 0 well practiced Ss were

used, since untrained Ss frequently make se tion errors which make it difficult to decide what, precisely, sho be regarded as a ‘non- selected item’.

(a) In a study of seleetion of letters by colour, digits were occasion- ally substituted fos half of the background ers, i.e., letters of the

,ich was to be rejected. A some similar technique has by Neisser in studies of visual segrch. The result was the

same as that obtainr;d by NEISSER (1967). When questioned imme- diUely after the presentation, Ss were not 0 inadly aware of anything but letters on the stimulus card. Wren ar, item of a wholly irrelevant colour was introduced, e.?., a black letter on a card used in selection between red and blue items, Ss often noticed “omething odd’ but could not give a de&rite description of what it was.

(b) In another experiment on selection b’ colour, the background a

items were kept constam - the same letters b&a; presented in the same

order - during trials ci through 23, after which these letters were present-&i as target items (letters of the ciiiour to be selected) on

290 J. M. VON WRIGHT

teal 24. me performance of Ss on trial 24 was, on the average, no better ‘han their performance ?n trials l-5 or 26-30 in which backg&und items wefe continuously varied. (This experiment has been reported in detail by VON WRIGHT, 1868bJ

(c) In a third experiment on selection by colour, a separate sound- for whole reports and the selection signal was omitt six Ss towards the end of a partial report series duri ntal session. Instead of the selection signal, the wh

report signal was given after a delay of 500 msec. On this trial the average zotal report was 0.5 letters, which may be compared with the average raw score of 2.6 letters for the ten preceding partial n-ids and with the average score of 3.4 letters in a subsequent report se,ries. Although th.is result is open to many different inter- pretations, it at lleast agrees with the description given by several Ss that pi&zessing of the items - the read-out from VIS - was ‘suspended’ until the signal arrived. When the whole report signal arrived 0.5 YCC after presentation, the unprocessed information in VIS was ‘out of reacv already.

(d) In one experiment Ss, instead of writing down their reports, took a recognition test immediately after each presentation. They were given a list of 8 letters, which on partial report trials consisted of the four target item.s, two background items and ‘two irrel<zvant items, and on whole report trials of six target items and two irrelevant items. Their task was to mark the target items. During the final session of the experiment each S was on two different occasions given the whole report signal - which indicated that all letters on the card should be reported - 0.5 set after ltbe usual partial report signal. When the initial task was to select by eolour, the number of background items correctly reported (recognized) did not exceed the number of irrelevant items reported. When selection was between consonants and vowels, some- what more background items than irrelevant items were reported, but the difference did not approach statistical significance+

NO definite conclusions can be made on the basis of these pilot studies. However, we failed to obtain evidence for learning effects relating to the non-selected (rejeoted) items as long as selection was easy and Ss well trained.

In the discussion of the results of experiment 1 it was argued that seleotion between consonants and vowels - like that between letters and nnmhrs and between letters in different orientations -.. i:s dacult

ully analyzed before R decision e to accept or to reje~~ it.. this argument is valid, Ss woul

more rejected items when selection

t.he decision to

are in many respects similar to those obtained in Ve listening (cf. TREISMAN, 1365, p. 106): $s are able

to select or to ignore items efficiently on the basis of simple and general physical characteristics, whereas selection is difficult when it requires a more detailed identification of the items presented. number of distinctive features to be taken into account: the simiEarity between the categories to be distinguished, the morz diticult is selection.

Intra-individual. variation in the level of performance in the present type of task seems to be largely determine by the ease of the s&ction process. When the discriminability between the selection categories varied, performance varied accordingly even when the discriminability between the selected items was constant. Conversely, when the discriminability between the selection categories was constant, per- formance level remained fairly constant even when the discriminability between the selected items varied. These results, together with the fact that there was no indication of a full analysis OE ihe rejected items when selection was easy, support ahe assumption that processing was al least. partly serial> the analysis by the selection attribute preceding tht: analysis and identification of the response attributes. When selection is easy, the initial scan of what is stored in VIS apparently serves primarily to specify the location of the items to be selected and further processed.

REFERENCES

AVERBACH, E. and G. SPERLING, 1961. In: E. C. Cherry (ed.). Symposium on infor- mation theory. London: Butterworth.

KAPLAN, I. T. and T. CARVEL:AS, 1965. Percept. Mot. Skills 21, 239-243.

292 J. M. VON WRWHT

~ti~m, O., 1904. Ber. 1. Kongr. exp. Psychol. 56-68. WNNLIW, T., 1966. &and. J. Psychol. 7, 189-196. LIWTI~, P. IL, 1970. In: A. F. Sanders (ed.), Attention and performance III, Acta

Psychol. 33, 93-105. NBISSER, U., 1967. Cognitive psychology. New York: Appleton-Century-Crofts.

, R. NO~ICH and R. LAZAR, 1963. Percept. Mot. Skills 17, 955-961.. SPWLMO, G., 1960. Psychol. Monogr. 74, No. 11. TREISMAN, A., 1966* In: B. M. Foss (ed,), New horizons in psych

worth: Penguin Books. ‘VOI?,I WRIGHT, J. M., 1968a. Quart. 9. em Psychol. ?#I, 62-68. ‘--, 1968b. Rep. Psychol. Inst., Univ. Turku, No. 31.