Recall cues in known-item retrieval

Recall Cues in Known-kern Retrieval

Bryce Allen University of Illinois, Urbana, IL 61801

The purpose of this study was to examine the effect of different kinds of cues on the recall of a text. Forty-four subjects read a short research paper. After a delay of three weeks, they answered questions on the paper. There were three types of cues: free cues, which en- couraged unrestricted responses; bibliographic cues, which asked subjects to recall elements of bibliographic description; and structural cues, which were based on the text-linguistic structure of research reports. Recall protocols were analyzed to establish the degree to which protocol vocabulary matched index terms and propositions taken from the article. Biblio- graphic cues produced relatively few matching terms, and short protocols. Free cues produced many matching terms and propositions, but very long protocols containing large numbers of non-matching terms and intrusions. Structural cues produced as many matching terms and propositions as free cues, but significantly fewer non-matching terms and intrusions. Implications of these findings for known-item retrieval and the design of information system interfaces are discussed.

Introduction

Information systems require input from users to serve as the basis for retrieval. To obtain this input, systems ask questions of, or present prompts to, their users. The responses of users depend in part on their needs and back- grounds, and in part on the way in which the interaction

with the system occurs. Belkin, Oddy, and Brooks [ 1,2] have examined the way users respond to very general questions about their “anomalous state of knowledge.” Dervin and Dewdney [3] examine three kinds of questions in the context of the reference interview: open, closed, and neutral.

Information systems, by their index structures and interfaces, place expectations on the kind of input which users may supply. Library catalogues, whether manual or automated, expect users to supply author, title, or subject. Document retrieval systems have expanded the range of possible input by increasing the number of indexed fields. However, the form of expected input is still extremely

Received January 26, 1987; accepted June 16, 1987.

01989 by John Wiley & Sons, Inc.

terse. Currently available interfaces [4] use menus to facili- tate the expression of that kind of input. In other cases, a human intermediary exists to translate more verbose, natural language expressions of user need to the input ex-

pected by the system. With the advent of some level of natural language processing (Boguraev and Sparck Jones

[5], Damerau [6,7], Thompson and Thompson [g] the scope of expected user input has increased in both form and content.

The kind of input which users provide to information

systems depends to some extent on the questions or prompts used by the system. In this article, the term “cues” is used for such questions and prompts. The purpose of this research is to investigate the kinds of user input which are elicited by three alternative forms of cues: open cues, bibliographic cues, and cues based on the text-linguistic structure of literature. The goal is to provide a basis for optimizing the kinds of cues used in information systems

so as to elicit user input which matches in form and content the input expected by the systems.

An example from an actual information retrieval setting may serve to illustrate the differences between these three

types of cues. In ongoing research, the cues have been ex- pressed as different questions on pre-search forms completed by users of online bibliographic search services.

Open cues correspond to general instructions to the user to describe the topic to be searched. Bibliographic cues corre-

spond to instructions to the user to provide keywords and synonyms which describe the topic, as well as any title known to be relevant. These instructions are typical of the kinds of questions normally used by library search services on pre-search forms. Structural cues give rise to a different kind of question, however. The user is asked to imagine an ideal article: one which would be right on topic, and which would meet the user’s information need. The user is then asked questions relating to the text-linguistic structure of that ideal article: for example, what methodology the authors might use in their investigations, and what findings and conclusions might result. In this example the effects of cues in the user-intermediary interaction is illustrated: they provide structure which directs the user’s input to the information system.

In known-item searches the US~Z’S responses to the cues

of the information system depend to some extent on his or

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE. 40(4):246-252, 1989 CCC 0002-8231189/040246-07$04.00

her memory. This study concentrates on a situation in which a user is trying to find an item previously read, and consequently in which the user’s memory of the item is of primary importance. Human memory for text has been thoroughly researched in psychology and education. In- sights from this research provide indications of the kinds of questions which may be asked of the user in order to

take best advantage of memory.

Schemata in Memory for Text

In his seminal book on human memory, Bartlett [9] ex- pounded two propositions which have been influential in

contemporary research in cognitive science. The first is that a person’s past experiences, actively organized into schemata, play an important role in how that person re- members an event. In the research of Bobrow and Norman [lo], Rumelhart and Ortony [ 111, Anderson [ 121, Adams

and Collins [ 131, and Rumelhart [ 141, schemata are de- fined as collections of concepts which form organizing structures for facts. During reading, schemata are activated and instantiation occurs: i.e. slots in the schema are filled

by instances from the text. In work in artificial intelligence, there is a similar idea. Minsky [15] developed the idea of “frames:” data-structures for representing stereo-

typed situations. By filling slots in frames, complex situations are reduced and conventionalized into compact rep- resentations. The idea of schema theory is that, when

reading a document, we reach into our store of experience, and understand what we read by fitting it into what we al- ready know.

Bartlett’s second contribution is that remembering is an imaginative construction or reconstruction of the event. When remembering a document after a delay, we use sche-

mata, plus any salient facts which have become attached to them, to reconstruct the document, Two kinds of experimental results are explained by schema theory. The first is

that people remember elements from text which are important to them (Anderson, Reynolds, Schallert, and Goetz [ 161 or which are congruent with their cultural background

(Steffenson, Joag-Dev and Anderson [ 171). They frequently will remember not only parts of the texts, but also elements which they think should have been in the text

(Sulin and Dooling [18]). The second is that people remember what is important in the text, either in terms of its organizational structure or its content (Meyer [ 191, Meyer and Rice [20], Thomdyke [21]). It appears that there are schemata held in common by members of a linguistic com- munity, which are used to understand textual materials (An- derson [12]). Minsky [22] calls these structures narrative and thematic frames. One narrative frame which has received a great deal of study is the story grammar (Mandler and Johnson [23], Rumelhart [24], Stein [25]). However another type of narrative frame which has been proposed by Kintsch and van Dijk [26-291 has potentially broader application. This is text superstructures: the typical structures of certain kinds of text which are used by readers to guide the understanding and recall of textual materials. An

example is the report schema: the standard form used for reporting experimental results in a variety of disciplines.

Although Johnson [30] reports a general absence of studies relating to cueing in the recall of prose passages, there are some interesting experimental findings in which schema-based perspectives were provided to subjects recalling narrative passages (Anderson and Pichert [31], Fass and Schumacher [32], Flammer and Tauber [33]). Recall of narratives was affected by the schema which was in- voked at time of recall, even if it was not the same per-

spective which was predominant at the time of initial contact with the passage.

Schemata in Information Retrieval

Schema-like structures have been used in some infor-

mation retrieval research. In user interfaces, Pollitt’s CANSEARCH system [34,35] makes use of thematic

frames to provide a guiding structure for user input. In knowledge representation, Rosenberg [36] describes a system which organizes information into episodic contexts to permit efficient retrieval. In automated gist extraction, the

TOPIC system (Hahn and Reimer [37]) makes extensive use of frame structures to generate content summaries of papers. However, there has been no application of text superstructures in information retrieval research, perhaps because these are high-level structures which are largely independent of the subject content of texts. This study in- vestigates cues based on these structures along with open cues and bibliographic cues. Since psychological research indicates that text superstructures are among the schemata

used by people in comprehending and remembering texts, it is hypothesized that cues based on these structures may provide an alternative to other types of cues in information

retrievai.

Methodology

Subjects

Forty-four students enrolled in M.L.I.S. or Ph.D pro- grams at the University of Western Ontario participated in the study. They were paid $7.50 for their participation.

Materials

A short paper on questionnaire distribution was copied from Psychological Reports [38]. The paper discussed a cost-effective alternative to mail distribution of questionnaires which resulted in a very high response rate in a field test. All elements of the paper were reproduced, including the authors’ names, the title, and the abstract.

Experimental Design

Each subject was instructed to read the paper carefully without turning back to reread. They were also informed that there would be a test of their memory of the contents

JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE-July 1989 247

of the paper at a later date. After a delay of 21 days, subjects were randomly assigned to three groups.

The first group received instructions to write down ev-

erything they could remember about the paper. Inclusive-

ness of memory was stressed, to avoid selection-by the subjects of what they thought was relevant. The subjects were asked to use the exact words of the paper if they could remember them, but otherwise to use their own

words. Because of the general nature of these instructions this group is identified as the free recall group.

The second group received instructions to respond fully and accurately to questions posed in writing:

1. Identify the author(s) of the paper.

2. Identify the title of the paper.

3. Write down any keywords which describe the paper.

Because these questions are related to bibliographic description, this group is identified as the bibliographic cues group.

The third group received instructions to respond fully

and accurately to questions posed in writing: 1. What purpose(s) did the author(s) have in writing the

paper?

2. What method(s) were employed by the author(s) in the investigations reported in the paper?

3. What were the results of the method reported by the

author(s)?

4. What conclusions did the author(s) reach in the pa-

per?

Because these questions are drawn from the report schema of Kintsch and van Dijk [27] this group is identified as the

structural cues group. In addition to these different kinds of cues, all subjects

completed a short questionnaire, giving demographic data such as age, gender, previous degrees, and familiarity with the subject area. These data were collected to control pos-

sible extraneous variables affecting the recall results. Also included was a question on how difficult subjects found the

recall task, which provided a subjective dependent variable to supplement the objective recall measures. This question is also similar to those asked in research on the self-assessment of comprehension and knowledge (for example Glen- berg, Wilkinson and Epstein [39]).

Processing of Protocols: Matching Index Terms

Protocols (the written responses of subjects to the recall cues) were broken into single words. Common function

words were eliminated using a short stop list of 28 words. The remaining words were stemmed using an algorithm based on Lovins [40]. The resulting stems were counted, to identify the overall length of the protocol and the frequency of occurrence of the word stems from the protocol. These word stems were then compared with a list of word stems drawn from the title and abstract of the article. The title and abstract word stems are the index entries which

would have been generated for the article in a generic bibliographic retrieval system. Three measures were calculated for each protocol: the number of word stems in the protocol which matched index terms; the number of word

stems in the protocol which did not match index terms; and the cosine similarity measure (Salton and McGill, [41]), calculated using the frequency of occurrence of terms as

the weight for the terms. The title and abstract of the article were used as a basis

for comparison because of the known-item retrieval sce- nario. It would have been unrealistic to check recall of a paper against an actual entry in a retrieval system which contained descriptors not seen by the subjects. However, in order to verify that the use of title and abstract terms from the article did not influence the outcome of the experi-

ment, the PsychInfo record for the article was retrieved.

This record contained descriptors, ibntifiers, and a some- what shorter abstract. All protocols were also compared to the index terms in this record, and the number of matching

and non-matching terms were calculated. T-tests were

used to determine if the mean values from comparisons with the “generic” index terms differed from the mean

values from comparisons with actual index terms. No significant difference was found. It is felt that this result justi- fies the use of the article’s title and abstract as a source for

index terms.

Processing of Protocols: Matching Propositions

The entire article, and all protocols produced by the structural cue or free recall condition, were divided into

propositions using the method described by Bovair and

Kieras [42]. This method decomposes connected discourse into verb frames consisting of a predicate and (typically)

two arguments. The arguments are either nouns linked by case assignment to the predicate, or other propositions. Propositional analysis of this kind makes it easier to com- pare the idea units in prose by eliminating variations in

syntax. Protocols produced by bibliographic cues were not

considered to be suitable for this type of analysis, because they consisted mostly of single words and phrases. In analyzing the text of the article, main propositions (those which are central to the topic of the paper) were distin- guished from other, supporting propositions. Main propo-

sitions were identified by their presence in the abstract, or by the fact that they occurred twice or more frequently in the text. This method of identifying main propositions was chosen as an objective alternative to the more usual method of independent ratings by expert judges. There were 70 main propositions, and 501 other propositions. The propositions from the protocols were matched against

the propositions from the article, using a gist scoring system. As indicated above, propositional analysis facilitates comparison of texts by standardizing syntax. Gist scoring takes this one step further by treating synonyms as equiva- lent. For example, the text contained the clause “each questionnaire was hand-delivered.” This clause is proposi-

tionalized as:

248 JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE-July 1989

Pl: HAND-DELIVER $ QUESTIONNAIRE P2: MOD QUESTIONNAIRE EACH

One protocol contained the clause “all questionnaires were delivered by hand.” This clause is propositionalized as:

P3: DELIVER $ QUESTIONNAIRE P4: NUMBER-OF QUESTIONNAIRE ALL P5: BY P3 HAND.

It is clear that P3 and P5 taken together match Pl, and that P4 matches P2 despite the differences in vocabulary. Accordingly, the protocol clause would be scored as hav-

ing two propositions matching those in the text. This type of scoring introduces a level of subjectivity into the analysis which must be acknowledged. In this study, however, the protocol analysis was used only to supplement the objective matching of keywords and index terms.

Findings

The results reported below were produced by analysis of variance, using the Tukey method for a posteriori comparisons. In certain cases, the data did not fit the assump- tions of analysis of variance, because of non-normal

distributions or heteroscedasticity. In those cases, the analysis was repeated using Kruskal-Wallis ANOVA by ranks, and the a posteriori comparison method described by Conover [43]. In all cases, the nonparametric analysis con- firmed the results of the intial analysis.

Table 1 indicates the performance of the three types of

cueing in terms of producing useful terms for information

retrieval. ANOVA indicates a significant difference in means

(p < 0.01). A posteriori analysis indicates no significant difference between free recall and structural cues, but differences significant at p < 0.05 between bibliographic

cues and both other types. Table 2 indicates the number of non-matching (super-

fluous) terms produced by the three cueing methods. ANOVA indicates a significant difference in means

(p < 0.001). A posteriori analysis shows that all three groups differ significantly from each other. This huge dif-

ference in the number of non-matching terms is one of the most important results of the research, and is discussed in more detail below.

Table 3 indicates the cosine similarity measure calculated between protocol and document index terms.

ANOVA indicates no significant difference in means. This result is also discussed in detail below.

TABLE 1. Number of word stems in protocol matching index terms.

Mean Standard Deviation

Free Recall 7.6 4.0

Bibliographic Cues 3.2 2.1

Structural Cues 6.9 3.7

TABLE 2. Number of word stems in protocol not matching index

terms.


Free Recall 50.8 27.0

Bibliographic Cues 5.9 3.8

Structural Cues 29.7 16.3

TABLE 3. Cosine similarity measure: protocols and document vocabulary.


Free Recall 0.19 0.08

Bibliographic Cues 0.25 0.14 Structural Cues 0.24 0.11

In addition to the above analyses, ANOVA was done treating all demographic variables (gender, age, level of

previous degree, subject of previous degree, library work

experience and familiarity with the subject area) as independent, and the three measures of recall as dependent. The only results which were significant at a = 0.05 were for levels of familiarity with the subject. However further ANOVA indicated that all levels of familiarity were found equally in the three experimental groups. These analyses indicate that demographic variables identified in the research did not contribute to the differences in recall. The differences are therefore attributed to the types of cues provided.

The types of cues also produced significant difference

in the level of difficulty with the task reported by the subjects. Table 4 summarizes these findings, which Kruskal- Wallis ANOVA (used for an ordinal dependent variable) found to be significant at p < 0.05.

A posteriori analysis indicates that there is no significant difference between the free and structural cues, but that the bibliographic cues were rated significantly more difficult (p < 0.05) than both other types.

Table 5 outlines the results of the matching of propositions. The first column gives the average number of main propositions in the recall protocols, the second the number of other propositions found in the text which were also found in the recall protocols. The third column gives the

number of propositions included in the recall protocols not

TABLE 4. Level of difficulty reported for recall task.

Median Mode

Free Recall 2 (difficult) 3 (easy)

Bibliographic Cues 1 (very difficult) 1 (very difficult)

Structural Cues 2 (difficult) 3 (easy)

TABLE 5. Propositions from recall protocols.

Main Other Intrusions

Free Recall 15.0 1.6 18.7

Structural Cues 14.5 7.2 7.4


found in the text of the article. These may result from in- correct recall, from inferences and conclusions drawn by the readers, or from intrusions of the readers’ schemata

into their memory for the text. T-tests on the means reported in Table 5 indicated that

there was no significant difference between the two methods of cueing in the number of text propositions, whether main or other, which were recalled by the subjects. How- ever, the means for the number of intrusions were signifi-

cantly different (p < 0.02).

Discussion

The effectiveness of cues for information retrieval may

be measured a number of ways. One important measure is the number of matching words. As indicated in Table 1,

both free recall and structural cues produced more than double the matching terms produced by bibliographic cues. However, the utility of additional matching vocabulary depends on the information retrieval setting. In large data files, searches using additional terms combined using a Boolean “and” will tend to produce smaller result sets. This is of particular interest when Blair’s [44] concept of futility point is considered. In small files, on the other hand, retrieval using fewer matching terms may produce satisfactory results, and the additional matching vocabulary will be superfluous. The number of matching terms

may be more significant for known-item recall than for

general subject searching.

Although there is no experimental evidence to verify the contention, it seems possible that a user seeking a

single known item would have a lower futility point than a user seeking general subject information. If this is the

case, cues eliciting more matching vocabulary may be of particular interest in known-item searching.

The second measure of cueing effectiveness is the number of non-matching terms provided. In a retrieval situa-

tion in which terms are being combined using the Boolean “and,” even one non-matching term in the search expres-

sion will result in a failure of retrieval. If unprocessed user input is to be incorporated into a search expression, as frequently happens in end-user situations such as library catalogues, it is of paramount importance to limit the number of non-matching terms. The results summarized above indicate that bioliographic cues do an excellent job of limit- ing user input. By contrast, in systems using human in- termediaries, the non-matching vocabulary may clarify the user’s need, resulting in an increase in retrieval effectiveness. Free recall resulted in an average number of non-matching terms an order of magnitude larger than bibliographic cues, while structural cues on the average occu- pied an intermediate position.

One measure which combines the number of matching and non-matching terms into a single measure is the cosine similarity measure. As operationalized for this research, the cosine measure is composed of the number and frequency of occurrence of matching terms, the length of the recall protocol and the length of the document surrogate.

Since the length of the document surrogate remains con- stant, the cosine measure varies with the number of matching terms and the length of the protocol. The three cueing

methods produced similarity measures which were not significantly different, because an increase in the number of matching terms tended to be matched by an increase in the overall length of the protocol.

The comparison of the levels of difficulty reported for the recall task is in line with the other findings. Subjects

found the task of recalling bibliographic elements significantly more difficult than free recall or recall using structural cues. This presumably relates to the level of exact-

ness which is presupposed by the bibliographic task: the limitation of non-matching terms.

The results of the gist matching of propositions from

the text with propositions from the protocols provides a pattern which is virtually identical to that obtained from keyword matching for the two types of cues analyzed. There was no difference in the number of central or other text propositions recalled, but the free recall condition provided subjects with an opportunity to introduce a number

of intrusions: propositions not found in the text. By contrast, structural cues appear to have constrained the subjects so that substantially fewer intrusions occurred.

These results relate to differences produced by the different tasks assigned to the three groups of subjects, not to a difference in memory for the article. These tasks were chosen because they are similar to those normally accom-

plished by users of information systems in generating input for the systems. However they were not presented to sub-

jects as information retrieval tasks; subjects were simply asked to respond fully and accurately to questions about an article they had read. This was done so that subjects would

not be influenced by their opinions about what information retrieval systems expect, and thus produce a “decision” difference in their responses. Future research will incorpo- rate additional tasks, including those which are identified

as pertaining to information retrieval, in order to measure the effects produced by subjects’ mental models of infor-

mation systems.

Conclusions

The three methods of cueing differed significantly in the numbers of matching and non-matching terms produced by subjects in recall protocols. It follows that the methods may be suitable for different types of information retrieval environments. If end-user input is to be directly incorporated into a query expression, as is frequently the case with library online catalogues and end-user search interfaces for bibliographic data files, then bibliographic cueing, which limits input dramatically, is most likely to

avoid non-matching terms. In a situation in which a human intermediary analyzes user input and creates the search expression, the additional input prompted by free or structural cueing is likely to be of greater use. In such a situation, structural cueing provides large absolute values of matching terms, and large proportions of matching terms


(on average, 21% of the terms in protocols produced by structural cues matched the document vocabulary, while

only 15% of the terms from free recall protocols matched the document). For known-item searching in an information system with some form of intermediary, structural cueing is the preferred method of the three. This conclu- sion is reinforced by the results of the propositional analysis of recall protocols. The high proportion of matching terms observed for structural cues is mirrored by a high proportion of matching propositions. Seventy-five percent of the propositions in recall protocols produced by structural cues matched propositions from the text, compared to 57% in the case of free recall.

This research used a recall experiment to provide an in- sight into the way users of information retrieval systems may respond to questions posed by the system. This may provide a basis for decisions in the design of such systems,

and in particular their user interfaces. However there are a number of extensions of this research which will be re- quired in order to establish its methodology and findings securely. In terms of known-item searching, this research needs to be replicated, and extended by providing a number of different tasks at the reading and recall stages. Such research would take into account both the encoding and retrieval schemata of subjects, and the influence of tasks which relate specifically to information retrieval. There is

as well an important extension of this research into the area of general subject searching. It is necessary to determine whether the results from this research in a relatively restricted type of information retrieval, that of known-item searching. can also be found in more general environments, such as bibliographic retrieval systems doing subject searches.

References

I. Belkin. N.J.; Oddy, R. N.; Brooks, H. M. “ASK for Information

Retrieval. Part I: Background and Theory,” Journal of Documenta- tion. 38:61-71; 1982.

2. Belkin, N. J.; Oddy, R. N.; Brooks, H. M. “ASK for Information

Retrieval. Part II: Results of a Design Study.” Journal of Documen- turion. 38:145-164; 1982.

3. Dervin, B.; Dewdney, P. “Neutral Questioning: A New Approach to

the Reference Interview,” RQ. 26: 506-513; 1986.

4. Hawkins, D. T.; Levy, L. R. “Front End Software for Online Data- base Searching. Part 3: Product Selection Chart and Bibliography,”

Online. 10(3), 48-57; 1986.

5. Boguraev, B. K.; Sparck Jones, K. “Natural Language Front End to

Databases with Evaluative Feedback,” In: Gadarin, Cl.; Gelenbe, E., eds. New Applications of Databases. London: Academic; 1984:

159-182. 6. Damerau, F. I. “Operating Statistics for the Transformational Ques-

tion Answering System,” American Journal of Computational Lin- guistics. 7:30-42; 1981.

7. Damerau, E I. “Problems and Some Solutions in Customizations of

Natural Language Database Front Ends,” ACM Transactions on Of fice Information Systems. 3:165-184; 1985.

8. Thompson, B. H.; Thompson, F. B. “ASK is Transportable in Half

a Dozen Ways,” ACM Transactions on Office Information Systems. 3:185-203; 1985.

9. Bartlett, F. C. Remembering: A Study in Experimental and Social Psychology. Cambndge, UK: Cambridge University Press; 1932.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

Bobrow, D. G.; Norman, D.A. “Some Principles of Memory Sche-

mata,” In: Bobrow, D. G.; Collins, A., eds. Representation and Understanding: Studies in Cognitive Science. New York, NY: Aca-

demic; 1975: 131-149.

Rumelhart, D. E.; Ortony, A. “Representation of Knowledge in

Memory,” In: Anderson, R. C.; Spiro, R. J.; Montague, W. E., eds.

Schooling and the Acquisition of Knowledge. Hillsdale, NJ:

Lawrence Erlbaum Associates; 1977: 99-135.

Anderson, R. C. Schemata as Scaffolding for the Representation of Information in Connected Discourse. Urbana, IL: Center for the

Study of Reading; 1977.

Adams, M. J.; Collins, A. Schema-theoretic View of Reading. Ur-

bana, IL: Center for the Study of Reading; 1977.

Rumelhart, D. E. “Schemata: Building Blocks of Cognition,” In: Spiro, R. J.; Bruce, B. C.; Brewer, W. F., eds. Theoretical Issues in Reading Comprehension: Perspectives from Cognitive Psychology, Linguistics, Artificial Intelligence and Educarion. Hillsdale, NJ: Lawrence Erlbaum Associates; 1980: 33-58.

Minsky, M. “Framework for Representing Knowledge,” In: Metz-

ing, D., ed. Frame Conceptions and Text Understanding. Berlin:

De Gruyter; 1980: I-25.

Anderson. R. C.; Reynolds, R. E.; Schallert, D. L.; Goetz, E. T.

“Frameworks for Comprehending Discourse,” American Educa- tional Research Journal. 14:367-381; 1977.

Steffenson, M. S.; Joag-Dev, C.; Anderson, R. C. “A Cross-cultural

Perspective on Reading Comprehension,” Reading Research Quar- terly. 15:10-29; 1979.

Sulin, R. A.; Dooling, D. J. “Intrusion of a Thematic Idea in Reten-

tion of Prose,” Journal of Experimental Psychology. 103:255-262; 1974.

Meyer, B. J. F. “What is Remembered from Prose: A Function of

Passage Structure.” In: Freedle, R. O., ed. Discourse Production and Comprehension. Norwood, NJ: Ablex; 1977: 307-336.

Meyer, B. J. E; Rice, G. E. “interaction of Reader Strategies and

the Organization of Text,” Text. 2:155-192; 1982.

Thorndyke, P. W. “Cognitive Structures in Comprehension and

Memory of Narrative Discourse,” Cognitive Psychology. 9:77- 110; 1977. Minsky, M. “Frame-system Theory, ” In: Johnson-Laird, P. N.; Wat-

son, P. C., eds. Thinking: Readings in Cognitive Science. London,

UK: Cambridge University Press; 1977: 355-376.

Mandlcr, J. M.; Johnson, N. S. “Remembrance of Things Parsed:

Story Structure and Recall,” Cognitive Psychology. 9: 11 I- 15 1; 1977.

Rumelhart, D. E. “Notes on a Schema for Stories,” In: Bobrow,

D. G.; Collins, A., eds. Representation and Understanding: Studies in Cogn;tive Science. New York, NY: Academic Press; 1975: 2ll-

236.

Stein, N. L. “The Definition of a Story.” Journnl of Pragmatics. 6:487-508; 1982.

Kintsch, W. “Memory for Text,” In: Flammer, A.; Kintsch, W.,

eds. Discourse Processing. Amsterdam: North-Holland; 1982: 186-

204.

Kintsch, W.; Van Dijk, T. A. “Toward a Model of Text Comprehen-

sion and Production,” Psychological Review. 85:363-394; 1978.

Van Dijk, T. A. Macrostructures: An Interdisciplinary Study of Global Structures in Discourse, Interaction and Cognition. Hills- dale, NJ: Lawrence Erlbaum Associates; 1980.

Van Dijk, T. A.; Kintsch, W. Sfrategies of Discourse Comprehen- sion. New York, NY: Academic Press; 1983. Johnson, R. E. “Retrieval Cues and the Remembering of Prose: A

Review,” In: Flammer, A.; Kintsch, W., eds. Discourse Process- ing. Amsterdam: North-Holland; 1982: 219-238. Anderson, R. C.; Pichert, J. “Recall of Previously Unrecallable In-

formation Following a Shift in Perspective,” Journal of Verbal Learning and Verbal Behavior. 17:l-12; 1979.

Fass, W.; Schumacher, G. M. “Schema Theory and Prose Reten-

tion: Boundary Conditions for Encoding and Retrieval Effects,”

Discourse Processes. 4: 17-26; 1981. Flammer, A.; Tauber, M. “Changing the Reader’s Perspective,” In:


Flammer, A.; Kintsch, W., eds. Discourse Processing. Amsterdam:

North-Holland; 1982: 379-391.

34. Pollitt, A. S. “Search Statement Generator for Cancer Therapy Re-

lated Information Retrieval,” International Online Informarion Meeting Proceedings. Oxford, UK: Learned Information;

1982:405-413.

35. Pollitt, A. S. “A ‘Front End’ System: An Expert System as an On-

line Search Intermediary,” ASLIB Proceedings. 36:229-324; 1984.

36. Rosenberg, S. “Expert Systems and the Design of Powerful User

Interfaces,” Proceedings of the 44th ASIS Annual Meeting. 18:285-

287; 1981.

37. Hahn, U.; Reimer, U. “Computing Text Constituency: An Algorith-

mic Approach to the Generation of Text Graphs,” In: van Rijsber-

gen, C. .I.; ed. Research and Development in Information Retrieval: Proceedings qf the Third Joinf BCS and ACM Symposium. Cam-

bridge: Cambridge University Press; 1983: 343-368.

38. Bush, D. W.; White. K. R. “Questionnaire Distribution: A Method

that Significantly Improved Return Rates,” Psycho/ogical Reports.

56:427-430; 1985.

39. Glenberg, A. M.; Wilkinson, A. C.; Epstein, W. “The Illusion of

Knowing: Failure in the Self-Assessment of Comprehension,”

Memory and Cognition. 10597-602; 1982.

40. Lovins, J. B. “Development of a Stemming Algorithm,” Mechani- cal Translation and Computational Linguistics. 11122-31; 1968.

41. Salton, G.; McGill, M. J. Introduction fo Modern Information Re- frieval. New York, NY: McGraw-Hill; 1983.

42. Bovair, S.; Kieras, D. E. “A Guide to Propositional Analysis for

Research on Technical Prose.” In: Britton, B. K.; Black, J. B., eds.

Understanding Expository Text: A Theoretical and Practical Hand- book for Analyzing Explanatory Text. Hillsdale, NJ: Lawrence Erl-

baum Associates; 1985: 315-362.

43. Conover, W. J. Practical Nonparametric Statistics. New York, NY:

Wiley; 1980.

44. Blair, D. C. “Searching Biases in Large Interactive Document Re-

trieval Systems,” Journal of the American Society for Information Science. 31:271-277; 1980.


Documents

Recall cues in known-item retrieval