Information Retrieval Systems in Academic Libraries

8/13/2019 Information Retrieval Systems in Academic Libraries

1/5

1

Information Retrieval Systems in Academic Libraries

The aim of this essay is to make some reflections about the way academic libraries use technology in order

to satisfy users needs. While outlining general theoretical principles we will analyse two academic librariesonline catalogues to which we were granted access.

Most of students and researchers using academic libraries carry out projects about specific topics on which

they need to gather further information. Motivated by an anomalous state of knowledge (ASK) (Belkin

1980; Belkin, Oddy and Brooks 1982) they perform an exploratory seeking (Rosenfeld and Morville 2007)

formulating a query whose answer, located in a separated stored database, is subjectively right. What is

needed in this kind of research environment is thus a text retrieval system that interact and communicate

with the user so that the way users represent their request can be compared and matched with the way

information represented in the databases by means of a dialogue which leads to retrieve the most relevant

documents in the users perspective.

In order to fulfill users information needs, on one side there are mathematical algorithms identifying the

document to be presented; on the other, important role is played by the system design (the way for

presenting results, how to group them, layout of the search-engine).

Several methods can enhance the possibility of a better result:

- Indexing techniques (documents are simply presented according to the precise number of termsmatching).

- Weighting terms (documents are ranked according to their total weight). In this case each term isbeing given a weight according to a combination of 3 different criteria (Robertson and Jones 1994).

- Iterative searching (documents, after going through statistical formulae to select a bettercombination of terms, are assessed by relevance). It can be performed by changing the request

term weight (relevance weighting) (Robertson and Jones 1976) or changing the request

composition (query expansion) (Robertson 1990). The latter reveals as a step toward more

interactive systems, in that search expansion starts from a document whose relevance can be

determined by the user (See Harman & Voorhees, 2006 for a review of studies of relevance

feedback in the TREC project).

- Other solutions helping widening the search results include: active bibliography, related documentsfrom co-citation, cited by, research index (Rosenfeld and Morville 2007).

For the development of more effective IR mechanisms (sensu Robertson) and for helping libraries adoptone for their users, evaluation with the pivotal role played by evaluation criteria involved in the test

processes become unavoidable. While in the past most of the experiments carried out to evaluate a IR

system would rely mainly on the physical output given by the mechanism, evaluated against the inversely

related concepts of precision and recall, nowadays many evaluation methods acknowledge the crucial

importance of the user as fundamental participant of the evaluation process. This shift of perspectives, due

to the advent of the cognitive, relevance and interactive revolutions (Robertson and Hancock-Beaulieu 1992)

is also associated with the fact that current libraries cata logues offer such a large amount of resources (lets

think to access to remote e-journals) to make difficult a laboratory-based experiment with relevance pre-

judgements of the whole set of documents.

Since main aim of a IR system is the satisfaction of the users ASK several researches have been carried outover the last twenty years in order to combine quantitative (mechanism centered) and qualitative (user-


2/5

2

centered) evaluation methods (Robertson and Hancock-Beaulieu 1992, Hancock-Beaulieu 1990, Hancock-

Beaulieu et al., 1991, Walker & Jones 1987 cited in Xie 2008). These studies advocate for including

interaction and not only effectiveness factors in the evaluation process; (Su 1992, 1994, 2003; Saracevic

and Kantor 1988a, b, c, Hersh Pentecost and Hickham 1996, Lancaster 1979 cited in Xie 2008).

CASE STUDY: the online catalogues of the department of Archaeology of the University College London

and University of Rome Sapienza were analysed.Results are summed up in the following tab:

QUERY SapienzaCatalogue UCLCatalogue

No. Documents retrieved No. Documents Retrieved

Prehistory Sahara Pottery 0 377

African Pottery 2 8061

Africa Pottery 1 9591

No. Documents retrieved Precision No. Documents retrieved Precision

Prehistory Africa 12 0.6 1771 0.6

By typing any 3 terms search on the Sapienza catalogue 0 results were obtained. It is quite evident as

in the Sapienza catalogueresearcher face an initial serious drawback.

By changing query and typing African pottery, only 2 documents (one relevant, one not relevant)

were retrieved by the Sapienza catalogue. Unexpectedly, by typing Africa, the system retrieved only

the not relevant one. Doing the same with the UCL catalogue, we noticed that the same book retrieved

by the Sapienza systemwas included in both searches. We could assume that the indexing system

on which the Sapienza catalogue is based does not include sophisticated stemming techniques.

As far as precision calculation concerns both systems scored 0.6 (first top 10 documents were

analysed). We think that precision should not be seen as a suitable criteria to assess a IR system since,

as already shown by Su (1994), in an academic environment a higher recall reveals to be more helpful

to students. Besides, an higher recall in the UCL catalogue led to clarify our ideas about what we were

actually looking for and change our queries.

An analysis of the systems design also provoked some reflections. In the Sapienza search engine

typed queries are not displayed anywhere once the results come out; the advanced search is not very

user friendly with too many available fields in the same screen. Besides in the UCL engine details asabstract and the fundamental recommendation are immediately readable. Furthermore two crucial

features offered by the UCL catalogue allowing users to refine their search by subtopic and expand

their research with suggested new searches is there missing.

In conclusion, this essay shows how strong IR systems should adopt a more user-centered oriented

perspective, bearing in mind that areas such as ranking algorithms, building up of efficient access

mechanisms, query processing, display and delivery of search results should be designed according to end-

users needs, at the heart of any IR; secondly how systems evaluation really need to consider not only the

effectiveness but also the relationship users create with the system, which if not emphatic, can lead themto opt for different sources.


3/5

3

References

http://trec.nist.gov/
http://trec.nist.gov/http://trec.nist.gov/http://trec.nist.gov/


4/5

4

http://www.ucl.ac.uk/library/

http://opac.uniroma1.it/SebinaOpacRMS/Opac?sysb=GR_1

Broder, A. (2002). A taxonomy of web search SIGIR Forum Fall, 36 (2). [Online] Available:

http://www.sigir.org/forum/F2002/broder.pdf(28 October 2012).

Belkin, N.J.(1980), Anomalous States of Knowledge as a Basis for Information Retrieval, Canadian Journal

of Information and Library Science, 5 pp. 133-143. University of Toronto Press. [Online] Available

http://comminfo.rutgers.edu/~tefko/Courses/612/Articles/BelkinAnomolous.pdf(27 October 2012).

Rosenfeld, L. and Morville, P. (2007), Information architecture for the World Wide Web (3rd

edition):

Designing Large-scale Web Sites, Sebastopol, CA.: OReilly.

Saracevic, T., & Kantor, P. (1988a)A study of information seeking and retrieving: I. Background and

methodology.Journal of the American Society for InformationScience, 39(3), 161-176.

Saracevic, T., & Kantor, P. (1988b). A study of information seeking and retrieving:II. Users, questions, and

effectiveness.Journal of the American Society forInformation Science, 39(3), 177-196.

Saracevic, T., & Kantor, P. (1988c).A study of information seeking and retrieving:III. Searchers, searches,

and overlap.Journal of the American Society forInformation Science, 39(3), 197-216.

Hersh, W., Pentecost, J., & Hickam, D. (1996). A task-oriented approach to information retrieval

evaluation.Journal of the American Society for InformationScience, 47(1), 50-56.

Hancock-Beaulieu, M. (1990).Evaluating the impact of an online library catalogue in subject searching

behavior at the catalogue and at the shelves.Journal of Documentation, 46(4), 318-338.

Hancock-Beaulieu, M., Robertson, S., & Neilson, C.(1991). Evaluation of online

catalogues: Eliciting information from the user. Information Processing and Management, 27(5), 523-532.

Su, L. T. (2003) A comprehensive and systematic model of user evaluation of Web search engines: I. Theory

and background.Journal of the American Societyfor Information Science and Technology, 54(13), 1175-

1192.

Harman, D. K., & Voorhees, E. M.(2006) TREC: An overview.Annual Review of Information Science and

Technology, 40, 113-155.

Lancaster, F. W. (1979). Information retrieval systems: Characteristics, testing, andevaluation. New York:

John Wiley & Sons.

Robertson, S.E.(1990). On term selection for query expansion,Journal of Documentation46 (4), 359-364.

Robertson, S.E. and M.M. Hancock-Beaulieu (1992). On the evaluation of IR system, Information

Processing and Management28 (4), 457-466.

Robertson, S.E. and K. Sparck Jones (1976). Relevance weighting of search terms. Journal of the American

Society for InformationScience May-June, 129-145.
http://www.ucl.ac.uk/library/http://www.ucl.ac.uk/library/http://opac.uniroma1.it/SebinaOpacRMS/Opac?sysb=GR_1http://opac.uniroma1.it/SebinaOpacRMS/Opac?sysb=GR_1http://www.sigir.org/forum/F2002/broder.pdfhttp://www.sigir.org/forum/F2002/broder.pdfhttp://comminfo.rutgers.edu/~tefko/Courses/612/Articles/BelkinAnomolous.pdf%20(27http://comminfo.rutgers.edu/~tefko/Courses/612/Articles/BelkinAnomolous.pdf%20(27http://comminfo.rutgers.edu/~tefko/Courses/612/Articles/BelkinAnomolous.pdf%20(27http://www.sigir.org/forum/F2002/broder.pdfhttp://opac.uniroma1.it/SebinaOpacRMS/Opac?sysb=GR_1http://www.ucl.ac.uk/library/


5/5

5

Robertson, S.E. & K. Sparck Jones (1994). Simple, proven approaches to text retrieval, University of

Cambridge technical note, TR356. [Online] Available: http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-

356.pdf (28 October 2012)

Su, L. T. (1992). Evaluation measures for interactive information retrieval. Information Processing and

Management, 28(4), 503-516.

Su, L. T. (1994) The relevance of recall and precision in user evaluation.Journal of the American Society for

Information Science, 45(3), 207-217.

Xie, I.(2008). Interactive information retrieval in digital environments.Hershey, PA: IGI Global [Online]

http://kutuphane.ankara.edu.tr/igi/Interactive%20Information%20Retrieval%20in%20Digital%20Environm

ents.pdf(29 October 2012)

Walker, S. & Jones, R.M.(1987) Improving subject retrieval in online catalogues. 1. Stemming, automatic

spelling correction and cross-reference tables. London: British Library.
http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-356.pdf%20(28http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-356.pdf%20(28http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-356.pdf%20(28http://kutuphane.ankara.edu.tr/igi/Interactive%20Information%20Retrieval%20in%20Digital%20Environments.pdfhttp://kutuphane.ankara.edu.tr/igi/Interactive%20Information%20Retrieval%20in%20Digital%20Environments.pdfhttp://kutuphane.ankara.edu.tr/igi/Interactive%20Information%20Retrieval%20in%20Digital%20Environments.pdfhttp://kutuphane.ankara.edu.tr/igi/Interactive%20Information%20Retrieval%20in%20Digital%20Environments.pdfhttp://kutuphane.ankara.edu.tr/igi/Interactive%20Information%20Retrieval%20in%20Digital%20Environments.pdfhttp://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-356.pdf%20(28http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-356.pdf%20(28