Column Editor

60

© Blackwell Science Ltd 2001

Health Information and Libraries Journal

,

18

, pp.60–63

Blackwell Science, Ltd

Research

Andrew Booth, School of Health and Related Research (ScHARR), University of Sheffield

Will health librarians and related informationworkers ever work together to create an internationalnetwork, such as the Cochrane Collaboration,dedicated to the purpose of preparing, maintain-ing and disseminating systematic reviews

1

of theeffects of health

information services and systems

?As we have remarked elsewhere ‘Informationscientists may be equipped to scan the horizonbut they possess silicon chips, not crystal balls’.

2

Nevertheless, it is possible to take an informedlook at developments in systematic reviews,together with the idiosyncratic characteristics ofour own evidence base, and to assess where futureprospects might lie. In previous issues this columnhas focused on obtaining funding for [September2000], and the critical appraisal of [December 2000],primary research. In this issue we turn the spotlightonto secondary research, namely systematic reviewand synthesis.

The Cochrane Collaboration?

Before discussing the likelihood of a separatecollaboration for health information science it isnecessary to consider whether or not the CochraneCollaboration would be an appropriate home forsuch an initiative. The Cochrane Library certainlycontains a number of full-text reviews

3, 4

orbibliographic references to topics falling withinthe wider domain of health information. A majorstumbling block, identified in preliminary dis-cussions with individuals involved in steering theCollaboration, has been that a Cochrane system-atic review is required to be concerned primarilywith an intervention’s

direct effects on health careoutcomes

. This admirably pragmatic tenet is nodoubt aimed at protecting the Collaboration frombecoming overly academic and detached from

the considerations of health care delivery thatshould quite rightly be paramount. In consideringto what extent the research reported in the healthinformation literature focuses on patient-focusedhealth outcomes, one would find that a largeproportion demonstrates effects of informationservices or skills training on the

knowledge

ofrecipients, e.g. ‘I went on a

trainingcourse and now I know about the explode andfocus features’. A smaller but significant proportionexamines the effects on the

attitudes

of participants,e.g. ‘Having attended a library open day I am nowmore likely to use the library services’. Still fewerreports concentrate on the effects on the observed

behaviour

(as opposed to self-reported behaviour!)of the subjects of a research study. Finally, analmost negligible amount focuses on whether thehealth of patients (i.e. their

health care outcomes

)actually benefits as a result of an information-related intervention.

It is only this final category that would interestthe Cochrane Collaboration. Such studies are lessplentiful precisely because it is so difficult to provesuch an effect. There are many confounding factorsin the chain between delivering an information skillscourse or providing an electronic textbook and thebenefit a patient might receive from the clinician’snewly acquired skills or knowledge, and this makesit problematic to establish any genuine ‘causeand effect’ relationship. It is this, far more thanthe well-documented preference of the CochraneCollaboration to focus on randomized controlledtrials, that appears to pose the most significantobstacle to the widespread inclusion of healthinformation topics in the Cochrane Library.Nevertheless developments in the recent years ofthe Cochrane Collaboration such as the recognitionof health economics and qualitative research methodsand the raised profile of the Effective Practice andOrganization of Care (EPOC) group suggest thatthis position is not necessarily to be seen as an

HLR309.fm Page 60 Wednesday, February 14, 2001 3:31 PM

Research



,

18

, pp.60–63

61

intransigent one. What characterizes these recentdevelopments, however, is the prior existence ofpowerful lobby groups organized around estab-lished international communities of researchers.The health library and information community,academics and practitioners, need to considerseriously whether a similar sustained effort isrequired to secure recognition of its own potentialcontribution to the Collaboration. Some hope isoffered by the inclusion in the Cochrane Library,under the auspices of EPOC, of a review protocolfor instruction in critical appraisal,

5

an interven-tion similar to our own core activity of instructionin literature searching.

The Campbell Collaboration?

Health information professionals inhabit a ‘westernfront’ between the ‘hard’ applied science of medicineand the ‘softer’ social science of librarianship. Muchof our activity is conducted within the domainsof education (students of nursing, medicine andother professions) or of training (postgraduateeducation, continuing professional development,etc). We also find ourselves torn between thecontrasting paradigms of the quantitativeresearch espoused by the biomedical communityand the qualitative approaches that are morecommon in the nursing and therapy professionsand, indeed, so typical of our own research. Our‘amphibious’ nature extends to us, at least at thispreliminary stage, the prospect of involvementin another international initiative, the CampbellCollaboration. This recent sibling to the CochraneCollaboration (http://campbell.gse.upenn.edu/)is a fledgling international network aimed at pre-paring, maintaining and promoting the accessib-ility of systematic reviews of the effects of

socialand educational policies and practices

. It first metin February 2000 and it has been strongly sup-ported by leading figures from the Cochrane Col-laboration. The range of domains and outcomesto be considered within the activities of such anoverarching organization is potentially muchbroader than that currently adopted by theCochrane Collaboration. The downside of thismight be a possible tendency for those withposition and influence within the National HealthService to view such educational interventions as

being removed from the main targets for theirinitiatives and funding. In short, placing theevidence base of health information squarelywithin the aegis of the Campbell Collaborationcould result in a return to the assumption thathealth information work should call on thetraditional reservoirs of postgraduate educationfunding rather than the newly opened streamsassociated with research and development orsupport to clinical care. Nevertheless any ‘flag ofconvenience’ likely to stimulate the developmentof an evidence base for health information servicesand systems should not be dismissed withoutserious investigation.

The Evidence Base of Health Librarianship

‘Ask not what the Cochrane/Campbell Collabora-tions can do for you—ask what you can do forthe Collaborations’. This misquotation fromJ. F. Kennedy’s inaugural address reminds usthat involvement in one of these well-organizedcollaborations can only come once we have startedto marshal our own information resources. Can ourcurrent evidence base sustain the rigorous methodsrequired for systematic review and meta-analysis?A feasibility study conducted for the HealthLibraries Group Research Working Party, thepredecessor to the current LINC Health PanelResearch Working Party, found that our evidencebase is scattered across a number of sources andthat it exhibits heterogeneity in the range of researchdesigns and outcome measures, together with poorresearch methodology.

6

This situation is exacerbatedby poor indexing of research designs and methodsand the prevalence of uninformative abstracts.

If it is unlikely that many review questions fromour domain will support a full-blown quantitativesynthesis of results (meta-analysis), what might bethe way forward? In a study that approximates mostclosely to the model espoused by the CochraneCollaboration, physicians’ preferences for informa-tion sources are examined.

7

In this review selecteddata from 12 studies published between 1978 and1992 were compared, quantitatively aggregated andsynthesized. The top five preferences from eachstudy were ranked and then cross-study similaritiesin rankings were identified and summarized. Thisreview may be flawed in that there is a simplistic


Research



,

18

, pp.60–63

62

assumption that rankings from different studiescan be pooled as if a difference between 1st placeand 2nd place in one study is equal to a differencebetween 1st and 2nd place in all the others. Never-theless, it does provide a powerful demonstration ofthe power of synthesizing data in such a manner.

Significantly, just as meta-analysis originatedfrom social sciences before migrating to medicineand being enthusiastically adopted as its own, analternative approach was derived from educationand is known as meta-ethnography.

8

This techniquewas originally used by its promulgators to synthesizequalitative data from a number of school inspectionreports. In this way emerging themes from acrossreports could be identified and summarized. Athree stage process is used that involves extractingthemes from each individual report, tabulating allthese themes into a single summary report andthen finally establishing common categories andsubcategories and equivalences across studies. So,for example, if the leadership characteristics of theheadmaster were seen to be a major factor in thesuccess of a number of schools, this would becomea category for analysis. Individual characteristics(e.g. sense of humour, approachability etc) wouldthen become subcategories. This approach could beapplied across a body of related reports of healthinformation research (e.g. all primary care informa-tion projects) to encapsulate our current knowledgeand to identify future directions for research.

If we can apply such a meta-ethnographicapproach to our professional literature it is clearwe could also use it to synthesize a myriad ofrelated pilot projects or individual case studiesthat never make their way into formal publishingchannels. If individual case studies of libraryprojects were to support this level of analysisthey would need to have fortuitously collectedlarge amounts of supporting data—an uncommoncharacteristic of most local initiatives! This sug-gests a way forward that parallels an approachused by the Cochrane Collaboration, namely,collaborative overviews using prospective datacollection. Some systematic reviews, notably thosein cancer, are regularly updated by the ongoing resultsfrom large trials.

9

This requires initial agreementregarding what data should be collected so as toensure consistency across studies. This is bestillustrated by an analogy from within our own

field. Suppose that the various ‘clinical librarian’projects currently springing up around the UK, orindeed the world, could agree on a minimum datasetto be used for their evaluation. This dataset mightbe based on criteria from a previous article.

10

Alternatively it might be the result of a processof consensus. Each participating librarian wouldagree to collect

at least

the data required by theminimum dataset. [They could, of course, collectany additional data that their local evaluationrequired.] In this way each additional evaluationwould not only draw strength from taking placewithin an acknowledged frame of reference butwould, in turn, also contribute to the growth of theknowledge base. Sounds simple doesn’t it?

Conclusion

This brief outline of the ways in which systematicreviews might relate to our field, indicates both thecurrent situation and possible future directions. Itis certainly possible to conduct systematic reviewsin health information topics where randomizedcontrolled trials exist. It is also possible to usemeta-analytic techniques (not necessarily full-blownmeta-analysis) to add value to an existing bodyof quantifiable research. Meta-ethnography offersthe possibility of extracting common themes orhypotheses for further investigation from a numberof related qualitative studies, either publishedor unpublished. Finally, agreement on commonstudy protocols for initiatives at a local level thatmight contribute ultimately to an internationalbody of evidence, as in our example from the clinicallibrarian movement, would seem to offer a practicalmechanism for ongoing research and evaluation.

Of course, to instigate such collaborative inter-national activity also appears to require identificationof an individual with a Celtic name (as in bothCampbell and Cochrane)! Who are we to say thatin years to come the prospect of a McKibbonCollaboration

11

or of a Marshall Collaboration

12

might not be realized?

References

1 Bero, L. & Rennie, D. The Cochrane Collaboration. Preparing, maintaining, and disseminating systematic reviews of the effects of health care.

JAMA

1995,

274

, 1935 – 8.


Research



,

18

, pp.60–63

63

2 Booth, A. & Walton, G. Some concluding trends and themes. In: Booth, A. & Walton, G. (eds.)

Managing Knowledge in Health Services

. London: Library Association. 2000: 289.

3 Thomas, L., Cullum, N., McColl, E., Rousseau, N., Soutter, J. & Steen, N. Guidelines in professions allied to medicine (Cochrane Review). In:

The Cochrane Library, Issue 4

. Oxford: Update Software, 2000 [updated quarterly].4 Freemantle, N., Harvey, E. L., Wolf, F., Grimshaw, J. M.,

Grilli, R. & Bero, L. A. Printed educational materials: effects on professional practice and health care outcomes (Cochrane Review). In:


. Oxford: Update Software, 2000 [updated quarterly].

5 Parkes, J., Deeks, J., Milne, R. & Hyde, C. Teaching critical appraisal skills in health care settings (Protocol for a Cochrane Review). In:


. Oxford: Update Software, 2000 [updated quarterly].

6 Booth, A.

‘Librarian heal thyself’: Evidence Based Librarianship, Useful, Practicable, Desirable?

<http://www.icml.org/tuesday/themes/booth.htm> accessed 09/11/00.

7 Haugh, J. D. Physicians’ preferences for information sources: a meta-analytic study.

Bulletin of the Medical Library Association

1997,

85

, 223 – 32.8 Noblit, G. W. & Hare, R. D.

Meta-ethnography: Synthesising Qualitative Studies

. (Qualitative Research Methods, Volume 11). London: Sage, 1988.

9 Clarke, M. J. Ovarian ablation in breast cancer, 1896 to 1998: milestones along hierarchy of evidence from case report to Cochrane review.

BMJ

1998,

317

, 1246 – 8.10 Greenberg, B., Battison, S., Kolisch, M. & Leredu, M.

Evaluation of a clinical medical librarian program at the Yale Medical Library.

Bulletin of the Medical Library Association

1978,

66

, 319 – 26.11 Haynes, R. B., Johnston, M. E., McKibbon, K. A.,

Walker, C. J. & Willan, A. R. A program to enhance clinical use of MEDLINE. A randomized controlled trial.

Online Journal of Current Clinical Trials

, 1993, (Doc. No. 56).

12 Marshall, J. G. & Neufeld, V. R. A randomized trial of librarian educational participation in clinical settings.

Journal of Medical Education

1981,

56

, 409 –16.


Documents

Column Editor