Understanding and using published standards for high quality systematic review searches

+The health librarian as a systematic review team

member: practical considerations

Ruth Mitchell, Cochrane Renal Group

Catherine King, National Centre for Immunisation Research and Surveillance (NCIRS)

Introduction• Standards

planning the search development of expert strategies reporting the search

• Practical outworking of each negotiation management of expectations documentation

Using standards to assist negotiations, expectations and documentation around who does what, why, when and how

Standards for conducting systematic reviews

Methodological standards for the conduct of Cochrane intervention reviews (Version 2.1)

80 standards 15 standards relating to searching for studies

Development led by Information Methods Retrieval Group

Finding what works in health care: standards for systematic reviews

17 standards with 71 ‘elements’ or steps 7 standards with 18 ‘elements’ relating to searching

Standards for planning the search

Standard 19 Planning the search requires authors: to plan in advance the methods to be used for identifying studies. to design searches to capture as many studies as possible meeting the

eligibility criteria to cover relevant time periods and sources to NOT restrict by language or publication status

Standards for planning the search

2.1 Establish a team with appropriate expertise and experience to conduct the systematic review

2.1.3 Include expertise in searching for relevant evidence

2.5 Formulate the topic for the systematic review 2.5.3 Use a standard format to articulate each question of interest

3.1 Conduct a comprehensive systematic search for evidence 3.1.1 Work with a librarian/information specialist to plan the search

strategy

Practical - Planning• Negotiation

Individual circumstances (e.g. Hospital/ Uni library) Continuum of involvement (guide to standards full involvement) Clarification of purpose – e.g. Masters, Cochrane Scope of review Reference interview – existing skills

Clarification often assists researchers as much as librarian

Practical - Planning

• Expectations Your own – team roles, timelines/time allocation Researchers – team roles, time required

• Documentation Discussions – supervisor/research team Confirm via email where possible

Standards for search strategy development and conduct Searching for studies - Standards 24-38

Sources to search – databases, prospective trial registries, grey literature, author contact reference lists… 8 maximize sensitivity with “reasonable precision”

Search strategy development – use PICO and study design 1

Use controlled vocabulary and free-text terms 1

Use appropriate search filters 1

Justify use of any restrictions 1

Documentation – reported searches should be reproducible 1

Rerun searches within 12 months before publication of review 1

Standards for search strategy development and conduct IOM Standard 3.1 Conduct a comprehensive search for evidence

Search strategy development, sources to search, updating searches 7

Peer review of strategy by an independent information specialist 1

Standard 3.2 Take action to address potentially biased reporting of research results

Grey literature, handsearching, contact researchers…3Standard 3.4 Document the search

Line-by-line description, database, date of search 1

Practical – Search development

• Negotiate scope PICO Scoping search Published/Grey literature/Clinical trial registries

• Expectation management Your own – monitoring time/involvement Team – “think my search is pretty good”.. Timeframes – “can I have all the searches by Friday?”

Practical – Search development

• Documentation

Account set-up (e.g. in OVID or similar) Saving – for researcher/self (always keep own copy!) Version control/ naming Searches – strategy, databases, interface, currency dates

searched

Standards for reporting searches

Methodological standards for the reporting of Cochrane intervention reviews

Still under development Cover the content required for both abstract and main body of a review e.g.

All sources searched Date of last search Any restrictions placed on search Who designed and carried out the search Full line by line copy of each strategy

Can be viewed at http://www.editorial-unit.cochrane.org/mecir

Standards for reporting searches

Standards for Reporting Systematic Reviews Standard 5.1 Prepare final report using a structured format5.1.6 Include a methods section. Describe the following:

Databases and other information sources used to identify relevant studiesSearch strategy

Item 7: Information sources Describe all information sources in the search (such as databases with dates of coverage, contact with study authors to identify additional studies) and date last searched.

Item 8: Search Present the full electronic search strategy for at least one major database, including any limits used, such that it could be repeated.

Practical - Reporting• Negotiate

What to include – guide using standards How much involvement in write up? Acknowledgment? – check final product

• Expectations Your own – finished now or still involved? Team – what expectations do they have re reporting

requirements/librarian involvement?

Practical - Reporting

• Documentation Reporting – ensure search can be replicated Check researcher has exact copies of searches + all

information as documented in the search development/running phase

Journals – PRISMA – full strategy (appendix/web extra)

Conclusions

• Standards – assist rather than hinder

• Reinforce key role of information specialist

• Provide framework for negotiation, expectation management and documentation

• Communication skills, diplomacy and sense of humour

ReferencesCochranehttp://www.editorial-unit.cochrane.org/mecir

IOMhttp://www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx

PRISMAhttp://www.prisma-statement.org/Other standards: AHRQhttp://effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-reports/?pageaction=displayproduct&mp=1&productID=318CRD http://www.york.ac.uk/inst/crd/index_guidance.htm

Acknowledgments• Cochrane Renal Group

• National Centre for Immunisation Research and Surveillance (NCIRS)

Activity - Scenario role play

2 scenarios to choose from Break into pairs

one person takes info specialist role one person takes client role (opportunity to channel

your inner worst/best client!) Aim is to negotiate issues and establish

expectations for the planning stage of searching for a systematic review

Feedback from 1 pair from each group