Finding and Evaluating Ren

al Evidence: Bridgingthe Knowledge GapShayna A. D. Bejaimal, R. Brian Haynes, Salimah Shariff, and Amit X. Garg

Nephrologists often search for clinical information to guide patient care, especially in the chronic kidney disease patient pop-

ulation where information needs are complex. Applying evidence-based principles while searching for current high-quality in-

formation can be challenging. This article details the ‘‘6S’’ framework to approaching evidence-based information resources

popularized by Haynes and colleagues. This hierarchal structure of resources includes ‘‘systems,’’ ‘‘summaries,’’ ‘‘synopses

of syntheses,’’ ‘‘syntheses,’’ ‘‘synopses of studies,’’ and finally, ‘‘studies’’ with the intent of providing nephrologists with an ef-

ficient method of retrieving accurate, high-quality information. Accurate clinical decision support systems are ideal but are yet

to reach their full potential. Summaries of best evidence for individual clinical problems are useful (UpToDate, DynaMed, Best

Practice, and PIER). Syntheses of articles are an efficient way to understand the available evidence for a clinical question. The

Cochrane Renal ReviewGroup is a helpful resourcewhich providesmany systematic reviews in renal research.When using bib-

liographic databases, suchPubMed, nephrologists can improve the precision of their search resultswith information filters. The

application of accurate and current clinical knowledge is increasingly important in renal patient care. Better ways of achieving

this will have large benefits.

Q 2012 by the National Kidney Foundation, Inc. All rights reserved.Key Words: Chronic kidney disease, Evidence appraisal, Information retrieval

From Division of Nephrology, University of Western Ontario, London,

Ontario, Canada; Department of Clinical Epidemiology and Biostatistics,McMaster University, Hamilton, Ontario, Canada; Department of Medicine,

McMaster University, Hamilton, Ontario, Canada; and Department of Epidemi-

ology andBiostatistics,University ofWesternOntario, London,Ontario,Canada.

A.X.G. is supported by a Clinician Scientist Award from the CanadianInstitutes of Health Research.

Address correspondence to Amit X. Garg, MD, PhD, London Kidney

Clinical Research Unit, London Health Sciences Centre, Room ELL-101, 800

Commissioners Road East, London, Ontario N6A 4G5, Canada. E-mail:amit.garg@lhsc.on.ca

� 2012 by the National Kidney Foundation, Inc. All rights reserved.

1548-5595/$36.00doi:10.1053/j.ackd.2012.01.003

Chronic kidney disease (CKD) is a condition for which

clinician information needs are complex.1,2 Accordingto the principles of evidence-based care, nephrologists’clinical decisions are best supplemented when they areable to garner accurate and current information reflectingthe best available evidence. Combining this with an under-standing of patient circumstances, as well as the realities ofhealth care delivery in the jurisdiction in which they work,a nephrologist can appropriately treat their patients.1,2

However, accessing accurate CKD evidence in a timelymanner is a growing challenge. The amount of new renalevidence is increasing exponentially and is dispersedacross more than 460 multidisciplinary journals.3,4 Thequality of renal systematic reviews may also be low.5

Sir Muir Gray of the U.K. National Health Service isquoted as saying, ‘‘Knowledge is the enemy of disease,’’making the point that better application of what weknow today will have a bigger impact on health and dis-ease than any single drug or technology likely to be intro-duced in the next decade.6 Thus, efficient knowledgedissemination in nephrology should be emphasized moreto facilitate increasingly positive impacts on the health ofCKD patients.

Nephrologists use a variety of sources to keep abreast ofthe medical literature and new research discoveries toimprove patient care.7 A survey of 100 Canadian nephrol-ogists indicated that the top 4 online resources used areUpToDate, PubMed, Google, and Ovid MEDLINE(Fig 1).7 This raises the question as to whether there isa preferred framework or hierarchy that nephrologistsshould consider when accessing information resources.In this article, we present the ‘‘6S’’ approach to evidence-based information access popularized by Haynes and col-leagues (Fig 2).8 This is a 6-level hierarchical structure ofevidence-based resources—from ‘‘systems’’ to ‘‘summa-

Advances in Chronic Kidney Disease, Vo

ries,’’ ‘‘synopses of syntheses,’’ ‘‘syntheses,’’ ‘‘synopses ofstudies,’’ and, finally, ‘‘studies’’—that intends to providenephrologists with an efficient method of high-quality in-formational retrieval. Physicians are recommended to be-gin their search for research evidence using resources atthe highest level of the hierarchy and, if their informationneed is notmet, thenwork their way down. An example ofsearching for the answer to ‘‘How does intradermal vs. in-tramuscular hepatitis B vaccine compare regarding effi-cacy and safety among chronic kidney disease patients?’’is shown in Figure 3. The highest level of evidence foundhere was in summaries, a nephrologist should start at thislevel and research further as needed through the otherresources.

Systems

Clinical decision support systems are designed to im-prove clinical decision making.9 Such systems may ormay not be computerized. The ideal system would seam-lessly integrate all important evidence from the medicalliterature about a clinical problem. At the point of care,

l 19, No 1 (January), 2012: pp 5-10 5

Bejaimal et al6

this information would be automatically linked to a par-ticular patient’s circumstances, potentially through anelectronic medical record (EMR).8 The system wouldbring forward recommendations to the health care pro-fessional that are specific to the patient’s health status,even if the physician is unaware that a knowledge gap ex-ists. The physicians would then decide whether to followthe recommendation, taking into account their clinical ex-pertise and patient preferences whenmaking the final de-cision.8 Such systems need to be up-to-date to stayeffective, and should do so with an explicit review pro-cess for finding and evaluating the best evidence.

There are a number of potential functions for thesesystems in CKD care, including drug prescribing,preventative care, and chronic disease management. In arecent systematic review on the use of clinical decisionsupport systems for kidney-related drug prescribing,we identified 32 studies. In 17 studies, systems were

CLINICAL SUMMARY

� Application of accurate and current clinical knowledge is

increasingly important in CKD patient care.

� Nephrologists must apply evidence-based principles when

searching for all forms of medical information.

� The ‘‘6S’’ approach of evidence-based resources provides

a hierarchical structure to high-quality evidence.

� Accurate clinical decision support systems are ideal;

however, they have yet to reach their full potential.

� Summaries of best evidence for individual clinical

problems (UpToDate, DynaMed, Best Practice, and PIER)

offer a great source for the care of renal patients.

� Nephrologists should use bibliographic databases such as

Google Scholar and PubMed with clinical query filters,

which can improve the precision of their search results,

allowing for more efficient information retrieval.

computerized, and in 15studies, they were manualpharmacist-based systems.10

Systems intervened gener-ally by prompting for drugdosing adjustments in rela-tion to the level of decreasedkidney function. Overall,the results from systemevaluations are promisingand warrant further study.Similarly, when high-quality studies of these sys-tems are considered for allareas of medicine, it can beconcluded that they gener-ally improve clinician per-formance.9 Looking ahead,however, these systems areyet to reach their full poten-tial. Many systems cover

a limited range of clinical problems and may not neces-sarily be based on the current best evidence. Ideally, inaddition to processes of care, more systems should betested for their impact on clinical outcomes as well. Inprevious studies such systems have had an inconsistenteffect on patient-important outcomes.9 The costs of thesesystems must also be taken into account. Thus, the designand evaluation of decision support systems is a complexundertaking. Nevertheless, such systems do hold themost promise of seamlessly integrating high-quality evi-dence into clinician decision making and workflow.8

Summaries

A summary of best evidence for individual clinical prob-lems within CKDmay be a nephrologist’s next best infor-

mation source for the care of renal patients. ‘‘Summaries’’or evidence-based clinical texts have a few distinctfeatures that set them apart from the traditional medicaltextbook. They require an explicit policy for finding,rating, and grading evidence; date-stamping of eachtopic; a schedule for updating; and in-line referencingof recommendations for care.8 Arguably the most popu-lar evidence-based text for such summaries in the renalcommunity is UpToDate (http://www.uptodate.com).UpToDate was founded by Dr. Burton Rose, a nephrolo-gist, which resulted in particular emphasis on renalconditions in the earlier iterations. UpToDate is a fre-quently updated, peer-reviewed information resourcethat uses paid writers for their summaries. It coversmost topics in the internal medicine subspecialties aswell as adult primary care, family medicine, obstetricsand gynecology, and pediatrics. The in-line supportingreferences that appear with summarized text include

key study descriptions anda link to MEDLINE ab-stracts. Although UpToDatehas lacked an explicit formalprocess for evaluating evi-dence, a new grading pro-cess is being implementedfor presented information.8

A similar resource is Dy-naMed (http://www.ebscohost.com/dynamed), whichprovides broad topic cover-age, including commonand uncommon diseases,symptoms, and informationon research, and is continu-ously updated.8

Other evidence-basedsummaries include BestPractice from the BritishMedical Journal Publishing

Group (http://bestpractice.bmj.com) that covers generalmedicine, including many common clinical problems.They review their evidence to high scientific standardsand post newly published evidence in updated tablesfor each topic. The Physicians’ Information and Educa-tion Resource (PIER) (http://pier.acponline.org/index.html) from the American College of Physicians (ACP) isanother evidence-based summary resource, with a focusin primary care.

Despite not being directly linked to patient data inEMRs, summaries can be run concurrently with EMRs tofacilitate the correct connection of evidence to a patient’sspecific circumstances, butwith opportunity for error.Ne-phrologists need to continue to use evidence-basedprinci-ples and judgment when using summaries and EMRs intandem.8

Figure 1. Online resources used by Canadian Nephrologists to guide patient care (sample of 115 nephrologists). Reprintedwith permission from Shariff et al, J Nephrol. 2011;24:723-732.7

Finding and Evaluating Renal Evidence 7

Synopses of Syntheses

Synopses of syntheses are concise 1-page structureddescriptions of systematic reviews—reports that systemati-

Figure 2. The 6S hierarchy of preappraised evidence. Abbreviatclinical evidence; PIER, Physicians’ Information and Educationevidence-based medicine; EBN, evidence-based nursing; DARwith permission of the American College of Physicians from DiC

cally assemble, critically appraise, and synthesize all relevantstudies on a specific question. Such synopses allow nephrol-ogists to access high-quality information without having to

ions: ACP M, American College of Physicians Medicine; CE,Resource; UTD, UpToDate; ACPJC, ACP Journal Club; EBM,E, Database of Abstracts of Reviews of Effects. Reprintedenso et al, Ann Intern Med. 2009;151:JC3-2, JC3-3.8

Figure 3. Working through the 6S hierarchal structurewith a renal clinical question. A nephrologist should start at the highestlevel of available information and work through the other resources as needed.

Bejaimal et al8

examine a long synthesis. These synopses are included insources such as ACP Journal Club (http://www.acpjc.org)and the Database of Abstracts of Reviews of Effects(http://www.crd.york.ac.uk/crdweb). To increase informa-tion retrieval efficiency, sources should provide enough in-formation to support a clinical action.8 For example, ACPJournal Club uses declarative titles and provides the key in-formation to allowadecisionmaker toproceed. Formorede-tailedunderstanding, the source provides links to the reviewor its original articles.

Syntheses

When properly conducted, a systematic review is anefficient way to understand the available evidence fora clinical question. Combining results quantitatively inmeta-analysis, when it is appropriate to do so, can help es-timate an effect with more precision. More than 400 renalsystematic reviews and meta-analyses have been pub-lished between 1990 and 2006, with the annual numberincreasing with time.11 The largest single collection of syn-theses is in the Cochrane Database of Systematic Reviews,where the traditional focus has been on preventative ortherapeutic interventions (http://www.cochrane.org/reviews/). Cochrane has now started to summarize diag-nostic test evidence as well. Cochrane reviews are oftenupdated as new literature is introduced. The CochraneRenal Review Group is managed from Sydney, Australia.This group is tasked with identifying pertinent high-quality renal clinical studies, analyzing the trial data, andpreparing structured reports for inclusion in the CochraneDatabase of Systematic Reviews. To date, the CochraneRenal ReviewGroup has more than 80 renal reviews listedon their Web site (http://www.cochrane-renal.org).

Another source of preappraised syntheses is Eviden-ceUpdates, sponsored by British Medical Journal Group(www.EvidenceUpdates.com). EvidenceUpdates includesreviews from a wide breadth of sources: all Cochrane re-views, syntheses from more than 120 clinical journals andreviews from the U.S. Agency for Healthcare ResearchandQuality, CanadianAgency forDrugs andTechnologiesin Health, and U.K. National Health Service Health Tech-nology Assessments. Other sources of systematic reviewsinclude Database of Abstracts of Reviews of Effects,Cochrane Library (http://www.thecochranelibrary.com),and Ovid’s Evidence-Based Medicine Reviews service(http://www.ovid.com/site/catalog/DataBase/904.jsp).An important caveat of all syntheses is that the quality ofrecommendations is only as good as the quality of the pri-mary evidence in the original studies.

Synopses of Studies

Synopses can also be made for individual high-qualitystudies. These are typically 1-page structured abstractswritten for those studies that have met methodologicalquality standards and are clinically relevant to their field.Nephrologists can find these synopses in ACP JournalClub.8 A simple search of the term ‘‘chronic kidney dis-ease’’ retrieves more than 110 citations. Examples ofavailable topics for the care of patients with CKD includeoutcomes of erythropoietin use, risk of CKDwith obesity,and the utility of computed tomographic angiography forthe diagnosis of renal artery stenosis.

Studies

It takes time to summarize data that are originally foundin primary articles. If nephrologists are only looking at

Finding and Evaluating Renal Evidence 9

syntheses and summaries, information from newly pub-lished clinical studies may go unseen. Nephrologists canthus use original studies to bridge knowledge evidencegaps. A key issue is how a clinician goes about findingprimary studies for a given topic area like CKD. Physi-cians can either search for, or be alerted to, new articlesin their field. Nephrologists can use evidence ‘‘refinery’’systems that identify studies for them. These systemsare generally organized according to a targeted clinicaldiscipline: Essential Evidence Plus for general practice(http://www.essentialevidenceplus.com/); ACP JournalClub for primary care, internal medicine, and subspe-cialties; and EvidenceUpdates for general medicine andall major specialties. Within these services, nephrologistscan subscribe to have alerts sent to them as and whennew articles within their clinical discipline are published,aiding in information dissemination.

Searching bibliographic databases provides anothersource for nephrologists to find studies, but articles re-trieved by this route must be sorted for relevance and crit-ically appraised by the searcher. A recent survey of 100randomly sampled Canadian nephrologists showed thatPubMed and Google Scholar are the 2 most frequentlyused, freely accessible bibliographic databases.7 Thebreadth of PubMed is apparent—it is a multidisciplinarydatabase with more than 21 million articles and 5200 jour-nals. Although the total amount of evidence available isvast, finding relevant nephrology articles can be difficult,despite frameworks designed to structure the question ofinterest.12 The use of PubMed search filters may providea solution to this problem.11 Filters are objectively derived,pretested strategies designed to retrieve articles for a spe-cific clinical purpose.13,14 Currently, PubMed provides2 types of search filters—methods-based and topic-basedfilters. Methods-based filters (known as clinical queries)were designed to retrieve articles on therapy, diagnosis,prognosis, and etiology. Methods-based filters can be ap-plied to any clinical discipline and are available for generaluse in PubMed (http://www.ncbi.nlm.nih.gov/pubmed/clinical). Topic-basedfilters, in comparison, are designed toretrieve articles within a specific discipline or topic.13,15,16

Examples of renal filters can be found at http://hiru.mcmaster.ca/hiru/HIRU_Hedges_Nephrology_Filters.aspx. Methods- and topic-based filters help remove irrele-vant articles to increase the precision of one’s search.15,17

More recently, Google Scholar has gained popularity asan alternative freely accessible bibliographic database.Google Scholar indexes material from many sources—peer-reviewed papers, theses, abstracts, and books—andits goal is to ‘‘provide a simple way to search foracademic literature’’ and ‘‘to sort articles the wayresearchers do.’’18 A study recently done by Shariff andcolleagues compared the performance of PubMed andGoogle Scholar when using nephrologist searches. GoogleScholar was able to find twice the number of relevantarticles within the first 40 results, compared with PubMed.

Google Scholar searches also demonstrated increasedprecision—within first few result pages received, 1 of 6 re-sults was relevant in Google Scholar versus 1 in 20 forPubMed. These results highlight the potential differencesbetween these 2 bibliographic databases and provide newinsights on how these databases can be best used by busynephrologists.

Conclusion

Knowledge is a key step on the path to positively affect-ing patient health. Bridging the knowledge gap betweenavailable current best evidence and nephrologists’ clini-cal practice can facilitate this.1 Being armed with the toolsto find the current best evidence enables this greatly.Using the 6S approach when searching for evidence canincrease the efficiency of one’s information retrieval,with the goal of realizing paralleled health benefits forpatients with CKD.8

References

1. Haynes RB, Devereaux PJ, Guyatt GH. Physicians’ and patients’choices in evidence based practice. BMJ. 2002;324:1350.

2. Levey AS, Andreoli SP, DuBose T, et al. Chronic kidney disease:common, harmful and treatable–World Kidney Day 2007. Am J

Nephrol. 2007;27:108-112.3. Fouque D, Laville M, Haugh M, et al. Systematic reviews and their

roles in promoting evidence-based medicine in renal disease. Neph-rol Dial Transplant. 1996;11:2398-2401.

4. Garg AX, Iansavichus AV, Kastner M, et al. Lost in publication: halfof all renal practice evidence is published in non-renal journals.Kidney Int. 2006;70:1995-2005.

5. Mrkobrada M, Thiessen-Philbrook H, Haynes RB, et al. Need forquality improvement in renal systematic reviews. Clin J Am Soc

Nephrol. 2008;3:1102-1114.6. Pang T, Gray M, Evans T. A 15th grand challenge for global public

health. Lancet. 2006;367:284-286.7. Shariff SZ, Bejaimal SA, Sontrop JM, et al. Searching for medical

information online: a survey of Canadian nephrologists. J Nephrol.2011;24:723-732.

8. DiCenso A, Bayley L, Haynes RB. ACP Journal Club. Accessingpreappraised evidence: fine-tuning the 5S model into a 6S model[editorial]. Ann Intern Med. 2009;151. JC3-2, JC3-3.

9. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerizedclinical decision support systems on practitioner performance andpatient outcomes: a systematic review. JAMA. 2005;293:1223-1238.

10. Tawadrous D, Shariff SZ, Haynes RB, et al. Use of clinical decisionsupport systems for kidney-related drug prescribing: a systematicreview. Am J Kid Dis. 2011;58:903-914.

11. Garg AX, Hackam D, Tonelli M. Systematic review and meta-analysis: when one study is just not enough. Clin J Am Soc Nephrol.

2008;3:253-260.12. Webster AC, Cross NB, Mitchell R, et al. How to get the most from

the medical literature: searching the medical literature effectively.Nephrology (Carlton). 2010;15:12-19.

13. Haynes RB, Wilczynski N, McKibbon KA, et al. Developing optimalsearch strategies for detecting clinically sound studies inMEDLINE.J Am Med Inform Assoc. 1994;1:447-458.

14. Jenkins M. Evaluation of methodological search filters: a review.Health Info Libr J. 2004;21:148-163.

Bejaimal et al10

15. Haynes RB, McKibbon KA, Wilczynski NL, et al. Optimal searchstrategies for retrieving scientifically strong studies of treatmentfrom Medline: analytical survey. BMJ. 2005;330:1179.

16. National Library of Medicine (US). PubMed Clinical Queries, 2008.Available: http://www.ncbi.nlm.nih.gov/pubmed/clinical. AccessedAugust 1, 2011.

17. Garg AX, Iansavichus AV, Wilczynski NL, et al. Filtering Medlinefor a clinical discipline: diagnostic test assessment framework.BMJ. 2009;339:b3435.

18. Google 2007. Google Scholar Beta: About Google Scholar. 2011. Avail-able: http://scholar.google.ca/intl/en/scholar/about.html. Accessed:July 7 2011.