Putting guidelines into practice: a tailored multi-modalapproach to improve post-operative assessmentsjep_1780 1..6

John A. Ford MB ChB DTM&H,1 Craig MacKay MB ChB (MRCS) BSC (Hons),2 Chris Peach,3

Paul Davies MB ChB MRCSEd MRCGP4 and Malcolm Loudon MB ChB MD FRCSEd FRCS (Gen)5

1Research Assistant, 2Surgical Trainee, 3Medical Student, 4General Practitioner, 5Consultant Surgeon, Department of General Surgery, AberdeenRoyal Infirmary, Aberdeen, UK

Keywords

audit, behavioural change, guidelines,multi-modal approach, post-operativeassessment

Correspondence

Dr John FordHTA GroupUniversity of AberdeenPolwarth BuildingForesterhillAberdeen AB25 2ZDUKE-mail: john.ford@abdn.ac.uk

Accepted for publication: 10 August 2011

doi:10.1111/j.1365-2753.2011.01780.x

AbstractBackground The Scottish Intercollegiate Guideline Network (SIGN) published Postop-erative Management in Adults in 2004, advocating post-operative assessments to optimizepost-operative care. Our aim was to improve post-operative assessments in a surgicalhigh-dependency unit (HDU).Methods A prospective audit of post-operative admissions to surgical HDU over two4-week periods was performed. Medical and nursing documentations were reviewed. Atailored multi-modal approach targeting specific barriers to change was used to implementchanges; education of staff, introduction of designated HDU bleeps and a post-operativeassessment pro forma. Re-audit was performed after 6 months.Main findings The first cycle included 72 patients and the second included 62 patients.Time to assessment improved after changes. Forty-six (74%) patients compared with 27(37%) patients before were assessed within 4 hours. The number of individual reviewsincreased and number of reviews due to nursing concerns decreased. Thirty-eight (61%)patients compared with 15 (21%) patients before were assessed through an individualreview and one (2%) patient compared with 23 (32%) patients due to nursing concerns.Documentation improved. Documentation of relevant past medical history, medications,allergies, complications and post-operative instructions improved from 2 (3%), 1 (1%), 0,8 (11%) and 26 (36%), to 18 (29%), 28 (45%), 20 (32%), 18 (29%) and 55 (89%),respectively. Difference between first and second cycles was highly significant throughout(P < 0.001).Conclusion Clinical practice was improved by a tailored multi-modal approach. Educat-ing staff, improving communication and documentation, and re-audit has shown significantimprovement. However, further improvements are required to reach best practice.

IntroductionPost-operative complications cause considerable morbidity andmortality [1]. The Scottish Audit of Surgical Mortality (SASM)reported 1140 post-operative deaths in 2009 [2]. In particular,patients appear to be at risk during the first 6 hours post-operatively [3]. Of the fully reported cases to SASM (n = 514),24.5% of cases had at least one area of concern (ACON). CommonACONs included post-operative fluid overload, poor communica-tion and hypotension during regional anaesthetic. Previous reportshave highlighted delay in recognizing a clinical deterioration as animportant factor [4]. SASM suggested that junior medical staffwere failing to recognize decline, failing to involve consultants atan early stage and not recognizing the dangers of not actingpromptly when decline is identified. These concerns are echoes ofother studies proposing a ‘failure to rescue’ patients from post-

operative complications [5]. Of further concern is a recent studythat suggested management, rather than incidence, of complica-tions effect post-operative mortality [6].

In response to SASM concerns, the Scottish IntercollegiateGuidelines Network (SIGN) group published guidance regardingthe management of post-operative complication [7]. The guidelinecovers assessment of the post-operative patient, good post-operative care and management of common complications. It isaimed at all multidisciplinary health care professionals, but espe-cially junior medical staff. The guidelines suggest that a post-operative assessment should be carried out ‘immediately’ and thata patient returns to the ward and should be performed by ‘thedoctor responsible in the first instance for patient care, usually thehouse officer’.

Aberdeen Royal Infirmary is a large teaching hospital with over1000 beds. There are four general surgical wards and one vascular

Journal of Evaluation in Clinical Practice ISSN 1365-2753

© 2011 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice 1

ward which have access to two high-dependency units (HDU).About 70% of high-dependency admissions are in the immediatepost-operative period [8]. Guidelines can be difficult to implement.Therefore, to assess and improve our service, we conducted a fullcycle audit of post-operative assessment using a multi-modalapproach to improve practice. Although SIGN suggests that allpost-operative patients should undergo a post-operative assess-ment, we targeted our audit to post-operative HDU patients.

MethodsA prospective full cycle audit was performed. Data were collectedover two 4-week periods and a 6-month period was allowed toincorporate implemented changes. Included patients were requiredto be: (1) admitted to HDU immediately after surgery; and (2)under the care of general surgical or vascular teams. Patients wereexcluded if admitted to HDU because of deterioration on the ward,spent time post-operatively in the intensive care unit or under thecare of another speciality. Audit criteria was defined as all post-operative patients requiring HDU care should immediately (withinan hour of admission) undergo a post-operative assessment byjunior medical staff. The audit standard was set at 75% to allow forunforeseen clinical circumstances.

Patients were identified by admission book held in HDU whichwas updated daily by secretarial staff. After a patient had beendischarged from HDU, data were collected from the medical andnursing notes. Variables collected included admission to HDUdetails, patient demographics, ASA level, type of operation (emer-gency or elective), initial nursing and medical assessment, grade ofmedical staff, reason for review, nature of review, and dischargefrom HDU details. Data were collected over a 4-week period.

Post-operative reviews were assessed quantitatively and quali-tatively. Quantity was measured by calculating the number ofassessments performed and the time difference between admissionto HDU and first assessment by medical staff. Reviews were clas-sified as routine individual, ward round or because of clinicalconcern on behalf of nursing staff. Quality was measured byassessing the documented review against SIGN post-operativeassessment criteria [7], including intra-operative complications,post-operative instructions, significant past medical history,current medications and allergies.

Strategy for changeAfter discussion with the multidisciplinary team, a multi-modalapproach was chosen to improve compliance.

• Education of medical and nursing staff. Results were presentedat weekly meetings and posters were made to remind staff aboutwhen and how a post-operative assessment should take place. Allnew medical staff were informed of the need to perform assess-ments and to be familiarized with the pro forma.• Improved communication. Designated HDU bleeps were createdas a single point of contact, making it easier for HDU staff tocontact junior medical staff.• Post-operative checklist. A pro forma was introduced to assistjunior medical staff, allowing quick and effective assessments ofpatients (Appendix 1).A re-audit was performed using the same method after a period of6 months. Statistical analysis was performed using chi-square testat 95% significance with SAS output [9].

ResultsSeventy-two patients were included before changes compared with62 patients after changes. Demographics are shown in Table 1.Baseline characteristics were comparable. More patients wereexcluded in the second group.

Time to assessment

Before implemented changes, only nine (12%) patients wereassessed within 1 hour, 27 (37%) patients were assessed within 4hours and 25 (35%) patients waited over 12 hours for their firstpost-operative assessment (Fig. 1). However, after changes wereimplemented, 25 (40%) were assessed within 1 hour, 46 (74%)were assessed within 4 hours and only 10 (16%) had to wait over12 hours.

Type of assessment

After changes, more assessments were completed as individualreviews rather than during ward rounds or because of clinicalconcern. Before changes, only 15 (21%) patients underwent indi-vidual review, 34 (47%) had their review during ward round and 23(32%) had their review because of clinical concern (Fig. 2).However, after changes, 38 (61%) underwent individual reviews,23 (37%) had their review during ward round and only 1 (2%)patient underwent review because of clinical concern.

Quality of assessment

Before implemented changes, there was very poor compliancewith the guidelines in regard to the nature of each assessment

Table 1 Patient demographicsFirst cycle Second cycle

Total 96 105Included 72 62Excluded 24 43Age (years) Mean 65 66

Range 33–88 24–91Sex (%) Male 36 (50) 36 (58)

Female 36 (50) 26 (42)Urgency of operation (%) Emergency 48 (67) 40 (66)

Elective 24 (33) 22 (35)Speciality (%) General surgery 50 (69) 42 (68)

Vascular surgery 22 (31) 20 (32)

Multi-modal approach to implementing guidelines J.A. Ford et al.

© 2011 Blackwell Publishing Ltd2

(Fig. 3). No assessments documented all criteria [compared with16 (26%) after changes], only 2 (3%) included past medicalhistory [compared with 18 (29%) after changes], 1 (1%) assess-ment included current medication [compared with 28 (45%)], noassessments documented allergies [compared with 20 (32%)], 8(11%) documented intra-operative complications [compared with18 (29%)] and 26 (36%) had post-operative instructions [com-pared with 55 (89%)].

Statistical analysis

Statistical analysis demonstrated a highly significant difference (Pvalue <0.001) in all categories (time to assessment, quality ofassessment and nature of assessment) when comparing time beforeand after implemented changes. There was a highly significantimprovement (P value <0.001) in all aspects of the post-operativeassessment (documented past medical history, medications, aller-gies, intra-operative complications and post-operative instruc-tions).

DiscussionClinical practice has significantly improved though a tailoredmulti-modal approach; education, improved communication and a

post-operative pro forma. After changes, almost 40% morepatients were being assessed within 4 hours, considerably fewerassessments were performed because of clinical concern andquality of assessment has considerably improved; however, furtherimprovement is still needed to reach our predefined audit standard.

Strengths and limitations

Collecting data prospectively improves the validity and resulted inno patients being lost to follow-up. Including a measure of quality,not only quantity, has obvious advantages. However, this audit hassome limitations. Our audit was dependent on clear documenta-tion. Patients may have been assessed without documentation andtherefore missed during data collection. However, the authorsmaintain that failing to document assessments is unsafe. Withoutdocumentation, staff are left to pass on instructions, often causingconfusion, and deteriorating examination findings may be missed.There were relatively small numbers in both groups but statisticalanalysis demonstrated significance.

Implemented changes

The audit team identified various contributing factors to poor com-pliance; therefore, a tailored multi-modal approach was necessary.

45

40

35

30

25

20

Perc

enta

ge o

f patients

15

10

5

0

Less than 1 1 to 3 4 to 8

Time (hours)

9 to 12 More than 12

Before

(n = 72)

After

(n = 62)

Figure 1 Time from admission to assess-ment.

70

60

50

40

30

20

Perc

enta

ge o

f patients

10

0

Individual Afternoon

ward round

Morning

ward round

ATSP*

Before

(n = 72)

After

(n = 62)

Figure 2 Type of assessment. *ATSP, askedto see patient due to clinical concern.

J.A. Ford et al. Multi-modal approach to implementing guidelines

© 2011 Blackwell Publishing Ltd 3

It was clear that compliance with the guidelines was poor in allrespects before implemented changes. There were several possibleexplanations. Junior medical staff were frequently unaware of theSIGN guidelines or did not understand what should be included ina post-operative assessment. Nursing staff reported a difficulty incontacting the appropriate medical staff.

Time to assessment improved

Undoubtedly, making staff aware of the guidelines and having adesignated bleep helped. We provided information about theguidelines on several levels: information in starter packs; distribu-tion of guidelines; departmental teaching sessions; designatedaudit meetings; discussion with, and letters to, individual juniormedical staff and nursing staff. The use of designated HDU bleepsproved popular with both medical and nursing staff. Nursing stafffound it easier to contact medical staff. Medical staff found iteasier to designate one person to be responsible for HDU each day.

Significantly fewer assessments because of

clinical concern

If junior medical staff are able to assess patients and manage anyimmediate problem, this pre-empts a request by nursing staff toreview. However it is possible that junior medical staff took theopportunity to complete a post-operative assessment when beingcalled to address clinical concern.

Quality of assessments improved

The guideline suggests that criteria such as past medical historyand current drug treatments are included in an assessment. Somefeedback by members of staff felt that it was inefficient tore-document these criteria. However, the authors feel that it isimportant to document any medical problems which may be rel-evant to post-operative recovery. Current drug treatments are alsoimportant to document a plan for post-operative pain, antibiotics,

anti-emetics, etc. Unsurprisingly, the pro forma was used in allcases where the post-operative assessment was fully performed.

Post-operative assessments have considerable benefits. Theadvantages are threefold: firstly, post-operative assessmentsprovide a clinical baseline for further assessments; secondly, itimproves junior medical staff’s ability to differentiate betweennormal and abnormal recovery; and thirdly, it provides an oppor-tunity to review analgesia, anti-emetics, fluids and post-operativeinstructions.

How does this study contribute tocurrent literature?

Implementing changes to improve practice can be challenging.Several systematic reviews have studied approaches to implement-ing guidelines [10–13]. In a systematic review, Bero et al. foundthat reminders and multi-modal strategies were consistently effec-tive measures [13]. This supports our implemented changes andreinforces the message that various interventions are often neededto address different problems. We found a lack of knowledgeamong junior medical staff and poor communication betweennursing and medical staff. One intervention was unlikely toaddress both these issues. However, multifaceted strategies are notalways successful and their success depends on various issues[14,15]. We advocate an extension of the multi-modal approach; atailored multi-modal approach designed to overcome pre-specificbarriers, as described by Cahill and Heyland [16] Instead of apply-ing a generic multi-faceted approach, specific barriers to behav-ioural change should be identified and a unique multifacetedstrategy devised.

Further research

The aim in compliance with this guideline is to reduce post-operative morbidity and mortality while speeding recovering andtraining junior medical staff. Parts of this guideline are unique asthey have been created on the consensus method. Although it

90

80

70

60

50

40

30

20

Per

cen

tag

e o

f p

atie

nts

10

0

Criteria documented

Full

Past m

edical h

isto

ry

Intra

-ope

rativ

e co

mplicat

ions

Allerg

ies

Med

icat

ion

Post-o

pera

tive

instru

ctions

Before

(n = 72)

After

(n = 62)

Figure 3 Quality of assessment.

Multi-modal approach to implementing guidelines J.A. Ford et al.

© 2011 Blackwell Publishing Ltd4

seems good clinical practice to perform post-operative assessment,currently there are no studies to support or refute post-operativeassessments in the reduction of morbidity or mortality. Therefore,further research is needed.

ConclusionCompliance with SIGN guidelines was poor even in patientsrequiring HDU care. However, clinical practice and concordancewith guidelines were improved by a tailored multi-modal approachby educating staff, improving communication through develop-ment of the bleep system and introduction of a post-operativeassessment pro forma. Although predefined audit standard was notreached, the strategy used shows promise and further audit cycleswould be needed to improve clinical practice further. This studydemonstrated that a complete audit cycle with targeted interven-tions can be effective in improving compliance with evidence-based best clinical practice.

References1. Rosen, A. K., Geraci, J. M., Ash, A. S., McNiff, K. J. & Moskowitz,

M. A. (1992) Postoperative adverse events of common surgical pro-cedures in the Medicare population. Medical Care, 30 (9), 753–765.

2. SASM (2010) Scottish Audit of Surgical Mortality Summary Report2009 Data. Edinburgh: ISD Scotland Publications.

3. Lee, A., Lum, M. E., O’Regan, W. J. & Hillman, K. M. (1998) Earlypostoperative emergencies requiring an intensive care team interven-tion. The role of ASA physical status and after-hours surgery. Anaes-thesia, 53 (6), 529–535.

4. SASM (2009) Scottish Audit of Surgical Mortality Summary Report2008 Data. Edinburgh: ISD Scotland Publications.

5. Silber, J. H., Williams, S. V., Krakauer, H. & Schwartz, J. S. (1992)Hospital and patient characteristics associated with death after surgery.A study of adverse occurrence and failure to rescue. Medical Care, 30(7), 615–629.

6. Ghaferi, A. A., Birkmeyer, J. D. & Dimick, J. B. (2009) Variation inhospital mortality associated with inpatient surgery. New EnglandJournal of Medicine, 361 (14), 1368–1375.

7. SIGN (2004) Post-Operative Management in Adults, A PracticalGuide to Post-Operative Care for Clinical Staff. Edinburgh: ScottishIntercollegiate Guidelines Network.

8. Turner, M., McFarlane, H. J. & Krukowski, Z. H. (1999) Prospectivestudy of high dependency care requirements and provision. Journal ofthe Royal College of Surgeons of Edinburgh, 44 (1), 19–23.

9. SAS Institute Inc. (2008) SAS Output. Version 9.2.10. Grimshaw, J., Eccles, M. & Tetroe, J. (2004) Implementing clinical

guidelines: current evidence and future implications. Journal of Con-tinuing Education in the Health Professions, 24 (Suppl. 1), S31–S37.

11. Grol, R. & Grimshaw, J. (2003) From best evidence to best practice:effective implementation of change in patients’ care. Lancet, 362(9391), 1225–1230.

12. Oxman, A. D., Thomson, M. A., Davis, D. A. & Haynes, R. B. (1995)No magic bullets: a systematic review of 102 trials of interventions toimprove professional practice. CMAJ, 153 (10), 1423–1431.

13. Bero, L. A., Grilli, R., Grimshaw, J. M., Harvey, E., Oxman, A. D. &Thomson, M. A. (1998) Closing the gap between research and prac-tice: an overview of systematic reviews of interventions to promote theimplementation of research findings. The Cochrane Effective Practiceand Organization of Care Review Group. BMJ, 317 (7156), 465–468.

14. Bessen, T., Clark, R., Shakib, S. & Hughes, G. A. (2009) A multifac-eted strategy for implementation of the Ottawa ankle rules in twoemergency departments. BMJ (Clinical Research Ed.), 339,b3056.

15. Mourad, S. M., Hermans, R. P. M. G., Liefers, J., Akkermans, R. P.,Zielhuis, G. A., Adang, E., Grol, R. P. T. M., Nelen, W. L. D. M. &Kremer, J. A. M. (2011) A multi-faceted strategy to improve the use ofnational fertility guidelines; a cluster-randomized controlled trial.Human Reproduction, 26 (4), 817–826.

16. Cahill, N. E. & Heyland, D. K. (2010) Bridging the guideline-practice gap in critical care nutrition: a review of guideline implemen-tation studies. Journal of Parenteral and Enteral Nutrition, 34 (6),653–659.

J.A. Ford et al. Multi-modal approach to implementing guidelines

© 2011 Blackwell Publishing Ltd 5

Appendix 1

Name ............................................. Ward Cons

Unit No. .......................................... Date

DOB .............................................. Time

Operation

Indication Current medications

Intra-operative history and complications

Blood loss

Significant PMH

HR BP RR SaO2 Temp Allergies

Urine Output Drain Output Pain score

Examination findings

AVPU

Plan (inc post-op instructions)Pain control:DVT prophylaxis:Nausea + Vomiting:Antibiotics:

Signed Bleep

Multi-modal approach to implementing guidelines J.A. Ford et al.

© 2011 Blackwell Publishing Ltd6