How to Perform Clinical Audit

How to perform clinical audit?

Dr. Mithali Abdullah @ Jacquline Sapen

MSc. OSHM (UUM), M.B.B.S (Aust.), OHD (NIOSH), CMIE (NIOSH)

Timbalan Pengarah (Perubatan)II

Hospital Sultan Abdul Halim, Sungai Petani, Kedah

WC

WHY

WHAT

HOW

Source: Start with Why (2009) Simon Sinek

What is your cause?

What motivates you

to do clinical audit?

Road Map of talk

Revisit of Clinical audit

Clinical audit cycle

Stages in clinical audit

INTRODUCTION

“Quality is not an accident, it is the result of

high intentions,

sincere efforts,

intelligent direction and

skillful execution”

Clinical Audit

Clinical audit is a specific form of audit that involves measuring clinical practice against standards. An audit within the clinical setting is not necessarily a clinical audit project.

Clinical audit would measure process( are we doing the things we should do?)

Clinical audit could also look at outcomes: such as monitor the success of a treatment which is known to work, rather than to find out whether it works.

Clinical audit vs Research

Clinical Audit

• Ask the question „are we following agreed or evidence based practice?‟

• Seek to find out if we are achieving the things we agreed we should achieve

Research

• Ask the question ‘what is the best practice?‟

• Researches will seek to find new knowledge

Audit Cycle The clinical audit process is known as the „audit cycle‟.

Choose a topic

Review standard

Collect data on current

practice

Compare data collected with

standard

Implement change to improve

Re-audit to look for

improvement

Source: NICE 2002

STAGES IN CLINICAL AUDIT

Stages in clinical audit

Stage 1: preparation: management & methodology

Stage 2: selecting criteria and standard

Stage 3: measuring level of performance

Stage 4: making improvements

Stage 5: sustaining improvements

Stage 1: preparation

Good preparation is crucial to the success of an audit project.

“If you fail to plan, you plan to fail.”

The audit team should decide in advance the audit project for the forthcoming year, time frame for data collection, design and format of the necessary data collection tools.

Ref: Principals for best practice in clinical audit. National Institute for Clinical Excellence. 2002. Radcliffe Medical Press Ltd.

2 areas to be considered in

preparation

Project management

• Team

• Leader

• topic selection,

• planning and resources,

• communication

Project methodology

• Study design

• Data issue: collection, analysis, storage

• Implementability

• Stakeholder involvement: patients, other HCW

• Support for local improvement

PROJECT MANAGEMENT

Stage 1: Step 1

The team

Identifying the skills and people needed to carry out the audit, and

training of staff and encouraging them to participate.

Involve the right people with the right skill from the start.

Certain skills are needed, which include:

1. Project leadership, Project management, project organization

2. Clinical, managerial, and other service input and output

3. Audit method expertise

4. Change management skills

5. Data management: data collection, data entry, data analysis, &

data presentation

6. Facilitation skills.

The Team

Audit team comprises :

staff from all relevant groups involved in the care delivery, audit staff

All audit team members:

1. must understand the processes of clinical care concerned

2. Must have a basic understanding of clinical audit

3. Understand the purpose of the audit

4. Committed to the plan and objectives of the audit

5. Understand what is to be expected of them – specific roles &

responsibility: leader, data management, communication, etc.

6. Must understand the ground rules for meeting

The leader

The leader

“Leadership is the art of getting someone else to do

something you want done, because he wants to do it.”

~ Dwight D. Eisenhower

Topic selection

1. Involve users (health care providers or patients) in the

process

2. Based on SMART crtiteria

Involve users (health care providers or

patients) in the process

Sources of user information:

1. Critical incident reports

2. Complaints or comments

3. Direct observation of care

4. Direct observations

5. Focus group discussion

Selection of topic: SMART

Seriousness: patient safety issues, high volume, high

risk, high cost

Measurable: any evidence-based standard, feasibility of

data management

Appropriateness: is the problem important? Is the

problem a priority?

Remediable: Is the problem concern amenable to

change?

Timely: should we deal with the problem now or later?

Defining the purpose of audit

The aims of the audit must be defined.

Key words that could be used include: to improve, to enhance, to

ensure, to change.

E.g. :

1. To improve blood transfusion process in the hospital

2. To ensure that Safe Surgery Save Life Initiative is implemented

3. To enhance the referral of stroke patients in the hospital

4. To increase the percentage of patients with controlled blood

pressure.

Purpose of audit

What aspects of care that concern us?

Which aspect of care that you would like to

audit?

• Is the treatment being administered correctly & does it have the desired effect?

Effectiveness

• Is this approach achieving the desired outcome with minimal effort, expense, & wastage ?

Efficiency

• Do all patients have equal access to care?Equity

Purpose of audit

• Is it easy for patients to get treatment? Waiting time?

Accessibility

• Is this the right management approach?

Appropriateness

• Is the treatment acceptable to patients? As per standard/guideline?

Acceptability

• Is the care being provided at the correct time?

Timeliness

Providing the necessary structure

1. A structured audit program is needed: committee structure,

feedback mechanism, regular audit meeting

2. A team of well-qualified audit staff.

Funding is important. This include resources for staff training &

administration, knowledge of clinical audit technique, data

management.

Time is another important barrier in successful audit. Participants

should be given the time to participate in the project.

Communication

Inform all those who are going to be involved.

Platforms for communication include:

memo/letter, e-mail, meeting

PROJECT METHODOLOGY

Stage 1: Step 2

Project methodology

Design of study: retrospective , prospective, cohort, case

studies, case review

Data collection: what, where, when, how

Sample : subjects to be selected. Inclusion & exclusion

criteria

Sampling technique & sample size

Type of data analysis: qualitative, quantitative

Study design: retrospective or cross

sectional data collection

1. Retrospective data collection:

look at previous records.

Could be difficult to trace relevant records.

Time consuming

Relevant information required may not be available: incomplete

records

2. Cross sectional data collection:

gives audit team more immediate feedback on its current

performance

Relevant information required would be easier to retrieve:

health care providers informed prior to study on documentation

of information.

Data collection

What: collect data that is related to audit criteria

Who(samples) from whom these data is gathered?

Patients? Healthcare providers? Inclusion/Exclusion

criteria

Who: by trained staff.

Where: unit/department

Data collection

When: inpatient, outpatient, on admission, after

discharge, during administration of medicine, during

observed procedure?

How: include observational checklist, questionnaires

(self-completed or interviewer completed), patient

medical records, interviews

Data storage: data collection format

An Audit on the Use of Granisetron Injection for

Chemotherapy-Induced Nausea and Vomiting in Hospital Sultanah Bahiyah

Pharmacy Department

Supervisor : Malathi Sriraman @ Jayaraman

Group Leader : Chan Huan Keat

Group members: Khor Seau Ting

Tan Say Li

34

Data collection pertaining to audit criteria

Quality of Care Selected

Rational use of granisetron injection in terms of:(a) Indication.(b) Dose.

Audit Criteria & Standard

NO Quality of Care Criteria Standard

1. Indication of granisetron injection

Only prescribed in moderately and highly

emetogenic chemotherapy unless it is justified.100 %

2 Dose of granisetron injection

Only prescribed at the lowest effective dose, which

is 1mg or 0.01mg/kg unless it is justified. 100%

Ref. Systemic Therapy of Cancer 2nd Edition, Ministry of Health Malaysia, 2007.

35

Objective 1: To determine the adherence of prescribers to recommended

indications and doses of granisetron injection by the updated guidelines.

Study

Design

Prospective study:

To check prescriptions for adherence to the

recommended indications & doses.

Inclusion

Criteria

All adult chemotherapy prescriptions

prescribed with granisetron injections

received from Jan-Mac 2011.

Exclusion

CriteriaPrescriptions with incomplete information.

Population

Size

N = 432 prescriptions.

All included.

Data

Collection

1. Receive & screen the prescriptions.

2. Record the required information

into Data Collection Form A.

Data

Analysis

SPSS 16:

(a) Descriptive analysis.

Data collection

You may like to know the factors that contribute to

substandard performance such as knowledge,

perception and practice.

Thus, design data collection tool to collect such data.

Include questionnaire (patients or HCW) &

observational checklist (to determine if knowledge is

translated into practice)

37

Objective 2: To assess prescribers’ perception of the granisetron

injection utilization in the prevention of CINV.

Study

Design

Cross-sectional study:

To study the prescriber’s perception using a

validated questionnaire modified from

Tajunisah M.E. et al (Cronbach’s α = 0.72). 8

Inclusion

Criteria

All doctors in 10 wards that provides

chemotherapy during Jan-Mac 2011.

Sampling

Method

Convenient sampling.

Sample

Size

n = 63 doctors.

Data

Collection

1. Distribute Data Collection Form B (self-

reported questionnaire) to wards.

2. Collect the forms back after completely

answered by doctors.

Data

Analysis

SPSS 16:

1. Descriptive analysis.

2. Inferential analysis: Chi Square tests.

8. Tajunisah M.E. et al, Physicians’ Perception and Adherence to Guideline on the Management of Chemotherapy

Induced Nausea & Vomiting (CINV). Hospital Pulau Pinang, 2009.

Names of Auditors:___________________Ward/Department:____________________Date:_______________

BHT1

BHT2

BHT3

BHT4

BHT5

BHT6

BHT7

BHT8

BHT9

BHT10

Total √ %√

1. Has fall risk assessment been completed within 48 hours of admission?

2. Is level of risk for fall documented in the record?

3. Is there an appropriate care plan for the patient based on the level of risk?

4. Is there evidence that the patient has been reassessed accordingly?

5. Is there evidence that there is action plan to reduce risk of falling?

6. If a fall occur, is there any evidence of the circumstances surrounding the patient?

7. For a patient who has fallen, is there any evidence that other strategy have been implemented?

8. Is there any evidence that doctors/ physiotherapist/occupational therapist have been involved?

Total √%√

PATIENT FALLS AUDIT FORM. 003/HSAH/UKL

Adapted from Barnet, 2001. 3 August 2009

Data protection

Data collection procedure must be ethical

When data is obtained directly form patients, auditors must explain

why the information is needed and what will happen to it, before

asking the patient‟s consent.

If the information is obtained from medical records:

i) patients consent to the retrieval of records and data being used

or

ii) data should be made anonymous before the audit start

or

Obtained ethical approval from national ethic committee.

Confidentiality of data

SELECTING CRITERIA & STANDARD

Stage 2

Stage 2: selecting criteria & standard

Clinical audit is a specific form of audit that involves measuring

clinical practice against standards

So, there must be criteria and standard.

Criteria: i) explicit statement that define what is being measured

ii) represent elements of care that can be measured

objectively

Criteria: an item or variable which enables the achievement of a

standard & the evaluation whether it have been achieved or not

Standard: level of care that need to be achieved for the particular

indicator.

Criteria

Classified into:

1. Structure: what you need

2. Process: what you do

3. Outcome: what you expect

Criteria

Structure: include

1. staff: no. of staff, skill mixed, ratio of staff to patients.

2. Equipment: technology, number of equipment

3. Space provided: number of OT functioning, room for

counseling

Criteria

Process refer to actions & decision by healthcare providers

1. Communication

2. Assessment

3. Education

4. Investigation

5. Prescribing

6. Therapeutic intervention

7. Evaluation

8. Documentation

The importance of process criteria is determined by the extent to which they influence outcome.

Department KPI Standard Achievement

Anesthesiology Percentage of

Elective And

Emergency Surgery

Patients Who

Receive Acute Pain

Service

>10% 5.1%

(204/4001 x 100%)

Process Criteria: pain assessment

Criteria

Outcome criteria are typically measures of physical or

behavioral response to an intervention, reported health

status, level of knowledge and satisfaction

More often proxy or intermediate outcome criteria are

used.

Department NIA Standard Achievement

Dietetic &

Catering

Incidence of Physical

Contamination of Food

Served to Patients

Sentinel events

2 cases

Indicator: Delay in

Response to In-patient

Referral by Dietitian

(for critical case)

Standard : ≤ 5

%

6.1%

(25/407 X 100%)

(Repeat SIQ. 11.8%:

Jul-Dec 2009)

Process Criteria: referral process

Outcome Criteria

How to develop valid criteria?

Valid criteria must meet the following:

1. Evidence based

2. Related to important aspects of care

3. Measurable

4. Specific

5. Achievable

6. Relevant

How to develop valid criteria?

Sources for developing criteria:

1. Existing guidelines: clinical practice guidelines (CPG),

national policy

2. Systematic literature review relevant to topic

3. Review of other practices elsewhere (when you

cannot find information in the literature)

MEASURING LEVEL OF

PERFORMANCE

Stage 3

Stage 3: measuring level of performance

Data storage

Data analysis

Comparing performance with standard

Any gap in performance (short fall in quality, SIQ)

52

Data analysis: compare with standard

38.0%

(164 Rx)

19.9%

(86 Rx)

42.1%

(182 Rx)

Moderate Emetogenic

(Level 3)

High Emetogenic

(Level 4)

Low Emetogenic

(Level 1&2)


1. Indication of

granisetron

injection

Only prescribed in moderately and highly

emetogenic chemotherapy unless it is justified.

100 %

38% non-compliant to

indication

53

99.3%

(429 Rx)

0.7%

(3 Rx)

Granisetron 3mg

Granisetron 1mg

Data analysis: compare with standard


2 Dose of

granisetron

injection

Only prescribed at the lowest effective dose,

which is 1mg or 0.01mg/kg unless it is justified.

100%

Only 0.7% compliant to

dose of 1 mg

MAKING IMPROVEMENT

Stage 4

Stage 4: making improvement

Analysis of problems: why is there a gap in

performance?

Identify areas to improve

Formulate strategies to overcome problems

Communicate action plan

Implement strategies

Evaluate strategies

SIQ

Is there gap in performance?

Is the gap in performance real?

Exclude error in data collection: inclusion criteria,

exclusion criteria, data analysis, data entry, data

calculation

Identify areas to improve

Conduct root cause analysis

Could be done earlier during data collection: has been

discussed in project methodology.

Literature review helps in identifying causes for

substandard performance.

Data collection tools designed to measure these causes:

knowledge, perception & practice

58

To assess prescribers’ perception of the granisetron injection utilization

in the prevention of CINV.

Lack of guideline enforcement

Lack of guideline availability

Lack of awareness

33.3% (21)

36.5% (23)

42.9% (27)

n=63 prescribers

There are no association between prescribers’ perception and all the factors tested including

departments and clinical experiences (p>0.05).

Prescribers’ perception:

Why are guidelines not widely-adapted in HSB

practice?

Formulate strategies to overcome

problems

Targeted

root cause

What to

change

How to

change

When to

change

Who is

responsible

Stage 1 of strategy for change

Targeted contributing

factors

AWARENESS

What to change? To improve the awareness among the

prescribers and pharmacists.

How to change? At INDIVIDUAL level:

۩ CME to update the prescribers on the

latest guidelines.

۩ CPE to educate the pharmacists on the

chemo prescriptions screening.

When to change? 2 months (Nov – Dec 2011).

Who is responsible? ۩ CDR Pharmacy unit.

۩ DIC Pharmacy unit.


Targeted contributing factors GUIDELINE AVAILABILITY

What to change? To ensure updated guidelines are

available in all the related wards and

daycare clinics.

How to change? At PRACTICE SITE level:

۩ To distribute the updated

antiemetic guidelines in poster

form to facilitate the rational

prescribing among the prescribers.

When to change? 2 months (Jan – Feb 2012).

Who is responsible? ۩ CDR Pharmacy unit.

۩ Hospital Health Education Unit.


Targeted contributing factors GUIDELINE ENFORCEMENT

What to change? To ensure the adherence to the

antiemetic guideline in the hospital.

How to change? At SERVICE level:

۩ Order form to require justification

from oncologists/ specialists in

non-indicated or high dose cases.

۩ CDR unit to prepare granisetron

injection in 1mg syringe form to

encourage the use of low dose.

When to change? 2 months (March – April 2012).

Who is responsible? CDR Pharmacy unit.


What to tell: why audit is done, problem statement,

criteria & standard, major & important findings,

strategies for change

How to communicate

Who to tell

When to tell

Where to tell


Many ways to do this:

1. meeting: intra-department, inter-department, hospital

level, state level

2. Training session: CME/CPD, seminar

3. Memo/letter

4. Bulletin, newsletter

5. Information board

6. E-mail

Present clinical audit and findings & what to do to improve

the performance

Implement strategies

Strategies for CHANGE is all about CHANGE

Observe the stages for change

Awareness Agreement Acceptance

ActionAccountabilityAssimilation

Stage 5: sustaining improvement

Do another cycle of data collection and analysis

Do trending of performance

4 kinds of motivationE

xtr

insic

So

meo

ne w

an

ts y

ou

to

do

it

Intrin

sic

Yo

u w

an

t to d

o it

Positive

Motivations towards a goal

Negative

Motivations away from something

“Do this clinical audit

& you’ll get a bonus”

“I really want to do this

clinical audit ”

“Do this clinical audit or

you’ll get low mark on

your annual appraisal”

“I really don’t want to

do this clinical audit ”

+

+ only this one create positive, sustainable motivation

Documents

How to Perform Clinical Audit