Improving Safety Performance Through Measurement

Improving Safety Performance Through Measurement

Improving Safety Performance Through Measurement

Mike ThomasManaging Director Prof. Andrew Hale

DirectorIOSH London Health& Safety Group

17 March 2008

OverviewOverview

Key questions

What is the role of performance measurement in a Safety Management System (SMS)? (“What gets measured gets done”, Drucker and HSE in HSG65)

How can measurement be used to improve performance and drive continual improvement?

How can we select appropriate key performance indicators (KPIs)?

How can we develop meaningful targets?

How can we promote performance measurement by line managers and link this to audits?

A Success Story: Corus NLA Success Story: Corus NL

Year -3 -2 -1 0 +1 +2

LTI Frequency/1000 man

17.3 16.1 14.3 7.9 0.9 1.9

Reported dangerous situations

186 237 642 1401 1643 1252

Maintenance department of an integrated steel works, employing about 1100 used an intervention driven by KPIs to improve performance.

Improvement driven by KPI’s for managers: • Compulsory + free choice, ‘proactive’ + ‘reactive’.• Targets/KPI + score (1-10 with 6 for hitting target) • KPIs given different weights & totalled for overall score/period• Reported on a ‘dashboard’ every 6 weeks

STOP-GO cards for workforce – dynamic task risk assessment

Key Performance IndicatorsKey Performance Indicators

Output: LTI frequency + Recordable frequency

Compulsory:

Reporting of dangerous situations (0.5pppy)

Dealing with dangerous situation reports (80%)

Toolbox meetings (including safety) (80%)

Choice (at least 1 of 4):

Behaviour observation rounds (80%)

Safety communication rounds (80%)

‘5 S’ housekeeping inspections (80%)

Risk assessment + plan of action (80%)

OHSAS 18001OHSAS 18001

Continual improvement

Checking and corrective action

OH&S Policy

Planning

Implementationand operation

Management Review

MeasurementMeasurement

Within 18001

“Checking and corrective action”

Includes

Measurement of proactive data

Measurement of reactive data

Internal audit (independent)

Reactive and proactive are key issues

Risk assessment & controlRisk assessment & control

Review & improvement

Risk management system Audit

EvaluationDirect risk controls:

Hard-, soft- & liveware

Primary processes

Inspection

Incident analysis

Risk assessment

Known risk

Comparison

Unknown riskReview RIE system

Types of indicatorTypes of indicator

Damage, injury, loss

Precursors – for each scenario, different precursors (incidents, leaks, breakdowns)

Re-design

Choice and design of (sub)system

Normal situation

with in-built hazards

Recovery

Deviations from normal situation

Loss of control(release of energy

+ exposure

Transmission

Damage process

Stabilisation

Elimination of hazard

Hazard control measures

Detection & recovery

Escape

Secondary safety

Rescue, damage limitation, treatment

Reporting

Learning

Choice & design of prevention &

control measures

Accident Deviation Model

For each hazard, different deviation scenarios, with different control measures, failing in different ways

LTI’s don’t predict disastersLTI’s don’t predict disasters

The lesson of Texas City:

BP became distracted from efforts to control process safety because they were lulled into a false sense of security by their very excellent control of lost time injuries.

The fallacy of Bird’s interpretation of Heinrich’s triangle

Heinrich’s thesisHeinrich’s thesis

Reasoning from top down

Each scenario or accident type apart

Average ratio over scenarios

1

29

300

Disabling Injury

Minor Injury

No injury

From Bird onwardsFrom Bird onwards

Overall statistics of a department, factory, or country

Different categories of severity

1

100

500

Property damage

(Bird 1966)

Disabling Injury

Minor Injury




Presence & readiness of risk control measures (hardware, software, people’s behaviour)

Management system processes to deliver the preventive measures

Resources and controls in the SMSResources and controls in the SMS

- Competence of staff- Availability of manpower- Commitment to safe operations & conflict resolution- Communication within and between teams- Procedures, goals & rules- Hardware & software

- Selection & training- Manpower planning- Incentives, supervision, appraisal, culture, management priority- Handover, briefing, communication channels- Task/policy analysis & design- Design, layout, maintenance

These processes are the ones to be audited

Problems with AuditingProblems with Auditing

Seen as a panacea

“Bolted on” – imposed from outside

May not match policy & own management system: independent of management

Passive process for manager

“Economical with the truth”

Wait for next audit

No ownership




Presence & condition of risk control measures (hardware, software, people’s behaviour)

Management system processes to deliver the preventive measures

Attitudes and values (culture) to use the SMS processes and the risk control measures

Safety climate/culture measurementSafety climate/culture measurement

Existing climate questionnaires: limited validation (TRIPOD is one exception) Interpretation is still more an art than a science – lack

of clarity over underlying models of culture Gap between measuring (profile) and improving

Safety culture maturity scales (Hearts & Minds): Similar criticisms Individual completion followed by group discussion

generates much useful debate

Improving PerformanceImproving Performance

Once you are clear on meaning & objectives

Look at measurement and improvement

Three key issues

How to define performance in measurable terms?

How can performance be measured?

How to extend to target setting and to drive improved performance?

How is Improvement Achieved?How is Improvement Achieved?

“What gets measured gets done”

Apply sound management techniques to safety

Performance standards - what people must do Who is responsible? – whose KPI? What are they responsible for? – tasks, processes When should the work be done? - plans What is the expected result? – (intermediate) outputs

Set targets and measure performance in terms meaningful to each individual

Criteria for performance measuresCriteria for performance measures

Validity

Reliability

Representativeness

Sensitivity

Openness to bias

Cost-effectiveness

NB. Exposure measures

Typical targets in the pastTypical targets in the past

“eliminate lost time (or all) accidents”

“reduce all accidents to n or by x% per annum”

“reduce number of days lost to n or by x%”

“reduce cost of claims or other losses by y%”

“eliminate notices as a result of enforcement action”

Obvious approach = what we want! Commendable, but typically, reactive

Accidents as performance measureAccidents as performance measure

Validity

Reliability

Representativeness

Sensitivity

Reporting bias

Cost-effectiveness

PM Exposure measure

Very high

High, but absence serious

Good if enough

Low for good organisation

Strong for minor

High cost of analysis

/man or /manhour

Problems with Reactive TargetsProblems with Reactive Targets

Problems - well recognised?

Not helpful to managers!

They accept the value and sense but … What must they do?

“Promotes” under-reporting and “manipulation” of outcome

What if zero accidents already?

Past performance may be a poor predictor of the future

Accident RatesAccident Rates

"A low accident or incident rate over a period of years is no guarantee that risks are being effectively controlled.This is particularly so in organisations involved in major hazard activities, where the probability of an accident may be low but where the consequences could be extremely serious.In this type of organisation, the historical incidence of reported accidents alone can be an unreliable indicator of safety or environmental performance and can lead to complacency."

Guidance on COMAH Regulations, UK

Reactive TargetsReactive Targets

Why still set? Easy option that sounds good! Mirrors UK Government’s targets Accident reduction IS the desired output of the SMS

Are Reactive “Targets” Still Valid? At a high level within the organisation as “aim” or

“vision” But NOT as targets for managers Need to move to better data: step 1

Minor injuries and health effects Near misses (surprises) or dangerous situations

Way Forward?Way Forward?

Aim

Reduce accidents but numbers of accidents are NOT used as exclusive targets for managers

Move to proactive measures

Set targets on proactive measures of performance

Compare with quality management

Contrast: Safety and QualityContrast: Safety and Quality

ISO approach to quality as:

An essential feature

Not an optional extra

Emphasise:

managing quality in, not inspecting defects out get the management processes right, but still measure defects

Apply this philosophy to safety

What is Proactive Measurement?What is Proactive Measurement?

Traditional measures (input measures): e.g. PPE being used Guards in place Documents reviewed and updated All employees trained Toolbox meetings held Maintenance conducted to plan

More dynamic measures (input + intermediate) Hazards identified and put right Competence tested & used Behaviour observed, discussed & improved

Planning and Target SettingPlanning and Target Setting

If you accept the philosophy, how can measures be translated into targets?

BS8800 approach

Annex C (BS 8800:1996)

Simplified version (BS 8800:2004, Annex D)

Planning and Implementing

Planning for SafetyPlanning for Safety

Measure Baseline Data

Measure Baseline Data

Devise Plan(Management Programme)

Devise Plan(Management Programme)

Topics withQuestions as Targets

Topics withQuestions as Targets

IMPLEMENT PLANIMPLEMENT PLAN

AimAim

ObjectiveObjective

Define Outcome Indicator

Define Outcome Indicator

Measuring SuccessMeasuring Success

IMPLEMENT PLANIMPLEMENT PLAN

Measure Outcome Data

Measure Outcome Data

Achieving Objective ?Achieving Objective ? NoNo

YesYes

Take Corrective Action as Necessary

Take Corrective Action as Necessary

Continue ProcessContinue Process

Measure Compliance with Programme (Targets)

Measure Compliance with Programme (Targets)

Meeting Targets ?Meeting Targets ?

YesYes

Combining DataCombining Data

Compliance Monitoring Data (Management Programme)

Outcome Data (Objective)

Improving Not Improving

Improving Not Improving

Why questions posed in plan?Why questions posed in plan?

Forces clear logical thought

Define the programme

Define the targets

Questions are a blue print for the managers

Questions can be used to measure success

“Correct” answer is “yes”

Link to audit (e.g. CHASE)

Proactive Monitoring and AuditProactive Monitoring and Audit

HASTAM’s CHASE system

Questions developed by planning process

Used by managers to measure performance

Used by specialists for audit

Audit by verification of

Managers’ answers

Recommended remedial action

Reinforces ownership by managers

ConclusionConclusion

Explained

Need for and effective means of setting realistic targets

Targets NOT based on “reactive” measures

Based on proactive measures of system

However, no conflict with reactive aims

Targets provide

Clear implementation mechanism for managers

Effective measurement of improved performance

ContactContact

Andrew HaleHastam LtdThe Old BakehouseMaldonEssex CM9 4LE

[email protected]