ACSA 2018 PDC - Getting to the Core of Your Safety ......Sumo Wrestling Entrenched in Japanese culture Tournaments involve 15 matches Winning record guarantees moving up Sumo Wrestling

3/23/2018

1

Getting to the Core of Your Safety

Program

Ryan Davis

Outline

Certification of Recognition (COR) in Canada

How incentives may affect stakeholdersHow COR maintenance may affect performanceHow to improve

3/23/2018

2

Learning Outcomes

At the end of this session, you will be able to:

1. Identify concerns in your own program(s).

2. Implement changes to programs to control incentives.

3. Examine and assess readiness to move to an HSMS.

COR In Canada

3/23/2018

3

COR in Canada

Established in 1989, through PIR

34 Certifying Partners in Canada29 Audit Tools Evaluated

COR Audit Tools

Practical Loss Control Leadership

Bird, Germaine, Clark

3/23/2018

4

0123456789

101112131415161718192021222324252627

AA

SP

AH

SA

AF

PA

…

AM

HS

A

AS

C

CC

SA

Enf

orm

WW

AA

AgS

afe

For

est…

go2h

r

Saf

ety…

MH

SA

AC

SA

AF

PA

SC

SA

BC

CS

A

Saf

ety…

CS

AM

MH

CA

NB

CS

A

NS

CS

A

HC

SA

S

NS

NY

MS

A

IHS

A

SA

SM

NL

CS

A

COR Audit Tools

COR Auditing - Methods

Health, Safety and Environmental Management Systems Auditing, 2nd Edition

Shematek, MacLean, Lineen

3/23/2018

5

56% 34% 10%

COR Auditing - Verification

COR Auditing - Scoring

All or Nothing (A/N)

(0, 10) or (10)

49% = 0 points

65% = 10 points

Range

(0-10)

49% = 5 points

65% = 7 points

Both based on evidence or ‘positive indicators’

3/23/2018

6


50%

4%

46%

A/NRangeBoth


>50% per element* +

> 80% overall* =

Certification!

3/23/2018

7

Studies on COR

Studies on COR

3/23/2018

8

Studies on COR

Studies on COR

3/23/2018

9

Studies on COR

Studies on COR

3/23/2018

10

Studies on COR

Incentives

3/23/2018

11

Sumo Wrestling

Entrenched in Japanese culture

Tournaments involve 15 matches

Winning record guarantees moving up

Sumo Wrestling

3/23/2018

12

7:7 8:6

80% 20%

Sumo Wrestling

Sumo Wrestling

3/23/2018

13

Sumo Wrestling - Rematch

7:7 8:6

40% 60%

Teachers in Chicago Public School

Policy change in 1996

Based on standardized tests

Had access to tests before submission

3/23/2018

14


Findings suggested that ~5% of teachers cheated


3/23/2018

15

Claims Suppression in Manitoba

Principal Findings:

> 5 day LTI: ~30% not reported

Misreporting LTI: ~14 – 35%

Awareness of claims suppression: ~36%


3/23/2018

16

Principal Findings:

“Participants saw what they considered the inherent incentivesin the experience rating system as the foundation for aggressive claims management, incentive systems tied to no time loss injuries and, the use of third party claims administrators.”

- Prism Economics and Analysis, 2013


Incentives in COR

3/23/2018

17

COR In Canada

Often a pre-bid requirement

COR is a COR

50% of COR tools exclusively A/N scoring

Incentives - Employers

External

COR required to bidWCB rebates>80% = Standard met

Internal

ConsistencyIncreased productionReduced Risk

3/23/2018

18

Incentives – Safety Professionals

Consultants

Get paidRepeat businessOut-of-scope work

Employees

BonusesJob securitySocial pressure

How COR may affect programs

3/23/2018

19

1. Measurement (Tool Itself)

2. Scoring Variance (A/N vs. Range)

3. Maintenance (Audit Cycle)

How COR may affect programs

Policy & Interview Exercise

3/23/2018

20

According to Shematek, et. al.:

“Probably the single most revealing component of the HSE audit information collection process is the audit in interview.”

- Shematek, MacLean, Lineen, 2016

Interviews

3/23/2018

21

Presentation Policy

Presentation Policy

3/23/2018

22

Presentation Policy

Each technique may be used to score a single concept:

Does it exist?

Do employees know about it?

Is it available?


3/23/2018

23

52% 17% 31%


Scoring Scenarios

3/23/2018

24

For records, points historically awarded on>50% positive indicators

Example - Monthly meetings:6/12 = full points5/12 = no points

Scoring Scenario

Company AMet Audit Standard

Records = 50% positiveGiven score: 80% (262/328)Actual score: 63% (207/328)

Company BFailed to meet standard

Records = 49% positiveGiven score: 66% (218/328)Actual score: 83% (272/328)

328 Pts Available, 110 points based on ‘records’

Scoring Scenario

3/23/2018

25

(Re) Certification• 80% Required*

Maintenance Year 1• 60% Required

Maintenance Year 2• 60% Required

COR Audit Cycle

80%60%60%80%60%60%80%

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7

COR Audit Cycle

3/23/2018

26

How COR may affect programs:

1. Measurement

2. Scoring Variance

3. Maintenance

How to improve

3/23/2018

27

“The better you get,the better you’d better get.”

- Getting Things Done, David Allen

External Stakeholders

WCB

Proactive element incentivesHigher rebatesRequire continual improvement

Certifying Partners

Alternative toolsAuditor workshopsRequire continual improvement

3/23/2018

28

Safety Professionals

Externally

Auditor selectionWork with CPsTie safety performance to business outcomes

Internally

Automate / OrganizeMeasure moreConsider readiness to move to an HSMS

3/23/2018

29

COR and CSA Z1000 Comparison

ACSA COR AuditSection 8

8.1 Formal Orientation

8.2 Confirmation of Completion

8.3 Certification Verification

8.4 Refresher Training

8.5 Job Specific Training

8.6 Supervisory Training

CSA Z1000-14Section 4.4.4.2a) be provided to all, based on their dutiesb) be conducted by competent persons;c) be based on principles of adult

learning;d) address differing levels of

responsibility, ability, language, and literacy;

e) consider participants’ culture and diversity;

f) include timely refresher sessions;g) include participants’ evaluationh) be evaluated and modifiedi) be documented.

3/23/2018

30

Conclusion

3/23/2018

31

Focus future research in 3 areas:

1. ongoing monitoring of program effectiveness;

2. investigate how certification facilitates change in OHS practices; and

3. assessment of audit tools to improve and incorporate evolving OHS best practices.

- McLeod, et. al., 2015

Thank you!

Ryan Davis

[email protected]

3/23/2018

32

ACSA Stakeholder Audit Tool Engagement Meeting

63

Survey - How could we improve Audit Tool?

Simplified questions Detailed questions Explanation of scoring Add range scoring Remove A/N scoring Combine elements Simplify guidelines Provideexamples Other (please specify)0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

3/23/2018

33

Summary of Changes

Element Weighting

Scoring

Notes Requirements

COR Audit Cycles

65

ACSA Sections1 Health & Safety Policy2 Hazard Assessment3 SWP4 SJP5 Rules6 PPE7 Preventative Maintenance8 Training9 Inspections10 Investigations11 ERP12 Stats13 Legislation

Partnerships Sections

66

1 Management Commitment

2 Hazard Identification

3 Hazard Control

4 Inspections

5 Training

6 Emergency Response

7 Investigations

8 Program Administration

3/23/2018

34

Element Weighting

67

# Element Min Max

1 Management Commitment 15 20

2 Hazard Identification 15 20

3 Hazard Control 15 20

4 Inspections 5 15

5 Training 10 15

6 Emergency Response 5 15

7 Investigations 5 15

8 Program Administration 5 15

Proposed Weightings – CP Tool

68

# Element Proactive Balanced Operational Reactive

1 Management Commitment 20 15 20 15

2 Hazard Identification 20 15 15 15

3 Hazard Control 20 15 15 15

4 Inspections 10 15 15 5

5 Training 10 10 15 5

6 Emergency Response 5 10 5 15

7 Investigations 5 10 5 15

8 Program Administration 10 10 10 15

100 100 100 100

3/23/2018

35

Proposed Weightings – ACSA Tool

69

# ACSA Weight Partners Ref. Weighting1 Health & Safety Policy 10 1 10%5 Rules 5 1 5%

13 Legislation 5 1 5%2 Hazard Assessment 15 2 15%3 SWP 4 3 4.0%4 SJP 4 3 4.0%6 PPE 3 3 3%7 Preventative Maintenance 4 3 4%9 Inspections 15 4 15%8 Training 15 5 15%11 ERP 5 6 5%10 Investigations 5 7 5%12 Stats 10 8 10%

100 100%


52% 17% 31%

3/23/2018

36

Scoring

All or Nothing:• 70% Positive Indicators required• 50% for first year only

Range:• Adding Range scoring for quantifiable verification

o (e.g. records, interviews)

• Rounding rules must be included by CP

71

Notes Requirements

Complete justification including ‘positive indicators’ for all questions.

Examples may include:

“20/25 (80%) of the hazard assessments reviewed identified health hazards in the assessments. Examples included exposure to fumes while welding…”

“Meeting minutes reviewed confirmed that 9/12 (75%) of the required monthly safety meetings took place per the ABC Construction Inc. policy.”

72

Documents

ACSA 2018 PDC - Getting to the Core of Your Safety ......Sumo Wrestling Entrenched in Japanese culture Tournaments involve 15 matches Winning record guarantees moving up Sumo Wrestling