How to Apply Risk-based Thinking to Quality Processes

7/25/2019 How to Apply Risk-based Thinking to Quality Processes

1/58

Page 1

ISO 9001:2015 - How to

apply Risk-based Thinking

to Quality Processes

Title VI-404842-TM ISO 9001:2015 - How to apply Risk-based Thinking toQuality Processes

Version 1

Author Michael Shuff

Issue Date 05 Aug 2015


2/58

Page 2

Summary

The new version of the ISO 9001:2015 standardis scheduled for final publication on September

23rd 2015. One of the new requirements is to show evidence of risk-based thinking (RBT) in the

quality management system. How do you do that? How are auditors likely to respond to the new

challenges that ISO 9001:2015 brings? How do you produce documented evidence of risk-based

thinking?

Although ISO 9001:2015 does not call for formal methods of risk management, it is likely that

anyone trying to understand RBT may turn to ISO 31000 and the list of risk assessment techniques

in particular. However, this is not as easy as it sounds. There are many techniques to choose from

and many may not be applicable to the sectors that ISO 9001 serves.

This white paper has two major sections. The first part provides a primer on many of the ISO

31000 risk assessment techniques and considers their applicability to quality management. The

second part provides a six-step methodology that you can follow to deliver evidence of a risk

based approach to quality. It is a practical methodology that is specific on inputs / outputs, and

what you need to do in-between. Several example templates are provided that could form the

basis for your documented information.


3/58

Page 3

1 Risk-based thinking as a requirement of ISO 9001 .................... ..................... ...................... ..................... ...................... . 6

1.1 A starting point for risk-based thinking applied to quality processes ...................... ..................... ..................... ..... 7

2 ISO 31000 Risk Management Techniques .................... ..................... ...................... ..................... ..................... .............. 11

2.1 Look-up Methods ...................... ..................... ..................... ...................... ..................... ...................... ................. 11

2.1.1 Checklists ..................... ..................... ...................... ..................... ...................... ..................... ..................... ... 11

2.1.2 Preliminary hazard analysis.......................... ..................... ...................... ..................... ..................... .............. 12

2.2 Supporting Methods .................... ...................... ...................... ..................... ..................... ..................... .............. 12

2.2.1 Structured interview and brainstorming............................ ..................... ..................... ..................... .............. 12

2.2.2 What can we learn from ISO 31000 risk assessment processes? ................... ..................... ...................... ...... 14

2.2.3 Are structured interviews and brainstorming 9001 requirements? ..................... ..................... ..................... 14

2.3 Other Supporting Methods ..................... ..................... ...................... ...................... ..................... ..................... ... 16

2.3.1 Delphi technique .................... ...................... ...................... ..................... ..................... ..................... .............. 16

2.3.2 SWIFT (Structured what-if ) ......................................................................................................................... 17

2.3.3 Human reliability analysis (HRA) ..................... ...................... ..................... ...................... ..................... .......... 18

2.4 Scenario Analysis ................... ...................... ..................... ..................... ..................... ...................... ..................... 20

2.4.1 Root cause analysis (RCA) ..................... ..................... ..................... ...................... ..................... ..................... 20

2.4.2 Scenario analysis .................... ...................... ...................... ..................... ..................... ..................... .............. 20

2.4.3 Toxicological / Environmental / Ecological risk assessment........................... ..................... ...................... ...... 21

2.4.4 Business impact analysis (BIA) ..................... ..................... ...................... ..................... ..................... .............. 21

2.4.5 Fault tree analysis ...................... ..................... ...................... ..................... ..................... ...................... .......... 22

2.4.6 Event tree analysis ..................... ..................... ...................... ..................... ..................... ...................... .......... 22

2.4.7 Cause and consequence analysis .................... ...................... ..................... ...................... ..................... .......... 23

2.4.8 Cause-and effect analysis ...................... ..................... ..................... ...................... ..................... ..................... 23

2.5 Function Analysis ...................... ..................... ..................... ...................... ..................... ...................... ................. 24

2.5.1 FMEA and FMECA ...................... ..................... ...................... ..................... ..................... ...................... .......... 24

2.5.2 Reliability-centred maintenance (RCM) ..................... ..................... ...................... ..................... ..................... 25


4/58

Page 4

2.5.3 Sneak analysis (SA) and sneak circuit analysis (SCI) ..................... ...................... ..................... ..................... ... 25

2.5.4 HACCP ................... ..................... ..................... ...................... ..................... ...................... ..................... .......... 26

2.6 Controls Assessment .................... ...................... ...................... ..................... ..................... ..................... .............. 26

2.6.1 LOPA (Layers of Protection Analysis) ...................... ..................... ..................... ...................... ..................... ... 26

2.6.2 Bow-tie analysis ..................... ...................... ..................... ...................... ..................... ..................... .............. 27

2.7 Statistical Methods ...................... ...................... ..................... ...................... ..................... ..................... .............. 27

2.7.1 Markov analysis ..................... ...................... ..................... ...................... ..................... ..................... .............. 28

2.7.2 Monte-Carlo analysis .................... ...................... ...................... ..................... ..................... ...................... ...... 29

2.7.3 Bayesian analysis ................... ...................... ...................... ..................... ..................... ..................... .............. 30

3 A Risk Management Methodology for Quality Management ...................... ...................... ..................... ..................... ... 32

3.1 Risk based thinking is the new 'preventive actions' for QMS .................... ..................... ...................... ................. 32

3.1.1 Planning and considering risks in quality system processes ................... ..................... ..................... .............. 33

3.1.2 What actions are required to plan for risks and opportunities? .................... ..................... ...................... ...... 34

3.2 The Six Steps ...................... ..................... ..................... ...................... ...................... ..................... ..................... ... 35

3.3 Step 1: Establish the Context ...................... ..................... ..................... ...................... ..................... ..................... 37

3.3.1 Scope and responsibilities for specific risk management activities ...................... ..................... ..................... 38

3.3.2 How should we document the "context of the organization"?...................... ..................... ...................... ...... 39

3.3.3 What information should the Statement of Context contain? ...................... ..................... ...................... ...... 40

3.3.4 Risk criteria for Quality Management Systems ..................... ..................... ...................... ..................... .......... 40

3.4 Step 2: Risk identification .................... ...................... ...................... ..................... ..................... ...................... ...... 41

3.4.1 Techniques for risk identification .................... ...................... ..................... ...................... ..................... .......... 42

3.5 Step 3: Qualitative risk analysis & risk evaluation ................... ...................... ..................... ..................... .............. 43

3.5.1 What is a `Qualitative analysis' of risk? ................... ..................... ...................... ..................... ..................... ... 43

3.5.2 Does ISO 9001:2015 require a qualitative risk assessment? ................... ..................... ..................... .............. 43

3.5.3 Sources of information for qualitative analysis........................ ...................... ..................... ...................... ...... 44

3.5.4 Summary: ..................... ..................... ...................... ..................... ...................... ..................... ..................... ... 46

3.6 Step 4: Semi-Quantitative risk analysis and risk evaluation ...................... ..................... ...................... ................. 46


5/58

Page 5

3.6.1 Methods for calculating risk factors .................... ..................... ...................... ..................... ...................... ...... 47

3.6.2 What is the value of the Semi-Quantitative approach in Step 4, following the Qualitative Assessment

conducted in Step 3? ...................................................................................................................................................... 48

3.7 Step 5: Risk treatment ...................... ..................... ...................... ..................... ..................... ...................... .......... 49

3.7.1 Example of Risk Treatment in a Quality Management System ...................... ..................... ...................... ...... 50

3.8 Step 6: Monitoring & review ................... ..................... ...................... ...................... ..................... ..................... ... 52

4 Summary and Conclusions ..................... ...................... ..................... ...................... ..................... ..................... .............. 54

4.1 Risk Assessment Methodology for applying RBT to QMS ..................... ...................... ..................... ..................... 55

4.2 Conclusion ................... ..................... ..................... ...................... ..................... ...................... ..................... .......... 57


6/58

Page 6

1 Risk-based thinking as a requirement of ISO 9001

Risk-based thinking is a sore point among many Quality professionals. Even so, identifying risk,

analysing the consequences, probability and level of risk (i.e. risk analysis) and risk evaluation using

formal techniques are becoming increasingly important tasks in the global business world.

ISO 9001:2015 incorporates what the

draft version of the International

Standard has termed "Risk-based

Thinking" in its requirements for the

establishment, implementation,

maintenance and continual

improvement of the quality

management system. If you are

already familiar with the DIS or read

the many discussions on the subject

that have appeared on LinkedIn

groups and elsewhere, you will

already be aware that formal risk

management is not mandated.

However, organizations can, in the

words of the TC 176 Committee's

draft standard (May 2014) "...choose to develop a more extensive risk-based approach than is

required by this International Standard, and ISO 31000 provides guidelines on formal risk

management which can be appropriate in certain organizational contexts".

We are sceptical about the subject of demonstrating risk-based thinking to a certification auditor

when they assess your quality management system. Of course, it is possible that you will not be

subject to an intensive grilling if the Standard does not require you to produce the outputs from

your risk assessment processes or evidence of a formal risk management system. Although if risk-

based thinking is required by ISO 9001:2015 to plan and control the quality management system

(QMS) and component processes and activities, it is unlikely to be ignored in the certification audit

process.

This begs the question:

How do you show risk-based thinking during a certification audit?

Risk-based thinking" assessment is likely to form a sizeable section of the ISO 9000 Guidance

documents when they are published along with the ISO 9001:2015 Standard. Waiting until

September may not be an option for those of you looking to transition from the 2008 Standard as

rapidly as possible, so we thought that it would be a good idea to look at how you might go about

this interesting task. The aim is to produce (a) evidence that you could show to an assessor [HEALTH

WARNING: nobody yet knows exactly what they will be asking for], and (b) a useful way of

identifying, evaluating and treating the kind of risks that apply to the processes used in Quality

Management.


7/58

Page 7

1.1

A starting point for risk-based thinking applied to quality processes

In our blog postISO 9001:2015The likely impact (Part II),we suggested the following basic

checklist of tasks:

Analyse and prioritizethe risks and opportunities in your organisation:

What is acceptable?

What is unacceptable?

Then plan actionsto address the risks. Ask yourself:

How can I avoid or eliminate the risk?

How can I mitigate the risk?

Then...

Implement the plantake action

Check the effectiveness of the actionsdoes it work?

Learn from experiencecontinual improvement

However, this list presupposes that you have identified risks and opportunities.

So if you have not done so yet, how do you approach risk identification in your context?

Read on...

Will ISO 31000:2009 help in taking a 'risk-based approach' to the quality management system,

component processes and activities?

Short answer: it can do, depending on your organization's context.

The ISO 9001 DIS says that ISO 31000 provides guidelines on formal risk management, which can be

appropriate in certain organizational contexts.

Those working for large, indeed global entities understand this. They have long since adopted riskmanagement methodologies and have risk managers on their team who are familiar with ISO 31000.

But what is ISO 31000 attempting to achieve, and is it relevant to the majority of organizations that

are trying to gain or transition to ISO 9001?

ISO 31000 describes an "overall approach to risk management, not just risk analysis or risk

assessment. It deals with the links between risk management process and both strategic direction

and day to day actions and treatments."1This on the face of it sounds an ideal recipe for risk-based

thinking. However, pick up the Standard and read it and this thought is quickly dispelled because ISO

1Project risk management guidelines: managing risk with ISO 31000 and IEC 62198, Dale F Cooper, et al, Wiley, 2014
https://www.cognidox.com/2015/02/iso-90012015-the-likely-impact-part-ii/https://www.cognidox.com/2015/02/iso-90012015-the-likely-impact-part-ii/https://www.cognidox.com/2015/02/iso-90012015-the-likely-impact-part-ii/https://www.cognidox.com/2015/02/iso-90012015-the-likely-impact-part-ii/https://www.cognidox.com/2015/02/iso-90012015-the-likely-impact-part-ii/https://www.cognidox.com/2015/02/iso-90012015-the-likely-impact-part-ii/


8/58

Page 8

31000 takes a generic approach that has to be developed - in considerable detail - to be useful in a

given context.

Great for the Strategic aims of the senior management, but not of any great value to the 'poorbloody infantry' of quality managers out there.

Perhaps the first (and frustrating) conclusion you will come to, having spent at least 120 ($180) on

your personal copy is that you also need to buy ISO.IEC 31010:2009Risk managementRisk

assessment techniques.

Therefore, your boss says, "OK, buy the one you actually need, but don't come back to me asking for

more. We've got by without 'risk-based thinking' in the past [insert number of years or decades];

surely we can do so this time?" You thank her or him for authorizing the purchase.

The PDF arrives on your computer. You open it. There are 92 pages, 6 of which in Annex A are acomparison of risk assessment techniques (some useful tables here) before you arrive at Annex B,

consisting of 61 pages describing the 31 risk assessment techniques. These seem suited for the kind

of people who enjoyed Mathematics (and Statistics especially) at school, but who may not be that

interested in helping you to design effective quality processes.

Yes, there is a worthy (absorbing even?) preamble about risk assessment concepts and processes.

There also a Clause describing how to select techniques for risk assessment, this starts with the valid

advice:

Risk assessment may be undertaken in varying degrees of depth and detail and using one or

many methods ranging from simple to complex. The form of assessment and its outputshould be consistent with the risk criteria developed as part of establishing the context.

[Clause 6.2]

There is no point in making life more complicated than it needs to be; thus:

In general, suitable techniques should exhibit the following characteristics:

it should be justifiable and appropriate to the situation or organization under consideration;

it should provide results in a form which enhances understanding of the nature of the risk

and how it can be treated;

it should be capable of use in a manner that is traceable, repeatable and verifiable. [Ibid]

Great!

By now, you are probably fired up with the possibility of finding a suitable risk assessment technique

that fits the context of your organization and its quality management system. You cannot wait to get

started on the job.

You turn to...

Annex A

(informative)Comparison of risk assessment techniques


9/58

Page 9

You quickly realize there are more risk assessment techniques than you thought existed, and even a

cursory reading suggests that some are complex. Notably the ones that are strongly applicable to

each step of the full risk assessment process; specifically:

risk identification;

risk analysisconsequence analysis;

risk analysisqualitative, semi-quantitative or quantitative probability estimation;

risk analysisassessing the effectiveness of any existing controls;

risk analysisestimation the level of risk;

risk evaluation.

Below is the list of the 31 tools. Depending on the industry you are working in, you will almost

certainly recognise at least some of them, even if you have not actually used any of the techniques

to assess risk.

Tools used for risk assessment

1. Brainstorming

2. Structured or semi-structured interviews

3. Delphi

4. Check-lists

5. Primary hazard analysis

6. Hazard and operability studies (HAZOP)

7. Hazard Analysis and Critical Control Points (HACCP)

8. Environmental risk assessment

9. Structure What if? (SWIFT)

10.Scenario analysis

11.Business impact analysis

12.Root cause analysis

13.Failure mode effect analysis

14.Fault tree analysis

15.Event tree analysis

16.

Cause and consequence analysis


10/58

Page 10

17.Cause-and-effect analysis

18.Layer protection analysis (LOPA)

19.Decision tree

20.Human reliability analysis

21.Bow tie analysis

22.Reliability centred maintenance

23.Sneak circuit analysis

24.

Markov analysis

25.Monte Carlo simulation

26.Bayesian statistics and Bayes Nets

27.FN curves

28.Risk indices

29.

Consequence/probability matrix

30.Cost/benefit analysis

31.Multi-criteria decision analysis (MCDA)

Table 1: Tools used for risk assessment

Not everybody will have the resources and capabilities within the organization to attempt some of

these - e.g., Fault tree analysis, Cause / consequence analysis, Monte-Carlo analysis, Bayesian

analysis.

Quality managers working for smaller enterprises (SMEs) may only dream of conducting analysis at

the level required by some techniques in the list. The sheer complexity of some types of risk

assessment will render the tool useless in most organizations employing between 1 and 250 people.

However, that does not mean to say that ISO 31010 isn't a valuable reference should you ever be

required to think about risk in these terms.

In the following sections, we will focus on some of these techniques.


11/58

Page 11

2 ISO 31000 Risk Management Techniques

Although risks and opportunities have to be determined and addressed, there is no requirement in

ISO 9001:2015 for a formal risk management or a documented risk management process. Even so,

the concept of preventive action is expressed in the 2015 wording through the risk-based approach

to formulating quality management system requirements. It follows that we will most probably want

to show our reasoning in this respect. In other words, how our thinking about risk led to these

actions?

In our view, this does not have to be an

onerous task even at the high-risk end of

the context spectrum. However, to

completely ignore the risks and

opportunities aspect of planning your

QMS [see 6.1], regardless of the degree of

risk involved, would surely be to risk a

major non-conformity?

ISO 9001 Risk-based thinking could(and

we are not saying that it should) be

demonstrated by showing the outputs

from one or more of the risk assessment

tools in ISO 31010 in your "documented

information".

To give you a flavour of what these tools

are intended to achieve and how they

work, we intend to describe a selection of

the 31 listed in ISO 31010. At the same

time and over the next two posts, we will attempt to link these tools to QMS processes in a

meaningful way; however, we do not anticipate our work in this respect to be in any way definitive

as a reliable reference. There is no common consensus on how best to employ risk assessment

techniques in quality management - at least none that we are aware of yet!

[That said, we are studying with interest the ICH guideline Q9 on quality risk management, which

provides principles and examples of tools for quality risk management applied to different aspects of

pharmaceutical quality. If you have experience of this guideline, I'd welcome your input!]

Note: the text is based on the contents of Table A.2Attributes of a selection of risk assessment

tools[Source: IEC/FDIS 31010:2009].

2.1

Look-up Methods

2.1.1

Checklists

This is a simple form of risk identification and a technique that provides a list of uncertainties that

need to be considered. Users can refer to a previously developed checklist, code or standard.


12/58

Page 12

Checklists and reviews of historical data are,

naturally enough, a sensible step if you are serious

about identifying the risks and opportunities in

accordance with the requirements of ISO 9001:2015Clause 6.1, and intend to plan and implement the

appropriate actions to address them. Although you

could enhance the quality of the output by

following a systematic process to identify risks by

means of a structured set of prompts or questions

for the experts - see structured interview below.

Personally, we would start by making a checklist of

the known issues in the environment that can (a) affect conformity of products and services [risk]

and (b) have the ability to enhance customer satisfaction [opportunity].

No ISO 9001 assessor is likely to fault you for making this much effort; whether or not you have

addressed these risks and opportunities in the design of your quality management system and its

associated processes.

However, it is also worth remembering that checklists are most useful when applied to check that

everything has been covered after a more imaginative technique that identifies new problems has

been applied.

2.1.2

Preliminary hazard analysis

This is a simple inductive method of analysis whose objective is to identify the hazards andhazardous situations and events that can cause harm for a given activity, facility or system.

Note: the term 'hazard' is always used in the context of physical harm.

At first sight, not a very promising tool but it does have advantages; namely: it is able to be used

when there is limited information; and it also allows risks to be considered very early in the system

lifecycle. In some organizational contexts, preliminary hazard analysis could be appropriate as a risk

assessment tool for quality when its use helps prevent Critical Non-conformities; which could, for

example, result in hazardous or unsafe conditions for individuals using, maintaining or depending on

the product.

2.2 Supporting Methods

2.2.1

Structured interview and brainstorming

This is a means of collecting a broad set of ideas and evaluation, ranking them by a team.

Brainstorming may be stimulated by prompts or by one-on-one and one-on-many interview

techniques.

So what should we plan to collect in terms of "ideas and evaluation"?

Let us remind ourselves first of what ISO 9001:2015 says we should do.


13/58


14/58

Page 14

2.2.2

What can we learn from ISO 31000 risk assessment processes?

ISO 31000 states that risk assessment attempts to answer the following fundamental questions:

what can happen and why (by risk identification)?

what are the consequences?

what is the probability of their future occurrence?

are there any factors that mitigate the consequence of the risk or that reduce the probability

of the risk?

Providing that you adhere to this basic structure, you are following the framework that is set out in

the International Standard ISO 31000:2009.

Rather than spending several days reading the Standard and having long meetings with colleagues to

see how it might be applicable, why not look for methods that would help you to meet the

requirements of ISO 9001?

For me, a good start would be:

Documenting the results of any 'consideration of risks and opportunities' exercise as evidence of

your management team's "risk-based thinking".

Even if it is clear from the design of your processes that you have taken account of Clause 6.1 and

determined the risks and opportunities that need to be addressed, having a record of your risk

assessment processes might prove useful, if only as a reminder to keep matters under review!

Then, evaluate the risk assessment tools (numbering 31 in total) in ISO 31010 to see if they are

applicable to your organizational context.

It's probably not the time to use them in anger yet (see below), but at least you will know they exist

and that some tools could help to identify risks and opportunities and be useful in carrying out risk

analysis (if you consider consequences, probability and level of risk) and risk evaluation?

2.2.3

Are structured interviews and brainstorming 9001 requirements?

No, absolutely not. Although if you don't currently use risk assessment tools to identify the typicaluncertainties that need to be considered, and there is no previously developed list available of

hazards, risks or control failures, either resulting from a previous risk assessment or past failures,-

where do you begin? This is likely to be a especially vexing question for organizations that are new to

ISO 9001 quality management and have to develop appropriate documented information for their

quality processes.

However, a cautionary note:

Before you despair and start writing out check-lists based on your own observations in an effort to

tick the box, remember that your colleagues in other departments and business units may already

be using some of the formal techniques of risk assessment and risk management process (in a 'silo-centric' way of course), without you even knowing about this.


15/58

Page 15

To quote from the Introduction to ISO 31000:2009:

"The current management practices and processes of many organizations include components of

risk management, and many organizations have already adopted a formal risk management processfor particular types of risk or circumstances".2

It follows therefore that it is worth interviewing them (in a structured or unstructured way) or

bringing them together for a brainstorming session - if only to find out what qualitative and

quantitative risk assessments have been made that could help you to address the requirements of

ISO 9001!

Whether or not though anyone is carrying out risk assessments, with or without the use of the tools

in ISO 31010, ISO 9001:2015 expects the organization to understand its context (see clause 4.1) and

determine the risks and opportunities that need to be addressed (see clause 6.1).

For example:

The ISO assume that one of the key purposes of a quality management system is to act as a

preventive tool, taking account of identified risks. Consequently, ISO 9001:2015 does not have a

separate clause or sub-clause titled 'Preventive action. Rather, the wording states unequivocally:

"The concept of preventive action is expressed through a risk-based approach to formulating quality

management system requirements".3

Although there are undoubtedly a number of quality professionals who feel uncomfortable talking

about risk in relation to preventive actions, assessing risk is something that managers in most (all?)organizations do already in one form or another. They may not always use the term risk to describe

their activities, - which could include for example conducting a sensitivity analysis of a financial

projection, or scenario planning for a project appraisal, assessing the contingency allowance in a cost

estimate, negotiating contract conditions, or developing contingency plans - ; but even so, thinking

about risks and opportunities is central to their work.4

IF it can reasonably be argued that managing risk is an integral part of good management (and we

think that it can) and that risk-based thinking is fundamental to achieving good business and project

outcomes and the effective procurement of goods and services, THEN identifying, analysing and

evaluating risk should be processes familiar to all quality managers?

Not everyone agrees with this statement of course, but understanding the context (see clause 4.1)

and determining the risks and opportunities that need to be addressed (clause 6.1) are requirements

of ISO 9001:2015. Therefore, before you reject the idea of using risk assessment tools because they

2ISO 31000:2009 - Principles and Guidelines on Implementation

3Draft BS EN ISO 9001 Quality Management Systems - Requirements, Date: 14 May 2014, A.4 Risk-based approach

4Project risk management guidelines: managing risk with ISO 31000 and IEC 62198, Dale F Cooper, et al, Wiley, 2014.


16/58

Page 16

are too complicated and "not part of your job", it is worth pondering this quote from the

Introduction to the ISO 31000:2009:

"The generic approach described in this International Standard provides the principles andguidelines for managing any form of risk in a systematic, transparent and credible manner and

within any scope and context".5

2.3 Other Supporting Methods

We have already looked at the following Look-

Up and Supporting Methods that are relevant to

risk identification:

Check-lists

Brainstorming

Structured or semi-structured interviews

Brainstorming and structured/semi-structured

interviews are techniques that are often used for

improving the accuracy and completeness in risk

identification; the Delphi methodology is

another.

2.3.1

Delphi technique

A structured collaborative communication technique, originally developed as a systematic,

interactive forecasting method which relies on a panel of experts. By combining expert opinions, the

aim is to support the source and influence identification, probability and consequence estimation

and risk evaluation. The experts answer questionnaires in two or more rounds. After each round, a

facilitator provides an anonymous summary of the experts forecasts from the previous round as

well as the reasons they provided for their judgments. In this way, experts are encouraged to revise

their earlier answers in light of the replies of other members of their panel.

Delphi can be used to estimate probability of adverse and positive outcomes: In the words of ISO

31010:

"Expert opinion can be used in a systematic and structured process to estimate probability. Expert

judgements should draw upon all relevant available information including historical, system-specific,

organizational-specific, experimental, design, etc. There are a number of formal methods for eliciting

expert judgement which provide an aid to the formulation of appropriate questions. The methods

available include the Delphi approach, paired comparisons, category rating and absolute probability

judgements."6

5ISO 31000:2009 - Principles and Guidelines on Implementation, Introduction, p.V

6ISO/IEC 31010:2009Risk managementRisk assessment techniques, p.15.
https://www.cognidox.com/assets/uploads/2015/04/84518197-84d8-4f0e-87a6-802042b7851f.png


17/58

Page 17

Despite the mention of probability above, Table A.1Applicability of tools used for risk assessment,

the Delphi method is marked 'NA' [NA = Not Applicable] for Risk Analysis to assess Consequence,

Probability and Level of risk - although personally we would agree with the commentary on page 29

[Clause B.3.2 Use] which states:

"The Delphi technique can be applied at any stage of the risk management process or at any phase

of a system life cycle, wherever a consensus of views of experts is needed."7

A true consensus approach that avoids the bias of dominant members of the team can be the wake-

up call that management needs to assess risk.

2.3.2 SWIFT (Structured what-if)

SWIFT is a system for prompting a team to identify risks, normally used within a facilitated workshop

and linked to a risk analysis and evaluation technique.

The first thing to understand about SWIFT is that it was originally developed as a simpler alternative

to HAZOP (Hazard and Operability Studies), a qualitative risk identification technique. HAZOP aims to

stimulate the imagination of participants to identify potential hazards and operability problems;

structure and completeness are given by using guideword prompts. The HAZOP technique was

developed to analyse chemical process systems and mining operation process but has later been

extended to other types of systems and also to complex operations such as nuclear power plant

operation and to use software to record the deviation and consequence.8HAZOP is intended for

high-risk organizational contexts where appropriate levels of resourcing are available to support its

use. SWIFT, on the other hand, has been purposely-design as a sort of 'HAZOP-Lite' needing fewer

resources. ISO 31010 regards the 'Resources and capability' requirement as "Medium", so this maybe a viable risk identification technique for use by most small to medium as well as larger quality

conscious organizations?

The system, procedure, plant item and/or change has to be carefully defined before the study can

commence. Both the external and internal contexts are established through interviews and through

the study of documents, plans and drawings by the facilitator.

The facilitator asks the participants to raise and discuss:

known risks and hazards;

previous experience and incidents;

known and existing controls and safeguards;

regulatory requirements and constraints.9

7Ibid., page 29.

8British Standard BS: IEC61882:2002 Hazard and operability studies (HAZOP studies)- Application Guide, published by BSI Group.

9ISO/IEC 31010:2009, B.9.3 Inputs, p.39.


18/58

Page 18

Discussion is facilitated by creating a question using a what-if phraseand a prompt word or subject.

The what-if phrases to be used are what if, what would happen if, could someone or

something, has anyone or anything ever. The intent is to stimulate the study team into

exploring potential scenarios, their causes and consequences and impacts.10

The risks identified are summarized and the team considers the controls already in place - assuming

that there are any - before confirming the description of the risk, its causes, consequences and

expected controls.

This information is then recorded.

What we particularly like about the SWIFT concept approach is the inherent discipline which forces

the team members to consider the effectiveness of the controls. Assessing risk is one thing, but

treating it is another entirely. They have to agree a statement of risk control effectiveness, which, if

it proves to be less than satisfactory, triggers the task of further considering risk treatment tasks andpotential controls.

The application of this team-based model does not have to be complex. ISO 31010 simply rates the

Complexity of the technique as "Any".11

2.3.3

Human reliability analysis (HRA)

Human reliability assessment (HRA) deals with the impact of humans on system performance, and

can be used to evaluate human error influences on the system.

At the risk of stating the obvious, human reliability is very important due to the contributions of

humans to the resilience of systems and to possible adverse consequences of human errors or

oversights, especially when the human is a crucial part of today's large socio-technical systems.

Contrary to the impression that you might receive by reading the relevant section in ISO 31010 -

specifically B.20 Human reliability assessment (HRA) - a variety of methods exist for human reliability

analysis. These break down into two basic classes of assessment method:

probabilistic risk assessment(PRA), and

those based on a cognitive theory ofcontrol.

In 2009, the Health and Safety Laboratory compiled a report12for the Health and Safety Executive

(HSE) outlining HRA methods for review.

10Ibid.

11Ibid., Table A.2 - Attributes of a selection of risk assessment tools.

12Review of human reliability assessment methods, Prepared by the Health and Safety Laboratory for the Health and Safety Executive

2009, PR679 Research Report, Julie Bell & Justin Holroyd, Health and Safety Laboratory; First published 2009.
http://en.wikipedia.org/wiki/Probabilistic_risk_assessmenthttp://en.wikipedia.org/wiki/Probabilistic_risk_assessmenthttp://en.wikipedia.org/wiki/Control_theoryhttp://en.wikipedia.org/wiki/Control_theoryhttp://en.wikipedia.org/wiki/Control_theoryhttp://en.wikipedia.org/wiki/Control_theoryhttp://en.wikipedia.org/wiki/Probabilistic_risk_assessment


19/58

Page 19

They identified 35 tools that constituted true HRA techniques and that could be used effectively in

the context of health and safety management.

Obviously, it is well beyond the scope of this article to define the merits and demits of all thesemethods. However, the HRA tools in the table below illustrates that there are a large number of risk

assessment techniques in the Health & Safety arena that could be applied elsewhere. It is also worth

reflecting that Risk Management is usually associated with the financial risk; however, risk

assessment techniques have other well-established uses including helping to maintain safe working

environments.

Without being specific at this time, we think that it is possible that some of these tools could be

adapted (if they haven't been?) to identify, analyse and evaluate risks and opportunities in the

design of quality processes. After all, corrective and preventive actions usually involve human

beings!

Acronym for Tool Expanded name

ASEP Accident Sequence Evaluation Programme

AIPA Accident Initiation and Progression Analysis

APJ Absolute Probability Judgement

ATHEANA A Technique for Human Error Analysis

CAHR Connectionism Assessment of Human Reliability

CARA Controller Action Reliability Assessment

CES Cognitive Environmental Simulation

CESA Commission Errors Search and Assessment

CM Confusion Matrix

CODA Conclusions from occurrences by descriptions of actions

COGENT COGnitive EveNt Tree

COSIMO Cognitive Simulation Model

CREAM Cognitive Reliability and Error Analysis Method

DNE Direct Numerical Estimation

DREAMS Dynamic Reliability Technique for Error Assessment in Man-

machine Systems

FACE Framework for Analysing Commission Errors

HCR Human Cognitive Reliability

HEART Human Error Assessment and Reduction Technique

HORAAM Human and Organisational Reliability Analysis in AccidentManagement

HRMS Human Reliability Management System

INTENT Not an acronym

JHEDI Justified Human Error Data Information

MAPPS Maintenance Personnel Performance Simulation

MERMOS Method d'Evaluation de la Realisation des Missions Operateur pour

la Surete (Assessment method for the performance of safety

operation.)

Table 2: List of HRA tools


20/58

Page 20

As ISO 31010 points out in the section on the 'Limitations' of HRA, many activities of humans do not

have a simple pass/fail mode. HRA has difficulty dealing with partial failures or failure in quality or

poor decision-making.13

2.4

Scenario Analysis

2.4.1

Root cause analysis (RCA)

Root Cause Analysis (RCA) uses a specific set of steps, with associated tools, to help find the primary

cause of the problem; so that you can:

Determine what happened.

Determine why it happened

Figure out what to do to reduce the likelihood that it willhappen again. RCA assumes that systems and events are

interrelated. An action in one area triggers an action in

another, and another, and so on. By tracing back these

actions, you can discover where the problem started and

how it grew into the symptom you are now facing.14

2.4.2

Scenario analysis

Scenario analysis is a process of analyzing possible future events by considering alternative

outcomes (sometimes called "alternative worlds").15

The technique can be used to identify risks by considering sets of scenarios that reflect (for example)

best case, worst case and expected case,in order to analyse potential consequences and their

probabilities for each scenario as a form of sensitivity analysis when analysing risk.

'The possible future scenarios or 'alternative worlds' are identified:

"...through imagination or extrapolation from the present and different risks considered

assuming [that] each of these scenarios might occur. This can be done formally or informally,

qualitatively or quantitatively."16

13ISO/IEC 31010:2009, B.20.6 Strengths and limitations, p.63.

14Root Cause Analysis, Tracing a Problem to its Root Origins, Mind Tools website:

http://www.mindtools.com/pages/article/newTMC_80.htm

15Scenario Analysis, Wikipedia: http://en.wikipedia.org/wiki/Scenario_analysis.

16ISO/IEC 31010:2009, Table A.2 - Attributes of a selection of risk assessment tools.
http://www.mindtools.com/pages/article/newTMC_80.htmhttp://www.mindtools.com/pages/article/newTMC_80.htmhttp://www.mindtools.com/pages/article/newTMC_80.htm


21/58

Page 21

2.4.3

Toxicological / Environmental / Ecological risk assessment

An ecological risk assessment tells what happens to a bird, fish, plant or other non-human organism

when it is exposed to a stressor, such as a pesticide.17

Aspects of the methodology, such as pathway analysis which explore different routes by which a

target might be exposed to a source of risk, can be adapted and used across a very wide range of

different risk areas, outside human health and the environment, and is useful in identifying

treatments to reduce risk.18

The strength of this analysis is that it provides a very detailed understanding of the nature of the

problem and the factors that increase risk. However, it needs good data that is often not available or

has a high level of uncertainty associated with it. Likewise, it is also resource intensive as is unlikely

to find many uses in quality management systems.

Pathway analysis, though, is a useful tool, generally, for all areas of risk and permits the

identification of how and where it may be possible to improve controls or introduce new ones.

If you are interested in following the steps of this type of environmental risk assessment process, we

recommend that you read 'Basic Information about Risk Assessment Guidelines Development',

published by the United States Environmental Protection Agency. See the web page link below:

http://www2.epa.gov/osa/basic-information-about-risk-assessment-guidelines-development

2.4.4

Business impact analysis (BIA)

A Business Impact Analysis identifies an organization's exposure to internal and external threats and

synthesizes hard and soft assets to provide effective prevention and recovery for the organization,

while maintaining competitive advantage and value system integrity.19

The analysis provided by a conscientiously-conducted BIA could be of value when determining "...the

external and internal issues that are relevant to the organization's purpose ... and that affect its

ability to achieve the intended result(s) of its quality management system"; as well as helping to

determine who are "the interested parties", and the requirements of these interested parties that

are relevant to the quality management system - see ISO 9001:2015 Clause 4 Context of the

organization.

If your organization already has a business continuity management (BCM) system based on the ISO

22301 Standard and since a BIA is a mandatory document, seeking out your Business Continuity

Manager to obtain the BIA report could be a sound move at this point. You will then have a valuable

17Ecological Risk Assessment: Technical Overview, Ecological Risk Assessment Process, U.S. Environmental Protection Agency website:

http://www.epa.gov/oppefed1/ecorisk_ders/index.htm#WITERAP

18ISO/IEC 31010:2009, B.8.2 Use, p.37.

19Elliot, D.; Swartz, E.; Herbane, B. (1999) Just waiting for the next big bang: business continuity planning in the UK finance sector. Journal

of Applied Management Studies, Vol. 8, No, pp. 4360. Here: p. 48
http://www2.epa.gov/osa/basic-information-about-risk-assessment-guidelines-developmenthttp://www2.epa.gov/osa/basic-information-about-risk-assessment-guidelines-developmenthttp://www.epa.gov/oppefed1/ecorisk_ders/index.htm#WITERAPhttp://www.epa.gov/oppefed1/ecorisk_ders/index.htm#WITERAPhttp://www.epa.gov/oppefed1/ecorisk_ders/index.htm#WITERAPhttp://www2.epa.gov/osa/basic-information-about-risk-assessment-guidelines-development


22/58

Page 22

item of documented information to show risk-based thinking because you will have assessed (by

means of the BIA) how key disruption risks could affect an organizations operations and

identified/quantified the capabilities that would be required to manage it.

If not, well ... you could consider conducting a BIA; although we would strongly recommend calling in

a qualified business continuity consultant.

2.4.5

Fault tree analysis

A technique used in safety engineering and reliability engineering, mostly in the aerospace, nuclear

power, chemical and process, pharmaceutical, petrochemical and other high-hazard industries. Fault

tree analysis (FTA) can be used to understand how systems can fail, to identify the best ways to

reduce risk or to determine or 'get a feel for' event rates of a safety accident or a particular system

level (functional) failure. It sounds more complicated than it actually is; however, it is a resource

hungry method.

If you are a Quality Manager in one of the above industries you will probably already be familiar with

fault tree diagrams produced from this type of analysis and you may well use the fault trees

developed by the organization to reduce or eliminate potential causes of non-conformities. They

start with the undesired event (top event) and determine all the ways in which it could occur, shown

graphically in a logical tree diagram.

Fault tree analysis is a time-consuming and costly exercise although it can be invaluable in

determining the probability of (undesirable) outcomes.

FTA can be used to:

understand the logic leading to the top event / undesired state.

show compliance with the (input) system safety / reliability requirements.

prioritize the contributors leading to the top event - Creating the Critical

Equipment/Parts/Events lists for different importance measures.

monitor and control the safety performance of the complex system (e.g., is a particular

aircraft safe to fly when fuel valve x malfunctions? For how long is it allowed to fly with the

valve malfunction?).

minimize and optimize resources.

assist in designing a system. The FTA can be used as a design tool that helps to create

(output / lower level) requirements.

function as a diagnostic tool to identify and correct causes of the top event. It can help with

the creation of diagnostic manuals / processes.20

2.4.6

Event tree analysis

A forward, bottom up, logical modelling technique for both success and failure that explores

responses through a single initiating event and lays a path for assessing probabilities of the

20Fault tree analysis, Wikipedia: http://en.wikipedia.org/wiki/Fault_tree_analysis


23/58

Page 23

outcomes and overall system analysis. Using inductive reasoning, ETA translates probabilities of

different initiating events into possible outcomes. It is arguably less resource intensive than fault

tree analysis (see Table A.2 in ISO 31010).

ETA can be applied to a wide range of systems including: nuclear power plants, spacecraft, and

chemical plants.21

Once again, if you are managing the quality system of a small enterprise in a relatively 'low risk'

context, this technique is unlikely to be for you.

2.4.7

Cause and consequence analysis

ISO 31010 describes the Cause and consequence analysis method as:

"A combination of fault and event tree analysis that allows inclusion of time delays. Both causes andconsequences of an initiating event are considered."

It starts from a critical event and analyses consequences by means of a combination of YES/NO logic

gates that represent conditions that may occur or failures of systems designed to mitigate the

consequences of the initiating event. The causes of the conditions or failures are analysed by means

of fault trees (see ISO 31010, Clause B.15).

Cause-consequence analysis does provide a comprehensive view of the entire system. However, it is

more complex than fault tree and event tree analysis, both to construct and in the manner in which

dependencies are dealt with during quantification, and so requires more time and resources.

2.4.8

Cause-and effect analysis

An effect can have a number of contributory factors that can be grouped in Ishikawa diagrams.

Contributory factors are identified often through a brainstorming process (see Part II of this article

for more information).

Kaoru Ishikawa popularized these diagrams in the 1960s, when he pioneered quality management

processes in the Kawasaki shipyards. The basic concept was first used in the 1920s, and is considered

one of the seven basic tools of quality control. Ishikawa diagrams are known as fishbone diagrams

because their shape is like the side view of a fish skeleton.

The basic steps in performing a cause-and-effect analysis are as follows:22

1. establish the effect to be analysed and place it in a box. The effect may be positive (an

objective) or negative (a problem) depending on the circumstances;

21 Event Tree Analysis, Wikipedia: http://en.wikipedia.org/wiki/Event_tree_analysis.

22ISO/IEC 31010:2009, B.17.4 Process, p.57.


24/58

Page 24

2. determine the main categories of causes represented by boxes in the Fishbone diagram.

Typically, for a system problem, the categories might be people, equipment,

environment, processes, etc. However, these are chosen to fit the particular context;

3.

fill in the possible causes for each major category with branches and sub-branches todescribe the relationship between them;

4. keep asking why? or what caused that? to connect the causes;

5. review all branches to verify consistency and completeness and ensure that the causes

apply to the main effect;

6. identify the most likely causes based on the opinion of the team and available evidence.

The results are displayed as either an Ishikawa diagram or tree diagram.

2.5 Function Analysis

2.5.1

FMEA and FMECA

This section covers FMEA (Failure modes and effects analysis) and FMECA (Failure modes and effects

and criticality analysis).

FMEA/FMECA is aninductive reasoning(forward logic) single point of failure analysis and is a core

task inreliability engineering,safety engineeringandquality engineering.Quality engineering is

especially concerned with the "Process" (Manufacturing and Assembly) type of FMEA.23

FMEA/FMECA identifies:

all potential failure modes of the various parts of a system (a failure mode is what isobserved to fail or to perform incorrectly);

the effects these failures may have on the system;

the mechanisms of failure;

how to avoid the failures, and/or mitigate the effects of the failures on the system.

FMEA/FMECA is a systematic analysis technique that can be used to identify the ways in which

components, systems or processes can fail to fulfil their design intent, highlighting:

design alternatives with high dependability;

failure modes of systems and processes, and their effects on operational success have

been considered;

human error modes and effects;

a basis for planning testing and maintenance of physical systems;

improvements in the design of procedures and processes.

FMEA/FMECA also provides qualitative or quantitative information for other types of analysis, such

as fault tree analysis, and is used in quality assurance applications. For example, it can produce a

semi-quantitative measure of criticality known as the risk priority number (RPN) obtained by

multiplying numbers from rating scales (usually between 1 and 10) for (a) consequence of failure, (b)

23Failure mode and effects analysis, Wikipedia:http://en.wikipedia.org/wiki/Failure_mode_and_effects_analysis
http://en.wikipedia.org/wiki/Inductive_reasoninghttp://en.wikipedia.org/wiki/Inductive_reasoninghttp://en.wikipedia.org/wiki/Inductive_reasoninghttp://en.wikipedia.org/wiki/Reliability_engineeringhttp://en.wikipedia.org/wiki/Reliability_engineeringhttp://en.wikipedia.org/wiki/Reliability_engineeringhttp://en.wikipedia.org/wiki/Safety_engineeringhttp://en.wikipedia.org/wiki/Safety_engineeringhttp://en.wikipedia.org/wiki/Safety_engineeringhttp://en.wikipedia.org/wiki/Quality_engineeringhttp://en.wikipedia.org/wiki/Quality_engineeringhttp://en.wikipedia.org/wiki/Quality_engineeringhttp://en.wikipedia.org/wiki/Failure_mode_and_effects_analysishttp://en.wikipedia.org/wiki/Failure_mode_and_effects_analysishttp://en.wikipedia.org/wiki/Failure_mode_and_effects_analysishttp://en.wikipedia.org/wiki/Failure_mode_and_effects_analysishttp://en.wikipedia.org/wiki/Quality_engineeringhttp://en.wikipedia.org/wiki/Safety_engineeringhttp://en.wikipedia.org/wiki/Reliability_engineeringhttp://en.wikipedia.org/wiki/Inductive_reasoning


25/58

Page 25

likelihood of failure, (c) ability to detect the problem. Note, a failure is given a higher priority if it is

difficult to detect.

2.5.2

Reliability-centred maintenance (RCM)

A technique that is used to achieve the required safety, availability and economy of operation (safe

minimum levels of maintenance), so that assets continue to do what their users require in their

operating context.

RCM allows you to identify applicable and effective preventive maintenance requirements for

equipment "...in accordance with the safety, operational and economic consequences of identifiable

failures, and the degradation mechanism responsible for those failures".24

RCM uses a failure mode, effect and criticality analysis (FMECA) type of risk assessment that requires

a specific approach to analysis in this context. From a quality management standpoint, it's worthbeing aware that RCM identifies required functions and performance standards and failures of

equipment and components that can interrupt those functions.

For more information, seeIEC 60300-3-11, Dependability managementPart 3-11: Application

guideReliability

2.5.3

Sneak analysis (SA) and sneak circuit analysis (SCI)

Sneak analysis is aimed at uncovering design flaws that allow for 'sneak conditions', i.e. those that

may cause unwanted actions or may inhibit a desired function, and are not caused by component

failure to develop.

Sneak analysis can locate problems in both hardware and software using any technology. The sneak

analysis tools can integrate several analyses such as fault trees, failure mode and effects analysis

(FMEA), reliability estimates, etc. into a single analysis saving time and project expenses.25 The

technique helps in identifying design errors and works best when applied in conjunction with

HAZOP. It is very good for dealing with systems which have multiple states such as batch and semi-

batch plant.

Sneak Circuit Analysis (SCA) is used in safety-critical systems to identify sneak (or hidden) paths in

electronic and electro-mechanical systems that may cause unwanted action or inhibit desired

functions. The analysis is based on identification of designed-in inadvertent modes of operation and

is not based on failed equipment or software. SCA is most applicable to circuits that can cause

irreversible events. These include:

a. Systems that control or perform active tasks or functions

b. Systems that control electrical power and its distribution

24ISO/IEC 31010:2009, B.22.1 Overview, p.66

25Ibid., B.23.2 Use, p.68.
http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134http://shop.bsigroup.com/ProductDetail/?pid=000000000030144134


26/58

Page 26

c. Embedded code which controls and times system functions.26

The SA process differs depending on whether it is applied to electrical circuits, process plants,

mechanical equipment or software technology, and the method used is dependent on establishingcorrect network trees.

2.5.4

HACCP

HACCP a systematic preventive approach to food safety from biological, chemical, and physical

hazards in production processes that can cause the finished product to be unsafe, and designs

measurements to reduce these risks to a safe level.27 HACCP has been recognized internationally as

a logical tool for adapting traditional inspection methods to a modern, science-based, food safety

system.28

HACCP is focused only on the health safety issues of a product ensuring that risks are minimized bycontrols throughout the process rather than through inspection of the end product. The seven

HACCP principles are the basis of most food quality and safety assurance systems, and the United

States, HACCP compliance is regulated by21 CFR part 120 and 123. The HACCP principles are also

included in the international standardISO 22000 FSMS 2005.This standard is a complete food safety

and quality management system incorporating the elements of prerequisite programmes (GMP &

SSOP), HACCP and the quality management system, which together form an organization's Total

Quality Management system.

Table A.1Applicability of tools used for risk assessment [see page 22 of ISO 31010], lists the HACCP

technique as "Not Applicable" for analysis of probability or levels of risk.29 However, the principle of

identifying the factors [risks] that can influence product quality, and defining process points wherecritical parameters can be monitored and hazards controlled, can be generalized for use other

technical systems.30

2.6 Controls Assessment

2.6.1

LOPA (Layers of Protection Analysis)

A technique for analysing whether there are sufficient measures to control or mitigate the risk of an

undesired outcome.

The basic steps are:

26Sneak circuit analysis, Wikipedia: http://en.wikipedia.org/wiki/Sneak_circuit_analysis

27Hazard analysis and critical control points, Wikipedia: http://en.wikipedia.org/wiki/Hazard_analysis_and_critical_control_points

28Ibid.

29ISO/IEC 31010:2009, Table A.1Applicability of tools used for risk assessment, p.22

30Ibid., B.7.2 Use, p.35.
http://en.wikipedia.org/wiki/Title_21_of_the_Code_of_Federal_Regulationshttp://en.wikipedia.org/wiki/Title_21_of_the_Code_of_Federal_Regulationshttp://www.iso.org/iso/catalogue_detail?csnumber=35466http://www.iso.org/iso/catalogue_detail?csnumber=35466http://www.iso.org/iso/catalogue_detail?csnumber=35466http://www.iso.org/iso/catalogue_detail?csnumber=35466http://en.wikipedia.org/wiki/Title_21_of_the_Code_of_Federal_Regulations


27/58

Page 27

A cause-consequence pair is selected, and the layers of protection that prevent the cause

leading to the undesired consequence are identified.

An order of magnitude calculation is then carried out to determine whether the protection is

adequate to reduce risk to a tolerable level.31

LOPA is a less resource-intensive process than a fault tree analysis or a quantitative form of risk

assessment, but is more rigorous than qualitative subjective judgements alone. It focuses efforts on

the most critical layers of protection, identifying operations, systems and processes for which there

are insufficient safeguards and where failure will have serious consequences. However, this

technique looks at one cause-consequence pair and one scenario at a time and, therefore, does not

apply to complex scenarios where there are many cause consequence pairs or where a variety of

consequences affects different stakeholders.

For more information, see:

IEC 61508 (all parts), Functional safety of electrical/electronic/programmable electronic safety-

related systems

IEC 61511, Functional safetySafety instrumented systems for the process industry sector.[PDF]

2.6.2

Bow-tie analysis

Bow-tie analysis is a simple diagrammatic way to display the pathways of a risk showing a range of

possible causes and consequences. It is used in situations when a complex fault tree analysis is not

justified or to ensure that there is a barrier or control for each of the possible failure pathways.

To understand how this works we recommend viewing a short video entitled "The Bow Tie Method

in 5 Minutes" by CGE Risk Management Solutions,32which explains the basics of the method for risk

assessment of hazards.

2.7 Statistical Methods

ISO 31010 lists the following statistical methods for risk assessment:

Markov analysis

Monte-Carlo analysis Bayesian analysis

31Ibid., B.18 Layers of protection analysis (LOPA), p.59.

32The Bow Tie Method in 5 Minutes, CGE Risk Management Solutions, YouTube:https://www.youtube.com/watch?v=P7Z6L7fjsi0
http://www.iec.ch/functionalsafety/http://www.iec.ch/functionalsafety/http://www.iec.ch/functionalsafety/http://webstore.iec.ch/preview/info_iec61511-1%7Bed1.0%7Den.pdfhttp://webstore.iec.ch/preview/info_iec61511-1%7Bed1.0%7Den.pdfhttp://webstore.iec.ch/preview/info_iec61511-1%7Bed1.0%7Den.pdfhttp://webstore.iec.ch/preview/info_iec61511-1%7Bed1.0%7Den.pdfhttps://www.youtube.com/watch?v=P7Z6L7fjsi0https://www.youtube.com/watch?v=P7Z6L7fjsi0https://www.youtube.com/watch?v=P7Z6L7fjsi0https://www.youtube.com/watch?v=P7Z6L7fjsi0http://webstore.iec.ch/preview/info_iec61511-1%7Bed1.0%7Den.pdfhttp://www.iec.ch/functionalsafety/http://www.iec.ch/functionalsafety/


28/58

Page 28

2.7.1

Markov analysis

A method named after a Russian mathematician, best known for his work on stochastic processes,

where a collection of random variables represents the evolution of some system of random valuesover time.

Markov analysis, or State-space analysis, is commonly used in the analysis of repairable complex

systems that can exist in multiple states, including degraded states33, and where the use of a

reliability block analysis would be inadequate to properly analyse the system.

The nature of the Markov analysis techniques lends itself to the use of software. There are several to

choose from on the market.

The Markov analysis process is a quantitative technique and can be discrete (using probabilities of

change between the states) or continuous (using rates of change across the states).

To quote ISO 31010:

"The Markov analysis technique is centred around the concept of states, e.g. available

and failed, and the transition between these two states over time based on a constant

probability of change. A stochastic transitional probability matrix is used to describe the

transition between each of the states to allow the calculation of the various outputs."34

The inputs essential to a Markov analysis are as follows:

list of various states that the system, sub-system or component can be in (e.g. fully

operational, partially operation (i.e. a degraded state), failed state, etc);

a clear understanding of the possible transitions that are necessary to be modelled. For

example, failure of a car tyre needs to consider the state of the spare wheel and hence

the frequency of inspection;

rate of change from one state to another, typically represented by either a probability of

change between states for discrete events, or failure rate () and/or repair rate () for

continuous events.35

The output from a Markov analysis is the various probabilities of being in the various states, and

therefore an estimate of the failure probabilities and/or availability, one of the essential

components of a system.

33ISO/IEC 31010:2009, Table A.2 - Attributes of a selection of risk assessment tools.

34Ibid. B.24.4 Process, p.70.

35Ibid. B.24.3 Input, p.70.


29/58

Page 29

2.7.1.1 Strengths and limitations of a Markov analysis

Markov diagrams for large systems are often too large and complicated to be of value in most

business contexts and inherently difficult to construct. Markov models are more suited to analysingsmaller systems with strong dependencies requiring accurate evaluation. Other techniques, such as

Fault Tree analysis (see Part IV of this blog post series), may be used to evaluate large systems using

simpler probabilistic calculation techniques.

States depend on current state probabilities and the constant transition rates between states - see

the state transition diagram in Figure 1 below:

Figure 1: Example of a state transition diagram

Apart from this obvious drawback (complexity), a true Markovian process would only consider

constant transition rates, which may not be the case in a real-world systems. Events are statisticallyindependent since future states are treated as independent of all past states, except for the state

immediately prior. In this way the Markov model does not need to know about the history of how

the state probabilities have evolved in time in order to calculate future state probabilities. However,

computer programs are being marketed that allow time-varying transition rates to be defined.

Markov analysis requires knowledge of matrix operations and the results are - unsurprisingly! - hard

to communicate with non-technical personnel.

If you would like to perform Markov analysis, you are advised to consultIEC 61165, Application of

Markov techniques.

2.7.2

Monte-Carlo analysis

Monte Carlo analysis consists of a broad class of computational algorithms that rely on repeated

random sampling to obtain numerical results. This method can address complex situations that

would be very difficult to understand and solve by an analytical method. Whenever there is

significant uncertainty in a system and you need to make an estimate, forecast or decision, a Monte

Carlo simulation could be the answer.

2.7.2.1 How does Monte Carlo analysis model the effects of uncertainty?

Systems are sometimes too complex for the effects of uncertainty on them to be modelled usinganalytical techniques. However, they can be evaluated by considering the inputs as random variables
http://webstore.iec.ch/webstore/webstore.nsf/artnum/036122%21opendocumenthttp://webstore.iec.ch/webstore/webstore.nsf/artnum/036122%21opendocumenthttp://webstore.iec.ch/webstore/webstore.nsf/artnum/036122%21opendocumenthttp://webstore.iec.ch/webstore/webstore.nsf/artnum/036122%21opendocumenthttp://webstore.iec.ch/webstore/webstore.nsf/artnum/036122%21opendocumenthttp://webstore.iec.ch/webstore/webstore.nsf/artnum/036122%21opendocument


30/58

Page 30

and running a number N of calculations (so-called simulations) by sampling the input in order to

obtain N possible outcomes of the wanted result.

Monte-Carlo analysis can be developed using spreadsheets, but software tools are readily availableto assist with more complex requirements, many of which are now relatively inexpensive.

Monte-Carlo analysis can be developed using spreadsheets, but software tools are readily available

to assist with more complex requirements, many of which are now relatively inexpensive.

Monte Carlo simulations require you to build a quantitative model of your business activity, plan or

process. This is often done by using Microsoft Excel with a simulation tool plug-in - a relatively

inexpensive set of tools.

To deal with uncertainties using Monte Carlo analysis in your model, you'll replace certain fixed

numbers -- for example in spreadsheet cells -- with functions that draw random samples fromprobability distributions. And to analyze the results of a simulation run, you'll use statistics such as

the mean, standard deviation, and percentiles, as well as charts and graphs.36

For risk assessment using the Monte Carlo simulation, triangular distributions or beta distributions

are commonly used.

Note that ISO 31010 Table A.1Applicability of tools used for risk assessment states this is tool is

strongly applicable for the Evaluation stage of risk assessment but not applicable (NA) for risk

identification or risk analysis.

2.7.3

Bayesian analysis

Referring again to Table A.1 from ISO 31010, Bayesian analysis is used in the risk analysis and risk

evaluation stages in risk assessment.37

In a nutshell, it is a statistical procedure

which utilizes prior distribution data to

assess the probability of the result. These

are often called conditional probabilities.38

There are many places that explain the

mathematics behind Bayes' theorem,

includingWikipedia,theStanford

Encyclopedia of Philosophy,and the

wonderful blogLessWrong.The definition

36Monte Carlo Simulation, web page on Frontline Solvers website

37ISO/IEC 31010:2009, Table A.1Applicability of tools used for risk assessment, p.22.

38ISO/IEC 31010:2009, p.26
http://en.wikipedia.org/wiki/Bayes%27_theorem#Statement_of_theoremhttp://en.wikipedia.org/wiki/Bayes%27_theorem#Statement_of_theoremhttp://en.wikipedia.org/wiki/Bayes%27_theorem#Statement_of_theoremhttp://plato.stanford.edu/entries/bayes-theorem/http://plato.stanford.edu/entries/bayes-theorem/http://plato.stanford.edu/entries/bayes-theorem/http://plato.stanford.edu/entries/bayes-theorem/http://lesswrong.com/lw/774/a_history_of_bayes_theorem/http://lesswrong.com/lw/774/a_history_of_bayes_theorem/http://lesswrong.com/lw/774/a_history_of_bayes_theorem/http://lesswrong.com/lw/774/a_history_of_bayes_theorem/http://plato.stanford.edu/entries/bayes-theorem/http://plato.stanford.edu/entries/bayes-theorem/http://en.wikipedia.org/wiki/Bayes%27_theorem#Statement_of_theorem


31/58

Page 31

that explains it best for me comes from the last of these - it is:

"The probability of a hypothesis C given some evidence E equals our initial estimate of the

probability times the probability of the evidence given the hypothesis C divided by the sumof the probabilities of the data in all possible hypotheses."

Bayesian inference is used in a wide range of fields from medical diagnosis to checking your inbox for

likely spam emails. However, is it any good for risk assessment?

Although it can appear to be objective, this is typically not the case. A Bayesian probability is really a

persons degree of belief in a certain event rather than one based upon physical evidence.

Because the Bayesian analysis approach is based upon the subjective interpretation of probability, it

provides a ready basis for decision thinking and the development of Bayesian nets (or Belief Nets,

belief networks or Bayesian networks).39The availability of software computing tools and what ISO31010 terms "intuitive appeal" has led to the widespread adoption of Bayesian nets. However, they

can be valuable wherever there is the requirement for finding out about unknown variables by using

structural relationships and data.

The inputs are similar to the Monte Carlo analysis above; namely:

define system variables;

define causal links between variables;

specify conditional and prior probabilities;

add evidence to net;

perform belief updating; extract posterior beliefs.40

Bayesian analysis can provide an easily understood model and the data readily modified to consider

correlations and sensitivity of parameters.

This technique could be successfully applied to Quality Management Systems. However, there will

be minimum sample size requirements for control charts that measure non-conformities (errors),

based on the average non-conformity rate in the quality processes being measured.

Lower error rates would therefore require larger sample sizes to make valid inferences because of

the properties of the binomial distribution.

Even so, we would be very interested to hear from Quality Managers who have applied Bayesian

analysis in this way to predict likely error rates in processes!

39ISO/IEC 31010:2009, B.26.1 Overview, p.26.

40Ibid. B.26.3 Input, p.77.


32/58

Page 32

3 A Risk Management Methodology for Quality Management

Those are some of the techniques covered in ISO 31000. In this section, we will apply them to a risk

management methodology suitable for quality standards such as ISO 9001:2015.

3.1

Risk based thinking is the new 'preventive actions' for QMS

To briefly recap the position to date:

ISO 9001 Risk-based thinking could (and we am not saying that it should) be demonstrated by one or

more of the risk assessment tools in ISO 31010:2010. However, that still leaves you with the

dilemma of selecting the most appropriate tools to help you to identify, analyse and evaluate risk in

your organizational context and with the resources at your disposal.

In ISO 9001:2015 there is no requirement for risk management. However, organizations can choose

to develop a more extensive risk-based approach, and the Standard refers to ISO 31000, which

provides guidelines that can be appropriate in "certain organizational contexts".

It remains to seen whether assessors for the various Certification Bodies will expect you to produce

documented evidence of risk-based thinking.

How will ISO Assessors attempt to assess RBT in Quality Systems?

The short answer is we do not know at present. However, as we have postulated, there are three

possibilities:

Option 1: They will ignore the risk-based thinking requirements of Clause 6 in the same way that

some claim preventive actions were ignored in the past. The counter to this is that Clause 6 in the

DIS requires "Processes for planning and consideration of risks and opportunities".

Option 2: They will regard the failure to show evidence of risk-based thinking in an organizations

quality processes as a non-conformity (perhaps even a major non-conformity) and will judge the

quality system to be ineffective because it has failed to reduce or eliminate the risks to process

outputs.

Option 3: Auditors will highlight in their report any good practices seen in the application of risk-based thinking to the planning and consideration of quality processes; showing how this has helped

to achieve continual improvement of the system and provide the assurance of conformity to

customer and applicable statutory and regulatory requirements.

You may decide differently, but in our view, Option 3is more likely in the majority of cases. Ergo, it

cannot hurt your case to show documented evidence of RBT, regardless of whether documented

information is a requirement or not.

However, it will be your assessor that decides this, not us!

Regarding Option 3 above, it is also worth reflecting upon the number uses of the words "continualimprovement" in the clauses of the new Standard.


33/58

Page 33

Aside from the definition that appears in Normative References, the term "continual improvement"

is used in Clause 5: Leadership, Clause 6: Planning, Clause 7: Support, Clause 9: Performance

Evaluation, and - unsurprisingly - in Clause 10: Continual Improvement; which states that:

"...the organization shall consider the outputs of analysis and evaluation, and the outputs

from management review, to confirm if there are areas of underperformance or

opportunities that shall be addressed as part of continual improvement."41

There is doubt about which of the three options above best describes the likely future response of

external auditors/assessors, but you can help put your organization in a position where Option 3 is

the more likely outcome, because your quality processes reflect the fact that you have taken

account of the risk and opportunities in your context.

3.1.1

Planning and considering risks in quality system processes

Notwithstanding the concerns about what ISO 9001 assessors may or may not be looking for with

regard to applying risk-based thinking (RBT), there are good reasons to put in place...

"Processes for planning and consideration of risks and opportunities"

There is already a significant precedent in the ISO family of management system standards that

explains the need for the risk-based approach.

BSI's Product Guide, ISO/IEC 27001 Information Security Management, sets out the case for RBT in

the context of improving information security:

"ISO/IEC 27001 takes a risk-based approach to the planning and implementation of your

ISMS, resulting in an appropriate and affordable level of organizational security. In this way,

it ensures that the right people, processes, procedures and technologies are in place to

secure your organizations information assets."42

We suggest that we could readily substitute "ISO 9001:2015" for "ISO/IEC 27001"; "ISMS" for "QMS";

"quality" for "organizational security"; and "achieve the intended results of the quality management

system" for "secure your organization's information assets" to arrive at the following:

"ISO 9001:2015 takes a risk-based approach to the planning and implementation of your

QMS, resulting in an appropriate and affordable level of quality. In this way, it ensures that

the right people, processes, procedures and technologies are in place to achieve the

intended results of the quality management system."

It is also worth bearing in mind that one of the key influences on the development of ISO

27001:2013 was the decision by the ISO to align ISO/IEC 27001 with the principles and guidance

given in ISO 31000 (risk management). This was deemed to be, in the words of BSI, "good news for

41ISO/DIS 9001:2014, 10.3 Continual improvement, p.63.

42ISO/IEC 27001 Information Security Management Securing your information assets Product Guide, October 2012 (modified May 2013)


34/58

Page 34

integrated management systems as now an organization may apply the same risk assessment

methodology across several disciplines".43

Earlier posts in this series have examined the different risk assessment techniques aligned to ISO31000 and described fully in ISO 31010:2009.

3.1.2

What actions are required to plan for risks and opportunities?

Clause 6 of ISO 9001:2015 is likely to be explicit about the need for planned actions to address risks

and opportunities in quality systems:

6.1.2The organization shall plan:

1. actions to address these risks and opportunities;

2.

how to:a. integrate and implement the actions into its quality management system

processes (see 4.4);

b. evaluate the effectiveness of these actions.

Actions taken to address risks and opportunities shall be proportionate to the potential impact on

the conformity of products and services.44

Although not all the processes of the quality management system will represent the same level of

risk in terms of th

Documents

How to Apply Risk-based Thinking to Quality Processes