Internal Controls Practices Group - WECC Internal... · 2019. 8. 1. · Poka-Yoke. Poka-Yoke...

Internal Controls

Practices Group

August 1, 2019

Ruchi Shah

Director, Risk Assessment

& Mitigation

Purpose

A platform to bring together WECC and

industry leaders to share practices regarding the

development and sustainability of internal

controls programs for NERC Reliability

Standards.

Requirement % of PNCs

CIP-007-6 R2. 6%

CIP-004-6 R5. 5%

CIP-004-6 R4. 5%

CIP-010-2 R1. 5%

CIP-006-6 R1. 4%

PRC-005-6 R3. 4%

CIP-002-5.1a R1. 3%

CIP-006-6 R2. 3%

CIP-007-6 R5. 3%

CIP-004-6 R3. 2%

PRC-024-2 R2. 2%

PRC-019-2 R1. 2%

Other 55%

Top 12 Most Violated Requirements

Cause Group Description % PNCs

A3B1 Skill-based error 31%

A3B2 Rule-based error 27%

A4B1 Inadequate management methods 12%

A4B3 Inadequate work organization and

planning

Top PNC Causes

How can Practices Group Help?

Understand how failures happen

Identify common ways to address the failures

Reduce Noncompliance

& Improve Reliability &

Security

Key Takeaways

▪ Gain a better understanding of Internal Controls concepts

▪ Implement ideas and practices in day-to-day operations

▪ Peer-to-peer learning

▪ Understand the concepts and tools

▪ Practice sharing

▪ Wrap-up

Contact:

Ruchi Shah

rshah@wecc.org

(801) 883-6881

Internal Controls

Practice Group

August 1, 2019

Harold Sherrill

Risk Assessment &

Mitigation

▪ Internal Controls Program Components

• Risk Assessment

• Design & Implementation

▪ Concepts

• Poka-Yoke

• Process Failure Mode Effect Analysis (PFMEA)

◦ Application of PFMEA

▪ Practice Sharing

• Introductions

Agenda

Program Components

Internal Controls Program

Reliability and Security

Program Components

Internal Controls Program

Reliability and Security

▪ Poka-Yoke/Mistake Proofing

▪ Process Failure Mode Effect Analysis

(PFMEA)

Concepts & Tools

Poka-Yoke/Mistake Proofing

Mistake proofing is about awareness, detection, and prevention of

errors that damage outcomes (i.e., reliability & security) and

compliance.

▪ Awareness—communicating the potential for mistakes and

designing the process to detect or prevent mistakes.

▪ Detection—allowing the mistake to happen, but providing a

way to uncover the mistake.

Proactive Focus:

▪ Prevention—keeping process mistakes from occurring in the

first place.

Poka-Yoke

Exercise

Poka-Yoke for Process Improvement

Human Error

An analyst uses a spreadsheet to track upcoming due

dates. They misread a date which caused an

important task to be late.

Possible Solutions

• Highlight near or past-due items

• Add a “days till due” count-down

How to Build a PFMEA

Sub-Process Action

Sub-Practice Function

Potential Failure Mode

Potential Causes of Failure

Potential Effects of Failure

Step 1)

Create an action statement from the language of the

requirement and place it in the Sub-Process Action

column. For instance, “…shall have documentation

for determining the facilities ratings…”

Sub-Process Action

Step 2)

Determine what the requirement is asking you to do.

In this example, you are being required to document

how you determine facility ratings. So, the Sub-

Practice Function is to “develop documentation.”

Sub-Practice Function

Step 3)

Detail the “way” in which you might fail to meet the requirement in the Potential Failure Mode column. In this example, you might fail by having “No or poor documentation suitable to effectively capture ratings.”

Potential Failure Mode

Step 4)

Now find the “cause” of this potential failure. One

cause of might be that you did not include guidance

on how exactly you will produce and maintain

documentation. In this example, the Potential Causes

of Failure might be “Failure to develop guidance

specifying how [the entity] shall have documentation

for determining Facility Ratings.”

Potential Causes of Failure

Step 5)

Finally, you must state the “effect” if you fail to

mitigate the Potential Causes of Failure. In this

example, the effect statement might be “Reliability

issues due to lack of understanding of facilities

ratings and subsequent limits for devices, lines, and

facilities.”

Potential Effects of Failure

The potential failure points and guidance questions give direction to registered entities for assessment of risk, while designing internal controls specific to NERC Reliability Standards and Requirements. The Registered Entity may use this document as a starting point in determining entity risk. It is not WECC’s intent to establish a standard or baseline for entity risk assessment or controls design.

Note: Guidance questions help an entity understand and document its controls. Any responses, including lack of affirmative feedback, will have no consequences on an entity’s demonstration of compliance at audit.

*Please send feedback to ICE@WECC.org with suggestions on potential failure points and guidance questions.

Intent of Failure Points

▪ Internal Controls Program Components

• Risk Assessment

• Design & Implementation

▪ Poka-Yoke/Mistake Proofing

▪ Process Failure Mode Effect Analysis

(PFMEA)

• Application of PFMEA

Summary/Questions

READY,

SHARE!

Internal Controls Failure Points

CIP-007-6 R1

SECURITY OBJECTIVE

To reduce the attack surface of Cyber Assets by disabling or restricting access to all known unnecessary ports.

To be aware of network-accessible (“listening”) ports and associated services accessible on their assets and

systems, whether they are needed for that Cyber Asset’s function, and disable or restrict access to all other

ports.

NIST Special Publication 800-53 (Rev. 4) CM-6

CIP-007-6 R1

Potential Failure Point (Part 1.1): Failure to develop a complete list of

Cyber Assets that require a process to identify all logical network-

accessible ports.

Potential Failure Point (Part 1.1): Failure to develop a process to

determine technical feasibility.

CIP-007-6 R1

Potential Failure Point (Part 1.1): Failure to develop a process to

identify all logical network-accessible ports.

Potential Failure Point (Part 1.1): Failure to develop a process to identify

which network-accessible ports are needed.

Potential Failure Point (Part 1.1): Failure to have a process to identify

ranges on logical network accessible ports.

CIP-007-6 R1

Potential Failure Point (Part 1.1): Failure to document ports identified

as “needed for operation” in configuration baselines.

Potential Failure Point (Part 1.2): Failure to develop a process to identify and

protect physical input/output ports.

CIP-007-6 R1

SECURITY OBJECTIVE

• Continuously acquire, assess, and act on new information to identify, remediate, and reduce

opportunities for attack.

• Review proposed configuration-controlled changes to the information system and approve or disapprove

changes considering security impact analyses.

• Proactively monitor and address known security vulnerabilities in software before they can be used to

gain control of or render inoperable a Bulk Electric System (BES) Cyber Asset or BES Cyber System.

NIST Special Publication 800-53 (Rev. 4) (CM-4)

CIP-007-6 R2

Potential Failure Point: Failure to have a procedure to update the patch

management process whenever there are changes to the entity’s applicable

Cyber Assets.

Potential Failure Point (Part 2.1): Failure to develop a complete list of

Cyber Assets that require a process to identify and track sources of

patches.

CIP-007-6 R2

Potential Failure Point (Part 2.1): Failure to develop a process/procedure

on how to identify and track sources of patches for applicable systems.

Potential Failure Point (Part 2.1): Failure to have a process or procedure to

evaluate patches for all applicable Cyber Assets, Systems, associated

software, firmware, and drivers.

CIP-007-6 R2

Potential Failure Point (Part 2.1): Failure to have a process or procedure

to install patches for all applicable Cyber Assets, Systems, associated

software, firmware, and drivers.

Potential Failure Point (Part 2.1): Failure to develop a procedure to

document updates of installed patches in baseline configurations.

CIP-007-6 R2

Potential Failure Point (Parts 2.2, 2.4): Failure to define or communicate start/end

dates for monitoring and mitigation timeline(s).

Potential Failure Point (Part 2.3): Failure to have a process for creating a

mitigation plan to properly deal with the vulnerabilities addressed by

each security patch.

CIP-007-6 R2

SECURITY OBJECTIVE

Each Responsible Entity must implement documented processes that collectively include:

• Deploying methods to deter, detect, or prevent malicious code;

• Mitigating the threat of detected malicious code; and

• Updating, testing, and installing identified methods that use signatures or patterns.

NIST Special Publication 800-53 (Rev. 4) SI-3

CIP-007-6 R3

Potential Failure Point (R3): Failure to develop a complete list of

Cyber Assets that require a process to prevent malicious code.

Potential Failure Point (R3): Failure to have a procedure that shows how the entity

will deploy methods to deter, detect, or prevent malicious code.

CIP-007-6 R3

Potential Failure Point (R3): Failure to develop a procedure that shows

how the entity will mitigate the threat of detected malicious code.

CIP-007-6 R3

Potential Failure Point (R3): Failure to develop a process to identify

methods in Part 3.1 that use signatures or patterns.

Potential Failure Point (R3): Failure to develop a process to update the

signatures or patterns.

CIP-007-6 R3

Potential Failure Point (R3): Failure to develop a procedure that shows

how to address testing and installation of signatures or patterns.

CIP-007-6 R3

SECURITY OBJECTIVE

Awareness of access events that report on:

• Successful login attempts;

• A limit of [organization-defined number] consecutive invalid login attempts by a user during a

[organization-defined period];

• A maximum number of unsuccessful login attempts; and

• Awareness of detection of malicious code.

NIST Special Publication 800-53 (Rev. 4) SI-3(1) & AU-12

CIP-007-6 R4

Potential Failure Point (R4): Failure to develop a complete list of assets

that require a process to log relevant events.

Potential Failure Point (R4): Failure to develop a procedure or process

that defines events at the device or system level for the specified types.

CIP-007-6 R4

Potential Failure Point (R4): Failure to develop a procedure or process that outlines how the entity will capture events.

Potential Failure Point (R4): Failure to develop a procedure or process that

defines an “alert.”

CIP-007-6 R4

Potential Failure Point (R4): Failure to develop a procedure or process that

defines a “failure of event logging.”

Potential Failure Point (R4): Failure to develop a policy that requires event

log retention at the device or system level for the specified types.

CIP-007-6 R4

Potential Failure Point (R4): Failure to define a qualifying “CIP Exceptional Circumstance.”

that defines “technical feasibility.”

Potential Failure Point (R4): Failure to define a “summarization” or a

“sample.”

Potential Failure Point (R4): Failure to define an

“undetected Cyber Security Incident.”

CIP-007-6 R4

that outlines how the identification of an undetected Cyber Security

Incident is to occur.

Potential Failure Point (R4): Failure to clearly define or communicate

start and end dates used to establish a period for review of log

outside of alert monitoring.

CIP-007-6 R4

CIP-007-6 R5

SECURITY OBJECTIVE

To manage system security by specifying technical, operational, and procedural requirements that protect the

Bulk Electric System (BES) Cyber Systems against compromise that could lead to misoperation or instability

in the BES.

• Enforce authentication of the intended individuals, groups, roles, or devices.

• Disable the identifier after business use is not required.

• Review accounts for compliance with account management requirements.

• Establish a process for protection of shared or group account credentials when individuals are

removed from the group.

• Ensure information systems support individual authenticator management by capability-defined

settings and restrictions for characteristics such as minimum password length, password

composition, etc.

• Enforce a limit on consecutive invalid login attempts by user on devices.

NIST Special Publication 800-53 (Rev. 4) IA-4, IA-5, AC-2, AC-7.

Potential Failure Point (R5): Failure to develop a complete list of assets

that require application of security controls outlined in R5.

CIP-007-6 R5

Potential Failure Point (R5): Failure to establish methods to enforce

authentication of interactive user access.

CIP-007-6 R5

Potential Failure Point (R5): Failure to identify the existence and potential

uses of default or generic account types that could be used to access

devices or introduce vulnerabilities for new and existing accounts.

Potential Failure Point (R5): Failure to identify individuals with access to

shared accounts.

CIP-007-6 R5

Potential Failure Point: (R5) Failure to develop a process to identify and

inventory all known default passwords.

Potential Failure Point (R5): Failure to change default passwords.

CIP-007-6 R5

Potential Failure Point (R5): Failure to develop methods to enforce password

parameters technically or procedurally.

CIP-007-6 R5

Potential Failure Point (R5): Failure to determine technical feasibility of

password change capability.

Potential Failure Point (R5): Failure to clearly define or communicate start

and end dates used to establish a period for password changes

Potential Failure Point (R5): Failure to create a technical feasibility

exception (TFE) and have it reviewed by WECC.

CIP-007-6 R5

Potential Failure Point (R5): Failure to establish a procedure on how

lockouts should occur.

Potential Failure Point (R5): Failure to establish lockout thresholds or alert

parameters after a specified number of unsuccessful authentication

attempts.

CIP-007-6 R5

FAC-008-3

SECURITY OBJECTIVE

To ensure that Facility Ratings used in the reliable planning and operation of the Bulk Electric System (BES)

are determined based on technically sound principles. A Facility Rating is essential for the determination of

System Operating Limits.

GENERAL FAILURE POINTS

▪ Potential Failure Point: Failure to develop a process

to ensure that the Facility Ratings methodology is

developed and followed.

▪ Potential Failure Point: Failure to develop a process

to track Facility status (i.e., new, existing, modified,

re-rates) and its Ratings.

▪ Potential Failure Point: Failure to develop guidance

specifying how you shall have documentation for

determining Facility Ratings.

FAC-008-3

Potential Failure Point (R1): Failure to develop guidance specifying how you will

have documentation for determining Facility Ratings.

FAC-008-3 R1

element ownership.

FAC-008-3 R1

element connectivity.

Potential Failure Point (R1): Failure to train personnel on developed

Facility Ratings.

Potential Failure Point (R1): Failure to develop a process for

identifying the most limiting element in a Facility.

Potential Failure Point (R1): Failure to define, communicate, and apply

technically sound assumptions used in developing Ratings.

FAC-008-3 R1

Potential Failure Point (R2): Failure to develop guidance specifying

how you will document methodology for determining Facility Ratings.

FAC-008-3 R2

element ownership for solely and jointly owned Facilities.

and evaluate element connectivity.

technically sound assumptions used in developing the methodology.

FAC-008-3 R2

Potential Failure Point (R2): Failure to develop a process for

identifying the most limiting element in a Facility.

Potential Failure Point (R2): Failure to develop guidance used in the

Equipment Rating determination process.

FAC-008-3 R2

Potential Failure Point (R3): Failure to develop guidance specifying how you

will document methodology for determining Facility Ratings.

FAC-008-3 R3

Potential Failure Point (R3): Failure to develop a process to identify element

ownership for solely and jointly owned Facilities.

FAC-008-3 R3

Potential Failure Point (R3): Failure to develop a process to identify and

evaluate element connectivity.

technically sound assumptions used in developing the methodology.

FAC-008-3 R3

Potential Failure Point (R3): Failure to develop a process for identifying the

most limiting element in a Facility.

Potential Failure Point (R3): Failure to develop guidance used in the

Equipment Rating determination process.

FAC-008-3 R3

Potential Failure Point (R6): Failure to have a Facility Ratings application

strategy that includes applicable components in R1, R2, and R3.

FAC-008-3 R6

Potential Failure Point (R6): Failure to train personnel who execute and implement Facility Ratings process.

Potential Failure Point (R6): Failure to develop a process to identify the

most limiting equipment of a Facility.

Potential Failure Point (R7): Failure to develop a process to track changes to the new, modified, or re rated Facility and its Rating.

FAC-008-3 R6, R7

Potential Failure Point (R7): Failure to develop a process to manage

requests for information that you are obligated to provide.

FAC-008-3 R7, R8

Potential Failure Point (R7): Failure to develop a process to identify the

most limiting equipment of a Facility.

Potential Failure Point (R8): Failure to develop a process to track changes to

the new, modified, or re rated Facility and its Rating.

FAC-008-3 R7, R8

FAC-008-3 R8

Potential Failure Point (R8): Failure to develop a process to manage

request for information that you are obligated to provide.

Potential Failure Point (R8): Failure to develop a process to identify a

Facility with a Thermal Rating that limits the use of the Facility under the

requestor’s authority.

FAC-008-3 R8

▪ Failure Points and Guidance Questionshttps://www.wecc.org/Pages/Compliance-UnitedStates.aspx

▪ National Institute of Standards and

Technology – Framework for Improving

Critical Infrastructure Cybersecurity Core

▪ SP-800-53 Security and Privacy Controls for

Federal Information Systems and

Organizations

Resources for Good Practices

FAC-003 & CIP-010

November 19, 2019 1:00pm – 5:00pm

November 20, 2019 8:00am – 12:00pm

California ISO

250 Outcropping Way

Folsom, CARegistration Link

Next ICPG Meeting

Contact:

RAM ICE Team

ice@wecc.org

Internal Controls Practices Group - WECC Internal... · 2019. 8. 1. · Poka-Yoke. Poka-Yoke...

Documents

Poka Yoke y Kanban

GETINGE POKA-YOKE AER A SAFER WAY TO REPROCESS ENDOSCOPESic.getinge.com/.../poka-yoke/brochures/getinge-poke-yoke-1617-en.pdf · Getinge POKA-YOKE AER | 3 SAFER, FASTER, MORE VERSATIL

Poka Yoke & Mistake Proofing

5.3 poka-yoke

The Poka-Yoke System - Elsmar Cove · PDF filePoka Yoke Slide 1 The Poka-Yoke System ... operations needs feedback and action. Poka Yoke Poka Yoke ... Poke_Yoke_050601-a.ppt

Kaizen Poka Yoke

Poka Yoke Documento

EL POKA YOKE

Poka yoke and Devops

Interne Schulung »Poka Yoke« - Fachinformation | Qualität · 16 Interne Schulung »Poka Yoke« 03.04.2018 1. Was steckt hinter Poka Yoke? 2. Anwendungszeitpunkte für Poka Yoke

Sps0650en Poka Yoke

ความส าคัญของ Poka-Yoke ในกระบวนการผลิต · เครื่องมือ Poka-Yoke มีส่วนช่วยให้เราหลีกเลี่ยงของเสีย

Poka yoke aplicaciones

Poka Yoke Presentation

Poka Yoke Kopempack

Sistema Poka Yoke

Lean Manufacturing Poka Yoke

Ejemplos Poka Yoke

Poka Yoke Explicado

07 Poka-Yoke