Agenda Compliance and Certification Committee

Agenda Compliance and Certification Committee September 16, 2015 | 1:00 p.m. - 5:00 p.m. CT September 17, 2015 | 8:00 a.m. - Noon CT JW Marriott New Orleans 614 Canal Street New Orleans, LA 70130 Introductions and Chair’s Remarks NERC Antitrust Compliance Guidelines and Public Announcement

Agenda Items

1. Administrative – Secretary and Patti Metro

a. Compliance and Certification Committee (CCC) Roster [LINK]

2. Committee Business

a. Consent Agenda

i. Meeting Agenda (Approve)

ii. CCC June 2015 Meeting Minutes* – (Approve) Patti Metro [LINK]

b. Review of CCC action items * – Jennifer Flandermeyer

c. NERC Board Enterprise-wide Risk Committee report – Patti Metro

d. NERC Board and MRC Update from August 2015 meetings*– Jennifer Flandermeyer

e. 2016 Work Plan Review and Discussion * - (Approve) Jennifer Flandermeyer

f. Independent Audits*

i. Planning for Independent Audit Schedule for upcoming NERC CMEP and ORCP – Mechelle

Thomas

3. Reliability Issues Steering Committee (RISC) Update * – Terry Bilke

4. Guidance Update – Patti Metro and Steve Noess

5. CIP v5 Technical Guidance Update – Helen Nalley and Ben Engelby

6. Subcommittee Updates

a. Nominating Subcommittee * – Helen Nalley

http://www.nerc.com/comm/CCC/Documents/CCC%202015%20Roster_02_12_15.pdf

http://www.nerc.com/comm/CCC/Agenda%20Highlights%20and%20Minutes%202013/DRAFT_CCC_Minutes_June%202015%20Meeting.pdf

Compliance and Certification Committee Agenda | September 16-17, 2015 2

i. CCC Sector openings and upcoming activities

b. ERO Monitoring Subcommittee (EROMS) – Ted Hobson

i. 2014 Stakeholders Perception Survey analysis and Board reporting (Approve)

ii. Self-certifications

c. Compliance Processes and Procedures Subcommittee (CPPS) – Matt Goldberg

i. CCCPP-010 updates and Risk-Based CMEP implementation progress

o Regional Entity coordination and involvement with CCCPP-010

ii. Status and Overview of 2016 CMEP / Risk Elements

iii. RSAW coordination

o Review RSAWs that are in process for new Standards

o CCC RSAW review criteria

iv. Quality Review form for Risk-Based CMEP principles

o CCCPP-010 updates and Risk-Based CMEP implementation progress

d. Organization Registration and Certification Subcommittee (ORCS) – Keith Comeaux

i. Risk Based Registration (RBR) update

ii. Organization Certification update

7. NERC Staff Reports Including Status of CCC Work Plan Deliverables

a. Risk-based Compliance Monitoring and Enforcement (CMEP)

i. Risk-based Compliance Assurance and Enforcement update – Steven Noess

ii. Organization changes for Risk-based Compliance Assurance and Enforcement – Steven

Noess

b. NERC funding and LSE function reference – Steven Noess

c. Future Outreach events Update - Marisa Hecht

i. Fall Industry Outreach event

ii. Fall 2015 Standards and Compliance Workshop update

iii. Small Entity Internal Controls Evaluation Exercise

d. Enforcement Update – Ed Kichline

e. Transmission Company Registration Update – Terry Brinker

8. Overview of ERO Auditor Training – Completion of IRA / ICE – Steven Noess

Compliance and Certification Committee Agenda | September 16-17, 2015 3

9. Member Round Table – Patti Metro

10. Review of Action Items and CCC Work Plan Deliverables

11. Future Meeting Dates

a. December 2-3, 2015: Atlanta, GA (NERC)

b. March 1-2, 2016: Atlanta, GA (NERC)

c. June 15-16, 2016: Folsom, CA (CAISO offices)

d. September 14-15, 2016: TBD – located with other standing committees

e. November 29-30, 2016: Arlington, VA (NRECA offices)

12. Adjourn

*Background materials provided

NERC Antitrust Compliance Guidelines I. General

It is NERC’s policy and practice to obey the antitrust laws and to avoid all conduct that unreasonably restrains competition. This policy requires the avoidance of any conduct that violates, or that might appear to violate, the antitrust laws. Among other things, the antitrust laws forbid any agreement between or among competitors regarding prices, availability of service, product design, terms of sale, division of markets, allocation of customers or any other activity that unreasonably restrains competition. It is the responsibility of every NERC participant and employee who may in any way affect NERC’s compliance with the antitrust laws to carry out this commitment. Antitrust laws are complex and subject to court interpretation that can vary over time and from one court to another. The purpose of these guidelines is to alert NERC participants and employees to potential antitrust problems and to set forth policies to be followed with respect to activities that may involve antitrust considerations. In some instances, the NERC policy contained in these guidelines is stricter than the applicable antitrust laws. Any NERC participant or employee who is uncertain about the legal ramifications of a particular course of conduct or who has doubts or concerns about whether NERC’s antitrust compliance policy is implicated in any situation should consult NERC’s General Counsel immediately.

II. Prohibited Activities

Participants in NERC activities (including those of its committees and subgroups) should refrain from the following when acting in their capacity as participants in NERC activities (e.g., at NERC meetings, conference calls and in informal discussions):

Discussions involving pricing information, especially margin (profit) and internal cost information and participants’ expectations as to their future prices or internal costs.

Discussions of a participant’s marketing strategies.

Discussions regarding how customers and geographical areas are to be divided among competitors.

Discussions concerning the exclusion of competitors from markets.

Discussions concerning boycotting or group refusals to deal with competitors, vendors or suppliers.

Any other matters that do not clearly fall within these guidelines should be reviewed with NERC’s General Counsel before being discussed.

III. Activities That Are Permitted

From time to time decisions or actions of NERC (including those of its committees and subgroups) may have a negative impact on particular entities and thus in that sense adversely impact competition.

NERC Antitrust Compliance Guidelines 2

Decisions and actions by NERC (including its committees and subgroups) should only be undertaken for the purpose of promoting and maintaining the reliability and adequacy of the bulk power system. If you do not have a legitimate purpose consistent with this objective for discussing a matter, please refrain from discussing the matter during NERC meetings and in other NERC-related communications. You should also ensure that NERC procedures, including those set forth in NERC’s Certificate of Incorporation, Bylaws, and Rules of Procedure are followed in conducting NERC business. In addition, all discussions in NERC meetings and other NERC-related communications should be within the scope of the mandate for or assignment to the particular NERC committee or subgroup, as well as within the scope of the published agenda for the meeting. No decisions should be made nor any actions taken in NERC activities for the purpose of giving an industry participant or group of participants a competitive advantage over other participants. In particular, decisions with respect to setting, revising, or assessing compliance with NERC reliability standards should not be influenced by anti-competitive motivations. Subject to the foregoing restrictions, participants in NERC activities may discuss:

Reliability matters relating to the bulk power system, including operation and planning matters such as establishing or revising reliability standards, special operating procedures, operating transfer capabilities, and plans for new facilities.

Matters relating to the impact of reliability standards for the bulk power system on electricity markets, and the impact of electricity market operations on the reliability of the bulk power system.

Proposed filings or other communications with state or federal regulatory authorities or other governmental entities.

Matters relating to the internal governance, management and operation of NERC, such as nominations for vacant committee positions, budgeting and assessments, and employment matters; and procedural matters such as planning and scheduling meetings.

# Date Responsible Due Status Comments and Next Steps52 Sep-14 Data Retention Project:

o CPPS will communicate with the Standards Committee to provide information on the results and recommendations. CPPS will report on the status at the December meeting.o Adina Mineo will communicate all recommendations to appropriate parties

Hect / Stewart Sep-15 Open Compliance and Audit specific have been added to the manual. Balance with Standards group and in progress.

63 Mar-15 Scheduling a hearing training for the new members. Metro / Stasko Dec-15 Open66 Jun-15 Post the updated CCC roster on the Committee Web page after the Board meeting. Metro 15-Sep-15 CLOSED67 Jun-15 CCC will review the requirements and history related to the CCC procedural documents and proposal to

move those to the scope of the EWRC.Clay Smith 15-Jul-15 Open

68 Jun-15 Follow up on process with NERC staff on how to file a complaint with the CCC beyond the procedural documents.

Patti Metro / Charlie Berardesco

15-Sep-15 Open Anonymous nature consideration.

70 Jun-15 Follow up on Ballot Pool Clean up Process with Standards Development Group. Ryan Stewart 15-Sep-15 Open71 Jun-15 Follow up on participation of CCC on Quality Reviews with Standards Development Group. Ryan Stewart 15-Sep-15 Open CCC participation and Translation issues72 Jun-15 Follow up on NERC funding to LSE reference in RoP section 1106 and will report back at September Steve Noess 15-Sep-15 Open73 Jun-15 Follow up with NERC and the Regions to see if the action to resolve the potential registration issue related

to Transmission development.Steve Noess / Scott

Quenneville15-Sep-15 Open Note to reference TRE process

74 Jun-15 Check on availability of NRECA offices for December 2016 meeting. Patti Metro 15-Sep-15 CLOSED75 Jun-15 Items for CIP v5 guidance documents to Ben Engelby by June 26, 2015 for July 1 meeting. CCC members 26-Jun-15 CLOSED

CCC Action Item List - As of September 2015

Agenda Item 2d Compliance and Certification Committee

September 16-17, 2015

Report of August 2015 Member Representatives Committee (MRC) and Board of Trustee (BOT)

Meetings

Information For informational purposes only Background These notes are provided by CCC attendees at the aforementioned meetings. The notes are not provided to accurately represent all agenda topics in full. The North American Electric Reliability Corporation (NERC) Members Representative Committee (MRC) and Board of Trustees (BOT) convened their quarterly meetings on August 12 and 13, 2015. The following are the most significant highlights from those meetings. NERC’s Board Chairman’s Report Mr. Gorbet’s remarks focused on the relationship with Canada and improvements we continue to see within the ERO. He confirmed the departure of Doug Jaeger as Vice Chair of the BOT with Bob Clarke stepping in to fulfill Doug’s term as Vice Chair. The Board took action for succession planning and announced that the next Board Chair was confirmed as Roy Thilly. NERC’s President Report NERC’s CEO remarks focused on the 12th anniversary of the 2003 blackout and the significant accomplishments for reliability since that time. As a group, we have seen the relationship building between industry and regulatory bodies as well as between Canada and the United States to recognize improvements to reliability and now security as the emerging space for response. Our nations are bound by many things – culture, economies, boundaries and many things but translation into targeting one means targeting both to a large extent. The state of reliability report is demonstrative of natural events as well as emerging security threats have emerged as a large influencer of reliability. In addition the changing mix of resources and how those influence reliability will require active management. On the reliability time scale, Mr. Cauley sees this as a tipping point for the industry in active response to reliability. Mr. Cauley expressed appreciation for the review of the ES-ISAC and the active role for improvement to the functioning of the ISAC with endorsement for the path that the ISAC is now on to address emerging threats. Risk Analysis will be important and while we cannot mitigate all threats and fix all problems, it will be important to risk assess and understand where we should

focus based on strong foundation of data and analytics to provide sound logic for approach and response. He suggested that system analytics will be critical as there will be a regulatory push for providing data and information from analytics but as an industry we need to present the data in a way that the proprietary nature is respected. Relationships with the regulators – at the public policy level there is more engagement with the federal space related to security both physical and logical. Mr. Cauley recognized that the public policy makers are aware of the protecting the interests of the public and that there could be directives coming to us as a result. This will be part of the continuing landscape going forward. Risks are shifting to new challenges making up a broader construct of managing reliability. Given that this is the case, it should not be assumed that NERC will address all issues. NERC can be an issue spotter to help assess the risk but others may step in with solutions such as NATF, NAGF, Trade Organizations, etc. An example of that would be the Grid Assurance response to spare equipment / emergency response related to transformers. Mr. Cauley sees opportunities to move other things into the industry arena to allow focus and clarity of roles – such as the spare equipment issues and NERCnet that has already transferred. Board of Trustees Strategic Planning Discussion As a Board, appreciation was expressed for the importance of continuing and valuing the risk basis in all actions and priorities. This is for three reasons – 1) allocation of resources and to focus where we need to be, 2) understand what risks are backed by data and analysis to clarify why this is a focus and 3) allows the organization to be forward looking and proactive in response versus historical views of what has occurred. Risk-Based CMEP allowed the organization to focus on the first issue and for the most part solve the resourcing issues, but the next two are focused on the additional value add of the organization. An important part of the vision for the organization is to use the data, analysis and expertise to highlight emerging risks and allow everyone (industry) to respond to those risks. The Board and NERC management are working toward the incorporation of items 2 and 3 to represent a broader environment in which they operate to clarify how to weave that into the fabric of what the NERC vision statement should encapsulate and synchronize with the priorities going forward. Board Remuneration Study Based on a recommendation from CGHRC, the Board of Trustees responded to the Remuneration Study Findings with an acceptance of the Consultant’s report and approval of the inclusive recommendations. This included an increase in the Board compensation. This Board is considered to be a working board spending approximately 9 hours each week on NERC activities. The Board members are responsible to give up substantial financial consideration with investments and consulting in the industry to serve. Responsibilities are set up differently from typical boards due to Appeals, Standards approvals and Penalty approvals. The basis for recommendations was largely looking at major utilities and ISO / RTO board remuneration. Background materials were included in the CGHRC meeting materials and agenda package. ERO Effectiveness Surveys CCC will provide the report to the Board on the Compliance section comments in November. The CCC approach for the survey already addresses the Policy Input Comments and will look for additional comments or improvement suggestions when the report is provided in November. The Board was requesting that the Public Power Comments received in the policy input were addressed by either the CCC work or NERC staff actions. Critical Infrastructure Protection (CIP) Version 5 Resolution of Outstanding Items

Agenda materials are included for this item in the MRC meeting package. This topic received substantial discussion on very specific perceived risk issues related to version 5 with requests from MRC members for coordination with NERC to drive resolutions quickly. The time for answers has passed. The risk with no response in mounting and is concerning to industry representatives. While other regional perspectives did not disagree with the discussion, the majority of the concerns expressed were voiced from the Western Interconnect. Input for 2016-2019 ERO Enterprise Strategic Plan Budget was approved with increases to reserves and an increase to the Director compensation was approved per the previous discussion to the remuneration study. The strategic plan discussion was largely covered in comments from the Chair and CEO with additional discussion from the full Board following. CIP-014 Effective date is 10-1-15. NERC continues working with the drafting team on guidance development related to the threat and vulnerability assessments and third party reviewers. Risk Based Registration FERC approved Purchase Selling Entity and Interchange Authority removals from the compliance registry. FERC is requesting more information on removal of LSE. In addition, the threshold for the Distribution Provider was adjusted. For work on Phase II, there have been four4 technical workshops conducted. Risk criteria and profiles have been reviewed and did not identify a group of lower risk entities. NERC is spending additional time on GO / GOP and TO/TOP to see if there are lower risk opportunities. There will be a report to close out Phase II to the Board in November. Compliance Guidance Meeting materials were included on this topic in the MRC agenda package. Group of industry personnel have been working to determine the correct approach going forward. Principles identified as well as broad categories of guidance for application guidance and practice guides. This proposed solution will be posted for industry review and comment. Compliance Assurance and Enforcement Metrics Presentation materials were included in the Board of Trustees Compliance Committee agenda package. The trend in numbers of violations is down as well as the serious risk violations. The ERO is moving toward a risk informed approach to monitoring and using data analytics to supplement the monitoring approach. Additional statistics were presented including that 75% of violations have been processed via compliance exceptions. The numbers for self-logging indicated that less than three percent of entities were using this method. EPA Rules / Reliability Impact Presentation materials were included in the MRC agenda package. There was active discussion around next steps for industry and NERC related to the impact of the rules. At the end of the discussion, it was recommended that NERC should remain a technical expert in participation efforts and actively engaged in consultation for impacts to reliability and resilience. Regional Consistency Tool Presentation materials were included in the Board of Trustees Compliance Committee agenda package. The tool that is managed by EthicsPoint was announced at NERC BOT and MRC. There have been write ups in the NERC newsletter and it has been posted on all 8 RE websites. There have been presentations at the regional meetings. This tool is not intended as the complaint

or concerns line but a communication channel back to the regions for inconsistencies. The Regional Entities and EthicsPoint do not track demographics for who submits. The ERO believes that MRRE will solve some of these items. ESCC Strategic Review of ES-ISAC Materials were included in the MRC and Board of Trustees agenda packages. The White House or Executive branch is working broadly on Information Sharing Analysis Organizations with additional expectations and definition. This work includes ISACs and will add expectations to the clarity of ISAC roles and responsibilities. NERC is participating in this effort. ISACs are not just looking at Cyber nor should they focus only there. The Board and Committees reviewed the results of the strategic review of the ES-ISAC. Board resolutions were approved to accept the ES-ISAC Strategic Review recommendations and to acknowledge that the Board still has legal and fiduciary responsibility for the ES-ISAC. ESCC will form an Executive Committee to manage the ES-ISAC and coordinate with the NERC BOT. Jim Fama, EEI, will act as secretary for the ESCC Executive Committee. Board Approvals All By-laws, Charter changes, personnel changes and standards were approved.

Mechelle Thomas, Director of Internal Audit and Corporate Risk ManagementCompliance and Certification Committee MeetingSeptember 17, 2015

NERC Internal Audit Update

Mechelle Thomas, Director of Internal Audit and Corporate Risk ManagementCompliance and Certification Committee MeetingSeptember 17, 2015

Agenda Item 2f Audit of NERC’s Compliance Monitoring and Enforcement and Organization Registration and Certification Programs

RELIABILITY | ACCOUNTABILITY3

Audit Requirement:Pursuant to Section 400, Paragraph 406 and Section 500, Paragraph 506 of NERC’s Rules of Procedure

Audit Objective: • Ensure NERC’s compliance with Compliance Monitoring and Enforcement

Program (CMEP) and Organizational Registration and Certification Program (ORCP)

Audit Scope:• CMEP and ORCP activities for the time period of 2013-2015

Audit Team:• Independent Auditor (serves as audit team lead)• CCC Observers• NERC Internal Audit

NERC CMEP and ORCP Audit


Audit Timing:• Audit kick-off - December 2015• Audit field work – April 2016• Audit completion – June 2016

CCC Observers• Patti Metro, Chair• Jennifer Flandermeyer, Co-Chair• Michael DeLoach• Ben Engelby• Lisa Milanes• Rick Terrill

NERC CMEP and ORCP Audit


Audit Roles and Responsibilities

CCC• Steering

Committee

• Provide feedback on scope

• Observer

• Review and provide feedback on audit findings

Independent Auditor• Finalize scope

• Conduct audit

• Report on findings

• Status and budget reporting

NERC Internal Audit• Coordination

• Communication

• Collaboration

NERC Staff• Provide requested

information

• Remediate audit findings as outlined in action plan


Coordination with CCC and EWRC

• Agreement on scope

• Presentation of audit findings to CCC and EWRC

• Bi-monthly coordination and audit status update calls


2015 Upcoming Audit Activities

• Audit Training

• Planning and Scoping

• Audit Team

• RFP and Independent Auditor Selection

• Formal audit kick-off


NERC RISC Update The NERC Reliability Issues Steering Committee (RISC) hosted NERC’s 2015 Reliability Leadership Summit

on August 25 in Washington, DC. Speakers included multiple utility CEOs and senior executives as well

as:

Colette Honorable, Commissioner – FERC

Philip Moeller, Commissioner – FERC

Patricia Hoffman, Assistant Secretary, Office of Electricity Delivery and Energy Reliability – DOE

The general risk areas discussed at the Summit were similar to those identified last year and are

reflected in the attached “heat map”. The Summit dialogue will be used to update the reliability risk

profiles developed by the RISC, which will then be presented to the NERC Board of Trustees at its

November 5, 2015 meeting in Atlanta. The report will be posted on the RISC website. NERC uses this

input as part of the 3-year workplan. Salient points and major themes from the Summit include:

Transmission is the hub of reliability and the enabler of most of the benefits of modern life.

A strong, resilient network mitigates many reliability risks.

The transmission system will work harder and operate differently (direction of flows both

horizontally and vertically will change) in 10 years.

Errors in models are an unaddressed risk.

o These errors will be magnified with increased penetration of dispersed generation.

o We are approaching a point where there’s need for a full AC model of each Interconnection.

Operators need common wide-area real-time indictors to make sure the “train doesn’t derail”.

NERC and the industry need to find ways to convert data to information and ultimately to wisdom.

Expected changes in generation mix don’t fit the infrastructure (gas lines and wind pockets typically

aren’t near backbone transmission or load).

The industry needs to develop interconnection standards for the aggregation of dispersed

generation (visibility, dispatchability, operator tools, forecasting).

o Differing standards between transmission and distribution interconnections will cause problems.

o This is likely something FERC will address.

The industry needs a better understanding of inverter technology.

Gas transmission will ultimately become the largest single contingency for the electric system.

o Reliance on fuel switching along with non-firm gas is a risk and some consideration needs to be

given to the probability of failures to start on the backup fuel supply during very cold weather.

o Loss of a major gas line during winter is a significant public safety risk.

Larger Balancing Authorities have fewer issues integrating renewables.

Seams remain an area of focus for FERC.

How do we better detect or predict the next new risk?

How can NERC work better with partners (vendors, forums, etc.)?

How to we keep the grid in a more resilient state (prepare for or recover from high-impact, low-

probability events)?

The industry relies on suppliers are from around the world and we need to work together to ensure

new equipment doesn’t require Technical Feasibility Exceptions (TFEs) out of the box.

http://www.nerc.com/comm/RISC/Related%20Files%20DL/Forms/Default.aspx

NERC RISC Update

There were several issues raised on the Electricity Sector Information Sharing and Analysis Center

(ES-ISAC):

o Due to interdependencies, there is a need for increased coordination with both the gas and

telecom industries.

o There is a dearth of real-time incident information flowing up to the ES-ISAC.

o The ES-ISAC needs to develop “standard products” of summary information for the industry.

o GridEx is an opportunity to strengthen the ISAC.

o Typically it takes days to confirm whether a system event was equipment failure or an attack

(theft, vandalism, intentional attack).

With regard to physical security, the goal should be to “devalue the target”. Planners should give

consideration to the impact of a physical attack when considering construction of very large

substations.

The grid is being used interactively with more things being controlled by computers over the

internet. The industry might want to move to a separate communication infrastructure for BES

operation (or at least have core functions on the separate communications network).

NERC uses a global approach to risk, yet each Region has its own set of key risks that should be the

focus of a given utility.

ES-ISAC needs to coordinate closely with both the telecom and gas ISACs.

NERC | ERO Compliance Metrics, Risk, and Reliability| December 2014 i

ERO Compliance Metrics, Risk, and Reliability

NERC CCC and RISC Recommendations to the NERC Board of Trustees

NERC | ERO Compliance Metrics, Risk, and Reliability | March 2015 ii

Table of Contents

Preface ....................................................................................................................................................................... iii

Executive Summary ................................................................................................................................................... iv

Chapter 1 – Previous Global Metrics Efforts and Limitations ....................................................................................1

Background .............................................................................................................................................................1

Chapter 2 – Proposed Metrics ....................................................................................................................................3

Introduction ............................................................................................................................................................3

Caveats ....................................................................................................................................................................3

CP-1 (Risk) Metric ...................................................................................................................................................3

CP-2 (Impact) Metric ...............................................................................................................................................5

Chapter 3 – Recommendations ............................................................................................................................... 11

NERC and Regions ................................................................................................................................................ 11

Registered Entities ............................................................................................................................................... 11

Appendix 1 – Leveraging Existing Compliance Metrics ........................................................................................... 12

Appendix 2 – Team Members ................................................................................................................................. 13

Appendix 3 – Definitions ......................................................................................................................................... 14

NERC | ERO Compliance Metrics, Risk, and Reliability | March 2015 iii

Preface

The North American Electric Reliability Corporation (NERC) is a not-for-profit international regulatory authority whose mission is to ensure the reliability of the bulk power system (BPS) in North America. NERC develops and enforces Reliability Standards; annually assesses seasonal and long‐term reliability; monitors the BPS through system awareness; and educates, trains, and certifies industry personnel. NERC’s area of responsibility spans the continental United States, Canada, and the northern portion of Baja California, Mexico. NERC is the electric reliability organization (ERO) for North America, subject to oversight by the Federal Energy Regulatory Commission (FERC) and governmental authorities in Canada. NERC’s jurisdiction includes users, owners, and operators of the BPS, which serves more than 334 million people. The North American BPS is divided into several assessment areas within the eight Regional Entity (RE) boundaries, as shown in the map and corresponding table below.

The Reliability Issues Steering Committee (RISC) is an advisory committee that triages and provides front-end, high-level leadership for issues of strategic importance to BPS reliability. The RISC offers stakeholder leadership engagement and input on issues that impact BPS reliability. The committee also advises the NERC Board of Trustees (Board), NERC standing committees, NERC staff, regulators, Regional Entities, and industry stakeholders to establish a common understanding of the scope, priority, and goals for the development of solutions to address these issues, including the use of solutions other than the development of new or revised Reliability Standards. In doing so, the RISC provides a framework for steering, developing, formalizing, and organizing recommendations to help NERC and the industry effectively focus their resources on the critical issues needed to best improve the reliability of the BPS. This report documents the result of the RISC’s continued work to define risks to the reliable operation of the BPS and provide guidance to the Board on activities that NERC should take to manage those risks.

FRCC Florida Reliability Coordinating Council

MRO Midwest Reliability Organization

NPCC Northeast Power Coordinating Council

RF ReliabilityFirst

SERC SERC Reliability Corporation

SPP-RE Southwest Power Pool Regional Entity

TRE Texas Reliability Entity

WECC Western Electricity Coordinating Council

NERC | ERO Reliability Risk Priorities | July 2015 iv

Executive Summary

This report outlines the work performed by the NERC Compliance and Certification Committee (CCC) and others based on a request by the RISC in June 2014 to identify ways compliance data can inform actions to reduce the risk to the bulk power system (BPS). This is based on a NERC Board request of the RISC in February 2013:

FURTHER RESOLVED, the Board hereby directs NERC management to work with the RISC and, as appropriate, NERC committee leadership to consider how NERC should utilize a data-driven reliability strategy development process that integrates with budget development and overall ERO planning (e.g., Standing Committee planning, department and employee goal setting).

The team that was formed to support the RISC request of the CCC focused on the following objectives:

Identify one or two high-level metrics to track BPS risk due to compliance violations.

Utilize these global metrics as well as currently available granular compliance metrics to reduce risk by:

identifying with some confidence which requirements, when violated, pose greater risk to reliability;

encouraging timely mitigation of Possible Violations (PVs) and sharing of lessons learned (mitigation and controls);

fostering a culture of self-inspection, self-correction, and self-reporting;

modeling after other industries focused on “lessons learned,” such as those whose foundation relies on safety triage; and

tailoring compliance monitoring and enforcement effort (both industry and the ERO enterprise) based on risk.

The team developed two compliance process (CP) metrics: CP-1 (Risk Focus) and CP-2 (Impact Focus). These metrics could offer significant value in achieving the objectives above. Detailed recommendations are found in Chapter 3. The proposed metrics, in conjunction with the self-correction model presented in the report, should prove effective in helping entities prioritize efforts to find and fix small issues before they lead to larger reliability disturbances or problems.

NERC | ERO Reliability Risk Priorities | July 2015 1

Chapter 1 – Previous Global Metrics Efforts and Limitations

Background From 2010 to 2012, there were two prior attempts at creating global compliance metrics focused on compliance violations and the associated risk to reliability: the Key Compliance Monitoring Index (KCMI) and the ALR CP-1. Both metrics had some value and were a useful start, but each had limitations. Both projects built on the idea of an intersection of compliance, performance, and reliability. The concept of that intersection is illustrated in Figure 1 below.

Figure 1: Conceptual Diagram of Risk Indices

Key Compliance Monitoring Index (KCMI) The first attempt at a global compliance metric was KCMI (originally named Standards-Driven Index, or SDI). This metric, meant to serve as a single performance measure on the state of the BPS, was based on a set of 26 high Violation Risk Factor (VRF) requirements that a team of subject matter experts deemed to be most important to reliability. The set of requirements were thought of as a Dow Jones average of important requirements. To measure the ongoing risk to the BES, the KCMI tracked the number of unmitigated KCMI-requirement violations in the ERO’s queue. The limitations of KCMI were:

The 26 requirements selected as “bellwether” requirements were largely based on judgment.

There was no feedback mechanism defined to retire less-impactful requirements and replace them with new, more consequential bellwether requirements.

The KCMI set did not include CIP requirements, which raised questions of a potential shortcoming.

By tracking unmitigated violations, the new observations in a recent quarter were masked by those already in queue. The metric was overstated by old requirements that were nearly fully mitigated but had not progressed through the administrative process of closing out the mitigation plan.

Chapter 1 – Previous Global Metrics Efforts and Limitations

NERC | ERO Compliance Metrics, Risk, and Reliability | March 2015 2

In short, the metric was inflated, was not sensitive to recent changes in recent violations, and did not vary much quarter to quarter due to the administrative process.

ALR CP-1 (Original) The other previously tested metric was called ALR CP-1. This metric was proposed in 2013 but never accepted by the Planning Committee or Operating Committee for use in evaluating the impact of registered entity compliance on reliability. The name was intended to conform to other NERC Adequate Level of Reliability (ALR) metrics. Rather than relying on a set of defined higher-risk requirements, this metric relied on the judgment of Enforcement staff as indicated in risk assessments filed with FERC. As a violation progresses through Enforcement, it is assigned situation-specific risk based on the facts and circumstances of the case. The most egregious violations are deemed “serious risk.” Somewhat similar to KCMI, ALR CP-1 proposed to track unmitigated serious-risk violations (as deemed by Enforcement) by quarter. There were limitations, some similar to the limitations of KCMI:

Including unmitigated violations inflated the metric due to administrative lag and also reduced the visibility of near-term changes in the number of violations.

As it can take years to close out a serious-risk violation, the approach to assess risk in the past may not reflect the current situation of the BPS.

While relying on the judgment of Enforcement staff was a useful start, the metric did not have a feedback loop. A review of the data showed significant differences between Regions on which standards had serious-risk violations. While there were likely legitimate reasons for the differences, the metric provided no mechanism to compare approaches and refine the process.

Compared to KCMI, there was also an additional administrative lag in the original ALR CP-1. KCMI was a straightforward derivation of a requirement’s VRF and Violation Severity Level (VSL). These two values are immediately known when a violation is identified. With ALR CP-1, the final risk value assessed by Enforcement was not deemed official for many months, if not years.


Chapter 2 – Proposed Metrics

Introduction The team recommends the development of two compliance metrics, focusing on the risk to and impact upon the BPS. The data to support these metrics would come from the dispositions of noncompliance each month. NERC Enforcement would calculate the metrics based on the factual descriptions and risk assessments of the noncompliance submitted by the Regional Entities. The key is to provide information of value to registered entities and the ERO Enterprise.

Caveats To avoid incorrect conclusions when reading about the suggested metrics, it is important to keep these caveats in mind:

Most violations have not resulted in BES events or disturbances.

A violation posing a serious risk to reliability may not have had an impact.

A PV that causes a state change to the BPS does not necessarily translate into a serious-risk violation.

This effort is not intended to affect or link to NERC’s Events Analysis process.

CP-1 (Risk) Metric Definition Compliance Process-1 (CP-1) is a quarterly count of PVs that Regional Entity staff determines posed a serious risk to the reliability of the BPS.

Description The team recommends using some of the concepts of the previously discussed ALR CP-1, with one modification:

1. Count violations in the quarter in which they began. NERC currently tracks serious risk violations based on the quarter in which they are filed. Under this metric, NERC will track filed serious risk violations based on the quarter in which the violations occurred. This will facilitate tracking of the trends of serious risk violations without regard to the length of time it takes to process the violations.

Benefits This metric has multiple benefits:

Requirements that most frequently lead to serious-risk violations can provide priorities for registered entities to develop internal controls.

Sharing lessons learned on the serious-risk violations (root causes, mitigation steps, and internal controls) should help registered entities reduce the number and impact of future violations of these requirements.

This metric does not rely on a static set of requirements.

The differences in the relative proportion of violations and the standards designated serious risk by the Regions may point to:



unique Regional risks that can then become part of the focus areas1 of the Regional CMEP Implementation Plans; and

opportunities for refinement and improvement of the risk assessment process.

Initial Observations Figure 2 depicts the trend in serious-risk violations.

Figure 2: Serious-Risk Violations

1 Focus Areas are standards that receive particular emphasis in a Region’s CMEP Implementation Plan.

0

5

10

15

20

25

30

35

40

45

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2

2007 2008 2009 2010 2011 2012 2013 2014 2015

Nu

mb

er o

f Se

rio

us

Vio

lati

on

s

Start Date of the Violation

Serious Risk Violations by Date Issue Occurred



Figure 3 depicts the standards and requirements that were filed at FERC as serious-risk violations in 2013 and 2014. Note that Critical Infrastructure Protection (CIP) requirements are some of the most often violated as well as those most often assigned serious risk. The requirements in the Figure 3 can provide one input for the development of internal controls.

Figure 3: Standards and Requirements with Most Occurrences of Serious-Risk Violations

This metric only considers the risk of individual violations. In some cases, however, a large number of minimal or moderate risk violations may present a serious risk to reliability, when aggregated. This metric will not reflect the risk involved with such cases.

CP-2 (Impact) Metric Definition Compliance Process-2 (CP-2) is a quarterly count of the number of noncompliance with observed reliability impact.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R10. R11.

R12. R15.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R3.

R3.

R3.

R3.

R3.

R4.

R4.

R4.

R4.

R4.

R4.

R4.

R5.

R5.

R5.

R5.

R5.

R5.

R6.

R6.

R6.

R6.

R6.

R6.

R6.

R8.

R8. R9.

0 5 10 15 20 25 30

CIP-007

IRO-005

CIP-005

TOP-002

CIP-006

TOP-004

CIP-002

PRC-005

TOP-004-2

CIP-003

CIP-006-1

TOP-006

TOP-008

CIP-007-1

COM-002

Instances of Serious Risk Noncompliance Filed Since 2010

Stan

dar

ds

Instances of Serious Risk Noncompliance by Requirement



Description While the CP-1 (Risk) metric is expected to provide value to the ERO Enterprise, it has two primary limitations:

1. While feedback loops among NERC and the Regional Entities should continue to improve the quality of the risk assessments, there is still some subjectivity in the assignment of risk.

2. Serious-risk violations are relatively rare occurrences, so their rarity provides limited opportunity for learning.

Figure 4: Final Risk Assessments (2013–2014)

Figure 4 depicts the risk breakdown of violations processed in 2013 and 2014. A small percentage of violations were deemed serious risk. Since BPS impact is a combination of probability and risk magnitude, the determination of actual reliability impact of a PV is another aspect that could provide significant benefit in trending. The team’s proposed CP-2 provides a measure of the observable BPS impact due to compliance violations. A common business approach to address a problem is to learn from similar but more mature processes and procedures in other industries. Industrial and utility safety programs provide a useful model that reduces the magnitude and frequency of serious safety mishaps by addressing the causes of more common, lower-level safety incidents. The theory behind this safety model is that major injuries are prevented by continuously finding and rooting out small issues. A parallel reliability model suggests that reducing serious impact on the BPS due to non-compliance can be achieved by addressing the causes of more frequent and more common lower-level impacts. Figure 5 depicts the alignment of the reliability model to the safety program model.



Figure 5: Safety and Reliability Models

At NERC Board meetings, FERC Commissioner Cheryl LaFleur has noted that she supports the theory of the Accident Pyramid. This theory says that major events are prevented by continuously finding and rooting out small issues. H.W. Heinrich2 developed the original concept that there is a domino effect whereby a cluster of small problems can form a chain of events that lead to major mishaps. He presented data that implies a direct relationship between the number of unsafe acts, minor accidents, and major injuries. Some of the most effective industrial safety programs are built upon the pyramid’s approach by developing a culture in which all employees have a stake in safety and are expected to continuously look for, report, and correct problems. The culture is achieved if the program is focused less on punishment and more on the correction of unsafe acts. This idea supports aggressive correction actions and intolerance of negligence. A common approach to track success in safety programs is by collecting simple observation reports. The flow of observation reports shows that people are actively looking for problems. The reports provide useful data on patterns of problems as well as a means to document corrective action. Similarly, in the electric power industry, the flow of self-reports and compliance exception log entries speaks to the maturity of the entity’s compliance culture, the attitude of self-monitoring and self-correcting, and the transparency to share lessons learned with others in the industry. To the extent that minor problems are aggressively found and corrected, there should be a decline in the more consequential mishaps higher in the pyramid. Using this approach to reduce the impact of standard violations on the BPS requires an assessment of the observable reliability impact of the noncompliance. If the guidelines for assessing impact are clear, it should not matter who records the impact determination or at what stage of the process. Accordingly, impact determinations would become part of the self-identified noncompliance process (including self-reported, self-certified, and self-logged noncompliance), performed by registered entities, as well as the evaluation of noncompliance discovered by Regional Entities.

2 Heinrich, H.W. (1931) Industrial Accident Prevention. McGraw-Hill: New York.



Figure 6: Impact Observations Mapped to the Impact Pyramid Tiers

Figure 6 maps the four data tiers that define the impacts used for CP-2. This metric only requires capturing a small amount of data along with each noncompliance. The observed impacts in the figure were used to advance the development of this report. Because of the subjectivity inherent in the definitions of observable impacts and the establishment of the tiers, it is expected the list will evolve over time based on experience.

Benefits Capturing the data (tier and type of impact) associated with the requirement for each noncompliance would offer several potential benefits:

Tracking the high-impact noncompliance occurrences using approaches proven in other industries and fields of study.

Identifying the requirements believed to be most often associated with observed impacts provides added value:

Lends to creating focus areas for CMEP Implementation Plans.

Likely input for development of internal controls and sharing of underlying causes and mitigation activities.

Providing feedback to the standards process.

Confirm that requirements slated for retirement have not had a record of impacts associated with their violation.



Requirements that are most violated yet have no impacts recorded may need adjustment of their measures or may need to be clarified to ensure they continue to address a reliability risk.

Initial Observations Figure 7 represents the occurrence dates of the violations filed in 2014 that had some observed impact on reliability. Tier 0 observations (no observed impact) are not depicted.

Figure 7: CP-2 Occurrences (2014-2015 Data)

Error! Reference source not found. shows the breakdown by requirement of the most frequently impactful violations filed in 2014 and 2015. A refined list of impactful requirements would be useful for registered entities seeking guidance on where to focus their internal controls program.

0

5

10

15

20

25

30

35

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

2009 2010 2011 2012 2013 2014 2015

Inst

ance

s o

f N

on

com

plia

nce

wit

h Im

pac

t

Impacts Filed Since 2014

Tier 1

Tier 2

Tier 3

4 per. Mov. Avg. (Tier 1)





Figure 8: Most Frequently Filed Standards and Requirements (2014-2015 Data)

0

1

2

3

4

R1 R12 R15 R4 R5 R6 R8 R9 R1 R2 R3 R4 R10 R11 R4 R5 R6 R1 R2 R5 R1 R2

IRO-005 TOP-004 TOP-002 TOP-006 COM-002

Inst

ance

s o

f Fi

led

Vio

lati

on

s w

ith

Imp

act

Most Frequently Filed Standards and Requirements with Impact


Chapter 3 – Recommendations

The team offers the following recommendations for achieving the objectives of the RISC’s request as well as additional benefits.

NERC and Regions

1. Move forward with proof of concept of CP-1 and CP-2.

a. Work with NERC CCC and PAS to refine the definitions of impact annually and consider proposed changes.

b. Create quarterly trends of CP-1 and CP-2.

2. NERC Enforcement and Reliability Risk Management staff work together to develop and share common-cause information (patterns) for impactful (CP-2) violations as well as root-cause information on violations that caused or contributed to system disturbances.

3. NERC and Regions work with the CCC to extract and post case-note-type data from self-reports, logs, and mitigation plans for the Top 20 lists (most violated, high impact, serious risk).

4. Use the CP-1 and CP-2 metrics as input to the CMEP’s Risk Elements and Focus Areas.

5. Several of the team members had not seen the presently published granular metrics. Raise the visibility of the compliance trends information by including them in the State of Reliability Report.

6. NERC and the Regions should periodically review differences among Regions’ serious-risk violations as an input to developing Risk Elements, identifying Regional specific risks, and increasing consistency in the risk assessment process.

7. Establish goals and approaches to encourage a culture of self-detection, self-correction, and self-reporting. Develop metrics to track this as well as the lessons learned from near misses and small disturbances.

Registered Entities

1. Review the Top 20 lists as possible starting points for the development of your standard and requirement level controls:

a. Most violated list.

b. Serious-risk (CP-1) requirements.

c. Impactful requirements (CP-2).

2. Aggressively self-inspect and self-correct.

3. Capture underlying causes, observed impact, and actions taken to correct noncompliance.


Appendix 1 – Leveraging Existing Compliance Metrics

NERC publishes quarterly enforcement-related metrics, several of which can be leveraged to evaluate the compliance maturity of industry and also to meet the goals of RISC’s request of the CCC:

Encouraging timely mitigation of PVs and sharing of lessons learned (mitigation and controls).

Fostering a culture of self-inspection, self-correction, and self-reporting.

Tailoring compliance monitoring and Enforcement effort (both industry and NERC) commensurate with the risk of the infraction.

Additionally, the team recommends adding Top 20 most-violated requirement graphs from both a risk (CP-1) and impact (CP-2) perspective to NERC’s quarterly compliance statistics.

http://www.nerc.com/pa/comp/CE/Pages/Compliance-Violation-Statistics.aspx


Appendix 2 – Team Members

• Aaron Hornick (NERC)

• Barb Kedrowski (NERC CCC)

• Ed Kichline (NERC)

• Farzaneh Tafreshi (NERC)

• Gizelle Wray (NERC)

• Heide Caswell (NERC PAS)

• Howard Gugel (NERC)

• James Stanton (NERC CCC)

• Margaret Pate (NERC)

• Matthew Varghese (NERC)

• Melinda Montgomery (NERC PAS)

• Michael DeLoach (NERC CCC)

• Paul Kure (NERC PAS, RF)

• Peter Raia (NERC)

• Stanley Kopman (NPCC)

• Terry Bilke (NERC CCC)


Appendix 3 – Definitions

Compliance Process – 1 (CP-1): A quarterly count of PVs determined by Enforcement staff to pose a serious risk to the reliability of the BPS. Compliance Process – 2 (CP-2): A quarterly count of the number of Compliance Exceptions or PVs with observed reliability impact. Major BES Disturbance: An event that results in Bulk Electric System instability or Cascading.

Compliance Metrics, Risk, and Reliability

Request of the CCC by the NERC Reliability Issues

Steering Committee (RISC)

September 2015

2 RELIABILITY | ACCOUNTABILITY

Agenda / Objectives

• RISC request of the NERC CCC

• Metrics and potential benefits

• Observations

• Recommendations


RISC Request

• NERC RISC asked the CCC for input on managing reliability risk by leveraging compliance data

• CCC-led team developed 2 high level metrics CP-1 (# of Serious Risk Violations by quarter of occurrence)

CP-2 (# of “Impactful” Violations by quarter of occurrence)

• Benefits Know with some confidence which requirements pose greater risk to

reliability

Enhanced risk-focused monitoring and enforcement

Enables risk reduction using approaches proven in other industries and fields of study


CP-1 (Serious Risk Violations by Quarter)

0

5

10

15

20

25

30

35

40

45

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2

2007 2008 2009 2010 2011 2012 2013 2014 2015

Nu

mb

er

of

Seri

ou

s V

iola

tio

ns

Start Date of the Violation

About 2% of the

1000+ violations

processed

annually are

deemed Serious

Risk


Top Serious Risk Violations

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R1.

R10. R11.

R12. R15.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R2.

R3.

R3.

R3.

R3.

R3.

R4.

R4.

R4.

R4.

R4.

R4.

R4.

R5.

R5.

R5.

R5.

R5.

R5.

R6.

R6.

R6.

R6.

R6.

R6.

R6.

R8.

R8. R9.

0 5 10 15 20 25 30

CIP-007

IRO-005

CIP-005

TOP-002

CIP-006

TOP-004

CIP-002

PRC-005

TOP-004-2

CIP-003

CIP-006-1

TOP-006

TOP-008

CIP-007-1

COM-002

Instances of Serious Risk Noncompliance Filed Since 2010

Sta

ndard

s

Instances of Serious Risk Noncompliance by Requirement


CP-2 Foundation

At NERC Board meetings, FERC Commissioner LaFleur notes she believes in the theory of the Accident Pyramid (larger events are prevented by continuously finding and rooting out small problems)

We can build metrics and a risk management approach that mirrors effective utility safety programs

Fatalities

Injuries

Near Misses

Unsafe Acts

Safety Model Compliance Parallel

Major BES Disturbances

Moderate Impact

Minor Impact

No Impact PVs


ALR CP-2 Data Collection

Find and Fix

these

To reduce the #

and magnitude

of these


Impactful PVs 2014 to Present

Note: There were no Tier 3 observations filed in 2014

0

5

10

15

20

25

30

35

Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4

2009 2010 2011 2012 2013 2014 2015

Inst

ance

s o

f N

on

com

plia

nce

wit

h Im

pac

t Impacts Filed Since 2014

Tier 1

Tier 2

Tier 3





Impactful Requirements

0

1

2

3

4

R1 R12 R15 R4 R5 R6 R8 R9 R1 R2 R3 R4 R10 R11 R4 R5 R6 R1 R2 R5 R1 R2

IRO-005 TOP-004 TOP-002 TOP-006 COM-002

Insta

nces o

f F

iled

Vio

lati

on

s w

ith

Im

pact

Most Frequently Filed Standards and Requirements with Impact


Recommendations (NERC)

• Establish the CP-1 and CP-2 data streams and associated “Top 20” lists and share root causes and lessons learned

• Use the CP-1 and CP-2 data as input to the CMEP’s Risk Elements and Focus Areas

• Use data to periodically improve the quality of Enforcement’s assessment of risk

• Establish goals and approaches to encourage a culture of self-detection, self-correction, and self-reporting

• Track progress over time


Recommendations (Registered Entities)

• Consider the “Top 20” lists as a starting point for the development of internal controls

Most violated Requirements

Serious Risk (CP-1) Requirements

Impactful Requirements (CP-2)

• Pursue logging authority and aggressively self-inspect and self-correct

• Capture underlying causes and actions taken to correct compliance exceptions

12 RELIABILITY | ACCOUNTABILITY RELIABILITY | ACCOUNTABILITY

Thanks to Team Members

• Aaron Hornick (NERC)

• Barb Kedrowski (NERC CCC)

• Chris Sweeney (NERC)

• Ed Kichline (NERC)

• Farzaneh Tafreshi (NERC)

• Gizelle Babik (NERC)

• Heide Caswell (NERC PAS)

• Howard Gugel (NERC)

• James Stanton (NERC CCC)

• Margaret Pate (NERC)

• Matthew Varghese (NERC)

• Melinda Montgomery (NERC PAS)

• Michael DeLoach (NERC CCC)

• Paul Kure (NERC PAS)

• Peter Raia (NERC)

• Stanley Kopman (NPCC)

• Terry Bilke (NERC CCC)


Questions ?

CIP Version 5Technical Guidance Update

Helen Nalley and Ben EngelbyCCC MeetingSeptember 16, 2015


• July 1 “Way Forward” Meeting Recap Memoranda Discussion and Outcomes

• CIP Version 5 Transition Advisory Group (VTAG) Activities Guidance (FAQs and Lessons Learned) Standard Revisions

Agenda


CIP Version 5 “Way Forward” Meeting Recap

Ben Engelby, ACES


• Open discussion on issues regarding NERC Memoranda • Assess reliability risks that CIP standards are meant to address • Address CIP V5 issues to support a consistent approach • Identify ways forward to support long-term solutions, including: Guidance development Standards development or interpretation processes Other solutions

July 1 Way Forward Meeting


• NERC Staff Gerry Cauley Mark Lauby Val Agnew Steven Noess Tobias Whitney Scott Mix Shamai Elstein

• Regional Entity Staff Ron Ciesiel (SPP)

• FERC Staff David DeFalaise

• CIP VTAG1

Helen Nalley (Southern) Philip Huff (AECC) Maggie Powell (Exelon)

• CIP V5 SDT Members Christine Hasha (ERCOT) Steve Brain (Dominion)

• Industry Stakeholders Lou Oberski (EEI) – Trades Barry Lawson (NRECA) – Trades Nathan Mitchell (APPA) – Trades Ben Engelby (ACES) – CCC Patti Metro (NRECA) – CCC Brian Murphy (FPL) – SC Chuck Abell (Ameren) – CIPC

Meeting Attendees

1VTAG Roster: http://www.nerc.com/pa/CI/Documents/V5TAG_Roster_070815.pdf

http://www.nerc.com/pa/CI/Documents/V5TAG_Roster_070815.pdf


• Programmable Electronic Device (PED)• Control Centers and Functional Obligations• Network Devices BES Cyber Systems• External Routable Connectivity (ERC) • Impact Rating Criteria 2.3 and 2.6• Generation Interconnection & Exempting non-Routable

Communications

Memoranda


• Background Guidance on meaning of “programmable electronic device,” as used in the

glossary term “Cyber Asset” “Any device that is electronic and capable of executing a set of

instructions”

• Issues Guidance expands the scope of devices Timing unreasonable to implement Unintended consequences

• Outcome: Withdraw Memo Additional guidance being developed Consider revising standard to define programmable electronic device

Programmable Electronic Device


• Background “[B]ased on the plain language…the definition of the term “Control Center”

…focus on the reliability tasks and functional obligations performed by a responsible entity, not on the functional registration of the responsible entity.”

“Any functional entity may own and operate a Control Center with BES Cyber Systems subject to the CIP Reliability Standards depending on the reliability tasks performed by its personnel.”

• Issues Unintended consequences that result in overly-broad application Standards need to be commensurate with the risks posed to the BES

• Outcome: Withdraw Memo Additional guidance being developed

Control Centers


• Background “Entities cannot categorically exclude network devices from the definition

of BES Cyber Asset (BCA)” “The exemption from applicability for ‘Cyber Assets associated with

communication networks and data communication links between discrete Electronic Security Perimeters’ does not apply to network devices with routable connectivity”

• Issue Creates “hall of mirrors” by requiring a firewall to protect existing firewall

• Outcome: Withdraw Memo Guidance with a range of acceptable approaches Consider revising standard to define network devices

Categorization of Network Devices


• Background “Routable connectivity requirements in the CIP version 5 standards apply

to natively serial-based BCAs modified to be externally accessible via a routable network.”

• Issues How to define associated Electronic Security Perimeter (ESP) with serial

device Not clear how to implement “and associated ESP” with serial devices Uncertainty to deal with ESP when there are long distances, such as where

a serial device may be communicating to a control center 500 miles away

• Outcome: Withdraw Memo Guidance including acceptable approaches for associated ESP Consider revising standard to address issue

External Routable Connectivity (ERC)


• Background “Criterion 2.3 apply to generation Facilities…not to specific systems or

subsystems within a generation Facility; nor do they apply to a particular set of BES Cyber Systems to be protected.”

“Criterion 2.6 designations apply to ‘[g]eneration at a single plant location or Transmission Facilities at a single station or substation location,’ not to specific systems or subsystems at those locations.”

• Issues Criterion 2.3 timing issues for this calendar year for must run generators Criterion 2.6 AVR status versus generator

• Outcome: Withdraw Memo Guidance on how to issue designations and timelines for implementation Convene group of RCs, PCs, TPs to get consistent designation outcomes

Impact Rating Criteria 2.3 and 2.6


• Background “[W]here a generator lead line is radial with no network flows and has the

sole purpose of connecting generator output to a networked Transmission system, the line would not qualify as a Transmission line to be included in the Criterion 2.5 calculation.”

• Issue Guidance should not be in the format of a memo

• Outcome: Withdraw Memo Convert memo into a FAQ

Generation Interconnection


NERC Version 5 Transition Advisory Group (VTAG)Activities to Support Implementation

Helen Nalley, Southern Company


• FAQ for Generator Interconnection• Compliance dates for newly identified CIP assets• Compliance dates for unplanned changes• Patch management • Shared BES Cyber Assets and common mode vulnerability

Guidance


• Applicability for Control Centers • External Routable Connectivity for serial devices • Glossary Terms Programmable electronic device Network devices

Standard Revisions


• Supply chain• Protecting communication links between Control Centers• Existing remote access controls• Protections for Transient Cyber Devices for Cyber Assets

associated with Low Impact Assets• Clarity of LERC

FERC NOPR


• CIP V5 Transition Webinars scheduled September 10 September 24

CIP Activities


CPPS Report

September 2015

CCCPP-010 updates and Risk-Based CMEP Implementation

• Background: CCC responsible developing Criteria (CCCPP-010-3) for ERO to use in assessing effectiveness of Regional Entities’ Compliance Monitoring and Enforcement Programs.

• Objective: review, and update if appropriate, Criteria for 2016 Implementation Year

• Next Steps: December 2015 Meeting: Share substantive feedback on the use/value of the Criteria

Status and Overview of 2016 CMEP

• NERC issued its piece last week. Regional Entity pieces to come out in November.

• Overview of evolution of Risk Elements and meaning of areas of focus

• Update of some Risk-Based Compliance Monitoring Framework Documents

• Periodic reviews between NERC and CPPS

RSAW Coordination and Quality Review

• Reviewed CPPS experience with reviewing RSAWs (and recent participation in COM-001-3 Quality Review)– CMEP Definition of “Guided Self-Certification”– Relationship between “Guided Self-Certifications” and RSAWs– NERC oversight of Guided Self-Certification forms/format

• CPPS Members’ participation in Quality Review focused on providing feedback on compliance feasibility of some requirements/measurements language– CPPS will look to provide “general” feedback to NERC Standards

Development personnel – Coordinate activity with Action Item #71?

Work Plan Review

• CCCPP-010-3• Experiences under Risk-Based Compliance

Monitoring and Enforcement Programs– Feedback to ERO– Lessons Learned, – Guidance, Bulletins, – Q&A documents, etc…

2016 ERO Enterprise CMEP Implementation Plan (IP)Steven Noess, Director Compliance AssuranceCCC, CPPSSeptember 16, 2015


• Purpose Annual operating plan for NERC and Regional Entities (REs) Implementation of risk-based approach for CMEP activities

• Followed process outlined in 2015 Risk Elements Guide, similar to creation of 2015 IP

• NERC release on or about September 1 of preceding year REs submit Regional IPs on or about October 1 NERC reviews and posts revised IP in November to include Regional IPs

• Updates occur throughout implementation year, as needed

IP Overview


• NERC IP provides details on: ERO Enterprise’s Risk-based Compliance Oversight Framework Prioritized list of Enterprise-wide risk focus areas o Map to associated Reliability Standardso Do not include all potential risks to BPSo REs consider local risks and circumstances within regional footprint

Guidance on Regional Risk Assessments Enforcement activities

IP Content Areas


• Refined, not recreate, the nine 2015 risk elements

2016 Risk Elements

Comparison of 2015 and 2016 Risk Elements2015 Risk Elements 2016 Risk Elements

Cybersecurity Critical Infrastructure ProtectionExtreme Physical Events Extreme Physical Events

Infrastructure Maintenance Maintenance and Management of BPS Assets

Monitoring and Situational Awareness Monitoring and Situational AwarenessProtection System Misoperations

Protection System FailuresUncoordinated Protection Systems

Long-Term Planning and System AnalysisEvent Response/Recovery

Planning and System AnalysisHuman Error Human Performance

Workforce Capability (N/A for 2016)


• Eight 2016 risk elements intend to focus and prioritize compliance monitoring efforts Not a comprehensive list of all risks to the reliability of the BPS Inherent Risk Assessments consider multiple factors, including ERO and

Regional risk elements, to focus monitoring

• Particular areas of focus under a risk element does not imply that: The identified NERC standard(s) fully addresses the particular risk

associated with the risk element The identified NERC Standard(s) is only related to that specific risk

element; All requirements of a NERC standard apply to that risk element equally.

Key Takeaways


• 2016 ERO Enterprise CMEP IP located on NERC website at under Compliance Resource Documents at: http://www.nerc.com/pa/comp/Resources/ResourcesDL/2016%20CMEP%20IP_V_1_09102015_Posted.pdf

Resources

http://www.nerc.com/pa/comp/Resources/ResourcesDL/2016%20CMEP%20IP_V_1_09102015_Posted.pdf


Enforcement Update

Ed Kichline, Senior Counsel and Associate Director, EnforcementCompliance and Certification CommitteeSeptember 17, 2015


Noncompliance Totals Continue to Decline


Noncompliance is Self-Identified


Self-Logging Compliance Exceptions and Find, Fix, Track, and Report

• Common procedure available on NERC.com

• 39 registered entities participating

• Self-Logging entities cover all functions

• 75% of all minimal risk noncompliance processed as Compliance Exceptions

• Compliance Exception is the overall leading disposition method

• Find, Fix, Track, and Report used primarily for moderate risk issues

Risk-based Enforcement
