1

Ex Ante Review Overview

Energy Division Staff and ContractorsEnergy Efficiency Industrial/Agricultural Programs

and Portfolio ForecastingCalifornia Public Utilities Commission

September 11, 2014

Ex Ante Review Overview

• Speaker:– Jaclyn Marks– Supervisor, Energy Efficiency

Industrial/Agricultural Programs and Portfolio Forecasting

– California Public Utilities Commission, Energy Division

2

Overview of Presentation and Topics Purpose: Information Exchange, Q & A encouraged

1. Overview of CPUC oversight of its authorized EE activities

2. Regulatory background on ex ante review process (emphasis on Custom projects)

3. Commission staff’s expectations when performing EAR

3

CPUC Oversight of IOU/CCA/REN Energy Efficiency Programs

• Speaker:– Jaclyn Marks

4

CPUC Organization

5

CPUC Organization• President and four Commissioners

– Governor appoints and Legislature confirms, are independent “Decision Makers”

• Administrative Law Judges – Hired as state employees but operate independently as

“Decision Makers”

• Division Management– Hired as state employees – Supervisors report to Program

Managers who report to Division Director who reports to Executive Director

• Commission staff – Hired as state employees and report to a supervisor for the area

they oversee6

Energy Efficiency Organizational Chart

7

CPUC Regulatory Process

• Rulemaking Proceedings– The main regulatory process within which the CPUC authorizes

and oversees the EE activities – this is a “legal” process whereby Decisions and Rulings are issued based on a legal and policy analysis of documents placed into the record

– Commission Decisions and Resolutions plus Decision Maker Rulings are the main vehicles for providing direction and making or clarifying/adjusting policy

– Record is established by Decision Maker rulings and official proceeding meetings and hearings

8

CPUC Regulatory Process

• Proceedings (continued)– Commission creates a proceeding via a Decision

• President assigns proceeding to a Commissioner • Chief ALJ assigns one or more ALJs to the Proceeding

– Party status requested via official request filing which can be accepted or denied by Ruling or Decision

– Parties to the proceeding can have input invited into the proceeding record

• The IOUs, CCAs, and RENs submit their proposed activities via applications

9

CPUC Regulatory Process• Commission Decisions for EE guidance and authorization

– D.09.09.047 authorized 2010-2012 programs and budgets including creating the ex ante review process

• ALJ Ruling dated 11/18/2009 in A.07-08-021 created the non-DEER deemed workpaper review process

– D.11-07-030 (the “ex ante” Decision) in response to IOU PfM• Clarified many ex ante policy issues such as early retirement/dual baseline• Created the custom measure and project review process (Attachment B)

– D.12-05-015 gave guidance on 2013-2014 EE applications• Provided extensive clarifications and amplification of ex ante policy

– D.12-11-015 authorized 2013-2014 programs and budgets extended the ex ante review process unchanged

– D.11-XX-XXX expected soon will authorize 2015 portfolios

10

CPUC Oversight of EE

• Questions?

11

Regulatory Background on Ex Ante Review Process (emphasis on

Custom projects)

• Speaker:– Jeff Hirsch– James J. Hirsch & Associates– Prime Contractor – Database for Energy

Efficiency Resources (DEER) and Ex Ante Regulatory Support – CPUC 12PS5125

12

Ex Ante Contract Team

• Primary players– JJ Hirsch & Assoc.

• Hirsch, Gandhi, Madison, Reeves, Addison, Hill, Maddox, Gates, Sweek, and others

– Navigant for stakeholder interaction organization support

– Energy Metrics (Rothenberg) for custom review

– Itron for custom review

– ERS for custom review

– Other for DEER, workpapers and regulatory support

13

Ex Ante Team Activities

• DEER– Assumptions, methods and values– Updated for Codes, Standard and Regulation changes as well

as to incorporation latest evaluation results and related research

• Non-DEER workpaper review and approval– New measures and updates to existing measures– Assumptions, methods, baselines, costs, EUL/RUL, NTG

• Custom measure and project review – main topic here• Other regulatory support

14

Ex Ante Interrelations

15

Non-DEER Workpapers

Measures not in DEER

DEERCommon EE measures;Savings estimates and parameters

Custom Projects

Site-specific;Values developed at completion

Ex Ante Values

Best available information and extrapolation

DEER methodology

Sub-selection of submission

Why Ex Ante Review?

• Ratio of ex post to ex ante values needs improvement.– Gross has fallen from ~.9 in 2000 to ~.6-.7 in 2008– Free rider percent has remained and 40%-50% or increased

• Evaluations have indicated there are areas where policy is not appropriately implemented and that assumptions, methods and data utilized are not the most appropriate.

• Commission staff has an oversight responsibility to ensure adopted policy is being followed .

16

Why review before agreement with customer is signed not later?

• Cost effectiveness involves both the savings estimates as well as the costs– Improving savings estimates can result in more effective use of

incentive $

• History of reaction to ex post evaluation– Review and oversight moved into the process to assist CPUC

and PA’s in finding issues and providing guidance designed to improve results

– Provide real time oversight to accelerate improvement

17

Objectives of the ex ante review process

• Improve the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness estimates

• Foster ongoing improvements to the quality and consistency the portfolio implementers’ own internal due diligence activities relating to ex ante values

18

Due Diligence

• Due Diligence is an investigation of a business or person prior to signing a contract, or an act with a certain standard of care, the process through which a potential investor can evaluate a major planned investment for its cost, benefits and risk.

• We are all involved in the due diligence effort.

19

CPUC DECISION 11-07-030• “Ex Ante” Decision adopted by CPUC in July 2011 and re-affirmed

by two subsequent Decisions in 2012 for use in 2013 and beyond• Appendix B describes the custom project EAR process• Sets minimum project documentation requirements• Applies to Commission Staff (CS) selected and non selected

projects• Allows review prior to customer agreement so as to approve all

values (savings, incentives, life, costs, etc.) used for EE savings “claims” and cost-effectiveness calculations

• Allows reviews of non-selected projects later (correct errors and set prospective requirements)

20

CPUC DECISION 12-05-015• Amplified and added clarifications and details to direction and

policy in D.11-07-030• Clarification of project classifications (NC/NR/ER/etc.)• Details of baseline definitions and selection including

requirements for use of early retirement classification• Details use of EAR “free rider” reviews to provide guidance

and set rules to improve “net” performance• Details of TRC cost calculation and how that values can limit

incentives• Clarification on expectation for above code/ISP activities and

no “like” or “regressive” baselines

21

Process of EAR for custom projects

• CMPA-Custom Measure and Project Archive• Summary list of custom projects uploaded to CMPA, bi-

weekly basis (can be more often).• Commission Staff reviews list, selects projects- selected

projects may be pre-application stage through claims stage.

• Utility uploads project documents to the CMPA, email notification sent to various parties.

• Commission Staff dispositions are posted on the CMPA, email notification sent to various parties

22

Ex Ante Review Reality

• Commission Staff review only a small percentage of custom projects.

• PAs generally appear to put more effort into projects Commission Staff pick for review than non-selected projects.

• Because PAs appear to put additional emphasis on “picked projects”, Commission Staff reviewed projects may not be representative of the full project population.

23

Future direction of the custom EAR process

• Improving the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness rests with the PAs reviewers.

• Commission Staff will be increasing efforts to work with PA reviewers and monitor the progress of changes in the PA review/due diligence process.

• Reviewed projects may be pre-application stage through claims stage.

• Commission Staff will focus more reviewers activities to assess the area where due diligence is inadequate.

24

Questions?

25

Commission staff’s expectations when performing EAR

• Speaker:– Keith Rothenberg– Energy Metrics– Subcontractor to JJH&A for custom EAR

26

Overview

• Role of the Commission staff representative reviewer• Documentation Expectations• What we expect and what we receive • Different project phases and purposes of Commission

staff review• Questions

27

Role of the Commission staff reviewer

• Not to supplement the Utility reviewer.• Reviewing the Utility reviewers’ due diligence efforts.• We rely on the Utility reviewers to accomplish the

Commission’s goals: – Improving the reliability of the savings estimates– Ensure project compliance with CPUC policy

28

What complicates documentation requirements?

• Q. For retrofit project types, why don’t we simply calculate the energy savings directly associated with replacing the existing system/equipment and use that value for the savings impacts?

• A. In some cases we do, however CPUC Policy attempts to balance the ratepayer’s interest and the customer’s interest with the realities of the circumstances driving the project.

• Result: Policy has created considerations for free ridership, rules associated with code compliance, ISP assessments, TRC considerations, fuel switching considerations, dual baseline rules, EUL, RUL, etc.

29

Project Types, Baselines, LifeProject Type

Measure Life

Basis

(RUL)/First Period Energy Savings

Baseline

(EUL – RUL)/Second Period

Energy Savings Baseline

NC (NEW)EUL Code or Industry

Standard BaselineN/A

ROBEUL Code or Industry

Standard Baseline1

N/A

NREUL Code or Industry

Standard Baseline1

N/A

ER (RET)

RUL/

EUL-RUL

Customer Existing Baseline

Code or Industry Standard Baseline1

REARUL or EUL

Customer Existing Baseline

N/A30

Early Retirement (AKA “Program-Induced Early Retirement”)

• Requirements described in guidance document “Early Retirement Using Preponderance of Evidence”

• Preponderance of evidence supporting that the program induced the replacement.

• Approved dual baseline energy savings calculation approach.

• Full measure cost, incremental measure cost for the second baseline.

• Proposed RUL of the existing equipment supported by evidence.

• Measure EUL with justification/source.31

Examples of the expected documentation

• Project type clearly described, not inferred. • Documentation to support baseline assignment • Facility description

– Is there co-generation on site? • Project description, including relevant associated

systems.• Existing system controls and operating status.• Existing system output capacities – current output and

maximum/design capacity.• Proposed modifications with schematic as applicable.

Preliminary savings calculations and supporting data with documentation to ensure replicability.

32

Expected documentation (Cont’d)- Calculation Method

• Written calculation methodology.• Precise step-by-step equations.• Associate the proposed methodology with specific

equipment and systems.• Energy savings principle for each measure should be

discussed.• Comprehensive and complete leaving only the final

verified variables and data to be determined after project completion.

• Generic methodology is not acceptable.• Documents that require a reviewer to hunt through

spreadsheets cell by cell to investigate the calculation methodology are unacceptable.

33

Expected documentation (Cont’d)- M&V Plans

• M&V plans, reports when required.• M&V required when impacts are difficult to estimate by

calculation alone.• System diagrams with measurement points.• Concise descriptions including measurement period,

measurement interval, measurement equipment.• Measurement equipment accuracy, result uncertainty.• May be necessary to collect additional baseline data

before the project commences.• Raw data archives.

34

Expected documentation (Cont’d)

• Manufacturer’s cut sheets when used to estimate ex ante savings or when needed to ensure replicability.

• Fuel switching considerations and any required analysis. • HVAC or process interactive effects values and methods

used to develop ex ante savings values.• Interactions between multiple measures that act to

increase or decrease savings-parametric analysis.• Dual baseline analysis for ER projects.• EUL/RUL value, analysis or source.

35

Expected documentation (Cont’d)

• Pre/post production output data when used in savings calculations and the source of such records.

• Pre-installation inspection report.• Post-installation inspection report.• Other Utility review documents.• Installation report where M&V performed.• Utility or implementer program manual (a single archive

of these documents should be referenced).

36

Pre-Installation Review

• Perform a “parallel” review with the Utility – What is parallel review supposed to be?

• Provide input to the Utility on the estimated custom measure or project ex ante savings

• Set precedents for future projects

37

Post-Installation Review

• Continued opportunity for CS to review and provide input on the accuracy of ex ante values.

• Review Utility compliance with earlier project specific guidance.

• “True-up” estimates with site specific M&V data/results.• Set precedents for future projects.

38

Utility Claim Review

• Conduct a review of energy savings and cost-effectiveness parameters for custom projects included into the Utility Quarterly Claim.

• Review project files to ensure the CPUC directed archiving of documentation is being followed.

• Appropriate default realization rates were applied to ex ante gross savings estimates for projects that were not reviewed.

39

Utility Claim Review (Cont’d)

• Recommendations made by Commission staff for reviewed projects were accurately reflected in the claim.

• Guidance provided in CS reviewed projects being followed in projects that were not previously reviewed by CS.

40

Early Opinions

• Utilities may submit to Commission staff a request for an early Commission staff review or opinion on a specific issue.– Appropriate for policy grey areas.– Eligibility issues.– Other areas where there can be uncertainty regarding

policy or Program compliance issues.• This should be used when either a utility or an

implementer has concerns or is uncertain how a particular project might be handled by CS review

41

Summary of CPUC Reviewers’ Role

• Not to supplement the administrator reviewer but to ensure proper administrator due diligence is being followed.

• Ensure program rules are being followed.• Ensure CPUC policy is being followed.• Verify that utility staff and contract reviewers perform

reviews in an appropriate manner balancing ratepayer and customer interests.

• Ensure that all required documentation is in the project file whether it is selected or not for ex ante review by CPUC staff reviewers.

42

Wrap Up

• Our work plays an important role in the implementation of the CPUC authorized ratepayer funded programs.

• As reviewers, we are all engaged in a due diligence effort on behalf of the rate payers as well as the Utility customers.

• The CPUC has authorized ex ante review to improve the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness estimates.

43

Wrap Up

• Commission staff are relying on IOU reviewers to achieve this goal. We are interested in working with you.

• There are changes to the EAR process that will allow CS to assess the progress in this area: categorization of comments, tracking of comments with project phases, project types, reviewers, review firms, etc.

44

Questions??

45

Your comments on the EAR process?

• How can CS improve the process?• What works and does not work?• What issues do Utility reviewers have?

46

Standardization of required documentation?

• Would a minimum required documentation template that is required to be included for every project be helpful?

• What is included in the application review packet that comes to your desk?

47

What additional information would be useful for Utility reviewers?

48

Documents Available Online

1. Industry Standard Practice Guidance Document

2. Project basis as Early Retirement/Replace-on-burnout /Normal Replacement/New Construction/Add-on Retrofit and Remaining/Effective Useful Life, and Preponderance of evidence Guidance Document

http://www.cpuc.ca.gov/PUC/energy/Energy+Efficiency/Ex+Ante+Review+Custom+Process+Guidance+Documents.htm

3. Commission Decisions and Rulings establishing policy and processes.

49