RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE Meeting 3 - Interim Discussion 10/17/2011 Skumatz Economic Research Associates, Inc. (SERA) The Cadmus

RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE

Meeting 3 - Interim Discussion

10/17/2011

Skumatz Economic Research Associates, Inc. (SERA)The Cadmus Group

SERA

TODAY’S AGENDA 10:00 Reintroductions 10:05 Walk through draft measure life guidelines for

next “classes” of measures (discussion throughout)

Deemed (values updates) Provisional, Simplified, Custom (Plus)

11:15 Action items between now and next meeting 11:20 Other comments / discussion 11:30 Adjourn

SERA and Cadmus Group

SERA

MEASURE LIFE GUIDELINES: ISSUE OF MEASURE “CLASSES”

RTF Savings & Cost Classes Deemed

Can deem UES, cost, load shape from eval, engineering

Provisional Interim cost & UES for tech, meas,

practices where deployment is designed to provide data or lead to values

Simplified M&V No site specific needed (calculator)

Custom M&V Infrequent – “generic” protocols

needed

Initially Proposed Lifetime Classes*

Hardware (only) Hardware, influenced

by behavioral Behavioral

reclassified because strong preference from RTF & neater mapping that direction

*respectively 1, 3, 5; even classes are associated PROSPECTIVE

SERA

CLASSIFICATION APPROACH

SERA

MEASURE LIFE GUIDELINES: DEEMED ADAPTATION

Deemed measures: Based on evaluation data / and engineering estimates. Sunset date / reinstatement.

Application to Measure Lifetime Guidelines: Many utilities use tables of “deemed” values for EUL estimates (historical, statistical, consensus, combo)

Discussion: Some studies, but often just transferred from another state. Rarely include important adjustment factors (weather, business). May bias.

Before measures can be “deemed” for RTF, we need 1) a review of the sufficiency of statistical studies underlying the values and/or 2) a preponderance of similar values elsewhere – adapted for RTF climate / conditions.

Treatment: Table of EUL values. Examples: Shell; others affected by behavior, etc.?

SERA

MEASURE LIFE GUIDELINES: PROVISIONAL ADAPTATION

Provisional measures: Interim values – insufficient evaluation data or engineering estimates to deem / deployment will collect data. Used for low / medium “risk” measures, supported by preliminary evaluations or reasonable engineering estimates. Allow “booking” w/later potential true-up.

Application to Measure Lifetime Guidelines: Can be applied to EULs realm for low to medium risk measures. We recommend a specific hierarchy for assigning these provisional values.

Discussion: We assume provisional measures will be where most of the “prospective” measures fall. In addition, measures with “gaps” that are low risk will be fall here.

Treatment: Hierarchy Examples: Cooktops (if low risk, poor data) potentially; also “next

generation” measures, and potentially any measure (including behavioral) in the short run.

SERA

MEASURE LIFE GUIDELINES: SIMPLIFIED CALC ADAPTATION Simplified calculation: Calculate using site specific input assumptions

and engineering calculations or calculators. Application to Measure Lifetime Guidelines: Apply to some

measures currently (e.g. CFLs via operating hours). IF sufficient enhanced EUL studies (using the custom M&V protocols) are conducted that include exogenous variables / assumptions in the model, we may be able to expand the list of lifetimes that can be addressed in this way.

Discussion: Can be very appropriate and serve as adjustable deemed values for certain classes of measures – after the relationship of causal effects that are uncovered in estimation work can be identified (e.g. potentially climate, operating hours, business type, etc.).

Treatment: Develop updating procedures. Examples: CFLs currently; later pumps, motors, maybe refrigerators or

HVAC equipment via O&M, climate, business type – that sort of thing.

SERA

MEASURE LIFE GUIDELINES: CUSTOM MEASUREMENT

Custom measures: Can’t deem, infrequent, not meaured, not suited to “simplified” (lack data), vary in operating conditions, interacting measures, etc. Generic protocols. If a large burden is imposed, collaboration or program design modification is

recommended; and an appeals process is provided.

Application to Measure Lifetime Guidelines: SR - use these guidelines to develop values to get to “simplified” (lack causal relation). LR, the description assigned above suits.

Discussion: Use to 1) assess the quality of estimates underpinning “Deemed” & “simplified”; In longer run, have guidelines for “Custom Measure/Behavior” and extra steps for “Custom-Behavior” (data / tracking).

Treatment: Guidelines from last time as core. Examples: Behavioral, process, many until moved to “simplified” or

(some) deemed…depending on “risk” and other factors.

SERA

WHERE WE ARE

SERA

MEASURE LIFE GUIDELINES:DEEMED – WHERE WE ARE…

Additional studies – gathering more!

Regions Gaps to date

Ag Com Ind Res Oth SumAIR 1 6 6 13BLDG 5 5 1 11COOK 3 3 2 8HVAC 4 15 10 11 4 44LITE 10 25 18 6 4 63MOTOR 6 8 10 3 27PROCESS 4 6 12 22PUMP 7 7 9 2 25REFRIG 4 1 9 14SHELL 8 8MISC 4 10 6 5 3 28SUM 36 89 80 41 17 263

SERA

MEASURE LIFE GUIDELINES: “DEEMED” – WHERE WE ARE…

Residential Measures Commercial Measures

Lighting – CFL Bulbs: 6-8 years, with some recent work starting to incorporate variations based on assumptions about hours per day that the bulb operates

Hardwired fixtures – 15-20 years for interior and exterior fixtures Lamps (table or touchier) – 5-10 years for most studies,

depending on type Occupancy sensors – 10-15 years HVAC replacement – 15-25 years HVAC and water heating in Energy Star – 15-25 years Room A/C – 11-15 years Programmable thermostat – 10-12 years Whole house fans – 25 years Attic ventilation fans with thermostat controls – 19 years Duct sealing and air sealing – each 15-20 years Insulation – 20-25 years Duct insulation – 20 years Windows – 20-35 years Pipe wrap – 10-20 years Tank temperature turn down – 4-7 years Weatherization (combination measures) – 20-25 years

Lighting – CFL Bulbs – 3.4-6 years, with some recent work starting to incorporate variations based on assumption on hours per day bulb that operates in business locations

Fluorescent fixture – 11-16 years Hardwired CFL – 10-15 years HID (interior and exterior) 13-15 years Occupancy sensors – 8-15 years Daylighting dimming – 9-10 years Packaged AC/Heat Pump – 12-15 years Chillers 19-23 years Economizers – 7-15 years Programmable thermostat – 5-10 years Energy Management Systems () – 10-15

years Motors – 13-20 years.

SERA

MEASURE LIFE GUIDELINES:DEEMED – WHERE WE ARE…

Raw studies Plus data on accepted values from

multiple states and utilities (CSG plus data from interviewees)

Gaps, differences, Hierarchy to adapt…

SERA

MEASURE LIFE GUIDELINES: DEEMED –WHERE WE ARE

General guidelines process for updating DEEMED values: Review EUL studies for quality, using the Custom

guidelines. Review in context for the measure / activity / program.

Examine influence of contributing factors (exogenous values). If no contributing factors are significant, move to next step. If contributing factors are significant, consider for inclusion as a “simplified M&V” approach.

Examine suitability for RTF region, using hierarchy from Appendix A: sector, program, NW/climate, etc. Wrong sector / wrong measures custom guidelines only option (to consultants

– decide if different procedures are ok for low risk, like provisional….)

Add to “deemed” table if appropriate (or “simplified” table)

SERA

MEASURE LIFE GUIDELINES: PROVISIONAL – WHERE WE ARE

Hierarchy for assigning “Provisional” Measure Lifetimes Measures without history, sufficient obs to support custom - the

following hierarchy is recommended: Examine # of years needed to make the TRC B/C=1. Examine manufacturer’s claimed lifetime (or panel review / Delphi /

or deem a share of the value; review “class”). If > one mfg, compute average. Make adj. for sector turnover If mfg EUL claims well beyond range for TRC>1, use; if not, not cost-

effective and don’t implement. Use value as provisional estimate until “true up”, (but monitor installed

measures). True-Up requirements & frequency; and risk assessment - to be

determined Risk based on: share of 6 year plan savings, how close to B/C ratio of 1; other

factors

SERA

MEASURE LIFE GUIDELINES: SIMPLIFIED CALC - WHERE WE ARE

Guidelines process for updating SIMPLIFIED M&V values: Review EUL studies for quality, using Custom guidelines in

context of other results available for the measure / activity. If exogenous factors significant, consider for inclusion as a

“simplified M&V” approach. (no effect deemed table) Examine suitability for RTF region, using hierarchy from

Appendix A: sector, measure, NW/climate, etc. Wrong sector / wrong measures guidelines only option

Develop appropriate calculation tool based on the key contributing factor (e.g. operating hours, HDD, business type(s), etc.).

Add to “Simplified M&V” table if appropriate (or deemed table if appropriate) .

SERA

MEASURE LIFE GUIDELINES: CUSTOM – WHERE WE ARE

Reviewed last time Data, sampling, Questionnaire, screening, precision, data

coll’n, modeling (multiple, exogenous variables, context); documentation. Working on measurement frequency (if expected measure life is 1-2 years; 3-5, etc).

Key change – specific request of exogenous / behavioral Handles measure AND measure + behavioral

Include “classes” of O&M / behavioral / influential factors in model Investigate for drivers Use “overall” now; consider refinements in case of new targets, etc. Use refined levels as input into “Simplified” (depending on N, quality)…

SERA


Operating hours

Climate – HDD, CDD

Coil cleaning / filter replacement, maintenance Q’s

Business type and business turnover variables,

Behavioral / use elements (frequency, etc.)

Other – to be completed

Lighting X X (affects op hrs)

X X

HVAC X X X X Refrig X X X X Motors, Pumps

X X

Cooktop X X Air compressors

X ? X

Other… X

SERA


Revisions to address comments & behavioral only Extra steps - Data collection – No ID of “uptake” / delivery

Complex - % of pop receiving; adopted; date stopped; EUL estimate Simpler – All pop received; only some adopted…? Working on equivalence…

Hierarchy of data collection preference – behavioral Random assignment Pre/post/control Pre/post, etc.

Data needs Behavior ever adopted; frequency, quality, # HH/employees adopting Whether behavior in place / operating; what share. If discontinued…

Date of discontinuation (failure / replacement / removal) of behavior Battery bracketing discontinuation date.. Why

Confirmation of influence, demographics / operating conditions and exogenous factors…

SERA

NEXT STEPS

SERA

MEASURE LIFE GUIDELINES: NEXT TIME

Next Time: Additional elements of Risk (mostly savings

shares so far) Refined guidelines incorporating comments EUL tables / numbers – thanks for

contributions & leads!! RUL TDF

Schedule: Next call 11/8/11

SERA

CONTACT

[email protected] 303/494-1178

SERA

LAST TIME’S SLIDES (IN CASE THERE ARE QUESTIONS!)

RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE

Meeting 2 - Interim Discussion

9/23/2011

Skumatz Economic Research Associates, Inc. (SERA)The Cadmus Group

SERA

TODAY’S AGENDA 2:00 Introductions, Progress / status 2:05 Schedule / Plan review 2:10 Walk through “Classes” 2:20 Walk through Class 1 draft guidelines

& discussion issues Edits plus direction for crafting the other classes

3:10 Criteria for evaluating EULs / comments 3:15 Next Steps, leads / contacts on reports, EULs,

interviewees 3:20 Other comments / discussion 3:30 Adjourn


SERA

SCHEDULE / PLAN Another document Two key outcomes

Guidelines EUL values

Lists activities / deliverables Meetings / calls Comments?

Call 10/17(M), 10-11:30; materials 10/11 call 11/8(T), 2:30-4; materials 11/1 In-person 12/6(T), 11-1; draft review; materials 11/28 In-person 12/13 (T), RTF meeting ; materials 12/9


SERA

CLASSIFYING THE EUL WORK “Buckets” for categories of measures / issues Two uses / states of guidelines

Retrospective - Review of existing EUL study OR conducting one

Prospective – new measure, planning purposes – or won’t have an EUL…

Three measure groups Hardware / measure (CFL, HVAC, motor, etc.) Behavior influencing measure (replacing filters, temp set) Behavioral (retrocx, light cycling, water temp)

Where does software fall…? Other issues

RUL, TDF


SERA

CLASSES FOR EUL GUIDELINES


Retrospective - Existing EUL studies, or about to conduct Prospective

Hardware Measures / UESs

Guidelines Class 1: methods / best practices for assessing EULs, conducting EUL study; hierarchy

Values, as available (ranked), to be included in an appendix;

Important gaps / research priorities to be noted To be discussed – treatment of “software” – like building

controls as part of retro-commissioning

Class 2 – planning, no EULs – guidelines on what to use…

Hardware+Behavioral

Class 3 – to be developed Class 4 –

Behavioral Class 5 – to be developed Class 6 – Other issues RUL, TDF

All classes NOT equally important / full focus

SERA

GUIDELINES – CLASS 1: HARDWARE, RETROSPECTIVE Overview / reporting

Clarity on measure / end use / sector, years, authors, utility, territory

Summary of method, sample size Discussion issues:

Specific Discussion Issues: How prescriptive – less than instruction manual…?

Level of expertise to assume?


SERA

GUIDELINES – CLASS 1: DATABASE MANAGEMENT Database management

Specify data sources, how ID’d / validation Data acquisition & attrition through steps to analysis –

counts / sources / rationale Data quality checks / procedures Data not used

Specific Discussion Issues: None


SERA

GUIDELINES – CLASS 1: SAMPLING & DATA COLLECTION

Sampling & Data Collection Sampling procedures & protocols:

Frame, sampling / basis / stratification. Sampling entity – premise, measure, etc. &

justification. Compare to others; ID & justify sample size.

Survey instrument: Key questions: in place & operating,

failure/removal/replacement date (or brackets for dates); why removed; confirm install; basic demographics; exogenous variables relevant to measure. Include instrument.


SERA


Sampling & Data Collection Survey data collection:

determine / justify data collection method (phone, etc.). Pilot.

List response rates, refusal tabulations / possible refusal surveys. Tests for bias.

Statistical descriptions: Descriptive statistics for key variables used;

comparisons.

Specific Discussion Issues: Measure, premise, etc. How prescriptive?


SERA

GUIDELINES – CLASS 1: DATA ANALYSIS, CONTEXT, RESULTS

Analysis, context, results Outliers, influential points, missing data and

treatment. Screening data for inclusion in final dataset / why, how many.

Exogenous variables affecting lifetimes: hours, business types, CZ / HDD / CDD, other

Modeling: Modeling software / procedures used; present

model statistics Alternative models / distributions tested, relative

performance Omitted / included variables & justification;

addressing variation, exogenous variables Results / standard errors / confidence band.


SERA


Analysis, context and results Context and comparisons for results Precision, measurement frequency (below)

Specific Discussion Issues: Acceptable reasons to omit? Confidence requirements – 80/10? 95/5? Needs for

reliability. History. Cost. Exogenous variables – rarely used. List the ones to

be included – or types by end use? Is 50% in place and operable right definition? What

are uses? Is that appropriate value? Frequency – vary by expected life range? Vary by

business turnover? Vary by contribution, uncertainty, TRC sensitivity?

Adjustments in Measure life by business / table?


SERA

USING EXISTING EUL STUDIES / VALUES

Use EUL? UES Sector Region Quality Elig Design

Best Same Same NW Strong Same Same

Good Same Same Other Strong Same Same

OK Same Similar NW Strong Similar Similar

Acceptable (if not critical?)

Same Diff Other Strong-Med

Similar Diff

Guidelines No No No No No No

ID gaps; recommendations to fill on priority basis: Contribution to council savings Sensitivity of TRC to EUL Number of existing studies, other Values in appendix?


SERA

COUNT SO FAR – REVIEW OF EXISTING EUL STUDIES

Sectors Ag – 16 Com’l – 17 Industrial – 21 Residential – 33 Low income – 0 Other – 2

Trails not strong for accepted EULs

End Uses Lighting – 36 HVAC – 25 Motors / drives – 10 Pump – 13 Water – 7 Cooking – 6 Refrig – 6 Process – 16 Misc - 3


SERA

ASSESSING QUALITY OF EXISTING EUL STUDIES


Same guidelines as Class 1 (since only hard measures thus far…) Overview Database management Sampling & data collection Data screening / modeling & analysis / context

implications and results

Looking for high quality in sample size, data coll’n, model quality / flexibility, consideration of OTHER variables

SERA

NEXT STEPS Revise Guidelines for Class 1 per today’s comments

Content, style, length, detail – use for guiding other classes Continue expert and background interviews / reports (data

request) Working on Hardware + Behavioral and Behavioral Review of prioritization

Portfolio, sensitivity Continue on EUL values assessment Look for patterns / Help inform prospective guidelines

Expect calls for info! Next meeting Call 10/17(M), 10-11:30; materials 10/11

More classes of Guidelines (1-4)


Documents

RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE Meeting 3 - Interim Discussion 10/17/2011 Skumatz Economic Research Associates, Inc. (SERA) The Cadmus