Introduction to Technical Reference Manuals (TRMs) for Kentucky Energy

Efficiency ProgramsPresentation for:

Kentucky EM&V Stakeholder Meeting #2March 24, 2016

Frankfort, Kentucky

Introduction LBNLissupportedbytheU.S.DepartmentofEnergytoconductnon-classifiedresearch,operatedbytheUniversityofCalifornia

q Provides technicalassistance tostates—primarily stateenergyoffices andutilityregulatorycommissions

q Assistance isindependent andunbiased

The presentationwasfundedbytheU.S.DepartmentofEnergy’sOfficeofElectricityDeliveryandEnergyReliability-National Electricity DeliveryDivision underLawrenceBerkeley

NationalLaboratoryContractNo.DE-AC02-05CH11231.

DisclaimerThispresentationwaspreparedasanaccountofworksponsoredbytheUnitedStatesGovernment.Whilethispresentationisbelievedto

containcorrectinformation,neithertheUnitedStatesGovernmentnoranyagencythereof,norTheRegentsoftheUniversityofCalifornia,noranyoftheiremployees,makesanywarranty,expressorimplied,orassumesanylegalresponsibilityfortheaccuracy,

completeness,orusefulnessofanyinformation,apparatus,product,orprocessdisclosed,orrepresentsthatitsusewouldnotinfringeprivatelyownedrights.Referencehereintoanyspecificcommercialproduct,process,orservicebyitstradename,trademark,

manufacturer,orotherwise,doesnotnecessarilyconstituteorimplyitsendorsement,recommendation,orfavoringbytheUnitedStatesGovernmentoranyagencythereof,orTheRegentsoftheUniversityofCalifornia.TheviewsandopinionsofauthorsexpressedhereindonotnecessarilystateorreflectthoseoftheUnitedStatesGovernmentoranyagencythereof,orTheRegentsoftheUniversityof

California.ErnestOrlandoLawrenceBerkeleyNationalLaboratoryisanequalopportunityemployer.

1

Topics

u Context/BackgroundonEEEM&VMethods

u IntroductiontoTRMs

u TRMexamples

u SettingUpTRMsandLessonsLearned

u EM&VandTRMResources

Intentisforinformalpresentationanddiscussion

2

Why talk about TRMs?uUpside:

q Savemoneyforconsumers,utilities,stateagenciesandthusthepublic• EM&Vactivities costless

q Createcertainty(andcontrol)forallinvolvedwithreliablesavingsvaluesapplicabletoKentuckymarkets

q SavetimeintheEM&Vprocess

uDownside,ittakessomecoordinationandeffortupfront

3

Context/Background

4Kentucky EM&V Stakeholder Meeting - LBNL Presentation by Steven Schiller - March 2016

Approaches: Determining Gross Energy Savings

u Oneormoremeasurement andverification (M&V)optionsfromtheIPMVP(A,B,Cand/orD)areused todetermine thesavingsfromasampleofprojects.Thesesavingsarethenappliedtoalloftheprojects intheprogram.q Typicallyappliedto“calculated” or“custom” measures(e.g.,commercial HVAC)q Tendtobeproject-by-projectassessment ofsavingsq Involveatleast somemeasurements fromsitecombinedwithanalyses

u ConductStatisticalanalysesoflargevolumesofmetered energyusagedata.q Typicallyappliedtomassmarket,residential programs(e.g.,weatherizationprogramq Usesutility billdataandoftenacontrolgroup

u Applydeemed (stipulated,default)valuesorcalculations thatarebasedonhistoricalandverifieddatatoapplicableprojectsand/ormeasuresq Typicallyappliedtowelldefined,“prescriptive” or“standard”measuresq Someverification activities involvedq WhereTRMsareapplied

5

“Typical” Combination for Determining Gross Savings

u Prescriptive programs use deemed savings values for savings (e.g., residential CFLs, residential weather insulation, commercial ventilation motors, commercial building lighting)

u Custom programs use M&V:

q Calculated ex-ante savings estimates and 100% site verification with spot measurements (e.g., commercial lighting, HVAC and controls measures)

q Another set of custom programs use pre- and post-measurements on a census of projects (e.g., industrial process measures)

u Residential direct-install program uses large scale billing data analyses

6

Introduction to Technical Resource Manuals - TRMs

Kentucky EM&V Stakeholder Meeting - LBNL Presentation by Steven Schiller - March 2016 7Kentucky EM&V Stakeholder Meeting - LBNL

Presentation by Steven Schiller - March 2016

Deemed Savings and Algorithm Resource Database – AKA “TRM” u Resource (document,database, website) thatincludes informationusedin

program planning, reporting andevaluatingofEEprogramswhichcaninclude:q Energyefficiencymeasuresmetrics– deemedsavingsvalues(demand,energy)q Engineering algorithms tocalculatesavings– deemedsavingscalculationq Parametersneeded forcalculating savings(e.g.,typicaloperatinghours,heatingdegreedays)

q Factorsforapplyingtosavings(e.g.,net-to-grossratios,measure lifetimes, projectcosts)

u Typicallyincludedocumentationof:q Assumptions (e.g.,baselines) usedtopreparevaluesq Calculations ofvaluesq Whatareappropriateapplications forapplyingvaluesandalgorithms

u Commonreference forutilityprogrammanagers, implementers, evaluators,andregulators

8

Definitionsu DeemedSavingsValue:(StipulatedSavingsValue,UnitEnergySavings).

EstimateofenergyordemandsavingsforinstalledEEmeasure‘perunit’:q Examples:

• kWhsavedperyearper12WattLED• thermssavedperlinearfootofweatherstripping

q UsedforwellunderstoodanddocumentedEEmeasures• Forexample:energy-efficientappliancessuchaswashingmachines,computerequipmentandrefrigerators,andlightingretrofitprojectswithwell-understoodoperatinghours

q Developedfromreliabledatasourcesandanalyticalmethodsq Areapplicabletothesituationbeingevaluated

u DeemedSavingsCalculation:Anagreed-to(stipulated) engineeringalgorithm(s)usedtocalculatetheenergyand/ordemandsavingsassociatedwithaninstalledEEmeasure(s).q Example:KWhsavedperyear=[(annualoperatinghours)x(kWpre – kWpost)]

9

Measure Spectrum

u Graphic from VEIC

TRMAlgorithm Deemed Deemed Deemed Deemed Custom

Variables Deemed Deemed with multiple options

One or more custom

Many/all custom Custom

Output Single deemed Multiple deemed Custom Custom Custom

100% Prescriptive

100% Custom

Measure Spectrum

• Standardization

• Simple, Consistent Implementation

• Low Admin Cost

• “Accuracy”

• Complex, Varied Implementation

• High Admin Cost

10

Deemed Savings

uTheyareusedalot!

uProbablythemostcommon(byfar)methodusedforutilityEEprograms

But,MustBeUsedWithCaution

u Havetobeappliedwhereappropriate –only!

u Theuseofdeemedvalues inasavingscalculationisanagreement toacceptastipulatedvalue,irrespective ofwhatactually“happens”

u Whenusingdeemedvalues, itisimportanttorealize thattechnologies alonedonotsaveenergy - itishowtheyareusedthatsavesenergy

11

TRM Advantagesu Savestimeandmoneywhileprovidingrelativeaccuracy– calculateonce,

versusoverandoveragainforeachprogram(andproject?),everyyear

q Allowsevaluators tobetterallocateresources

u Pre-vetted, pre-approved values– reduceregulatory riskandprovidecertaintyforregulator,utility,implementerandcustomer intermsofincentivepayments

q Withvaluesvetted forKentucky

u Maintainsconsistency

q Planningandevaluationvalueswillbecalculatedusingthesamemethodologywhileallowingforutilityspecificinputs

q Evaluationfindings (e.g.,billinganalysis,metering, surveydata)informTRMupdatesallowingutilities topoolevaluation resources; theUniformMethodsProjectcanbeastandardwaytodevelop values:http://energy.gov/eere/about-us/ump-home

12

TRM and Deemed Savings Cautionsu Cantheybemisused?Yes!

q Onlyasgoodasthedata,analysis,andQCthatgoesintothem(garbage in…..)

q Accurateonaverage• Shouldbe- canbeevenbetter thancasebycaseM&V)• But,accurate foreachprojectandcustomer (probablynot…..)

u Watchoutfor:

q Applyingvaluesonlywhere theyareapplicable!q Systematicbiasesq Interactive andstackingeffects (multiplemeasures insamefacility)

u Bestwithtransparency anddocumentation includingaguideonhowtousethedataandalgorithms

13

Applicability Conditions (concept)

14

• 10hoursperday• Baseline=60WattsRetailLED

• 8hoursperday• Baseline =40Watts

Multi-FamilyCommonAreaLED

• 2.5 hoursperday• Baseline =40WattsPoint ofSaleLED

• 4hoursperday• Baseline=75Watts

LowIncomeResidentialDirectInstall

• 2.75hoursperday• Baseline=18Watts

ResidentialNewConstruction

Bottom Lineu TRMs:

q Creategreater savingscertaintyandconsistencyforsavingsvalues,andperhapsmoreaccuracy

q Arewidelyassumedtoreduceastate’sEM&Vcostsq FocusEM&Vresources

u StatewideorregionalTRMsarebecomingessentiallyastandardpractice– bitmoreonthislater

u However,

q Aswithanytoolneed tobeusedcorrectlyandwithcautionq Require (a)agreement amongstakeholders, (b)somestartup researchandcosts,and(c)timetogetgoing

15

TRM FormatsCanbeindifferentformats

u Onlinedatabase

u Downloadable database (mostcommon):

q ElectronicDatabase– oftenExcelworksheets butcanbe(shouldbe….)moresophisticateddatabase tools,provides lookupvaluesfortrackingsystem

q PDF – textformatwithcommonsections foreachmeasure protocol;mostcommonformatforrecentTRMs

q Word – textformat,similartoPDFs

16

TRM Coverage and Administration: Geographic or Jurisdictional Options

u Regional/Statewide

q Used tospecifythebasis fordeterminingsavingsvaluesclaimedbyanyprogramadministrator (e.g.,utility)inaregionorstate.Oftendeveloped throughamultiplestakeholder process

q Administered byregionalnon-profit, statecommissionoragency,advisorycommittee,programadministrators

u ProgramAdministrator (e.g.,utility)

q Used tospecifythesavingsvaluesclaimedbyasingleutility.Oftendeveloped bythatutility

q Administered byutility

17

Quick History of TRMsu 1990s – Thefirstdatabases ofsavings

q NorthwestPower&ConservationCouncil’sRegional Technical Forum(RTF)UnitEnergySavings(UES)WorkbooksDatabase

q CaliforniaPublicUtility Commission(CPUC)DatabaseforEnergyEfficient Resources(DEER)

u 2000s – ContinuedworkbytheRTFandCPUCandnewdocuments calledTechnical ReferenceManuals

q Morestatesstartedtodevelopthese resourcesfortheuseofallutilities withinthestate

u Now

q Moreandmorejurisdictions areadoptingTRMs– about20stateorregional TRMs

q Movement tocreateregional ifnotnationalstandardizationofresources

q U.S.DOEsupportingeffortsatstandardization

18

Today’s National Picture

About half the states have TRM resources (including New Mexico)

19

A Few State Examplesu ArkansasTRM

q CompilationofdeemedsavingsvaluesforelectricandgasenergyefficiencymeasuresbeingimplementedbytheArkansasIOUs

q Providecommonframeworkandsetofreferencepointsforconductingcost-effectiveDSMProgramevaluations.q Describethetypesofinformationthatmustbecollected, frequencyforconductingevaluations,andthekeymetricsthat

mustbereportedq Keydefinitions,recommendationsregardingdatacaptureandEM&Vreportingformats.

u EnergyTrustofOregonTRMq PrimarilyfullydeemedvaluesinthedatabasedevelopedinconjunctionwithanalysiscompletedbyRegionalTechnical

Forum;somecalculatorsq Usedforprograms,andmadespecifictoOregonIOUterritoryq UpdatedasneededwithEM&Vresults

u MichiganEnergyMeasuresDatabaseq Provideuserswithaccurateinformationonpotentialtechnologiesormeasuresthatcouldbeusedinanenergy

efficiencyprogramsq ProvidecustomizedmeasuresforMichiganspecificweatherandloads.q AllowforconsistentapplicationofinformationacrossMichiganforenergyefficiencyplanningandgoalmeasurement.q Allowforconsistencyofassumptions.q Providedocumentationforregulatoryreviewandreconciliationprocesses.

u PennsylvaniaTRMq Primarilypartiallydeemedalgorithmswithinputsbasedonlook-uptablesorcustomer-specificapplicationdataq Usedforprogramplannersandclaimedsavingsq Updatedannually

20

Lets run through some examples…..

21

Setting Up and Using TRMs and

Lessons Learned22Kentucky EM&V Stakeholder Meeting - LBNL

Presentation by Steven Schiller - March 2016

Planning, Implementing, and Evaluating Efficiency Programs

ES-3November 2012 www.seeaction.energy.gov

ďĞĐŽŶƐŝĚĞƌĞĚǁŚĞŶĚĞƚĞƌŵŝŶŝŶŐŶĞƚƐĂǀŝŶŐƐEĞƚƐĂǀŝŶŐƐĂƌĞĚŝƐĐƵƐƐĞĚŝŶŚĂƉƚĞƌϱ

ͻ ƐƟŵĂƚĞƐŽĨŶŽŶͲĞŶĞƌŐLJďĞŶĞĮƚƐ;EƐͿThese are the impacts ĂƐƐŽĐŝĂƚĞĚǁŝƚŚƉƌŽŐƌĂŵŝŵƉůĞŵĞŶƚĂƟŽŶŽƌƉĂƌƟĐŝƉĂƟŽŶĂƐŝĚĞĨƌŽŵĞŶĞƌŐLJĂŶĚĚĞŵĂŶĚƐĂǀŝŶŐƐdŚĞƐĞƌĞƐƵůƚƐĐĂŶďĞƉŽƐŝƟǀĞŽƌŶĞŐĂƟǀĞ^ŽŵĞĞdžĂŵƉůĞƐŝŶĐůƵĚĞƌĞĚƵĐĞĚĞŵŝƐƐŝŽŶƐĂŶĚĞŶǀŝƌŽŶŵĞŶƚĂůďĞŶĞĮƚƐƉƌŽĚƵĐƟǀŝƚLJŝŵƉƌŽǀĞŵĞŶƚƐũŽďƐĐƌĞĂƚĞĚĂŶĚůŽĐĂůĞĐŽŶŽŵŝĐĚĞǀĞůŽƉŵĞŶƚƌĞĚƵĐĞĚƵƟůŝƚLJĐƵƐƚŽŵĞƌĚŝƐĐŽŶŶĞĐƚƐŐƌĞĂƚĞƌĐŽŵĨŽƌƚĨŽƌďƵŝůĚŝŶŐŽĐĐƵƉĂŶƚƐůŽǁĞƌŵĂŝŶƚĞŶĂŶĐĞĐŽƐƚƐĚƵĞƚŽďĞƩĞƌĞƋƵŝƉŵĞŶƚŽƌŝŶĐƌĞĂƐĞĚŵĂŝŶƚĞŶĂŶĐĞĐŽƐƚƐĚƵĞƚŽŶĞǁĂŶĚŵŽƌĞĐŽŵƉůĞdžƐLJƐƚĞŵƐEƐĂƌĞĚŝƐĐƵƐƐĞĚŝŶ^ĞĐƟŽŶϳϵ

ES.4 ENERGY SAVINGS ESTIMATES AND UNCERTAINTY: “HOW GOOD IS GOOD ENOUGH?”ĂĐŚŽĨƚŚĞďƵůůĞƚƐĂďŽǀĞŝŶ^ĞĐƟŽŶ^ϯĚĞĮŶĞƐĂŶĞƐƟŵĂƚĞ ǀĞƌƐƵƐĂŶĞdžĂĐƚǀĂůƵĞdŚŝƐŝƐďĞĐĂƵƐĞĞŶĞƌŐLJĂŶĚĚĞŵĂŶĚƐĂǀŝŶŐƐĂƐǁĞůůĂƐŶŽŶͲĞŶĞƌŐLJďĞŶĞĮƚƐƌĞƐƵůƟŶŐĨƌŽŵĞĸĐŝĞŶĐLJĂĐƟŽŶƐĐĂŶŶŽƚ ďĞĚŝƌĞĐƚůLJŵĞĂƐƵƌĞĚ/ŶƐƚĞĂĚƐĂǀŝŶŐƐĂŶĚďĞŶĞĮƚƐĂƌĞďĂƐĞĚŽŶĐŽƵŶƚĞƌĨĂĐƚƵĂůĂƐƐƵŵƉƟŽŶƐhƐŝŶŐĐŽƵŶƚĞƌĨĂĐƚƵĂůĂƐƐƵŵƉƟŽŶƐŝŵƉůŝĞƐƚŚĂƚƐĂǀŝŶŐƐĂƌĞĞƐƟŵĂƚĞĚƚŽǀĂƌLJŝŶŐĚĞŐƌĞĞƐŽĨĂĐĐƵƌĂĐLJ ďLJĐŽŵƉĂƌŝŶŐƚŚĞƐŝƚƵĂƟŽŶ;ĞŐĞŶĞƌŐLJĐŽŶƐƵŵƉƟŽŶͿĂŌĞƌĂƉƌŽŐƌĂŵŝƐŝŵƉůĞŵĞŶƚĞĚ;ƚŚĞƌĞƉŽƌƟŶŐƉĞƌŝŽĚͿƚŽǁŚĂƚŝƐĂƐƐƵŵĞĚƚŽŚĂǀĞďĞĞŶƚŚĞƐŝƚƵĂƟŽŶŝŶƚŚĞĂďƐĞŶĐĞŽĨƚŚĞƉƌŽŐƌĂŵ;ƚŚĞ ĐŽƵŶƚĞƌĨĂĐƚƵĂůƐĐĞŶĂƌŝŽŬŶŽǁŶĂƐƚŚĞďĂƐĞůŝŶĞͿ&ŽƌĞŶĞƌŐLJŝŵƉĂĐƚƐƚŚĞďĂƐĞůŝŶĞĂŶĚƌĞƉŽƌƟŶŐƉĞƌŝŽĚĞŶĞƌŐLJƵƐĞĂƌĞĐŽŵƉĂƌĞĚǁŚŝůĞĐŽŶƚƌŽůůŝŶŐ;ŵĂŬŝŶŐĂĚũƵƐƚŵĞŶƚƐͿĨŽƌĨĂĐƚŽƌƐƵŶƌĞůĂƚĞĚƚŽĞŶĞƌŐLJ ĞĸĐŝĞŶĐLJĂĐƟŽŶƐƵĐŚĂƐǁĞĂƚŚĞƌŽƌďƵŝůĚŝŶŐŽĐĐƵƉĂŶĐLJ dŚĞƐĞĂĚũƵƐƚŵĞŶƚƐĂƌĞĂŵĂũŽƌƉĂƌƚŽĨƚŚĞĞǀĂůƵĂƟŽŶƉƌŽĐĞƐƐŚŽǁƚŚĞLJ are determined can vary from one program type to another and ĨƌŽŵŽŶĞĞǀĂůƵĂƟŽŶĂƉƉƌŽĂĐŚƚŽĂŶŽƚŚĞƌ

ĞĐĂƵƐĞƚŚĞŝŶĚŝĐĂƚĞĚǀĂůƵĞƐĂƌĞĞƐƟŵĂƚĞƐƚŚĞŝƌƵƐĞĂƐĂďĂƐŝƐĨŽƌĚĞĐŝƐŝŽŶŵĂŬŝŶŐĐĂŶďĞĐŚĂůůĞŶŐĞĚŝĨƚŚĞŝƌƐŽƵƌĐĞƐĂŶĚůĞǀĞůŽĨĂĐĐƵƌĂĐLJĂƌĞŶŽƚĚĞƐĐƌŝďĞĚdŚĞƌĞĨŽƌĞĞǀĂůƵĂƟŽŶƌĞƐƵůƚƐůŝŬĞĂŶLJĞƐƟŵĂƚĞƐŚŽƵůĚďĞƌĞƉŽƌƚĞĚĂƐĞdžƉĞĐƚĞĚǀĂůƵĞƐƚŚĂƚŝƐďĂƐĞĚŽŶƚŚĞŝŵƉĂĐƚĞǀĂůƵĂƟŽŶǀĂůƵĞƐĂƌĞĞdžƉĞĐƚĞĚƚŽďĞĐŽƌƌĞĐƚǁŝƚŚŝŶĂŶĂƐƐŽĐŝĂƚĞĚůĞǀĞůŽĨĐĞƌƚĂŝŶƚLJ DŝŶŝŵŝnjŝŶŐƵŶĐĞƌƚĂŝŶƚLJĂŶĚďĂůĂŶĐŝŶŐ ĞǀĂůƵĂƟŽŶĐŽƐƚƐǁŝƚŚƚŚĞǀĂůƵĞŽĨƚŚĞŝŶĚŝĐĂƚĞĚĞǀĂůƵĂƟŽŶŝŶĨŽƌŵĂƟŽŶ ĂƌĞĂƚƚŚĞŚĞĂƌƚŽĨƚŚĞĞǀĂůƵĂƟŽŶƉƌŽĐĞƐƐĂŶĚůĞĂĚƐƚŽƉĞƌŚĂƉƐƚŚĞŵŽƐƚĨƵŶĚĂŵĞŶƚĂůĞǀĂůƵĂƟŽŶƋƵĞƐƟŽŶ,ŽǁŐŽŽĚŝƐŐŽŽĚĞŶŽƵŐŚdŚŝƐƋƵĞƐƟŽŶŝƐĂƐŚŽƌƚǀĞƌƐŝŽŶŽĨĂƐŬŝŶŐ;ϭͿǁŚĂƚůĞǀĞůŽĨĐĞƌƚĂŝŶƚLJŝƐƌĞƋƵŝƌĞĚĨŽƌĞŶĞƌŐLJƐĂǀŝŶŐƐĞƐƟŵĂƚĞƐƌĞƐƵůƟŶŐĨƌŽŵĞǀĂůƵĂƟŽŶĂĐƟǀŝƟĞƐĂŶĚ;ϮͿŝƐƚŚĂƚůĞǀĞůŽĨĐĞƌƚĂŝŶƚLJƉƌŽƉĞƌůLJďĂůĂŶĐĞĚĂŐĂŝŶƐƚƚŚĞĂŵŽƵŶƚŽĨĞīŽƌƚ;ĞŐƌĞƐŽƵƌĐĞƐƟŵĞŵŽŶĞLJͿƵƐĞĚƚŽŽďƚĂŝŶƚŚĂƚůĞǀĞůŽĨĐĞƌƚĂŝŶƚLJ

dǁŽƉƌŝŶĐŝƉůĞƐĂƌĞŝŵƉŽƌƚĂŶƚǁŚĞŶĐŽŶƐŝĚĞƌŝŶŐŚŽǁŐŽŽĚŝƐŐŽŽĚĞŶŽƵŐŚ;ϭͿĞŶĞƌŐLJĞĸĐŝĞŶĐLJŝŶǀĞƐƚŵĞŶƚƐƐŚŽƵůĚďĞĐŽƐƚͲĞīĞĐƟǀĞĂŶĚ;ϮͿĞǀĂůƵĂƟŽŶŝŶǀĞƐƚŵĞŶƚƐƐŚŽƵůĚĐŽŶƐŝĚĞƌƌŝƐŬŵĂŶĂŐĞŵĞŶƚƉƌŝŶĐŝƉůĞƐĂŶĚƚŚƵƐďĂůĂŶĐĞƚŚĞĐŽƐƚƐŽĨĞǀĂůƵĂƟŽŶĂŐĂŝŶƐƚƚŚĞǀĂůƵĞŽĨƚŚĞŝŶĨŽƌŵĂƟŽŶĚĞƌŝǀĞĚĨƌŽŵĞǀĂůƵĂƟŽŶ;ŝĞĞǀĂůƵĂƟŽŶƐŚŽƵůĚĂůƐŽďĞĐŽƐƚͲĞīĞĐƟǀĞͿdŚĞǀĂůƵĞŽĨƚŚĞŝŶĨŽƌŵĂƟŽŶŝƐĚŝƌĞĐƚůLJƌĞůĂƚĞĚƚŽƚŚĞƌŝƐŬƐŽĨƵŶĚĞƌĞƐƟŵĂƟŶŐŽƌŽǀĞƌĞƐƟŵĂƟŶŐƚŚĞďĞŶĞĮƚƐ;ƐĂǀŝŶŐƐͿĂŶĚĐŽƐƚƐĂƐƐŽĐŝĂƚĞĚǁŝƚŚĞĸĐŝĞŶĐLJŝŶǀĞƐƚŵĞŶƚƐdŚĞƐĞƌŝƐŬƐŵŝŐŚƚďĞĂƐƐŽĐŝĂƚĞĚǁŝƚŚĞƌƌŽƌƐŽĨĐŽŵŵŝƐƐŝŽŶŽƌĞƌƌŽƌƐŽĨŽŵŝƐƐŝŽŶŶĞƌƌŽƌŽĨĐŽŵŵŝƐƐŝŽŶŵŝŐŚƚďĞŽǀĞƌĞƐƟŵĂƟŶŐƐĂǀŝŶŐƐǁŚŝĐŚŝŶƚƵƌŶĐĂŶƌĞƐƵůƚŝŶĐŽŶƟŶƵŝŶŐƉƌŽŐƌĂŵƐƚŚĂƚĂƌĞŶŽƚĐŽƐƚͲĞīĞĐƟǀĞĂŶĚŽƌŽǀĞƌƉĂLJŝŶŐĐŽŶƚƌĂĐƚŽƌƐĂĚŵŝŶŝƐƚƌĂƚŽƌƐĂŶĚƉĂƌƟĐŝƉĂŶƚƐŶĞƌƌŽƌŽĨŽŵŝƐƐŝŽŶŽŶƚŚĞŽƚŚĞƌŚĂŶĚŵŝŐŚƚďĞĂƐƐŽĐŝĂƚĞĚǁŝƚŚƵŶĚĞƌĞƐƟŵĂƟŶŐƐĂǀŝŶŐƐŽƌŶŽƚŝŵƉůĞŵĞŶƟŶŐĞĸĐŝĞŶĐLJĂĐƟŽŶƐďĞĐĂƵƐĞŽĨƚŚĞĚŝĸĐƵůƚLJŝŶĚŽĐƵŵĞŶƟŶŐƐĂǀŝŶŐƐďŽƚŚŽĨǁŚŝĐŚĐĂŶƌĞƐƵůƚŝŶƵŶĚĞƌŝŶǀĞƐƟŶŐŝŶĞĸĐŝĞŶĐLJĂŶĚƌĞůLJŝŶŐŽŶŽƚŚĞƌĞŶĞƌŐLJƌĞƐŽƵƌĐĞƐƚŚĂƚŚĂǀĞƚŚĞŝƌŽǁŶƌŝƐŬƐĂŶĚƵŶĐĞƌƚĂŝŶƟĞƐƐƵĐŚĂƐĨƵĞůĐŽƐƚƐĂŶĚĞŶǀŝƌŽŶŵĞŶƚĂůŝŵƉĂĐƚƐ

ES.5 EVALUATION AND THE EFFICIENCY PROGRAM PROCESS ƐƐŚŽǁŶŝŶ&ŝŐƵƌĞ^ϮƚŚĞĞĸĐŝĞŶĐLJƉƌŽŐƌĂŵƉƌŽĐĞƐƐĐŽŶƐŝƐƚƐŽĨƉůĂŶŶŝŶŐŝŵƉůĞŵĞŶƟŶŐĂŶĚĞǀĂůƵĂƟŶŐĂĐƟǀŝƟĞƐdŚƌŽƵŐŚŽƵƚƚŚŝƐƉƌŽĐĞƐƐƐĂǀŝŶŐƐǀĂůƵĞƐĂƌĞƚLJƉŝĐĂůůLJŝŶĚŝĐĂƚĞĚďĂƐĞĚŽŶĞƐƟŵĂƚĞƐƉƌĞƉĂƌĞĚĂƐƉĂƌƚŽĨĞĂĐŚĂĐƟǀŝƚLJ KŶĞǁĂLJƚŽĚĞƐĐƌŝďĞƚŚĞƐĞƐĂǀŝŶŐƐŝƐǁŝƚŚƚŚĞĨŽůůŽǁŝŶŐĐůĂƐƐŝĮĐĂƟŽŶƐĂůƐŽĚŝƐƉůĂLJĞĚŝŶ&ŝŐƵƌĞ^Ϯͻ WƌŽũĞĐƚĞĚƐĂǀŝŶŐƐǀĂůƵĞƐƌĞƉŽƌƚĞĚďLJĂƉƌŽŐƌĂŵŝŵƉůĞŵĞŶƚĞƌ

ŽƌĂĚŵŝŶŝƐƚƌĂƚŽƌďĞĨŽƌĞƚŚĞĞĸĐŝĞŶĐLJĂĐƟǀŝƟĞƐĂƌĞĐŽŵƉůĞƚĞĚ

FIGURE ES.2: Workflow and reporting for planning, implementing, and evaluating efficiency programs

PLANPROGRAMS

PROJECTED SAVINGS

IMPLEMENTPROGRAMS

CLAIMEDSAVINGS

EVALUATEPROGRAMS

EVALUATEDSAVINGS

ATRMIsUsed inAllThreeActivities

23

Structure for Defining Evaluation Activities – including TRMsu EM&V Framework – Primary

document that lays out top level structure. This is perhaps the principle document that all stakeholders can focus on and provide high level input. When used -q This is where the TRM concept gets

defined

u Portfolio (annual) Plans – Indicates major evaluation activities that will be conducted during the evaluation cycle

u Evaluation Research Plans – Created for the major EM&V activities

u Site Specific M&V Plans – For custom project sites that are analyzed and inspected

Kentucky EM&V Stakeholder Meeting - LBNL Presentation by Steven Schiller - March 2016 24

How TRM Efforts Get Initiatedu Mostare"ordered" orjustagreed tobyaCommissionorperhaps anadvisory

boardifthere isathird-party EEadministrator

u WiththeseTRMsoperatinginabouthalfthestates, commissions, stakeholders,utilities,and/orgroupofimplementers areaskingandsaying:

q Whyarewerecalculating orre-justifyingthesamesavingsvaluesoverandoveragain?q Whydoesoneofourutilities use“x”andanother ”y”forthesavingsforthesamemeasure?

q Howcanweincreasecertaintyofsavingsthroughouttheprocess– i.e.howmanageriskforutility customersandutilities?

q Howcanwesavetimeandmoney?

u Thebarriersareusuallymoneyandprocess:q Itsalmostcertainlycheapertodooneforthestateversusoneperutilityorimplementer, butthosecostsareburied,versusasingle largerline item

q Insomecases utilities andimplementers dopreferthattheCommission approvestheTRM- toavoidsecondguessing, i.e.toprovidecertainty

25

Getting a TRM Process Started 1. Research(reviewotherstates’andregionsTRMefforts)

2. SetObjectives:

a. Usedforplanning,reportingand/orinplaceofex-postsavingsdetermination?

3. Decidewhatinformationisneeded:

a. Forexample,grossand/ornetsavingsvalues,costdata,effectiveusefullife

b. Deemedsavingvaluesonlyoralsocalculation tools?Includeworkpapersforcustommeasures?

4. Answersomequestions:

a. Who’sdatabase?Howisitreviewed? Howisitapproved?

b. Startfromscratchorstartwithanotherstate’ssystemandmodifyforKentucky?Aregionaleffort?c. Buildlargesystem(lotsofEEmeasures)frombeginningorstartsmall(justhighpriorityEEmeasures)and

buildupasdatawarrants?d. Whatformat– on-line,spreadsheet,pdf,etc.e. Whodevelops,verifies,andmaintainsdata?

f. Whatarecriteria for“good” dataandhowrigorouslyitisverifiedandappliedappropriately.Howarebaselinesdefined?

5. Thensetbudgetsandtimeframes– andaframeworkand/orworkplan

26

UsingTRMDataFromOtherStatesu Manystatesusedatafromother states,butwill

document thatisapplicable totheirownstate(climate,market,baselines, operatinghours,measurecharacteristics)

u Scopingstudywasconducteddeveloping regionalTRMs and includedanassessment ofsavingsvaluesfor20measures coveringdifferent fuels, sectors,end-uses inmultipleTRMs, findings:q Savingsestimates varybyorderofmagnitude acrossTRMs

q Maindriversofvariancesare:• Differingbaseline assumptions (e.g.,hoursofuse,weather,prevailingcodes)

• Sourceofsavingscalculations (building simulationversusengineering algorithm)

• Parametersincluded inalgorithm(e.g.,useofHVACinteractionfactorforlighting)

27

So How Much Does it Cost?u ItDepends!u Itdependson:

q Timing– howquickyouwantitandhowoftenupdated

q Qualityq Scope

• HowmanyEEmeasures• Primaryorsecondaryresearch instate– orjustupdatetoanother

• Levelofdocumentation• Format• Levelofreview

u BallParkCostRanges– ItDepends!!

u Development - $50,000- $200,000

q Could itbeless?…perhaps; coulditbemore?….ohyes

u Updates - $10,000to$50,000percycle

q Mid-Atlantic TRMupdateis$75,000peryear,balance largerscopeandreviewprocesswithveryefficient/experienced team(thisisalsoaboutwhatitcostformodifying itforforusinanotherstate)

u Canbecombinedwith:

q Scopeofanevaluatorq Trackingandreportingsystems

28

Updating Processu Typicallydoneeveryone,twoorthreeyears

u Reviewandsummarizeotherjurisdiction’sTRMupdateprocessesforcomparisonandguidance

u RecommendanoverarchingstrategytoupdatetheTRMinatimelyandappropriatemanner,tobestmeettheneedsoftheorganizationsusingit

u Interviewstakeholderstoidentifyneedsandschedulesrelevanttotheupdateprocess,commonalitiesthataremutuallysupportiveofasingleprocessandschedule,aswellasanyuniqueneedsorsituationsthatnecessitateextraattention.

u IdentifymeasurestobeaddedorupdatedinthenextroundofTRMmeasuredevelopment

Lessons Learnedu Defineupdatecyclethatmatchesplanning

cycles(orplanningandreportingifretroactiveapplication)

q Typicallyannualoreveryotheryear

q Berealistic ontimerequiredtodoupdates

u Usesavingsverificationandevaluationresultstoinformupdates

u Developprocesswhereoldmeasuresaresystematicallyreviewedthroughannualupdateprocess

u Maintainareferencelibrarytotrack:

q Changesq Feedback

q Errorcorrectionsq Newinformationincludingnewmeasure

suggestionsandreferences

29

Who Does Evaluation (and TRMs)

u Administrationoftheevaluation function:

• 37%utility administration• 36%administration bytheutilityregulatorycommission oracombinationof

thecommission andutilities• 27%administration bysomeothergovernmentagencyorthird-partyentity

u Moststates(79%)relyonindependent consultants/contractors toconducttheactualevaluationswith21%usingutilityand/orgovernment agencystaff

“ANationalSurveyOfStatePoliciesandPracticesFortheEvaluationOfRatepayer-FundedEnergyEfficiencyPrograms”MartinKushler,SethNowak,andPattiWitteFebruary2012ReportNumberU122.www.aceee.org

30

Lessons Learned –TRM Development Process

u Establish definitions formetrics (gross,net,incremental savings,lifetime, etc.)andmeasures

u Clearlydefinerolesandresponsibilities ofdifferentparticipants

u DefineprocessforinputandapprovalofTRMandupdates

q Strivefortransparency,wideinputandlimitedlegal/regulatoryhoopsrequiredtomakechanges

u Decidewhethervaluesaretobe“expected values”or“conservativevalues”(rememberEEsavingsareestimates)

u Providesomeguidanceonselection criteriaforwhatmeasures gointheTRM

u Processguidanceshouldmake itclearwhatassumptions areusedandforwhichpurposes

u Decidehowvaluesareused– “lookingback”or“goingforward”

31

Looking Back or Going Forward?q FortheaffectedmeasurestheperunitenergysavingsarebasedonthevaluesfoundintheTRM–

butwhichversionforwhatpurposesandwhen?– TRMvaluescananddochange– mistakesfound,betterdata,baselineschange,etc.

q Forexample:– TRMupdatedinNovember2013andNovember2014andNovember2015– ProgramplanssubmittedinAugust2014– usedNovember2013TRMvalues– ProgramapprovedinDecember2014- withNovember2013TRMvalues– Programimplementedin2015– whichTRMversionshouldbeusedforclaimedsavings?– Programevaluationcompletedin2016- whichTRMversionusedforevaluatedsavings?– What’sfairtoutility?What’sfairtotheratepayers?What’srightforsystemplanners?

q Pointsouttwoissues:• Shouldlineupprogramplanning,approvalswithTRMupdates – itwouldhavebeenmuchbetter

ifthe2014TRMupdatewasdoneinsummerof2014versusfall• Shoulddecideinframeworkwhetherutilitiesgetcreditforsavingsbasedonlookingback or

goingforwardTRMversions– CommonapproachisusingTRMvaluesvalidattimeofprogramapproval

32

EM&V and TRM Resources

Kentucky EM&V Stakeholder Meeting - LBNL Presentation by Steven Schiller - March 2016 33Kentucky EM&V Stakeholder Meeting - LBNL

Presentation by Steven Schiller - March 2016

EM&V and TRM Resources

u DOE/EPASEEActionEM&VResourceswebsite:http://www4.eere.energy.gov/seeaction/topic-category/evaluation-measurement-and-verification

u U.S.DOEUniformMethodsProjectwebsite:http://energy.gov/eere/about-us/ump-home

u TRMreviewanddevelopmentscopingstudy:http://www4.eere.energy.gov/seeaction/system/files/documents/emvscoping_databasefeasibility.pdf

34

SEE Action Impact Evaluation Guideu Describes common terminology,

structures, and approaches used for determining (evaluating):q energy and demand savings q avoided emissions q other non-energy benefits

u Does not recommend specific approaches - it provides:q context q planning guidanceq discussion of issues

i

Energy-Efficiency Program Impact Evaluation Guide An introduction to and summary of the practices, planning, and associated issues of documenting energy savings, demand savings, avoided emissions, and other non-energy benefits resulting from end-use energy-efficiency programs. A RESOURCE OF THE STATE AND LOCAL ENERGY EFFICIENCY ACTION NETWORK

http://www1.eere.energy.gov/seeaction/

35

Discussion/Questions

Contact:SteveSchillerSeniorAdvisor,LBNLSRSchiller@lbl.gov

36

KentuckyEM&VStakeholderMeeting- LBNLPresentation byStevenSchiller- March2016